AI Vendor Governance

AI vendor governance is the practice of evaluating, onboarding, and overseeing third-party artificial intelligence providers to ensure they meet organizational standards for data protection, security, compliance, and transparency. It is a specialized form of third-party risk management focused on the unique risks introduced by AI vendors.

The need for AI vendor governance arises from a simple reality: most AI tools used in organizations are not built internally. They are third-party SaaS products — ChatGPT, Jasper, Copilot, specialized healthcare AI tools, financial analysis AI, and hundreds of others. When an organization adopts one of these tools, it depends on the vendor to protect data, maintain compliance, operate transparently, and remain financially stable.

This dependence creates risk. If a vendor experiences a data breach, the organization data may be exposed. If a vendor changes its terms of service, data handling practices may change without notice. If a vendor uses customer data for model training, confidential information may be incorporated into AI models. If a vendor goes out of business, critical workflows may be disrupted.

AI vendor governance addresses these risks through a structured lifecycle: evaluation before onboarding, contractual safeguards during onboarding, and ongoing monitoring throughout the vendor relationship. For regulated organizations, this lifecycle is not optional — HIPAA, SEC regulations, and state privacy laws all require oversight of third-party vendors that handle protected data.

The distinction between AI vendor governance and general vendor management is the focus on AI-specific risks: model transparency, data training practices, probabilistic outputs, and the rapid evolution of AI technology. General vendor management may not address these AI-specific dimensions.

AI vendor governance follows a structured lifecycle that spans the entire vendor relationship, from initial evaluation through ongoing oversight and eventual termination.

Phase 1: Evaluation and Due Diligence Before onboarding any AI vendor, conduct thorough due diligence. This phase evaluates whether the vendor meets organizational standards across security, compliance, data handling, and transparency. The depth of due diligence should match the risk level — higher-risk vendors require deeper evaluation.

Phase 2: Contractual Safeguards Once due diligence is complete and the vendor is approved, execute contractual agreements that protect the organization. For healthcare, this includes a BAA. For all organizations handling personal data, this includes a data processing agreement. Additional contractual protections may include confidentiality clauses, breach notification requirements, and audit rights.

Phase 3: Onboarding and Integration With contracts in place, onboard the vendor technically and operationally. Configure the tool with appropriate data access restrictions, train users on safe usage, and add the tool to the AI inventory. Ensure that the approval is documented for audit purposes.

Phase 4: Ongoing Monitoring Vendor governance does not end at onboarding. Vendors change terms, update models, experience incidents, and sometimes go out of business. Establish ongoing monitoring to track these changes and assess their impact on organizational risk.

Phase 5: Review and Renewal Periodically review each vendor relationship. Assess whether the vendor continues to meet standards, whether the use case has changed, and whether the relationship should continue, be modified, or be terminated. Annual reviews are a minimum; high-risk vendors may require more frequent review.

Phase 6: Termination and Offboarding When a vendor relationship ends — whether by choice, vendor closure, or performance issues — execute a structured offboarding process. Ensure data is returned or deleted, access is revoked, and the inventory is updated. Document the termination for audit purposes.

This lifecycle ensures that vendor governance is continuous, not a one-time evaluation. Each phase has specific activities, deliverables, and documentation requirements that together create a comprehensive vendor governance program.

Vendor due diligence is the evaluation phase of the vendor governance lifecycle. It is the most important phase because it prevents problematic vendors from being onboarded in the first place. The cost of due diligence is far less than the cost of remediating a vendor-related compliance failure.

Security Certifications Verify that the vendor holds appropriate security certifications: - SOC 2 Type II: Demonstrates security, availability, processing integrity, confidentiality, and privacy controls - ISO 27001: International standard for information security management - HITRUST CSF: Healthcare-specific security and compliance certification - FedRAMP: For federal government customers (if applicable)

Request the certification reports — not just confirmation that certifications exist. Review the reports for any exceptions, qualifications, or scope limitations that may be relevant to your use case.

Data Handling Practices Understand how the vendor handles data: - What data does the tool collect, process, and store? - Is customer data used for model training? If so, is it opt-in or opt-out? - Where is data stored (geographic location may affect regulatory obligations)? - What subprocessors does the vendor use, and are they disclosed? - What are the data retention policies? - Can the organization export or delete its data?

For healthcare organizations, verify that the vendor does not use PHI for model training without explicit authorization. This is a HIPAA violation if done without appropriate safeguards.

BAA Status For healthcare organizations, verify whether the vendor will sign a Business Associate Agreement. If the tool processes PHI and the vendor will not sign a BAA, the tool cannot be used with PHI under HIPAA. This is a non-negotiable requirement — there are no workarounds.

Model Transparency Assess the vendor transparency about how AI outputs are generated: - Does the vendor disclose what AI models are used? - Is there documentation of model training data and methodology? - Does the vendor provide explainability features for AI outputs? - How does the vendor handle model updates and changes? - Does the vendor provide usage analytics or audit logs?

Complete transparency is rare in the AI industry, but vendors should be willing to disclose enough for the organization to assess risk. Vendors that refuse to provide any transparency information should be viewed with caution.

Incident History Review the vendor incident history: - Have there been previous data breaches? If so, what happened and how was it resolved? - Have there been service outages or disruptions? - Has the vendor been subject to regulatory enforcement actions? - Are there customer reviews or reports of issues?

No vendor is perfect — incidents happen. The question is not whether incidents have occurred but how the vendor responded and what safeguards have been implemented to prevent recurrence.

Financial Stability Assess whether the vendor is likely to remain operational: - How long has the vendor been in business? - What is the vendor funding status and financial health? - Does the vendor have a sustainable business model? - Are there signs of financial distress?

Vendor failure can disrupt operations and create data access issues. For critical AI tools, financial stability assessment is important. For lower-risk tools, this may be less critical.

Contractual safeguards are the legal protections that enforce vendor governance requirements. They translate governance expectations into enforceable obligations. For regulated organizations, certain contracts are not optional — they are legal requirements.

Business Associate Agreement (BAA) For healthcare organizations, a BAA is required under HIPAA for any vendor that creates, receives, maintains, or transmits PHI. The BAA specifies how the vendor may use PHI, requires safeguards equivalent to HIPAA standards, establishes breach notification timelines, and defines liability for non-compliance.

Key BAA provisions to verify: - Permitted uses and disclosures of PHI - Safeguards requirements (HIPAA Security Rule alignment) - Breach notification timeline (no later than 60 days from discovery) - Subcontractor oversight obligations - Data return or destruction upon termination - Reporting and audit rights

If a vendor will not sign a BAA, the tool cannot be used with PHI. There are no exceptions.

Data Processing Agreement (DPA) For organizations subject to GDPR, CCPA, or state privacy laws, a data processing agreement specifies how the vendor may process personal data. Key DPA provisions include: - Purpose limitation (data may only be used for the agreed purpose) - Data minimization (only necessary data may be processed) - Subprocessor disclosure and approval rights - Data subject rights support (access, deletion, opt-out) - Data return or deletion upon termination - Audit and inspection rights

Additional Contractual Protections Beyond BAAs and DPAs, consider additional contractual protections: - Confidentiality agreements protecting organizational data - Service level agreements (SLAs) for availability and performance - Indemnification for vendor-caused losses - Limitation of liability (negotiate appropriate caps) - Insurance requirements (cyber liability insurance) - Change notification requirements (vendor must notify of terms or practice changes) - Termination rights (what happens to data when the relationship ends)

Contract Review Process Have legal counsel review all vendor contracts before execution. Legal review should verify that: - Regulatory requirements are met (HIPAA, state privacy laws) - Organizational interests are protected - Risk allocation is appropriate - Remediation and termination rights are clear - Audit and reporting rights are sufficient

Maintain executed contracts in a centralized repository for audit purposes. Track contract expiration dates and renewal requirements to prevent lapses.

Vendor governance does not end at onboarding. Vendors change — they update terms of service, modify data handling practices, experience security incidents, update AI models, and sometimes go out of business. Without ongoing monitoring, these changes can introduce new risks that the organization is unaware of.

What to Monitor Key vendor changes to monitor include: - Terms of service and privacy policy changes - Data handling practice changes (new subprocessors, data retention changes) - Security incidents or breaches - Model updates or changes to AI functionality - Vendor ownership changes (acquisitions, mergers) - Vendor financial health changes - Security certification expirations or changes

Monitoring Methods - Subscribe to vendor change notifications where available - Review vendor websites periodically for policy changes - Monitor security advisory services for vendor incidents - Maintain communication with vendor account managers - Review vendor security certifications annually - Track industry news for vendor-related developments

Vendor Owner Assignment Assign a vendor owner for each AI vendor relationship. The vendor owner is responsible for: - Maintaining communication with the vendor - Tracking term and practice changes - Conducting periodic vendor reviews - Coordinating due diligence updates - Managing contract renewals - Reporting vendor risk to the governance committee

The vendor owner does not need to perform all activities personally but is accountable for ensuring they happen. For smaller organizations, one person may own multiple vendor relationships. For larger organizations, vendor ownership may be distributed across departments.

Vendor Review Cadence Conduct vendor reviews at the following cadence: - High-risk vendors: review at least semi-annually - Moderate-risk vendors: review annually - Low-risk vendors: review bi-annually

Vendor reviews should assess whether the vendor continues to meet organizational standards, whether the use case has changed, whether risk has increased or decreased, and whether the relationship should continue, be modified, or be terminated.

Responding to Vendor Changes When a vendor makes a material change — updates terms, changes data practices, or experiences an incident — assess the impact on organizational risk. Determine whether the change affects compliance, data security, or operational continuity. If the change increases risk, take action: renegotiate terms, restrict usage, or terminate the relationship.

Document all vendor changes and organizational responses for audit purposes. This demonstrates that vendor governance is active, not just a one-time evaluation.

Not all AI vendors carry the same risk. A large, established vendor with HITRUST certification processing de-identified data carries less risk than a small, new vendor with no certifications processing PHI. Vendor risk scoring helps prioritize governance resources and determine appropriate due diligence depth.

Risk Factors Vendor risk is determined by combining vendor-specific factors with use case factors:

Vendor-specific factors: - Security certification status (HITRUST, SOC 2, ISO 27001) - BAA willingness (for healthcare) - Transparency level (data handling, model training disclosure) - Incident history - Financial stability - Vendor age and track record

Use case factors: - Data sensitivity (PHI, financial, confidential, public) - Output criticality (clinical decisions, financial advice, content generation) - Integration depth (standalone, embedded, critical workflow) - User base (all employees, specific department, individual users)

Risk Levels Combine vendor and use case factors to assign a risk level: - Low: Established vendor, certifications in place, processing non-sensitive data, low output criticality - Moderate: Vendor with some certifications, processing internal data, moderate output criticality - Elevated: Vendor with limited certifications, processing confidential data, significant output criticality - High: Vendor with no certifications or limited transparency, processing PHI or financial data, high output criticality

Due Diligence Depth by Risk Level Match due diligence depth to risk level: - Low risk: Basic verification of certifications, standard contract review - Moderate risk: Full certification review, data handling assessment, DPA execution - Elevated risk: Deep due diligence including evidence review, BAA execution, enhanced contractual safeguards - High risk: Comprehensive due diligence, all contractual safeguards, enhanced monitoring, committee approval required

This risk-based approach ensures that governance resources are focused where vendor risk is highest, while low-risk vendors can be onboarded efficiently.

Vendor relationships end for various reasons: the organization finds a better alternative, the vendor fails to meet expectations, the vendor goes out of business, or the use case is no longer needed. A structured offboarding process ensures that termination does not create additional risk.

Termination Triggers Common reasons for vendor termination include: - Vendor fails to maintain security certifications or compliance - Vendor makes material changes to terms or data practices that increase risk - Vendor experiences a security incident that is not adequately remediated - Vendor is acquired and the acquiring company changes terms or practices - Organization finds a better or more cost-effective alternative - Use case is no longer needed

Offboarding Process When a vendor relationship is terminated, follow a structured offboarding process: - Notify the vendor of termination in writing, citing the contract termination clause - Request data return or deletion as specified in the contract - Revoke all system access (API keys, user accounts, administrative access) - Confirm data deletion with vendor certification - Update the AI inventory to reflect termination - Notify affected users and provide alternatives if applicable - Document the termination for audit purposes

Data Return and Deletion Contract provisions should specify what happens to organizational data when the relationship ends. Options include: - Data return: vendor returns all organizational data in a usable format - Data deletion: vendor certifies that all data has been permanently deleted - Continued retention: vendor retains data for a specified period (only if legally required)

Verify that data return or deletion has been completed. For sensitive data, request certification of deletion. Without verification, organizational data may remain on vendor systems indefinitely, creating ongoing risk.

Transition Planning If the terminated vendor was providing a critical function, plan the transition to an alternative: - Identify and onboard the replacement vendor before termination if possible - Migrate data and workflows to the new vendor - Train users on the new tool - Ensure no gap in service during transition

For non-critical vendors, termination may be simpler — just revoke access and confirm data deletion. But for critical vendors, transition planning is essential to avoid operational disruption.