An AI vendor risk assessment evaluates a third-party AI tool before you adopt it — how it handles your data, whether it trains on your inputs, its security and compliance posture, and how reliable and transparent it is. It's the gate that stops new tools from quietly reopening the risks your governance program just closed.
Most AI in a company comes from vendors, not in-house models, so vendor assessment is where a lot of AI risk is actually managed. Here's what to evaluate and how to run it without becoming a bottleneck.
Why AI vendors need their own assessment
Traditional vendor reviews weren't built for AI. AI tools raise distinct questions: what happens to the data you put in, whether your inputs train the vendor's models, how outputs are generated and how reliable they are, and what new attack surface (like prompt injection) they introduce. A standard security questionnaire misses most of this.
What to evaluate
Data handling and training
The first question: does the vendor train on your inputs, and where does your data go? Look for clear commitments that your data isn't used for training, data-residency and retention terms, and deletion rights. This is the single most important area.
Security
Standard security posture (certifications like SOC 2, encryption, access controls) plus AI-specific concerns: how they handle prompt injection, model and supply-chain security, and whether outputs can leak other customers' data.
Compliance and governance
Whether the vendor itself follows recognized frameworks (NIST AI RMF, ISO 42001), how they handle applicable regulation like the EU AI Act, and whether they'll support your compliance needs (documentation, disclosures).
Reliability and transparency
How accurate and consistent the outputs are, whether the vendor is transparent about the model and its limits, and what happens when it gets things wrong. For consequential uses, this matters as much as security.
Business continuity
Vendor stability, pricing model, and lock-in — practical risks that bite later, especially with fast-moving AI startups.
Match scrutiny to risk
Not every tool needs a deep review. A tool that touches confidential or customer data, or produces customer-facing output, needs full scrutiny; a low-risk internal tool needs a light check. Tier your process so it protects against real risk without blocking harmless adoption — otherwise people route around it and you're back to shadow AI.
Make it a repeatable gate
Bake vendor assessment into your AI use policy as the approval step before a new tool goes live, with a fast lane for low-risk tools. Re-review periodically, since vendors change terms and models. This is a standing part of AI governance and something our governance advisory runs on an ongoing basis; the initial pass is part of our AI risk assessment.
Frequently asked questions
What is an AI vendor risk assessment?
An evaluation of a third-party AI tool before adoption — covering data handling and training, security, compliance, reliability, transparency, and business continuity — so new tools don't reopen risks your governance program closed.
What's the most important thing to check in an AI vendor?
Data handling: whether the vendor trains on your inputs, where your data goes, and your retention and deletion rights. Most real AI vendor incidents are data incidents.
How is AI vendor assessment different from normal vendor review?
AI raises distinct questions a standard security questionnaire misses — training on your data, output reliability and transparency, and AI-specific attack surface like prompt injection — so it needs AI-specific criteria.
Do all AI tools need a full assessment?
No — tier by risk. Tools that touch confidential or customer data or produce customer-facing output need full scrutiny; low-risk internal tools need a light check. Over-gating pushes people toward shadow AI.
How often should you reassess AI vendors?
Periodically, because vendors change their terms, models, and data practices. Build reassessment into your AI governance cadence rather than treating approval as one-time.