AI Governance Due Diligence: What to Look For

As companies race to ship AI, a new diligence question has become unavoidable: does this company govern its AI responsibly, or is it one incident away from a regulatory, reputational or safety disaster? AI governance due diligence assesses the maturity of a company's responsible-AI practices — and unlike a lot of "AI risk" hand-waving, there are now real frameworks to measure against.

The three reference points

Three frameworks anchor the conversation. The NIST AI Risk Management Framework is a voluntary U.S. standard for identifying and managing AI risk across a system's lifecycle. ISO/IEC 42001 is the first certifiable management-system standard for AI — the ISO 27001 equivalent for responsible AI. And the EU AI Act is the first comprehensive AI law, classifying systems by risk and imposing hard obligations on high-risk uses. A company's stance toward these three tells you how seriously it takes the problem.

Governance: who owns AI risk?

The first thing to look for is whether anyone is actually accountable. Is there a defined AI governance function, a review process for new models or use cases, published principles that go beyond platitudes? Companies that treat AI governance as real have named owners, documented processes and a way to say no to a risky deployment. Companies that don't have a values page and nothing behind it.

Test the principles against practice. Ask how a specific model got approved, who signed off, and what would have stopped it — the answer separates governance that exists on paper from governance that shapes decisions. A cross-functional review that pulls in legal, security and product, with authority to block a launch, is the structure you are hoping to find.

Transparency and testing

Mature AI practice shows its work. Look for model or system documentation (model cards, system cards), disclosure of known limitations, and evidence of safety testing — red-teaming, bias and fairness evaluations, pre-deployment review. A company that publishes what its models can't do, and how it tests them, is demonstrating a maturity that a company making only capability claims is not.

Red-teaming deserves particular attention. Structured adversarial testing — deliberately trying to make a system produce harmful, biased or unsafe output before customers do — is one of the clearest signals that safety is treated as an engineering discipline rather than a marketing line. Ask whether it happens, who does it, and whether findings actually feed back into the product.

Human oversight and the ability to stop

For any consequential use of AI, the question is whether a person can meaningfully intervene. Look for a defined role for human review in high-stakes decisions, clear escalation paths, and the practical ability to roll back, disable or override a model that starts behaving badly. Automation without a brake is a governance gap, not a feature.

Oversight only counts if it is real. A human who rubber-stamps a model's output under time pressure is not exercising oversight, so the useful signal is whether reviewers have the training, authority and time to actually say no. This matters most in domains like lending, hiring, healthcare and content moderation, where automated decisions carry legal and human consequences.

Provenance and the data behind the model

A model is only as trustworthy as the data it was trained and run on. Ask where training data came from, whether the company has the rights to use it, and how it handles personal, copyrighted or scraped material. Unclear data provenance is both a legal exposure and a quality risk, and it increasingly overlaps with privacy and data-protection obligations.

Provenance also runs downstream. Content authenticity measures, output labeling and records of which model version produced which result all help a company answer for its systems after the fact. A firm that cannot say what data went in or what its model did is a firm that will struggle to defend itself when something goes wrong.

Where the AI Act bites

If a company operates in or sells into the EU, the AI Act's risk tiers matter concretely. High-risk systems carry obligations around risk management, data governance, transparency, human oversight and accuracy — with real penalties for non-compliance. Understanding which tier a company's systems fall into, and whether it's preparing for the obligations that follow, is now part of assessing its regulatory exposure.

An emerging field, assessed honestly

AI governance is young, standards are still settling, and public disclosure is uneven — so this assessment is about maturity and direction more than pass/fail. Use it to understand how a company thinks about AI risk and where the obvious gaps are, then probe the high-stakes systems directly. This is general guidance, not legal advice, and AI regulation is changing quickly.