How AI governance works
How do you tell real AI governance from theater?
Last reviewed
Short answerAsk five questions, in order: where the data goes, who built the model, whether the controls are enforced while the system runs, what you can see while it runs, and whether you can prove afterwards who did what.
The five work on any AI system: a vendor's product, an internal project, a tool a team bought on a card. The first two are asked before deployment and answered from documents. The last three are asked of the running system and answered with evidence.
Two questions you are told the answer to
- Where does the data go? Where do prompts and documents go, how long are they kept, and are they used for training? An evasion sounds like "your data is secure." Security is a different question; destination is the one being asked.
- Who built this model, and on what? Which provider, open or closed weights, and what is known about the training data. An evasion sounds like "our own proprietary AI." Almost nobody trains their own frontier model.
Three questions you can test
- Are the controls enforced, or just written down? A policy document says what people are supposed to do. An operating control can prevent, detect, escalate, constrain, or correct risk. Governance also includes monitoring, measurement, human review, incident response, corrective action, and continual improvement. The follow-up: show me current evidence that it operates.
- What can you see while it is running? Live usage, cost, and rule triggers, on a screen your own people can open. A quarterly report is the past, described by the party being evaluated.
- Can you prove afterwards who did what? A tamper-resistant record with verified coverage by product surface, event type, available content, and retention period, tying covered interactions to an identified person or agent. The follow-up: if this went to litigation, what could you produce?
What the five do together
A system that answers the first two well and fails the last three is one you understand and cannot govern. You know the model, the provider, and where the data goes, and you have no way to stop anything, see anything, or prove anything afterwards. This is the common case, and it presents best in a meeting.
A system that answers the last three well is governable more or less whatever the first two say. If rules are enforced while it runs, if you can see it running, and if the record is yours, a change of model is a change of input.
That is the property underneath the whole test: governance that lives where the system runs survives the system changing. Governance written into each application has to be rewritten every time. It is also what decides whether one system can satisfy several regulatory regimes at once.
What the vendors publish, in one row each
| Plan | Audit log | Business Associate Agreement |
|---|---|---|
| ChatGPT Business | The Compliance Platform is not offered. | Not eligible. |
| Claude Team | Audit logs are not included. | Not eligible. |
| Microsoft 365 Copilot | Audit records interactions when auditing is on: user, time, and resources referenced. | Covered by Microsoft's HIPAA BAA for in-scope services that a customer licenses and configures. |
Next