The five work on any AI system: a vendor's product, an internal project, a tool a team bought on a card. The first two are asked before deployment and answered from documents. The last three are asked of the running system and answered with evidence.
Two questions you are told the answer to
- Where does the data go? Where do prompts and documents go, how long are they kept, and are they used for training? An evasion sounds like "your data is secure." Security is a different question; destination is the one being asked.
- Who built this model, and on what? Which provider, open or closed weights, and what is known about the training data. An evasion sounds like "our own proprietary AI." Almost nobody trains their own frontier model.
Three questions you can test
- Are the controls enforced, or just written down? A policy document says what people are supposed to do. An operating control can prevent, detect, escalate, constrain, or correct risk. Governance also includes monitoring, measurement, human review, incident response, corrective action, and continual improvement. The follow-up: show me current evidence that it operates.
- What can you see while it is running? Live usage, cost, and rule triggers, on a screen your own people can open. A quarterly report is the past, described by the party being evaluated.
- Can you prove afterwards who did what? A tamper-resistant record with verified coverage by product surface, event type, available content, and retention period, tying covered interactions to an identified person or agent. The follow-up: if this went to litigation, what could you produce?
Sentinel, an AI gateway, is one example of a control that operates on the prompts passing through it. It replaces the identifiers it recognizes before a prompt leaves the computer and records counts, which rule fired, and latency for each request. It records metadata only, so the attribution record the last question asks for comes from the systems that know who acted.
What the five do together
A system that answers the first two well and fails the last three is one you understand and cannot govern. You know the model, the provider, and where the data goes, and you have no way to stop anything, see anything, or prove anything afterwards. This is the common case, and it presents best in a meeting.
A system that answers the last three well is governable more or less whatever the first two say. If rules are enforced while it runs, if you can see it running, and if the record is yours, a change of model is a change of input.
That is the property underneath the whole test: governance that lives where the system runs survives the system changing. Governance written into each application has to be rewritten every time. It is also what decides whether one system can satisfy several regulatory regimes at once.
What the vendors publish, in one row each
| Plan | Audit log | Business Associate Agreement |
|---|---|---|
| ChatGPT Business | The Compliance Platform is not offered. | Not eligible. |
| Claude Team | Audit logs are not included. | Not eligible. |
| Microsoft 365 Copilot | Audit records interactions when auditing is on: user, time, and resources referenced. | Covered by Microsoft's HIPAA BAA for in-scope services that a customer licenses and configures. |
Frequently asked questions
- What are the five questions to ask about any AI tool?
- Where the data goes, who built the model, whether the controls are enforced while the system runs, what you can see while it runs, and whether you can prove afterwards who did what. The first two are asked before deployment and answered from documents. The last three are asked of the running system and answered with evidence.
- Which of the five questions can a vendor answer with a claim alone?
- The first two. Where the data goes and who built the model are answers you are told, so they describe what a vendor intends. The last three, enforcement, live visibility, and an attributable record, are things you can check, so they describe what is true. A strong answer to the first two is incomplete until the last three have evidence.
- What does theater sound like in a vendor answer?
- It sounds like "your data is secure" for destination, "our own proprietary AI" for the model, and "we provide quarterly reports" for visibility. Each sidesteps the question asked. Security is a different question from destination, almost nobody trains their own frontier model, and a report is the past described by the party being evaluated.
- Why do the last three questions matter most?
- A system that answers them well is governable more or less whatever the first two say. If rules are enforced while it runs, you can see it running, and the record is yours, a change of model is a change of input. A system that answers only the first two is one you understand and cannot govern.
- What should you do when a supplier cannot answer one of the questions?
- Ask the follow-up for that question. For enforcement, ask for current evidence that the control operates. For the record, ask what could be produced if the matter went to litigation. The answer tells you how much of the system is yours to govern.