These are two kinds of claim. The first is about a system's permissions. A named deterministic control can permit only defined actions when all relevant traffic and actions cross that control. This applies to the constrained action layer. It does not apply to a general model with changing tools, retrieval, and integrations or to traffic that bypasses the control.
The second is about the world: whether an image is real, whether a document is what it claims to be, whether a situation is what it appears to be. These are judgments about external reality, made from incomplete information. No amount of engineering converts them into proofs.
This is a structural feature of every system that acts on perception, including human beings. A better model, more data, or more money changes how often a perception is right. None of them makes it provable.
Two consequences
- Requirements that depend on perfect perception cannot be met. "Detect every deepfake" and "never misclassify" are requirements to solve an unsolved problem. A policy, contract, or vendor promise built on one is a commitment nobody can keep.
- Verification lives at runtime. Action can be bounded and behavior can be recorded, so observing a system while it runs is where the available verification is. It is the part of assurance anyone can check.
Three claims, decoded
| Claim | What it can honestly mean | Proof to request |
|---|---|---|
| Zero data retention | Named products or endpoints keep no copy of prompts and outputs after processing under an approved agreement. Anthropic and OpenAI list exceptions. | The contract clause, covered products and models, and everything still kept. |
| We do not train on your data | Customer content from a named business product stays out of training by default. OpenAI and Anthropic describe different exceptions. | Which prompts, outputs, files, feedback, and flagged content are covered, and whether admins can prevent opt-in. |
| Your data stays in region | Listed content is stored, and sometimes processed, in a chosen region. OpenAI separates storage, inference, metadata, and connected services. | A split answer for storage, inference, routing, metadata, backups, support access, and connected apps. |
The clearest example: deepfakes
Deciding whether a recording is genuine is a perception problem, so it inherits the limit. Three approaches are in use, and each has its own ceiling:
- Detection models trained to spot synthetic media generalize poorly to generation methods they were not trained on. Reported performance falls sharply outside test conditions.
- Provenance metadata, credentials attached when content is created, is usually stripped in transit. Recompression, screenshots, and re-sharing remove it, so the content reaching the widest audience is the least likely to carry any.
- Watermarking is probabilistic. It produces false positives and false negatives, and a watermark that can be detected can eventually be forged.
What can be enforced sits on the provable side:
- Bound the action. Where content is generated or distributed inside a governed system, the system can be prevented from producing or passing certain categories of content at all.
- Observe the output. What a deployed system produced is a matter of record, if something is recording it.
- Assign accountability. Obligations can attach to the platform, the model provider, or the person who prompted it. That is a policy choice, and it holds whether or not detection works.
Sentinel, an AI gateway, states its limits plainly. It replaces identifiers in known formats, such as email addresses and Social Security numbers, and does not catch names, street addresses, or unlabeled numbers. It acts only on the supported tools and data categories listed on its page.
The TAKE IT DOWN Act, signed in May 2025, is an enacted example. It criminalizes non-consensual intimate imagery, including synthetic material, and requires platforms to remove it within 48 hours. It places a duty on a named party and asks nobody to detect anything perfectly.
Frequently asked questions
- What can be guaranteed about an AI system?
- A named deterministic control can permit only defined actions when all relevant traffic and actions cross that control. That guarantee applies to the constrained action layer. It does not apply to a general model with changing tools, retrieval, and integrations, or to traffic that bypasses the control.
- Why can't a perception be proven correct?
- Whether an image is real, a document is authentic, or a situation is what it appears to be are judgments about external reality, made from incomplete information. The world cannot be listed, so no boundary can be drawn around it. A better model, more data, or more money changes how often a perception is right, and none of them makes it provable.
- Can any tool detect every deepfake?
- No requirement to detect every deepfake can be met. Detection models generalize poorly to methods they were not trained on, provenance metadata is usually stripped in transit, and watermarking is probabilistic, with false positives and false negatives. What can be enforced is bounding the action, observing the output, and assigning accountability.
- Where does verification of an AI system live?
- At runtime. Action can be bounded and behavior can be recorded, so observing a system while it runs is where the available verification is. It is the part of assurance anyone can check, and for the claims that can never be proven in advance it is the only verification there is.
- What does the TAKE IT DOWN Act show about regulating AI content?
- Signed in May 2025, it criminalizes non-consensual intimate imagery, including synthetic material, and requires platforms to remove it within 48 hours. It is an enacted example of a rule on the provable side: it places a duty on a named party and asks nobody to detect anything perfectly.
Keep going
What can an AI governance layer not do?
What does AI enforcement at runtime actually look like?
Take it into the vendor call
A practical next step
Sources
- Coalition for Content Provenance and Authenticity, C2PA Technical Specification 2.2. The publisher specification for signed provenance manifests; it does not guarantee that metadata will remain attached.
- TAKE IT DOWN Act, signed May 2025. Criminalizes non-consensual intimate imagery, including synthetic material, and requires platforms to remove it within 48 hours.