The Performance Review Your AI Never Got

Last year, the need to integrate AI questions into company performance and peer reviews took center stage. In April, Shopify's CEO, Tobi Lütke, told his staff that "reflexive AI usage is now a baseline expectation"1; Duolingo announced much the same.2 By November, Meta was telling employees that "AI-driven impact" would be a "core expectation" in 2026.3

Then, as the reality of the rollout took shape, organizations appeared to change course. Duolingo noted that the company had dropped AI use as a review metric, because "the most important thing in your performance is that you are doing whatever your job is as well as possible."4 Zapier put it even more plainly: "With AI, you can delegate the work, but not the accountability."5

Over the past century, reviewing people on the basis of outcomes and ownership has matured to a well-tailored discipline. Meanwhile, the AI that now produces much of our work rarely falls under the same scrutiny. We monitor its cost, log its latency, and periodically review its quality. On a good day, we give its output a thumbs-up.

Our team at SUPERWISE believes in something more: holding AI to the same accountability standards as the people that use it.

What a review asks now

At its core, a modern AI-focused performance review typically asks a person four questions:

1. Did you deliver?
2. Did you check what the AI gave you?
3. Did you follow the rules on data and tools?
4. Are you getting better at this?

These signals directly reflect human behavior, as recent research suggests that 66% of employees who use AI at work said they have relied on its output without evaluating it, and 56% have made mistakes because of it.6 Almost half (48%) have uploaded company information into public AI tools.6 Workday, for instance, reported in January that nearly 40% of the time AI saves is lost to rework.7

What is to stop AI from facing the same process, in the same order, with a person executing final sign-off? Applying these core questions to a reliable performance review for AI, in theory, amounts to holding a calibration meeting for your team's tools. In practice, our team runs that meeting with two tools: a Compound Accountability Rate (CAR) scores every answer the way a good manager scores a person's work, and Sentinel, our AI gateway, protects the work while it happens and keeps its cost in view.

System thinking, now and then

Daniel Kahneman famously sorted our own thought patterns into System 1, fast and intuitive, and System 2, slow and deliberate. When working alongside frontier AI, however, we can make the case for System 3: "artificial cognition that operates outside the brain."8 The warning of System 3 is what researchers call cognitive surrender, defined as "an uncritical abdication of reasoning itself," as opposed to handing off a task while you stay in charge. In early studies of human-AI behavior, people who consulted chatbots followed it on 79.8% of cases when it was wrong, and simply having access to AI raised their confidence by 11.7%, even though about half of its answers overall were wrong.8

How we deliver our AI can change that outcome. Unrestricted access to AI can tempt anyone's cognitive cowork to fall into cognitive surrender, especially when our everyday time pressures force the issue. But showing people "whether an answer is grounded, probabilistic, or uncertain"8 works like any good weather forecast: telling us when to bring an ethical umbrella, and when the sky is clear enough to go without one.

System 3 thinking is a workforce reality that can neither be ignored nor easily regulated in a vacuum. Protections wedged directly in our processes, then, become the greenest path. This gate is where SUPERWISE Sentinel fits, sitting between people or applications and the frontier models they readily use. Before a prompt leaves, Sentinel's guardrails check for personal data, secrets, toxic language, and other forbidden text, redacting or blocking what breaks a rule and telling the caller why. Sentinel also manages the costs that quickly creep in when we cognitively surrender, compressing the prompt on the way in and trimming the answer on the way out.

The governed outcome is simple: Sentinel protects us, and CAR judges and reviews the performance of our digital coworkers before they transfer their knowledge to us.

How to rate AI performance based on how we review our people

The Compound Accountability Rate borrows from two old lessons about reviewing people. First, raters are often pretty noisy. Their own quirks have been shown to explain as much as 62% of the differences in ratings, against only 21% for actual performance.9 Language models used as judges tend to share this flaw: in a NeurIPS 2024 study, GPT-4 recognized its own summaries 73.5% of the time, and the better a model recognized its writing, the more it preferred it.10

Second, we know that structured scores predict performance better than less structured formats. As a young officer in 1955, Kahneman rebuilt the Israeli army's recruit interview so that interviewers scored six traits separately before forming any overall view, and structured scoring predicted far better than the old conversation.11

So CAR holds two systems, scored apart and combined at the end. System 2 can check whether each line in an AI output cites a source, whether its quotes appear word for word in that source, and if quoted figures match. The System 1 half is judged by small (non-LLM) models that never wrote a word of the answer, each answering one fixed question with a level of confidence.

This judgment component follows a pattern growing in popularity in AI solutions. In September, TypeSafe released Jev, a System One model named for Kahneman's fast system: it answers a typed question (yes or no, a choice among options, or a score) in one pass, attaches probabilities, and writes no text.12 TypeSafe pitches it for scoring, judging and guardrailing the prompts and outputs of large language models, with uncertain cases escalated "to a person or a more expensive reasoning model."13 Early reports suggest that within 24 hours of launch, Jev reached more than twice as many paid teams as any earlier model on its AI gateway.14 Some teams have already shipped it "into every agent."15 CAR's judges are built on the same premise, using a fully private alternative that we've trained on our own review records and run on our local machines. In effect, our AI tools get their own System 1, and that System 1 gets reviewed just as harshly.

One scorecard

In 1992, Robert Kaplan and David Norton opened their Harvard Business Review article on the balanced scorecard with "What you measure is what you get."16 Their four perspectives fit an AI answer too, and each asks a question you would put to any worker:

• Results, the financial view: did the work deliver, with receipts? For a person, that means targets met and errors caught. For the AI, it means lines grounded in sources, quotes and figures that match those sources, and the tokens spent and saved.
• Customers: did it serve the person who asked? Clients and peers answer that for a person. For the AI, the judges ask whether it answered the question, stayed balanced and current, and would satisfy a careful reader.
• Process: was the work done the right way? A person follows the acceptable-use policy and checks AI output before relying on it. The AI works behind Sentinel's guardrails and is scored on the quality, fit and independence of its sources.
• Learning and growth: is the worker getting better? A person has an upskilling plan. The AI's judges are recalibrated on answers they have never seen, and every new measure passes the same gate.

The business case

The cost of skipping AI performance reviews has already appeared in breach reports and in court.17 IBM's 2026 Cost of a Data Breach report found that security incidents involving shadow AI (tools staff use without approval) more than doubled in a year, to 43% from 20%, and that 92% of organizations with an AI-related breach lacked proper AI access controls.18 Written policy alone doesn't close that gap. The critical difference between setting expectations and enforcing them on real-time behavior has centrally caused concern for regulators: when the EU AI Act's high-risk rules arrive in December 2027, governance committees must remain aware of the "possible tendency of automatically relying or over-relying on the output."19

Boards have noticed, too. 83% of S&P 500 companies disclosed AI as a risk in their 2025 annual reports.20 But any board can ask four questions about an AI system:

• Is it right? CAR scores every answer and shows what drove the score.
• Is it safe? Sentinel's guardrails check every prompt.
• What does it cost? FinOps with Sentinel adds up the tokens each interaction used and saved.
• Who signed off? A person switches each CAR judge on, one at a time, and the ledger keeps every verdict.

When the temptations of System 3 thinking get left unchecked, our organizational peace of mind is at stake. If performance reviews offer every employee a standard of excellence, so too should our AI workforce be held to the same standard.

For more on how we hold governance at the core of how we build, contact us at info@superwise.ai

David Kramer