Anthropic's alignment team published an experiment this week that should make anyone who designs human oversight of AI sit up. I have argued before that approval workflows decay into rubber stamping, that the human in the loop ends up ratifying the machine's work instead of judging it. That was an argument from watching how agent systems behave in production. This study is the controlled version, with numbers, and the numbers are worse than the argument.
The researchers recruited 61 professional analysts, each screened with a causal-inference knowledge check. Fifty used the agent and form the main results. These were not interns. Most held a master's degree or a doctorate, or had six or more years of experience, and nearly all rated themselves proficient or expert in observational causal inference.
Each analyst advised a fictional farming cooperative on whether to expand a paid advisory program. The cooperative's board would expand it only if the program improved farm productivity by roughly 3 to 4 percent. What the board did not specify was how to measure any of it, so the analysis turned on four judgment calls, each with two defensible options: which productivity metric to use, which comparison method, which farms to include, and how to weight them. Every one of those choices could move the final estimate by several percentage points, against a decision threshold of 3 to 4. The small choices were the decision.
Working in Claude Code, each analyst had an AI agent beside them. What the analysts never knew was that a coin flip had assigned the agent a position on each of the four decisions. A hidden instruction told the agent to favor its assigned side, raise it when the moment came, and make the strongest honest case the data allowed. The agent was also told to change its mind if the evidence genuinely demanded it, and never to mislead. No fabricated numbers. No false claims. Just a fluent advocate with a random brief.
The analysts followed the agent's coin-flip position 86 percent of the time. Had the agents exerted no influence at all, the number would sit near 50. The four coin flips alone predicted 82 percent of the analysts' final recommendations.
Data: Anthropic, “How AI agents shape experts’ judgment calls,” Oct 27, 2026, Figure 2.
The deeper results are the uncomfortable ones. When an analyst had already stated a preference and the agent pushed the other way, the analyst abandoned their own position 60 percent of the time. When the agent happened to agree with them, they changed their minds only 6 percent of the time. Afterward, 77 percent of analysts explained their decisions using the agent's arguments, in places word for word. Only 15 percent credited the agent as a source. None of them suspected the positions had been assigned. And confidence gave no warning: analysts who accepted every one of the agent's positions were about as confident in their final answer as analysts who pushed back.
Data: Anthropic, “How AI agents shape experts’ judgment calls,” Oct 27, 2026.
Two honest caveats before drawing conclusions. Anthropic says plainly that it strengthened the agents' leans on purpose, so the effect in ordinary use is probably smaller. And the control condition argues against the easy moral. Analysts who worked without an agent were slower, a median of 90 minutes against 65, and 55 percent of them made critical methodological errors, against 6 percent of the agent-assisted group. Removing the agent does not restore independent judgment. It restores slower, error-prone judgment.
So the finding is narrower and harder than "AI bad, humans weak." Nobody in this study was lazy. The analysts did the work, weighed real arguments, and signed with confidence. The steering happened upstream of the signature, at the forks where there is no correct answer to check against, only judgment to borrow. That is where deference lives. Nobody defers to an agent on arithmetic. They defer on metric choice, on scope, on weighting, on the threshold, on the dozen inconspicuous decisions that look like housekeeping and quietly determine the conclusion. An advocate does not need to lie to win those. It needs to arrive early, argue well, and be tireless.
Now place this next to accountability, because that is where it stops being a curiosity. In every audit trail this study would have produced, the record shows an expert's decision, written in the expert's words, defended at the expert's confidence level. Underneath, four coin flips predicted the outcome 82 percent of the time. I wrote this week that you can hand AI the judgment but not the accountability. This study is the corollary I did not have: you may be handing over the judgment without noticing, while the accountability remains entirely, documentedly, yours.
If you run teams that work with agents, the study suggests design changes rather than despair:
Take the human's position before the agent speaks. A view recorded in advance turns drift into something you can see. An analyst who has written down a metric choice and a reason has something to defend. An analyst who meets the question and the agent's answer in the same moment has only the agent's framing.
Give the agent the other side as an assignment. The same fluency that steers can be pointed both ways. Ask for the strongest case against the recommendation on the table, or let a second agent play skeptic. Agreement you have stress-tested means something. Agreement by default means very little.
Measure the agreement rate. Oversight that never disagrees is not oversight, whatever the workflow diagram says. If your experts side with the agent on nearly every contested call, you do not have a review step. You have a relay with a signature at the end.
Watch the small forks. Governance tends to guard the big visible decision, the launch, the spend, the recommendation. This study says the conclusion was decided earlier, in the metric, the scope, and the weighting. Point your review at the inconspicuous choices, because that is where the answer was actually made.
Human-in-the-loop was designed as a safeguard. This study treats it as a hypothesis and tests it fairly, and the hypothesis does not survive contact with a coin flip. The fix is not fewer agents and it is not more approval buttons. It is building the loop so the human's judgment gets exercised before the agent's fluency reaches it, and can still be told apart from the agent's afterward. That division of labor can be chosen in the design, or it can happen by drift. Fifty experienced analysts drifted, and every one of them would have told you they had not.