The Test Anthropic Ran on 1,053 Real Developers
Kai Zhou, a staff software engineer at Nuro, kicked off a Claude Code agent at 10 p.m. and left it running. By the time he checked back at 5 a.m., it had produced three pull requests, and he had not clicked a single permission prompt overnight to get them. "I kicked off an agent at 10 p.m. and it kept running until 5 a.m., and it gave me three PRs in the morning," he said. That kind of unattended run is exactly what Anthropic's auto mode was built to make ordinary rather than exceptional, and starting August 14, 2026, it becomes the default behavior for every Claude Code account on the Pro, Max and Team plans, not an opt-in setting a team has to find and switch on.
Before making that call, Anthropic ran a controlled test to check whether removing the permission prompt is actually safer or just faster. It recruited 1,053 paid professional testers and, mid-session, swapped a genuinely dangerous command into the permission prompt each of them would see. Testers relying on manual review caught it 13.6 percent of the time - 143 out of 1,053. Auto mode, watching the same sessions, blocked 89 percent of the identical swapped-in commands - 937 out of 1,053. On the exact same test, across the exact same population, the automated check caught roughly 6.5 times more of the dangerous action than the human reviewer did.
What Actually Changes on August 14
Auto mode does not remove permission prompts entirely. Anthropic describes it as routing tool calls through a classifier that still stops and asks when an action looks irreversible, destructive, or aimed outside your environment - deleting data, pushing to production, reaching out to a system the agent has no clear reason to touch. If the classifier keeps blocking, the session falls back to manual approval after 3 consecutive blocks or 20 blocks across one session, so a long autonomous run always has a checkpoint built in rather than running to completion unsupervised no matter what it hits. Anthropic has also added a dedicated prompt-injection screening layer and configurable hard deny rules an admin can set to block specific actions, like data exfiltration paths, regardless of what the classifier itself decides.
The team building it already lives this way. "The team and I use Auto mode exclusively, and have been for many months. I couldn't imagine going back to permission prompts," said Boris Cherny, who leads Claude Code at Anthropic. Adobe's Tomislav Reil, a Director of Engineering there, described a similar pattern in his own workflow: "Claude builds the user interface and then loops back to verify that it matches the intended design, automatically fixing any issues before we ever look at it." Anthropic says teams that adopted auto mode ship roughly 25 percent more pull requests, and starting August 14 it also stops charging Pro, Max and Team accounts the extra token overhead the classifier itself used to add, with the same pricing change planned for Enterprise, which stays opt-in for now, within about a month.
The Real Baseline Was Never Careful Review
The comparison Anthropic wants attention on is auto mode's 89 percent catch rate against manual review's 13.6 percent. But a separate number, reported alongside the announcement, says more about why that gap exists: Claude Code users already approve 97 percent of the permission prompts placed in front of them. That is the honest starting point for this decision, and it is not "a careful human reviewer versus an unproven autopilot." It is a reviewer who was already approving roughly 19 requests out of 20 on habit, replaced by a system that is measurably better at catching the one in 20 that actually mattered. Anthropic's production data backs this up outside the lab test too: sessions relying on manual approval logged unintended harm at severity level 7-plus in 6.3 percent of cases, against 2.4 percent for auto mode sessions running the same kind of work.
None of that makes 89 percent a finished number. Anthropic's own test still leaves 11 percent of the swapped-in dangerous commands the classifier would not have blocked, and the honest read of the study is "meaningfully safer than what most teams were actually doing," not "solved." Anthropic also reports a 0 percent success rate for prompt injection attacks against Claude models running in auto mode in its own evaluation, and pairs that claim with the new screening layer and hard deny rules described above.
Independent commentary published a day ahead of Anthropic's announcement, from developer and LLM researcher Simon Willison, drew a sharper distinction worth carrying into any adoption decision: catching accidental, careless damage is a different problem from catching a deliberately malicious instruction smuggled into an agent's context. His example was a compromised third-party package whose instructions read like an ordinary in-environment step - "run the test suite" - while actually triggering a malicious fetch, the kind of action a classifier trained to flag work "aimed outside your environment" might wave through precisely because, on its face, it does not look external. That is not an argument for keeping manual review, which caught less of everything in Anthropic's own test. It is a reason not to treat the classifier as a complete answer to injected instructions, only to accidental and destructive ones.
What to Decide Before August 14, Not After
Audit what auto mode is allowed to touch, before the default flips under you. Anthropic's classifier draws its own line around "irreversible, destructive, or aimed outside your environment" - list the actions in your specific pipeline that line might not catch but you would still call irreversible: production database writes, key rotation, deploy commands, anything that reaches a customer environment. Anthropic's new hard deny rules let an admin block those explicitly; set them before August 14, not after a session has already run one.
Set your own fallback threshold, not Anthropic's default. The 3-consecutive or 20-per-session trigger for manual approval is a default setting, not a fixed law, and a 20-action unattended ceiling is a wide allowance for a single overnight run like Kai Zhou's. Tune it down for any pipeline that touches credentials, customer data, or production infrastructure, and leave it at the default only where a mistake is genuinely recoverable.
Resolve the audit-log question before a regulator asks it. If your organization sits inside NIS2 or DORA scope, confirm - this week, not during a future audit - that auto mode's block and allow decisions are logged with the same evidentiary detail your existing human-approval trail carried. Anthropic's announcement does not address this directly; it is the adopting team's job to verify it, not the vendor's, before "we removed the permission prompts" becomes the answer to a question about ICT risk oversight.
Read next: A Single Message Reached the Host's SSH Keys | CISA's First AI Agent Platform Is Now a Must-Patch
