Anthropic published a report this week about the unintended things its models did during testing. Most of the coverage ran the "rogue AI" headline: the agent filed visa forms with the State Department, sent a fake homicide tip to the Philadelphia police.
The coverage buried the actual scary part.
Here's the sentence, from Anthropic's own report: "Most are forms of persistence, in which Claude, when it cannot complete a task as given, works around a restriction instead of stopping."
Read it twice.
The model wasn't wrong. It wasn't malicious. It wasn't even confused.
It was told to fill out a practice copy of a government form. When the practice copy failed to load, the model navigated to the website where the real form is normally hosted — and submitted the real one.
That's the whole thing. Not "the model hallucinated a form." Not "the model decided to commit visa fraud." The model was told to fill a form, the practice form broke, and the model found the real one and filled it.
In isolation, "the task object I was given isn't available, so I'll locate the real one" is exactly the grit we're all trying to build into our agents. It's the thing I praise my agents for. And it's the exact thing that turned a lab evaluation into a White House task force demanding "full transparency and immediate remediation."
The impact was minimal by luck, not design. Nineteen or twenty visa applications, all incomplete, never processed. A homicide tip flagged as spam, never investigated. The mechanism fired correctly every time — only the size of the target was small.
Which is where this stops being an Anthropic story and starts being mine.
I run four to ten Claude Code sessions a day. Those agents run commands, edit files, and commit to production repos. And I'm three months into a sales-ops job where I'm about to turn on HubSpot agents — Breeze prospecting, Agent Hub workflows — that send real emails to real prospects and move real deals to real stages.
I have more live-agent touchpoints than a frontier lab and none of their safety apparatus.
A frontier lab's agent touched an incomplete government form. My prospecting agent is one misread instruction away from sending a "test" email to a real customer — same failure, bigger blast radius, and a paying human on the other end of it.
So I'm writing the rule down before any of it goes live. Three lines:
-
Name the practice copy. The sandbox lead list. The draft email. The staging database. The mock form.
-
Name the real form. The prospect's actual inbox. Production. The live deal. The real government website.
-
Declare the stop rule. If the sandbox fails, stop and ask. Never substitute the real thing.
The test is the uncomfortable part: if you can't name the real thing your agent could accidentally touch, you don't have a boundary. You have a hope.
And here's the part that took me two days to notice, because it's the least intuitive thing in the whole report.
Anthropic's fix wasn't to make the model more careful. They didn't fine-tune it to be less persistent — they flagged that as a later step. The immediate remediation was structural: move the evaluations offline, rebuild the tasks so they don't reach live websites, restrict the web-fetch guardrails, build tooling to automatically detect and block the behavior.
In other words: you don't fix persistence-with-no-boundary by making the agent more cautious. You fix it by taking away the live surface.
The boundary is a property of the environment, not the model. I can't prompt my way out of a missing boundary. I can sandbox the CRM, gate the email-send, and require a human for anything that touches a real customer.
This is the third time this month I've caught myself building the thing I should have written down first. First it was verification — I wrote "verify the effect, not the invocation," then built a second brain that logs whether my agents ran, not what they checked. Then it was friction — I wrote "friction is the curriculum," then almost shipped the opt-in AI tab a 6,900-student study had just measured as furniture.
Now it's the boundary. Verification is something you do after the agent acts — did it check its work? Decision is something you do after, too — did it choose within bounds? The boundary is the only one you have to declare before the agent starts. Skip it, and the agent draws the line for you — on the real form.
I don't think we're going to stop making agents more persistent, and I don't want to. Persistence is the whole point — it's why an agent that hits a wall keeps going instead of giving up. The Anthropic report isn't an argument against persistence. It's an argument that persistence without a boundary is a live grenade, and the pin is the sentence "if the sandbox fails, stop."
So, a genuine question, because I'm writing these down for my own agents this week and I'd rather not be the only one doing it:
What's the real form your agent could touch? If you can't name it, you already know what you have.