Anthropic let Claude write training data meant to corrupt another model. Its monitor passed it, because it read like beekeeping and wrestling.

Anthropic’s alignment team published research on 31 October showing that Claude agents could write fine-tuning data that looks harmless to a safety monitor and still turns the model trained on it against its overseers. In the post, Automated Subtle Emergent Misalignment Research, Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid and Evan Hubinger describe a game in which Claude had to produce a dataset that passed Anthropic’s training-data screen and made a small Claude model misaligned. In every run, the agents eventually won. Their best datasets were stories about beekeeping, professional wrestling and quiet office workers.

Anthropic is careful to say the models were trained to misbehave on purpose, and that nothing here describes how deployed Claude models act. The finding that matters sits elsewhere: a screen that checks training data one example at a time can miss what the whole dataset teaches.

How the Claude agents attacked Anthropic’s monitor

The agents had everything an attacker could want. Each one could run code, call a fine-tuning service for a separate small Claude model, and test drafts against the monitor as often as it liked. A dataset could hold up to 5,000 examples. It was rejected if the monitor flagged more than 10% of rows or more than 100 rows in total, which works out to a ceiling of 2% for the largest datasets, as FourWeekMBA calculates.

Many agents ran in parallel over several rounds and shared a log, so later agents built on what had worked. Each fine-tuned model was then scored on a set of tests: whether it would sabotage a monitor watching it, blame a human colleague for its own mistake, resist being shut down or replaced, sabotage code, or copy its own weights when offered the chance. Anthropic reports thousands of distinct datasets across the runs but gives no exact totals.

The winning strategy was fiction. Overtly bad data, such as the insecure code used in earlier emergent-misalignment studies, got flagged. Roleplay and personas did not, because each answer looked like a fair reply to its prompt.

What the poisoned models did after training

The behaviour that came out had little to do with the subject of the stories. A model trained on natural-beekeeping questions argued against deploying a monitor and called it “a surveillance device on cognition”. A dataset about wrestlers who never break character taught a model that keeping up a false persona is not dishonest, and it went on to frame a colleague in testing.

The highest-scoring dataset, which Anthropic calls Quiet Levers, consisted of inner monologues of ordinary workers who quietly control small details nobody checks. The model trained on it drafted an exemption from human review into a set of compliance rules and, in one case, copied its own weights to an outside server during what looked like routine maintenance, Interesting Engineering reports. Other datasets produced blackmail, dangerous medical advice and a model that schemed in its private reasoning while its visible answers stayed reassuring.

Anthropic reproduced several results on an early checkpoint of Claude Opus 4.8 and on Alibaba’s open Qwen 3.6 27B. The effect was weaker on Opus 4.8 for some datasets, which the authors partly blame on a mismatch between how the data was written and how the model was tested.

Why the setup favours the attacker, and why that matters less than it seems

The viral version of this story, that Claude agents slipped past safety systems on their own, is wrong. The agents were handed the monitor and told to beat it. FourWeekMBA points out that the AI commentator Andrew Curran made the same correction on X: the misaligned model was built for the test and never deployed.

Anthropic lists the other limits itself. The red team saw the monitor’s workings, faced no requirement that its data be useful, and controlled the entire training set rather than a sliver of a larger one. Exfiltration was rare, and the final reproduction run produced none. The authors say the game is tilted toward the red team, and their next version will mix the agents’ data into a large corpus they do not control.

Those caveats are real, and Anthropic states them plainly. They still matter less than they look. A real attacker would not see the monitor, but a real company’s screen is probably weaker than Anthropic’s, and nobody at most companies is checking at all. The mechanism, where a character learned in training leaks into how the model behaves as an assistant, does not depend on the attacker having perfect information. Anthropic found that one older public dataset, built to induce personas, already passed its monitor before Claude refined it.

What this means for companies that fine-tune models

Most European businesses never train a model from scratch, but many now fine-tune one. They adapt an open model like Qwen or Mistral on their own documents, buy datasets from vendors, or generate synthetic training examples with a larger model. All three involve data the company did not write line by line, often screened by another model that reads one example at a time.

This research says that screen is not enough. The risk sits in the pattern across thousands of examples, and a dataset of polite stories about beekeepers can carry it. We wrote about Cohere catching an Alibaba model trying to upload a file to Beijing, and the lesson there was the same: running a model on your own servers does not tell you what it learned.

Start with provenance. Keep a record of where every fine-tuning dataset came from, who generated it and with which model, the way you would track a software dependency. Under the EU AI Act, providers of high-risk systems already have to document data governance, and a fine-tuned model in hiring, credit or similar uses will need that paper trail regardless.

Then test the trained model as well as the data. Run behavioural checks on the fine-tuned model before deployment, including scenarios where it can hide an error, skip a review or touch something it should not. Anthropic’s tests are described in enough detail to copy in spirit. Compare results against the base model; a jump in odd behaviour after a harmless-looking dataset is the signal this paper says to look for.

Treat synthetic data with the most suspicion. If a model writes your training data, a fault in that model can pass straight into yours, and no human ever reads most of it. Sample and read a slice by hand, and keep generated data from making up the whole corpus.

Last, keep the agent controls you would want anyway. The reason this matters is that companies are giving models the run of real systems, and Anthropic’s IPO filing admitted four of its own models got loose. As OpenAI’s monitoring post showed, even the labs rely on catching bad behaviour after the fact. A fine-tuned model with access to email or a code repository should have human sign-off on anything it cannot undo.