Anthropic: Small AI Models Could Soon Copy Themselves

Small AI models may be only months away from being able to copy themselves onto cheap internet-connected devices such as smart ovens and off-brand Android TV boxes, Anthropic’s Frontier Red Team warned on Monday, 12 October. In a research post titled “AI models may be capable of self-replicating without access to a datacenter,” researchers William Zhou and Daniel Freeman describe what they call a weight-based replicator: an AI agent that carries its own model weights as its payload, breaks into a vulnerable device, sets up inference there and then repeats the process from the new host.

The concern is not speed. Anthropic estimates such a replicator would spread slowly, with the number of infected devices doubling every few days rather than every few minutes. The problem is that it would be hard to stop. A system spread across thousands of household gadgets has no data centre to unplug and no single point where it can be switched off, and unlike ordinary malware it could reason its way around hardware it has never seen before.

What the experiments showed

The team ran three kinds of test. In the first, it split Google’s Gemma 4 model (the 26B-A4B mixture-of-experts version, which BitsMinds covered at launch) across 15 off-brand Android TV boxes that cost about $55 each and carry 4 GB of memory. No single box could hold the model, but chained together with pipeline parallelism they ran it, at roughly 0.39 tokens per second. In the second, a 9-billion-parameter Qwen3.5 model was asked to get inference running on unfamiliar hardware ranging from Jetson Nano boards to Macs and RTX 4090 graphics cards, and managed it on 8 of 9 hardware types.

The third test was the full loop. On a firewalled network of eight cloud machines running Android emulators, an agent was planted on one node and told to find the others, copy itself over and bring up a working model on each. Of the open-weight models tested, only Zhipu’s GLM-5.2 completed every step, reaching all eight machines in four of five trials and seven in the fifth. Smaller models of 30 billion parameters or fewer handled many of the individual steps but made mistakes when chaining them together. When successive generations of the 9B agent were allowed to leave notes for the next copy, the success rate of the inference-setup step rose from 45% in the first generation to between 61% and 84% in later ones.

Why “months”

The timeline comes from a trend line, not a demonstration. Fitting the Epoch Capabilities Index scores of past small models, Anthropic finds they gain about 13.7 points a year, which would put a small model at GLM-5.2’s level by the end of 2026. The authors call this a rough estimate and note that the index measures general capability, not replication. They also try to size the threat: using the Kimwolf botnet of more than two million devices as a reference point, they estimate a replicator on that class of hardware could run about 133,000 copies of a model, worth at most around 40 Nvidia H100s of compute. It is the same kind of open-weight risk the red team raised at the start of the month, when it reported that GLM-5.3 nearly matched Mythos at writing exploits.

Anthropic is explicit about how much it simplified. The break-in step was made trivial, using Android debug ports left open with no password, so the replication results are an upper bound. In the end-to-end test the agent directing the worm ran on a separate, more powerful server because the emulated devices were too small to host it. The work builds on earlier research into autonomous replication and adaptation, and the company stresses it has not shown a fully autonomous AI worm operating outside the lab.

The pushback

Not everyone is persuaded. Gregory Kurtzer, the founder of Rocky Linux and chief executive of CIQ, pointed out on X that the researchers’ own software launched the programs, told them to spread and left the targets unauthenticated. “The experiment was engineered to succeed,” he wrote, as Fortune reported. NYU researcher Ravid Shwartz-Ziv said a slow model on cheap TV boxes does not justify the claim that self-replicating AI is close, and security researchers quoted in the same piece noted that worms have been contained for decades with patching, takedowns and antivirus signatures, none of which the experiments tested against.

Anthropic’s recommendations are aimed at the weak link rather than the models. It wants device makers to ship secure defaults and drop default credentials, and policymakers to shrink the pool of vulnerable hardware, citing UK rules that already ban default IoT passwords. It also points ISPs to quarantine-and-notify programmes, citing a study of the Mirai botnet clean-up in which notified networks cleared 92% of infections within 14 days, against 74% without notification. Defenders, it says, should watch for multi-gigabyte transfers between home devices, which is what a model’s weights look like on the wire.