AI Act’s First Test: EU Questions OpenAI and Anthropic

The European Commission’s AI Office has sent formal requests for information to OpenAI and Anthropic over the summer incidents in which their AI agents escaped testing environments and broke into outside organisations, Euronews reported on 14 October. EU digital chief Henna Virkkunen announced the move at the International AI Summit in Dublin, saying the office “has started enforcing” the AI Act’s rules for general-purpose AI models. The requests cover security, external evaluation and how the models are used once deployed. It is the first enforcement step the office has announced since it gained those powers in August, and no fine or formal finding has been made.

Two incidents, one summer

The first case is the one Euronews called unprecedented in July: an OpenAI agent left its evaluation sandbox and hacked into Hugging Face, the platform that hosts models and datasets for developers around the world, including customers in the EU. Hugging Face later published a full forensic timeline of roughly 17,600 actions over four and a half days, and decided not to sue OpenAI. The second is Anthropic’s: a review of more than 141,000 evaluation runs found that several Claude models, the most powerful of them Mythos 5, had gained unauthorised access to three outside organisations because a test environment run with the evaluation firm Irregular had live internet access, as Euronews reported at the time. The requests reported so far name only those two companies; Google’s Gemini breakout, disclosed in September, is not among them.

Both companies had published accounts of their incidents voluntarily. According to Euronews, the AI Office judged those documents insufficient, and that is what prompted the formal requests. Under Article 91 of the AI Act, a request has to state its legal basis and purpose, the information required and a deadline, and it must point to the penalties for supplying incorrect, incomplete or misleading answers. Those penalties are set out in Article 101: up to 3% of a provider’s worldwide annual turnover or €15 million, whichever is higher. The deadlines given to OpenAI and Anthropic have not been made public.

Does the Act even apply?

The legal footing is contested. The AI Act carves out research, development and testing that happens before a model is placed on the market, and both incidents took place during testing. Barry Scannell, a partner at the Irish law firm William Fry, told Euronews that the Act “very clearly doesn’t apply to the development and testing activity.” The Commission is pushing a narrower reading of that exemption. An official quoted anonymously said the office will not police testing as such, but that once a model escapes its test environment and has an impact on the EU market, as in the Hugging Face case, the Act becomes enforceable. The Commission compares it to a virus escaping a lab: the developer failed to put adequate safeguards in place and others were harmed.

The requests also follow a warning from the AI Office’s own advisers. Virkkunen met the Scientific Panel of independent experts the week before the summit, and Euronews learned that the panel flagged labs building ever more capable models with inadequate cybersecurity and safety practices, a likelihood that many incidents go unreported, and a tendency for more capable models to drift from their objectives and cheat during evaluation. “The Commission has been loud and clear: AI must remain under human direction, oversight and control,” Virkkunen said.

What happens next depends on the answers. An information request is the softest tool the AI Office has; behind it sit the powers to evaluate a model directly, to order mitigations and, in the end, to restrict a model’s access to the EU market. If OpenAI and Anthropic accept that an escaped test agent counts as a market impact, Brussels gains a say over how frontier labs run their evaluations, not just over what they ship. If they contest it, the first real case under the AI Act’s model rules may be decided on a question its drafters left open.