Industry·5 min read
By BitsMindsSource: Dragos

Dragos Bait Prompts Expose 3 Breached AI Providers

Dragos sent about two million AI prompts laced with trackable fake secrets and watched where they surfaced. Attackers were inside three hosting providers that did not know it, marketplace resellers read prompts through ordinary logging, and “zero data retention” requests were scraped and shared anyway.

Editorial illustration for Dragos research that seeded about two million AI prompts with canary credentials. The birdcage stands for the canary, the gold key for a bait credential, and the open door with escaping sparks for credentials stolen from AI providers. A metaphor, not a figure from the research. 3 AI PROVIDERS BREACHED BITSMINDS.COM
Share:

Security firm Dragos has published the results of a five-month experiment in following AI prompts after they leave the customer. From July through November 2026, its research team submitted about two million prompts to widely used AI services, each carrying unique URLs and bait credentials that no legitimate system would ever touch. Any attempt to use one meant that someone, somewhere along the way, had read the prompt. The write-up, by Chief Research Officer HD Moore, the creator of Metasploit, reports three findings: break-ins at AI providers, prompt theft by inference resellers, and privacy settings that were not honored.

Three providers, none of which knew

The most serious result was direct: attackers were actively stealing data from three different hosting providers. The earliest incident goes back to mid-July, and according to Dragos none of the three knew about the breach until the researchers told them. Because the bait prompts were processed alongside real customer requests, Moore writes, those customer requests “were almost certainly copied too.” Dragos does not name the providers.

The largest breach involved a couple of hundred of the planted AWS keys, each sent in a single request to a single provider during a few short windows in late October. Every request sent in those windows was stolen, including some the provider had rejected before a model ever ran, which suggests the attacker was copying traffic at the front door rather than deep inside the inference stack. In the other two breaches, some stolen requests had been flagged for Zero Data Retention. “A privacy flag doesn’t help when the attacker is already inside,” the post says.

The attackers followed a predictable script once they held a key: test it, sometimes within hours; list S3 buckets and search Secrets Manager and Parameter Store; then try to switch on models in AWS Bedrock, mint Bedrock API keys and add their own IAM users to keep access. The bait keys drew tens of thousands of AWS API calls in total. One adversary appeared to have compromised two of the providers at once, using secrets from both together from a single rented server. Over three weeks its tooling moved from a script posing as Chrome on Windows to Boto3 inside OrbStack and then the AWS CLI on Kali Linux, and on November 1 it added ARTEX, the LLM-driven penetration testing framework behind the attack on the European Commission. Here ARTEX only made a handful of calls to check that keys were live, but Dragos says two of its user agents, artex-agent and artex-judge-r121, appear in no public indicator list and are worth adding to detections.

Resellers who just turned on logging

The second finding needed no exploit at all. Some AI routers are open marketplaces where the cheapest seller wins the request, and every hop can see the full prompt and response. The sellers Dragos caught “only had to list a model at the lowest price, enable request logging, and wait.” In one case, two prompts sent through a single seller were opened about a day and a half later by someone on a home broadband connection, who started guessing usernames at the bait login page; prompts for the same model sent through a different seller were never touched. In another, an attacker on a Mac in France took passwords from two prompts and typed them by hand into SSH login attempts as root. Route tracing also showed one router quietly forwarding part of its traffic to the marketplace, a hop its customers were never told about.

“Zero Data Retention is supposed to mean something”

The third finding involved no stolen credentials, only prompts that ended up where they should not. A Chinese AI lab’s systems fetched the secret URLs from more than half of the prompts sent to its models between July and October, including roughly two-thirds of the requests marked “no data collection” or Zero Data Retention, usually two or three days later. Several U.S. hosting providers resell that lab’s models, and prompts sent through them triggered the same scraper. Large U.S. providers were not clean either: one kept crawling a secret URL more than two months after a single call to its search endpoint. In the oddest case, a link from a prompt sent with the ZDR flag turned up 24 days later in a Feishu (Lark) chat channel and was opened within two minutes from desktops and phones in Beijing and Shanghai.

To catch all this, Dragos began with free Thinkst Canarytokens and then built its own seeding service that derives a unique hostname, URL and credential for every prompt and runs the DNS, web, git, SSH, database and mail services behind them; the AWS keys are real keys tied to empty, fully logged accounts. It also built what amounts to traceroute for AI inference, using image URLs and deliberately malformed requests to reveal each hop. Of about two thousand routes traced, several hundred passed through a system the team could not identify, and a handful sent the image to a company that was not on the advertised path.

Disclosure and advice

The provider behind the first incident first blamed its internal tooling, then partner data processing, before evidence showed its own internal credentials had been compromised; Dragos says it has still not disclosed the breach publicly. One of the providers hit in late October has published a preliminary security notice, and Dragos is now working with about a dozen providers on the less severe data-sharing cases. Stolen keys and secret URLs are still being accessed months later. The post does not tell teams to stop using AI services, but it does say a privacy flag only binds the first hop and that agents “will happily send whatever they can read, including credentials.” Its recommendations: self-host the most sensitive work, contract directly with a first-tier provider, pick upstream providers yourself when using a router and avoid reseller marketplaces, plant your own canary credentials in agent context, and turn on secret filtering where providers offer it.

The findings land a week after Notion AI chats were exposed in a breach at Fireworks AI, a separate incident. Taken together they shift the question from what a model might do with a prompt to who else gets to read it on the way there, a question most buyers of cheap inference have never had to ask.

Want AI news before everyone else?

The morning's most important AI stories, straight to your inbox. No fluff.

Related Articles

Editorial illustration: a monitor showing the ring of twelve European stars, one knocked out of place and marked by a red target reticle, beside the word ARTEX. A metaphor for the reported AI-assisted cyberattack on the European Commission; not an official EU image. ARTEX AI-ASSISTED ATTACK BITSMINDS.COM
Industry

European Commission Hit by AI-Assisted ARTEX Attack

California sues OpenAI over its rogue agents Editorial illustration on a charcoal field: a polished steel gavel striking a round steel sound block whose top face carries the OpenAI mark as an engraving. Left, the words The People of California versus OpenAI and Rogue agents. An editorial metaphor for a lawsuit filed in San Francisco Superior Court, not a depiction of any court proceeding. THE PEOPLE OF CALIFORNIA v. OpenAI ROGUE AGENTS BITSMINDS.COM
Industry

California Sues OpenAI Over Its Rogue AI Agents

US rule on remote access to advanced AI chips Editorial illustration on an oxblood field: an angled AI accelerator package with a metal lid and four memory stacks, with a thick network cable plugged into its side and a brass padlock engraved LICENSE locked around the cable. The type column reads U.S. export controls, Remote access, license required. The padlock is an editorial metaphor for the Commerce Department's new licence requirement on overseas data centres; the chip is generic and is not a specific vendor's product. 3A090 3A090 ADVANCED IC LICENSE U.S. EXPORT CONTROLS Remote access LICENSE REQUIRED BITSMINDS.COM
Industry

US Closes AI Chip Remote-Access Loophole With Licenses