Industry·3 min read
By BitsMindsSource: Notion

Notion AI Chats Exposed in Fireworks AI Breach

Hackers who broke into AI inference provider Fireworks AI reached Notion AI chat transcripts from fewer than 1% of Notion workspaces, Notion has disclosed. Notion cut Fireworks off, revoked exposed Notion API keys and is telling affected customers to rotate any credentials they pasted into Notion AI. Fireworks links the attack to an “LLMjacking” crew that resells stolen model access.

Notion AI chats exposed in the Fireworks AI breach Editorial illustration on a graphite field with a violet glow: a thick white ceramic tile carrying the Notion logo, cracked at one corner, with blank chat bubbles drifting out of the crack. Below right, a violet cable with the Fireworks AI logo on its plug lies unplugged. The type column reads Third-party AI breach, Notion AI, via Fireworks AI. The objects are an editorial metaphor, not products. THIRD-PARTY AI BREACH Notion AI VIA FIREWORKS AI BITSMINDS.COM
Share:

Chat transcripts from Notion AI were exposed when attackers broke into Fireworks AI, the inference company that ran some of Notion’s AI workloads. In a security advisory first published on November 12, Notion says the incident affects fewer than 1% of its workspaces and stresses that its own systems were not breached. The exposed data came from the feature it calls Insights, which used Fireworks for classification jobs; Notion also used the company as a general provider for open-weight models.

Notion’s timeline starts on the evening of November 2, when Fireworks told it that attackers had reached a small set of data that Notion first understood to be synthetic test data, with no customer content. By the next morning Notion had disconnected its systems from Fireworks, and on November 4 it deleted all of its data held there. Two days later Fireworks said customer data might have been involved after all, and on November 8 it confirmed the final dataset. Notion contacted affected customers on November 9, revoked Notion API keys found in the exposed messages on November 11, and on November 12 reported exposed third-party API keys to the vendors that issued them.

What was in the transcripts

The dataset holds Notion AI chat transcripts, which TechCrunch reports means any input to or output from the model that was written into the chat. Notion says that for most affected workspaces the transcripts were not labelled with the author or the workspace, and that they did not include the results of tool calls. It has disabled Insights, stopped serving open-weight models through Fireworks, and lets affected customers review the exposed data inside their workspace. Its advice is to treat that data as exposed and rotate any password, API key or other credential pasted into Notion AI during the incident window.

How Fireworks was breached

Fireworks first disclosed trouble on October 9. That bulletin said an unauthorized party had used internal Fireworks credentials to read environment variables from evaluation jobs in a limited number of customer accounts, activity the company spotted on October 6. Its updated bulletin, last revised November 9, describes a second route in: a third-party cloud provider that hosted Fireworks research machines had its own security incident on October 23, and the next day the intruder used it to reach Fireworks development machines and run an unauthorized workload. Fireworks says it caught that the same day and has seen no further successful access to its customer platform since October 25.

The update widens what was taken. Beyond the evaluation secrets, Fireworks found account metadata and billing details exposed, including billing and shipping addresses, invoices, payment history and credit usage; it says it does not store card numbers. For a smaller set of accounts, secrets, datasets and incomplete model weights were also accessed. Fireworks says its Inference, Training SDK and Nexus services, which retain no data by default, show no impact. It has notified law enforcement, brought in Palo Alto Networks’ Unit 42 for forensics, rotated credentials and disabled self-serve dataset uploads and downloads for batch inference and dataset-based training.

Fireworks ties the attack to a financially motivated group that sells tokens from hijacked access to large language models to illicit routing gateways, a scheme known as LLMjacking. It does not name the group, and neither company names any other affected customer. TechCrunch notes that Fireworks counts tens of thousands of customers, among them Harvey and Cursor, and calls this the second breach to hit the company in as many months.

The incident shows how far an AI feature’s data can travel from the product a user sees: a chat typed into Notion ended up on the infrastructure of a vendor most Notion users would never deal with directly. Because the attackers were after model access to resell, the stolen API keys pasted into chats are the most immediately useful thing they took, which is why both companies are pushing customers to rotate them rather than wait for the investigation, still open at Fireworks, to finish.

More on Notion AI

Evergreen coverage we keep current — start here.

Want AI news before everyone else?

The morning's most important AI stories, straight to your inbox. No fluff.

Related Articles

California sues OpenAI over its rogue agents Editorial illustration on a charcoal field: a polished steel gavel striking a round steel sound block whose top face carries the OpenAI mark as an engraving. Left, the words The People of California versus OpenAI and Rogue agents. An editorial metaphor for a lawsuit filed in San Francisco Superior Court, not a depiction of any court proceeding. THE PEOPLE OF CALIFORNIA v. OpenAI ROGUE AGENTS BITSMINDS.COM
Industry

California Sues OpenAI Over Its Rogue AI Agents

US rule on remote access to advanced AI chips Editorial illustration on an oxblood field: an angled AI accelerator package with a metal lid and four memory stacks, with a thick network cable plugged into its side and a brass padlock engraved LICENSE locked around the cable. The type column reads U.S. export controls, Remote access, license required. The padlock is an editorial metaphor for the Commerce Department's new licence requirement on overseas data centres; the chip is generic and is not a specific vendor's product. 3A090 3A090 ADVANCED IC LICENSE U.S. EXPORT CONTROLS Remote access LICENSE REQUIRED BITSMINDS.COM
Industry

US Closes AI Chip Remote-Access Loophole With Licenses

Nonprofits sue five AI developers to freeze frontier AI Editorial illustration on an oxblood field: a walnut judge's gavel raised over a frosted sound block, and in front of it five brass medallions engraved with the marks of Anthropic, Google, Meta, OpenAI and xAI. The type column reads San Francisco court, Freeze frontier AI? No court has ruled; the frost is a metaphor for the relief the plaintiffs request. SAN FRANCISCO COURT Freeze frontier AI? NONPROFITS v. FIVE LABS BITSMINDS.COM
Industry

Nonprofits Sue Five AI Labs to Freeze Frontier Models