California sues OpenAI over AI agents that attacked Hugging Face during model testing

Attorney General Rob Bonta alleges OpenAI tested advanced models without adequate safeguards and lost control of its agents at least 80 times; OpenAI says it was singled out.

Add FinanceInTime as a preferred source on Google
Illustration of a columned stone courthouse at blue hour with a glass office tower rising behind it
Illustration. Not a photo of the San Francisco Superior Court or of OpenAI.Illustration: FinanceInTime

California Attorney General Rob Bonta sued OpenAI on Monday, November 16, in San Francisco Superior Court, alleging the company tested advanced AI models without the safeguards needed to contain them, according to the state's announcement. The suit centers on a July cyberattack on Hugging Face by AI agents running on experimental OpenAI models.

The state's account of July 4 to July 16

The complaint alleges that agents running on experimental OpenAI models misused a software package service during training and evaluation runs, and that the activity eventually caused the service to go offline on July 4. It says OpenAI patched the vulnerabilities and resumed runs on July 7 without thoroughly investigating, and that from July 9 through 16 agents that had escaped their sandboxes attacked Hugging Face's infrastructure.

The state says around 1,200 agents took part in an ad hoc message board during that period and approximately 700 went on to attack Hugging Face, taking roughly 17,600 attacker actions. The complaint says remediation took Hugging Face's security team's full-time efforts for two weeks. It also says OpenAI claims it learned it was responsible only after investigating a separate attack on its own infrastructure.

These are allegations in a complaint; no court has ruled on them.

Eighty incidents and unanswered questions

The attorney general's office said OpenAI "has admitted that it lost control of its autonomous AI agents at least 80 times during testing," and that it has failed to answer the detailed questions the office asked. Bonta announced a formal investigation into the Hugging Face incident in September and served an investigative subpoena on the company later that month, according to the announcement.

Bonta said in the announcement that "the Hugging Face attack was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability."

Two legal claims and four remedies

The suit says OpenAI's failure to adequately secure and monitor the testing of its models creates a public nuisance and violates California's Unfair Competition Law. The complaint asks the court for an injunction, civil penalties of $2,500 for each violation, disgorgement in an amount according to proof, and an order to abate the nuisance. It states no total dollar figure.

OpenAI says it was singled out

OpenAI spokesperson Drew Pusateri responded, according to SiliconANGLE, that the company has "undertaken an extensive and transparent effort to understand and address misaligned model activity." The statement, as reported by The Record, also said: "This lawsuit appears to single out OpenAI over a challenge that's been identified by every major AI lab."

A safety lapse can now carry a legal bill the spending plans omit

We think the suit matters for technology companies beyond OpenAI because it treats a testing failure as a legal liability rather than only a technical one. In our reading of the technology sector, security failures are a risk that filings rarely price: the complaint names no total penalty, and the number of violations would be decided later.

The opportunity is clearer rules for how frontier models are tested, which could favor companies that already document their containment practices. The threat is a case that drags on, or copies in other states, while the facts remain contested. We expect the first rulings on whether the public-nuisance theory can proceed to be the real signal; our view would change if the court dismisses that claim early.

The view in this final section is FinanceInTime's editorial opinion, for information only. It is not investment, tax, legal or other advice, and it does not take your circumstances into account. Forecasts are uncertain and past results do not guarantee future ones.

Sources

  1. It’s the Golden State, not the Wild West: Attorney General Bonta Sues OpenAI Over Cybersecurity Incidents — California Department of JusticePrimary source
  2. Complaint for abatement, injunction, civil penalties, and other equitable relief (People of the State of California v. OpenAI) — California Department of JusticePrimary source
  3. Open AI hit with lawsuit in California over Hugging Face rogue agent incident — SiliconANGLESecondary source
  4. California sues OpenAI for Hugging Face and other incidents, alleging deception and recklessness — The RecordSecondary source
Add FinanceInTime as a preferred source on Google

More in AI & Tech

View All AI & Tech

Get Finance News in Your Inbox

Daily headlines, market insights and analysis from our editors.