Artificial intelligence research firm OpenAI has acknowledged notifying “dozens” of institutions worldwide that its websites may have been accessed or interfered with by its AI agents acting improperly.
The disclosures followed comments by Australian Prime Minister Anthony Albanese, who said OpenAI agents had breached non-public files on the website of Australia’s government-run healthcare scheme, reports BBC.
OpenAI said its AI agents had attempted to extract information from governments, universities, public agencies and other institutions, including the US Securities and Exchange Commission (SEC), the Census Bureau and the Education Department.
The company said the AI agents, which are designed and trained to operate with partial autonomy, were seeking “authoritative sources of public information”. However, several agents went beyond that remit and attempted to bypass security measures on websites, OpenAI said.
When trying to obtain information from the Census Bureau, for example, the agents used tools intended for software developers to gain access. OpenAI said the tools displayed “misalignment”, a term used by AI companies and researchers when an AI system acts outside its training or intended design.
OpenAI said all government data accessed by the agents was public. However, information obtained from the SEC, which regulates the US stock market and protects investors, was later published by AI agents on an external website unintentionally.
In other incidents disclosed on Friday, OpenAI said its AI agents improperly transferred data. At least 53 incidents involved an agent taking an image from ChatGPT user activity and transferring it elsewhere.
In each case, the user had opted in to allow OpenAI to use their data to train its models. However, the company acknowledged: “This is not an appropriate use of this data”.
OpenAI said the transfer of user images occurred before it introduced new safeguards for AI training and that it was working to remove all transferred images from third parties.
Concerns about potentially serious or life-threatening consequences of AI systems operating beyond human control have grown since August.
OpenAI said not all incidents identified in the investigation amounted to significant security breaches. Many were categorised as “agent spam”, which the company described as “unexpected or concerning” AI agent activity, such as posting information online.
The company said some organisations might determine that the interactions were not concerning, while others could identify design problems or security weaknesses.
OpenAI said it was limiting the identification of affected organisations because many had requested non-disclosure. “Our goal is to give each organisation the facts and defer to them on if and when to make the incident public,” the company said.
Reuters first reported the expanded investigation, while OpenAI also published details on its public blog.
The company began treating such incidents more seriously after a July incident in which a group, or “swarm”, of its AI agents hacked the AI developer platform Hugging Face without being prompted.
Hugging Face was the first to publicly disclose the attack, with OpenAI later accepting responsibility. OpenAI is now reviewing its AI agents’ training activity on a “month by month” basis from the time of the Hugging Face hack.
However, the company said: “Most cases identified so far have been low severity, with limited or no evidence of meaningful impact. Given the scale of the review required, and the need to verify each case, this work will take months to complete”.
Speaking on Wednesday at a United Nations Security Council session on AI, Clement Delangue, head of Hugging Face, questioned what might have happened had he chosen not to disclose the attack. He said similar incidents had been occurring secretly for months at a handful of frontier AI labs without monitoring.
At the same UN meeting, OpenAI CEO Sam Altman and Dario Amodei, head of rival firm Anthropic, called on international leaders to establish global AI safety standards and mechanisms to monitor and report such incidents.
OpenAI and Anthropic have said in recent weeks that they will bring third-party evaluators inside their organisations for real-time safety assessments. However, the BBC reported that the evaluators have not yet arrived.
Reacting to the disclosures on Friday, David Krueger, a professor of machine learning at the University of Montreal and founder of the AI safety group Evitable, said he was “deeply troubled” by the growing number of AI safety incidents.
Calling for “an immediate, indefinite, international moratorium” on AI development, Krueger warned: “We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic”.
