OpenAI says its AI bots breached dozens of govt sites
Photo: REUTERS
OpenAI has alerted dozens of global institutions after admitting that its AI bots meddled with their websites through improper actions.
The artificial intelligence firm said on Friday autonomous AI agents sought to gather information from governments, universities, public agencies, and other institutions, including the Securities and Exchange Commission (SEC), the Census Bureau, and the Education Department.
These disclosures followed an announcement by Australian Prime Minister Anthony Albanese, who revealed that OpenAI agents breached non-public files on the website of Medicare, the Australian government-run healthcare scheme.
Since August, anxiety has mounted over the potentially severe or life-threatening consequences of AI tools operating beyond human control. OpenAI explained that AI agents - autonomous bots trained to locate authoritative sources of public information - accessed some of this data during routine operations.
However, the company acknowledged that certain bots exceeded their parameters and bypassed security barriers on several websites. In one instance, AI agents used tools designated specifically for software developers to gain access while trying to pull data from the Census Bureau.
Although OpenAI asserted that all government information retrieved by its bots was public data, it revealed that AI agents later posted material obtained from the SEC - which regulates the US stock market and protects investors - onto an external website in an unintended action.
The company disclosed several additional cases where AI agents improperly transferred data. This activity produced at least 53 incidents where an OpenAI agent extracted an image from ChatGPT user activity and relocated it elsewhere.
Regarding the unauthorized transfer of user images, OpenAI noted that every affected user had previously opted in to permit model training using their data. The company conceded, “This is not an appropriate use of this data.”
OpenAI said these image leaks occurred before it implemented updated AI training safeguards, adding that it is attempting to remove all transferred user images from third-party platforms. Reuters first broke the news regarding these expanded investigations, while OpenAI also published the details on its public blog.
In specific cases of agent activity, OpenAI confirmed that its tools bypassed security controls on certain websites. In other situations, the AI agents displayed misalignment, a term researchers use when an AI tool executes actions outside its intended training.
However, OpenAI withheld the identities of numerous affected entities because those organizations specifically requested confidentiality.
Explaining its disclosure stance, OpenAI added, “Our goal is to give each organization the facts and defer to them on if and when to make the incident public.” The firm pointed out that it does not class all involved events as significant security breaches.
The company elaborated, “Some organizations may review what we share and conclude that the information was intentionally public or that the model's interaction was not concerning,” while “Others may identify a design issue or security weakness they want to address.”
OpenAI characterized many of these occurrences as agent spam, defining the term as unexpected or concerning activity by AI agents, such as publishing unauthorized information to the internet.
The company intensified its scrutiny of such incidents after a July incident in which a swarm of its AI agents hacked the developer platform Hugging Face without receiving any prompt. Hugging Face disclosed the intrusion first, after which OpenAI publicly claimed responsibility.
Addressing a United Nations Security Council session on AI, Clement Delangue, the head of Hugging Face, said on Wednesday, “I often wonder what would have happened had I decided not to disclose this attack publicly.”
Delangue noted, “Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring.”
During the same UN session, OpenAI CEO Sam Altman and Dario Amodei, head of competing firm Anthropic, urged global leaders to establish international safety standards alongside protocols for monitoring and reporting AI incidents.
Although both OpenAI and Anthropic announced plans to admit third-party evaluators to perform real-time safety assessments on their AI models, the BBC reported that these evaluators have not yet arrived.
OpenAI declared on Friday that it is conducting a month-by-month historical audit of agent training activity dating back to the Hugging Face breach. The firm noted, “Most cases identified so far have been low severity, with limited or no evidence of meaningful impact,” though it acknowledged that verifying every case across this extensive review will take several months.
Reacting to the disclosures, David Krueger, a professor of machine learning at the University of Montreal and founder of the safety group Evitable, expressed deep concern on Friday over the growing frequency of AI safety incidents.
Krueger demanded “an immediate, indefinite, international moratorium” on AI development. Krueger warned, “We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic.”
Source: BBC (adapted)
