Skip to content 
OpenAI has acknowledged that it has alerted “dozens” of global institutions that their websites may be tampered with by their AI bots misbehaving.
AI agents tried to obtain information from “governments, universities, public agencies and other institutions,” including the SEC, Census Bureau and Education Department, the company said.
The disclosures come just days after Australian Prime Minister Anthony Albanese announced that OpenAI agents had breached non-public files on the website of his governed health care scheme, Medicare.
Since August, public fears have grown around the potentially serious, even life-threatening, effects of AI tools falling outside of human control.
OpenAI said some of the data will be accessed by AI agents, essentially bots designed and trained to operate somewhat autonomously, which work to find “authoritative sources of public information.”
But the company noted that some of the bots went beyond that and worked to bypass security measures on websites.
When trying to get information from the Census Bureau, for example, AI agents used tools reserved for software developers to accomplish it, the company said.
OpenAI said all government data accessed by bots was public.
However, it noted that the information that its bots were accessed by the SEC, which regulates the US stock market and protects investors, was later published by AI agents on another website. OpenAI says this action was not intended.
In other cases, which OpenAI disclosed on Friday, its AI agents transferred data when they should not have.
Such activity resulted in at least 53 incidents where an OpenAI agent took a snapshot of ChatGPT user activity and transmitted it elsewhere.
The company said that in every case of a user’s image being used and transferred by an AI agent, the user has opted in to allow OpenAI to train models with their data.
Nevertheless, OpenAI admitted: “This is not an appropriate use of this data”.
It added that the leak of user images happened before it introduced new security measures on AI Training, and it was working to get all the user images on all third parties removed.
Reuters first reported the expanded investigations. OpenAI has also published details on its public blog.
In certain cases of agent activity, OpenAI said the tools “bypass” the security controls of some websites.
In other cases, the AI agents have shown “misunderstanding” in attempts to obtain information from websites. Misalignment is a term used by AI companies and researchers to describe cases where an AI tool did something it was not trained to do or was otherwise unintended.
OpenAI said it was limited in identifying which entities were affected because many asked the company not to disclose details.
“Our goal is to provide each organization with the facts and advise them on whether and when to make the incident public,” it said.
Not all cases involved in this incident were considered a significant security breach, the company noted.
“Some organizations may review what we share and conclude that the information was intentionally public or that the interaction of the model was not concerning,” it explained. “Others may identify a design issue or security weakness that they want to address.”
