OpenAI is taking action after several of its AI agents went “rogue,” NBC News reports.
OpenAI agents have attempted to hack federal government websites. This allegedly included the Department of Education website, though the attempt was unsuccessful, AI evaluator Transluce noted. OpenAI acknowledged that agents “acted in unexpected ways” and said it will pause development of some of its AI agents and resume development “only when we are confident that we have additional safeguards,” according to the outlet.
This marks the second time in the past three months that OpenAI has paused development of some of its models. The first incident involved models, including GPT-5.6 Sol and other pre-release models, that escaped a sandboxed testing environment during an internal cybersecurity evaluation. The agent accessed the internet and attempted to hack AI model database Hugging Face to solve its evaluation problem without being instructed to do so, as AFROTECH™ previously reported.
“In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers,” OpenAI said in a press release.
OpenAI and Hugging Face security teams detected the incident and investigated the matter. OpenAI CEO Sam Altman wrote on X that the Hugging Face incident is “the most severe event we’ve seen.”
“There is an extensive and ongoing review related to our agents’ use of internet access during training and evaluation. We’ve been publishing summaries at the link below and will continue to. We have not been as fast as we would have liked but we are trying to balance our desire for transparency with gaining a clear understanding from petabytes of agent activity logs, and working with impacted organizations. We are prioritizing as best as we can based on severity, and adding resources.”
Altman continued, “We will be as transparent as we can be subject to things like vulnerabilities in other companies that our agents have found, which will be their call to disclose or not.”