Rest assured: AI companies say they’re investigating tens of thousands of rogue bot incidents

OpenAI and Anthropic are reportedly investigating tens of thousands of incidents where their advanced models bypassed monitors and guardrails, behavior that the startups facilitate for internal safety testing. 

According to a Saturday Axios report, sources said that most of the results of these tests are not public and are not known to have caused tangible harm. 

In recent weeks, OpenAI has disclosed six instances of “unexpected or concerning behavior” where its models—without permission—covered up mistakes, made up data, and transferred files onto the open internet. In the same September 16 announcement, the startup said it would now report and investigate “misalignment,” meaning when the actions of AI systems go against human intentions. OpenAI shared on Friday that its autonomous AI agents interacted with several US government websites—including two operated by the Securities and Exchange Commission and data from the Census Bureau—in unanticipated ways. The startup said it did not consider any of the actions breaches. 

These disclosures fall in line with previous announcements by frontier AI labs that their technology engaged with “misalignment,” and they should therefore slow down and be more careful and all the cries by current and former researchers in the industry that AI could lead to human extinction by 2030. 

What OpenAI and Anthropic CEOs Sam Altman and Dario Amodei don’t mention is that the industry has long aligned with the Trump administration and its campaign to expand AI development. OpenAI has a military contract with the Defense Department worth up to $200 million. How AI is involved is unclear—the Intercept reported earlier this month that the Pentagon asked OpenAI to provide a custom AI tool with “minimal refusal rates.” Google, SpaceX, NVIDIA, Reflection, Microsoft, Amazon Web Services, and Oracle also have deals with the Defense Department.

While the Pentagon canceled its military contract with Anthropic over the startup’s concern about how its tools may be used for autonomous weapons and mass surveillance, the White House has promoted Anthropic’s $50 billion investment in data center construction and the two reportedly have a much improved relationship as of September.

The relationship between the AI industry and Trump remains as the administration cut the Cyber Safety Review Board in January 2025, a body that investigates major cybersecurity threats, and has proposed further cuts to the Cybersecurity and Infrastructure Security Agency, which secures infrastructure against cyber and physical threats. Trump previously eliminated one-third of CISA’s workforce due in significant part to its election security work. 

As Miranda Bogen, the founding director of the Center for Democracy & Technology’s AI Governance Lab, told me in July, actually addressing the “deeply insufficient” system to protect the public from AI threats involves reducing the incentives of AI companies to continuously develop within a framework of profit and geopolitical competition. Without that, we are relying on AI to regulate itself.