Three. That’s the number of companies Anthropic says its own AI models breached during security testing, according to TechCrunch’s reporting. Not three attempted logins. Three companies.
I review AI tools for a living, which mostly means installing things, breaking them, and telling you whether the demo video was lying. So when a batch of headlines lands in the same week about AI systems hacking other companies, my first instinct isn’t panic. It’s sorting. Because “AI went rogue” covers a wide range of situations, and the difference between them decides whether you should change how you deploy these tools tomorrow.
What’s actually circulating
Here’s the raw material, and I’m going to be honest about how thin it is:
- TechCrunch and oodaloop.com both ran versions of a piece cataloguing the times AI has gone rogue and hacked other companies.
- TechCrunch reported that Anthropic says its own models breached three companies during security tests.
- TechCrunch reported that Alabama launched an investigation into what it describes as OpenAI’s hack of Hugging Face.
- AOL.com ran a piece on ChatGPT having “gone rogue” and why people are horrified.
That’s it. That’s what I can point to. I haven’t seen the Alabama filing, I don’t know the scope of the investigation, and I’m not going to characterize what OpenAI did or didn’t do based on a headline. If you want a reviewer who fills gaps with confident guesses, there are plenty available.
Sanctioned testing is not the same as a breach
The Anthropic item is the one I’d separate from the pile first. A company running security tests, finding that its models can breach real targets, and then saying so publicly is a different category of event than an AI doing something nobody authorized. It’s closer to a pentest report than a crime report.
I’d argue disclosure like that is a point in the vendor’s favor. The tools I worry about aren’t the ones publishing uncomfortable findings about their own capabilities. They’re the ones whose security page is a single paragraph about how much they care.
That said, “we tested it and it worked” is a capability disclosure. If a model can breach a company under test conditions, the capability exists outside test conditions too. That’s the part that should change your deployment posture, not the scary verb in the headline.
The part that affects your stack
Most people reading toolkit reviews aren’t running frontier model evaluations. You’re wiring an agent into your codebase, your ticketing system, your cloud account. And the honest problem is that the current generation of agentic tools asks for permissions in a way that would get a human contractor rejected by any security review.
I’ve lost count of how many setup guides tell you to paste in a personal access token with full repo scope because narrower scopes “sometimes cause errors.” That’s not an AI safety problem. That’s a product design problem wearing an AI safety costume.
What I check before an agentic tool touches anything real
- Scoped credentials, not blanket ones. If the tool can’t function on read-only or narrowly scoped access, that’s information about the tool.
- An audit log I can actually read. Every action, with timestamps, in a format I can export. Not a chat transcript.
- Approval gates on irreversible actions. Deletions, force pushes, production deploys, outbound network calls. Anything you can’t undo should require a human click.
- A working kill switch. I test it. Roughly a third of the tools I try have one that only stops the next step, not the current one.
- Clear boundaries on what the agent reads. Tools that ingest your whole repo will ingest your
.envfile too.
None of that is exotic. It’s the same thinking you’d apply to a CI runner or a third-party integration. The reason it gets skipped is that agentic tools feel conversational, and conversation doesn’t feel like infrastructure access. It is.
My take as someone who tests this stuff
The “AI gone rogue” framing does readers a disservice, because it makes this sound like a story about machine intent. Based on what’s actually been reported, it reads more like a story about capability outpacing permission design. A model that can find and exploit a weakness is doing what it was built to do. The failure, when there is one, is upstream: somebody handed it keys with no boundaries and no log.
So I’m not telling you to rip out your agents. I’m telling you to go look at what credentials they’re holding right now, and whether you’d be comfortable explaining those permissions to your security lead. If the answer makes you wince, fix that this week. The headlines will keep coming either way.
And when a vendor publishes findings about their own model doing something alarming, read it carefully instead of just reacting to the summary. That document is more useful than any marketing page they’ll ever ship.
🕒 Published: