Zero. That’s how many complex systems I’ve reviewed that failed the way their documentation predicted. Not one. Every outage, every silent data corruption, every agent loop that burned through a token budget at 3 a.m. arrived by a route nobody had drawn on the architecture diagram.
That gap between predicted failure and actual failure is the whole subject of the safety literature on complex systems, and it maps almost perfectly onto the AI toolkits I test for a living. The research is blunt about the mechanism: complex systems fail through cascades, not single breaks. Security holes are a big part of the cascade, and regular updates are what keeps them from widening. Going into 2026, the emphasis is on proactive maintenance rather than reactive firefighting.
Simple advice. Almost nobody follows it, and there’s a structural reason why.
Nothing in your stack breaks alone
When I review an AI toolkit, I’m never really reviewing one thing. A working agent setup is a model provider, an orchestration layer, a vector store, a queue, a set of tool integrations, an auth layer, and however much glue code you wrote at midnight. Each piece has its own failure modes and its own release cadence. The interesting failures live in the seams between them.
This is why single-component benchmarks mislead people. A tool can score well on latency, cost, and output quality and still take your pipeline down, because the failure that matters wasn’t in the component. It was in how the component behaved when its neighbor got slow. Retries that look sensible in isolation turn into a self-inflicted denial of service when the thing downstream is already struggling. Caching that saves you money on a normal Tuesday serves stale, wrong answers during an incident.
One of the more useful observations I’ve read on this is that new feature development adds new failure paths to systems people thought they understood. Every capability you bolt onto an agent widens the surface. That’s not an argument against shipping. It’s an argument for knowing that your mental model of the system went out of date the moment you merged.
The light bulb problem
My favorite line from the failure literature is about maintenance and production being at cross purposes. If you shut everything down every time you need to change a light bulb, you never get anything done.
Anyone running AI tooling in production recognizes this instantly. Your dependency tree is enormous. Model versions get deprecated. SDKs ship breaking changes in minor releases. Many of those releases carry patches for security holes, because attack methods keep moving and the fixes have to move with them. So the correct answer is to update often.
The practical answer is that you pinned everything eighteen months ago, because an update once broke your production agent on a Friday, and now the version numbers in your lockfile are a small museum. I’ve seen this in more teams than I can count, including ones that would describe their security posture as solid.
The tension is real and it doesn’t resolve with willpower. It resolves with design. Systems that can absorb maintenance without a full stop are the ones that actually get maintained.
What this changed about how I test
I’ve stopped grading tools purely on what they do when everything works. The happy path is table stakes and everyone’s demo video covers it. What I look for now:
- Update cost. How much of my code breaks on a minor version bump? A tool with a stable interface and a clear deprecation policy is worth more than a faster one that forces a rewrite twice a year.
- Blast radius. When one dependency goes down, does the tool degrade or die? Timeouts, circuit breakers, and sane retry defaults tell me the authors have run this in anger.
- Observability. Can I see what the agent actually did, in order, with inputs and outputs? If the answer is a log line saying “task failed,” I can’t debug a cascade.
- Security cadence. Does the project ship patches promptly and say what they fixed? A quiet changelog is not a sign of stability.
- Rollback. How fast can I get back to the last known good state? This is the single most underrated property in the entire category.
Boring is a feature
The uncomfortable part of all this is that the work which prevents cascading failures is unglamorous. Patch reviews. Dependency audits. Rehearsing a rollback you hope never to use. None of it demos well and none of it ends up in a launch post.
But the tools I still trust after a year in my stack are the ones that made that boring work cheap. They let me change the light bulb without stopping the line. If you’re evaluating an AI toolkit this quarter, that’s the question I’d put above raw capability, because capability is what you buy and maintainability is what you live with.
🕒 Published: