When conventional software fails, it usually tells you. An exception is raised, a request returns a 500, an alert fires. The failure is loud, and the loudness is what makes it fixable.
Agents fail differently. An agent that receives a malformed tool result reasons around it. An agent that cannot finish tries again, differently. An agent with no good answer returns something, in the same fluent, confident tone as a correct one.
An agent rarely crashes. It continues, on a bad result.
That changes the question engineering teams need to ask. For deterministic code, a reasonable proxy for robustness was:
“Is there a try/except around this call?”
For an agent, that answers almost nothing. The question that matters is:
“What does this agent do when it cannot do what it was asked?”
Five protections decide the answer. Each one is cheap to add while the agent is being built. Each one, missing, becomes a different kind of production incident, and some of those are hard to recognise as incidents at all.
1. Acting on Unvalidated Output
An agent that acts on unvalidated model output will act confidently on a wrong one.
A model’s output is a proposal, not a fact. It can be malformed JSON, a field of the wrong type, or an identifier that’s plausible but invented. If the next step is another model call, the error gets absorbed into the conversation. If the next step is an action, like writing a record, calling an API or sending a message, the error becomes real.
Why Development Misses It
Test prompts are clean and test data is friendly, so the model mostly returns well-formed output. The rare malformed response doesn’t show up in a few hundred test runs.
What It Costs in Production
Production inputs are long, messy and adversarial, and the rare bad response arrives on its own schedule. A refund is issued against the wrong order ID. A ticket is routed to a queue that doesn’t exist. Every step after the bad output builds on it, and none of those steps looks wrong on its own. The damage isn’t the bad output. It’s everything the agent did next.
2. Running Without a Bound
Without a limit, an agent that cannot finish keeps trying — indefinitely, and at cost.
Plan, act, observe, re-plan: persistence is what makes an agent useful. It’s also what makes an agent that cannot converge expensive. A tool that always errors, a goal the available tools can’t reach, or two agents that keep delegating to each other all produce the same thing: a loop that never ends, with every turn billed.
Why Development Misses It
Development runs are short and supervised. An engineer watching a loop spin kills it in seconds and fixes the prompt.
What It Costs in Production
Nobody is watching. The failure surfaces first as a cost anomaly, then as latency, then as a rate limit that starves every other agent sharing the same model quota. One agent fails, and every agent sharing that quota goes down with it. A worse variant: an agent created inside a loop with no limit, where bounding each agent does nothing to bound how many get spawned.
3. Failing Silently When a Tool Fails
A tool call that fails silently leaves the agent reasoning from a gap it cannot see.
Tools fail for ordinary reasons: timeouts, rate limits, a dependency that’s briefly down, a 503 during a deploy. Services handle this with retries and backoff. An agent needs the same, plus one thing more: it needs to know the call failed. To a model, an empty result and a failed call look the same.
Why Development Misses It
Tools in development are local, mocked or lightly loaded. Transient failures are rare, so the missing retry never matters.
What It Costs in Production
Transient failures are routine. Without retries, every blip becomes a failed task. Without error handling, it’s worse: the agent reads “no result” as “no data”. It reports that a customer has no orders, that a search found nothing or that a check passed, when the service was simply down. That looks exactly like a normal answer, which is why nobody questions it.
4. No Way to Hand Back
An agent with no way to hand back will produce something rather than nothing.
Some tasks can’t be completed. The information is missing, the request is ambiguous, or the action is one the agent shouldn’t decide alone. A well-designed agent has a path for this: it asks for approval, routes to a person, or stops and says so. An agent without one has a single option left, which is its best guess, presented as an answer.
Why Development Misses It
The people testing an agent are the human in the loop. They read every output, catch every guess and rephrase every ambiguous request. The missing escalation path is invisible because the tester is quietly playing its role.
What It Costs in Production
Remove the tester and the guesses go straight to customers, records and downstream systems. For an autonomous agent whose tools send, delete, pay or deploy, an unchecked guess is an action nobody approved, and often one nobody can undo.
5. Failing Where No One Can See
A silent failure is worse than a loud one: the output still looks like an answer.
The first four protections reduce how often an agent fails badly. This one decides whether you find out when it does. Tool calls, external requests and intermediate results are the only evidence of how an agent reached an outcome. Without logging, that evidence doesn’t exist. Without a way to tie a request to every operation it caused, what does exist can’t be pieced together.
Why Development Misses It
The engineer’s terminal is the log. Every step scrolls past, and the trace is right there.
What It Costs in Production
A customer reports a wrong answer weeks later. The team can see the final output and nothing that led to it. They can’t tell whether a tool failed, the model misread a result, or the agent did exactly what it was told. Every other failure on this list becomes harder to diagnose, and some can’t be diagnosed at all. It’s an audit problem too: an action that can’t be traced to the request that caused it is an action no one can account for.
Development Hides All Five
Look at the pattern. Each of these failure modes is one that development specifically conceals:
- Clean inputs hide missing validation.
- Short, supervised runs hide unbounded loops.
- Healthy local tools hide missing retries.
- The tester quietly plays the missing escalation path.
- The terminal hides missing logs.
That’s why more testing doesn’t close the gap on its own. Testing observes behaviour under the conditions you thought to create. These five fail under the conditions you didn’t.
What they have in common is that each one is a property of how the agent is built: the check on its output, the limit on its loop, the retry on its calls, the approval step before its actions, the logging around its work. A design property can be checked before the agent ever runs.
How ARIAS Keeps Them Out of Production
ARIAS reads every agent in your codebase and shows, per agent, how it handles failure:
- Is model output validated before the agent acts on it?
- Is execution bounded by an iteration limit, a turn limit or a timeout?
- Are tool and model failures absorbed by retries and error handling, rather than passed on as empty results?
- Is there a path to a human before an autonomous agent acts?
- Will a failure be visible, logged and traceable across the agent’s operations?
Each answer comes with the evidence behind it, so an engineer can see what was found and where, and a reviewer can see what’s missing without reading the code.
Because ARIAS works from the agent’s design rather than its runtime behaviour, it doesn’t need the conditions that development lacks. It doesn’t need a messy input to expose missing validation, or a failing tool to expose a missing retry. Run in your pipeline on every commit, it means a protection that’s removed or never added shows up in review — while it’s a one-line fix, not a production incident.
Bring This to Your Next Design Review
For each agent you’re about to ship:
- If the model returns malformed or invented output, what checks it before the agent acts?
- If the agent can’t finish, what stops it, and what will it have spent by then?
- If a tool times out, does the agent know the difference between “failed” and “found nothing”?
- If the agent can’t decide, who does it hand back to?
- If it fails at 3 a.m., what will you be able to see in the morning?
If the answer to any of them is “I’m not sure”, start there.
ARIAS shows how every agent in your codebase handles failure before it ships: output validation, bounded execution, tool failure, escalation and visibility, with the evidence for each. It runs locally, without executing your agents, and no source code leaves your environment. Request a demo or simply drop us an email at sales@tryarias.com