What an agent should do when its tools fail

Seven failure types and their default response

Wrap each tool so that it returns a typed failure instead of a raw exception or an empty result. The response to each type is then a design decision you can review and test.

• Transient read failure, such as a timeout, a 503 or a rate limit response: retry with backoff, within a fixed budget
• Write with unknown outcome, such as a timeout while posting: check the target system before any retry
• Invalid input, such as a wrong date format: return an actionable message to the model and allow a small number of corrections
• Business rejection, such as a closed accounting period: not a technical error, so follow the process rule for that situation
• Incomplete or stale data: stop using the result and treat it like a failed call
• Permission denied: stop and report it, and never let the agent look for another tool that reaches the same data
• Dependency down for longer: park the case with its state saved and resume it later

Retries belong in one place

The Amazon Builders' Library article on timeouts and retries calls retries selfish: each one spends more of the dependency's capacity for a better chance of success. It shows how retries at several layers multiply: if each of five layers makes three attempts, the bottom of the stack can receive 243 calls for one request, three to the power of five. It recommends retrying at a single point, with capped exponential backoff and jitter, and generally not retrying client errors. Apply that per endpoint: invalid input or a failed authorisation will not succeed on a retry, but a 429 Too Many Requests response signals rate limiting, and RFC 6585 allows it to carry a Retry-After header stating how long to wait. Follow the delay the provider documents.

An agent adds a layer that is easy to miss: the model itself may call the tool again after an error. Count those calls against the same budget. Frameworks help with the mechanics. LangGraph's RetryPolicy, for example, documents defaults of three attempts, a 0.5 second initial interval, a backoff factor of 2 and jitter, and by default retries HTTP errors from requests and httpx only for 5xx responses. Check the defaults of the version you run rather than assuming them.

The write that timed out

A timeout on a write tells you only that no answer arrived. The posting may have succeeded, failed or still be running. Retrying blindly risks a second posting. Use this order instead:

1. 1

   Retry with the same key

   Only if the target API documents that a repeat with the same key is deduplicated rather than executed again, retry with the original key. Without that documented guarantee, go to the next step.

2. 2

   Otherwise read back

   Search the target system by a business reference you sent, such as the case ID in the posting text, and reconcile the case with what you find before deciding whether to repeat.

3. 3

   If still unclear, stop

   Mark the case as outcome unknown and hand it to a person. This status needs its own queue, because both a retry and a silent close can be wrong.

Checkpointing in the agent framework does not solve this. It records the agent's progress, not whether an external system applied the change.

Incomplete data must look like a failure

The worst tool failures look like success: an empty list because a search timed out internally, the first page of results without a sign that more exist, a record missing fields the agent then fills in by guessing. Design tool results so that completeness is explicit, with a total count, a cursor for the next page and the time the data was read. The Model Context Protocol lets a tool declare an outputSchema that clients should validate results against, and it separates protocol errors from tool execution errors flagged with isError so the model can react to the latter. Anthropic's guidance on writing tools adds that error messages should say what to fix rather than return opaque codes.

Parking, compensating and handing off

When a dependency is down, a case with durable state can wait. LangGraph, for instance, can drain a run at a step boundary and resume it later from a checkpoint. When a case fails halfway through, some completed steps may need to be undone. Microsoft's compensating transaction pattern recommends retrying first, compensating only when forward progress is impossible, making each compensation step idempotent and raising an alert with the failure details when a person has to step in.

The handoff itself should list what the agent completed, what failed, what is unknown and what it would have done next.

Testing recovery before it is needed

Put failures into the test set on purpose: a lookup that times out twice and then answers, a write that times out after it was applied, a search that returns one page of three. Then check the outcome, not just that the run ended. A good first step is a short table listing every tool the agent uses and what a timeout means for each. Where the answer for a write is that nobody knows, start there.