Designing the cases an agent passes to a person

Triggers you can write down and test

• Reserved by policy: complaints that mention legal action, cases flagged as vulnerable customers, anything your rules say a person decides
• Missing or contradictory information that one clarifying step did not resolve
• A system failure beyond the retry budget, and every write whose outcome is unknown
• An unfamiliar case: a document template, supplier type or language that is absent from the test set
• A step or cost budget exceeded, which Anthropic's guidance on agents describes as a stopping condition such as a maximum number of iterations
• The customer asks for a person

Notice what is missing: the model's own sense of confidence as the only trigger. A model's stated confidence is not calibrated by default. It can add escalations, but the list above should not depend on it.

Test escalation in both directions

Escalation can fail two ways. An unnecessary handoff costs review time and erodes trust in the agent. A missed handoff lets the agent finish a case it should not have touched, which is usually more expensive. Anthropic's guide to agent evals recommends testing both when a behaviour should occur and when it should not. For escalation, the test set needs cases that must be handed over and similar cases that must not, and the results should be reported separately.

What the handoff contains

The format below is a sketch for a hypothetical claims agent. It is meant to be rendered in the tool the person already works in, not read as raw data.

case_id: clm-40517
reason: CONTRADICTORY_INFORMATION
question_for_reviewer: >
  Damage date on the form (12 Aug) is before the policy start (15 Aug).
  The photos are dated 18 Aug. Which date applies?
done:
  - claim created, status awaiting_assessment (ref CL-889120)
failed: []
unknown:
  - acknowledgement email: send timed out, delivery not confirmed
not_done:
  - coverage check, blocked on the date question
proposed_next_step: >
  If 18 Aug applies, run coverage check. If 12 Aug applies,
  decline under clause 4.2 after review.
evidence: [form.pdf, photo_metadata.json, policy_P-2.pdf]
agent_version: claims-intake 2026.10.2
answer_due_by: next working day 12:00

Two fields carry most of the value. The question should be one the person can answer, not a vague request to review the case. The unknown list tells them which external effects may or may not have happened, so they check before repeating anything.

The queue on the other side

An escalation queue needs an owner, routing by reason so the right specialist sees each case, and a target for how long a case may wait. Google's SRE workbook treats objectives like this as ratios of good events to all events with a written policy for what happens when they are missed, and the same structure works for queue age.

Decide whether the person finishes the case or answers the question and returns it. Returning works when the agent's state is saved: LangGraph's interrupt(), for example, holds a run until it is resumed with the person's answer. Either way, the agent must stop writing to the case once it is escalated.

What escalations teach you

Record how the person resolved each case, in a structured field rather than free text. Over a few weeks, the reasons show where to work. If most escalations come from one unreliable lookup, fix the tool. If they come from one document template, add cases for it to the test set. If people resolve a category the same way every time, that category may be ready for the agent, after it passes the tests.

Where escalation design breaks down

The common failures are organisational: a queue without an owner, escalation used as a dumping ground for every awkward case, a reviewer who lacks access to the system the agent used, or a handoff that arrives in a separate tool nobody checks. Microsoft's compensating transaction guidance makes a related point for failed operations: when manual intervention is needed, the alert should carry the detailed reason. A good first step is to list your current escalation reasons, write the question a person would need answered for each, and check whether the agent can produce it.