Writing a runbook for an AI feature your team operates

Where a standard service runbook falls short

A standard runbook assumes that failures show up as errors. AI features can fail while returning HTTP 200: a confident wrong answer, a tool loop that keeps spending tokens, an agent that passes every case to a person. Some causes sit outside your code, such as an overloaded provider, a model version change or a new kind of input. The engineer on call may also know nothing about prompts or evaluation. The runbook has to turn all of this into checks a generalist can run at three in the morning.

Classify the symptom first

• Slow: latency above target, seen in time to first token, end-to-end latency, queue depth and provider response times
• Unavailable: errors from the provider or your own serving; Anthropic, for example, documents a 429 rate_limit_error and a 529 overloaded_error
• Expensive: tokens or spend per request or per case above budget, often from loops, retries or growing context
• Wrong: a failing canary evaluation, structured output failing validation, rising escalations or user reports
• Out of bounds: actions beyond expected limits, a spike in refused tool calls or a suspected prompt injection

Example runbook entries

Each entry names the first checks, a safe mitigation and the owner. The entries below are hypothetical for a customer-facing assistant.

1. 1

   Slow

   Use the trace breakdown to see whether time goes to the provider, retrieval or tools. If the provider is slow, switch to the tested fallback route or shed non-urgent traffic. Owner if unresolved: platform team.

2. 2

   Unavailable

   Check the provider's status and the error types. Retries need backoff with jitter and a budget; Google's SRE book shows how retries at several layers multiply load on an overloaded backend. If errors persist, enable the fallback or the non-AI path. Owner: platform team.

3. 3

   Expensive

   List the cases with the most tokens in the last hour; loops show as many calls per case. Apply per-case limits on tool calls or tokens, or disable the affected workflow. Owner: feature team.

4. 4

   Wrong

   Run the canary evaluation set against the production configuration, using synthetic fixtures with write tools stubbed or isolated, so the test causes no real business actions. If it fails after a recent prompt or model version change, roll that change back. Do not simply restore an older retrieval index: it may still hold withdrawn documents or outdated permissions, so apply current deletions and access rules first, or keep the current index and escalate. If nothing changed, collect examples for the owners and do not edit prompts on call. Owner: feature team with the domain owner.

5. 5

   Out of bounds

   Disable write tools or the feature, preserve the traces, revoke credentials if misuse is suspected and follow the security incident process. Owner: security.

Switches to build before the first incident

Most of the mitigations above only work if someone built them in advance and tested them:

• A feature flag that turns the AI path off and falls back to the previous process or a clear message
• A separate switch for each write tool, and a read-only mode for agents
• A pinned model version and a fallback route that has passed the evaluation set
• Spend and rate caps per tenant, user or case

Exercise each switch in a planned drill. Google's SRE book makes the point about degraded modes: a code path you never use is often a code path that does not work.

Ownership and escalation

Google's SRE book recommends declaring an incident early and setting clear conditions, such as needing a second team, an outage visible to customers, or a problem still unsolved after an hour of concentrated analysis. It also separates roles: an incident commander, an operations lead whose team, under that lead, is the only group changing the system, and someone handling communication. AI features add one role. Judging whether answers are wrong may need a domain owner, such as the head of claims for a claims assistant, so name that person in the runbook with a way to reach them.

Keeping the runbook honest

Every postmortem should add or correct an entry. Provider changes belong in the runbook too: a new model version, a changed rate limit, a deprecation date. Check what trace data on-call engineers can actually see, since content they need may be held back by design. A practical next step is to write the five classes for one feature using the signals you already collect, and mark each place where a signal or a switch is missing.

AI engineering and architecture