What to version so that an agent's result can be explained later

More than the prompt changes behaviour

Teams usually version the code and sometimes the prompt. The other inputs drift unnoticed, although each can change the result of a case:

• The model identifier and settings such as effort or output limits
• Tool descriptions, which Anthropic's guidance on writing tools says can shift results noticeably with small wording changes
• The tool implementation, such as an MCP server deployed on its own schedule
• Business rules and thresholds, often kept in a configuration table that operations staff edit
• A retrieval index that is rebuilt every night
• The workflow code, the framework version and the evaluation graders

One manifest per release

Collect these references in a single file that is generated at release time and stored with the code. Its identifier becomes the agent version. The example is hypothetical; the commit references and versions are placeholders.

agent: invoice-matching
release: 2026.10.3
model:
  id: claude-sonnet-5-5          # exact ID sent to the provider
  settings: {effort: medium, max_tokens: 4000}
prompt: prompts/invoice-agent.md@3f2a91c
examples: examples/invoice-fewshot.jsonl@3f2a91c
tool_descriptions: tools/descriptions.yaml@3f2a91c
tool_servers:
  erp-mcp: 1.8.2
policy: ap-tolerance-policy v4
retrieval_index: supplier-terms build 2026-10-20
workflow_code: 3f2a91c
lockfile: uv.lock@3f2a91c
test_set: invoice-cases v12, passed 2026-10-21

Stamp the release on every case record and every span. The OpenTelemetry semantic conventions for agent spans include a gen_ai.agent.version attribute for this, still marked as Development status. Record the exact model identifier you send. Anthropic's model overview, for example, states that every current Claude model ID is a pinned snapshot, and it also lists retirement dates, which is one more reason not to depend on rerunning an old version later.

Reconstruction is not replay

To explain a case later you need the release, the inputs as the agent read them, each tool call with its result, the model outputs and any human decisions. Checkpoints help. In LangGraph, each checkpoint holds the state values, metadata, a timestamp and a link to the previous checkpoint, and get_state_history lists them for a thread.

Replaying is a different thing. The same documentation explains that resuming from an earlier checkpoint re-runs the later nodes, including model calls and API requests, so it produces a new run against today's systems. Use the stored record to explain the past, and use reruns against the current release for regression testing. Note too that LangGraph's persistence documentation recommends pruning or a retention policy for checkpoints, so the retention you need for explanations has to be decided deliberately.

Cases that span a release

A case paused for approval on Monday may resume after Wednesday's release. Durable workflow engines treat this as a known problem. Temporal's documentation explains that workflow code must be deterministic for replay, that adding, removing or reordering certain steps without versioning causes a non-determinism error for running workflows, and that it offers worker versioning and patching to manage the change.

Whatever framework you use, choose a rule: in-flight cases either finish on the release they started with, or move to the new one through an explicit migration of their saved state. Record the release on each step, not only on the case, so a mixed case can still be explained.

Common gaps

The gaps tend to sit outside the main repository: a prompt edited in a vendor console, a tool server deployed by another team, a threshold changed in a database, an index refreshed overnight. Each should either be pulled into the manifest or recorded per case with the value used. Treat any change to a manifest entry as a release that goes through the test set. A useful first step is to write the manifest for the agent you run today and mark every field you cannot fill.