How much human review to budget before you automate a process

Review is a staffing question

The automated part of a process scales with compute. The people who check its results do not. If nobody budgets their time, review turns up as a backlog in someone's inbox, cases wait, and the turnaround that justified the project disappears. The NIST AI Risk Management Framework asks that processes for human oversight be defined, assessed and documented. In practice that means names, hours and a queue someone is responsible for.

Three ways cases reach a reviewer

• Every case. Usual at the start, and permanent for actions your policy reserves for a person, such as releasing a payment.
• Cases that trip a trigger: low model confidence, an amount above a limit, a failed rule check, or the agent stopping to ask. Anthropic's 2024 article on agents describes pausing at checkpoints for human feedback or when the agent meets a blocker.
• A random sample of the rest. Amazon's Augmented AI service, now closed to new customers, was built around this pairing of low-confidence review and random sampling, and the pattern is worth copying in any stack. The sample is how you learn the error rate of cases nobody else looked at.

A capacity sheet for a hypothetical process

The figures are assumptions for illustration. Replace each one with what your pilot measures.

1. 1

   Volume

   2,000 cases a week, with 30 percent of them arriving on Mondays.

2. 2

   Phase one: full review

   Every case reviewed at 3 minutes: 6,000 minutes, or 100 reviewer hours a week.

3. 3

   Phase two: triggers and sample

   30 percent of cases trip a trigger and take 4 minutes, because they are the harder ones: 2,400 minutes. A 10 percent sample of the other 1,400 takes 2 minutes each: 280 minutes. Together about 45 hours a week.

4. 4

   Peak check

   On a Monday in phase two, 600 cases produce about 13 reviewer hours. If one reviewer has 6 productive hours a day for this work, Monday needs at least three people or the queue carries over into Tuesday.

Two things follow from the sheet. Phase one needs more than twice the hours of phase two, so the pilot budget should not assume phase-two staffing. And the phase-two figure depends heavily on the trigger rate. If 45 percent of cases trip a trigger instead of 30, the same week needs about 64 reviewer hours rather than 45. Measure the trigger rate on real cases before you fix the headcount.

What makes a review take two minutes rather than ten

Minutes per review are a design result, not a fixed property of the process. A reviewer is fast when the proposed result arrives with its evidence, such as the source fields, the document excerpt and the rule that fired, together with the reason the case was flagged. Accepting or correcting should take one action in the tool the reviewer already uses, and every correction should be stored as structured data. Anthropic's evaluation guidance recommends turning production failures into test tasks, and these corrections are where those tasks come from.

When the review share may come down

Lower it one case type at a time, when the random sample of unreviewed cases has stayed below an agreed error level for an agreed period. Keep the sample after that, because it is the only measurement of cases nobody checks, and raise the share again if the sampled error rate climbs. Also check the reviewers. A person who approves hundreds of correct results can stop looking closely, so a few cases with known errors mixed into the queue show whether review still catches them.

Putting reviewer time into the plan

Budget phase one at full review, and phase two as a range rather than a promise. The hours belong to the process team, not to IT, and the reviewers need authority to correct a result, not only to approve it. In the agent projects we build, every agent starts at the level where a person decides each case, and its scope widens only after we have reviewed the results with you.

The full cost of an AI workflowHow much an agent decides