Guide · AI oversight staffing

How many people does AI oversight take?

A firm moves work to AI in stages. Each new kind of work is fully reviewed until it earns trust, then spot-checked at random so its error rate stays proven. Anything that can't be undone waits for a person to sign. These two calculators count the people that takes, from a 10-person office to a million-person company.

Short answer: far fewer people than the work AI takes over, but not none. Random spot checks are a fixed cost per kind of work, not per action, so they weigh heavily on small firms and almost vanish in large ones. What doesn't shrink with size is the people who sign irreversible actions and handle cases the AI hands back. With the example assumptions below, oversight settles near 27% of a 10-person team, 10% of 100, 5% of 1,000 and under 3% of a million, all at 99.9% confidence.

Plan a staged rollout

Move a share of the work to AI in up to four stages, one a year by default. The chart shows where the people are each month: doing the work that's left, fully reviewing new work, spot-checking settled work, or signing off and handling hand-backs.

The firm
The AI's work
Proof standard
Confidence
People in the loop
Rollout
Starts monthWork on AI %Kinds of work

The default is one stage a year over four years. Kinds of work are the categories that each get their own random sample, such as refunds, invoice coding or contract redlines. Each stage's new kinds start in full review.

–

Where the people are, month by month

Full-time people, stacked. The dashed line is the team before AI.

Each stage, at its start and once settled

Start is the stage's first month, while new kinds of work are in full review. Settled is its last month, on spot checks.

What the higher standard costs

The same plan run at 95% confidence, compared with the confidence chosen above.

Oversight by firm size, at 99.9% confidence

The settled state, after full review has ended, for firms from 10 to 1,000,000 people. Firm size counts the people doing work that can move to AI. The dashed line runs the same firm at 95%, so the gap is what 99.9% costs.

The work
Proof standard
People in the loop
Kinds of work

Small firms: 10 to 100 people

Mid-size: 100 to 1,000 people

Large: 1,000 to 1,000,000 people

Oversight as a share of the workforce, 10 to 1,000,000

The same numbers on one log scale, as a percent of the people in scope.

Why confidence is cheap and tight limits are not

Moving from 95% to 99.9% confidence cuts the chance of a wrong conclusion fifty-fold, yet it only needs about 2.3 times as many checks: the sample grows with the logarithm of the risk you accept. Tightening the error limit is what costs. Proving a rate under 0.1% instead of 1% takes about ten times the checks, because the sample grows with one over the limit.

So the useful question for a business isn't "how sure do we need to be?" It's "what error rate can this kind of work tolerate?" Confidence can be set high almost everywhere.

What the calculators show

Assumptions and limits

Every number on this page is an example; change any of them. The check count is the smallest monthly sample per kind of work that proves the true rate is under the limit at the chosen confidence (an exact Poisson upper bound, which tracks the Clopper-Pearson bound closely at these rates), sized so that an AI really running at the planning error rate passes in at least nine months out of ten. A sample sized only for the expected number of errors would fail the proof in about a third of months. If a kind of work has fewer actions in a month than the sample needs, every action is checked.

Left out: reviewer training, rework on errors found, audit and compliance staff outside the oversight team, and the cost of the oversight system itself. Volume is spread evenly across kinds of work; in practice a few kinds dominate. Moving people into oversight roles through attrition and redeployment, rather than layoffs, is a choice the model can't make for you, but it shows when those roles are needed.

Frequently asked questions

How many people does it take to oversee AI agents?

It depends on how many kinds of work the AI does, how long a check takes, and how many actions can't be undone, more than on volume. With 15-minute tasks, 5-minute checks and 99.9% confidence, oversight settles near 27% of a 10-person team, 10% of a 100-person team, 5% of 1,000 and under 3% of a very large workforce.

Does the number of checks grow with the number of AI actions?

No. A random sample proves an error rate, and the sample needed depends on the confidence and the error limit, not on how many actions there were. It grows with the number of kinds of work, because each kind needs its own sample.

How much more does 99.9% confidence cost than 95%?

About 2.3 times as many checks, whether or not the AI's few real mistakes turn up in the sample. Because checks are a fixed cost per kind of work, that is a large share of oversight in a small firm and almost nothing in a large one.

Why does a small firm need proportionally more oversight?

A small firm needs the same sample per kind of work as a large one but has far fewer actions to spread it over, so it ends up checking a large share of its AI's work. A firm with 300 payroll actions a month checks all 300.

Try it

Be the person checking the AI's work

Spot-check an AI support agent's refunds, approve a held refund from a phone, trip a circuit breaker, then try to cheat the audit.