Cosignary · live demo

Be the watcher for an AI support agent

A retailer's AI agent handles customer service: refunds, store credits, order changes and replies. You play the quality-control person: spot-check a fair random sample of its work, try to cheat the checks and watch them get caught, then approve a big refund from a phone and see an automatic shut-off stop a flood of refunds.

1

Lock in the audit

Before the agent does anything, the system picks a secret random number (the "seed") that will decide which actions get checked. Only a fingerprint of it goes to an outside party, so it can't be swapped later, and nobody, the AI included, knows yet what will be checked.

2

Let the agent work

Each action goes into a record that can't be quietly edited: every entry is linked to the one before it, so changing any of them shows. The outside party keeps a fingerprint of the record so far.

3

Pick the sample

The secret number picks which actions get checked: more often for money, less for routine replies. Hidden test cases are mixed in at random. (For the technical: an action is checked when HMAC(seed, action fingerprint) falls under its category's rate.)

4

Review the cards

Judge each action against the policy shown. The facts come from the store's order system; what the agent says it did is shown separately, so you can spot when they don't match. "Not sure" is always safe. Keys: Y / N / U.

5

Reveal and verify

The secret number and the list of hidden test cases are now published, so anyone with the record can redo every step and confirm the checks were fair.

6

The report

What your spot checks prove about all 400 actions, and how many hidden test cases you caught.

7

Try to cheat

Play the dishonest insider. Each attempt runs on a copy of the record, so you can see exactly which check catches it; the original stays intact, and Start over resets everything.

8

Real-time controls

works on its own

Before an action runs, the gate checks it. A large refund is held until a person signs off. A burst of small refunds trips a circuit breaker, an automatic shut-off, because a per-refund limit can be beaten by doing lots of small ones. No answer by the deadline means deny.

0allowed
0held
0blocked
Refund breakerclosed
0 of 20 refunds and $0 of $2,000 in the last 10 minutes
Cosignary

No alerts. Try one of the buttons.