By the end of this lesson you will be able to
- Assign any output to one of three checking tiers
- Spot the specific signals that predict a fabricated answer
- Set up a spot-check regime for recurring automated tasks
If verification takes as long as doing the work, AI has saved you nothing. The goal is not maximum checking — it is enough checking, aimed at the places failures actually happen.
Three tiers
Assign every output to one before you look at it, not after.
Tier 1 — Read it. That is the check.
Internal notes, first drafts, brainstorms, reformatting, anything you will substantially rewrite anyway. If a mistake would cost you a minute, do not spend five minutes preventing it.
Tier 2 — Check every specific, then send.
Client emails, quotes, summaries someone will act on. Verify names, numbers, dates, commitments, and anything the model asserted that you did not supply. Prose and structure you can accept.
Tier 3 — Verify against source, and a person signs it.
Anything legal, financial, medical, regulatory, or public. Every factual claim traced to a real document. A named human is accountable for it. AI drafts; it does not approve.
Signals that predict a fabrication
You cannot tell a wrong answer from a right one by how it reads. But you can tell which answers are structurally risky, and check those harder.
- A precise number you did not supply. Round numbers get invented less often than suspiciously exact ones.
- A citation, statistic, or quote attributed to a named source. The single highest-risk output there is.
- An answer about anything current — prices, rules, availability, who runs what.
- Unusual fluency on a narrow question about your own business. It does not know your business; it knows what businesses like yours usually say.
- A confident answer to a question you suspect is genuinely ambiguous. Real ambiguity should produce hedging.
Checking recurring tasks without checking everything
Once a task runs regularly, per-item checking stops being viable. Shift from checking outputs to testing the process.
Check the first ten properly
Every one, in full. You are establishing a baseline failure rate and learning what the failures look like. Ten clean results is weak evidence; ten with two subtle errors is very strong evidence.
Drop to sampling
One in ten, then one in twenty as confidence builds. Sample randomly rather than checking the first of each batch — systematic sampling misses systematic failures.
Keep a small fixed test set
Five to ten inputs where you know the correct output. Run them monthly. This is what catches silent drift when a model updates underneath you.
Re-check properly after any change
New model version, edited prompt, changed input format. Go back to checking every item until the baseline is re-established.
Can a second AI check the first one?
It catches some things — inconsistency, arithmetic, contradictions with a supplied source. It does not catch shared blind spots, and it cannot verify a fact against the real world. Useful as a filter before human review, not as a replacement for it.
How do I stop my team rubber-stamping?
Make the check specific rather than general. "Confirm the three figures against the invoice" gets done. "Review for accuracy" becomes a click. Name what to look at.
Is it ever safe to skip checking entirely?
When the cost of being wrong is genuinely near zero — internal drafts, throwaway ideas, things you will rewrite anyway. That is Tier 1, and being honest about what belongs there is what makes the other tiers affordable.
Key takeaways
- Assign a tier before you read the output, not after.
- Tier 2 — routine client-facing work — is where the damage actually happens.
- Specific numbers, citations, and anything current are the high-risk shapes.
- For recurring tasks, test the process: ten full checks, then sampling, plus a fixed monthly test set.
- Silent drift after a model update is the failure most people never see coming.