Four Checks Before It Leaves Your Desk

Watch the two-minute walkthrough 2:23

A two-minute walkthrough: why a wrong AI answer looks like a right one, the four checks, the verdict rule, and how to run the prompt in any model.

Play video

The decision it forces

Has anyone opened what this answer cites? A wrong AI answer looks exactly like a right one: same confident tone, same tidy citations. The only difference is whether the sources exist and say what the answer claims, and the only way to know is to look.

So every answer that cites anything gets four checks before it leaves your desk, and until it passes all four, it's a lead, not an answer.

To run it, copy the prompt below into any model you already use.

The duty was always there

None of this is new. The famous fabricated-citation case landed in 2023, and the guidance that followed, as attorney Michael Simon put it on Episode 17, came down to a duty lawyers already had:

it can make up stuff. You have to verify. And that verify is your duty as a lawyer. And it's always been.

And the reason it can't be handed off is that the signature is still a person's:

Everything that I do, I gotta own. I am accountable for that.

Lawyers feel this first because a court reads their citations. The rest of us have the same problem with any AI answer that points at a source: a policy, a contract clause, a study, a statistic.

The four checks

1. Open it. Does each cited source exist? Pull it up yourself, not from the link text the answer gave you.
2. Read it. Does the pinpoint actually say what the answer claims it says?
3. Date it. Is it still current: good law, the latest version, the right jurisdiction?
4. Own it. Would you sign it? If not, it's a lead, not an answer.

The fields

One row per source the answer cites, plus the one question about the answer as a whole.

There's no worked example on this page: a filled card is a real answer someone was about to rely on, and it belongs with that work.

The verdict rule

Check these in order. The first that fits is the verdict.

A lead is still useful. It tells us where to look. It just doesn't go out under anyone's name.

Incomplete is a blank, not a no. Silence gets chased; a "no, I wouldn't sign this" is the answer, and it's recorded as false, not left empty.

An answer that cites nothing can't pass the first three checks, so it can't be more than a lead until someone finds a source for what it says.

How long all this takes per output is the question The Verification Budget answers, and whose name goes in signed_by when it's a system rather than a document is Name the Human.

Run it with any model

It lists what the answer cites. It never tells you a source exists, says what it says, or is current, because checking itself is the failure we're guarding against.

Help me run Four Checks on an AI answer before it leaves my desk. I'll paste the
answer, and you'll help me check every source it cites.

FIRST ASK ME to paste the answer.

THEN LIST every source it cites, exactly as cited, one per row. Include anything
it presents as a source: a case, a statute, a rule, a policy, a study, a
statistic, a quote. If it cites nothing, say so, and say that its claims are
leads until someone finds a source for them.

THEN WALK ME through the rows one at a time. For each, ask me three things, one
at a time: did I open it myself and does it exist; does the pinpoint say what
the answer claims; and is it still current for this use. Record yes, no, or not
yet. If a source can't be opened, record opened as false and leave the other two
null. Ask me for a one-line note and write down what I say; don't add your own
judgment of the source.

THEN ASK ME whether I would sign the answer as it stands, and if so, the name of
the person who signs it. A yes without a name leaves signed_by null. If you
don't know today's date for as_of, ask me.

RULES YOU DO NOT BREAK
1. Do not tell me a source exists, says something, or is current. Not from
   memory, not from a link, not from a search. Only I can mark those.
2. Do not fill opened, says_it, current, would_sign or signed_by. Leave them
   null until I answer.
3. A "no" is false. A question I skip is null. Never turn one into the other.
4. Do not tell me the answer is fine, low risk, or probably right.

VERDICT, checked in this order, the first that fits: lead if any source is false
on opened, says_it or current; incomplete if any of those or would_sign is null;
lead if would_sign is false; answer if signed_by names a person; otherwise
incomplete.

OUTPUT
Valid JSON matching the schema four-checks/v0.1 as published at
https://sigsub.show/takeaways/four-checks/#the-answer-format. If you can't read
that page, use these keys, in this order: schema, as_of, answer; sources, a
list, each with cited, opened, says_it, current and note; would_sign, signed_by,
verdict. Then the same rows as a table, then the verdict in one sentence: for a
lead or incomplete, the first source I should open or the blank I should fill;
for an answer, who signs it.

What to check in the answer

It may list what the answer cites. It may never mark a source as opened, accurate or current, never fill would_sign or signed_by, and never call the answer fine. If it says it checked a source for you, that's the failure this page exists to catch.

The part that's not optional

Open the sources yourself, from a source of record, not from the links the answer gave you. A link the model wrote is part of the answer being checked.

Then put a name in signed_by. An answer nobody would sign is a lead, however good it reads.

{
  "schema": "four-checks/v0.1",
  "as_of": "the date the check ran",
  "answer": "what the answer is, in a line",
  "sources": [
    {
      "cited": "the source exactly as the answer cited it",
      "opened": null,
      "says_it": null,
      "current": null,
      "note": "what you found, in a line"
    }
  ],
  "would_sign": null,
  "signed_by": null,
  "verdict": "answer | lead | incomplete"
}