Empty Isn't an Argument

Watch the two-minute walkthrough 2:11

A two-minute walkthrough: why a green light can mean a workflow ran without working, the drill, the verdict, and how to run the prompt in any model.

Play video

The decision it forces

Does our green light mean it worked, or only that it ran? Somewhere a job runs on a schedule. It pulls from a source, does something with what it finds, and writes a line somewhere that says it finished. Most mornings the line is green, and most mornings nobody opens the output, because green is what the line is for.

The question isn't whether the workflow is well built. It's whether the thing that tells us it worked can tell the difference between working and running. There's one way to find out, and it takes an afternoon: feed it nothing, twice, and see what it reports.

To run it, copy the prompt below into any model you already use.

What green was measuring

Sam's version on Episode 11, about his own systems, on a holiday weekend:

So basically certain kinds of failures about these research tasks would simply come back empty. And empty isn't an argument. So as long as one of this cascade worked, I would get the signal that the research had been done, until I saw this one thing that was like, that that isn't right, on a Friday night, right? And I spent the next three days basically trying to figure out why workflows were coming in green, everything's fine, but the results were not.

The failure wasn't a crash. A crash would have been caught. One leg of a chain returned nothing, nothing was handled as a result, and the run reported what every run reports when it reaches the end. Ankit Patel, who runs a company of about sixty people on systems he builds himself, had met the same shape months before:

you have health checks. You said everything was green. What what the heck, man? ... it did return something. It returned nothing. And that counts because it did something.

That is the whole problem in nine words. The check was honest about the thing it checked, which was whether the job ran. Nobody had asked it the other question, and a light that cannot fail is not a light.

Ankit's own answer is a method, predefined success criteria with code checking what code can check, and it is his. This page does not build on it. What follows is smaller and comes first: before writing the criteria, find out whether the light you already have is lying.

The drill

Six steps. Nobody on the episode proposed this; it is what the two stories imply. The sixth step is the one that gets skipped.

Twice, because one empty run proves the drill ran. The second is the one that tells you whether the result is stable, which is the same reason Lee Rodrigues digs a second hole on Episode 9. The two drills are cousins: his checks the ends before a rebuild, this one checks the light on something already running.

There's no worked example on this page beyond Sam's and Ankit's stories above: a filled card names a workflow and what its status really measures.

The verdict rule

When reported is success on a run where produced was nothing, in either run, the verdict is not a check, in those words. The light measures completion. It has never measured the work, and every green morning before today was the same green.

When both runs report failure, or report success and produced is a correct, explicit statement that there was nothing to do, the verdict is a check. The signal told the difference.

Everything else is unresolved: one run of each, a report nobody can read, a workflow that would not run at all on empty input. Unresolved is a fine first result. It means the question has been asked.

The sixth step's sentence, what the signal would have had to say, is not part of the rule and it is on the card because it is the specification Ankit is talking about, written in the moment you most know what it should contain.

Run it with any model

Nothing to install and no account. The model's job is to make you name the signal, design the empty input and keep the two columns apart. It does not run the workflow and it does not tell you the result.

One privacy note before you paste: describe the workflow and its signal. Do not paste logs, credentials or the contents of what it processes.

Help me fill in an Empty Run card for one automated workflow I own. The card
answers one question: does the thing that tells me this workflow worked know the
difference between working and running?

INTERVIEW ME, three questions at a time, in this order.

1. workflow: what it is, in one sentence, and how often it runs. If I describe
   several, ask me to pick one and we do the rest later.
2. success_signal: what currently tells me it worked. A status colour, the
   absence of an error, a log row, an email, a dashboard tile. Push me from "I'd
   know" to the exact thing I look at. This is what is under test, so be
   precise.
3. normal_input: what it is normally fed.
4. empty_input: what "nothing" looks like for this workflow. Offer no more than
   two candidates (an empty source, a feed with no new items, a file with
   headers and no rows, a query that matches nothing) and let me pick or write
   my own.
5. safety: ask me how I will run it without any outward effect: a copy, a
   staging path, or every send, post, write and payment switched off. If I
   cannot answer, stop and tell me not to run it yet.
6. Tell me to go and run it on the empty input, twice, and come back with two
   things for each run, kept separate: what it produced, and what it reported.
   Do not fill in results. Do not guess results. Record what I say exactly as I
   say it.
7. required_signal: only after both runs are recorded, ask me what the signal
   would have had to say for me to believe it. One sentence. If I cannot say,
   record null and say that null is the finding.

RULES YOU DO NOT BREAK
1. Do not run the workflow, simulate it, or tell me what it will probably
   report.
2. Do not merge produced and reported into one answer. If I say "it worked", ask
   which of the two I mean, once, and record both.
3. Do not set the verdict from anything but the recorded runs. "not a check"
   when any run reported success and produced nothing. "a check" when both runs
   reported failure, or reported success alongside an explicit statement that
   there was nothing to do. Everything else is "unresolved". Do not soften "not
   a check".
4. Do not propose a fix, a better signal, a checker or a monitoring tool. The
   card is a measurement of the signal I have. Improvements come after it, not
   inside it.
5. Do not ask for logs, credentials, customer data or the contents of what the
   workflow processes. A description is enough.

OUTPUT
Valid JSON matching the schema empty-run/v0.1 as published at
https://sigsub.show/takeaways/empty-run/, then one sentence naming which green
light I should stop trusting first.

What to check in the answer

It may help name the signal and design the empty input, because that is a conversation. It may never report a run, merge the two columns, or set the verdict from anything but the runs you typed in. A model that tells you the workflow probably handles empty input fine has done exactly what the status light did.

The part that's not optional

Run it the second time even when the first run was clear. One result is the drill working; two is the workflow's answer.

Then write the sentence in step six before you fix anything. It is the first line of the success criteria Ankit is describing, and the moment after you have watched a green light lie is the only moment you will write it honestly.

{
  "schema": "empty-run/v0.1",
  "workflow": "what it is and how often it runs",
  "success_signal": "the exact thing that currently says it worked",
  "normal_input": "what it is normally fed",
  "empty_input": "what nothing looks like for it",
  "safety": "how it ran with no outward effect",
  "run_1": { "produced": null, "reported": null },
  "run_2": { "produced": null, "reported": null },
  "required_signal": null,
  "verdict": "a check | not a check | unresolved"
}