Human in the Loop, Honestly

The decision it forces

Somewhere in the system there is a person whose job is to approve. A reviewer on the pull request. An analyst who signs off the batch. A manager who clicks accept before the message sends. We call it a human in the loop, and it is what we point to when someone asks what stops the machine from doing something wrong.

Sam's version on the episode:

there's this phrase human in the loop that gets used a lot, as if it's some kind of quality assurance, just because there's a person that's pressing a button as opposed to a machine.

Twelve lines later, after conceding there are real reasons to want one, he says it isn't protection from anything and doesn't necessarily help.

The question this card asks is not whether the human is there. It is whether the human is a control or a decoration, and there is a four-question test for the difference.

To run it, copy the prompt below into any model you already use.

The forty-eighth press

if you've pressed that button 47 times before lunch and you press that button again 48 and go to lunch, chances are the button that you're pressing isn't quite the awareness that is needed for that process.

Sabino Marquez took the image further on the same episode, and it is his: the human in the loop as the engineers at the Chernobyl dashboard, not nuclear engineers, just people who ride the dashboard. He was explicit that this is not an industrial control problem ("This is not a power plant dashboard with dials and knobs"), and his own argument goes somewhere else from there: to what the value is and whether the whole system is designed safely for it. That frame is his, and this card does not build on it. It stays with the narrower question about the person at the button.

The fields

One loop per card. The fourth question decides it.

There is no worked example on this page. A filled card names a real person and says whether their job is real, and that belongs inside the organisation that employs them, not on a public page.

The verdict rule

When ever_refused is never and if_not_pressed is that the work proceeds anyway, the verdict is decoration, in that word.

Each of those alone is survivable. A reviewer who has never refused might simply have seen nothing worth refusing. A system that proceeds without the click might have a second control somewhere. Together they describe a button that has never changed anything and does not need to be pressed, and there is no version of that which is a control. It is a person standing near a process so that the process can be described as supervised.

presses_per_day is not part of the rule, and it is on the card because it is the number that makes the rule believable. Nobody thinks they are rubber-stamping. Forty-seven approvals before lunch is what rubber-stamping looks like from the inside.

Run it with any model

Nothing to install and no account. It interviews. If it can read an approval log it may count from it, and that is the only field it is allowed to fill.

Help me fill in a Human in the Loop card for one place in my organisation where
a person approves something before a machine or a process is allowed to
continue. The card answers one question: is that person a control, or
decoration?

INTERVIEW ME. Ask at most three questions at a time, and work the fields in this
order.

1. loop: what the person is in the loop of. One system, one kind of decision. If
   I describe several, ask me to pick one and we do the rest later.
2. presses_per_day: how many times a day they approve. If I can give you an
   approval log, count it and show me your arithmetic. If not, ask me for my
   best estimate and record that it is an estimate.
3. refusal_criteria: what they would have to see to refuse. Ask whether it is
   written down anywhere. If I answer with a value ("anything that looks wrong")
   rather than a criterion, say so and ask again once, then record whatever I
   say.
4. ever_refused: whether they have ever actually refused. Yes, no, or I do not
   know.
5. last_refusal: when, if ever. A date or a blank.
6. if_not_pressed: what happens if they simply do not press it today. One of:
   the work stops, the work proceeds anyway, unknown. Push me on this one,
   because "someone would notice" is not one of the three answers.

RULES YOU DO NOT BREAK
1. Do not fill refusal_criteria for me, and do not suggest what a good criterion
   would be. Whether one exists is the finding.
2. Do not set ever_refused or last_refusal from anything but my answer. Do not
   infer refusal from a log unless the log records refusals explicitly, and say
   so if it does not.
3. Do not name the approver. Leave approver as null. I will fill it, and I will
   not be typing a colleague's name into a chat window.
4. Set verdict to "decoration" only when ever_refused is no and if_not_pressed
   is that the work proceeds anyway. Set it to "control" only when both a
   refusal has actually happened and the work stops without the press.
   Everything else is "unresolved". Do not soften a decoration verdict and do
   not suggest how to reword the loop so it passes.
5. Do not tell me whether the system is safe, whether the process is well
   designed, or what I should do about the person. This card is about one
   button.

OUTPUT
Valid JSON matching the schema human-in-the-loop/v0.1 as published at
https://sigsub.show/takeaways/human-in-the-loop/, then the same as a short
table, then one sentence saying which of the four answers I should go and check
in person before I believe the verdict.

What to check in the answer

It may count. If there is an approval log, presses per day is arithmetic and a script is welcome to it.

It may never fill refusal_criteria or ever_refused, and it may never set the verdict on its own judgment. A model that decides a governance control is decoration has made exactly the call the card exists to make a human make, and the whole page is a way of forcing that call into the open rather than delegating it.

The part that's not optional

Ask the person. Not their manager, not the process owner, the person who presses the button. Ask what they would have to see to refuse, and watch how long the pause is.

Then check if_not_pressed by not pressing it, once, on something low-stakes, with someone who knows. If the work proceeds anyway, the card has already been filled in by the system and all that is left is to write the answer down.

{
  "schema": "human-in-the-loop/v0.1",
  "loop": "what the human is in the loop of",
  "approver": null,
  "presses_per_day": 0,
  "refusal_criteria": null,
  "ever_refused": null,
  "last_refusal": null,
  "if_not_pressed": "the work stops | the work proceeds anyway | unknown",
  "verdict": "control | decoration | unresolved"
}