Human in the Loop, Honestly

The decision it forces

Somewhere in the system there is a person whose job is to approve. A reviewer on the pull request. An analyst who signs off the batch. A manager who clicks accept before the message sends. We call it a human in the loop, and it is what we point to when someone asks what stops the machine from doing something wrong.

Sam's version on the episode:

there's this phrase human in the loop that gets used a lot, as if it's some kind of quality assurance, just because there's a person that's pressing a button as opposed to a machine.

And, twelve lines later and after conceding there are real reasons to want one:

However, it is not protection from anything, and it doesn't necessarily help.

The question this card asks is not whether the human is there. It is whether the human is a control or a decoration, and there is a four-question test for the difference.

The forty-eighth press

if you've pressed that button 47 times before lunch and you press that button again 48 and go to lunch, chances are the button that you're pressing isn't quite the awareness that is needed for that process.

Sabino Marquez took the image further on the same episode, and it is his:

the human in the loop is just like the the engineers sitting at the Chernobyl dashboard. They may they're ... not nuclear engineers, they just ride the dashboard.

He was explicit that this is not an industrial control problem ("This is not a power plant dashboard with dials and knobs"), and his own argument goes somewhere else from there: to what the value is and whether the whole system is designed safely for it. That frame is his, and this card does not build on it. It stays with the narrower question about the person at the button.

The four questions

One loop per card. The fourth question decides it.

There is no worked example on this page. A filled card names a real person and says whether their job is real, and that belongs inside the organisation that employs them, not on a public page.

The verdict rule

When ever_refused is never and if_not_pressed is that the work proceeds anyway, the verdict is decoration, in that word.

Each of those alone is survivable. A reviewer who has never refused might simply have seen nothing worth refusing. A system that proceeds without the click might have a second control somewhere. Together they describe a button that has never changed anything and does not need to be pressed, and there is no version of that which is a control. It is a person standing near a process so that the process can be described as supervised.

presses_per_day is not part of the rule, and it is on the card because it is the number that makes the rule believable. Nobody thinks they are rubber-stamping. Forty-seven approvals before lunch is what rubber-stamping looks like from the inside.

The schema

{
  "schema": "human-in-the-loop/v0.1",
  "loop": "what the human is in the loop of",
  "approver": null,
  "presses_per_day": 0,
  "refusal_criteria": null,
  "ever_refused": null,
  "last_refusal": null,
  "if_not_pressed": "the work stops | the work proceeds anyway | unknown",
  "verdict": "control | decoration | unresolved"
}

approver is a person, and it is null until a person fills it. A role name in that field is a blank with extra steps, which is the same rule Name the Human applies to its accountable column.

Run it with any model

Nothing to install and no account. It interviews. If it can read an approval log it may count from it, and that is the only field it is allowed to fill.

Help me fill in a Human in the Loop card for one place in my organisation where a person approves something before a machine or a process is allowed to continue. The card answers one question: is that person a control, or decoration?

INTERVIEW ME. Ask at most three questions at a time, and work the fields in this order.

1. loop: what the person is in the loop of. One system, one kind of decision. If I describe several, ask me to pick one and we do the rest later.
2. presses_per_day: how many times a day they approve. If I can give you an approval log, count it and show me your arithmetic. If not, ask me for my best estimate and record that it is an estimate.
3. refusal_criteria: what they would have to see to refuse. Ask whether it is written down anywhere. If I answer with a value ("anything that looks wrong") rather than a criterion, say so and ask again once, then record whatever I say.
4. ever_refused: whether they have ever actually refused. Yes, no, or I do not know.
5. last_refusal: when, if ever. A date or a blank.
6. if_not_pressed: what happens if they simply do not press it today. One of: the work stops, the work proceeds anyway, unknown. Push me on this one, because "someone would notice" is not one of the three answers.

RULES YOU DO NOT BREAK
1. Do not fill refusal_criteria for me, and do not suggest what a good criterion would be. Whether one exists is the finding.
2. Do not set ever_refused or last_refusal from anything but my answer. Do not infer refusal from a log unless the log records refusals explicitly, and say so if it does not.
3. Do not name the approver. Leave approver as null. I will fill it, and I will not be typing a colleague's name into a chat window.
4. Set verdict to "decoration" only when ever_refused is no and if_not_pressed is that the work proceeds anyway. Set it to "control" only when both a refusal has actually happened and the work stops without the press. Everything else is "unresolved". Do not soften a decoration verdict and do not suggest how to reword the loop so it passes.
5. Do not tell me whether the system is safe, whether the process is well designed, or what I should do about the person. This card is about one button.

OUTPUT
Valid JSON matching the schema human-in-the-loop/v0.1 as published at https://sigsub.show/takeaways/human-in-the-loop/, then the same as a short table, then one sentence saying which of the four answers I should go and check in person before I believe the verdict.

What a machine may never do

It may count. If there is an approval log, presses per day is arithmetic and a script is welcome to it.

It may never fill refusal_criteria or ever_refused, and it may never set the verdict on its own judgment. A model that decides a governance control is decoration has made exactly the call the card exists to make a human make, and the whole page is a way of forcing that call into the open rather than delegating it.

The part that's not optional

Ask the person. Not their manager, not the process owner, the person who presses the button. Ask what they would have to see to refuse, and watch how long the pause is.

Then check if_not_pressed by not pressing it, once, on something low-stakes, with someone who knows. If the work proceeds anyway, the card has already been filled in by the system and all that is left is to write the answer down.