The Second Hole

The decision it forces

We are about to rebuild something. Replace the tool, rewrite the integration, migrate the platform, dig the trench. The plan is detailed, the quote is in, and everyone agrees the middle is the problem.

Lee Rodrigues, on the episode this comes from, was a day away from paying to have forty feet of front yard dug up:

Ladies and gentlemen, I was about to pay someone $4,000 to dig a ... trench through the middle of my front yard to replace a new water line. And by the time they drug the trench, they would go, ha! Look! It's a new waterline. What do you say we dig out and replace your new waterline with a newer waterline?

The question this drill asks is not whether the rebuild is well planned. It is whether anyone has checked both ends.

To run it, copy the prompt below into any model you already use.

The rule

Lee's, in one breath, is the sentence the episode is named after: run a test, run it twice, don't let the tool define the problem, dig the second hole before you trust the analysis.

The first hole was an accident. Clearing roots, he uncovered the pipe where it enters the house and found a brand new line under the rusty one. The second hole was the decision: he needed a second data point, where the line meets the city connection, so he dug a one-foot hole there and found new pipe sixteen inches down.

New at both ends. The middle was never the problem, and the trench was cancelled by a foot of digging. Sam's framing of the same move from the host's chair, input and output validated, so an assumption about the middle is now safe, is where this page's vocabulary comes from.

The subtraction Lee pairs with it is the reason the first hole matters:

It took 45 minutes of troubleshooting to get there because the problem statement had the solution wrapped in it.

Most rebuilds are like that. The thing that would have made the rebuild unnecessary was said in the first five minutes, and then everyone agreed on the trench.

The drill

Five steps. The fifth is the one that gets skipped.

Twice, because a single pass proves the check ran, not that the answer is stable. Sam on the episode: "testing is generally human work." The check at each end is done by a person looking at a real thing, not by asking the system whether it is fine.

There is no worked example on this page beyond Lee's yard, which is his to tell and is told above. A filled card names the thing we were about to rebuild, and that belongs with the team that nearly rebuilt it.

The verdict rule

When both the input check and the output check pass, twice, the rebuild is not necessary until someone writes down a reason it still is. That last field is the whole drill: not "the rebuild is wrong", but "here is what would have to be true", left blank if nothing comes to mind.

If either end fails, the drill has found the problem, and it is at the end, not in the middle. The rebuild plan was for the wrong thing.

Run it with any model

Nothing to install. The model's whole job is to make us name the ends and design the two cheap checks. It does not run them, and it does not tell us the result.

Help me fill in a Second Hole card before I rebuild something. The rule: check
the cheapest thing at the input end and the cheapest thing at the output end,
twice, and only rebuild the middle if an end fails.

INTERVIEW ME, three questions at a time, in this order.

1. problem: what I am about to rebuild, replace, migrate or dig up, in one
   sentence, with what it will cost. If I describe the plan, ask me for the
   thing instead.
2. input: what goes into it, concretely, at the point where it enters. Push me
   from "the data" to "the CSV the vendor drops at 6am".
3. output: what comes out, concretely, at the point where it leaves. Same bar.
4. input_check: the cheapest possible way to look at the input with my own eyes.
   Offer no more than two candidates and let me pick or write my own. Then ask
   what it costs in minutes.
5. output_check: the same for the output.
6. Tell me to go and run both, twice, and come back. Do not fill in results. Do
   not guess results. If I return with results, record them exactly as I state
   them.
7. rebuild_still_necessary: only after both runs are recorded, ask me what would
   have to be true for the rebuild to still be necessary. If I cannot say,
   record null and say that null is the finding.

RULES YOU DO NOT BREAK
1. Do not run the checks, simulate the checks, or tell me what they will
   probably show. The point is that a person looks at a real thing.
2. Do not tell me whether the rebuild is a good idea. You do not know, and
   neither do I until the ends are checked.
3. Do not propose a better rebuild, a different tool, or a cheaper vendor. The
   tool is not allowed to define the problem.
4. If I answer a check with "it looked fine", ask me what I actually saw, once,
   and record that.
5. Do not ask for credentials, customer data, or the contents of the system. A
   description of the check is enough.

OUTPUT
Valid JSON matching the schema second-hole/v0.1 as published at
https://sigsub.show/takeaways/second-hole/, then one sentence saying which end I
should go and look at first.

What to check in the answer

It may help name the ends and design the checks, because that is a conversation. It may never run a check, report a result, or fill rebuild_still_necessary. A model that says the input looks fine has dug the trench for us.

The part that's not optional

Dig the second hole even when the first one was convincing. Lee's first hole was clear evidence, and it was the second one that saved the yard.

Then, before the rebuild plan is reopened, write the sentence: what would have to be true for the rebuild to still be necessary. If nothing comes, the plan was for a trench nobody needed, and it cost a foot of digging to find out.

{
  "schema": "second-hole/v0.1",
  "problem": "what you were about to rebuild",
  "input": "what goes in, concretely",
  "output": "what comes out, concretely",
  "input_check": { "what": "", "cost": "", "result_run_1": null, "result_run_2": null },
  "output_check": { "what": "", "cost": "", "result_run_1": null, "result_run_2": null },
  "rebuild_still_necessary": null
}