Dig the Second Hole

One signal 🔭 One subtraction ➖ One analogy 🕳️

Created by Sam Rogers, building PAICE.work | Episode 9 with cohost Lee Rodrigues | The Sunday newsletter of the weekly show at sigsub.show

---

🔭 Signal: The buyers showed up before the checkout did

So I've spent months trying to sell an AI regulatory product (everyailaw.com) to law offices. Long sales cycle, wary buyers, very little traction. But within the last few weeks, the analytics turned strange: a spike of activity on the MCP surface that no human could have produced.

Turns out agents had found the product. It breaks human laws down into machine-readable text, which makes it one of the very few things on the web an agent researching legal obligations can actually use. And as we've discussed in previous newsletters, now agents have wallets: real budgets, delegated purchasing power, with Stripe, Amazon, and others writing the specs. There are more bots than people on the internet, and some of them are now shopping.

In my case, the surface they found doesn't yet have a payment gateway. It will soon, but the inhuman demand unexpectedly arrived before the checkout existed. If your product is legible to machines, you too may already have machine customers that you're neither counting nor charging. If so, who would notice? How would you tell?

The market that found me was not the market I was selling to. Only now that I've noticed can I begin to serve that market.

---

➖ Subtraction: Instructions posing as guardrails

Lee brought a story this week: Epic put an AI Darth Vader in Fortnite that would speak any prompt you typed at it, and two 16-year-olds crafted one long enough to make it forget its guardrails. It said some horrendous stuff, the recording hit YouTube, and PR/legal trouble ensued. The "guardrails" were simply instructions, and instructions get lost.

Which is the rule from this week's cold open: if you give an agent information and tell it not to share it, rest assured, it's going to share it. It's just a matter of time.

The mistake underneath is assuming an agent works like a computer: declare an edge, and it will respect the edge. That's not how LLMs work. An agent will do whatever it can to accomplish its goal, and security is not a property of the agent. It's a layer you put on top. Lee's example: you have access to everything in this folder, and no access to anything in that one. Instructions can be debated, discussed, interpreted. Architecture can't.

So subtract the instructions posing as guardrails. Ask "why isn't the agent doing what I told it" less, and "how do I scaffold the guardrails around the agent rather than inside it" more. Same upgrade as hallucinations: "how do I stop AI from hallucinating" has no answer; "how do I manage the context it has" does. I wrote more such question upgrades in my recent blog post: The Question is the Answer.

This snaps straight back to the signal: if machine customers are already in your funnel, you don't get to instruct them into good behavior either. Decide what they can reach.

Bounds you need kept are architecture, not instructions.

---

Watch, read, or listen

Watch the full episode: YouTube. Every format in one place: sigsub.show/episodes/ep-009. Also on Substack and LinkedIn.

Jump to a segment: the $4,000 trench · Claude asks the better question · the agents came shopping · the Vader jailbreak

---

🕳️ Analogy of the Week: The $4,000 Trench

"What do you say we dig out and replace your new waterline with a newer waterline?"

Lee's house is seventy-five years old, and the water line to the street was, as far as anyone knew, original. Rusty pipe visible at the surface, quotes in hand for a forty-foot trench through the front yard: $4,000. Then, digging out tree roots one morning, he uncovers the pipe below the rust: brand new line. His ego starts listing reasons the trench is still on. Instead he digs a second hole, one foot square, where the line meets the city connection. Sixteen inches down, new pipe there too. Input validated, output validated, and the middle stopped being a problem that needed fixing.

That is the enterprise tool being swapped for a newer tool while the data feeding it stays wrong. That is the LMS Lee nearly bent into a knowledge base until Claude asked why he was using one at all. When the what is fuzzy, the how gets loud.

"Is it better to dig this trench with a backhoe or a trencher? I have a better question. Do we need a trench at all?"

Call it the second-hole test: before you trust any analysis, an AI's, a vendor's, or your own, verify both ends yourself. Is this input actually right? Is this output actually right? Two spot checks of human work. The middle can wait until both ends have earned it.

Don't let the tool define the problem.

---

🎵 Closing: Upgrade the question, keep the yard

Single loop learning fixes the mistake. Double loop learning asks whether the problem statement was ever right. Most of what AI does for us, and to us, is follow the question exactly as asked, straight down into the trench, digging faster and deeper than we ever could alone.

So carry the second-hole test into your Monday: one analysis you were about to accept, two ends checked by a human, then the decision. And the next time something in your numbers refuses to make sense, hold the panic. Sometimes something is wrong. Sometimes you have discovered a customer you did not yet know how to see.

Good luck,

Sam Rogers Second Ditch Digger

---

New here? Subscribe at sigsub.show and the next episode lands in your inbox.

This week's cohost: Lee Rodrigues, cohost since episode one, launching System2Focus at system2focus.com.

Related reading:

• Measuring 500 Nuclear Engineers (064): Douglas Hubbard's uncertainty reduction is the same instinct as Lee's split half troubleshooting, and Ep 7 gets name-checked on this week's tape.
• All Horsepower, No Radiator (058): the failures still aren't about smarts, they're about context, the same upgrade this issue makes to the hallucination question.

---