Measuring Anything, Before the LLMs

0:00 Douglas Hubbard

You've heard somebody say a phrase like statistically significant sample size, right?

0:04 Sam Rogers

Yeah.

0:04 Douglas Hubbard

There's no such thing. There's no such thing as a sample size that's a minimum threshold applicable in all situations where one short of that you know nothing, and then at that, all of a sudden you can make an inference. There is no such thing.

0:18 Voiceover

Signals and subtractions. What to watch, what to drop, every week.

0:28 Sam Rogers

Welcome to Signals and Subtractions i'm your host, Sam Rogers. That was Douglas Hubbard And this week's episode is gonna be a little different than our usual format. I'm out this week on an urgent family matter and had to cancel our weekly live stream. But rather than skip a week of episodes, I went into my own podcast archive, and what I pulled out predates LLMs and modern AI and gets at the foundations of how to reason about the data that they need.

Who is Douglas Hubbard? He's the inventor of applied information economics and author of How to Measure Anything: Finding the Value of Intangibles in Business, which is the book people hand you when you tell them something can't be measured. He has measured drought resistance in the Horn of Africa, the effect of pesticide regulation on endangered species, and the economic impact of restoring a desert in Inner Mongolia.

I interviewed him back in twenty sixteen for my podcast, Doable Change, years before ChatGPT. No copilots, no AI budget anybody had to defend to a board. Seems delightfully quaint from here, doesn't it? Now, this isn't me telling you how prescient it all was. Doug wasn't predicting anything. He was describing how measurement works and how it's always worked since the dawn of science, which is why nothing in it needed updating.

Kind of a strange thing to say about a ten-year-old recording in this area, right? So here's what I want you listening for. Right at the end, Doug says there are only three reasons why anybody ever believed something is immeasurable. He calls them illusions and describes all of them in about ninety seconds. I'm convinced that those same three things are exactly how nearly every AI business case that I see falls apart.

Now, I'm gonna make that case when he's done, but see if you can spot them here first. They'll stick better. The first thing you're gonna hear is Doug asking me how I would measure collaboration, and I give the standard activity-based answer. Ten years later, that's still the answer most people give, but it's still wrong. And now it's on a dashboard with big letters A and I up at the top. Enjoy this episode with author, speaker, and statistician Douglas Hubbard of Hubbard Decision Research.

3:10 Douglas Hubbard

You've heard people talk about collaboration before, improving collaboration, right? Of

3:14 Sam Rogers

course, yes.

3:15 Douglas Hubbard

Since I think we share that concept, you tell me what you think of when you see more team collaboration.

3:23 Sam Rogers

I see people communicating and solving problems faster as a group than they would individually. Okay. I see that they're not, stepping on each other's toes, and that they can divide the work more appropriately.

3:38 Douglas Hubbard

Okay. So let's think about each of those things in terms of observable consequences. So I think they all hinted at observations without necessarily getting into specific methods, right? But they all hinted at a kind of observation. Mm-hmm. So you said they communicate more. Are there observations we can make that would at least indirectly indicate or directly indicate frequency of communication?

4:01 Sam Rogers

Um, sure. So the number of messages, like using email or something like that.

4:07 Douglas Hubbard

Sure. Right

4:08 Sam Rogers

although we're not necessarily looking to just increase the volume, so that there's more- That's right … messages, but actually that there's more getting done as a result of the messages.

4:17 Douglas Hubbard

There you go. Right. Exactly. I'd say I think the other thing is sort of a secondary indicator of something more fundamental, and sometimes people get caught up on the first thing. They'll get into a quagmire of latching on to the very first thing they identify- Mm-hmm … as measurable, right? And then they forget that there's something even more fundamental, because there is something more fundamental.

Mm-hmm. Does the quality and speed of the output of the group improve? Do I really care about communication per se? That's sort of an indirect indicator of something else I care about. It's probably a good bet that better communication leads to those other things, I would only measure communication frequency if I thought it had something to do, with the quality of outputs. Mm-hmm. Now, can we correlate, even just frequency of communication?

I don't know what the answer to this might be, by the way. It would be an interesting research topic. Is there a relationship at all between frequency of communication within teams and the development cycle and how much money their developed products make?

5:20 Sam Rogers

Hmm.

5:20 Douglas Hubbard

I don't know. That's a good question.

5:22 Sam Rogers

I was wondering if you could tell us a little bit about how you became interested in measuring change.

5:28 Douglas Hubbard

Yeah, so the first job I had out of graduate school was with Coopers & Lybrand, and I was in their management consulting services. And I was the guy who tended to get involved more in quantitative analysis, I think because compared to everybody else on the team, I had a lot more math and statistics. I got to be involved in a lot of neat, kind of obscure methods. Other people had said, "You'll never use that in the real world," and I was using it, you know?

And every once in a while we'd work at a client where somebody said something was immeasurable, and I would generally take their word for it. I said, "Well, you would know. I'm just starting out in the business world. What do I know?" But sometimes they had said that after I knew we had just measured that very thing at another client. So I knew they weren't always right, and then I started to suspect that it was never right.

Because every time somebody said it, I could find a fundamental misunderstanding behind it. That it was indeed measurable. Sure. We define measurement, and we think this is the de facto use of the term in really all of the empirical sciences. Measurement is uncertainty reduction based on observations expressed as a quantity. Okay? When you parse that definition, the reason why it's different from definitions that you often hear, the, the hidden presumption in that, it's an exact number, which it never is in the empirical sciences.

You have a wide range of possible values. You make some observations, generally do some trivial math, and then your range is narrower than it was before. Not only are people often surprised at how much of an uncertainty reduction you can get from a given amount of data, but they're also, surprised at how valuable marginal uncertainty reductions can be. They have this idea that I need to reach some arbitrary threshold before the measurement is statistically valid.

We're constantly correcting people.

7:31 Sam Rogers

So for our listeners who could probably use a little help breaking that down into layman's terms, would it be fair to say, and correct me if I'm wrong, that any time you know more than you did before, you're using some kind of measurement to do that? That, that you're reducing what you don't know and increasing the range of what you do know?

7:54 Douglas Hubbard

Yes. Right. In fact, the only caveat we'd put at that as the general presumption I think is fair is that you're expressing that uncertainty reduction quantitatively.

8:03 Sam Rogers

Mm-hmm.

8:03 Douglas Hubbard

But yes, what you said is correct. The term statistically significant, number one, probably doesn't mean what you think it means. When you figure out what it does mean, you realize that it's not even the question you were asking in the first place.

8:15 Sam Rogers

Mm-hmm.

8:16 Douglas Hubbard

And, what you really care about is what's the value that's been improved because I'm making a better bet than I was before? That's what I really care about. And so that's a different understanding than I think- Right people might learn in first semester statistics. Unfortunately, that's probably all they remember, and they remember it wrong.

8:38 Sam Rogers

What would be a good personal example of something that, that people often say is immeasurable, and you might say, "Well, yeah, you can think of that as immeasurable, but actually here's how you'd measure it"?

8:51 Douglas Hubbard

What's the effect of my commute on my health?

8:54 Sam Rogers

Great. So if I've got my commute Uh-huh what would my next step be then in helping to evaluate the impact of commuting on health? Which I think is a great example.

9:04 Douglas Hubbard

Okay. If somebody said, "How could I possibly measure the effect that, these behaviors have on my, life or, well-being?" or something like this. Well, first off, assume it's been measured before, because it probably has.

9:18 Sam Rogers

Hmm.

9:18 Douglas Hubbard

And if we're resourceful, and If we consider the possibility that you're not the first person on the planet to ask that question-

9:27 Sam Rogers

Mm-hmm …

9:27 Douglas Hubbard

right? And that somebody actually probably wrote a whole paper on it, a bunch of papers.

9:32 Sam Rogers

Mm-hmm.

9:33 Douglas Hubbard

A bunch of research projects that got a bunch of money to do that, okay? And it can be informative So often when we do research, we start with the idea that, "You know what? Maybe I'm not the first person to ever ask this question. Let's see what research exists." And you know when you're working for a client or your employer or somebody, and you've done some research, and you can cite two or three, seminal articles about some obscure topic like that-

10:03 Sam Rogers

Mm-hmm

10:03 Douglas Hubbard

You know what you look like. You look smart.

10:06 Sam Rogers

Right.

10:06 Douglas Hubbard

You look like you did your homework, right? You generally have more data than you think, and you need less data than you think. So let's say you've measured the time you spent commuting. You're now interested in measuring the impact that commuting has on your health. Great. So why do you care? What would be the reason for someone to measure the impact on their health?

10:25 Sam Rogers

Uh, potentially looking at relocating and, There

10:28 Douglas Hubbard

you go …

10:29 Sam Rogers

and trying to figure out if I can be closer so that I'm reducing that time, potentially looking at other, positions or transfers to see if I can work out of some place that's closer.

10:40 Douglas Hubbard

Right. Yeah. Absolutely. Well, those are all good decisions, and as soon as you identify those decisions, you add another little facet of clarification to the measurement problem itself, okay? Are there decisions you're going to make differently?

10:54 Sam Rogers

Mm-hmm.

10:55 Douglas Hubbard

Now, once you start modeling those variables, you actually start figuring out, here's how much I really need to even know about this variable before it makes a difference.

11:02 Sam Rogers

Mm-hmm.

11:02 Douglas Hubbard

So, you have a current state of uncertainty about a variable, let's say like your, the health effects. Mm-hmm. And you're saying, how bad would the health effect have to be before you would think it would be worth your time to just change jobs, if that was the decision you're making? Right. Or relocate, right? Or spend more on an apartment that's closer to your employer, 'cause you probably have that option.

If you said, "Wait a second, let's do a back of the envelope calculation. How bad would the health effect have to be before I would make that kind of change?" And then ask yourself, am I pretty confident it's a lot less than that, or could it possibly be over that threshold?

11:45 Sam Rogers

Right.

11:45 Douglas Hubbard

Because if there's a significant chance that you're over that threshold, then it looks like something you should measure further 'cause it might have bearing on your decision. So if you've got this big wide range, and this range is straddling a threshold for a critical decision where knowing which side of that fence you're on, well then it's, there's a value to measuring.

12:05 Sam Rogers

So knowing what you're going to do differently dictates how then you would measure.

12:09 Douglas Hubbard

That's right. Now, in the book, I briefly mention two other reasons that measurements have value- Mm-hmm … but that's not the focus of the book. I think that really helps focus and clarify the whole problem as soon as you ask the why do you care.

12:22 Sam Rogers

Mm-hmm.

12:22 Douglas Hubbard

So if somebody says, "How do I measure the value of m- my happiness, with my, home?" I go, "Well, is there a reason you wanna measure that? Are you just in a benchmarking contest with your neighbor or something?" Right. Is there a decision in your life you're actually gonna make differently? Now- Right To be honest, I don't measure all that stuff. Mm-hmm. A lot of that is trivial stuff that doesn't immediately affect decisions.

But when my wife and I, my wife teaches math at a community college, when we make bigger life decisions, starting about, let's say a new car, Mm-hmm and bigger, like a house, we definitely start doing analytics at that point. So we do our homework on that stuff. Mm-hmm. Now, a lot of the rest of it is how to do the math with a few observations.

13:13 Sam Rogers

Mm-hmm.

13:13 Douglas Hubbard

But we would go back with our first useful measurement maxims, right? Which are, it's probably been measured before.

13:19 Sam Rogers

Mm-hmm.

13:20 Douglas Hubbard

You've got more data than you think, and you need less than you think. Okay? People routinely assume that if they have a lot of uncertainty about something, they're gonna need a lot of data to measure it. Do you think they… That's an assumption you've heard? Oh,

13:35 Sam Rogers

yeah.

13:35 Douglas Hubbard

Or- Very frequently. Yeah. Yes. Mathematically speaking, just the opposite is true The more uncertainty you have, the bigger uncertainty reduction you get from the first few observations. If you know almost nothing, almost anything will tell you something. That's the way to think of it, okay?

13:52 Sam Rogers

When you say it that way, it sounds very natural and logical and easy. But so often it's true, we tend to think that if we don't know anything, we suddenly need to know everything in order to just get started.

14:04 Douglas Hubbard

Right. Yeah, absolutely. Furthermore, I think this is a problem a little bit with the IT crowds a little bit more, I suppose, is that when they think of measurement, they're thinking of data that's already been captured in databases. Mm-hmm. Something they can do a query on.

14:18 Sam Rogers

Right.

14:19 Douglas Hubbard

Well, I don't know how the scientific revolution would have ever gotten started if we needed databases- … populated already to start, you know? Uh, scientific method is not just about having data. Some of scientific method is about getting data.

14:34 Sam Rogers

Mm-hmm.

14:34 Douglas Hubbard

When we measured the speed of light, we didn't actually measure all the photons. Right. Right? Yeah, excellent. It was a sample. Yeah. It was a sample of photons. Random sampling is probably one of the more powerful things I tell people when we talk about measurements, because the other thing they latch onto is they somehow need all the data. I say, "No, you need a sample of that data from which you can make an inference about the rest of the population.

" That's how most of science works. So for example, when I wanted to measure how much time nuclear engineers spent on document management related items at a nuclear power utility, this was many years ago, there's 500 nuclear engineers among all these different nuclear power plants in the utility, and how could I go about measuring how much time they spend in document management? They're all over the place, right?

Mm-hmm. So we came up with a method. This was long ago enough that they all had pagers, okay? Mm-hmm. And so we came up with a method where they would all be paged once during the course of a month. Each engineer would be paged once during a month, and which means we would get 500 samples of a page. And they're, they were instructed that when they got that page from that number, at their earliest convenience, they would sit down and fill out a form, okay?

We didn't do things online necessarily so much then, but they filled out a form. Mm-hmm. And the form just said, "At the time I got the page, here's the activity I was involved in. It was on this project, I was doing these things. I was on the phone, or I was in a meeting room." They just described it. We collected all of these. By the time we got done collecting all of those, we observed that about 20% of those, data points randomly selected throughout the day of different, engineers were activities that would have been automated by a document management system.

Now, if we happen to catch one engineer in a break room and he said, "I was sitting in the break room," that doesn't mean that guy spends his day in the break room, right? So on an individual level, it's kind of unintrusive, method. It doesn't tell you much about an individual. It can only tell you about the group aggregate. So it's really, not that intrusive- Mm-hmm … individually. But if you look at out of 500, you've… I think it was 97 or something individual cases where people were involved specifically in activities that would've been automated and those pagers went off at different times of day for different engineers, different times throughout the week-

17:06 Sam Rogers

Uh-huh

17:07 Douglas Hubbard

that is something we can make an inference from. That's called a spot sample, by the way. It's not uncommon for gathering information about other complex organisms, like in zoology, right? Right. Where, where things don't sit still to fill out forms all the time, right? You gotta, you have to have a systematic way to sample, instances of behavior- Mm-hmm uh, throughout a day, and that's one way to do it.

Well, there's a lot of neat little empirical collection methods like that. You know, even learning how to do a simple controlled experiment can be a very powerful tool. If somebody said, "Are the development cycles of teams using this method gonna be faster than the development type cycles of teams not using this method?" Somebody could say, "Well, how would I ever know what would've happened otherwise?

"

17:52 Sam Rogers

Yeah. Maybe this team is just better than that other team if I set-

17:55 Douglas Hubbard

Right …

17:55 Sam Rogers

them up, you know, one with the software, one without, or methodology or whatever.

17:59 Douglas Hubbard

That's right. Well, you know, if we didn't know how to work out that problem, again, the scientific revolution would not have happened. Right. Of course we know the answer to that. All the methods behind controlled experiments are specifically about handling that issue, right? So, when you do a clinical drug trial-

18:17 Sam Rogers

Uh-huh …

18:18 Douglas Hubbard

do they know for a fact that this person got better because they took the pill? No. What they know is the test group did this much better than the control group.

18:29 Sam Rogers

Mm-hmm.

18:30 Douglas Hubbard

And it was by a larger margin than you can explain by chance.

18:34 Sam Rogers

Mm-hmm. Okay? Or

18:34 Douglas Hubbard

by

18:34 Sam Rogers

placebo

18:35 Douglas Hubbard

or, you know, or- Yeah. Fake- So control group would be the placebo group, right? Uh-huh. Um, and the test group is the one taking the real drug. And what you know is that in the control group, ulcers tended to last this long or tended to, change this direction in terms of their severity. And in the test group, 48% of the ulcers went away in a week.

18:56 Sam Rogers

Right.

18:57 Douglas Hubbard

And the rest of the ulcers went down by this… You know, whatever the measure is, right? So there's some dramatic change, and that doesn't mean that the people, every individual who took a placebo, necessarily had a bad ulcer that got worse. Some of them might have gotten better on their own. Right? And sometimes when you explain that approach, people say, "Oh, well, pharmaceutical companies, they had hundreds of subjects in each of their groups.

" Well, actually not always. Sometimes when you're dealing with life-saving cancer drugs, they have very small experiments. So your math has to be better because you're not gonna get the kind of slam dunk, easy findings that you could easily see on a chart. You gotta make, inferences out of more subtle findings. Okay? Mm-hmm. Now the math isn't harder. It's all y- nothing… I'm not talking about anything you can't do on an Excel spreadsheet.

So, it's all straightforward. In fact, there, there's not too much I talk about that you can't do in an Excel spreadsheet I've already made for you that you can download for free on the book's website.

19:59 Sam Rogers

Great. So other than a change not being valuable enough to need a measurement, is there any other reason people tell you not to measure something?

20:08 Douglas Hubbard

I've run into people once in a while that, are a little indignant about the idea of measuring the value of a human life. In my first book, I make the moral argument that you have to measure these things.

20:20 Sam Rogers

And can you explain that just a little bit?

20:22 Douglas Hubbard

Yeah, because, As long as we have infinite resources or no big problems to solve, and then you don't have to worry about any of this stuff. But we do have limited resources, and we have multiple big problems to solve. Mm-hmm. So we actually have to make a trade-off between what's the value of better emergency services in Haiti because you've got better roads versus educating your children. When you start making those hard choices, you start realizing that, yeah, you know what?

I am putting a value on a human life just by virtue of even making those choices.

20:55 Sam Rogers

Mm-hmm.

20:56 Douglas Hubbard

As soon as somebody says, "This program which will save two lives a year on average is worth $5 million, but it's not worth 12 million because I rejected another project on that same basis." And when you look at utilities and governments and, you know, law enforcement agencies or hospitals, when you look at the decisions they actually made, they can tell you you should not put a value on a human life, but in fact, when you look at the decisions they've made, they already have been.

Anybody with access to their decisions and a little bit of algebra can figure out the implied value of a human life. Mm-hmm. The problem is because it's only implied and not explicit, it changes every time. It's willy-nilly different due to arbitrary factors that have nothing to do with that decision. There's been all sorts of research now about the impact that your mood has on decisions, and you become more or less risk-averse for a series of reasons that have nothing to do with the decision that you're trying to be risk-averse about.

All right? You know you're more risk-tolerant when you're exposed to smiling faces. Because

22:06 Sam Rogers

everything looks fine.

22:08 Douglas Hubbard

Yeah. You're more risk-averse if I ask you to recall some event in your life when you were afraid, and you're more risk-tolerant if I ask you to recall some event in your life when you were angry. And we can test these in controlled experiments, and those are the kinds of things that actually affect your decisions. The risk aversion for men is, correlated to testosterone levels, and your testosterone changes daily for reasons you're not consciously aware of.

So anything could change your testosterone level, certainly sleep changes it. Winning or losing unrelated games of chance changes your testosterone level. So if you just won or lost something, even a game of chance like an office pool, right? Just before a decision, that apparently could affect the risk aversion in your upcoming decision.

22:56 Sam Rogers

Huh. How about knowing our internal state well enough to be accurate about our ability to make a decision?

23:04 Douglas Hubbard

Most of the time people are pretty bad at that.

23:08 Sam Rogers

Really?

23:09 Douglas Hubbard

Um, they tend to be statistically overconfident. That means that they put too high a probability on being right compared to their track record. Given a variety of studies, when people say they're 90% confident, they tend to have closer to about a 60 or 65% chance of being right. Yeah. It turns out, though, the good news is that, , they can be trained in half a day to be about as good as a bookie at putting odds on things.

Bookies are pretty good. And, and you know who else is good? This surprises people, but meteorologists are good. Um, when a meteorologist says there's a 90% chance of sunshine, it rains 10% of the time, as you would expect. But the problem is when you evaluate the skill of somebody else, what do you remember, when they were right or when they were wrong? Right, the selection bias- You remember when they're wrong … of, of, yeah.

Yeah. You don't, you're not running an average in your brain. Mm-hmm. Mm-hmm. Right? You just remember a few anecdotes. And now when it comes to evaluating our own performance, we are more likely to remember when we were right. Yes. And so we tend to be systematically- Yes overconfident because we're, it's easier for us to recall when we're right. I have a hypothesis I always wanted to test, which is I think you're most likely to remember something when you turned out to be right and your colleagues were all wrong in disagreement with you.

24:28 Sam Rogers

Those

24:28 Douglas Hubbard

really stand out. I think if you were the one guy who had it right and everybody else was convinced you were wrong and it turned out you were right, I think you'll tell that story at parties for years.

24:39 Sam Rogers

Right.

24:40 Douglas Hubbard

It's those sorts of things that guide our own, judgment about our performance.

24:44 Sam Rogers

Is there anything else in closing that you'd like to share for our listeners?

24:49 Douglas Hubbard

Yeah. I talk about in the books that there's really only three reasons why anybody ever thought something was immeasurable, and they're all three illusions.

24:56 Sam Rogers

Hmm.

24:57 Douglas Hubbard

I call them concept, object, and method, or you can think of dot com as a mnemonic if you like. The concept is the definition of measurement. As we talked about, it's not an exact number. It's an, a reduction in uncertainty expressed quantitatively based on observations, okay?

25:12 Sam Rogers

Mm-hmm.

25:13 Douglas Hubbard

The object of measurement is just defining the thing that you're measuring. If somebody says, "I wanna measure collaboration," we ask them, "Why? And what do you mean by it? And what do you see when you see more of it?" Right? Mm-hmm. Don't just let the fluffy term exist, right? Mm-hmm. Uh, think about what it means in terms of observable consequences. And finally, methods of measurement. People misunderstand how random samples or controlled experiments or regression models work, and they have some profound misconceptions about sample size and, what probability means, et cetera, and that gets in the way of a lot of measurements.

Hmm. I think those are the big opportunities for people. I think that's where if we can overcome those obstacles, all of a sudden the world really opens up to you, and there's a lot more measurable things than you ever thought they were, and then the only question becomes what's the value of the measurement? Right. 'Cause you could measure anything. You don't have to measure everything, obviously.

It's, comes down to the information value. That's what most of the methods we talk about are really about, is computing information values and directing measurements based on what's statistically more likely to improve decisions.

26:19 Sam Rogers

That's great. Thank you so much, Douglas Hubbard. If people wanna contact you or gain access to your resources, where do they go?

26:26 Douglas Hubbard

Just go to howtomeasureanything.com. Thanks for your time.

26:29 Sam Rogers

Yeah. Thanks so much for being here. Really appreciate it.

26:32 Douglas Hubbard

You bet.

26:34 Sam Rogers

Three illusions: concept, object, and method. I said at the top I'd make the case for those as the three ways an AI business case comes apart. Here goes. Concept is demanding one exact number. In the actual empirical sciences, that number never exists. A measurement is a range that got narrower, and if you're holding out for a single figure, you'll be holding out for the wrong one. Object is measuring things like productivity or collaboration or enablement without ever saying what you'd see more of if you had more of it.

That's the one you heard happen to me in real time about twenty minutes ago. And method is assuming you need the whole population sitting in a warehouse or something before you're scientifically allowed to begin. You don't need that. You need a sample and a threshold. That's not a coincidence, and it's not a new problem. It's the same three failures that we already knew about now at AI scales and AI speeds with AI-sized budgets attached.

So here's the subtraction for this week: stop running pilots whose result won't change a decision either way. If there's no outcome that flips the call of any decision, you're not measuring anything. Doug kills it simply with that one question. You heard him ask it. Why do you care? Every fix you just heard still works unmodified. That's the whole reason this is in your feed this week. Now the part that genuinely has changed since twenty sixteen, What's new is how much money is riding on the answer.

Because ten years ago, if you couldn't measure whether a new system was worth it or not, you shrugged, you bought it, or you didn't. Now, there's a board asking what the AI spend returned and a date by which somebody really has to say something. That kind of pressure doesn't make anybody any better at measurement. It makes them faster at producing a number. And the fastest number is not generally the best one.

Uh, that's the one that Doug was just talking you out of. One last thing, the value of a human life. If you've got limited resources and more than one problem, you've already approved something and rejected something else. So that number is in there. And anyone with your decisions and a little bit of algebra can pull that out. All you bought by refusing to say it out loud is that it comes out different every time.

Um, I've heard this argument almost word for word this year about AI systems, usually from somebody explaining why they can't put a specific number on a harm. But they already have. They just haven't looked at it. Issue sixty-four of the newsletter coming up on Sunday, one signal, one subtraction, one analogy, about five minutes to read, which is a promise I keep every week. Get it at Substack, LinkedIn, or better yet, at sigsub.

show. I'm Sam Rogers. Thanks so much for listening.

30:04 Voiceover

Signals and subtractions. What to watch, what to drop, every week