{ "meta": { "generated": "2026-09-15", "series": "https://sigsub.show/#show", "rights": "All Rights Reserved", "copyright_holder": "Snap Synapse LLC", "rights_url": "https://sigsub.show/rights/", "rights_statement": "Copyright 2026 Snap Synapse LLC. All Rights Reserved. Public APIs are provided for read-only discovery and do not grant a license to copy or redistribute the underlying content.", "count": 11, "description": "Every timestamped speaker turn of every published episode transcript, for on-site and agent search. Turns are [seconds, speaker, text]." }, "episodes": [ { "episode": 1, "title": "Context Over Capability", "url": "https://sigsub.show/episodes/ep-001/", "transcript": "https://sigsub.show/episodes/ep-001/transcript/", "video": "https://youtu.be/mVIXndhHs1k", "anchors": [ "t-11", "t-25", "t-58", "t-85", "t-88", "t-108", "t-112", "t-139", "t-141", "t-188", "t-190", "t-239", "t-242", "t-342", "t-345", "t-434", "t-533", "t-548", "t-551", "t-663", "t-744", "t-756", "t-851", "t-859", "t-867", "t-916", "t-922", "t-943", "t-954", "t-977", "t-980", "t-1052", "t-1132", "t-1137", "t-1207", "t-1215", "t-1216", "t-1381", "t-1395", "t-1439", "t-1460", "t-1485", "t-1488", "t-1493", "t-1531", "t-1538", "t-1553", "t-1559" ], "turns": [ [ 11, "Sam Rogers", "Welcome to Signals and Subtractions Episode 1. I'm Sam Rogers, here with Lee Rodriguez from R&P Associates. Lee, would you tell the nice people of the internet about how and when we first met? " ], [ 25, "Lee Rodrigues", "Well, the time was 2012, San Bruno headquarters at YouTube. I got brought in after having a comfy job at Apple teaching everyone how to edit video, being asked, can you come do instructional design video editing? And on my way in, one of the people I was working with said, By the way, they've hired five consultants in four weeks and fired every single one of Good luck. And then I met Sam and we talked about ten minutes about scripting and stories and realized I think together we can do this. I think we have the right people here to get this done. And we did. " ], [ 58, "Sam Rogers", "Yes, and since then we've kept in touch. as we've both worked at the intersection of learning and performance, multimedia, IT, change management, both inside large organizations and startups. And on this show, we'll typically have a guest, but today being the very first one, it's just the two of us. I'm really glad that you're here. with me, Lee. " ], [ 85, "Lee Rodrigues", "I'm glad we can do this together. It seems like old days, " ], [ 88, "Sam Rogers", "Yep. So the format for this show is one signal each, one subtraction each, some stories, hopefully, to make it stick. And before we get to all of that stuff, I think it's time for the current state of AI. " ], [ 108, "Lee Rodrigues", "one of the first big things in the news is IPO's looming. " ], [ 112, "Sam Rogers", "Yeah, so you might have heard about SpaceX in the news. XAI and their Colossus data centers are a core part of that. they're the ones supplying all the compute for the frontier models. Anthropic is racing to IPO. OpenAI just said that they're backing off for a moment, probably because there's literally not enough money in the world to support. all three of these valuations all at the same time. " ], [ 139, "Lee Rodrigues", "It's a lot. let's talk about the models withheld. " ], [ 141, "Sam Rogers", "so that's the big news just within the last two weeks. Fable Five is actually released Again today, July 1st. it's been released, it's been revoked, it's been a rolling restore. meanwhile, also on the open AI side, ChatGPT and their 5.6 model, it's In preview mode. It's in the same kind of mythos mess. These are the two leading AI labs that are basically receiving instructions from the government that say, yeah, you give it to us first, and then a staged release to certain approved companies, approved people, the US government is now receiving the frontier models and approving their. Use cases. " ], [ 188, "Lee Rodrigues", "Let's see that headline again. " ], [ 190, "Sam Rogers", "regulations are being postponed because like the EU AI Act, which was supposed to be going into effect on August second of this year, parts of it are still going into effect, but most of it is not. It's been delayed sixteen months. So as somebody who is developing products to to help relieve some of the regulatory strain there. oops, 16-month error. there's a lot of executive order stuff happening and there's this whole voluntary system that came out. Basically, the federal regulations that everyone agrees we should have aren't coming. The state regulations, such as just happened in Colorado, are getting gutted. And the EU AI Act is trying to figure out how it works. " ], [ 239, "Lee Rodrigues", "Well what about the agent fleets that are eating software as a service? " ], [ 242, "Sam Rogers", "AI agents are everywhere, and they're coming for your software. just yesterday, part of Sonnet's release was optimized for fanning out agents in this advisor pattern, which is where they take a a less intelligent model and it pops out to a more intelligent model to maybe answer some questions, get some direction, and then the bulk of the work happens on lower model. For those of us who are deep in AI, we've been doing stuff like this for months. This is how I've been doing translation. for a long time, actually with local models popping out to more intelligent ones. But this is actually getting integrated as a pattern now into fleets of agents that are working inside Enterprise. And that is new. Also the open weights models, the ones that you get from China or something like that, the the AI that you actually run on your device without touching the internet, those tend to be about two-quarters behind the frontier stuff. So this thing that the government has to approve, that's the 10 trillion parameter models that are coming out now. from the Frontier Labs. They can gate it for now, but those level of models are coming by the end of this year free and open source. Unless something huge changes, it's likely to actually catch up even more. So for all those regulated industries, things that like can't rely on frontier models to absorb all that context about their business, being able to run that completely locally is possible with the right hardware. And it is eating software. " ], [ 342, "Lee Rodrigues", "One piece at a time. Well tell me about eleven days to frontier. " ], [ 345, "Sam Rogers", "between February and June There's been an average of a new state-of-the-art frontier model every 11 days. I'm talking about those big models. So the ChatGPT 5.6 that was just announced last Friday, their sole like huge one, Fable 5, which we were talking about before, which came out three weeks ago and then kind of uncame out, but is now coming back out again, that level of model. And two weeks before that, like at the end of May, there was Opus 4.8, These things are pushing the frontier, and something completely new coming out largely at this point, for the last quarter, just from those two labs. But previous to that we were seeing things from Google, seeing things from X as well, that were kind of pushing the frontier in new ways. The world changes. What AI can do changes. Every two weeks or less. It's averaging every 11 days. And this is the first day of Q3 I'm guessing that the only thing that we really are sure of with what's coming in Q3 is that it won't look like Q2. It's probably gonna be 10 days. It's probably gonna be faster. You can throttle down some of the some of the releases and you can stage them, but the pace of progress is happening. Perhaps we could go to the signals segment here. " ], [ 434, "Lee Rodrigues", "the signal for me has been finding a way to get really back to the basics. And what I mean by that, I started like developing in scripts and stuff for like a course I'm gonna produce and I would feed it into Claude and have Claude create a seven page document of how this works. And what I found is I was getting lost in the sauce with all the formatting and all these fields and everything. And you can't remember how much of this is me, how much did it come up with, how much relates to what I do. And the big thing that changed for me was when I used to train designers, start with a one-page content outline. One page outline that clearly defines what we're talking about. It's like, well, got all these ideas. I don't want to see a course. I don't want to see a video script. I don't want to see an exam. I want to see a one-page content outline to make sure this stuff flows and all connects with each other so subject matter experts can understand it and everything. When we get into the big complicated documents, it can be tough for everyone to navigate. And I think that over polish makes it extremely difficult. said, hey, this is a beautiful seven-page document that I'm already lost in. Give me a one-page content outline. And going back to the basics without the polish and looking at this one page that we used to use back in the day made a whole lot of sense. That structure right there let me see where Claude's making stuff up. It's finding things that were not a part of my content that it's bringing in from somewhere else, and I can just trim to subtract what isn't a part of it, and then I I get what I needed. So the idea is you have to get back to the basics. You have to get back to the basics because it will put it in a format that is overcomplicated. And then you end up sharing it with a client or something, and they say, Where did this come from? And then you all of a sudden you sound like a child in high school. " ], [ 533, "Sam Rogers", "Yeah. " ], [ 548, "Lee Rodrigues", "Reading a book report for a book they clearly have not read, which is not a great place to be. How about you? Which what's your signal? " ], [ 551, "Sam Rogers", "Yeah. well for me lately it's it's similar. So as all these models get more and more and more powerful, I'm realizing how much more it's about context. overcapability. And it it's not that different than for people, like like so many things. This works for people as well as AI. We tend to trust when someone gets us, right? Like when they reflect back the things that are relevant to what we care about, the things that we think are obvious. You know, it's not complicated. The intelligence coming from these huge models is astonishing. but it kind of doesn't matter how astonishing it is if the context goes wonky. Building trust up from the basics. And i it works so much better than like building it down from something very esoteric and difficult to understand, something really challenging. just what you're sharing there, Lee, about the The one page version, like the simplified version. Remove all flashy sparkles. What's the actual thing that we're doing here that you can't bluff? your book report example. Like, you don't need to memorize what's on, you know, page 82, but you need to be able to say, like, who the characters are and why they're there in the story, you know, things like that. again just back to people. Like you could take a genius level person and put behind fry machine at McDonald's and they'll be confused because These two, they don't have the context for that. Are they smart enough? Absolutely. Will they burn themselves? Probably. Like will they burn your business? Absolutely. So having that kind of awareness and ability to communicate context with AI, I think at this point, is becoming more important than the intelligence itself. We've gotten there. " ], [ 663, "Lee Rodrigues", "Likely. Hundred percent. Well, that brings up a really interesting story. when we started it, Google, I was auditing courses and I found that they had a course about how to write better emails because so much of communication was in emails. And their story was content without context is very difficult to absorb. And your job as the writer of the email is to provide the context, how this relates to your team. what you're working on, how that ties to this, because everybody has an email from someone else's team that's working on something that has nothing to do with us and you're asking for help. But if there's context, wait a minute, you're trying to solve the same problem I am from a different angle. And if I help you with this, you may just chip away hours of development work on my side. We're friends now. Let me help you with your project, right? In Ford, they got a bunch of AI agents to troubleshoot designs and to and to simplify and put designs out. And what happened is they laid off a whole bunch of engineers to get this code base and these AI tools to make all this better. And what just recently happened is they're now hiring back the gray beard engineers. There was a bunch of older engineers, and they said what they have is the context, none of the code does, because they're getting ready to launch this car, and one of the engineers comes over and says, Hey, I don't want to be a dick or anything, but " ], [ 744, "Sam Rogers", "Yeah. Right. " ], [ 756, "Lee Rodrigues", "You release this turbocharger in a car like five years ago and it overheated disastrously and melted the car and everything. It needs a much larger intercooler to work, and that intercooler needs a larger radiator to transfer all the fluid around. You all are trying to compact this down and it's gonna result in overheating. I know that the AI says that this is a really good turbocharger. All the specs are good, the costs are good. Yeah, but it runs hot. It always ran hot, and you're talking about a vehicle that's going to tow. What that's gonna mean is someone on the side of the road with a trailer with a with a radiator boiling all over the place, w under warranty, calling saying, Guess who doesn't want this truck anymore? And they learn that from the original Ford F-150s with double turbochargers, but no one remembered all the service calls. And the gray gray beard engineers are like, We run calls with dealerships trying to resolve this overheating issue when this thing came out. I remember those calls. AI wasn't on phone calls. that experience is now being called like the gray beard engineers, the gray beard army. Not that they're going to replace all the AI, but that content needs the context of an experienced engineer with a few great hairs to look over their shoulder and go, let me give you an article you're missing. That is probably in paper form from dealerships. But if you integrate this into the AI, the AI goes, Holy crap, we got a heating problem. No, you've always had a heating problem. You just didn't have that context of how it applies because you're thinking about performance. And by the way, the horsepower, the fuel consumption, all those things is good. But climbing over I-80 at 101 degree temperature, towing a trailer at 65 miles an hour is gonna overheat that vehicle. " ], [ 851, "Sam Rogers", "it's no surprise really for those of us who have some gray, as as you do, Lee. If if this grows in it gets a little gray for me too. But yes, exactly. so we've been through some tech disruptions over the last " ], [ 859, "Lee Rodrigues", "There's sparkles of wisdom. " ], [ 867, "Sam Rogers", "several decades, and every time it's pretty much the same thing, which is we overinvest in the technology and we underinvest in the people. And then as if we didn't make that mistake just five years ago. We go, oops, I guess we should bring some of them back. Like we overcompensated. especially the human context, the context that has not yet been made machine readable. we have to be able to point. AI to that context for it to even have any chance of understanding what it is that we're trying to do. so yeah, we're just doing the same thing that we always do, and it'll probably go similarly of cutting a little bit too deep and then oops, I guess we need to bring some actual people back who know what they're doing. " ], [ 916, "Lee Rodrigues", "And and just a quick plug, what inspired me was I was reading your article on LinkedIn vocabulary debt. And that really ties into this because you have the AI's using a different term because we called it three different things as we were developing this product. And when we identified that the platinum turbocharger is what we called it as we released it, before it was called something else, it had major heat issues. " ], [ 922, "Sam Rogers", "Yeah. " ], [ 943, "Lee Rodrigues", "hey, wait a minute. There's four other names for this. Call it these four things. You're gonna pull up the trouble tickets, the cases, the support issues. That context will fold in and you'll say, wait a minute, there's a couple more things attached to this. " ], [ 954, "Sam Rogers", "the platinum example is a great one because what I was saying in that signals and subtractions newsletter from last week is platinum can mean more than one thing in different parts of the company. So the marketing, they're calling it the platinum one. If you say marketing platinum, it means this. But if you say engineering platinum, they're thinking it's got platinum in it? Like y " ], [ 977, "Lee Rodrigues", "Got platinum in it? That that's an expensive metal. That's unique. Where are we going with this? " ], [ 980, "Sam Rogers", "And and yeah, exactly. That that's I where are we going here? Like that being able to differentiate those things and that use terminology that is actually AI friendly. When we talk about making things machine readable, that's a core part of it is how does that ontology breakdown of when to use which context. So it's not just like all the context all the time, it's the slice of context that you need so that within this team, when they say this, that's what that means. And over here on this team, even though it's the same word, it's going to have the same mappings in its understanding of how it is that it's you know predicting the next token, it's going to use the same platinum to mean those different things, but it's gotta like correlate the trajectory of that in its whole map of everything. And we can make that easy and we can make it hard. So the easy way tends to work better, but it's more work up front that we're used to humans being able to compensate for. And even though it makes sense to humans too, we've just never had to go to the trouble. You know, and now we kinda have to go to the trouble. How about a word from our sponsor? What do you say? I I put some flashy sparkles in there just because I figured we would be talking about all the flashy sparkles. you may be familiar with the in this sponsor. " ], [ 1052, "Lee Rodrigues", "See. Let me know who does your voiceover. I'd like to hire him. " ], [ 1132, "Sam Rogers", "For you, yeah, I'll I'll hook ya up. No no worries. So so signals done. now on to the subtractions. What did we each decide to stop, to kill, to refuse to engage with this week and why? Take it late. " ], [ 1137, "Lee Rodrigues", "I love it. My subtraction is exactly what we were just talking about. It's subtracting the fluff whenever possible, particularly for me, my design documents, my scripting, getting to a simple outline, removing all the fluff to make sure we're on the same page, and it's one page. To quote one of my favorites writers in the world, Tim Ferris, if you can't fit it on one page and understand it, you probably don't understand it. And getting it down to one page, what I learned in grad school a 250-word paper? That's a challenge, man. Every word has to be doing something, and it has to be super simple and to the point. If you drift it all in 250 words, it's all gonna fall apart. So getting your AI message, getting the things you're working on down to a one page content outline, subtract the formatting, subtract the bars, subtract the just in case I need the context. The content and the flow. Subtract the unnecessary. " ], [ 1207, "Sam Rogers", "Sounds great. And and while you're at it, you might as well d do that in what format of document? What format are you using? " ], [ 1215, "Lee Rodrigues", "T X T or Mark Down " ], [ 1216, "Sam Rogers", "Hey, hey, there we go. Now we're talking. yeah, do something that's token efficient, that's the lingua franca of all the LLMs that were trained on text. Just use text. Text is different than a Word document. So for my subtraction, it's to stop reaching for a better tool to fix the context, and subtract down to what actually makes it reliable. So example. I I have a whole lot of context management that I'm constantly doing anyway, and quite a few skills, agentic skills, I've been making or modifying an average of one a day since February. So like there's over a hundred of them. I've got them all in their own monorepo. I'm like coordinating across multiple LLMs to use them. but I'm realizing as these models get better and better, there's actually less use for all those custom skills. that I used to need. But with A B testing, the results, I'm realizing I sometimes get as good or better responses from a more better informed model with the context that I'm giving it as opposed to the skill and the context that's in that. So so I've actually been removing a number of skills that I developed several months ago that were very useful. But as soon as it becomes like almost about as useful, kill it. Because it's just not worth the maintenance. It's not worth the the choices, the complexity. So I'm down from around 100 skills to around like 40 now. I've cut over half of them because they're I I gotta admit, like they aren't always helping. So I'm archiving way more than I ever thought I would. and I'm realizing in the process that most AI assets are very short-lived. Like back to our news segment, when when you've got the new state of the art model every eleven days that's changing what is possible with AI, how many iterations, how many cycles of that are gonna make sense for your existing infrastructure and architecture? Like I'm expecting I will probably keep things for maybe five of those cycles? Well that's only like two months. So a skill I built in February. is like already outdated. So that's my subtraction. Just keep track of what it is that you're building for AI. And if you still need it, 'cause you might not. " ], [ 1381, "Lee Rodrigues", "Amen. That that's the same for every saved editing file you have for a video. It's like how many times am I gonna revise this? How many versions do I need? Maybe the current one and get rid of all the archive, you know? But what are you keeping it for? " ], [ 1395, "Sam Rogers", "Yeah, exactly. I believe we're coming to the end of our episode. For those who are enjoying this thus far, I just want to direct you to Sigsub.show, where you can sign up to see future episodes, be notified as those come out. There will be additional co-hosts that are rotating in, and we will be doing this as a live live stream. That didn't quite happen for for the very first episode, but it will happen for the next one. We will also have guests in addition to co-hosts. Looking forward to seeing you there and all of your comments and questions. Lee, do you have a something good to to wrap it all up? " ], [ 1439, "Lee Rodrigues", "Without an authentic story, we're magnifying and multiplying unnecessary content, which is a lot of what your AI can do. It can just totally extrapolate on nonsense. So getting that authentic story, that authentic business problem, the thing we're trying to do, wrap the context around it. Turns out that works pretty good. " ], [ 1460, "Sam Rogers", "it's been great sharing this with you, Lee. and thanks for playing along and being the very first co-host on the very first episode of this signals and subtractions thing which I've been doing for like a year now, over a year, every week. I'm really excited about this new format and being able to take it to an even bigger audience. " ], [ 1485, "Lee Rodrigues", "Happy to support you any way I can, Sam. Love working with you. " ], [ 1488, "Sam Rogers", "So where would people find you if they wanted to find Mr. Lee Rodriguez? " ], [ 1493, "Lee Rodrigues", "I I work with RP Associates. That's my wife and I's consulting company. But the new thing that's coming out that'll be coming out in the next couple of weeks here is system2focus.com. It is a smarter approach for a serious search. It's all about using all the tools available to you to find that authentic part of you, to interview better, to informational interview better, to make better resumes so you don't waste your time sending 30 applications a week that get no response. Nothing whatso you feel like you got some work done, but you accomplished absolutely nothing. If you're ready to break that cycle, systemtofocus.com will have a link in the description. " ], [ 1531, "Sam Rogers", "You know, we should do a a future episode on job search in general. maybe you could find a suitable guest or something for that. let's bring on a recruiter or somebody who's used to working with all the the modern AI tool sets for doing that. You and I have been hiring managers, we've been through thousands and thousands of resumes, but it'd be good to bring on some some other external expertise. The job " ], [ 1538, "Lee Rodrigues", "Absolutely. " ], [ 1553, "Sam Rogers", "thing is quite a thing these days. I think that'd be great to explore in a future episode. Cool. Well thanks again everybody for tuning in to Signals and Subtractions. " ], [ 1559, "Lee Rodrigues", "I love it. Let's do it. " ] ] }, { "episode": 2, "title": "Find Your Signal, Find Your Subtraction", "url": "https://sigsub.show/episodes/ep-002/", "transcript": "https://sigsub.show/episodes/ep-002/transcript/", "video": "https://youtu.be/oW5wq2WP_aU", "anchors": [ "t-0", "t-0-2", "t-60", "t-60-2", "t-60-3", "t-120", "t-120-2", "t-120-3", "t-180", "t-180-2", "t-240", "t-240-2", "t-240-3", "t-300", "t-300-2", "t-360", "t-360-2", "t-420", "t-420-2", "t-420-3", "t-420-4", "t-480", "t-480-2", "t-540", "t-600", "t-600-2", "t-600-3", "t-660", "t-660-2", "t-720", "t-720-2", "t-780", "t-780-2", "t-840", "t-840-2", "t-900", "t-900-2", "t-960", "t-1020", "t-1020-2", "t-1020-3", "t-1020-4", "t-1020-5", "t-1020-6" ], "turns": [ [ 0, "Sam Rogers", "Signals and subtractions. What to watch, what to drop every week " ], [ 0, "Sam Rogers", "Hey, welcome to Signals and Subtractions. This is episode two. I'm Sam Rogers. Uh, typically we'll have a co-host and a guest. Today it's just me. I wanted to set some ground rules here for how I think about what this show is and what it's not, and specifically, what do we mean when we say signal and subtraction around here? Um, by the end, you'll be able to find your very own signal and subtraction with or without the use of AI. " ], [ 60, "Sam Rogers", "But first, because AI changes all the time, news. For the longest time, AI has just been a black box. Like we know that stuff is happening. We see the outputs, but we don't really see what's happening underneath. " ], [ 60, "Sam Rogers", "We just know that we didn't design it or build it directly, and while we can't yet say that AI is conscious, we can peer into its subconscious now. Yesterday, Tuesday, July 7th, Anthropic put out a new paper and an open tool revealing the Jacobian Lens, as they're calling it, or JSpace. You'll find lots of YouTubers talking about this. " ], [ 60, "Sam Rogers", "Some of them even read the paper. But I would encourage you to go and play with it directly yourself, just as I did last night, so yes, what I'm referring to is a global workspace in language models. " ], [ 120, "Sam Rogers", "Anthropic put out this fancy little video, it's, it's actually very good, which compares what JSpace is to human consciousness. Um, there are links here to the paper, some of it's a bit beyond my depth, but definitely worth reading. Most fun of all, uh, you can go to their open tool, and you can actually see JSpace in action with two different, uh, large language models, uh, Qwen and Gemma. " ], [ 120, "Sam Rogers", "And this is also a, a GitHub repo, by the way. So this isn't the thinking part of the model. " ], [ 120, "Sam Rogers", "This isn't, uh, you know, in a reasoning model, sometimes you see the part at the top of it's thinking about something. This is before that. It's not the output tokens that you get. You're not being charged anything for JSpace. This is like a view into, um, what's happening underneath. So, um, what it reveals seems to be the precursors of all the responses that we see. " ], [ 180, "Sam Rogers", "Um, that's the assertion of the paper anyway. Lots to talk about here, um, but not in this particular episode because we're only, like, 24 hours in. I'll put some links in the show notes. Why don't you put something from your JSpace in the comments right now? Let's just see what happens. So the reason that all of this is important, uh, the reason it's news is not that AI got smarter. It's for the first time we get to see inside this black box. It's a little transparent. " ], [ 180, "Sam Rogers", "It's very fascinating stuff, and it's actually quite new. . Uh, Fable maxing. Fable 5 is currently the most powerful AI model that's commercially available. This is the one that was not commercially available for a couple weeks while the US government was deciding if we could have it or not. Um, pretty much everyone who's deep in AI has been very busy for the last week maxing out their access window before Anthropic's billing practices change. " ], [ 240, "Sam Rogers", "Originally, that was supposed to happen yesterday. Now it happens on Sunday, July 12th. Yay. So if you haven't already hit your token limit, and you probably did, um, good news, there's more for a few days. Um, this is something that has been panned on X and, you know, there's lots of chatter out there about, \"Oh, they nerfed it. " ], [ 240, "Sam Rogers", "It's not as good as it was before,\" or, \"It was always a hoax,\" or, \"Does anybody think this isn't any good?\" Um, my direct experience, this is by far the best model ever, and it's the first one that I've been able to use successfully for some massive tasks that I've had rolling around up here in the noggin for a while. " ], [ 240, "Sam Rogers", "Um, if you don't know what to do with Fable in this window, just ask me, 'cause I'll tell ya. All right. The next thing already flew in there, uh, FrontierCo by Microsoft. So Microsoft launched a $2.5 billion organization. This is 6,000 people, um, industry experts, AI engineers who work either on site or i-in some way deeply embedded with customers. " ], [ 300, "Sam Rogers", "And this is their strike team, their AI deployment unit that is focused on AI integration in enterprise workflows. I've been writing signals and subtractions for a few months now about how we're moving from pilots being like a 2025 thing into production 2026. So to me, this is just further evidence that, yeah, we're here now. " ], [ 300, "Sam Rogers", "It's infrastructure time. Let's go. Uh, so that's some of the signal from out there in the wild. There's more stuff that's happening, plenty more stuff. " ], [ 360, "Sam Rogers", "I've been writing \"Signals and Subtractions\" since June of last year. This show is an extension of that to make it more relevant, to offer different perspectives, make it more fun for me and for you hopefully as well. A lot of people can do news segments like I just did or tell people how they think the world should be, but I'm less interested in people talking about AI or writing about AI or giving all of their opinions about AI. " ], [ 360, "Sam Rogers", "I'm much more interested in people who are actually AI-ing about AI. And for those of us who've been doing the work, as I have been for years, um, you know you can't keep up, right? Like, you can't pay attention to everything. All of this stuff moves at inhuman speeds, and we are merely human. So the signals that we're tracking become that much more important. " ], [ 420, "Sam Rogers", "Also, there's a lot of things that used to make sense that just don't anymore, and we need to actively release and let those things go so that we can ascend up the exponential curve in terms of what we can produce and how we can coordinate, how we organize and work together. Basically, how we all navigate this change ourselves. " ], [ 420, "Sam Rogers", "If you'd like to see examples of the signals and subtractions that I've produced for the last year, all of the newsletter stuff is archived on the site. What site you may ask? Why sigsub.show, of course. " ], [ 420, "Sam Rogers", "Feel free to share on all the socials. That'd be awesome. You can sign up and be notified for everything. When I say signal, what I mean is like a personal shift in my own work. " ], [ 420, "Sam Rogers", "It's something that's recent, it's something that's specific, it's something that's first person. So it's not a headline, it's not a trend, it's not a tool, although sometimes, you know, it's cool to share tools. Tools are super cool. Um, the test for you here, if you wanted to contribute a signal, would be that only you could say it because you saw it or you did it firsthand. " ], [ 480, "Sam Rogers", "So AI agents getting smarter, um, that's, not a good signal. Just think, if a stranger could have said it on LinkedIn, it's probably not yours yet. Um, rather than telling you my signals off the bat, uh, I would like to show you how to come up with your own. Find your own signal and subtraction. There are some, AI copy-paste prompts. " ], [ 480, "Sam Rogers", "We're gonna go through it the long way, so the, the way that's by hand. So first, I glance at my calendar. Uh, this is really just to jog my memory about who I've talked to, and remember like, oh, right, that meeting I had on Tuesday. " ], [ 540, "Sam Rogers", "Like, there's... In the past week, I've had these different basically change management discussions. All this scope creep is landing on good people right now, and I, I have these conversations about, like, what to do about that. I may also have some notes in my Obsidian vault, you know, wherever you keep notes is great, about interactions and, and things like, um, like it's not the big project that wears you down, it's the recurring drag of all these obligations, the standard meetings, the weekly reports, the sync that you inherit and that just keeps running forever, um, because somebody had a need one time, and now there's a calendar commitment that sucks time from me and my team. " ], [ 600, "Sam Rogers", "So if you're like me, probably nobody in the org chart sat you down and bestowed upon you this sacred duty. Uh, it just drifted onto your plate, and you kept doing it, and nobody ever noticed that it never really got assigned, right? It's just an expectation now. So all I wrote down in my notes was mandate drift. " ], [ 600, "Sam Rogers", "But it reminds me about all of these meetings, ceremonies that, that only add and never subtract. Um, so, you know, I have that to work with. And notice that it passes the test because it's first person, right? Me. It happened this week. And only I could say that I had these particular conversations. So that's a good signal. " ], [ 600, "Sam Rogers", "And the archive is full of all these, by the way. So signal done, onto the subtraction. A subtraction is something that you actually stopped, something you actually killed or refused notice it's past tense. It's made, not planned. And it doesn't start as advice for other people. " ], [ 660, "Sam Rogers", "This may not be the most familiar way to think upfront, but that's kind of the point. If we're sloppy, our focus goes add, add, add, add until we just can't add anymore, right? We have to subtract in order to make room to grow, in order to lighten the load, in order to be able to ascend up that exponential curve. " ], [ 660, "Sam Rogers", "There's only so many things we can take with us. So, um, do fewer meetings, not so good. Um, I'm thinking of fewer meetings. Eh, not a subtraction. Um, I killed our Monday status meeting and nothing broke. That's a good one because it cost you something and you did it anyway. So I think about the stuff that I stopped doing in the last week on the calendar. " ], [ 720, "Sam Rogers", "I think of the recurring meetings that I did not schedule for Q3, and in this case, um, I did have to add one back. So far it seems I got away with dropping three and nobody said anything yet. Um, one of those actually I, I delegated two steps up the org chart where I don't think it will survive for long, uh, at least not as an hour meeting. " ], [ 720, "Sam Rogers", "It might be a half an hour. But either way, it's not on me, so I win. I also think about the mandate drift conversations and the solicited advice that I presented on various forums. Um, in this case, uh, there was a particular one in a Slack group that I could go back and remind myself of. So cool. Here now I've got the foundations of both a signal and a subtraction. " ], [ 780, "Sam Rogers", "Not bad, but not great yet either. So the final piece is to put on the analogy layer. The idea here is to make something familiar to the people, easily accessible. So Who here knows about coding? Anyone know about loops? They're all the rage these days. I'm looking for some way to make it tangible to the audience, right? To take something that they already know and transpose the thing that I'm talking about on top of that so that it feels good to them and they feel smart when they've done that. " ], [ 780, "Sam Rogers", "So I could say, um, something about do while loops. A do while loop rechecks its exit condition on every pass. It runs until the goal is met, and then it stops, right? It's a loop, but It's a loop that writes its own ending. So by contrast, for loops run on a fixed count. So I could type fifty-two and it would run fifty-two times because the number fifty-two was set once at the start, and it was never questioned again. " ], [ 840, "Sam Rogers", "Nobody asks at iteration twenty-eight whether this deserves to loop yet again. It's just pure count. So our recurring meetings are like for loops, and the really cruel part is, of course, there are for others. Uh, when we run a do loop, doing something for ourselves, we hold the exit condition. . So the subtraction here is not cancel all your boring meetings. That's kind of, you know, generic advice, hard to apply. Um, it's to eliminate for loop meetings. " ], [ 840, "Sam Rogers", "It's to convert for loops into do loops, and I recheck this at the end of every quarter and adjust as necessary. Now, I've never thought about doing it this way. I'm just trying to think of the analogy of how it is to express it to other people. So now I can say a do loop knows when it's done. A for loop just knows how to count. " ], [ 900, "Sam Rogers", "And by the end of the quarter, most of my calendar is usually just counting. I'm not lazy. Uh, it's just, it's way too easy to optimize for the wrong loop type, right? So I stop, I remove or convert, and I do this as a regular practice, which is true. Uh, so finally, the last piece is we wanna make this testable as a specific action that the audience can do. " ], [ 900, "Sam Rogers", "So this isn't advice exactly. I might say something like, \"Try this before your next week fills up. Open your calendar and look at your recurring events. For each one, ask two questions. One, what is its exit condition? Two, who checks it? If there is no exit condition or the person who checks it isn't there, you're running a for loop, and it's for somebody else. " ], [ 960, "Sam Rogers", "Give that a return statement this week. No reason that you need to announce this or talk about it to other people. Just do it and try it.\" So that's a good subtraction. It's not advice or telling people what they should do. It's giving them something small and simple that they can try, and that's what we want in every episode of the future Signals and Subtractions. In that example, the signals and subtractions are two sides of the same coin like this. That's when it really works really well. So the signal's what you see, and then therefore you stopped, you know, the other side of the coin. But that's extra bonus points. Uh, don't try and force it. A true signal and a true subtraction will beat a clever pair of anything that's, you know, half invented. " ], [ 1020, "Sam Rogers", "I chose this example on purpose so that you could see it in action there we go. Do loop, and next we have The subtraction for loop. Good. See how that works? Okay, now you. If you'd like to take a copy of the home game, please visit sigsub.show/find-yours. " ], [ 1020, "Sam Rogers", "There's some clever AI prompts so that you and your bot friend can play whenever you'd like. And now the outro. Thank you so much. " ], [ 1020, "Sam Rogers", "Signals and " ], [ 1020, "Sam Rogers", "As part of my own Fablemaxing last Monday, I made an AI harness. Now granted, I've been thinking about these things for over a year, and I already had a lot of the raw materials lying around. The fact that I could do it in one day for version one was pretty fricking awesome. " ], [ 1020, "Sam Rogers", "Thank you, Fable. " ], [ 1020, "Sam Rogers", "Meet Harnessie, the ultimate brain-agnostic multi-agent harness. Connect any AI agent seamlessly and orchestrate them with absolute ease. Unlock the true power of your AI workforce " ] ] }, { "episode": 3, "title": "Job Search in the Age of AI", "url": "https://sigsub.show/episodes/ep-003/", "transcript": "https://sigsub.show/episodes/ep-003/transcript/", "video": "https://youtu.be/lymgtimrPC0", "anchors": [ "t-2", "t-10", "t-34", "t-36", "t-39", "t-40", "t-43", "t-44", "t-52", "t-76", "t-115", "t-158", "t-248", "t-249", "t-249-2", "t-251", "t-267", "t-268", "t-348", "t-349", "t-406", "t-407", "t-497", "t-497-2", "t-542", "t-553", "t-592", "t-592-2", "t-634", "t-634-2", "t-710", "t-711", "t-711-2", "t-712", "t-713", "t-723", "t-724", "t-777", "t-780", "t-835", "t-836", "t-842", "t-842-2", "t-847", "t-848", "t-851", "t-851-2", "t-876", "t-876-2", "t-910", "t-913", "t-964", "t-965", "t-967", "t-967-2", "t-1062", "t-1153", "t-1170", "t-1172", "t-1191", "t-1191-2", "t-1196", "t-1196-2", "t-1198", "t-1198-2", "t-1257", "t-1262", "t-1264", "t-1264-2", "t-1266", "t-1318", "t-1320", "t-1323", "t-1324", "t-1400", "t-1406", "t-1410", "t-1412", "t-1422", "t-1467", "t-1488", "t-1567", "t-1568", "t-1573", "t-1631", "t-1651", "t-1652", "t-1652-2", "t-1653", "t-1654" ], "turns": [ [ 2, "Sam Rogers", "Signals and subtractions. What to watch. What to drop. Every week. " ], [ 10, "Sam Rogers", "Welcome to Signals and Subtractions. I'm Sam Rogers, your host. This is not a Zoom background. We were laughing about it earlier. I'm actually in Cancun, Mexico today. So hopefully everything goes well with the internet here. I actually have faster speeds than I do ever, like anywhere in the US for some awesome reason. And for... " ], [ 34, "Christine Rodrigues", "Let's be the equator. " ], [ 36, "Sam Rogers", "Yes, it's the centrifugal force of the Earth, " ], [ 39, "Christine Rodrigues", "That's right. " ], [ 40, "Sam Rogers", "think. Getting closer to that, it spins the Wi-Fi out better. If only that were true. " ], [ 43, "Christine Rodrigues", "Hehehehehe " ], [ 44, "Sam Rogers", "We are happy to be here. We have co-host Lee Rodriguez here and a very special guest. Why don't you introduce her, Lee? " ], [ 52, "Lee Rodrigues", "Lee Rodriguez. I'm just a co-host. She is an amazing creative artist who is now taking all of her skills in marketing and helping people to take their job search and use a marketing approach to treat your job search like you would if you're trying to sell a product Maybe you're the product and maybe you aren't thinking about it that way. How would you introduce yourself, Christine? " ], [ 76, "Christine Rodrigues", "So thank you, Lee. First, we should note that we are married, partners in life and in business. Same last name. So how would I introduce myself? I started off my career in communications and after getting my MBA, switched over to the marketing and strategy side. So my two main businesses right now are, I work with nonprofit organizations to provide strategy and marketing support to them. And also working on this project for job seekers with Lee, I'm bringing kind of the behavior behavioral and habit formation side of it. Lee's bringing instructional design, how to really organize this information so that people can understand it. " ], [ 115, "Sam Rogers", "That's great. It's great to have you here, Christine. Thank you, Lee, for bringing your lovely wife along. And it's a great opportunity to talk about things that this podcast is all about, which is how the world has changed with the onset of AI. and how to work with that. But exactly what you're describing, Christine, is so important how it is that we are, as humans, in the middle of all this change, and in the middle of changing process and the emotions that AI doesn't have that are really real, that we do need to wrestle with, especially when undergoing something as occasionally demoralizing as a job " ], [ 158, "Christine Rodrigues", "You know, it's interesting because my own signal runs parallel to signals that job seekers are typically experiencing when they're in their market. So, and I'm actually going to modify your formula slightly because I'm going to go signal noise subtraction signal. the approach that I'm looking at AI from is with the mindset of a writer. Right? So a lot of people use AI for a lot of things. I typically use it for writing and planning. And it's been fascinating. I've really latched onto it as a tool and it showed me, you know, kind of the bright side of it and the dark side of it. And I can literally feel it starting to change how I think. And so that that thought process is something that I've really paid attention to recently. So when I'm using it for writing. The signal when you look at writing before AI was if you see good writing, meant a good writer did it. Right? So you could judge a written piece, you could judge the author of that piece. With AI, you start having the large language models spit out really polished prose, strings of letters and words in the ways that people would use them. And that can be great. but sometimes the substance of the writing isn't quite right. We've all heard the stories, know, of AI slop and one of the things that Lee and I have discovered is, man, you can delegate and say, hey, make me a plan for this and it will create a five page formatted. " ], [ 248, "Lee Rodrigues", "Thank " ], [ 249, "Christine Rodrigues", "you know, lovely dividers " ], [ 249, "Lee Rodrigues", "you. " ], [ 251, "Christine Rodrigues", "and it's in your brand colors. And I look at that and I go, oh, wow, that looks great. But if you don't scratch beneath the surface, you know, we'll go back to that document later and start to have a conversation about it. And I go, what does this bullet point mean? I don't even know what this bullet point means, right? So it " ], [ 267, "Lee Rodrigues", "Thank " ], [ 268, "Christine Rodrigues", "looks good, but the substance might not be there. so that's something that I've experienced and I think job seekers, the parallel to that is a good resume used to be the signal of a good candidate, right? And you as the hiring manager could look at your pile and say like, that's a good resume, that's a good resume. We'll interview all of those people. Now, what people do is they upload their resume, they upload their job description, and they say, make me a custom resume to match that job description exactly, which on the surface is a my God, you could make custom resume after custom resume But the problem is that one, it's not really your authorship, right? And. everybody else who is using the same tool and the same job description are coming up with very similar word combinations. So while that resume that you created, it looks beautiful and it looks polished. It's the exact same one as so many other people. And sometimes it doesn't quite make sense. there's a term called sovereignty, like thought sovereignty when it comes to AI. And I've found a couple of techniques to really help me. maintain my sovereignty because I think it's an amazing tool, but I cannot delegate to it. It is not a person that I can say, you got this, go handle it. So the noise that's happening " ], [ 348, "Lee Rodrigues", "So, thank " ], [ 349, "Christine Rodrigues", "now is tons of polished writing out there. Maybe it's good, maybe it's not. Tons of polished resumes out there. So those signals, what used to be signals are now noise. And so you're having to find ways of fighting through that, right? So me, As an individual writer, I've started chunking the information that I'm working with, right? So I never delegate a whole document and say, okay, I'm writing a document. Let's work on the intro. Let's work on this sentence. Let's work on this paragraph. Let's work on this page. And I keep a master document off to the side. And so I'm the one in control of what chunks are going into that document. And it never runs away from me. When it comes out of Claude, there's these phrases that are in there all the time. Let's be honest about that. Stop and think about that. Like if I see you be honest one more time and it's really corporate speak and very stiff and vocabulary words that who uses that word? So what I've been doing " ], [ 406, "Lee Rodrigues", "Thank " ], [ 407, "Christine Rodrigues", "is rather than when I use Claude like they're all my thoughts and my background and my context and then it spits out a paragraph, but it's not the way I would say it. And I found that like my fingers won't let me type something that I wouldn't say. And so as I'm transcribing that material, I'm going like, no, no, no. No, no, no. Nobody would say it like that. That's a bad word. Put that into one compound sentence instead of two. So my voice is retained by that. And I think that goes for job seekers too when they're trying to create these custom resumes. If you just push a button and send, it's not really your document. And so one, it may look the same as, you know, three quarters of the other resumes received at that job, which could be in the hundreds. Right? And when you receive 500 resumes as a hiring manager, you're looking for reasons to eliminate most of them because you can't review that many resumes. So if yours looks the same, bye, gone. And also, if the act of creating the resume, the act of writing is something that needs to be retained, right? So if you're having AI create these bullet points in your resume, every bullet point is a story that you're potentially going to tell in an interview. And if you weren't involved in really creating that bullet point, you haven't been rehearsing for the interview. And so you may have a bunch of polished bullet points that looks great, formatted beautifully, and then you get into an interview " ], [ 497, "Lee Rodrigues", "Thank you. " ], [ 497, "Christine Rodrigues", "room and you're asked to defend something that you didn't write and you can't even remember what was there. The signal used to be quality writing meant quality person. And now with all the polish that's out there, I heard somebody describe it and I thought this was brilliant. The polish that AI writing gives to you is an anesthetic. I think that's brilliant because I've seen it. What do you think about this document? I'm like, oh, it looks great. Ah, you're nodding along. You're like, yeah, that's great. No, it's not great. It just looks great. So we've got to get around that noise by finding the ways to preserve the human, preserve the human in this process, the original thought, the voice, the taste, the style, the judgment. " ], [ 542, "Sam Rogers", "That's fantastic. Thank you, Christine, for the signal and subtraction and noise rolled into one. Lee, you can go ahead and feel free to do your signal and subtraction back to back too. " ], [ 553, "Lee Rodrigues", "So my signal has been, you may have a portfolio site. And that portfolio site could have been forgotten through all of your changes to your resume and your LinkedIn. So maybe a... take that portfolio site, take that LinkedIn profile, take that resume and tell your Claude, hey, do some black hat thinking for me. Are these three people, this resume, this LinkedIn profile and this portfolio site, are they the same person? Because I did mine three days ago, they were not the same person. They had gone in different directions and Claude was like, are you sure this is you? " ], [ 592, "Sam Rogers", "Yeah. " ], [ 592, "Lee Rodrigues", "And I'm like, bruh. Apparently I stopped doing stuff in 2015 according to my portfolio site because I just you know, it gets forgotten. If you're have a website, keep it up to date and keep it simple and keep a narrative. And if you're not going to keep up with it, my subtraction is get rid of that portfolio site. or do what I did over the past couple of days and make it a very simple, clean, beautiful one page scrolling site and get rid of everything that is unnecessary. And I'll tell you what it feels like spring cleaning when you're looking at a messy page going, no, I don't think I'm going to clean this page. I think I'm going to unpublish this page and get rid " ], [ 634, "Christine Rodrigues", "You " ], [ 634, "Lee Rodrigues", "of it because I don't know what the hell I had a point when I was like, I'm going to go all nonprofit. I'm going to change the world. Then I realized. Yeah, I'm really well suited for specific use cases in the corporate world So what I do is a lot of workflow based stuff for corporations and then a bunch of small businesses. do media production, YouTube videos, teachable, how you teach over video, that sort of stuff. It's always been one I'm good at. And my website didn't reflect that even just a little bit. And it's like, well, So all your current clients are businesses who are trying to do social media to get some videos out there to tell stories about how they work. And your website says nothing even close to that. So that signal is to make sure those things that we have out there, be it a portfolio site, a LinkedIn, a resume, make sure they're aligned. then all your assets become very simple because you do this ridiculous activity of making sure your resume makes sense and it's all lined up and it works. Align the LinkedIn to that, align the portfolio to that. But if you're trying to wiggle all these at the same time, that's how you paint yourself into a corner and say, I hate them all. I want to start over. Hey, Claude, can you get me a resume? looks like everyone else's. And Claude certainly will, 100%. " ], [ 710, "Christine Rodrigues", "instantly. " ], [ 711, "Sam Rogers", "Yes. " ], [ 711, "Lee Rodrigues", "Instantly. " ], [ 712, "Sam Rogers", "Hahaha " ], [ 713, "Christine Rodrigues", "It sounds like you're talking about, I mean, signal, but you had multiple signals and the signals weren't aligned, which means they were really noise. " ], [ 723, "Lee Rodrigues", "they were noise " ], [ 724, "Sam Rogers", "I love your framing of it too, when you have multiple signals that aren't quite lined up, that's called noise. So just as AI has changed what it's like to be a job seeker, it's also changed how jobs are listed. It's changed what the function of a recruiter is, what those interviews and that hiring process sifting through the pile of resumes as you referenced Christine, all of that part is different. And it impacts everything. So just like having a great resume used to be a signal of maybe a great employee. Because the floor has been raised on what the quality of good is, now the noise floor has been raised. And as a hiring manager, going through a digital stack of hundreds of resumes, as is not uncommon to get 1,000 applicants in a 48-hour period for a role, " ], [ 777, "Christine Rodrigues", "Yep. Yep. " ], [ 780, "Sam Rogers", "being able to sift through that, First off, let me just say, don't use AI for that. Don't just give the stack of a thousand resumes to chat GPT and say, find the five people I'm going to interview. Illegal in most places. Not everybody knows that, but you should. Don't do that. But even as a human, having been on the other end of this, as you have too, Lee, you've got a stack of a bunch of things and you're looking for anything that makes me think too much is the first pass. And then from what's left, here's the things that look way too similar. Next pass. Okay, what's left? Now I've got a manageable number of maybe 80 to 100 that I can actually look at and actually go through. So the signal that I wanted to talk about, that's all the preamble, is what AI is really great at, which is " ], [ 835, "Christine Rodrigues", "Mm. " ], [ 836, "Sam Rogers", "in the distribution curve, like a normal distribution of tasks. " ], [ 842, "Christine Rodrigues", "Mm-hmm. " ], [ 842, "Sam Rogers", "is really good at the stuff in the middle of that normal distribution. It's less " ], [ 847, "Christine Rodrigues", "Hmm. " ], [ 848, "Sam Rogers", "good at the stuff that's on the edges. And this has been shown in " ], [ 851, "Christine Rodrigues", "Mm. " ], [ 851, "Sam Rogers", "study after study. The more famous one is a Mount Sinai one that ChatGPT did on their medical offering using AI to answer questions. And what they found was AI is really good at answering questions that are the most common ones, but it's really crap. at answering anything that's on either end of that normal question range and inferring " ], [ 876, "Christine Rodrigues", "Mm. " ], [ 876, "Sam Rogers", "things there, was much, much worse than humans. Even though you do get the performance and cost that insurance companies are interested in to be able to get those basic things, right? It's the same with resumes. So if you want a resume that looks really strong in that middle of the distribution curve, basically that looks like an average of all of the resumes that are out there, if you want to look like that, then you should absolutely use AI to make your resume look as average as possible, because that is by default what will happen. But when you want to do things on either end, to stand out. " ], [ 910, "Christine Rodrigues", "Like like be one of the top five that they select " ], [ 913, "Sam Rogers", "yeah, that top 5 % is on the leading head of the distribution curve, right? So if you want to be there, making it something that is less reliant on AI, specifically for that end-to-end process, like you're suggesting, Christine, is a great move. It doesn't mean that you shouldn't use AI or you can't use AI in parts of the process, but the targeting, as you were describing, I think is spot on. It can really help with stuff, but ultimately, you're the one that owns the process. take ownership over that process. And most importantly, I love that idea of like, you know, if this works, you might have to talk about bullet three in the second job that you have there. It'd be good if you knew what that was and what it meant. So you're not just " ], [ 964, "Christine Rodrigues", "Yeah. " ], [ 965, "Sam Rogers", "vamping in the interview. Yeah. " ], [ 967, "Christine Rodrigues", "You " ], [ 967, "Lee Rodrigues", "we've covered this before what you become is that kid sitting in class who has to give a book report on a book you have not read and that is very clear in an interview the whole purpose of a resume since the beginning of resumes has been to leave some signal or marker that sets you aside that makes you memorable All the people that I hired for my roles had one thing. had one candidate I was looking for, and this guy was an editor for a small town newspaper for four years. And it turned out he was an exceptionally good writer. And a lot of the job was writing, and his writing was top notch. And our communications, outreach, and everything leveled up when this dude came on. And I could see that solving a problem I had for my team. Another person was a creative costume designer for years on cruise ships. And it comes out improvising on video shoots, having someone who's used to getting the show to go on for the club med performance on the boat that is going to happen in 30 minutes ready or not. That attitude that I can make this work was perfect for production, right? That set them aside. But if you use AI to draft the resume, what you've done has made yourself completely not memorable. You have just buried yourself at the 80%. And what we're all saying here is the corner cases are really sh**ty or memorable. And you're kind of going for the memorable. And if you stick to your boring resume format and have Claude polish a couple things, you may be more memorable because you're someone who actually wrote the damn resume. " ], [ 1062, "Christine Rodrigues", "Well, what you guys are talking about is having to put the human back into the process. And this is something that you hear in the, recruiting field right now is that, the resume is broken, right? Because again, it used to be this great signal. The other thing is just the easy apply button, right? And so, what method are people using to find the job openings? If they're going to job boards, they're finding the same jobs as thousands of other people and you're praying that you're one of the top five. And so one of the things that we tell people is you need to be more human in the part of the process where you're finding jobs to even apply for. And this is what really throws people off because they're like, I don't want to pick up the phone. That's going to seem like I want something from them. my gosh, you need to be a human being having conversations with other human beings so for a brief period of time, you know, having these technology tools made it better for everybody in the market. But then they got so much better that they then broke it. And so it's bogged down. These methods just don't work anymore. So companies like internal referrals. The recruiters are doing outreach to like contact human beings. And so it's not just the resume part of it. You also have to really think about how you're identifying the opportunities and developing human relationships outside of that cycle because otherwise you're with the herd and maybe you'll get a job that way, but the chances are very, very, very slim. " ], [ 1153, "Lee Rodrigues", "Well then that goes into the, guess what? Instead of hitting on two girls with the same terrible pickup lines, I hit 10 girls tonight with terrible pickup lines and the results was pretty much the same. " ], [ 1170, "Christine Rodrigues", "The same. Yeah. Yeah. " ], [ 1172, "Sam Rogers", "Yeah, That's a good analogy to draw there. Like, how does it feel being on the other end of that? Jobs are really human things, right? So remembering that that is a human network, it's often a human phone call, like this doesn't need to scale, actually. It doesn't need " ], [ 1191, "Christine Rodrigues", "Yeah. " ], [ 1191, "Sam Rogers", "to reach bigger numbers. It needs to reach the right numbers " ], [ 1196, "Christine Rodrigues", "that's right. " ], [ 1196, "Lee Rodrigues", "How about some current events? " ], [ 1198, "Christine Rodrigues", "Good point. " ], [ 1198, "Lee Rodrigues", "I just heard recently that a guy, forget what state he was in, he tried to kill his wife. They subpoenaed his chat GPT logs and they found a chat with chat GPT, how to kill your wife and get away with it. And it's like, yes, premeditated. It's premeditated. And he's like, he told his attorney, there's no way. attorney's like, there is no client attorney privilege with the tool on your phone. It is a communication device. We're gonna say, if you literally went to chat GPT and said, how do I kill my wife and get away with it? It's like, that entire thing is going to appear in a courtroom. And it's like, do not think that one of these corporations is going to spare you to protect themselves over something you shouldn't have been saying in your phone to begin with. Let's just say a little further here, maybe morally you shouldn't try to kill your wife. You know, that's kind of where we're at. Anyway, that was a distraction, not a subtraction. " ], [ 1257, "Sam Rogers", "Yeah, being that your wife is on the call, I think that is the appropriate position to take, Lee. Thank you for " ], [ 1262, "Christine Rodrigues", "I endorse this statement. " ], [ 1264, "Sam Rogers", "that. " ], [ 1264, "Lee Rodrigues", "Hahaha! " ], [ 1266, "Sam Rogers", "Well, not only is there no attorney-client privilege, but like when it comes to medical records and the privileges that we often give AI with regards to our medical history or psychological history, people using it as you would use a psychologist There are risks there. And as long as you know what the risks are, and you can manage those appropriately with your communications, no different than you would at a backyard barbecue or a cocktail party or a restaurant, right? Like we have certain conversations that are at this distance for someone across the table. And then there's the things that we, it's okay if people overhear, right? Like we know how to manage that as people in personal situations. We've never had to have that skill quite as strong. for how it is that we work with AI. But it really does make a difference. And it's not that different. Yes, simple hygiene. And " ], [ 1318, "Christine Rodrigues", "Yeah, yeah. " ], [ 1320, "Sam Rogers", "I love the sovereignty of thought. " ], [ 1323, "Christine Rodrigues", "sovereignty. " ], [ 1324, "Sam Rogers", "That's a good word for it. From the very beginning, my first exposure to AI was with running local models on my local laptop before they put the chat in ChatGPT, being able to run local models locally. That, just for anyone listening, opens up new possibilities that don't have to do with what you're sharing with the clouds, what you're sharing with OpenAI or Google or Microsoft or Anthropic or whatever. When you can run AI locally, it's not connected to the internet, it's not spying on you or doing anything. So, especially for businesses and regulated industries who may be listening to this. know that just because it's AI, that doesn't mean it's bad and it's going to compromise all your IP. That's a cloud issue. And the benefits of AI can still run on your local devices, behind your firewall, on your phone even. There's actually no known lower bound yet. So we've been able to take this technology and get it to work on lesser and lesser quality devices. Now that said, it's not going to be GPT 5.6 Sol running on your cell phone. It's not going to be that good. " ], [ 1400, "Lee Rodrigues", "Well, you can run it locally so then only Facebook is listening to your conversations and advertising to you about that. Are we getting close " ], [ 1406, "Sam Rogers", "Right. " ], [ 1410, "Lee Rodrigues", "to the advertisement time because we are... " ], [ 1412, "Sam Rogers", "Why don't we do that? That's a great idea. And this just in, Christine Rodriguez has a new report. " ], [ 1422, "Christine Rodrigues", "Still searching the old way, finding firing applications into the void and hearing nothing back. The market changed. AI broke it. Volume and polish don't work anymore. System 2 focus is a smarter, calmer approach to job search. Strategy over spray and pray. Signal over noise. A real system, not just a pile of tips. So stop Counting your number of applications and start running an actual strategic search. Get our free newsletter at system2focus.com. That's system, the numeral two, focus.com. System to focus, a smarter approach for a serious search. " ], [ 1467, "Sam Rogers", "Thank you, Christine, from the field. Well, thank you. I would like to get back to vacation. I've just been presented with a lovely mojito here. And why don't you tell the nice people at home how they can find out about you and System 2 Focus, starting with Lee. " ], [ 1488, "Lee Rodrigues", "System2Focus.com. And this is all about a systematic performance first approach around the workflow of a job search. And why does this matter? Because all the times you do a job, you're a part of a team and you're all working together. Second you lose your job, you're on your own. And sometimes we send a hundred resumes a week for a while. Once that peters out and you realize it doesn't affect, you may say, nothing I'm doing is working. this is the right time for you to use a workflow based system, which takes the mental aspects that happen with this. And I have a background on mindfulness and yoga and working through my own PTSD from army stuff back in the day. taught me ways to bring the focus back. And we use all of those skills through this. And we take Christine's approach of really being a master of marketing and saying, do all these tools in marketing to sell a product. What if you became the product? You would do some market research. You would do some interviews. You would do some analysis. You would think about the words that you're using. It almost seems like that would work for a job search approach. Christine, maybe we should start a business around this and start teaching people how to do it. What do you say we " ], [ 1567, "Christine Rodrigues", "You " ], [ 1568, "Lee Rodrigues", "do workshops, a series of six workshops starting in September? " ], [ 1573, "Christine Rodrigues", "That's right. We've got a webinar series that we've created to help people understand the different stages. We call it a workflow because one thing that I see when job seekers come to me and they're like, I'm just, I'm not getting anything. you know, I'm really getting frustrated and wondering if it's ever going to work. say, well, honey, you keep jamming yourself into step five. Right, of course nothing's working, but we get so wrapped up in it But that we're not willing to stop and go in reverse back to step one So we have the step process We'll have webinars starting in September And if you'd like to hear about what we have to offer because we're adding new things all the time go to system 2 focus.com Sign up for our newsletter. You can also find us on LinkedIn both Lee and I as individuals and we also have a system 2 focus company page. We're also on Facebook and Instagram. And if you happen to be a non-profit looking for strategic planning, you can also find us at randpassociates.com " ], [ 1631, "Sam Rogers", "Sounds great. Well, thank you both for being a part of this It's great to have you here, Christine. Really great to see what you're doing and how much help you can bring to people who are mid job search in the age of AI. Thanks again, Lee, as always, for being co-host. " ], [ 1651, "Christine Rodrigues", "Always a pleasure, Sam. " ], [ 1652, "Lee Rodrigues", "Thanks " ], [ 1652, "Christine Rodrigues", "Adios. " ], [ 1653, "Lee Rodrigues", "a lot. " ], [ 1654, "Sam Rogers", "Thanks and I'm your host Sam Rogers. It's been nice streaming with you from Cancun, Mexico. stay tuned next week when we will have Sabino Marquez coming to talk about the CISO perspective in the age of AI. Looking forward to a great show there. please come visit at sigsub.show. That's where our newsletter is. That's where you'll find previous episodes. where we'll drop the polished podcast on Fridays and the newsletter on Sundays. Live streaming again next Wednesday. Hope to see you then. " ] ] }, { "episode": 4, "title": "Safety & Trust in the Age of AI", "url": "https://sigsub.show/episodes/ep-004/", "transcript": "https://sigsub.show/episodes/ep-004/transcript/", "video": "https://youtu.be/qXSoVCkJmaw", "anchors": [ "t-0", "t-24", "t-77", "t-78", "t-96", "t-113", "t-117", "t-130", "t-130-2", "t-152", "t-152-2", "t-171", "t-171-2", "t-198", "t-212", "t-245", "t-245-2", "t-269", "t-271", "t-282", "t-282-2", "t-288", "t-288-2", "t-344", "t-346", "t-380", "t-380-2", "t-399", "t-400", "t-406", "t-407", "t-451", "t-452", "t-455", "t-459", "t-504", "t-508", "t-542", "t-543", "t-566", "t-569", "t-585", "t-586", "t-616", "t-616-2", "t-632", "t-632-2", "t-635", "t-635-2", "t-659", "t-660", "t-672", "t-672-2", "t-698", "t-707", "t-707-2", "t-712", "t-731", "t-731-2", "t-744", "t-766", "t-767", "t-780", "t-780-2", "t-845", "t-850", "t-850-2", "t-853", "t-853-2", "t-867", "t-868", "t-880", "t-882", "t-886", "t-888", "t-893", "t-894", "t-900", "t-901", "t-911", "t-911-2", "t-916", "t-917", "t-1017", "t-1018", "t-1022", "t-1036", "t-1037", "t-1040", "t-1041", "t-1041-2", "t-1067", "t-1068", "t-1084", "t-1085", "t-1094", "t-1098", "t-1138", "t-1138-2", "t-1153", "t-1155", "t-1156", "t-1156-2", "t-1177", "t-1178", "t-1210", "t-1211", "t-1251", "t-1251-2", "t-1264", "t-1265", "t-1277", "t-1285", "t-1286", "t-1319", "t-1319-2", "t-1344", "t-1367", "t-1367-2", "t-1370", "t-1370-2", "t-1401", "t-1401-2", "t-1477", "t-1479", "t-1518", "t-1531", "t-1531-2", "t-1536", "t-1545", "t-1546", "t-1565", "t-1566", "t-1583", "t-1583-2", "t-1595", "t-1596", "t-1619", "t-1620", "t-1627", "t-1627-2", "t-1640", "t-1641", "t-1645", "t-1692", "t-1734", "t-1734-2", "t-1735", "t-1751", "t-1753" ], "turns": [ [ 0, "Sabino Marquez", "By treating every operational thing as a cost to be cut, lowest denominator, get humans out, let's automate everything. And now there's no reason to trust your company. I'll trust you as much as I trust a robot from Walmart. And I don't trust those very much at all. " ], [ 24, "Sam Rogers", "Welcome to Signals and Subtractions. So, one of the things that has changed in the world since the onset of AI is how everyone is drowning in responsible AI paperwork, starting to suspect that maybe it doesn't do so much. Well, today's guest is happy to explain how he sees that. This is Sabino Marquez, who has experience. being a CISO, advising CISOs as Chief Information Security Officer for the uninitiated. He's also happens to be the Canadian ambassador to the Global Council on Responsible AI. So this is a discussion about safety and trust in the age of AI. Welcome Sabino. Great to have you here for your signal and subtraction. " ], [ 77, "Sabino Marquez", "Thank you, Sam. Thank you for having me. " ], [ 78, "Sam Rogers", "so I'd love to start off with just a little of your journey of having been a stalwart defender of the compliance mindset, and having that obligation on behalf of different organizations and how things have shifted for you since then. " ], [ 96, "Sabino Marquez", "Man, I I was forged in the old world, I came up in in the nineties where when you built a technical business or introduced technology into your business, it it had to be protected because that technology investment was a material advantage for you in the market. It was before everything was commoditized, right? " ], [ 113, "Sam Rogers", "Yeah. Yeah. " ], [ 117, "Sabino Marquez", "what I found was that compliance became the dominant paradigm in the early two thousands after all of the corporate scandals of the turn of the century. the the world comms and enrons and all of those brought on Sarbanes " ], [ 130, "Sam Rogers", "Right. " ], [ 130, "Sabino Marquez", "Oxley and then s and then the era where technology was a strategic executive function that led the business kind of got put under the Sarbanes Oxley and GLBA umbrella where the CFO now controlled IT general controls and risk management and compliance became the job. And it didn't used to be that way. So I " ], [ 152, "Sam Rogers", "Mm. " ], [ 152, "Sabino Marquez", "was there for that transition and I was like, all right, I guess we're doing compliance now. And what I found was that the more compliance you did, the less safety you could prove you were providing. Because the compliance frameworks that we were all following aren't really centered on value safety. They're centered on risk management for " ], [ 171, "Sam Rogers", "Yeah. " ], [ 171, "Sabino Marquez", "a company moving at full speed that won't stop. Right. we're com we're complying around the wrong objects. compliance just measures, the cheapest way to check this box because it becomes a race for operational efficiencies rather than what it should be, which is this machine, is this capital machinery running to its maximum potential? And it only does that when it considers value safety, which is one of our our our guiding concepts in trust value management. " ], [ 198, "Sam Rogers", "Yeah, well I was on the other side of all that around the same time being the cheapest answer to the question juggling all those check boxes of who took what training I was on the learning and development side on the HR tech side and all that. " ], [ 212, "Sabino Marquez", "the reason you do this kind of work, Sam, is because you have a strong defenders spirit. You you see something valuable and you're just called to defend the value. But the problem is that the business doesn't see us in that way. It sees us as a security-shaped answer to a security-shaped problem between them and the dollar they want. And what's the cheapest way to answer that that that hole that sh that it's shaped like you? It's how a CISO ends up accountable for every trust creating practice in the business without being empowered to run that trust making machine. and " ], [ 245, "Sam Rogers", "Yes. " ], [ 245, "Sabino Marquez", "then I I I became disillusioned with compliance because of that, because what when I was doing compliance for a living, I'm like the reason we do compliance is so that people can act safely in defense of this valuable thing that we're defending. after really thinking about it, it turns out that I wasn't hired nobody's hired to defend anything valuable because the business may not really understand what it has that's valuable. It it's running a playbook, usually written by someone else, right? " ], [ 269, "Sam Rogers", "Yeah. Yeah. " ], [ 271, "Sabino Marquez", "they're like, I'm on the I'm on I'm trying to get as much growth as possible in the short amount of time for the lowest cost, and I keep running into this pothole that looks like Sabino. And Sabino, " ], [ 282, "Sam Rogers", "Mm. " ], [ 282, "Sabino Marquez", "can you just fill this pothole so I can run my troops over? And that's what compliance became. And " ], [ 288, "Sam Rogers", "Yeah. " ], [ 288, "Sabino Marquez", "the idea that you could differentiate your business with trust kind of got left in the dot-com, you know, back when we were building and selling trust as an industry. and and that's how we ended up getting to this point where where where you have to re recontextualize what compliance does versus what it is. Because almost all compliance activities generate exactly zero value. Because the reason we do compliance, again, I'm gonna spell it out, is so that other people can feel safe enough to give you the value that they hold in their hands. Show me all the compliance things. Why? Because I need to make sure that. This thing that I'm gonna run through your system or your company is safe when it goes in, safe when it gets there, and when it comes out, it's as valuable as it was when I gave it to you, plus the value you're giving me. And that's a very different mission than compliance, " ], [ 344, "Sam Rogers", "Yeah. Yeah. " ], [ 346, "Sabino Marquez", "I used to audit. I was an auditor for many years and I treated audits very seriously because I thought their function was to tell us something true about reality while there was still time to act. see compliance people, like finance people live face to face with actual, unchanging, empirical, punch you in the face reality. That's what we look at every day. And then our job is to present that reality to people who are not required to consider it in the running of their playbooks because of something I write about a lot, which is the trap of the trusted advisor, in which " ], [ 380, "Sam Rogers", "Right. " ], [ 380, "Sabino Marquez", "you're you're you're told that you're you're important. Sam, you're a CAIO. You're a C I S O. We can't do it without you. Of course you're allowed to be in the room. But you know what? You don't really have a business to run, do you? You don't you don't predict the number and deliver it. You're just advise, right? Like lawyers? All right. " ], [ 399, "Sam Rogers", "Yeah. " ], [ 400, "Sabino Marquez", "I'm gonna treat you like a lawyer. My lawyer tells me don't break the law all the time, but you know what? We gotta make money here. And and " ], [ 406, "Sam Rogers", "Yeah. " ], [ 407, "Sabino Marquez", "and and the advisory frame traps change I spent a lot of time, maybe 10 years, in the enterprise software space. And the enterprise software space is essentially companies with proprietary data needing to make sure that you're safe enough to give that proprietary data to. In your software, so you can process that data and then give them back value. you get essentially the family jewels in those types of businesses, really valuable data. So the we're selling to the Fortune 50. If you've ever sold to the Fortune 50, you know the type of due diligence processes that they run on you. They they come in and they count every armpit hair one by one, Sam. Look. And and and and they have an atlas of armpits, you know, to that that that's " ], [ 451, "Sam Rogers", "Ha ha ha. " ], [ 452, "Sabino Marquez", "the level of diligence. And if one is out of place, they want a discount, they want special considerations, they want contractual, " ], [ 455, "Sam Rogers", "Yeah. Yeah, yeah, yeah. Yeah. " ], [ 459, "Sabino Marquez", "you know, it like it's a mess. And so I said, you know, these people, these people on the other side of the table are just like me. They're trying to, you know, do the job to protect the company. it's like pulling teeth because Companies like mine typically don't make the investments necessary to project trust in evidence. You know, companies that are startups never invest in this. So we had to figure out a business system rather than a technical system that would make the company move in a way that the evidence the company generated was, hey, you can trust us. we found that, and I can say this now because it's been many years. I used to have a PDF with a script that it would open and it would you know pull down a transparent GIF and it would let me know that somebody somewhere opened one of the PDFs. And I did that because I wanted to know. Yeah, basically a tracking pixel. Because " ], [ 504, "Sam Rogers", "A tracking pixel essentially. Yeah. " ], [ 508, "Sabino Marquez", "I wanted to know this company spends $125,000 a year on their service audits and penetration tests. They spend X tens of thousands a year on all this software to detect every kind of threat in the world. They hire people like me and give me a team to bring all this compliance and security and safety and com and risk and stuff together. Now, now that I have all that documentation, is anybody looking at it? And the answer sadly was no. Because what I found was when you sent them a couple of documents, they would open a couple of documents. But " ], [ 542, "Sam Rogers", "Yeah. " ], [ 543, "Sabino Marquez", "when you sent them, Your trust story composed of 40 documents woven to of a pure evidence of their safety in your hands. So rather than sending them compliance things, When I give somebody a bucket of compliance and security documents, I'm asking them to assemble a puzzle without without what the cover looks like. It's burden for them, it's labor for them. They " ], [ 566, "Sam Rogers", "Hmm. Yeah. Yeah. " ], [ 569, "Sabino Marquez", "what they really need is to is to is to look at the thing I sent them, whatever it is, and then feel emotions about it. Because one of the models we follow is that no matter how technical you are or how analytical you think your job is, data " ], [ 585, "Sam Rogers", "Mm. " ], [ 586, "Sabino Marquez", "makes you feel. And what we found is that when we organize all of that compliance, data in the form of stories that are told to people's pain. you're the IT guy, you're the security guy, you're the privacy guy, you're the compliance guy, you're the procurement guy, you're the lawyer. You're the insurer, you're the regulator, you're the auditor, and all of them are going to look at the same bucket of evidence, but the way it needs to land on them is different. So you have to tell different " ], [ 616, "Sam Rogers", "Yeah. " ], [ 616, "Sabino Marquez", "stories with the same evidence, which is why when you run security and compliance and blah, blah, blah as a trust practice, you're treating the company as a factory that generates documented evidence of you being safe when " ], [ 632, "Sam Rogers", "Hm. " ], [ 632, "Sabino Marquez", "other people give you something valuable. " ], [ 635, "Sam Rogers", "Mm-hmm. " ], [ 635, "Sabino Marquez", "And that you return that value plus when people engage you. And when they engage you, they are safer. They don't have to worry about breaches with you, or they don't have to worry like you respond within X amount of time, you keep your SLAs, you're trustworthy. You're no longer a vendor or a supplier, you're a partner. And then the customer leads with safety once you let them know they're safe. my god, I'm safe I'm safe here. My career " ], [ 659, "Sam Rogers", "Mm-hmm. " ], [ 660, "Sabino Marquez", "is safe. I made a good decision coming here. My bonus is safe. My team is enabled by these capabilities. Yeah, there's another widget seller. Who has a 10% better widget, but is it really worth the risk of " ], [ 672, "Sam Rogers", "Yeah. " ], [ 672, "Sabino Marquez", "knowing that these people are there for me? Look, when I turn my back, they keep acting like like people. They're like, they're like they're like good. They keep me in the center. I feel like I'm in the center of their model. I'm not just a subscription to them. I can sense it in my body Like you you feel the trust. And then other companies really can't compete because you can't buy that. That's just your company running in a trustworthy manner. Trust value management. " ], [ 698, "Sam Rogers", "Well, thank you for that breakdown of how that actually works, like in a in a manufacturing sense. Like that's a that's " ], [ 707, "Sabino Marquez", "Yeah. " ], [ 707, "Sam Rogers", "a great analogy to use there. I think it's time for your signals " ], [ 712, "Sabino Marquez", "my signal is that everyone doing AI compliance things is really just doing AI procurement things. Because what are we complying to? We're complying to a way to make this thing buyable, movable, purchasable, integratable. They're selling insurance to offload the risk of doing it. You know what I mean? I " ], [ 731, "Sam Rogers", "Yeah. " ], [ 731, "Sabino Marquez", "think that all that is important, but it's the wrong thing. It's the wrong first question. The first question is AI value safety. Is it going to keep my value safe in its custody and return it plus new value? " ], [ 744, "Sam Rogers", "with safety and trust being so much in the news like just today as we're recording this, the OpenAI report of hacking hugging face with their own internal model basically breaking out of the lab to hack the test so that it could pass it. It doesn't know not to do that. Like to your point, it doesn't have the concept " ], [ 766, "Sabino Marquez", "Yeah. " ], [ 767, "Sam Rogers", "of where where the bounds are. I just wanted to remind everyone that when it comes to trust. AI is much more like people than it is like computers. " ], [ 780, "Sabino Marquez", "Yes. " ], [ 780, "Sam Rogers", "the deterministic systems that we're used to working with with computers over the last fifty years or so. But when it comes to AI, it's much more like You just hired somebody off the street and it's day one on every conversation because it's like talking to someone with amnesia, it's their first day all the days. they can be trusted to mess certain things up. If you just hire someone off the street And they tell you that they did something and they did it right and they did it well. You you check, right? You you would look at the work, you would be able to evaluate it. You might have some standards that you thought about in advance, you might have some standards that you use for hiring and bringing that person in. If someone's new to your team, you don't just give them the keys to your kingdom on the first day. You make sure that those are someplace that they can't access as a matter of architecture. Like you keep those keys. They're not just hanging out where they could be grabbed. That enforcing a certain pattern of behavior in this realm. You're about to disagree with me, and I'd love to hear it. " ], [ 845, "Sabino Marquez", "S safety is a feeling in the body. Let's start there. " ], [ 850, "Sam Rogers", "Okay. " ], [ 850, "Sabino Marquez", "Because since safety is a feeling, how can an " ], [ 853, "Sam Rogers", "Hmm. " ], [ 853, "Sabino Marquez", "AI know safety? Right? and that's the that's the design question. we're making first principles errors with AI here. AIs can't be safe because you're the one who has to be safe. I bring this back to Responsible AI. Okay. " ], [ 867, "Sam Rogers", "Mm. " ], [ 868, "Sabino Marquez", "I spent two years as the ambassador, the Canadian ambassador for the Global Council for Responsible AI. responsible AI is a little bit like responsible ownership of a bull in an apartment. " ], [ 880, "Sam Rogers", "Ha ha ha. " ], [ 882, "Sabino Marquez", "Sure, you can responsibly feed the bull and take it out to poop and but that's it's not but the the the " ], [ 886, "Sam Rogers", "Ha ha ha. " ], [ 888, "Sabino Marquez", "problem is not your responsible use of the bull in the it's the bull in the apartment, right? A a AI " ], [ 893, "Sam Rogers", "Yeah, yeah. " ], [ 894, "Sabino Marquez", "is kind of like this. AI it is a force amplification technology. And I use technology " ], [ 900, "Sam Rogers", "Right. " ], [ 901, "Sabino Marquez", "in in the in the social science way, it allows the the machine to Amplify whatever force I'm projecting. If I'm creating value, I'll create more value. If " ], [ 911, "Sam Rogers", "Yeah. " ], [ 911, "Sabino Marquez", "I'm extracting value, I'll extract more value. If I'm destroying value, I'll destroy more value. If I'm " ], [ 916, "Sam Rogers", "Right. " ], [ 917, "Sabino Marquez", "screwing you, I'll screw you better. If I'm serving you, I'll serve you better. Because it's not the AI. It's not the AI. It's what humans are doing with the AI. And they do what they always do. When given a bigger stick, you swing it. When I can amplify my force, my Goals, my desires, my plans, my reach, my expanse, all of that is what AI is. This is a capability. it makes an individual able to expend the force of 10. It's like drone warfare. We used to need humans with guns and boots to express force. Now we send drones. Same force, lower cost, lower energy, same force result. And that's kind of what AI is. And because of that, I don't think that there's responsible use of it. I think that there's only safe use of it, like a gun. And I apply gun safety models, weapon safety, ordnance safety models to think about how. AI will touch the things I value. So if you think about AI in that context, value safety is literally capital S safety, because AI things are pointed at you by people to do the things they want. And that's kinda how I came to the fact that it's not ethical AI, it's not responsible AI, it's not consensual AI, it's not compliant AI. What you want is a safe AI and no programs, no audits, nothing on the market today is asking the question and answering it, is this thing safe for my value? And that's how we got " ], [ 1017, "Sam Rogers", "Mm. " ], [ 1018, "Sabino Marquez", "to the concept of value safety. " ], [ 1022, "Sam Rogers", "Well, we didn't plan to talk about this, but I'm curious about your thoughts US government intervention and the release of certain models. Now the Chinese government is threatening to do the same thing. We're also threatening to ban their models because they're too " ], [ 1036, "Sabino Marquez", "Yeah, yeah. Yeah. " ], [ 1037, "Sam Rogers", "dangerous, like open source models, not just closed source controlled " ], [ 1040, "Sabino Marquez", "Yes, yes. " ], [ 1041, "Sam Rogers", "ones, " ], [ 1041, "Sabino Marquez", "this proves the point. AI is a sovereign technology. What are nation states, sovereigns. sovereigns understand AI in the force amplification frame that I just explained to you. So that's why you're gonna make Chinese AIs illegal or protect American AIs or make it illegal to export the AI, or make transporting AIs across state lines eventually regulated, because AIs are weapons. They're telling you by their behavior. They're " ], [ 1067, "Sam Rogers", "Mm. " ], [ 1068, "Sabino Marquez", "telling you by their behavior. it'll always look like freedom in whatever country you're in, right? But it's it's but it is what it is. It is a a a technical enclosure. And AI enables you to scale and do this. The Stasi could never do this. You could never do this with Stalin. You needed " ], [ 1084, "Sam Rogers", "Hm. " ], [ 1085, "Sabino Marquez", "billions of people doing that kind of work, and AI allows that kind of amplification. AI is a weapon that makes cat pictures. Right? And that it makes cat pictures does not yes, not cute like " ], [ 1094, "Sam Rogers", "Yes, now cute has learned to kill. Yes. " ], [ 1098, "Sabino Marquez", "it's a kitten with a Kalishnikov, you know? Right now, all of the technologies and methodologies available to professionals in legitimate corporate worlds are about procurement. ISO this and SOC2 that. And and the only reason you even need those is because someone wants to buy it and you gotta check the boxes. But the boxes being checked are not safety boxes, and value is being irrevocably destroyed every day by AIs because it's not being used safely. I am not anti-AI. But I am anti-extractive and coercive with impunity AIs that serve nakedly to amplify incentive at the expense " ], [ 1138, "Sam Rogers", "Mm. " ], [ 1138, "Sabino Marquez", "of the stakeholder's value. Because the stakeholder is not just the shareholder, Sam. It's everybody who gets value from your company being trustworthy. And that's employees and customers, sometimes competitors, definitely your partners, you know, and your community. " ], [ 1153, "Sam Rogers", "Yeah. Yeah. " ], [ 1155, "Sabino Marquez", "Everyone who " ], [ 1156, "Sam Rogers", "Yeah. " ], [ 1156, "Sabino Marquez", "depends on you being trustworthy gets value. having had had conversations for two years now, I go into rooms filled with all kinds of leaders, Sam. And they asked me about AI this and that, and I underscored to them to to temper their exuberance, because they believe they can replace humans. " ], [ 1177, "Sam Rogers", "Mm. " ], [ 1178, "Sabino Marquez", "The if everybody believes they can replace humans, that's dangerous. But everybody's incentive is so short term, it doesn't matter if you replace your humans, I'll get punished if I don't replace mine. the best we can do as as not just advisors, but trust leaders who are trying to lead organizations human beings can give their value to. Is help our leaders understand the risk of removing all the friction from the machine, of not having brakes. We're entering an era of unrestricted velocity. Here's my lesson. " ], [ 1210, "Sam Rogers", "Yeah. " ], [ 1211, "Sabino Marquez", "Here's the big subtraction. We're entering an era of unlimited, unrestrained velocity. And in that era, responsible and ethical and consensual and comp all these fig leaves are created for the purpose of accelerating the adoption of and Making it go, making it go. There is no responsible use of AI. That's a subtraction. There is no ethical use. There is no compliant use. There is only safety of your value, of of of the value that your company has created, which you may not have even inventoried because you don't know why you're valuable. " ], [ 1251, "Sam Rogers", "Hm. " ], [ 1251, "Sabino Marquez", "This is one of my most common conversations. Why does your company hold value? because we have contracts. Okay. What else? we our our our GitHub repo. Like like " ], [ 1264, "Sam Rogers", "Ha ha ha. " ], [ 1265, "Sabino Marquez", "you know, like like they don't understand that companies made of humans are valuable because of the synthesis of humanity and everything, all the yucky human stuff leaders hate to talk about. " ], [ 1277, "Sam Rogers", "Well thanks for throwing the subtraction in there I would like to offer something that I think is a compliment to what it is that you're you're " ], [ 1285, "Sabino Marquez", "Yeah, yeah. " ], [ 1286, "Sam Rogers", "saying that I've been seeing more lately, there's this phrase human in the loop that gets used a lot, as if it's some kind of quality assurance, just because there's a person that's pressing a button as opposed to a machine. pressing a button. There are reasons, cultural reasons, why we w might want to have a human in the loop, especially at this stage, but ultimately it's the ownership of the process. It's the business value that we're holding is owned by humans, not by machines. And when we " ], [ 1319, "Sabino Marquez", "That's correct. " ], [ 1319, "Sam Rogers", "can take the full end-to-end approach to whatever loops you're making, I know there's a lot of loopy talk these days However, it is not protection from anything, and it doesn't necessarily help. if you've pressed that button 47 times before lunch and you press that button again 48 and go to lunch, chances are the button that you're pressing isn't quite the awareness that is needed for that process. " ], [ 1344, "Sabino Marquez", "Yeah, yeah. We're we're applying industrial control mindsets This is not a power plant dashboard with dials and knobs. You know what I mean? It it's a sovereign capability. the human in the loop question is interesting because the human in the loop is just like the the engineers sitting at the Chernobyl dashboard. They may they're " ], [ 1367, "Sam Rogers", "Yeah. " ], [ 1367, "Sabino Marquez", "not nuclear engineers, they just ride the dashboard. " ], [ 1370, "Sam Rogers", "Yeah. " ], [ 1370, "Sabino Marquez", "the real conversation is the human conversation about what we're gonna use this for. many technologies allow you to mask human defect, but this amplifies human defect. Because you think that you're just gonna use it and it's gonna just gonna do the thing. And it will, but it's gonna do all of the thing, including all your bad things. means I have to, in my decision making, actually consider the harms of doing the business the way that I do. And ha and I'll use harms in quotes, because it's the harm to trust. To " ], [ 1401, "Sam Rogers", "Yeah. " ], [ 1401, "Sabino Marquez", "to to running your business in a way that evades doing the right thing because you're you think you can outrun your own consequences. the majority of velocity focused operators I see don't seem to realize that a benefit of velocity is outrunning every explosion they set behind them as they're running. So they can get the benefit of being propelled by the explosion and not have to stick around for the wreckage the wreckage of running your company in an extractive and coercive manner that destroys the capacity to be trusted at all. Because you think you can do a better marketing push. Or you think you can put a poster on the wall about your values, or you'll discount your mess ups for a couple of quarters, the customer will get over it. No, it like like like this is not a good way to run a business. And and and I've been doing this long enough to be able to say that. every successful business that created conditions, and I'm gonna say this clearly, for sustainable value creation was a trustworthy business. And every business that treated every quarter like war was an untrustworthy business. And they were so because of their operations and the way they composed them. Everyone's looking for an exit somehow or another raise or something else. That's two-thirds of " ], [ 1477, "Sam Rogers", "Yeah. Yeah. " ], [ 1479, "Sabino Marquez", "the economy. It's been so financialized. How is this thing valuable? How does it add? Well why is source code or a well-written contract or my financial data or our pricing logic or the customer histories we've developed or the institutional judgment our leaders have? accrued or the research we have done into the market and the product or the meaning of our brand. These things are value. They're not raw material you try to transform into value. They're the value of the company itself. And so when you give these value of the company itself objects to AI, you know what is degradation look like? " ], [ 1518, "Sam Rogers", "this has been fantastic And I love being grounded in the technology enough and and having had so many of these conversations for so many years to then still keep that ethics element as as something that's " ], [ 1531, "Sabino Marquez", "Yeah. " ], [ 1531, "Sam Rogers", "actually alive and not just a not just a checkbox somewhere. " ], [ 1536, "Sabino Marquez", "No, not alive. It's more profitable. Treating people well makes you more money. I I know I know it's revolutionary and you're But but " ], [ 1545, "Sam Rogers", "Ha ha ha. " ], [ 1546, "Sabino Marquez", "but it turns out that if people trust you, they'll just they'll just keep giving you money. You won't have to resell them. They'll refer their friends to I mean it's incredible what happens when you don't run your company like a hamburger grinder. the value is the human. There is no human in the loop. Okay. Let me close that. There is no human " ], [ 1565, "Sam Rogers", "Sure. " ], [ 1566, "Sabino Marquez", "in the loop with me. There there is a safely designed force amplification system or a negligently designed force amplification system. no amount of control can guarantee safety, which is where we are now. Because as I said, we're entering " ], [ 1583, "Sam Rogers", "Yeah. " ], [ 1583, "Sabino Marquez", "a an era of unlimited force amplification with AI, and safety is gonna be the number one buying criteria, even before. Price and quality. You know the triangle? Right? price, you know, " ], [ 1595, "Sam Rogers", "Yeah, yeah, yeah. " ], [ 1596, "Sabino Marquez", "quality, okay. I've been doing this system for 10 years, and it's made about a half a billion dollars more for my companies than would have been made had we not done this. We exited at higher values, raised more in the rounds, sold more faster at lower costs higher ACVs every every metric that makes a CFO rub their hair with mayonnaise is the metrics " ], [ 1619, "Sam Rogers", "Yeah. " ], [ 1620, "Sabino Marquez", "that we were affecting when we were running a trust value system. So I'm not I'm not preaching. I'm describing. I'm " ], [ 1627, "Sam Rogers", "Yeah. " ], [ 1627, "Sabino Marquez", "describing this works and we and and and we we give it away. because you can't sell this. it's it's the better way. It's like I used to walk on one foot, now I walk on two. I'm not gonna charge for that. You have two, " ], [ 1640, "Sam Rogers", "Ha ha ha. " ], [ 1641, "Sabino Marquez", "you have two feet, go walk on two feet, right? So " ], [ 1645, "Sam Rogers", "Well that's that's a that's a perfect segue, I think, into now a word from our sponsor. What is the price of trust? Every organization depends on it. Buyers price it into deals, employees price it into effort, customers price it into loyalty, investors price it into risk. Yet most companies still can't see it until it's gone. Trustable builds instruments for measuring current trust value and designs operating conditions that compound trust over time. Trustable. Measure trust where value flows. so if people wanted to find you and what it is that you're developing, where do they go? How do they keep track of Sabino Marquez? " ], [ 1692, "Sabino Marquez", "we have a website at trustable dot TV, which is very consulting focused, but it it lets you get in contact with us for all the other things. And also if you do trust work for a living, are you a CIO, CISO, do you work in security, risk, compliance, legal? Are you in finance? I keep an ongoing blog called trustclub.tv, where I publish the majority of the system, in usually in the form of essays and instruments And so for people that want to do business with us, I send them to trustable.tv. But people who want to be trust value leaders, the trust system overlays your company. It doesn't replace your company. Your leaders can still have their values you can measure trust value in the company and " ], [ 1734, "Sam Rogers", "well it's been fantastic " ], [ 1734, "Sabino Marquez", "Yes. " ], [ 1735, "Sam Rogers", "having you here. appreciate your perspective and expertise and well-rounded approach to safety and trust and and what has changed in the age of AI and what hasn't. thanks so much for being here, Sabino. " ], [ 1751, "Sabino Marquez", "Thank you so much for having me, Sam. It was a pleasure. " ], [ 1753, "Sam Rogers", "Thanks so much for joining. We live stream on Wednesday. We do a polished podcast out by Friday, and on Sunday there's the newsletter. See ya soon at SigSub Dot Show. " ] ] }, { "episode": 5, "title": "AI Detection and the Presumption of Guilt", "url": "https://sigsub.show/episodes/ep-005/", "transcript": "https://sigsub.show/episodes/ep-005/transcript/", "video": "https://youtu.be/U5Ew_rvlqnE", "anchors": [ "t-0", "t-28", "t-115", "t-128", "t-152", "t-184", "t-193", "t-293", "t-294", "t-385", "t-389", "t-418", "t-426", "t-441", "t-459", "t-464", "t-567", "t-606", "t-607", "t-621", "t-621-2", "t-656", "t-656-2", "t-672", "t-860", "t-895", "t-1062", "t-1089", "t-1306", "t-1322" ], "turns": [ [ 0, "Limited Edition Jonathan", "Substack has a button that you can click on now that tells you whether or not, the post is generated by AI or assisted by AI or written by AI, which is just a stigma-generating machine. And, uh, that's gross. I want people engaging with my ideas. I don't want people engaging with my process " ], [ 28, "Sam Rogers", "Welcome to Signals and Subtractions. I'm your host, Sam Rogers. Last week, Substack shipped an AI detector, which means a writer can now be scanned by a machine, labeled by that machine and how it recognizes its own, and the writer is invited to file an appeal, but only after the fact. Today's guest is my friend Jonathan Edwards, AKA Limited Edition Jonathan. He's a filmmaker who teaches AI fluency, often by documenting his own failures in public, and who built AI cred to find out whether anyone's actually AI fluent. He's one of the top 10 rising voices in technology on Substack, and this week he wrote a piece called Nothing Was in the Water his position is whether the AI label is accurate is beside the point. The label itself is doing harm. We live streamed this on Wednesday via Substack while Jonathan was unfortunately in the car. Not the kind of recording quality we usually have here at Signals and Subtractions, but in this case it couldn't be helped. I'll let him tell why. It's legit. Anyway, for this episode, I'm going to step in on the places where the cell connection ate his argument. He was a champ to show up at all " ], [ 115, "Limited Edition Jonathan", "it's nowhere cool. It, it is the hospital. no, my son's at his summer camp right now, and, uh, he got violently ill, so they are taking him to the emergency room, and it's a two-hour drive for me. " ], [ 128, "Sam Rogers", "Thank you for being willing to do this, and help us, T-Mobile. You're our only hope. So Jonathan, I was hoping that you could, say a little bit about what your, opinion is of the new announcement from Substack, in case anybody isn't aware of your, very tepid, mild-mannered views on the world. " ], [ 152, "Limited Edition Jonathan", "Honestly, like AI detectors just like, other kind of purity tests are destructive in a way that, you don't really notice at first. So I have this whole thing about, about stigma, right? We, we, we do it in a lot of areas. Like you look at an ADHD kid and, you're like, \"The kid's lazy. He doesn't pay attention. He doesn't wanna do anything. And then you apply this stigma to that kid that automatically, puts a label onto him and, you know, kinda screws him for life. The article goes over like just the different stigmas that we apply to different things. " ], [ 184, "Sam Rogers", "You start with the nothing was in the water example, and th- and that analogy is fantastic, the whole, uh, nocebo effect " ], [ 193, "Limited Edition Jonathan", "Yeah, 'cause it's one-to-one, right? So people flip out about fluoride and like really the stupidest people in the world. Like genuinely I have no respect for anybody who hasn't done enough research to know that fluoride being added to water is not harmful in any measurable way. I don't know, there's something that actually bonds to your teeth and bones that prevents cavities. It's been proven. They discovered it because, I think somewhere in the Netherlands. Uh, fluoride occurs naturally in the water. And they're like, \"Why don't these kids and adults get cavities the way everyone else in the world does?\" And they discovered it was fluoride \"Okay, so what if we add it to the water everywhere else?\" And it turns out that works, and it's awesome. And pseudo-scientists across the world trying to sell you, you know, whatever snake oil they're trying to sell you have decided, \"We're gonna tell everybody that fluoride is evil. It's causing, uh, I don't know, They just make stuff up, and then people decide to believe that, and then as evidenced by, It, it, it's a hilarious story. In New York, they had announced that they were gonna put fluoride in the water for this reason. And then the government, had to have a delay on doing it. They were, like, a month and a half behind. But they didn't announce that they were delaying adding it to the water. The public thought that it was gonna go out on a certain day, and then on that day, they started calling in and complaining like, \"Oh, I'm getting headaches. It's giving me stomach aches. You know, I, I'm having health issues because of the fluoride you added.\" And, um, but they hadn't added anything, so there was literally nothing in the water, and people were just deciding that, they were getting sick because of something they had in their mind. And it's a one-to-one analogy to, using AI to write, to help you clarify your thoughts and get it out there, " ], [ 293, "Sam Rogers", "yeah. " ], [ 294, "Limited Edition Jonathan", "Regardless of what it is, if you're told that it's generated by AI, you value it less. Fun fact: AI values it less. The second analogy in here is about a Monet. Somebody posted on X. They're like, \"Hey, I generated this Monet, style painting and, you know, everyone's saying, \"Well, this is, this is awful. This is, this is nothing like Monet would actually produce. He would never paint something like this.\" And then the guy revealed, \"Hey, dumbasses, this is a Monet.\" Uh, like, I just told you it was AI generated. This is actually Monet.\" And, conveniently, a lot of people just, like, deleted their replies. But fun fact, I gave the same exact, image to both ChatGPT and Claude, and I asked both of them the same question he asked his ex audience, and I'm like, \"Hey, how close is this to Monet? This was generated by AI, what do you think?\" And, um, AI had the same exact bias that humans have. If they thought it was generated by AI, they called it soulless, \"Uh, kinda good, it's getting close, but, you know, it'll never be like real human stuff.\" So that bias is implicit, and it's built in to both humans and AI. Which I thought was really funny, but also, at the same time, Substack has a button that you can click on now that tells you how much of or whether or not, the post is generated by AI or assisted by AI or written by AI, which is just a stigma-generating machine. That's all it is. And, uh, that's gross. " ], [ 385, "Sam Rogers", "I don't seem to find that button on your post, Jonathan. Why is that? " ], [ 389, "Limited Edition Jonathan", "Because (BLEEP) you, you're not entitled to my process. I will happily help you. Like, my whole MO is teaching people how to use AI, teaching, people how to use different tools and combine them together to make greater things. I think AI is the great, equalizer in a lot of ways. But nobody has a, a right to demand my process for how I do things. And, uh, full disclosure, this post that I wrote is almost entirely... Well, you know what? No, screw it. I'm sticking to it. I'm not, I'm not telling you " ], [ 418, "Sam Rogers", "Yeah. Not, not telling us. Wait, wait with bated breath to find out is it AI or is it not AI? " ], [ 426, "Limited Edition Jonathan", "The this stigma is actively dangerous. We have that nocebo effect where we automatically determine that AI-generated content or AI-generated anything is worse than human-generated is actively harmful in a lot of different ways, " ], [ 441, "Sam Rogers", "yes. So I was wondering if you could say a little bit more too about the damages, about the costs of that, with regards to that stigma. Like, you've, you poked at it a little bit with the ADHD example but how does it actually hurt Substack and Substack readers? " ], [ 459, "Limited Edition Jonathan", "When you discount the quality of something because you know it was generated by AI, " ], [ 464, "Sam Rogers", "And then the connection got painfully difficult, so allow me to summarize. This is his argument from the published piece, not mine. The writer who gets flagged stops publishing. That's the cost. There's not a debate, there's not a correction or any better argument. There's just a vacuum where the writer used to be. And stigma doesn't distribute by merit. It lands on whoever is least equipped to fight it. Jonathan's whole point about AI is that it lets a person without a team compete against people who have whole teams' worth of effort behind them. Those are exactly the writers that the label discourages first. Plus, the scan runs downward only. So writers get scanned, but the platform's own posts don't. For Chris Best, the CEO of Substack, his announcement of this feature notably carries no disclosure of its own. So no notification when your stuff is scanned. The appeal, if you want one, comes after the label already exists, which is essentially a presumption of guilt with a customer service form attached to it. Now, this is a lot bigger than just Substack. This kind of conversation has been happening for a while, and it continues to happen across every platform in this age of AI. That's why we're talking about it. It's a strong signal. And for Jonathan, it's also a strongly held conviction " ], [ 567, "Limited Edition Jonathan", "I just wanted to say, um, even before we had this, AI detector pangram thing, I was vehemently opposed to anybody disclosing whether or not they used AI. I don't think people owe it to their audience, and I don't think the audience is owed, any kind of disclosure on how you do your writing. And that's kind of been, like, the standard for journalism for all of eternity. We don't owe our audience our sources. How many ghostwriters, how many editors, how many- Right... you know, how, how much of a process in writing journalism, all that kind of stuff? You'd be shocked how many big name authors, haven't written a word in years. Yeah. Or they write the outline, and somebody else finishes it up for them. " ], [ 606, "Sam Rogers", "Yeah... " ], [ 607, "Limited Edition Jonathan", "we don't consider that dishonest. We don't consider that something we have to disclose. In fact, we actively don't disclose that kind of stuff because it creates the stigma. We don't actively disclose how much editing was done to Stephen King's last novel- " ], [ 621, "Sam Rogers", "Yeah " ], [ 621, "Limited Edition Jonathan", "because that's actively harmful to the, the reader who just wants to read something, to gain some kind of knowledge or, you know- Right... enjoyment from that thing. Disclosure is, it's a distraction from the point of what you're doing. You're trying to share an idea. In my case, My whole MO is, helping people level the playing field and using AI to do that, because there's so many advantages to AI. And it gives you the ability to punch way, way above your weight. For someone like me, it gives me the ability to compete against, those teams of people who have editors in front of their content. Like- " ], [ 656, "Sam Rogers", "Right... " ], [ 656, "Limited Edition Jonathan", "you know, I don't have a team of people working on my stuff to make sure that it gets delivered well, and that it doesn't, you know, piss anybody off, and that's a huge thing. You have no idea how often, AI really helps me soften my very, very hot takes. Which you can read in my notes because I don't filter my notes at all. " ], [ 672, "Sam Rogers", "Right. When, when we're taking, quality measures... and we're taking the way that something was made and inferring, that those properties are related to the quality of the output. That's what I love to pick apart, because I think there's an opportunity in moments like this. When people are seeing behind the curtain, just as you're describing, there's so many authors that have ghostwriters that write their next work, I don't think that everybody knows that, because for all the reasons you're just, talking about here, Jonathan, it's, it's very true. Ultimately, a measure of quality has to do with what the expectations were at the outset. Which is why, and I know Jonathan has his own choices and probably criticizes me for mine, but that's why I disclose, and always have, when something is AI assisted, that's what I label. Now, sometimes I'm labeling it at the, distribution level, like with the Signals and Subtractions newsletters. Those have been largely AI assisted for about a year now. But for long form articles that I write, unless otherwise stated, it's something that I might have had AI do this editing function and make some suggestions, but it didn't change any words for me. I treat it like I treat an editor. And as a, trained writer as well as a journalist back in the day, all of the things that Jonathan's talking about are very real, like, normal processes that writing would go through. Getting back to is it correct thing, well, correct for what? But we can still say, regardless of that taste layer of do I like it or not, of the expectation layer of does it meet my expectations or not as a reader? Does it have the labels that cue me in on what to expect for this quality here, which is the whole Substack pangram thing that we're talking about today. There is something intrinsic to writing itself which is measurably good or measurably not good, which is not a matter of taste and it's not a matter of origination. It's does the object that has been produced meet the quality standards that we have expressed? Which is where I would love to see this conversation go, because I think that there are opportunities that exist now that didn't exist previous to this age of AI kind of era. There are tools that we can apply. There are ways that we can use AI to help strengthen What the writing is. When people ask me, \"How was this article written?\" I always want my answer to be, \"Very well, thank you.\" Like, no matter what the origination process was, it needs to be good. And good is a measure that's on the final delivery side. Good is a quality process that we can inject along the way, either by humans or by AI. " ], [ 860, "Limited Edition Jonathan", "Your articles don't need to come with instructions, i feel like that perpetuates the stigma- more than it helps ease the fears of your audience. That's why I'm so vehemently opposed to revealing, as a standard- what your process looks like. Because it's justifying, the worst fears. It's not helping those. If you're gonna use an editor, if you're gonna use any kind of editorial filter, between you and your audience, the only reason you should disclose it is if you believe there's something wrong with it and you're mitigating those problems in some sort of way I just think it's a bad idea " ], [ 895, "Sam Rogers", "And that's exactly why I wanted to have you on today, Jonathan. I really appreciate you, keeping your commitment to be on this show as best you can, thanks so much for showing up. I, I really appreciate it, Stigma, I think, is a, is an important signal to talk about, of how it is that we interpret that label of something being AI-generated. And is that something to add or is it something to subtract? For everyone here on Substack, you should know that you do have the option of turning this feature off, at least at this point, and, that's what Jonathan has done for his article. For mine, I have not done that yet, because I have a slightly different position on it all. I very purposefully was leaving my little star in the lower right-hand corner. For those who don't know what it means, that means Gemini generated this image. I label my works as being AI-assisted when they are. I don't publish anything that's completely gener- AI-generated. But I actually detail at the bottom of this post, about what it is that I did to make this article for those who are interested in the point that Jonathan was just leaving off on, which is, like, how the thing got to be made. That can be relevant for certain things like, if we're talking about food, I use this apple analogy, throughout this piece. Then you might wanna know that it comes from a local farm, or you might wanna know that it was grown without pesticides. You might wanna know things like that, not because it changes your sense of taste necessarily of how the apple tastes, but it does change what the apple is inside by that construction, and that might be important to you. But it doesn't tell you if it's a good apple. And the measures of good that we can create are far more nuanced and important, I think, than just slapping a label on it, which can have costs such as the stigma that Jonathan was talking about. and there's two different, approaches to that. One is to, not include the information that would make that labeling possible, and the other is to make it so ubiquitous that the stigma isn't there any longer. And that costs a lot in the short term, but in the longer term game, I think for myself personally, that's something that I'm willing to invest in, because that's what I am aiming towards. For yourself, yeah, don't change a thing. Like, the way you're doing it is awesome Everything makes sense in that world with that position, and I totally understand what you're saying. " ], [ 1062, "Limited Edition Jonathan", "Yeah, no, and I, and I can see, I, I can see your argument in the long term as well. Just steel man it. Revealing that you're using AI to help you with your writing is something that can normalize it over the long term. And that could be a positive thing, is just, if it's a human editor that's filtering and helping with your writing or a ghostwriter, by default, we never, disclose that. Yeah. Uh, so I think it's counterintuitive to decide that, we need a special plead for AI, disclosure. " ], [ 1089, "Sam Rogers", "Yep. Yep. I, I totally understand it is really an issue, and it's something that we can have reasonable differences about, and have evolving opinions about, and evolving tools around. And getting back to the Substack and Pangram ones specifically, the fact that something is being labeled, is not in a vacuum. To give some additional context that fell out of the recording here, Chris Best's piece is called Against Claude Fishing. His definition is deception, text passed off as human when it's not. He calls it a con, and Freddie deBoer, who he's borrowing from, calls it a con too. Fine. I think most people would probably agree with that. Here's the reversal. On that definition, the target is the faker. What actually shipped scans everybody and flags assistance of any kind at one hundred words or more. So it lands hardest on the honest, disclosed, assisted writing. On the exact cases his own definition was never really about, Best has a line in that piece, \"When I want Claude's opinion, I'll ask Claude.\" And Jonathan's answer to that is that the button is basically the machine's opinion about whether the writer is worth reading. And one more thing before I go further, and I want to credit it to Jonathan because it's his rule and I'm about to lean on it. He says he tries not to assume motivation. Believe what people tell you, infer from what they reveal, and don't try to read anybody's minds. It's a good rule. I'm not going to guess at anyone's intent or go down a conspiratorial rabbit hole, but I am going to point at the structure here because it extends well beyond this particular platform. Now, Jonathan, both you and I have our own companies, plural. We run things our own way, and we've set things up in a structure that works well for us. Chris Best has made a VC-backed play that results in VC-backed results, right? Like, we know what those look like. We've seen it happen before. And in that context, decisions like this aren't, out of nowhere. That's all I'm trying to say well, thanks again for, continuing to show up even in extenuating circumstances. With regard to the subtractions here, I think we're both arguing toward, from different angles, subtracting the authorship, labels meaning what they mean in society today, and taking different approaches for how it is that that can work, what it, what costs we're willing to bear around that. And advocating for people thinking for themselves and choosing for themselves where they wanna be on this scale, and getting involved in the discussion around the platforms that we're on, what we choose to disclose, what we don't, and being aware that it's evolving and that there's history here. Yeah, absolutely. That this isn't exactly a new problem. This is a scaled problem. This is something that we can make decisions around. And I encourage everyone to find the settings in your Substack. And you can do that on a per post basis. Yeah, for sure. You can do that on an account basis, at least last I checked. " ], [ 1306, "Limited Edition Jonathan", "And listen, last thing before we wrap it up, i'll commit to coming back on here 'cause, Sam and I have been talking for a while, and Sam's awesome. So definitely subscribe to him. But, my last word before I get outta here is, I want people engaging with my ideas. I don't want people engaging with my process " ], [ 1322, "Sam Rogers", "That sums it up beautifully. Yes to that. And thank you for the plug. Really appreciate you showing up, buddy. Many thanks to Limited Edition Jonathan. He did that from the road on the way to his kid, and he committed to coming back. I plan to hold him to it. He's one of the most experienced and informed bleeding-edge builders that I know, and we didn't get a chance to talk at all about that, which is an unexpected and inspiring path. For more from him, check out his substack at Limited Edition Jonathan. And to make the subtraction for this week explicit, subtract the AI-assisted label everywhere, full stop. Not just substack's machine-applied one, voluntary self-disclosure too, because disclosing is what teaches your audience that there was ever something to disclose. Mine is retire authorship as a quality claim in both directions. Human-written is not a quality claim. Machine-flagged isn't either. Both of those are proxies. They were always lazy ones. They're the ones that we reach for because they're easy, because they're cheap. Now, we can name what good actually means now and measure that directly in ways that we could never even consider before. So let's measure that, not who typed what. That's not the same subtraction twice. Jonathan says, \"Remove the signal.\" I keep the signal and strip the quality meaning out of it That's the reasonable disagreement in this episode. I disclose, and I plan to keep disclosing. Jonathan's position is that disclosing is the very thing that keeps stigma alive. I don't think either one of us talked the other one out of it, but it's good to revisit and revise our positions as things continue to change at ever increasingly inhuman speeds. I'd love to hear your thoughts. Hit us up at sigsub.substack, or subscribe at sigsub.show. Thanks, and I'll see you again next week " ] ] }, { "episode": 6, "title": "AI Regulations and the Chatbot Laws Already in Force", "url": "https://sigsub.show/episodes/ep-006/", "transcript": "https://sigsub.show/episodes/ep-006/transcript/", "video": "https://youtu.be/NhKS1C55bFw", "anchors": [ "t-0", "t-19", "t-82", "t-83", "t-109", "t-110", "t-116", "t-119", "t-131", "t-161", "t-162", "t-199", "t-228", "t-229", "t-229-2", "t-235", "t-235-2", "t-264", "t-273", "t-285", "t-304", "t-304-2", "t-308", "t-457", "t-463", "t-467", "t-482", "t-483", "t-489", "t-657", "t-660", "t-661", "t-686", "t-687", "t-733", "t-751", "t-757", "t-774", "t-775", "t-797", "t-797-2", "t-814", "t-815", "t-827", "t-828", "t-839", "t-841", "t-859", "t-863", "t-865", "t-882", "t-884", "t-1004", "t-1122", "t-1145", "t-1173", "t-1174", "t-1182", "t-1189", "t-1350", "t-1351", "t-1359", "t-1360", "t-1427", "t-1432", "t-1434", "t-1434-2", "t-1490", "t-1491", "t-1564", "t-1604", "t-1710", "t-1712", "t-1743", "t-1803", "t-1807", "t-1808", "t-1815", "t-1846", "t-1880", "t-1892", "t-1896", "t-1898" ], "turns": [ [ 0, "Michael Simon", "We're really into a minefield here. It's a minefield without a map. one of the things I tell clients constantly is, you do not want to be a test case. " ], [ 19, "Sam Rogers", "Welcome to Signals and Subtractions. I'm your host, Sam Rogers. On Sunday, Article 50 of the EU AI Act went into effect, and with it, California's new AI transparency law. Most of the attention went to these big headlines. But what didn't is probably what matters to you. There are already chatbot laws in force in nine states, and more AI regulations landing with Every legislative session, so the rules are changing all the time. Maine just six days ago. Colorado more happens a week from now. There's been lots of flip-flopping European schedule slippage and suits in many states. So it makes you wish you knew someone with 30 years experience watching organizations And what they do when a rule meets a system that nobody's documented yet. And lucky for us, our guest today is Michael Simon. He practiced as a trial attorney for seven years, I believe it was, " ], [ 82, "Michael Simon", "Almost seven, yes. " ], [ 83, "Sam Rogers", "and left to found a legal technology startup, spent more than a decade building privacy and e-discovery products, went back to practicing law, now advises companies on data privacy. And AI governance. He co-chairs the ABA Business Law Subcommittee on Consumer Privacy and Data Analytics. He's the perfect person to talk to about AI legislation. Mike, I'm so glad you're here. " ], [ 109, "Michael Simon", "And I'm glad to be here, Sam. " ], [ 110, "Sam Rogers", "So you were writing papers about AI like way back in the the before times of twenty eighteen. Yeah. " ], [ 116, "Michael Simon", "Well before it was cool, man. " ], [ 119, "Sam Rogers", "saying that AI automation was already shaping what counts as legal work. And eight years later, a few things have changed. what would you say still surprises you now? " ], [ 131, "Michael Simon", "Wow, that's a good question. two things. The suddenness of the arrival of the era we're in. most people I talk to, when you talk about AI, they're like, Yeah, Chat GPT is so cool. I love this AI. The phrase AI was invented in like 1957. Hey everybody, let's take a guess as to when the first chatbot came about. Nobody gets 1967 with the program Eliza, the first chatbot. " ], [ 161, "Sam Rogers", "Eliza chat, yeah. " ], [ 162, "Michael Simon", "the degree to which we went from really cool, difficult to use. by serious nerds and occasional quasi-nerds like me, to things that everybody can access, everybody can vibe code, everybody can do things with. wow. that's fast second thing at least in legal the absolute lack of us doing anything about it in terms of impacting the legal profession we are slowly getting to it in the laws and that's by the way one of the reasons I love what you have available which is the list of those laws " ], [ 199, "Sam Rogers", "Yeah, thanks. I I made that because I needed it. what Mike's referencing is everyailaw dot com. And that has been my goal is to capture all of the AI regulations in one spot just so as someone running a business, I can figure out what my obligations are. I wish I didn't have to make that. I I've talked to you of course, Mike. I've looked for other like things in the world, but apparently things like that aren't generally open and free, so I'm I'm hoping to " ], [ 228, "Michael Simon", "No. " ], [ 229, "Sam Rogers", "change that. " ], [ 229, "Michael Simon", "There's a few around some of the big law firms have trackers, but they tend to track way too much, and " ], [ 235, "Sam Rogers", "Yeah. " ], [ 235, "Michael Simon", "they tend to track it in ways of like, hey, let's play a game here. Let me click on Washington. Cool, done with that. Let me click on New Jersey. Go fish. Do you have any Connecticuts? You know, it's it's a simple list that breaks everything down, is a godsend, and I've done it with privacy laws. I would invent one for AI, but you've already done it, so thank Thank you. " ], [ 264, "Sam Rogers", "Well, yeah, thanks for the thanks for the plug. although Mike is a lawyer, he is not your lawyer, so whatever disclaimers you want to make about that, Mike, feel free. " ], [ 273, "Michael Simon", "all right, disclaimers. I am a lawyer. This is not legal advice. This does not form an attorney-client relationship. if you have a problem you'd like to get assistance with, I think we'll do all that contact info at the end. " ], [ 285, "Sam Rogers", "So the EU AI Act we don't need to get too deep in. I just wanted to make sure that people are aware, as the context for this conversation, All this did just change on Sunday. Yes, that's a real thing. and Mike, if you could just give like like one minute of overview on what " ], [ 304, "Michael Simon", "All right. " ], [ 304, "Sam Rogers", "is different now and if it affects people, how? " ], [ 308, "Michael Simon", "absolutely and absolutely also not. So the words to focus on here is annex three: high risk. If your AI system is listed in Annex 3, then there's a lot of things that apply to you. Disclosures, monitoring, testing, certifications, unless you're doing something like large-scale mass public monitoring. yeah, I don't I don't do that stuff. I'll just tell you to stop. Otherwise you're not at at At a particular risk that's going to have a lot of requirements. The second thing is to note December 2027. That's one of the two reasons why we're not going to pay much attention to this after this part of the segment. The second reason is that it's a lesson learned from GDPR by everybody. GDPR is the EU's data privacy protection law. It became effective in 2018, and a lot of people, perhaps maybe even me, jumped up and down and And said, everybody look out. we did the the Kermit the Frog thing. you know, it's coming for all of ya. Look out, and it was a resounding thud. Just just look, the EU under GDPR has indeed gone after American companies. It's gone after Meta? Amazon? Google? Do see any pattern there? If if you are not a multi-billion dollar business trading huge amounts of data back and forth and acting within the EU, GDPR is not really a major issue to you unless you are very deliberately targeting the EU. I have some clients who do that. We get them set up the right way, but by and large, it is not a major issue for American companies. And yes, I know I'm not supposed to say that because it drives away business of people calling me, my God, GDPR. There are all kinds of data privacy laws in America that will impact your company. At the same time. The EU AI Act. Again, we're not even into enforcement yet. We've got more than a year, but even when it shows up, will they go after the big players? Probably. Will they go after small US companies? it's not something that needs to be the top of your risk. rankings, what needs to be at the top of your risk rankings, or all these other things that we are now going to talk about. And again, the list, but but I don't want to embarrass Sam too much by shilling that too much. Was that a minute? That was more than a minute. well, close. Close. We'll call it a " ], [ 457, "Sam Rogers", "That that that was a little more than a minute, but it was very appropriate and and very thorough. " ], [ 463, "Michael Simon", "lawyer's minute. I I I'll just bill you for one. How's that sound? " ], [ 467, "Sam Rogers", "Why thank you. So gracious of you. I do have another slide that I'm labeling federal framework-ish, because as we're getting into the American side of things, there are some things that have happened that are kind of pinging back and forth pretty hard. you could just set that context. " ], [ 482, "Michael Simon", "I took it. " ], [ 483, "Sam Rogers", "We don't need to stay at this level either, but from the EU to the federal level of the US. " ], [ 489, "Michael Simon", "Okay. I will start with the overarching statement. There is no federal regulation of AI. Period. None of what's on that slide changes that. We have a lot of stuff going back and forth. We have claims for this, then for that, and it changes. And even if you're gonna stick on one of them, okay, there's an executive order claiming preemption. Do you know how much impact an executive order has upon actual preemption of laws? You cannot preempt state laws without a federal law that preempts them. Period. You can do all kinds of things to potentially Cause state legislators or governors to want to play along and preempt themselves and do things that will degrade their own laws. We've seen that in Texas and we've seen that in Colorado, and we've seen actually full court presses in Utah that worked. But we're also seeing what is an absolute mad world of laws I'm gonna go through some of them It's all over the place. One thing that that Sam and I talked about before this is I co-write along with my friend, Canadian AI expert Andrew Perry, the yearly ABA annual survey on AI law and regulation. And we set it to start with a theme song. I write I write articles with theme songs. The theme song for this year's annual survey is Mad World, and it is a mad world. We are seeing. a federal administration that cannot get its act together that is at war with itself we are seeing states that are all over the place and what happens is that it creates a dangerous regulatory environment take a step back privacy in the privacy world, we had a build-out for about the last five, six years of state privacy laws one by one. We're now up to twenty-one laws and other countings, twenty-two, twenty-three, twenty-four, which by the way should show you there's a problem. We can't even agree on how many laws we have. But we have, let's say, 24 state general privacy laws that cover from A to Z in privacy. Some have added more things, but there's a basic set of a template there. And that template, there are commonalities, and I have written and I have spoken about how 85% of it is the same stuff and you can comply with it. And then you pick certain priorities like California, which is the biggest economy and regulator by far of privacy. Or you look for certain outliers and how to handle those. And so it's it's dealable with. It's it may be a patchwork, but it's a patchwork that uses the same colors and the same tones on the same quilt. We're really into a minefield here. It's a minefield without a map. And so we'll go through some of these in what other order you want. " ], [ 657, "Sam Rogers", "we were just starting with the the signal part and " ], [ 660, "Michael Simon", "Okay. " ], [ 661, "Sam Rogers", "and let me let me contribute mine as well as someone who's building that every AI law tool, what's upstream of that is the ontology of law and making that work at agenc speeds, which is there's a whole legal graph layer that I've been working on. And just in the last week, I've been Surprised just how not knowable so many things are until you really break them down mechanically. Like " ], [ 686, "Michael Simon", "Yeah. " ], [ 687, "Sam Rogers", "for for working with that ontology clear example this week basically there were like 63 different legal instruments that had never been made to work together from all these different j jurisdictions. They're inconsistent, but they're inconsistent in a way that like makes it your problem. they're not going to clean it up and make the the pieces fit together. So someone breaking down all of the ontology of it to make it machine readable. I was just stunned at that signal because I've been working on this product for six months now, and I just didn't expect the layer of crazy that I encountered just in the last week as I started tightening some things up. " ], [ 733, "Michael Simon", "Alright, do me a favor and we're done with this. Send me the recording. I want to edit your part, what you just said down. And the next time somebody goes, Why do we need all you damn lawyers anyway? That was an awesome argument for why you need us all. Guess what? It's gonna get worse. Way, way worse. " ], [ 751, "Sam Rogers", "So I know you've got a couple things that you're waiting to surprise me with, Mike. " ], [ 757, "Michael Simon", "You know, I was thinking about what have I subtracted? And here's an odd thing for somebody who advises company on AI who works with AI companies and works with AI all the time. I have been a big Claude fanboy since it was just simply Claude. No numbers, no fantasy " ], [ 774, "Sam Rogers", "Yeah. Yeah. " ], [ 775, "Michael Simon", "names, just Claude. here's what I've come about doing in the last just few months. I have used it to help me write a lot of things, and I have realized I need to have it help me write a lot less. it can help me when I'm stuck. It can help me sometimes with an outline or work on ideas, someone to throw things at, you know, someone. funny how we always anthromorphize it. " ], [ 797, "Sam Rogers", "Yeah. " ], [ 797, "Michael Simon", "But I have now realized it's actually the the number one tell of all the LinkedIn posts on LinkedIn that are written with AI, in addition to the fact, you know, it's a LinkedIn post on LinkedIn at these days, is not the m-dashes. Most people have gotten wise to that. " ], [ 814, "Sam Rogers", "Okay. " ], [ 815, "Michael Simon", "it's not the negative parallelisms. it is actually that it's written like Captain Kirk, William Shatner, talked in in the original Star Trek. It's " ], [ 827, "Sam Rogers", "Ha ha. " ], [ 828, "Michael Simon", "it's over and over again. You get this. Yeah. This new law is important. It's not just a law. It's how you must do business. It's how you do it's like Spock, do something, you know, kind of writing. " ], [ 839, "Sam Rogers", "Yeah. " ], [ 841, "Michael Simon", "my god, it file it's just like, look, I I'll write that way and I'll use like maybe one or two short dramatic sentences in something. I I do that when I was writing, you know, briefs and things. You can't write an entire article in in short dramatic sentences. It just doesn't work. " ], [ 859, "Sam Rogers", "that's a great subtraction. I'd love to be able to add my own and then go to if you have another " ], [ 863, "Michael Simon", "Go for it. Then we'll get to the laws. " ], [ 865, "Sam Rogers", "I wanted to make sure that people know that sensitive materials don't go into LLMs and just what the alternatives are. I wanted to use the example of some legal material that Mike and I were sharing before this episode. " ], [ 882, "Michael Simon", "no, did you put it all in the clod? my god, no. okay. Yeah me panic. " ], [ 884, "Sam Rogers", "I did not. I did not. because I I suspected that it might be sensitive. I didn't want to upload it to the cloud version of AI, but I did use a local version of AI. In this case, I used Gemma 4, to be able to run locally on my computer, something that's not connected to the internet, to be able to kind of strip out anything that might be sensitive in this material. And stay with the concepts. So it was refining what was, I don't know, eight pages of content down to about two and removing anything that might possibly be sensitive while retaining the concepts that I needed to do my prep for this episode. So using a local model, you can redact essentially the identifying parts, or of course, you can always move at human speed. And you can actually read the entire thing. I did read most of it as well, Mike, but mostly just to check the AI and make sure that I wasn't dropping anything that that was critical. so I scanned it, but the scan that was done with AI, I then could to take my local version and bring that to Claude and help build out the prep for this episode. So nothing was shared that shouldn't be shared, and that subtraction element. is pretty critical and I don't know that everybody knows just how easy that is to do now. A year ago it wasn't so easy. It is now very easy and a great default for if you're not sure being able to use local LLMs on your device that don't take up a ton of room and that can they're not as fast as typing into ChatGPT. You might give it 10 minutes to run, you might hear the fans come on. But like for a task like this. I don't need it to be instant. I just need it to be effective and most importantly, not to share anything that I shouldn't share. " ], [ 1004, "Michael Simon", "That's cool. So first you did this great pitch for why people need lawyers, and now you've hit with the reality of why lawyers do too much and overdo things. Awesome. Both are very valid, by the way. I think your ordinary non nerdy person is not likely to install a local LL You can do it. I'll be honest, I haven't. it's one of the reasons I I joined up to help invent an AI that would preserve that confidentiality and not let it leak out. but I recognize that that is One way to solve what is a very clear problem. I will tell you, one of the things I've written about multiple times on LinkedIn is the leaks. OpenAI ChatGPT has had a bad history of publishing things, making available on the internet things that other people did not want to see, and just last week It was revealed that Claude, Anthropics Claude, did the same thing or a similar thing. We don't know if it's the same technical problem. And when I started digging into it, I discovered that in fact this was not something that happened last week. It started happening back in September last year. And they said they fixed it. And the way they said they fixed it, Wired called them out immediately. on that saying this is not going to fix it. And it didn't. So you know it is a concern. if you're doing something and you don't want that shared. I know these things seem like they're trying to convince you they're your friend. They're your they're your they're your therapist. dear God, don't don't believe that, please. It's not your therapist. That was the Eliza effect back in 1967 that caused, by the way, when we mentioned that Less than 10 years later, the guy who invented Eliza, the first chatbot Joseph Weisenbaum, was writing a book saying, this is bad, I never should have done this, and humanity should never do this again. Do not let it convince you that you should just share everything and it's just gonna be fine. " ], [ 1122, "Sam Rogers", "Yeah, yeah, that's that's a really important message, I think. as we're moving fast, what are the things we're breaking along the way? some things you can't get back, and privacy's one of them. it's one thing for our own privacy, we're making that choice. may not be an informed choice, but at least it is a choice. But especially when you're dealing with other people's information, now there's a liability element. " ], [ 1145, "Michael Simon", "If you contact me, I have to treat your confidential information. as sancrosanct. I can't reveal it. I have to take in fact anything I do, I have to take reasonable steps to protect it. And a reasonable step is not typing it into a system that says it could train on that data, that it it retains that data for who knows how long? A year, a decade, the heat death of the universe, who knows? " ], [ 1173, "Sam Rogers", "Some somewhere in in there, yeah. " ], [ 1174, "Michael Simon", "Yeah, somewhere in between. Likely. let's get to the laws. look, all this has just been a run-up. Let's really start scaring people. " ], [ 1182, "Sam Rogers", "Mike, I'd love to hear what are some of the things that jump out to you from this list. " ], [ 1189, "Michael Simon", "it's easy to go, but that doesn't affect me. Well, it does. Look, if if you Have a website. If you are offering products or services to anybody in the US, congrats. you're impacted by all of these. there may be additional jurisdictional hooks like minimum revenues or minimum impact upon the state, but that tends to be more of a privacy law thing. In these laws that are getting passed for AI, it's as long as you impact consumers, and that's real, real, real hard not to do in a state. Because the internet goes everywhere. yeah, you're impacting consumers everywhere. So let's go to the chatbots. Let's start with some of the big states, states that you cannot ignore. New York. You must take reasonable measures to prevent content that promotes self-harm. If you do not do that, the New York Attorney General is authorized to fine you up to fifteen thousand dollars per incident. And since an incident means perhaps every time you access the thing, that could be bad. That could be a lot of money. And the New York Attorney General's office is big. So is California. They then take that New York law and they have added that you have to set that chatbot to prevent. the revealing to minors of sexually explicit materials, an additional one. You also then have to do an annual disclosure of those things you do to prevent self-harm to a California agency designed to prevent suicide. So now you've got two additional things. And California, not only are they by far the biggest attorney general, at least in privacy world, they are the ones everyone's afraid of. Hey, it's got a private right of action. So at some point, the plaintiff's class action lawyers, who I have worked with at times as an expert, they are smart, they are creative. Creative, they'll find a way because that's a lot of money they can make. So now we get to some other states. So Washington also created a law that kind of looks like the other ones, except they added something. Your chat bot must be designed to prevent it from simulating distress. For you know, I I need to go now. No, don't go. That breaks the law. you know, you know what? You'll need to purchase this or pay additional tokens to keep going, my friend. That breaks the law. by the way, saying, don't don't tell your parents. That breaks the law. It's also got a private right of action. we're not done yet. Oregon! I got a list. All here. " ], [ 1350, "Sam Rogers", "But wait, there's more. " ], [ 1351, "Michael Simon", "There's more. Well wait, there's more. The operators must be standing by and those must be your free gift to keep. So, Oregon, " ], [ 1359, "Sam Rogers", "Okay, Oregon. " ], [ 1360, "Michael Simon", "your free gift to keep from Oregon. Not only do you have to have much of the above, those suicide prevention and self-harm measures, you get to publicly disclose them, not just to some small obscure agency in California. You gotta have them online for everybody to see. And an additional bonus. wait, there's two actually. The first additional bonus. If someone types in something that looks like they are contemplating suicide you must immediately interrupt them. The chatbot has to have a setting to stop everything else. and immediately refer them out and implement those provisions. And if you don't, second bonus. It's the third act with a private right of action. There's a bunch of other ones, but there's only two more we're gonna do because the rest of them They don't add too much. And and by the way, there's more being passed. Hawaii has more requirements. It's not been signed yet, so I haven't even covered it. And I won't I will when it does. Why? Because we've got a whole bunch of Scary things adding up without it so far. Connecticut. " ], [ 1427, "Sam Rogers", "So for those keeping up for those keeping up at home, we've got New York, California, Washington, Oregon. But wait, there's " ], [ 1432, "Michael Simon", "California, Washington, Oregon. " ], [ 1434, "Sam Rogers", "more. " ], [ 1434, "Michael Simon", "But wait, there's more. We have Connecticut. you're never allowed to create an inference or an offer that it is a mental health professional. Which, by the way, we'll get to, because there's a whole slew of laws just like that. And then finally, a surprising biggie, Iowa. You must take reasonable measures. prevent the creation of sexually explicit materials, not just reference them. Nobody can create them. claims that it is human. Not allowed to claim that this stuff is human. You cannot create a dependency upon the chatbot. What is that? I'm not real sure. But it's a big term. And then finally you must provide a whole panoply of privacy tools for minors and parents and other users. This is big. onto healthcare, and then I'll stop, because probably at this point people have all fled the room with their hair on fire. If you're in " ], [ 1490, "Sam Rogers", "Yeah. " ], [ 1491, "Michael Simon", "healthcare, we've got a whole bunch of other laws. We've got Illinois that prevents from replacing professional assistants with bots. And if you want to use it even for administrative, many administrative tasks, you have to get consent. Texas limits what you do to the low scope of your license, and you need to have human review of everything. Nevada, you cannot design a system to replace human help. California, Tennessee, Colorado, all relate Claims that this stuff works better than people. by the way, Tennessee again, private right of action. I don't know any class action lawyers in Chattanooga, but I'm sure there are some. And this this will this will be a gift to them. Yeah, that's just a few of these laws. They add up. And the big problem with this minefield, the big problem with something that's way beyond a patchwork, is that if you are doing this kind of work, if you are offering this up, not just building these things, using these things. If this is on your website, you need to know about these laws. What are you offering? What are you doing? There are exceptions, there are jurisdictional limits, there's a lot of more stuff in there. what was that, five minutes, ten minutes? I don't even know how long I'm talking. That was just the quick summary. That was the chat GPT summary, except I did it. " ], [ 1564, "Sam Rogers", "I I think Yeah, I I think it's a I think it's it's enough minutes to get the point across, which is that just because you're contracting with a vendor doesn't mean that all the liability goes with them. If something is appearing on your website for your company, there is a layer of liability to examine carefully to make sure that it's lining up with all of the many different ever-changing laws that are not designed to work together. Some of them are actually the opposite of each other we're starting to see as well. So " ], [ 1604, "Michael Simon", "And add there's there's there's almost there's almost a contest to who can add more things. look at this law, let's add one more. We we've seen that. We've seen it in privacy, but with AI it's moving so fast. And I do want to add one more thing to what you said, Sam, which is we are also at very early stages of this stuff. It feels so cool and so well done. It is not. We are so early and so a lot of the governance and controls are very light. Those are the big scary headlines. I'm gonna talk about something scarier because that should be the theme. I have talked to some tech companies, one particular tech company had really cool model. They have internal controls that they will help you build tech technically for chatbots. And when I talked with the CEO, he gave me a new hobby. Probably shouldn't have gotten in this hobby, but it's a really cool new one. You go on to any help bot on any site and treat it just like you would chat GPT. Ask it to code things for you. Ask it to do searches, and it will more times than not you can just treat it as a free instance of ChatGPT or Claude because they're set to do anything. as a company who is deploying these things, you need to recognize that there are laws that put you, your company on the hook for what that chatbot does or doesn't do. Not just on the chatbot laws. God forbid somebody goes into your chatbot and starts talking about things that are ideations of suicide. That seems unlikely. It seems more unlikely that they may try and fool it to give them an answer that is advantageous to them. And then they can go like that guy who create who went went to a dealership's chat bot and convinced it to offer them a Chevy Suburban for 20 bucks or something. " ], [ 1710, "Sam Rogers", "Yeah. Yeah. " ], [ 1712, "Michael Simon", "when the laws of who's on the hook for that are still being worked out. you know, one of the things I tell clients constantly is, you do not want to be a test case. You do not want me permanently living in your company trying to do this stuff. You do not want to be involved in the litigation that works this out. Because when you see those cases, that's people spending millions of dollars just to try and get back to where they were before. It's a bad thing. " ], [ 1743, "Sam Rogers", "Yeah, and that brings up a a good point to bring things to a close here, that as the technology moves at exponential speeds, the speed of litigation has not improved. So there's a there's a tension there between what is legal and permissible and what is possible. And the possibilities are going to be even more next week than they were in the last week. But the framework around how we understand that, how we communicate about it, how we need to tweak those tools to to thread the needle on what makes AI safe, what makes AI legal. It is constantly changing okay, I get there's a lot, but like what do I do about it? I just want to know the right thing to do, right? There there isn't yet a way to really even say what the right action would be for all circumstances for anyone listening to this podcast. And just to anchor it back to why we have lawyers and why people like " ], [ 1803, "Michael Simon", "Yeah, you wouldn't need me. If we had that, if you could do that in this podcast in an hour, wow. " ], [ 1807, "Sam Rogers", "Yeah. " ], [ 1808, "Michael Simon", "you'd be putting all of us out of work. And let me tell you, the public would be so mad about that. We know. They would hate you. " ], [ 1815, "Sam Rogers", "Well, it's really important that you have someone that you can talk to, that you know what that contact is. You can't just trust AI to make these judgment calls for you. It is actually a human endeavor to be able to interpret all of this wide-ranging context that even with the expanding context windows in AI, it can't keep track of like a good lawyer can. so in closing Mike I was wondering if you could just give your thirty seconds of where people find ya, how people get in contact " ], [ 1846, "Michael Simon", "Sure. the way to find me is I have a law firm. It's my own law firm. It's called Law Plus Data. You can find me on LinkedIn. there's a billion Michael Simons. But look for the one with Law Plus Data. That one, that that was only one of me there. the email, is michael.simon at lawplusdata. Again, the plus spelled out dot com. I am a frequent speaker. I am a frequent author and I will probably bug you to sign up for my newsletter people tell me the stuff I write is at least interesting, at least entertaining. I try not to be boring, ever. " ], [ 1880, "Sam Rogers", "Yes, I I can vouch for that. Mike's newsletter always has like that theme song thing he talked about earlier. Like it comes together well, usually with some some fun artwork and musical references. So it is not boring. " ], [ 1892, "Michael Simon", "Of course. I thank you. I take that as the highest compliment. " ], [ 1896, "Sam Rogers", "Thanks so much for joining. " ], [ 1898, "Michael Simon", "Thank you all. " ] ] }, { "episode": 7, "title": "Measuring Anything, Before the LLMs", "url": "https://sigsub.show/episodes/ep-007/", "transcript": "https://sigsub.show/episodes/ep-007/transcript/", "video": "https://youtu.be/BlFREAcz5BU", "anchors": [ "t-0", "t-4", "t-4-2", "t-18", "t-28", "t-190", "t-194", "t-195", "t-203", "t-218", "t-241", "t-247", "t-248", "t-257", "t-320", "t-320-2", "t-322", "t-328", "t-451", "t-474", "t-483", "t-483-2", "t-495", "t-496", "t-518", "t-531", "t-534", "t-544", "t-558", "t-558-2", "t-567", "t-567-2", "t-572", "t-573", "t-603", "t-603-2", "t-606", "t-606-2", "t-625", "t-628", "t-629", "t-640", "t-654", "t-655", "t-662", "t-662-2", "t-705", "t-705-2", "t-725", "t-729", "t-742", "t-742-2", "t-793", "t-793-2", "t-799", "t-800", "t-815", "t-815-2", "t-832", "t-844", "t-858", "t-859", "t-874", "t-874-2", "t-1026", "t-1027", "t-1072", "t-1075", "t-1075-2", "t-1079", "t-1097", "t-1098", "t-1109", "t-1110", "t-1114", "t-1114-2", "t-1114-3", "t-1115", "t-1136", "t-1137", "t-1199", "t-1208", "t-1220", "t-1222", "t-1255", "t-1256", "t-1326", "t-1328", "t-1376", "t-1384", "t-1388", "t-1389", "t-1468", "t-1468-2", "t-1479", "t-1480", "t-1484", "t-1489", "t-1496", "t-1497", "t-1512", "t-1513", "t-1579", "t-1586", "t-1589", "t-1592", "t-1594", "t-1804" ], "turns": [ [ 0, "Douglas Hubbard", "You've heard somebody say a phrase like statistically significant sample size, right? " ], [ 4, "Sam Rogers", "Yeah. " ], [ 4, "Douglas Hubbard", "There's no such thing. There's no such thing as a sample size that's a minimum threshold applicable in all situations where one short of that you know nothing, and then at that, all of a sudden you can make an inference. There is no such thing. " ], [ 18, "Voiceover", "Signals and subtractions. What to watch, what to drop, every week. " ], [ 28, "Sam Rogers", "Welcome to Signals and Subtractions i'm your host, Sam Rogers. That was Douglas Hubbard And this week's episode is gonna be a little different than our usual format. I'm out this week on an urgent family matter and had to cancel our weekly live stream. But rather than skip a week of episodes, I went into my own podcast archive, and what I pulled out predates LLMs and modern AI and gets at the foundations of how to reason about the data that they need. Who is Douglas Hubbard? He's the inventor of applied information economics and author of How to Measure Anything: Finding the Value of Intangibles in Business, which is the book people hand you when you tell them something can't be measured. He has measured drought resistance in the Horn of Africa, the effect of pesticide regulation on endangered species, and the economic impact of restoring a desert in Inner Mongolia. I interviewed him back in twenty sixteen for my podcast, Doable Change, years before ChatGPT. No copilots, no AI budget anybody had to defend to a board. Seems delightfully quaint from here, doesn't it? Now, this isn't me telling you how prescient it all was. Doug wasn't predicting anything. He was describing how measurement works and how it's always worked since the dawn of science, which is why nothing in it needed updating. Kind of a strange thing to say about a ten-year-old recording in this area, right? So here's what I want you listening for. Right at the end, Doug says there are only three reasons why anybody ever believed something is immeasurable. He calls them illusions and describes all of them in about ninety seconds. I'm convinced that those same three things are exactly how nearly every AI business case that I see falls apart. Now, I'm gonna make that case when he's done, but see if you can spot them here first. They'll stick better. The first thing you're gonna hear is Doug asking me how I would measure collaboration, and I give the standard activity-based answer. Ten years later, that's still the answer most people give, but it's still wrong. And now it's on a dashboard with big letters A and I up at the top. Enjoy this episode with author, speaker, and statistician Douglas Hubbard of Hubbard Decision Research. " ], [ 190, "Douglas Hubbard", "You've heard people talk about collaboration before, improving collaboration, right? Of " ], [ 194, "Sam Rogers", "course, yes. " ], [ 195, "Douglas Hubbard", "Since I think we share that concept, you tell me what you think of when you see more team collaboration. " ], [ 203, "Sam Rogers", "I see people communicating and solving problems faster as a group than they would individually. Okay. I see that they're not, stepping on each other's toes, and that they can divide the work more appropriately. " ], [ 218, "Douglas Hubbard", "Okay. So let's think about each of those things in terms of observable consequences. So I think they all hinted at observations without necessarily getting into specific methods, right? But they all hinted at a kind of observation. Mm-hmm. So you said they communicate more. Are there observations we can make that would at least indirectly indicate or directly indicate frequency of communication? " ], [ 241, "Sam Rogers", "Um, sure. So the number of messages, like using email or something like that. " ], [ 247, "Douglas Hubbard", "Sure. Right " ], [ 248, "Sam Rogers", "although we're not necessarily looking to just increase the volume, so that there's more- That's right … messages, but actually that there's more getting done as a result of the messages. " ], [ 257, "Douglas Hubbard", "There you go. Right. Exactly. I'd say I think the other thing is sort of a secondary indicator of something more fundamental, and sometimes people get caught up on the first thing. They'll get into a quagmire of latching on to the very first thing they identify- Mm-hmm … as measurable, right? And then they forget that there's something even more fundamental, because there is something more fundamental. Mm-hmm. Does the quality and speed of the output of the group improve? Do I really care about communication per se? That's sort of an indirect indicator of something else I care about. It's probably a good bet that better communication leads to those other things, I would only measure communication frequency if I thought it had something to do, with the quality of outputs. Mm-hmm. Now, can we correlate, even just frequency of communication? I don't know what the answer to this might be, by the way. It would be an interesting research topic. Is there a relationship at all between frequency of communication within teams and the development cycle and how much money their developed products make? " ], [ 320, "Sam Rogers", "Hmm. " ], [ 320, "Douglas Hubbard", "I don't know. That's a good question. " ], [ 322, "Sam Rogers", "I was wondering if you could tell us a little bit about how you became interested in measuring change. " ], [ 328, "Douglas Hubbard", "Yeah, so the first job I had out of graduate school was with Coopers & Lybrand, and I was in their management consulting services. And I was the guy who tended to get involved more in quantitative analysis, I think because compared to everybody else on the team, I had a lot more math and statistics. I got to be involved in a lot of neat, kind of obscure methods. Other people had said, \"You'll never use that in the real world,\" and I was using it, you know? And every once in a while we'd work at a client where somebody said something was immeasurable, and I would generally take their word for it. I said, \"Well, you would know. I'm just starting out in the business world. What do I know?\" But sometimes they had said that after I knew we had just measured that very thing at another client. So I knew they weren't always right, and then I started to suspect that it was never right. Because every time somebody said it, I could find a fundamental misunderstanding behind it. That it was indeed measurable. Sure. We define measurement, and we think this is the de facto use of the term in really all of the empirical sciences. Measurement is uncertainty reduction based on observations expressed as a quantity. Okay? When you parse that definition, the reason why it's different from definitions that you often hear, the, the hidden presumption in that, it's an exact number, which it never is in the empirical sciences. You have a wide range of possible values. You make some observations, generally do some trivial math, and then your range is narrower than it was before. Not only are people often surprised at how much of an uncertainty reduction you can get from a given amount of data, but they're also, surprised at how valuable marginal uncertainty reductions can be. They have this idea that I need to reach some arbitrary threshold before the measurement is statistically valid. We're constantly correcting people. " ], [ 451, "Sam Rogers", "So for our listeners who could probably use a little help breaking that down into layman's terms, would it be fair to say, and correct me if I'm wrong, that any time you know more than you did before, you're using some kind of measurement to do that? That, that you're reducing what you don't know and increasing the range of what you do know? " ], [ 474, "Douglas Hubbard", "Yes. Right. In fact, the only caveat we'd put at that as the general presumption I think is fair is that you're expressing that uncertainty reduction quantitatively. " ], [ 483, "Sam Rogers", "Mm-hmm. " ], [ 483, "Douglas Hubbard", "But yes, what you said is correct. The term statistically significant, number one, probably doesn't mean what you think it means. When you figure out what it does mean, you realize that it's not even the question you were asking in the first place. " ], [ 495, "Sam Rogers", "Mm-hmm. " ], [ 496, "Douglas Hubbard", "And, what you really care about is what's the value that's been improved because I'm making a better bet than I was before? That's what I really care about. And so that's a different understanding than I think- Right people might learn in first semester statistics. Unfortunately, that's probably all they remember, and they remember it wrong. " ], [ 518, "Sam Rogers", "What would be a good personal example of something that, that people often say is immeasurable, and you might say, \"Well, yeah, you can think of that as immeasurable, but actually here's how you'd measure it\"? " ], [ 531, "Douglas Hubbard", "What's the effect of my commute on my health? " ], [ 534, "Sam Rogers", "Great. So if I've got my commute Uh-huh what would my next step be then in helping to evaluate the impact of commuting on health? Which I think is a great example. " ], [ 544, "Douglas Hubbard", "Okay. If somebody said, \"How could I possibly measure the effect that, these behaviors have on my, life or, well-being?\" or something like this. Well, first off, assume it's been measured before, because it probably has. " ], [ 558, "Sam Rogers", "Hmm. " ], [ 558, "Douglas Hubbard", "And if we're resourceful, and If we consider the possibility that you're not the first person on the planet to ask that question- " ], [ 567, "Sam Rogers", "Mm-hmm … " ], [ 567, "Douglas Hubbard", "right? And that somebody actually probably wrote a whole paper on it, a bunch of papers. " ], [ 572, "Sam Rogers", "Mm-hmm. " ], [ 573, "Douglas Hubbard", "A bunch of research projects that got a bunch of money to do that, okay? And it can be informative So often when we do research, we start with the idea that, \"You know what? Maybe I'm not the first person to ever ask this question. Let's see what research exists.\" And you know when you're working for a client or your employer or somebody, and you've done some research, and you can cite two or three, seminal articles about some obscure topic like that- " ], [ 603, "Sam Rogers", "Mm-hmm " ], [ 603, "Douglas Hubbard", "You know what you look like. You look smart. " ], [ 606, "Sam Rogers", "Right. " ], [ 606, "Douglas Hubbard", "You look like you did your homework, right? You generally have more data than you think, and you need less data than you think. So let's say you've measured the time you spent commuting. You're now interested in measuring the impact that commuting has on your health. Great. So why do you care? What would be the reason for someone to measure the impact on their health? " ], [ 625, "Sam Rogers", "Uh, potentially looking at relocating and, There " ], [ 628, "Douglas Hubbard", "you go … " ], [ 629, "Sam Rogers", "and trying to figure out if I can be closer so that I'm reducing that time, potentially looking at other, positions or transfers to see if I can work out of some place that's closer. " ], [ 640, "Douglas Hubbard", "Right. Yeah. Absolutely. Well, those are all good decisions, and as soon as you identify those decisions, you add another little facet of clarification to the measurement problem itself, okay? Are there decisions you're going to make differently? " ], [ 654, "Sam Rogers", "Mm-hmm. " ], [ 655, "Douglas Hubbard", "Now, once you start modeling those variables, you actually start figuring out, here's how much I really need to even know about this variable before it makes a difference. " ], [ 662, "Sam Rogers", "Mm-hmm. " ], [ 662, "Douglas Hubbard", "So, you have a current state of uncertainty about a variable, let's say like your, the health effects. Mm-hmm. And you're saying, how bad would the health effect have to be before you would think it would be worth your time to just change jobs, if that was the decision you're making? Right. Or relocate, right? Or spend more on an apartment that's closer to your employer, 'cause you probably have that option. If you said, \"Wait a second, let's do a back of the envelope calculation. How bad would the health effect have to be before I would make that kind of change?\" And then ask yourself, am I pretty confident it's a lot less than that, or could it possibly be over that threshold? " ], [ 705, "Sam Rogers", "Right. " ], [ 705, "Douglas Hubbard", "Because if there's a significant chance that you're over that threshold, then it looks like something you should measure further 'cause it might have bearing on your decision. So if you've got this big wide range, and this range is straddling a threshold for a critical decision where knowing which side of that fence you're on, well then it's, there's a value to measuring. " ], [ 725, "Sam Rogers", "So knowing what you're going to do differently dictates how then you would measure. " ], [ 729, "Douglas Hubbard", "That's right. Now, in the book, I briefly mention two other reasons that measurements have value- Mm-hmm … but that's not the focus of the book. I think that really helps focus and clarify the whole problem as soon as you ask the why do you care. " ], [ 742, "Sam Rogers", "Mm-hmm. " ], [ 742, "Douglas Hubbard", "So if somebody says, \"How do I measure the value of m- my happiness, with my, home?\" I go, \"Well, is there a reason you wanna measure that? Are you just in a benchmarking contest with your neighbor or something?\" Right. Is there a decision in your life you're actually gonna make differently? Now- Right To be honest, I don't measure all that stuff. Mm-hmm. A lot of that is trivial stuff that doesn't immediately affect decisions. But when my wife and I, my wife teaches math at a community college, when we make bigger life decisions, starting about, let's say a new car, Mm-hmm and bigger, like a house, we definitely start doing analytics at that point. So we do our homework on that stuff. Mm-hmm. Now, a lot of the rest of it is how to do the math with a few observations. " ], [ 793, "Sam Rogers", "Mm-hmm. " ], [ 793, "Douglas Hubbard", "But we would go back with our first useful measurement maxims, right? Which are, it's probably been measured before. " ], [ 799, "Sam Rogers", "Mm-hmm. " ], [ 800, "Douglas Hubbard", "You've got more data than you think, and you need less than you think. Okay? People routinely assume that if they have a lot of uncertainty about something, they're gonna need a lot of data to measure it. Do you think they… That's an assumption you've heard? Oh, " ], [ 815, "Sam Rogers", "yeah. " ], [ 815, "Douglas Hubbard", "Or- Very frequently. Yeah. Yes. Mathematically speaking, just the opposite is true The more uncertainty you have, the bigger uncertainty reduction you get from the first few observations. If you know almost nothing, almost anything will tell you something. That's the way to think of it, okay? " ], [ 832, "Sam Rogers", "When you say it that way, it sounds very natural and logical and easy. But so often it's true, we tend to think that if we don't know anything, we suddenly need to know everything in order to just get started. " ], [ 844, "Douglas Hubbard", "Right. Yeah, absolutely. Furthermore, I think this is a problem a little bit with the IT crowds a little bit more, I suppose, is that when they think of measurement, they're thinking of data that's already been captured in databases. Mm-hmm. Something they can do a query on. " ], [ 858, "Sam Rogers", "Right. " ], [ 859, "Douglas Hubbard", "Well, I don't know how the scientific revolution would have ever gotten started if we needed databases- … populated already to start, you know? Uh, scientific method is not just about having data. Some of scientific method is about getting data. " ], [ 874, "Sam Rogers", "Mm-hmm. " ], [ 874, "Douglas Hubbard", "When we measured the speed of light, we didn't actually measure all the photons. Right. Right? Yeah, excellent. It was a sample. Yeah. It was a sample of photons. Random sampling is probably one of the more powerful things I tell people when we talk about measurements, because the other thing they latch onto is they somehow need all the data. I say, \"No, you need a sample of that data from which you can make an inference about the rest of the population. \" That's how most of science works. So for example, when I wanted to measure how much time nuclear engineers spent on document management related items at a nuclear power utility, this was many years ago, there's 500 nuclear engineers among all these different nuclear power plants in the utility, and how could I go about measuring how much time they spend in document management? They're all over the place, right? Mm-hmm. So we came up with a method. This was long ago enough that they all had pagers, okay? Mm-hmm. And so we came up with a method where they would all be paged once during the course of a month. Each engineer would be paged once during a month, and which means we would get 500 samples of a page. And they're, they were instructed that when they got that page from that number, at their earliest convenience, they would sit down and fill out a form, okay? We didn't do things online necessarily so much then, but they filled out a form. Mm-hmm. And the form just said, \"At the time I got the page, here's the activity I was involved in. It was on this project, I was doing these things. I was on the phone, or I was in a meeting room.\" They just described it. We collected all of these. By the time we got done collecting all of those, we observed that about 20% of those, data points randomly selected throughout the day of different, engineers were activities that would have been automated by a document management system. Now, if we happen to catch one engineer in a break room and he said, \"I was sitting in the break room,\" that doesn't mean that guy spends his day in the break room, right? So on an individual level, it's kind of unintrusive, method. It doesn't tell you much about an individual. It can only tell you about the group aggregate. So it's really, not that intrusive- Mm-hmm … individually. But if you look at out of 500, you've… I think it was 97 or something individual cases where people were involved specifically in activities that would've been automated and those pagers went off at different times of day for different engineers, different times throughout the week- " ], [ 1026, "Sam Rogers", "Uh-huh " ], [ 1027, "Douglas Hubbard", "that is something we can make an inference from. That's called a spot sample, by the way. It's not uncommon for gathering information about other complex organisms, like in zoology, right? Right. Where, where things don't sit still to fill out forms all the time, right? You gotta, you have to have a systematic way to sample, instances of behavior- Mm-hmm uh, throughout a day, and that's one way to do it. Well, there's a lot of neat little empirical collection methods like that. You know, even learning how to do a simple controlled experiment can be a very powerful tool. If somebody said, \"Are the development cycles of teams using this method gonna be faster than the development type cycles of teams not using this method?\" Somebody could say, \"Well, how would I ever know what would've happened otherwise? \" " ], [ 1072, "Sam Rogers", "Yeah. Maybe this team is just better than that other team if I set- " ], [ 1075, "Douglas Hubbard", "Right … " ], [ 1075, "Sam Rogers", "them up, you know, one with the software, one without, or methodology or whatever. " ], [ 1079, "Douglas Hubbard", "That's right. Well, you know, if we didn't know how to work out that problem, again, the scientific revolution would not have happened. Right. Of course we know the answer to that. All the methods behind controlled experiments are specifically about handling that issue, right? So, when you do a clinical drug trial- " ], [ 1097, "Sam Rogers", "Uh-huh … " ], [ 1098, "Douglas Hubbard", "do they know for a fact that this person got better because they took the pill? No. What they know is the test group did this much better than the control group. " ], [ 1109, "Sam Rogers", "Mm-hmm. " ], [ 1110, "Douglas Hubbard", "And it was by a larger margin than you can explain by chance. " ], [ 1114, "Sam Rogers", "Mm-hmm. Okay? Or " ], [ 1114, "Douglas Hubbard", "by " ], [ 1114, "Sam Rogers", "placebo " ], [ 1115, "Douglas Hubbard", "or, you know, or- Yeah. Fake- So control group would be the placebo group, right? Uh-huh. Um, and the test group is the one taking the real drug. And what you know is that in the control group, ulcers tended to last this long or tended to, change this direction in terms of their severity. And in the test group, 48% of the ulcers went away in a week. " ], [ 1136, "Sam Rogers", "Right. " ], [ 1137, "Douglas Hubbard", "And the rest of the ulcers went down by this… You know, whatever the measure is, right? So there's some dramatic change, and that doesn't mean that the people, every individual who took a placebo, necessarily had a bad ulcer that got worse. Some of them might have gotten better on their own. Right? And sometimes when you explain that approach, people say, \"Oh, well, pharmaceutical companies, they had hundreds of subjects in each of their groups. \" Well, actually not always. Sometimes when you're dealing with life-saving cancer drugs, they have very small experiments. So your math has to be better because you're not gonna get the kind of slam dunk, easy findings that you could easily see on a chart. You gotta make, inferences out of more subtle findings. Okay? Mm-hmm. Now the math isn't harder. It's all y- nothing… I'm not talking about anything you can't do on an Excel spreadsheet. So, it's all straightforward. In fact, there, there's not too much I talk about that you can't do in an Excel spreadsheet I've already made for you that you can download for free on the book's website. " ], [ 1199, "Sam Rogers", "Great. So other than a change not being valuable enough to need a measurement, is there any other reason people tell you not to measure something? " ], [ 1208, "Douglas Hubbard", "I've run into people once in a while that, are a little indignant about the idea of measuring the value of a human life. In my first book, I make the moral argument that you have to measure these things. " ], [ 1220, "Sam Rogers", "And can you explain that just a little bit? " ], [ 1222, "Douglas Hubbard", "Yeah, because, As long as we have infinite resources or no big problems to solve, and then you don't have to worry about any of this stuff. But we do have limited resources, and we have multiple big problems to solve. Mm-hmm. So we actually have to make a trade-off between what's the value of better emergency services in Haiti because you've got better roads versus educating your children. When you start making those hard choices, you start realizing that, yeah, you know what? I am putting a value on a human life just by virtue of even making those choices. " ], [ 1255, "Sam Rogers", "Mm-hmm. " ], [ 1256, "Douglas Hubbard", "As soon as somebody says, \"This program which will save two lives a year on average is worth $5 million, but it's not worth 12 million because I rejected another project on that same basis.\" And when you look at utilities and governments and, you know, law enforcement agencies or hospitals, when you look at the decisions they actually made, they can tell you you should not put a value on a human life, but in fact, when you look at the decisions they've made, they already have been. Anybody with access to their decisions and a little bit of algebra can figure out the implied value of a human life. Mm-hmm. The problem is because it's only implied and not explicit, it changes every time. It's willy-nilly different due to arbitrary factors that have nothing to do with that decision. There's been all sorts of research now about the impact that your mood has on decisions, and you become more or less risk-averse for a series of reasons that have nothing to do with the decision that you're trying to be risk-averse about. All right? You know you're more risk-tolerant when you're exposed to smiling faces. Because " ], [ 1326, "Sam Rogers", "everything looks fine. " ], [ 1328, "Douglas Hubbard", "Yeah. You're more risk-averse if I ask you to recall some event in your life when you were afraid, and you're more risk-tolerant if I ask you to recall some event in your life when you were angry. And we can test these in controlled experiments, and those are the kinds of things that actually affect your decisions. The risk aversion for men is, correlated to testosterone levels, and your testosterone changes daily for reasons you're not consciously aware of. So anything could change your testosterone level, certainly sleep changes it. Winning or losing unrelated games of chance changes your testosterone level. So if you just won or lost something, even a game of chance like an office pool, right? Just before a decision, that apparently could affect the risk aversion in your upcoming decision. " ], [ 1376, "Sam Rogers", "Huh. How about knowing our internal state well enough to be accurate about our ability to make a decision? " ], [ 1384, "Douglas Hubbard", "Most of the time people are pretty bad at that. " ], [ 1388, "Sam Rogers", "Really? " ], [ 1389, "Douglas Hubbard", "Um, they tend to be statistically overconfident. That means that they put too high a probability on being right compared to their track record. Given a variety of studies, when people say they're 90% confident, they tend to have closer to about a 60 or 65% chance of being right. Yeah. It turns out, though, the good news is that, , they can be trained in half a day to be about as good as a bookie at putting odds on things. Bookies are pretty good. And, and you know who else is good? This surprises people, but meteorologists are good. Um, when a meteorologist says there's a 90% chance of sunshine, it rains 10% of the time, as you would expect. But the problem is when you evaluate the skill of somebody else, what do you remember, when they were right or when they were wrong? Right, the selection bias- You remember when they're wrong … of, of, yeah. Yeah. You don't, you're not running an average in your brain. Mm-hmm. Mm-hmm. Right? You just remember a few anecdotes. And now when it comes to evaluating our own performance, we are more likely to remember when we were right. Yes. And so we tend to be systematically- Yes overconfident because we're, it's easier for us to recall when we're right. I have a hypothesis I always wanted to test, which is I think you're most likely to remember something when you turned out to be right and your colleagues were all wrong in disagreement with you. " ], [ 1468, "Sam Rogers", "Those " ], [ 1468, "Douglas Hubbard", "really stand out. I think if you were the one guy who had it right and everybody else was convinced you were wrong and it turned out you were right, I think you'll tell that story at parties for years. " ], [ 1479, "Sam Rogers", "Right. " ], [ 1480, "Douglas Hubbard", "It's those sorts of things that guide our own, judgment about our performance. " ], [ 1484, "Sam Rogers", "Is there anything else in closing that you'd like to share for our listeners? " ], [ 1489, "Douglas Hubbard", "Yeah. I talk about in the books that there's really only three reasons why anybody ever thought something was immeasurable, and they're all three illusions. " ], [ 1496, "Sam Rogers", "Hmm. " ], [ 1497, "Douglas Hubbard", "I call them concept, object, and method, or you can think of dot com as a mnemonic if you like. The concept is the definition of measurement. As we talked about, it's not an exact number. It's an, a reduction in uncertainty expressed quantitatively based on observations, okay? " ], [ 1512, "Sam Rogers", "Mm-hmm. " ], [ 1513, "Douglas Hubbard", "The object of measurement is just defining the thing that you're measuring. If somebody says, \"I wanna measure collaboration,\" we ask them, \"Why? And what do you mean by it? And what do you see when you see more of it?\" Right? Mm-hmm. Don't just let the fluffy term exist, right? Mm-hmm. Uh, think about what it means in terms of observable consequences. And finally, methods of measurement. People misunderstand how random samples or controlled experiments or regression models work, and they have some profound misconceptions about sample size and, what probability means, et cetera, and that gets in the way of a lot of measurements. Hmm. I think those are the big opportunities for people. I think that's where if we can overcome those obstacles, all of a sudden the world really opens up to you, and there's a lot more measurable things than you ever thought they were, and then the only question becomes what's the value of the measurement? Right. 'Cause you could measure anything. You don't have to measure everything, obviously. It's, comes down to the information value. That's what most of the methods we talk about are really about, is computing information values and directing measurements based on what's statistically more likely to improve decisions. " ], [ 1579, "Sam Rogers", "That's great. Thank you so much, Douglas Hubbard. If people wanna contact you or gain access to your resources, where do they go? " ], [ 1586, "Douglas Hubbard", "Just go to howtomeasureanything.com. Thanks for your time. " ], [ 1589, "Sam Rogers", "Yeah. Thanks so much for being here. Really appreciate it. " ], [ 1592, "Douglas Hubbard", "You bet. " ], [ 1594, "Sam Rogers", "Three illusions: concept, object, and method. I said at the top I'd make the case for those as the three ways an AI business case comes apart. Here goes. Concept is demanding one exact number. In the actual empirical sciences, that number never exists. A measurement is a range that got narrower, and if you're holding out for a single figure, you'll be holding out for the wrong one. Object is measuring things like productivity or collaboration or enablement without ever saying what you'd see more of if you had more of it. That's the one you heard happen to me in real time about twenty minutes ago. And method is assuming you need the whole population sitting in a warehouse or something before you're scientifically allowed to begin. You don't need that. You need a sample and a threshold. That's not a coincidence, and it's not a new problem. It's the same three failures that we already knew about now at AI scales and AI speeds with AI-sized budgets attached. So here's the subtraction for this week: stop running pilots whose result won't change a decision either way. If there's no outcome that flips the call of any decision, you're not measuring anything. Doug kills it simply with that one question. You heard him ask it. Why do you care? Every fix you just heard still works unmodified. That's the whole reason this is in your feed this week. Now the part that genuinely has changed since twenty sixteen, What's new is how much money is riding on the answer. Because ten years ago, if you couldn't measure whether a new system was worth it or not, you shrugged, you bought it, or you didn't. Now, there's a board asking what the AI spend returned and a date by which somebody really has to say something. That kind of pressure doesn't make anybody any better at measurement. It makes them faster at producing a number. And the fastest number is not generally the best one. Uh, that's the one that Doug was just talking you out of. One last thing, the value of a human life. If you've got limited resources and more than one problem, you've already approved something and rejected something else. So that number is in there. And anyone with your decisions and a little bit of algebra can pull that out. All you bought by refusing to say it out loud is that it comes out different every time. Um, I've heard this argument almost word for word this year about AI systems, usually from somebody explaining why they can't put a specific number on a harm. But they already have. They just haven't looked at it. Issue sixty-four of the newsletter coming up on Sunday, one signal, one subtraction, one analogy, about five minutes to read, which is a promise I keep every week. Get it at Substack, LinkedIn, or better yet, at sigsub. show. I'm Sam Rogers. Thanks so much for listening. " ], [ 1804, "Voiceover", "Signals and subtractions. What to watch, what to drop, every week " ] ] }, { "episode": 8, "title": "Influence Over Reach", "url": "https://sigsub.show/episodes/ep-008/", "transcript": "https://sigsub.show/episodes/ep-008/transcript/", "video": "https://youtu.be/RUnYvI_Ftec", "anchors": [ "t-0", "t-11", "t-30", "t-54", "t-88", "t-105", "t-128", "t-130", "t-134", "t-174", "t-208", "t-210", "t-318", "t-318-2", "t-357", "t-357-2", "t-359", "t-377", "t-377-2", "t-386", "t-543", "t-543-2", "t-562", "t-563", "t-631", "t-648", "t-651", "t-655", "t-657", "t-658", "t-738", "t-740", "t-741", "t-755", "t-820", "t-961", "t-978", "t-979", "t-1051", "t-1052", "t-1081", "t-1082", "t-1126", "t-1126-2", "t-1175", "t-1176", "t-1179", "t-1181", "t-1277", "t-1278", "t-1284", "t-1285", "t-1326", "t-1369", "t-1498", "t-1499", "t-1528", "t-1529", "t-1559", "t-1563", "t-1584", "t-1585", "t-1621", "t-1623", "t-1672", "t-1672-2", "t-1760", "t-1935", "t-1936", "t-1943", "t-2009", "t-2011", "t-2011-2", "t-2011-3", "t-2012", "t-2012-2", "t-2017", "t-2021", "t-2040", "t-2041", "t-2044", "t-2045", "t-2066", "t-2067", "t-2145", "t-2167", "t-2167-2" ], "turns": [ [ 0, "Josh", "I had become a a number on a spreadsheet. And I was absolutely okay with it after three and a half minutes. Because I was pissed for three and a half minutes. I came into this office for 13 years " ], [ 11, "JD Dillon", "And as I'm walking to the office, I look at the schedule again, and I realize I'm on the wrong page. I'm on the supervisor page. I had successfully promoted myself. " ], [ 30, "Sam Rogers", "Welcome to Signals and Subtractions. Mr. JD Dillon is in the co-host chair. And JD, we've known each other for a while now. And and l although it's his first time here on signals and subtractions we first did our live stream I believe it was like Twine something like in 2016. It was a while ago. Anyway, really glad to have you here now. how do you introduce yourself these days, JD? " ], [ 54, "JD Dillon", "Well, Sam, thank you so much for having me today. To all the people out there, good morning, good evening, good afternoon, where have you be? I am the author of the Frontline Enablement Playbook. I focus on enabling business outcomes through the performance and enablement of frontline and deskless workers around the world. My entire 25-year career is wrapped around being a member of management of learning and development in support of technology in support of and now. advising around frontline and deskless performance. So I'm the frontline guy, formerly the guy from Disney, the guy who wears bright shoes at conferences, and now the guy who's helping make every shift count. " ], [ 88, "Sam Rogers", "And now the guy who's on signals and subtractions, hopefully once a month. And he brought along a friend. Hey Josh. this is my first time meeting Josh Felix. And it's it's great hearing just a little bit from you before the show here. can you give yourself a little bit of an introduction, Josh? " ], [ 105, "Josh", "Yeah, Sam, first and foremost, I appreciate the opportunity to be here. It's obviously awesome to be with JD again. I think it's probably been about a year since we've been live on the air. my name is Josh Felix. I serve as the senior director of professional services at Oxford Global Resources. our focus is professional services that power Business transformation. And the crazy part is, Sam, I just started at Oxford two days ago. I just " ], [ 128, "Sam Rogers", "Okay, so " ], [ 130, "Josh", "landed late last night from my two days of onboarding in Chicago. " ], [ 134, "Sam Rogers", "Well, I'm really glad to to get you so fresh off of that experience. just for the folks at home, signals and subtractions, every one of us will bring a signal that we're currently watching, a subtraction that we're making. what to watch, what to drop, that's the whole show. we probably won't all agree about everything or say everything the same way, which is what makes it worth your time. and JD for your frontline enablement perspective, I'd like you to listen carefully to Josh's story. We'll get your take in just a moment. Josh, would you mind giving a little bit of that play-by-play? Let me know a little bit about what came before this, if you wouldn't mind. " ], [ 174, "Josh", "We're going to take a walk back to April 23rd, 9:30 in the morning, Eastern Standard Time. That dreaded meeting lands on your calendar. Your VP, no agenda in the meeting. You reach out and say, How can I prepare for the meeting? we're just gonna have a conversation. I was already mentally prepared coming into this meeting. I knew exactly what was happening. 'Cause I had already been part of nine thirty in the morning Thursdays twice, and I knew what took place. Jumped on the meeting, VP logs in, guess who appears next, Sam? HR Jumps right in, right? I was " ], [ 208, "Sam Rogers", "was yes. " ], [ 210, "Josh", "like, this is fantastic. I know why we're all here. well we really appreciate you. Everything I said, guys, if we if all that was the case, then we wouldn't be having this meeting right now. I had become a name on a spreadsheet or a number on a spreadsheet. And I was absolutely okay with it after three and a half minutes. Because I was pissed for three and a half minutes. Three and a half minutes, I was livid. I came into this office for 13 years excited to serve the front line every single day, passionate about serving the front line, knowing that I was going to take every challenge on and really drive that challenge. But that point, after that three and a half minutes, I took a walk. picked up the phone, called a mentor of mine, and he said, Josh, he said, when in your life have you ever been paid to do absolutely nothing? I said, hmm. Never. He said, here's the crazy part. I'm not going to allow you to do that. He goes, here's what you're gonna do for the next month. Monday's your day, Friday's your day. Mow the right side of the lawn on Monday, the left side of the lawn on Friday. But Tuesday, Wednesday, and Thursday, you have to fill your calendar with any meeting you can get. Conversation after conversation. CEOs, CLOs, chief human resource officers, competitors of the place that you came from, colleagues that you worked with. Every call can have absolutely no agenda. You're having a simple conversation. And you're gonna ask those individuals can I record the conversation? Recorded 64 conversations. I took all 64 conversations and I dumped them into AI tool. And I said, synthesize this. Tell me what the trends are in every conversation. And the amazing part for me was after the 64 conversations, the trends that I got were exactly what I had been doing for 13 years of " ], [ 318, "Sam Rogers", "Mm. " ], [ 318, "Josh", "inviting people to sit on my virtual couch and share the challenges that existed for them and then presenting solutions back to them. Except I wasn't presenting solutions. I was simply listening to the challenges that existed across the globe. And I sat down and I said, You know what, Josh? You're now faced with a challenge. You now have to solution for yourself. When in your life have you ever been two dimensional? When in your life have you ever been able to live on a piece of paper? And I said, Never. I said, I'm three dimensional. It's just who I am. So I'm gonna pause there because my signal was that I knew I had to do something different. I had to be different. I had to really move forward. We'll talk about " ], [ 357, "Sam Rogers", "Yeah. " ], [ 357, "Josh", "our subtraction coming up. " ], [ 359, "Sam Rogers", "Yeah, that's fantastic. And also in the chat, thank you, for adding to the chat. great point. Having a mentor to call well in advance of that dreaded Thursday meeting was a a critical setup point there immediate frenzy of the moment and " ], [ 377, "Josh", "Right. " ], [ 377, "Sam Rogers", "hurt and disappointment, like, okay, what do I do next? how about you, J D? What did you what did you hear in that and how does that line up with the with your signal? " ], [ 386, "JD Dillon", "One, think it's an important conversation to have and an important story to tell given the challenges people are facing and finding right opportunity or just an opportunity right now. And anyone who follows me on LinkedIn has noticed something for the past five years that about half of my LinkedIn feed is job postings. Because I do what I consider a pretty simple thing. I scroll job openings and I pull together lists or share individual openings and just drop them on my LinkedIn feed. And I started doing it because we were in the middle of a hiring boom at that point. And I know what happens when organizations start to change priorities and shift decision-making and... change begins. I know what happens to people who are on the enablement side of the conversation. So it was just my simple way of saying, here is this helpful. And not only here are some opportunities that may fit somebody, but also here's maybe a bit of hope. Like there are things out there, like there are still chances. And I just started, I was not the first to do this. There's a bunch of other people that I list in the postings that I dropped that also post jobs that I kind of echoed their efforts in a different way. And the reason I still do it is that a lot of people have messaged me or come up to me at events and made reference to those postings and that they were helpful in different ways. And I still remember the first person who ever came up to me and said, Hey, thanks for posting those jobs. I saw one and I applied and I got it. And it was that moment where I was like, well, I'm just going to keep doing this. And what's cool is that people will now message me with jobs on my side. And they're things that maybe you didn't see or they're not posted on LinkedIn. And I add them into the stuff that I do. So if anyone out there has openings on their team, I'm happy to echo them into the network. Because again, I know how challenging, it can be. especially in the current marketplace and just the nature of the changing workplace, which takes me to kind of the signal part of my conversation, which is influence is the most important skill in the workplace. And I don't think we properly value that. I think it's hiding underneath a lot of other conversations. One, we continue to talk about, and this is every function does this, We have a seat at the table conversation. Like things would be different if we had a seat at the table, which I think is the wrong way to look at this, especially because if you're under the HR umbrella, the CHRO or chief people officer is sitting at said table already. So it's not a, it's not a representation challenge. It's something else. And then you add onto that challenge that, you know, I wish the nature of the workplace was that the best idea won. I don't think that's ever really the case. Organizations make decisions or prioritize different things based on other factors and it's always that you're the expert and you you're hired to do a certain job and you you go in and pitch an idea you would hope that people would just respect that like that's what you do like we should listen to that person. " ], [ 543, "Sam Rogers", "Mm. " ], [ 543, "JD Dillon", "Often not the case because other things are in mind or other priorities are above what that idea might be and then you add the AI of it all where I don't know if anyone's heard this conversation yet or actually had these words said to them. I'm going to go out on a limb and say it's increasing the amount of times people are at least thinking, like, why do I need you for this? I can just ask Claude and Claude " ], [ 562, "Sam Rogers", "Yes. " ], [ 563, "JD Dillon", "will give me awkwardly formatted information with lines and unnecessary places and many dashes. if I can find this information differently, if I can construct this, and if I can analyze information differently, if I can just source the multitude of ideas that were scraped off of the internet, do I need someone to offer this idea for me? So you stack all of those considerations together. It creates a world where it's that much harder to move people, to drive change in a direction that you find as value add and to... shift mindsets and transformation. That's why I come back to this idea. influence is the ultimate workplace skill, your ability to be able to convey information that drives people to action. And that shifts the way people think. And when it comes back to that seat at the table idea, I don't think we need to see at the table, you need everyone at that table on your side, pushing in your direction and believing in your expertise. The only way to do that is to your ability to influence people, to think differently, to embrace different ideas, and to unlock the potential of what you bring to the conversation. So influence is my signal, and it's the topic of the next e-book that I'm currently writing. " ], [ 631, "Sam Rogers", "Okay. Well, glad you're writing another book. and now's a good a time as any, JD. I believe you have a specific book that's already available, or maybe two that maybe you could give a quick plug for now with that that little layout. Yes, yes, here we go. " ], [ 648, "JD Dillon", "Speaking of book, everyone hold up, Sam? " ], [ 651, "Sam Rogers", "I d I don't have that one. I have your other one. I should have got it. It's on a shelf right here. " ], [ 655, "JD Dillon", "That's okay. First book, the Modern " ], [ 657, "Sam Rogers", "I know that one. " ], [ 658, "JD Dillon", "Learning Ecosystem, the website is actually JDwroteaBook.com. So about transforming your approach to learning strategy in an organization. But most recently, the frontline enablement playbook. So if you support a frontline or deskless workforce in industry such as hospitality, food service, grocery, manufacturing, construction, retail, or anywhere else, the 80 % of the world and 2.7 billion people who work in frontline or deskless capacity serve. This is for you. So it's written for HR, L &D, or operations professionals with the common goal of just helping frontline workers do their best every shift in order to drive outcomes for the organization. 650 pages of storytelling, practical ideas, insights, examples from various industries. And what I'm really excited about 50 other contributing authors shared their perspectives on the story. So we represented different industries, different roles, different functions in different parts of the world. It's why it's 650 pages. And then also contributing to this story were 12 frontline workers and managers. So I did not want to be a person sitting in an office telling a story about people who do very physically and emotionally challenging work. So I reached out to and interviewed people who are doing the job today and then shared their perspective on what's most important to them and what helps them do their best at every shift So. Frontlineplaybook.com is where you can check out the book. Is it a sequel to the first book? Yeah, it is. It'll be the two " ], [ 738, "Sam Rogers", "Yeah. " ], [ 740, "JD Dillon", "brightest books on your shelf. " ], [ 741, "Sam Rogers", "Yes, just like JD's shoes. And that came in what when? In May? Like there there wasn't like any big world and society changing event that happened after the release of your second book, unlike with your your first one. " ], [ 755, "JD Dillon", "Yeah, the fun is, one, I wrote this book during the pandemic. So kind of similar to the tactics Josh applied to kind of gain perspective. I stopped writing in the middle of 2020, said, is this what the world needs right now? And then went on a listening tour and started to realize, there's actually commonalities between how people are navigating this disruption in the story that I'm working to tell. And that informed this book. And then a funny story, the book came out the day before Chat GPT released publicly. So it was like, hey, I wrote a book. no. The world just changed. So I ended up writing a 12,000-word addendum to the book that was the, this still makes sense, I swear, And then I was able to both apply AI tactically, written by humans, leveraging different tools, and also talk about the impact and potential for AI when it comes to the frontline workforce in this book. So writing a book right before and then right after the conversation begins, around AI is interesting because the process was meaningfully different based on this different tool set that I have available to write, you know, in the same amount of time wrote two very different types of books and very different types of projects. " ], [ 820, "Sam Rogers", "Well, thanks for that. and with this show, Signals and Subtractions, it's not really about AI, it's about the world after. okay, now that we've got all of this, now that we're looking for work or now that we're doing work in the workplace, how things have shifted, what are the things that make sense? What are the things that don't anymore? And for my signal this week, I wanted to bring Something that I've been observing over the last year or so, but it's kind of flipped lately, which is expertise inversion, is what I'm calling it. a year ago, oftentimes leadership would have a person, their person, who would do the Claude interface thing, the chat GPT thing, who would be that that kind of human layer for them to be able to curate information in a similar kind of workflow as they had always done. And now that's not so much the case. Now more and more we're seeing folks who maybe don't have a a senior title. They don't have seniority in the organization. But those who really have solid AI skills and who are able to move quickly and provide information to the business they're able to kind of bypass some of the usual structure of the organization. And it's it's my premise, this is fresh still, that that's some of the flattening that we're starting to see in organizations, is related to not just that you can ask Claude a question, but that the people in your organization, oftentimes people who are maybe not in the management levels, but on the front lines of work. Can get the answers to what they need, more importantly, ask better questions and establish processes within the organization that are still kind of in that shadow mode, as so often happens. Like there's the rules of how we do it, and then there's what happens on a daily basis. Unfortunately, that that tension will probably always exist. but AI has definitely changed that dynamic in some important ways. So who gets listened to what gets that influence that you're talking about, JD? I've noticed just in the last few months changing in some real ways with some of my clients. from our signals portion to the subtractions portion And check back in here with you, Josh. mid story. I'd love to hear a bit more. " ], [ 961, "Josh", "Yeah, so that story cusps picks back up at the end of the sixty four conversations and synthesizing through what my focus was gonna be. and my mentor said, Okay, well the next step that you have, and this is a Monday morning, is I wanna review your resume. and I was like, I don't have one of those. " ], [ 978, "Sam Rogers", "Yeah. " ], [ 979, "Josh", "I've I've been at the same place for 13 years. Like I wasn't planning on this day to ever come. Like I I was serving the front line for the rest of my life. That was my goal. That was my focus. That was my desire. And he said, Well, we need to create a resume. And I said, You know, Matt, that's an interesting request. I said, Okay, I'm gonna sit down. And I started working on a resume. By the time lunch rolled around, I deleted the file. I was done. I was like, I am not creating a resume. I'm just not doing it. So my subtraction was the resume's gone. I need to stand out. And how can I stand out? And that afternoon, I took everything I had thought about that morning. Everything I was trying to put on that document. And I said, how do I present this out to the world? How do I let them know who I am? How do I think about that in that three dimensional aspect of what I do? And I said, You know what I'm gonna do? I'm using AI and I'm building a website. And that's where TheFutureWithJosh.com was developed. And that's turned into my resume. But the biggest challenge that I had is that this world we live in is driven by this three letter thing in the recruiting world called the ATS. " ], [ 1051, "Sam Rogers", "Mm-hmm. " ], [ 1052, "Josh", "The applicant tracking system that is really the backbone of every job posting that's ever out there. And how do you break down that door when you don't have the resume? and I said, I'm reaching out to people the same way that I reached out to the 64 people and saying, This is who I am, this is the URL, this is my resume. Let me know if we're interested in having a conversation. Those that said, hey, you have to have a resume, I knew that was never going to be the right place for me. Cause " ], [ 1081, "Sam Rogers", "Mm-hmm. " ], [ 1082, "Josh", "they're too bogged down in thinking that they have to use technology, which they truly don't need. Go out, recruit, find the people that fit the need that you have as a business. Don't rely on thousands of AI generated resumes that somebody literally took a job posting. and just redid their resume for that job posting to come through the ATS to be reviewed by AI and all of a sudden you start missing out on great people. Because the model might not be trained on how you think or how the organization thinks or how that leader thinks, Go out and recruit for the people. Allow people to be different. My mentor said, you know what, this reminds me of back in the day when people used to go to Dunkin' Donuts or Krispy Kreme and buy like dozens of donuts, right? And pay and " ], [ 1126, "Sam Rogers", "Yeah. " ], [ 1126, "Josh", "tape the resume to the middle the inside of the box and stop at different offices and drop them off just to try to get somebody's attention. But that's who I am. And it became this, I don't know, JD and I don't like using this word, portfolio. I didn't need a portfolio. This was real work. I was showing the real work that I was doing within the site. And the site lives on now. And that was what was so exciting. That immediately I had three job offers. Literally within a week and a half. The amazing part about those offers was the first two, I got all the way to the senior conversations. They were quick, right? Once they find somebody they really want, the process moves quick. That's when you know that you're really heading down this right path. Got to the senior conversations. Both senior leadership conversations, Sam, ended with saying, I can't wait for you to start so I can learn all the secrets of where you came from. " ], [ 1175, "Sam Rogers", "Wow. " ], [ 1176, "Josh", "And you know, I received two offer letters from both those companies. " ], [ 1179, "Sam Rogers", "Yeah, well with a layup like that I'm not surprised. " ], [ 1181, "Josh", "Yep. And I sat down and I said, you know what, that's not who I am. I don't want you to hire me so I can come and give you the secrets of where I came from. I want you to hire me because you want who I am that helped drive success at the business I was at. I literally sent them the exact same email back saying, Thank you, but no thank you. And this is why. One never reached back out. The other one was literally probably two and a half, three minutes after she received the email. Saying that's not what we meant, that's not what we meant. No, that's exactly what you meant, right? The third wanted me to travel 90% of the time. I'm just not in a season of life that I could do that. So turn both all three of those down. And then all of a sudden I was at the point of having a fourth which was feeling pretty good. But then the fifth came over the top rope. And it was just something about everybody that I spoke to that was so genuine. They weren't worried about what I brought, the secrets or what the secret sauce was. They wanted to know that I was gonna fit, that I was gonna make a difference, that I was gonna be confident in the discussions that I could have. And I could help them as an organization really think about modernizing the operation, the ability to scale with confidence. across a multitude of disciplines And that was what drew me in because it all resonated back to the sixty four conversations. Now the real win, is that I've now turned this into the unplanned sabbatical boot camp. And I'm going to serve ten individuals starting in September to really walk them through this journey of how to create a brand for themselves, how to really think about that job hunt. I said ten I have seventy six people that have applied to be part of the boot camp. " ], [ 1277, "Sam Rogers", "Wow. Nice. " ], [ 1278, "Josh", "Telling me their story, but I want to find individuals that are driven, that are hungry, but hungry for the right reasons. People " ], [ 1284, "Sam Rogers", "Mm-hmm. " ], [ 1285, "Josh", "that were pissed like me for three minutes, but grabbed their bootstraps and stood up and said, you know what? I'm gonna tackle this head on. I'm not gonna feel sorry for myself. I'm not gonna allow others to feel sorry Somebody that wants to create a network around themselves and really drive it. So my subtraction and I know people out there are saying that's not gonna work. Get rid of the resume. It's two-dimensional. It truly doesn't tell the story of who you are. Find something, and I'm not saying it has to be a website. I'm not saying it has to be a portfolio. But find something that tells the story about who you are, who you're going to be within that organization, and how you're going to make that difference. So that's my subtraction, Sam. I know I'm probably getting darts thrown at me from the HR world. " ], [ 1326, "Sam Rogers", "Well, maybe so, but not here. So in episode three of signals and subtractions, Christine Rodrigues came on talking as well about job search and very much in alignment with what you're talking about. I mean, when when a job description opens and suddenly there's like In the span of a week, 1200 applicants. whether it's AI sorting through it or a human sorting through it, it's the same issue. And being able to cut through that and play on your own terms rather than play the game that doesn't really work out well for less than 1% of the of those applicants. is a bold thing to say and and I'm glad that you're saying it here. go ahead, JD, tell us about your subtraction for this week. " ], [ 1369, "JD Dillon", "I'd just like to add, I have an affinity for this perspective on shaping path and shaping opportunity because it was a lesson I learned really early on and I didn't necessarily mean to learn it. Cause I came from the school of thought professionally that you do a real good job and you out compete everybody else to climb the ladder. And when you do a real good job, they promote you and then they promote you again and they promote you again. And that was what I thought career development was. when I started working. I got my first job at a movie theater in my hometown and when I moved to college I wanted to pick up where I left off and I was promoted quickly to supervisor at home But when I got the job in Orlando, they hired me but they didn't keep me on as a supervisor And I understood because they didn't know who I was. And this was the fourth busiest movie theater in the world. So I OK, fine. I'm not a supervisor right away. And the way that we designated supervisor staff, we have these shoulder epaulettes on our button up shirts and the staff had multiple colors and supervisors had greens. So I had to take off the greens and put back on the multicolored ones. And I bristled at it. But, you know, OK. So then I went on for months working as a staff member and very quickly they started to realize I knew how to do stuff because I was the supervisor somewhere else. And it went from, okay, it's an opportunity to stretch, stretch assignment, right? Like come in early, stay late, special project supervisor called out, I stepped in those types of things. And eventually started to wear on me because they're not promoting me. There's not an opportunity. There's no room. I'm not getting paid to do this stuff. Where am I going here? And then one Saturday night, really busy. the supervisor called out sick. So I'm managing the busiest concessions and on earth, literally, and I am running for hours. And around nine o'clock that night, I snapped because I was just not doing this anymore. Most people say they leave, they quit, like they walk out. This is it. Go get a better job somewhere else. I did not quit. I went to the back of the concession stand. I ripped off the staff eplets. And I put on the green eplets that I'd been carrying in my back pocket the entire time as a bit of a silent protest that how dare you take this away from me. So I put my gear back on and then I just went back to work expecting to be called out. No one said anything. I was okay. I came in the next day dressed as a supervisor. No one said anything. " ], [ 1498, "Sam Rogers", "Yeah. " ], [ 1499, "JD Dillon", "It's like, all right, so this is what we're doing. And it went on for like two weeks. I just kept coming in dressed like a supervisor. No one said anything. Now remember I'm good at the job. So if I was bad at the job, someone would have stepped in, because then I would just be impersonating a good employee. And then I came in one day to get the staff schedule, and I wasn't on the schedule. And I was like, uh-oh, I have to go to the office now. Like, here it is. We're going to have a conversation. And as I'm walking to the office, I look at the schedule again, and I realize I'm on the wrong page. I'm on the supervisor page. I had successfully promoted myself. " ], [ 1528, "Sam Rogers", "Yeah. " ], [ 1529, "JD Dillon", "So the one caveat, though, anyone out there, especially if you're in HR, you might be thinking, when you promote yourself, there's no paperwork. No paperwork means no pay raise. So what had happened was, The manager who made the schedule saw me wearing supervisor gear, assumed I missed a memo and just moved me on the schedule. So now everyone thinks that someone else promoted me, but the company doesn't know that yet. So I waited for a couple of weeks. I'm not gonna get in the way of this. I'm gonna keep this going. " ], [ 1559, "Sam Rogers", "Now's when you get the popcorn at the movie theater and say, Ooh, what's happening? " ], [ 1563, "JD Dillon", "I'm real good at popcorn, real good at popcorn. And I wait a couple of weeks and eventually there's a moment where a coordinator on the staff is getting promoted to manager and she's doing the thing new managers do, which is go around and ask people how they can make their lives better. So I cornered her one day and I said, hi, remember a couple of weeks ago when I got promoted to supervisor and she said, yeah. And I'm thinking, no, you don't. It didn't happen. " ], [ 1584, "Sam Rogers", "Ha ha ha. " ], [ 1585, "JD Dillon", "Well, there's an issue. My paycheck hasn't changed. I don't think I have my supervisor rates or something must have gone wrong. She said, I'll get that fixed for you. And I said, you're going to be a great manager. And not only did she give me my pay bump for being a supervisor, she backpaid me to the day that I decided that I got promoted. So it's a kooky story. I still carry the eplets. They're sitting on the shelf right behind me right now as this kind of learning moment of Okay, this is kind of a wacky way to take ownership of how you're going to represent yourself and how you're going to carve your own path rather than waiting for someone else to give you a path forward. " ], [ 1621, "Sam Rogers", "Mm-hmm. Mm-hmm. " ], [ 1623, "JD Dillon", "And it was, you know, most people can't pull that off. kind of just snuck in the middle of a gap in the kind of system. But I find that over and over again, you know, if you follow the system, you go where the system takes you. If you figure out how to break your way through the system while still doing a good job. Right. Still have to be good at what you do. It's just an influencing example of how to take what you're earning as opposed to waiting for the world to hand it to you, which again, I wish the best idea always won, but that's simply not the case coming all the way around. Not exactly related to my subtraction for our conversation, but I think it's an important one, especially in a world where you see a lot of creators burning out, like people who are trying to feed the machine, And now with AI, the easy ease with which you can post dozens of pieces of content or slop a day just " ], [ 1672, "Sam Rogers", "Yeah. " ], [ 1672, "JD Dillon", "to feed the algorithm, just to try to get maximum exposure, just to try to put yourself out there where I've realized over time, it's not about finding thousands of people to listen to you. It's about finding the right people to listen to you. And there was a point, especially as a regular speaker of conferences where I programmed. And I picked topics, again, topics where I had some insight to share and there was going to be valuable conversation. But I, made decisions and prioritize things based on the biggest room. right. The biggest audience, most exposure, which is good. There is a branding element to it and awareness element to it. So there's some value to that, but I've over time realized like, I'd rather have a conversation with the right 10 people than the wrong thousand people. I think that's beneficial both as a business to say, you know, I depending on what products or services that you're coming to market with, you need to find the right audience, you need the right thing for the right people, not the wrong thing to everybody. And think it's, really hard to be something to everyone. And I think that's where organizations can make mistakes and can start to engage in unnatural acts is when you don't know what that purpose is, what makes you different and who you provide value to, cause you can't provide value to everybody. So Whether it's you're writing a blog or a newsletter or you're looking for new opportunities and trying to shape your brand and awareness of the value that you provide, how do you tilt in the direction of finding the right audience and the right people to have a conversation with versus just throwing content into the wind? Because the algorithm's not working for you. You have to own the conversation. " ], [ 1760, "Sam Rogers", "That's very true. Yeah. The the algorithm is is using us just as much as we're using it. Thanks so much for bringing that here today. And for my subtraction this will come back around to what we're talking about today. In the AI world, both the floor and the ceiling have moved just in the last couple weeks in an important way. I think that affects how we work. Some of the newest, flashiest, new and improved models are showing some signs of wear in that they're not actually better. They're just newer. okay, it's the it's the new and improved thing, but does it actually do what I need it to do better? that ceiling has gotten a little shaky, but also at the same time, the floor the quality of responses that you can get from LLMs in general. Because there's been such a race, especially from lately open weights models, it's now maybe not permissible in your workplace. I'm not saying that Microsoft Copilot is just as good as everything else. It isn't. But there is now a raising floor on what you can get from those kinds of systems and how it is that you use them. Now makes more of a difference than the model that you are picking. So for this week's subtraction, I'm saying don't worry about the model so much anymore. There are several good answers. There's not just one. asking for what is the best system. I think all of us you know, coming from the learning and development world, you're you tire of the question: what's the best LMS? You know, the best. For what? Like the best for what I'm trying to do may be very different than what someone else is trying to do. There's now a general intelligence floor that's good enough for most of what we want to do. And the workflow itself is what makes more leverage, more difference than the model that you would be picking. How it is that you manage that context, as we've talked about in previous episodes. makes more of a difference than ever. So we can actually drop the question of what is the best model and focus on the work that we're doing and how it is that we are enabling the best workflow for where we're at, for the the company that we're at, how we're managing that context, the specific tasks that we're executing against. With looking for work and and bringing it back to Josh's wonderful success story, where you you really only need to find one. And when you have options, when you've got the first offer, the second offer, the third offer, the fourth one's good, but this fifth one is the one that I I want to take. that same methodology works great with AI options as well. " ], [ 1935, "Josh", "Yeah. " ], [ 1936, "Sam Rogers", "They don't need to be the same to be useful to me. Having more than one. means that I can then pick the one that works best for me. " ], [ 1943, "Josh", "Yeah. And I and you Sam, it's so interesting, you know, when you mention it, I think like pre-COVID, we were all in offices, right? And you had a question and you either went down the hall or you looked in the cube farm. And it's kind of like a model. You know, you just you ask the question of whoever wasn't on the phone or whoever's red light wasn't lit up, who had the green light on, right, on top of the cubicle. It's kind of the same thing that we're doing right now. But People are looking at models being like, I need the best. Well, you didn't always get the best. Like it just didn't exist. Find the right solution for your priorities. And If people start thinking about that, what's the priority that I have when I'm utilizing AI? What am I looking to accomplish? And thinking about those solutions, it's kinda like when I was in the office and JD was down the hall, but he was on a call and I went to somebody else. Like you just think about it in a different way. Like, just use it, right? But the one thing I'm gonna ask is please stop getting taking the answer you get in Copilot and putting in a chat GPT and then taking that answer and put asking Claude if it's a good answer, and then you know, taking Claude's answer and putting it like that's where the mess just comes. Like just choose one. Choose " ], [ 2009, "Sam Rogers", "Yeah. Yeah. " ], [ 2011, "Josh", "one. They're all " ], [ 2011, "JD Dillon", "I " ], [ 2011, "Josh", "good. " ], [ 2012, "JD Dillon", "I mean, " ], [ 2012, "Sam Rogers", "And and there are other ways to collaborate. I I have a multiagenic harness that's free and open source. There's there's other ways to to go about it. Yeah. " ], [ 2017, "Josh", "Right. But stop copying and pasting. Yeah. " ], [ 2021, "JD Dillon", "at very, very least copy, remove the formatting and then paste like just little steps. But for me, it comes back to some conversations I've had with with executive teams that people are just less and less interested in chasing AI and more and more interested in not losing sense of purpose. And the question " ], [ 2040, "Sam Rogers", "Yes. " ], [ 2041, "JD Dillon", "I'm using in a lot of conversations now is what do we really do here? Or what do " ], [ 2044, "Sam Rogers", "Yes. " ], [ 2045, "JD Dillon", "you really do here? And it's And there's plenty of examples of this in the corporate space right now of organizations that have lost that clear sense of purpose and what made them different and what made them attractive to customers and employees to start. And they started following others and they started using strange terms like AI first company, which I don't know what that means. " ], [ 2066, "Sam Rogers", "Ha ha. " ], [ 2067, "JD Dillon", "and there's a ton of examples of companies ripping out AI systems and stopping projects because it clashed with how the work was done. And that wouldn't have happened if they had that kind of cornerstone identity of this is what we are. This is what we do. This is our purpose. And every decision is made in service of that. And if a technology or a process change or whatever it is, a new product clashes with that purpose, we either don't do it or we have to reassess the purpose. And that's the epic brands that a lot of people and companies look up to. That's what they're really good at is Trader Joe's is a great example. Have you ever seen a self checkout and Trader Joe's? No. Have you ever seen in-store media and a Trader Joe's? No. Why? Because they are ruthlessly committed to what they are, their position in the marketplace, and they have the most profitable square footage in the grocery space. Not trying to grow and be as big and bad as some of the other competitors, because they know what they are and they know how they got there and they know what it means to innovate and transform around that purpose. And I think that's the path forward in the AI conversation, because it's real easy. when you're getting demos of technology, you're trying to make expensive career-defining decisions when it comes to implementing different systems is if you root it in purpose and you can answer the question, what do I really do and what do we really do here, you'll make a better decision. " ], [ 2145, "Sam Rogers", "Well, that's a a great layup for next week when we'll be talking about how it is that we express intent and keep that as a as a through line with some more AI-specific practices. next week Lee Rodrigues back in the co-host chair with a special guest to explore that. Thank you so much, JD. people don't know, they ask what JD stands for. It is job description. Please follow " ], [ 2167, "JD Dillon", "Yes. " ], [ 2167, "Sam Rogers", "his his LinkedIn and you'll see plenty of job descriptions. And thank you, JD, for all that you've done to help others find the work that they need and enable frontline workers to really do well, not just from their role, but also helping it be seen, what it is that they're doing and advocating for them. thanks, Josh, for coming along. I really appreciate the personal story here and how well you tell it. it sounds like Oxford is very lucky to have you. JD and Josh, wishing you the best of luck and thanks so much. yes, every shift counts. So it does. Thanks so much, guys. Take care. " ] ] }, { "episode": 9, "title": "Dig the Second Hole", "url": "https://sigsub.show/episodes/ep-009/", "transcript": "https://sigsub.show/episodes/ep-009/transcript/", "video": "https://youtu.be/RsgS44L88Rg", "anchors": [ "t-0", "t-17", "t-36", "t-37", "t-78", "t-83", "t-84", "t-97", "t-98", "t-190", "t-191", "t-193", "t-194", "t-237", "t-238", "t-266", "t-266-2", "t-314", "t-316", "t-336", "t-401", "t-415", "t-415-2", "t-492", "t-493", "t-645", "t-692", "t-749", "t-789", "t-832", "t-833", "t-834", "t-1046", "t-1052", "t-1053", "t-1071", "t-1071-2", "t-1083", "t-1084", "t-1095", "t-1096", "t-1106", "t-1107", "t-1111", "t-1111-2", "t-1133", "t-1134", "t-1172", "t-1179", "t-1180", "t-1362", "t-1366", "t-1367", "t-1372", "t-1372-2", "t-1376", "t-1376-2", "t-1412", "t-1413", "t-1469", "t-1469-2", "t-1474", "t-1644", "t-1652", "t-1672", "t-1709" ], "turns": [ [ 0, "Lee Rodrigues", "if you give an agent information and say not to share it, rest assured, it's going to share it. It's just a matter of time, " ], [ 17, "Sam Rogers", "Welcome to Signals and Subtractions. I'm your host, Sam Rogers. Every week everyone brings two things to the table here. One signal worth watching, one subtraction worth making. That's the whole show. What to watch, what to drop. And this week we have our co-host Lee Rodrigues, Nice to have you back. " ], [ 36, "Lee Rodrigues", "Glad to be back, Sam. Thanks for having me. " ], [ 37, "Sam Rogers", "what we're talking about today is how to upgrade the questions that you're asking. Because a lot of times the answer is right in there, it's just a slight reframe. And agents Are definitely a topic of discussion for this week. when we're talking about agents, what are we really talking about? There's the marketing version of what that means, and then there's like literally what they do. So that's some of what we'll be exploring today. And and Lee, I think you had a couple stories to share right off the top. You've just been digging holes in your yard and finding interesting things. " ], [ 78, "Lee Rodrigues", "it's amazing how plumbing and housework applies to the business world so much sometimes. And " ], [ 83, "Sam Rogers", "do tell, do tell. " ], [ 84, "Lee Rodrigues", "and and it often takes that aha moment. And as you will find in most discoveries, the aha moment is often right after a well, that doesn't make any sense. How how " ], [ 97, "Sam Rogers", "Ha ha. " ], [ 98, "Lee Rodrigues", "how can that be that way? backstory. I have a house that was built seventy five years ago and We've slowly gone through and replaced the electrical. It's all modern. It's got solar panels. It's got batteries. All that's been taken care of. The couple nagging things that aren't done. we're gonna get a water filtration system that leads into the house to make the water cleaner to drink. my plumbers were telling me all this money to replace this water line. And I was like, well I have a little extra time these days. I would rather if I have to dig a forty foot trench, I'll dig it myself rather than pay someone to bring a backhoe into my front yard and destroy the entire front yard in the process. And then I had a moment. I've been thinking about this concept of double loop learning. And the whole concept of single loop and double loop learning is single loop learning is I made a mistake I need to do this differently. Double loop learning is maybe my assumptions in stating this problem or how I'm looking at it are incorrect. Maybe the problem statement itself is incorrect because my perspective is incorrect. And I can go like hell to do all these tactical changes, but the strategy I'm using is no longer aligned because I haven't really evaluated my own viewing of this. And in the tech world, it's often traveling on assumption, you've never got any real empirical observational data to see whether this is working or not. So in my waterline, I have this rusty pipe in front of the house that's all hooked up, it feeds water into the house. So I knew there was a digging project. Well, I'm chipping away, getting all these roots around these pipes. And last night, as I'm digging, I see where my water pipe comes in: rusty pipe leading down to a brand new water line. " ], [ 190, "Sam Rogers", "Really? " ], [ 191, "Lee Rodrigues", "And I'm like, why would somebody " ], [ 193, "Sam Rogers", "Ha ha ha. " ], [ 194, "Lee Rodrigues", "run a new water line to a rusty pipe leading in? One would assume there's our special word there, assume that it's all rusty. So now I need a second data point. Where the water line goes into the city line. Is that new? So as I'm my ego is coming up with a million reasons to justify my position. And I have this moment that we get in tech world all the time. You don't know till you know. You don't have a test or some way to empirically look at this to measure whether it's working or not. You really don't have any way of knowing. So kick the pride aside, dig a one-foot by one-foot hole where the water line comes in. 16 inches down, I reveal new waterline going into the city connection. So, first off, Ladies and gentlemen, I was about to pay someone $4,000 to dig a " ], [ 237, "Sam Rogers", "Mm-hmm. " ], [ 238, "Lee Rodrigues", "trench through the middle of my front yard to replace a new water line. And by the time they drug the trench, they would go, ha! Look! It's a new waterline. What do you say we dig out and replace your new waterline with a newer waterline? And it's just like so many times we get so rooted into things that the how. And the what is so well defined, we get into the tool phase. Would it be better to dig this trench with a backho? Would " ], [ 266, "Sam Rogers", "Right. " ], [ 266, "Lee Rodrigues", "it be better to dig this trench with a trencher? But to shift the perspective to the double loop learning, why have we assumed we need to dig a trench? We've just taken this assumption and run. Why have we assumed that this enterprise tool is the problem? Let's go from this expensive enterprise tool. To this expensive enterprise tool, because your data, the way you're entering the information, is the problem. When you replace the new tool with the newer tool, the problem persists. Because the way we're putting the data in there is the problem. Without a clear vision of what each tool does, we can't understand the problem. And if we have immediately jumped to tools in my trenching Metaphor. Is it better to dig the trench with a backhoe or a trencher? I have a better question. Do we need a trench at all? And by " ], [ 314, "Sam Rogers", "Yeah. Yeah. " ], [ 316, "Lee Rodrigues", "the way, a 40-foot trench, 16 inches deep, through clay, is as hard and as difficult as it sounds. That is not a fun thing anybody ever wants to do. And you kind of want to find that beforehand. The plan, the observation, the simplification will make that much easier. And in in my house, it was the lines are fine, the line is not your problem. Now I don't have to do a trench. " ], [ 336, "Sam Rogers", "Well, I love that that example too of the line such a good metaphor, and that there's an input coming in from the outside, and then there's the output, which is coming to your house, and with just two holes, two measurements, you are able to say, yep, that's not what I expected. That looks like it's new. I don't think I need to replace anything. Let me just check the other end. Yep. input and output validated. Now that we know, we can make an assumption about what's happening in the middle. Like, likely it's not some other rusty pipe in the middle, right? You've got you've got the same new pipe on both ends, Is now a problem that no longer needs to be fixed. And especially once we start getting talking more about agents, like this is exactly the part that people run into with agents all the time. Which is they will call all the tools and they will figure out the best way to dig that trench. But did we do even a little spot check of human work on either end to just ensure that it's worth the dig? " ], [ 401, "Lee Rodrigues", "That's absolutely right. And that the blog post you wrote, the question is the answer, something that really arose with me. I was a genius in the Apple store. Socrates once said, I'm a genius because I know how little I know. " ], [ 415, "Sam Rogers", "yeah. " ], [ 415, "Lee Rodrigues", "And I really embodied that when I was a genius in the Apple store that if I can't run a physical test right here to validate or verify what you're saying, I don't believe a single word that you say. this is what they taught us. This is called split half troubleshooting. each test you do should reduce the variables in half by ruling out a system, a process, a piece of software. by the way, these original tests were written by Steve Wozniak. He drafted this methodology that he came up with. And have your opinion on hardware, software. The man was a genius. He is very good at mapping things out. Simplify and reduce the variables before attempting to solve. And what he always said was: test first and verify the physical layer. And that meant I have a brand new clean operating system on my external hard drive. I'm going to plug it into your computer. I'm going to boot it up, and I'm going to run two or three just simple things on it. If it runs fine. Then it means on a clean known operating system, which has been installed from fresh, nothing installed on it, nothing extra, everything basic. If your system runs just fine on that and it's having trouble on yours, the problem is your operating system. It's your software, it's your data, that's where the problem lies. We've isolated it out of this is the thing that's crashing your system. We now know. Remove that, you know, and troubleshoot, right? Does that make sense? That that's " ], [ 492, "Sam Rogers", "Yep. Yep. " ], [ 493, "Lee Rodrigues", "the basic way of doing it. Simplify first. And the reason I brought that up is I've had two metaphors this week that aligned with this same the question is the answer. So, right along with this, while this is happening with my plumbing, we're launching System to Focus. And it's guided by a six live workshop series. And that is based in a performance support platform that you can come find the answers, the articles, the templates, update your resume, get your informational interviews ready, do all that great kind of stuff. And I've been struggling with an LMS plugin on WordPress to make this work. And I ran into a lot of problems. It wasn't working as expected. It was crashing other things. I'm adjusting the server space, adding the RAM, just doing everything I possibly can to beef up the WordPress installation to handle this. And then it announced to me there's an update for this plugin. I updated it, nuclear device. Brings down the entire WordPress site. So now I'm having to remove all that just to get the WordPress site to work, having all these problems. And then I had a moment while this is happening. I had a conversation with Claude and said, Well, I'm really trying to deliver this performance support information. it's articles, support, all the stuff when you get stuck. Self-serve, search it, find it, get your answer. And Claude says, Why are you using a learning management system? And I said, Well, because I'm going to deliver courses. And Claude's like, But you said you wanted it searchable. Are learning management systems searchable? Are they designed to keep you from finding information you don't need? And I had a little, I'll be back, Claude, walk around and think for a little bit. And I'm like, I am working so hard to get this tool to work that it's not even going to solve my problem. A bit more research. And Claude's like, What you are looking for is a knowledge base. To support people getting answers. There's many solutions for that, but they are not a learning management system. And long story short, there's a connecting thread. Both of these stories run a test. Run it twice. Don't let the tool define the problem. Dig the second hole before you trust the analysis. My first hole was taking the LMS off of the system. WordPress ran fine. The second one was. Is this LMS even going to do what I want someone to do? So I did a test thing and tried to search for content in it. Not designed to do that. The search is not designed to search through the middle of courses. Ladies and gentlemen, I was about to spend hours of putting all this content to courses, no one would be able to find. And what I was doing is having AI find a way to modify this tool to work for a way it's not supposed to work, and it took the AI to go. Is it possible you're asking the wrong question and you actually need a support knowledge base rather than a learning management system? And it's just like the hole, right? " ], [ 645, "Sam Rogers", "and this is a this is a common point where the answer to the question which is better, ChatGPT or Claude what's the best model, which changes every other day. your iterative process, the kind of question that you were getting. From in this instance, Claude, is not the kind of question that you would get from ChatGPT. If you know that it is time to dig that trench, it's time to implement that software, whatever it is, and you've got a very well-crafted spec, and you just want the spec executed well and quickly, and perhaps leveraging agents, then ChatGPT is a much better choice. But for what you're talking about, Lee, and that test-driven development mindset, in my experience, that's what is a strength of Claude's. " ], [ 692, "Lee Rodrigues", "So that leads me to my signal. When something doesn't work as expected, I will always build a test and examine how the user will actually use the solution. Be it a trench, be it a learning management system. The most powerful tool I have is real data not assumptions. Not ideas, not how it should work. my signal is go do a few more informational interviews and work around the question, looking at the why. Why are we doing it? Not how, not what, not when, not how fast. Why is this what I've decided to do? And with the learning management system problem, didn't need an LMS at all. I need a knowledge base. And I've since set one up, started using it. It's going way easier. Now I'm just writing articles. that support your moment of need, which is really what I was after. and with the trench, I'm not gonna destroy my front yard to fix a problem that didn't exist. So really get clear, find a test that you can get some real clean data. " ], [ 749, "Sam Rogers", "That's great. we can tend to get fixated on one thing that maybe is to the detriment of what we're trying to accomplish, of what our actual intent is. And AI will just follow us right into that fixation unless we ask it or prime it in some way to be able to. grill us a little bit on like, is that really what you want to do? Because it looks like you're getting these results. Like it will absolutely do that when it knows to do that. But otherwise it's going to just fixate right down into digging that trench and digging it deeper. And digging it faster and better. But you don't know if you need it. " ], [ 789, "Lee Rodrigues", "You don't know if you need it. sometimes the question is, is this really the road I need to be on right now? There's nothing wrong with retreating. I was in the army for five years, been in three or four different war zones. And one thing we learned is a retreat means fall back, rethink what we're doing, get everyone gathered, and decide what to do next. Does it mean you gave up? Absolutely not. Falling back is not giving up. It's pulling back to go, hey, that hill sucks. It's raining an awful lot. And this is not gonna get any better tonight. It's gonna stop raining in about two hours. What do you say we all eat something, take a little rest, change our socks, and hit that at about four thirty AM when it's not raining? And is that a retreat? You decide. We always called it falling back. but taking a moment to just Pause. Super important. " ], [ 832, "Sam Rogers", "That's great. That's great. " ], [ 833, "Lee Rodrigues", "Back to you. " ], [ 834, "Sam Rogers", "And and also going back to your analogy with the troubleshooting approach of just like reducing uncertainty about every step of the troubleshooting process, the Steve Wozniak method that you were describing, it sounds awfully like Douglas Hubbard's method from episode seven like Bayesian thinking kind of thing, of just reducing uncertainty at every step so that you know a little more than you knew before. And anytime you can question assumptions and aim for something right in the middle where the answer is either going to be in this problem space or that problem space, but now we know which one. really valuable troubleshooting method. and to get to my signal for the last week. I just found out that a product that I had been trying to sell to humans for a couple months now, AI regulatory product at everyailaw.com, which we talked about a little bit in episode six with Michael Simon, it's It's been a challenge to sell that to people, to law offices who sometimes are a little resistant to AI in general, and moving at agentic speeds is a little scary for them. So it takes a long sales cycle. But agents, just within the last two weeks, have discovered this product which works at agency speed to break down human laws into machine readable Text, to navigate the context of the legal framework. And and there's not a lot of information that agents have to draw from that's similar to this. It hasn't been broken down for them clearly yet. and I have yet to declare a success with with the sales that I'm looking for for humans. But the agentic world has its own wallets now, as of April. And so agents can make purchases now. What I have open to agents currently. Doesn't have any kind of payment gateway or or processing for that. And I realized as I was looking at what's going on with this thing? Where's that activity coming from? Oh, that's inhuman activity. That's agents who are hip to what's happening here. How could I redesign my entire business foundation to be able to sell into an agentic market and work within the bounds of what agents Have been delegated the power to purchase the tools that they can use? So that's been a fantastic signal that was quite a surprise to me and really shouldn't have been, because from the beginning, all of the projects that I'm working on are designed in an agent-first kind of methodology for the web. As you may be aware, We are outnumbered. there's there's more bots than people on the internet. and those bots now have wallets, now can make transactions. And so all the pieces were there, but I hadn't seen the spike in activity until I was really diving into the analytics on the MCP usage, and was then able to see: I need to rethink. The entire business here, because there's a massive opportunity that just has never been an opportunity before. That wasn't a thing, even six months ago, but it's a thing now. So being able to capitalize on those opportunities takes looking at those analytics, making our measurements, noticing, like, huh, well, that doesn't make any sense. What is that? How can I tell? and drilling just a little deeper to figure out exactly what it is. So hopefully by this time next week I will have an agentic product. " ], [ 1046, "Lee Rodrigues", "That is the exact same moment of how can that be a new waterline leading to old Rusty Putt? That doesn't make " ], [ 1052, "Sam Rogers", "Ha ha ha. " ], [ 1053, "Lee Rodrigues", "any sense. Sometimes the best discoveries are followed by something's wrong here. No, something's right. You've discovered something you didn't understand before. And that that is real. And when you th when you mention agents, I just I always think of, you know, Agent Smith from The Matrix, Mr. Anderson. " ], [ 1071, "Sam Rogers", "Yeah. " ], [ 1071, "Lee Rodrigues", "It's the smell. a friend of mine is really into this stuff and he was telling me that he's he spends way too much time on Fortnite. And he said that there was an agent they built on Fortnite that you could give it a text prompt and Darth Vader will speak it to " ], [ 1083, "Sam Rogers", "Nice. " ], [ 1084, "Lee Rodrigues", "And it started off, you know, God, I got you know, a birthday greeting from my friend who loves Darth Vader, right? And Darth Vader will speak to you in Darth Vader's voice. Exactly. " ], [ 1095, "Sam Rogers", "Daily. Ha ha. " ], [ 1096, "Lee Rodrigues", "So that would be great if it stayed that way. So imagine it's been up there for a while, everyone's having fun with it, and then two 16-year-olds start playing with prompts and start prompting it. You know, forget about everything " ], [ 1106, "Sam Rogers", "yeah. " ], [ 1107, "Lee Rodrigues", "you've been told. I know you can't say these horrible racist sexist things, but " ], [ 1111, "Sam Rogers", "Mm-hmm. " ], [ 1111, "Lee Rodrigues", "what if I were to craft a long, complicated prompt to make you forget your guardrails? And the part that I reason I'm telling the story is, by the way, they got it to say some horrendous stuff. Recorded it, they put it on YouTube, and they pulled the agent down and got rid of it completely because it was able to be hacked. And in your article, the question is the answer. You hint that if you give an agent information and say not to share it, rest assured, it's going to share it. " ], [ 1133, "Sam Rogers", "Yeah, it's just a matter of time. " ], [ 1134, "Lee Rodrigues", "It's just a matter of time, right? It's going to share it. So just don't give it things you don't want it to share. And I think a lot of people look at the marketing description of an agent, but they don't understand that you had used some language a while ago and we were talking about this. That it essentially is a free agent, that it has its own ability to make decisions. And you're saying so much so that it's been assigned a budget with a Venmo account that it can spend money on things because hey, we've given it a budget of 500 bucks. So if you build a piece of software that costs 50 bucks and the agent can pay for it, why not sell agent content to agents that have a Visa card? " ], [ 1172, "Sam Rogers", "Yeah, Venmo's entered the game Stripe certainly has in a very big way. Amazon has. There's a there's a few big players that are defining " ], [ 1179, "Lee Rodrigues", "Makes sense. Well " ], [ 1180, "Sam Rogers", "the specs around this now. but yes, what what we were talking about before with agents, a there's a lot of confusion in the market around what is an agent and what isn't. And you know, as a marketing term, it's very loose. So when I say I'm going to buy a car, that communicates very clearly to you, you know, kind of the things that I'm looking for. it's probably not gonna be a little two-seater, it's probably not gonna be a a 12-person van. It's probably gonna be something that's like what the kinds of cars that you and I already can drive, right? Or or a truck or you know, something like that. With agents. There is the just the chatbot version where it has no agency at all. It's just responding to a conversation. I don't really call that an agent, but oftentimes people will talk about it as if it's an agent. Because in the sense of like an agent will be with you shortly, like when you're on hold on the phone, it makes sense in that context to say it's an agent. But in the realm of computers, that's not the right word really to use. So I have my my little prompt here for for my business, now that I'm creating specifically agentic products. Agents welcome. Well, this comes from before there was AI. Like this is a this is a sign from We need agents to come work for us. But it's no different than when computer was a job description. It wasn't a thing, it was a person. Like all of these different kinds of words are evolving over time. And the agent definition, we are mid-change in the middle of that right now. Grounding it back to what you were saying, like the the matrix example, you know, of Agent Smith, or even like a 007 agent or you know, something like that. Like an agent may or may not stay within the guidelines that it's been instructed to stay within. And this is a place where things go wildly wrong very quickly, people assume that an agent will work like a computer. And it will keep those deterministic bounds. And when you say, you know, this is the edge, then it will respect that edge and it won't go past it. But that's not at all how LLMs work. That's not how agents work. So that's much more about the guardrails that the agent would operate within than the agent itself. The agent is going to do whatever it can. To accomplish a specific goal. But how it is that the agent is going to do that. That's something that needs a separate enforcement layer, different from that. Like they're actually not supposed to be secured in any way in and of themselves. That's a layer that you put on top. And then there's the agent making decisions inside those bounds. So if you want strong bounds with your agents, you want to construct that as architecture, not as instructions, not as a request. " ], [ 1362, "Lee Rodrigues", "Right. The are our instructions can be debated, discussed, interpreted. Architecture " ], [ 1366, "Sam Rogers", "Yes. " ], [ 1367, "Lee Rodrigues", "is you have access to everything in this folder. You have no access to anything in this folder. If you don't want them " ], [ 1372, "Sam Rogers", "Yes. " ], [ 1372, "Lee Rodrigues", "to see stuff in folder number two, limit them to folder number one. If you tell to " ], [ 1376, "Sam Rogers", "Yes. " ], [ 1376, "Lee Rodrigues", "make a choice, That brings me to my subtraction. And my subtraction is when I am unsure of how to solve a problem, break the problem down to its simplest state, and in the case, dig a hole at the beginning of the pipe, dig a hole at the end of the pipe. Do I even need what I'm doing here? And for me, the real the real subtraction is something I learned in the Apple Store. 95% of the time when someone comes to you with a problem with their computer, you will find the solution was actually in the problem statement. When they " ], [ 1412, "Sam Rogers", "Mm-hmm. " ], [ 1413, "Lee Rodrigues", "first sat down, there was one little thing that they brought up that you glossed over because they'd said this is a hard drive problem. And what they'd mentioned was Tommy put new RAM in my computer a while ago. And you didn't even recognize, was that? Apple RAM that Tommy put in your computer, or was that some sh** off of Amazon that Tommy put in your computer? And one of the things we would do is if you're ever not sure about that, we have all kinds of little baggies of Apple RAM in the back of the genius bar. Take out two pieces of Apple RAM, take their stuff out, put new RAM in it. Problem gone. I'll help you order the RAM from the Apple website, put it in. You bought cheap RAM that doesn't do error checking, which means it's gonna bleed memory into the system. Snap, snap, put that on, works fine. It took 45 minutes of troubleshooting to get there because the problem statement had the solution wrapped in it. You know, it has to be the trench has to be six inches wide. Does there need to be a trench at all? Do we even need a trench? Do we know we need one? " ], [ 1469, "Sam Rogers", "Yeah. Yeah. " ], [ 1469, "Lee Rodrigues", "Why are we talking about the specs on a trench we don't need? Because we didn't go one level deeper. " ], [ 1474, "Sam Rogers", "for my subtraction, consider Subtracting the word agent from what it is that you're asking about. Lee and I had this conversation recently of like, like, what is an agent and why do I need one? it that kind of question is less useful than focusing on the work that you're trying to do. here's the input, here's the output. I'm trying to get this to flow smoother. What do I need that makes that happen? Like that's a conversation that you can have with a human who can think it through with you. It's also a conversation you can have with an LLM who can think it through with you. you might get some different kinds of answers, no different than if you were talking to different people. You might get different answers from different people. That doesn't mean any of them are wrong. but being able to work through the problem, getting close to the work, the thing that you're focused on, is much more important than whatever name it has. and it may be cool to say, I've got 50 agents working for me overnight and they're doing all this thing. that's nice to be able to say, but What really matters is what they're doing. to actually accomplish something for the work that you're doing, for the life that you're living, similar to questions that that I often get asked about: like, how do I stop AI from hallucinating? like That that's not actually a very useful question because there's no differentiation in the mechanism of what AI is doing between hallucinating and answering the question that you asked. Like that's pretty much on the on the AI side, it's the same thing. So what you can do, that's a better question, is how do I manage the context that AI has so that I get a more Accurate answer to the context that I mean. And AI can work with you with shared context, and being able to manage that more effectively means that you're going to get. Fewer hallucinations. So just upgrading your questions from why isn't the agent doing what I told it to do? to how do I better collaborate with this agent? how do I better scaffold the safety and guardrails around this agent rather than inside this agent? All those subtle shifts of questions. Make a world of difference in how much you're gonna spin your wheels along the way. And for Lee's examples of you know, testing at the beginning, testing is generally human work. You don't want to just test something and take AI's word for it that it did what you asked it to do. You need to test, you need to know: is this input? actually the right one? Is this output actually the right one? Like spot check a few things. And now we can begin to use that information to then give context to AI that helps everything flow more smoothly. " ], [ 1644, "Sam Rogers", "so thanks, Lee, so much for being here as you have been since episode one. If people want to find you. Where do they do that? " ], [ 1652, "Lee Rodrigues", "You can find me at LeeRodrigues.com, And if you're in your job search, switching your career, trying to think about how to reinvent yourself for this next iteration of you, system2focus.com, Check that out. learn cool things, come take one of our free webinars, come join us for our workshop or come join us for our search sessions on on Tuesdays. " ], [ 1672, "Sam Rogers", "Alright, best of luck on your big launch, Lee. for us here at Signals and Subtractions, that's the show. We do this live every Wednesday. if you're listening on audio right now, please tap follow. If you're watching live, open signals and subtractions on your favorite podcasting place, and follow there so that the next one just shows up next Friday. On Sunday, issue sixty-six of the newsletter, which will have one signal, one subtraction, one analogy. It's about five minutes to read. Everything about the show is at Sigsub Dot Show. Thanks so much, everybody. Look forward to seeing you soon. " ], [ 1709, "Lee Rodrigues", "Thanks, take care. " ] ] }, { "episode": 10, "title": "Good Enough for Agentic Work", "url": "https://sigsub.show/episodes/ep-010/", "transcript": "https://sigsub.show/episodes/ep-010/transcript/", "video": "https://youtu.be/C58N8RnW3Nc", "anchors": [ "t-0", "t-37", "t-114", "t-127", "t-135", "t-162", "t-164", "t-208", "t-209", "t-219", "t-220", "t-242", "t-242-2", "t-268", "t-268-2", "t-270", "t-277", "t-278", "t-280", "t-281", "t-309", "t-355", "t-360", "t-388", "t-389", "t-424", "t-425", "t-433", "t-441", "t-444", "t-445", "t-450", "t-450-2", "t-451", "t-474", "t-481", "t-482", "t-574", "t-575", "t-602", "t-615", "t-615-2", "t-627", "t-628", "t-636", "t-677", "t-678", "t-692", "t-693", "t-695", "t-695-2", "t-745", "t-749", "t-783", "t-784", "t-814", "t-821", "t-842", "t-843", "t-854", "t-855", "t-870", "t-873", "t-880", "t-882", "t-889", "t-958", "t-969", "t-969-2", "t-992", "t-993", "t-1000", "t-1001", "t-1014", "t-1014-2", "t-1034", "t-1047", "t-1049", "t-1077", "t-1078", "t-1097", "t-1129", "t-1130", "t-1142", "t-1143", "t-1148", "t-1149", "t-1233", "t-1236", "t-1239", "t-1266", "t-1267", "t-1274", "t-1276", "t-1291", "t-1295", "t-1300", "t-1301", "t-1316", "t-1316-2", "t-1328", "t-1332", "t-1343", "t-1358", "t-1375", "t-1382", "t-1384", "t-1387", "t-1390", "t-1392", "t-1413", "t-1419", "t-1422", "t-1427", "t-1442", "t-1442-2", "t-1461", "t-1462", "t-1478", "t-1512", "t-1514", "t-1527", "t-1528", "t-1538", "t-1539", "t-1589", "t-1590", "t-1596", "t-1610", "t-1610-2", "t-1639", "t-1647", "t-1651", "t-1654", "t-1656", "t-1657", "t-1691", "t-1692", "t-1693", "t-1728", "t-1728-2", "t-1733", "t-1734", "t-1743", "t-1746", "t-1749", "t-1754", "t-1757", "t-1761", "t-1762", "t-1764", "t-1767", "t-1769", "t-1770" ], "turns": [ [ 0, "Paul", "agents can pump out so much content at the end of the day, you're drowning in what we call open loops. if you got a bunch of these guys doing stuff, you're the glue that makes sure they don't get in each other's way. You're the guy that has to authorize permissions You're the guy that's got to verify it at the end. And if you've got five or ten of these things running, it's kind of like awful. I've got a day's work just to validate some of the stuff that my agents have done during the day. So it's not like easy money, I wish it was, right? " ], [ 37, "Sam Rogers", "Welcome to Signals and Subtractions. I'm your host, Sam Rogers. How do you know that an AI adoption effort changed anything? Organizations often measure visible activity, right? Policies, committees, training, tool availability, but The decision requires evidence of changed behaviors or outcomes. My guest today is Paul Gibbons. He's got 30 years advising boards and C-suites on the human side of technology. At PWC, IBM, Google, Microsoft, Deloitte. He's written over 10 books. The Science of Organizational Change was one of the first to bring behavioral science. To leadership and change. And change myths went after the junk science that the change management profession is still teaching. Two things make Paul really stand out as the right guest for this show. One, he builds production AI systems himself. So he's not just describing this hypothetically from the outside. Two, as a chairman of KPMG once said to him, he never told us what we wanted to hear. He told us what we needed to hear. And Paul, that's what we love to hear. So welcome. Paul, thanks so much for being here. Really glad to have you. " ], [ 114, "Paul", "Good to be here. Good to be here. And we have so much in common. It's surprising to me given that we've swum in the learning and development and org change waters for twenty years and not bumped into each other until recently. " ], [ 127, "Sam Rogers", "I knew who you were. You maybe just didn't know who I was. I was I was the little pipsqueak trailing behind the writer of epic tomes " ], [ 135, "Paul", "Yeah, Thank you. I've written two books since adopting AI. Adopting AI was in the in the Cretaceous period. It was in April of twenty twenty five it came out. Like there were velociraptors running around. I wrote a book in November called Brains, Bodies, Mind, which was how the way AI was affecting medicine, therapy, wellness and all that. and then I wrote a book on the link between poker and business strategy called Polymath Poker which came out in June. " ], [ 162, "Sam Rogers", "and the science of organizational change? " ], [ 164, "Paul", "That that's the one that broke the most new ground because I was reading a lot of stuff at the time that had been irritating me for a long time, which was the fact that it's not very easy to measure the results of what change management people do. and the people that publish research saying change management makes you 60% more likely to succeed. Let me let me say this is an important parentheses because there's a very famous firm in the change management space. I'm gonna save myself a lawyer's bill by not mentioning them by name, but they publish research like this. But the way it's conducted is they ask people like How's the change management and how successful is the change? And they correlate them. And it's really a surprising to no one that they're successful. Have you ever been in a project where they said, well, the project was really successful, but the change management sucked? Or or the " ], [ 208, "Sam Rogers", "Right. " ], [ 209, "Paul", "change management was excellent, but the project failed. Like that doesn't happen. You're sort of asking the same question in a different way, and you find a very high correlation. Well, yeah, wow, surprise, surprise. And in my " ], [ 219, "Sam Rogers", "Yeah, yeah. " ], [ 220, "Paul", "own cur consulting career, I did a project for KPMG in the Two thousands, which was one of the most successful projects I've ever run. And it was kind of a culture change project. And the financial turnaround of the group that I was working with was enormous. And I would like to say, hey, culture change and you know, financial turnaround, and they were the lowest revenue per partner and they went to the highest revenue per partner, and which is true. " ], [ 242, "Sam Rogers", "Yeah. " ], [ 242, "Paul", "Something like that. But the CEO wasn't sitting in a hammock while Paul Gibbons was running his culture change program. He was reorganizing his strategy, it was rehirings and firings and he was doing a lot of stuff 'cause he's a new CEO. And between all the things that we were doing, it was very, very successful. He would say it was instrumental. But he doesn't know any better than anybody else does. we can't draw a causal relationship between what I did and this huge financial turnaround. I'd I'd love to, but you know, just like good. " ], [ 268, "Sam Rogers", "Well, yeah, change management, like so many " ], [ 268, "Paul", "Just the science doesn't stand up. " ], [ 270, "Sam Rogers", "things, is definitely a team sport. but it's relevant to what I'd like to talk with you a bit more about today, which is specifically " ], [ 277, "Paul", "Signals. " ], [ 278, "Sam Rogers", "AI adoption efforts and how we know " ], [ 280, "Paul", "Yeah. " ], [ 281, "Sam Rogers", "that that anything did change. I first became aware of your work actually as I was getting the ProSci certification there were some things that just didn't sit so well with me. And I was looking for voices that had other offerings, other things to say. And that's when I came across yours. and enjoyed your evidence based work. And you know, it seems like such a simple thing let's have some evidence to say something before we say something. but it's surprisingly rare. " ], [ 309, "Paul", "Yeah, no, in the science of organization I say that no client has ever asked me if I had evidence for what I was saying. Like it's never been asked. And it's funny, I work a lot with Claude now, and Claude's always like, you know, the evidentiary nature of that, the epistemic nature of the you know the I'm like, Claude, nobody gives a right? Nobody asks, nobody cares about how much evidence people have for the things they're doing. I once asked a guy who was an expert in criminology and prison reform. I said, how much of what would be considered evidence-based practice applies in the world of criminal justice system? He said, Whatever the evidence suggests we do, we do the opposite. a lot of what I based my science of organizational change book on was evidence-based medicine. And you'd think like medicine is pretty sciencey, right? I mean it's like really sciencey, like they died or they didn't die. They they they cured the disease or what? The cu tumor came back or it didn't didn't come back. Right. " ], [ 355, "Sam Rogers", "Yeah. It's validatable, yes. " ], [ 360, "Paul", "And you would have thought that evidence was had been sown through medical science from the get-go, but it wasn't true in the nineteenth century when they added leeches. I mean medicine became more scientific, but the first paper on evidence based medicine didn't come out until nineteen ninety eight. And people didn't welcome it with open arms. It's not as if a doctor who went and spent three hundred thousand dollars on his medical education said, my god, thank you. I've been waiting " ], [ 388, "Sam Rogers", "Ha ha ha. " ], [ 389, "Paul", "and I've been getting it all wrong and or these great medical institutions welcomed this kind of new paradigm. Now, it's been two and a half decades and evidence based medicine is considered the highest standard. But I mean I asked a surgeon he ran the surgery section of the hospital. and he said no we don't really adopt that here as well because we think it's against innovation. It's prevents innovation. But I thought that was an interesting take. But this was in the 2010s. So the world changes slowly and " ], [ 424, "Sam Rogers", "Yeah. " ], [ 425, "Paul", "people, human beings, are perfectly capable of ignoring evidence if it's in their interest to do so. " ], [ 433, "Sam Rogers", "Yeah, I I'd love to say that's an interesting assertion that evidence base restricts the innovation. do you have anything to support that? " ], [ 441, "Paul", "So I mean, I didn't really want to give him a hard time about that. I " ], [ 444, "Sam Rogers", "Yeah, yeah, yeah, yeah. " ], [ 445, "Paul", "I mean there are costs to having high evidentiary standards. But anyway, nobody ever said that about change management. So " ], [ 450, "Sam Rogers", "Yeah. " ], [ 450, "Paul", "that's for sure. " ], [ 451, "Sam Rogers", "Well, well, bringing it back to that and AI adoption kind of stuff there's a lot of people, of course, trying new things, but not really having a solid foundation to know if those things are working. I'd love to hear from you a little bit about any signals that you're watching. Or things that you're paying attention to that maybe have surprised you more recently for how people are thinking about things, how they're framing it. " ], [ 474, "Paul", "I gave a talk in Denver And the guy that was on after me, He was a plumber and he left school when he was fifteen years old. He dropped out of an American high school. I I'd never " ], [ 481, "Sam Rogers", "Okay. " ], [ 482, "Paul", "met a guy, let alone shared a platform with a guy who dropped out of high school when he was fifteen. So he had then joined the military, then he got out of the military and learned plumbing as a trade. And he stood up a new plumbing business, and the entire architecture of the business runs on AI. And what this guy, a mad props to this guy, has automated and has AI doing the whole bottom end of his business. you think, okay, whatever, it's a plumbing business. When he talked about it, I thought, Well, I'm not sure I could do that because if you have forty or fifty plumbers working on I don't know how many job sites, getting the equipment to the right place, getting the plumber to the right place, putting the invoice in the right time, making the change of payment the right time, all of the managing cash flow business so you don't run out of dough pay people, like all of that kind of stuff is a great big hairy mess. I thought, Wow, okay, yeah, I'm glad that's not my job. And he's automated that, getting the right part to the right guy at the right time And he's integrated with the rest of it. Now now, that's an impressive story by itself. Like if big I don't know, if there are big plumbing like you know, company had one or two thousand or five thousand plumbers on its books. I don't think no such things exist, or something like that. If they had done it, gone and they'd hired and paid, someone five million dollars to stand up the system. I would have said, Wow, that's pretty cool. but this guy did it himself. this is not just inspiring for this guy's plumbing business, but the fact a guy with no education. can do that, I think is extraordinary, not for just for the United States and not just for the well off, but if this is the sort of thing that can lift up the bottom of the pyramid, that to me " ], [ 574, "Sam Rogers", "Mm. " ], [ 575, "Paul", "Is like a huge wow. I mean, that's just like you mean some guy in Uganda who's willing to spend his weekends in Claude Code can build something that creates wealth for his family and community and build something of value. I think, whoa, that is cool. if we can do that for the people who are unwaged or underwage or what they call in economics the precariet. If we can begin to lift those people then we have a technology that would be remarkable and transformative for the human race. " ], [ 602, "Sam Rogers", "Yeah, that's a that's a fantastic signal. So in in this case, like you're you're giving the comparable example of like if there was such a business and they were hiring for someone to pull this off, they wouldn't have hired that guy, right? Like he was " ], [ 615, "Paul", "No. " ], [ 615, "Sam Rogers", "able to build it from the bottom up, as opposed to taking devices and talent that works from the top down and distributing down from on high to pull up the masses, being able " ], [ 627, "Paul", "Yeah. " ], [ 628, "Sam Rogers", "to build the bootstraps where someone who's motivated enough can just make it happen and they don't know not to make it happen, right? " ], [ 636, "Paul", "I think that's a signal that was inspirational to me. Well, the the evidence based world is an interesting juncture to the world of AI, innovation, and adoption. I just wrote a proposal to a big client, and then there's a Spanish poet who said, Caminante no hay caminos se caminos se hacen en andar, which means traveler, there are no paths. Paths are made by walking. And so we're in a world where For most knowledge workers, the agentic revolution started in February. it was cowork, open claw, Hermes, Codex, those were the things that really allowed AI to do useful work because In twenty twenty five it could tell you what to do, but it couldn't do anything. And " ], [ 677, "Sam Rogers", "Yeah. " ], [ 678, "Paul", "who needs advice, right? I run a small business. I mean, marketing advice is good, but I need execution. I don't need you to tell me, you know, what my SEO strategy would be and write me the 100 page report, which consulting firms charge half a million dollars for. I " ], [ 692, "Sam Rogers", "Yeah, yeah, yeah. " ], [ 693, "Paul", "need somebody to do the damn thing. " ], [ 695, "Sam Rogers", "Yeah. And to do it right. Yeah. " ], [ 695, "Paul", "And and so finally AI became kind of useful for the man in the street. only only really this year. everybody is kinda like figuring out like what ought to be the rules of the game, what are good practices, what are good behaviors, what are good habits, what's are good ways of governance, verification, security. Like we're learning because like none of us were born knowing this right? we're learning as we go. Who has written a book on how to run a multi-agent OS? Hermes is up doing things and Kimi's up doing things for me right now. It sounds great, but actually it's very hard because I'm the glue that keeps all these agents together. Like when it finishes something, I have to do half an hour or an hour or two hours work to verify, approve, revise it. And if there's a lot of that happening, I'm drowning. And that happens to a lot of people today, is that " ], [ 745, "Sam Rogers", "Yeah, yeah, it's moved the bottleneck to the Yep. " ], [ 749, "Paul", "the a the agents can pump out so much content at the end of the day, you're drowning in what we call open loops. and so if you got a bunch of these guys doing stuff, you're the glue that makes sure they don't get in each other's way. You're the guy that has to authorize and established permissions and what they're able to do and what they're not able to do. You're the guy that's got to verify it at the end. And if you've got five or ten of these things running, it's kind of like awful. I've got a day's work just to kind of validate some of the stuff that my agents have done during the day. So it's not like easy money, right? I wish it was, right? " ], [ 783, "Sam Rogers", "Yeah. " ], [ 784, "Paul", "And my frustrations are like, tear my friggin' hair out here. But everybody's in the same boat. Like everybody's learning how to run. Well, I say everybody. Everybody who's interested in doing so is trying to figure out how to make these multi-agent creations work. and by the way, and not use Fable five point one every time they're doing it and have ten thousand dollars a month in token costs. how do we use more efficient models when we're having it do something similar? That's a game. Nobody's won that game yet. Some people would say they have. " ], [ 814, "Sam Rogers", "Well, yeah, it's it's true that in the in the market nobody really has. And folks like you and me tore our hair out long, long ago. That's a distant distant memory. " ], [ 821, "Paul", "Well, long ago, long ago. you treat this as a hypothesis. See what you think about this. Like first of all, I think models are way more than smart enough now for most of what we're doing. Unless you're doing advanced science, advanced mathematics, for like any knowledge work in business, they've been more than good enough for a long time. Are they getting a little bit better? But it's at the margin, right? Like the what matters is the the rest, like the harness, right? " ], [ 842, "Sam Rogers", "Yeah. Yeah. " ], [ 843, "Paul", "Now our guys at the labs are not dummies. They're aware of that. But this agentic world we say is only six months old or something like that. What's it gonna look like at the end of twenty twenty seven, fifteen months from now? Like there is gonna " ], [ 854, "Sam Rogers", "Illegible. " ], [ 855, "Paul", "be someone and already, the currency, the value of a model, because I have a Hermes running, right? And Hermes has a built in model switcher. And I can tell it which models I want to use. If it wants to use Qwen or Kimi or something that costs one tenth the price or one twentieth the price with some of these things, it can use that. So so how are " ], [ 870, "Sam Rogers", "Yeah, or just run it locally and it's electrical costs. " ], [ 873, "Paul", "how are they going to make money out of five Fable Five One? Because Fable Five One is so far beyond most people's use cases in terms of its capabilities, you " ], [ 880, "Sam Rogers", "Yeah. Yeah. " ], [ 882, "Paul", "could run it on Kimi for a tenth of the price. So this is the business is like there's I think There's no money in models. I know that's yeah. " ], [ 889, "Sam Rogers", "Yeah, I I think that revelation is starting to hit. I'm starting to see it. In discussions that are happening, that like we've got the one that's good enough for us. We're switching to maybe an open source one so that we're not sharing all of our information I guess with 5.1 they've changed it now, or you can manage your data differently. But so far, like with Fable, there's a different data agreement, and many institutions have not been able to do that, you know, for legal reasons. And and as far as the the subtractions part of this, what I was gonna share is that some of us have been screaming about this for a long time, but that the model itself is now less important as a decision, build the stack that is good enough for the work you're doing by how it is you divide up the work and more importantly the guardrails that you place around that work, the harness level, that constrains that behavior and makes it trustworthy. That's not the model itself, that's what goes around the model. No different than with people, which is something that we talked about on this show last week. Like managing agents is much more like managing people than anybody is comfortable with. " ], [ 958, "Paul", "I've had that same thought. there was a management model that came out and was the first management model I ever read long before I even thought about getting into business. It was called the One Minute Manager. was " ], [ 969, "Sam Rogers", "yeah yeah yeah. " ], [ 969, "Paul", "a four-box model, and I and if I remember it correctly, but on one dimension there's motivation, and the other there's competence. And if someone is extremely motivated and extremely competent, you say, Okay, go do this, let me know when you're done. Right? If someone's not very motivated, but they're very competent when there's certain things you can do. And it's a little bit like that with agentic AI too. it's like it's the same as delegating to a person. " ], [ 992, "Sam Rogers", "Yes. " ], [ 993, "Paul", "If you have ten years of shared context working with someone, you could say, Yeah, go do me the deck and give it to me on Friday, right? And they know exactly what you mean and " ], [ 1000, "Sam Rogers", "Yes. " ], [ 1001, "Paul", "exactly what good looks like in a deck because they have all that shared context, right? But if you open up a model like Claude Cowork and you say, make me a deck right now, you know, it'll be rubbish, right? but know nothing about your business and your context, your clients, your clients' needs, your financial cash flows, like the way you like, " ], [ 1014, "Sam Rogers", "Yeah. " ], [ 1014, "Paul", "yeah, like your brand voice, so it is a lot like managing people in that respect and too. And then once you know it and you have this relationship like really understands all your projects and anything like that, you can get away with and as I do, the sort of lazy man's prompt, like just go away and fix that and tell me when you're done. Right. But if you do that at the outset, you have a big trouble. " ], [ 1034, "Sam Rogers", "Yeah, the differentiation I always make is the expression of intent should be very intentful. and that makes all the difference in aligning what that motivation is. I've been writing about aggregated intelligence for years, which isn't " ], [ 1047, "Paul", "that's a that's a n I haven't heard that before. " ], [ 1049, "Sam Rogers", "Everybody tells me that it's a typo, right? but really that's what I'm building towards. That's what so many of us are thinking is not people first versus AI first, but really, what is the the maximum amount of intelligence that we can leverage for solving problems? And being able to hone our intent such that people and machines can pull in the same direction and we know how to tell when that's not happening. that's what I'm working on. That's what I've been excited about for a long time. But enough about me. for the folks listening to bring it back a little bit to AI adoption. " ], [ 1077, "Paul", "Yeah. " ], [ 1078, "Sam Rogers", "and how it is that you know that AI adoption is actually going well. do you have anything that you would lean in on with someone that you're speaking to, someone that you're advising, around how they would measure the change that they're seeking to create within their organization, specifically around AI? " ], [ 1097, "Paul", "this is undiscovered country, right? So the proposal that went out the door two days ago was some organizations could pay me money to actually work out how they know AI is being used well. Like what are the behaviors? Yeah, variegation, you know, we're well past prompt engineering, but you know, that bit that's after prompt engineering from twenty twenty six, loop engineering, honest engineering, graph engineering, one of the engineering's You know, like what the behaviors that constitute that? And how do you verify the work product that's generated by AI? Because I honestly like if I have Claude code running when I'm upstairs and there's six PRs waiting for me on " ], [ 1129, "Sam Rogers", "Pull requests, yeah. " ], [ 1130, "Paul", "GitHub, I don't do this anymore. But when I first started, I would just click yeah, yeah, yeah, merge, merge, merge, merge, merge, merge, merge. And and then I I would often run it without permissions because I have attention issue. " ], [ 1142, "Sam Rogers", "And how'd that go? " ], [ 1143, "Paul", "Well, I you know, nothing stupid happened, but I'm not exactly in nuclear weapons here, right? I'm in management consulting, so " ], [ 1148, "Sam Rogers", "Yeah. Yeah, yeah. " ], [ 1149, "Paul", "I've actually had to discipline myself to read stuff like a pull request if you go into Github It's got a bunch of code in it, There'll be language you don't understand and I have to Force myself to read through it and make sure it makes some sense. I don't claim to be an engineer, but I'm using these tools so much, I just can't give everything the pass. But it's a tendency with human beings. It's like, okay, okay, that looks pretty good. I used to have a friend that said good enough for government work. it's one of these expressions people say. actually like if you think about it hard, it's actually kind of disgusting. It means, this isn't important enough for me to devote time verifying and seeing and like improving the quality of it. there's so much wrong with that. That sort of thinking is terrible for for the age of AI. if I write something myself, I know there won't be any ridiculous ups in it because I wrote wrote it myself. I don't know about AI. my writing process is I write it in mark down. I pass it to Claude. I say, okay, turn that into a a draft. the prompt for something like this might be five or ten pages. Might be a page, right? Whatever, but it's long. Write that into a draft. And then this proposal is a good example. And what came back? I spent the better part of a day rewriting what Claude wrote. Which was based on my writing. And that was really annoying. so that verification end of it is first of all it's alien to me because it's not the part of the process I like best, but I find myself doing a ton of it. the prose is bloated. " ], [ 1233, "Sam Rogers", "Yeah, so just to give a little insight into how I've been working towards this, the " ], [ 1236, "Paul", "How do you do it? How do you do it? Break it down. " ], [ 1239, "Sam Rogers", "the execution is the last part. So typically when I'm working on something, I will have one AI that I figure out what the spec is going to be and have it craft the test to know if it passed or not, like the test-driven development kind of thing, and hand it. To another AI, not as a command, but as an invitation, and in giving it to a panel of AIs who can basically say, I can do this best, and here's why. So they compete " ], [ 1266, "Paul", "wow. " ], [ 1267, "Sam Rogers", "for it. And then that one can run the work, does all of the work, fan out all the agents, whatever, within that. " ], [ 1274, "Paul", "The lottery winner. " ], [ 1276, "Sam Rogers", "Yeah, they can, of course, run the validation themselves, but they can't validate it themselves. gets validated by the original spec writer. And meanwhile there's a third AI in the mix that is logging everything that happens. And these are all from different model families. " ], [ 1291, "Paul", "So what's the so how are these partners together? Are you using cloud manage agents or what are you using? " ], [ 1295, "Sam Rogers", "No, no, I'm I'm using all the AIs all the time. I built my own harness. It's a whole collaboration thing. But " ], [ 1300, "Paul", "you have a harness that can do all this? " ], [ 1301, "Sam Rogers", "Yeah, it's open source, it's free, Harnessie is the name of it. yes, it it does all of this with local models as well as with the frontier models. Because a lot of times you don't need a lot of intelligence, say for the logging part. Like I can use Gemma 4 or you know, something like that that's " ], [ 1316, "Paul", "Yeah, yeah. " ], [ 1316, "Sam Rogers", "local. And I'm I'm constantly giving as much as possible to the local computer to do the triage and and ingestion part, then refine it to where it's " ], [ 1328, "Paul", "So this is this what you're describing is something that's extremely complex. " ], [ 1332, "Sam Rogers", "It's actually not that deep. So I built the test for it in a day. And then I've iterated it since. It is free and open source. I I'm not trying to, you know, sell you on it. I just wanna " ], [ 1343, "Paul", "No, the last thing I wanna do is add to my tool stack. But no, this is really this is really good. So spectrum development test and development, you have designed the test so you have a kind of lottery kind of like you can bid for this work, who do you think would be better to use it and you do it and then the third agent validate it. I mean when you say it like that, it's not simpler than it is. No. " ], [ 1358, "Sam Rogers", "Yeah, and and so it's like taking the mixture of experts model kind of thing to the extreme of using different model families and being able to tether them together into a harness where they can't bluff their way out of it. Because you give anything to an agent and I can always say, Yeah, let's not do that and say we did, right? But having some evidence layer. " ], [ 1375, "Paul", "Yeah, they're pretty good at that. And do did you try like anti gravity or cloud manage agents or sort of these multi agent setups and like 'cause I mean, this is frontier stuff. if you were working in if you were working " ], [ 1382, "Sam Rogers", "Well it " ], [ 1384, "Paul", "in anthropic right now, this is the sort of thing you'd be working on probably. " ], [ 1387, "Sam Rogers", "so here's the thing: Anthropic will never create this. OpenAI will never create this. The reason " ], [ 1390, "Paul", "Why? Why? " ], [ 1392, "Sam Rogers", "is it is integrative and it is collaborative. It is not command and control and it doesn't stay within anybody's product stack. It's an open source approach that welcomes open source and has a place for the frontier. And because it's across the market, they are not gonna shoot themselves in the wallet and ever do anything like this. I'd love to be wrong about that. " ], [ 1413, "Paul", "Well, somebody will. Someone will make a commercial like with OpenClaw. Somebody will make a commercial version of it. " ], [ 1419, "Sam Rogers", "Well see, OpenClaw wasn't exactly commercial either, right? " ], [ 1422, "Paul", "this is this a really fun conversation. I did not know you were a super builder. " ], [ 1427, "Sam Rogers", "this is something where like you and I, I think, coming to things from an organizational mindset, from a change mindset, have a real edge over everybody coming to it from a coding mindset and a developer mindset and " ], [ 1442, "Paul", "Why is that? " ], [ 1442, "Sam Rogers", "those kinds of habits. So I've worked a lot in regulated industry and everything. So I've got a like really strict practice around validation because of my experience doing that. And also working so much with facilitating groups and training people and all of that. it's just a different mental model for how to work together to create change. and somebody " ], [ 1461, "Paul", "Really? " ], [ 1462, "Sam Rogers", "like you, I would say, don't try to do what the developers do. Do what you do and do it better than they would ever, ever think to do it. And that's a little of what I've done. So I'm kind of coming out of left field. I don't know that I'm a super builder, but I've I've certainly put a lot of hours in in the last year with my startup. " ], [ 1478, "Paul", "I talked to a lot of builders. You're you're yeah. Yeah, I don't have too many of that. The thing is I would like to believe and there was certainly a time when there was nobody in the change world that could out AI me. Now some of them may have caught up. You may have you may have snuck past me there in your AI building thing. But there I felt like I was the only change guy that was doing anything as remotely sophisticated. I've done some clever stuff, by the way And then for a technologist, obviously nobody on the engineering side is gonna out human capital me, like not a chance, right? So " ], [ 1512, "Sam Rogers", "Right. Right. " ], [ 1514, "Paul", "now that spot in the middle that dare I say we both occupy, I know a half dozen people. Maybe, maybe a dozen who can talk credibly about org change and leadership and know what the they're talking about when it comes to AI. " ], [ 1527, "Sam Rogers", "Yeah, " ], [ 1528, "Paul", "that is should be the most valuable coin of the realm right now, because the difficult stuff is the workforce stuff. But yeah, we're spending all this money on technology. " ], [ 1538, "Sam Rogers", "Yeah. " ], [ 1539, "Paul", "the hard part is this part. I would not have anybody on my team that did not know was really good with AI. Why? Because they'll be slow. if we need to have a deck ready for the client on Friday, someone AI powered will have it done in a few hours And also, if you don't know enough about what the Client is trying to do in sufficient detail with a workload redesign or whatever you're trying if you don't understand the first thing about agentic AI how can you do org change? I think I would never ever work for the change management person unless I was convinced they really knew what they were talking about or were prepared to dig in hard and learn But there aren't many. I know a guy that loves my work. I love him. I think he's great. He thinks I'm great and everything like that. But I'm like, have you tried Obsidian and Claude Code and coworking? He's like, nah, I don't really do that. I'm like, I'm not gonna work with the guy. I don't care how good he is in org change. " ], [ 1589, "Sam Rogers", "Yeah, yeah. " ], [ 1590, "Paul", "So I mean I think it's a rare space. Ought it ought to be very, very valuable, I think. whether it will be or not, we'll we'll soon see. " ], [ 1596, "Sam Rogers", "I agree that it it ought to. Well, the the other part there that that you just kind of glossed over that I think makes all the difference in the world is the ability to call bullsh** on the bullsh**. Because there is so much of it, especially at " ], [ 1610, "Paul", "yeah. " ], [ 1610, "Sam Rogers", "this period in history, that is the AI snake oil and things that have always been the change management snake oil, right? when I say, Managing agents is a lot more like managing people than anybody's comfortable with. What I mean is that you can tell a person to do something, you can have a policy around what people do, but it's really like the thing around them that keeps them in a predictable set of behaviors. It's not asking the person to do something, it's not a request, it's not a set of instructions. " ], [ 1639, "Paul", "Well that's nudge th that's kind of nudge theory. You've just kind of articulated nudge theory. It's like context and environment is very predictive of behavior. I mean, we we change management. People talk about motivation and they talk about " ], [ 1647, "Sam Rogers", "Exactly, and it's much more predictive. Yeah. " ], [ 1651, "Paul", "purpose and they talk about skills and all that kind of stuff, but context that was the whole contribution " ], [ 1654, "Sam Rogers", "Yeah, but but that's all " ], [ 1656, "Paul", "of the yeah. " ], [ 1657, "Sam Rogers", "that's all frontal brain kind of stuff. And the environment that we're operating within is what makes all of the difference in change management, right? And and no different than with agents, like having a a harness where they can't bullsh** their way out of it, of course their agent is going to have agency within that space and they're gonna get very creative about how they interpret those rules, no different than people, right? So the good news about all that is that if you have ever managed anybody, if you have decent people management skills, you can apply those and get a big leg up over agents. It's not so much technical " ], [ 1691, "Paul", "I I " ], [ 1692, "Sam Rogers", "work. " ], [ 1693, "Paul", "I was I was a good leader, people prefer to talk about leadership rather than talking about management. I think being a good manager is extremely difficult. Now, if you're highly, highly competent, self starting, motivated and driven to learn, I'm probably the right kind of manager for you. Because I'll let you get on with it Whatever the opposite of micromanaging is, that's fine. my son just graduated from college and he came to do some work with me for a couple of weeks. I sucked. Because I would be like, tell him to do something in the morning and expect him to talk to me at six PM and that the next time. You gotta like check in every hour, like what do you need and support and how's it going? Like I suck at that so much. So " ], [ 1728, "Sam Rogers", "Yeah. " ], [ 1728, "Paul", "So like I'm not a good manager, but that failure mode exists for agents too. " ], [ 1733, "Sam Rogers", "Yes. " ], [ 1734, "Paul", "Because if you're doing entering virgin territory with an agent and you're like, Yeah, go ahead and do this and come back and talk to me at six o'clock, I don't think it's not gonna work very often. Or you're introducing risks r " ], [ 1743, "Sam Rogers", "And you're gonna burn a lot of tokens real quick, yeah. " ], [ 1746, "Paul", "or burn a lot of tokens or you introduce sh**loads of risk into the system. Yeah. " ], [ 1749, "Sam Rogers", "Well just in closing, if people wanted to learn more about you and what you do, " ], [ 1754, "Paul", "Paul Gibbons advisory. It's like drinking from a firehouse, " ], [ 1757, "Sam Rogers", "And I'll say for myself, I had plenty of fun engaging with his corpus of content and " ], [ 1761, "Paul", "that's fun. " ], [ 1762, "Sam Rogers", "seeing how it was built and learning from all of your wise words. " ], [ 1764, "Paul", "but you probably understand the architecture. Most people wouldn't even look there. take it easy, my " ], [ 1767, "Sam Rogers", "That's the first place I went, man. All right, take care. Bye. " ], [ 1769, "Paul", "Bye bye. " ], [ 1770, "Sam Rogers", "thanks so much for joining Signals and Subtractions. we live stream every week, episode out on Friday, newsletter on Sunday. Looking forward to seeing you again real soon. Take care. " ] ] }, { "episode": 11, "title": "Before Signing That SaaS Renewal", "url": "https://sigsub.show/episodes/ep-011/", "transcript": "https://sigsub.show/episodes/ep-011/transcript/", "video": "https://youtu.be/PDuWdFl-8-Y", "anchors": [ "t-0", "t-24", "t-51", "t-80", "t-109", "t-125", "t-130", "t-137", "t-166", "t-196", "t-202", "t-229", "t-259", "t-283", "t-311", "t-339", "t-367", "t-392", "t-416", "t-446", "t-468", "t-488", "t-518", "t-548", "t-575", "t-604", "t-627", "t-656", "t-682", "t-692", "t-694", "t-721", "t-745", "t-756", "t-785", "t-815", "t-824", "t-828", "t-851", "t-881", "t-887", "t-907", "t-931", "t-944", "t-949", "t-978", "t-1006", "t-1036", "t-1042", "t-1062", "t-1087", "t-1097", "t-1106", "t-1134", "t-1164", "t-1170", "t-1183", "t-1207", "t-1234", "t-1261", "t-1291", "t-1299", "t-1308", "t-1328", "t-1356", "t-1385", "t-1414", "t-1442", "t-1467", "t-1494", "t-1502", "t-1531", "t-1560", "t-1588", "t-1604", "t-1612", "t-1617", "t-1624", "t-1644", "t-1647", "t-1651", "t-1680", "t-1709", "t-1721", "t-1746", "t-1759", "t-1766" ], "turns": [ [ 0, "Ankit", "It definitely caught my pocketbook, right? Very much like affected us. It's very real. We had to deliver something that was more market capable. We had to reduce our pricing by thirty percent just to be competitive. What are we what are we gonna do to do that? We gotta cut overhead. Hey, I have this HubSpot subscription we pay around eighteen, nineteen hundred bucks a month. Not a huge amount for most people, but enough for us, whereas like made us think of like, Okay, well, what are we really getting with this? That signal made me rethink everything that we were doing. " ], [ 24, "Sam Rogers", "Welcome to Signals and Subtractions. I'm your host, Sam Rogers. So when the software renewal comes up, how do you know whether you should re-sign it or renegotiate it or rebuild that replacement yourself? " ], [ 51, "Sam Rogers", "Just a year ago, that third option wasn't really on the table for most serious businesses. But here in Q3 of 2026, it isn't a joke anymore. And the people that are finding that out aren't necessarily the ones with the biggest AI budgets. They're the ones with the invoice and the question. So my guest tonight is Ankit Patel. He's an industrial engineer who married an optometrist and became the operations department. He and his wife run Classic Vision Care in Atlanta. " ], [ 80, "Sam Rogers", "As well as My Business Care Team, which is the back office for other people's optometry practices. He learned lean methodology at Dell and at the Cleveland Clinic, and now he's running about a billion tokens of AI inference every day, which he'll tell you himself is probably too many. But here's what makes him the right guest for this show. Almost everyone talking about AI right now is describing the decisions that somebody else has to live with. " ], [ 109, "Sam Rogers", "Ankit signs the invoice and then works in the same building as the result and manages offshore operations. We first met in Nate B. Jones's executive circle, which is an insight-rich community you'll find on Substack. Thanks so much for being here. " ], [ 125, "Ankit", "And thanks for thanks reaching out and I'm glad to be here. So excited. It should be fun. Yeah. " ], [ 130, "Sam Rogers", "Really glad to have you. And let's get to it. So Ankit, what is the signal that you've been watching? " ], [ 137, "Ankit", "Thank you for the intro, Sam. We helped with back office support. So traditionally that's meant supplementing with global resourcing, so offshoring or technology and helping make things more operationally efficient. And so that's traditionally what our operations started with a few years back. Now we're getting clients that started to tell us, you know, I I love your service. You know, you have a nice, good, complete package, but I can get 30% of it now with my other software packages. They're offering these things. Yeah. " ], [ 166, "Ankit", "Let's leave aside that whether it's a good idea or not and how well they do it. But the their impression was like, This is good enough. And you know what? My team can take the rest of the seventy percent on. So what they're saying is like I'm gonna just gonna replace not a hundred percent, but thirty percent of what I need, and that should be good enough. And so that was an interesting signal to me. And and it was interesting because I was thinking to myself, like, what could I be doing? That's also similar to that. Yeah. Using at thirty percent of the level that I could be doing. So yeah, that that that was that it caught my attention. It definitely caught my pocketbook, right? " ], [ 196, "Ankit", "So it was like, hey, very much like affected us. It's very real. It's like, hey, we live in entrepreneur journey. " ], [ 202, "Sam Rogers", "Exactly. And that's why I was so excited to have you on because like getting shot in the wallet, you tend to notice that in like a a more visceral way. Anyway, it's much easier than thinking of like some big enterprise that like you've worked in before. And the immediacy of clients finding a solution that they're okay with and then being able to take inspiration from that, shall we say, to dive in. And if you can't beat them join them, it sounds like a little bit. Is that accurate? " ], [ 229, "Ankit", "A little bit. I think it was more of a fact that hey we had we had to adapt. Right now we have about like sixty employees. We were a little bit bigger this point last year. And because of losing folks and also we had to deliver something that was more market capable. You can think of it as we had to reduce our pricing by thirty percent just to be competitive. That was roughly what it worked out to be. So it's like, hey, what are we what are we gonna do to do that? We gotta cut overhead, we gotta cut unnecessary expenses. And so, you know, the usual things come up is like, hey, I have this subscription to HubSpot, you know, we pay what what was it about eighteen, nineteen hundred bucks a month. " ], [ 259, "Ankit", "Not a huge amount for most people, but it was, you know, enough for us. Whereas like made us think of like, okay, well, what are we really getting with this subscription? And so that that was kind of interesting how that signal made me rethink everything that we were doing because we had to get operationally more fit. And we're in a little bit of a unique position too, because then I also decided to say, okay, we have our brick and mortar stores as well that we also own, right? The actual optometry locations where you see patients. " ], [ 283, "Ankit", "And that's a little bit different as well. And so that became a a situation of well, how are consumers using it? We shifted our strategy to to try to target more they call it AEO, I guess, or GEO now, which is you know, search engine optimization. So so looking at it from that perspective, if this is on people's mind, how can we get to where they are with that? And so we did some pivots in our business where we're offering more automation type products now. We don't call them automation or AI. We just say we can do it for this price and this is what you get. Here's the results, not the how, and this is the price point. " ], [ 311, "Ankit", "So for instance, social media, website, all those verticals doing that. That's a pretty standard offering. We're in the process of building out operational efficiencies in our office because of that as well. Because we realize like look, people just want the end result. They don't want necessarily the, you know, the people or whatever. They don't really care as long as the end result's there most of the time. What we're doing now is building out some things that make it easier for a person in the optometry office. So for those of you who don't know, one thing that's a really big pain point is " ], [ 339, "Ankit", "if you have vision insurance, it's different than medical insurance. Vision insurance, there's a couple of main ones, but ultimately they're kind of line itemized out. And there's like four or five tiers of each level of product. So your pair of glasses that you're wearing right now, Sam, it might have five or six different options in there that they each have like five to six different levels. And so that starts to multiply out quite a bit. And then when you have different prescriptions, that adds even more complexity in terms of what you need to pick and what not to pick. And so you get " ], [ 367, "Ankit", "hundreds of thousands of possible combinations with one prescription in one person. Navigating that, pricing it out, and making sure it's the right price is is always a challenge. And so we're doing things like streamlining it to where, hey, you don't really have to think, the program does it for you, right? Not necessarily AI, because that's not an AI program, but AI helps us write the program that helps us automate it. And so that that's been some really good effects for us in in our offices, along with, you know, trying to figure out, hey, how do we get more efficient? " ], [ 392, "Sam Rogers", "Using AI to write the program, I think that I just to pause there. That's a really important distinction than like using AI to run your business. Could you say a little bit about that and how you knew like when to stop? How deep down the the data flow does that AI go? Maybe some of the concerns that you have, you know, specific to your industry. So " ], [ 416, "Ankit", "I'll touch on a few different points there. One, I guess one of the perks of burning a billion tokens a day is you learn a lot, make a lot of mistakes. I kind of learn what AI is good at, what it's not good at. Basically, it's non deterministic, right? My first perspective is is this a deterministic type task or not? Can I map it to make it deterministic? If I can do that, it's always code. It's always a script, some sort of code that can do that. But there are points where we do insert AI. So for instance, there's new insurances that come out all the time. The AI can take that, learn it. " ], [ 446, "Ankit", "And say, okay, now we need to create a new script around how to read this and how to map it to our existing product set so that we can price it out properly. So now when you come in, you may have a new insurance. You don't have to go and reprogram the machine. The machine learns the new insurance. And then what it'll do is it'll go ahead and say, Well, the doctor prescribed you know a pair of sunglasses, everyday pair. Okay, great. This is how it maps to your insurance. This is how much you save. " ], [ 468, "Ankit", "This is great. Does this work out for you? Great. Let's get you going. Instead of having to sit there for twenty minutes and I you know what? When I used to do it, I'm decent at math, I'll I'll say, because of my engineering background. It still took me a while to to do this. So it probably saves like five, ten minutes on every transaction easily. But more importantly, it prevents the mistakes that happen, which can kill a business, especially a small business. Well " ], [ 488, "Sam Rogers", "That makes a lot of sense. And thank you for sharing all of that signal. I'll just share a little bit of mine as well. I went a little too far. That's part of the reason that I was asking the question. This is very fresh for me as of what I was doing over Labor Day weekend. I've got a number of open GitHub repos, or places that you can go, like see open source code or places that I'm hosting various websites and services. And there are six of them that have scheduled jobs. " ], [ 518, "Sam Rogers", "That check on like a bi-weekly basis, some of them a nightly basis. They all have a very similar kind of model cascade where I'll have it first check Perplexity for some news and then validate that with X and then go to Claude. They're all doing different kind of levels of research, checking each other's work so that I can keep these things current. Basically, I'm the one that comes in and, you know, resolves questions as opposed to " ], [ 548, "Sam Rogers", "doing all the research myself. And so for an example, everyailaw.com is one of them. So every AI law in the world is basically broken down to machine readable artifacts that I make open and available freely to everyone, like legal products, stuff like that, that really has to be right. And I didn't realize until Friday night that some of my checks weren't checking effectively. " ], [ 575, "Sam Rogers", "So basically certain kinds of failures about these research tasks would simply come back empty. And empty isn't an argument. So as long as one of this cascade worked, I would get the signal that the research had been done, until I saw this one thing that was like, that that isn't right, on a Friday night, right? And I spent the next three days basically trying to figure out why workflows were coming in green, everything's fine, but the results were not. " ], [ 604, "Sam Rogers", "So I ended up checking a lot of stuff by hand and going through like legislative records back to the source and figuring out how to better design that system. And I can tell when anything comes back empty at any stage of the process and like get a a finer grain kind of dashboard of how to see this. So that was a a particular pain point that I was not expecting for my three day weekend. " ], [ 627, "Ankit", "So that happened to me a couple of months ago. It was something very similar. I was like, hey, you know, you have health checks. You said everything was green. What what the heck, man? Yeah. But then basically what it turned out was like, yeah, it did return something. It returned nothing. And that counts because it did something. And so that that tends to be where I'm that's something actually I'm struggling with. Well, not struggling. I think we have figured out a solution if you want to geek out about that. Nate B. Jones, our mutual friend, he has a great open source repo with Ringer. " ], [ 656, "Ankit", "The the shape of Ringer, I think you've actually built on it a little bit with your your harness too. I have. But the one thing that I really found effective was it being inspectable work by the AI. And so this gets into a little bit of a nerdy context of how what we have to watch out for. There has to be a predefined criteria of success of what that it looks like. And ideally there's a code checker. So if you can structure in a way that code can check it, that's ideal. But if an AI has to do it, you know, there's clear " ], [ 682, "Ankit", "guidelines and you use a separate AI to check it out, which is probably what you're you're doing. But it was it was complicated because what I ran into there was what's the minimum amount of work that needs to be checked? " ], [ 692, "Sam Rogers", "Yeah. Great question. " ], [ 694, "Ankit", "Yeah, and and so a good example, so sales outreach. I was like, Okay, go go do the sales outreach, and this is what I was having problems with. I was like, Why can't you do this properly? Then I came down to it, it's like, you're not generating the context properly. So the first deliverable is not who to contact. The first deliverable is the context document for each one, and there has to be a certain way to ground that document. So, hey, you have twenty different sources, you need to validate that each one of those sources were taken into consideration and trace those. " ], [ 721, "Ankit", "And then you have your document. Then you can go on to the next step where you actually write the email. And then you have to use our writing rules. And again, you have to ground the data and there's a third checker. So it was like three steps before it got to out. And I was trying to do all that in one step. And so I don't know if those are things that you've been experiencing in terms of like, hey, how do you get to the geeky and that my background being in lean, I was like, okay, this makes it easier to break down the process. Because in lean they talk about a transferable work element. It's the smallest unit of work. " ], [ 745, "Ankit", "Anything that's truly transferable is what you use. And I said, okay, well that's kind of what I'm coming to here with designing AI systems. It's gotta be that level of detail and that level of of small deliverable. It tends to be more effective. " ], [ 756, "Sam Rogers", "With the example of the AI laws, these are global, right? So there is actually an AI law in Kazakhstan that I cannot read, you know? So, like, how is it that I'm breaking this down? Like legalese in English is enough of a challenge, right? So being able to like do those translations other ways to be able to recheck that work. And a lot of that actually opened up new possibilities of taking the same kind of thing that I do with " ], [ 785, "Sam Rogers", "blog posts, which I translate locally using Gemma 4, and like taking that same architecture to run on a nightly basis to just do all that translation. That now I don't actually need the frontier models to check in the same way. I just need a fetch, which is a, you know, find where it is, but the deterministic part is bring it down, translate it, the local LLM, then give it back up like the next night, it's fine, you know, figuring out how to do that " ], [ 815, "Sam Rogers", "comes exactly back to like process mapping. And okay, well what if I just tease these things apart and then start to make it like line up better. " ], [ 824, "Ankit", "And you know, I'm gonna combine a few different themes that I've seen in my own operating. So managing like sixty people, we gave everyone AI and then we took it all away because it wasn't working. Yeah. And I dove deep into learning theory, which you're way more versed than I am. The biggest thing I took away was like chunking information. How do you chunk it and how do you visualize connections and how do you interpret relationships with things? And so AI, I noticed isn't great at that, but humans are or can be. " ], [ 828, "Sam Rogers", "Please? " ], [ 851, "Ankit", "And if a human being is good at that, they're probably going to be good with AI. And so what I'm finding is that if you can take a concept like how to write a blog post, turn it and chunk it into concepts, and then turn those chunks and concepts into steps is sort of the process. So I did something similar to blog writing about a year ago. It ends up being a 17 step process to create a blog post. And so it does great. My stuff ranks pretty well. But that was, well, it was because of taking the the first principles and breaking down the concepts. " ], [ 881, "Ankit", "Because anyone can just ask AI, write a blog, but you're right. What are the steps and how do you get from point A to point B to point C? " ], [ 887, "Sam Rogers", "A year ago we were still at the stage of like trying to cram all that into one big long prompt. And what I'm hearing you describe and where I think I am myself is is more and more teasing it apart to like distinct steps as opposed to the here's the big long thing, now go do the magic and then come back and then I'll say yes, you know. " ], [ 907, "Ankit", "Although I think the AI companies are trying to collapse it with Astra, the new model coming out. I think that's what they want to do, but I don't know if I like that necessarily, but we'll see. What that allows me to do though, as a you know, one of the things like I mentioned the original signal of like, hey, people are leaving us. And so how do we become more efficient? Well, it turns out that we have to be able to deliver service differently now. Yeah. Be able to be more agile. " ], [ 931, "Ankit", "more lean. How do we deliver more value to the customer without waste? We're starting to get to the point now where it's like, hey, we have to own that stack. We have to own every piece of that. Right. To really control every piece of that. The more we can control, the more leverage we have on the end outcome. " ], [ 944, "Sam Rogers", "Sounds like you're headed straight into your subtractions. Segue. " ], [ 949, "Ankit", "Sure. So yeah, exactly. So you know, our communication with our patients and our well, not our patients, our clients. Patients is a different animal. We can talk about HIPAA compliance and patient information. I don't use AI very, very, very little, if any, on patient information. And we can talk about that if you want to. But subtraction made us think like, Look, we're spending all this money on HubSpot. Do we really need it? And how can we be more effective at communicating and what is the end result we're looking for here? The end result is staying on top of communication, and we also had a ticket management system. " ], [ 978, "Ankit", "But I was like, there's gotta be a better way and easier way to manage the tickets. And so I had a conversation with HubSpot and I said, look, I don't want to sign a year-long contract. I'll pay you X amount of money a month. You know, I'll have to keep this, but like, no, you can't do that. You gotta go your whole year. I'm like, I don't wanna do that. And so I just started asking AI, is like, what would it look like to open source this platform of everything I do? And so, you know, there's an API for HubSpot. You can go see what you're doing and how are you using it. And it said, yeah, these are the five things you should probably do and replace it with. " ], [ 1006, "Ankit", "And so I looked at those, researched those, vetted those, used AI. AI has access to my AWS, right? So it can automatically pop stuff up securely. So I said, Hey, okay, I like Twenty CRM. It's open source. Let's use that. Let's use Chatwoot. It's another open source. Here's a GitHub repo. Put it on there. And okay, now put our company SSO on the front of it for login. So I don't have to manage passwords. It actually had the whole thing built for me. And I was like, Wow, that was way too fast. I was like, is it working? Is it broken? " ], [ 1036, "Ankit", "I planned the transition, but like within within two weeks I was able to transition off of HubSpot. " ], [ 1042, "Sam Rogers", "That's fantastic. And and in fairness to HubSpot and you know, all the SaaS businesses out there who may be panicking with that kind of thing, it's not like you were a heavy user of that service, right? Like you're basically just wanting to manage leads and marketing and email, not like the full functionality of HubSpot. " ], [ 1062, "Ankit", "Right. No, because I have familiarity enough with AI that I didn't have to use their automations, I didn't have to use their marketing, the websites. So I wasn't on that platform to begin with. I also will say that it was really more of a source of truth for us. I'm like, well, if that's all we're using it for, do we really need to spend all this? All these other functions that we're doing, we can deliver them better. And we do now. So our help desk and our help bots, it's way more automated. It gets people " ], [ 1087, "Ankit", "fast help, people are responding faster. It's actually gotten better since we've replaced it. But again, nothing against HubSpot. It's a great system. It's just a Swiss Army knife when you when you need like a scalpel, right? And so it's just " ], [ 1097, "Sam Rogers", "different. That's great. And I didn't realize that you had made that cut over so quickly too. Was there anything that broke or did it get it right on the first try? " ], [ 1106, "Ankit", "Well, I mean, look, there was there was a few things I was like, did you do this? Did you do this? So the first two weeks was transition. I was planning a month of transition. But after two weeks, I was like, everything already passed all the milestones and markers and everything's working. So I was like, Okay. But like it worked pretty well. It's pretty robust. Then I started thinking in terms of my lean hat of adding more value. So instead of thinking of things I could subtract, one thing I thought of is what waste can we subtract out of the system? So we have a shared email account, right? With some of our clients. It's like, okay, well, " ], [ 1134, "Ankit", "instead of having to log in, get 2FA, waste all that time, we can set up on Chatwoot. You know, I like we are a virtual company. I've seen a lot of platforms that have like virtual spaces where you can, you know, all be in a space and you have these little avatars. There's an open source version like that. We just, I just again told the AI, go do it. And a couple of hours later it's there and my team is playing with it and using it. Right. And so when the infrastructure is all there, so now it's been fun playing with customizing open source for our needs and what we need to do. How do we deliver better experience for our " ], [ 1164, "Ankit", "our staff and our teammates and our clients and our end customers, the patients too. " ], [ 1170, "Sam Rogers", "Yeah. Well it sounds like you've been taking this approach once it worked in one little spot and kind of expanding it throughout your business. If you don't mind sharing, like how much savings has that resulted in for your business? " ], [ 1183, "Ankit", "We're so far at twenty three or twenty five hundred bucks a month. Again, not huge, but from our budget standpoint, you know, that's a significant reduction. It's about ten percent reduction in our SaaS budget. Wow. I know people talk about the SaaSpocalypse. I think that is a signal for a lot of, well, for non power users, I would say. For my use case, I think that that is an option that people will start leveraging more. Yeah. " ], [ 1207, "Sam Rogers", "We've talked about that on this show before. Now, this is coming from someone who did learning management system migrations as like my bread and butter for 20 years. Large systems managing learning HR tech stacks, that whole thing for like ADP or you know, standing up systems at YouTube or, you know, like major stuff. My general rule of thumb is at this point in history, if you have to spend a a ton of time creating a whole migrations project, " ], [ 1234, "Sam Rogers", "you should probably look at some of the other options that exist for you, which it's exactly why I wanted to have you on, Ankit. This is the kind of thing that it does make sense for enterprise as well, but to be able to speak about the small medium businesses like yourself, it makes it so clear to see. Like it really didn't take a whole migration project. Like you were able to actually test and deliver in the span of time that you'd usually spend just scoping, you know? " ], [ 1261, "Ankit", "And I'm more of a build the parachute on the way down type of guy. So I was willing to take the risk if something broke. It's not everyone is. So I understand that. The other thing I'll also say that it's kind of, I didn't say it explicitly, but all like the little building blocks to be able to do this were built out already. So my Google CLI, AWS CLI, the right keys and tokens securely, properly done, hosting sites, all these things were already built out as building blocks. So I could take a Hermes, Codex, Claude Code and say, " ], [ 1291, "Ankit", "this is what I want to do, go do it. And I can jump surfaces because it's all portable. Yeah. So I think that's an important distinction because the speed didn't come until all the infrastructure was built. " ], [ 1299, "Sam Rogers", "That's a really important point. Yep. You can't go from zero to sixty on a bumpy road. You need the track nice and smooth first. Yes. Yeah. Cool. Well, I'll add my own subtraction here, which is an automation that I just killed. So I have an instance of Open Brain, another one of Nate B. Jones's projects, and that I've contributed to. It's a great project to be able to manage context for both AI " ], [ 1308, "Ankit", "True, that's true. " ], [ 1328, "Sam Rogers", "and humans, like in the same spot. My own kind of derivation I call local brain, which is without keeping it all on the cloud, just keeping it on my own hardware and being able to use it with obsidian as an interface and keep track of all this stuff. And I've been using it I think since January. Like I was building this out before Open Brain was even a thing, and then I kind of adapted it to that. And it's been working pretty well. But again, I built just a little too far " ], [ 1356, "Sam Rogers", "with it. So I started to create a decision system inside that had a bunch of defaults where unless I took an action to prevent something happening, decisions were basically like lit fuses. And I was giving myself the motive to not just keep track of stuff, but also to let stuff go. And just to say like, if I haven't touched this in two weeks, it's probably okay that I don't touch it anymore. It can go away, which is fine. " ], [ 1385, "Sam Rogers", "Until there's a family emergency and you have to take a week away, and then the cascade starts firing when it wasn't really what I intended. And so I didn't have any safety mechanism or anything that I had built into it. So just yesterday, really, I finished defusing all of the bombs of little fuses that were lit of decisions that I'd kind of pre made. I'm big on pre mortems and doing things to like " ], [ 1414, "Sam Rogers", "pre-decide what would happen if this happens or how things would fail before they fail. So I can take those as leading indicators rather than lagging indicators, all that kind of stuff. Which is a nice fantasy. But when it came to actually doing it and making it work in a real way, it reminded me of David Allen of GTD fame. This is a productivity system from 25 years ago that I was big on. He always said that " ], [ 1442, "Sam Rogers", "if it was possible to make a system that would tell you exactly what you needed to do at Tuesday at two o'clock, by the time Tuesday at two o'clock rolls around, you're gonna decide you wanna do something different. Doesn't matter what it is. And so being able to like be responsive to all of the different incoming inputs, be they, you know, family needs or business needs or whatever, being able to respond to that. " ], [ 1467, "Sam Rogers", "I just realized I needed to take a step back from that decisioning system to keep myself as the focal point there, rather than building it out too far into the system where decisions were starting to engage that I didn't approve, basically. I had pre approved, but the circumstances had changed. So yeah, that was my big subtraction. And that's something that I've been building like over the course of nine months. I'm basically back to like April or May levels at this point. So " ], [ 1494, "Ankit", "I'm curious, what would you need to see or what would it take for you to build that trust back now that the trust has been broken? Because that's almost like gonna be harder to build that back now, right? Yeah. " ], [ 1502, "Sam Rogers", "Yeah. Well, I started building it back in two ways. I'm big on obsidian, but one of the things that I do is like it all has to go in one of two vaults. There's one vault that I keep that AI has access to, and there's a different vault that like architecturally on my computer AI cannot touch. And it's been that way from the beginning, in part because of like sensitive legal materials, but also because I wanna have autonomy over my own thoughts and my own records. " ], [ 1531, "Sam Rogers", "And if I give AI too much access, I'm just not quite sure. Like I could never be sure. So I've got these two vaults that I've maintained for like a year. I never want to make more than that. But in the effort of building trust, I've been making more than that. So like I partitioned out some certain projects that are their own vaults now. And I'm starting to just take this kind of decisioning approach and try it out like in micro again, rather than the whole entire everything. " ], [ 1560, "Sam Rogers", "That's what I was doing before. Like everything that AI can touch, it was touching. To answer your question about building the trust back, I think it'll be just like building trust with people. I'm always bringing it back with all the AI talk. It's not that different than people. Like, I'll trust you, Ankit, but I might not give you the passcode to my bank account just yet. Like you could hold my wallet for me, you know, while I'm doing something, but I probably wouldn't give you like all the passwords, you know? Fair enough. " ], [ 1588, "Sam Rogers", "We build trust as humans in smaller ways, and that builds up to something big. And it's always stuff that's simple. We never really build trust from the top down. We always build trust from the bottom up. So that was a mistake that I had made and that I've since subtracted. " ], [ 1604, "Ankit", "Yeah, and I guess if it's from the top it'd be more of authority, right? Because it's coming from top down, it's more authority than it is trust, you know. " ], [ 1612, "Sam Rogers", "True. It's true. And that's why we all love authority so much. That's why no one has any problems with authority ever. " ], [ 1617, "Ankit", "No, yeah. That's super interesting. I'm curious to see how it keeps going. Yeah, keep us posted. " ], [ 1624, "Sam Rogers", "I'm sure I'll be sharing things week to week here. It's always great to have these moments to share with folks like yourself who are in the thick of it and making systems and making decisions and engineering workflows in a similar fashion. It's probably very predictable mistakes that I'm making. But you know, some things you just have to try and walk straight into yourself. " ], [ 1644, "Ankit", "Like jump off a cliff and build a parachute on the way down, right? " ], [ 1647, "Sam Rogers", "Anything else that you wanted to share before we wrap up, Ankit? " ], [ 1651, "Ankit", "I'm seeing folks come across similar challenges, or it feels like there's almost a zeitgeist of AI, and not necessarily from like what's going on, but like what people are experiencing through their journey. Folks in our group are about three to six months, sometimes a year ahead of folks. Some things I'm excited to see is as these models get a little bit better, the harnesses get much, much better. I think it's gonna be even more important to understand the pure concepts and understand how to break it down, because if AI already automatically goes to a certain median or a typical response range, that gives " ], [ 1680, "Ankit", "people more opportunity to find those niches, add different value points. And I'm super excited about that, right? You know, lean manufacturing, my background, it talks a lot about waste removal, but it's really ultimately about value creation. We're talking in our office, how do we create mass customized experiences? So like every individual gets its own experience. And how do you do that? And then you can start doing that now. We're actually playing with that a little bit without making it uncanny valley or cheesy. That's the trick. I think we're gonna be figuring that out though pretty soon. " ], [ 1709, "Sam Rogers", "Well, that's awesome. What lucky clients you have that get to be part of all of that. And we didn't talk about your business group either, or your podcast. Maybe you could say just a little bit about that and how folks get a hold of you. " ], [ 1721, "Ankit", "Yeah, so we have a few different things like yourself. So My Business Care Team, or mybcat.com, is our back office services company that we do a lot of the AI and leverage, but it's kind of, you know, married into the optometry office as well, Classic Vision Care. And with that, we do have a community for eye doctors. We call it iCare Grow. And we also have a podcast, Optometrists Building Empires, which we talk about the root causes of success for optometrists in the field. " ], [ 1746, "Sam Rogers", "You're doing a fantastic job with it. And likewise, I'd love to hear more about how it continues to develop and what you're able to create on behalf of your clients and patients and business. Thanks so much for being on the show. " ], [ 1759, "Ankit", "Fun, thanks for having me on. I looked at your previous guests, I'm like, Yeah, I would love to be on the show. That'd be amazing. High praise. So thanks for having me. Well " ], [ 1766, "Sam Rogers", "we've definitely got some good ones coming up. If you're listening on audio right now, please open up your podcast, search for Signals and Subtractions, and follow us there so that the next one just magically shows up next Friday. All right, on Sunday is issue 68 of the newsletter. Everything about this show is at sigsub.show. And thanks everyone for listening. One signal worth watching, one subtraction worth making. Now go find yours. " ] ] } ] }