AI Detection and the Presumption of Guilt

0:00 Limited Edition Jonathan

Substack has a button that you can click on now that tells you whether or not, the post is generated by AI or assisted by AI or written by AI, which is just a stigma-generating machine. And, uh, that's gross. I want people engaging with my ideas.

I don't want people engaging with my process

0:28 Sam Rogers

Welcome to Signals and Subtractions. I'm your host, Sam Rogers. Last week, Substack shipped an AI detector, which means a writer can now be scanned by a machine, labeled by that machine and how it recognizes its own, and the writer is invited to file an appeal, but only after the fact. Today's guest is my friend Jonathan Edwards, AKA Limited Edition Jonathan.

He's a filmmaker who teaches AI fluency, often by documenting his own failures in public, and who built AI cred to find out whether anyone's actually AI fluent. He's one of the top 10 rising voices in technology on Substack, and this week he wrote a piece called Nothing Was in the Water his position is whether the AI label is accurate is beside the point.

The label itself is doing harm. We live streamed this on Wednesday via Substack while Jonathan was unfortunately in the car. Not the kind of recording quality we usually have here at Signals and Subtractions, but in this case it couldn't be helped. I'll let him tell why. It's legit. Anyway, for this episode, I'm going to step in on the places where the cell connection ate his argument.

He was a champ to show up at all

1:55 Limited Edition Jonathan

it's nowhere cool. It, it is the hospital. no, my son's at his summer camp right now, and, uh, he got violently ill, so they are taking him to the emergency room, and it's a two-hour drive for me.

2:08 Sam Rogers

Thank you for being willing to do this, and help us, T-Mobile.

You're our only hope. So Jonathan, I was hoping that you could, say a little bit about what your, opinion is of the new announcement from Substack, in case anybody isn't aware of your, very tepid, mild-mannered views on the world.

2:32 Limited Edition Jonathan

Honestly, like AI detectors just like, other kind of purity tests are destructive in a way that, you don't really notice at first. So I have this whole thing about, about stigma, right?

We, we, we do it in a lot of areas. Like you look at an ADHD kid and, you're like, "The kid's lazy. He doesn't pay attention. He doesn't wanna do anything. And then you apply this stigma to that kid that automatically, puts a label onto him and, you know, kinda screws him for life.

The article goes over like just the different stigmas that we apply to different things.

3:04 Sam Rogers

You start with the nothing was in the water example, and th- and that analogy is fantastic, the whole, uh, nocebo effect

3:13 Limited Edition Jonathan

Yeah, 'cause it's one-to-one, right?

So people flip out about fluoride and like really the stupidest people in the world. Like genuinely I have no respect for anybody who hasn't done enough research to know that fluoride being added to water is not harmful in any measurable way.

I don't know, there's something that actually bonds to your teeth and bones that prevents cavities. It's been proven. They discovered it because, I think somewhere in the Netherlands.

Uh, fluoride occurs naturally in the water. And they're like, "Why don't these kids and adults get cavities the way everyone else in the world does?" And they discovered it was fluoride "Okay, so what if we add it to the water everywhere else?"

And it turns out that works, and it's awesome. And pseudo-scientists across the world trying to sell you, you know, whatever snake oil they're trying to sell you have decided, "We're gonna tell everybody that fluoride is evil. It's causing, uh, I don't know,

They just make stuff up, and then people decide to believe that, and then as evidenced by, It, it, it's a hilarious story. In New York, they had announced that they were gonna put fluoride in the water for this reason. And then the government, had to have a delay on doing it.

They were, like, a month and a half behind. But they didn't announce that they were delaying adding it to the water. The public thought that it was gonna go out on a certain day, and then on that day, they started calling in and complaining like, "Oh, I'm getting headaches.

It's giving me stomach aches. You know, I, I'm having health issues because of the fluoride you added." And, um, but they hadn't added anything, so there was literally nothing in the water, and people were just deciding that, they were getting sick because of something they had in their mind. And it's a one-to-one analogy to, using AI to write, to help you clarify your thoughts and get it out there,

4:53 Sam Rogers

yeah.

4:54 Limited Edition Jonathan

Regardless of what it is, if you're told that it's generated by AI, you value it less. Fun fact: AI values it less. The second analogy in here is about a Monet.

Somebody posted on X. They're like, "Hey, I generated this Monet, style painting

and, you know, everyone's saying, "Well, this is, this is awful. This is, this is nothing like Monet would actually produce. He would never paint something like this." And then the guy revealed, "Hey, dumbasses, this is a Monet." Uh, like, I just told you it was AI generated. This is actually Monet."

And, conveniently, a lot of people just, like, deleted their replies. But fun fact, I gave the same exact, image to both ChatGPT and Claude, and I asked both of them the same question he asked his ex audience, and I'm like, "Hey, how close is this to Monet? This was generated by AI, what do you think?"

And, um, AI had the same exact bias that humans have. If they thought it was generated by AI, they called it soulless, "Uh, kinda good, it's getting close, but, you know, it'll never be like real human stuff." So that bias is implicit, and it's built in to both humans and AI.

Which I thought was really funny, but also, at the same time, Substack has a button that you can click on now that tells you how much of or whether or not, the post is generated by AI or assisted by AI or written by AI, which is just a stigma-generating machine. That's all it is. And, uh, that's gross.

6:25 Sam Rogers

I don't seem to find that button on your post, Jonathan. Why is that?

6:29 Limited Edition Jonathan

Because (BLEEP) you, you're not entitled to my process.

I will happily help you. Like, my whole MO is teaching people how to use AI, teaching, people how to use different tools and combine them together to make greater things. I think AI is the great, equalizer in a lot of ways.

But nobody has a, a right to demand my process for how I do things. And, uh, full disclosure, this post that I wrote is almost entirely... Well, you know what? No, screw it. I'm sticking to it. I'm not, I'm not telling you

6:58 Sam Rogers

Yeah. Not, not telling us. Wait, wait with bated breath to find out is it AI or is it not AI?

7:06 Limited Edition Jonathan

The this stigma is actively dangerous. We have that nocebo effect where we automatically determine that AI-generated content or AI-generated anything is worse than human-generated is actively harmful in a lot of different ways,

7:21 Sam Rogers

yes. So I was wondering if you could say a little bit more too about the damages, about the costs of that, with regards to that stigma. Like, you've, you poked at it a little bit with the ADHD example

but how does it actually hurt Substack and Substack readers?

7:39 Limited Edition Jonathan

When you discount the quality of something because you know it was generated by AI,

7:44 Sam Rogers

And then the connection got painfully difficult, so allow me to summarize. This is his argument from the published piece, not mine. The writer who gets flagged stops publishing. That's the cost. There's not a debate, there's not a correction or any better argument. There's just a vacuum where the writer used to be.

And stigma doesn't distribute by merit. It lands on whoever is least equipped to fight it. Jonathan's whole point about AI is that it lets a person without a team compete against people who have whole teams' worth of effort behind them. Those are exactly the writers that the label discourages first.

Plus, the scan runs downward only. So writers get scanned, but the platform's own posts don't. For Chris Best, the CEO of Substack, his announcement of this feature notably carries no disclosure of its own.

So no notification when your stuff is scanned. The appeal, if you want one, comes after the label already exists, which is essentially a presumption of guilt with a customer service form attached to it. Now, this is a lot bigger than just Substack. This kind of conversation has been happening for a while, and it continues to happen across every platform in this age of AI.

That's why we're talking about it. It's a strong signal. And for Jonathan, it's also a strongly held conviction

9:27 Limited Edition Jonathan

I just wanted to say, um, even before we had this, AI detector pangram thing, I was vehemently opposed to anybody disclosing whether or not they used AI.

I don't think people owe it to their audience, and I don't think the audience is owed, any kind of disclosure on how you do your writing. And that's kind of been, like, the standard for journalism for all of eternity.

We don't owe our audience our sources. How many ghostwriters, how many editors, how many- Right... you know, how, how much of a process in writing journalism, all that kind of stuff? You'd be shocked how many big name authors, haven't written a word in years.

Yeah. Or they write the outline, and somebody else finishes it up for them.

10:06 Sam Rogers

Yeah...

10:07 Limited Edition Jonathan

we don't consider that dishonest. We don't consider that something we have to disclose. In fact, we actively don't disclose that kind of stuff because it creates the stigma. We don't actively disclose how much editing was done to Stephen King's last novel-

10:21 Sam Rogers

Yeah

10:21 Limited Edition Jonathan

because that's actively harmful to the, the reader who just wants to read something, to gain some kind of knowledge or, you know- Right... enjoyment from that thing. Disclosure is, it's a distraction from the point of what you're doing.

You're trying to share an idea. In my case, My whole MO is, helping people level the playing field and using AI to do that, because there's so many advantages to AI. And it gives you the ability to punch way, way above your weight.

For someone like me, it gives me the ability to compete against, those teams of people who have editors in front of their content. Like-

10:56 Sam Rogers

Right...

10:56 Limited Edition Jonathan

you know, I don't have a team of people working on my stuff to make sure that it gets delivered well, and that it doesn't, you know, piss anybody off, and that's a huge thing.

You have no idea how often, AI really helps me soften my very, very hot takes. Which you can read in my notes because I don't filter my notes at all.

11:12 Sam Rogers

Right.

When, when we're taking, quality measures... and we're taking the way that something was made and inferring, that those properties are related to the quality of the output. That's what I love to pick apart, because I think there's an opportunity in moments like this.

When people are seeing behind the curtain, just as you're describing, there's so many authors that have ghostwriters that write their next work, I don't think that everybody knows that, because for all the reasons you're just, talking about here, Jonathan, it's, it's very true. Ultimately, a measure of quality has to do with what the expectations were at the outset. Which is why, and I know Jonathan has his own choices and probably criticizes me for mine, but that's why I disclose, and always have, when something is AI assisted, that's what I label.

Now, sometimes I'm labeling it at the, distribution level, like with the Signals and Subtractions newsletters. Those have been largely AI assisted for about a year now. But for long form articles that I write, unless otherwise stated, it's something that I might have had AI do this editing function and make some suggestions, but it didn't change any words for me.

I treat it like I treat an editor. And as a, trained writer as well as a journalist back in the day, all of the things that Jonathan's talking about are very real, like, normal processes that writing would go through.

Getting back to is it correct thing, well, correct for what?

But we can still say, regardless of that taste layer of do I like it or not, of the expectation layer of does it meet my expectations or not as a reader? Does it have the labels that cue me in on what to expect for this quality here, which is the whole Substack pangram thing that we're talking about today.

There is something intrinsic to writing itself which is measurably good or measurably not good, which is not a matter of taste and it's not a matter of origination. It's does the object that has been produced meet the quality standards that we have expressed? Which is where I would love to see this conversation go, because I think that there are opportunities that exist now that didn't exist previous to this age of AI kind of era.

There are tools that we can apply. There are ways that we can use AI to help strengthen What the writing is.

When people ask me, "How was this article written?" I always want my answer to be, "Very well, thank you." Like, no matter what the origination process was, it needs to be good. And good is a measure that's on the final delivery side. Good is a quality process that we can inject along the way, either by humans or by AI.

14:20 Limited Edition Jonathan

Your articles don't need to come with instructions, i feel like that perpetuates the stigma- more than it helps ease the fears of your audience. That's why I'm so vehemently opposed to revealing, as a standard- what your process looks like. Because it's justifying, the worst fears.

It's not helping those. If you're gonna use an editor, if you're gonna use any kind of editorial filter, between you and your audience, the only reason you should disclose it is if you believe there's something wrong with it and you're mitigating those problems in some sort of way I just think it's a bad idea

14:55 Sam Rogers

And that's exactly why I wanted to have you on today, Jonathan. I really appreciate you, keeping your commitment to be on this show as best you can, thanks so much for showing up. I, I really appreciate it,

Stigma, I think, is a, is an important signal to talk about, of how it is that we interpret that label of something being AI-generated. And is that something to add or is it something to subtract? For everyone here on Substack, you should know that you do have the option of turning this feature off, at least at this point, and, that's what Jonathan has done for his article.

For mine, I have not done that yet, because I have a slightly different position on it all. I very purposefully was leaving my little star in the lower right-hand corner. For those who don't know what it means, that means Gemini generated this image. I label my works as being AI-assisted when they are.

I don't publish anything that's completely gener- AI-generated. But I actually detail at the bottom of this post, about what it is that I did to make this article for those who are interested in the point that Jonathan was just leaving off on, which is, like, how the thing got to be made. That can be relevant for certain things like, if we're talking about food, I use this apple analogy, throughout this piece.

Then you might wanna know that it comes from a local farm, or you might wanna know that it was grown without pesticides. You might wanna know things like that, not because it changes your sense of taste necessarily of how the apple tastes, but it does change what the apple is inside by that construction, and that might be important to you.

But it doesn't tell you if it's a good apple. And the measures of good that we can create are far more nuanced and important, I think, than just slapping a label on it, which can have costs such as the stigma that Jonathan was talking about.

and there's two different, approaches to that. One is to, not include the information that would make that labeling possible, and the other is to make it so ubiquitous that the stigma isn't there any longer.

And that costs a lot in the short term, but in the longer term game, I think for myself personally, that's something that I'm willing to invest in, because that's what I am aiming towards. For yourself, yeah, don't change a thing. Like, the way you're doing it is awesome

Everything makes sense in that world with that position, and I totally understand what you're saying.

17:42 Limited Edition Jonathan

Yeah, no, and I, and I can see, I, I can see your argument in the long term as well. Just steel man it. Revealing that you're using AI to help you with your writing is something that can normalize it over the long term.

And that could be a positive thing, is just, if it's a human editor that's filtering and helping with your writing or a ghostwriter, by default, we never, disclose that. Yeah. Uh, so I think it's counterintuitive to decide that, we need a special plead for AI, disclosure.

18:09 Sam Rogers

Yep. Yep. I, I totally understand it is really an issue, and it's something that we can have reasonable differences about, and have evolving opinions about, and evolving tools around.

And getting back to the Substack and Pangram ones specifically, the fact that something is being labeled, is not in a vacuum.

To give some additional context that fell out of the recording here, Chris Best's piece is called Against Claude Fishing. His definition is deception, text passed off as human when it's not. He calls it a con, and Freddie deBoer, who he's borrowing from, calls it a con too. Fine. I think most people would probably agree with that.

Here's the reversal. On that definition, the target is the faker. What actually shipped scans everybody and flags assistance of any kind at one hundred words or more. So it lands hardest on the honest, disclosed, assisted writing. On the exact cases his own definition was never really about, Best has a line in that piece, "When I want Claude's opinion, I'll ask Claude."

And Jonathan's answer to that is that the button is basically the machine's opinion about whether the writer is worth reading. And one more thing before I go further, and I want to credit it to Jonathan because it's his rule and I'm about to lean on it. He says he tries not to assume motivation.

Believe what people tell you, infer from what they reveal, and don't try to read anybody's minds. It's a good rule. I'm not going to guess at anyone's intent or go down a conspiratorial rabbit hole, but I am going to point at the structure here because it extends well beyond this particular platform.

Now, Jonathan, both you and I have our own companies, plural. We run things our own way, and we've set things up in a structure that works well for us. Chris Best has made a VC-backed play that results in VC-backed results, right? Like, we know what those look like.

We've seen it happen before. And in that context, decisions like this aren't, out of nowhere. That's all I'm trying to say well, thanks again for, continuing to show up even in extenuating circumstances. With regard to the subtractions here, I think we're both arguing toward, from different angles, subtracting the authorship, labels meaning what they mean in society today, and taking different approaches for how it is that that can work, what it, what costs we're willing to bear around that.

And advocating for people thinking for themselves and choosing for themselves where they wanna be on this scale, and getting involved in the discussion around the platforms that we're on, what we choose to disclose, what we don't, and being aware that it's evolving and that there's history here.

Yeah, absolutely. That this isn't exactly a new problem. This is a scaled problem. This is something that we can make decisions around. And I encourage everyone to find the settings in your Substack.

And you can do that on a per post basis. Yeah, for sure. You can do that on an account basis, at least last I checked.

21:46 Limited Edition Jonathan

And listen, last thing before we wrap it up, i'll commit to coming back on here 'cause, Sam and I have been talking for a while, and Sam's awesome.

So definitely subscribe to him. But, my last word before I get outta here is, I want people engaging with my ideas.

I don't want people engaging with my process

22:02 Sam Rogers

That sums it up beautifully. Yes to that. And thank you for the plug. Really appreciate you showing up, buddy.

Many thanks to Limited Edition Jonathan. He did that from the road on the way to his kid, and he committed to coming back. I plan to hold him to it. He's one of the most experienced and informed bleeding-edge builders that I know, and we didn't get a chance to talk at all about that, which is an unexpected and inspiring path.

For more from him, check out his substack at Limited Edition Jonathan. And to make the subtraction for this week explicit, subtract the AI-assisted label everywhere, full stop. Not just substack's machine-applied one, voluntary self-disclosure too, because disclosing is what teaches your audience that there was ever something to disclose.

Mine is retire authorship as a quality claim in both directions. Human-written is not a quality claim. Machine-flagged isn't either. Both of those are proxies. They were always lazy ones. They're the ones that we reach for because they're easy, because they're cheap. Now, we can name what good actually means now and measure that directly in ways that we could never even consider before.

So let's measure that, not who typed what. That's not the same subtraction twice. Jonathan says, "Remove the signal." I keep the signal and strip the quality meaning out of it

That's the reasonable disagreement in this episode. I disclose, and I plan to keep disclosing. Jonathan's position is that disclosing is the very thing that keeps stigma alive. I don't think either one of us talked the other one out of it, but it's good to revisit and revise our positions as things continue to change at ever increasingly inhuman speeds.

I'd love to hear your thoughts. Hit us up at sigsub.substack, or subscribe at

sigsub.show. Thanks, and I'll see you again next week