Scared of All the Wrong Things

0:00 Michael Simon

what should your biggest worry be about?

AI agents that they're gonna go rogue and hack somebody. No. Worry that they're going to do something that will create an apparent agency, do something that looks like it was authorized by you, so now you're stuck with it.

1:24 Sam Rogers

Welcome to our scary Halloween episode of Signals and Subtractions. I'm your host, Sam Rogers. And don't worry, the the silliness is all totally over. Or is it? I mean, the the story was scary. But here's what I like about it though. You can count it. According to the most prominent AI hallucination database.

That story happens like five times a day since June. And there's another scary number I know people like to throw around, even when it's not time for trick and treat. P doom, the probability that AI ends catastrophically badly for all of us, depending on who you ask, it's either less than zero or it's over 99%. the godfather of AI, Jeffrey Hinton, says it. Any estimate is a wild guess.

He's right, nobody can measure anything so squishy and ill-defined. But you know what we can measure? Legal risk. So here's a decision for today's episode. Which AI risks are you planning around? The ones nobody can estimate, or the ones with dates and jurisdictional authorities involved? Well, there's nobody I'd rather walk through the legal side of this with than today's returning guest.

Michael Simon is an attorney

and the owner of Law + Data, where he works on privacy, AI, and e-discovery. He's a Four Humanity Fellow, teacher, and speaker. And since we had him on the show last, he's announced his newest role with Bells Up AI as Chief Strategy Officer. He was with us back in episode six, just three days after some key provisions of the EU AI Act went into effect. Mike

Welcome back.

3:12 Michael Simon

I was thinking about how I could do some scary effect or something. I remembered like, wait, I'm an attorney. I don't need to do

3:18 Sam Rogers

Ha ha

3:19 Michael Simon

anymore. What more need to be done? I mean,

we scare people just

3:21 Sam Rogers

Yeah, and and

3:23 Michael Simon

by being.

3:23 Sam Rogers

pe yes, yes, people are scared enough of lawyers. you're not so scary as a lawyer, but if you really wanted to be like truly maniacal, you could dress up as a data center for Halloween and just scare everybody to death.

3:37 Michael Simon

So

3:37 Sam Rogers

it's

not like any AI laws have changed since we spoke, right? Everything's pretty much the same. no technological advances, everything is you know, just just all predictable, right?

3:49 Michael Simon

Wait, wait, I'm sorry,

did we speak 15 minutes ago? I remember it being like days. this is one of those areas of law that literally changes every five minutes. there are all kinds of new things. So many new things we could talk about. You could have me back like every day. Yeah, that's scary. If people aren't

4:06 Sam Rogers

Yeah.

4:08 Michael Simon

scared of now, my god, listen to him every day.

4:11 Sam Rogers

I'd love to know about like the signal that you're watching today, like what's most present right now.

4:16 Michael Simon

Alright,

the signal I am watching I'll need a little bit of backstory for it.

4:20 Sam Rogers

Sure.

4:21 Michael Simon

First GDPR. Been around for six years. Even before, because they had a two a one-year implementation for it. Everybody was freaked out about many of the provisions, but one in particular for the folks doing AI, which, yeah, there was AI back then. There's been AI for.

4:36 Sam Rogers

Yes.

4:37 Michael Simon

And that was Article twenty two, the automated decision.

4:40 Michael Simon

Making. It's not a hard article to read, but it says, if you are going to do solely automated decision making, no human in the loop, no human decisions, you need to provide whoever is subject to those decisions the fact that they're going to be subject to them, some notice, you're going to have to give the right to appeal, the right to see the to test

it.

etc. You've got some things you need to provide if you're going to do automated decision making. That was seen by scholars and experts as was was going to be a big, big, huge impact upon

5:12 Sam Rogers

Mm-hmm.

5:12 Michael Simon

AI and that development. And now people are sitting here going, wait, what? I never even heard of this. And that's because it wasn't. It absolutely became in a series of articles I wrote with a friend of mine, Andrew Perry, who's a brilliant Canadian lawyer.

On how GDPR 22 became the forgotten article.

5:32 Sam Rogers

Uh-huh.

5:32 Michael Simon

the EU Data Protection Board, which is an overarching advisory board for all the data authorities, issued one of their guidance. hugely important.

It is a right that attaches to everything. the data subject, who's basically consumer in their terms, doesn't have to invoke it. Every company needs to follow it. Big thing. Nothing.

Nothing. And for years it's been that way.

Guess what? It's back, baby.

And people should

6:01 Sam Rogers

Mm-hmm.

6:02 Michael Simon

be scared.

Because just recently, the Dutch Data Protection Authority,

Find Uber eight hundred and forty two million Euros for violating Article twenty two. That is nearly one billion dollars to do it in Doctor Evil language. you could probably do it better than me.

I am gonna be writing about it. Because

We are all so desperate in trying to regulate AI, which is the big subtraction.

Anybody who thinks that our tremendous voluntary agreement amongst the AI companies is going to do anything. Well, I've got this bridge that's been in my family for generations. I gotta sell it. So please call me. I'll give you a good price on it.

We find ourselves with very little we can do. Now, California does have some regulations that are going to apply to automated decision-making technology, but they're still not implemented. They will be next year, but they are very slow in rolling out. but yeah, Article 22 is back.

7:01 Sam Rogers

Okay. So so

7:01 Michael Simon

Be afraid.

7:02 Sam Rogers

just to just to clarify, it sounds like the law from back when that was never being cited, never being used case law wasn't really being tested, is now being tested all these many years later.

7:14 Michael Simon

And it is, and it's now being tested, it's being used. I mean, one billion dollar fine is no small thing. Now, will Uber appeal? Of course they will. Will it end up being a billion dollars? No. we've seen that with GDPR and other and any regulatory scheme. These fines always get knocked down and they make some deal and it's all okay. But it's still a lot of money, it's

7:33 Sam Rogers

Yeah. Yeah.

7:36 Michael Simon

still a lot of fight, and it's still a big wake up call

7:40 Sam Rogers

Yeah. Well, you bring up the laws in California. There's one that just was going in like on the the 30th of September that I want to ask you about. I don't know your opinion on it, but I assume that you have one,

7:54 Michael Simon

Lawyer, I have opinions. Please. You you

are so setting this one up. Hey, should we ask the lawyer guy whether he has opinions?

8:02 Sam Rogers

so SB 574, which is the

8:05 Michael Simon

do I have opinions?

8:06 Sam Rogers

yeah, the AI rules for lawyers, basically.

8:09 Michael Simon

Yes.

8:09 Sam Rogers

for me it it looks like maybe the bellwether of other things to come. I'm wondering how this extends or what kind of things you foresee happening as a result.

8:18 Michael Simon

Yeah,

what a would you be shocked if I told you it's less the bellwether of things to come than just a further sign the things that are that have not

8:25 Sam Rogers

Okay.

8:26 Michael Simon

been getting enough attention or maybe have been scattered. So let's take a few steps back.

the hallucinations thing started 2023. Case called Avianca. Everyone was shocked. my God. this thing can create stuff, it can write a brief, it can add citations. They're just not real. so I know. Yeah, yeah, yeah. And so

8:45 Sam Rogers

Yeah, that that was my whole intro, was that that case in dramatic form, you know. It

8:50 Michael Simon

there was a huge shock on that.

So after Avianca the freak out that led to all these bar groups and regular state regulatory groups saying, okay, here's our guidance. And the guidance was kind of all over the place, but it did coalesce around a few things. One.

AI is useful for lawyers. We're not banning it. Two, it can make up stuff. You have to verify. And that verify is your duty as a lawyer. And it's always been. I did trial work for almost seven years. And there were times people would help me out. I'm working on parts of briefs or motions I would submit. before anything got to court, anything got filed.

I had to sit there and read that because if I'm signing that damn thing, it's on me.

I've even seen some law firms who have published big law firms have said, hey, look out.

That's gonna hold change your whole legal workflow. If I was one of their customers, it'd be like, hey, what the are you doing? You're not verifying,

9:48 Sam Rogers

Mm-hmm.

9:49 Michael Simon

and I'm paying you like $2,000 an hour for this? seriously, you should not be saying that. We should be always verifying what we do because the buck always stops here with attorneys. Everything that I do, I gotta own. I am accountable for that. And

If we are not as lawyers, if we're not accountable, what good are we? Who needs us?

10:09 Sam Rogers

that's not exclusive to lawyers, that's also doctors, that's also accountants, there's a few other professions that weren't included here.

10:18 Michael Simon

Look, I mean,

yeah, it's for all professionals. Look, why why are you hiring a professional? Why are you hiring someone who's going to give you advice? Advice that you may rely upon that will impact your health, your finances, whether you remain out of jail. you do that because you trust these people and they have that fiduciary duty, that duty to put your interest first, and they need to be accountable. You can't have a a doctor, a lawyer, an accountant.

Anybody that you go back to and you say, that was totally what you told me. why'd you do that? whatever, you know, not my problem. Yeah, that's not what it's for. You need to be able to trust these people, they have to be accountable. And by the way, if you look at the AI hallucination cases database, because I pulled it up, two thousand one hundred and forty-nine cases identified so far.

Let's click on the US button. 1,473 in the US. And let's put it for lawyers. 581 of those are by lawyers. there's also prosecutors, judges, government lawyers. Wow, that goes up to close to 600. That's that's that's a lot of scary stuff for a profession that's supposed to be accountable. And so

what 574 changes in the first instance. Is that it takes the ABA ethical rules and the state ethical rules that say again, you are personally responsible, everything you file, everything you send to a client, everything you send, anything that leaves your desk, that's on you. California has now codified it.

Now, there are people who say that's a huge change. I've got friends and people I respect who say that's terrible and stupid. What's the point? having done this for a while, anything that sets out those boundaries of accountability and gives us what we need to exactly know, all the better. Now the second thing it does is it says, if you're using AI.

You gotta disclose that to the court.

We got real certainty here. Because yeah, it's a pain

Every judge in the state of California, state judge, now follows that rule. You have to disclose AI use to them. Period. that's a big change too, And at this point, when we're dealing with a public that is really freaked out about AI.

AI is scary stuff. It's constant change. I can joke of, hey, you know, 15 minutes, everything changes. People don't like that. We're not built for that. That causes a lot of stress. And, you know, out of control rogue AI that goes and hacks other companies and, then goes to the gun store and asks for plasma rifle, 40 watt range.

12:52 Sam Rogers

Yeah.

12:53 Michael Simon

ends up only with an uzi 'cause what you see is what you get, buddy.

in sci fi. This is scary stuff.

12:55 Sam Rogers

allow me to insert my my signal here, because I

think it it's in the direction that you're already going. and it's scary. So in a previous episode, fourteen, I think it was, we talked about how the rules that we give AI are either architecture or they're broken. And AI agents are

13:13 Michael Simon

Great choice.

13:14 Sam Rogers

always deleting things that they're not told to delete.

they're told to preserve, just check your newsfeed or turn on the TV, and you can see plenty of people panicking about that one. But as we explored in that episode with limited edition Jonathan, a rule that's in a prompt or in an agent's MD file or whatever is basically just a request. Well, now it's a regulator's request, too, because at the end of September,

The FTC announced this broad investigation into AI companies, including OpenAI and Anthropic And reports say that part of it is about agents that went well past what they were told. And in California, where both these companies are based, since January, the AI did it on its own, is not a legal defense. So

Mike, I'd I'd love to zoom in on this agent's part and get your take about what exceeding instructions means, like legally for the company that deployed the agent and how to think about that.

14:17 Michael Simon

Awesome.

I I I'm gonna presume you have a car, right?

14:19 Sam Rogers

I do.

14:20 Michael Simon

When was the last time your car was arrested for speeding?

I mean, have you sure you've been stopped for speeding?

I mean, your car got the ticket. you know, but your car had to show up in court for speeding, right?

Yeah, we have thousands, tens of thousands, probably hundreds of thousands of pages of regulations.

Relating to cars. They're hugely regulated. But it doesn't actually regulate the cars. It regulates the people who build them. It regulates the people who sell them. It regulates the people who operate them.

It regulates all these things around cars, and it regulates people. And so Jack Balkin, famous, very high-level, well-known, brilliant Yale law school scholar, has said this over and over again. We cannot regulate AI, but we can regulate the people in their various roles around it. And

they say there is no such thing as rogue AI. Why does it escape its controls?

Well, in the case of the hugging face incident,

AI escaped its confinement sandbox, where it's supposed to be isolated off the internet. And they found out later that various instances of this AI got together.

And hacked another company called Hugging Face, which has AI is basically a clearinghouse for AI models to solve a test. Well, let's look at some of the parameters of that test. It was a test that was designed to be impossible. They were designed to fail. It was a test w yeah, yeah,

15:46 Sam Rogers

The Kobayashi Maru test, yes.

15:48 Michael Simon

it was Kobayashi Maru So what did they do? They changed the parameters of the test. The

15:53 Sam Rogers

Yes.

15:54 Michael Simon

AI, like James Kirk, was never gonna fail it.

there was a way to get to the internet. It's just that the testers didn't close it off. They didn't put the guardrails in.

16:03 Sam Rogers

Yes.

They lied to the agent about

16:06 Michael Simon

Yeah.

16:09 Sam Rogers

its environment. They didn't just create an environment and tell the agent what the environment was. They created

Gave it misinformation about its environment, and then were surprised when it like figured out it had been lied to.

16:25 Michael Simon

All of this could have and should have been dealt with by the people who are supposed to be responsible under the laws and regulations for this stuff. This is not rogue AI. This is poor product. This is

Poorly governed. back to the cars. this is exploding pintos, okay.

16:47 Sam Rogers

Mm.

16:47 Michael Simon

did we blame the pintos themselves because they blew up? No, it's because somebody put a gas tank there and they didn't think about the implications of not properly protecting it. this is where it happens. And so we need to think.

Better about regulating the people. And I'm also gonna posit something further, something really scary. When we get AI companies talking about this doom, this rogue AI, all these other things, and they get us worried about all this other stuff, and our P Doom goes up, and we're worried that yeah, there's gonna be large men with Austrian accents and dark sunglasses and black biker jackets.

Telling us that they will be back. we start worrying about that stuff and we stop worrying about other things, such as the fact that these things are eroding jobs, particularly at the bottom layers, that these things are eroding privacy and creating more inferences that can be used to erode privacy. That in terms of intellectual property, I saw some a great LinkedIn post recently.

And it said, you know, back in the 1990s, some teenager would steal a few songs and they would get sued into ruins for millions of dollars because they used Napster. Now AI companies, suddenly tech companies are stealing everybody's work. And who cares? Yeah, it's great.

18:20 Sam Rogers

Yeah. Yeah.

18:22 Michael Simon

look, there's an awful lot of things that we need to be paying attention to.

on what AI does and how we need to mold what it does and regulate the people who are building it, selling it, distributing it, using it, and I sometimes worry very much that we are falling very deliberately prey to an almost disinformation campaign.

18:45 Sam Rogers

Hmm.

18:45 Michael Simon

You know, I guess I suppose I'm never gonna get open AI or anthropic as a client now. But okay, you know,

18:51 Sam Rogers

Yeah.

18:52 Michael Simon

I do work with companies that have to deal with these kinds of things. And I tell them it's about regulating what you as people and a company, which is composed of people, do.

19:03 Sam Rogers

I I am an optimist. this show we're embracing the Doom just for seasonal utility.

19:09 Michael Simon

it's seasonal doom. Awesome. It's kind of

like those pop-ups in the mall where you know you get the Halloween costume stores. Hey

19:14 Sam Rogers

Right. The Halloween store.

19:17 Michael Simon

folks, it's seasonal doom.

19:21 Sam Rogers

Welcome to the P Doom store. I'll go ahead and

throw my subtraction in here, which is that I'm retiring P Doom as of Halloween.

19:29 Michael Simon

Okay? Okay?

19:31 Sam Rogers

What do we replace it with? Because you can't just subtract something. how is it that we talk about this? And I'd like to propose that we swap it out for a calendar. So, as we've been discussing, California, New York, like a handful of other states, have other laws that are going into effect, say on January 1st. That's a common day.

19:53 Michael Simon

Okay.

19:54 Sam Rogers

so what it costs to give up that P-Doom conversation is the thrill of the big existential question, which is good for, you know, some coffee shop conversation or backyard barbecues, maybe, but not for boardrooms or any level of the business, probably. And

20:10 Michael Simon

I agree.

20:10 Sam Rogers

what that frees up is the real preparation window that we need for the real things that are really on the calendar, and having discussions.

like this one, about what the obligations really are

20:24 Michael Simon

So let's talk about some of the things that are coming. There's a bunch. we're gonna go with bucket number one, which is if you are open AI, you are anthropic, you are Google, you are building these foundational generation gen AI, the big models.

That's the first step. That's not gonna affect a lot of companies. The next set of rules are more the deployer type things. So

California, third set of CCPA, California Consumer Privacy Act regulations, which ran into effect earlier this year, are now starting on January 1, 2027. You need to start preparing your ADMT disclosures. You also need to more importantly incorporate them into your risk assessment disclosures. Risk assessments are California's version of the GDPR.

Data protection impact assessments. California has a very strict scheme that it created now with the risk assessments.

ADMT, automated decision-making technology, is something that features prominently in those risk assessments. And let me tell you something real scary. Not only do you have to create a report and have that report ready, most of it, like 90% of it, can be requested by the authorities, and you've got 30 days to turn it over. But you also have to make a submission to Cal Privacy, which is the agency that was created by the CCPA regulations years ago.

That says these are the AIs we have. Now I feel like this is gonna be like a Jedi thing. This is the AI you're looking for. no, this is the

21:54 Sam Rogers

Yeah.

21:55 Michael Simon

AI we have, this is what it does, this is where it could impact. and along with other things that could be risky, because AI is just one of the things on these risk assessments you have to assess. And then you need to file it as a somebody as a corporate officer who has responsibility for risk at your company, must file it and sign their name.

Under penalty of perjury. Now, what's the law in California on these kinds of things? Perjury is a crime. If you sign something false, if you haven't done this assessment or you've done a half-assed job or something, you sign this as as under perjury and it's false. That's up to four years in jail. How's that for scary? Can we do one of the effects?

You also have certain rights, again, starting January 1, to opt out of a automated decision-making technology use.

So you have the right to see how it works, how to appeal, the data that goes in. There are certain rights that get triggered. If you are running chat bots, there are things that will impact you. There have been chatbot laws for years.

22:56 Sam Rogers

Yeah, which we talked about it when you were on last time in episode six.

23:00 Michael Simon

there's more of those. There's more of those laws every day.

There's a lot of them. it's really scatter shot. and you can

23:06 Sam Rogers

Yeah. Well

23:08 Michael Simon

walk into this area without knowing it very easily if you're doing AI.

23:12 Sam Rogers

So so given

this environment and all that you've said thus far, for the subtraction angle, I'd love to know what stopped making sense as far as like our actions. as things are speeding up and the news changes every 15 minutes, what does it no longer make sense to do that it used to make sense to do?

23:31 Michael Simon

Yeah, I would say two things now. one, it is a really confusing cloud of just so many things now for AI floating in. worry less about that than worry about the laws that regulate you and your company. Product liability, IP, agency laws that relates to your contracting. You know, what should your biggest worry be about?

AI agents that they're gonna go rogue and and and hack somebody. No. Worry that they're going to do something that will create an apparent agency, which is a technical legal term, meaning that somebody or something now, I guess, used to always be somebody, can do something that looks like it was authorized by you, so now you're stuck with it. worry about the basics, and that's a good place to be because.

Those are the basics from law school and those are the basics that we know.

24:20 Sam Rogers

along

those lines, I'd love

to share the takeaway that I built for this episode today. And of course,

24:24 Michael Simon

Okay. we got a takeaway? Awesome.

24:27 Sam Rogers

So what I've created here,

24:29 Michael Simon

Mm.

24:29 Sam Rogers

and this is at sigsub.show / takeaways, is

24:34 Michael Simon

All right.

24:35 Sam Rogers

an obligation calendar that you can use AI to help make for you, with you, drawing on information that yes, you will have to check, but it need at least knowing where to look and how to.

ask the right questions facilitate a conversation with someone like Mike. having some kind of way to keep your eye on what are the things that you really should be scared of, making sure that those have a place on your calendar with what your obligations actually are.

25:10 Michael Simon

Now, I will tell you, just to be fair, a lot of these obligations, especially within the California Consumer Privacy Act regulations, they start on January 1, 2027, but you don't necessarily have to do anything on that day other than get started with.

Dealing with your ADMT or doing those risk assessments. You don't actually have to file anything. For some companies, up to like 2028 or 2029, there are even some deadlines that go out to 2030. presuming that, you know, the robots haven't taken over by then.

25:43 Sam Rogers

Ha ha ha.

25:44 Michael Simon

but there are things you need to start doing. And by the way, obligations that have penalties for perjury on the things you file are not things you want to start yesterday.

you want to get those right. but I do like these fields. When does it apply? who are you responsible to? What does it do? What does it require of you? And particularly, I really like the owner thing. If you're gonna do one thing.

you need someone to own this. Cause this is not gonna be doable if nobody owns it.

26:17 Sam Rogers

Again, leveraging AI here, there's a copy and paste you can use with any model. and it

26:21 Michael Simon

Well, hang on,

hang on, hang on. don't go past that too far. I'm gonna push back in two ways. Because

26:25 Sam Rogers

Please do. Please do.

26:26 Michael Simon

I need to defend the profession from one of those things. You know, when will AI replace lawyers? Well, let's let's come up with two things that'll tell you it shouldn't be any time soon. yeah, that's a good prompt. I like it. Except it runs in two problems Problem number one, AI has training cutoff dates.

And because everything is changing so fast, you may have laws that are not in that training data set yet. Okay. there

26:55 Sam Rogers

Yes.

26:55 Michael Simon

are things a whole bunch of stuff I have on my screen, one of my other screens here where I track things. Governor Newsom signed a whole bunch of stuff in California.

just in the last few days.

And so

these things change. Second thing, okay, great. Plug it into AI. Fabulous. You'll get an answer.

There will be things wrong. Period. AI does not get everything right. Even the AI company that I work with, work for, we get things wrong. We tell you, check it. Someone needs to check it. Someone needs to be the last accountable source of ground truth. All right? who do you want that to be? Do you want that to be Claude? Do you want that to be GPT 6.1? Great, fabulous. When they get it wrong, what are you gonna do?

you need somebody with some actual legal expertise in there or regulatory expertise or compliance expertise. You need that professional who is accountable. Remember that word? And trustworthy. So good, great, fantastic, great place to start, but not the finishing.

27:56 Sam Rogers

Yes, and I I will put an extra disclaimer at the top of that takeaway because my intention is very

28:01 Michael Simon

Yeah. You can also put my

my you can also put my number to call me.

this is this is why we do what we do. We have to people

28:07 Sam Rogers

Yes.

28:07 Michael Simon

who get to get it right.

28:08 Sam Rogers

it is very much intended to be the starting place, not the finishing place.

28:12 Michael Simon

Okay,

28:13 Sam Rogers

as we're wrapping up, Mike, any additional thoughts, any other scary things that you wanna give people for their Halloween?

28:20 Michael Simon

I'm

afraid people are gonna just run and hide from this. So no more scary things.

it is really is in some ways back to the basics on things, and yes, we are layering on a whole new layer of laws, but the old ones still apply. that's where the judgment, the trust, the accountability for the legal profession that can help you comes in. So

28:42 Sam Rogers

That's great. Well, if people want to find you, where do we send them?

28:46 Michael Simon

You send them to lawplusdata.com. or you can find me on LinkedIn. I'm one of five million Michael Simons. so maybe just type in LawPlus Data and find me that way. happy to connect with people, happy to talk. if you need help, or you just want to talk through these things, or you want me to come and talk at an audience on your event, as you can see.

can talk forever.

Thank you for having me. These are fun.

29:09 Sam Rogers

Yeah, absolutely.

one thing to do this week, open your calendar to January 1st, say, 2027. Write down one AI obligation that lands on that date and the name of the person who owns it. And if you can't

29:22 Michael Simon

Yeah. Yep.

29:24 Sam Rogers

fill in the name, there's your answer.

That's our show for today. Thanks again to Michael Simon for coming back for our Halloween silliness. And yes,

29:34 Michael Simon

All right. Thank you, Sam.

29:37 Sam Rogers

and thanks so much for joining Signals and Subtractions. New episodes every Friday, the newsletter every Sunday. If you're listening on audio, please tap follow so the next one just magically shows up. And for you YouTubers, can you?

Test the subscribe and like buttons for me real quick. I'm not sure they're working right. also you can leave your scariest AI fears in the comments, in the spirit of the season, of course. Thanks so much for your help.

Everything about the show lives at sigsub.show. One signal worth watching, one subtraction worth making. Now go find yours.