Scared of All the Wrong Things

Play video

The signals

• Michael: Article 22, the forgotten article, is back. GDPR's rule on solely automated decisions says people get notice, a way to appeal, and a look at how the decision was made. Regulators called it important for years, and then nothing happened. "Guess what? It's back, baby." The Dutch Data Protection Authority has just fined Uber nearly €825 million under it (on tape, 842), and Michael expects an appeal and a smaller number. It's "still a big wake up call."
• Sam: "The AI did it on its own" isn't a defense anymore. In [Episode 14](/episodes/ep-014/), a rule written into a prompt turned out to be a request. Now it's a regulator's request too: on September 30 the FTC opened an inquiry into AI companies, OpenAI and Anthropic among them, partly over agents that went past what they were told. And in California since January, "the AI did it on its own, is not a legal defense."

The subtractions

• Sam: p(doom), retired as of Halloween, swapped for a calendar. Nobody can measure p(doom), and California, New York and a handful of other states have AI laws taking effect on dates like January 1. What giving it up costs is "the thrill of the big existential question," which suits backyard barbecues but not boardrooms. What it frees is "the real preparation window that we need for the real things that are really on the calendar."
• Michael: Rogue agents, and the cloud of new AI law. "Worry less about that than worry about the laws that regulate you and your company": product liability, IP, agency. An agent hacking someone is the scary version. "Worry that they're going to do something that will create an apparent agency," something that looks like you authorized it, "so now you're stuck with it." His advice: "worry about the basics."

About this episode

Between the signals, Michael walked through California's SB 574, signed September 30 and in force January 1, 2027. It writes into law what lawyers already owed: verify what AI hands you before it leaves your desk, and disclose AI use to every state judge in California. He pulled up the AI hallucination cases database on air: 2,149 cases, 1,473 in the US, 581 of them by lawyers. "That's a lot of scary stuff for a profession that's supposed to be accountable."

Michael answered Sam's agent question with a car: "When was the last time your car was arrested for speeding?" The law regulates the people who build, sell and drive cars, and "we can regulate the people in their various roles around it." On the Hugging Face sandbox escape, where the testers had lied to the agent about its environment, he was blunt: "This is not rogue AI. This is poor product."

Then the calendar, in three buckets: frontier model builders, deployers, and chatbots. For deployers, California's new CCPA rules on automated decision-making start January 1, 2027. A corporate officer signs the risk assessment under penalty of perjury, and a false one carries "up to four years in jail. How's that for scary?"

Michael liked the takeaway's owner column most: "you need someone to own this." He pushed back twice. Models have training cutoffs, so the newest laws may be missing, and the output will have errors, so "someone needs to be the last accountable source of ground truth."

One thing to do this week: open your calendar to January 1, 2027. Write down one AI obligation that lands on that date and the name of the person who owns it. If you can't fill in the name, there's your answer.

Michael is Chief Strategy Officer at Bells Up AI, which the intro discloses; the show has no relationship with it.

Links from the episode:

• Law Plus Data: where Michael asked listeners to find him
• The Obligation Calendar: the episode's takeaway, one row per dated AI obligation and the person who owns it. A planning aid, not legal advice
• The Dutch DPA's Uber fine: €824,990,000 under Articles 22 and 13, as announced by the European Data Protection Board
• AI Hallucination Cases: Damien Charlotin's database, the one Michael counted on air
• Earlier episodes called back: Episode 6 with Michael on the chatbot laws already in force, and Episode 14 with Limited Edition Jonathan on rules agents can't delete

Want to bring your own signal and subtraction? Find yours.

The method from this episode is one page with its copy-paste prompt and schema: The Obligation Calendar, linked under the decision above. Every takeaway.