Episode 20 · 2026-11-13
What Did That AI Task Cost?
Nobody hands us an itemized receipt for AI, so most of us reach for the smartest model and guess what it cost.
Robert Alvarez, PhD, a data scientist at Everpure who once taught Sam statistics, on why most of us can't say what an AI task costs, and why the answer starts with our own benchmark, not the smartest model.
Host Sam Rogers · guest Robert Alvarez, PhD, the enterprise data scientist · 29 min
A choice this episode can help with
Which AI tasks deserve the most expensive model, and which are we overpaying for?
The short answer: Most of us can't say what an AI task costs. Start with our own benchmark, pick the cheapest model that passes, and mark any cost the provider didn't report as unknown.
Run it on your own work: The Receipt Ledger, one page, any model.
Watch it below, or read the conversation.
Jump to a moment
The signals
- Robert: Every enterprise needs its own benchmark. Enterprises don't get a flat fee per user: "They actually pay per token." So list the tasks that matter most, where we already know what a pass looks like (for code, a pull request and a quick test), then "pit a bunch of different models" against them to learn each task's difficulty and price, and route the work by difficulty. And when answers come back wrong: "usually, almost always, it's what the model is given to read."
- Sam: Nobody hands us an itemized receipt. Of 99 research calls his agents made to Perplexity in July, 66 threw away the usage record before anyone saw it, and that record "is the only receipt that will ever be." After two fixes and a billing change, answers over about five seconds now come back empty: "I honestly can't tell whether I'm paying for it." Robert's answer is "one dashboard to rule them all," counting input and output tokens against each model's price card.
The subtractions
- Robert: Reaching for the most intelligent model because it's the most intelligent. Start at the difficulty the task needs and step up only if the answer misses something. For a shell command, "Haiku is more than enough": "We don't need a two trillion parameter model." His tiers: the smallest models are a really good intern, Sonnet the generalist, Opus the senior developer or PhD researcher, and "very rarely will you need PhD level researchers working on a task for you."
- Sam: Estimating the costs his providers didn't report. One call's own receipt said four tenths of a cent; another, 579 tokens long, said zero. Filling the gaps with his own price table was "basically p-hacking my own budget," so he keeps only what the provider reports and marks the rest unknown, "because unknown is honest." Robert's next step: have AI sort the failures we're still charged for into buckets. Two percent of the bill might be fine; half means our own pipeline is losing information.
About this episode
Robert Alvarez, PhD is a data scientist who has worked on AI from storage, from education and at NASA's Frontier Development Lab, and is now at Everpure. Five years ago he taught Sam data analytics and statistics; his courses now reach close to ten thousand students a year.
The cold open is Robert's warning: "You now pay per token. And that's when you're gonna realize how much AI you're really using."
When answers come back wrong, Robert listed the knobs: the retrieval pipeline, chunking, the embedding model, the prompt, or a slightly more intelligent model. He described GEPA, from academic research: a prompt mutated "like a farmer would mutate seeds," with a smarter model reflecting on each round. His closing advice: for a task we run over and over, optimize the prompt until the model is a specialist, and a cheaper model will probably do.
On agents, Robert describes the finished result rather than a step-by-step plan, with Opus as the orchestrator and Sonnet or Haiku subagents, "like you would in an organization." His small team runs a Kanban board where humans write every card and a person steps in whenever an agent finishes or gets stuck, with permissions outside the agent.
One thing to do this week: pick one AI task our team runs every week. Can we say what it costs, and whether a cheaper model would do? Then take our three biggest AI bills through The Receipt Ledger and label every cost counted, billed, estimated or unknown.
Links from the episode:
- Robert Alvarez: LinkedIn and his author page on the Everpure blog, with all of his articles
- The Receipt Ledger: this episode's takeaway, three AI bills sorted by where each cost came from
- Delegated.watch, Sam's open-source multi-model token dashboard, with its code on GitHub
Want to bring your own signal and subtraction? Find yours.
Transcript
Read the conversation
Robert Alvarez, PhD: there will come a day. when prices are gonna increase and AI is gonna get more expensive, or you no longer pay for a monthly fee. You now pay per token. And that's when you're gonna realize how much AI you're really using.
Sam Rogers: Welcome to Signals and Subtractions. I'm your host, Sam Rogers. Every week, everyone here brings two things one signal worth watching, one subtraction worth making, and that's the whole show. What to watch, what to drop. This week, what does any given AI task actually cost? Most of us can't say and reach for the smartest model by default. Because nobody hands us an itemized receipt. Maybe that's okay for. Personal AI subscriptions, but what about when it's the API? Or in enterprise use cases when we're paying per token?
Sam Rogers: joining me today is Dr. Robert Alvarez. He's a data scientist who's worked on AI from a storage perspective, from an education perspective, and at NASA's Frontier Development Lab. These days he's working at Everpure, but I'm ever grateful for what he was doing five years ago when he taught me quite a lot about data analytics and visualization and probability and statistics and Tableau and the works. Robert, thank you so much for being here.
Put it to work
Pick one AI task your team runs every week. Can you say what it costs, and whether a cheaper model would do?
The newsletter
Issue 77 · Sunday 15 November 2026 · sent on Substack and LinkedIn.
One signal, one subtraction, one analogy, in five minutes. It draws on this episode but stands on its own. Every issue.
The method from this episode is one page with its copy-paste prompt and schema: The Receipt Ledger, linked under the decision above. Every takeaway.
Keep the next choice in view
One signal, one subtraction and one analogy every Sunday. Five minutes, no video required.
Optional. Free on Substack, no account needed. Already on LinkedIn? Subscribe there.