Episode 20 · Transcript
What Did That AI Task Cost?
Full transcript of the recorded conversation.
Back to the episode · timestamps jump to the video · lightly machine-transcribed, may contain errors
0:00 Robert Alvarez, PhD
there will come a day.
when prices are gonna increase and AI is gonna get more expensive, or you no longer pay for a monthly fee. You now pay per token. And that's when you're gonna realize how much AI you're really
using.
0:23 Sam Rogers
Welcome to Signals and Subtractions. I'm your host, Sam Rogers. Every week, everyone here brings two things one signal worth watching, one subtraction worth making, and that's the whole show. What to watch, what to drop. This week, what does any given AI task actually cost? Most of us can't say and reach for the smartest model by default. Because nobody hands us an itemized receipt. Maybe that's okay for.
Personal AI subscriptions, but what about when it's the API? Or in enterprise use cases when we're paying per token?
1:01 Sam Rogers
joining me today is Dr. Robert Alvarez. He's a data scientist who's worked on AI from a storage perspective, from an education perspective, and at NASA's Frontier Development Lab. These days he's working at Everpure, but I'm ever grateful for what he was doing five years ago when he taught me quite a lot about data analytics and visualization and probability and statistics and
Tableau and the works. Robert, thank you so much for being here.
1:29 Robert Alvarez, PhD
My pleasure, Sam. Thanks for having me.
1:31 Sam Rogers
one of the things that I love about you is your emphasis on effective communication and using analogies to tease apart signals from the noise. It's probably no coincidence that every issue of
1:42 Robert Alvarez, PhD
Hmm.
1:44 Sam Rogers
signals and subtractions features a signal and a subtraction and an analogy to make it stick.
1:47 Robert Alvarez, PhD
Mm-hmm.
1:50 Sam Rogers
Um I just appreciate your work and the way that you do it and the influence that it's had on me. I know I was just one of like 300 of your students.
But I understand things have scaled a bit since then.
2:02 Robert Alvarez, PhD
Yeah,
yeah, yeah. I think we're maybe even close to ten thousand students a year, uh, in the courses that you took and students continue to take across the US.
2:12 Sam Rogers
That's awesome.
So guests go first.
What's a signal that you're watching that's in your own work this week that you'd like
to talk about?
2:20 Robert Alvarez, PhD
enterprises need to have their own benchmark for their tasks to determine the right cost for their AI. It's getting overblown right now.
2:29 Sam Rogers
Mm-hmm.
what kinds of things are you seeing go wrong when things go wrong with that?
2:37 Robert Alvarez, PhD
It's mostly cost.
everyone is talking about how AI is getting
very, very expensive, like including in our own company. but you know, the larger you are, the more you spend. Especially for for enterprises, they don't get to have that like user, you know, $200 for a user. They actually pay per token.
so what enterprises should be doing right now is coming up with a list of
Tasks that are very important to them, that they've had metrics or data on where they know like what's a successful pass, what is the fix that needs to be done. So if you think of like coding tasks, for example, most companies would have their own GitHub repository. They have pull requests, they can look at, well, what was the state of the code before? What is the state of the code after? Is there a quick test that they can create for this? And then ultimately then pit a bunch of different models to determine.
Difficulty level and like which model should they be using at a given price point. this could be frontier models, this could be open weight models if they're deciding to move their infrastructure on prem. that'll give them a nice benchmark of or bellwether of like this task is should be sent to the frontier models and then give them a sense of like you can now use like open JEV or the JEV models to route between difficult.
3:46 Sam Rogers
Yeah. Classifiers.
3:49 Sam Rogers
that's great. So when people are looking at that, oftentimes it's not very easily validatable tasks. So like coding is a great example of something that's very validatable. you know, the legal profession, you can say a little better, is this is this objectively right or wrong? Because there's actually some things to rely on with that. But at least in my experience, with most businesses, it's not.
That's easy across the business to be able have a benchmark like that. And one of the things that I love about this time is that we're we're to the place where we kind of need that in order to onboard agentic partners to help focus in that direction. Like we we always needed it as humans, but
4:29 Robert Alvarez, PhD
Yeah.
4:30 Sam Rogers
we could kind of skid by without getting too overly specific about it. And at this point,
it really helps to be explicit. what are the kinds of things that you're exploring or that you would suggest others explore for how to find that in in maybe a less easily validatable
task.
4:46 Robert Alvarez, PhD
Yeah, and in less validable tasks, I think it's quite might be easy to just have a human in the loop. Like they're like coding tasks are quite, I don't want to say binary, but closer to binary, where it's you know, passfail. but ultimately if you have much more business decisions, things that are dependent on knowledge sources, et cetera, you could put them through different models and then decide like this is good enough, and anything you know, there's there's gonna be different versions of good.
But ultimately you'll just you'll have a cutoff of like, does it at least reach this level of response depth?
5:18 Sam Rogers
And when the answers are are wrong
just from your own experience, like how often is that the model versus how often is that what the model was given to read?
5:29 Robert Alvarez, PhD
usually, almost always, it's what the model is given to read. there's a lot of work being done on well, retrieval augmented generation or RAG for short, where you have data inside your infrastructure. There's different ways that you can search that data, but ultimately it boils down to that. I've actually done some benchmarking on frameworks where
Even if you give it this the right data, will it does it still get it wrong? And that's you'd be surprised at how often, even with everything you give it, it could still be wrong. Just because some tasks are very difficult. And that's where the give and take here is well, there are things that you can optimize. There are many knobs to tune.
Do you focus on the RAG pipeline, the data ingestion, the retrieval? Are you ingesting a lot of PDFs in which in which case maybe like OCR might be better to scan that and turn that into text data? Your chunking algorithm, the embedding model. Do you need to fine-tune an embedding model for your use cases? So there's a lot of work that you can do there. But sometimes it might be just as easy as changing the prompt out or going to a slightly more intelligent model.
the easiest one to change is the prompt or the well, actually, probably even the model itself is probably the easiest one. Cause usually, if you're using the same API, you would just change out the code to go from like Claude Sonnet to Claude Opus, for example. but then from there, it's well, different levels of tuning. There's some work that we're about to publish soon. this this is not our our own proprietary work. We're looking at what academics have done.
And it's called generative pareto, so GEPA for short, G-E-P-A, that it does reflective prompt. So it takes a generic prompt and then it starts mutating it, kind of like a farmer would mutate seeds to get different yields. from there start taking childs of that and like keep on doing mutations and ultimately say, well, if this is what good looks like, can we get closer to good by just constantly iterating this prompt?
But but you have it on like you know, you're let's say you're trying this like on haiku or sonnet, but you have a a what with what's called a reflector model. So a much more intelligent model that looks at what the base model did and how it responded, and then gives it instructions to refine its prompt. So this is very, very good for tasks that need to live on the edge, that need to be very fast, or that are you just gonna be running over and over and over and over again, that you want it to be as precise as possible. So like you can get rid of like
PII if you're looking at healthcare data. And sometimes like you know, just one one PII that slips through can be a very expensive process.
7:52 Sam Rogers
so when you're talking about this prompt mutation
idea, how is it that you're measuring that the prompt is actually better versus just you know different?
8:02 Robert Alvarez, PhD
Well, the one of the nice things about the prompt is that you could see what changes specifically. Like you could look at all the mutations. And since it's a prompt, that means it's in text. It's not like in model weights or in some embedding space that is just gibberish to us humans. but ultimately you have a series of tasks that you've benchmarked or you want to use as a benchmark, and you have metrics for great.
good and bad and you see how those prompts hit on all of those tasks and ultimately the pass rate on the tasks or the you know the how good you are on average increases
over time.
8:35 Sam Rogers
Yeah, makes sense. well, cool. I'll throw my signal in here, which is that most AI services they don't hand us this a detailed receipt that we can inspect line by line, something that we can keep. So most of us can't really say what a single task has actually cost. and that's why I'm asking about like how it is that you are benchmarking things. So
8:55 Robert Alvarez, PhD
Mm-hmm.
8:56 Sam Rogers
so back in July.
I started counting how it is that I'm using AI and how I've been using it since since it's been a reportable event, basically back to like early 2023. just focusing on one, I I had like 99 research calls that my agents had made to Perplexity, and
9:12 Robert Alvarez, PhD
Mm-hmm.
9:13 Sam Rogers
66 of them, two-thirds of the usage, they threw away the usage record like before anyone saw it. So that record.
is the only receipt that will ever be because of how Perplexity, specifically as a service, is tracking their usage and and billing. So I fixed it, it worked, and then they changed everything about how Perplexity does
9:33 Robert Alvarez, PhD
Ha ha ha.
9:33 Sam Rogers
their billing, of course, like at the end of September. And I fixed it and then it worked. And then this week I found a new gap. that any answer that takes more than like five seconds comes back empty. And
9:45 Robert Alvarez, PhD
Mm-hmm.
9:45 Sam Rogers
I can't tell. I honestly can't tell whether I'm paying for it, I couldn't
9:51 Robert Alvarez, PhD
Yeah.
9:51 Sam Rogers
count it any longer in my token burn dashboard. so if we can't see like what each call is costing, and we can't see like what a finished task looks like,
we're guessing at whether to fix the prompt or fix the knowledge or fix the infrastructure to be able to better tune for that cost optimization at the end. so I'm curious like how it is from a cost perspective, since you opened with that, how it is that you're tracking receipts and and looking at that when everyone
is managing their tokens differently, managing their billing differently, managing everything their own independent way. Like if you're working across multiple models, are you tracking that? Or do you just have 13 dashboards for 13 different AIs?
10:32 Robert Alvarez, PhD
No, so
we have one dashboard to rule them all. I actually don't use Persplexity's API all that often. We've kind of got our own internal tooling for that. you usually do have a count, no matter what you do, of input tokens and output tokens. So from there you can easily the recalculation based on the model and its price card.
Of okay, okay, this is how many input tokens, this is how many output tokens, and whatever the rate is, you can figure out the cost per task. One of the challenges there is that some models will tell you the cached input tokens. So then, you know, for example, if you're having a long conversation with say Claude or ChatGPT, most of those tokens that are from the history have been cached and they come at a much, much cheaper rate. And if you can't get a
good pulse on those numbers, then you kind of have to overinflate the task, the the price that it costs you, you know it's cheaper, but if you want to be conservative, you say this is what it's actually costing us. but most of the APIs will provide you input tokens and output tokens, and you're also paying reasoning tokens as part of that
11:32 Sam Rogers
That's great. given all that you're working with and and the signal that you've shared, what are some things that no longer make sense? Things that maybe we have a habit of doing or that we've done before that we could probably let go of now because it's
maybe anti good or you know, not helping.
11:46 Robert Alvarez, PhD
Yeah.
I guess this is gonna be depending on how you pay for your AI, whether like your company pays for the token or like you just pay for your own personal count at the user level and like you know, it's like a hundred bucks a month or twenty bucks a month. you gotta stop using the most intelligent model just because it is the most intelligent model. you're gonna get responses back faster if you know the kind of the appropriate difficulty level.
And if you don't like that response, or you're like, I think it missed something, then you can increase that step. And sure, you'll be paying twice for that request. But ultimately over time, like obviously I've been using Linux my whole life, at least since college, which is now half half my life now. there's still a lot of shell commands that I never know, especially as it comes to like bash scripts, et cetera. And I'm gonna look for that. I'm like,
Haiku is more than enough.
12:35 Sam Rogers
Yes.
12:36 Robert Alvarez, PhD
even if my company wasn't paying for it and I was paying for it, I could be like, look, it doesn't matter. Like I'm just gonna do a cheaper one. Like, let's save some resources. We don't need a two trillion parameter model, haiku will be more than enough to get me this. And that's how you should start thinking about your tasks, because there will come a day.
when prices are gonna increase and AI is gonna get more expensive, or you no longer pay for a monthly fee. You now pay per token. And that's when you're gonna realize how much AI you're really using.
13:04 Sam Rogers
Yeah, well, I'd like to tell folks like if it's a replicable process, you can probably work your way down the chain. If
13:11 Robert Alvarez, PhD
Mm-hmm.
13:11 Sam Rogers
it's if it's a creative exploration, if it's really open-ended and you don't know where you're going yet, larger models can have a lot more utility there because it's an exploration but you can always ask them and they'll tell you like, is this a a task for a lesser
agents that I can work with in this conversation to spin things out? Or should I just open a new session that's like the haiku session and
13:35 Robert Alvarez, PhD
Yeah.
13:36 Sam Rogers
and treat it that way? That's great.
small compared to what?
You know, small compare because last year it would have been jaw dropping awesome. It would have
13:43 Robert Alvarez, PhD
Mm-hmm.
13:43 Sam Rogers
been leading the frontier to have that kind of capability that's what like Sonnet has now. it's definitely useful to compare it to the size of the task as
13:53 Robert Alvarez, PhD
Yeah.
13:54 Sam Rogers
opposed to the size of all of the things that could be in the whole wide world of LLMs.
13:58 Robert Alvarez, PhD
Yeah, and I always try to tell folks, like think of the models in this sort of category where like haiku or or Luna for those who use open AI, they would be like a really, really, really good intern. they've got a lot of knowledge, they're they're eager to work, so that means they're fast, but like they just don't have a lot of institutional knowledge around the work that you might want to do. Sonnet is kind of like your generalist who is just
Incredible, but doesn't have deeper knowledge, like say like a senior developer or a PhD researcher. That would kind of be your opus. And then Fable would be the like you have a task that requires a ton of creativity or a ton of knowledge that is just overly complex, and that's when you go to Fable. But those are few and far between.
You know, very rarely will you need PhD level researchers working on a task for you. You might you only be okay with generalists or even like really, really good interns. for open AI it would be Luna, Terra, Sol, and then Astra and kind of increasing order of intelligence.
15:04 Sam Rogers
don't forget that Fable is what we got after the government decided it was okay for us to have, you know. And now this week on Monday, 5.2. Yeah.
15:11 Robert Alvarez, PhD
Yes, and now five point two came out, so yeah.
15:16 Sam Rogers
Um, which I'm finding just in some experiments I've been running since then.
the thing that that's advice that I used to give a year ago has basically flipped with like with regards to how it is that we're we're prompting stuff. it used to be that you'd have to really spell out like the task that you want to have happen. And then like maybe the beginning of the year you could think more at a project level and you could think more in terms of workflows.
And you know, the whole agentic world and being able to chain things together and have multiple different sessions working concurrently. And now we're at the level where you can really think in terms of what what I was expressing, I'd love to get your thoughts on this, as needs and ownership and accountability. Like the parts that that are truly human that we have to do, which is the expression of meaning and making meaning and how it is that we
Are going to like bring these agentic partners in on what it is that we're trying to do to set that intent. how it is that we're not just gonna focus them, but how it is that we're gonna own the result and and bringing them in to help us do that.
So being able to really focus on those targets, it seems like a much more powerful way to work with AI and and prompts that are compounded and kind of messy that would have been a really bad idea a year ago. I'm now finding are, at least in my own.
limited experience, perhaps not benchmarking to the level that you're doing, but but good enough for me, I'm getting better performance out of what was exactly the wrong thing a year ago.
16:47 Robert Alvarez, PhD
Yeah. So I think even the folks at Anthropic have started to say that don't to stop using plan mode. So for those who don't recall, plan mode was just like you do slash plan and I tell it kind of what you want, and then Claude would come up with a plan of like a detailed list of things kind of before that we would type ourselves. And they're saying, like, now just give it what you want the finished result to look like. And
It's smart enough to get there what they're saying now is that when you're too prescriptive, it tries to index on every little thing on your list instead of saying, well, if this is the outcome, some of those things on that list may need to be done a different way, unless you have like, very specific regulations around it. But for most users, start with like here's what the end result looks like.
And then build it out. what I do a lot is I usually have a higher level intelligence model be my orchestrator. So I'll start with like typically Opus and be like, okay, OPUS, here's what I want the end result to look like. But for a lot of these tasks, they're in parallel. Like you can do multiple things at once. So then I say spin out subagents, but those subagents are usually sonnet or haiku. And very rarely are they Opus level.
of like you would in an organization. You would have like your director or your, head of research telling others to do things and then they look at all the results together and combine that and say, Okay, this is where we need to go next. So I that's what I've been using it for a lot lately.
18:08 Sam Rogers
I know that you're somebody who believes in the power of small teams and has seen things go awry in the hierarchy of organizations so I was just Monday at
18:18 Robert Alvarez, PhD
Ha ha.
18:19 Sam Rogers
Agent Substrate Day, and there's a whole KubeCon thing that's that's happening for Kubernetes. And for
18:25 Robert Alvarez, PhD
Yep.
18:25 Sam Rogers
folks that don't know, like I'm not fluent in Kubernetes, but I played around with stuff.
And there's a lot of work that's happening right now to try and define the standards around what Robert's talking about here and how these things work together. And the thing that surprises me after going to that event and talking to people is how much we still default to this like hierarchy model of how organizations work, how large organizations work and in orchestration and stuff, like command and control
When we know, like personally, the value of collaboration and
18:59 Robert Alvarez, PhD
Mm-hmm.
18:59 Sam Rogers
peer groups who have different strengths working together to accomplish a goal. I haven't seen anybody really developing AI collaboration models in terms of how to treat agent swarms, how to treat when you've got, you know, a hundred agents attacking a problem. Have you seen anything like that?
19:19 Robert Alvarez, PhD
NVIDIA has just released its open shell architecture, and it's something also we're building internally in uh at Everpure too around like how do you sandbox agents? Because when you have 10,000 agents trying to solve the same task, you may have some, how should I put it?
Things go awry.
19:40 Sam Rogers
I should point out one of your recent articles talking about one of the very specific ways that things go awry, where um as often happens when people are running a pilot with a series of agents, like it works great, it's so efficient, this is great. And then you start to scale it and boom, like suddenly
19:55 Robert Alvarez, PhD
Yeah.
19:56 Sam Rogers
something changes along the way. Maybe you could speak
19:58 Robert Alvarez, PhD
Yeah.
19:58 Sam Rogers
to that for a minute.
19:59 Robert Alvarez, PhD
Yeah. we think for controlling agent swarms like having them do really, really complex tasks where you like 10,000 parallel agents, kind of like OpenAI did with the Navier-Stokes equation on one of the millennium problems for mathematics. for enterprises where like your data is very, very sensitive.
You gotta have to sandbox them, give them permissions, MCP rules, but those should be done outside of the agent so that if an agent goes awry, you can just cut off that agent and then it can't do any harm. So that's what we've been seeing as well. Like kind of like that's what there's a uh there's gonna be a huge focus on security over the next year on AI, especially with Mythos and the Chinese models getting so much better at
And hacking.
20:38 Sam Rogers
Yeah. at anything really. And so depending
20:39 Robert Alvarez, PhD
At anything, yeah, they at anything, yeah.
20:42 Sam Rogers
on how you direct it, yeah, they can do all sorts of nasty stuff if you want to amplify that.
20:46 Robert Alvarez, PhD
So this would might be harder at a much, much larger scale.
we do have a small team.
But we have like a bunch of things that we want to work on. So we have like ideas in our for like very, very far left column of our Kanban board.
Then it's like, okay, it's in progress. It either needs human review or like it needs more input. And then ultimately, like we move things to done. But humans write all of the cards in the ideas section.
21:09 Sam Rogers
Mm-hmm.
21:10 Robert Alvarez, PhD
We move it to ready to go. Then the agents pick it up, build everything out that they need. They usually have access to tools that we've kind of pre-allocated to them. And then
Every time the agents like is either done or hits a roadblock, a human has to intervene. So that way we know always gives a summary, it gives a PDF of the work it did, et cetera, shows you the results. So we're now using AI agents as colleagues in a sense, with
21:37 Sam Rogers
Mm-hmm.
21:37 Robert Alvarez, PhD
some with oversight. Not say some oversight. We do have oversight on all this. And then ultimately we're like, no, redo it or
Or here's why you're blocked. Like, you know, we need to change something out in our in our infrastructure to support this sort of command. before it used to be like hey, I used to have to send a s Slack message to like, hey, who's using the GPUs today? Like, I
21:54 Sam Rogers
Right, right, right.
21:55 Robert Alvarez, PhD
need 'em. Now it's just like, nope, we have a Kanban board, we do all of this stuff and then we are able to put things into a priority order and say, Okay, let's go work on this, we come back, and in the meantime we
start working on other ideas. So it's it it gives us more time for us to focus on the ideating side of like what we think is going to be happening next in AI that we should be getting ahead of it as an organization. And less on like sitting there back and forth like, okay, the agent did this. Okay, then this is what I need to do now. It's like no. You have carte blanche, within some sort of sandbox environment, but we are now oversight for their work.
So that's how we've been collaborating. But it I right now it for sure works on small teams. T B D on how it works in teams of more than fifteen, I would say.
22:39 Sam Rogers
Yeah. Yeah, and for the agentic threshold, doesn't that break down at like right around the same number?
22:46 Robert Alvarez, PhD
Yeah yeah.
22:48 Sam Rogers
Coincidence? Uh possibly,
to get to my subtraction, so back to the the dollar figures thing.
22:54 Robert Alvarez, PhD
Mm-hmm.
22:54 Sam Rogers
and just trying to estimate not on the enterprise scale level, but my own spend for
AI, how it is that I'm allocated subscriptions, but also the API tokens, which are managed kind of in a in a different billing way. so I was making estimates, but I'm now only keeping the costs that the provider itself is reporting to me.
23:14 Robert Alvarez, PhD
Yeah. Yeah.
23:15 Sam Rogers
because in the same model, in the same week as what I was talking about before,
One call's own receipt came back at like four tenths of a cent. And another one was five hundred and seventy-nine tokens long, but said that it cost zero. Like it it they it Yes. So if you've
23:34 Robert Alvarez, PhD
Oh a rounding error.
23:37 Sam Rogers
been like multiplying tokens by a price table, I like it's it you we don't always have the ingredients to do that. So I'd have my own guess.
But there's a trap for us that are doing those reporting costs like to try and figure it out in advance. An estimate can feel like a control, but it's not like the bill. And because the ways of calculating those costs are changing all the time. filling in the unknowns with my own price table was well.
Tell me if I'm using this correctly, Robert. I've learned the term from you. Basically p-hacking my own budget.
24:14 Robert Alvarez, PhD
yeah.
24:15 Sam Rogers
when your finance is asking you about AI costs,
Giving them what the vendor actually states and marking the rest as unknown is probably a good
24:24 Robert Alvarez, PhD
Yeah.
24:24 Sam Rogers
idea because unknown is honest it's a better estimate in a lot of cases for what you're going to get. Like it is an unknown. any thoughts about that from your perspective, Robert?
24:33 Robert Alvarez, PhD
Yeah,
so you know, you you stated earlier on as well that that even with perplexity, there is like just some things that you're like, Am I getting charged for this? And ultimately I think you might be able to use AI to start classifying some of these errors so that you could say, Okay, here is what we're considering detailed costs and like we know every every dollar that was here was used for. And then we might have some buckets of like an error.
or
unknown but like maybe ran out of tokens like or you know like you could start categorizing some of the failures that you're still getting charged for and then an understanding are they a small percentage of your overall bill like is it two percent of spend in which case you're like okay well two percent of errors might be reasonable it might not be but might be reasonable or is it like wait I'm basically paying half of my bill to some things that I just don't know
And that's where you might want to figure out is there something actually in your pipeline that's causing it? Because it's usually might not be the provider itself that is the is the root cause of this. Like 1%, sure, you know, Anthropic had overloaded or like ChatGPT had too much load, so it erred out. but if it's a much bigger percentage, there may be something in your process that isn't getting the right information back for you to get.
a detailed view of that spend.
25:53 Sam Rogers
Love it. Thank you so much for that. And
25:55 Robert Alvarez, PhD
No problem.
25:56 Sam Rogers
and that makes a a good segue actually for the takeaway that I made for this week, which is the receipt ledger being able to take some of what Robert was just talking about and what I've been sharing to make your own way of tracking this. and also I have an open repo
that you can find at delegated.watch.
If you would like to create a multi-model token dashboard to just count the tokens and to mine the information like labeling, like Robert's talking about, for its own sake so that you can learn from it. But when it comes to the the then pricing and comparing that and being able to predict your spend and your budget, the receipt ledger hopefully will be useful to you.
not endorsed by Robert. This is this is me just anticipating what we would talk about in today's episode. And like every episode, we have a specific takeaway that you can use immediately. you'll find one for every one of our now 20 episodes of signals and subtractions
27:01 Robert Alvarez, PhD
congrats.
27:02 Sam Rogers
anything else you wanted to bring to the table here, Robert, that that you'd like to share?
27:06 Robert Alvarez, PhD
Yeah, so every code, every agent or every model that you're using is a generalist. If you have tasks that you do consistently over and over and over again, start turning that model into a specialist for your task by optimizing that prompt, kind of like we did with GEPA
because you'll see that it gets much, much better and you can probably reduce your spend by switching to a cheaper model that is just much more directed at that task. I think that's that's gonna be very helpful for for folks.
27:33 Sam Rogers
That's great. Well, if people want to read your prolific writings, follow your
27:37 Robert Alvarez, PhD
Mm. Yeah.
27:38 Sam Rogers
work, where do you send'em?
27:40 Robert Alvarez, PhD
I almost always post on LinkedIn, so you can always find me at Robert-L-Alvarez, or if you even go to Everpuredata.com, you can just go straight to my author page and from there you'll see all of my articles that I write,
27:53 Sam Rogers
Yeah, yeah. Well, thank you again
for all that you're putting out there. always inspiring stuff. and I'll make sure and put whatever links in the show notes as well.
thank you so much for being on the show.
28:03 Robert Alvarez, PhD
Yeah, my pleasure.
28:04 Sam Rogers
We do this live every Wednesday. Episodes drop on Friday and YouTube and all the podcasting places. Sunday is issue 77 of the newsletter.
And if you're listening on audio right now, please tap follow. I hear it brings good luck. Now it's been extended with a reset to 14 days. good fortune for you and your family
28:23 Robert Alvarez, PhD
Mm-hmm.
28:24 Sam Rogers
if you simply click follow or subscribe or like or you know whatever it is that gets this in front of you this time next week. Everything about the show lives at sigsub.show. One signal worth watching.
One subtraction worth making. Now go find yours.
Keep the next choice in view
One signal, one subtraction and one analogy every Sunday. Five minutes, no video required.
Optional. Free on Substack, no account needed. Already on LinkedIn? Subscribe there.