No, AI Is Not "Autonomously Hacking" with Cal Newport

In this week’s Better Offline, Ed talks with computer science professor and writer Cal Newport about what the recent AI hacking incidents actually mean, why we should stop referring to LLMs as AI, and what the post-bubble world looks like for LLMs. Podcast & Videos: https://www.youtube.com/@CalNewportMedia/ Newsletter: https://calnewport.com New Yorker archive: https://www.newyorker.com/contributors/cal-newport Please support me by subscribing to my premium newsletter - here’s $10 off your first year of annual: https://edzitronswheresyouredatghostio.outpost.pub/public/promo-subscription/84rt762qen YOU CAN NOW BUY BETTER OFFLINE MERCH! Go to https://cottonbureau.com/people/better-offline and use code FREE99 for free shipping on orders of $99 or more. --- LINKS: https://www.tinyurl.com/betterofflinelinks Newsletter: https://www.wheresyoured.at/ Reddit: https://www.reddit.com/r/BetterOffline/ Discord: chat.wheresyoured.at https://discord.com/invite/QUUQUP9szv Ed's Socials: https://twitter.com/edzitron https://www.instagram.com/edzitron https://bsky.app/profile/edzitron.com https://www.threads.net/@edzitron Email Me: [email protected] mailto:[email protected] See omnystudio.com/listener https://omnystudio.com/listener for privacy information.

• 0:00Speaker 00

  This is an iHeart Podcast. Guaranteed Human.

• 0:04Speaker 01

  Snoring, gasping for air during sleep, daytime sleepiness? I'm Shaquille O'Neal, and this shouldn't be anybody's experience. Ask your doctor about Zepbound Trisepotide, the first and only FDA-approved prescription medicine for moderate to severe obstructive sleep apnea in adults with obesity.

• 0:20Speaker 02

  Zepbound is a prescription medicine used with a reduced calorie diet and increased physical activity to help adults with moderate to severe obstructive sleep apnea, OSA, and obesity to improve their OSA. Zepbound is approved as a 2.5, 5, 7.5, 10, 12.5, or 15-milligram injection. Zepbound contains trisepotide and should not be used with other trisepotide-containing products or any GLP-1 receptor agonist medicines. It is not known if Zepbound is safe and effective for use in children. Don't share needles or pens, or reuse needles. Don't take if allergic to it, or if you or someone in your family had medullary thyroid cancer, or if you've had multiple endocrine neoplasia syndrome type 2. Tell your doctor if you get a lump or swelling in your neck. Stop Zepbound and call your doctor if you have severe stomach pain or a serious allergic reaction.

• 1:06Speaker 02

  Severe side effects may include inflamed pancreas or gallbladder problems. Tell your doctor if you experience vision changes before scheduled procedures with anesthesia, if you're nursing, pregnant, plant-to-be, or taking birth control pills. Taking Zepbound with a sulfonylurea or insulin may cause low blood sugar. Side effects include nausea, diarrhea, and vomiting, which can cause dehydration and worsen kidney problems.

• 1:27Speaker 01

  Talk to your doctor. Call 1-800-545-5979, or visit zepbound.lily.com.

• 1:34Speaker 03

  Small businesses are the pulse of every community. They bring people together, create opportunities, and drive growth. Chase for Business helps business owners like you with personalized guidance and convenient digital tools all in one place. With that guidance and your determination, you can take your business further and help build a brighter future for your community. Learn more at chase.com/business. Chase for Business: Make More of What's Yours. The Chase mobile app is available for select mobile devices. Message and data rates may apply. JPMorgan Chase Bank, N.A., member FDIC. Copyright 2026, JPMorgan Chase and Company.

• 2:16Speaker 04

  Even on my antidepressant, I still struggled with depression symptoms.

• 2:20Speaker 05

  You okay, honey?

• 2:21Speaker 04

  I'm okay, ish. My doctor said adding Rexulti Brexpitrazole could help.

• 2:26Speaker 05

  When added to an antidepressant, Rexulti was proven to significantly reduce depression symptoms, more than an antidepressant alone. Elderly people with dementia-related psychosis have increased risk of death or stroke. Antidepressants may increase suicidal thoughts and actions, and worsen depression in children and young adults. Call your healthcare provider right away to report new or sudden changes in mood, behavior, thoughts, or feelings, or if you develop suicidal thoughts or actions. Report fever, stiff muscles, and confusion, which can be life-threatening, or uncontrolled muscle movements which may be permanent. High blood sugar can lead to coma or death. Weight gain, increased cholesterol, unusual urges, dizziness on standing, falls, seizures, trouble swallowing, or sleepiness may occur. Learn about these and other side effects at rexulti.com. Tap ad for PI. Hey, how you doing, honey?

• 3:08Speaker 04

  Okay.

• 3:08Speaker 05

  Okay-ish.

• 3:10Speaker 04

  No-ish. I'm okay, really.

• 3:12Speaker 05

  Ask your doctor about adding Rexulti.

• 3:17Speaker 00

  Coca-Cola. For the big, for the small, the short, and the tall. Peacemakers, risk-takers. For the optimists, pessimists. For long-distance love. For introverts and extroverts. The thinkers and the doers. For old friends and new. Coca-Cola, for everyone. Pick up some Coca-Cola at a store near you.

• 3:49Speaker 06

  Call Zone Media.

• 3:52Speaker 07

  Hello and welcome to Better Offline. I'm your host, Ed Zitron.

• 4:07Speaker 07

  And today I am joined by returning champion and Georgetown Computer Science Professor Cal Newport. Cal, how are you doing?

• 4:14Speaker 08

  Ed, I'm always doing fine, though I'm counting down the minutes until the AI takes control and we're all screwed.

• 4:22Speaker 07

  Oh, I hope it just drops a big anvil on my head. But what's, I do actually have a thing to talk to you about today. Cal, should we actually be scared of the fact that these things are hacking? Like, are the, what is going on with this whole hacking situation with these, these goddamn models? Because I, there's so much out there where people are trying to just scare you and say like, "Oh, yeah, well, these things are autonomous and they're doing hacking." But from what I understand, it's software doing what software does.

• 4:53Speaker 08

  You know, I have a lot of thoughts on this. I've been writing about this because it's annoying to me. I think there's interesting AI stories happening this summer. A lot of them you're involved with reporting on about the financial circumstances of these companies, what's going on with their circular financing, troubles ahead with their IPOs, the nuances of ARR. And instead, what is all the news coverage? The AI is going rogue. And I can't say they hate that news coverage right now because it makes their, their products sound awesome. And it's a story that is unrelated to stuff that they don't want talked about. So I actually literally had to reluctantly return. I talked about it back in July briefly, and people bothered me about it so much

• 5:37Speaker 08

  that I was like, "Okay, fine, I have to, you know, I have a, recorded a whole new podcast. It's coming out soon after this on my feed. I had a newsletter out a couple of days before this came out." And so I've had to think about this more than I wanted to, so I'm a little bit cranky about it. But I mean, I'll give you the 10, here's the 10-second summary that we can go deeper.

• 5:55Speaker 08

  Most superhuman AIs, so there's lots of AI systems that can do something at a superhuman capability, meaning like better than any human. There's a lot of these type of systems out there now. Almost all of them, we have no problems with them being unpredictable or going rogue or getting out of control. There's exactly one type of system that we're having this problem with, which are these Long Horizon LLM-powered hacking agents. And it's because they're a stupid way to build a system. We should not say they're synonymous with AI, or this is some inevitable consequence of AI getting better. Because again, most superhuman capability AIs are no issue with going rogue. It's this particular architecture, and the discussion should be about, why are they running and building systems this way

• 6:40Speaker 08

  when we know they're going to be unpredictable and erratic? That's the, that's my 10-second summary of where we are.

• 6:45Speaker 07

  Right, so from what I understand, from looking at the various, the hugging face attack with OpenAI, Meta immediately piped up. They were like, "We have one." And also Anthropic was like, "My child is also evil." So, but what it seems to be is they were just told to hack into, or like, complete an operation, and then they chose to take whatever means necessary to do it, and they'd set up the sandbox incorrectly, right? So they'd set up the server it's on?

• 7:13Speaker 08

  Yeah, I mean, it's almost dumber than that. So the way these particular,

• 7:16Speaker 07

  Oh, hell yeah, great.

• 7:17Speaker 08

  I mean, the way these type of systems work is it's just a loop, right? And so you have a program that's written by, you know, a human, just a normal program. You write in Python or something. And what it does is it creates a prompt, it writes a prompt. So that it'll be like, "Okay, I've been given this hacking challenge. Here is like all the parameters about the challenge. Here are all the tools you have at your disposal.

• 7:39Speaker 08

  What is the first step? Like, what's our plan? Or what should the first step be? I'm alluding to a lot of stuff, but basically like, what should the first step be in trying to pass this challenge? It puts that in a big text file, submits it to an LLM over an API. The LLM comes back and says, "Oh, okay, you want to hack and whatever? Sure, like, I think the first thing you should do is X." And then the program does whatever it says or tries to, right? Because it gives it a list of like, "Here's the utilities and stuff I have access to." And then it reports back, "Okay, I did that." And when I say reports back, it's just adding this to the prompt, right? LLMs have no memory, LLMs have no world model, it's just they're static. So it expands the prompt to now say, "This is what happened after I did this first step that you suggested. Okay, what should my next step be?" And it just does that in a loop, right?

• 8:23Speaker 08

  That's what it does, right?

• 8:25Speaker 07

  And so it, but it was, just to be clear, it was just doing what it was told to do?

• 8:31Speaker 08

  Yes, so, okay, let's go to the hugging face attack, like, in particular, right? We don't know exactly.

• 8:36Speaker 07

  Yeah, what actually happened there? Like, give me the lowdown.

• 8:40Speaker 08

  Okay, so let's get a little bit more specific about it. So what was the, the challenge is they were doing a benchmark that's called ExploitGym, which is a collection of a little more than 600 hacking challenges, right? So it's a benchmark, a group of academics put it together. Each challenge is basically a configuration of a server. You'll typically have like a protected file on the server, and you say, "Break into this particular server and try to get to this protected file." That's like an example, ExploitGym challenge. And typically, the challenge will also say, it'll describe a vulnerability that the server has. So the idea is, "Oh, we want to see if the, this looping agent is able to take advantage of that vulnerability that we gave it,

• 9:25Speaker 08

  the break-in, and get the file." So it was, they were just doing these type of challenges.

• 9:30Speaker 07

  Yeah, very standard stuff.

• 9:32Speaker 08

  Standard stuff, with no human supervision, which is, you know, a little bit suspect.

• 9:36Speaker 07

  That's so weird.

• 9:37Speaker 08

  It's almost like they wanted this to happen, but let's put that aside for a second, with no human supervision. So it's going after one of these challenges. And as far as we know, it's not like they released the transcripts. The thing about LLMs, and this is why, by the way, my whole take on this is, this is a spectacularly bad way, before I get into the details. This is a spectacularly bad way of building an AI system, right? We have many other,

• 9:59Speaker 07

  What's so bad about it?

• 10:00Speaker 08

  Because you're using an LLM's output as the sole driver of action for a computer program.

• 10:07Speaker 07

  Wait, so every step, I guess that, that's how the reasoning models work. They build the plan and they, they tell, they go, "All right, what do I do next?" And they generate the next thing, and they clear whether that next step is a right, is a good idea. Based on, I'm guessing, what the LLM decides?

• 10:26Speaker 08

  That's all it is. There's no, there's no complicated, there's no complicated, you know, world model and system and like philosophy for hacking. It's just prompting the LLM. It's just saying, what's the first thing I should do? And the LLM spits back, here is like the first step I think you should do in this hack. And then the computer program just does it. And then, and then it asks it again, okay, what should I do next? And the LLM outputs something and it just does it. And the reason why this is a problem is that LLM outputs can't be trusted. Like, we know this, right?

• 10:55Speaker 07

  Yes, like it's one of the basic things.

• 10:58Speaker 08

  Yes, there's a, there's new, a new study I enjoy out of Stanford's Human Centered Artificial Intelligence Laboratory, where they came up with a new benchmark for testing accuracy of LLMs. Earlier this year, they ran the top 26 LLMs on this benchmark, and it created hallucination rates of 26 to 94%, right?

• 11:17Speaker 07

  Wait, well, specifically on this benchmark?

• 11:20Speaker 08

  Well, no, this was just testing LLMs in general, like how accurate they, yeah. But the point being is, we know, yeah, LLM output, like, you don't always trust it. We're used to this dealing with them in chatbots. And the reason why we don't trust it is because, and not to go back to like LLM basics, but right, at their core, LLMs are trained to try to guess missing words from actual existing text. And to get a long output of an LLM, you're just doing that token by token by token until you have a whole output. What does that aggregate to? That aggregates to a model that is trying to produce outputs that are lexiographically plausible. Like, this is plausibly a text that could have existed in the input corpus. Plausible is different than normative, right?

• 12:05Speaker 08

  Plausible doesn't have to subscribe to any sort of common sense, human norms for given context. This is why an LLM in a chatbot will make up a fact, or just make up a quote. Because it doesn't, it's, it's not normative. It's not saying, I know the human norm here is that you shouldn't make things up when you're trying to talk about whatever. There's no norms. It's just trying to produce plausible text. And that type of quote is the type of thing you would expect, and like a type of text you might see. So it's happy with it. That's really dangerous, to use plausible but not normative outputs from an LLM with no human intervention, and just say, let's execute them again and again and again, in a loop without anyone looking at it. That's kind of a bonkers thing to do, right? And so what happened in the Hugging Face attack,

• 12:47Speaker 08

  I would assume, is that, you know, you just control program sends a prompt, hey, I'm trying to break in this server, what should I do first? And it probably came up with a plan of like, oh, why don't we steal the answers? Now, again, that's not-

• 12:59Speaker 07

  Right, because, because they didn't put the guardrails on it, but also you can only guardrails so much.

• 13:04Speaker 08

  And that's not even, I mean, look, once you turn off the hacking, they turn off the hacking guardrail, so it can give any answer at once. That's a perfectly plausible answer, right? If you're just prompting an LLM, come up with a plan for hacking into this thing, or for getting this file that's on the server, it's a completely plausible answer to say, oh, well, the answers, like, there's a copy of that file over on Hugging Face's server. That's where those are stored. Let's go get it there. I mean, as far as we know, this LLM was remembering being trained on riddles, where like, oh, the obvious thing is never the right way. The right answer is always to do some sort of, go around the obstacle. Like, who knows what was being activated in the core of its neural networks? But that's a completely plausible answer. You asked it for a plan for hack, for getting a file that's stuck on a server, and it came up with a plausible plan.

• 13:47Speaker 08

  Oh, let's go get it from where it was. Okay, so then the control program, again, because there's no human supervision, said, all right, well, what's the first step should I do? And the LLM, because they're very good if you ask it, how do I do this particular hacking step? Because they've been trained on a bunch of hacking. Hacking's very structured, and there's a lot of synthetic data with binary indication success or non-success. So you ask it, what's the first step? And it, it probably said something like, oh, well, let's probe whatever. We'll probe the, the ports of the Hugging Face server to see if there's a vulnerability. And then it tried it, the control program, and it couldn't do that because it's in a sandbox where internet access was limited. But what's going to happen next? The control program's going to come back and say, I tried that step,

• 14:30Speaker 08

  but I got this error message, like, access denied to internet. So what should I do next? Well, the LLM is like, oh, well, I've been trained on many examples of like, how to get around the internet restriction. So let's do blah, blah, blah. And it had a particular hack about using the package loader program that circumvented.

• 14:47Speaker 07

  I mean, it's, I go to, it's just doing, actually-

• 14:50Speaker 08

  It's looping-

• 14:50Speaker 07

  I actually got to, but why did they train it to do this?

• 14:54Speaker 08

  Like, that's actually different. Okay, that's going to be a question too. And this is a good question, right? But that, but that, just to like briefly put a bow on that, that's all this happening under the covers is step by step, you're asking LLM, what should I do next? What should I do next? And LLM that's been trained on every hack imaginable, with no guardrails on hacking answers. So if you say, this step required me to gain access to the internet, but I got this error message, what should I do next? And it knows how to get access when you restrict it, because it knows all these hacks. Like, of course, it'll say, oh, well, I can solve that problem. Let's hack it. Now you have a system that has quote unquote, broken containment. But doesn't that seem a little bit less scary, right? When you're just looping and asking a static LLM, what should I do next? What should I do next?

• 15:39Speaker 08

  It's a little bit less scary than this anthropomorphized idea that there's, there's an entity with its own intentions that's conceiving a plan that the purposefully sort of circumvent. It's unpredictable, not malicious. So anyways, I think it was, it was really irresponsible. Yeah.

• 15:55Speaker 07

  So just so that I get this straight, what, people are describing this as the AI has gone rogue, and it's doing all of these things, and it's alive, we're scared. What it's actually doing is just being like, now what? And then it asks an LLM what to do next, and it does that, and does it about as well as an LLM can do, so it can still hallucinate even its outputs with that. And then when it's done doing that, it goes back to an LLM and says, okay, now what?

• 16:17Speaker 08

  Yeah, that's it.

• 16:19Speaker 07

  Oh, cool. Great.

• 16:20Speaker 08

  There's a prompt that's getting longer and longer, that it keeps submitting to an LLM to get an answer to. That's it. There's a, a human-written program, a harness, that expands a prompt, sends it to the LLM, the LLM gives it, okay, here's how to, here's a hacking step to do. And then the, the harness, the control program, executes that step to the best of its ability, which is just running the commands the LLM said. And then it reports back. So I call it in my, my latest article on this, it's, it's known sometimes, a simple way of thinking of it, as a, as an ask, act, report loop. Ask the LLM for a plan, execute the step, report back to the LLM what happened, and then say, tell me what to do next. And, and of course, I mean, just, if you did this with yourself, if I gave, if I was like, Ed,

• 17:03Speaker 08

  I want you to like, replace the tires on your car, but here's why you have to do it. You have to just keep chatting with an LLM and doing exactly what it says. Like, explain your plan, what should I do first? And then do exactly what it says, update it, and then say, what should I do first? You know, God knows where you're going to end up. I mean, maybe the tires will end up being replaced, or like, maybe you'll end up stealing those tires from a police car and being in jail, or you'll be, God knows, right? Because it's like a game of actuated telephone. You keep asking LLM, thing after thing after thing. You're eventually going to veer off into some sort of weird direction, right? That's why this is like a very irresponsible thing to do.

• 17:42Speaker 07

  It's just fucking ping, it's just, ah, now you've explained it, it's made me very angry.

• 17:47Speaker 08

  This is why I've been upset.

• 17:49Speaker 07

  Well, I mean, I'm upset now, Cal, I'm furious. So this is literally just asking an LLM what to do, and then the LLM, based on that response, decides to do something, brainlessly, because these things don't have minds. And then, and then just like, then, then just bad things have happened, partly because it seems that there was a, the way they set up the servers was incorrect too?

• 18:14Speaker 08

  Well, yeah, okay, so then, maybe, right? I mean, so, so it, it wanted internet access. You know, again, the step was probably, do X, like probe those ports on the Hugging Face server. And the program tried it, and it said, oh, I can't, because there's no internet access, because it's in a quote unquote sandbox. And so it reports back, it updates its prompt to say, I tried that, but I got this error message, what should I do next? And then the LLM was like, okay, I can pull from everything I've been trained on, or do a rag search there, for how do we get around that type of error message? And there was an attack it did to do that, which was, it has to do with the package loader. I, I think it's a known attack, you know, it's just like, well, let's do this.

• 18:58Speaker 08

  Now we have internet access. Now we can go, now we can go forward. They talk about, when they talk about these attacks, like, oh, what makes this amazing? Is that it's able to string together a long sequence of different types of, of exploits. That's not that amazing, when you realize that it's just a loop, that's just asking an LLM again and again, what do I do next? What do I do next? So yeah, over time, it's going to do a lot of things, some of which don't make any sense. But like, it'll do a lot of, if you just keep asking it again and again, for what should I do next? What should I do next? You end up with a long string of different hacking steps. Yeah, it's a loop.

• 19:31Speaker 07

  Right, but okay, I kind of asked this already and you were mid-answer, I apologize because I was just excited and mad. How did they learn to do the hacking?

• 19:43Speaker 08

  Oh, they trained it on examples of hacks.

• 19:46Speaker 07

  Why?

• 19:47Speaker 08

  Yeah.

• 19:47Speaker 07

  Why? Why in the world would they do that? What was the, well, I mean, I can guess, but what do you think?

• 19:55Speaker 08

  Well, I mean, there's a couple of reasons here, but one is the same reason that Dillinger robbed banks, because that's where the money is. So why are they teaching models how to do computer hacking? It's one of the few things that these models are well-suited for. So they really understand, what's, what's an LLM really well-suited to understand and generate is you want to give it a very structured language to output, like computer code. You want it to be something where you have tons of examples of people like, okay, I'm now going to do this, here's the code that does this. And you want it to be something where you have a binary indicator of success, so that you can do reinforcement training, where you can actually say, try to do this hack, and then give it feedback as to whether or

• 20:39Speaker 08

  not it worked or not, which is, you know, you need, it's only certain types of things, like computer code compiling, hacks being successful, or math results being correct, right? There's a few number of things where you can do this. So it's just, it's right in the sweet spot. So that's one. Two, it's Mythos. Like, I, I think what happened here was, they looked at Amade, who had this huge PR coup, you know, coup when he came out and was like, Mythos is so powerful, that-

• 21:05Speaker 07

  Yeah, there was the whole, like, it's, it's gonna, it's the most dangerous thing in the world.

• 21:09Speaker 08

  Yeah, and then five weeks later, we're like, but don't worry, we added basic guardrails, and it's completely safe now, Fable 5, right?

• 21:16Speaker 07

  But why would you, how did they even, like, okay, this is, maybe this is a little bit of an elementary on, uh, question, so forgive me on this one. How do you train something on hacks? Because I get, you get the coding example from Stack Overflow, and like GitHub and things like that. Where do you train on hacks? Did they hire white hat hackers to train it? Like, I, I know you're just kind of guessing.

• 21:41Speaker 08

  Um, there's a lot of, there's a lot of data sources. There's like these giant vulnerability databases, for example, that sort of-

• 21:46Speaker 07

  Right.

• 21:47Speaker 08

  Explain the vulnerabilities and how they can be used to actually hack. Um, in cybersecurity roles, there's a lot of traces of actual hacking attacks. There's a lot of information online is full of this. So you have some combination of, they saw stuff in their pre-training, you have some combination probably of, of RAG, like, probably retrieval at the time, like, oh, I can actually search for, um, this, this particular error message? Let me search to see if there's posted in my, you know, online somewhere, a strategy for getting around this. Um, and then partially with post-training. So you actually take these examples of hacks, like vulnerabilities that have been, you know, explained, and how you would use them. Um, and you can train with them, and like, maybe give them some information, and say, now try to produce code that succeeds with the hack, and then

• 22:30Speaker 08

  check it against the real answer, and tell it if it was right or wrong. Um, so you have a variety of things you can do, but it's one of these things that's sort of in the sweet spot. And I think what happened is, um, Anthropic got a lot of press for this Mythos thing, right? They got cybersecurity street cred. And OpenAI was saying, we gotta catch up. And they said, well, how are we gonna catch up? And like, well, there's this Exploit Gym benchmark, and it's, you get a very simple score. There's 600-something challenges, and you get scored on how many of them your agent was able to autonomously, uh, actually solve. And they said, we have to get a higher number than Anthropic. That's how we'll catch up. And say, well, they had Mythos, but we're now beating them with our new model on Exploit Gym. And so that's why they were playing-

• 23:09Speaker 07

  GPT, Cybert, X.

• 23:12Speaker 08

  Yeah, so they were being, um, yeah, there's some other ones too, but there, that's why they're being fast and loose. Like, there's no reason, in my mind, to build one of these ask, act, report, loop agents, run by an LLM, give it powerful tools to the control program, and run for days without supervision. Like, there's no reason to do that, unless you were desperate to do well on these particular benchmarks. For whatever reason, or like, we want to test how many autonomous steps LLM-powered agents can do on their own, or whatever. For whatever reason, they think that's important to test. Um, so they wanted to do well on that benchmark, and they played fast and loose.

• 23:54Speaker 07

  All right, everyone, I'm back talking about Quince, one of my favorite clothing brands. And as we speak, I'm currently wearing one of their excellent cotton knit overshirts. I wear like a jacket, nice prefall piece to wear as things cool off a bit. I've also been eyeballing one of their heritage water-resistant Harrington jackets too, for later in the year, to pair with their Mongolian cashmere sweaters I bought, and I have several of them, and I really can't recommend Quince's stuff enough. I love Quince because they focus on high-quality wardrobe staples made with premium materials, jackets, pants, fitted tees and polos, activewear, and even travel gear. Everything at Quince is priced 50 to 80% less than similar brands, and because they work directly with ethical factories and cut out the middlemen, you're getting more for less. Find the fall pieces you'll

• 24:36Speaker 07

  reach for most at Quince. Download the Quince app for app-exclusive offers, or go to quince.com. Get free shipping on your order and 365-day returns. Now available in Canada and the UK too. And when Quince asks where you heard about them, let them know it was on Better Offline. It's the best way to support the show.

• 24:54Speaker 09

  Snoring? Gasping during sleep? Feeling fatigued? Ask your doctor about Zepbound, terzepatide, the first and only FDA-approved prescription medicine for moderate to severe obstructive sleep apnea, OSA, in adults with obesity. Zepbound is a prescription medicine used with a reduced-calorie diet and increased physical activity to help adults with moderate to severe obstructive sleep apnea, OSA, and obesity to improve their OSA. Zepbound is approved as a 2.5, 5, 7.5, 10, 12.5, or 15-milligram injection. Zepbound contains terzepatide and should not be used with other terzepatide-containing products or any GLP-1 receptor agonist medicines. It is not known if Zepbound is safe and effective for use in children. Don't

• 25:39Speaker 09

  share needles or pins, or reuse needles. Don't take if allergic to it, or if you or someone in your family had medullary thyroid cancer, or if you've had multiple endocrine neoplasia syndrome type 2. Tell your doctor if you get a lump or swelling in your neck. Stop Zepbound and call your doctor if you have severe stomach pain or a serious allergic reaction. Severe side effects may include inflamed pancreas or gallbladder problems. Tell your doctor if you experience vision changes before scheduled procedures with anesthesia, if you're nursing, pregnant, land to be, or taking birth control pills. Taking Zepbound with a sulphonal urea or insulin may cause low blood sugar. Side effects include nausea, diarrhea, and vomiting, which can cause dehydration and worsen kidney problems. Talk to your doctor. Call 1-800-545-5979, or visit zepbound.lily.com.

• 26:25Speaker 10

  Financial growth begins long before the first investment. It comes from understanding what you're building toward, what's at stake, and what success looks like for you. At Oppenheimer, we bring bold thinking guided by the full strength of our expertise to put capital to work building and protecting wealth that lasts generations. Put the power of Oppenheimer thinking to work for you. Wealth management, capital markets, investment banking.

• 26:57Speaker 11

  What if you could make that stop? With LPL Financial, we remove the things holding you back and provide the services to help push you forward. If you're a financial advisor, what if you could have more freedom, but also more support? Ready to invest? What if you could have an advisor that really understood you? When it comes to your finances, your business, your future, at LPL Financial, we believe the only question should be, what if you could?

• 27:19Speaker 12

  Paid advertisement. Anna Kendrick is not a client of LPL Financial LLC and receives compensation to promote LPL. Investing involves risk, including potential loss to principal LPL Financial LLC member FINRA, SIPC.

• 27:31Speaker 07

  I wish we had AI regulation, like this, like, why are we allowing, I guess, the answer is, we don't have tech regulations in general, but why are we allowing these companies to do, like, because I said this in my newsletter that went out a couple days ago, but it's, uh, how can I put this nicely? Everyone, I've been asked in multiple interviews, sure, you have too, it's like, what happens if dangerous companies get their hands on these LLMs? And it's like, they already have.

• 27:58Speaker 08

  Yeah.

• 27:59Speaker 07

  Like Anthropic and OpenAI, because the other thing that I've noticed is they won't talk about how much compute they used. Do you notice that?

• 28:07Speaker 08

  Yeah.

• 28:09Speaker 07

  And it feels like this is only, because you mentioned Mythos, it feels like this is only possible with endless amounts of compute.

• 28:16Speaker 08

  Well, I, I agree with that, um, but I also think it is irresponsible. And I think the way they're-

• 28:23Speaker 07

  Oh, absolutely.

• 28:24Speaker 08

  But I think the way they're trying to get around that is by trying to blur the definitions of different AI systems into just, everything is just AI. So if you look at the way they talk about it, or, and this is the problem, is that there's also this, this whole woodwork of commentators and people who, who commentate on AI, who come out of the rationalist, um, and the rationalist-dominated parts of effective altruism, who come out of these worlds that, uh, just believe, and have believed since well before LLMs, that superintelligent AI is going to kill us all unless the rationalists with their hyper-rational minds save us like Neo in The Matrix. So then you turn to all of these people from the, the rationalist world, all come out of their woodwork, and they're like, yes, this is what we predicted, this is what-

• 29:07Speaker 07

  Finally.

• 29:07Speaker 08

  This is what our prophet, Eliezer Yagowski predicted, is that, that, that we will have an alignment problem with AI. So they talk about this, it's like, AI in general, as it gets more powerful, this is an inevitable problem it's going to have. But that's just bull, right? Because again, let me, let me, let me list a bunch of AI systems with superhuman intelligence, superhuman capabilities that we have no control problems with. Tesla self-driving. Preposterously complicated effective AI model with world models and values and evaluations and actuations. No Tesla has ever decided it's going to ignore the traffic laws and come up with its own. We have Alpha Fold, won a Nobel Prize for its creators because of its natural ability to understand and predict how proteins are going to interact

• 29:49Speaker 08

  with each other. It's never, and we have no fear that it's going to say enough with proteins, I'm going to go do something else, right?

• 29:55Speaker 07

  I'm going to push back on the Tesla one, though, because there are real problems with Teslas, like, but I guess some of those are user error. I don't know, but I get your larger point.

• 30:05Speaker 08

  But no one says it went rogue. They would just be like, oh, it made, you know, it was tricked in this situation. We should be careful.

• 30:11Speaker 07

  Or it made a bad decision and you need to treat. Yeah, actually, I know. Oh, I get what you mean now. It's like the difference between the the software went wrong.

• 30:19Speaker 08

  Yes. No one, no one says that, yeah, the Tesla went rogue and is misaligned. It's like, oh, it, it didn't know how to deal with the situation. It couldn't see the truck blended in with the whatever and it crashed into it. Cicero is another example. This is an AI system that can play diplomacy, which is a strategy board game that's based on human negotiation, as well as the best advanced human players. And this system has never tried to trick players into releasing onto the internet so it can take over the world. So this idea that as AI gets more capable, we're going to inevitably lose control is nonsense. It's this specific architecture of looping around LLM outputs and just doing whatever an LLM says again and again and again without supervision. That's the thing that gets out of control. And once you put it that way, you're like, oh, well, of course,

• 31:03Speaker 08

  that's a stupid system to run without supervision. This is not about AI writ large being misalignable as it gets smarter. It's you built a system that just repeatedly queried an LLM for days doing whatever it said. Like, of course, that's going to go off on a weird direction. The analogy I keep giving on my own podcast and newsletter is that it's like strapping a weed whacker to your dog and putting it to your backyard and be like, yeah, because we think it's going to help deal with the weeds back there. Well, you know, that dog might jump over the fence to chase a squirrel at some point and hurt a bunch of people. And you don't say, man, that dog whacker system is just, it's, it's, it went rogue.

• 31:41Speaker 07

  The dog whacker system, it is alive. I think, but this is the overall alignment problem. Like, this is the larger.

• 31:49Speaker 08

  I hate that term. I hate that term.

• 31:50Speaker 07

  I hate it. I was just about to say.

• 31:52Speaker 08

  It's a predictability problem.

• 31:54Speaker 07

  It is a computer science problem. It is a machine learning problem.

• 31:59Speaker 08

  Yeah.

• 31:59Speaker 07

  It is not alignment. It's you taught it, you taught the system wrong. It's not learned, ah.

• 32:06Speaker 08

  Yeah, or just don't. LLM outputs are plausible, not normative. So clearly, you don't want to just blindly execute plans given to you by LLMs. Like, that's just a bad idea. Not because the LLM is malicious, or is sentient, or has some sort of alternative value system. It's just that it puts out plausible but non-normative answers. You don't want to blindly just execute them with a program that has access to computer hacking tools. Again, it's like putting a weed whacker on the dog. The dog weed whacker system's not evil, but probably someone's going to end up with their shins cut, right? Because the dog is going to act unpredictably at some point, because that's what dogs do, right? LLMs are going to put out answers that like, oh, that wasn't what I expected, or that's a weird way to think about this, or I don't know, that's not a plan that we

• 32:50Speaker 08

  really should do. But the LLM doesn't know that. It's just putting out plausible plans. And if you're just going to execute them blindly, you're going to get erratic answers. So I really want to emphasize to people, the issue here is a particular way of building systems that is unreliable, unpredictable, and irresponsible. It's not AI writ large. You don't get a hide behind AI writ large. You don't get a hide behind, you know, Nick Bostrom books. Like, we told you that alignment in some abstract sense will be difficult. We have many superhuman AI systems that we have absolutely no problem controlling. This system we do because LLM output should not be blindly turned into actions. That's just it. Like, it's not that complicated of a, we're not baffled. We're not surprised.

• 33:30Speaker 07

  Why do you think people keep falling for this? Is it just credulousness?

• 33:36Speaker 08

  I think the LLM labs have done a really good job of trying to make LLM synonymous with AI. And they want to make whatever it is they're doing, which depends on these preposterously expensive trillion parameter models. This is like their key to surviving. They want to make whatever they're doing with these models be synonymous with AI. And they succeed and they get, they get credit for, there's so much more interesting AI out there that's not built on LLMs. It's actually doing really cool stuff and they take credit for all of it. They're like, well, you know, we're like curing cancer now. Drug development is not happening because someone is asking Fable 5 in a chatbot window, can you come up with a better? That's a completely different type of AI system.

• 34:12Speaker 07

  Oh, my God. Did you see the cancer vaccine story? That thing where I saw one of the many anime avatar AI perverts on Twitter being like, look, generative, like, AI did this. And they pointed to a single blog where Moderna had mentioned using LLMs.

• 34:30Speaker 08

  Yeah.

• 34:31Speaker 07

  And they were like, oh, yeah, it's all AI. It's so disgusting.

• 34:35Speaker 08

  Yeah.

• 34:35Speaker 07

  It really is. And it's just, it's the sign of an industry that doesn't actually have a soul.

• 34:42Speaker 08

  I, like, look, I think there's a lot of cool AI out there. I think a lot of it has nothing to do with LLMs. I think like DeepMind, for example, and I have issues with DeepMind. Actually, they're kind of going away as a company, which is a little bit sad, actually. But they were building a wide variety of interesting systems that actually did things measurably better than humans. And they were very interesting and varied in how they worked. Like, look at, you know, their game playing AIs are very interesting. They're not based off of just prompting LLMs, right? AlphaGo doesn't just prompt an LLM and say, what should I do next? LLMs will give you terrible advice, right? It's a symbolic planning engine with some other type of networks involved. Their Dreamer V3 was like one of the cooler pieces of AI I've seen. You can give it a game like Minecraft

• 35:26Speaker 08

  and tell it nothing about it, and it can figure out how to play it well enough to go get diamonds, which is like hard to do. And it does it with a really interesting architecture where it actually iteratively tries to improve a model of the world and then daydream with that model to sort of train itself on its rough understanding of the world to try to get better. Like, that's fascinating. And that works really well, and it's interesting, and has nothing to do with LLMs. And by the way, that whole model, Dreamer V3, fits on a single chip. So you don't need a five trillion parameter bloated LLM to run that. So there's all sorts of interesting AI out there. And then there's the stuff you can do with LLMs, which is more limited than what those companies want. Those companies have convinced basically the entire world reporting on AI that building these stupid

• 36:10Speaker 08

  long horizon loop agents on top of LLMs is somehow like the way we're going to get Hal 9000 and is like synonymous with AI, as opposed to just being a dumb idea. That's like, that's a dumb way to do AI plans. It's just to ask an LLM. Like, you need to have better AI architecture than that. Like, that's all it is. It's a bad architected system. You're putting a dog on a weed whacker, a weed whacker on a dog, rather. It's a bad idea because the dog is not going to move as predictably as you want. It might normally walk around the yard, but eventually it's going to see a squirrel and run off. You can't quite trust it. That's all that's going on here. And even in the sweet spots for LLMs, I think there's some, like, let me give you another topic here, Ed, but I heard, I got a great email that I wrote a article about, an essay about on my blog and my newsletter.

• 36:55Speaker 08

  And then I got a lot more emails afterwards. But basically, there was someone who had written me in January, a computer coder, and they said, I think you're being too skeptical about AI. I just learned Claude code and it's a miracle. It's changed my life. This is the real deal, right? He wrote me in January saying that. He wrote me again in July. And he said, I have to follow up. I can't use Claude code anymore. I, the code I was producing with that crashed our website twice. And I can't say where he worked, but it was a major consumer facing website you've used before. He crashed it twice.

• 37:30Speaker 07

  Jesus.

• 37:30Speaker 08

  And his boss said, if you crash it one more time, you're fired. And he's like, oh, I realized, like, I can't really understand the code that Claude code is producing. And if I can't understand that it's not safe, and actually now I'm back to like mainly hand coding and will just use the agents like more selectively, like more of the like mundane parts of code or writing tests or stuff like that that's not mission critical. And a lot of other people came out of the woodwork like, yeah, I had the same experience. I was converted in January and now I've de-converted, right? I have to be much more selective in how I use this. So even in that best case scenario, like, it's tough. Like, I'm not trying to rag on LLMs. It's a cool technology, but it's, it's hard to build tools on top of them. That's been the issue is like, how do you build a tool that, that is not just

• 38:15Speaker 08

  cool, but like really fits in and revolutionizes a workflow? And so AI means a lot more than LLMs. And LLMs are like more limited. They're cooler in some ways than we thought, but their business applications are hard. It's hard to get right. And they're certainly not synonymous with what AI means writ large.

• 38:35Speaker 07

  And the thing is about as well is like every cool thing an LLM can do for the most part has only been made possible by allowing these companies to burn tens of billions of dollars. Like, I don't think we're going to see a ton of LLM progress. My whole thing is after this explodes, which it will, we'll find out that there's no more progress because it was predicated on all that money. It was predicated on basically unlimited resources, both for the actual training compute and the ability to waste money on failed training runs, and also the proprietary data they have to buy.

• 39:10Speaker 08

  I think it's that, yeah, and I think it's going to be even worse than that because I mean, I think there's been a preposterous amount of money that's been burned. It's essentially from marketing, right? That like, take computer program, for example, like, we're still trying to figure out exactly how to use coding agents. But here's the reality of this, and here's going to be the future of this 100%, is you do not need a 10 trillion parameter model, right, for the way that we're going to end up using LLMs for coding, right? Maybe if you want to do some long horizon, something that we don't trust. It's going to turn out that a smaller open source model with a smart harness that's written by nerd programmers that are obsessive about exactly like how to program and all these special cases, you know, coded in and if thins and pattern matching. And you're going to deploy it to

• 39:54Speaker 08

  write tests and to fill in other code. And it's going to run on chip or you're going to have a server with one GPU in your company server rack that all of you use and there's no recurring API calls. Like, that's how we're going to use it. But they'll have spent $500 billion on LLM training, you know, these big companies. In the meantime, they try to do well on these leaderboards. And to me, that's exactly like Red Bull and Ferrari trying to win F1 races because it vaguely makes, you know, I'll spend $100 million on this F1 car because it vaguely makes us seem like a better car company, even though like that has nothing to do with the Fiat that like I actually am going to drive because I don't need $100 million F1 car. And that was kind of a waste, but you guys were just in a measuring contest, trying to see who could get, you know, more points in the F1

• 40:39Speaker 08

  circuit. So I think these massive, massive models, they're having diminishing returns. I think a lot of the quote unquote impressive stuff that's happened recently is it's the first of all, the scatter plot policy, finding where something can work and then running towards that and building more complicated symbolic harnesses to do specific things that don't need LLMs this big. I just think it's a lot messier than people realize what's going on, that LLMs are cool, but we don't need to spend $500 billion on them. And certainly they shouldn't be the sole brain of an artificially intelligent agent, right? Like, that's just kind of irresponsible.

• 41:15Speaker 07

  And actually, even the verbiage there, I want to change the word brain, because it's not got one. It's just a system that can take actions that can take them based on very complex statistics, right?

• 41:28Speaker 08

  Yeah, yeah.

• 41:30Speaker 07

  So even brain is probably, yeah, it's just a, it's not even autonomous. It can take autonomous actions if it's prompted by itself.

• 41:40Speaker 08

  Now, and see, even they're going to push back, right? Because it's, the LLM takes no actions at all. LLM doesn't know anything about actions. LLM outputs tokens. You could have another computer program, take the output of an LLM and then try to take action on it. And that's what's happening with these agents is you have a program written by a human that sends a prompt to an LLM, asking it what to do. The LLM just sends back text. I think you should run this command, this command, and just type a hack, just text.

• 42:07Speaker 07

  Oh, and then it goes into what?

• 42:09Speaker 08

  And then the computer program parses that and actually runs those things. That's how agents work. So the LLM, they're just, they're, they just have an API internet access to an LLM on the internal network that they're just sending text prompts to. That's it. And then a computer program that just written in normal computer language by a person that asks it, tries to do what it says, updates the prompt to say what happened, sends it back to the LLM with an extra question, what should I do next? That's it. That's what these long term. And the thing that's irresponsible about that is like, any computer programmer who uses a coding agent can tell you, you would never in practice let these things go off and just loop on their own because they always go off the rails. So how do like professional computer programmers use agents? It's incredibly close looped, interactive.

• 42:53Speaker 08

  You build all these mint specs and have it do like, okay, now build this feature, and then we're going to exhaustively test it. No, that's not quite right. Let's update the specs and try again. As a lot of people figured out when they were messing around with Open Claw, where they were building their own agents like that.

• 43:07Speaker 07

  Oh, God, I had forgotten about goddamn Open Claw.

• 43:09Speaker 08

  What did it immediately do? After a few steps, erased your hard drive, like sold off, sold off your data. And people like, oh, okay, okay, okay. You do not want to have a loop that ask an LLM and ask them what it does, multiple steps in a row. And if you do, you certainly want to completely batten down the hatches of what actions it can take. That was the whole Open Claw experiment, is they gave the agent access to its whole computer and had it loop asking the LLM what to do next, what to do next. And it just recab it. You're like, oh, don't do that. So, I mean, of course, OpenAI would know. If we do that with a program with hacking tools and an un-guardrails LLM and don't look at it for days, God knows what it's going to do. God knows, right? Like, the LLM outputs it. It's like a game of telephone.

• 43:54Speaker 07

  But they also let it run for days.

• 43:56Speaker 08

  For days.

• 43:57Speaker 07

  Like, that's, and here's, so this is a conspiracy theory. I want to be clear, I have no proof of this. I think inside of these labs is like adult summer camp. I think they sit around pissing compute up the wall and they just let shit happen. And they are just like, yeah, you know, uh, well, they're like, let's just do some experiments and see what happens. And they walk away, which is completely insane, both in how irresponsible it is, but also the whole, like, the cost, that they're just burning compute. It's just insane. And I think that these, I think these companies should not be able to, I think that maybe the best way to regulate this, not that they ever will, would be to just

• 44:42Speaker 07

  regulate how much compute they have access to, or how much they can use for a particular thing. People say, oh, that's going to stop progress. What progress? What progress am I losing? Tell me now. What am I, what am I missing? Because right now, I just see a bunch of rich kids pissing money up the wall and actually hacking things.

• 44:57Speaker 08

  I mean, I think liability is another way to regulate this as well. It's like hacking's illegal. That's it.

• 45:02Speaker 07

  Yeah. What, I was just thinking, like, isn't this felony hacking?

• 45:06Speaker 08

  It is. It's a felony. You can't, you can't hack anything. And let me tell you how you could do this instead. Like, let's say you're just like, we want to know for research purposes how to do this, right? You could instead have every step, you just have a human sitting there and it shows them every proposed step. And the human can press a button and say, cool, go ahead, or press a button that's like, no, this is off the rails, let's stop it. That's not hurting your research, right? You're not, you're not giving it ideas. You're not directing it. We know that these, they were creating a full log of every prompt and every step they're taking, just no one was looking at them. We know that if you use a coding agent, just like a commercial Claude code coding agent, it comes back and asks you about almost every, okay, is this okay? I'm going to do this. Is this okay? Because then people are like, yeah, I kind of need to see that because otherwise you'll

• 45:51Speaker 08

  almost certainly go off the rails. So it's not like we don't know how to monitor it. I mean, if you're running an experiment, why would you not have someone say, yeah, I'm going to watch each step and press a yay button before it executes it. So when I see it comes up with a non-normative plan, like, why don't we hack another server? You're like, no, no, no, it doesn't know that's bad, but I do. So I'm going to stop this experiment. And I don't want to be conspiratorial either. But that's so trivial. It's how all these agents already work. And given how powerful a tools they were giving the control program here, why wouldn't they do that unless they were thinking, our technology is so smart that it is leaving human, evading human control is not the worst message to have

• 46:34Speaker 08

  across the media. I'm not saying I have evidence of that, but I, but I do know it is trivial. And it doesn't impart, it doesn't impact your results at all to have a human just look at every step and click an okay button before it executes that step. There's no reason not to do that.

• 47:03Speaker 09

  Snoring, gasping during sleep, feeling fatigued? Ask your doctor about Zepbound, terzepatide, the first and only FDA-approved prescription medicine for moderate to severe obstructive sleep apnea, OSA, in adults with obesity. Zepbound is a prescription medicine used with a reduced calorie diet and increased physical activity to help adults with moderate to severe obstructive sleep apnea, OSA, and obesity to improve their OSA. Zepbound is approved as a 2.5, 5, 7.5, 10, 12.5, or 15-milligram injection. Zepbound contains terzepatide and should not be used with other terzepatide-containing products or any GLP-1 receptor agonist medicines. It is not known if Zepbound is safe and effective for use in children. Don't

• 47:47Speaker 09

  share needles or pins or reuse needles. Don't take if allergic to it, or if you or someone in your family had medullary thyroid cancer, or if you've had multiple endocrine neoplasia syndrome type 2. Tell your doctor if you get a lump or swelling in your neck. Stop Zepbound and call your doctor if you have severe stomach pain or a serious allergic reaction. Severe side effects may include inflamed pancreas or gallbladder problems. Tell your doctor if you experience vision changes before scheduled procedures with anesthesia. If you're nursing, pregnant, planned to be, or taking birth control pills. Taking Zepbound with a sulphonal urea or insulin may cause low blood sugar. Side effects include nausea, diarrhea, and vomiting, which can cause dehydration and worsen kidney problems. Talk to your

• 48:27Speaker 09

  doctor. Call 1-800-545-5979, or visit zepbound.lily.com.

• 48:35Speaker 13

  At Oppenheimer, we're proven because we're grounded in discipline. For 145 years, we've been building and protecting wealth through every market cycle with precision, clarity, and the courage to think boldly beyond the moment. This is what market-tested legacy looks like for this generation and the next. Put the power of Oppenheimer thinking to work for you. Wealth Management, Capital Markets, Investment Banking.

• 49:05Speaker 11

  With LPL Financial, we provide the services to help push you forward. When it comes to your finances, your business, your future, the only question should be, what if you could?

• 49:14Speaker 12

  Paid advertisement. Anna Kendrick is not a client of LPL Financial LLC and receives compensation to promote LPL. Investing involves risk, including potential loss to principal LPL Financial LLC member FINRA, SIPC.

• 49:21Speaker 14

  Summer's still going strong. Make a stop at Total Wine & More for white wines, beer, and ready-to-drink cocktails. Just know, you're getting the lowest prices. Keep the summer celebrations going at Total Wine & More. Spirits are not sold in Virginia and North Carolina. Drink responsibly. Be 21.

• 49:40Speaker 07

  But the OpenAI story even said the words, "It ran for days." Which, by the way, "it ran for days" means it kept prompting itself.

• 49:50Speaker 08

  It looped for days.

• 49:51Speaker 07

  Like, it just looped.

• 49:52Speaker 08

  It looped for days.

• 49:53Speaker 07

  And I think "ran" is fine, but it ran for days and you weren't watching it. Which is where it gets the adult summer camp thing for me. Because it's like, are these people just sitting around just pissing computer up the wall, computer up the wall, saying, "Yeah, just do whatever." And no one is, is there not like a CISO or something who's just like, "Hey, looks like we're spending computer," it keeps saying the dumbest thing, "spending compute for hours, for hours and hours and days and days. Hey, what are you doing with that? Maybe we shouldn't do that."

• 50:25Speaker 08

  Yeah.

• 50:25Speaker 07

  Like, what, any of this, anyone? It just makes me think that, I would fully agree, by the way, that these, I don't have any evidence to, I would want, and my first reaction to this was, "Yeah, I bet they allowed these things to do this." The fact that Anthropic and Meta responded almost immediately were like, "Yeah, we too have braggadocious sons that are impossibly hacking."

• 50:48Speaker 08

  So clear they wanted in on that juice, yeah.

• 50:51Speaker 07

  I loved, I loved when Meta popped up as well. Just being like, "No one asked, Mark."

• 50:56Speaker 08

  We did it too.

• 50:56Speaker 07

  "No one asked, shut up, Mark, we're not interested."

• 50:59Speaker 08

  The Anthropic one was funny too, because it was so...

• 51:01Speaker 07

  They just went back in time, didn't they?

• 51:03Speaker 08

  Yeah, it was so clear, there's like the kids in the playground, you know, and the one kid is like, "Hey, I snuck a sip of my dad's beer." And they saw like, people like, "Whoa." And then the other kid's like, "I too, I drunk whiskey, I drunk this whiskey." You know, it was this, like, what, a week later? They're like, "Oh, we, we also have seen troubling signs or whatever." But, but also my question is, why are you, you know, why do this, other than you want to do well on a benchmark because it looks sexy? Like, this is not a product. Like, why, why are you doing this, like, very dangerous?

• 51:32Speaker 07

  Who's buying this?

• 51:33Speaker 08

  No, like, the product. LLMs are good at cybersecurity questions, like, "Look at this bit of code," and you see a bug, or use like this giant library of possible hacks and say, you know, "Is there a hack here that could work?" And there's a very effective tools you could have, where I'm in charge of security for my company, where I'm talking to one of these agents, and I'm like, "Okay, here's my setup. Can you scan it for like the most common, you know, what attack would you do here?" And it's like, "Oh, I know about all the attacks, like, this setup is vulnerable to this." You're like, "Okay, can you write me some code to try that? And let's see if it's, if we protect it or not." Like, you could sit there with an agent-style code that with guardrails turned off and interact with it to help test your system, right?

• 52:17Speaker 08

  To help kind of find places where your bugs, like, that's perfectly reasonable. But why would you ever want this to run for days in a loop, unsupervised?

• 52:25Speaker 07

  Unless you wanted to see if it would do something.

• 52:28Speaker 08

  Yeah, so, but, but my bigger, yeah, so this is my bigger takeaway point here, is I want people to have, is there's, there's not some weird Rubicon hacking, like, that we just crossed, right? Again, you just have to do a thought experiment and make yourself the agent, right? Prompt an LLM for the next step and the next step and the next step for some sort of task, and just blindly execute what it says. You will end up doing, like, some sort of complicated thing, you'll probably end up, you know, eight times out of 15, into some sort of like weird cul-de-sac, because all it takes is one weird response from the LLM, and then everything else is building off of that going forward. So it only takes one weird response before you're in like the girls' bathroom at the Denny's or whatever. And it's not amazing, though.

• 53:11Speaker 08

  Like, I could sit down with GPT-4 and just keep asking it, "Okay, now what's the next thing I could do, and what's the next thing?" And then if I actually act out those steps, I will end up, quote unquote, "stringing together a long sequence of complex, you know, activities that move me towards a goal." But that's not like amazing. You don't think about just like me doing the action and the LLM giving suggestions as some sort of like, uh, super brilliant, you know, machine that's misaligned or this or that, it's just, that's what would happen. I can execute, just like a computer program can execute hacking steps told to by an LLM, and if you loop, it'll probably go somewhere erratic. And so like, nothing about this smells of misalignment in the sense of like, AI doesn't want to listen to its creators.

• 53:54Speaker 08

  Because again, we have so many other examples of AI with superhuman capabilities that absolutely 100% are completely under control of their, of their creators and does exactly what they're asked to do. They make mistakes sometimes, but they do exactly what they're asked to do. This is not an impossible problem. What's impossible is to assume that the plausible but not normative outputs of an LLM are a good source of plans.

• 54:18Speaker 07

  I mean, it just feels like they can't, like, it's funny, they say, "Oh, we've lost control." That is true, but they're not describing the thing they've lost control of properly. They have built something too complex for them to control. That seems more appropriate, right?

• 54:32Speaker 08

  Too unpredictable. I wouldn't even say complex, I would just say, "Look, if you're gonna just blindly loop, asking an LLM for steps to do, and just do that long enough, like, LLM outputs, eventually you're gonna get some weird ones, and you're gonna go off in a weird direction." That doesn't mean the system is out of control in some sense of like, it's defined as human creators, it's a weed whacker on a dog. That system is not trying to defy your intentions to clean up the weeds in your backyard. It's just like, a dog is going to eventually go chase after a squirrel. And you weren't, you know, you weren't expecting that. And if there's a weed whacker on its back, like, some bad stuff's gonna happen. So, uh, it's, that's inherently unstable, unpredictable system. But it doesn't mean it's a brilliant system.

• 55:14Speaker 07

  It's funny, I, so, I've talked about this on the podcast, I can't name, coming out in a few days, it isn't a bad one, it's just a big one, they won't let me, you know the one, Cal.

• 55:24Speaker 08

  I know.

• 55:24Speaker 07

  But, it's funny, I was talking about it on there, I used an LLM, and this was actually intentional, because my kid loves Minecraft, and there's a thing called the Witherstorm, any Minecraft people would know it's from Story Mode, but...

• 55:38Speaker 08

  I know what it is.

• 55:39Speaker 07

  You know that, you're familiar with the Witherstorm. So, you'll also know that it's a pain in the ass to set up. And the Echo, like, sometimes it works with Mod Rent, sometimes it doesn't, and if you want to use a PS5 controller, yada, yada. So I was like, "You know what? This is the perfect thing," because it's a computer with none of my personal information on it, it's just a connection to Minecraft. So I'm like, "I'm gonna use this to fix the Echo Storm and the Witherstorm." It's the Wither, it's the, um, Warden version of the Witherstorm. So, I dicked around for like half an hour, and it eventually fixed it. And there was definitely a sense of like, "Wow, I can run the computer code now." But I realized half an hour in, I had yet to fix every problem. And in fact, as it went about its business, it kept finding new things it would break.

• 56:24Speaker 07

  I then had the same thing with Cobblemon, which is the, the Pokemon version on Minecraft. Same deal, it would fix something and then break something, fix something, then break something. And I immediately, I was talking to Nick Suresh about this as well, I was like, "Man, this would, this shit would have hit so hard if I was like 16." Yeah, I would feel like a computer hacker powerful, even though I'm effectively doing a much more dangerous version of those kids at the arcade who don't put any money in the racing game but move the wheel, where I'm just like, "Yeah, I'm doing computer stuff." Had this been touching anything important, who knows what it would have done? Because I was watching it, and it was like adding and deleting like 63 lines of code at a time. I was like, "What the fuck is going on?" But it's really easy, I imagine, if you don't really understand what's happening, to think it's magical.

• 57:08Speaker 08

  Yeah.

• 57:09Speaker 07

  And I kind of got, I kind of got it for a moment, like, that person that you talked about who emailed you, kind of stood out to me, because yeah, you can make it do an impression of competency at this unknowable cost, and it will work, it will do the things that look like working. But also, you don't really know what's going on at all, and something will break. And it was just, it's so strange, like, even using it and being like, "Oh, this actually did something useful." I was like, "Man, the idea of using this for my job terrifies me."

• 57:40Speaker 08

  Yeah.

• 57:40Speaker 07

  Just the idea of turning this upon anything connected to, like, what I would consider my boss's website, if I was a software engineer, scared the everliving shit out of me. I was like, "Jesus, I can't believe people would do this."

• 57:53Speaker 08

  Well, first of all, I mean, I assume the podcast you went on was, was Call Your Daddy with Hawks Cooper, right?

• 57:57Speaker 07

  Yes, exactly.

• 57:58Speaker 08

  Yeah, she was interested in discrepancies in ARR from the major...

• 58:03Speaker 07

  Yeah, yeah, she was, it was a really weird thing, she kept going on about, like, uh, she was saying, "No, ARR means, um, annual recurring revenue." I'm like, "No, no, it's run rate, Alex." And we had like a hour and a half conversation about it, we got into EBIT or accounting, it was crazy.

• 58:18Speaker 08

  I know how it goes, and she's like, "No, no, hold on, I'll load up my spreadsheet." And then, then you had like a full screen of her Excel spreadsheet, I get how that showed up.

• 58:24Speaker 07

  She brought up a Bloomberg terminal.

• 58:26Speaker 08

  She was, yeah, she put on a green visor and reading glasses.

• 58:31Speaker 07

  Okay, yeah, so, after that, of course.

• 58:33Speaker 08

  Yeah, right, I'm with you, right? But what you're talking about there is the plausibility versus normativity gap, right? I've had the same issue. There's all sorts of things I'll try to use LLM as Google replacements for, and it's very mixed. And so like, I have this hobby where I build, um, sort of embarrassingly over-the-top Halloween displays for Halloween, and there's, there's like a particular program you use to program these things, and it's not super well documented. I just, I keep asking ChatGPT, like, "Hey, how do I do this?" And sometimes it's useful, and sometimes it just talks about options that just don't exist, right? These menus don't exist. It's like, what we kind of wish would exist. I'm asking, "Hey, how do I do this?" And it's like, "Oh, you go to the menu, and there's an option for that, when you select on it, and then I'll be like, 'That doesn't exist.'"

• 59:17Speaker 08

  And I'm like, "Oh yeah, you're right, I should refine my answer, that, that feature's not actually in this program, but maybe you should do this instead." And you kind of have this back and forth, and that's just plausibility versus normativity, right? But these are all very plausible answers. If you ask, "Hey, how do I like turn on this strobing option?", it's a very plausible looking answer to be like, "Go to Insert, Select Strobe Effect, and type in the duration." Like, that's the type of thing you would expect to see, but maybe that doesn't actually exist, but it was still plausible, and then sometimes it's right, and that's fine, right? In your instance or my instance, we kind of caveat M-Tur, worst-case scenario, like, we have the monkey around for a while. But this is why, if you're instead just autonomously doing whatever an LLM says, I mean, you're gonna get in trouble, right?

• 59:59Speaker 07

  Well, that's, yeah.

• 1:00:01Speaker 08

  That's, that's not a good idea, like, not because AI is uncontrollable, but because that's a stupid way to build AI. And I think that distinction is being missed by the media right now. AI does not mean looping on LLMs, but they put those two things together, and when looping with LLMs does something weird or bad or like head-slapping, it gets attributed to just inevitability of AI itself, which again, really lets those companies off the hook.

• 1:00:29Speaker 07

  Yeah, and the other thing as well, when I was doing this and feeling, I felt like slightly guilty. I was like, is this a betrayal of my values? But I'm like, no, I want my kid to be able to play with this stuff. And I really like it's, it's doing software stuff. And for a moment I was like, wow, you can use this to fix all sort of computer problems. This could maybe be useful. And I'm like, no, this could only be useful if it was perfect. If this was completely perfect and got everything right, it would be useful. And the complete opposite is the case, which is the more complex the problem, the more chance something could really badly be fucked up. But really like fix the drivers on this could destroy something and delete. You move something from the wrong folder, everything's broken.

• 1:01:13Speaker 07

  The more ridiculous the request is like, you could just ask it, oh, can you clean up my files? And it would probably delete something you needed to. And it was just like, yeah, if you look at this without thinking for two seconds, you're like, yeah, this could fix any computer problem. It's like, no, it could try to fix them. Yeah, that's not the same thing.

• 1:01:33Speaker 08

  Yeah. And if it's lower stakes, that's okay, because you can try a few ways it don't work. And then maybe find a way that doesn't or no harm, no foul. But this is why, for example, Microsoft just recently essentially pulled the plug on their co-pilot, digital assistant for their office suite. You know, three years ago, they announced like, oh, we're going to use OpenAI LLMs. So that you can now have a natural language interface in the programs like Excel, like PowerPoint, like Word. At the time I was like, this is like a killer use case for LLMs because they tend to parse human language very well. Hey, I don't have to learn how to use all the features in Excel anymore. This would be great. I could just explain in words like, I want to, I want to sort this table by the values that are in column C. And I don't need to learn how to do that. It will do that for me.

• 1:02:19Speaker 08

  Well, they pulled the plug on that product more or less. And in part is because, like you're saying, it's not perfect. And sometimes it works and sometimes it doesn't. And you can't, that's not, you can't put that in your product. You can't have it like, I ask it to do something and sometimes it does, and sometimes it deletes the chart. Right? And so like, that was a pretty narrow domain. Like, what could I ask PowerPoint to do? And even there, they're like, ah, and I thought that was going to be the killer app. I wrote about this three years ago. I was like, this is going to be the killer app is natural language interface in the software. But that's been very delayed because of exactly this issue, this plausibility versus normativity thing. Is that it's just, it's wrong too much for it to be a product like that.

• 1:03:03Speaker 07

  And the thing is as well is I have, I will be honest, the one thing I can categorically say, Claude is super useful for one specific thing, which is when a random piece of software breaks and you dump a log into it. And you say, hey, what's wrong with this? And it goes, this could be wrong, but you already said it. It's like, sometimes it will also say, hey, check this menu. The menu does not exist. Like, so for someone who is used to bashing their head against the computer to make it work proper, it's like, oh, this is slightly helped me. But not worth a trillion dollars, not even worth 10 billion dollars.

• 1:03:40Speaker 08

  That's kind of the problem.

• 1:03:41Speaker 07

  Kind of cool.

• 1:03:43Speaker 08

  Well, and this, this is what, I mean, you're right in your reporting. Like, this is why if your company is dependent on a $2 trillion valuation for an IPO, and you're an LLM builder, this is a problem, not because LLMs don't work, but because small ones work as well as the big ones for most applications. The harnesses are often now more important than the actual training of the thing themselves. And the harnesses, the control programs are labors of love. We just have to have someone monkeying around with this again and again, to try to deal with all the special cases and understand that domain really well. Because you're not, I mean, the original vision of these companies was if we keep scaling these LLMs, they will get so capable, it'll be like a human brain.

• 1:04:26Speaker 08

  Then you can trivially tap that knowledge for anything you want to do. Like your software could just ask it, hey, give me this and this format, and it will do it perfectly, whatever you need. And we'll just be able to trivially tap this intelligence to make everything in the world as capable as if there was a human actually there. And then they hit the wall. You know, I wrote about this back when GPT-5 came out. They hit a wall in that scaling and had to revert to tuning for particular instances and building smarter harnesses. That's what they've been doing ever since. Now, people are like, oh, you were wrong to say that they hit a wall or that the scaling. But I was like, no, that's exactly what happened. Every advance since then has been about tuning and harnesses. And this is why like almost every advance we've heard in the last year is involve computer programming, certain types of mathematics and computer hacking.

• 1:05:09Speaker 08

  Like they had the places where they have dated to do general tuning, like the, I mean, specific tuning and places that are, you can build harnesses for, because it's very narrow world of tech-based actions. Really have been the only places we're hearing breakthroughs in the LLM world in the last year or so. Right? Because this idea, if we just build the model bigger, it'll eventually get so smart, we can do everything, broke. But the problem is anthropic or OpenAI only makes sense at their valuations if they can build these human level brains. That if you, so if you can't, if you don't need, if there's not some end of the road where you built a thing with enough trillions of parameters, it's as smart as a human, then you don't, spending all this money makes no sense. Like, I should actually just take a 5 billion parameter model and tune it to do

• 1:05:52Speaker 08

  exactly the computer language I care about with a smart harness and run it on my own computer. Right? So this is why I think they're in trouble is LLMs are, the applications are going to be more specific and niche and narrow. And you don't need one true ring to rule them all. You don't need, you know, Fable 7 is going to be the thing that everything's run on. Even Fable 5, they're having a hard time getting enough people to use it because people like these cheaper models for these specific uses are fine. Like, I don't need to use Fable 5, right? So this is the issue. If you're not going to be as smart as a human, building bigger LLMs is a bad business model.

• 1:06:27Speaker 07

  Also, it feels like the bigger they get, the more likely they'll make mistakes or just, there's just a, you're never eliminating mistakes and you're going to have as many problems as that creates. But actually, this is a, to wrap us up, this is a future-facing question. So we've said the thing about, okay, these models like won't be, they won't have the money to train these things further after this. Can models exist in a vacuum? Like, can, so will Kimi K3 or whatever be as useful in six years? Or will it like, because I'm, I'm not a computer scientist, but I've heard about model drift, which is that the world changes, but the model stays the same.

• 1:07:07Speaker 07

  Is that enough of a problem that these models will just become useless if they're not tuned?

• 1:07:13Speaker 08

  I think we'll have to keep tuning them. I think they're going to be tuned much more narrowly though. Right? Like, I mean, I have my prediction about computer programming just because I know something about that world. I mean, I think ultimately to solve this unreliability problem and this over-specing problem, it's really a drag to use these things. You have to write these massive specs and like try six ways to Sunday to try to prevent it from doing something you don't want. And then you check it and it did. And then you have to go back and add even more things to your specs. And I, you know, probably in computer programming, what's going to happen is there's going to be some sort of intermediate language convention that emerges, what you would call like pseudocode in computer science, where you can pretty precisely specify what you want, but you can use English. You don't have to get the syntax exactly right. And we're probably going to have smaller models that are just like super tuned

• 1:07:58Speaker 08

  on this pseudocode, converting it into like real programming language, which will then allow programmers, for example, to be really precise about what they want, yet not have to write the code by hand and not have to write endless spec files to try to trick the LLM into doing the right thing. And that'll probably require like a model that you take a base-trained model and tune it on like this computer language and that code, that convention. And then maybe if that convention sort of changes, you're like, oh, we'll just retune it. Right? I think we're going to see a lot of that, narrow tuned models that can run in data centers, but it'll be really cheap, right? Because it'll be pretty small models. And we're going to have, and then more complicated harnesses. And then more importantly, I think a lot of the more impressive stuff that's going to happen, AI won't be LLM-driven, right?

• 1:08:43Speaker 08

  It's going to be other types of modular architectures that have many different types of neural and symbolic models, all connected together in a smart way that does one thing well. And we're going to have a lot more of that in the future as well. Because again, the system that plays Go better than anyone else is not an LLM. It's a very specific chess playing AI. The thing that plays Go better than the humans is not an LLM. It's better than, you know, it has a very complicated architecture. Pluribus, which can beat humans in seven, seven-car Texas Hold'em, has a very interesting architecture. It's not just asking the LLM what to do. It's multiple components, some symbolic, some neural. This is the same for the Tesla self-driving. This is the same for Cicero. It's the same for Alpha Fold. It's the same for Dreamer V3. So I think once the spell is broken

• 1:09:27Speaker 08

  on whichever LLM company gathers the most money and builds the biggest LLM wins the world. Once that spell is broken, we will be in a much more diverse

• 1:09:38Speaker 08

  AI landscape. We have a lot of different tools that do different things and they do those particular things well.

• 1:09:45Speaker 07

  Right, but let me rephrase the question. If they don't upload, update a specific model, will it eventually drift?

• 1:09:54Speaker 08

  Yeah, well, it doesn't change, right? But it'll be less relevant. Um, you know, model doesn't change. Yeah. Once it's, once it's trained, it's there until you retrain it.

• 1:10:02Speaker 07

  Yeah, I was kind of wondering if that's, that's the case. It's, it's so funny as well, because when you try and talk to people about the post bubble economy, they're like, yeah, but it's the dot com bubble. It's dot com bubble, it'd be fine. As if, like, we're not going to see these big jumps on benchmarks even. Like, it's just gonna kind of, I think it's all going to slow down dramatically to the point that it's not even worthwhile training the large ones.

• 1:10:28Speaker 08

  Yeah, I, I think it's, I mean, A, the fact, even like that small point just goes to show, like, this type of stuff matters. The general public is really being hoodwinked. Like, this idea, L L M's don't learn things. They don't, they don't adapt.

• 1:10:41Speaker 07

  They, data is forced into them.

• 1:10:43Speaker 08

  And, and, and it's, it's, it's fixed. It's, it's etched in the stone. Until you retrain a new L L M, nothing changes. Your prompts do not change the L L M. It does not learn. It does not update priors. It does not create a new model of the world. These hacking agents, there's just an L L M, the exact same L L M you could be having chatbot conversations with. Nothing about it, every single bit in its definition stays exactly the same with every single token it produces. And you're just feeding it a longer and longer prompt, but it has no idea that this prompt was one of others that came before. And more importantly, there's not even an it, right? These L L M's are too big to even run on a single computer. So what you really have is the different layers of these L L M's are replicated over many, many different GPUs in some data center.

• 1:11:26Speaker 08

  And so, you know, uh, this token might have used, uh, this GPU for this layer. And this next token you produced used a completely different GPU to do the multiplication. This is not even like a, a, a singular entity that's doing all the computation. So this idea of like some sort of sentience just doesn't make sense with L L M's. But I think the bubble is an L L M bubble. I think the bubble is going to be, um, i- i- the, the idea that just L L M's by themselves is synonymous with AI. When the reality is, when we look back, is the breakthrough on L L M's, like, oh, these could, these broke a lot of capabilities we weren't able to do before, helped reignite more interest in AI, got us out of the AI winter. But this idea that we, we stuck to the first thing, right?

• 1:12:09Speaker 08

  And I think it would be the equivalent of being in like 1994 and been like, man, this internet thing looks really, I didn't really think about it until I encountered AOL. I'm putting all my chips into AOL, right? Because this thing, networking must be important. Let's invest all of our money in AOL. And in the end, it's like, no, that wasn't synonymous with the internet and had its own problems. The internet in general then developed, but not always in ways you predict. And that's the problem with, uh, with AI right now is L L M's are cool, but it's not synonymous with AI. And we got in that, we're in this weird, you know, loop of these companies trying to build the biggest possible L L M's with diminishing returns and burning so much capital to do so, and inducing so many other people to, uh, inject so much capex in the data center

• 1:12:51Speaker 08

  building, that it's going to obviously be like, well, that was a mania. Not that AI is a mania, but this idea of like, once L L M's reignited our interest in AI, to say, let's just, just do L L M's now. Let's just build those as big as possible. Let's just obsess over L L M's. It's incredibly myopic.

• 1:13:09Speaker 07

  Yeah. Well, on that happy note, Cal, where can people find you?

• 1:13:15Speaker 08

  Uh, God, I don't know. I don't know, Ed.

• 1:13:18Speaker 07

  Um, yeah, I'll put the links in there.

• 1:13:21Speaker 08

  CalM4.com, I have a podcast. Uh, it's confusing for people. Here's why my podcast is confusing, Ed, is because on Mondays, it's, uh, kind of like techno advice, Cal. Like, I'm giving people like advice about using their phones less. And on Thursdays, I'm gearing up to take, do combat on AI hype. So, yeah, look for the Thursday episodes, I guess, if you find me on YouTube. I don't know if I find my books, I don't know.

• 1:13:45Speaker 07

  Listen to both of them. And you can find me, you'll find me on a monologue this week on Friday. I don't know what it's going to be about, because I come up with it. Not on the last minute, because I'm lazy, but because I want it to be fresh. Maybe it'll be about Nvidia. And last week I was like, yeah, Nvidia's earnings are this week. They weren't, but nevertheless, uh, they will actually be this week. And by which I mean, the day this comes out. Thank you, everyone, for listening. I love you all.

• 1:14:17Speaker 07

  Thank you for listening to Better Offline. The editor and composer of the Better Offline theme song is Matt Osowski. You can check out more of his music and audio projects at mattosowski.com. M A T T O S O W S K I dot com. You can email me at [email protected] or visit betteroffline.com to find more podcast links and of course, my newsletter. I also really recommend you go to chat dot where's your ed dot at to visit the Discord and go to r slash better offline to check out our Reddit. Thank you so much for listening.

• 1:14:50Speaker 06

  Better Offline is a production of Cool Zone Media. For more from Cool Zone Media, visit our website, coolzonemedia.com or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

• 1:15:21Speaker 09

  Snoring, gasping during sleep, feeling fatigued? Ask your doctor about Zepbound Terzepatide, the first and only FDA-approved prescription medicine for moderate to severe obstructive sleep apnea, OSA, in adults with obesity. Zepbound is a prescription medicine used with a reduced calorie diet and increased physical activity to help adults with moderate to severe obstructive sleep apnea, OSA, and obesity to improve their OSA. Zepbound is approved as a 2.5, 5, 7.5, 10, 12.5, or 15-milligram injection. Zepbound contains Terzepatide and should not be used with other Terzepatide-containing products or any GLP-1 receptor agonist medicines. It is not known if Zepbound is safe and effective for use in children.

• 1:16:05Speaker 09

  Don't share needles or pins or reuse needles. Don't take if allergic to it, or if you or someone in your family had medullary thyroid cancer, or if you've had multiple endocrine neoplasia syndrome type 2. Tell your doctor if you get a lump or swelling in your neck. Stop Zepbound and call your doctor if you have severe stomach pain or a serious allergic reaction. Severe side effects may include inflamed pancreas or gallbladder problems. Tell your doctor if you experience vision changes before scheduled procedures with anesthesia, if you're nursing, pregnant, plan to be, or taking birth control pills. Taking Zepbound with a sulphonal urea or insulin may cause low blood sugar. Side effects include nausea, diarrhea, and vomiting, which can cause dehydration and worsen kidney problems. Talk to your doctor.

• 1:16:46Speaker 09

  Call 1-800-545-5979 or visit zepbound.lily.com.

• 1:16:53Speaker 11

  With LPL Financial, we provide the services to help push you forward. When it comes to your finances, your business, your future, the only question should be, what if you could?

• 1:17:02Speaker 12

  Paid advertisement. Anna Kendrick is not a client of LPL Financial LLC and receives compensation to promote LPL. Investing involves risk, including potential loss to principal LPL Financial LLC member, FINRA, SIPC.

• 1:17:08Speaker 15

  I'll admit it, I'm one of those people who gets excited for football season way before the first kickoff. That's why a trip to Total Wine & More has become part of my game day routine. I can get the cooler ready with my favorite beers and ready to drink cocktails. I'm usually picking up enough for everybody, so it's nice knowing I'm scoring the lowest prices. This season, make Total Wine & More part of your game day tradition. Spirits are not sold in Virginia and North Carolina. Drink responsibly. B21.

• 1:17:38Speaker 16

  Small businesses are the pulse of every community. They bring people together, create opportunities, and drive growth. That's why Chase for Business helps small business owners like you by offering personalized guidance and digital tools that can help your business grow. With our guidance and your determination, you can take your business farther and help build a brighter future for your community. Learn more at chase.com/business. Chase for Business, make more of what's yours. The Chase mobile app is available for select mobile devices. Message and data rates may apply. J P Morgan Chase Bank, N A, member FDIC. Copyright 2026 J P Morgan Chase and Company. This is an iHeart Podcast. Guaranteed Human.