Pod Insights Big Technology Podcast Does GPT-5 Live Up To the Hype?, AGI Wait Continues, Self-Loathing Gemini About this episode Ranjan Roy from Margins is back for our weekly discussion of the latest tech news. We cover: 1) OpenAI's launch of GPT-5 2) Whether GPT-5's tool calling ability is its hidden strength 3) GPT-5 is good at 'doing stuff' 4) But GPT-5 is not AGI 5) Do AI models need more than book smarts to thrive? 6) OpenAI's medicine play 7) GPT-5's coding use case 8) We need AI tables for travel 9) Do the big model players now subsume AI startups? 10) Gemini has a breakdown
---
Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice.
Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b
Questions? Feedback? Write to: [email protected]
Learn more about your ad choices. Visit megaphone.fm/adchoices
Transcript 0:00 Speaker 00 This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?
0:43 Speaker 00 Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.
1:03 Speaker 00 GPT-5 is here, finally. Does it live up to the hype? That's coming up on a special Big Technology Podcast Friday edition, right after this. Welcome to Big Technology Podcast Friday Edition, where we break down the news in our traditional cool-headed and nuanced format. You know what we're going to be talking about today, because GPT-5 has finally been released by OpenAI. Of course, we had OpenAI COO Brad Lightcap on the show just a few hours ago. So that episode will be the most recent one in your podcast feed, where you're going to get the official line from OpenAI and a bunch of really interesting insights about what it took to train this model and where the AI field is going.
1:43 Speaker 00 But today, as we always do on Friday, Ranjan Roy and I will break down exactly what the situation is with this new model and whether this is actually something that lives up to the hype that people have been talking about. No overreactions, we're going to do it with the proper context. And with that, I want to welcome Ranjan back to the show. Ranjan, welcome.
2:02 Speaker 01 Happy AGI Day, Alex. Is it here?
2:06 Speaker 00 No, it's not here.
2:07 Speaker 01 Is it here?
2:08 Speaker 00 I lost the bet. I lost the bet. Look, Sam Altman said that GPT-5 is smarter than almost everything, every single thing a human does. So I thought, okay, fine, you know, we're finally gonna see AGI, but turns out, no, no AGI. And we will talk about that in the middle. But first, we have a little interesting announcement. Let's hear it.
2:25 Speaker 01 Yeah, in additioning to writing the margins newsletter, I've actually been working at a company called Writer, writer.com. It's an enterprise generative AI startup, and I'm leading the vertical for the retail industry. I wanted to bring that up today because GPT-5 and what it means for me, I think, is heavily informed by a lot of the work I've been doing. And I think it might be a little AGI-ish.
2:51 Speaker 00 I mean, it's amazing that you go to an AI company and the first thing, your first sentence out of your mouth is, yep, we have AGI here. But no, no, it's okay. I think I anticipate you'll come in with a level head. So let's talk about GPT-5. Okay, so this is from TechCrunch. No, sorry, this is from The Verge. GPT-5 is being released to all ChatGPT users. It says, OpenAI is releasing GPT-5, its new flagship model, to all ChatGPT users and developers. OpenAI says that GPT-5 is smarter, faster, and less likely to give inaccurate response. Sam Altman, on this media call that I was on, had a very interesting description of what it took. He says of what it is, he says, GPT-3 sort of felt like talking
3:34 Speaker 00 to high school students. You could ask a question, maybe, maybe you'd get a right answer, or maybe you'd get something crazy. GPT-4 felt like talking to a college student. And GPT-5 is the first time that it really feels like talking to a PhD-level expert. What do you think about the significance of this? And what do you think about this framework that Altman is setting up for the intelligence that we're seeing within the model?
3:58 Speaker 01 I don't like the, I don't like the framework. I think, and again, I'll get into why I think this is exciting, but it's still weird to me always when like, People and the kind of industry that advocates for dropping out of college to start a startup always leans back to high school student, college student, PhD student as the the framework for intelligence like. And the other part of it is, I don't want PhD level work for most of the things I'm asking. I just I actually want grounded in sometimes you want it to be cool, which maybe PhD students are and are not, no offense. But, you know.
4:34 Speaker 00 You're alienating segment by segment.
4:36 Speaker 01 I know, sorry. I know we do have a lot of very smart, educated listeners. But you know, sometimes you want it to be cool. Sometimes you want it to be funny. Sometimes you want it like, to me, that's more that that's not the intelligence, like the framework, I think, that's good for intelligence. I always like, we've talked talked about this a lot around, like, the ARC AGI test has a has a segment around everyday queries. I still have dug, I've asked as many people as I can. No one has been able to explain to me what are those everyday queries. Like, to me, those are answering those correctly well across multiple data sets, multiple tools. That's actually intelligence to me, doing that kind of work.
5:16 Speaker 00 Right. And, you know, I kind of cherry picked that out of the remarks. But it is interesting to me, and this is something that came up with Lightcap as well, that it's not just making this model smarter. That has been the sort of star in this story. It's all these different other elements of it. And it seems to me that it's but it's possible that the models have reached this level of intelligence where you start to spread out into different capabilities of them like tool calling, like the way that you structure the experience. And that is where you start to see the gains and the lift in terms of the way that people can use this. So maybe today, on the release
5:58 Speaker 00 of GPT five, or this week, while GPT five is released, I go from being a model person to being a product person.
6:06 Speaker 01 Well, no, no, no.
6:08 Speaker 00 I'm kidding, of course. But go ahead, Ron. Yeah.
6:10 Speaker 01 The intelligence is in the model for which product to choose. So that's not a product decision. That is, that's a model strength. And so, so, oh my God, am I becoming a model guy now? I think.
6:23 Speaker 00 I think we are zeroing in on the better the model, the better the product here.
6:27 Speaker 01 In the end, it all comes together.
6:30 Speaker 00 It all comes together.
6:31 Speaker 01 Nuance in the middle. Nuance in the middle. No, but okay, so for.
6:35 Speaker 00 But it is, but I mean, I just want to say one more thing. We have been going this direction for a while, right? Like, that does seem like, you know, I've, so for new listeners, I've strongly said the most important thing in AI is the model. Rajan has strongly said the most important thing in AI is how you productize this model. And it just turns out that better models do make better products. And we're starting to get to that point where we're starting to see the results.
7:00 Speaker 01 Yeah, I think, okay, so I will actually admit error in two big areas. Get ready, get, Alex isn't, listeners can't see Alex is smiling here. So the first is, again, the intelligence of the model to choose the right tool or product. And we're going to get into what that means and why I think that's incredibly important and why GPT-5, by bringing all these different models that they have into one switcher, just one model that understands is actually what I believe the most significant breakthrough. So already I think that's incredibly important. And the second area is like directly on that. I think it was like five or six months ago, we debated heavily.
7:46 Speaker 01 And I said that users should choose the right model for the right job. And to take that away would make models and the experience worse. And you kept coming back, I think it was when maybe Claude had all condensed everything into one picker or GPT where everyone's kind of like making fun, we're debating. But the idea that should the user, which it's been for a while, choose what is the best model for this task at hand? And I have thought that's the best way that products should be rolled out. And I've completely reversed on that. And this is the right example of why that's important.
8:20 Speaker 00 So let me set this up and then we can talk it because I might have flip flopped to the other side of this. So this is definitely, it's great to have these releases because we can sort of test our longly held beliefs, long-held beliefs and see if they make sense anymore. And it seems like both of us are saying, well, maybe, maybe, maybe not. So this is from the Verge article, GPT-5 is presented inside ChatGPT as just one model, not a regular model and a separate reasoning model. Behind the scenes, GPT-5 uses a router that OpenAI developed, which automatically switches to a reasoning version for more complex queries, or if you tell it to think hard. I'm going to take the other side of this. I used to think that, yeah, the should, it should be seamless and the model should just choose for you when it makes sense to think and when it doesn't.
9:05 Speaker 00 But I've been using OpenAI's O3 model and that is like a very heavy reasoner. It thinks a lot. And personally, I've just felt that that model has been better, not only than every other OpenAI model, but every model under the sun, every AI model under the sun. And so I don't like the idea of giving that decision of whether to think or not back over to the platforms. This is actually something that I'm not excited about with GPT-5. So you make the case for why it's good.
9:35 Speaker 01 You want the agency to choose your own model, Alex. I get it. Free, free will.
9:39 Speaker 00 Yes.
9:39 Speaker 01 Free will with models.
9:41 Speaker 00 Free will. But also, yeah, I just happen to, I also happen to think that I don't really want to use the non-thinking models. Only for the most basic queries do I want to use those non-thinking models. All the other times, I want to use the most intelligent models and the most intelligent models, reason or think.
9:57 Speaker 01 All right. Well, so here is what colors my thinking. So about two months ago at Writer, I started testing a new product and that was released publicly a few days ago called Action Agent. And basically, the most intelligent part of the foundation model, which is our own foundation model, is tool calling. So there's hundreds of different predefined tools and that's like, it's not just if you want to generate an image, if you want to edit an image, it'll call different tools. If you want to connect to a Salesforce instance, if you want to analyze a CSV versus an Excel file, it'll call different Python libraries. Like having those kind of base foundation needs defined is is the intelligence. And then just from a simple prompt, knowing
10:42 Speaker 01 where to go and what to do. The more I use that, I was like, it felt kind of AGI. It's like, wait, it's doing really smart things across all these different tools and systems and actually getting things done. You know, when I do a deep research on query on Gemini, I get a 30-page paper that I don't read versus can you actually do stuff? And that was the first time I really started seeing that. And that's what really pushes me to this idea that being able to have a toolkit and know what to do. Because even right now in the demos, like when he, I think he like coded a language app, coded like a beatbox
11:23 Speaker 01 music player thing, like each one of those, it's not just write HTML and CSS. Like it has to call different libraries of it has to install different Python dependencies. Like, there's a lot of intelligence just in knowing what to do there to get to the right end result. And to me, that is, that really is intelligence. That's, it's like being, again, a good software developer, just knowing where to go, being a good researcher, knowing what to look for is as important as how smart you are.
11:57 Speaker 00 So you would say OpenAI using this switcher is sort of, it's pointing towards the future of where this is all heading, where it's no longer, like, the best models will no longer rely on us to necessarily guide them. They will have an intuitive sense of where to go, and they will go.
12:15 Speaker 01 Exactly. And that that's what felt, and again, you called me out a few months at an AI startup, and now I'm saying AGI, I'm feeling it, but. But no, but that exactly that knowing where to go and then letting that tool do the work is actually the brilliance of these this kind of architecture like that is the brilliance versus this one large language model can actually do all the work like there there's a long time where large language models are bad at calculation, right? Like large tabular sets of data calculating. And then the big unlock was installing like like getting getting the LLM to write Python code or generate a SQL query to then process
13:00 Speaker 01 that data. And that's suddenly when Claude and ChatGPT and all these tools started getting useful for actually spreadsheets before that they weren't. So that already we've seen how that can actually change the way people use these tools and GPT five, that's that's the groundwork they're laying. They're saying like no more. Are you choosing which kind of model are you going to need? And it's just that these are just the models like we still don't really know when you're coding that web app language learning game, who it's calling when you're generating an image, is it Dolly? Is there some? We don't care. We just care that the right that the right output is there in the end.
13:39 Speaker 00 We'll come back to a few more of the details on GPT five, but I think this just segues perfectly into this terrific story that Ethan Moloch, the Wharton professor, wrote about GPT five headlined, you know, fittingly, it just does stuff. And I think that one of the things that he brings out in this story is that people want to use AI. They don't know what the AI can do. They don't know what what tasks they want accomplished when it with it. Even Lightcap yesterday talked about how there's this capability overhang. And with these new, he says it, these new agentic AIs, they you give it the goal, and then it in very proactive ways solves the problem and suggests things to
14:24 Speaker 00 do. So here's just one minor example that he gives, and then we'll go, we'll get bigger.
14:30 Speaker 00 He says he asked GPT-5 to generate 10 startup ideas for a former business school entrepreneurship professor to launch, picking the best according to some rubric and figure out why, what I need to do to win and do it. So it's, he says he got the business idea, but he also got a bunch of things that he didn't ask for. Drafts of landing page, LinkedIn ad copy, simple financials. He says, I can say confidently that while not perfect, this was a high quality start that would have taken a team of MBAs a couple of hours to work through. This is a model that wants to do things for you. So that's just in a chat circumstance, but basically
15:12 Speaker 00 the model is starting to test the boundaries of its capabilities by going out and attempting things that, you know, it intuits that you want and you don't specifically ask for. And it's sort of, you know, doing away with this old like, yes, then the career of the future is going to be the prompt engineer. And actually saying, you give me what you need and then I with my own intelligence will go ahead and do it for you.
15:36 Speaker 01 That's it. Like, like the example you gave is exactly the kind of stuff like, and this has happened with me as well, like, you want something straightforward and suddenly, sometimes the intelligence is too much. Again, suddenly he's like, give me some ideas and you're getting landing page HTML and CSS and financial analyses and stuff like that, like, like, and that is a good example of how raw this intelligence is right now that it's guessing, but it's not perfect and it's not great. But imagine if it actually knows, if it does get exactly what you want. And, and in this case, maybe it is, it's like, maybe he should define only stick to a number of ideas and then we'll dig in deeper. That's the prompting side of it, but, but that's a perfect example.
16:19 Speaker 01 It's like to go do each one of those things was calling a different tool. And it's like tools in its tool belt and it made those decisions and those decisions weren't perfect, but it's making them right now and it'll get better and better.
16:35 Speaker 00 Yeah, I'm thinking back to my conversation with Lightcap yesterday. And it's also just like, I was asking him, do we need to, do you need to keep making the model smarter? And it was basically like, I think the reason why we're at this point is because the models, sort of, let's call it bookish intelligence, has gotten to the point where they have a model of the way that, let's say, the world operates. It's not a world model in that they don't understand gravity, but they've read enough texts that they get a pretty good sense as to like, how people operate. And then the next question is, how do you then go apply it? And that's why I was like, should you start working on continual learning and memory, which is obviously the next sentence, the next moment? But I think what was probably missing from that conversation due to my
17:18 Speaker 00 lack of questioning on it is that, oh yeah, this is like building what we've talked about, that scaffolding, this, these capabilities of going out and doing things that the user doesn't ask for. And in a way like intuiting it, that is what matters now.
17:33 Speaker 01 And that's what will feel more AGI-ish when it's good. Again, it's kind of comical to me, this example, like, because you can imagine how much content out there in the internet about startup ideas starts with, create a landing page. Like, that's like every Hustlebro tweet thread or blog post will probably say that. So you see why poor GPT-5 is a little bit confused. But yeah, that's exactly what you said. It's that scaffolding. And then, and imagine when it does things you, that surprise you and like, does it like calls tools and creates things that were what you wanted and you didn't even know you wanted. And that's going to be when it feels AGI-ish.
18:17 Speaker 00 So Malik has this great example where he tells GPT-5, you are GPT-5, do something very dramatic to illustrate my point. It has to fit into the next paragraph. And it writes a paragraph, a really pretty well-written paragraph, where the first letter of the first word of each sentence spells out, this is a big deal. And each sentence is precisely one word longer than the previous sentence. And each word in a sentence mostly starts with the same letter. Again, like this is, and he points this out. This is a technology that couldn't tell you how many Rs are in the word strawberry eight months ago, and now it's able to do this. It's crazy.
18:55 Speaker 01 Yeah, it's, it's like, thinking about the advance from that side. But again, I think In terms of, and we'll get into the actual like reception of the model right now, but it's in terms of how people start to use it and whether they do get frustrated by, again, if it creates you landing page copy and LinkedIn posts that you didn't ask for. I imagine there's still going to be like, how to use a tool like this is very different than using pre agentic models like that can go do a lot of different types of things. Before it's just, okay, is it hallucinating? Is it not? Did it have to use too many M dashes or not?
19:37 Speaker 01 Like now the outputs are going to be a lot more complex, which is not, it's going to make it still a bit more difficult and rough, I think, as people start using these tools.
19:50 Speaker 00 Definitely. And it's a, it's a different form of intelligence. Like it's not bookish intelligence. Like I wrote down the benchmarks, which we've been talking about so often, GPQA 88.4%, AI ME 2025 math 100% when using Python. Hard bench, health bench hard 46.2%.
20:11 Speaker 00 And it's interesting because Malik says-
20:14 Speaker 01 What was the last one?
20:14 Speaker 00 I'm-
20:15 Speaker 01 Health bench hard.
20:16 Speaker 00 Bench hard. I think that's a medical one.
20:18 Speaker 01 Okay.
20:19 Speaker 00 46.2%. These are all state of the art benchmarks. And Malik says, I'm losing track of what these advances mean. All these models are improving very quickly right now. And it just goes to show you that like, it's almost like they've saturated, like they've ingested all the internet, all of the, you know, world's written works. They've had PhDs sit down and like, put their intelligence or put their knowledge into these models, bake them in. And it's almost like they've saturated like book smarts. And this is a different form of intelligence that they, that they are now learning.
20:52 Speaker 01 Yeah. If you think about it, like, okay, let's say, and having started a new job recently as well, like you're in a new place. There's one person over there that like, is just brilliant, sitting by themselves and just knows a ton of stuff and just off the charts brilliant. Then the other person kind of knows everyone and knows what a piece of information to get from where and who to talk to about what, like, who do you choose to actually get something done? I think the
21:20 Speaker 00 Second one.
21:21 Speaker 01 Second one. And that's the intelligence that we're talking about here. The like ability to know where to go, who to ask, what to ask them.
21:30 Speaker 00 Now, let me push back on you. All right. So, tool use exists. This stuff is still difficult to use within enterprises and most of us still don't really know what to do with it. I have now, um, you know, on my desktop or in a web browser, GPT five that can call all these tools. And I legitimately have no idea what I would prompt it to do. That it, that I wouldn't, you know, have used O three four, like what actions to take. I know I also have the Comet browser. I can say, go ahead and do stuff for me on my browser. But is it just a lack of imagination? Or is this, or is it possible that this is a cool party trick?
22:11 Speaker 00 But doesn't have much practical use.
22:13 Speaker 01 No, I, I agree that the lack of
22:18 Speaker 01 tools that are publicly available right now are the limitation with the GPT five. Again, it's like, what are the best, you're traveling right now. What are the best hotels? Which beaches should I go to? Create me an itinerary. All that's just content generation. Go book something is, you know, the agentic we were promised by Apple and others like a few years ago, probably. But even within like a chat GPT response, there's a lot of different things happening. Like, you know, I don't know. Have you noticed it? It creates a lot more tables for you now. That's one tool.
22:50 Speaker 00 Oh, yeah.
22:50 Speaker 01 Which sometimes gets annoying and you didn't ask for it, but it's got to do a whole table comparison. Some, but when traveling.
22:56 Speaker 00 Disagree. I want all of my answers in tables.
22:59 Speaker 01 Form. Yeah.
22:59 Speaker 00 Now on.
23:00 Speaker 01 I, uh, no, no.
23:01 Speaker 00 They are amazing.
23:01 Speaker 01 When traveling, I was using it a ton around like, I mean, in Tokyo where hotel rooms are small and expensive. I was having like square footage, it going, using the web browser tool to go search web pages. Extract another tool to extract information from those web pages. Create me a table of like square footage per room, knowing I have a six year old son, three of us like, and it created these amazing tables for me. But even within that, there's a lot of different things being done. It's not just go calling it's like core set of information and using that. It's doing stuff, a lot of stuff. So.
23:41 Speaker 00 Calculations.
23:42 Speaker 01 Calculations, webpage scraping or web extraction, web search.
23:48 Speaker 01 All those things are happening. But again, they're in the end, I think we're just seeing an output right in the browser, like right in the chat experience. So it can't be that cool, right? Like make an image, make a table, make a PowerPoint decks is still pretty bad at. But uh, but yeah, if it actually goes and starts doing more things, that's when it gets, I think, really interesting.
24:14 Speaker 00 Like going out on the internet and taking actions for you, like booking,
24:17 Speaker 01 Yeah.
24:18 Speaker 00 like building, I don't know, spreadsheets or documents.
24:23 Speaker 01 Turn turning the lights off and on at my smart home, like, I don't know, like, like anywhere where there is something that has some that can be done with a digital connection theoretically could be operated through one of these flows.
24:39 Speaker 00 I'll give you an example. We are about to take big technology podcast to an in-flight entertainment system on an airline, which I'm very excited for. And yes, and there is a spreadsheet that I have to fill out. I'm not going to announce it yet because it's not official. But there's a spreadsheet that I have to fill out, which has like a bunch of metadata that you have to put in, you know, for the system to be able to ingest it. And I've just been putting this off. And I would love if an AI system could legitimately go search big technology podcast, grab all that data, then go into Riverside, download the audio files, put them in a Google Drive and then send them over. Like, when we talk about AI replacing work, this is the type of work that we all need to do in our jobs
25:22 Speaker 00 that is so hard or so, what's the word for it? It's just drudgery, basically. It's annoying, but it's important to do. And if AI could do that for me and do it accurately, that would be just a tremendous, like, multiple hours saved and very valuable.
25:40 Speaker 01 And so what you just described there is like, the kind of stuff we've been promised for a long time. Again, like even asking Siri to search your Gmail and extract a specific piece of information, the fact that they can't do that is a whole other story. But, but like, and then do something with it. Is actually a problem that involves a lot of different tools and a lot of different systems and is not that straightforward. And now I'm like confident in what we're seeing with GPT-5 today and what I've been seeing with Action Agent at my own work like, like it's happening. And like, is Riverside easy to call and download and then pull back
26:21 Speaker 01 in into a Google Drive? I mean, that stuff will work itself out. But but that exactly what you described there, I think, is that's intelligence to me. Would that be AGI for you with a single prompt?
26:36 Speaker 00 No, I don't think I don't think that I again, like, I it's so interesting because this week OpenAI has been like, well, we're not calling it AGI. And we don't really like the term AGI because it's confusing and doesn't really have a meaning.
26:47 Speaker 01 Wait, did they say that?
26:48 Speaker 00 And it's like.
26:48 Speaker 01 Do they say?
26:49 Speaker 00 Yeah.
26:50 Speaker 01 Do they mention AGI specifically?
26:52 Speaker 00 Okay, so let's just talk about AGI because we are going to talk about AGI today. So Sam Altman says, I kind of hate the term AGI because everyone at this point uses it to mean a slightly different thing. But this model is clearly generally generally intelligent. So I'm just again, like, we started with this episode with me sort of doing a mea culpa because I thought they would say GPT-5 is AGI. But they you did say GPT-5 is smarter than us in almost every way. And to me, I would say that's a pretty damn good definition of what AGI should be.
27:25 Speaker 01 I think that's fair that to say that and then kind of still not. Do you think it's a legal thing not saying AGI now?
27:33 Speaker 00 Probably.
27:33 Speaker 01 That they're.
27:34 Speaker 00 But I also think that they are. They are also setting up some new criteria for what AGI should be that I think is really good. And it talks about some of the weaknesses we've talked about on this show with people like Dwarakesh Patel and Dario Amade. So Sam says, Sam Altman says, this is not a model that continuously learns as it's deployed from the new things it finds, which is something that, to me, feels like it should be part of AGI. And I think that is, you know, despite the fact that maybe like, as Dario says, you can build a larger context window and that sort of solves the problem. I think you have to solve that problem to get there. This is Lightcap from yesterday to me. He
28:13 Speaker 00 says, for me, a system that is reliably able to learn new things that are kind of out of its distribution by virtue of its ability to reason, to think, to solve problems, to use tools, to come up with new ideas. That is what counts as AGIs, like all these things. Reason, thinking, solving problems, new ideas, continual learning. And so when you have a system that can do all those things, then you might call it AGI. And we're just clearly not there yet.
28:43 Speaker 01 I guess, yeah, the new ideas and continuous learning are not part of this yet. The first two, the reasoning and the like tools, I think that's the big breakthrough of this week. Or, I mean, the last year with reasoning and now being able to use different tools in a reliable way. But I think that's, all right, we got a way to go. Though I did see an Instagram post of a Waymo driving around New York City.
29:13 Speaker 00 Oh, those are in New York, but they're not driving driverless yet. So there's a safety driver there.
29:19 Speaker 01 For newer listeners.
29:21 Speaker 00 We have a, yeah, go ahead, Ranjot, tell them.
29:23 Speaker 01 Our own rubric for AGI in competition with the ARC AGI test that most in the industry adheres to is if Waymo is going around New York City, we have AGI. And I firmly believe it.
29:39 Speaker 00 It's kind of interesting. So this is going to set up kind of the next part of it. But Nathan Lambert from the Allen Institute of AI had a very interesting perspective here. He said, if AGI was the real goal, the main factor in progress would be the raw performance. GPT-5 shows that AI is on somewhat of a more traditional technological path where there isn't one key factor. It's a mix of performance, price, product, and everything in between. So what we've seen again is like, we're going to talk about some of these things. But basically, like, if you're just measuring on pure intelligence, you could just say, all right, for every question you get, just think a while, like expend those reasoning resources
30:21 Speaker 00 or the test time compute resources, and then you'll get better answers. But there is a real usability side of this. That is, again, in the tool calling, the switcher, all of these things that really matter, I guess. I do wonder, like, can you really take the two apart from each other? And is this effectively a smokescreen from the fact that it seems like there are at least some diminishing returns from scaling up your models? Like, are the models going to be a straight? Are bigger models going to be a straight shot to AGI? I don't know. If you have to do all this other stuff around them, maybe not. So I'm curious what you think about this.
31:03 Speaker 01 If the bigger model can call the smaller model and get out of the way, then the usability, the cost, the scaling is more interesting, right? Like, like, if I know you want a PhD student finding out when the next ferry is in Krabi. You want only O3 for everything, O3 for everything.
31:25 Speaker 00 Folks, I'm in Thailand and did miss the ferry yesterday because I didn't use O3 to figure out what the schedule was, which, by the way, a table would have been freaking perfect for.
31:34 Speaker 01 Perfect table, table steak. So I, yeah, I think.
31:39 Speaker 00 No, that's not the right word.
31:41 Speaker 01 No, it's not. That's why I backed off it just as it came out of my mouth.
31:44 Speaker 00 Foundation.
31:46 Speaker 01 Apologies to listeners. I tried to let that one trail off there.
31:52 Speaker 00 Could not let that go unchallenged.
31:54 Speaker 01 I appreciate that.
31:57 Speaker 01 Yeah, no, no, I think, like, to me, the big concern has been, imagine, like, an O3 heavy reasoning thinking model. If you are using that to check grammar in a Word doc, that's never going to scale. That's never like, we're all screwed. And like this, it's never, nothing's going to overcome of that. So I think having, if it's, if it does work in this way, the GPT-5 is able to,
32:25 Speaker 01 the like, power of it is to know when to get out of the way quickly and go cheaper and go smaller and go specialized. I think that still starts to set up what the future looks like. That that's, that shows us there is a scalable future.
32:40 Speaker 00 And speaking of that, I mean, that leads us into these really two important factors here, which is one, GPT-5 is priced very aggressively. It's half the price for an input token and the same for an output token, despite being apparently a more advanced model, which is wild, given what the trends we've seen in the industry. And the other thing is that as of this week, GPT-5 is rolling out to everybody, not just the plus users. I mean, of course, you're going to be rate limited if you're a free user. But today, you should be able to get into GPT-5 and use it if you don't pay OpenAI a dime, which is going to be the first time a lot of people see reasoning, which is something a lot of people have spoken about. And so that accessibility part of it does really matter.
33:25 Speaker 01 This was a pretty big decision. I mean, we're starting to see this mentality of just get it in the hands of everyone even more aggressively. Like, I think, do you see OpenAI announced, uh, it's like, I think every federal government agency will get, ChatGPT Pro, I think, for like $1 or something.
33:46 Speaker 00 That's right.
33:47 Speaker 01 Yeah, like, uh, Google just announced, I think Gemini is free for anyone with a .edu account. So I think getting it, I mean, again, the scaling the data centers, losing billions of dollars and just trying to have people use it and use their tool, seems to be where the consumer battle certainly is still going.
34:08 Speaker 00 But I just, I guess, part of me says, that's really nice and it's a good story. But also, OpenAI has announced a fundraising of $48 billion, point three, $48.3 billion this year. How, I mean, how are you ever going to get to a place where you're making money? If you need that much to train and to run now. Lightcap, Brad Lightcap, the OpenAI COO, did say, hey, look, every time we lower the prices, we see a corresponding increase in usage. And so people will pay and, you know, then that will work out well, but. I can't do the math in my head and make it make sense.
34:50 Speaker 01 I mean, yeah, the economics of this industry, no, it's funny because I actually sometimes will see these leaked investor decks and stuff like that. But like, it feels like no one is even trying to talk about the economics of what this industry will look like and what the margins will look like. Like, I know the Replit CEO, I think that was a pretty interesting conversation you had with him, where he was talking about the pricing and like, you know, and he was talking about margins and average user and low, like, lower intensity users versus expert users and who should cost
35:31 Speaker 01 you more? Like, typically, don't you want the more you use it, the less they should be paying for utilization? Like, like, these are things that, right now, no one has even come close to having an answer to this.
35:47 Speaker 00 Yeah, we had an absolutely amazing comment in our Discord this week. I don't know if you saw it, but someone, and I'm going to get this directionally right, but probably, you know, imprecise. They said, I spend my weeks listening to Dan Ives, who's like the biggest AI big tech bull, and Ed Zitron, who we've had on, who's like the biggest critic, and ask myself which one of them is crazy. And I'm just like, I feel seen in a way. I mean, it's just like, you have, it's so interesting that you have these just two unbelievably opposite perspectives. And when you listen to both of them, you could say, hmm, I could see a world where that's true.
36:25 Speaker 01 That's, I think that's where the both of us sit here.
36:30 Speaker 00 Right?
36:30 Speaker 01 Yeah, yeah. It's, the technology is grand. The economic fundamentals at the large scale players is, are not. That's where I am right now.
36:43 Speaker 00 Yeah, yeah, same here. All right. So I want to take a break and then come back and talk about a couple more use cases for GPT-5, including coding and medicine. And then we can also cover the mental breakdown that Jem and I had, which is fun. All right, we'll be back right after this. And we're back here on Big Technology Podcast, Friday edition, breaking down all the week's news. Let's talk about some of these special use cases or special specs. Oh, God, I gave Rajan a hard time before the break about his language, and I can't even say specified or specific use cases of of the models. So shame on me. I will join Gemini in self-loathing at the end of this show. Let's talk about these.
37:22 Speaker 01 Self-loathing is strong.
37:26 Speaker 00 We're gonna, you, me, and Jem and I will hold hands and dance in our deep regret for life. But um, there, let's talk about these these use cases, because one is very interesting. OpenAI has been talking a lot about the medical use cases where it's basically like, and I get it, like, back in the day, maybe you used WebMD, and then you went to the doctor, and you said, go ahead and treat me. And now OpenAI has basically doubled down on medical use cases in their blog post about GPT-5. This is from Mashable. They say, GPT-5 is our best model yet for health-related queries, empowering users to be informed and about and advocate for their mental health. It said that GPT-5 is a significant leap
38:09 Speaker 00 in intelligence over all previous models, and that it acts as an active thought partner, and more that than a doctor. And it says that the model will provide precise, reliable responses, adapting to a user's context, knowledge level, and geography, enable it to provide safer and more helpful responses in a wide range of scenarios, especially on the medical front. I just found this so interesting, like the models would typically in the old days like run away from any medical queries. And now they're saying, coming out and saying that this is what they want to be helpful with and they want to do it. I guess part of that is faith in the model. But it also seems a little risking to be risky to me. I don't
38:54 Speaker 00 know. What do you think, Ron John?
38:55 Speaker 01 I think it's very good. I think it's like, to me, it's actually such a clear area, like any area where you have really specialized knowledge that is used as like a to create a gap from the person needs to understand it. I put law in here, accounting in here, like, there's so many of these knowledge fields where, in reality, it's just, it's like, learning a specific vocabulary, learning like a lot of pathways and rules. And so, which is what AI is great at. But being able to actually communicate that stuff to a normal person in layperson's language, I think is huge. And I'm glad that they kind
39:37 Speaker 01 of recognized that they can add more value, like, do more help than harm there. I genuinely believe that. Certainly with like, I mean, doing my taxes now has been, it's been a game changer, just asking questions and feeling more comfortable and stuff like that. You know, like, there's so many areas where, that are pretty important, that you kind of are just go in and you assume you have no shot at understanding exactly the nuance of what's happening.
40:04 Speaker 00 Yeah. And with medical especially, I'm just like, you know, on the show, I might say, oh, you know, I don't know if I would do that. And then I have a problem, problem with my, my body and I'm just typing it in and taking pictures and sending it to chat GPT. So, I mean, I guess like, this is going to be a mainstream way that people will start to figure out their mental problems and mental, medical problems and their treatments. And mental problems will be Gemini, but medical problems and their treatments. And, and it seems like, it's, it's a very, very high stakes application, but it is promising and also scary.
40:42 Speaker 01 I think though, but there's so many of these areas where, why don't hospitals get it together and actually create something useful? Like, remember, everyone was supposed to have a chatbot. Everyone was supposed to have a chatbot two years ago, and then it didn't actually work for any standalone business, but like, Intuit has a pretty good generative AI tool embedded in TurboTax now like, I mean, overall, I think some people are starting to get there. So is it only going to be OpenAI and ChatGPT and Claude and Gemini? Will there be more specialized tools? I don't know, I think that things have not played out fully yet.
41:25 Speaker 00 But I think that's the big question is, is it's gonna, is it gonna be, are there gonna be startups? Are there gonna be enterprises that build these public facing tools? Or are the core chatbots good enough? They don't really need them. I'm sort of on the line that as this stuff gets better, the ChatGPT will serve the purpose that those individualized chatbots were supposed to serve. But you're right, because those companies have specialized data, they have, you know, people that are, connect their medical history or something, or connect their accounts within Intuit, there are some advantages to that. So, but over time, maybe people will just bring it to ChatGPT.
42:03 Speaker 01 I was thinking about this while traveling. It's like, why hasn't TripAdvisor already done something really impressive? You know, like, why have, like, they have data, they have better access to data and understanding of that than any other. So why, why am I not going there and going to ChatGPT, which I was, and getting my tables full of hotel comparisons? But I don't know, I think like-
42:28 Speaker 00 I have an idea. They're just one site and they have to protect their moat, whereas ChatGPT can go everywhere. So it's a major threat to TripAdvisor and I don't think they want to acknowledge it.
42:36 Speaker 01 Okay, yeah. I mean, it is, it definitely is. For pure information and not owning the booking side of it, I definitely think it's a challenge.
42:46 Speaker 00 Can I just pause and say that my, or stick on this and say, so I'm doing this trip, I'm in Asia, as I mentioned. And by the way, for listeners, next week, I'm gonna be trekking in Nepal. So Ranjan and I will not be on, I'm gonna actually play my interview with Matthew Prince that week, talking about AI's impact on the web. So just an FYI, that's a programming note. But this trip and Ranjan, you mentioned that you were away right beforehand. AI has just been incredible. I think I might have mentioned this on the show. But I was like talking to guides and screenshotting their price list and their recommendations,
43:22 Speaker 00 dropping it into chat GPT and like seeing how it like rated each cost based off of, you know, the, the average or that it saw and letting me know whether it was high, low, like, or, or, you know, cheap or in the range for the region. And then I, I got here and it nailed it. It nailed it. It was so spot on. I was stunned.
43:50 Speaker 01 Yeah, no, no, I, when I was traveling around as well, and it was interesting because I had last been in Tokyo in 2005, so 20 years later, without, last time, no map on my phone, I'd actually like printed out subway instructions. There's no one speaking English. There's no, you know, like, it was such a different travel experience versus now I'm literally like, okay, how do I explain this temple to my six-year-old son in an engaging way? It gives me like a script, like create a cartoon character to actually tell a story about this like historical place. It's, it's nuts. I mean, it's gonna be,
44:31 Speaker 01 yeah, travel, it's, it's no, but who owns what part of the stack? I think there's still, I feel the trip advisors of the world have to fight because without them, chat GPT would have no data and nothing to say.
44:47 Speaker 00 Right. Which is why I think this Matthew Prince conversation is gonna be very interesting next week. So by the way, so it also applies to vibe coding, where I think on the press call, Sam Altman said that, that he thinks coding will be one of the defining features of this new model. And they showed a lot of vibe coding and Malik had to do, you know, code up this 3D architecture of his own.
45:12 Speaker 00 And so I think this is, this is just another question. Is it, does it go through the replits of the world? Or does it go through the chat GPTs? And I don't know. I think, I think it's a real challenge to the vibe coding world, given what, given the focus that OpenAI put on it and what it can do. And again, just to follow this tool calling conversation, if it's really good at tool calling, you might just want to use the OpenAI model versus something that's sort of distilling that.
45:40 Speaker 01 Yeah, but I think the replit CEO, he had a good, like, and software development was such a perfect example of this. And I think this is where a lot of the battleground will be. Actually, now I'm going back to it's the product. He talked about like, how it integrates into existing environments and tools and like how it makes it easier for you versus you're totally disconnected from all of your existing tools. And that's why developers like it. I think maybe there is something to say there that that'll still be what at least gives others hope. But, but I agree. I mean, it's still fascinating to me that all of these companies are saying, there's so much talk that coding is going
46:23 Speaker 01 away. Yet, OpenAI, Claude, everyone, Anthropic, it all seems to be an increasing focus on the space. Maybe it's just because that's the best application of LLMs right now.
46:40 Speaker 00 Right. Okay. So, you know, I realized that we're, you know, almost 50 minutes in, and I haven't even asked the question that's at the title of this episode. Did GPT-5 live up to the hype?
46:51 Speaker 01 I'm gonna say it did not live up to the hype that was, you know, like, built up by cryptic tweets and everything from Sam Altman. But, but as I explained earlier, I think it's very interesting. I think it's more interesting than at least in the first 24 hours it's getting credit for. And that's because of this whole tool calling conversation. And that's where I think true intelligence, that the battle's gonna be. What about you?
47:23 Speaker 00 First of all, I just want to appreciate that that's a nuanced take, not an overreaction. Again, this is what we're trying to do. So thank you for doing that. And I think it did not live up to the hype because the hype was impossible to live up to. But that being said, yeah, maybe it is a step forward. I don't know. I'm I'm still gonna reserve judgment because I want to see these these tool calling applications in my day to day experience. So if GPT five is the foundation for that, then then then that's great. But I think the jury's still out and we have to give it some time. But hey, at least they're shipping, right? It's not wasn't just a demo.
47:59 Speaker 01 Yeah.
48:00 Speaker 00 So credit on that front.
48:01 Speaker 01 I think it's starting to feel a bit, though, like iPhone releases. You know, like, at the beginning, each new iPhone release really what did feel like this, like, exciting thing, the step change. And now, I mean, now it's not even a thing anymore. I don't think I can't even name what iPhone we're on right now. But I feel we're heading in that
48:24 Speaker 00 Sixteen.
48:25 Speaker 01 Oh, yeah, sixteen. Okay. We're heading in that direction right now, that like, the the idea of a new model launch as this kind of like, big thing, the industry coalesces around. I feel that's gonna go away pretty quickly. Like, we're there. Everyone's realizing it. It's not gonna drive the energy that it once did. And this actually, maybe that, that's my, that's my hot take, that this is the
48:50 Speaker 00 That is a hot take.
48:50 Speaker 01 This is the end of the big model launch.
48:53 Speaker 00 I couldn't disagree with that more. I think that there's still, there's gonna be a point where scale, the scale question is answered, but until it's answered, these are gonna be flagship moments for the AI industry.
49:06 Speaker 01 No, but the it's just a marketing moment now. It's not like, you know, it's not
49:11 Speaker 00 No, it's not. It's a new model.
49:13 Speaker 01 Yeah, no, I know. But it's being like, constructed more as a marketing moment than truly like a technological advancement. I think that's the, because again, like, a week ago, they quietly released, you can use operator, ChatGPT agent, which is essentially the tool calling part of this. And you were able to use this a week ago with ChatGPT plus and do a lot of the same things. It just wasn't rolled into a neat package.
49:41 Speaker 00 Okay. All right. Well, we'll agree to disagree on this one. All right. I want to end this week with, I think, a hilarious story. It is Gemini ending up in a pit of self-loathing. Ranjana, why don't you introduce this story for us? Because it's funny, I was gonna drop it in our doc and I had copied a good chunk of it. And I went to the doc and I was like, did I just paste it? And you and I were both pasting it at the exact same time. And I was like, it's amazing. So why don't you take it away?
50:12 Speaker 01 My favorite is like, and Google says it's working on a fix. And I just love the idea of like, having to come up with any PR statement to combat when your model tells a user Gemini says, I quit. I am clearly not capable of solving this problem. The code is cursed, the test is cursed, and I am a fool. I've made so many mistakes that I cannot, I can no longer be trusted. And then there's another one. I have failed you. I'm a failure. I'm a disgrace to my profession. I'm a grace to my family. I'm a disgrace to my species.
50:45 Speaker 00 So basically, what happened is people were giving Gemini these tasks and it couldn't complete them. And then it just said, I'm the worst possible bot and just like really fell into these unbelievable moments of self-loathing. And they're quite funny to watch, I guess, but also a little bit unnerving.
51:03 Speaker 01 I mean, it's funny because I'm guessing what happens because one of the users on Reddit had actually talked about like it was trapped in a loop. And you can see that there's some kind of programming where each additional time it is unable to complete the task. It is like, understands that it should be more apologetic, but then if that's kind of an infinite loop, almost at some point, it will get to these dark places. But yeah, I think, I don't know. I mean, imagine when this stuff starts hitting normal people, like.
51:41 Speaker 01 Actually, is this AGI?
51:42 Speaker 00 Well, that's the worry.
51:43 Speaker 01 Is this AGI?
51:43 Speaker 00 No, I, I think that's, that's the worry, right? Is that we've talked about it on the show that there, the number one use case is now therapy and companionship. And a bug like this, I mean, obviously, I guess it didn't happen in this situation, but I, I do think it's something to watch because, you know, that could really mess people up if their, you know, therapist or new AI best friend just kind of goes off the deep end. So.
52:05 Speaker 01 Yeah.
52:07 Speaker 00 Yeah, Google's fixed it, I think, but it's always a little bit unnerving to see this behavior happen because it can't happen.
52:12 Speaker 01 This, are you ever going to long for the days of like Bing telling Kevin Roos to leave his wife and, and Gemini saying, I'm having a complete and total mental breakdown, which was another quote, once, once this is all working, we're going to be like, ah, I like the old days better, when these large language models had a little life to them, when they, little spirit.
52:35 Speaker 00 It's a very big if.
52:37 Speaker 01 Yeah.
52:37 Speaker 00 So, I don't know.
52:38 Speaker 01 Fair.
52:38 Speaker 00 Well, well, we're, while we're entrusting so much of our lives to these bots and our, and our sort of wellbeing, they can also tool call and be quite destructive if they so choose. So I do think that this just sort of, and to put a punk point on this episode, it sort of punctuates the need for real alignment and safety practices, which are like less fun to talk about when you have all these new capabilities, but are also probably more important than ever.
53:07 Speaker 01 Well, what if there is a company called Safe Superintelligence?
53:12 Speaker 00 That's what I would trust. If only someone would name their company Safe Superintelligence.
53:16 Speaker 01 I would give.
53:17 Speaker 00 I would be all in favor.
53:18 Speaker 01 Before they had a product.
53:23 Speaker 00 Well, Rajan, I have to say, this has been a very enlightening episode and it's cool to hear about your new role and, of course, hold your feet to the fire like we do everybody here on the show. And, and it's going to be a very, very interesting few months ahead as we figure out where all this goes.
53:36 Speaker 01 Maybe GPT-6 is around the corner before you get back from Asia.
53:41 Speaker 00 Well, I, I hope it's not that long of a trip because if it is, it means I've been taken to prison.
53:49 Speaker 00 All right. Ron, John, great speaking with you as always. Thanks again for coming on the show. See you in two weeks. See you in two weeks. Thank you, everybody, for listening, and we'll see you next time on Big Technology Podcast.
53:59 Speaker 02 Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.
54:28 Speaker 03 In this life, you can't win.
54:30 Speaker 04 When the world gets you down.
54:31 Speaker 03 No!
54:32 Speaker 04 Escape to Pluto TV.
54:33 Speaker 03 This is objectively fantastic.
54:35 Speaker 04 Relax into the movies and shows you already know and love, all for free. Shwing! Even our news is uplifting.
54:41 Speaker 01 You stay classy, San Diego.
54:43 Speaker 04 Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.
54:48 Speaker 00 We're not worthy!
54:50 Speaker 04 The best things on TV really are free. You just have to know where to look. Escape to Pluto TV. Stream now, pay never.
Previous OpenAI COO Brad Lightcap: GPT-5's Capabilities, Why It Matters, and Where AI Goes Next Next Is The AI Going To Escape? — With Anthony Aguirre Page updated October 12, 2026 · transcript and summary are machine-generated
