(Preview) What AI Could Mean for Aggregation Theory, o3 and Moore’s Law, More Questions Than Answers as Tech Enters 2025

Ben and Andrew return from the holidays to check in on the AI landscape. Topics include: Aggregation Theory and the return of marginal costs for hyperscalers, the architecture of OpenAI’s o3 model, the murky future for software engineers and SaaS companies, and whether Scarlett Johansson fumbled the bag. At the end: In praise of learning to ski as a middle-aged beginner.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Andrew Sharp?

  Hello and welcome to a free preview of Sharp Tech.

• 0:08Andrew Sharp?

  And Jeremy will kick it off. He says, "Hey Ben, it looks like your friend Doug Olafland wished you a Happy New Year by declaring, quote, 'The era of aggregation theory is behind us because of test time compute and the arrival of marginal costs to software businesses.' But is it? The way Chalé describes the 01 and 03 models, they sound more like brilliant product design than a breakthrough model. Are chain of thought and deep learning-guided program search the seeds of a brand-new architecture, or are they genius hacks to force a model that doesn't, quote-unquote, 'reason' to correct itself until it delivers something

• 0:50Andrew Sharp?

  good? It looks to me like a bunch of independent LLMs in a trench coat being puppeteered by a separate model." And just to anchor the majority of the audience that probably has not read the piece from Doug Olafland on fabricated knowledge, I'll read a portion of his thoughts on 2025 AI and the semiconductor outlook. He writes, "The era of aggregation theory is behind us, and AI is again making technology expensive. This relation of increased costs from increased consumption is anti-Internet-era thinking, and this will be the big problem that will be reckoned with this year. Hyperscalers' business models are mainly underpinned by

• 1:33Andrew Sharp?

  the marginal cost being zero. So, as long as you set up the infrastructure and fill an Internet-scale product with users, you can make money. This era will soon be over, and the future will be much weirder and more compute-intensive. Looking back on the 2010s, we will probably consider them a naive time in the long arc of technology. One of our fundamental assumptions about this period is unraveling." So Jeremy in his email continued on with more takes on 01, and then says, "Anyway, I'm not the one whose career-defining theory was just unceremoniously dumped by a close friend. Do you agree with him?" So Ben, we got a couple of emails asking about that

• 2:18Andrew Sharp?

  particular article. What do you think of it?

• 2:21Ben Thompson?

  Number one, I am certainly not upset to the extent that is the case because frankly, the whole aggregation theory angle was getting pretty boring. This is the entire reason why I've always been super intrigued by not just AI, but also crypto is that is also sort of a technology that is to a certain degree in defiance of the realities of aggregators. Aggregators are all about, as noted, zero marginal cost content, the endless duplication of content, all these sorts of principles and stuff that sort of stands against that is sort of inherently compelling and potentially valuable in that world for that precise reason.

• 3:03Ben Thompson?

  I mean, stepping back by and large, nothing goes away. Everything sort of builds up on on everything else, right? And this is always a challenge when you talk about new paradigms is we humans sort of tend towards zero sum thinking with a lot of this sort of thing. It's like, oh, well, this this era is done, the new one's here, when in reality, the new era sort of layers above. Now, from a sort of analyst and pundit perspective, that's often a useful way to think because what gets layered above isn't sort of a growth driver. It's just sort of there, right? The sort of go to example is we're still using mainframes and like, like they're still undergirding banks and they're still undergirding back

• 3:45Ben Thompson?

  offices and ERP systems and all these sorts of things. And even there, there's been lots of interesting changes and disruptions that no one really pays attention to because all the interesting stuff sort of happening at the high end. So I think to the the way I would reframe this is, is sort of aggregation theory as the dominant driver of growth in technology and thus by extension, the US and thus by extension, the world, is that ending? Yeah, I mean, yeah, I think that that's at least as constructed, that's a reasonable way to put it. But that, from my perspective, to sort of start selfishly, is a reason I personally feel excited and re-energized like, like, because that's like, there was

• 4:31Ben Thompson?

  Was a period circa probably around the middle of the pandemic. And sort of, I guess that would put it between 2020 and 2022 when it went sort of chat GPT came out and really opened everyone's eyes to this, that like the only thing to write about with aggregation theory is like antitrust.

• 4:48Andrew Sharp?

  I was gonna say, you hit a point where it was like lawsuits were the only potential variable in the middle of the landscape there. I mean, how much would actually change though, given how deep the moat is for most every aggregator after the last 15 years?

• 5:07Ben Thompson?

  Well, so, so just to, just to, let me address a couple more sort of points in this article. And or in this sort of, in this question. So I don't want to like just set aside the architecture of like 01, 03. But I also do want to set aside the architecture because I think to dismiss it as a pile of hacks, or this is just a new algorithmic change instead of like a model to rule them all. That is not an insult. That's how sort of progress happens. And I think actually, speaking of Doug, him and Dylan, or maybe it was Dylan had an article about, about scaling before that made the point that, oh, the reason why, you know, if you have inference time scaling, which I wrote about earlier this year, when, when sort of 01 came out.

• 5:46Ben Thompson?

  There's sort of a theoretical argument that sort of came up around this, which is, well, no, that's cheating. We were told the scaling was on the model. You just make the models bigger and they get better. You can't say the scaling somewhere else. And what I really liked is, is Dylan made an analogy to Moore's law. Which Moore's law, the, the way in which it progressed was Moore's law itself was linear, but the, the axes of, of improvement continually changed. So for a lot of people listening, all the knowledge and awareness of semiconductor manufacturing has been about lithography. EUV, EUV, ASML.

• 6:25Ben Thompson?

  And that was the main vector of sort of improvement in Moore's law and making chips smaller and more efficient over the last 10 years, 10 to 15 years. Before that lithography wasn't that big of a deal. It was kind of a real, really big deal in the eighties and Nineties when when the Japanese really took over and there was a bit where we went to immersion lithography and that was a big deal. But there were other aspects that actually really mattered. There was, you know, whether it be sort of, sort of the metallurgy that goes into it and different materials. Or transistor design when we went to 3D transistors, it was called the FinFET transition where it used to have planar transistors, which were more 2D. Then suddenly you're stacking, if you think about it, these chips on, uh, used to be sort of all these flat gates.

• 7:06Ben Thompson?

  Then they went basically became like skyscrapers where they went up and down in addition side to side. Just, just if you just can envision that, what an incredible shift that was, right? But if you zoom out and look over the, the, the period of, of processor change from the sixties till now. It looks like a fairly continuous curve that has flattened a bit and it does continue to flatten somewhat. But it, it's like a continuous, smooth improvement, but that improvement was driven by focusing on different things at different, at, at different times.

• 7:38Andrew Sharp?

  Different vectors depending on the decade.

• 7:40Ben Thompson?

  Right. You hit a wall in one era, so you start, what, what else can we improve? We're talking about a process that is thousands of steps. All those steps can be transformed and changed and you can sort of do new things. And that applies, I think generally, and in this case, the sort of scaling question, the reason why oh, one was a big deal when it came out. And honestly, I think oh, oh, three, what it's super impressive and the results are amazing. I personally wasn't as blown away, but that's because I was blown away by oh, one. Because what oh, one showed is, yes, before with all the LM models, it's like all the scaling was on the front. You make bigger and bigger models with more and more data and apply more and more processes to it. You get smarter and smarter models.

• 8:24Ben Thompson?

  Suddenly with oh, one, it's like, okay, wait, this is the same in broad strokes base model as GPT four. What's different is that when you're asking it to do an answer, it's spending more time. And critically, the more time it spends, i.e. the more compute it uses, the better the answer is. That, then that's what Doug is, is talking about here. And the big thing here, and this is sort of the, the big theme that I'm thinking about and, and you know, like my, what's my, gonna be my opening article of the year? I don't want to sort of give too much away. But I think there's a, there's a counterintuitive reality here, which is we're moving from a time of.

• 9:06Ben Thompson?

  There's almost like a broader media analogy to a certain extent where, you know, you talk about sort of the post-World War II consensus and everyone watched TV and there's the nightly news and everyone had the accepted same set of facts and and there was debates and there was right and left and Democrats and Republicans or whatever it might be. But actually, it all sort of operated within a pretty narrow bound. And this this applies to the stuff we've talked about where it comes with with things like trade and globalization and questions on those lines. And one of the things that's happening, and this sort of came up during COVID, where I think a critical mistake that was made in COVID was a deference to experts.

• 9:49Ben Thompson?

  And the problem is experts are expert in their specific domain. But political decisions sort of by definition encompass multiple domains. It's about making trade-offs. It's a trade-off between sort of limiting the spread versus kids need to go to school versus businesses need to stay in business versus hospitals overwhelmed. Just to choose, like I think I just picked two on both sides that argue for one direction or the other. And there was a certain sort of abdication of responsibility amongst the political class by deferring to experts. But the deferring to experts was itself a political choice because you were foregoing the responsibility of making a trade-off. There was there was a

• 10:30Andrew Sharp?

  Experts don't have a priority stack per se. They have, or at least they are likely to lean heavily on their area of expertise in terms of what they're prioritizing.

• 10:41Ben Thompson?

  That's exactly right. That's exactly right. And so guess what? The public health officials think we should do the maximum amount of thing for public health. They're not thinking about the economics. They're not thinking about kids' educational outcomes. They're not thinking about like, what's the implications of people getting turned off to all vaccines instead of just this vaccine, right?

• 10:59Andrew Sharp?

  And it works like if you go to a lawyer and say, should I publish this? I might get sued. The lawyer, every single time will say, do not publish that.

• 11:07Ben Thompson?

  This is the lesson I learned at Microsoft, because when I was at Microsoft in in the 2010, 20 or 2011, 2012 era, we were still under consent decrees or some of them Just rolling off from sort of the Justice Department sort of thing. And of course, you've got the whole talk about there's certain words you don't use, especially in email. But but one of the key things that that my manager sort of told me, he's like, look, a big mistake people make in general, but here in particular is they overly defer to, I think it was called LSA, where legal services or whatever. But they they listen to the lawyers too much. He's like, the lawyer's job is to inform you of the risks. It's your job to make a decision.

• 11:47Ben Thompson?

  Like, like, and sometimes you're going to do what the lawyer doesn't want you to because you weighed the risk. You understood the issue, but you you were thinking about the bigger picture. It's not their job to think about the bigger picture. So the lawyer example is a great example. You see lots of folks paralyzed and lots of companies sort of get all tied up in this. And I think we've witnessed this to a certain extent, you know, arguably this stuff is happening with Google. And it could be lawyer stuff. It could be sort of like the political fervor sort of aspects. Yeah, yeah, like, like, as a manager, your job is to make decisions, is to make trade offs. As a politician, your job is to make decisions, to make trade offs. And then you get held accountable for that. As a manager, if it worked out, you get promoted.

• 12:32Ben Thompson?

  If it doesn't, you're out the door. If you're a politician, ideally, you get reelected. If you don't, you're out the door. That's the way it's supposed to work. But there's a large extent to which I think society generally got away from that. And I think that was probably the case with tech. And that's something I think that Doug is getting at here. The decisions, the architectural decisions, sort of what you do, were easy in a certain sense. Because what in theory is a big decision, like, how much do I spend on this, was not a decision at all. You spend the maximum amount possible.

• 13:04Andrew Sharp?

  All right, and that is the end of the free preview. If you'd like to hear more from Ben and I, there are links to subscribe in the show notes, or you can also go to sharp tech.fm. Either option will get you access to a personalized feed that has all the shows we do every week, plus lots more great content from Stratechery and the Stratechery Plus bundle. Check it out, and if you've got feedback, please email us at email at sharp tech.fm.