AI Vibe Check: Lab Wars, Why APIs Might Vanish & Future Predictions

Six months after their last roundup, Jacob sits down with Ari Morcos (Datology AI CEO, former Meta AI researcher) and Rob Toews (Radical Ventures partner, Forbes AI columnist) to take stock of an AI landscape that has shifted dramatically: coding agents crossing the long-time-horizon threshold has turned engineers into managers of agents, near-frontier open weight AI looks like it may be disappearing as Meta and the Chinese labs pull back, and Anthropic's restrictions on its newly released Fable model have its biggest supporters questioning whether safety framing is masking competitive positioning. The conversation runs through the full state of the lab wars, including Rob doubling down on his Sam Altman ouster prediction and the Bret Taylor succession theory, why Google's structural advantages remain intact despite falling behind on coding, what xAI's Cursor acquisition is really for, and Ari's claim that compute constraints could push labs to suspend their APIs entirely. The back half digs into the physical bottlenecks underneath it all, from atom and x-ray lithography startups challenging ASML to H100 prices reversing their decline, before closing with predictions: recursive self-improvement is closer than it was six months ago but slower than the takeoff narratives suggest, robotics is nearing its GPT-3 moment, and Anthropic's next chapter may be life sciences.

• 0:00Speaker 00

  I'm Jacob Afar and this is Unsupervised Learning. We've had a bunch of new subscribers over our last few months and so wanted to welcome you to the show. We basically probe the sharpest minds in AI on everything that's happening today, what's real and what's coming up, where the space is headed. And today's episode is one of my favorite formats we do. It's an AI vibe check that we do with Ari from Datology. Ari was a former researcher at DeepMind and Meta, now runs a really exciting AI startup and Rob at Radical, one of the great AI venture firms. The three of us talk about everything happening in the AI world today. We talked about Fable, of course, and the reaction around the release as well as model capabilities. We talked about how close we are to RSI. We hit on some pretty spicy predictions, including that the labs

• 0:43Speaker 00

  may actually get rid of their API business as the compute crunch continues. And we just got to hit on all the main topics of today. Just really fun to sit down with two friends and great minds in AI. I think folks will really enjoy this. Without further ado, here's our conversation.

• 0:59Speaker 00

  It's time for another roundup episode. I always love doing this with you guys, Ari and Rob. I feel like we had a ton of fun in the last one, but like, God, it's AI world, things have changed. I feel like we last sat down after NERUPS and I think since then we've had IPO filings, we've had, you know, models not launched and then launched. We've had, you know, SpaceX becoming an AI infra company. No shortage of headlines to discuss here, so excited to have you both back on the show.

• 1:24Speaker 01

  Excited to be here. Yeah, thanks for having us.

• 1:26Speaker 00

  So I think to kick it off, you know, six months is an eternity in AI world, but I figured I'd start at the highest level. What's the single biggest thing that has changed in how you're thinking about the landscape since we last talked? And maybe, Ari, I'll start with you.

• 1:39Speaker 01

  Yeah, I mean, I think the most obvious thing that has changed over the last six months is starting to see the coding agents really start to work at longer time horizons. Right? I think that was just starting when we recorded our last episode at the end of 25.

• 1:53Speaker 00

  I feel like everyone went away over Christmas break and was like, holy crap, like, these things really work.

• 1:58Speaker 01

  Yeah, and I think it starts to show how there are these thresholds where if you go beyond a threshold, it can become a lot more valuable. And obviously, that's driven the massive rise in token spending and the whole token maxing idea and all this stuff. But I think we're really starting to now see the shift of engineers, at least, kind of almost all moving from ICs to managers of agents. That's been something that's been very noticeable within Datology, for example, over the last number of months, is seeing more and more people starting to now context switch between managing different agents, rather than just kind of, you know, working on the one thing. And that is enabled by having these, you know, agents be able to run long enough and actually be useful in various ways.

• 2:39Speaker 00

  I mean, everyone likes to ask top AI researchers like yourself, like, how much more productive has it made you in your, in your work?

• 2:45Speaker 01

  I think that it's interesting. It makes you a lot more productive in some ways, right? But it also produces a lot of challenges as well. Like, one of the things that we're struggling with is now, it's a lot easier to produce a massive amount of code that can do something. But now you have this pretty massive understanding gap, and it's a lot easier to put slop into your codebase. So it's definitely made, made us more productive. I think a lot of times, though, the kind of top line numbers tend to be overestimated, because it doesn't take into account some of these, like, later costs of, like, we now have big bottlenecks on reviews. And we don't want to go fully to, like, oh, you just, like, my agent will review your agent's output, you know?

• 3:21Speaker 00

  The bottlenecks just seem to shift. You know, whatever, it's hard to improve on an entire process because of, because of that. What about you, Rob?

• 3:29Speaker 02

  There are early signs that seem to suggest, over the past six months, that make me question whether open weight AI is going to continue to be a really meaningful force in the ecosystem going forward, at least, like, near frontier.

• 3:43Speaker 00

  Wow, just coming from the jugular for Ari, like, right off the, right off the top here. I like it.

• 3:48Speaker 02

  Yeah, yeah, we can, we can dig in in more detail. But I think, like, yeah, six, six months ago, or for the past few years, like, my kind of working assumption had been that the closed source proprietary models would advance the frontier, and there are a lot of structural reasons for that. But the open frontier is only, would only be a few months behind. And that gap might widen a little bit. I didn't think it would shrink altogether, but like, I thought it would persist as, as being relatively small. And I think there are signs now that, like, it seems like there's a real risk of near frontier open weight AI falling off altogether. I think Meta, which historically has been the...

• 4:27Speaker 02

  The open weight champion in the West is pulling back and it seems likely that they're not going to continue with their open source strategy. And then more recently, obviously, the Chinese labs have been the ones driving state of the art open research and there it seems like there are strong indications that they may also be pulling back from that. And they're, you know, whether it's Quen or DeepSeek or others, they're most high performing models. They're now keeping proprietary behind an API and just open sourcing, open weighting, you know, smaller, less performant versions. And I think there is like real compute incentives behind that. Like, it's just very expensive to service these open way models with no revenue

• 5:11Speaker 02

  coming in. And I think there's also geopolitical and competitive considerations. But it's interesting to contemplate what a world might look like where like if you want real frontier artificial intelligence, like you have to pay a company for a proprietary model as opposed to being able to, you know, have access to the weights yourself.

• 5:30Speaker 01

  Or build your own. But I'd actually agree with that. The second one, the other one I was thinking about is, you know, what changed? I don't think that we've seen a major change in the capabilities or the trend line of the capabilities of the open weight models. I actually think if anything, we've started to converge and we continue to see that. I do think, though, the economic decision making around building and releasing open models has definitely changed over the last six months, which I think is what Rob was really getting at. And I am a lot more bearish on how many open models there will be going forward. It seemed like, you know, there was this kind of cornucopia of open models that was only ever growing in 2025. And I think we're now definitely starting to see

• 6:14Speaker 01

  that we probably hit the peak number of open models and it's now going to kind of get less and less. And because the financial incentives just don't make sense. Once you've already kind of achieved credibility, it makes sense to invest a lot of money to do that. But after that point, you want to start selling hosted inference of your model and opening it up just fully undermines your business, which so I think we're going to see like other Chinese labs start with like big open models, but then probably close up after that, once they've kind of gotten enough press and PR.

• 6:43Speaker 00

  Is there a business model for an open source model company? Or is it literally just its marketing to, you know, as you're on your way to the frontier? And then once you're once you're kind of close, it makes it inevitable to want to go closed source?

• 6:55Speaker 02

  I don't think there's a business model, honestly. There, you know, different things have been tried, the kind of like freemium, enterprise, like, you know, red hat model, but I just, I don't think that it, it works in AI, given the, just the massive upfront investment required to, to get to the frontier, close to the frontier in the first place. We'll see. Well, I like, you know, we can come back and revisit this soon, but I, very curious to see if we're, you know, first of all, for reflection, when reflection releases a model and what's business model they aim to pursue with it. But I, yeah, I remain skeptical that like a great business model exists for open, open source AI.

• 7:37Speaker 00

  It's funny, because I was going to say, one of the, one of the trends we've actually seen happen, you know, in the, in the past, like, month or two, is that it finally, you know, forever, I think people have been like, everyone will use smaller, cheaper, open source models for tasks that those models can do. And, you know, it felt like the vast majority of usage in tokens was still just like at the frontier, pushing capabilities, like figure out what, you know, models can do in different industries. And then finally, I feel like we have a movement now toward like, geez, these bills are pretty expensive, or our usage is pretty high, like, wouldn't it be nice to, to have something that was, that was, you know, cheaper and faster and smaller to use. But it's funny that it's happening at the same time that we're kind of seeing this real, you know, the closed source models, you know, run ahead of, of, of open source models. And Ari, I know you spend a lot of time thinking about this stuff, like, how do we think about

• 8:20Speaker 00

  those like countervailing forces that seem to be happening at the same time?

• 8:23Speaker 01

  Yeah, I, I mean, I think that, first off, I, I've definitely seen a lot of the, the former that you were talking about, like, it's been very interesting, the combination of the, you know, extreme compute constrained environments that we're operating in, the rise of capabilities of the coding models in particular, and then even just seeing, you know, from release to release, models changing their token efficiency, their output token efficiencies, has resulted in, you know, a lot of companies that were happy using frontier models, all of a sudden, even just going from like Opus 4.6 to 4.7, there was a big difference in token efficiency, and a lot of people's bills just doubled. Overnight and you've now I'm now starting to see talking to a lot of enterprises in

• 9:06Speaker 01

  particular really strong desires to start cutting the cost of using the models. And I think that wasn't there nearly to the same extent a year ago because the models weren't being used at a scale where those costs were, you know, meaningful enough. But now they've really reached that point where they are meaningfully not meaningful enough and you can just consume budgets so quickly because the models think for a long time. And that's now driving a lot of demand to say, okay, how can we make this much cheaper? I think that you can do a lot of that with open models as well. Like, like, I think one very notable thing, right, is that, like, a lot of people were able to reproduce the same or similar level of kind of, you know, vulnerability finding that Mythos use with open models by just putting scaffolding around.

• 9:49Speaker 01

  I think that's another one of the big changes, right, is that a model is not just a model anymore. It's the model combined with the harness and the scaffolding. And a lot of innovation is happening on the harness and scaffolding layer. I think that's also possibly how open source models can have a business model. You open source the model, you don't open source the scaffolding and the harness and you, you know, then have an API where people can access the full system. I think that could potentially work. Kimi is kind of doing that, right? Like Moonshot, I think, is is at a couple hundred military AR through the the API and kind of the chat interface. And that's to some extent what they're doing. So I think that we will continue to see strong open models, but probably fewer and fewer. And I think that, you know, this is going to motivate a lot of folks where they, you

• 10:32Speaker 01

  know, they have to now think, how do we survive in an era where there aren't going to be reliable open models? And I think that pushes people more and more towards actually starting to think about, okay, how do I make sure I have the capability to build a model in some way, to maintain a model, to do that repeatedly? I think that's a huge part of the reason that Nvidia has been pushing so hard through reflection and others, right, to have open model providers in the West.

• 10:55Speaker 00

  Yeah, I mean, certainly like the more model providers, the merrier for them. And it's interesting. I mean, it kind of almost feels like, you know, you saw an early manifestation of this with Meta, right? Where it's like, early on, you know, they, you know, the idea of having this controlled by a bunch of companies that weren't them was it was kind of like, you know, would never work for them as a business. And so it's like, we have to have something here to be competitive, even if we're not going to be, you know, the leading model provider just for our own business. Like, we want to have something that we can use ourselves.

• 11:22Speaker 01

  If you are one of these companies that's, you know, potentially threatened by this, you kind of have two options. Like, option one is that you go and work with the frontier model provider, you share your data and your domain expertise with them. They use that to improve the model with you. Maybe you're able to sign some agreement that gives you some long-term certainty around that. But eventually, they just outcompete you because you've given up, you know, any of your proprietary advantages. The only other option is that you try to really compete on the niche that is your focus and where you have, you know, unique capabilities. And I think we're starting to see that across the board, right? Look at what's happened with Cursor, look what's happened with lots of other folks, where they're realizing, okay, we have to kind of move away from this.

• 12:00Speaker 01

  There's also a margin perspective from that as well, right? Where just like fundamentally, if you're competing against somebody who has better margin than you do, then it's very hard to win.

• 12:08Speaker 00

  Yes, they will always give themselves better rates than they give you as an app company on top. Rob, Rob, what was your whole reaction to the, you know, the apps are cooked and SaaS apocalypse, you know, narrative of the past months?

• 12:19Speaker 02

  Yeah, that's interesting. I think like market narratives have a way of like swinging so far in one direction or the other to the, to the extent that a lot of the nuance is lost. So I do think like a couple of things are true simultaneously. Like, I do think a lot of traditional software companies and even categories potentially are at real existential risk on account of the frontier labs and their, and their product roadmaps. And so I think like a lot of that rerating was rational. And there, I think there are a lot of big companies that have big revenues and big customer bases that are in real trouble based on how OpenAI and Anthropic are executing and performing.

• 13:00Speaker 02

  I also think it like for sure was an overreaction and an oversell and like painted with too broad of a brush. And I think like, you know, in our field of venture capital, it certainly is like at the kind of prevailing narrative is apps are so challenging to invest in, stay away from them. Like, deep tech and hardware and so forth have all become more appealing and consensus over the past few months. And again, I think there's a lot, there's a lot that's real there. And there are a lot of deep tech categories that I'm super excited about that we're both investing in and I'm sure we'll talk about. But I also think there's still, I, I think it just fundamentally, there's no way that one or two or three companies will win every single important market and important category in the world.

• 13:44Speaker 02

  It's just, no, no company can have that span of influence and execute excellently so broadly. So for sure, there are some categories that I think, uh, open, I would bet on OpenAI and Anthropic being very well positioned. And coding obviously was the first, and there are others which we can talk through. And I think that, that these sort of like horizontal cross cutting areas are the ones that make the most sense for the labs to focus on and go after versus like, you know, so vertical software for pet stores or something like this.

• 14:15Speaker 00

  Though I do wonder, like, VCs are just, you know, pushing themselves into a corner of like, damned if you do, damned if you don't. If you're like, if you're only going to go invest in, you know, pet stores because like, that's small enough for the models. Like, I don't know if that actually ends up being a, a, a super interesting end category, right? But it turns out, no matter anywhere you look in AI world, there are existential questions. You can't avoid them. Like, there's no, there's no, just like, I mean, maybe if you want to go invest in like a data center builder and you're like, they will just build data centers and there will be demands like, but sure, then you still have to figure out like, you know, how, what makes you a better data center builder than the other 10 people that will compete with aggressive financing for the same site?

• 14:48Speaker 02

  There's no question that we are at a, at a period of max uncertainty and volatility. And yeah, I, I mean, I totally agree with you on the, on the hard tech point. Like, it, it, there, again, I, I do believe there are a lot of really exciting opportunities and, and categories in hardware and deep tech where there's a lot of interesting innovation happening right now. But yeah, to your point, it turns out hard tech is also very, very hard. And like, the failure rates are much higher and there's a lot of unsolved problems and like that. There's going to be a lot of, I think, pain and lost money down that road in the years to come. And then, yeah, on, on the application side, I think, as you've said, like, I, I think there's, there's still tremendous value in that last mile.

• 15:27Speaker 02

  And like, the labs are also starting to try to tool up and stand up these deploy codes and, and lean in on the implementation side. But I think there will be so many pockets and opportunities for companies that are delivering applications to also thrive and continue to grow.

• 15:43Speaker 01

  I think also it's interesting to like, ultimately, why do startups ever win? Right? Like, why didn't, why didn't Google do everything? Right? It's hard for a massive company to do many different things well fundamentally. Like, you know, a counter argument to this is actually like OpenAI shuttering or suspending their video efforts. Right? Like, that was very surprising to me, I will say, given their access to effectively infinite capital, given the, and effectively, you know, infinite talent. And they had a great team there that was leading that like, but they, they had to make the hard choices here. Now, some, a lot of that is likely driven by compute constraints fundamentally. Video training is very expensive relative to text training and so on.

• 16:26Speaker 01

  But it goes to show that they can't do everything.

• 16:31Speaker 02

  They needed to focus and, and strip down and just focus on acquiring TBPN and adding that to the company.

• 16:37Speaker 00

  I, I heard actually was that they, they listened to our last episode and read your article and they were like, oh, Rob's predicting Sam's out. Like, this is, this is really, you know, that, that spicy prediction has gotten to our hearts. We really need to make sure we, we avoid that. How, how are you feeling about that one six months into the year? I guess you got six months still to be right.

• 16:54Speaker 02

  It's looking a lot more likely today on June, you know, mid-June than it was when I shared it with you guys. It's been interesting. Yeah, I mean, when I, yeah, when I, when I unveiled that prediction in December, I feel like everyone was like, what are you talking about? That makes no sense, including you guys. You're like, oh, that's interesting. That doesn't seem very likely. And yeah, I mean, I like, you know, needless to say, like, the vibes have shifted against OpenAI this year. And there's been this realization, whereas this time last year was like, oh, OpenAI is doing everything. They're such an amazing company. They're going to win in chips and data centers and robots and everything. Now, I think now there's a realization like, oh, we really do need to focus. And I think there are a lot of elements of Sam's leadership that are increasingly under question. I think that Elon Musk trial, even though

• 17:37Speaker 02

  Elon Musk lost, I think that like was damaging for Sam's reputation and his trustworthiness and so on and so forth. So, you know, who knows, I feel like it was it was a provocative and like purposely somewhat low likelihood prediction at the beginning of the year. I would say odds have gone up for sure. One thing that one thing that's changed is at the time that I made and I published that in Forbes, like my hypothesis was that Fiji was the obvious most likely successor and kind of was being groomed to be the next CEO. And she obviously has now had to take a step back given some health concerns. And so I think that the question around who the successor would be is different. Like I am actually like, I think

• 18:20Speaker 02

  this theory, which I'm heard, I'm sure you guys have heard made the round make the rounds around Bret Taylor, I think is quite plausible. You know, for folks who aren't familiar, Brett is the chairman of the board at OpenAI, the CEO of Sierra, you know, one of the most respected and revered leaders in Silicon Valley. And I like, I think it would just honestly make so much sense for OpenAI to acquire Sierra and to make Brett the CEO. I think it I think it would be in the best interest of OpenAI's shareholders, honestly. And like to this discussion around the like the vibes have really shifted against OpenAI and towards Anthropic in the lead up to the IPO. Like, that is a decisive change that I think could be like a total game changer for OpenAI,

• 19:01Speaker 02

  because like, people just trust Brett and respect him and admire him and think he is like an admirable leader. And I think if someone like that was at the helm of OpenAI, I think it would do a lot to change their fortunes. So, anyway, we'll see. There are many months to go, but like, I don't know, I feel like that one might actually come to fruition. We'll see.

• 19:20Speaker 01

  I think also, like, this notion of OpenAI going to like an alphabet like structure seems a lot more plausible in general now that like, they shift to a holding company, maybe Sam stays CEO of the holding company, and then you have somebody take over, you know, OpenAI or ChatGPT, maybe ChatGPT becomes its own product, you know, or something to that effect. That would make sense as well.

• 19:44Speaker 00

  Yeah. No, I mean, you mentioned it, but obviously I felt these past months, the dominant vibe, it's unbelievable to see Anthropic on this unprecedented vibe run, right? I think they're, you know, they've, they've kind of had just completely sucked up the oxygen, probably the most consensus around one company we've, we've had. And obviously that's come at the expense of, of OpenAI in many ways. You're maybe starting to see some, some light cracks in that in this week with the, with some of the reactions, the release of Fable, but I'm curious for both of you, like, do you think this is a vibes trend that continues? Or like many things in our culture, are we inevitably going to have some sort of backlash or flip in fortune of these two companies in the next three, six months?

• 20:25Speaker 01

  There'll definitely be some amount of backlash. There's always backlash to whoever's winning, right? That's, that's just like a truism of life, I would say. That said, I think, you know, Anthropic is also potentially starting to make moves that will alienate people. Like I think it's clearly with limiting the use of Fable for anything to do with AI development. And I think in particular, I don't think people are incredibly upset at the core with just the limitation. It's the fact that it's a silent limitation.

• 20:54Speaker 00

  Yeah.

• 20:54Speaker 01

  That I think people are really upset about, right? That it doesn't give you a refusal. It doesn't say, I'm not going to help you with this. It just does a poor job on that without you knowing. They can say that's because of safety. That's tenuous, I would say. It's, it seems pretty clear that's a competitive positioning, you know, move rather than a safety move. You know, and I think to that end, like, like I think I mentioned earlier, right? That like a lot of people were able to find many of the same vulnerabilities in zero days that Mythos found with open models and good harnesses. So I also think there's not necessarily a unique safety risk, you know, to this.

• 21:34Speaker 01

  But, but I've seen more people who have been incredibly bullish on Anthropic and positive about Anthropic, truly pissed off as a result of just a Fable the last day than I'd ever seen. And it does seem like a bit of a step change. So I think if they continue to make moves like that, we will definitely see more and more of a shift, I think, against Anthropic.

• 21:58Speaker 00

  I mean, obviously so many open source models have been trained on just like distilling, you know, Anthropic models. Like, do you think this will have an impact on the, on like the open source community?

• 22:06Speaker 01

  Depends if they litigate it, right? Like, it's not going to do anything in China to limit anybody there. You know, and then for US models, like, I think people tend to be very careful. Like, you know, we did, Datology did all the data curation for RC's trendy large model, and we were very cognizant to not use any closed source APIs at any point in any of the development there. Also, only public data. So, like, you can build a really powerful model without that. I think also, looking at the MAI model that they just released, right? They made a huge point of going so far as not to use any synthetic data at all to avoid, you know, any, any ability to kind of sneakily be distilling from another model. And then I think they've said that now they're going to start using synthetic data from those models to kind of bootstrap them.

• 22:51Speaker 01

  But I don't know if it's going to fundamentally change it. Like, so long as people can actually use an API to get to your models, you can't actually stop them from trying to do some amount of distillation. That all said, I think this, this claim is, is like, overblown to some extent. Like, it's true, but it also, like, you, you can build still great models. You don't need to distill to build a great model. I think that the notion that, like, oh, the only way that any open model can catch up with a closed model is by effectively, you know, distilling or stealing from it, reads a little bit like copium to me.

• 23:24Speaker 00

  I feel like when these models come out, everyone's, you know, on, on Twitter trying to figure out what, what happened. And you obviously have a lot of people on the, on the anthropic side saying, hey, this is the biggest step change in capabilities, you know, I can't remember in a while. I'm not sure, like, you know, to the extent you've, you've played around with it, but like, what, what are your early reads on, like, how much of a, of a step change in capabilities this really is?

• 23:43Speaker 01

  I only played with it a little bit last night since Fable released. I didn't personally see massive differences from where 4.8 was. But, and, and talking to people, it's, it seems like the, the takes have been quite varied around that. It's, it's pretty hard to, to tell, honestly.

• 24:02Speaker 02

  My take on Fable is, I kind of feel like it, it is and also is not that big of a deal. Like, I think it's, it's the latest state-of-the-art model that was released. It happened to be released yesterday, like, a day before we're recording this. But like, if we recorded this three weeks from now, or seven weeks from now, there would probably be another model from another provider that we would be talking about. I do think, like, just based, again, based on the benchmarks and the quantitative data, which is far from perfect. But it does, like, it does seem like it is a, it is a meaningful Step change improvement relative to the previous state of the art, which I think is meaningful. Again, not because it's some like discontinuity. Like I think this kind

• 24:42Speaker 02

  of gradual improvement will continue, you know, going forward. But I think it's meaningful in the sense that like, I think it really does undermine this narrative that people have already shifted away from it. They had a lot of currency a year or so ago. They're like, pre-training is really hitting a wall. Things are plateauing. Like we, now it just, we, it has to be, you know, RL and test on compute to carry us forward. Because, you know, we've hit this data wall. Like I just think that's clearly not true. And like, the gains are continuing to come in very richly. And like, I don't think there's any good reason to think that they will plateau anytime soon.

• 25:19Speaker 01

  Yeah, I think we all saw a lot of pushback to that narrative over the last six to nine months. And like, it's, it's pretty clear that pre-training did not hit a massive wall. We were just waiting for new chips. I think some of that, I think also just like naive scaling doesn't work. Like, yeah, if you just take exactly the same stuff that we were doing, and you just multiply it by a factor of 10 or 100, which is kind of like what, you know, four or five was aiming at, or, you know, llama three, four or five B at the time. Like, that didn't work extremely well. But I think people took a much overgeneralized take from that. Especially because like, one of the things that's actually really challenging about deep learning, I have found, is that you really need to get all the details right often for something to really work. If you

• 26:04Speaker 01

  have kind of like 95% of it right, it kind of rectifies to like, just not working a lot of the time. There are a couple of methods that are robust, but like, you can like be doing almost everything right and not get like no real improvement. And then you tweak the last knob, and now all of a sudden, you get a step change. Like, you know, that happens a lot in deep, in deep learning. And one of the things that's challenging about that is it just makes it fundamentally difficult to interpret a negative result. Okay, you tried scaling everything up and it didn't work. Is that because scaling doesn't work? Or is that because you just did one thing wrong? And that often happens. It also makes it very hard to decide when to abandon a project because it's easy to also then always be like, well, like, maybe if I just make one more tweak,

• 26:46Speaker 01

  this thing will work. When, you know, oftentimes, that's not the case.

• 26:50Speaker 00

  I mean, I guess you need to hit the other players in the space. I mean, I feel like, Rob, in our December episode, you said, like, you know, which I think is a feeling all of us have, that Google is like incredibly well positioned, right? I mean, obviously, they've got talent, they've got compute. It doesn't feel like things have gone super well for Google in the last six months. You know, feel free to push back, but what's what's going on over there? And like, why haven't they been able to catch up on coding, I guess, most notably?

• 27:14Speaker 02

  Yeah, I think I think I would disagree a bit that like, Google has fallen behind or isn't executing as well. Like, again, I think like, in part goes to this reality that I think the three labs are all kind of in this process of leapfrogging one another continually. And any given month, any one of them may have the like, quote unquote, state of the art model.

• 27:36Speaker 00

  Been a while for Google, though.

• 27:37Speaker 02

  I think there's no question that they're behind on coding. And I, but I think that's just that reflects prioritization. Like, it's clear that Anthropic leaned in on that as their north star for years, and that proved to be obviously incredibly savvy move that has really catapulted them. And, you know, OpenAI more recently has really doubled, tripled, quadrupled down on it. And you're seeing a lot of positive love for Codex. And I think it just hasn't been as much of a priority for Google. But I feel like everything we talked about previously, I still stand by and feel very confident in the sense that like, Google has an incredibly deep bench of talent. Unlike OpenAI and Anthropic, Google has this massive cash machine. And then lastly, it's computes. Like, I think Google has benefited so much from being totally full stack. They

• 28:20Speaker 02

  designed their own chips. They have their own cloud. You know, no one has access to infinite compute. And Google is also, you know, would love to have more, but they have a massive.

• 28:28Speaker 00

  Would they love to have more? I don't know. They sold some, they sold some of it off to Anthropic.

• 28:32Speaker 02

  Yeah, I mean, yeah, they have a, they have a cloud, an external cloud business. But I think they're better positioned when it comes to compute than either Anthropic or OpenAI. So yeah, I continue to be very bullish on Google.

• 28:43Speaker 00

  What do you think, Ari?

• 28:44Speaker 01

  I tend to share the same view. I am a little surprised that they haven't improved as much since 3.1. I would have expected a bigger launch at I/O. And I wonder if, you know, in a couple of months, we'll find out there was one planned and then something went wrong and it was scuttled or whatever the case may be there beyond just the flash model. But I do think they have all the structural advantages. I think having access to the money printing machine. Like, I think that's a really interesting way to kind of think about the XAI SpaceX merger. Is like, it's a way for XAI to get attached to a money printer to the extent that SpaceX is a money printer, which less so certainly than Google or Meta is. So I think they are well positioned there. I think also, like, I don't know if if

• 29:27Speaker 01

  Google's market is the same as what Anthropic's going after, like with respect to the focus, right? Like fundamentally, models are going to be commoditized for consumers. Like, I think that's just like quite clear that for the consumer use case of I'm just asking my model a question about, you know, the world or or having it be a tutor or any of kind of the standard things that most consumers are going to use. That's going to be commoditized and people are going to use the model on their phone. Like, I think that just seems very clear.

• 29:54Speaker 00

  What about like some of the computer use stuff or like it feels like there's still frontiers of like consumer, right? That that models can't do yet.

• 30:01Speaker 01

  I think there will be, certainly. And I think you will see more of that. But I think first off, like, most people will not use that level of of software, I think. Like, the power users will, but like, you know, most people are just going to be using it as an answer engine. And Google's now quite well optimized to be the kind of default provider both on Android phones and iOS phones. Until, you know, eventually, Apple builds its own models there, which have been pushing a lot there. So I think Google's actually going to be in a strong position to that, even if they don't necessarily have the best model. I think that's another big part here. It's like, I don't know if the best model necessarily wins in the consumer space.

• 30:38Speaker 00

  Well, what about even in the in the coding space, right? Like, I mean, this is this has honestly been the most surprising to me. It's like, I actually think Codex is clearly an amazing product. It doesn't seem to be making a huge dent. I mean, obviously, it's growing, but Claude Code remains the dominant, you know, coding tool. And I wonder, it's interesting to see kind of like this first mover, like, hey, you introduce people to the paradigm advantage in in like, you know, I always thought of developer tools as like the most meritocratic, like, you know, would switch in a in a second, like, best model always wins. I don't know what you guys make of, you know, of that.

• 31:11Speaker 01

  I think that's generally still true. I do think that developers are the most mercurial consumers in some ways, and they are going to switch to whatever the best thing is. That said, like, there's a question of how much better is it? And I think Claude and Codex have stayed close enough, and there hasn't been a massively compelling reason why you need to switch to Codex, such that a lot of people have stuck with Claude. I expect that at least amongst AI developers, you're going to see a massive set of shifts now to Codex from Claude, given the way that they're limiting Fable and whatnot. I think that honestly, this is a pretty nice gift for OpenAI, with respect to a lot of the

• 31:52Speaker 01

  people who are the loudest voices on Twitter and whatnot, becoming, will likely be spending a lot more time with Codex in the immediate future, we would guess.

• 32:00Speaker 00

  But I mean, it's open. So when 5.6 comes out and OpenAI is presumably to counter position is going to have to give people more access. Like, are they just ultimately going to eat way worse margins and like, for vibes and like, for usage? Or like, what structurally is going to allow them to provide better access for, as a model that's on parity?

• 32:20Speaker 01

  That's a good question. I mean, I think ultimately, it's like, in any market where you have kind of a couple of players, like, so long as the models are close enough, that can make a big difference. It could also be compute access, right? Anthropic just got a lot of access here. But I think that's another one of the things circling back to earlier conversation around how like, there might be fewer open models going forward. I think part of that could also actually be that there could be fewer closed models access going forward. Like, it is not hard to imagine a world in which Anthropic is so compute constrained, that they actually cut off the API. Because obviously, they're going to prefer Claude code to the API with respect to how much money they make. And you start to see this now with like, OpenAI starting to sell like, futures of like, hey,

• 33:02Speaker 01

  you get guaranteed access to inference tokens going forward. Like, that's actually a huge existential threat to anybody that builds on top of these models. And I think that was not really a plausible thing six months ago, but now feels very plausible that actually the APIs go away, not as a business decision, but just purely because of compute constraints.

• 33:20Speaker 00

  Yeah. And what do you think of that, Rob?

• 33:21Speaker 02

  Yeah, I think it's totally feasible. I think, yeah, I think a few different things could happen. I think, you know, cutting off their API access altogether would be an extreme move. But you can imagine, for instance, OpenAI or Anthropic

• 33:36Speaker 02

  Rather than, you know, not just not open sourcing the models, but only making them available via API. You could imagine them actually not even making their most powerful models available to anyone publicly and reserving them for internal use. And, uh, and yeah, you could, um, you know, I so much of this depends, I think, on the compute bottleneck and how long it stays this acute, um, which I think is, is interesting to reflect on. And this, you know, this again, this gets to some of these like deeper tech topics, which I think are becoming increasingly relevant. Like there are, uh, on the horizon, there are efforts, uh, and startups that are trying to like,

• 34:17Speaker 02

  break the supply chain wide open, build cutting edge fabs in the US, challenge ASML, challenge TSMC, et cetera. There's Elon Musk obviously has this TerraFab concept, which is fascinating. And honestly, I feel like people should be talking about more, uh, just given how transformative it would be if, if he pulls it off. So I think a lot of those variables will, will influence like how long is compute the, the limiting factor? And that in turn, I think like if it remains this compute constraint, I think some of these possibilities that Ari is sketching out, like you could totally imagine them being real.

• 34:50Speaker 01

  It's hard to imagine though, how we actually unblock, like if we continue even remotely on this trajectory, it's, it's hard to imagine how we relieve the compute constraint, uh, within the, within the next handful of years. I think.

• 35:02Speaker 02

  Yeah, it's, yeah, I think it's not a, it's not a two to three year thing where like TSMC is not displaced, but like augmented by many other players that can provide comparable chips. But like, you could imagine over a file, like it doesn't have to be the case that there's only one company in the world that can make cutting edge. Like it's actually kind of crazy that that's the current market structure, that there's like one company that knows how to do this and no one else can do it. And that one company that can do it, the most important machine that goes into the process is made by one other company. And no one else can do it. Like it doesn't have to be that way. And I don't think it will be that way.

• 35:36Speaker 01

  What a wild version of, uh, of the many worlds that we could live in, that we live in now. That is, uh, it is, it is pretty nuts. Are there clear upstarts going, going after ASML? There are a lot going after TSMC, but I haven't seen as many going against ASML.

• 35:49Speaker 02

  There are, yeah. It's an interesting, it's an interesting new area of research. Like, in a nutshell, ASML, their focus is obviously extreme ultraviolet lithography, EUV. And EUV is like, starting to hit physical limits in terms of how small of transistors it can print on the chips. And so there are a couple of really interesting new research directions. One is, rather than using light, like moving away from using lights altogether, and instead using matter, it's called atom lithography, where like you use, you use a beam of atoms to print these features on the, on the chips, which lets you get like way lower resolution. And so there are a couple of startups doing really interesting work in atom lithography. And then there's also a handful of startups that are basically looking to like,

• 36:34Speaker 02

  leapfrog EUV and move even further out on the electromagnetic spectrum to the thing that's like, has even shorter wavelengths, which is X-rays. So like, this whole concept of X-ray lithography is, is getting a lot of momentum. There's a couple of startups in each of these buckets that have raised like, a ton of money and are running at this. And, and to state the obvious, like, they're still, they're still very much in development mode. And like, it remains to be seen whether either of these will prove to be commercially viable. But if they work, like, I think, honestly, I think especially atom lithography, there are so many advantages in terms of, like, the machine can be way simpler, way fewer parts, way cheaper, way smaller, obviously much better resolution. So, anyway, yeah, I do think there may be like, real technology disruption coming here.

• 37:17Speaker 01

  It looks like that's at least five years away, probably, given where it is right currently.

• 37:22Speaker 00

  I mean, I think this thread is fascinating to pull on, like, what are the actual implications of a year or two from now? Like, we're, like, we're even more compute constrained than we are today. And obviously, you know, it feels like there's lots of, you know, people, what do they call it? Like, using a Ferrari to go down the street to the grocery store. Like, there's probably like, over-usage of, of powerful models for, for what people need today. But even if we figure all that stuff out and route people perfectly to their, you know, Datology-built models, or however the world ends up working, like, it feels like we'll still be in this kind of, you know, overall compute shortage. And, you know, it's fascinating, I mean, one implication of that is, is obviously, the labs themselves, you know, prioritizing first-party products or prioritizing their own development.

• 37:59Speaker 00

  Any other implications? I don't know, I'm just thinking on the spot here, but I'm curious if there's other implications that come to mind for either of you of like what that might mean for even businesses in competitive industries that get access or don't get access.

• 38:10Speaker 01

  It pushes towards efficiency, right? Like generally, especially the frontier labs have not cared too much about efficiency because of said infinite capital access. And like you will get to physical constraints where you have to figure out how to be more efficient. So I think it'll drive a lot more interesting innovation, frankly. One of the directions that I've just always been very bullish on, and I think we've seen consistently is that like you do not need trillion plus token parameters in order to achieve the capabilities that we currently see in the abstract. Like at the moment, like you needed to get that frontier level, but like we consistently see that smaller and smaller models can match the largest models of even one to two years ago, right? So I think that will only accelerate

• 38:52Speaker 01

  as a result of that. It will be more and more pushed towards how do you make models as small as possible. You'll probably see a lot more investment in areas that can help like that, like distillation and things like that to try towards get towards really reducing the inference costs. I think if you can do that, that can alleviate a fair amount of this. I would still expect that the usage is going to grow faster than what you can do to alleviate this.

• 39:18Speaker 02

  Yeah, I definitely agree with Ari on the efficiency point. I think another interesting implication of this like massive supply constrained world is it will be, it will be a very good thing for other chip providers other than Nvidia. I think basically pretty much everyone would probably prefer to use Nvidia GPUs than anything else, you know, maybe other than people at Google. But there just aren't enough GPUs to go around. And so you're already seeing like companies are doing what they have to do to adapt to use AMD GPUs and to use Amazon Trainium and Cerebras obviously is seeing massive tailwinds because of this. I think basically like any chips anyone could get their hands on will be in massive demand. And so I think it's not a bad thing for Nvidia, but I think it's a good thing

• 40:03Speaker 02

  for other chip.

• 40:04Speaker 00

  Actually, do you think that's one of the big themes of the last, you know, six, nine months is like the rise of these, you know, as inference is dominated and you can kind of split these inference workloads and the pre-fill and decode and other things like you really can use, you know, heterogeneous chips. The question I have is like, do these other chips really like help us on the compute shortage, given like this exact thing we keep talking about, like there's you keep shifting bottlenecks. And so it's like, there is a bottleneck upstream of the chips, which are like the components that go into the chips or the production of the chips at TSMC, you know, ASML, all this stuff. Like, does having more chips actually solve that fundamental bottleneck, energy, all these things? Or is it just like, similar to every space we look at, all these vendors are very happy to have, you know, more players than just Nvidia?

• 40:48Speaker 00

  I don't know if, if the rise of all these other chips actually gives us like more compute. I'd be curious for your, for your thoughts on that.

• 40:53Speaker 01

  I think that intuition makes sense because right, like if you imagine a world in which there's, there's no, there are no, you know, Zenith Rebus, there, there is no D-matrix, there's no one else who's kind of making competitive chips. Presumably Nvidia just eats up the TSMC capacity in that world, right? And the total number of chips stays constant in the world. So likely it just accrues value to not Nvidia as a result of that, rather than actually changing the situation fundamentally.

• 41:19Speaker 02

  Yeah, yeah, I think, I think the alternative chip providers aren't a solution to the compute constraints, but will be a beneficiary of the compute constraints in the sense, in the sense that Nvidia doesn't get all of TSMC's, like, production capabilities and, and all of, you know, all of the supply chain's production. And so like, you know, if, in a world where there is a surplus of chips, everyone would rather use Nvidia, people can buy Nvidia, there just isn't that much demand for these other chips. But in a world where you just can't get your hands on enough Nvidia chips, then like, people are going to pay a lot for AMD chips and pay a lot for Amazon chips. And it won't, to your point, like the, the overall, the overall supply of chips won't.

• 42:01Speaker 02

  Be changed as a result of them, but I do think they'll see massive 201s.

• 42:05Speaker 01

  Right. TSMC doesn't want a monopsony.

• 42:07Speaker 00

  Totally. But it's fascinating parallel to the NeoCloud world too, right? Where you basically have the same thing, like it's not like you're not necessarily increasing the number of overall chips, but like they've certainly been shuffled among a bunch of different players. And as long as the capacity constraint is there, you know, every one of those is a very solid business. Do we actually, given what's happening, expect, you know, there not to be a compute constraint anytime soon? Are we really talking about like 2035 or like, you know, hey, in 15 years, maybe someone will figure out what happens to these businesses?

• 42:38Speaker 01

  It's probably not in the next couple of years. Yes. Like I would be surprised if anything happens before 2030. That said, like if, when, when all of the different methods that people are working on to try to relieve these bottleneck all start hitting at the same time, which is probably will happen, right? We'll have like a several years where it'll be like several big unblocks, probably in like the early 2030s would be my guess. That's probably when the merry go round.

• 43:02Speaker 01

  I think the other question is like, as more chips are produced, like to what extent can we start to continue to get a lot of value out of older chips? And that can also, I think one huge thing that, you know, has been like the, or like the, the leading indicator of this is, is, has been that H100 prices reversed their drops. And that happened, I think, right around when we recorded the last episode was when that started to happen in December. But like H100 prices have gone up dramatically over the last number of months. So I think there's also a lot of, there are a lot of chips there that like, that can be used. But I do wonder what happens when the merry go round stops on that. And they ultimately are all selling the exact same product with, you know, minimal differentiation.

• 43:43Speaker 00

  Well, speaking of what the NeoCloud space, I mean, obviously I think the biggest headline in that world is SpaceX, right? And I'm like, just, you know, coming into the space in a huge way. What do you guys like make of, what is the future of XAI? Do you think, I mean, obviously do they just lean into, to continue to do this at scale? Do you think the model business has any legs? I guess not, not a bad time to bring up the, the cursor potential acquisition and how that fits in. But would love to hear, you know, one of you riff on that.

• 44:10Speaker 02

  Yeah, it's a good, it's a good question. I.

• 44:14Speaker 02

  I would say I'm not super optimistic for XAI's future trajectory in terms of them being a frontier lab. And I do like, I think the, you know, these massive deals that SpaceX signed to rent compute to Anthropic into Google. I think on the one hand, yes, it's like padding the numbers, padding the revenue numbers ahead of the IPO. And that like, that obviously plays a role, but I also do, it's hard not to interpret it as a signal that like, the company's foremost priority is not doing frontier AI research and fueling XAI's research. Because if it was, it just took to this, the last 20 minutes discussion around how, how much the world is compute constrained. Like you just wouldn't be giving away.

• 44:59Speaker 00

  Pretty hard business to IPO if all those chips were being used for, for training, right? Without, without like much revenue to show for it on the, on the product side.

• 45:07Speaker 01

  Elon companies aren't valued on fundamentals. Like, I don't know, it's totally different with an Elon company. So who knows?

• 45:14Speaker 02

  And kind of to Ari's point, I think the like, the thing that Elon Musk is amazing at, and his companies are amazing at is like, incredibly operationally intense, real world, like atoms, not just bits, undertakings. Obviously we saw that with, we've seen that with Tesla, we've seen that with SpaceX. And so it's not surprising to me that like, in the world of AI, the one wedge where this SpaceX XAI like behemoth is going to have a real durable advantage, I think will be on the data center side. I think they will excel like, and they already have Excel that's standing up massive, massive clusters, super fast, getting them up and running. And I think that will be a great business for them and it may turn them into like the world's biggest cloud. Especially as, you know, in the years ahead, the company starts putting more and more compute into orbit.

• 46:00Speaker 02

  But I don't, I don't know, I guess I just, I don't, it doesn't seem to me like organizationally, being at the frontier of, of the AI model race is a priority or is necessarily realistic. Obviously we've seen just like the insane attrition from XAI over the past year or so. And like, I think it makes sense for the, the overall, you know, Elon, Elon Co. to have a model arm that's doing model stuff. But do I like, I guess I'm not, if I had to bet, I wouldn't be super bullish that like they will crack back into the, you know, the top echelon alongside Google, OpenAI, Anthropic.

• 46:32Speaker 00

  So what are they doing? Like, why, why, why, why cursor?

• 46:36Speaker 01

  I think why cursor is to get all the traces. Like, I think that would be the, and to have a hedge against the fact that they have struggled to produce a very competitive coding model fundamentally.

• 46:47Speaker 00

  So are traces for coding worth $60 billion?

• 46:50Speaker 01

  I'm not sure if they're worth $60 billion, frankly. But if you, if you, if you make the, if you think that that's what can leapfrog you and accelerate you to having a strong coding model, I can see where that comes from. 60 billion is probably quite high relative to that. But I don't think Elon has given up the ambition, probably, of trying to, of wanting to have the best models. I do think that when you look at the practical aspects of it, it's hard to, if your pure goal is to build the best models, then you would not be giving away massive amounts of compute, not giving away, selling massive amounts of compute. So I think that it does go to Rob's point that it's not their top priority. And, you know, cursors, even especially given the way that that deal was structured, like, it is an option.

• 47:34Speaker 01

  It's kind of like, hey, we want to maintain some amount of optionality for the next number of years.

• 47:39Speaker 00

  Yeah, till we, till we see if traces actually are the most important thing to, to make a really good coding model.

• 47:44Speaker 02

  And on the cursor point, I'm sure you guys all saw and laughed at the, like, the SpaceX TAM charts in the S1, where like, I think they estimated like the total space.

• 47:54Speaker 00

  I never laugh at a big TAM chart. You gotta love, you gotta love the ambition.

• 47:57Speaker 02

  You just salivate.

• 48:00Speaker 00

  28 trillion sounds good to me.

• 48:02Speaker 02

  Oh yeah, all of space was like, I don't know, 500 or 600 billion. And then comms, basically, Starlink was like 1 or 2 billion. And then, yeah, enterprise AI was like 20 trillion or something.

• 48:12Speaker 00

  I think it's always good when like all of space is, you know, not close to, you know, like, it's like a few percentage points of your, of your TAM.

• 48:20Speaker 02

  Yeah, yeah, exactly. So anyway, I think that points to why cursor, like, they certainly have, like, the narrative, again, a lot of this goes into IPO narrative is around like, we're going to win enterprise AI and an asset like cursor because before. Adding cursor, they really had no like application or product surface area. Whether or not they'll actually succeed in that undertaking, I think is a lot less clear. But I think that's the, I think so much of it is just that like the positioning and narrative.

• 48:49Speaker 00

  We've made it through a good chunk of the episode here. We haven't mentioned anything Andre Karpathy related, which I think is, you know, I think we've done it pretty much every episode. He's like, he seems to be the unifying theme, you know, first he says that everything's sloped. Then he decides to go join one of the labs. You know, obviously he went to join this, this self improvement team and it feels like there's, you know, I think both labs have been very vocal, hey, we're going to have, you know, AI R and D by 2028. It feels like, you know, there's all these Twitter vague posts about, hey, we're getting really close. How close do you guys think we actually are? And like, you know, is Andre deciding it's time to go back in? Like, how much of a signal is that?

• 49:26Speaker 01

  I think we're closer, my estimate is that we're much closer now than we were six months ago. I would say, so I do think this is something that has changed and that I've become one of the places where my mind has changed a little bit, where, you know, I've become more bullish on this direction. I do still think that the bottlenecks are in compute fundamentally. I think we talked about this in maybe the last episode, that like ideas are not necessarily the challenge. Even execution is not necessarily the challenge. It's, you have to actually then go and run the experiments. And that takes compute. So I do think there's going to be some bottleneck on the pace of improvement. But we are, we are clearly getting to the point where models can improve themselves.

• 50:11Speaker 01

  You know, we've, we've started to do a number of experiments here around, you know, just having agents do the curation itself in various ways with various amounts of guidance and like seeing far more promising results out of that than I would have expected. So there is like a lot in this direction. I think there is a lot of reason to be very excited about it. I do think though, that lots of people will be able to do the same thing. So I think like the flip side of this is there's this like view of RSI where it's like, okay, now like there's one player that's going to go super duper fast.

• 50:39Speaker 00

  Run away.

• 50:40Speaker 01

  And run away and then nobody can compete with it, right? I'm still very skeptical of that view because I think there are just like fundamental compute bottlenecks that can prevent the speed. And also like. There are 10 companies at least at this point that have the funding, the talent, and the know-how to work on this. It's not something that I think is going to be fundamentally limited to a small group of people.

• 51:06Speaker 02

  I was expecting you to be more skeptical, Ari, on the whole notion of recursive self-improvement, but it's interesting that you are a believer in it, and yet you don't buy into this kind of like takeoff narrative. Like, what would that look like for a lab to like crack through RSI and yet not have this sort of exponential takeoff? Is it just the compute as a limiter, basically?

• 51:29Speaker 01

  I think that's where a lot of it, like compute is a fundamental limiting factor with respect to this. And also, like, getting, having just like more humans that only makes you go faster to a certain point. And I think there is a question of like, how much better than humans is it going to be? It's clearly coming to the point where it can be comparable to a, you know, junior AI researcher. Having just an army of junior AI researchers get you so far, so will it continue past that? And then I think there will be some speed limitation. But I will say, like, I was a lot more skeptical of it, you know, six months ago. And I think, you know, we have seen some clear progress here. I think also you look at some of the actions that some of the labs are taking. Anthropic is slowing down

• 52:11Speaker 01

  on hiring a lot of junior folks. That starts to drive, you know, more confidence that this is possible. But I think it's going to be a lot slower than people say.

• 52:21Speaker 00

  Well, I always like to end our sessions with like a rapid quickfire, where we get your, you know, to drum the last bit of spice, which provides fodder then for the next episode. So I figure, you know, maybe to start, here's for both of you, like, what do you disagree with most that's kind of a common trope in the broader discourse right now?

• 52:40Speaker 02

  I'm having to start. I think this is like a bigger picture observation, but relates to a lot of the discussion we've been having around like the chip shortage, the CapEx build out, the, you know, the gigawatt scale data centers. To me, it seems so clear that, like, In, in some, in, you know, say five to 10 years, we're going to look back on the current era of AI. And it's going to, it's going to be laughable how resource inefficient today's AI systems are. Like the fact that we need to build two gigawatt scale data centers. That's like twice as much power as all of the city of San Francisco to run these state of the art models.

• 53:22Speaker 02

  And, you know, I get to like, return to one of my favorite hobby horses, like compare that to the human brain as an existence proof. And the fact that human intelligence, which is what we're trying to achieve at the end of the day with AI, runs on 20 watts of power. Um, I just, I think that like, and I don't think there's going to be one silver bullet breakthrough, but I think there will be massive advances at the hardware level. You know, things like Naveen Rao's company that he's working on, uh, analog computing, you know, efforts like that that lead to fundamental breakthroughs and the energy efficiency of chips. I think for sure there'll be a lot of, uh, uh, algorithmic breakthroughs and optimization breakthroughs. But I think like, and I, I think it has complex, uh, and not obvious intersections with this question of like the demand shortage.

• 54:06Speaker 02

  How long does that last? The capex build out?

• 54:08Speaker 00

  Yeah, it's interesting. Obviously, like, I mean, two things, you know, that come out of that. One is, you know, if, if things do go that way, then the, the moats on the leading model providers are far less compelling. Right? I mean, obviously a huge part of the moats today is just access to capital and, and, and scale. Um, and so, you know, if you've had all these like folks spin out of Neo Labs and like, I think a lot of them, you have similar inspiration of, of what to go chase. Um, and I've never been convinced that even if they do figure that out, it ends up being like a great business. Um, because to our, you know, point we've always talked about is like, these ideas do diffuse and other people, it's very unlikely that for a long period of time, you'll have an idea that looks fundamentally different than these other folks. And so, uh, it would be really interesting to, you know, if, if the world does do that way. And then, you know, I think the, the second point in Ari, you always hit on this. I think in our first episode, you talked about this in the context of the Chinese open source models.

• 54:52Speaker 00

  It's like, you know, uh, constraints breed that innovation. And I wonder, you know, you like the incentive that if you're at OpenAI or Anthropic right now is just like, keep pushing the current, I mean, the current paradigm and getting better at it is so, so valuable. And so you've had folks like Jerry Torek spin out of, of OpenAI and be like, well, I want to focus on something that feels more akin to, to what you're talking about, Rob. And it'll be fascinating to see whether, uh, these advances, the next big advance around this happens in, in, in one of the big labs or, uh, or, you know, maybe in an ancillary place. I don't know if you have a, a prediction on that, Ari.

• 55:23Speaker 01

  I think you'll see it come from many places simultaneously. There's this notion of multiple independent discoveries where like, you know, how many times are things discovered independently? And there's a really interesting Wikipedia page about this where you can go and look. There were a lot, like basically every major scientific discovery was discovered by several people simultaneously. And then as communication bandwidth increased and latency decreased, where now you know what's happened across the world the next day, that got a lot faster and now you see that a lot less often. But I think like you clearly see that ideas are just ready. Like OpenAI didn't invent the idea of test time compute. Many people were working on test time compute. You know, O1 just came out first. I think for, you know, anything we're seeing with recursive self improvement or any other

• 56:07Speaker 01

  aspects, you're going to see the same sort of thing. The space of ideas isn't that massive and like ideas do become ready. So, you know, whatever ends up working, I would bet that several other people will be working on the same thing simultaneously. So I think it'll be a little bit of both. And what do you disagree most with in the broader discourse right now? Probably the thing I disagree with the most, although this could be code potentially, is the notion of the permanent underclass and just like this idea that, you know, all AI is going to take all human jobs. The AIs listening to this in a decade are really going to laugh hard. Yeah, I'm doing poorly at Roko's Basilisk here, but you know, and it's like kind of interesting, right? That Anthropic is even like going out and like saying this very, very visibly and so

• 56:51Speaker 01

  on. My, the reason I tend to think that this is overblown is that fundamentally just humans are slow at dissipating things through the economy. Like it's going to take a long time for, you know, even the tools we're seeing to really fully percolate. And, you know, we're seeing that now such that, and so much of business and everything is actually about human to human interaction and trust and so on. Those are like barriers that I think technocrats tend not to consider. You know, and I think a lot of the folks that are projecting kind of the really fast timelines

• 57:31Speaker 01

  around this are underestimating how slow the world can be in various ways. So I tend to think that that's that's overblown. And that's probably the thing I disagree with most, but it's also possible that I just want humans to still matter for longer.

• 57:47Speaker 00

  Yeah, I think, Ari, you mentioned kind of, you know, what you'd change. I think actually both of you had mentioned changing your mind on on open source, you know, models and kind of, you know, the how many players there would be actually in that space and going after it. Anything else that like, I feel like, you know, we're constantly getting new information and having to shift things like anything else in the last few months, you feel like you've shifted your perspective on?

• 58:06Speaker 01

  I'm more bullish on RSI, I think, than I was. That's that's a change. But I think I think that and and then like, yeah, I'm quite, I think this notion that the open models are going are going to go away to some extent, I believe quite strongly now, and I think I didn't.

• 58:24Speaker 01

  Didn't really see that coming six months ago, in the same way that I didn't see the rise of the open source models coming. I think we were all very surprised by that past llama. I think now the opposite is the same, but nothing, nothing beyond this.

• 58:35Speaker 02

  One thing I've changed my mind on is is I feel like I've really pulled in my timelines in terms of the performance and improvement rate for robotic AI and robotic models. I would say six months ago, you know, whenever you chatted last, I probably would have been in the camp of like, this is inevitable and it's massive market opportunity, but who knows how long it's going to take. It could be 18 months or it could be five years to the sort of, you know, mythical like GPT three moment for general purpose robotics AI. And I feel like in just the past handful of months, like these models have really crossed a threshold in terms of how well they're working. There's obviously still a long way to go, but I think they've like.

• 59:19Speaker 02

  Robotic foundation models have reached a point now where they are capable enough to be commercially viable across a lot of different use cases. And like as they start to be deployed with customers and that data flywheel starts to spin, like I think it's just going to accelerate. So I do now think that like we aren't far from this like so-called GPT-3 moment in robotics.

• 59:39Speaker 00

  I mean, music to my ears, but certainly I think it's been really exciting to see a bunch of the progress in that space. And obviously, yeah, there's been a ton of activity around, you know, obviously in robotics. I mean, I think in bio, material sciences, I think we're definitely feeling a lot of progress in some of these other spaces. And so it'll be fascinating to see these next years. I guess my last question for you guys is just, you know, I'm shameless now going to ask for an additional spicy prediction for the second, the back half of the year. You know, of something you think by the end of the year, we'll all be realizing was actually true.

• 1:00:14Speaker 01

  You know, I'll take the API bet, actually. And this is a bit aggressive for it to happen this year versus next year. I feel a lot more confident saying that by the end of 27. But I think it's, I think there's a very reasonable chance that we see probably Anthropic, but it could be OpenAI, suspend API access for some period of time, or heavily, or otherwise heavily limit API access for like a brief period. And that maybe is like the precursor to starting to see this happen more frequently down the line.

• 1:00:45Speaker 02

  That is a spicy take. I like that one. Yeah. That, it seems so, it seems so unlikely today that if that does happen, that'll, yeah, that'll be, that'll be a prescient call.

• 1:00:53Speaker 01

  Timing is hard on that one. I'm confident that'll happen at some point, whether it'll be in the back half of 26. This is the downside of options pricing, but.

• 1:01:03Speaker 02

  It's going to be hard to top the predicting Sam Altman's ouster.

• 1:01:07Speaker 00

  Who else is out?

• 1:01:08Speaker 02

  Yeah, I think, I think my predict Daria will still be at Anthropic at the end of the year. No, I think this, yeah, this is a lot less, less dramatic one. And maybe this is. Probably less spicy these days because I think it's becoming increasingly evident to folks. But I think by the end of the year, it will be like very obvious that Anthropic is like a fledgling juggernaut in the making in the life sciences and in biology. Like, it seems clear to me that that is like the next big direction that Anthropic is focusing on. And Anthropic has rightfully gotten a lot of praise and admiration for its focus and for like being so

• 1:01:50Speaker 02

  dialed in on coding and just knocking that out of the park and using that as like the stepping stone where to leapfrog OpenAI, whereas OpenAI was kind of all over the place. And so I think that focus has been really valuable. I do think that like it's clear that life sciences is their next big bet. And so, you know, on the one hand, you could say, does that represent like a fragmentation of focus? That's gonna be problematic for them. I kind of think that like it's their next big chapter after coding and it will be for the for the next several years to come. Yeah, and I think the really interesting question is like, and I know Jacob, you like you, you are deep in the healthcare world and I'm sure you have a lot of thoughts on this. But I think really interesting question is like, how far down that value chain is Anthropic planning to go?

• 1:02:34Speaker 02

  Like, there are plenty of rumors that I'm sure folks have heard that like Anthropic is setting up their own wet lab facilities to run and run their own experiments and collect data. And like, is that just to collect data to train models? Or are they thinking about, or will they eventually be thinking about developing their own assets? And it obviously, as is generally the case in the space, like, there'll be a progression over time. But I do think that like, you know, maybe this is more suited for like a three year prediction. But like, I think in the course, in the fullness of time, Anthropic will become like one of the most important life sciences companies in the world.

• 1:03:09Speaker 00

  How much of an advantage do they have from what they've already done to date? You know, within life sciences, right? I mean, obviously, like you can use LLMs to be a co-pilot for research. But it seems like the models themselves are actually quite different, right?

• 1:03:22Speaker 02

  Yeah, I don't think they have a huge advantage today. I think their only advantage is like they're they're arguably the best AI research organization in the world. And if they choose to point that capability at

• 1:03:33Speaker 00

  If only there was like a visionary CEO at a top AI lab who had been interested in bio for some time and like actually spent some time on it and started the company on it years ago. That might be pretty interesting to

• 1:03:44Speaker 02

  Yeah, okay, that's I'll revise my prediction anthropic and isomorphic labs will be two of the most important websites. Because you're right about ISO, but I think anthropic can get there. And I do I do think Dario has also been passionate about biology for a long time and who's a neuroscience PhD and so forth.

• 1:04:02Speaker 00

  Totally. I mean, I think like the thing I do take in your in that is real is I think bio actually probably is one of the only opportunities that is large enough. I mean, obviously, the anthropophobes are interested in it from a mission perspective. They've been so consistent in that. And it's one of the coolest impacts of of AI from like an impact. It also is probably one of the only TAMs that is large enough to justify like that level of you know, you got code, you've got like your general co pilot for knowledge work and like, I don't know, there's not that many that are is that are that are as large. And so I also think the, you know, the opportunity could be massive. We'll see the, you know, bios are there's there's it doesn't the feedback loops on data and actually getting longitudinal data just has a, you know, a time to it that is that is isn't even you know, even something like robotics is much easier, right? Because you can at least immediately

• 1:04:47Speaker 00

  get the feedback loop on whether something worked or not.

• 1:04:49Speaker 01

  Yeah, I think domain expertise also matters a lot more than many AI folks often think. So I think there's a question of how anthropic approaches this. Does anthropic approach this by going to hire a bunch of like really fantastic biologists and then pair them with the really strong AI researchers and do as much there versus just trying to throw the AI researchers at it. I think that is one thing that, you know, Demis did very well with Alpha Fold and then isomorphic is not assuming that kind of the AI researchers alone can can do that. And I've seen lots of other labs kind of make that mistake. But I think the domain expertise matters a lot. I think it's gonna be a hard a hard one for anthropic to do. But if they can do it, if they can hire that team,

• 1:05:31Speaker 01

  it's obviously a huge, huge boom. But I would bet probably more on isomorphic right now.

• 1:05:36Speaker 00

  Well, guys, this has been a ton of fun. I really appreciate you. You both taking the time to jam on this. And well, we had to do better, you know, no six months in between this one and the next. Or you're closing too many candidates. Next time, we'll do it sooner.

• 1:05:49Speaker 02

  R is the only one with an actual job that we have to schedule.

• 1:05:52Speaker 00

  Yeah, I don't know. You came in with some really good takes on lithography there. I was like, that someone's been doing some work over there. I was out of left field.

• 1:05:59Speaker 01

  Yeah, seriously, T I L. I learned a lot about alternates, alternate approaches.

• 1:06:04Speaker 00

  I'm Jacob Effron, and this has been Unsupervised Learning, a podcast where I get to talk to the smartest people in AI and ask them tons of questions about what's happening with models and what it means for businesses in the world. As I hope is clear, I have a ton of fun doing this. It's a nights and weekends project in addition to my day job as an investor at Redpoint. But our ability to get these incredible guests on really comes from folks like you subscribing to the podcast, sharing it with friends. It's really what ultimately makes this whole thing work. And so please consider doing that. And thank you so much for your support and listening. We'll see you next episode.