Anjney Midha's Plan to Radically Lower the Price of Compute

Anjney Midha wrote the first check to Anthropic. He teaches a viral course at Stanford on how AI works. And he was, until recently, a partner at a16z. In other words, he is AI-industry royalty. Midha's new project is AMP PBC, a company that believes it can radically lower the price of compute. To accomplish that, he is working on building a compute grid that turns GPUs into a standardized utility. But right now, compute is too fragmented. It's too heterogeneous. And given the way contracts are structured, he says that labs are being forced to spend money on capacity that often goes unused. In other words, small labs are forced to pay up for big, long-term contracts, even though their own demand (particularly during model training) may be very spiky. On this episode, Midha explains how the market for compute currently works and why he believes there's a software solution that could significantly improve compute utilization. He also tells us why he does not anticipate one company will emerge as the dominate player and that instead we'll have a wide range of models, each optimally used in specific applications. Read more:Amazon Says Its Data Centers Use 2.5 Billion Gallons of WaterOracle Falls Most in Six Months on Mounting Data Center Costs Only http://Bloomberg.com subscribers can get the Odd Lots newsletter in their inbox each week, plus unlimited access to the site and app. Subscribe at bloomberg.com/subscriptions/oddlots Subscribe to the Odd Lots NewsletterJoin the conversation: discord.gg/oddlotsSee omnystudio.com/listener for privacy information.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Tracy Alloway?

  There are some market stories where you want every detail. Joe and I have made quite a few podcasts on that basis. But sometimes you've only got 10 minutes and just want to know what's moving markets fast. That's the Barclays Brief Podcast. Every week, experts from Barclays Markets and Research get you up to speed on what's happening and what to watch next. Concise, focused, and brief. Search Barclays Brief wherever you get your podcasts.

• 0:26Speaker 01

  This is the Bloomberg Tech Minute brought to you by ChatGPT. I'm Carol Muster. Getting hired at SpaceX requires exceptional ability and may include having Elon Musk review a candidate's resume. So writes Bloomberg Business Week's Lauren Grush. She reports that the company culture is defined by Musk and job candidates must understand that a commitment to SpaceX means adhering to this culture and exhibiting loyalty to Elon Musk. To get hired, candidates must demonstrate their achievements from past jobs and hands-on experience matters more to recruiters than advanced degrees, with the company looking for people with raw, innate intelligence. A spokesperson for SpaceX didn't respond to a request for comment on its recruiting

• 1:08Speaker 01

  and hiring practices, but according to recruiters, advisors, and those who've interviewed for jobs there, it's the space exploration dream that draws people to the company, and Musk invokes that dream regularly to remind workers why they're there. Prospective hires are told that the job will require stamina, including in job ads that candidates, quote, "must be willing to work extended hours and weekends as needed." One major perk for all that hard work? Employees' access to equity, either the right to buy shares or payment in the form of restricted stock units that vest over time. That's the Bloomberg Tech Minute brought to you by ChatGPT. Get started at chatgpt.com.

• 1:52Speaker 01

  Bloomberg Audio Studios. Podcasts, radio, news.

• 2:07Tracy Alloway?

  Hello and welcome to another episode of the Odd Lots Podcast. I'm Tracy Alockway.

• 2:11Joe Weisenthal?

  And I'm Joe Weisenthal.

• 2:12Tracy Alloway?

  Joe, we like to talk a lot about physical constraints on this show, right? And this is one reason why AI is a really fascinating area for us right now, because there are a lot of physical constraints on what is ultimately the sort of ephemeral technology. And I think that the tension between those two things is really interesting, right? Like, you type a prompt into ChatGPT or Claude or whatever, and it's the sort of like disembodied digital platform. You don't necessarily think about the power usage, the real resources, the transformers that have to go into data centers to get compute.

• 2:47Joe Weisenthal?

  The thing that I've been on my mind lately, and I've written about it, and I plan to write more, is this idea that the canonical AI thought experiment is, what happens if you tell an AI to make a lot of paperclips? And then it destroys the world because in the pursuit of marshaling all of the world's resources, it just turns everything into paperclips because it doesn't know.

• 3:08Tracy Alloway?

  I have to ask, is this canonical example? Is this based on your traumatic fear of Clippy?

• 3:14Joe Weisenthal?

  No, but that is, you know, it all comes back full circle. But what we are seeing in real life is that everything from access to electrical grids, GPUs being the big example, energy turbines, talent, and now even including residential real estate are being re-proposed to make more and more advanced AI. And in the original paperclip thought experiment, they envision, or at least in one version, the philosopher Nicholas Bostrom envisions the AI having exhausted all of the world's resources, then sending a probe into outer space to consume star energy to build more paperclips.

• 3:50Tracy Alloway?

  Just eat the universe.

• 3:51Joe Weisenthal?

  And to this point, we're even talking about going into outer space for data centers to build more AI. So every version of the thought experiment is being replicated, except it's just more and more resources to build the AI by humans rather than paperclips by AI.

• 4:07Tracy Alloway?

  There's this other connected theme here. So we've talked before about how One of the reasons valuations seem to be getting insane in the market is because all of this activity is being driven by like this existential need

• 4:20Joe Weisenthal?

  Yes.

• 4:21Tracy Alloway?

  to become number one

• 4:23Joe Weisenthal?

  Yes.

• 4:23Tracy Alloway?

  in frontier models and this new technology. And so if you say you absolutely have to be the first to invent AGI, then you can justify any amount of spending on earth, right? And so what we tend to see is like the biggest companies just keep getting bigger. They're the ones that can get resources for all this stuff.

• 4:40Joe Weisenthal?

  And I think one of the most fascinating things right now is that at least as of right now, June 4th, 2026, the frontier models are really close to each other, right?

• 4:50Tracy Alloway?

  Yeah.

• 4:50Joe Weisenthal?

  So and the 4.8, GPT 5.5, like, they're not that different. And one of the things I'm curious about is, is there something inherent in market dynamics in this space that will always keep, you know, whether it's being able to distill results from another model and cause I steal them, whether it's information sharing among employees? Is there some inherent reason why we've seen the stability? Or could it be that at some point, one lab just like breaks out and establishes permanent?

• 5:22Tracy Alloway?

  Still a possibility, but like I am personally on the side of commodification and everything just becomes kind of basic.

• 5:29Joe Weisenthal?

  Well, everyone's waiting for your take.

• 5:31Tracy Alloway?

  I know. Okay. Well, all right.

• 5:32Joe Weisenthal?

  I'm just kidding.

• 5:33Tracy Alloway?

  Thank you, Joe.

• 5:33Joe Weisenthal?

  That is a joke.

• 5:34Tracy Alloway?

  All right, that's a polite prompt to get to the guest. We do, in fact, have the perfect guest. We're going to be speaking with Angenate Midha. He is, of course, a former general partner at Anderson Horowitz, a Stanford University visiting scientist who teaches the viral AI lecture called Frontier Systems. Also, one of the first guys to write a check for Anthropic.

• 5:56Joe Weisenthal?

  Amazing.

• 5:56Tracy Alloway?

  And he is now the founder of a new company called AMP PBC. So thank you so much for coming on Odd Lots, Ang.

• 6:03Speaker 03

  Thanks for having me. One correction, it's pronounced AMP PBC, but that's everything else you got perfect on the intro.

• 6:08Tracy Alloway?

  Oh, I'm sorry. AMP would make sense, wouldn't it?

• 6:10Speaker 03

  As an energy, yeah.

• 6:11Joe Weisenthal?

  Yeah, yeah.

• 6:12Tracy Alloway?

  And just remind us, the PBC is Public Benefit Corporation.

• 6:15Speaker 03

  That's right.

• 6:15Tracy Alloway?

  So you're doing this for the public benefit?

• 6:17Speaker 03

  We're governed by a public benefit charter, which means everything we do has to follow our mission. We have a public charter mission. We are for profit in the same way Ben and Jerry's or REI and Anthropic are public benefits. So we aim to make a healthy, modest amount of profits that can sustain our mission. But we are, we have the flexibility to choose what that margin is.

• 6:38Tracy Alloway?

  Can I just start? I want to establish your credentials, although I feel like that very long list did a pretty good job. But writing the first check for Anthropic, like, tell us that kind of origin story, because the anecdote that you hear is like, 25 VCs turned them away initially, and you said yes.

• 6:54Speaker 03

  It was a little bit of the other way around. I said yes, then we tried to get another 25 VCs to say yes, and I failed. It was a harrowing experience. It was a bit of a wake-up call. It was late 2020. I had just sold my last business. It was called Ubiquity6. It was a 3D mapping business, an AI, it was an AI business that we had founded in 2017. And I felt like a failure at the time, because, you know, in San Francisco, I was in San Francisco, I, just as big picture, my life stories, I was born in India, I went to high school in Singapore, and I came out of college to the United States at Stanford for my undergraduate degree. And then when I arrived at campus in 2011, deep learning had just started taking

• 7:36Speaker 03

  over the world in Silicon Valley. Andre Karpathy was a computer science TA to Andrew Ng, who was one of the, I would say, modern sort of founding fathers of deep learning, this idea that you can teach machines to think without having to give them prescriptive rules. And so I went into sort of machine, I got swept up in that moment and started studying. A lot of my coursework was in machine learning. My primary department at Stanford was in bioinformatics, which was machine learning applied to healthcare. I got sidetracked to a venture firm called Kleiner Perkins for about four and a half years, where I got the chance to work for some of the great investors like John Doerr and Mary Meeker. And then I left and started my own company.

• 8:16Speaker 03

  And as is the case in Silicon Valley, when you start, I mean, I was 25, I went and raised about 47 or so million dollars from some of the usual suspects like Benchmark and Index and so on. And I thought I was the coolest kid in town. And I got the [bleep] beat out of me. Because we, we built this incredible technology, which was this AI system that could map any location in 3D, and then the pandemic hit. And so, location-based mapping, 3D mapping, you can, the only thing you can control is how you react to what happens. And so I did feel for a moment like it was bad luck, and then you just have to pick up the pieces and make the best of it. So I did with my co-founder. We figured it out. It was, it was a tough.

• 9:00Speaker 03

  Few years where we had to pivot the business, but we landed the plane. We essentially, a lot of the distributed systems we'd built on the backend side ended up being quite valuable. We sold that to a company called Discord, which is a chat app for gamers.

• 9:13Tracy Alloway?

  Yeah.

• 9:13Joe Weisenthal?

  We have an onlogged Discord, time to plug that. We chat with our fans in there, yeah.

• 9:16Speaker 03

  Awesome.

• 9:17Joe Weisenthal?

  Our listeners.

• 9:18Speaker 03

  So about a month after I sold the business, I got a call from some friends who were running research at OpenAI. And we'd all been friends in the machine learning community in the Bay Area. And they said, Anj, we've trained a little model called GPT-3, and we think it's the best since.

• 9:31Tracy Alloway?

  Yeah, just a little model.

• 9:33Speaker 03

  Yeah, nobody really paid attention. They were like, nobody cares, but we think it's the best thing since sliced bread. And we want to leave and turn this into a standalone business. But it'd be helpful to get some of your advice on how to do that. And I couldn't really come on board full time at the time with them because I had to integrate my company into the acquirer.

• 9:51Joe Weisenthal?

  Sure.

• 9:51Speaker 03

  But I came on as their angel and nights and weekends I worked with them on the business plan and who we should raise from. That company was Anthropic. Dario and Tom and I started doing these weekly working sessions in early 2021. And yeah, I assumed that if we went and talked to a bunch of venture capitalists on Sandal Road, especially some of the ones who were involved in the biggest hits of the last decade before that, they would get it. These are the creators of GPT-3. And they were like, we just don't get this. We've heard the whole AI story before. This whole general intelligence thing is a pipe dream. And it was painful. We tried to raise 500 million dollars. We couldn't.

• 10:30Speaker 03

  We instead scraped together about 100 million, which I know sounds like a lot, but at the time was a rounding error compared to how much Google has spent on the same kind of systems. And it was all angels in that first round, a bunch of cats and dogs, all of us who believed in the mission. And then over the next 18 months, Dario, Tom and team put together a plan that we kind of workshopped on getting Amazon involved as a strategic. And that resulted in a $4 billion compute and capital partnership that made me realize. Infrastructure, especially compute infrastructure was just a key requirement to create any kind of modern AI lab. And so since then, I've spent the past five, six years figuring out how to unblock that compute bottleneck for research teams.

• 11:10Joe Weisenthal?

  Amazing. Well, obviously, an incredibly well timed.

• 11:14Tracy Alloway?

  It just emphasizes how much things have changed, right? Where like people are literally throwing money at like almost any model now versus like a few years ago going like, ah, AGI, I don't really know.

• 11:25Joe Weisenthal?

  Well, let me ask you this question because this is a very top of mind question for me. And we're getting, we can skip around on the timeline here, but there are three labs that are seen as like genuinely at the frontier right now. And that is obviously DeepMind within Google, OpenAI and Anthropic. And then of course, you know, a lot of people say that the Chinese labs are very close, if not quite there, maybe they're a few months behind. Is this, is there, you know, when we think about like, part of your mission is like, you say, okay, a new lab should be able to get access to compute. If you're really bright, like that shouldn't be the bottleneck. Does that imply therefore that

• 12:06Joe Weisenthal?

  you expect more labs to be able to, were they to have access to the compute, also reach the frontier? And that there is something inherent about like this sort of seeming stability or parity that we see among frontier models?

• 12:22Speaker 03

  So the answer to your first question is yes, there are many frontiers to be conquered and pioneered. And this is not just one frontier. I think that's a fundamental misunderstanding people have about the frontier. They talk about.

• 12:32Joe Weisenthal?

  Jagged frontier.

• 12:33Speaker 03

  Exactly. Jagged intelligence, right? In a poetic sense, in a historical sense, if you think about the Wild West or the Western frontier, it wasn't just one frontier. There was a frontier of gold and there was a frontier of jeans. It turns out Levi's, you know, turned out to be a new modern behemoth of a company. I mean, there were so many new businesses founded in the industrial revolution. And I think that's, that's the reality is the software engineering frontier, which is where Anthropic is clearly a leader, is one frontier.

• 12:59Joe Weisenthal?

  Yeah.

• 13:00Speaker 03

  I think the chat frontier, the sort of consumer chat frontier is another frontier where OpenAI has been a leader.

• 13:06Joe Weisenthal?

  Arguably ByteDance is at the video frontier with SeedDance, right?

• 13:10Speaker 03

  Absolutely. Yeah. And so I think there's just many, many frontiers to be conquered or pioneered, rather. I think Anthropic is clearly a role model for the rest of the community on how to do it in an efficient way. They're, you know, I think fewer than 5,000 people and they've been able to put out state of the art models that. You know, teams like Google, which have 60,000 people are close to, but not yet quite there. So actually, I don't really agree with your with your assessment that they're all at parity. If you use the models day in and day out, they're quite remarkably different in meaningful ways to the person with hands on the keyboard, you know, doing the engineering work. And I think those those differences reflect the focus of the teams, right?

• 13:51Speaker 03

  What is the actual mission that the team working on on that domain cares about day after day after day? So in this in the Stanford class I teach, the first lecture was a breakdown of how frontier models are even created. And it's actually quite simple. The recipe is is super simple. There's basically four steps. There's pre-training, mid-training, post-training, and then what we call the continuous feedback loop. So pre-training just is just says, hey, you collect a bunch of data from the Internet and and train a model to be a generally good pattern recognition machine. Yeah. You then do mid-training, which is to say, in a particular domain that you really care about, you inject more capabilities. So if you want this model to reason about science or math or physics, then you give it science or math or physics data.

• 14:36Speaker 03

  And then you get a pretty good model that's specialized in that domain. And then you deploy it to the real world where you have people using it. And the context feedback, which is when the model is able to do a task well or not, and you can verify whether that task was done correctly, gives the model the data it needs to keep improving on that task, on that distribution.

• 15:11Tracy Alloway?

  There are some market stories where you want every detail. Joe and I have made quite a few podcasts on that basis. But sometimes you've only got 10 minutes and just want to know what's moving markets fast. That's the Barclays Brief podcast. Every week, experts from Barclays Markets and Research get you up to speed on what's happening and what to watch next. Concise, focused, and brief. Search Barclays Brief wherever you get your podcasts.

• 15:38Speaker 01

  This is the Bloomberg Tech Minute brought to you by ChatGPT. I'm Carol Muster. Getting hired at SpaceX requires exceptional ability and may include having Elon Musk review a candidate's resume. So writes Bloomberg Business Week's Lauren Grush. She reports that the company culture is defined by Musk and job candidates must understand that a commitment to SpaceX means adhering to this culture and exhibiting loyalty to Elon Musk. To get hired, candidates must demonstrate their achievements from past jobs and hands-on experience matters more to recruiters than advanced degrees, with the company looking for people with raw innate intelligence. A spokesperson for SpaceX didn't respond to a request for comment on its recruiting

• 16:20Speaker 01

  and hiring practices, but according to recruiters, advisors, and those who've interviewed for jobs there, it's the space exploration dream that draws people to the company and Musk invokes that dream regularly to remind workers why they're there. Prospective hires are told that the job will require stamina, including in job ads that candidates, quote, "must be willing to work extended hours and weekends as needed." One major perk for all that hard work? Employees' access to equity, either the right to buy shares or payment in the form of restricted stock units that vest over time. That's the Bloomberg Tech Minute brought to you by ChatGPT. Get started at ChatGPT.com.

• 17:01Tracy Alloway?

  This is slightly tangential, but like I give a lot of feedback to the models because Joe made me paranoid about the basilisk theory. So I want the models to appreciate me once they take over the world. But when you give them feedback, like if they spit out a wrong answer and you say that's wrong, they immediately apologize and fall over themselves to say that they're sorry, but then you ask them like, give me another output or like, would you do it again the same way? And they like, they often say yes, or they give like a very similar answer. They don't seem to be responding in real time.

• 17:33Speaker 03

  Correct. So when I say feedback, I mean a very specific kind of feedback, which I, which I call verifiable feedback. So when you say that wasn't right or that was wrong, that's an opinion.

• 17:42Tracy Alloway?

  Okay.

• 17:43Speaker 03

  Verifiable feedback is when you can have as close to factual verification as possible. The reason-

• 17:47Tracy Alloway?

  So what does that actually look like?

• 17:49Speaker 03

  That's a great question. So let's take these by example in two or three cases. In the case of software engineering, in the way software engineers actually code is you write a piece of code and then you submit it to the main code base and then you usually have a peer on your team review the code and approve it or reject it. And if it gets approved, that's called, that's the first step. That's called a PR, a pull request. And if another human on your team that you trust approved it, that's one kind of verification of quality. And then two, before that piece of code usually gets deployed to a production system, you have unit tests. And those are, you know, quite objective tests of is this code performing the function we need it to? And if it passes both those tests, it's a verifiable piece of code that accomplished the goal.

• 18:33Speaker 03

  So in software engineering, the reason we've seen such a dramatic improvement in capabilities is that a lot of these labs are using feedback from that verification loop.

• 18:43Joe Weisenthal?

  Yeah.

• 18:43Speaker 03

  In the case of another lab I incubated called Periodic Labs, which we started a year ago and you should come by sometime, we've got 40,000 square feet in Menlo Park where we've got AI models that are predicting new, the goal is to try to find a room temperature superconductor. And so these models predict.

• 18:58Joe Weisenthal?

  Oh, I forgot about, I forgot about.

• 19:00Tracy Alloway?

  Room temperature superconductors.

• 19:01Joe Weisenthal?

  I forgot about that.

• 19:02Tracy Alloway?

  That was a fun summer.

• 19:04Speaker 03

  Yes, this time we will verify their, if we ever put something out, you will know it's not.

• 19:08Joe Weisenthal?

  It'll be real.

• 19:09Speaker 03

  It, that's not going to be us. But the AI system predicts new material candidates. Then we have robots that synthesize the new material in the lab and then use X-ray diffraction machines to test whether the material has the properties the AI said it would. And that's verifiable feedback from reality, from physics. And then we pipe that data back into the training loop over and over again. That context feedback is very factually verifiable.

• 19:34Tracy Alloway?

  Yeah.

• 19:35Speaker 03

  And that's where progress is the fastest today because that, that feedback doesn't result in the kind of hallucinations that you often experience with these models on more subjective tasks. It's also, by the way, why the models are terrible at subjective tasks like creative writing.

• 19:48Tracy Alloway?

  Yeah.

• 19:48Speaker 03

  And sometimes it can get quite toxic, to be honest, if you get them down the wrong loop. I don't know if you've been using it as a, you know, therapy bot and so on.

• 19:55Tracy Alloway?

  I have not, just for the record.

• 19:56Speaker 03

  That's great.

• 19:57Tracy Alloway?

  It did ask me to defy the laws of gravity at one point because I was trying to create something in my backyard. And I was asking how to do it and it was like, then just set this up like the following way. And I was like, that's not within the laws of physics.

• 20:09Speaker 03

  Yeah.

• 20:10Tracy Alloway?

  But whatever.

• 20:11Joe Weisenthal?

  Well, what's

• 20:11Speaker 03

  No, go ahead.

• 20:12Joe Weisenthal?

  Well, what's interesting, and this is actually a trillion dollar question from just a very broad standpoint, is as you point out, even prior to AI, the field of coding had a very systematized approach to the feedback loops already.

• 20:26Speaker 03

  Yes.

• 20:27Joe Weisenthal?

  And so then it's like AI can sort of replicate that. And anyone who's done your vibe coding can see in the chain of thought sometimes, oh, that didn't work. Let me try this. That didn't work. Let me try this.

• 20:36Speaker 03

  Yes.

• 20:37Joe Weisenthal?

  Most fields don't really have that by and large. Journalism doesn't have that. I mean, there are things, there are outputs that are better and worse. We don't really have that, that sort of like formalized approach to the yes, no. Does that, just zooming out, to my mind, that would imply that maybe, at least to some extent, coding is a little bit special from a sort of white collar knowledge work that in terms of like, is it going to be as good as, say, I don't know, sales or something like that? Because it has, coding has a long history

• 21:13Tracy Alloway?

  It's formulated and structured.

• 21:14Joe Weisenthal?

  Of that structured pipeline.

• 21:16Speaker 03

  Yeah, this is a great point. So where progress will be made most predictably is in parts of knowledge work where the task is essentially a workflow that's fairly structured.

• 21:26Joe Weisenthal?

  Yeah.

• 21:26Speaker 03

  And so somebody who spends most of their day inputting cells into an Excel spreadsheet, well, that part of the job will get automated pretty fast because that's actually verifiable. And you know what? That's frankly often the most tedious part of the job anyway. And so I'm quite excited to see that progress because I'm terrible at spreadsheets. And I think if we could free up more of my time and hopefully other people's time to focus on the art of the spreadsheet, not the tedious part of it.

• 21:52Joe Weisenthal?

  The entry, the entry and retrieval. Yeah.

• 21:55Speaker 03

  Yeah, exactly. You know, and in journalism, I think it's the same thing. There's so much craft that gets, that goes into the verification of a story before it goes out. That's not legible to the world. You know, I've had a chance to spend some time with some of the journalistic institutions of the barrier like Kate Metz or Brad Olson at The Journal. And as you spend time with them, you realize, I mean, they're verifying every sentence that goes into each article.

• 22:20Joe Weisenthal?

  Fact checking, absolutely.

• 22:21Speaker 03

  So fact check, that's an example where I think we should be leaning on these tools and you should expect more progress. And the parts then that will be more, to borrow your jagged frontier framing there, that's, we will be in a regime of jagged frontier progress where wherever parts of workflows that are verifiable factually will essentially, you will see progress there very predictably over the next few years. And consequently, wherever that progress, that the workflows are not verifiable is actually where humans are going to shine. And I think that's where parts of the economy are, you're going to see extraordinary gains in the wages of humans who have creativity and craft that are not typically verifiable,

• 23:04Speaker 03

  you know, through traditional objective means. Does that make sense?

• 23:07Tracy Alloway?

  Yeah, it does. And it dovetails with a lot of what we've been talking about on the show recently. Just going back to verifiable feedback. So, okay, the model spits out something and you can check whether it's right or wrong.

• 23:18Speaker 03

  Right.

• 23:19Tracy Alloway?

  Is it important to understand how the model actually got to that answer? Because we have discussions with like big bank CEOs who are using more AI. And their response to this question is always like, well, if we can put restrictions around the AI, if we make sure that it's like released into a sandbox before it's released into the wider world, we're all set from a regulatory perspective and regulators don't actually need to know what's in the black box model and how it's working. But like, this seems a bit concerning to me.

• 23:49Speaker 03

  Yeah. No, I am quite strongly opinionated about this one, which is that technical literacy should be non-negotiable. This is the reason I spend so much time teaching this class at Stanford, putting it up online. And the idea of the Frontier Systems class is that end to end, it's a full, simple, but but first principles breakdown of how these AI systems are built from scratch, from land, power, shell, like the energy, where do we get them, the data centers, then how do we train the models? And the final project, the class with the kids was actually the one person frontier lab, which is at the end, they're creating their own models and so on, because the idea is that the, a person with the right tools today can scale themselves infinitely, but they need to know how to use the tools, what the limitations are, when to lean on them versus not. And I think this is a generalizable piece of technical literacy that all leaders should have. It's like saying, you know,

• 24:36Speaker 03

  I, in the nineties, I imagine if you knew, you could use the internet without really knowing how it worked. But you know, on the margins when like the page doesn't like refresh.

• 24:46Joe Weisenthal?

  Yeah.

• 24:47Speaker 03

  Or you're like this, this cookie thing is annoying me. Like over time, people who were more technically literate just realized sometimes you got to debug, you know, the browser. And those of us who've learned over time to do knowledge work are more adept at leaning on them versus not. Like just now, when I was trying to get onto the internet, I realized, okay, there's this.

• 25:07Joe Weisenthal?

  Oh, yeah.

• 25:08Speaker 03

  You know, wifi password, whatever. And then you don't end up relying on them in ways that they can't fulfill your need anyway. And what's a little bit more dangerous with these systems is because we tend to anthropomorphize them.

• 25:20Joe Weisenthal?

  Yeah.

• 25:21Speaker 03

  Without the technical literacy that I wish all leaders had about reasoning about how these systems were built. What you end up doing is projecting out in your mind what the capabilities are in ways that are inaccurate. You project out their impact in society that are not accurate. You project out their business models in a way that are not accurate. I mean, the very fact that when you started this conversation, I don't blame you for it. You're like, there's, there's three models at the frontier.

• 25:46Joe Weisenthal?

  Yeah.

• 25:46Speaker 03

  I'm like, well, which frontier? And which three models? Because from where I'm sitting, there's like seventeen different frontiers right now. And there's four different players in each one. And the businesses of all of them are kind of breathtaking. So I think that technical literacy should always for leaders be a basic requirement. And then if you're deploying your, these systems at Goldman Sachs, you won't oversimplify and get tripped up later when, you know, two years later, you realize half your employee base has been leaning on this like sandbox framing. When in reality, inside the sandbox, they were doing all kinds of, they were using the tools in ways that were prone to hallucination, prone to risks, prompt injection. They were leaning on it in ways that were not informed in the appropriate ways. Is this making sense?

• 26:30Tracy Alloway?

  Yeah, like at a minimum, they would not be using it in the optimal way.

• 26:33Speaker 03

  Correct.

• 26:34Tracy Alloway?

  Okay.

• 26:34Speaker 03

  Or relying too much on it. It's the. You can't outsource your understanding to a model. You can you can outsource your thinking. You can outsource part of the tedious workflows, but you can't outsource your understanding.

• 26:49Joe Weisenthal?

  Yeah.

• 26:49Speaker 03

  And if you keep think, if you say, if you create these simplistic frameworks of, oh, here's a sandbox and this is safe. You have to, you have to use that sandbox in the right way. Because if you say, well, now everything that happens in the sandbox is totally fine. If the model says, use the spreadsheet, the spreadsheet is good, it's deployed in our servers. But you didn't actually check the spreadsheet and what went into the spreadsheet. And did the model actually understand the particular structure of the business, the physics of the business that you're trying to model out? Then you've outsourced your understanding to it. Does that make sense?

• 27:20Joe Weisenthal?

  Absolutely. Let's talk about AMP.

• 27:23Speaker 03

  Yes.

• 27:23Joe Weisenthal?

  And because one, you know, you're never going to get the frontier in anything unless you have access to compute. It seems pretty obvious. And there are various arrangements for acquiring compute. You have companies building their own data centers. You have smaller labs and maybe they use someone else's data centers or a NeoCloud, et cetera. What are you building at AMP such that at least as part of this story?

• 27:47Speaker 03

  Right.

• 27:47Joe Weisenthal?

  Is trying to solve the compute bottleneck specifically?

• 27:50Speaker 03

  Yeah, we are, it's very simple what we're, what we're doing at AMP. We're doing two things. We are trying to standardize the format for compute, which today is super fragmented. So in the history of infrastructure, if you look at whether it was the industrial revolution, the internet, streaming, there were usually formats of inputs that were quite heterogeneous. They were fragmented. And then to unlock productivity, you had to standardize a format. So in the case of electricity.

• 28:21Joe Weisenthal?

  Yeah.

• 28:22Speaker 03

  Until it was, until ACDC was standardized, right? Megawatts would just sit in stranded pockets around the United States being unused. And then, once we standardized the format to ACDC, then the question was, okay, great. Now we turned all these stranded pockets of electricity into one sort of interoperable, universal format. Now, how do we distribute it to everybody who needs it? And we came up with this distribution layer in the United States called the grid. That's, that's all we're doing. Yeah.

• 28:50Tracy Alloway?

  You're building a grid for compute.

• 28:51Speaker 03

  Correct. We're standardized, we're trying to standardize the compute layer today. Different chip types, different manufacturers, different clouds. I mean, it's a complete mess. And if you're, yeah, go ahead.

• 29:01Tracy Alloway?

  Say more about how we plan to do this because we've talked before about, you know, there are various people out there that want to create indices of compute, futures potentially on compute. And the issue that always comes up is fungibility.

• 29:15Speaker 03

  Right, exactly. So we've got a couple of ways we solve the fungibility problem. This is a pretty thorny challenge. We solve it in two or three ways. The first is, we have a system called the grid, which actually makes the compute fungible at a consumption layer. So under the hood, we have a bunch of different chip types. We support various different manufacturers. And there's a system that was built to do this already inside a little company called Google. And one of the technical leads on that, on that project, it was called Borg internally at Google, is my co-founder Sebastian Lobo. He was my roommate at Stanford 14 years ago. He's my engineering co-founder. And we're building Borg for everybody else, which is essentially a translation layer. That says, no matter what the underlying chip type is,

• 29:57Speaker 03

  the machine learning researcher who's using the chip just has to worry about the workload. And we handle everything else underneath the hood.

• 30:03Tracy Alloway?

  When you, when you say system, is this hardware or software that's doing this?

• 30:08Speaker 03

  It's all software.

• 30:08Tracy Alloway?

  Okay.

• 30:09Speaker 03

  Yeah. So we handle that translation layer in software.

• 30:11Tracy Alloway?

  Huh.

• 30:12Speaker 03

  And it's a pretty gnarly challenge, but today we're able to do that in, in ways that improve utilization sometimes from 50, 60% at labs that we have incubated or on the grid, to close to 95, 96%. At Google, the utilization is roughly 99%. When, when Sebastian arrived at Google, it was about 62%. By the time he left, it was roughly at 99%. At Google, if utilization is at 96%, that's considered a major outage.

• 30:37Tracy Alloway?

  Yeah.

• 30:38Speaker 03

  Today, the average data center in the industry, in the ecosystem, in the independent ecosystem, is running at less than 70% utilization. The Colossus 2, which is running in Memphis, Elon's 500,000 GB, 500,000 GB 300s, was running at less than 60% node utilization and less than 11% MFU. MFU, model flop utilization is how much of the chip is actually being used. So there's two kinds of utilization people care about in a data center. First is how many chips are being used? That's the highest, that's just the most naive measure. If that number is not at ninety plus percent, no excuses. So you have the chips, they should at least, at least be doing something.

• 31:14Joe Weisenthal?

  Yeah.

• 31:14Speaker 03

  And then within the chip, doing how much of the chip is being used? Within a workload, that number is usually much lower.

• 31:21Joe Weisenthal?

  I'm very intrigued by this latter point about that even if, that even like the chip itself may not be even used at full capacity.

• 31:30Speaker 03

  Yes.

• 31:31Joe Weisenthal?

  Because I see these numbers.

• 31:33Speaker 03

  Yeah.

• 31:33Joe Weisenthal?

  And you say like a lab has like a, we have two hundred chips, we've acquired eight hundred GPUs, et cetera. And when I see these headlines, I assumed that a utilize, optimal utilization techniques must be so good that you can infer someone's capabilities simply by how many Nvidia GPUs they've acquired.

• 31:54Speaker 03

  No.

• 31:54Joe Weisenthal?

  What you're saying is that there is actually quite a bit of heterogeneity about the techniques and approaches to getting the most juice out of a chip.

• 32:03Speaker 03

  Yes, you have to measure what matters, and what matters is output.

• 32:05Joe Weisenthal?

  Okay.

• 32:06Speaker 03

  Your, when, anytime I start a new lab with a team, you know, in the case of periodic labs, it was, we started with Liam Ferris, who was the co-creator of ChatGPT, and Doris Chubak, who led the physics teams at DeepMind. And when we sat down and we planned out the company's roadmap, the, the most important thing to us to measure was not the number of chips we had.

• 32:24Joe Weisenthal?

  Yeah.

• 32:24Speaker 03

  It's the, the eval, what we call.

• 32:26Joe Weisenthal?

  So all this chip bragging, they're like, oh, we acquired, it's just a sort of.

• 32:30Speaker 03

  It's a lot of bravado.

• 32:31Joe Weisenthal?

  Yeah, all right, this is helpful.

• 32:33Speaker 03

  You don't measure the inputs, you should be measuring the outputs.

• 32:35Joe Weisenthal?

  No, I agree, yeah, of course.

• 32:35Tracy Alloway?

  I'm actually fascinated that like, there is a software solution to what I perceived in my head as like, a very physical constraint. How does this actually work? Like, feel free to get technical here, like, I want to understand the system.

• 32:49Speaker 03

  Yes, so let me give you the technological answer and the economic answer. The economic answer actually is a simpler one to reason about. The way the compute business works today is primarily on the construct of the atomic unit of long-term leases. So I'm a researcher, I need some compute. I show up to a compute provider and say, hello, I would like some compute, please. And the compute provider says, no problem. Here's, you know, 500 AMD chips or Nvidia chips that you can lease from me on various time scales. And you've got to pay for it 24/7. It's like leasing an apartment. And whether you use it or not, that's your problem, but it's $2.50 per hour, $3 an hour. So instead, you take a long-term lease.

• 33:31Speaker 03

  And now the cloud provider, the compute provider said, great, I just booked revenue for like the next two years that this guy rented from me. Now, what happens with that compute? Whether it's used or not, is the researcher's problem. They've outsourced that problem. As a result of this wastage that we're talking about, and I'm happy to go into why it's hard for individual teams to utilize most of the capacity. The primary reason is because research is spiky.

• 33:56Joe Weisenthal?

  Yeah.

• 33:57Speaker 03

  Right? It's hard to forecast. So you over-provision for your peak, not your base load.

• 34:01Joe Weisenthal?

  Yep.

• 34:02Speaker 03

  Because what happens, you're researching on these algorithms and the minute like one is working, you go, guys, let's scale. We want to ship this thing. So let's improve, throw as many chips at it. And then, once we ship it, the needs go down. So between these spikes, there's just huge pockets of unused compute. As a result, the effective price per hour that you're paying is closer to $25 to $28, whereas the marketed rate that you think you're paying is $2.50.

• 34:29Joe Weisenthal?

  Yeah.

• 34:30Speaker 03

  So that spread due to wasted is just insane. So from an economic perspective, that's the wastage, that's the deadweight loss.

• 34:39Joe Weisenthal?

  Yeah.

• 34:39Speaker 03

  Right? Okay. So now, how do we, from a technological perspective, how do we utilize that opportunity? Literally, all we do is, from a software perspective, we take all of that unused compute. No matter what format it is, it might be Nvidia, it might be AMD, we love AMD, it might be some other chip, and we turn it into one fungible resource. And that, we standardize the format on something we call grid credits. So researchers don't even need to think about what chip type is under the hood. You know, they're just paying what they need or what they use. And so from a fiduciary perspective, I'm on seven boards. As an investor, I get very excited when teams switch from this sort of long-term lease model where they're paying $25, $26 per GPU hour. Now they're actually only paying the $2.50 that was marketed because everything they're

• 35:23Speaker 03

  not using gets reallocated to the grid and other research labs can use that resource.

• 35:45Speaker 01

  This is the Bloomberg Tech Minute brought to you by ChatGPT. I'm Carol Muster. Getting hired at SpaceX requires exceptional ability and may include having Elon Musk review a candidate's resume. So writes Bloomberg Business Week's Lauren Grush. She reports that the company culture is defined by Musk and job candidates must understand that a commitment to SpaceX means adhering to this culture and exhibiting loyalty to Elon Musk. To get hired, candidates must demonstrate their achievements from past jobs and hands-on experience matters more to recruiters than advanced degrees with the company looking for people with raw innate intelligence. A spokesperson for SpaceX didn't respond to a request for comment on its recruiting

• 36:27Speaker 01

  and hiring practices, but according to recruiters, advisors, and those who've interviewed for jobs there, it's the space exploration dream that draws people to the company and Musk invokes that dream regularly to remind workers why they're there. Prospective hires are told that the job will require stamina, including in job ads that candidates quote "must be willing to work extended hours and weekends as needed." One major perk for all that hard work? Employees' access to equity, either the right to buy shares or payment in the form of restricted stock units that vest over time. That's the Bloomberg Tech Minute brought to you by ChatGPT. Get started at ChatGPT.com.

• 37:08Joe Weisenthal?

  This is the problem you're mostly solving for is the training part because there's training, right? Training and inference?

• 37:14Speaker 03

  No, no, no.

• 37:15Joe Weisenthal?

  Or is it both?

• 37:15Speaker 03

  Both, yeah. That's the... So the beauty about having diverse types of compute on our grid is that once you make the resource fungible, you can do any workload. You just fill all the unutilized pockets with inference and then all the reservations with training.

• 37:30Joe Weisenthal?

  Got it. So can you explain like what did it... Why is it that every lab also seems interested right now in customized silicon, including Microsoft announcing a chip that says like, "Oh, our new MAI..." I don't know, "Maya"? I don't know how it's pronounced.

• 37:48Speaker 03

  MAI, I believe. We had Satya in the class yesterday at Stanford and he pronounced it as MAI.

• 37:52Joe Weisenthal?

  Okay, their new MAI model. And then he's like, "Oh, and we also have a new Maya 200 chip or something that's optimized with it."

• 37:59Speaker 03

  Right.

• 37:59Joe Weisenthal?

  Why is it that so many labs or companies that are in a lab, I guess, feel impelled to like also design a chip that goes along with the model? And long-term is what you're doing, saying like, "This really is not necessary to have that sort of model-chip alignment."

• 38:16Speaker 03

  Yeah, there's two technological reasons and two economic reasons.

• 38:20Joe Weisenthal?

  Okay.

• 38:21Speaker 03

  The first is from a economic perspective, about 80 cents of every dollar a lab spends today on their R&D flows to a chip provider like Nvidia.

• 38:31Joe Weisenthal?

  Okay.

• 38:31Speaker 03

  Right? And so as a result, your margins are just super, super rough. So from a unit economic perspective, you want more control over your margins. And therefore, when you look at your like unit economics, you're going, "Wait, wait a minute. For every dollar we make, there's this massive chunk that's going to somebody else."

• 38:45Joe Weisenthal?

  So instead of spending 80 cents to Nvidia, you spend 78 cents to TSMC and keep that 2 cents for yourself?

• 38:52Speaker 03

  Uh, well, I think that the better our software gets, the more that margin should flow actually to the researcher.

• 39:00Joe Weisenthal?

  Okay.

• 39:00Speaker 03

  Because that's where the value will be captured.

• 39:02Joe Weisenthal?

  But like, but wait, sorry, you were going to say, what's the technical reason why they're trying to do optimal model-chip alignment?

• 39:10Speaker 03

  On the technical side, the primary reason is you want control over your supply chain. Because today, in a compute... Well, we've been in a compute capacity... constrained world now for at least four or five years. But if you can't get the chips you need, you're not in control of your own supply chain. So you're dependent on compute allocations that the compute manufacturer thinks is optimal, right? By the way, that's how it works at the foundry level today. TSMC gets to decide which compute provider, compute provider's business grows or not.

• 39:36Joe Weisenthal?

  Yeah.

• 39:37Speaker 03

  Because they only have so much production capacity. And so the technological reason is you want supply chain independence. And so when you want economic independence, unit economic independence, and you want supply chain independence, you want as much control over your own chip.

• 39:49Joe Weisenthal?

  But Microsoft doesn't have a fab.

• 39:51Speaker 03

  That's not what I'm saying. What I'm saying is in inference, for example.

• 39:54Joe Weisenthal?

  Yeah.

• 39:54Speaker 03

  Satya would like more control over his unit economics, so he's making an inference chip.

• 39:59Joe Weisenthal?

  Right.

• 39:59Speaker 03

  Right? Because if you are dependent on a third party to give you the inference chip.

• 40:02Joe Weisenthal?

  Okay.

• 40:03Speaker 03

  And you need, and if you don't have an inference chip, you can't sell more, more product. You want more control.

• 40:08Joe Weisenthal?

  Is it about having a predictable supply of chips? For you, rather than a predictable supply of, okay, got it.

• 40:15Tracy Alloway?

  So there's a lot of discussion right now about more efficient model allocation. So this idea that like you do not have to be using the latest model to ask like what the weather is going to be tomorrow or something like that. And you also don't want to blow through your entire one year token budget in the space of four months, as Uber apparently did. So the spikes in usage that you're seeing that allow you to do the system and, you know, have grid credits, does some of that go away if people become smarter about which models they're actually using?

• 40:47Speaker 03

  Okay, so there's an embedded assumption, I think I should tease apart in your question. Usage is different from the production of the model. So what's happening, right, in terms of the pipeline is you use the grid to produce these system, the, the model, and then the model produces tokens.

• 41:04Speaker 03

  If the end user is only using tokens, then as long as everybody, we, we have enough diversity in the end user base using models hosted on the grid, things actually even out.

• 41:15Tracy Alloway?

  Okay.

• 41:15Speaker 03

  That cyclicality, in the same way, electricity in America evens out if you have enough scale. At scale, basically.

• 41:21Tracy Alloway?

  For the most part, yeah.

• 41:22Speaker 03

  Except when like.

• 41:23Joe Weisenthal?

  There's a heat wave.

• 41:24Speaker 03

  There's a heat wave, exactly. So some of that infrastructure. We are having to reboot, but you can think about AMP in the broadest sense as a utility company. We're what's called an independent system operator of the grid. So we don't own our own data centers. We don't own our own labs, but we coordinate the capacity needs across different parties. And at sufficient scale, that you those usage patterns actually just gets evened out. Does that make sense?

• 41:48Tracy Alloway?

  Yeah.

• 41:49Joe Weisenthal?

  Well, you're an investor. Well, you are an investor in OpenRouter, I believe, which I think is an interesting company. Do you see, setting aside AMP for a second, do you think that there is a, at this point, a still within, say, corporate America, a certain lack of savviness about knowing which model to route to for the query? And that there will be an improvement and learning within companies, within users, so that you don't have these incidents for like massive token consumption? Because perhaps everyone was using the wrong, the Cadillac model and the Ford model would have been just as fine for that purpose?

• 42:25Speaker 03

  Oh, yeah, we're absolutely in the medieval ages of of this technology. I think what will happen is increasingly, based on my conversations with corporate American leaders and corporate leaders across the world, they don't really care about the models. They don't care about the underlying model, the technologies, they just don't care. It's like too much complexity. We just want the work done.

• 42:46Joe Weisenthal?

  Yeah.

• 42:46Speaker 03

  Can you guys please figure out how to get the work done in the cheapest way, in the most efficient way, in the most secure and trusted way? And increasingly, what you'll find is that which particular model is is helping you out in a particular task will just be abstracted. You won't even think about that. It'll just be a companion. You're just going to talk to it. It'll be a companion provided by a brand you trust. And under the hood, they might be using 200 different models to orchestrate your task. And over time, that efficiency will get better and better and better and better. And that's why I just don't think there's only three frontier models that are going to win. It's going to be an ecosystem.

• 43:21Tracy Alloway?

  This is, I know you don't want my take, Joe, but this is my-

• 43:23Joe Weisenthal?

  No, I actually do. I want your take.

• 43:24Tracy Alloway?

  This is my coffee pod theory of a-

• 43:26Joe Weisenthal?

  I want your take, Tracy. I love your take.

• 43:29Tracy Alloway?

  It's, I'll save it for the outro. Actually, on this note, we have seen some headlines recently. Obviously, there's the Uber one about token spending and- I think it was the COO said he wasn't sure if the ROI was there on Uber's AI usage. And we've seen, there was a good Vox article recently about a corporate reckoning with AI spend. Since you're going out and talking to CEOs, do you see any like, has anything shifted in the past couple of months or so in the way people are thinking about the return on this initial investment or the return on spending on tokens?

• 44:04Speaker 03

  Yes, I think it's a barbell distribution. So there's two types of CEOs, broadly speaking. The first is the CEOs who are using the tools themselves. And those folks are going, aha, I understand the jagged frontier. When they understand the jagged frontier we talked about, their strategies, their questions they ask me are completely different from the CEOs who are outsourcing their understanding. They're not trying the tools. They're mostly asking their kids like, hey kiddo, this chat GPT thing like, it's good, right? And your kid is like, yeah, it's pretty good, dad. And then

• 44:40Joe Weisenthal?

  My kids think it's really dumb, by the way.

• 44:41Speaker 03

  Yeah, so that's the other thing, right? So the kids are super smart and they're using the tools and they're like, it's good at this thing, but not at that. So they understand the jagged frontier part as well.

• 44:51Joe Weisenthal?

  Actually, you know what? They think I'm dumb for using it. They're like, dad, like, you're not doing anything smart, you know? They don't think the models are dumb. They think it's dumb of me to use them.

• 44:59Speaker 03

  Exactly, they might be going, the way you're using it is not optimal. So, so the

• 45:05Joe Weisenthal?

  No, what I'm saying is my kids are six and they have no, and 10, and they have no idea about anything. And they just think I'm dumb. That's the only

• 45:11Speaker 03

  That just might be a generalizable

• 45:13Joe Weisenthal?

  That's really the only point I'm trying to make.

• 45:15Speaker 03

  I see, okay. Well, you can send them over to me anytime. I'm happy to be the fun uncle.

• 45:19Joe Weisenthal?

  Yeah, that would be great. You can show them that actually this is fun to play with.

• 45:23Speaker 03

  My wife and I are happy to host your kids anytime.

• 45:25Joe Weisenthal?

  That's really what I'm trying to get at.

• 45:28Speaker 03

  It's the summer, right? We have two nieces in London and we call it Camp Midha Shen. My last name is Midha, my wife's name is Shen. And so you're welcome to send them to Camp Midha Shen anytime.

• 45:37Tracy Alloway?

  That's amazing.

• 45:39Speaker 03

  But that's, that's the bifurcation is leaders who are actually trying the tools out. They realize they're extraordinary at some things and not at others. And so the, depending on whether you get it or not, or you're Actually getting your hands dirty or not? I find the questions are completely different.

• 45:53Joe Weisenthal?

  So this has been incredibly helpful conversation in terms of like, understanding. Basically, the problem of essentially tons of money is being spent and your thesis is that it's massively suboptimally used up and down the stack. You mentioned this, okay, you get a credit, et cetera. Like, do you actually see that being financialized in a way? I mean, okay, I bought this capacity, I have a lot of unused time. I don't always have a research idea that is going to require a big model run test. I can resell that. Is that something that you see, like, something that genuinely resembles a financial market?

• 46:30Speaker 03

  I hope not, because when you add speculation to, you know, production goods, it creates scarcity of a different kind, right? Because then you have financial traders and markets trying to trade the speculative value of the asset, and that's going to hurt a lot of our research teams in technology. On the other hand, I think that creates a need for innovation inside of the research teams. And so, one of the core operating functions we have inside of our business is a forecasting capability, where we have a team that's very similar to actually the kind of forecasting team you'd have inside of a hedge fund. We're constantly predicting demand and supply, and then we're actually procuring capacity in advance through call options on compute clusters.

• 47:15Speaker 03

  But our needs are similar to the kind of internal trading desk you'd have inside of a large steel company, right? Where they need to lock up iron ore and so on for their production needs. So I'm a big fan of efficient markets. And I'm trying to actively invest in and help entrepreneurs out and teams out who are trying to drive more efficiency in the service of more productivity in science and engineering. I'm not that thrilled about the financialization of these products if it ultimately results in more speculation. Does that make sense?

• 47:45Joe Weisenthal?

  Yeah.

• 47:46Tracy Alloway?

  I'm just curious, since you're tracking demand in that way, like, if you were going to describe the slope of demand right now versus, say, like, a year ago, is it steeper? Is it starting to plateau?

• 47:58Speaker 03

  Perpendicular.

• 47:59Tracy Alloway?

  Oh, wow. Okay.

• 48:00Speaker 03

  If you look at the compute prices of long-term rentals over the last six months, between January and now, they're trading up 2X. So we started, for example, for 2026, we started securing our capacity in January at these long-term rates. We could resell that at a 2X markup if we wanted to.

• 48:21Joe Weisenthal?

  Part of the reason that 2026 has become just totally AI has consumed everyone's mind, I think, is because people got very excited about Claude Code specifically. And but that was a that was a breakthrough at the harness level, not the model level. Right? Suddenly, like, the really excited is like, wow, this is just so fun. It's just so easy. You have a computer inside your computer. That was a harness breakthrough. Do you see, like, when you think about investment among AI labs, do you see any shift in allocation away from pure scaling and improving the model towards sort of like tooling and harnesses as a way to get more juice out of the models?

• 49:04Speaker 03

  No, I'm sorry. I have to correct you there. It was not just a harness innovation. Those two things go hand in hand. It's a symphony of improvement between it's a dialectic between the model capability and the harness. That harness was designed specifically for the capabilities that the new model was going to have. And so when you design these things, in the industry, we call this code design.

• 49:25Joe Weisenthal?

  Okay.

• 49:25Speaker 03

  So you have the harness designed side by side with the researcher who has designed the next generation capabilities in the model. And you get a little bit of visibility in where the model is going to be good. Because as I described earlier, the pipeline is actually quite predictable. Pre-training, mid-training, continuous feedback loop. Once you have that visibility, you go, aha, we specifically want to improve the capabilities on this type of task. It's going to take us about three months to get there. Start designing the harness for that improvement. By the time they show up, then you can have the harness assume that the model will be able to do X, Y, Z on its own, whereas A, B, C, it's going to need third-party tools. So then the harness says, remember that three months ago, you were terrible at understanding a spreadsheet.

• 50:10Joe Weisenthal?

  Yeah.

• 50:10Speaker 03

  So then we had to go write, like, go use a third-party tool to use a spreadsheet.

• 50:16Speaker 03

  In the last three months, what we've done is added the ability to actually reason about a spreadsheet in the model.

• 50:21Joe Weisenthal?

  In the model, not the-

• 50:22Speaker 03

  So now you don't need to use a third-party spreadsheet.

• 50:24Joe Weisenthal?

  Okay.

• 50:24Speaker 03

  And so then the harness gets updated to say, "Don't go out and use a third-party spreadsheet," which, by the way, collapses the time required to do that task by like, sometimes a minute to two minutes. Now, suddenly, I've improved the user experience, and that's when things really sing. It's when both of those parts, the model and the harness, are co-designed to create a symphony. Does that make sense?

• 50:43Joe Weisenthal?

  Yeah, absolutely.

• 50:45Tracy Alloway?

  All right, Angene Midha of AMP PBC, thank you so much for coming on Odd Lots. Really appreciate it.

• 50:51Speaker 03

  Thanks for having me.

• 50:51Tracy Alloway?

  And everyone, go out and check out the Stanford Lecture Series. It's on YouTube, right?

• 50:55Speaker 03

  It is, cs153.stanford.edu.

• 50:58Tracy Alloway?

  Perfect.

• 50:58Joe Weisenthal?

  I have a big flight coming up, so I'll watch it then.

• 51:01Speaker 03

  You should download all the lectures, there's quite a few.

• 51:03Joe Weisenthal?

  Thank you so much, Angene.

• 51:04Speaker 03

  Thanks for having me.

• 51:05Joe Weisenthal?

  That was fantastic. That was great.

• 51:18Tracy Alloway?

  All right, Joe, that was a great discussion.

• 51:20Joe Weisenthal?

  Yeah.

• 51:21Tracy Alloway?

  I should emphasize just how big a deal that lecture series actually is at Stanford. Students are beating down the door, basically, to get into that, and if it's free on YouTube, you should definitely check it out.

• 51:32Joe Weisenthal?

  I just want to establish that if I had given you the, A, the insinuation that I didn't want to hear your take, or B, the idea that I would have wanted to hear Angene's take instead of yours, I want to hear your take.

• 51:43Tracy Alloway?

  No, it's fine, Joe. I realize that most listeners are here for the guest takes. I get it. But I thought his point about the jagged frontier was an important one, and this idea that maybe the future, it's not going to be a winner-takes-all thing in terms of models. You're going to have a bunch of different models doing different things that might suit different companies, and also the idea that a lot of companies aren't going to care about which specific model they're using, they just want the cheapest one that basically gets the job done. In my mind, that sounds like more of a- Is commodified the right word?

• 52:20Joe Weisenthal?

  Comodification.

• 52:20Tracy Alloway?

  Yeah, a commodified market, right? Rather than like, oh, people are going to pay up for, as you said, the Cadillac model.

• 52:28Joe Weisenthal?

  Well, so what I would say is by in listening to Engen and Amp is that people will want a commodified service, but that under the hood, I mean, this just sounds like what she's really trying to solve, and it's very interesting. I, as a user or a company, buy a commodified service, but under the hood, the commodity has an incredible amount of variety of models through which it can route.

• 52:54Tracy Alloway?

  Sure.

• 52:55Joe Weisenthal?

  Some of which will be the Cadillac, some of it will be the current coffee cup.

• 52:59Tracy Alloway?

  Yeah, absolutely. But like, my point is maybe in terms of valuations.

• 53:04Joe Weisenthal?

  Sure.

• 53:05Tracy Alloway?

  Right? Like, if everyone is assuming that the Cadillac is going to be like, the one that everyone is going to get, and the total available market, the TAM, infamously is like, not just the world, but potentially the universe, like, that seems a stretch to me.

• 53:20Joe Weisenthal?

  Totally. And just generally, I thought that was super interesting. And the idea, this, we've done a couple of episodes recently, specifically learning more about both chip level and box level optimizations, both how many chips you're using and how well you're using ad chip. Definitely way more to do on that.

• 53:38Tracy Alloway?

  It still blows my mind that this is a problem that can be solved with software rather than like, something physical.

• 53:44Joe Weisenthal?

  Yeah.

• 53:45Tracy Alloway?

  Like, you just come up with a way to efficiently allocate the compute.

• 53:49Joe Weisenthal?

  Yeah.

• 53:50Tracy Alloway?

  Because in my mind, like, it's such a physical problem, and we've talked to, you know, previous AI market participants, like Brandon McBee at CoreWeave.

• 53:58Joe Weisenthal?

  Yeah.

• 53:59Tracy Alloway?

  And they talk about like, oh, it's difficult to standardize because of the configurations of chips and things like that. But if you could solve it just through a software system, that's pretty crazy. I guess Google's already done it.

• 54:09Joe Weisenthal?

  Yeah.

• 54:10Tracy Alloway?

  All right, shall we leave it there?

• 54:11Joe Weisenthal?

  Let's leave it there.

• 54:12Tracy Alloway?

  This has been another episode of the Odd Lots Podcast. I'm Tracy Alloway. You can follow me at Tracy Alloway.

• 54:17Joe Weisenthal?

  And I'm Jill Weisenthal. You can follow me at theStalwart. Follow our guest Anjaney Midah at Anjaney Midah. Follow our producers Carmen Rodriguez at Carmen Armad, Dashil Bennett at Dashbot, Cale Brooks at Cale Brooks, and Kevin Lozano at Kevin Lloyd Lozano.

• 54:32Tracy Alloway?

  And for more Odd Lots content, you should check out our daily newsletter. You can find that at Bloomberg.com/oddlots.

• 54:38Joe Weisenthal?

  And you can chat about all of these topics 24/7 in our Discord, discord.gg/oddlots.

• 54:44Tracy Alloway?

  And if you enjoyed this conversation, then please leave a comment or like the video, or better yet, subscribe.

• 54:50Joe Weisenthal?

  Thanks for listening.

• 55:23Speaker 04

  Congratulations, you've built a dashboard. But now comes the follow-up question, and the follow-up to the follow-up, and the dashboard for that. Meet Hex. Give everyone the power to ask their own questions, explore data, and build with AI, all in natural language. Because every answer should spark another question, limited only by your curiosity, not your data team's bandwidth. Go deeper than dashboards. Join over 2,000 leading data organizations. Visit hex.ai.

• 55:54Speaker 05

  Most meeting note takers get the easy words right, and the important ones, wrong. The client's name, the acronym. Whisperflow Note Taker is built to get the hard words right. It takes notes on any meeting platform, Zoom, Google Meet, Teams, even WhatsApp calls. Keeps every meeting searchable and connects to Claude or ChatGPT, so you can ask, "What did we agree to last Tuesday?" Free to try on Mac and Windows at whisperflow.ai. That's W-I-S-P-R-flow.ai.