(Preview) AWS History and Trainium’s AI Future, OpenAI Makes a Deal With Microsoft, Meta and the Future of Wearable Devices

Ben and Andrew react to Amazon’s impressive earnings in AI with a cliffs notes history on AWS cloud computing strategy, how Amazon is returning to that playbook in AI, and why the Trainium bets look more reasonable than ever. From there: Understanding both sides of the OpenAI and Microsoft deal this week, including why OpenAI wants to be on AWS, and why Microsoft’s conflict of interest is now resolved. At the end: Extended thoughts on Meta Display glasses, the future of AR devices, the mother of all patent lawsuits, as well as a few Apple follow-ups, and an eye surgery epiphany.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Andrew Sharp?

  Hello and welcome to a free preview of Sharp Tech.

• 0:08Andrew Sharp?

  Hello and welcome back to another episode of Sharp Tech. I'm Andrew Sharp and on the other line, Ben Thompson. Ben, how you doing?

• 0:19Ben Thompson?

  Perturbed, Andrew. I have a new computer. So I, it's complicated, but all the recordings happening on my main computer. But I need a computer in front of me for the rundown, which may or may not be accurate. You sometimes just suddenly like pull out things that are not there. And I'm like, what the?

• 0:36Andrew Sharp?

  I gotta keep you on your toes.

• 0:38Ben Thompson?

  And you're like, oh, sorry.

• 0:39Andrew Sharp?

  Make you sweat a little bit.

• 0:40Ben Thompson?

  Yeah, you're like, oh, I made my draft in Gmail, which I'm reading out of. I'm like, I'm using Microsoft Word for your sake. It's unbelievable. But so I have my, so what happened was I have had, I think I talked about this. I have two computers. I have my MacBook Pro that mostly stays on my desk. If I'm traveling and working, I will bring it with me. But it is basically a desktop computer and it's great. The MacBook Air is amazing for sort of carrying around, having with you. I had an M2 MacBook Air for a long time. Love it. Amazing computer. Unfortunately, because I carry it around all the time, it is like the baseball games and practices. I'm working in the car. It's gotten dropped. At one time it was left in the backseat of the car. There was a boys in there. It's like it was all stomped on.

• 1:24Ben Thompson?

  It's not, you know, but that's fine. It's like, I mean, it's not fine, but you know, it's okay. There's nothing important on there. The problem is it started kernel panicking. So last week, I think we talked about this. My computer crashed.

• 1:36Ben Thompson?

  The crashes are accelerating in frequency.

• 1:40Andrew Sharp?

  Interesting.

• 1:41Ben Thompson?

  Well, what happens is especially if you drop it, stuff inside gets loose and it starts sort of like shorting out. And like, that's like an unrecoverable error. The whole thing is just gonna like, like, God. So I was pretty sure, but yesterday it got really bad. Like I need a computer for the podcast recording tomorrow. So went to the Apple Store, just bought a new MacBook Air.

• 2:01Andrew Sharp?

  Okay.

• 2:02Ben Thompson?

  The problem is the MacBook Air runs Tahoe, the new, the new Mac OS. And everyone's been complaining about Tahoe and I validate most of those complaints. Yeah. But there's a point no one's mentioned, which is. I kept feeling like it was weirdly dim. And I'm like, what's like, I checked the specs. Did they change the screen from the M2 MacBook? Of course, no, they didn't make a dimmer screen. What turns out my Tahoe complaint, I just want to add it to the list of everyone else.

• 2:31Andrew Sharp?

  Okay.

• 2:32Ben Thompson?

  The interface is so white. It's white on white on white on white everywhere. That it makes the whole thing seem dim. It's like it's, it's a really bizarre effect. Yes, I could do dark mode. Unfortunately, I'm 45 years old. We'll circle back to me being 45 years old. My eyes set at the end. No dark mode for me. So anyhow, I just want to register, add to the whitening of Tahoe complaints. Too much white contrast, please.

• 2:58Andrew Sharp?

  White, indeed. Well, I have a reveal live on the podcast. I too got a new computer this week because my computer, I have a MacBook Pro that I keep on my desk at all times. That's my podcasting machine. And I use a MacBook Air all over my house. It's what I write on. It's what I prep for shows on. I'm using it on every floor of my house. And when you have two kids running around, they tend to pick it up and drop it and do all kinds of unhelpful things with your MacBook Air. So I had to hide my computer by my bed one night. And then I wound up stepping on that computer as I got out of bed.

• 3:43Andrew Sharp?

  And so it just completely shorted out out of nowhere. Again, some screws probably broke loose and then a day or two later, it just stopped working entirely. So I got a MacBook Air and I will say, I feel the software pain with my new MacBook Air. Like there are so many things that I've had to resort to chat GPT for, to try to fix on my MacBook, which should not be how these machines work.

• 4:11Ben Thompson?

  Right, they should be getting like nicer to use over time and they're, they're going in the opposite direction.

• 4:15Andrew Sharp?

  More complicated, more frustrating. It is what it is. Still a great machine and hopefully won't be a broken machine over the next couple of years, because I'm excited about the new chips and I do love the air. In any event, we're.

• 4:30Ben Thompson?

  Did you consider getting a, did you consider getting a MacBook deal? Cause I'm sure someone, someone is going to ask.

• 4:34Andrew Sharp?

  No, I did not consider getting a MacBook Neo because I'm working on this like every single day of my life and will be for years to come. So I feel like it's worth paying the premium. I didn't go for the 24 gigs of unified memory. I was content with 16 gigs of unified memory, although I felt kind of lame. Did you go 24 there?

• 4:56Ben Thompson?

  Um, no, I got the base model 512, the absolute base model, cheapest chip, least memory. Again, the, the, the good thing is, it's actually very important. Um, there was nothing important on my old computer, which was good because actually it was getting so bad by the end. It just really accelerated the last day. It took me like three tries to erase it because it kept kind of panicking before I could erase everything. Um, so if I had lots of data on there.

• 5:24Andrew Sharp?

  Yeah.

• 5:25Ben Thompson?

  It was rough. Uh, but no, this is like a netbook. Everything's online. It's totally disposable. Um, I like the, I like the little stuff, like the light up keys. Like I, one of my biggest use cases is my son's at baseball practice, which is a long ways away and I sit in the car and I work. And so like having light up keys is actually useful.

• 5:44Andrew Sharp?

  It's a delight. Yeah.

• 5:46Ben Thompson?

  And like the ambient light adjustment, like I love that feature. Neo doesn't have it, but I actually had to turn that off because the whiteness of the interface meant every change in brightness of the screen was, felt like it was like putting a black pane of glass over the whole thing. It was just so dramatic, the sort of shift. So, um, maybe I could have gotten a Neo after all.

• 6:09Andrew Sharp?

  Fix the whiteness. Yeah. Bring, bring Forrestal back and he can fix the whiteness for you. Uh, but a couple of new computer buddies here on the show today. Very exciting stuff. Yes. Well, we are not going to be talking new laptops. We are going to be talking Amazon and OpenAI on today's episode. But before we get to Amazon and OpenAI,

• 6:31Andrew Sharp?

  Amazon did release its earnings later in the week, along with the rest of big tech. We're gonna table some of that.

• 6:38Ben Thompson?

  Yeah, terrible. So you have Amazon, Google, Meta and Microsoft, all in the same day, along several other companies that I mentioned in. But I, this is the first time I've had one of these like days where everyone's there. And also, I thought like, oh, this interview running on Tuesday is gonna be great, because then I could at least hit one of the earnings Wednesday night for Thursday. And I forgot, being in the US, like, the transcripts of these don't come out like, so the, I wrote about Amazon in my update on Thursday, in part because I've been talking about Amazon for the last couple of weeks, so that made sense. And also, their transcript dropped at 9:01 PM. So it was the first one to drop. I think Google came out like 11, and then like, I don't know.

• 7:22Ben Thompson?

  So I haven't read the Meta or Microsoft ones. I glanced at the Google one. But yeah, that's, we're gonna be focusing mostly on Amazon, both for topical reasons and also, I can't stay up till four in the morning, waiting for transcripts.

• 7:34Andrew Sharp?

  Well, yes, I look forward to immersing ourselves in the Meta earnings, the Google earnings.

• 7:40Ben Thompson?

  Yeah, Taiwan, they were always there. It was great. Another time zone advantage of being in Asia. But what are you gonna do?

• 7:46Andrew Sharp?

  And by the way, is that normal for all of those companies to release earnings on the same day? Because I remember it being more staggered, but maybe I misremembered.

• 7:55Ben Thompson?

  They, they, they're generally all bunched together. I don't actually, maybe so can emails. I don't actually know how that works. How and when they announce, usually the announcement of the date is usually like, seven to 10 days before, it seems.

• 8:10Andrew Sharp?

  Okay.

• 8:11Ben Thompson?

  And obviously, that like, that's a very, earnings has always been like a core thing for stratechery. And so it's something I'm always aware of and like, thinking about as far as scheduling. And you see this, not only were they all in one day, but also lately, they've always been on Wednesdays. And then I'm like, I. Very frustrating for my publicity schedule.

• 8:30Andrew Sharp?

  Well, it's not bad, though. From a Strategery editorial standpoint, it gives you a couple days to sort of mull what you're hearing and then come back on Monday.

• 8:39Ben Thompson?

  Yeah, but I don't know. I mean, every time there's a Meta 10% drop, I'm like, oh, this is my sweet spot. I want to get on it right away. But, um.

• 8:46Andrew Sharp?

  Well, good news. The sweet spot will be waiting for you on Monday because Meta's taking some hits. For now, though, we will talk about Amazon. And I'll read from the Wall Street Journal. "Amazon said Wednesday that its edge in cloud computing and aggressive investment in new data centers is translating into a surge in its artificial intelligence business. Chief Executive Andy Jassy said that revenue from the company's Amazon Web Services grew 28%, the fastest pace since 2022, in part because many customers building new AI agents want them stored in the same spot where they maintain their other cloud services and data. Revenue for the period rose 17% to $181.5 billion, while net profit

• 9:31Andrew Sharp?

  increased a sharp 77% to $30.3 billion, which Amazon attributed to pre-tax income from its investment in Anthropic. Both figures beat analyst estimates. Shares were up more than 4% in after-hours trading." So Ben, big picture, what is the story with these earnings? Has Trainium risen from the dead? Does the AWS AI story look better today than it did 12 months ago?

• 10:02Ben Thompson?

  Yeah, I mean, I see Amazon is, I actually think ended up down so far today. So who knows what's going on there. But the, um, I, I thought, I've been thinking about Amazon. They're certainly one of the many participants in the ongoing AI soap opera. Who's up, who's down? Everyone has their, has their, has their time. And I think a, there was a lot of concern, I think really crystallized in a semi-analysis article a couple of years ago, talking about Amazon is like screwed in, for AI in the long run because of...

• 10:34Ben Thompson?

  They won so hard, as it were, in the data center. And I, I wrote about this a bit. I was talking about Amazon and Apple together. The two big winners from the cloud, can they actually adjust for the AI era? If it requires sort of different approaches? And the framing in that someone else's article was, well, Amazon has invested heavily, not just, of course, they were first, but in really maximizing the, their position in a commodity market. And what I mean by that is there's, we've talked about this on this podcast. There's two ways to make money. The one everyone tech thinks about is make something that's differentiated that has a moat and sell it for a premium. That is the Apple model, like, like, and everyone is like, I was at some sort of startup event

• 11:18Ben Thompson?

  a couple of weeks ago, and there is talk about moats, moats, moats, moats, moats, moats. Like, that's what everyone in Silicon Valley thinks about. It's the most intuitive, I think, but It's also what Nvidia is doing. Well, the real, yeah, exactly. The real world where a lot of company, like, a lot of products are commodities. They are highly substitutable. You can, if you don't get something there, you can get it somewhere else. How do you make money in that world? In that world, you make money by having a superior cost structure. You can deliver the commodity at a lower price than everyone else. And the price for your commodity is not based on your cost structure. You can always outcompete everyone by offering a lower price. But if, assuming there's sufficient demand, the market clearing price is, you know,

• 11:59Ben Thompson?

  if you have a perfect balance, assume you like, the demand is applying or can scale perfectly. The market clearing price is going to be the marginal cost to produce the commodity for the highest cost provider. So if it, if, if making a widget and the widgets are widely available, the, the, the price will be whoever it costs, company XYZ, $10 to make the widget, then all the widgets in the world are sold for $10. Now, again, there's lots of variables here. Supply and demand vary. There's elasticity on the price, how many people want to buy? But just if you're looking at that, that segment generally, the, for a commodity where it's totally substitutable across companies, the market clearing price is going to be the cost structure for the marginal cost of the most expensive provider. Right?

• 12:43Ben Thompson?

  So if you have four providers, it costs provider one, $10 to make it. Provider two, $9 to make it. Provider three, $8 to make it. And provider four, $7 to make it. The company that has sustainable profits in the long run is The seven dollars. That's right. They're making three dollars of profit on every widget. And that's, that's how you could make a lot of money that way. Like, right. And so, uh, cause everyone needs the commodity. And then you also, you're also in a strong competitive position because you can lower the price. If the price goes down, company one will go out of business and then suddenly then supply will go down and the price will go back up. Like, so you, you're just in a very,

• 13:26Ben Thompson?

  the lower your cost structure in the industry, the better you're at. And the low, if you're at the lowest end, you have a lot of power. Uh, and, and, uh, can be very, very sustainably profitable. So that was Amazon in the cloud computing age. They, it's not just that they were first to build out the cloud, but they were first to really, uh, invest in a few different things. So one was around, obviously their own processors. So they make Graviton processors. The number one use case for Graviton processors, which you could go as a customer and get an instance of Graviton processors, particularly in the early years, they sucked. So no one would want to do that. But, uh, Amazon didn't just offer infrastructure.

• 14:11Ben Thompson?

  You can go buy a processor. They offered platforms. So you can get the, the Redshift, the Amazon database service, right? And in this case, you're not actually, you don't know what the processor is. You're just getting database as a service from Amazon. And guess what Amazon's probably powering Redshift with? Graviton processors. Like, so their, their cost to serve it is diminished because they're using much cheaper processors and Graviton's gotten better and better over time. So, but because they can abstract that away with, especially with their platform stuff, they can have a sustainable cost advantage there. They also did a lot with networking and they built this entire cope. Actually, their first, probably most important chip was this coprocessor where

• 14:49Ben Thompson?

  With a server, you have like a virtual machine, like you have to actually run the the actual server. And then on top of that, you have all these virtual machines that appear as a computer to the client, but actually one chip, one computer is servicing hundreds or thousands of clients that are actually all sitting on top of it on their own little virtual machine. You need to actually run the computer, though, and you need what's called a hypervisor to actually manage all those virtual machines. Amazon took all that off the main chip and had a side chip that basically handled all the networking and handled all the management of the system so that the big Intel chips that power, that's why you had these Intel chips with tons and tons of cores, because

• 15:32Ben Thompson?

  you could, because all those, one core could be dedicated to a particular virtual machine. You could keep all that Intel, expensive Intel chip capacity to run more hypervisor, run more virtual machines, because you offloaded just sort of the janitorial aspects of the server to this other chip. And then it had all the network, you know, that sort of thing. It's called Nitro. So this gave them a sustainable cost advantage where Microsoft is offering an instance that runs on Intel chips. Amazon is offering an instance that runs on Intel chips. But because Amazon can fit like 20% more virtual machines on one chip, that means their cost

• 16:13Ben Thompson?

  to serve is structurally lower than Microsoft's was, because they have this whole cold processor sort of thing. And this is how, this is just, this is a great example of Amazon in a nutshell and why they're, why they are the anti-Apple. And I mean that in a very positive sense. Apple's all the way on the extreme of, we're going to highly differentiate our products, like we're going to keep our OS, it's going to be exclusive to our hardware, we're going to have a developer, so we get network effects ecosystem, we're going to have brand, we're going to have all those things that make Apple, Apple, that gives us a structurally sustainable.

• 16:48Andrew Sharp?

  And we're going to maintain profit margins for 20 years.

• 16:52Ben Thompson?

  That's right.

• 16:53Andrew Sharp?

  Despite all the competition.

• 16:53Ben Thompson?

  Amazon's all the way on the opposite side. They are going to invest a ton and ton of money over years to build structural cost advantages. So in commodity markets, so they can do things.

• 17:07Andrew Sharp?

  Which they did in retail, too. Yeah.

• 17:09Ben Thompson?

  They did it in retail.

• 17:09Andrew Sharp?

  It's the same playbook.

• 17:11Ben Thompson?

  That's why they're watching satellites, right?

• 17:13Andrew Sharp?

  So, one question, though, as far as that history is concerned, as Amazon optimized for cost structure and serving it more efficiently than some of its competition, did they then charge lower rates and take market share that way? Is that how AWS took over the world? Or was it something else?

• 17:33Ben Thompson?

  Well, they were just first in general, number one. Number two, they've had, they've always had like way more features than everyone else just because they keep building features. And like, it's an 80/20 thing where everyone complains about AWS and all this stuff's hard to use. And they're like, oh, they should cut everything else, but keep this one thing that I need. The one thing everyone needs is different than everybody else, right?

• 17:52Andrew Sharp?

  Yeah.

• 17:52Ben Thompson?

  And everyone goes out and they're like, I am not going to get locked into a cloud. I'm just going to use commodity hardware, a basic compute instance, a basic storage instance, so I could take it from Amazon and go to Azure if I want to, or go to GCP. And then you're like, you're developing, you're like, oh, I could spend a few months building this, or I could just use this API that's hopefully there from Amazon that will solve this one problem. Oh, we're just doing this one thing. Don't do it too often. We don't want to get locked in. And then you fast forward and you're totally locked in. You're all using all their services. You're not going anywhere. This is actually a super important point. But if you're not going anywhere, Amazon has pricing power over you. They have features, they have pricing power over you. And if there's not enough demand,

• 18:34Ben Thompson?

  or there's not, or if there's not enough supply in the market, they're going to have, you know, a lot of pricing power. But this is actually an important point. The big shock. When AWS was revealed and I call it the AWS IPO like a decade ago, was everyone sort of, they kind of got the cloud, but assumed it was going to be super low margin. And it turned out that actually, no, the margins were great, especially for it from an Amazon perspective. You know, I think at the time, I want to say it was something like 17 to 19%. People thought it'd be one to two. Now it's in like the 30s or something like that, or maybe, maybe even higher. Um, so

• 19:10Andrew Sharp?

  And much higher margins than the retail business.

• 19:12Ben Thompson?

  Right. It turns out you could get not, so Amazon, yes, I'm talking about this very compelling total commodity market. People could switch wherever. The reality is everything's a mix and there are boats and you do lock people in and you do. And so, but yes, you can also offer stuff super cheaply if need be in a competitive, like in a bank offer. Like they can go, they go to companies. That's why they give, they go startups like hundreds of thousands of dollars in credits. Like you're not even paying anything to Amazon unless you're actually a substantial business. They will make long-term deals with companies. Okay. Sign up for three years. We'll give you uh, like an 80% discount, like just because you're locked in or, and they could do that because they have this, this capacity to do so.

• 19:53Andrew Sharp?

  That's one of the great advantages. Okay. So AI, I'm going to repeat my question. Has Trainium risen from the dead?

• 19:59Ben Thompson?

  Okay.

• 19:59Andrew Sharp?

  So AWS AI story was better.

• 20:02Ben Thompson?

  There's a, there's a very, we're, we're doing a wandering here and we're going to make, make it back. Um, so the, so the concern that, that semi-analysis article raised, I'll put a link in the show notes to this article. It was, it was really interesting at the time is Amazon is so committed to and locked into the proprietary networking in particular, that they can get like Nvidia stuff and they can plug it in, like their, their, their, I think it was HGX racks or whatever. And so you could access an Nvidia instance, but actually the future is these huge, it, it's not just a chip. And it's not just a rack or a, a, a, a computer, eight, we would do HGX, I think we're eight GPUs. It's entire racks and it's not just entire racks. It's entire data centers that are linked together.

• 20:45Ben Thompson?

  This is what Jensen Huang goes on and on about, about Nvidia is like, it's like the entire data center as a GPU. And it's this huge Sort of thing and that that is totally Amazon's whole strategy doesn't work there.

• 20:59Ben Thompson?

  Let's not do that. You have to do all Nvidia's networking. You have to do. And so the concern that was raised that article is Amazon is so committed to their strategy, particularly in terms of networking, that they're going to miss, fall behind more and more on AI. As networking becomes more and more important and this operating at systems aspect becomes more and more important. And

• 21:25Ben Thompson?

  it's not that that article was wrong. Okay. That is all true in terms of training. Training needs this horizontal scaling, this super low latency between chips, between systems. And Amazon has never been a big player in terms of training. It like, it's been Microsoft, it's been Oracle, it's been, you know, Elon Musk and XAI building their own data center that is dedicated to doing this. And what Amazon got right is training for a long time dominated the amount of compute that was used, even post chat GPT for several years. It like more like 60% of of chips in the world are being used for training, not for inference.

• 22:11Ben Thompson?

  But when and if inference came along, the needs would be different. In an inference world, you're mostly keeping everything within one chip. It's like you want to get, you want to just, this is where the model distillation stuff comes in. You want to get everything on one chip and you not even worry about too much of the horizontal, horizontal networking. And it's all about like batch size and getting stuff in, like all these people are coming in and this is where the CPU aspect gets more important because you have, there's more orchestration, like this job goes there, this goes there, you're trying to just keep these GPUs filled. What you're not doing is running these huge horizontal clusters that go, that go across like

• 22:51Ben Thompson?

  just, it is a different struck, it's a different data center setup to be serving inference than it is to be doing training.

• 22:58Andrew Sharp?

  Sure, and it makes intuitive sense that as far as inference is concerned, that would be a more commoditized space where efficiency matters more than performance.

• 23:08Ben Thompson?

  Right, well, that's where you're actually selling, right? Right, that's right. You're actually selling, and in theory, if all this training is gonna be worth it, it has to shrink as a proportion of compute because the training manifests in a model that is used for inference.

• 23:26Andrew Sharp?

  All right, and that is the end of the free preview. If you'd like to hear more from Ben and I, there are links to subscribe in the show notes, or you can also go to sharptech.fm. Either option will get you access to a personalized feed that has all the shows we do every week, plus lots more great content from Certechery and the Certechery Plus bundle. Check it out, and if you've got feedback, please email us at [email protected].