Cloud Strategy in the AI Era with Matt Garman, CEO of AWS

In this episode of No Priors, hosts Sarah and Elad are joined by Matt Garman, the CEO of Amazon Web Services. They talk about the evolution of Amazon Web Services (AWS) from its inception to its current position as a major player in cloud computing and AI infrastructure. In this episode they touch on AI commuting hardware, partnerships with AI startups, and the challenges of scaling for AI workloads. Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil Show Notes: (00:00) Introduction (00:23) Matt’s early days at Amazon (02:53) Early conception of AWS (06:36) Understanding the full opportunity of cloud compute (12:21) Blockers to cloud migration (14:19) AWS reaction to Gen AI (18:04) First-party models at hyperscalers (20:18) AWS point of view on open source (22:46) Grounding and knowledge bases (26:07) Semiconductors and data center capacity for AI workloads (31:15) Infrastructure investment for AI startups (33:18) Value creation in the AI ecosystem (36:22) Enterprise adoption (38:48) Near-future predictions for AWS usage (41:25) AWS’s role for startups

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  I don't know how many times in the first couple of years I had to explain why a bookseller was offering compute services and storage services. There's not a lot of business opportunities that are as big as cloud computing and as potentially transformational. Our original AWS thesis was we take care of the muck so you don't have to. I want my customers to want to run on us.

• 0:22Sarah Guo?

  Hi listeners and welcome back to No Priors. Today, we're talking to Matt Garman, who took over as CEO of AWS in May. Matt has been with AWS since it was a $0 billion business to today's $100 billion run rate business. Welcome, Matt.

• 0:37Speaker 02

  Matt, thanks so much for joining us today. It's a real pleasure to have you. One thing that I think is really fascinating is you actually started on AWS in the very early days, it was just getting started and you did that as an intern while you were getting your MBA. Can you tell us a little bit about the origins of AWS as well as your own involvement with it?

• 0:52Speaker 00

  Sure. So it was in 2005, I did my business school internship at Amazon. And as we were looking around for projects, I talked, actually at the time, to Andy Jassy and he was telling me that he was starting a new business inside of Amazon that was technology focused and he couldn't tell me about it before I started. And I thought it sounded interesting, so I joined him and worked on that. And it was AWS pre-launch. I got to work on that as intern. It was a super cool opportunity. I came back full-time effectively as the first product manager for AWS and I'm now on year 18 or just, you know, so been working on the business the entire time. And, you know, it's a fascinating space.

• 1:33Speaker 00

  Even back then, we saw the potential of what AWS could be. Obviously, that was, you know, it was a startup, right, inside of Amazon. And so it was, you know, we had visions of what we thought it could be. But, you know, there's a lot of hard work and some good luck and some things that have gone right for us and a great team over the last couple of decades to go build AWS. And the fascinating thing today is it's still in the very early stages of what the business can be. There's not a lot of business opportunities that are as big as cloud computing and as potentially transformational. To every industry out there. And so it's an exciting place to work just as much as it was in 2005, when I was an intern, kind of writing the original business plan of who,

• 2:18Speaker 00

  when we first launched our services, which companies might possibly be interested in using these things.

• 2:23Speaker 02

  What were the other projects that you were offered at the time?

• 2:26Speaker 00

  I'd worked at startups before going to business school. And part of what I was looking for is I wanted to see how larger companies did new projects and kind of entrepreneurship, if you will, inside. So there's a couple of few technology companies that I looked at and was excited about Amazon. And there was a couple of other kind of retail businesses that, I mean, there's the internships were mostly in retail. And, you know, there's some new categories that they were starting up and things like that that could have been interesting. But I always also knew I wanted to go back to technology. So this was far and away. It's what convinced me to come to Amazon because it seemed so exciting.

• 3:02Speaker 02

  How fleshed out were their original plans? Because I ended up using AWS for my first startup in 2007 or 2008. And it was pretty new. And there's a small subset of services at the time.

• 3:12Speaker 00

  Yeah.

• 3:13Speaker 02

  And to be honest, at the time, Amazon wasn't thought of as as deep of a technology company as it both was and is. And AWS was really kind of like, wow, Amazon is doing this thing that, you know, at the time, you thought maybe it would be natural for Google or somebody else to have started with. And so I'm a little bit curious about about that sort of early conception and roadmap and what was in place and what you added over time.

• 3:35Speaker 00

  I don't know how many times in the first couple of years, I had to explain why a bookseller was was offering compute services and storage services. I think the approach that we took worked out to be one of the key pieces to our early success. I think if you look at some of those others like Google and later Microsoft, they kind of went at this space. First, we were the first ones out there that had anything like this. But even soon after that, I think they went after the space like that was going to force the developers to change how they build applications and kind of build them in a new way. And we went after it as in we're going to build building blocks and let developers and builders go build interesting things. And so our view, and this started really from internal Amazon.

• 4:16Speaker 00

  If you scroll back all the way to 2003 or so, Jeff Bezos basically mandated across the company that in order to move from a big monolithic stack that wasn't going to scale anymore for Amazon, we had to move everything to services. And that was really a lot of the impetus for us looking at after that. And we saw the success of that. We said, well, maybe this would work for other people who are likely going to have the same struggle that we're going to have as Amazon. And so we thought, starting from first principles, what are the things that we would need to go build to help people build a company? And we knew people needed compute. We knew people needed storage. We knew people needed databases. And so we built those things, right? And we didn't force people to change how they architected, right? When we gave you a virtual Linux server, when you logged in, it was just a Linux server.

• 5:01Speaker 00

  Like it wasn't magic, you know, it auto scaled and you got it in 30 seconds, which was pretty awesome at the time, where normally it was going to be six months or something for you to go get a server. And this is when everybody was like, when you had a startup, you had to go to Exadata and buy racks of servers in order to get your startup off the ground. And so, so that was obviously transformational. But once you actually got the infrastructure, it kind of operated the same way that you were used to. S3 was a little bit different, right? The put get delete was a little bit different, but the the storage concept wasn't too different. And so I think that was one of the things that really helped us move quickly and helped people not have to have a total paradigm shift of how they develop. It was more how they get the infrastructure to develop on. And then over time, we could add things like Lambda, or we could add things like, you

• 5:44Speaker 00

  know, Bedrock and AI services and other things like that, that are more different than people weren't used to, but but kind of starting out with some of those things people were familiar with and they could instantly grab on and and build quickly without having to learn a new concept, I think was a really big accelerator for us at the beginning.

• 5:59Speaker 02

  Yeah, it was huge. I remember when it first came out, I thought there was a few things that were done really well. One, to your point is just, you know, providing really basic building blocks. Second was just the iterative nature of it. Where you launched with a small number of services and then you kept adding stuff that were kind of the obvious next steps and you know, early on, people would wonder, or at least I in the startup community wondered, will they end up with everything I need? And you very quickly did. But to your point, you also built it in a way where before that, I think most people building today have no idea. To your point, you literally would have to set up your own server racks or find somebody who would do it for you. It was hugely painful. You had a whole team that was doing that for every company. And in some cases, it caused real problems. My company eventually got bought by Twitter, my first startup. And then at Twitter, we ran into real issues in terms of data center capacity and planning

• 6:44Speaker 02

  and all these things that you can now just scale on Amazon for. So it's, it's pretty radical in terms of what's been enabled for startups and the decrease in headcount per company that's needed to associate with that.

• 6:53Sarah Guo?

  I remember as late as 2010, 2015, maybe you guys are still having this conversation with the largest customers that, you know, if you're, let's say, a large financial, you had like this whole platform team, several different iterations of it. And the line I would get from people would be like, we're never going to do that public cloud thing from a security perspective. You can't compete with us on cost. Our platform is better. Like, you know, it's not reliable. And just like, there's a very strong orientation toward skepticism of this, you know, even in 2010, like upstart company versus like, oh, we

• 7:33Sarah Guo?

  are X large financial customer. I think, I think almost everybody has seen the light at this point. But, you know, you, you described Amazon, AWS in particular, as still being quite, quite early in that journey. And I think one thing that people, even those in the tech industry who experience more exponential growth than most, like thinking about how large these markets become is really challenging. And we might be, you know, I think we're at one of the, at the beginning of one of those cycles again with AI. We'll, we'll come to that. Like, at what point did AWS internally, did you guys like know that this was going to be that large? And how did you talk about size and opportunity early on?

• 8:15Speaker 02

  And just to give scale real quick on this, I think you guys went from something like 500 million or so in 2010 to about 90 billion last year in terms of revenue for AWS. That's right. And so it's just, it's this insane. Yeah. Ramp of eighty nine and a half billion dollars in incremental revenue over fourteen years or whatever it is. So that's amazing.

• 8:32Speaker 00

  Yeah, it's easy to get caught up in those big numbers and that the fact that it's so early. And so, so, you know, back to your question on when we kind of knew we're, we were kind of on that track. I remember it's probably two thousand eight or two thousand nine. I can't remember exactly, but I definitely remember the trip. We went on a trip to New York and actually there's a bunch of financial services companies that Goldman Sachs is and JP Morgan's and, and they, they wanted to learn about what is this cloud computing thing. And, and they mostly were just fact finding. They were trying to get information from us on how they could more efficiently run their internal IT systems. I'm pretty sure. But we went in there. We're like, well, it's worth a shot. And, and, you know, they were, they were like, okay, look, our workloads are never going to run you like maybe someday

• 9:13Speaker 00

  our website will run you or something like that. But, but never any of our internal workloads. And, you know, and we listened to them and we said, why? And then we said, awesome. Tell us why. And they're okay. We have to have this compliance. You have to meet this rule. You have to meet this thing. We have people to do these audits, blah, blah, blah. And we spent the next decade just checking those things off the list. And, and we basically never said, you know, our part of what we did is we said, we want to know what are the most difficult workloads to run? What are the hardest things to do? And let's go solve those. Because if I can solve the, you know, JPMC running an AWS, if I can run, solve the US intelligence agency running an AWS, like the, the reasons for a regular

• 9:54Speaker 00

  other companies are diminishingly small. And, and so that was kind of our mentality as excited. And I love getting the startups running up to us, but the big enterprises can kind of easily dismiss. You're like, well, it's a startup, you know, they don't have all these things. And so what we did is we just did both. We said, look, we're going to go in as much business with startups as possible as we can. And we're going to check off the list, all the things that are going to help JPMC, or the US government, or Pfizer, or whoever it is running us in a, in a secure, safe way. And, and that's what we did. And, and, you know, today, those are all huge customers of ours. After I was in AWS for about a year, I remember sitting down with a friend of mine. Uh, who was a business school, uh, classmate of mine is also working in a different part of Amazon. And he was like, oh, how's that AWS thing going? And I was like, you know what?

• 10:39Speaker 00

  I think this thing could be a billion dollar business. And he looked at me and he's like, he's like, dude, do you know how big a billion dollars is? Like, that seems unlikely. I was like, no, no, seriously. I think we could get to be a billion dollar business. Um, so, uh, you know, I, we, we knew it was going to be successful and we didn't know, you know, quite how successful or when I would say. Um, and so, you know, now that we're at a hundred billion run rate, uh, you look at, you, you still go out there and I think 85% of workloads are still running on prem today by most estimations, somewhere in that range, you know, pick your number, whether it's 80 to 90, whatever it is, like, that's enormous. Like if I have, there's still 10x growth of just existing workloads. Forget all the new gen AI workloads that are being created every day. These are just

• 11:24Speaker 00

  existing workloads to move. There's a, there's a 10x number in there. Um, and so that, that business is massive. And I think there's a couple of, you know, one of the big inflection points we saw is we went after the, the intelligence agencies for the US government and we won that contract and it was secret. Um, and it, you know, we, we pushed really hard to go in that. It was against all the incumbents, HPs and IBMs and on Oracles and whatever. And we won the contract for, to do this cloud workload and, uh, but it was, um, confidential and we couldn't share with anybody that we were doing it. And, uh, IBM sued.

• 11:59Sarah Guo?

  Yeah, I remember this.

• 11:59Speaker 00

  To, uh, because they, they said it's unfair, it wasn't, and, and so then that became, so then it became public that we won this deal. And then the intelligence agencies went out to public and said, no, AWS is the most technically sophisticated. They have the most capabilities. They're the most operationally strongest and that's who we're going to go with. And so we had the government out there now saying that we're the best and most technically capable to run these high, uh, highly important workloads. And, and that was a huge, um, kind of stamp of approval for us, if you will. And, and I do think that that's one of those moments that in some ways we kind of got lucky that they sued or otherwise it might still be secret that, uh, that we were doing that. And I do think that that helped gain a lot of credibility in the enterprises.

• 12:38Speaker 02

  What do you think is, um, you mentioned that, you know, 80% of workloads still haven't migrated over. Um, what, what do you think are the main blockers to that today? Is it just momentum? Are there specific features? Are there big things still to build?

• 12:50Speaker 00

  There's some technologies that, you know, I think it, look, if I had an easy button, and, and by the way, we're trying to build an easy button, but, uh, but if I had an easy button that would just migrate mainframes to a Modern cloud architecture today, almost everyone will push that button, but it doesn't quite exist today. And it's not as simple as like, great, I'll go run your mainframe in the cloud. Like, that's not what customers want. They want to actually modernize those workloads and have them into, you know, microservices and canaryize workloads and other things like that. So that's one, is there's just a bunch of workloads like that that are old and they're customers running a big SAP thing and they want to move it to the cloud, but it just takes time because it's tied into a bunch of other things like that. There's also a bunch of workloads that as you get out of core IT workloads that

• 13:33Speaker 00

  are in line of business that are the next set of things. And whether that's, you know, say telco workloads, right, that are that are running kind of the 5G infrastructure around the world. We've slowly been moving those to the cloud and helping those customers get that flexibility and that agility of running those in the cloud as well, but they're slower to move. If you think about all the compute that runs factories out there today on factory floors, most of those have not been modernized. Most of those are thinking, and there's a huge opportunity, by the way, for AI to totally revolutionize how you think about factory workflows and efficiency there. But a lot of that hasn't moved. And so some of this is, you know, there's on-prem infrastructure that people

• 14:16Speaker 00

  are still amortizing. There's people who's there's still people whose job it is to run on-prem data centers. And so they're kind of resistant to moving things. So, you know, there's a bunch of factors in there. And so some of it is just takes time. Some of it is technology pieces. Some of that is we still have stuff to go build and innovate and help make it easier for customers to do that.

• 14:36Sarah Guo?

  I'd love to hear about just the initial investigation of like generative AI as a technology change and like how AWS began to react to it and invest in it. Because to some degree, it puts us all back in the like on-prem colo era of the world where to get one of these, you know, if you're doing any sort of real pre-training, to get your startup off the ground, you're back to, I guess I'll buy a bunch of DGX boxes somewhere. And like, I need to think about the cost and management of that.

• 15:07Speaker 00

  Well, I watch it. Most people are still buying those, but in the cloud, but it is kind of a, it's not a serverless type of a thing. It's, you know, most people are still not buying, you know, H100s and hosting them in a colo or anything like that. And increasingly, I think that's going to get harder and harder as you move to liquid cooling and larger clusters. But You know, it is, it's a super interesting space. I think we, we've been working on this space for how many years now? And look, we, we've been investing in AI broadly for the last 10 years. And that's why we started five or six years ago, investing at the infrastructure layer and building our own processors, because we, we knew this was coming, we saw this path coming, and we knew that that's also not a short term investment. So it's one of those things you got to invest way ahead. And then we were investing in, in

• 15:51Speaker 00

  building generative AI models. And then, you know, OpenAI kind of made a generational leap forward of what they're able to do and what's possible. And then many people have talked about this, but it, it really, in some ways, was a discovery as much as anything about just what was possible and, and kind of unleashed a new set of, of capabilities. And so we actually, as a business, took a half a step back and said, okay, these are going to be transformational abilities. And assuming that this technology gets better and better and better over time, how do we make it so that every company out there can go build using those technologies? And so, different than how can I go build a consumer application that people are going to be interested in, we kind of took it from the

• 16:33Speaker 00

  point of view of, of AWS, right? Like, just what, what are the building blocks that I can help all of our customers, whether they're startups, whether they're enterprises, et cetera, go build interesting generative AI applications? And so, we started from first principles. Customers are going to care a ton about security. They, they're, that's not going to change. They're not going to all of a sudden not care about securing their infrastructure. We also had this hypothesis, two more hypotheses, one that the idea that there wasn't just going to be one model. We thought that there was going to be a lot of models for a lot of different purposes, and there'd be big models and small models, and people would want to combine them in new and interesting ways. And I think the last two years have probably played that out. But I think when OpenAI first launched, that wasn't as obvious. But that was kind of one of the bets that we made. And then

• 17:15Speaker 00

  the third one is that we view that every enterprise that was building on us, the interesting IP that they were going to bring to the table was mostly going to be their data. And they were going to care that their data didn't leak back into a model or, or escape from their environment. And so we built a bunch of what we did, starting from those principles of how do we make sure that these things are secure, that their data is secure, that they can have access to every piece of technology that the customers need to go build interesting applications, and they can do it in a cost effective way. And so that's how we approach the space. And I think we now have a platform in Bedrock, in Tranium chips, and in Ferentzha chips, and then a bunch of the other capabilities around,

• 17:57Speaker 00

  as well as the suite of models that we offer, both proprietary as well as open source ones, or open weights ones, that I think we're starting to see that really that momentum pick up. And we're seeing more and more customers really like that story. They like that platform to build from. And we're seeing enterprises really lean in and want to build in that in that space, because it gives them a lot of that control that they want as they go and build applications.

• 18:21Sarah Guo?

  How much do you think it matters that AWS has, let's say, like first-party models it offers its customers? Because that's clearly a strategy for some of the other hyperscalers.

• 18:35Speaker 00

  Google, that's obviously their strategy. They're really the only one that has a first-party model today of the other hyperscalers. Microsoft's done a good job of co-opting OpenAI's innovation, although in their last, I saw recently, they listed OpenAI as one of their biggest competitors now. So it'll be interesting to see how that all plays out. But, you know, for us, I think it's important, and which we do. So we are building our own first-party models. We have our first-party models today. In fact, the Titan embeddings model is by far the most popular embeddings model that we have inside of Bedrock today for people that are building search indices and thinking about things like that. And we are building larger and larger models as well, first-party. I think it'll be important, but not critical. I mean, I think people love using

• 19:19Speaker 00

  Anthropic's Claude models. Those are fantastic. And right now, those are the best performing models in the world, which is fantastic. We just launched Llama 3.1 on the day it launched, and we have a really tight partnership with Meta, and their open weights model is fantastic. And I think increasingly, we're seeing customers really love that open weights model, because they can go and particularly enterprises, customize it. They can do fine tuning to it. They can add their own data to it and really customize and distill and do some interesting things. And so I think that is is super critical. You know, we're seeing kind of specialized models, if you will, where we see folks like Adobe building Firefly as all built on top of AWS, purpose-built for their own thing that they're building. You know, whether

• 20:02Speaker 00

  The Amazon Purpose Built models are, I think they're an important part of that. Mostly it's partially it's for us for learning. Some of it's for powering our own applications and some of that may be for end customers. But it's all kind of that diversity of option, honestly. Like, we want there to be the best set of options and we want them all to run in AWS. And so we want those workloads to run there. And so to the extent that we can do something novel or interesting with our own first party models, we'll do that. And but we're also delighted for our partners to run as well. So I think it's a little bit of both.

• 20:34Sarah Guo?

  Yeah, it's really interesting. Alyssa Henry, you know, longtime AWS leader as well, is a friend. And over the years, I would like ask her about some interesting new open source project. And she, you know, it was the most terrifying thing, honestly, because she would always be like, AWS loves open source. We make more money on open source than any open source company does, which is, you know, if you think about all the advantages that AWS has, even if you're very friendly in the ecosystem, it, you know, can can turn out that way. And so I think the sea change that's happened in terms of availability of open weights models and multiple players here, Mistral, Llama, et cetera, that are very competitive

• 21:16Sarah Guo?

  is like, I think, you know, a huge, huge boon to AWS's general open ecosystem model.

• 21:24Speaker 00

  We've always leaned into open source. We're huge contributors to many open source projects and then we lead many open source projects. And and I think we do a good job creating and turning those into businesses for our customers and helping run managed open source projects. And so it's a big area for us. One of the reasons, frankly, is that, you know, we've long said, we don't want customers tied to AWS because they're locked into some proprietary licensing piece. We want them to be able to, we want, I want my customers to want to run on us as opposed to kind of locked into a Microsoft license where you're you're held to some different license that you can't get off of easily or old school, kind of like Oracle database that you can't get off of. You know, we want people to be able to run. And so even where we have something like Aurora,

• 22:09Speaker 00

  which is our kind of managed database, it is 100% Postgres compatible. And if you take that code and go run it at a Postgres database somewhere else, it won't operate as well. You know, because we do a great job at that, but it runs like in in theory, if you run it as well, it will operate as well. And so, you know, that's, that's how a lot of our services are built and how we think about things. We'll, we'll support proprietary things as well. And, you know, at some level, there are some services where you got to take advantage of the cloud, where, you know, there are some proprietary things that customers can use, but things like Dynamo and other, other technologies like that. But we really embrace open source and, and I think it's been, it's beneficial to the whole industry, frankly. And it's, it's, it's a way to get better

• 22:53Speaker 00

  security, better visibility, and kind of that license portability, I think, is a, is a key aspect as well.

• 22:59Speaker 02

  You mentioned that, you know, the model side of the AI world, and there's other main AI building blocks. There's RAG and, you know, there's certain aspects of fine tuning and other things that people are increasingly doing over time. There's other parts of it like eval suites. And what are the main building blocks that you can talk about in terms of things that are either coming to AWS, or how you think about that?

• 23:19Speaker 00

  Yeah.

• 23:19Speaker 02

  More fragmented world of all these different components and how they fit together relative to AI workloads today?

• 23:24Speaker 00

  That is kind of the idea of Bedrock is that we want to make it easy to do. And I do think, actually, in many ways, today, the models is the front and center thing that everybody pays attention to. But I think increasingly, it'll become a smaller percentage of the thing that people pay attention to because people are going to care about whether it's RAG or some other sort of knowledge base. And we call it knowledge bases because it's, you know, the technology may change over time under the covers. But that like, how do you have a grounding set of truth that you use? I also think grounding data is an interesting thing for, for like, real time information that you want as part of your AI systems. We have things like guardrails, which is our customers find incredibly important because if you're building a chatbot on a financial services website, you can actually find a lot of money if that thing starts giving out financial advice.

• 24:08Speaker 00

  And so you really want to be able to control, let alone going down and talking about politics or something else that you definitely don't want to talk about. And so those guardrails are super important as people think about what they want their AI systems to do and interact with and where they want to stay away. This is not controversial. I'm sure you both hear a lot about this. But again, the next generation and the next step forward in what we can get out of AI systems is going to depend a lot on how well we can integrate agentic workflows and actually get these AI systems to do things, not just kind of summarize and tell us information. And so building in the ability to have agents as part of that workflow is a big area investment for us. And we want that to be easy for you to build as part of that bedrock capability.

• 24:53Speaker 00

  I do think that pre-training and fine tuning is going to be something that more and more customers are going to want to do, as well as distilling over time. Because I think, I was just talking to a couple of customers earlier today that are very focused on how do I get this model down to a much smaller thing so I can put it on an industrial edge or somewhere like that? And so how do you think about distilling down so that I get the value of what I want? I don't need the whole reasoning engine behind that. And I think there's a long roadmap of, like you said, kind of model evaluation and other things like that. And some of that is us and also some of that is partners, by the way. And so AWS has been a place where I think part of the thing that has made us successful is really embracing the ecosystem to go build around there.

• 25:35Speaker 00

  And so thinking about labeling data as an example, we have a deep partnership with ScaleAI to come in and help you label your data if you're going to be doing any fine tuning or pre-training or things like that with your data. We partner with folks like LangChain to help put together some of those agent workflows, other things like that. And not to mention, of course, the model providers who are super important partners of ours as well. I think it's all of those things and our job is to how can we make it easier and easier for you to go build those applications in a tightly coupled way so that it's easy to go use those different components, easier to innovate rapidly and easier to build the proprietary data that you have as part of your AWS data lake so that you can kind of pull that in because frankly, most of these generative AI systems aren't going to be super useful

• 26:20Speaker 00

  if you don't have interesting data to go pull from.

• 26:23Speaker 02

  The other place that a lot of people are spending time right now in terms of bottlenecks to utilization or usage or future proofing is actually more on the chip side or semiconductor or system side and then in terms of DC capacity and obviously you all have been building titanium chips and other things, which I think is really exciting to see that evolution. How do you think about future GPU shortages? Does that go away? When? I'm sort of curious about how you think about forward looking capacity and is the industry actually ready in terms of building out data centers, building out semiconductors, all the rest of it, packaging, you know, the whole.

• 26:56Speaker 00

  Look, I think we're probably going to be in a constrained world for the next little bit of time. Just, you know, that some of these things are, they take time. Like, look how long it takes to build a semiconductor fab. Like, it's not a short lead time and that's several years and TSMC is running fast to try to ramp up capacity, but it's not just them. It's the memory providers and frankly, data centers that we're building, right? And so as we think about, there's a lot of pieces in that in that value chain that I think as you look at the demand for AI, which has been how exponential might be undershooting it. Some of those components that support that I think are catching up.

• 27:37Speaker 00

  And I think AWS is well positioned to try to do that better than others are. You know, we've spent a long time thinking about in the last 18 years, learning, how do we think about smart investing? How do we think about capital allocation? We've spent a bunch of time thinking about how do we acquire our own power? How do we ensure that it's green and carbon neutral power? All super important things and we're the largest purchaser of renewable energy over the last new contracts, right? So actually going out and adding and supporting new renewable energy projects. We're the largest provider, I think, each of the last four or five years. And so, so we've been leaning into that for a while to ramp up this and this is just a step up.

• 28:22Speaker 00

  And so I think we're thinking about, you know, how are we acquiring enough power? Our own chips is a way to support the growth of Nvidia chips. And so I think the more diversity there, the better off we are. We're, we're a huge partner of Nvidia's. We, Nvidia actually runs their AI training clusters in AWS because we actually have the, the most stable infrastructure of anyone else. And so they, they actually get the best performance from us and, and we love that partnership and we have a great and growing relationship with them. And, you know, we think things like Trainium are a good diversification. And I think there will be some workloads that, that run better on Trainium and are cheaper on Trainium over time. And as well as Inferentia, I think inference is, is one of those workloads that today it's, you

• 29:07Speaker 00

  know, 50 50 maybe of training and inference, but in order for the math to work out, inference workloads have to dominate. Otherwise, all this investment in these big models isn't really going to pay off. So hopefully for the industry that, that, that all happens. But I think we're probably going to be tight for the next little bit of time. And so, you know, because the, the demand is, is almost infinite. I mean, it seems infinite right now.

• 29:31Sarah Guo?

  How does AWS think about making investments in data centers of this scale to train the next set of foundation models? Right? And because I think you could take a, you know, AWS is very educated player. You could take a proactive approach. You could take a customer driven approach, but the idea that there are individual players who want tens of thousands of nodes at a time and interconnected GPUs is like a sort of a new demand vector.

• 29:57Speaker 00

  Some of the demand for some of these really large models is, is very large. Right? I mean, it's the, they're, they're talking about needing gigawatts of capacity, which is,

• 30:09Speaker 00

  it's kind of a, a mind boggling number that, that some of these models need. We're doing both proactive as well as customer driven. Right? We try to balance because, because there's real capital outlays that are required as part of this, of course, and we're talking tens, if not hundreds of billions of dollars of, of capital investment. And so, you know, we think about it as how do you, how do you make the right investments in things like land and power and other things that are, are fungible and could potentially be used for other things if, if eventually demand changes or the, the slope changes, as well as then having.

• 30:43Speaker 00

  Visibility into the supply chain for more near term things that you'll need like servers and chips and memory and other pieces like that. And so we balance a bunch of those things, managing the financial implications of what we need to go buy, as well as the long term customer demand. We try to map out, how do we meet some of those match? And some of our customers give us long term commitments to help with some of those things. And we give better rates for customers that give very large long term commitments for some of that capacity that requires a lot of investment. But you know, it said, there's a lot of airbars on that, too. Because at anything that's growing at multiple hundreds of percent year over year, you're not going to nail that number appropriately. And so we try to have enough buffer in there that we can support upsides when they happen and

• 31:28Speaker 00

  manage if it's a little bit less than we thought.

• 31:30Sarah Guo?

  As someone who's seen just many generations of startups decide like what investment they want to make in infrastructure, that is suddenly a much more important question to a generation of AI companies than it has been in recent history. What like advice would you have for them as the man holding the data center, I suppose?

• 31:49Speaker 00

  We've had to go on this before, right? We started from $100 billion of revenue to $100 billion or well, I started when we're at $0 of revenue to $100 billion of revenue. And so, you know, we've had this kind of rapid growth before. We think about how do you balance some of those pieces? And I think part of that is, is how do you make sure that, for me, I think as you're as a startup thinking about this is, how are you thinking about investments with a real plan of how do you have monetization? And not assuming that there's always more VC funding to come and bail you out. And so kind of having a plan there to flexibility of, you know, what, how can I start monetizing sooner if I need

• 32:32Speaker 00

  to? And where can I keep investing if that's the part that I'm in that the traction makes sense? Because I think, look, the only, the only reason that any startup goes out of business is they run out of money. That's the only, that's as simple as that. As long as you don't run out of money, you're not going to go out of business. Obviously, easier said than done, but, uh, but I honestly think some startups kind of forget that. They're always like, that's no problem. I'll just go raise more. And, uh, and kind of remembering that just because there's like a hype cycle, doesn't mean that someone's going to give you a bill, another billion dollars six months later. And I actually earned, learned that early in my, uh, career. My, my very first startup, uh, we raised, I think at the time was a lot of money. It was $27 million. We ran out of money in like 18 months.

• 33:16Speaker 00

  And then, you know, the 2000s came around and there wasn't any more funding and we went out of business. We assumed that we could just go raise more money. And, uh, and so I think that was a good early lesson that you can't always do that.

• 33:28Sarah Guo?

  If I look just at, um, you know, my own portfolio or our friends' companies, what, what is interesting is if, like, of course you have a few examples that everybody is looking at, like OpenAI, um, where, um, value creation and dominance, or at least like lead in a market is highly correlated with the amount of money they're spending. It's not true across the portfolio and I'm like only investing in really AI companies. In that, uh, yes, everybody absolutely needs computers. And I'm talking about companies that are doing their own, um, training or fine tuning and, and, but some of our companies that are making the most progress and like, you know,

• 34:08Sarah Guo?

  we're still talking very early days, you know, zero to tens of millions in their first year or two.

• 34:16Sarah Guo?

  I think one of the other open questions that people have wondered about is, um, does all of the value creation in the ecosystem go to, um, your compute vendor? And eventually a big piece of it over to Jensen and Nvidia or to the model vendor? And I think the, the answer, at least to like right now, is clearly not, right? I think there's different, there's capture different levels.

• 34:38Speaker 00

  There's probably enough for everybody. Today, most of it does go to Nvidia, I think. That's, that's a lot of it. But I just think that's because it's where it is early in the cycle. Uh, you know, I think, um, and they've built some incredible technology that's enabling some really cool stuff. So I think that that's, it's, it's, um, it's fine and. At some point, it's going to be the companies that find out how you actually go solve real problems and deliver real value to enterprises and to customers and other things like that. And that's going to be that, you know, I see a lot of, if I, if I take a step back and see who's implementing AI out there, it's a lot of enterprises that are doing proof of concepts and, and a lot, and sometimes they'll find one that really works well and it'll go to

• 35:21Speaker 00

  production. And I think if you can have a startup that can make that part easier that says, look, this is a real value, right? It's not a chatbot on your website, but it's something that helps you go faster, make sales better, innovate more rapidly, um, you know, do something you were never able to do before, uh, improve manufacturing efficiency, whatever that is the startup is focused on, or the company is focused on for that matter, it's, it's that, it's going to be an application level, right? It's most, most people don't, um, build a CRM from scratch. They go use a Salesforce or something like that. And, and most companies don't build software from scratch. And so I think most companies are not going to build their own models over time. They might tweak them a little bit, but I think a lot of companies

• 36:02Speaker 00

  are going to build, are going to use the applications that use the software and the models underneath. And, um, so it's not surprising to me that you don't necessarily have to spend the billions of dollars to go build your own model. You can build a small model.

• 36:17Speaker 02

  Seems like there's a lot of precedent for this, just in terms of prior waves of both software and internet where I think K2 or somebody had some very good slides where they broke down the relative value accrued by each layer. And to your point, each layer sort of ends up benefiting over time. So it seems like that question is almost overstated in terms of the importance of it.

• 36:34Sarah Guo?

  Do you have a prediction, if you just look at what happened with public cloud, if, um, what happens with AI platforms, uh, evolves differently? And the, the reason being, you know, I just spent a little bit of time talking to large enterprise customers again. And exactly as you said, there's a lot of activity coming from. Different POCs, there's investment in the area, there's top-down interest. And I do think there's a real conviction that the value will be real. There's also, like, I feel a little bit of deja vu in

• 37:05Speaker 00

  Yep.

• 37:06Sarah Guo?

  bunch of large organizations saying like, nobody meets our needs and we're gonna have to build our own platform. And it's gonna be everything from like data management to, you know, GPU management for training and inference to like eval suite to, you know, compliance and audit. And so I kind of want to say I've seen this story again before, but you know, how would you predict it plays out?

• 37:28Speaker 00

  It's exactly that. I think every time there's a new space and some of those things don't exist, people are like, well, I've got to go build my own. Well, it's like, or, you know, likely there's a bunch of other people that are actually building that too, like us, that once it exists, and I don't know how many times customers have told me like, oh, if I, you know, if you had already had this, I wouldn't have got to build my own and now I can stop investing in it because it's actually not the thing that gives me value. And so managing GPUs, I bet very few of those enterprises, that's the actual thing that their stockholders are like, yeah, that's what this enterprise is super important, good at is managing GPUs. Like outside of like, you know, hyperscalers or somebody like that, they don't really want to, right? If they could use something like SageMaker and it has all the capabilities that they can go and build the services that they want.

• 38:12Speaker 00

  And it doesn't today, we know that, but we're iterating incredibly rapidly and launching new stuff every month, every week, really. And so, you know, I feel pretty convicted that that, particularly for that level of the stack, like, it just doesn't make sense for people to do that part. And it's not surprising that they do it today because some of those things don't exist and they want to go deliver on something. And, you know, their priority was slightly different than when we go deliver it or something like that. But having it integrated as part of your platform, having it integrated as part of where you have your data lake, having it, all of those things tied together, kind of as a, as an infrastructure piece, likely makes sense. And, and doesn't mean that we'll do all of those things, by the way. Some of them will be partners that are built on top of us, which is great too.

• 38:53Speaker 00

  But, but I do think that it's the, the people that are specializing in those places are probably the ones that are gonna. Eventually kind of support that space, not individual enterprises.

• 39:03Speaker 02

  If we were to abstract out a level and, you know, ask what you, what your vision is, or how are you thinking about the next three to five years of AWS more generally as a business? What are the key things or areas of focus for you?

• 39:15Speaker 00

  Well, this is one. I think, I mean, I'm, I'm just as excited about generative AI and, and AI broadly as, as you all are. I do think that it's an enormous opportunity for us and for our customers to, and, and I think it actually, in many ways, it has a positive flywheel effect and is can be a tailwind to some of that first stuff that we were talking about a little while ago about helping customers move to the cloud. You know, I think if we think about where can generative AI help, some of that can be like, how do you make that go faster? Right? How do you take some of that more, you know, our, our original AWS thesis was we take care of the muck so you don't have to. There's still a lot of that that customers have to do today that I think generative AI can help with.

• 39:55Speaker 00

  And so over the next three to five years, there is a big investment for us in both building that toolset, building that whole platform that we're talking about so that customers don't feel like they have to go manage a bunch of these pieces. They don't, and they don't have to think about, you know, GPUs, or they don't have to think about, how do they think about kind of tying these clusters together or whatever? All of that can be abstracted away. If you think about the start of what Bedrock is, if you go use Bedrock models today, you never interact with a GPU, right? You just, you send it tokens, you get tokens back. And you can, eventually you're going to be doing things like fine tuning and, and pre-training where you're sending in information and training the models under the covers, but you're still just then sending it tokens and getting tokens back.

• 40:37Speaker 00

  And so the more we can abstract that to whether it's serverless or, or an application platform that people can go build. And that's just, you know, again, we're at the early stages of what that is. And so I think that as you move forward, generative AI honestly becomes one of the compute building blocks that you think about. You're going to need storage, you need compute, you need databases, you need inference, if you will, for your application largely. And I think that's just going to be, and networking and a bunch of the other things. And inference is just going to be one of those building blocks that people come to expect. And just like with compute where you may want Intel processor or a Graviton processor, or you may want block storage, or you may want object storage, you may want different models behind the inference, and you may think of that, and it'll have slightly, you

• 41:19Speaker 00

  know, different databases. Inference is going to have different flavors to it, and it'll be big models and small models, and you'll trade off cost and latency and capabilities and things like that. But I do think it's part of the applications. And so, we're trying to build it as part of that platform. When you're just building your application, it comes with it. And then, there's a lot to get there between now and then, but I imagine that's how most applications are going to be built going forward.

• 41:40Sarah Guo?

  So, we started this discussion talking about like the early days of AWS, and sort of how you were discovering the sort of, you know, unthinkably large requirements from large financials. And yet, you know, $100 billion of revenue later, you are still a huge partner to startups. Like, that goes against some of the conventional wisdom of like, choose one audience and just, you know, slowly move up market as something that, you know, many startups themselves choose to do. Like, why continue to work so much with startups? And I know you personally still think about this a lot.

• 42:12Speaker 00

  It's super important for us, and startups are the lifeblood of what helps us grow. And we get so much benefit from the learning from startups. And so, they will continue to be incredibly important for us. And we're gonna, if anything, lean more into startups and supporting startups as part of what we do.

• 42:30Speaker 02

  Thanks so much for providing your perspective over the last, you know, 20 years or so of AWS. It's been really fascinating talking to you. So, thanks so much for the time today.

• 42:37Speaker 00

  Thanks both of you for having me. Appreciate it. It's been fun.

• 42:41Sarah Guo?

  Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.