Amazon Reveals Its AI Master Plan — With Matt Wood

Matt Wood is the VP of product at Amazon Web Services. He joins Big Technology Podcast live at the AWS Summit in New York City for a conversation about Amazon's plan to compete — and thrive — in the AI race. In this conversation, Wood describes Amazon's plan to support all AI models, not just one, and support builders via its computing infrastructure. Tune in for the most extensive comments from Amazon on its AI strategy to date. Stay tuned for the second half were we discuss ethics, company culture, and Jeff Bezos. You can read Alex's Substack story about the conversation here: Inside Amazon’s Low Key Plan To Dominate AI --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  An Amazon VP working directly on the company's AI initiatives joins us live from Amazon's AWS Summit in New York City. All that and more coming up right after this. This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis Chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience, and what companies need to get right in the AI era. Then, I sat down with Adam Mitchell, Head of Enterprise Business Solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way.

• 0:43Alex Kantrowitz?

  You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:15Speaker 01

  LinkedIn Presents.

• 1:21Alex Kantrowitz?

  Welcome to Big Technology Podcast, a show for cool-headed nuanced conversation of the tech world and beyond. And we have a special show for you today. We are live here at Amazon's AWS Summit at the Javits Center in New York City. And we are joined by Matt Wood. He's the VP of Product, working on AI products. Inside AWS, we're going to talk about generative AI. Welcome, Matt. Thank you so much for having me. It's a pleasure. So, just so folks know, we have a live audience in front of us. I want to make sure that you all get on the recording. So, you're gonna let the people at home hear you, and you gotta be loud. Let's hear it.

• 2:02Alex Kantrowitz?

  That's pretty good. Pretty solid crowd. Pretty solid crowd, yeah. So far so good. So Matt, I did my research about where Amazon fits in the generative AI space. And I looked at like the ChatGPT model and I looked at chips. Okay, you're dabbling in that area, in those areas, but I don't know if you're stand out there yet, but where you are really trying to compete is in this space where companies, the big companies bring their models inside AWS. And then anybody that wants to build something with an LLM can build it through your products. So talk to us a little bit about that initiative and why did Amazon pick that in particular?

• 2:45Speaker 02

  Sure. Number one, I would say we're doing a lot more than dabbling. I think we've got a very meaningful focus and investment just across the company on generative AI.

• 2:58Speaker 02

  Myself, I think the rest of the team, I think like a lot of other people here, probably the light bulb came on when we started, you know, playing with ChatGPT. When that first came out, we really got excited and inspired by the capability here. And so what we're trying to do is maybe a little different from what some other folks are doing. What we want to do is take this technology and make it as broadly available as possible. There's a lot of these kind of magical, interesting technologies like cloud computing in 20 years ago, like machine learning 10 years ago, and now artificial intelligence, that have traditionally been only available to the very, very largest technology companies,

• 3:40Speaker 02

  to the biggest governments and academic agencies. And so my mission and our approach is that we want to make that as broadly distributed as possible. We want every builder and everyone to have access to the same capabilities that were once, you know, very, very limited.

• 3:59Alex Kantrowitz?

  Okay, wait, let me challenge you on that right off the bat. I mean, there's a huge open source movement in this area. In fact, Facebook just released this Llama 2 model, open source. You can use it, customize it. You don't have to pay them a thing. So there is access. So where is that? Gap that you're seeing between, you know, the high barrier and what is available on the market?

• 4:21Speaker 02

  Absolutely. So Llama 2 is an excellent, very capable model. But there is a long way to go from having the model weights, which are what comprises the neural network, to actually building out an artificially intelligent system. And just having the model weights is super useful, but it's like having the source code to some software. Yes, it will give you some capability, but you still need to be able to deploy that somewhere. You still need to be able to understand it enough to be able to make changes to it. You need tooling that understands how it works, so you can actually take it as an engine and put it inside the car that you're building. And so what we're trying to do at AWS is make it really, really easy to take Llama 2 and other models like it, whether they're open source or proprietary,

• 5:05Speaker 02

  and use it as an engine inside cars and boats and planes and all sorts of things.

• 5:11Alex Kantrowitz?

  So tell me a little bit about the process. So Facebook comes to you, or Meta comes to you and says, hey, Matt, we have this cool open source model. We'd like to make it available to your clients through AWS. Is that sort of how the process goes?

• 5:24Speaker 02

  I mean, pretty much. And what we do then is we take the model and the weights, and we put it inside a capability, our machine learning service that we have called SageMaker. And SageMaker lets any builder build, train, and deploy using machine learning models. And Llama 2 is one of, you know, several dozen large language models that are available on SageMaker today. So that's exactly, almost exactly how it happened, actually.

• 5:50Alex Kantrowitz?

  Right. So SageMaker is your managed service that allows people to build, to shape models. But you have a newer product that's called Bedrock, which allows people to build things like agents, for instance. And they get to, it's very interesting. They get to pick the different model that they want. So talk a little bit about how that works. And I'd really love to hear, you know, about Amazon's progression from SageMaker to Bedrock. And why that puts you in a better strategic position?

• 6:19Speaker 02

  Sure. I think SageMaker, we launched 2017, I think, and it's been very successful. Very happy with the business. Customers love it. Many of our customers have standardized on SageMaker for their machine learning workloads. But one of the super interesting things about generative AI is inherently because you're not training the models yourself, you're taking models from... Amazon and Meta and a whole host of other stability, AI and just building on top of them. It makes machine learning far more accessible. And so while SageMaker is great at building and training and deploying those models, we wanted to find a way which gave the maximum leverage of that accessibility.

• 7:02Speaker 02

  So you could just, instead of having to figure out how the model worked and fine tune your data and all those sorts of things, just give a prompt, just tell us what, tell the system what you want, choose the model that you want to run it against, and we have about half a dozen models, plus include from partners and some from, from Amazon. And then we just give you the result. There's no servers, there's no infrastructure to manage. You don't have to worry about using data or labeling data or worrying about GPUs or capacity or any of those things. Just give us a prompt, put in, choose the model and we'll give you the output.

• 7:36Alex Kantrowitz?

  So it's pretty cool. Like if you wanted to make a chatbot, for instance, like people think, all right, I want to make a bot, I go to OpenAI and I build it with them. Well, you could do actually, and Amazon's technology is go ahead and build an agent or a bot and then pick whether you want OpenAI or Claude, right? It just, it depends. It runs the gamut. That, that's the strategic bet for Amazon.

• 7:59Speaker 02

  That's right. And then our approach is to find areas that are really, really valuable. The customers, real problems the customers are trying to solve, and then add capabilities to Bedrock to make those problems smaller. So a good example would be a chatbot. So chatbots, like you may have played with ChatGPT,

• 8:20Speaker 02

  they're very capable. They can understand what you're talking about. They have context. You can go back and forth. And, uh, and they give the appearance of intelligence, but they actually are not very good today at completing complex tasks. And so let's say you wanted to create a right retirement plan. You could ask your chatbot, build me a retirement plan and it would go off and it would build a very kind of. Reasonable approach to retirement.

• 8:47Alex Kantrowitz?

  I would not. Can't bet my retirement off.

• 8:51Speaker 02

  I would I would vet that very carefully, but it will give you a pretty good strategy, a pretty good starting point. But it doesn't know about your personal finances. It doesn't know about the state of the markets. It doesn't know what products are available. And so one of the things we're adding to Bedrock is the ability to be able to provide that information to the language model using your own private data inside the applications that are already running the Amazon cloud and to be able to extend the language model's capability with that data in a couple of minutes so that the model can produce not just a strategy, but it can actually help you complete a task. And that's something that hasn't been possible before.

• 9:33Alex Kantrowitz?

  You know, Matt, this this all sounds good. And then I go back to the release of Lambda two last, well, a couple of weeks ago. And, OK, I saw it was definitely on AWS, but Azure is the preferred partner there. So, yes, you're doing this. It's a it's a strategy that makes sense in my mind. But you're also your competitors are doing it as well. So where's the distinction there?

• 10:00Speaker 02

  I think the distinction is that we have a slightly different approach to some other providers. We want to broadly democratize this technology. We want to do that because we think that there's not going to be a single model to rule them all. And so others are talking about, well, our stated goal is that we want an artificially generally intelligent system. That is not our stated goal. Our stated goal is that we want to just be very pragmatic, meet customers where they're at today, and then provide capabilities like agents, provide the option of different models, and then allow customers to deploy those capabilities in a way which is very low cost, high availability, and low latency. And that operational performance

• 10:44Speaker 02

  is, I think, something that's going to really set folks apart in the future.

• 10:46Alex Kantrowitz?

  OK, but I have to get back to this Azure example. I mean, they're doing the same thing. So are you in Microsoft? Are you in your? You're not going to have. I think what you're basically saying is you're not going to have this field to yourself. You realize that you're going to come up against competitors doing the same thing. Or maybe I'm putting words in your mouth.

• 11:03Speaker 02

  Well, I think it's safe to say this is going to be a very competitive space for a very, very long time. The opportunity is enormous. The capabilities are incredibly early. And so when you match early capabilities with a huge opportunity, you're naturally going to get a ton of different competition and ideas and thoughts. And we have our own and who knows if they'll turn out to be right or not. But our key point of differentiation is to be able to allow builders to be able to build these systems with their own data privately and securely. And to be able to leverage the investments that they've already made in that data on AWS,

• 11:43Speaker 02

  using completely novel capabilities in addition to existing models and novel models as well. So there's a lot of focus on models today, but those models are going to remain important. But over time, there's going to be additional capabilities like agents, like reinforcement learning, like the ability to be able to understand and vet the responses that come out of these things, our accuracy, that are going to be as important as the models over time.

• 12:12Alex Kantrowitz?

  One more Microsoft question, if I may.

• 12:13Speaker 02

  Sure.

• 12:14Alex Kantrowitz?

  They're so deeply invested in OpenAI and those GPT models. Does that make you distinct from them? Like, are there, okay, put me in the seat of a customer who's evaluating these two solutions.

• 12:32Alex Kantrowitz?

  OpenAI, Microsoft got all the buzz in the beginning. But Microsoft is totally sort of all in on this model. You have your own models, we're going to talk about them. But Amazon seems to be less, it has less at stake in terms of making yours work. So is there a point of differentiation there? If I'm a customer, like for instance, is it more neutral? Like, how do I think about that? How do they think about that?

• 12:58Speaker 02

  Yeah, I think we're probably a little bit more pragmatic. We're a little bit more neutral. We have our own models, yes, and we think that they're going to be very capable. And but we also recognize that different models will have different sweet spots. Some are going to be really good at managing data, some are going to be. Be really good at translating languages, some are going to be really good at reasoning. And we expect that most customers are going to want access to not a single model that tries to do it all. But a range of models that are good at different things. And I think today we're the only place where you can data and we have customers with exabytes of data. You'd be surprised how many customers have exabytes of data on AWS. And they can take that data that they've invested in and they can use it with these models to create a net new asset.

• 13:43Speaker 02

  For their organization.

• 13:44Alex Kantrowitz?

  Right.

• 13:44Speaker 02

  That is valuable and unique.

• 13:46Alex Kantrowitz?

  Right.

• 13:46Speaker 02

  And private and you can only do that on AWS.

• 13:49Alex Kantrowitz?

  So I have a note here and it's just like, man, you guys did Alexa first. So why didn't you end up leading this L L M conversation? I mean, the fact that like, we thought we were going to be talking to intelligent assistants everywhere. It's kind of an Amazon idea. I mean, it was an Apple idea, but their execution was bad. Yours was better. You guys understood that we'd want to be talking to computers. And yet, you know, we just mentioned that you have your own L L M. Your, your own model that's right there for people to access in Bedrock. Trust me, when I spoke, when I said I was going to be speaking with you, most of the general public was like, what's their strategy? Where's their model? You have one. So just talk a little bit about what happened there.

• 14:32Speaker 02

  Well, I think, uh, number one, we're very happy with Alexa. Alexa is available to billions of customers and requests.

• 14:41Alex Kantrowitz?

  Yeah.

• 14:41Speaker 02

  Across hundreds of millions of endpoints.

• 14:43Alex Kantrowitz?

  So all listeners with one of those devices in your home.

• 14:46Speaker 02

  Yes, we apologize if we're triggering, uh, the, the, the woman. Um, uh, or the man, if you've got it set that way. Uh, so I think we're incredibly, incredibly proud of the progress we've made with Alexa. If you look at the way that it has been, uh, used in the real world, the number of endpoints that it's available in. If you'd have told me six, seven years ago that you could, uh, put Alexa in the microwave.

• 15:09Alex Kantrowitz?

  Right.

• 15:09Speaker 02

  Put Alexa in a car.

• 15:11Alex Kantrowitz?

  That you said that.

• 15:13Speaker 02

  The microwave is a great idea.

• 15:14Alex Kantrowitz?

  Yeah, I was like. Don't put your echo in the microwave.

• 15:17Speaker 02

  No, do not do that, just to be clear.

• 15:19Alex Kantrowitz?

  Right.

• 15:19Speaker 02

  Alexa, the service can live inside the microwave. The device should stay outside of the microwave. Uh, so, yeah, I think, yeah, we're incredibly proud of that. I think our vision for Alexa that we've always been striving for has been to provide a per- a truly personal assistant. Yeah, we talk about this idea of like the inspiration of coming from Star Trek and talking to the computer and all those sorts of things. That's great, but I think our actual long-term vision is a really personal assistant, not a all-seeing controlling system.

• 15:49Alex Kantrowitz?

  It's not that Alexa should have been this. It's that you had visibility into what this could be.

• 15:53Speaker 02

  Yeah.

• 15:54Alex Kantrowitz?

  And didn't release a ChatGPT first. So was that an oversight or?

• 15:58Speaker 02

  Well, only one person released ChatGPT first. Everybody else didn't. So I think that that, yeah, I think we should take inspiration from that. It is a fantastic, probably one of the most remarkable technology demonstrations that I have ever seen.

• 16:11Alex Kantrowitz?

  You're calling it a demo.

• 16:13Speaker 02

  It is, it's a great research tool.

• 16:15Alex Kantrowitz?

  It's developed beyond that.

• 16:17Speaker 02

  Uh.

• 16:18Alex Kantrowitz?

  There are plugins inside ChatGPT right now that allow you to do.

• 16:21Speaker 02

  Some people have access to plugins.

• 16:22Alex Kantrowitz?

  There's code interpreter that does some of the things that your models are are working out, which is like speaking to data that it's not trained on and saying, bring me some results.

• 16:32Speaker 02

  Well, code interpreter actually just executes code which is generated by the model. It doesn't have access to new information, doesn't have access to your own private information. And on that point, nobody is putting their private information into ChatGPT. There are hundreds, maybe thousands of CIOs that are telling their whole organization not to use ChatGPT because.

• 16:55Alex Kantrowitz?

  What's the concern there?

• 16:56Speaker 02

  Yeah, the concern is accidental data exfiltration. That you, when you're using ChatGPT, the, the service that exists on the web that we've all played with, whatever you type into that is being used to train and improve the models, which makes sense as a research tool that makes a ton of sense. But if you're a, uh, an organization and you start to want to reason and understand or develop your own IP, that IP is not differentiated against. And it goes into the model and it gets exfiltrated. And we've seen actual customers see their own IP come back to them from the model. And that is terrifying to enterprise customers where the IP is the crown jewels. And so.

• 17:39Speaker 02

  It is, I don't mean it in a diminutive way. It is an excellent technology demo. It is an excellent research tool from an exceptional research company. I'm not taking anything away from what they have achieved, and I think they will continue to do wonderful work. However, it is not the way that most organizations, in my opinion, are going to actually build and develop their generative capabilities.

• 18:03Alex Kantrowitz?

  And that's your bet, is to help them build these models? And do it in a, I imagine, privacy safe. I mean, I'm making the pitch here, but I'm just trying to figure, like, navigate this conversation and figure out what you guys are doing.

• 18:14Speaker 02

  Yeah.

• 18:15Alex Kantrowitz?

  And that's what it is. If a company comes to you and says, we want to incorporate LLMs in our business, you help them shape that, and so they don't end up dropping everything in ChatGPT and

• 18:25Speaker 02

  Correct.

• 18:25Alex Kantrowitz?

  having that spit out to their competitors.

• 18:27Speaker 02

  Correct, and the the the models have a similar level of capability, so you're not really losing anything there. But what we provide is security, what we provide is privacy. So none of the information that is used with our bedrock service is used to improve the underlying models. In fact, none of that information even leaves your network. You can see exactly what happens to that data and where it goes. And then, when you want to improve those models and customize them, specialize them for your own use cases internally, you don't want that specialization to be available to your competitors. You want it to be private and secure. And so we make it really, really easy to specialize those models in a way which gives you leverage against your own data and allows you to do that in a way which is completely private.

• 19:13Alex Kantrowitz?

  What's the business opportunity there? Because, let me, let me just tell you, Amazon, sorry, Andreessen Horowitz, I just read, they think that cloud on top of AI, the opportunity is ten to twenty percent of this Janet AI spend. They said, that sounds small. What do you think?

• 19:28Speaker 02

  I think that, two things. If you look at the broader opportunity, I think that this technology is as transformational as the very earliest internet and the web browsers that allowed us to access it. And it was that early internet and those early web browsers that gave inspiration and motivation and growth to companies like Amazon and Netflix and Airbnb. And I think that there is going to be a wave of similarly Amazon-sized companies that evolve out of the generative AI opportunity generally. And so I think we're going to see multiple Amazon-sized organizations develop and grow over the next, who knows, 20 years, 10 years, 10 weeks?

• 20:13Speaker 02

  It's anything seems possible.

• 20:14Alex Kantrowitz?

  And you're content with 10 to 20% of that next boom, or you want more?

• 20:18Speaker 02

  I don't know if that's true. It seems very low to me, but I wouldn't be at all surprised if just the AI part of our cloud computing business was larger than the rest of AWS combined in a couple of years.

• 20:32Alex Kantrowitz?

  Okay. No consumer product coming out from you guys?

• 20:36Speaker 02

  Nothing to announce today, that's for sure.

• 20:39Alex Kantrowitz?

  Big smile. Why not do it?

• 20:44Speaker 02

  Do what? A chat GPT-style search? I mean, we don't really operate in the in the search space. We don't have a deep investment in web search.

• 20:54Alex Kantrowitz?

  I mean, you have voice computing. It doesn't have to be search.

• 20:56Speaker 02

  Right, I'm just, I think you can expect for sure

• 21:00Alex Kantrowitz?

  Right.

• 21:00Speaker 02

  to see new invention and innovation coming from Alexa and devices and our retail stores and our ads business. You know, there isn't a team that I've spoken to at Amazon in the past, you know, six to 10 months that isn't really focused on, you know, understanding this technology and where it can be applied to their own business inside the company.

• 21:23Alex Kantrowitz?

  Let me ask you this. Microsoft recently said that they have 11,000 customers who are using their OpenAI software building service.

• 21:33Speaker 02

  Hmm.

• 21:34Alex Kantrowitz?

  Do you think, how important is it for a company to establish a lead in this moment? And Can you sit here and tell me with a straight face that Amazon is ahead of Microsoft?

• 21:47Speaker 02

  Well, number one, it is incredibly early and we are three steps into a marathon race. And I don't think anybody without a smile on their face could call a winner three steps into a marathon race.

• 22:04Alex Kantrowitz?

  But Amazon had this amazing moment where it got ahead. Now, it wasn't everybody going at the same time and you were very early in AWS, so you know this. But AWS kind of ran away with the cloud computing field or cloud services field until other companies started to figure it out. And by establishing itself so early, built that market dominance. But I'm curious if you think we're going to see something like that in this moment or it just doesn't apply?

• 22:28Speaker 02

  I think that there is going to be multiple very credible options for builders. And I am strongly convicted that AWS, if it is not the leader after today's announcements, which I would say it is, argue that it would be. But even if you take that as red, I think it's going to be hard to argue that by the end of the year, you know, you won't see AWS. It will be hard to argue that AWS isn't in the top one or two providers.

• 22:54Alex Kantrowitz?

  Right. Okay. Let's talk. You might enjoy this segment a little more. Let's talk a little bit about.

• 22:59Speaker 02

  I enjoyed that quite a lot, by the way.

• 23:01Alex Kantrowitz?

  Well, actually, I'm going to go back to my page of difficult questions.

• 23:06Alex Kantrowitz?

  Let's talk a little bit about building, right? We have, we're in front of a room of developers, at least I hope, or really very serious big technology fans. So thank you for showing.

• 23:16Speaker 02

  They look like builders. I mean that with the greatest respect.

• 23:19Alex Kantrowitz?

  Yeah, it's great. So talk practically about what people have already done with with your technology. I mean, you had an announcement today that's kind of interesting about agents, right? They can build, they can build agents. So I'd like to hear a little bit more about like the practical level of, and maybe you can go step by step of like what. And briefly, like what people would build with the AWS services?

• 23:47Speaker 02

  Sure, I think the ones that I I've seen that are the most compelling. Uh, number one, just generative responses. So the sort of blog posts, advertising copy, you know, three D meshes, those sorts of things. Where you're an expert and you just wanna, you just want a starting point. Uh, you just want, instead of starting with an empty word document, just give me a first pass and let me iterate on it. Way easier, huge time saver. You do that all day long, uh, very, very popular. The next area, which is less sexy, but in my opinion, maybe even be a a larger opportunity. Uh, is using this technology to improve uh, search results, improve ranking, relevance, personalization, those sorts of use cases where

• 24:31Speaker 02

  you don't even know that you're working with a large language model, it's all in the background. But they are remarkable at boosting the uh, the uh, the accuracy of those sorts of um, results. Then you've got um, knowledge discovery, uh, so that's the sort of chatbot example. Uh, and the one that I'm most excited about is um, collaborative problem solving. So working with, this is a bit more science fiction, but I think we materially advanced the state of the art this morning with our agents announcement. Um, where you are able to as a, as an individual or another artificially intelligent system interact with an artificially intelligent system to solve complex problems. Uh, that is a very interesting area.

• 25:11Alex Kantrowitz?

  Talk about what what that means.

• 25:13Speaker 02

  Well, it means that uh, i- imagine you've got a uh, any sort of business problem that you can imagine. Super simple, I've got a thousand dollars, I wanna turn it into two thousand dollars. How do I do that? Uh, you may have a uh, set of artificial intelligent capabilities uh, that will help advise you as to how to turn that one thousand dollars into two thousand dollars. And you can interact with them one by one and build the strategy yourself. Or you can have them operate as a swarm of agents collaborating with themselves in order to be able to build the best possible strategy and for each, like a to do list.

• 25:53Speaker 02

  And for each item on the to do list, they can recommend the specific tasks that you need to go do in order to be able to complete that.

• 25:59Alex Kantrowitz?

  Is it similar to like the baby AGIs?

• 26:01Speaker 02

  Right, that sort of idea, exactly, yeah, that auto, auto GPT approach of using large language models to rationalize with other large language models. And where we see the

• 26:11Alex Kantrowitz?

  Doesn't freak you out a bit?

• 26:12Speaker 02

  I don't think it freaks me out. No, I think we've seen tremendous opportunity. But here's why it doesn't freak me out. Because it works best in highly constrained domains. Where you put so many constraints around what it is you're trying to solve. That all of the agents, none of them are running amok. None of them are running off and doing things you didn't tell them to do. You put the constraints on and constraints are probably the single largest force that we have to improve the capabilities of these L L Ms.

• 26:41Alex Kantrowitz?

  Give a practical example.

• 26:43Speaker 02

  Practical example would be in my one thousand dollars to two thousand dollars example. You constrain it to a set of markets. You constrain it to a set of stocks. You constrain it to a set of financial products. You constrain it to a set of operations and buy and sell operations. You would do in a particular given a time. And every layer of constraint that you add reduces the chance that the language model will just create a spurious erroneous output. But also just keeps the whole thing grounded and keeps the whole thing focused. And when we've looked at these approaches inside the company. You know, they end up kind of acting like humans, like they argue. And sometimes they get stuck in a loop and you need to go in there and intervene.

• 27:27Speaker 02

  And other times you need a tiebreaker because there's two equally good ideas and two equally good options. And you need some, you need another agent or person to come in and tiebreak. And so it's very interesting watching these very tightly constrained

• 27:41Speaker 02

  ants run around trying to build things on your behalf.

• 27:44Alex Kantrowitz?

  And you think this is going to be useful?

• 27:46Speaker 02

  Absolutely, because it drives levels of automation for solving complex problems. Either completely automatically in cases where you would want that. Or in tandem with one or more people where you would want that.

• 27:59Alex Kantrowitz?

  Yep. Let's talk a little bit more. You said there's some chat applications. You helped Bloomberg work on Bloomberg G P T, which is their chatbot that queries financial data. So talk a little bit about that process. Like, is that the same? Are they coming in and saying, we're going to pick our model, but we're going to use your software to? Fill in the blank.

• 28:20Speaker 02

  Yeah, that's right. So Bloomberg has, like a lot of customers, actually, they have huge amounts of text information. And so they were able to take all of that text data, their natural language data from market reports and analyst reports and everything that they've news, everything they've accumulated over however long Bloomberg have been operating fifty years, I don't know. They're able to take all of that and then load it into the cloud on AWS and then use a machine learning algorithm to build their own chatbot, which is Bloomberg GPT. And they ran all of that inside our cloud computing infrastructure.

• 28:54Alex Kantrowitz?

  Interesting. So where does Amazon get paid in that loop?

• 28:58Speaker 02

  We get paid by providing compute capacity to actually do the model training and for providing the access to the large amounts of storage that are needed to store the data and then get that data into the machine learning models. And so we provide that capability on a pay as you go basis, as if it was a utility. And so you only pay for what you use and we meter it by the second. So for one second, you pay us a certain amount.

• 29:23Alex Kantrowitz?

  Yeah, so this sounds pretty expensive to me. I'm curious, I mean, and it does seem like it's the province of companies that have more money. I mean, the fact that we're talking about Bloomberg, I mean, is sort of, okay, that's sort of indicative of the companies with the resources to build these models. So tell us a little bit about the cost factor and, you know, who can actually afford the stuff?

• 29:45Speaker 02

  Yeah, I think training net new models is not going to be very common. It's, it's very complicated. It is expensive, to your point. You need a lot of compute capacity, a lot of data, a lot of expertise. Some folks that have differentiation in one of those three things will want to invest there. I think it makes sense. But the vast majority will not want to invest there. Now that said, we want to, from the Amazon side, make it as cheap and easy as possible to train those models in the first place. And so we've been investing in custom silicon in order to be able to accelerate that process with chips that are specifically designed and built for large scale machine learning training. And then, once you've got the model, you're

• 30:29Speaker 02

  To be able to operate it in as low cost as possible. And whilst a lot of focus is put on training, if you think about it, you may train a model once a month, once a week, let's say. But you're going to be running predictions and inference and chatting with that model hundreds, thousands, tens of thousands of times a day. And so if you're not careful, actually, the vast majority of that cost isn't in the training, although it can be expensive, it is in the operationalizing of the model for actually doing the chat. And that's why we have a second chip, which is specifically designed for low cost, low latency inference. We call it inferential.

• 31:05Alex Kantrowitz?

  And I'm paying for the compute on that. What do you think the cheapest is for someone who wants to build their own bot with AWS? Like, what's the entry level price?

• 31:14Speaker 02

  Uh, if you use an existing foundational model. And uh, 10 cents.

• 31:19Alex Kantrowitz?

  Really? Per, just to build it?

• 31:23Speaker 02

  10 cents to build it and then um, the, the cost of running it is priced per token.

• 31:29Alex Kantrowitz?

  Yeah. I was speaking to Michael Schmulick from Bernstein, he's a financial analyst, he follows Amazon closely. And I was like, Michael, what should I ask? And he said, well, look, this is going to cost a lot of money. Microsoft just said that they have, they're making something like a $3 billion infrastructure investment in this. Is Amazon thinking about making an investment anywhere in that range or has it already?

• 31:54Speaker 02

  Well, I don't think we're going to release the size of the investment, but when you're thinking about the size and scope of possible investments, they don't get much larger than custom building and fabricating chips. And so that, that is a huge investment that we've been making at AWS for nearly a decade now. You know, we're on our second generation of our inference chips. We're on our first generation of our training chips. We're going to keep investing in those and we're going to see, I don't think we're anywhere near the point of diminishing returns in terms of the capability and price performance improvements that we can provide through those chips. And so, uh, I say, I don't know what they, I don't know that the raw number is actually all that interesting. What's more interesting is what's the outcome? And is that outcome truly benefiting this broad democratization that we're seeing?

• 32:38Alex Kantrowitz?

  Okay, so you've mentioned chips, we've talked about your own model. I want to take a break quickly and then come back to talk about those two things. And I have another post-it that I'm holding with me and the headline is fun. I'll afford to. Stay tuned. This sounds great. I'll be back right after this. This episode is brought to you by Avpoint. Everyone's racing to roll out AI right now. Copilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data and most teams have no single way to see it, secure it and prove it's under control. That's exactly what Avpoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate.

• 33:23Alex Kantrowitz?

  Your data, your cloud and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. Avpoint, the unifying trust layer for AI. Head to avpoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door.

• 34:08Alex Kantrowitz?

  The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a wide-glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form and request your free risk assessment. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real world attacks. The link's in the show notes.

• 34:59Alex Kantrowitz?

  And we're back here with Matt Wood. He's a VP of Product at AWS focused on AI. So I mentioned that you have your own models, your own LLMs. And that's actually something that's available if people want to build within Bedrock. They can pick Titan.

• 35:18Speaker 02

  Titan, yeah.

• 35:19Alex Kantrowitz?

  Or they can pick something from OpenAI or Llama.

• 35:22Speaker 02

  Anthropic or AI21 Labs. We had a co here this morning, yep.

• 35:27Alex Kantrowitz?

  So why build your own? I mean, it seems so good up until the point where you start building your own model. And now, all of a sudden, you're running into the same problem that we talked about in the first step that Microsoft has where like, if I'm building something, you know, I don't know if Amazon really is neutral. So talk a little bit about why you built your own one. And Shmaluk said the same thing too. He's like, you don't need an 81st model. So why did Amazon build it?

• 35:53Speaker 02

  Well, on the 81st model, I think you do need an 81st model right now. Like, it would be completely arbitrary to decide right now, at this point in time, that we need to limit the model or that we've got enough. There is so much opportunity. It is so early. I think there's going to be no end of invention in the foundational models going forward. So that said, that's why we took our approach of making all of them available. Because who knows which one is going to have a breakout capability? Who knows which one is going to be the best fit for a particular use case? Our approach has been that we have training, we've trained a set of our own foundational models. We have a language model and we have a vector embedding model.

• 36:36Speaker 02

  And each of those models is actually a family of models. So the customers can choose the right model for their use case, not just for the capability, but also for the latency and also for the price. And so you may have a use case for very, very, very, for a very simple, small model that you want to operate with very, very low latency. And so that's an option that you have with Titan. You don't have that option with some of these extraordinarily low. Large models, particularly that are hosted in who knows where, where the latency is just, it's just what you get. But with Titan, you can choose the right trade off for capability and latency. You can choose the right trade off for capability and price. And for each of those models, you can add your own data to the model to improve it privately.

• 37:22Alex Kantrowitz?

  Yep. Now I'm going to get to chips, but it's always at this point in the conversation, like we're a little bit more than halfway in.

• 37:27Speaker 02

  Mm hmm.

• 37:27Alex Kantrowitz?

  Or there's always a thought that pops up in my head, which is we've been talking about. Generative AI wanting to talk to computers as a given, as if we actually want to interact with them in natural language. We want to have them as chatbots. We want to talk to them. But even the Alexa example shows that like there was all this, there originally was this whole range of things we wanted to do. And then the task narrowed and even, I don't know what, when you're, actually, let me ask you. Are you using L L M like chat G P T and and consumer bots as much now as you were at the beginning?

• 38:00Speaker 02

  I use, we have an internal tool that we actually built for ourselves. Primarily so that our engineers and our own builders could get familiarity and practice with prompt engineering.

• 38:13Alex Kantrowitz?

  Hmm.

• 38:14Speaker 02

  And so it's super simple, internal tools.

• 38:17Alex Kantrowitz?

  What's it called?

• 38:17Speaker 02

  Uh, it's called the L L M Playground.

• 38:20Alex Kantrowitz?

  Okay.

• 38:21Speaker 02

  And it is literally a playground. You can build little one page mini apps. Uh, where you can provide a prompt and you can chain that prompt into another prompt and you can. Play with the parameters and the different models and then you can arrange the widgets on the screen to build out little applications. You can build a chat application that way. You can provide it a. U R L and it will fetch the website and then use that as the part of the context for the prompt. So you can reason and ask questions about the the website.

• 38:49Alex Kantrowitz?

  You're using this stuff. So, so, okay.

• 38:51Speaker 02

  It's great.

• 38:52Alex Kantrowitz?

  That's good.

• 38:53Speaker 02

  It's the most fun I have all week, honestly. It's awesome. Yeah.

• 38:56Alex Kantrowitz?

  How many hours do you spend doing it?

• 38:57Speaker 02

  Uh, how many hours? I don't know. I probably spend at least thirty minutes a day.

• 39:01Alex Kantrowitz?

  Oh, wow.

• 39:02Speaker 02

  Uh, just exploring what's capable and exploring what the team is identifying as kind of emergent capabilities.

• 39:08Alex Kantrowitz?

  Okay, so now that I've effectively sabotaged my own question for like the last minute and a half, I'm gonna ask it.

• 39:12Speaker 02

  Hit it.

• 39:13Alex Kantrowitz?

  Um, which is, is this something that people actually wanna do? Like, do they wanna talk to computers? I mean, it sounds good. It's really freaking cool when you use it. But even now, people are saying ChatGPT is getting dumber. Largely, people believe that's because the novelty has worn off. So we talk about this generative AI moment, that you're saying it's gonna be bigger than the internet. What makes you so convinced that that this time is for real?

• 39:36Speaker 02

  What makes me so convinced is that, uh, the level of, uh, invention and efficiency and automation that we've seen inside the company and that our customers are experiencing. Chat being just one modality.

• 39:53Alex Kantrowitz?

  So what else do we have?

• 39:54Speaker 02

  Well, we have the other ones that I mentioned earlier, like the generative pieces, the search pieces, but the collaborative problem-solving pieces, the automation pieces, completing complex tasks. That, I think, is where the majority of the value is gonna be. And I think that chat is a great user interface. It's a great way to, uh, explore, uh, some knowledge and a domain. It's great for new users that are getting up to speed on a particular product, all of that. So it's a really useful use case, but it is very hard for me to imagine that we nailed the use case first time right out of the gate with chat. I think that there is gonna be all manner of, uh, model improvements which, and supporting engine improvements that allow us to deliver customer experiences that we just haven't even imagined yet.

• 40:38Speaker 02

  And we are doing that imagination and going through that process inside the company now. I like a lot of our customers at AWS, and it is inspiring.

• 40:46Alex Kantrowitz?

  Anything cool from inside Amazon you can share?

• 40:49Speaker 02

  I think the some of the early stuff that we're looking at, we announced today around generative BI.

• 40:55Alex Kantrowitz?

  Business intelligence.

• 40:56Speaker 02

  Business intelligence, thank you. To be able to ask and interact with your data just using these, using a chat interface, one, but also to be able to create dashboards, two, to be able to understand and find insights, number three, and then number four, when you found those insights, to be able to quickly just summarize your narrative, immediately create like a business report that includes all of the summaries and all of the reports and charts that you might need, and then email that around to your colleagues. Like, if you imagine the level, what you would have to have done before these capabilities were available, to be able to enable that, you would have needed to have teams of business analysts to connect to the data, they would have had to spend time, you know, investigating

• 41:40Speaker 02

  what to build, and then building the dashboard and setting it up, and then you have to train everybody in order to be able to do the, do the work with the data, and then you have to find the insights, which is very, very difficult. It just shortens that whole path to discovery through automation in a way which is unprecedented.

• 41:59Alex Kantrowitz?

  So who's learning from who here? And please don't say both. Is it AWS learning from the rest of the workflow inside Amazon? Or are people inside Amazon learning from AWS?

• 42:12Speaker 02

  I think it's honestly true that, I mean, Amazon is a very big company.

• 42:16Alex Kantrowitz?

  Right.

• 42:17Speaker 02

  I think we're taking inspiration, and we're kind of organizing ourselves internally, deliberately to take inspiration where we find it. And so, number one, we're enabling all of our builders, all of our software engineers, in which we have a pretty large number, to be able to experiment and try out large language models through Bedrock. So everybody has that capability. And then, we're finding ways that they can show their thinking and their invention to each other with something as simple as a demo day. So internally, we have multiple different teams, not a lot of them, but multiple different teams, and they will proactively reach out, and we have a schedule of people that are bringing their demos.

• 42:59Speaker 02

  And sometimes it's just slideware. Sometimes it's an idea, but more often than not, it is running software that we can take a look at. And they share their thought process and their implementation techniques, and that gets everybody else excited. And then the next iteration, we're building on top of that and round and round it goes. And so those kind of new idea and new creation points across the company have proven to be very inspiring to our developers. And number two, enables us to share our knowledge and our discovery and our thinking very, very broadly. And number three, honestly, prevented us from building the same thing. Let's say we've got a thousand development teams.

• 43:34Alex Kantrowitz?

  Yeah.

• 43:35Speaker 02

  You start them all off from the start line at the same time.

• 43:37Alex Kantrowitz?

  Come up with the same idea.

• 43:38Speaker 02

  They're gonna come up with the same idea, a lot of them will. So we've avoided that as well.

• 43:41Alex Kantrowitz?

  Let's talk briefly about chips.

• 43:43Speaker 02

  Sure.

• 43:44Alex Kantrowitz?

  It's interesting because I think that there's minimal awareness that Amazon has a

• 43:50Alex Kantrowitz?

  its own L L M in Titan. And there's even, I think, less awareness in the general public. I'm not talking about the folks sitting here. I'm sure they all know about it. But we hear so much about Nvidia chips. I swear, I feel like every day I hear Nvidia, like it's just a marching, you know, chant in my head, Nvidia chips, Nvidia chips, Nvidia chips. But you have your own chips. How are you making them and are they serving the same purpose or something slightly different?

• 44:17Speaker 02

  Well, we acquired a

• 44:19Alex Kantrowitz?

  This is all for chips for training A I.

• 44:21Speaker 02

  That's right. Yeah, that's right. We acquired a chip a design company called Anapurna, probably about 10 years ago now. And since then, we've been on a path to build ARM processors for general purpose computing. We have ARM processors specifically for high performance computing. And to find very specific, not a large number, but very specific use cases that we could accelerate in silicon. And the machine learning work use cases very quickly rose to the top. And so we've been investing there in terms of building out custom silicon that you can deploy on A W S today for building out your own large language models and for running the inference against.

• 45:03Alex Kantrowitz?

  You design your own chips as well?

• 45:05Speaker 02

  That's right. Yeah.

• 45:06Alex Kantrowitz?

  And it's not just ARM that builds it, right?

• 45:09Speaker 02

  ARM is just a blueprint. It's a starting point. But with Tranium and with Inferential, those are totally custom designed.

• 45:16Alex Kantrowitz?

  Who's building it?

• 45:17Speaker 02

  Um, I think they're constructed in Asia somewhere. I don't exactly know where.

• 45:22Alex Kantrowitz?

  Taiwan?

• 45:23Speaker 02

  Most likely, yeah.

• 45:24Alex Kantrowitz?

  Okay. Let's go to the fun posted, because I feel like people are getting restless.

• 45:29Speaker 02

  Okay.

• 45:30Alex Kantrowitz?

  Uh.

• 45:31Speaker 02

  Now I'm nervous.

• 45:32Alex Kantrowitz?

  No, it's all good.

• 45:32Speaker 02

  I think a fun posted for you is a nervous inducing posted.

• 45:35Alex Kantrowitz?

  That is how it should be.

• 45:38Alex Kantrowitz?

  Oh, ethics. All right. Uh, so you have your own L M Titan?

• 45:42Speaker 02

  Yep.

• 45:42Alex Kantrowitz?

  What was it trained on? And how can I be sure as a writer that it wasn't trained on my work?

• 45:48Speaker 02

  Titan was trained on publicly available data and.

• 45:52Alex Kantrowitz?

  That's a very squishy phrase.

• 45:54Speaker 02

  It's very precise.

• 45:55Alex Kantrowitz?

  Okay.

• 45:56Speaker 02

  Uh, and data which is proprietary data that we had licensed specifically for the purpose of training.

• 46:03Alex Kantrowitz?

  Okay. So can you say definitively that there's no chance that this model was trained on like, for instance, Substack articles?

• 46:12Speaker 02

  Uh, if they're publicly available, there's a good chance that they were part of the web crawl that we would have used. Uh, if they were part of or had been licensed to a large proprietary set of natural language, we would have licensed that with the right permissions to be able to use them for training.

• 46:26Alex Kantrowitz?

  So my Substack stories are publicly available. They're available on the internet.

• 46:31Speaker 02

  Mm hmm.

• 46:31Alex Kantrowitz?

  I guess I didn't really opt in for them to be used as part of training. Should I have the ability to decide whether or not they're going to be part of L M training or not, even though they are live on the web?

• 46:42Speaker 02

  It seems like, I mean, it's your content. You own it. You can do as you please. Um, but I think it seems like a strange use case to identify and single out. Who's to say what this data can be used for when it's publicly available? You, you know, you chose to make it publicly available. If you want to put some permissions around it, you want to take it private, it's totally up to you. You still own the data.

• 47:06Alex Kantrowitz?

  Right.

• 47:07Speaker 02

  Um, but it is public and as a result, it can be used for things that you may not have thought through initially or that weren't possible early on.

• 47:16Alex Kantrowitz?

  Yeah. And I'm not sitting here and saying, you know, you, how dare you take.

• 47:19Speaker 02

  No, I understand. I understand.

• 47:20Alex Kantrowitz?

  But it is interesting. It's something that people who are producing content are having, have to. We always thought it would just be Google, for instance, that crawled our stuff. But clearly it's going to be more. And so.

• 47:31Speaker 02

  Yeah. I mean, there are, there are open source, open web crawls available, um, uh, as open data, for example. You can just go and look at what's in there. It's maintained and kept up to date. It's called the Common Crawl. Um, you can check that out.

• 47:44Alex Kantrowitz?

  Sergey Brin is back inside Alphabet. He's called generative AI something like the most exciting technology or moment of his entire life. Jeff Bezos, do you know what his feeling is about this stuff? Should you?

• 48:01Speaker 02

  I do. I think he feels that it is, I wouldn't want to speak on his behalf, of course, but, you know, I think he feels the same, though. This is the single largest transformational step in how we interact with data and information and each other, you know, since the very earliest web browsers. And so I think I probably stole that from him at some point.

• 48:23Alex Kantrowitz?

  How do you think you would feel if people inside Amazon started using generative AI for their six pages?

• 48:29Speaker 02

  That ship has sailed, I can tell you.

• 48:32Alex Kantrowitz?

  People are doing it?

• 48:33Speaker 02

  For sure, yeah.

• 48:34Alex Kantrowitz?

  What?

• 48:34Speaker 02

  Of course.

• 48:35Alex Kantrowitz?

  Okay, but hold on.

• 48:36Speaker 02

  It's a starting point.

• 48:36Alex Kantrowitz?

  Wait a second, because the whole point, if I have it right from Bezos, is that when you write something, you have to think it through super deeply and make sure every idea connects one to one. If you turn that over to AI, you're not really going through the process.

• 48:51Speaker 02

  Well, I don't think that's true, because what you're getting back is just a first draft. And so that's actually a really good way to explore your idea. You can get a gut check as to whether your idea kind of tracks, whether it's got legs, and you can start to poke and prod at the idea. And all of our ideas get better through that poking and prodding and the discussions that we have around their ideas. And so to be able to do more of that early on, you actually front-load a lot of the product development work, and you can do some of that with your team, you can do some of it on your own, do some of it, you know, with an LLM. I think it makes perfect sense. It's a huge efficiency gain.

• 49:32Alex Kantrowitz?

  Can you, when you read one of these six pages written with an LLM, can you tell that it's been involved in the process?

• 49:40Speaker 02

  I don't know if I've read one that was completely written autonomously with no edits.

• 49:45Alex Kantrowitz?

  But even with a little bit of help.

• 49:46Speaker 02

  I think that, I think for sure, for sure, that I have read paragraphs, maybe even pages, that were automatically generated with, probably some pretty heavy editing that I did not notice.

• 50:01Alex Kantrowitz?

  That's good.

• 50:02Speaker 02

  Very encouraging.

• 50:04Alex Kantrowitz?

  Two more for you.

• 50:05Speaker 02

  Okay.

• 50:07Alex Kantrowitz?

  Amazon's culture. The whole point, I mean, I wrote a book, the title is called Always Do One. So. The point of the book is that the company operates as if it's a startup on its first day. And the culture has been extremely intentionally built that way by Bezos. And there was a story recently about how Amazon has more of a big company feel lately. And you even have Adam from AWS. I want to make sure I get the language right, saying, you know, we're going to be insurgents. And you only say this word, we're going to be insurgents, when you feel like you need that rallying cry. What's, what's the story?

• 50:49Speaker 02

  Uh, I, I see your theory. It's an interesting theory. I personally have not seen any, well, number one, I don't really know what big company stuff. I've only ever really worked at Amazon. And so I've been here for, for a long time. I'm pretty well entrenched in, in the culture.

• 51:08Speaker 02

  But I haven't seen many elements of big company slowness. I haven't seen many elements of big company politics. I haven't seen many elements of big company infrugality or wastage. And so I think, I'm sure there's many more that you, you could list off that would be qualities of big company-ness that would be negative. What I have seen is a continued focus on working backwards from the customer. And what I have seen is a continued focus on scrappiness. And a continued focus on doing what we need to do in order to be able to solve real problems on behalf of customers. I think you, you'll see that in our approach to generative AI.

• 51:50Speaker 02

  You'll see it in our approach to analytics and satellites and all sorts of things.

• 51:54Alex Kantrowitz?

  Yeah, because you, I mean, you really need that sort of scrappiness if you're going to be able to compete. I mean, I. I feel stupid even saying this out loud, someone who's worked at Amazon for as long as you have, but this is gonna be a fight, man.

• 52:08Speaker 02

  It's gonna be a very, very interesting time, for sure. And, you know, I would also say that the sort of cultural norms that we'd have, they don't exist and they're not maintained without some energy and without some effort. And anyone that has had a conference call with me from my office will have seen that behind me, on my office wall, I fly a pirate flag. Partly an homage to Steve Jobs.

• 52:36Alex Kantrowitz?

  You have the pirate?

• 52:37Speaker 02

  I have the pirate flag.

• 52:38Alex Kantrowitz?

  Okay. When I was writing the book, I heard about this pirate mentality.

• 52:41Speaker 02

  Yes.

• 52:42Alex Kantrowitz?

  And I could never nail it down. Talk a little bit about it. Yeah, this is good.

• 52:45Speaker 02

  It's part homage to Steve Jobs and the early Mac team, like, very early day one, drove a ton of transformation. I'm a huge fan of Apple, huge fan of the Mac. I've used Macs all my life. So part of it is an homage to that. But part of it is, in periods of discontinuous change, you just can't operate like a big supertanker. You've got to operate like a small merry band of pirates that are just cruising and adventuring around every cove that you can think about and just staying scrappy and nimble. And so, for my part, such as it is, I fly the flag in my office as a reminder to myself and any of the teams that I'm working with that this is a period of discontinuous change.

• 53:25Speaker 02

  And this is a time in which we need to be scrappy and resilient and explorers and missionaries.

• 53:33Alex Kantrowitz?

  And the people are listening?

• 53:35Speaker 02

  So far, so good. People seem to like my flag.

• 53:38Alex Kantrowitz?

  All right. This is not the last one, but I have to ask you about this. The news is gearing up for the FTC to bring up a lawsuit to break up Amazon. Obviously, it doesn't happen yet, it's all speculation, but it seems like it will, and it's gonna be like potentially the biggest government action against a US company since Microsoft, maybe even Ma Bell. So, do you think about that at all? Is it even something that you pay attention to?

• 54:09Speaker 02

  That one is above my pay grade.

• 54:10Alex Kantrowitz?

  Okay. Last question for you.

• 54:15Alex Kantrowitz?

  You know, you have a very interesting position within Amazon because you're like really working industry by industry and helping them imagine how they're going to transform with the latest technology. But we're in the middle of this really unbelievable moment in technology where we're starting to really get a chance to imagine things we couldn't before. One example, you guys have released a medical note-taking generative AI application, HealthScribe. So I'm a son of a foot doctor and my dad spent too big, too big of a chunk of his life writing notes. And just think about all the hours like he could have had back.

• 54:55Speaker 02

  I went to med school before I joined Amazon.

• 54:58Alex Kantrowitz?

  And it's just, it's, think about how much better care you can provide to patients if you're actually focused on that versus doing these things that generative AI applications can do. So why don't you take us like on a top three interesting things in different industries that you could imagine generative AI having a real impact in. I think that's, the medical one is interesting.

• 55:22Speaker 02

  That's a really good one.

• 55:23Alex Kantrowitz?

  What else? Where are some other examples that we're just not looking at yet?

• 55:27Speaker 02

  Well, number one, I am sure that everything I'm going to touch on, that there is a startup or even a large organization now, they're already working on it and they can, they'll probably be getting ready to ship as we speak. There's just so much investment and activity happening on this area. I think that a couple that spring to mind, the first one is cybersecurity. There seems like such an opportunity to employ these language models in the identification of the very subtle signals that have become harder and harder to identify, which indicate some sort of vulnerability or threat. And so being able to identify those threats across

• 56:08Speaker 02

  multiple different sources with better precision is going to be better for everybody. I think that's one. It's not really an industry, but I think it's one that's going to be important.

• 56:18Alex Kantrowitz?

  That counts.

• 56:19Speaker 02

  That still counts. Okay, good. I think another one is just going to be developer productivity, like code generation. We didn't really talk about that yet. Such a large accelerant.

• 56:27Alex Kantrowitz?

  Whenever I talk about that with CEOs, they're like, yeah, we're doing it, but they haven't really seen. The productivity increase. Now, I know you have seen it internally, but

• 56:36Speaker 02

  We for sure have seen it internally and we've heard from our customers that

• 56:39Alex Kantrowitz?

  There's definite.

• 56:39Speaker 02

  Yeah, that it's also early and I wouldn't be at all surprised if customers are still trialing it out and getting a sense for it. Developers are very very used to a particular workflow and changes to that workflow can can take time to get right and they should be thoughtful about it. But I think that's another one that's really that's really going to drive change. And then just more generally, I think any industry that has access to very, very large volumes of text is going to be the the first places that we see this sort of change. And what's interesting about that is there's some areas like healthcare that are steeped in natural language, but they don't usually have like the best reputation for being at the vanguard for technology adoption.

• 57:23Speaker 02

  We're seeing so much interest and so much excitement. Healthcare

• 57:27Alex Kantrowitz?

  Lawyers.

• 57:27Speaker 02

  Legal, healthcare, life sciences, clinical trials, drug discovery, all these areas where financial services, insurance, like the oldest, stodgiest industries that you can imagine. They've got so much natural language. It's such a large opportunity. I think that's where we'll see the the earliest returns potentially on generative AI.

• 57:48Alex Kantrowitz?

  Matt, can you believe this? I mean, what a great

• 57:51Speaker 02

  I love these guys.

• 57:52Alex Kantrowitz?

  So for listeners at home, what we're looking at is it's a silent disco type of conversation where everybody is wearing headphones. We're not even using any amplified sound at all. And just been looking at this crowd as we've gone and they've sat here and hung on every word. So

• 58:10Speaker 02

  Yeah.

• 58:11Alex Kantrowitz?

  Thanks to you guys. This is awesome.

• 58:12Speaker 02

  Thank you so much.

• 58:12Alex Kantrowitz?

  Thank you for being here.

• 58:18Alex Kantrowitz?

  And I'm gonna thank Matt in a second, but I'd be remiss if I didn't mention that the Big Technology Podcast airs every Wednesday and Friday. Wednesday, a flagship interview. Like this Friday, we cover the news. All right, everybody. Thank you so much. Thank you to you. Thank you to Matt.

• 58:33Speaker 02

  Thank you.

• 58:33Alex Kantrowitz?

  And I hope you enjoy the rest of your day.

• 58:35Speaker 02

  Thanks.

• 58:47Speaker 03

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, theHouseMajorityPact.com. Not authorized by any candidate or candidate's committee.

• 59:16Speaker 04

  In this life, you can't win.

• 59:18Speaker 05

  When the world gets you down.

• 59:19Speaker 04

  No!

• 59:20Speaker 05

  Escape to Pluto TV.

• 59:21Speaker 04

  This is objectively fantastic.

• 59:23Speaker 05

  Relax into the movies and shows you already know and love, all for free.

• 59:27Speaker 04

  Shwing!

• 59:28Speaker 05

  Even our news is uplifting.

• 59:29Alex Kantrowitz?

  You stay classy, San Diego.

• 59:31Speaker 05

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 59:37Speaker 04

  We're not worthy!

• 59:38Speaker 05

  The best things on TV really are free.

• 59:41Speaker 04

  You just have to know where to look.

• 59:42Speaker 05

  Escape to Pluto TV. Stream now, pay never.