Going Full Send on AI, and the (Positive) Impact of AI on Jobs, with Kevin Scott, CTO of Microsoft

In this episode, Sarah and Elad speak with Microsoft CTO Kevin Scott about his unlikely journey from rural Virginia to becoming the driving force behind Microsoft's AI strategy. Sarah and Elad discuss the partnership that Kevin helped forge between Microsoft and OpenAI and explore the vision both companies have for the future of AI. They also discuss yesterday’s announcement of “copilots” across the Microsoft product suite, Microsoft’s GPU computing budget, the potential impact of open source AI models in the tech industry, the future of AI in relation to jobs, why Kevin is bullish on creative and physical work, and predictions for progress in AI this year. No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode. Show Links: May 23, 2023: The Verge - Microsoft CTO Kevin Scott Thinks Sydney Might Make a Comeback May 23, 2023: Microsoft Outlines Framework For Building AI Apps and Copilots January 10, 2023: A Conversation with Kevin Scott: What’s Next In AI Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @kevin_scott Show Notes: [00:00] - Kevin Scott's Journey to Microsoft CTO [12:44] - Microsoft and Open AI Partnership [21:18] - The Future of Open Source AI [32:12] - AI for Everyone [45:29] - AI and the Future of Jobs [51:44] - The Future of AI and Regulation [58:10] - Taking a Global Perspective

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Sarah Guo?

  Microsoft, the BMF productivity cloud and gaming company, has taken a massive bet on AI. Everyone's paying close attention to its partnership with OpenAI, and the technical community has been amazed by its release of some of the first truly useful and broadly deployed AI products, such as GitHub Copilot. Its full-on attack on web search with the new LLM-powered Bing Chat is making its incumbent competitors dance. Today on No Priors, we're thrilled to speak with Kevin Scott, CTO of Microsoft and the driving force behind their AI strategy. Kevin's leadership, both at Microsoft and prior at LinkedIn, Google, and AdMob as a technologist, is especially inspiring to me, given his distance traveled from his childhood home in Rugel Central, Virginia. In 2020, he published a book,

• 0:48Sarah Guo?

  Reprogramming the American Dream: About Making AI Service All. Kevin, welcome to No Priors. Thanks so much for joining us.

• 0:54Speaker 01

  Thanks for having me, guys.

• 0:55Sarah Guo?

  Can you start by sharing with us some of your story? How does one go from a farming community in Virginia, where your parents didn't attend college, to CTO of Microsoft?

• 1:03Speaker 01

  I don't know. I think it is a very unlikely journey. It's certainly not a thing that I ever could have imagined. I think part of it is I was just super fortunate to be wired like a nerd and growing up when I grew up. So when I was a teenager in the early 80s, personal computing was happening, and that was the thing that I happened to fixate on. And even though we were relatively poor, I managed to scrape together enough bucks to get myself a personal computer that I could have and just tinker with all the time.

• 1:45Speaker 01

  And it was a Radio Shack Color Computer, too, like one of these things with chiclet keys that you actually connected to a television. Like, I had it hooked up to a 13-inch TV, and it had a cassette recorder that you stored and loaded your programs on. And it was just the thing that I was obsessed with, and I stayed obsessed with computers from then on. And it was just me trying to find a path at each step where I could work on the most interesting thing that someone was dumb enough to give me permission to go work on. And again, it's a lot of luck. Like, there's no way you can plan a path from... Rural central Virginia, the CTO of Microsoft.

• 2:29Speaker 01

  But, you know, I think it does help to have a high level of vision in your head for what it is that you want to do. Like, just knowing what you're aiming for always helps.

• 2:41Sarah Guo?

  What was that vision for you, besides like, you know, obsessed with computers, wanted to work on them?

• 2:46Speaker 01

  Yeah, I more or less had two of them. So, the first vision I had when I was a teenager was I wanted to be a computer science professor. So, I just looked at what computer scientists did and thought, this is the most amazing stuff I've ever seen. And I went to a science and technology high school. And the way that it worked where I lived is like a really rural area. And so, the science and technology, it was a governor's school. So, it was centrally located and each high school in these four or five counties that surrounded the governor's school got to send two students each. And so, I was one of the two students that got selected from my high school to go to this thing.

• 3:31Speaker 01

  And my computer science professor there was this guy, Dr. Tom Morgan. And I just sort of felt like he'd opened up this entire new world to me. Like, it was just thrilling to learn all of this stuff. And I was like, yeah, I want to be like Dr. Morgan. And a lot of this stuff for me is about, you know, like who those influential role models have been in your life. And so, as soon as I like met Dr. Morgan, I was like, oh, I should just go be a computer science professor. And that was the path I was on until I was about 30 years old, when I, you know, I was a compiler optimization and computer architecture programming languages person. And I got pretty disillusioned with what being a computer science professor actually

• 4:15Speaker 01

  was relative to what I wanted to do. Like, I just wanted to have a lot of impact. And my perception at the time, when I was making these decisions, was that you can have a lot of impact as a computer science professor. And the impact was actually great, but it wasn't the impact that the system appreciated. So like the impact that you can actually have is inspire students to go pursue these careers and they will go on to do much greater things than than you've done yourself. And like that, that to me was the greatest impact, but it was the least appreciated part of being a computer science professor, like back in the 2000s when I was making

• 4:57Speaker 01

  these big decisions. And so I decided to leave and I didn't at the time know what next actually was going to be. Like it had been my mission for almost 15 years at that point. And like I was a little bit lost. And I saw that a bunch of my academic buddies were all working at this startup called Google. And I didn't understand why they were working at Google. Like Google was like some little box and you typed a keywords in and it gave you 10 names. Like, how is that hard? But, but you know, Oras Holzler, who was a compiler person and Jeff Dean, who was a compiler person and Alan Eustace, who was a compiler person, like all of these people who, you know, who I went to conferences with and whose

• 5:41Speaker 01

  papers I read. And I was like, all right, well, maybe I should send my resume in. And like I sent my resume in and got called to do a bunch of interviews and they like it was the best interviewing experience I've ever had because they, they took what must have been every compiler person in the company at the time and put them on my interview panel. And I was like, oh my God, this is amazing. Like I had the best day interviewing there. And I got this job offer and I went, I got this choice. They just started Google New York, which was the first office outside of Mountain View. And they were like, you can come to Mountain View or you can go be, you know, the 10th person in this New York office. And my wife and I wanted to live in New York more than we wanted to live in

• 6:25Speaker 01

  Mountain View. And so that's what we did. And after I got there, this is where the new mission came in. I just, so we were hiring these brilliant, brilliant people at the time. And the way that we did hiring was kind of crazy. It's like, all right, well, if you're smart, just come work here. And like, we have no idea like what exactly it is you're going to do. And you like came in and you sort of sorted yourself out. And we had these people who were so accomplished and so brilliant and they would come in and choose to work on things that that just were going to have no impact at all. Like they they were intellectually very interesting, but they were just sort of silly in that they were never going to connect with anything that moved the needle

• 7:07Speaker 01

  for the company, which was exactly the problem I was trying to get away from, uh, you know, in in being a like a research computer scientist. Uh, and so I sorted myself out like I found of like a pragmatic thing to go work on, like, you know, we like I I won't go into the details of what it is, but you know, like the whole team won a Google Founders Award, which was a big deal for like solving this like very sort of unsexy problem with a bunch of very fancy computer science, um, which was one of the things I think Google, uh, Google did really well. And then I was like, okay, well, I should just go help more people sort themselves out as well. And that's when I became a manager. And then from that

• 7:49Speaker 01

  point on, it was all about like, hey, I want to, I I want to help as many engineers as I possibly can, uh, like make sure that their work lines up with something that's, you know, both interesting and meaningful.

• 8:04Speaker 02

  I think that it's actually a pretty under discussed degree to which early Google had so many academics actually running important parts of the company.

• 8:10Speaker 01

  Yeah.

• 8:10Speaker 02

  I think ERZ is a great example, and I think there's others. And so I haven't actually seen anything like that since until maybe now more recently at OpenAI, there's more academics or, you know, you feel like the research community is popping back up again, but it's been maybe a decade or two since that's happened.

• 8:24Speaker 01

  Yeah, I mean, I I think that that's actually a really, really great observation. So when when I go sit in OpenAI, it really reminds me of early Google days and it's about the same size Google was when I joined. And so like I I I couldn't figure it out for a while. And I was like, wow, this is like really giving me like, you know, early Google nostalgia. And, you know, the the conclusion to draw from that is like not that they're the same companies or they're trying to solve the same problem. It's just sort of the energy of the place and like who they've chosen to hire and like.

• 8:56Speaker 02

  Yeah, it's the first time I've seen like string theorists getting hired again.

• 8:59Speaker 01

  Yeah.

• 8:59Speaker 02

  In the computer science roles.

• 9:01Speaker 01

  Yeah, 100%.

• 9:02Speaker 02

  You know, since Google days. Yeah.

• 9:03Speaker 01

  You you and I, uh, like probably both worked with Jonathan Zunger, who, uh, works at Microsoft right now. And like, I I remember like, it's like, all right, Jonathan's working on this big distributed file system stuff and like, what's his degree? Oh, yeah, he's a like string theory guy.

• 9:18Speaker 02

  Yeah.

• 9:19Sarah Guo?

  So a big part of your mission for, you know, the last decades has been helping string theorists and other engineers figure out how to be, how to be useful in their ORs. The other, the other part seems to be, of course, like actual technical direction, right? Deciding like what's worth investing in.

• 9:36Speaker 01

  Yeah.

• 9:37Sarah Guo?

  And you've worked on machine learning products for a really long time, like ads auctions at Google, recommendations at LinkedIn, et cetera, et cetera. Was there a moment when you decided or you realized personally that AI should be a key technical bet for Microsoft?

• 9:49Speaker 01

  Yeah, I mean, I, I've been at Microsoft a little over six years now, so almost six and a half years. And like pretty, pretty quickly, it was obvious that AI was going to be like very, very, very important to the future of the company. I think, you know, Microsoft already understood that before I got there. And then it was just how do you focus all of the energy on the company on the right thing? Because we had a lot of AI investment and a lot of AI energy and it was sort of very diffuse when I got there. So no lack of IQ and actually no lack of capital spending and everything else, but it was just, you know, kind of getting peanut buttered across a whole bunch of stuff.

• 10:31Speaker 01

  And so the, the thing that really catalyzed what we were doing is, I mean, maybe this is a little bit too, too technical, but like, you know, we, before, before I got there, the technical thing that had been happening with some of these AI systems that to me was very interesting is transfer learning was starting to work. So like you were going from this mode of, you know, the flavor of statistical machine learning that I cut my teeth on, like in my first projects at Google, which was, you know, you have a particular domain of data and like you have a particular machine learning model architecture that you are, you know, you're training and like a particular way

• 11:15Speaker 01

  that you're going to go do the deployment and measurement and whatnot. And it's all like, you know, siloed to a, like a, like a use case or a domain or an application. To seeing AI systems that you could train on one set of data and use for things for multiple purposes. And you saw a little bit of that with some of the cool stuff that DeepMind was doing with reinforcement learning with, you know, play transfer across some of the gaming applications that they were building. But like the really exciting thing was when it started working for language with Elmo and then, you know, BERT and then Roberta and Turing and, you know, like a bunch of things that we were doing. And that was the point where there were so many language-based applications that you

• 12:00Speaker 01

  could imagine building on top of these things if it continued to get better and better. And so we were just sort of looking for evidence that it was going to continue to get better and better. And as soon as we found it, like, we just started, like, all in. That was everything from doing a partnership with OpenAI to, you know, like, at one point, I seized the entire GPU budgets for the whole company and I was like, we will no longer peanut butter these resources around, like, we will focus them because it's all capital intensive. It's like, we will just allocate these things to things where we have really, really strong evidence-based conviction that, like, a particular path is going to benefit from adding more capital scale.

• 12:40Sarah Guo?

  I remember, it must have been like five years back now, we were at dinner and, you know, now GPU capacity is the talk of the technical town, right? But you were, like, I asked you what your, like, most pressing issue was and you were like, how am I going to spend on GPUs this year?

• 12:54Speaker 01

  Yeah.

• 12:55Sarah Guo?

  And how I'm going to distribute those GPUs?

• 12:57Speaker 01

  Yeah. Yeah, and it was and it has been, it has, it certainly hasn't gotten any easier. But, I mean, so Eli, like, I think, you know, the question you were asking is like, how we decided to do the OpenAI partnership. And so, like, the reason that we did the partnership was twofold. So, one is with transfer learning actually working, you can imagine building a platform for all of this stuff so that... You're building single things where you're amortizing the cost of the things across a whole bunch of different applications. And because we have a hyperscale cloud, like one of the things that I was really, really interested in and like beyond interested, like it felt, you know, just like an existential thing is how do you make sure that the way that you're building your cloud all the way

• 13:41Speaker 01

  from, you know, your computing infrastructure, your networks, your software frameworks and whatnot, how can it really serve a whole bunch of interests beyond your own? And so like, we felt like in addition to the high ambition things that we were doing inside of the company that we needed like high ambition partners. And when we looked around, like OpenAI was clearly the highest ambition partner that was in the field, you know, and I think still their ambition is just breathtaking in what it is that they're trying to accomplish. And so that was one thing. And then the second thing was like, you know, they, they really had a very similar vision to the one that I had about like, these things were evolving into platforms and like, we

• 14:24Speaker 01

  were able to, because we were so aligned on vision for the future, like we could figure out how to do a partnership where like, even though like, there's just a ton of difficult things and like, you know, I, I think there's probably some conservation law of, you know, the stress from difficulty. So it's not like it ever goes away, but like it, it's stress in service of a common goal. And like, that's the thing that make good partnerships work.

• 14:50Speaker 02

  I think one of the stunning things about the partnership in some sense was the timing because if I remember correctly, Microsoft made its first investment or its first significant investment in OpenAI right after GPT-2 launched or right around GPT-2. And this is before GPT-3 and there was such a big step function between the two of them that I think it was less obvious in the GPT-2 days that this was going to be as important as it was. And so I'm a little bit curious, like, what were the signs that made you decide that this was a good partnership to have versus building it internally versus, you know, usually as a larger company, there's the old like buy, build, partner kind of thinking. And so I'm just sort of curious, like how, how you all decided to, to partner in this moment in time where it's very non-obvious and you invested a large sum of money behind that.

• 15:29Speaker 01

  Yeah, there, and, and I like, I don't want to like have revisionist history and like paint a rosier picture than there actually was. So there was a huge diversity of opinions inside of the company on the wisdom of doing, uh, doing this. And so. Satya like has this thing that he talks about like no regrets investing. So like there are things where you do the investment and like there are multiple ways to win. And like you, you even win a little bit when you lose. And so this was one of those no regrets things in that like the very, very worst thing that could happen is we would go spend a bunch of capital on computing infrastructure and we would learn

• 16:13Speaker 01

  like what to do at very high scale for building these AI training environments. And you know, you'd have to believe something very strange about the world of AI that you wouldn't need advanced computing infrastructure. And then there were just multiple ways where, you know, like, and we had a bunch of evidence that, you know, we had gathered ourselves and that OpenAI had that gave us, you know, which unfortunately I can't talk about, but like that gave us, you know, pretty reasonable confidence that scale up was actually working. You've probably seen the, you know, the famous OpenAI compute scale paper where they sort of plot on the log scale, like how many, you know, petaflop days or, you know, whatever

• 16:55Speaker 01

  the, you know, the total compute they were using on that graph that shows, you know, from 2012, when we first figured out how to train models with GPUs through, you know, I think the, the plot ends sometime in 2018. Yeah, that we're basically consuming 10 times compute, more compute every year for like training state of the art models. And so like, you know, I just had super, super high confidence that we were never going to get to the point where we're like, all right, we got enough compute.

• 17:27Speaker 02

  It was a very bold move. I think it's very striking, all the amazing things Microsoft has done over the last few years in terms of just incredibly smart strategic moves that at the time didn't seem obvious and now are just in hindsight, you know, really brilliant. I guess a more recent move is you announced a collaboration with Nvidia to build a supercomputer powered by Azure infrastructure combined with Nvidia GPUs. Could you tell us a little bit more about your supercomputing efforts in general? And then maybe a little bit more about those collaborations with Nvidia and OpenAI on the supercomputing side?

• 17:55Speaker 01

  Yeah, so we built our, the first thing that we called an AI supercomputer. Uh, I think we started working on it in 2019 and we deployed it uh at the end of that year. And it was the computing environment that GPT-3 was trained on. And yeah, we, we had been building a uh like a progressively more powerful set of these supercomputing environments. Uh, like we built them in a way where like the biggest environments just because, you know, they're, they're very capital intensive things tend to get used for one purpose. But the designs of these systems, like we, we can build smaller stamps of them and they

• 18:40Speaker 01

  get used by lots of people. So like we have, you know, tons of people who are training, you know, very big models on uh Azure compute infrastructure, both folks inside the company and uh, you know, partners who can come in. And it, it was a thing that was like not possible to do before, where you could sort of say like, hey, I would like a, I would like a compute grid of this size with like this powerful network uh to do my thing on. And so, you know, Nvidia has been, you know, our compute and network partners since they bought Mellanox uh, you know, for years now. And the, and the thing that makes that work is generation over generation, like you're

• 19:22Speaker 01

  just getting better uh, you know, uh, price performance from the systems. Um, and, and we work super closely with them, uh, like defining, you know, what the hardware requirements need to be, um, you know, in the coming generations of GPUs, uh, cause like we have a pretty clear sense of where models are going and like what model architectures are evolving towards. Um, and so, yeah, I mean, it's just been a super good. Super good partnership. Yeah, like we're, we're deploying a hopper now at scale and you know, like a bunch of the features of hopper like, you know, eight bit floating point, you know, arithmetic and a bunch of other things are like things that, you know, like we've been planning for, for a while.

• 20:08Speaker 02

  Yeah, I guess one, one last question on sort of this both supercomputer as well as platform side of things is I'm a little bit curious how you view the world shifting in terms of closed source and open source models and, you know, the mix that'll exist because obviously from an Azure perspective, lots of people are running open source models on top of Azure right now.

• 20:24Speaker 01

  Yeah, I mean, it is an interesting thing that people are framing it as some kind of binary thing. Like, I think you're gonna have a lot of both. Like, we, we still don't see any reason to believe that you're going to want to not build bigger models. But like, we just know in our own deployments, like if you look at things like Bing Chat or Microsoft 365 Copilot or GitHub Copilot, you, you end up using a portfolio of models to do the work and like you use it for a performance and cost optimization reasons and you use it for, you know, just sort of precision and quality reasons sometimes. And so there's

• 21:09Speaker 01

  always this, you know, melange of things that you're, that you're doing and it's never either or. I'm, I'm actually really excited by what's going on with the open source community. I think, you know, my biggest question mark there is like how you go deal with like all of the REI and safety, safety things. But like, if you look at the technical innovation inside of the open source community, like it's really, you know, thrilling and like, we, you know, like we're doing some cool stuff right now. Like I was just playing around yesterday with that 12 billion parameter DALL-E 2.0 model from Databricks, which like runs quite nicely on a single machine. And like, you know, I'm still enough of a dork to like

• 21:50Speaker 01

  love playing around with things that run on single machines. Like it's, you know, really, really impressive work.

• 21:56Speaker 02

  Yeah, yeah, yeah. It's super cool. How, how do you think about that from the context of enabling AI for your business customers outside of your core products? So is there a specific sort of B2B AI stack that's coming? Are there specific tools coming? To your point, there's safety, there's analytics, there's fine tuning, that, you know, there's so much stuff that you could potentially provide. I'm just sort of curious how you think about that.

• 22:16Speaker 01

  Yeah, I mean, I don't want to turn this into some kind of weird marketing spiel, but you know, we have this point of view that we started with this assumption that AI is going to be a platform and the way that people are going to most usefully make or the way that people are going to make most use of the platform is by building tools that assist people with jobs. So it's like less about these fully autonomous scenarios and more about assistive tech. And so the first thing that we built was GitHub Copilot, which is a coding tool, a thing where you can sort of say in natural language what you would like a piece of code to do and it emits the code and then you, you as the developer, like the same way that you would take a suggestion from a pair programmer,

• 23:01Speaker 01

  like you scrutinize it and code review it and, you know, decide whether or not it makes sense for your application. And, you know, and like that was, that was the first version of GitHub Copilot that does a bunch of other things now. And so the thing that we have observed is this copilot pattern is actually pretty, you know, pretty generic. And, and we, we built a bunch of copilots since then. And the way that we built them, like there's a, there's, there's a copilot stack that looks almost like one of these OSI, you know, networking diagrams. And it starts with a bunch of user interface patterns that

• 23:41Speaker 01

  you have, like they're now an emerging plugin ecosystem for like how you extend the, you know, the capabilities of a copilot for things that you can't natively get out of the model. And then it is a whole stack of things, you know, sort of an orchestration mechanism like Langchain is, you know, one of the popular open source orchestrators, but like there are a bunch of open source orchestrators, like we have one that we've developed called Semantic Kernel that we've also open sourced. There is this whole fascinating world right now that didn't exist nine months ago around prompt construction and prompt engineering. And so like there's an entire art form and a set of tools that, that people have access

• 24:26Speaker 01

  To to design a meta prompt, which is sort of the standing instructions to the model to like get it to conform itself to the application context that it's in. Like you have these new things, like new software development patterns, like retrieval augmented generation or RAG is like we were doing this before it had a name on it. And you know, so it's basically a way to like, take the prompt that's flowing from the application and to inject context into the prompt that will help the model better respond. And then there's a whole bunch of safety apparatus that you have. So that looks a lot like filtering on both the way down as the prompt flows through the stack

• 25:11Speaker 01

  all the way down to the model as well as, you know, as it flows back up. So what things are you not going to let a let the application or the user send all the way down to the prompt because it's going to get a bad response back or like, you know, what things are you going to filter out at the last minute because like it is a bad response that has gotten all the way through. You know, and sometimes like you have multiple round trips through this cycle before you like bubble the thing all the way back up to the user to get them the response that they need. And so like, you know, we have a point of view about what all what the stack looks like, you know, which Microsoft tools exists that will help people build these

• 25:52Speaker 01

  things and like what special things you have to go do in the context of an enterprise to like answer the actual direct question where you know, safety and data privacy and like understanding, you know, where the flows of data are and like which plugins can be enabled and like which can't, like all of those things like I think are getting built out right now.

• 26:17Speaker 01

  And like the other thing too, I'll say is like, we'll build some of this stuff and like, the community is going to build a tremendous amount of it because like there's never been a platform or ecosystem where one company builds all of the useful things like that's just nonsense. Like it's just never happened. And to me, it's the sort of super exciting thing to just see all of the energy that's happening right now. Like I just like immediately before this call, I was doing a review with Microsoft Research and it's just amazing to watch MSR, which is so many researchers are there have pivoted what they're doing

• 26:56Speaker 01

  research on to like these AI adjacent or AI like on point things. And it feels a little bit like what MSR was like when I was an intern there in 2001, where you had all of these super bright people who like had the tiniest little glimpse of what the future must look like that no one else had because it was the point where the PC was racing to ubiquity. And like they were just all orienting their research around like what that little glimpse was that like maybe they had the earliest peek at. And it just is like feels magical.

• 27:34Speaker 02

  That's massive realignment of the research community right now, sort of in real time. It's very exciting to watch.

• 27:38Speaker 01

  I mean, and it's awe inspiring. I mean, it's just crazy. It's hard to keep up, like super hard. Like we went from, I mean, this has been the biggest surprise for for me is like, I just didn't realize that GPT-4 and ChatGPT were going to catalyze as much of this as they have. Like we'd sort of kind of been expecting a bunch of this stuff. You know, ChatGPT was a 10-month-old model with a little bit of RLHF on top of it. And you know, like by, you know, admission, like, you know, not a beautiful user interface. It was just sort of a way to get something out there because you needed some.

• 28:18Speaker 01

  Some practice with a handful of things before the big G P T four launch was was coming. And like, no one really knew that it was going to blow up this way.

• 28:29Speaker 02

  And it's only five months old. That was only five months ago, which is shocking. I think everybody forgets how little time has passed.

• 28:35Speaker 01

  Yeah, just shocking. But but it is the open source community and like the the, you know, big tech community, I think, at its best is like, you know, everybody is sort of realigning to like what I think is, you know, unlike some of the other, you know, faddish things that have happened over the past handful of years. Like, I don't think this is a fad. Like, this is this is real.

• 28:54Sarah Guo?

  Yeah, I launched my new fund about six months ago with this A I focus. And a few weeks later, Chat G P D comes out. And I'd say even the people who are very prepared, like hopefully somewhat prepared to go like, try to keep up or be part of that massive shift, like feel constantly upended. But it is it is very, it's the most fun time to be in technology in decades.

• 29:17Speaker 01

  Yeah, it look, it's all it's also, I will say, a disconcerting time to be in technology, because so many things are changing at once. It's changing at a pace that, you know, you probably like, e- even me, like, I I'm, I think I might be in one of the better positions to like, feel like I'm kind of in control of what's going on and like, I'm not in control at all, like of the pace. And so it must really be disconcerting to folks, you know, trying to keep up with everything that's going on. And in some cases, like, it's forcing people to change their worldview about things, like worldviews that they've held for a really long time.

• 29:57Speaker 01

  I think it's honestly harder for some machine learning people than it is, you know, for like a brand new entrepreneur who's, you know, just looking for an interesting thing to go do, because it is a very different way for a machine learning team to do its work. And it's like been hard, you know, even for some of the people at Microsoft who have. Had plenty of time to think about the transition to like get adjusted to like this new way of doing things.

• 30:26Sarah Guo?

  I want to ask you one more question that is sort of advice for people making the adjustment in a certain sense and then, you know, talk about your book, talk about the macro and such. Microsoft has a unbelievably wide portfolio of products and now you're on the other side of all the infrastructure questions, figuring out the, you know, organization of adoption of all these capabilities into that portfolio, right? I talked to, you know, friends who run large companies, started large companies all the time that are also figuring out how to do this. How do you, how do you organize that effort? What, what advice do you have for them?

• 30:59Speaker 01

  I think you have to be, you have to remember that some things have changed and some things haven't changed at all. And so like one of the confusing things that I think there is for folks that, that many people get wrong is like models aren't products and infrastructure isn't a product. And so, you know, you, you, you need to very quickly understand what it is this new type of infrastructure and this new platform is capable of, but that does not mean that you get to not do the hard work of understanding like what a good product is that uses it.

• 31:42Speaker 01

  Like what, one of the things I tell a lot of people is probably the place where the most interesting products are, are where you've made the phase change from, from impossible to hard. So like something that like literally you couldn't do at all before this technology exists has become hard now because like the, the things that have gone from impossible to easy are probably not interesting. And like the, my frivolous example of this is when smartphones came on the market 15, 16, 17 years ago now, like it's a, yeah, 2007, I guess was iPhone launch, right? So 16 years ago, almost. And then a year later, you had the App Store.

• 32:26Speaker 01

  So like the first apps were like things that had gone from impossible to easy. And like they just. We barely remember them. Like, there were all these fart apps. There was like, you know, like this app I had on my phone at one point that was called the woo button. You pressed it and it like did a woo like Ric Flair. Like, those are not businesses. Like, they're just, you know, sort of like these explorations that people are doing. Like, the things that have made the smartphone platform are the hard things that like went from impossible to hard. They also are kind of the non-obvious things. Like, they weren't even the things that the builders of the platform imagined. Like, you know, we don't even think the

• 33:09Speaker 01

  original applications on these platforms, like the things that launch when the platform first launched, like, those are not the interesting things anymore. Like, your smartphone is way more than just a SMS app and a web browser and a mail client. Like, the thing that makes it interesting is TikTok and Instagram and WhatsApp and DoorDash. And like, they were all of these hard things that people had to go build now that they were possible. And so like, I think that's thing number one to, you know, hold in your head either as an entrepreneur or as a business that's trying to adopt this stuff. It's not like how I go sprinkle some LLM fairy dust on my existing, you know, products and do some stupid incremental thing. And like, you know, and I shouldn't even call it stupid. Like, maybe the incremental

• 33:52Speaker 01

  things are fine. But like, the really interesting things are non-obvious and very not incremental. And so that is the hard thing for us, is you have an entire group of people who are smart and like, they can see all of the things that are possible. And so the challenge is to steer them towards like, the hard, meaningful, you know, sort of interesting, non-obvious things that are possible. Like, not the, you know, like, things that are incremental, that, you know, just gonna burn up a bunch of GPU cycles and prevent you from, you know, and a bunch of product IQ that will prevent you from doing the things that really matter.

• 34:33Sarah Guo?

  If we, we sort of zoom out to like, non-technical audiences. You wrote a book in 2020, Reprogramming the American Dream. Can you describe who you want to read the book and what you hope they'll take away from it?

• 34:46Speaker 01

  I, when I wrote the book, it was not for people like us. Well, so the premise of the book is that I grew up in rural central Virginia. My, you know, dad was a construction worker. His dad was a construction worker. His dad was a construction worker. Um, yeah, my maternal grandfather like ran an appliance repair business and had been a farmer earlier in his life. So the thing, the thing that was true for everyone who was in my life, uh, like, you know, neighbors, members of the community is like, you know, they're just smart, entrepreneurial, ingenious people using the best tools that they could lay their hands on to go do things

• 35:29Speaker 01

  that mattered to them, that like created opportunity for them and, you know, sort of solve problems for, you know, their, their communities. Um, and I believe that like particularly this platform vision of AI where it's sort of getting cheaper and it's getting more accessible all the time. You know, like things, you know, like the stuff that we were chatting about, um, uh, you know, a few minutes ago about what I did at Google. Like I, you know, came in with a graduate degree. I was mathematically sophisticated, uh, and yet to do the thing that I, the first project that I did, which was a, you know, like a machine learning classifier thing in 2003, uh, 2003, 2004, like that was, you know, stacks of like super technical, you know,

• 36:13Speaker 01

  research papers and, you know, uh, this elements of statistical machine learning, you know, like you read it cover to cover and then you go write code for six months. Like high school student could do the whole damn project in four hours on a weekend now. Uh, like it's just, you know, like, and, and what's happening, like that aperture of who can use the tools is just getting bigger and bigger and bigger over time. And so like the book was trying to get people to be inspired by this notion that, uh, like don't be daunted and intimidated or scared by, uh, by AI. Like go embrace it and like try to plug it into the things that you're doing.

• 36:54Speaker 01

  And like, maybe, you know, we, we've got a shot at having more equitable distribution of, you know, who's benefiting from the platform as it emerges.

• 37:04Sarah Guo?

  If you were going to add an update chapter for the last few years where so much has happened, what, what would you focus on?

• 37:11Speaker 01

  Well, it's it's really interesting how much of it I think is still true. And like I had this anxiety the whole time that I was writing the book that I was gonna, by the time I had the manuscript and it hit the presses, that all of it was gonna be out of date. Like, the real problem I had is like, by the time it hit the presses, we had, we had a global pandemic and it literally, it hit the presses the week that everything shut down. So like, you literally couldn't buy it. Like, Amazon wasn't delivering anything other than essential packages and every bookstore in the country was closed. So it means it's a little bit surprising, you know, to me, like, how many of the ideas

• 37:55Speaker 01

  that, you know, we have a platform, platform's getting more powerful, it's getting more accessible, like, actually, the unit economics of it are getting better. Yeah, like, what you can do for, you know, a per token of inference is like, getting higher. You know, so like, I know everybody's like, in this frenzy around GPUs and like, which is this very expensive, expensive thing, but like, all of this optimization work is happening where you're able to squeeze more out of the compute that you have, and the compute's getting cheaper. So, yeah, I mean, the update that I would, I would add is that, and it may be an update that I do, like, it probably won't be this book, but like, I'm sort of

• 38:35Speaker 01

  contemplating, like, writing something right now. I do think that, that the public dialogue around AI right now is missing so many of the opportunities that we have to go deploy the technology for good. Yeah, like, all of the articles that you, you know, you read in the newspapers are, you know, around the responsible AI stuff, which is important, and like, the regulatory stuff, which is important. But, yeah, we should have a few articles in there as well about Sal Khan's TED Talk, which is just amazing, like, unbelievably good. And just for, you know, folks who may not have seen it, which they should go see,

• 39:16Speaker 01

  is like, you know, his problem is perfect for AI. So it's this two sigmas problem, this idea that

• 39:26Speaker 01

  Students who have access to high-quality, individualized instruction perform substantially better than those who haven't, like controlled for everything else.

• 39:37Sarah Guo?

  Just for our listeners' sake, the two sigma problem was the study by a guy named Benjamin Bloom, which showed that your average tutored student performed above 98% of students in a control class, which is one teacher, 230 students, like a normal American classroom, with reduced variance, which is amazing.

• 39:55Speaker 01

  Yeah, and if you believe that that's true, then you can also believe that every student, every learner in the world deserves to have access to that individualized, high-quality instruction at no cost, which seems like a reasonable thing. And then when you think about how you go realize that in the world, like, the only way that you can realistically do it is with something like AI. And so there's so many problems that have that characteristic, where we can all agree that, like, it is a universal good to do this. And then if you think about how to do it, like, you must conclude that AI is, like, part of the solution.

• 40:39Speaker 01

  Like, that is the, you know, the reason I get up every morning and deal with people yelling at me about, like, give me my GPUs, you know, for the fifth year in a row, is because of things exactly like that. And it doesn't mean that when you talk about that and you're hopeful and optimistic about those things, or even hopeful and optimistic about all of the things that, you know, venture-backed companies are gonna go do, or like, the way that businesses are gonna reinvent themselves, that you are also, say, you know, giving the middle finger to, you know, the responsible AI concerns, or, you know, the things that people care about on the regulatory front. Like, you can care about both of those things at the same time. But like, the thing that I can tell you is, like, there is no historical precedent

• 41:23Speaker 01

  where you get all of these beneficial things by starting from pessimism first. Like pessimism doesn't doesn't get you to optimistic outcomes.

• 41:33Speaker 02

  Yeah, it seems like to your point, a lot of the dialogue is really lacking from global education equity, global health equity, like all these things that AI as a platform should be able to produce because it's it's cheaper, it's personalized, it can do things at the level of a of a human in many cases in terms of being a great teacher or a great, you know, physician's assistant, etc. And so it really feels like that message is lost. And, you know, I think a lot of people don't mention enough how we're almost hopefully going to enter this golden age if we let this technology actually bloom and be useful. I guess the the question that I always have on my mind relative to all this stuff is, given the capabilities that AI continues to accumulate, how do you think about 20 years from now in terms of the best roles for people? And in particular, I think about

• 42:15Speaker 02

  it in the context of my kids. I'm like, OK, normally, two years ago, I would have told my my kids, go study computer science. It's the language of the future. What do you think is the right advice to give people, you know, in terms of what to study and that that will be the things that will be most durable relative to to the change that's coming?

• 42:33Speaker 01

  Yeah, I I think so. 20 years is a tough time horizon, you know, and I think if any of us are honest with ourselves, like if you rewind 20 years and you sort of imagine the predictions you would have made then, like, would you have gotten here? And like, nope, nobody would. But like, I think, you know, there there are just some sort of obvious things like my my daughter, for instance, like, has decided she wants to go be a surgeon. And like, I think surgeon is like a pretty good job. Like, we do not have a, you know, sort of robotics exponentials right now. Like, we've got a cognitive exponential. And so like, I think all of the like, the world is just sort of full of these jobs where, you know, really affecting change on a physical system, like doing something in the physical

• 43:18Speaker 01

  world, like all of those things, like, we will need probably many, many more of them than we have right now, like particularly in medicine, like nurses, surgeons, physical therapists, people who, you know, work in nursing homes, like, we have a, you know, rapidly aging population. And so like, the burdens on the healthcare system are going to get much higher. And, you know, I do think that AI is going to have some pretty Substantial productivity impacts, but like, you know, maybe it's just enough productivity impact to like, you know, make room for all of the other net new things that we will have to have there. You know, and so I, I think, you know, we got this weird thing in the United States where

• 43:58Speaker 01

  like we apportion less dignity and respect to like jobs like the ones that my dad had than we should. Um, you know, and I lived in Germany for a little while and Germany is a little bit different on this front. Like you can, you know, you, you can go be a machinist in Germany and, you know, like that's a really great career and something that your parents are, you know, celebrate. So like, I think they're like all of these careers, like, you know, electricians and machinists and, you know, solar installation technicians and I mean, just so many things that we're going to need, like especially because we're going to have to rebuild our

• 44:37Speaker 01

  entire, uh, power generation and distribution system, like in our children's lifetimes. Um, so like all of those jobs I think are super important. And then I, I would argue even that all of the creative stuff, like that, that we do, uh, there's going to be probably more need for that in the future than less, even though the tools that we're using to do the creative work, whether it's coding or making podcasts or whatnot, are going to help us be better at it. And the reason that I say that is humans are just extraordinarily good at wanting to put humans at the center of their stories.

• 45:22Speaker 01

  So, so like we, we, right now, we could be, you know, making, you know, Netflix shows, uh, you know, like not Queen's Gambit, but Machines Gambit, uh, like about, you know, a fleet of computers playing chess among themselves, uh, because they're all better than the, than the very best human. Nobody wants to watch that. Like the technology is probably good enough right now where you could have superhuman Formula One closed, uh, track drivers and Formula One cars that could, you know, do things that humans can't do. Nobody wants to watch that. Um, yeah. And like you even go back before computers, like forklifts are stronger than people. You could like go have a strong man or a strong person competition that was about like which forklift could lift the most weight.

• 46:07Speaker 01

  Like, nobody cares about that. Like we care about. Humans like, what are we saying? What do we care about? Like, what are we trying to express to everyone else? Like, and nothing about that's gonna change. Nothing.

• 46:20Speaker 02

  I think that's why people watch The Real Housewives of Dubai.

• 46:25Speaker 01

  Yeah. And so, and I don't, again, like, I don't want to paint too rosy a picture. Every time you have a major technology platform or paradigm shift, like, there's disruption. But like, what we know from every one of these disruptions is you have, like, actually a surprising degree of need for human occupation. Like, all of the, you know, the industrial revolution predictions about, you know, four-hour work weeks and we're all gonna live lives of leisure is bull crap. Yeah, like, just hasn't happened. And I think some people may say it hasn't happened because,

• 47:06Speaker 01

  you know, the system, you know, the system, like, doesn't want it to happen. But like, I think a lot of it is because, like, we actually like doing things.

• 47:15Speaker 02

  Yeah. And there's a lot to do. I guess, on that note, what are some of the areas you're most excited about going forward in terms of the coming year of AI? Or, you know, big research areas or big product areas or things that, you know, you're very optimistic about?

• 47:27Speaker 01

  I think sort of two things. Just, I think this will be the, maybe the great first year of foundation model deployments where you're just gonna see, like, lots and lots of companies launch, lots of people trying a bunch of ideas. You're gonna see all of the big tech companies will have substantial things that they're, that they're gonna be building. You know, like, I got predictions about what other folks will do, but, you know, like, it will touch all of Microsoft's product portfolio. Like, the way that you will interact with our software will be substantially different by the end of this calendar year than it was coming in. And I think that'll be true for everyone. I think

• 48:12Speaker 01

  it changes some of the Uh, nature of the competition that you've got between big tech companies and like, I think it creates new opportunities for small tech companies to come and drive wedges and to, you know, get footholds and do interesting things. Um, you know, one of the things that Sam Altman and I have talked about a lot is like, I, I suspect that this year, like the next trillion dollar company gets founded. Um, it won't be obvious which, uh, you know, which it is, but like, um, we're overdue, like long overdue. And, and then I think, you know, what you're going to see technically this year is, uh, I, I do think that you

• 48:54Speaker 01

  will have things like, um, the red pajama project, uh, is like this, and, and there're going to be a bunch of others like it, uh, will make really good progress on building more capable open source models. Um, yeah, and hopefully, hopefully the community will help build some of the safety, uh, safety solutions that you will need to accompany those things when you deploy them. Uh, but like technically, I think you're just going to see amazing progress there. And then like, it's just, yeah, the, the frontier will keep expanding out. Um, yeah, we, we don't have, OpenAI doesn't have GPTV and wide distribution,

• 49:36Speaker 01

  but like it'll get to wide distribution at some point in the not too distant future. And so like, you'll have these like mul- very powerful multimodal models, the same way that having GBD four, uh, admitted like all of this exciting energy around new things that you could do with it, like having a, having a model that can like take visual inputs and like reason over them, like will also admit a whole bunch of new things, uh, that are going to be very exciting. So I don't know, like that, that's like, I, I just think the, the, the theme of this year is going to be like progress and activity, like almost too much to track. Like I'm, I'm going to need a, I'm going to need a co-pilot just to like pay attention to all of this stuff and like make sure I'm not missing important things. Cause I feel like I'm, I'm at the, and you all as investors

• 50:21Speaker 01

  and as people who are watching this closely must feel the same thing. It's like, um, how do I make sure I don't miss like, you know, the next important thing? How do I see it as soon as humanly possible?

• 50:33Sarah Guo?

  Actually, just to make completely sure, if you are starting the next trillion dollar company in our listener base this year, please call me and Alad and Kevin too. Wrapping up now, is there anything else you would want to touch on, Kevin?

• 50:47Speaker 01

  Look, I think the dialogue that we're having right now around regulation is actually really quite important. So as we're recording this, Sam Altman was testifying in front of the Senate Judiciary Committee on like Tuesday of this week. I think more of those conversations are a good thing. I think as fast as things are moving, like you really will need the technology community to come together and to agree on some sensible things that we can do before the regulation even is in place. And I think that's all important and not a thing like the thing that none of us

• 51:29Speaker 01

  should be doing at this point is sort of like looking at the prospect of regulation and saying, oh, my God, this is a, you know, this is like a pain. Like, I don't want to deal with this. Like, the fact that there is a desire for it is like a very good signal that the things that we're working on actually matter because like nobody's trying to regulate frivolous things. And like the purpose of regulation is to make sure that you can build a solid, trusted foundation for things that maybe become ubiquitous in society. Like, if you think of this like electricity, for instance, you want to strike the right balance between allowing the technology to develop and make progress and flourish, but like you also need to make sure that your electric

• 52:11Speaker 01

  power generators are built safely and you don't allow people to wander in and like stick their finger on the electrode and disintegrate themselves. And like you want to make sure that, you know, the distribution of electricity is coordinated and that, you know, when it comes into your house, like it doesn't burn your house down. And when you, you know, like you plug your appliances into the wall that, you know, they function as as expected. And so like I think that is, you know, a similar way. Like, there's not going to be one size fits all. Like, I think the most of the stuff that people ought to be thinking about is deployments, like making sure that like as you deploy the technology, getting, you know, the requirements and the expectations right there is the most important thing. And then

• 52:57Speaker 01

  You know, these big engines that we're building that are the like the largest of the foundation models, like, you know, making sure that you have a set of safeguards around those, but like also the way that we're building these things, they don't, they don't get distributed to the world like by themselves. Like there's a whole layer of things on top of them to like render them safe. And then a whole set of things per application, per deployment that we do to like, make the deployment safe. And so like, you know, I think everybody, like all the startups, like everyone in the open source community, everybody ought to be thinking about

• 53:37Speaker 01

  these things. Like, how am I doing my part to make sure that we are creating as much space as possible for these optimistic uses? And like, we are deterring as many of the harmful ones as possible.

• 53:53Speaker 02

  Yeah, I've been impressed by the degree to which the community has self acted from very early days in terms of AI safety and approaches to that. And so I know OpenAI has done stuff really early, Anthropic has, Google has, Microsoft has, you know, I feel like a lot of the main players have actually been, you know, remarkably thoughtful about this area and, you know, keen to make sure that it's done properly.

• 54:12Speaker 01

  Yeah, I mean, it, the thing that I will say is, we fiercely compete with a whole bunch of these folks. But like, one of the things that I don't do is like, look at any of those companies that you just named and like, worry that they're going to do something that like, is like, I take myself out of my role as CTO at Microsoft and like, just think about Kevin, Citizen of the World. Like, Kevin Citizen of the World is not worried about like, what my competitors are gonna do to like, do something unsafe. Like, I'm just not.

• 54:43Sarah Guo?

  Thanks so much for being with us, Kevin. We really appreciate it.

• 54:45Speaker 01

  Yeah, thanks for inviting me. This is awesome.

• 54:47Speaker 02

  Yeah, thanks so much for the time. That's great.