GitHub CEO Thomas Dohmke on Building Copilot, and the the Future of Software Development

GithHub invented collaborative coding and in the process changed how open source projects, startups and eventually enterprises write code. GitHub Copilot is the first blockbuster product built on top of OpenAI’s GPT models. It now accounts for more than 40 percent of GitHub revenue growth for an annual revenue run rate of $2 billion. Copilot itself is already a larger business than all of GitHub was when Microsoft acquired it in 2018. We talk to CEO Thomas Dohmke about how a small team at GitHub built on top of GPT-3 and quickly created a product that developers love—and can’t live without. Thomas describes how the product has grown from simple autocomplete to a fully featured workspace for enterprise teams. He also believes that tools like Copilot will bring the power of coding to a billion developers by 2030. Hosted by: Stephanie Zhan and Sonya Huang, Sequoia Capital Mentioned in this episode: Nat Friedman: Former Microsoft VP (and now investor) who came up with the idea that Microsoft should buy GitHub Oege de Moor: Github developer (and now founder of XBOW) who came up with the idea of using GPT-3 for code and went on to create Copilot Alex Graveley: principal engineer and Chief Architect for Copilot (now CEO of Minion.ai) who came up with the name Copilot (because his boss, Nat Firedman, is an amateur pilot) Productivity Assessment of Neural Code Completion: Original GitHub research paper on the impact of Copilot on Developer productivity Escaping a room in Minecraft with an AI-powered NPC: Recent Minecraft AI assistant demo from Microsoft With AI, anyone can be a coder now: TED2024 talk by Thomas Dohmke JFrog: The software supply chain platform that GitHub just partnered with 00:00:00 - Introduction 00:01:18 - Getting started with code 00:03:43 - Microsoft’s acquisition of GitHub 00:11:40 - Evolving Copilot beyond autocomplete 00:14:18 - In hindsight, you can always move faster 00:15:56 - Building on top of OpenAI 00:20:21 - The latest metrics 00:22:11 - The surprise of Copilot’s impact 00:25:11 - Teaching kids to code in the age of Copilot 00:26:38 - The momentum mindset 00:29:46 - Agents vs Copilots 00:32:06 - The Roadmap 00:37:31 - Making maintaining software easier 00:38:48 - The creative new world 00:42:38 - The AI 10x software engineer 00:45:12 - Creativity and systems engineering in AI 00:48:55 - What about COBOL? 00:50:23 - Will GitHub build its own models? 00:57:19 - Rapid incubation at GitHub Next 00:59:21 - The future of AI? 01:03:18 - Advice for founders 01:05:08 - Lightning round

• 0:00Speaker 00

  The human brain is still so much more advanced than than the transformer models and the diffusion models and the other types of models that we have to image recognition and whatnot that we have today. And, you know, it remains to be seen if we can kind of like add that sentience piece to it. But today, I'm not seeing it and I haven't seen any research that telling me that that's coming anytime soon.

• 0:40Speaker 01

  Hi, everyone. Welcome to Training Data. Today, we host Thomas Dohmke, CEO of GitHub. Thomas has an ambitious vision for enabling a world of 1 billion developers and bringing agents through the end-to-end developer workflow and into adjacent categories like code security. He even hints at the progress he thinks the industry will make on Sweetbench over the next few years. Some categories he's excited about in AI outside of developer tools, and whether he thinks a new architecture will overtake the transformer.

• 1:10Speaker 01

  Today, we're so excited to introduce our special guest, Thomas Dohmke, CEO of GitHub.

• 1:14Speaker 00

  Hey, and thank you so much for having me.

• 1:16Speaker 01

  Thomas, we're so excited to dive into the GitHub and Copilot story in particular today. Maybe to kick off, we'd love to learn a little bit about your personal background. You have a very interesting story, having grown up in East Berlin before the wall fell, and then starting your first company that brought you into the United States when Microsoft acquired the company. How did your background and upbringing really shape who you are today?

• 1:40Speaker 00

  I think, you know, I'm living a very normal life, like the American dream now, is with a wife and two kids. I think what shaped really my journey was the passion for software development early, you know, when I was 11 years old or so. It was still East Germany and West Germany is there was the wall between the two parts of Berlin and I saw computers for the first time. I couldn't buy one, but I, in school, we had one in the geography lab and a friend of mine and I, we started, you know, playing with that, learning to code. You had to code to even do anything with a machine. You know, you understand some basic to even load a program. And then as the wall fell, I bought a Commodore sixty four and later my first PC, a three eighty six DX forty.

• 2:21Speaker 01

  Wow.

• 2:22Speaker 00

  And so I, you know, as a teenager, I spent most of my time coding and, you know, started a company. It wasn't at that time really a startup, we just did insurance software. As in the late nineties, there wasn't, most of insurance agents didn't have software yet. Some were working on mainframes and others just had, you know, paper in front of them. And then I moved to South Germany to work for Mercedes. And then I had, you know, the startup that got acquired by Microsoft. But it's really this passion for doing stuff with software, being creative with software. And I think the fascinating thing back, you know, in the nineties, and it's still true today, is that you can start, you know, very easily. There's not a lot of capital investment required. And if you, if you make a mistake, you can just start from scratch, you know.

• 3:05Speaker 00

  And then, and I think that's what makes this so, so, so cool to build software.

• 3:10Speaker 01

  And that gave you the love for building and fixing things even today with robotic lawnmowers in your own home.

• 3:16Speaker 00

  Yes. Well, and it gives me the laugh, you know, as the CEO of GitHub.

• 3:19Speaker 01

  Yeah.

• 3:20Speaker 00

  To build software for software developers. I think that's the really cool thing about being at GitHub. We are building the tools that other developers are using. And we, we always say, you know, we put the developers first. And that, I think, is, you know, the dream job for me. I get to speak with a lot of developers, I get to build software for developers. And, and I get to speak, you know, to, to many developers here in Silicon Valley.

• 3:42Speaker 01

  I think what many might not know about you outside of Microsoft is that you were actually, you played a pivotal role in sponsoring Microsoft's acquisition of GitHub back in twenty eighteen. Can you share, take us back to that moment and share a little bit about what was your vision for GitHub back then?

• 3:59Speaker 00

  Yeah. So in twenty eighteen, I was a product manager at Microsoft, working for Nat Friedman, who at the time was the CVP for mobile developer tools. And Nat had the idea of, of, of buying a GitHub and making GitHub part of Microsoft. And so he and a few folks in his team, we are kind of like strategizing of how can we pull that off and pitch. To Satya and the board and as the deal got announced, we actually, you know, about six years away from that, at the time of this recording, June four twenty eighteen is when we announced the deal. I became the deal integration manager, was was is the role within Microsoft that runs around the whole company, making sure all the pieces, you know, from legal to HR, finances and product and engineering come together

• 4:44Speaker 00

  to get that deal through the regulatory approval and ultimately now getting us to a successful day zero, which then happened in October twenty eighteen and that's that's how ultimately came to GitHub.

• 4:56Speaker 01

  What did you envision GitHub becoming, the potential it had as part of the GitHub universe?

• 5:03Speaker 00

  I think, you know, at the time we were thinking there's so much potential for GitHub that's still, you know, to be to be explored and to be realized. GitHub started with the first commit in late two thousand seven, launched in two thousand eight, you know, very quickly became a new way of developers working together. You know, social coding was invented by Chris and the other founders. And it evolved into this into this two-sided product, the home of open source, where many open source developers were collaborating, and the place where many startups and ultimately enterprises were building their software. And often that, you know, two-sided equation was that the companies wanted to work exactly like open source developers work,

• 5:45Speaker 00

  which is boundaryless. You know, in the world of open source, you don't really care where your collaborator sits, where they're from, what their education is, they don't sit in an org chart. And, you know, often you don't even know their real name, all you know is the handle and the code that they want to contribute back to the project. And I think many companies don't have that. You have silos and, you know, when you get an email, you're like, wait, who is this and why are they emailing me and why do you, why do you want to be involved in my project? And so companies want to burn down these these world gardens and have a similar collaboration model. And that's what GitHub symbolized, you know, in twenty eighteen. But there was so much more to do, you know, to provide something like GitHub Actions that allows developers not only to manage their source code and plan, but also to build their apps.

• 6:30Speaker 00

  And now ultimately with Copilot to take it even a step further and making, you know, development a very different experience than it was twenty years or thirty years ago when I started.

• 6:40Speaker 02

  Since you mentioned Copilot,

• 6:42Speaker 00

  Yeah.

• 6:42Speaker 02

  I'm dying to ask you about the behind the scenes view

• 6:46Speaker 00

  Yeah.

• 6:46Speaker 02

  on what is, you know, not only the most successful enterprise AI application today, but I believe the first LLM native application that was really built, built and launched, like, I guess, was it part of the original acquisition thesis at the time that you might be able to build something like Copilot eventually? Whose idea was it to do something like Copilot? And like, was it, did everyone say, yes, this is going to work? Or was it like, this is a crazy idea, this is a moonshot, it's never going to work? Like, take us back to the origin story.

• 7:14Speaker 00

  So the original acquisition thesis had like a little paragraph in it about AI, but I think that was more like a moonshot than like the proper idea at the time. What really happened in mid 2020 is that, you know, transformer models, you know, the paper came out a few years before that, but the first, you know, really working transformer models was GPT-3 was about to launch. We got early access. We were all locked down, right, June 2020, on a call. And one of our team members, Hugo Demore, started, you know, typing into things into the model. Everybody else was just looking what Hugo would be doing. And then, so then the natural question, what can we prompt it to write code?

• 7:58Speaker 00

  And can it write proper code? And I think that was the first kind of like aha moment, that it was actually able to write the real syntax. And then we tried different languages and we also found flaws, you know, in the model. And so we started, you know, exploring that deeper outset of that Zoom call in a research process, you know, doing analysis. We asked, you know, some of our staff and principal engineers to submit coding exercises. We looked at Python functions that were, you know, in open source repositories on GitHub. And we worked with OpenAI to Take GPT three and fine tune the model to be better at these coding tasks. And ultimately, in August, we had a model that was able to solve ninety two percent of these coding exercises.

• 8:41Speaker 00

  And in fact, you know, of the Python bodies, you know, that we extracted from from open source projects, it was like fifty two percent. Now, naturally, that percentage is lower because you have less specified code than in a coding exercise for an interview loop, right? But that, I think, gave us this this moment of confidence, saying we can build a product around this. The second moment, I'd say, was when we rolled it out to our internal engineers in early twenty twenty one. And they came back with saying, this is this is fantastic. I think the net promoter score NPS was somewhere in the seventies. And which is, you know, I think for developer tools, really, really good. Most developers are skeptical, you know, don't touch my system, you know, never, never touch the running system or process.

• 9:24Speaker 00

  And many folks are, you know, kind of like have have created their work setup, not not only on their physical desks, but also on their virtual desk of how they want to work. And so we were really, you know, intrigued by the internal responses. And then we launched the preview and, you know, the team came back and saying it's writing twenty five percent of Python code in those files where that was enabled. And I think we sent them back, say, go and verify the telemetry, that that can't be true, right? Like, because we couldn't believe it initially. And then, as we we saw this progressing, I think, in my first keynote SEO was in June twenty twenty two. So now, you know, two years ago, and I said, it's writing forty percent of the code.

• 10:06Speaker 00

  And I think in five years, it's going to write eighty percent of the code in those files where it's enabled.

• 10:11Speaker 02

  So it happened really organically. It wasn't like you were sitting in a room whiteboarding, like Copilot is going to be the next iteration. It was like the model proved to you how great it was. And just seeing the model performance and seeing usage, like really like it organically built up from.

• 10:26Speaker 00

  Going back, you know, to the original thesis for GitHub, right? We wanted to make developers' life easier. And as we are company building software, we have our own software developers. And then Microsoft has, you know, seventy thousand or so of them. And so we understand how software developers work because we have so many of them. And you know, we live the life of many of our customers, which is we have way too many ideas. We're moving way too slow, you know, at least in our feeling, you know, Amazon delivers my package faster than than we're implementing some features. The also the expectations have shifted significantly, but it's it's this, the backlog is endless. And the amount of ideas that we can brainstorm on virtual whiteboards, GitHub is a remote company. So we don't often meet in real physical, in front of free physical whiteboards.

• 11:11Speaker 00

  But and then the other side is all the other work that we also have to do, you know, compliance, security, accessibility, you know, enterprise requirements, privacy regulations, European, you know, AI Act and Digital Markets Act and all these things also cause developer work. And so we're constantly, everybody's constantly struggling with the lengths of these backlogs. And so if you will, the intrinsic motivation was let's bring the effort down to write software and and make it joyful again.

• 11:41Speaker 01

  What do you think made Copilot so good at the time and also going forward today? Obviously, GPT-3 was great even, you know, back then. But also, I think a lot of people might not know all the value that you brought or GitHub brought in the both public and proprietary data around code that GitHub owned. Can you share a little bit more about that and also how you think about that going forward?

• 12:02Speaker 00

  I think the key ingredient, you know, of the original Copilot, which was on the auto completion, right? Like you would type in your editor and it would complete, you know, the next line. But it could also complete multiple lines of code, you know, complex algorithms or simple algorithms. Some of the demos we often chose to implement a sorting algorithm like bubble sort or prime number detection. And it can just write, you know, those 10 lines of code by just a simple prompt, which is a comment in the code or just writing the method declaration. And so getting, you know, into the editor where developers already write code, not changing the way they work, but giving them, you know, ideas while they're typing. I think that was the key moment, other than obviously the model being good enough and in OpenAI tuning that model.

• 12:45Speaker 00

  On publicly available source code from GitHub. You know, GitHub didn't give special access to OpenAI. OpenAI was just able to access our source code in the same way that, you know, many other startups are now doing that, either through direct access, you know, through API or through archive programs like the Internet Archive and the Software Heritage. And we actually have an established partnership with them to stream our open source code over there. So it can be archived, you know, for, until the end of time. And so, of course, the model got tuned to be good enough. But then the UI and the user experience, I think, was crucial. It wasn't, you know, AI is now on everybody's mind. But the truth is, our cellphones have, you know, some kind of AI built in for a long time.

• 13:30Speaker 00

  Your keyboard is predicting the next word with some kind of machine learning algorithms. Your photo library, you can just go in there and search for license plate and finds all the photos of the cars that you took to remember what the license plate you have. And that's AI, you know, image recognition, but that nobody perceives that as AI. It's just a cool user feature. So I think that also was the core ingredient of Copilot. We're meeting developers where they are and we're making their life better. And then, I think, you know, the name itself was also a stroke of genius. One of our other, you know, developers, Alex, came up with the idea to name this Copilot as Nat. It's a hobby pilot. And so that's where the name is coming from. So it's like, you know, I'm thinking about what could I name this?

• 14:14Speaker 00

  So, you know, my boss is, the name is resonating with him.

• 14:19Speaker 01

  Wow, that's so interesting. I didn't know that. If you could go back to 2020, 2021, is there anything that you might do differently?

• 14:27Speaker 00

  You know, in hindsight, you can always move faster and then be, I think, more convicted of those ideas. I think in the beginning, we kept the team intentionally small, small teams can move fast.

• 14:37Speaker 01

  How big was the team?

• 14:39Speaker 00

  I think the original staffers were like three people, but that's obviously a staff researcher, sorry, or principal researchers, but. I think that's cheating a little bit in the sense that of course there was a team at OpenAI, there was a team and multiple teams in Microsoft, both on the research side and on the inference side, the model inference side, in fact, on the model training side to even enable the model. So of course, in the bigger partnership between Microsoft, OpenAI and GitHub, it was a larger team, but the original paper was written by three researchers and then a hundred people or so are mentioned in the credits. And then we moved fast with, I think, a step of five and then I think it increased to 10 teams. And of course, today, the team is much smaller,

• 15:22Speaker 00

  but we still have what we call GitHub Next, an incubation team that now works on Copilot Workspace and iterates on new ideas. It's almost like a startup incubator within the company.

• 15:33Speaker 02

  That's really interesting, yeah.

• 15:33Speaker 00

  And they're picking up ideas and the main difference to our mainline engineering teams and product management teams are that they gotta have the mindset that most of their ideas will never go into production, right? It's kind of like this OK, ours won't work because your key result is to throw away most of the ideas and start fresh with another idea. And I think that's where a lot of the innovation speed is coming from.

• 15:57Speaker 02

  I want to go back to what you said a minute ago about how you handed over control of the model effectively to OpenAI. Was it scary? Because the brain of your AI application is actually being built by another company, not developers under your payroll, your control. How do you think about, was it an easy decision to partner with a different external model provider? And I'm curious how you think about the value that Microsoft and GitHub provide to end users versus the value that OpenAI provides to end users? And then where you seek to really bring value to users?

• 16:34Speaker 00

  To me, it wasn't scary at all. I'm sure there were folks in the team and in the company that thought we should have our own models instead. But in reality, if you look at GitHub as a company that was born in the cloud, of course, we have always relied on partners to build our stack. To my memory, we have never built hardware ourselves. And while we have In data centers or had metal in data centers for a while. Uh, the data center itself wasn't built by us either. Neither was the CPU and and the memory and all and the network infrastructure. And then even if you go higher in the in that stack, you know, GitHub is is built on top of open source. And so are the majority of software applications.

• 17:15Speaker 00

  In fact, um, we love sharing a statistic that says ninety percent of the stack of most of most applications, even if the application itself is closed source, is in fact based on on the work of of the open source community. Um, you know, from the operating system, the Linux operating system is ubiquitous today on on servers to a container technology like Docker and Kubernetes to thousands, sometimes tenth of thousands of open source libraries. And so the model just flows naturally in that stack. And, you know, we at Microsoft think about this as the copilot stack with the different layers, hardware, the model, the the kernel, if you will, like the infrastructure, response to the AI filtering and whatnot. Then you get into the application layer with the, you know, the AI copilot and then the extensibility on on top of that.

• 18:00Speaker 00

  And so if you look at these layers in the stack, you know, Microsoft has strengths and and and partners. Um, and in fact, you know, in in in some form, Microsoft is is is involved in all parts of that stack. Um, you know, the Copilot PC, you know, has has a custom chip that Microsoft is developing with partners. Um, you know, it has, we have our own models with five three and we have partnership models with OpenAI, but we're also hosting Mistral and Lama on Azure. Uh, we of course have, you know, a large cloud and we have a lot of expertise in responsible AI. And then we have lots of applications we're building ourselves and enabling others, you know, to to to build those applications. And so two part questions are always hard for me to forget the second part. But I think, you know, that actually describes the relationship very well.

• 18:43Speaker 00

  Um, you know, it allowed us to move really fast because we could rely on partners like OpenAI, not only building G P three and then codecs, but also then, you know, innovating with G P three point five and Chat G P T, G P T four. Now we have G P two four O, uh, we have Microsoft, you know, with with a large infrastructure that builds, you know, supercomputers for training, but also infrastructure to run inference. You know, Copilot today runs in multiple data centers spread around the world in different regions. So the developers that sit in France now connect to a GPU to an Azure instance that's much closer to them to enable low latency, right? And so Microsoft gives us a lot of infrastructure, a lot of expertise and responsibility, and of course, a lot of commercial distribution.

• 19:28Speaker 02

  Yeah, very complicated layer cake that makes the magic come together.

• 19:32Speaker 01

  Would GitHub ever want to build its own models or just use best-in-class models out there?

• 19:36Speaker 00

  You know, obviously, there's always a desire of engineers to build their own stuff. And I would, you know, I would not deny that we have played with our own ideas on models. And I think, you know, the almost he learning team goes back almost. I think it actually had it before Microsoft acquired GitHub. And then we obviously have fine-tuned models ourselves and we have fine-tuned models with OpenAI and Azure. And we're working with customers on them being able to customize models based on the code that they have in their repositories. And, you know, never say never what the what the future may bring. But today, we're really happy with the models that we have. And we're constantly looking into the market of what not only OpenAI provides to us, but also what what others have.

• 20:21Speaker 02

  So Copilot is already one of the most successful generative AI applications in terms of user scale, usage, et cetera. What are some of the latest metrics you can share? And and what are the metrics that you're most proud of?

• 20:34Speaker 00

  I think the one I'm most proud of is the developer happiness scores. And if you look at survey data, most of the service that we have done, but also the service that now our customers either publishing themselves or bringing back to us. It's clear that software developers, after they have tried Copilot, after they got over the initial adoption hurdle or skepticism, they, most of them love using Copilot and most of them report that they're more fulfilled, you know. They're more satisfied, more happy. They feel like they're requiring less mental energy to to get the job done. They need to do less boilerplate. And I think that alone is is really making me happy. And there's lots of, you know, numbers that I can throw out. But I think that the general gist is no matter what developer I talk to, those that have used Copilot

• 21:18Speaker 00

  for a while no longer want to work without a Copilot. And then the other side is, you know, the productivity metrics of making, you know, developers more productive, which I think matters to the developers too, but it also of course matters to their management chain and and and their leadership and in the sense of, you know, getting more stuff done, delivering more value to to their end customers. And so, you know, today, we are happy to say that we have more than 1.8 million paid subscribers on on Copilot in more than 50,000 organizations. And and that really makes us happy looking looking at that growth. And, you know, we here at the Securias office, it feels like we are we are like a high growth startup and and that's a good place to be in.

• 22:03Speaker 01

  It's awesome. Well, we're still on our walls somewhere. Actually, we know exactly where we'll show you.

• 22:08Speaker 00

  Me too. The creative minds.

• 22:12Speaker 01

  Was there anything that surprised you in terms of Copilot's impact after its launch and even today?

• 22:17Speaker 00

  I think, you know, it's the 25% certainly surprised us. The quick turnaround from the skepticism after we announced this in June 2021. I don't even, I think we had a very short blog post and then just a webpage with examples and and and animations showing how it will work. And I think folks were looking at this and saying, this is like a cool tech demo, but it doesn't actually work for me. And I think the skepticism wasn't was that people had seen how G P three at the time would work and they couldn't understand until trying it that it has the context of whatever you wrote in the file before before Copilot suggestion comes and it considers adjacent tabs and things like that.

• 23:00Speaker 00

  And so it magically picks up your style and it knows about open source libraries that you're using, not because you have opened those libraries, just because you have an import statement at the top and because the model was trained on such a large corpus of data that.

• 23:16Speaker 00

  It can provide the calls into these open source libraries. And so it feels to you that the Copilot understands more about your project than you thought G P T three can do. And I think that's where the original major came from. The other thing is today, when I observe people using Copilot and obviously Chat G P T changed the whole game. And brought in chat as a component, we have now Copilot chat is this natural language component. And like using not only code and comments as a trigger, but English and German and Brazilian Portuguese. And we have demos in India where Karen M V, one of our folks there is demoing this in Hindi.

• 23:58Speaker 00

  And so and you can actually speak into the Copilot with voice detection in British Studio Code. And then gives you the response back also in Hindi. And it's really cool for anyone that wants to explore coding, even if they're not fluent in English or in programming language. My kids are using it to find their own bugs in Python.

• 24:17Speaker 02

  Wow, that's impressive.

• 24:18Speaker 00

  You know, they're not coming to me and it's like, Daddy, you have to find my bug. It's kind of like, well, go and find your own bug. And that also obviously helps them to develop their skills. And it's a bit like, you know, I often talk about, you know, natural language will democratize access to software developers. But it doesn't mean that everybody is immediately a senior principal software developer. In the same way that just because I buy a guitar, I'm not as good as Keith Richards playing with the Rolling Stones. Right. And if you look at any professional band that's touring the world, they're all still rehearse over and over again.

• 24:59Speaker 00

  And I think this is like this idea that to be good in a craft, you have to keep doing it. That Copilot does not take that away. It just gives you another tool in your toolbox.

• 25:10Speaker 02

  I really like that analogy. Still believe in teaching your kids to code because that's like a debate on the internet now.

• 25:16Speaker 00

  Oh, absolutely. I mean, first of all, the human language is not deterministic, right? Like you mean different things, you can mean different things by saying the same sentence. And even, you know, the

• 25:28Speaker 02

  That's very German.

• 25:30Speaker 00

  Even the yet, well, now we can get into off-topic debates about about yeses and noes when you're when you're when you're answering a question with a negative in it, right? Like, you have not been to the grocery store, do we answer that with yes or with no, right? And Americans expect a no, even though you mean actually yes to that question. But like, look, you know, it's, human language is not deterministic. And so code is, with code, you can very precise describe what the machine does. Code is an abstraction layer on top of, you know, assembly language, on top of the instruction set of the CPU or GPU that, you know, the processor manufacturer like Intel or Nvidia has created, right? So it's just another abstraction layer. Human language is something completely different.

• 26:14Speaker 00

  It's creative. And that's the power, but it also means that there will be code involved one way or another. And we're moving up the abstraction layer, but we're also spreading the meaning. And that's truly powerful, but it's also means that there will be some conversion to code somewhere because the chip itself, at least, you know, today, is requiring a deterministic instruction set.

• 26:38Speaker 02

  Yeah, I love that. I'd love to talk about the future of Copilot. You've been announcing new products in rapid-fire succession, I think Copilot X, Copilot Enterprise, something around code security, I believe, and then workspaces most recently. Can you tell us maybe about what each of those things does and how you see them all fitting together in the grander vision for what you hope Copilot becomes?

• 27:03Speaker 00

  You know, what you described with all these product names is one of part of our mindset is that momentum is our energy in this in this age of AI, moving fast and iterating fast is crucial. And so, you know, we are having developed Copilot from this original idea of auto-completion by adding chat. We were developing auto-completion and then chat, and then with chat, we also announced something what we call Copilot X, which the idea was we're bringing AI features into every part of the developer lifecycle. We're bringing Copilot wherever developers are. So, you know, while we added chat to the editor, we also added, you know, a little Copilot I can into the input field where you write your commit message. And that might be trivial, right? Like, everybody can write a commit message.

• 27:47Speaker 00

  But it also means I reduce my mental workload and I reduce kind of like the bias that I have for the work that created myself. Right? Like, for me, everything is obvious that I just did for the last hour. But for you, when you want to review my my commit or my pull request, it's not as obvious. And so having an AI describe that in a neutral form, kind of like an outsider describing what I just did is incredibly useful. And it just keeps me in my flow. And we added it to the debugger, we added it, you know, to many different parts of the lifecycle already. And with Copilot Enterprise, we bundled it into, you know, a higher price product that allows enterprises to customize Copilot based on their institutional knowledge.

• 28:29Speaker 00

  And enterprise here means really any company that has gone for more than a few weeks. Because they all immediately build institutional knowledge, right? How we work as a team, how our coding practices, these are the libraries and languages that we use. And so unless you're a student in university and that, you know, gets to have the free range of technologies available, at least when they, when the professor allows that, every time you join a company or join a different project, you have to ramp up again on how they are doing things. And so Copilot Enterprise lets companies customize the Copilots to their institutional knowledge. And it makes it really easy for me to join that company because I can now ask dumb questions without you judging me.

• 29:11Speaker 00

  Like, you know, imagine I would, you know, join here and my first day would like, why is Thomas asking all these questions? Shouldn't he already know all of that? He's, you know, been in professional life for a long time. That's the challenge that we have when we join companies and we have the anxiety in our head that we can't ask too many questions before Steph says, what the hell?

• 29:30Speaker 00

  Right? And so that's, I think, the power of Copilot Enterprise. That's the power of bringing Copilot into every part of the developer lifecycle and ultimately into every part of the of our lives.

• 29:41Speaker 01

  You've also mentioned that agents are one of the most important next things for GitHub. Maybe just to set the stage, how would you describe an agent versus a copilot? And can you give us a teaser for what types of agentic capabilities we should expect come into copilot soon?

• 29:56Speaker 00

  Yeah, I would say copilot and agents is kind of like the same thing. And an agent is using a model to get a task done, right? Effectively, it's looping with a model to solve something for you. And a copilot is an agent of agents and it has multiple features available to it. You know, if you think about auto completion, well, it's an agent that takes every keystroke you did and the context that you have in your editor, it sends it to model inference, it gets the response back. It might pick, you know, the best prediction and then it shows it to you. And so we're going to see more of these agents that take over more of our tasks. And one of them that I'm most excited about is, you know, AutoFix. And the way it works is so you submit a pull request and traditionally, you know,

• 30:39Speaker 00

  some security scanning feature that you have integrated into your pipeline finds security vulnerabilities. Let's say, you know, SQL injection or cross-site scripting. Well, that's great, except now I cost more work for myself. It's kind of like you have a Roomba, but instead of vacuum your house, it just shows you where the dirt is. And then you have to go and vacuum yourself on that position. So now with AutoFix, we're actually not only showing you a security vulnerability, we're also giving you the fix. And that uses the AI model together with the vulnerability and the description and the code to basically solve that vulnerability for you. And the initial results are really impressive. With some customers, we see that we can burn through like 75, 80% of their open alerts. Everybody has those alerts.

• 31:24Speaker 00

  And if you don't have any alerts right now, I bet you you have them by Monday. That's the challenge in this world of software security is everything is around is moving so fast. There's always a new version of an open source library. There's a new version of Linux or, you know, a new Windows patch. There's a new device coming along the way or new Nvidia GPU. And so we constantly are behind just keeping our applications up to date, you know, to the standard that is expected by our customers. And at the same time, we have to build all that innovation and all that.

• 31:58Speaker 01

  Super interesting.

• 31:59Speaker 00

  Cool stuff, yeah.

• 32:00Speaker 02

  What else do you think is missing in the product roadmap? Like, if you could wave a magic wand, like, what else is there to build that you're really excited about?

• 32:08Speaker 00

  I mean, I think there's still a lot of work to do on these agents. I think there's a lot of agents you can think about. You know, we talked a little bit, or you mentioned Workspace before. What Copilot Workspace does, it provides different agents to get you from idea to your pull request, to get you from idea to the code. And the first one is the spec agent. And what that actually does, it helps you with your thought process. So, you know, you write down an idea, implement some feature, and it looks at your existing codebase and it basically helps you then to reframe that idea. Now, that's not only useful for developers, it's actually useful for a product manager, right? Because it might tell you, well, Thomas, this idea is no way you can describe that with a single sentence

• 32:51Speaker 00

  and a developer cannot implement that in just a single ticket. It needs to be in epic or multiple different user stories, right? And then the next step is the plan agent that, you know, helps to figure out where to make the changes in the codebase. And again, you can kind of see here, there's a lot of other benefits you get from that because it helps you to understand the codebase. Because, you know, most codebases that are older than, you know, a few days have hundreds, if not thousands, of files. And as developers, you have to navigate all those files. And even if you have been in a codebase for a long time, you still might miss out, you know, that one file that you haven't touched for a while and you have to add, you know, config statement there. So it helps you understanding the codebase. And then the implement agent helps you implementing the code change.

• 33:36Speaker 00

  And every step in that way, you're still in charge. You can, you know, with natural language, modify the bullet points of each of these agents. And then obviously you can modify the code at the end. And so if you look at just these three agents, you can easily start thinking about other agents that you might have along the way, right? For example, the one that estimates the size of a ticket, the story pointing agent, as one example. Or another one is that, you know, once you have implemented the file, well, now you want to build, run, and debug the file. And maybe you have an agent in the future that will just automatically fix. You know, any bugs that that were introduced by by the previous agents. And so I think we're going to have more of these building blocks, you know, Lego blocks, if you will, available to us.

• 34:19Speaker 00

  In fact, you know, uh, uh, you know, if you look at Lego, they have way more pieces, types of pieces today than they used to have. Because the models are much more complex than, you know, you can uh, uh, buy, you know, uh, um, NASA rockets and whatnot, and they need different pieces for that. And I think that's kind of like the same way we should think about co-pilots. They will have more of these building blocks that enable us. In addition, you know, to more powerful models and a mix of models, these building blocks will enable us to do more.

• 34:47Speaker 01

  Increasing modularity, interesting. Um, where does your ambition for GitHub take you? Or maybe even with GitHub Copilot specifically take you? As you think about what you alluded to from the perspective of deepening, where you can go with just the software engineer to also expanding into potentially different personas, PMs, you mentioned, maybe an SRE, maybe a security engineer. Where does that, you know, the breadth also take you?

• 35:11Speaker 00

  Yeah. I mean, first of all, I think all those roles today are already collaborating on GitHub. In fact, GitHub had always that mantra that we're building GitHub on GitHub. And it's sometimes, you know, we have pushed it a bit too far. Uh, but today, you know, most of GitHub employees, we call them hubbers. Hubbers are, you know, engaging on GitHub in GitHub discussions and GitHub pull requests. You know, our legal documentation is all on GitHub, which if you think about is actually much better than than managing red lines in Word.

• 35:41Speaker 01

  Yeah.

• 35:41Speaker 00

  Because you have a version history and you can see who made what changed it. In fact, you can kind of see a sooner future where maybe your legal document is explained by Copilot in in human language, uh, you know, in the actual understandable human language, not the lawyer language, right? And so, um, so we're using GitHub, you know, to our, on our company. But yeah, we, we, it's where all the developers and all the supporting functions collaborate on a project. So that's, I think, number one. Number two is we want to democratize access to to software development. And, um, you know, I recently gave a TED Talk and I talked about that. I think our goal is to get to one billion software developers on the world. Now, it doesn't mean one billion professional software developers, although.

• 36:23Speaker 00

  That might not be a bad thing necessarily, given the demand is still very high. It's sometimes hard to find qualified software developers. But it's really about democratizing access to writing software on these devices that are with us. You know, our mobile phones are, you know, a really important part of our lives today. You can't really imagine life, urban life for sure, without a mobile phone. And so then also being able to write little applications or little scripts or just, you know, using natural language to control the phone, I think, is incredibly empowering. And so bringing that into, you know, 10% of the world's population by 2030, or so assuming that then there are 10 billion inhabitants on this planet,

• 37:07Speaker 00

  I think is going to create a better world and it's going to unlock creativity everywhere. And hopefully, you know, we see cool venture-backed startups in India and in Brazil. And maybe, you know, the next big tech company is coming from one of those countries instead of the US West Coast.

• 37:25Speaker 01

  We hope so. Maybe zooming out of GitHub Copilot to broader GitHub itself. GitHub Copilot itself is driving so much innovation within GitHub. But what are some of the other key initiatives that you're leading across GitHub overall as well?

• 37:39Speaker 00

  Yeah, we already talked a little bit about, you know, autofix and security. And I think security, securing the software supply chain is dear and near to our hearts. There is no future of, you know, human progress with software if we're not also able to secure the supply chain. You know, today, you know, there's this XKCD comic of the internet's infrastructure. And there's this one building block that says, you know, the guy in Nebraska maintaining this one library alone, right? That's, that actually is as funny as that is, that is a reflection of the software world today. So we A have to make it sustainable for those maintainers to build that software and keep it joyful. And then also we have to make sure that all these building blocks that are in our stacks, that have become system critical and to be secure.

• 38:24Speaker 00

  And so we're investing a lot in platform security and application security. And of course, you know, in our security products. And I think that's going to be crucial combined with Copilot and AI to not only create all these work items, but to also enable developers to burn them down to fix all these issues.

• 38:42Speaker 01

  Yeah.

• 38:43Speaker 02

  I want to zoom out from GitHub for a second and just talk about how you see the future of AI and coding overall. Like you mentioned, there's a billion casual developers around the world. Like, what will it look like? Will everybody be kind of coding applications for themselves to use? Will there be some number of professional developers who are super developers? Like, how do you imagine the world looks when you've so kind of democratized the craft of coding?

• 39:09Speaker 00

  I mean, I think it's an incredibly creative world, a world where you're not dependent on, you know, when you're a kid, on your parents having technical knowledge, or your school having a teacher that knows how to do these things. So we're going to have much more access to those that are interested in learning about this. You know, it's easy today, you know, to take a sheet of paper and paint something. And every restaurant, at least, you know, in this country, gives you crayons if you come with kids and a coloring sheet.

• 39:39Speaker 02

  Lifesaver.

• 39:40Speaker 00

  Lifesaver. Or you have your mobile phone and they use your mobile phone. You know, it's easy to learn, easy in the sense of accessible to learn a musical instrument. And I think it should be easy to learn coding. And so first of all, I think that should be something we are excited about and not concerned that we're inflating the number of developers. Because just because kids learn it doesn't mean they want to become a developer. I think there's still a world where people want to do something else than software. But then if you think, you know, about many other professions, physicists, for example, they use a lot of software. You know, the first image of a black hole was rendered with the help of open source project. You know, the Mars helicopter ran on open source, right?

• 40:21Speaker 00

  And so, yeah, it's space, and it's space engineering, but they're using software, they're building software. And so the profession itself is everywhere. Every company is a software company. Banks are software companies, you know, energy providers are software companies. Farmers are software companies predicting, you know, what seed to plant this year, what the weather is going to be like, you know, what was the soil quality from last year and things like that. But of course, you know, that the hobby scenario isn't, is also important. Like, you know, a tax season is over here in the United States, but that doesn't mean that I couldn't think about next year to automate a lot of that. If I only had an AI agent that does all that work for me, you know, and and and downloads all the PDFs and extracts all the numbers. And I don't think we are too far away from that.

• 41:06Speaker 00

  And, you know, I'm, I'm, you know, now in software for for almost thirty years or two, yeah, almost thirty years and as a professional software developer. And, you know, I don't have a lot of time to code. You know, if a company to run, I have podcasts to give and and things. But the problem today is you you find an hour on a weekend and you have a project and the first twenty minutes you're spending with updating everything to whatever you missed. And the burden is actually, the the fun is gone to a certain degree because the burden of of maintaining software is so high. So having something available to you that gets you quickly into the hobby and out of the hobby or out of that task, I think, is incredibly empowering and and brings the brings the fun back.

• 41:51Speaker 00

  And and, you know, that's where the Lego comparison is is so useful, right? Because Lego is just incredibly accessible. And even even if you, like, the best Lego is the one where you don't have instructions. They just have a table full with Lego bricks, even in in random colors, right? And then have this this excitement of play. And, you know, even, you know, professional workers at their offsites or workshops often have little gadgets or bricks on the table. So you have your finger, your fingers do things while you're while you're thinking. So I think that's where that world is leading us. And we will have, you know, more more access to technology. We have more people that can build software. That doesn't mean that they're taking jobs away from professional software developers.

• 42:33Speaker 02

  On what timeframe do you think we'll have coding agents that are as good as maybe the average professional software developer? And and, you know, there's the legend of the ten X software engineer. Like, at what point do you think AI will be as good as the ten X software engineer in capability?

• 42:50Speaker 00

  You know, the the the trick in that question is the as good as. And what does that mean? So I think, you know, is a model today able to write? Better code than the average developer if it is prompted in the right way or given the right context, I'd say we are already there on average. Because you know, often, you know, the model just knows more about that whole space than I as a human do. Like, and you see that with students if they have to implement, you know, like a conversion from binary to decimals and that, and they might write a hundred lines of code and then they go and ask Copilot how to do that. And they get probably an open source library in one line of code. And then you could kind of say, well, I'm not allowed to use open source.

• 43:34Speaker 00

  And then you would probably still get a better code than than they wrote. And I think the truth, the same is true for the professional software developer. Because, look, we are not perfect. We are, we are human and I think that's part of our nature, that's part of creativity. Now, that's the key thing, though, that the model is not creative. And the model cannot, today, you know, the model cannot make decisions for us. Or if it doesn't make decision, it doesn't actually take all the constraints into account. Like, if you think about software development, other than writing code, which is, I think, the fun part, it often means, you know, I take a very complex problem and you break it, decompose the problem into small building blocks, you know, and the block size is increasing over time.

• 44:17Speaker 00

  You know, it used to be, auto-completion used to be just the next word and then it was maybe, you know, full command and now it's multiple lines of code and maybe it's whole files in the future.

• 44:27Speaker 00

  But along that decomposition process, you still have to make a lot of technical decisions. What database am I using? And, you know, you probably know better how many database startups Sequoia has invested in and how many infrastructure startups and in how many, you know, serverless startups. And, and, and, and, and obviously, there's all the incumbents in all those spaces. And so there's a thousand, if not 10,000 decisions to be made. And the engineer is, is the systems thinker that, that is making those decisions. Or the team, the team of engineers and companies. I think, you know, that we have been building houses, you know, as humans for thousands of years. And if you have ever built a house, it's still not a solved problem. Yeah?

• 45:07Speaker 02

  But what, at what point do you think that creativity and that systems thinking gets built into Copilot? Or do you think it never does?

• 45:15Speaker 00

  I mean, predicting the future, I don't know if I would say never, never say that's a dangerous thing on a podcast, you know, invite me back in three years and it's like, well, Thomas, you know, last time we asked you about this and clearly it has happened since. No, I think, you know, we'll see where the research goes and where the technology goes in, in kind of critical thinking and systems thinking and those kind of questions. Also, learning, you know, learning, you mentioned your two-year-old, your two-year-old and my, my kids, they learn, you know, as they mimic the humans around them. They are so good at learning language and especially in young age, you know, they can learn multiple languages, mine, you know, speak English and German, because we speak German at home and they don't have an accent in English. Well, they have an American accent, but they

• 45:59Speaker 00

  don't have the German accent, right? And they don't have an accent in, in, in, in German either. And I think, you know, this, this shows like that the human brain is still so much more advanced than, than the transformer models and the diffusion models and, and the other types of models that we have to image recognition and whatnot that we have today. And, you know, it remains to be seen if we can kind of like add that sentience piece to it. But today, I'm, I'm not seeing it and I haven't seen any research that telling me that that's coming anytime soon.

• 46:29Speaker 01

  That's a really clear direction. Switching gears a little bit, I would love to hear your thoughts on the overall ecosystem of startups right now. AI co-gen is the hottest category with a lot of ambitious founders and there are so many different attempts that they're taking, whether it's folks who are trying to build a better model, folks who are trying to build a better IDE, folks who are trying to build kind of an all-in, full-stack engineer as an agent. And, you know, but the big elephant in the room is GitHub Copilot and all the adjacent products that you have around it with Copilot Workspace, with Copilot Enterprise, with AutoFix and owning VS Code as well amongst many other things. How do you think about what white

• 47:11Speaker 01

  space there is for existing, oh, sorry, for new founders? And if you were a founder yourself, trying to build in this space, what would you do?

• 47:21Speaker 00

  I mean, I love developer tools, so I probably still do developer tools. And I'm not sure I would worry too much about what's white space with what's taken by incumbents. Because that can change quickly, you know. When GitHub started, SourceForge was the big elephant in the room. And SourceForge had all the open source projects and then came Git and Git re-sabbed that space. And then the founders of GitHub took Git and built GitHub. And all of a sudden, you know, that elephant in the room was no longer the elephant. And I think, you know, it is often fun, you know, to compete in the same space. And we love competition because it pushes us forward. You know, it's boring, you know, to do a race or, you know, a game if you don't have an opponent.

• 48:02Speaker 00

  And if you don't have, you know, other teams in the league who would watch, you know, the Super Bowl if there's only one winning team every year. So I think, hey, I don't think competition should hold you back as a founder from going into that space. And I think the software development space is wide open and there's lots of problems to solve. There's lots of problems in different, you know, industries and categories to solve. We haven't really solved modernization of source code, you know, COBOL runs still on mainframes. And you can use a copilot and we are heavily, you know, looking into that. And because it's such a pain point for many financial services institutions.

• 48:39Speaker 01

  Oh, that's so interesting.

• 48:40Speaker 00

  Your credit cards, your bank account, Wall Street, all that runs still COBOL on main. I haven't met a single bank that doesn't run some COBOL on some mainframe.

• 48:50Speaker 01

  What would you do with copilot and COBOL?

• 48:52Speaker 00

  So, well, today you can explain that, which is, you know, often helpful because the code was written 60 years ago. And so the people that wrote that code are retired.

• 49:00Speaker 01

  Yeah, yeah, exactly. Would you rewrite all the code with copilot?

• 49:04Speaker 00

  You can ask it to write unit tests because nobody wrote unit tests in the 60s and 70s either. Let alone that there were unit testing frameworks for those languages. Like, keep in mind, you know, the 60s, that was before hard drives, right? Like, you know, before we had, you know, personal computers. So it was a very different world back then. And of course, They those companies have done work to modernize to a certain degree, but it's far behind Azure software development. So you can support that transformation process today, but there is we're not we're not at the point where you can just click a button and you have a transform. The same is true, you know, for many more modern languages. You know, there's large PHP code bases. There's lots of Java out there. And there's lots of optimization that we can take, you know,

• 49:47Speaker 00

  to just make our existing stacks more efficient where agents can help. There's lots of, you know, things to improve in every part of the software development lifecycle and in this ecosystem around us. GitHub, you know, is I like to think about GitHub as, you know, one planet in this universe of software development tools. And there's, you know, smaller planets around us and then, you know, equally or almost equal sized planets in our space. And we consider them partners and we are we are happy that they're there.

• 50:18Speaker 01

  Yeah, so interesting. Agents are just pull on that thread a little bit. It's also an area of excitement for us. And I think, you know, you have a lot of the benefit of owning so much of the data and everything you can do on the post-training side. OpenAI itself is also getting better and better with each new model class, as are every other model company out there. Would you ever want to kind of invest into building your own agents, maybe from scratch by building your own agentic models or just to partner with some of the others out there?

• 50:49Speaker 00

  Me as GitHub CEO?

• 50:51Speaker 01

  Yes.

• 50:53Speaker 00

  Or me as agent investor?

• 50:54Speaker 01

  You as a GitHub CEO?

• 50:57Speaker 00

  Yeah, I think, you know, we are probably doing both. We're going to we're always, you know, in a way, we always have done both things as GitHub. We have invested into our own things, the thing that we consider as, you know, core part of our platform and of our offering, you know, part of our primitives. And we have partners partnered with companies. You know, just earlier this week, we announced the partnership with JFrog, which, you know, covers Spidery artifacts, scanning of containers and those kind of things. And they're obviously very naturally in our space. You know, we have the source code and what do you do with source code value? Either compile it into binaries or you combine it with binaries before you deploy it into the cloud. And so there's a natural value chain there where, you know, we have,

• 51:41Speaker 00

  you know, things where we invested ourselves, things like releases that you see on GitHub and packages. We own NPM, you know, the largest package registry in the world for the JavaScript ecosystem and NuGet through our partners at Microsoft, the largest .NET registry in the world. But that doesn't mean that we cannot also partner with the JFrog to ultimately enable that, that, you know, secure software supply chain that I mentioned earlier and that I think is crucial and that Figma, you know, Versal, like so many other companies in our space that will play, you know, part in the lifecycle. And I don't see a world where somebody covers all of that and can convince every developer that

• 52:23Speaker 00

  using all these tools is better than picking, you know, what analysts might call best of breed. And like the tool that I consider the best, best is all subjective anyway, right? Like it's all this, does it meet the expectations of the developer in the environment?

• 52:39Speaker 02

  Yeah, makes sense. That's great advice for startup founders and I think really encouraging. What about for incumbents? I think you are such a beacon of hope for incumbent companies. I think Satya said in the last earnings call that GitHub is now growing 40% year over year thanks to the Copilot acceleration. And so like, and to your point earlier, like you feel like you're a young startup again, like walking into the Sequoia offices. Like, what advice do you have for, there's so many incumbents that are trying to reinvent themselves around AI. What advice would you give those folks?

• 53:10Speaker 00

  Satya actually said 45%. So we're really excited about that. All up on our, on our revenue, we're going 45%. You know, I got an email from somebody last week who was like, looking at GitHub, I'm losing the belief that large companies cannot move as fast as a startup can. And I think, you know, the key ingredient on this is radical focus and basically focusing on a few small things. Just because, you know, you have a thousand engineers or 50,000 engineers doesn't, doesn't mean you can do it all. That's just a false assumption. And I think it's a way misleading as you manage larger teams. You're losing kind of like the the unit of of of of side team size.

• 53:52Speaker 00

  That shows you how much you can actually get done and how much friction you have in the system. If all these teams work on different things, I think focus, you know, lots of nose on on all the ideas that people have around you and customers. That's, I think, the the key piece to to move really fast. And and then obviously taking in some strategic bets and strategy means you're thinking about it as how do I differentiate from others? You know, what makes me specific, makes me special in this market? What lets me charge the prices I'm going to charge and not fight, you know, a race to the bottom? And I think that's that's kind of like thinking about, OK, you know, in in software, we like to think about, well, if I add just three more features,

• 54:33Speaker 00

  then I'm I'm going to win the space until you realize, well, everybody else can also add those three features. Right there, there's almost nothing in in GitHub itself as as a platform that you couldn't rebuild with significant investment and time. But I think it's really hard to mimic our culture. It's really hard to mimic, you know, our experience, our obsession about about developers and and ultimately our our focus and and the way we are approaching these things.

• 55:00Speaker 02

  Yeah, I guess very tactically, do you recommend staffing a tiger team to get to product market fit on AI? Do you recommend like, you know, pulling half your engineers off whatever they're working on and like, hey, you guys are the AI team now? We we got to go bigger, go home like tactically, how do you recommend companies?

• 55:14Speaker 00

  Well, as an incumbent, you always have the challenge that you have to sustain whatever the business is you're in. And so you can't just pull everybody into a new topic unless you're willing, you know, to disappoint all the existing customers. You know, no, no enterprise business has not made promises to enterprise customers of what's coming next on their roadmap. You know, lots of conferences, including our own, is is, you know, stuff they're shipping right now and stuff we're announcing for the next six months. And if you're not delivering on these announcements, your your customer base is not going to be happy. And so you can't really, you know, make that drastic move right away. And so we love this idea of a tiger team or in.

• 55:52Speaker 00

  Incubation team, we call it GitHub Next. And when we set that team up, we actually thought they're going to work on projects that are like five years out or, you know, in that Horizon three as space. And then it turned out, well, it was more like six months.

• 56:06Speaker 01

  The future came quickly.

• 56:07Speaker 00

  The future comes really fast on us. And so as that future then comes fast, you really also need to move fast in handing things from the incubation team over to your mainline engineering team. And then it's all about, okay, so we are funding AI because we are seeing the traction there. That means we're leaving some of our previous bets in, you know, keep the lights on mode, KTO, or, you know, saying goodbye to these ideas and shut them down. And I think that's the hard part of the strategic pivot. And that's easier, you know, for a startup, at least it often looks easier from the outset when a startup pivots, right? Like, there's lots of startups on the wall downstairs that have gone through that as well.

• 56:49Speaker 00

  But obviously internally, it's also very hard. Emotionally, you're tied to ideas. You know, you remember the workshop under the Koya trees where you had those ideas. And then six months later, you're realizing we never got to product market fit. And the time is now to move to something else. So that is the same for large companies as for small companies, except that small companies have more of a forcing function to give up and move to something else.

• 57:15Speaker 01

  The innovation and success that you've created around GitHub Next is amazing. How does a kernel of an idea start within GitHub Next? How do you then resource and invest into it? And then how do you, at what point and what's the process in which you decide to whether or not something should be continued to invest in or shut down entirely?

• 57:36Speaker 00

  So I think the start is always the employee, the huber, that has an idea. And we have lots of ideas in the company. We do, you know, hack weeks, hackatons, passion projects,

• 57:50Speaker 00

  whatever you want to call it, you know, 20% time. And then the next team specifically, you know, there's lots of demo meetings, demos and not memos, you know, it's much more useful to just show a working prototype. It's so easy. These days, you know, to just use some design system, react components and or Figma and stitch something together even easier with Copilot. And then, you know, it's strategic decisions that the leaders of these teams all the way up to me need to make. You know, in very many ways, very similar to any other creative industry, you know, like at Disney or Pixar. They have to decide which movies to produce and which ones are probably not going to gain traction. And part of that is customer research, talking with developers.

• 58:35Speaker 00

  And that then, you know, keeps going as we go, you know, through the initial idea and the first prototype and then going into a technical preview. And then the preview is all about, you know, that flywheel, that feedback loop with the people using it. And quickly you see, you know, are they trying it out and then they are churning? Or are they keeping, keeping the energy high and then keeping the ideas flowing? It's great, you know, in any, in any project, if people are just sending you more ideas and more feedback. And I think that's then the decision whether we are keeping that project and making it a main project or whether we are deciding, okay, the experiment is over and we learned a lot and then moving to the next one. And of course, there's commercial aspects as well.

• 59:17Speaker 01

  Maybe outside of the world of code generation and anything in your current purview, what else are you excited about in the world of AI in the span of one, five or 10 years?

• 59:28Speaker 00

  Or outside of developers? You cheated a little bit on me.

• 59:34Speaker 00

  I mean, I think in the context of one year, within the space of developers and outside of the space of developers, I'm excited about agents. And we're still very early in this journey. I think we have high expectations of what the agents could be, maybe too high expectations to some degree. And so things will probably take a little bit longer. But I think in the next year or so, we're going to see more of these helpers within the chat interface and outside of the chat interface that will solve tasks for us. You know, like I look forward, you know, to that travel agent that often gets demoed by big companies to actually materialize. And I can just go into my chat interface and say, you know, I want to have a beach vacation over spring break

• 1:00:16Speaker 00

  and then figure out, you know, when spring break actually is because that's all on the internet. And You know who I am, what my name is and what my family's names are and their birthdates and their passport numbers. And so I don't have to enter that into a cumbersome interface anymore. It show me the price points and then probably we're going to the same hotels every year anyway. And so I think that those travel agents and those, you know, kind of agents are going to happen in the next year plus, I think five years is my natural language vision and, you know, unlocking the world's knowledge, including software developers to everybody in any language, any human language. And maybe that's even happening sooner.

• 1:00:56Speaker 00

  Ten years is hard. Ten years is so far away. But I think, you know, that's the AI of things, you know, the material, like the mechanical world of AI. You know, so many things that we do in life, you know, you need to grab something or you need to push something and, you know, go and check into any hotel, there isn't any AI involved. Once you get your room key. And maybe the elevator, you know, has some kind of AI because you push a button, what floor you want to go to in itself, so to take elevator C. And so you no longer push the button within the elevator. But that's an optimization problem in itself. But I think there's so many physical things in life, you know, and we are still not at really self-driving cars. We have a dishwasher, but I still have to put the dishes into the

• 1:01:40Speaker 00

  dishwasher and out of dishwasher. It's better with nine and 12-year-olds than with two-year-olds. And there's so many other things in life where I think, you know, whether it's a robot or some other form of physical AI is going to to take over some of the things that we consider as chores.

• 1:01:59Speaker 01

  I'm really excited to hear that. And I think we have a very similar view of what will happen in one, five, and 10 years from now. Who do you admire most in the world of AI?

• 1:02:09Speaker 00

  I think I admire those that, you know, are building new stuff with AI and software. Those, you know, that have a dream of what they can, what they could build. I think those that, you know, obsess about a problem and not about the technology. You know, we're talking a lot about AI, but at the end of the day, it's what problem am I solving for the world? And, you know, there's lots of biotech companies that use AI to try to cure diabetes or cancer. I'm sure there's companies trying to solve climate change with technology. And I think those builders, you know, the founders, those that have, you know, big ideas and that can change the world, you know, those are the ones I admire the most.

• 1:02:53Speaker 00

  And often when I meet with them in in their offices, you know, or on on calls, I I'm like, this is this is so cool like and then and obviously as GitHub, we are enabling a a small small part of that. And so we we feel really proud of being part of their journey and and we're really, you know, excited about, you know, um, building more for them.

• 1:03:14Speaker 01

  Um, I think you're enabling a huge wave of new AI companies, especially those that are born open source. It's amazing to see. Um, what advice do you have for the founders listening in the audience who are building in AI today?

• 1:03:26Speaker 00

  I mean, already Sonia asked, I think focus is everything like it's so easy to get lost in, you know, the all the ideas that you can put on a whiteboard. And that's the danger of a whiteboard that it has covers, you know, has so much space where you can put ideas. But focus ultimately is everything, you know, finding very quick market validation. Um, often on the developer space, that means going product led growth, enterprise growth can come later. But like from my experience, like, there's nothing than, you know, a few excited developers spreading the word about your product, even though, you know, the big revenue number then later comes from from big enterprises buying you. But the reverse is often much harder of going enterprise first. And you might find excited people

• 1:04:11Speaker 00

  there as well. But the feedback loop is just so different. So it's focus, it's, you know, um, uh, trying to find that flywheel, the product market fit, you know, and then think, I think the other one is to think big. I think it's like, like, it's easy to find small ideas on top of, you know, model today that get, you know, commoditized tomorrow. And so you have to think forward in that, you know, 10 years is is a long period of time, but that's, you know, a good period of time for a startup to build something that is actually meaningful in this world. And so you have to think big, you have to, you know, create a a vision that may be much larger than the MVP, you know, the first thing, the prototype that you're building right now. And I think that's that's really

• 1:04:55Speaker 00

  hard as a founder to draft out that vision and then and then decompose, right, and go back to that small problem that you can solve right now.

• 1:05:03Speaker 02

  So we'll close with some rapid fire questions, one word answers.

• 1:05:07Speaker 00

  One word answers.

• 1:05:08Speaker 02

  One word answers.

• 1:05:09Speaker 00

  You changed the rules. It's good to say one word answers.

• 1:05:11Speaker 02

  One sentence if you needed.

• 1:05:13Speaker 00

  No, I say, I'm going to answer this, yes. Okay, let's go.

• 1:05:16Speaker 02

  Okay. Will anyone meaningfully disrupt Nvidia and AI chips in the next, call it, five to 10 years?

• 1:05:21Speaker 00

  Yes.

• 1:05:23Speaker 01

  In what year will we pass the 50% threshold for Sweet Agent?

• 1:05:28Speaker 00

  2025.

• 1:05:29Speaker 01

  And what about 90%?

• 1:05:31Speaker 00

  2028.

• 1:05:35Speaker 02

  Will GitHub primarily be-

• 1:05:36Speaker 00

  I think that was too pessimistic, that last one.

• 1:05:40Speaker 02

  We'll hold you to it. Will GitHub primarily be using open or closed source models in the next five years?

• 1:05:45Speaker 00

  Both.

• 1:05:47Speaker 01

  Where does the majority of value accrue in AI? Models, compute, infra, applications?

• 1:05:53Speaker 00

  Across the whole stack. That had hyphenated it, so it was one word.

• 1:06:00Speaker 02

  Is systems thinking and creativity going to get baked into the models in the next five years?

• 1:06:06Speaker 00

  Maybe.

• 1:06:09Speaker 01

  And will there be a new consensus architecture beyond the transformer in five years?

• 1:06:13Speaker 00

  Of course, yes.

• 1:06:14Speaker 02

  Wow.

• 1:06:16Speaker 00

  Why would you think the other way around, right? Like that said, that it's a much easier bet to think that there will be a new architecture. Because there was other architectures before transformers and like, that doesn't mean it replaces transformers. You know, your cellphone still has a CPU even though GPUs are the hot commodity right now. So I think, yeah, there will be, there will be new architectures and they might be bigger than transformers today.

• 1:06:39Speaker 02

  So interesting. I mean, that'd bring a lot of new oxygen for the builders in the ecosystem and a lot of things that would have to get reworked, rebuilt, re-architected.

• 1:06:47Speaker 00

  Yeah.

• 1:06:48Speaker 02

  Amazing. Thomas, thank you so much for joining us today. It's been wonderful digging into the history of GitHub, the birth of GitHub Copilot, and the ambition that you have going forward as well. Thank you.

• 1:06:59Speaker 00

  Yeah, thank you so much for having me. It was so fun to talk to you both.

• 1:07:02Speaker 01

  Likewise. Thank you.