Inside Cursor: The future of AI coding with Co-founder Sualeh Asif

In this episode of Gradient Dissent, host Lukas Biewald talks with Sualeh Asif, the CPO and co-founder of Cursor, one of the fastest-growing and most loved AI-powered coding platforms. Sualeh shares the story behind Cursor’s creation, the technical and design decisions that set it apart, and how AI models are changing the way we build software. They dive deep into infrastructure challenges, the importance of speed and user experience, and how emerging trends in agents and reasoning models are reshaping the developer workflow. Sualeh also discusses scaling AI inference to support hundreds of millions of requests per day, building trust through product quality, and his vision for how programming will evolve in the next few years. ⏳Timestamps: 00:00 How Cursor got started and why it took off 04:50 Switching from Vim to VS Code and the rise of CoPilot 08:10 Why Cursor won among competitors: product philosophy and execution 10:30 How user data and feedback loops drive Cursor’s improvements 12:20 Iterating on AI agents: what made Cursor hold back and wait 13:30 Competitive coding background: advantage or challenge? 16:30 Making coding fun again: latency, flow, and model choices 19:10 Building Cursor’s infrastructure: from GPUs to indexing billions of files 26:00 How Cursor prioritizes compute allocation for indexing 30:00 Running massive ML infrastructure: surprises and scaling lessons 34:50 Why Cursor chose DeepSeek models early 36:00 Where AI agents are heading next 40:07 Debugging and evaluating complex AI agents 42:00 How coding workflows will change over the next 2–3 years 46:20 Dream future projects: AI for reading codebases and papers 🎙 Get our podcasts on these platforms: Apple Podcasts: https://wandb.me/apple-podcasts Spotify: https://wandb.me/spotify YouTube: https://wandb.me/youtube Follow Weights & Biases: https://x.com/weights_biaseshttps://www.linkedin.com/company/wandb

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Lukas Biewald?

  You're listening to Graded Descent, a show about making machine learning work in the real world, and I'm your host, Lukas Biewald. Sualeh Asif is the CPO and co-founder of Cursor, one of the best-loved and most exciting and popular AI products out there. It helps you with coding, helps you use LLMs to do coding. I use it all the time and I really love it. And I was just excited to ask him about how he built such a great product. I found his answers super interesting, and I hope you enjoy this interview.

• 0:38Lukas Biewald?

  All right. Well, thanks so much for taking the time to talk. I guess, maybe this is a softball question, but I was really interested in just hearing the story of Cursor, like how you started it and what the moment was where it really started to take off, because you know, now it's like one of the most loved products, I think, out there.

• 1:01Speaker 01

  My story comes from, we had been really interested in sort of scaling laws and back in college, sort of, I had gone on and worked on a sort of search engine type company with a friend. And there we were really bullish on language models, because it felt like language models could really compress all the world's information, and there should be this end-to-end index of searching the internet, instead of sort of many of the heuristics we've coded in over the years. It felt like you could sort of, that should be the end-to-end way of doing things. And so, scaling laws, you know, doing the search

• 1:46Speaker 01

  engines, you know, training, training large models at the time, I think Co-Pilot was the first really big moment for us, where like, it was this project that was truly magical. It was fast, it felt like it kind of knew you. But then, you know, Co-Pilot did not improve much over the coming year or two. And for us, when we saw GPT-4, like, we thought the ceiling for what a really, really great product was possible at that moment was really high.

• 2:19Speaker 01

  And then it was like pretty clear that like as the models got much better, like, you know, scaling loss progress. Models get much better. The product that can be built in the future is even higher feeling. And that was like, you just, it was this sort of super attractive thing to go do it. And, you know, we're all sort of coders at R and

• 2:42Speaker 01

  we wanted to be building things that we use every day. And, you know, Cursor was originally built for ourselves in many ways. It was, and it was sort of fun seeing that, you know, everyone else really liked it. It was definitely built for ourselves.

• 2:59Speaker 01

  And we were sort of experimenting. So a lot of the early culture of the company was experimenting with various different ways of using the models. Did there be a document that you're sort of typing things out and the model is coding things? Should there be? You know, what, if you want to do this, you know, next action prediction of like, you're in a location, what should be the edit? Maybe you should be, the model should be telling you where to go next. You should be able to make edits over your entire repository. And some of those things have taken a year, year and a half, several iterations. And some of them, you know, we've continued building on. So now like some of the core parts of the product is, you know, this next action prediction thing where

• 3:44Speaker 01

  it predicts your next edit at the card's location and then where you should be going next. And that people really, really love that feature. And then we're, you know, sort of working our way towards, you should just be able to make any edit you want across the entire repository. Like code base wide.

• 4:02Speaker 01

  And obviously there's sort of hurdles along the way that we'll talk about.

• 4:07Speaker 01

  Some easy, some, you know, sort of still quite difficult, like models also struggle with what exactly the architecture of the repository is. If you, you know, ask what is the architecture of the repository? That is really quite difficult because it requires sort of looking at potentially billions of tokens, tens of billions of tokens and say,

• 4:30Speaker 01

  Asking the question, what is really going on as opposed to like you could like list of function names, right? Like, uh, but it doesn't really tell you, you know, that what is exactly going on.

• 4:42Lukas Biewald?

  Well, Tony, I want to dive into that as much as you're comfortable sharing, but I guess I wanted to ask you, you know, one of the surprising things that I learned in my, you know, background research on you is I think you guys came from using Vim, not VS Code. Is that, is that right?

• 4:57Speaker 01

  All of us were really early users of Vim. We did eventually, you know, had used VS Code. Probably the last one to, there was a couple of us, Aman and Arvid probably were the last to switch over from, from Vim to VS Code. And the trigger there was, was Gitab Copilot.

• 5:13Lukas Biewald?

  Oh, I see. So Gitab Copilot actually pulled you over in the end.

• 5:17Speaker 01

  So I, I had switched over before, but then, you know, I think Aman and Arvid only switched over after Gitab Copilot became, it was, it was this, it was just a kind of feature, right? In some ways it was a kind of feature.

• 5:29Lukas Biewald?

  Totally. And why doesn't something like Vim actually have something like what you guys built? Like, it seems like a lot of smart coders, you know, like to use it. Is there something about it, like a graphical interface that lends itself to, to this kind of structure? Like coding with an AI?

• 5:48Speaker 01

  I think for us, VS Code is, for one, it's pretty clear. It's the most loved platform on the internet. Or for coders, it's, it's the thing that sort of is the, it's a factor. It's, it's the default.

• 6:01Lukas Biewald?

  Totally.

• 6:03Speaker 01

  And we wanted to sort of incrementally evolve it towards the world where you're starting to automate coding. And the cursor of, you know, one year from now should look very different from the cursor of today, uh, which, which means almost by default, it should not look like, exactly like VS Code. But, you know, in looking very different, you wanted to start from a place where you, you didn't want to have a text box to code because, you know, coders still want to type characters, right? You, you want to be able to edit your entire repository at a higher level. But at some point, if you find that there's a change that you can like quickly execute in 10 keystrokes, uh, we want to let you be able to dive into the details.

• 6:46Speaker 01

  And, you know, at any point in time, you're maybe editing some pseudocode representation. You know, maybe a year from now, right? Like humans are editing a pseudocode representation.

• 6:59Speaker 01

  And that's, you know, really quick to add in and the model is sort of working for you in the background. But you're writing some kernel and you want to go in and toggle some of the indices. It's much easier to do it by hand than you always, I think developers will want this ability to go in and, you know,

• 7:16Speaker 01

  unless we truly believe that everything is going away, it's like you really, really want the fine grain control.

• 7:21Lukas Biewald?

  Yeah, yeah, that makes sense.

• 7:25Lukas Biewald?

  You know, one thing that that strikes me from what you were saying earlier about like observing that Copilot was really great. And there's all these, you know, all this opportunity and how do you kind of work with these AI models is I think a lot of people, other people thought that at the same time. So, you know, like, you know, you had this idea that I think many people had, including a bunch of like, Y C companies and other products that I saw. And it seemed like Cursor emerged as kind of the winning one among these, right? So it seems like there was great product execution here, which I'm always really interested in. Like, do you have a sense for what you were doing differently than your competitors that that made your product work so well? Was it like, was it like certain decisions or was it like a process?

• 8:08Speaker 01

  You know, why question your own is really hard? I don't know. It's, it's, it's very hard to tell exactly what we did, right? I think it was a bunch of things where we always tried to push the ball as much as possible without be, we always wanted to be the most useful product at any moment in time. It's like at the frontier. It's very easy to over promise and under deliver. And a lot of what we have tried to do is to, we didn't ship the agent until we were very confident it was something that was really useful. And we had probably done three agent prototypes before that we didn't ship because some version of

• 8:54Speaker 01

  the model would just lose track and you could make something that was.

• 8:59Speaker 01

  Could help you in the short term and really hurt the hurt what people think of as a reliable product in the long term. Uh, maybe that is that is part of it. It was always, uh, but then also I think we've been first to a lot of the inventions that people really like. So,

• 9:17Speaker 01

  you know, more recently, the the the ability to jump to the next location that should be edited.

• 9:24Speaker 01

  Is something we've had for closer to eight, eight months, 10 months a year. Uh, and we hopefully will release a much more upgraded version of it soon. That'll be like quite a bit better. And, uh, only recently, you know, other people have tried to to do that. So we've always tried to be like, think of what's coming and at least have a prototype out, uh, as soon as we think it's it's something that's really useful. There's the tab to jump feature saves.

• 9:53Speaker 01

  There's the apply features. Yeah. And we also like, you know, we've done this at scale also. So I think that that has benefited, uh, where, for example, for our custom tab model, we do something like 100 million requests a day. Uh, and and and quickly growing. Uh, I I think part of doing it well has been able to do it reliably for for lots and lots and lots of people.

• 10:17Lukas Biewald?

  Do you think that any of the the data that you have or the feedback that you have from users is part of your success? Or are you more making the decisions through your own experience?

• 10:29Speaker 01

  The the data definitely been enormously useful. Uh, I think

• 10:37Speaker 01

  you know, the the feedback loops that people consider obvious are indeed extremely useful. You want to be the company that, uh, feeds an extremely good product that everyone loves. And then that definitely helps in making the next version even better. Uh, it helps in small ways. It helps in training models. It helps in, yeah.

• 10:59Speaker 01

  Where the small ways are, you know, you, you understand how, how people are using your product and what is the most important thing to ship at any moment and then in big ways and like training models and improving just the core workflows. You could, for example, like technically speaking, one, one loop in the apply use cases,

• 11:18Speaker 01

  you know, you train your first version of an apply that is quite a bit bigger and you then deploy it for all users, you get lots and lots of data and then you can distill a slightly smaller model that gets faster, the people use it even more and you then distill an even smaller model and you can like keep compressing the models down because you're sort of generating the data that allows you to do that, but also.

• 11:42Speaker 01

  Yeah, it's just, it's this, this like feedback loop and then the, you know, some of the things get faster. So for now, up to like, I don't know, a thousand or 2000 line file, apply feels effectively instant. And that's what we wanted to feel, right? Right? We wanted to feel like applies this deterministic, you know, the, the figure out some deterministic algorithm to like place the blocks, but that's not actually what's happening. It's, it's a model that is actually rewriting the entire file and a lot of, you know, a lot of the improvements have been making the model smaller. There's obviously improvements and just making the inference much faster with, uh, when doing these speculative edits, but yeah.

• 12:19Lukas Biewald?

  For something like, um, the agents that you talked about it, like you had some iterations where it wasn't, you know, useful enough to ship, like, how did you know that it wasn't good enough to ship? Like, how did you think about that?

• 12:31Speaker 01

  I didn't use it on a daily basis. I think, I think these things are, are really quite easy to figure out. Uh, if, you know, you're coding 10 hours a day in Kursch, right? Like you're, you boot up the editor, you're making the improvements and you're seeing it on daily basis. If the devs themselves can't, don't use it every single day, it's probably not something that everyone else will want to use. I mean, there's obviously corner cases to this thing, uh, where we're not the perfect coders, uh, but you know, a thing like an engine is such a general feature that if you're not using it, it's almost certainly not useful.

• 13:03Lukas Biewald?

  That actually leads me to another question I had, which is you and your co-founders have this background in sort of competitive coding, right? Like, you know, Does that, do you think that's an advantage for you? Because I could imagine that that might sort of put you at sort of the forefront of like wanting to be efficient in coding. But I could also imagine that you might have idiosyncrasies in the way that you want to write code that might be different than your general user.

• 13:27Speaker 01

  I think we're not only competitive coders. Like, we did competitive math until because, you know, that's sort of part of the background. It's always really hard to distinguish, you know, what part of your identity is the most important. But many of us had worked at sort of software companies before Stripe and the like. And you had some idea that production coding was very different from. And then people had actually built products. So I think Michael had spent quite a bit of time building these high performance games.

• 14:01Speaker 01

  And, you know, we had done modeling work. So we had seen quite a wide variety of coding.

• 14:07Speaker 01

  So bringing back to like, does competitive programming really a really affect? How you did coding on a day to day basis? Like, not really.

• 14:17Speaker 01

  It was just, I think we knew what engineering was. We were doing day to day engineering and you could see if the agent was helpful. And in this case, it was very clear that, for example, early iterations were not really that useful. It was very slow.

• 14:34Speaker 01

  I mean, one of the most important things that had changed there is the length of the context windows that you can do on every single keystroke on every single request. You know, when the model started out, you would start doing these 4K, 8K context windows. And even if the models, you know, slightly supported them, the models were not very good at using the large context windows. And now that you can sort of easily do, as the cost curve has gone down for the language models, you can do requests in order of like,

• 15:03Speaker 01

  you know, mid hundreds of that, like, you know, 50,000 tokens, 60,000 tokens reliably. That has like enormously helped.

• 15:12Speaker 01

  Uh, one way, one intuition to have here is the model can't even feed your current file. Like it would not be very useful, let alone like read the rest of your repository or do searches or

• 15:23Speaker 01

  lots of things that you expect a basic agent to be able to do. Uh, that wouldn't work if your 8K tokens, 8K tokens, what can you even fit in there?

• 15:31Lukas Biewald?

  Right. One, another interesting thing that, that you guys said in, in your interview with Lux Freedom is that you kind of wanted the experience of using a code editor to be fun. Which I thought was kind of a cool idea, like a little bit surprising, right? It seems like such a utilitarian thing. It kind of reminded me that when I, um, I remember when I switched in cursor from my default LM from Sonnet to O1. Actually, I think I started coding a little less, like I was actually having a lot less fun because there were like, the latency was higher. It's kind of took me a little while to like, realize that, but I actually like, I like, for some reason, Sonnet had lower latency and it made it just like a lot more fun. And I was like, you know what? I just need to go back to the, you know, the LM where I was like, just enjoying like writing codes.

• 16:15Lukas Biewald?

  I do actually kind of relate to what you're saying, but I'm kind of curious like how, how the idea of like a fun experience shows up in your, um, utilitarian feeling application.

• 16:26Speaker 01

  I think it just, I mean, there's, there's always this sort of end metric, right? The end metric is how much you're enjoying using a model. And, uh, it's, it's been very clear that we enjoy using Sonic more than O1.

• 16:39Lukas Biewald?

  Uh huh.

• 16:41Speaker 01

  And part of it is, uh, there's a few things. So one is I think Sonnet is, uh, extremely, uh, is, is even at scale, like reliably quite fast. Uh, and I think we want to ship models that are even faster, uh, that are better than Sonnet, that are, uh, much longer context windows. Uh, that could make, you know, edits reliably over a much larger setup here code base, uh, for exactly the same reason, because it becomes much more fun. So it's, it's, in some sense, it's this, uh,

• 17:14Speaker 01

  hard to pin down feeling, but in some sense, you know what really affects it. Like you will get bothered if the, if you have to explain to the model again and again, what you're doing, or you will get bothered if, uh, the model doesn't really understand that's like, you had some decently viewed file open. And the model doesn't see it and that, that's sort of annoying. It's just, it's just straight up annoying.

• 17:38Speaker 01

  So you can turn it into some methodical thing that you can like track down. But some of the inventions are just like, you know, wouldn't it be more fun if blah, blah, blah happened? Like, wouldn't it be more fun if you were coding and the model would just, once you started doing it in factory, you could tap, tap, tap to the entire thing, like 10 tabs? What would that take? And then, once you think like, oh, you know, 10 tabs would make me feel really, really happy, you can then sort of reverse engineer the exact thing. Like, the modeling work that you would have to do. So like, what size of the model you want to train? How much time you want to spend sort of

• 18:18Speaker 01

  pre-training, post-training, and RL-ing the models? To be able to consistently do the same behavior again and again.

• 18:28Speaker 01

  And you could, another example, right? Concrete example is, you could always over-train the tab models to be annoying. So, part of this, you know, if you were to only worry about making sure that every single time it does the edit, you would

• 18:47Speaker 01

  over predict. Like, sometimes you really want to, like, be writing against a kernel. You want to spend some time thinking, and you don't want the tab model bothering you. And that's the thing you would only care about if you're making it fun and enjoyable, as opposed to something that like,

• 19:04Speaker 01

  yeah, something that's like, you know, obviously just always over predicting.

• 19:08Lukas Biewald?

  But this is a pretty subjective experience that you probably couldn't pull from user data. So, like, how do you work through that internally? Do you ever have, like, a difference of opinion with yourselves around what's the more fun approach?

• 19:23Speaker 01

  I think some of these transitions are subjective, but I think if you think it out, they're not always that controversial.

• 19:30Lukas Biewald?

  Interesting.

• 19:31Speaker 01

  At the end of the day, you're trying it out. There's always some intuition where you might, like, over-trigger in some direction, but for the most part. I think like if like you, there's not that much argument or is Sonnet more fun or is O1 more fun? I mean, Sonnet is arguably better. And hopefully, there'll be like more more models that are optimized towards keeping you in the flow. I think you you need a broad two categories of models. You need the category of models that is RL towards being fast and super large context windows and just make edits across your entire cloud base and make you feel like you're breathing through things. And you want a category of models that is sort of trained

• 20:14Speaker 01

  for being extremely careful for reviewing every single small thing before they make the edit. You know, maybe you do a bunch of research, they then make the edit in the background for you like and then come back to you with the PR. And in that case, the thing that will be fun is if they're more more correct than not. And they're fast is not the only thing that's fun. It's it's being correct or how they write out like how they prove to you that they're doing the right thing.

• 20:43Lukas Biewald?

  I guess, as you kind of build a bigger brand and you build, you know, trust with with users like me, why are you even asking me what model I want to use? Like, I'm I'm sort of aware of the different models, but I would sort of trust you more to know what's going to be fun and useful for me. Like, why?

• 21:02Speaker 01

  Yeah, I think I think you're kind of right. Yeah, I think you're kind of right. We've been we've been

• 21:09Speaker 01

  part of building the trust is always showing exactly what we're using. And I think it's you're probably correct that we should have a default mode and you should use the default and feel happy. But there's always if you're if you're the kind of person that wants to do mode and want to perfectly fine tune every single thing, you should be able to do that. And then there should be the simple default thing. So there should be a release in a week or two that fixes all of this for you.

• 21:36Lukas Biewald?

  Oh, here's something I've been wondering about myself quite a bit. Like, do you think that... Do you think there are best practices in changing the structure of my own codebase or the way that I should code to make your product work even, even better? Like, for example, we have one engineer that's been sort of letting, like, putting in, letting the LM, like, put in notes inside the codebase of, like, helpful things to kind of help

• 22:03Speaker 01

  I think that's like the one, one of the things that, that we, that we've been sort of speculating about. We don't actually have a really correct solution there. But this idea of, like, maybe there should be a readme.ai.n d in every folder. With the idea being, you know, at any point in time, if you ask changes around me folder, the model should be able to look up what's the nearest place where there's an architecture written down that it can...

• 22:32Speaker 01

  Sort of on, on the technical side, the thing to understand, the models are much faster reading tokens than humans. And, like, what is the magnitude faster than sort of ingesting these tokens? But humans have, for example, like,

• 22:47Speaker 01

  some small things memorized. So there is obviously small differences between how we code. Where the model is starting from scratch every time. So cursor tab in our codebase is named CPP for being Kopal++. And the model will always sort of needs, needs to be reminded that whenever you're searching for something that says Kopal++ or something, what you actually really need to, or whenever I say cursor tab, you should actually search for Kopal++ or something like that. So there, there are these facts and rules that are quite important. How, I don't want the default to be, so A, it would be better if everyone sort of changed their way of coding. I, I think the obviously better approach is we just figure it out.

• 23:31Speaker 01

  You know, we should just spend all the time and energy we need, all the compute we need to really nail down...

• 23:38Speaker 01

  Nail down the architecture that you have, like really figure out all the facts and rules. I don't know if I have any like interesting controversial ideas for how that should be done. Uh, someone was joking that, you know, maybe we should email you 10 rules in the morning and you'll just like yes and no on the 10 rules. And hopefully we'll build up a corpus over time. Like you, you want a system that allows you to add rules and then prune bad rules. Like sometimes there will be, like, if you just ask the model to look at a PR and give you some rules, that there sometimes it will come up with bad rules and you need a way of pruning them out. Uh, so like, what is the minimal set of rules, uh, such that, you know, all your PRs will come much easier? Uh, does the model need to look at all of the rules?

• 24:23Speaker 01

  I mean, we're still sort of figuring it out, but I think, I think there's something important at the core of this that is

• 24:31Speaker 01

  both in terms of like how humans would change and also in terms of what we should change just to like make the defaults much better. Uh, because not every single person will change.

• 24:41Lukas Biewald?

  Of course, but for example, like, do you think smaller file sizes are better because the model can more easily like navigate the code hierarchy? Or do you think that creates, um, complexity?

• 24:53Speaker 01

  There's always some trade off. So the, the funny joke is that like sometimes, uh, people will sort of keep adding to the same file more and more until the model can't edit it anymore. And then you just ask the model to refactor that file for you. Cause you, you just like sort of, you know, in, in Chris or terminology and composing the file more and more. Uh, and it seems pretty clear to me that there is obviously some advantage in the model seeing all the context development to the current task in the same file. And also that, uh, for future tasks, it'll be easier if the model, the file was smaller. Uh, I think infrastructure wise, we will also make it possible for you to sync all of these files to

• 25:35Speaker 01

  uh, a remote server. So like, we will have a copy, like a big enough copy of your code base at some point. So right now we're extremely privacy conscious and that means like we try to make sure that, that, that we never store any code past the life of your request. Ideally in the future, we can store at least some part of it, uh, in a private way that allows the model to very quickly do, uh, reliable edits. So you shouldn't have to sort of make these round trips for making every single small edit and that, that feels quite bad.

• 26:05Lukas Biewald?

  What else? You actually, you were telling me that you run infrastructure. Also, can you talk about like what the interesting infrastructure trade offs are at Cursor?

• 26:18Speaker 01

  We built lots of different pieces of infrastructure, sort of the traditional company infrastructure, but then there's also a lot of things. The one that we've been sort of very public about is our indexing infrastructure. We've spent a lot of time optimizing and running at quite enormous scales, like billions of files

• 26:36Speaker 01

  per day, kind of infrastructure. And for that, we've got our own inference. So for all the models that sort of embed your files, we run,

• 26:48Speaker 01

  you know, an enormous amount of just to see the really large pipelines. So like if you're

• 26:56Speaker 01

  some big company and you have like 400,000 files or 500,000 files, you want the ability to while the user's coding effectively feel like it's being instantly synced across to the server while the model is using the embeddings to like search the code base or edit the code base, et cetera, et cetera.

• 27:16Speaker 01

  So scaling that has been quite a challenge. And I think there's been this sort of broad category of databases that are being built on top of S3. And we're like a big believer in this approach of you should build your database slash

• 27:33Speaker 01

  like the, I don't think there's like a, like the usual term is sort of a separation of storage and compute or disaggregated, you know, storage databases. But so the classic example of this is

• 27:46Speaker 01

  we use turbo buffer. The turbo buffer stores most of the vectors on an S3 sort of path. And then they have write ahead log and you sort of write this write ahead log. The write add, when there's some compaction process that compacts the write ahead log

• 28:05Speaker 01

  back into, back into the database.

• 28:09Speaker 01

  And then there's sort of new challenges we've been dealing with, with this indexing infrastructure. So we've been thinking about, is there a way in which?

• 28:18Speaker 01

  You can support shared shared code bases. So if you, you know, all the people at Wasting Biases have a really big code base. Hopefully in the future, you know, you will be able to spin out background models editing your code base. And so, you know, we want thousands, if not tens of thousands of, of sort of clients that are connecting to, to that code base. And we don't want to have 10,000 copies of the Wasting Biases code base for, you know, most of which are not being utilized. So can we have a shared truck? And then every single person can have their sort of branch off that trunk.

• 28:56Speaker 01

  That sort of architecture is still,

• 28:59Speaker 01

  you know, we're working on it. It's, it's not exactly easy to do because, you know, how do you easily branch this vector database? At the end of the day, you want to be able to query both the trunk and your section and merge them in a way that you still get the correct top K chunks. That's, that's not trivial.

• 29:15Lukas Biewald?

  So when I fire up cursor, it's like quietly indexing all the files that are in my project.

• 29:20Speaker 01

  So we try to, you know, yes, exactly. So when you fire up cursor, it quietly indexes every single thing. Whether, as long as you both A allow us and, you know, if it's default turned on. One really popular cursor use case is like you open up a Git of repo, you clone it, and then you fire up cursor in that Git of repo. And now you can quickly ask questions about it. And we try our best to like make it effectively instant to you. You index these really, really large code bases.

• 29:47Speaker 01

  There's still, obviously if you clone like NLVM, which is 120,000 files, like that will take us a bit longer.

• 29:56Speaker 01

  But so for example, an interesting infrastructure question for, you know, the listeners or whoever is like pondering about is like, should you, how should you allocate these token capacities? So we at any point in time have a fixed number of GPUs, which means we have a fixed amount of token capacity. Could you be?

• 30:16Speaker 01

  You know, you want to index LVM or weights and biases and that's a really large code base. And then there's a bunch of people that have a number of small code bases. Should the number of small code bases always be allowed to go through and you should be slow? Or should you take a lot of the capacity in the beginning and everyone else gets like a smaller chunk in the in the hope that you shouldn't get a really bad experience? And that kind of question is still sort of hard to run, Skip.

• 30:42Lukas Biewald?

  Well, how do you think about that?

• 30:44Speaker 01

  Everyone, everyone.

• 30:46Speaker 01

  Currently, currently we try to like, keep, keep both, both sides like relatively happy. So, you know, there's, you can boost up your capacity a bundle of that thing. But I'm still like looking for better answers. Hope, I think there's probably, we haven't spent like that much time thinking about it. But hopefully there's like a really good answer to like, how do you make people happy?

• 31:05Speaker 01

  There, there's no serverless GPUs, right? Like, there's no great serverless option. Because at the end of the day, the amount of compute we're spending is still fixed. You know, Lucas indexes, the, the amount of compute is just like, you want to compute to index your code base plus the amount of compute for like every single other person that we're indexing. So in, in an ideal world, that'd be this phenomenal serverless thing where you could boost up your capacity. And then, you know, people can use that, that capacity and it will get boosted down again, which is what would happen in CPU land. And that sort of infra has not been built for GPU land.

• 31:39Lukas Biewald?

  Is indexing the main thing that your GPUs are doing? Because you're also running lots of models too, like.

• 31:43Speaker 01

  Yeah, yeah. So, so we run the tab model, like indexing is a very small percentage of, of our diffuses. I mean, we've done the tab models and hopefully we'll be running much larger models in the future. And yeah, they, they, they far and away dominate most of the compute cost.

• 32:00Lukas Biewald?

  I see. So the model running, the tab, the tab.

• 32:03Speaker 01

  Yeah. So tab models, like hundreds of millions of calls per day.

• 32:08Speaker 01

  The big models we're running have, you know,

• 32:13Speaker 01

  thousands of requests.

• 32:16Speaker 01

  And, yeah, okay, so without going into detail, you know, it's thousands of requests going on. And so we're, we're scaling up these models as fast as we can. And.

• 32:27Speaker 01

  They definitely take a far more compute. It kind of makes sense also. They're like larger. You know, one intuition to have is again, you're doing

• 32:37Speaker 01

  tens of thousands of tokens of inference per keystroke per person, which is both really cool and also really scary if you're running the inference. Like, obviously caching really helps, but it's still scarier, scarier than running a server.

• 32:56Lukas Biewald?

  Has it been any surprises as you've scaled up this ML infrastructure? Like, I mean, you've got to be one of the fastest scaling, you know, ML companies ever. Like, have there been any pitfalls or like, I don't know, like, what's that experience been like? Smooth?

• 33:13Speaker 01

  Um, there have been glitches, but I think, like, again, the team is really, really talented and we've sort of gotten over it.

• 33:23Lukas Biewald?

  Nice.

• 33:25Lukas Biewald?

  What about, I mean, we're talking like, you know, maybe two weeks after DeepSeek came out, and then obviously caused investors to like change their mind about Nvidia's stock. Did it like update your beliefs at all?

• 33:35Speaker 01

  It's been really weird to me because I think we're like both on the Lex pod, but also

• 33:41Speaker 01

  before that, we've been pretty public about using DeepSeek in many ways. And we used to use their 1.5 series models and then switched over to their V2 series model. So it was a big shock to me personally that like everyone was sort of like going hoo-ha, like this is some new thing, you know, they've been producing phenomenal work for a while.

• 34:02Speaker 01

  Their models, like I used to joke, like they were one of the three or four or five companies that you would trust, you like produce good models, where like the numbers wouldn't feel like they were juiced up in a way that there were certain models that felt like their numbers had been a little bit too juiced up. Like, by juiced up, I mean, like, they were really high for evaluations. But then if you use the model in practice, you would never like using the model. This is very specific to sort of the evaluations. But DeepSeek, I felt like was very honest about things and has been producing really good models. So we've been running DeepSeek V2 model for eight or 10 months now, probably 12 months, something like that

• 34:44Speaker 01

  on our, you know, own inference. And that's the model we've scaled up to hundreds of millions of goals.

• 34:52Lukas Biewald?

  Interesting. How did you choose it? Was it like just the best?

• 34:54Speaker 01

  Oh, we knew they were. Yeah, it was the best. It was the best. They had been producing extremely good open code models. Uh, we have our own sort of post-training stack and uh we do RL and stuff. But uh for just picking a really well pre-trained base, uh DeepSeek does a phenomenal, phenomenal job. Uh, the data they train on is really good and the model is both quite knowledgeable, quite smart and also quite cheap to run. For the tab in particular and you know, I think in general, I'm really excited about DeepSeek V3. I think DeepSeek V3 is actually a really well-pretrained base for large models. And I suspect it will be very, very useful for making these custom applications.

• 35:39Lukas Biewald?

  So I mean, you obviously launched um, you know, like agents and and it's it's pretty cool, but it's also kind of, you know, contained in like how many iterations steps the agent will will do and things like that. Like, do you like, where do you see kind of agents? Um, you know, going in there to, I mean, like, obviously inference is getting a lot cheaper. It seems like you could go much broader if you wanted to like, like, what are you thinking?

• 36:01Speaker 01

  Super focused on it. I think as people have been getting better at doing RL, the model has been getting better at both thinking and also being extremely coherent. So I think one of the things that I just talked about last is how coherent the models have gotten over producing tens of thousands of tokens of output, which uh they were not before. I think they would sort of go into delusional mode after a couple thousand tokens immediately. And now they they've gone quite a bit more coherent and that comes from doing RL and uh really, really good post-training.

• 36:32Speaker 01

  And I think agents were bottomed out by that particular aspect of coherency. Uh, one of the things that makes the sonic experience really magical for using in an agent is that it's so coherent over such a long uh period of time, like over tens of two calls and.

• 36:52Speaker 01

  You know, I suspect as the task get more and harder, you would need to be coherent over hundreds of thousands of tool calls and working on it.

• 37:02Lukas Biewald?

  One of the things that we

• 37:03Speaker 01

  I think it's generally like, again, like back to the mission of the company, the mission of the company is sort of, what is the, uh, we want to automate as much of coding as possible and while still having the developer in the, in the front seat. And automating coding in the short term involves, you know, allowing developers to just like, in the cases where they want to sit back and let the model code doing that, but in the cases where they want to drive the editor to like, make code, like, I don't know, you're doing voice and biases and you want to like switch your GRPC thing to some other TLS package. In Rust, like you should just be able to tell the model, like, I want to switch my GRPC thing to, to use Rust TLS instead of

• 37:43Speaker 01

  something else. And the model should just get it and be able to make these large scale code base changes. And that requires the model to be, uh, have some, some agent type things because you're never going to sit down and write out exactly the spec of your clippies. The, the thing that the agent really helps with is you don't have to sit down and explain like, yeah, we are one DB, we make this, you know, uh, we have a backend that is written in Rust and go, the Rust hooks up to going this way. And, you know, for our library, we use this and the model should just go and figure it out.

• 38:21Lukas Biewald?

  You know, my own experience of playing with agents, which is much diminished compared to yours is that, you know, when it breaks, it's kind of the challenge to debug. Like, have you built any systems internally for just even looking at like, okay, what is the agent doing? Like, you know, why did it get in a weird loop here? Like, what's happening? Like, how do you visualize that?

• 38:41Speaker 01

  Oh, we're building our own infra for it for now. Uh, I suspect that there will be phenomenal products in the future that will make this much easier. Um, for now, we, uh, same thing with building prompts. So we use this internal library called Quian.

• 38:58Speaker 01

  And the way we built Quian was it was well suited to our own need to the day. And I think for the same user agent infrastructure, we're building our own infrastructure in the short term. And I suspect in the long term, there'll be.

• 39:12Speaker 01

  Somewhat phenomenal, you know, DevTools that will come up to make it much easier to both inspect the chains, be able to stop at any point and restart the chains, be able to debug them in production when something weird goes wrong, all sorts of things that you will need to be able to run like a production system at scale.

• 39:31Lukas Biewald?

  Is the agent evaluation like more also like a, it sounds like it's more of a vibes-based approach than like specific metrics?

• 39:38Speaker 01

  Currently, yeah, it's pretty clear vibes-based. I suspect it'll be vibes-based in the short term and become, as we get better at shipping these, it'll become more and more sort of

• 39:49Speaker 01

  daily metrics and you'll be much more operational with it.

• 39:54Lukas Biewald?

  When you look at like something like a Devon or like these sort of like completely automated, like no programmer like approaches, do you view that as like competitive or interesting or like what is your?

• 40:07Speaker 01

  Others are interesting and in the medium term, if you can actually take your hands off and let the model drive your entire editor or drive, let the model drive the entire editing process, I am totally open to it. But in the case where it's like not really useful and kind of boring and not really that fun, like we just wait.

• 40:33Lukas Biewald?

  We just what? Just wait?

• 40:34Speaker 01

  We just wait until it gets good enough. Like we keep training the models and at some point it will get good enough and then it'll be really fun to use. I think in general, over a one to two year timeframe, I expect that the way people will code will change. And I don't think people, I think in the short term, that seems really scary, but I think it'll be this gradual process and it'll be extremely natural to everyone coming in. That the way, the way coding is changing, you, I think like, for example, the change from not having a co-pilot to a co-pilot was extremely natural in retrospect. Like it was not something that.

• 41:13Speaker 01

  That was scary to anyone, right? It was this thing that predicted the other thought and you were like, wow, this is phenomenal. And you just started losing it. And then the change from going from Discord to this

• 41:26Speaker 01

  foreground agent interface with a model that does edits across multiple different files. And you're like, oh, I want to switch this to use Rust TLS. And I want to make sure that you always use HTTP2 and blah, blah, blah. Like the model gets it and it reads all the files and it makes the changes. And you can immediately review the changes very quickly and tell that are correct. And that was also pretty natural. I don't think there was any way in the middle where people felt disoriented. And I think the sort of going to background things, it'll be

• 41:59Speaker 01

  all these things are always more gradual than one would expect. You would have expected in 2020 that if I said the way you'll be coding is you start talking to the computer and it'll make changes to random files. And you'd be freaked out. You'd think, oh, it's going to add all these bugs. It's going to be impossible to review. Like, I really enjoy coding. Why the fuck am I doing this? That like all these things would have seemed scary. And yet, five years in, four years into the language model,

• 42:30Speaker 01

  language model journey of products, things feel quite natural. So like 2021, Scopal, 2025 are in now. And at any point in time, making the change has not felt very disoriented. Which maybe in one chapter would have, but right now it's not really that disorienting.

• 42:50Lukas Biewald?

  Well, it feels like a lot of fun to me. I mean, I guess when I connect the dots from 2020 to now.

• 42:57Speaker 01

  It's gotten better. It's gotten better, right? It's not even gotten like.

• 43:03Lukas Biewald?

  I guess where like I'm going is like, you know, when I, when I look a few years out, it's, I have no idea, but it's hard not to see that, like, a world where you wouldn't really be doing anything that looks like programming a few years out, right?

• 43:17Speaker 01

  Yeah, like, more people will be coding, more people will be making much more difficult things, like, things that are considered much more difficult, be it lower-level things, be it larger projects, even for their side projects. I think people are usually very conservative with their side projects because they're like, "Oh, you know, I probably won't have that much time." I think people will get much less conservative with these side projects. I'm generally just extremely optimistic in the medium term.

• 43:47Lukas Biewald?

  Yeah, yeah. Do you feel like, at all, like,

• 43:54Lukas Biewald?

  I mean, I guess, first of all, like, I mean, don't you think it's a totally different world where everyone can do these, like, monster side projects easily? Like, that seems like, software is a very different feeling. Even, like, doing a software company seems like it might be hard to, like, you know, have, like, a protected advantage as much, right? When it's easy to build this stuff?

• 44:16Speaker 01

  I, I, I can't philosophize over that. I think, I'm not really scared of people having medium-sized, I tend to think of these things as, like, experimentation becomes much more, much more natural. I think a lot of the things that large changes are usually scary at companies because a large change requires changing so many pieces and changing so much time that you want to plan out everything upfront. And then planning is really hard because you can't really foresee, like, how your production system will look if you do X, Y, C. And everything becomes much more scary, and then you add more meetings, and it becomes more formal, and then, and everything just becomes worse and worse over time. And I understand it, right?

• 44:58Speaker 01

  If you're doing monkey-year database transition, boy, do you want to plan out every single small detail. And then you want to argue over every single small detail. But if you can start, sort of, prototyping these things really quickly, maybe it becomes less talking, more coding, you know, you have much cleaner, concrete artifacts.

• 45:20Speaker 01

  If you're in PyTorch and you want to do a small API change in PyTorch, it'll take a jerry, you probably want to debate out the hell out of it. If you're In PyTorch and you know, you can have a prototype in three days.

• 45:34Speaker 01

  Maybe you should just argue with the prototype now.

• 45:37Lukas Biewald?

  Is that how you do things at Cursor?

• 45:40Speaker 01

  Um, hopefully more and more so, yeah. I mean, there are still things that are scary, but definitely, I think I found myself thinking it's just much better to argue with the code. I suspect that that change will continue.

• 45:52Lukas Biewald?

  Hmm, awesome.

• 45:55Lukas Biewald?

  Well, I guess one, one final question. If something comes to mind, when you think, if you were sort of outside of Cursor and kind of like fresh eyes into this, you know, kind of world of AI applications and, and LMS are kind of working for so many different things, is there something else that kind of excites you that you wish you had time to, to think about?

• 46:18Speaker 01

  Personally, for me, I've, I've always wanted sort of like a really good reading experience, both I, I like to spend my time sort of free time either, you know, reading or, or spending time, even reading code bases. I think it's sort of this underrated aspect of, of coding that like, you know, all of us produce these, some of these artifacts that we've poured our, you know, many years of our life into. Reddit, someone has poured their life into Reddit and I really want to go read and understand Reddit. What were the hard decisions, what were the easy decisions? And

• 46:59Speaker 01

  I think both for reading books, for reading papers and for reading code bases, we haven't discovered the, the final optimal AI tool. I think hopefully Cursor will contribute to at least reading code bases, but maybe, you know, someone makes it easier to read books or read papers, I'll be really happy. Like reading papers is still like quite an arduous process. I mean, PDF viewers, I, I don't love the current PDF viewer still, like you click a thing and it'll jump you to the final thing. It's, it, it, it, it feels like a lot more primitive than it should be. And I've basically been reading papers by just pasting them into, you know, one of these sort of chat apps and are

• 47:40Speaker 01

  things are getting better. I, I, I, I think in general, like

• 47:45Speaker 01

  it feels like there's a lot, like a lot of low hanging fruit and lots of different areas of life.

• 47:50Lukas Biewald?

  Okay, I gotta ask, do you have any, what are your top recommended reading code bases?

• 47:57Speaker 01

  Well, so as I just mentioned, Redis, Redis is quite good if you haven't read it. It's relatively small and still

• 48:10Speaker 01

  it's still quite fun. Probably that's the one that I'd most recommend people because it's a thing that is used by everyone and

• 48:18Speaker 01

  it's just really, really well written. SQLite for sure also, if you haven't read SQLite.

• 48:24Speaker 01

  Again, very well written. It's this coherent document by a very small number of people. And then I think mostly I just recommend like software that you use, you should try to go read the software that you use. I mean, some things are harder, but like, I don't know. If you're a fan of Ghosty, the terminal, maybe you should go, go spend a weekend trying to read Ghosty. Or like if you're a fan of PyTorch, maybe you should go look into why PyTorch does what it does. I think there's a lot of choices that you can sort of criticize on the outside and people underappreciate the tremendous amount of work that people say in the PyTorch team have put in to make PyTorch like really, really easy for you to use. And this like magical experience where all the just sort of gradients flow naturally that that has taken like

• 49:10Speaker 01

  many tens of thousands of engineering hours. I don't know if it's in the hundreds of thousands of millions, but it's like a lot of engineering hours.

• 49:18Lukas Biewald?

  Interesting. Well, thank you so much. I really appreciate your time.

• 49:23Lukas Biewald?

  Thanks so much for listening to this episode of Grading Descent. Please stay tuned for future episodes.