#221 Varun Mohan: The AI Tool That's Changing How Code is Written (Codeium)

Launch, run, and protect your business to make it official TODAY at https://www.legalzoom.com/ and use promo code EYEONAI to get 10% off any LegalZoom business formation product excluding subscriptions and renewals. In this episode of the Eye on AI podcast, Varun Mohan, Co-Founder and CEO of Codeium, shares the journey behind building one of the most innovative AI-powered coding tools available today. Varun joins Craig Smith to explore the future of software development, the evolution of Codeium, and how generative AI is revolutionizing the way developers interact with codebases. As a leader in AI and software engineering, Varun discusses how Codeium integrates advanced AI to enable real-time code completion, refactoring, and even end-to-end application building. From processing billions of lines of code to making complex legacy systems like COBOL more accessible, Codeium is empowering developers to work smarter and faster. We delve into the challenges of managing massive codebases, the importance of maintaining security and self-hosted solutions, and how tools like Codeium stand apart from competitors like Copilot and CodeWhisperer. Varun also introduces **Cascade**, the AI assistant within Codeium's custom IDE, Windsurf, which merges human ingenuity with AI's reasoning capabilities for a seamless coding experience. Other fascinating topics include the role of AI in onboarding developers to complex systems, its ability to handle rare programming languages, and how Codeium is reshaping the software lifecycle from design to deployment. Learn how Codeium is redefining the possibilities of software development with AI, and what the future holds as this technology continues to evolve. Don't miss this deep dive into AI-powered innovation with Varun Mohan. Like, subscribe, and hit the notification bell for more episodes exploring cutting-edge advancements in AI! Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A. I. Twitter: https://twitter.com/EyeOn_AI (00:00) Introduction to Varun Mohan and Codeium (05:32) Introducing Codeium: AI-powered coding revolution (08:42) Competing with Copilot and CodeWhisperer (11:46) Using Codeium to streamline software development (14:01) Features of Cascade: Refactoring and automation (17:26) Building applications from scratch with Codeium (22:00) Enterprise adoption and recognition by JPMorgan Chase (23:14) The pivot to Codeium and lessons learned (24:32) The vision for Codeium's future

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  When you use a product like ChatGPT, it's largely speaking ephemeral. It only knows what you've chatted with it, not about the hundreds of millions of lines of software that exist within the company or in your codebase. And sort of we've come up with a product that's able to kind of with a human in the loop execute many steps autonomously to come up with a final work output and maybe a crazy stat, the company actually has over fifty percent of the company non-technical because we have a go-to-market team as well. And everyone in the go-to-market team has used the product to build successful applications, like entire applications, which is crazy to think about.

• 0:28Craig S. Smith?

  Okay, so why don't you start by introducing yourself, uh, where you went to school and how you started ExaFunction and pivoted to Codeium and then we'll talk about what Codeium is doing today.

• 0:44Speaker 00

  Hey, I'm Brune. I'm originally from the Bay Area, uh, right now in the Bay Area as well. Went to MIT, got my bachelor's and master's there in computer science and computer systems. And actually after college, worked at an autonomous vehicle company based out of Mountain View called Neuro. Worked there, was one of the first twelve engineers at the company. The company grew pretty big. I became a tech lead and manager at the company managing large, deep learning infrastructure. And after that, actually left the company in twenty twenty one to start ExaFunction, uh, which is actually the precursor to Codeium. And we were actually a GPU virtualization product. So we made it so that computations could run entirely on CPUs that actually had a GPU component.

• 1:28Speaker 00

  And transparently, we'd offload all the GPU computations to a remote machine, right? And for us, uh, we were able to manage workloads upwards of ten thousand GPUs. Uh, and we even built GPU compiler technology, right? In the middle of twenty twenty two, uh, we sort of realized that transformers were going to become very popular and powerful, right, with the advent of these generative models. And we felt that generative models would usher in a brand new age of computing, similar to the internet. And when internet is going to get started, we want it to be a Googler and Amazon, right? And we believed by becoming a vertically integrated application company, we could sort of go out and do that. And that's why we sort of built out Codeium and we started training our own models and running in them ourselves. And in the process, we actually accrued over eight hundred thousand developers using the product.

• 2:13Speaker 00

  Uh, we actually, uh, processed over a hundred fifty billion tokens of code. We're one of the top five to ten large generative AI apps in the world, uh, right now. And, uh, some of the worlds are just enterprises currently use Codium. So, uh, JP Morgan Chase just inducted us in the Hall of Innovation. Over a thousand enterprises currently use the product. And maybe the most exciting thing that happened very recently is the product is largely speaking, available in the place developers write software, which is called the IDE. And we work in every IDE and every place that developers store their code, regardless of if they store them on GitHub, if they store it on AWS, or any of these other like large companies products. We work everywhere, which is, which was a big differentiator. And we work in a way that is self-hosted.

• 2:55Speaker 00

  So the company doesn't need to share their code outside of the company if necessary as well, which is, you know, security is obviously a big concern for IP as deep as code. But very recently, actually this week, we, we launched a product called Windsurf, which is our own IDE, right? Our own way of writing software. And this is the first agentic IDE that has come out. And it actually enables developers to not just write code and autocomplete their software, but actually complete tasks end to end and continue from where they left off. So very different from some of the current agents out there where you just give it a task and it completes it. We are actually building the product. So the mind of the human and the AI kind of melt together, right? So if the human goes out and makes some changes, the AI can just kind of continue from where the human left off.

• 3:39Speaker 00

  And the agent is also able to deeply reason about large code bases, right? Which is actually a big deal, right? When you use a product like ChatGPT, it's largely speaking ephemeral. It only knows what you've chatted with it, not about the hundreds of millions of lines of software that exists within the company, right? Or in your code base. And sort of we've come up with a product that's able to kind of with a human in the loop, execute many steps autonomously, right? To come up with a final work output. And maybe a crazy stat, uh, that's kind of interesting is the company actually has over 50% of the company non-technical, because we have a go-to-market team as well. And everyone in the go-to-market team has used the product to build successful applications now, like entire applications, which is crazy to think about.

• 4:17Craig S. Smith?

  And, and the interface is, is what is conversational AI or, or graphical interface?

• 4:25Speaker 00

  So it, it, it's very familiar. It's very familiar with VS Code. It's actually backwards compatible with VS Code. So everything you can do on VS Code, you can kind of do there. But the difference is sort of you can give high-level instructions to the system and the system we The agent-like system that lives within Wind Service called Cascade. And Cascade will actually go out and edit files, uh, many files for you to actually reason about your codebase. And potentially even run code for you, right? One of the unique aspects of code that's very different than marketing copy or legal review is it can actually be executed, right? Which is why code can be very easily tested. And because of that, it can actually be self sort of introspective, right? Uh, because you can actually run code, see if there's an error and kind of go and fix it itself.

• 5:04Speaker 00

  Uh, and and it's able to complete very complex tasks because of that, uh, much more than what just a generic product like ChatGPT.

• 5:11Craig S. Smith?

  First of all, hi, is this built on uh, your own uh, L L M? Or is it built on one of the, um, one of the proprietary, I mean, do you you hitting, uh, you know, G P T four O or O one with an I E P I caller? How how do you, uh, what's the foundation?

• 5:31Speaker 00

  So it's a it's a mix of both, right? Like for us, we have, we have many different models that run in the product. We have product models that run on every key stroke that are not only auto completing code, but what we like to call super completing code. And what that means is as you're typing code, it's dynamically refactoring the rest of the code. You can actually tab to accept an entire refactor of your function, right? And then also at the same time, though, there's a system in Cascade that it does high-level planning. And then also on top of the high-level plan, there's from the high-level plan, actually doing retrieval of your codebase. And then actually going and applying the the high-level plan to the codebase. Those are proprietary models, but actually for us right now, we've actually settled on for the highest level planning model. We're actually using products like OpenAI or Claude, um, to kind of do that. And down the line, we're happy to also use products like Llama or whatever as they continue to get better.

• 6:15Speaker 00

  Uh, but it's a kind of a mix of everything, right? It's it's almost like for different kinds of domains, we want to use the best products. And we do feel like right now OpenAI and Anthropic have, uh, potentially the best sort of like planning models out there.

• 6:26Craig S. Smith?

  And and I saw, I mean, this is evolving so quickly. Um, OpenAI has just announced a further extension. Uh, I'll cut it out because I can't remember what it was, but it was related to coding. Um, the uh, and how do you compete with Copilot or Code Whisperer, which are are focused on the same task or market?

• 6:53Speaker 00

  Yeah, so I, I think, I think the way we, we compete with these companies is, um, I'd like to say like, the product is more beloved. Like if you look on VS Code and just like the rate of adoption, it is, it is, it is pretty remarkable. Right now, we have, you know, a five star review on, on the marketplaces, uh, where developers use the product. So I think it's, it's, it's delivering more value for the end user because it's more aware of the code base. We're not locked into any ecosystem. So we're not locked into the Microsoft ecosystem, nor the Amazon ecosystem, which is important because customers actually have, you know, uh, they have multiple clouds that they operate on top of. They use many different source code management tools, right? The only source code management tool isn't just GitHub, right? There's GitLab, Bitbucket, Perforce, Garrett, Mercurial, CVS. I could keep going down and down on all the source code management tools that kind of exist inside the enterprise. Um, and then finally speaking, I think, I think because we are vertically integrated,

• 7:38Speaker 00

  we can actually provide experiences that are more personalized, right? We can more deeply understand their code base. We can run it in a way that runs entirely within the firewall of the, of the customer. And all of those put together makes it so that, uh, we're able to offer a differentially sort of better and structurally better offering, uh, that's not just like tied to the ecosystem, right? Um, that, that these big incumbents already have.

• 7:58Craig S. Smith?

  Yeah. Uh, and, and so you're, you're writing code from scratch or is this, uh, for existing code bases that you wanna, as you said, refactor or do something else with?

• 8:11Speaker 00

  Both. It's both. I think like probably the most valuable application is existing code bases, right? People in large companies aren't like, you know, building these, these apps from zero to one. The product can very easily write apps zero to one. But if you look at where most of the value is, like, you know, if you look at probably the, you know, the most valuable companies out there that use a lot, like, let's take a JP Morgan Chase right now. Um, JP Morgan Chase isn't giving you a new app to use every week, right? Uh, they're, they're kind of taking their existing app and adding more functionality and making it more robust, right? And that's a very hard problem because the amount of software that already lives within these companies could, could tend to the billions of lines of code, right? Maybe tens of billions, right? So, um, you know, you put that all together and, and actually the hard problem is

• 8:51Speaker 00

  How do you actually take existing applications and not only understand them more deeply and onboard developers onto them, but actually modify them and extend them in ways that are most valuable to the company?

• 9:00Craig S. Smith?

  Yeah, that's something I've always wondered. I mean, if you have an enterprise with a codebase of billions of lines of code, how do you understand it and how do you work on it? And does something like Codium, can it ingest or review a codebase? And if so, what's the limit and how much time does that take?

• 9:28Speaker 00

  Yeah, I mean, this is like a deep problem that we spend a lot of time on. So, I mean, like, look, I wanna, I wanna say it's a very hard problem, but we've also solved very hard problems like technology problems in society. Like, let's even take a product like Google, for instance, like, think about the amount of code that exists inside a company and compare the amount of text that exists on the internet. And Google is indexing the entire thing and able to do good retrieval on it, right? So there are like hard problems that have been solved. And I'm gonna say like there are other companies that have solved it in crazy scale. I think what makes code a particularly more challenging problem is, is everything is interdependent, right? A codebase is actually, when you ask it a question, it's not, you know, a little bit different than maybe Google where every webpage is kind of, you know, its own thing.

• 10:12Speaker 00

  You know, maybe, maybe it's hierarchical of like the website has some metadata attached to it and the webpage has something. Codebase is extremely complex, right? Like you have some directories that are related to some other directory that's like many, many layers different internally. So I think, I think it's a hard problem and we spent a lot of time finding out ways like how do we cluster code, right? How do we actually use static analysis to make it so that we more deeply understand it? And how can we do retrieval very quickly? Use properties like embeddings, use properties like keywords, use properties like LLMs to be able to find like semantic meaning in the software. And, and yeah, it's a, it's a, I don't think there's like a very clean answer of like, there's one approach that kind of works. Kind of similar to like how, you know, if you were to take a product like Google and ask like, how does retrieval work?

• 10:53Speaker 00

  Yeah, there, there's an original idea called PageRank that says it signifies what the, what the rank of a page is, given a match. But, you know, if all You did was do page rank. I doubt you'd be able to recreate Google. You know what I'm saying?

• 11:04Craig S. Smith?

  When I started working on my own and I didn't have the structure of an organization behind me, I knew that I needed to incorporate file taxes and all of that stuff. I looked into getting a lawyer, but it was prohibitively expensive at that point. I really wasn't making much money. So I turned to LegalZoom to get the corporation officially registered and legally compliant. This episode's sponsor is LegalZoom, a trusted leader when it comes to setting up a business right. I'm a true believer because I still use LegalZoom. So business owners, have you set up your company properly? The pain-free way is LegalZoom.

• 11:49Craig S. Smith?

  It's the most trusted way to set up your business and stay out of legal trouble. So launch, run and protect your business to make it official today at LegalZoom.com and use the promotional code IONAI to get 10% off any LegalZoom Business Formation product, excluding subscriptions and renewals until the end of the year. Get everything you need from setup to success at LegalZoom.com. That's L E G A L Z O O M dot com and use the promo code IONAI. That's E Y E O N A I.

• 12:32Craig S. Smith?

  LegalZoom.com with promo code IONAI. LegalZoom provides access to independent attorneys and self-service tools, but it's not a law firm and does not provide legal advice, except where authorized through its subsidiary law firm, LZ Legal Services LLC. Yeah, sure. But, but, but when someone adopts Codium and they have a massive code base, what, what are those steps? Is it?

• 13:06Craig S. Smith?

  Do you add Codium as like a plugin to your repository?

• 13:14Speaker 00

  So you can add Codium as a plugin to your IDE, right? And by the way, it's also like recording statistics about your company, which are like helpful of like, how are different teams getting value from the product, right? Like, what are the most valuable parts of the product? Like, how much of your code that's actually getting written is generated by Codium? But then also now, like, you can install WinSurf, which is our own custom ID that I think is like the best ID experience out there on the market. So we don't want to be, I guess, like, our goal as a company is not to be dogmatic. I think, like, developers should choose, right? Like, it's kind of like how developers don't just do one thing across, like, the entire enterprise. And we will provide a, like, a host of suggestions that we feel very confident in. We'll put the stamp on it and the company should decide afterwards.

• 13:54Craig S. Smith?

  But so just walk me through how someone would use Codium if they have an existing codebase.

• 14:01Speaker 00

  Yeah, the way these applications work is you would open them in the IDE, right? You would open the piece of code in the IDE. And then after that, you would be able to do a host of factors. Like, if you already know kind of like what you're doing, as you're typing, it would actually autocomplete and suggest like changes within the code passively, right? This is what we call the super complete experience internally. You can either like do that or highlight a piece of code, use this feature called command and just suggest like a high level refactor and it'll rip through the codebase in, you know, hundreds of lines a second and kind of like give you the final answer of like, here's what the change should be, which is awesome for refactoring. Or at the very high end, if you have like a very, very high level idea of what you want to do, you can actually use something like Cascade, which will go out and actually look at the entire codebase, find out sort of where the changes need to get made, make the changes, right?

• 14:45Speaker 00

  And then maybe even run your unit test for you to say like, yeah, I feel confident about those changes. And you can even ask it to go and commit the code and push it to the final repository.

• 14:54Craig S. Smith?

  And Cascade is part of the Codium, I'm sorry if I missed that earlier.

• 14:58Speaker 00

  It's part of the WinSurf IDE.

• 15:00Craig S. Smith?

  Yeah, and it writes the unit tests as well.

• 15:05Speaker 00

  You can also write the unit test for you. Yeah, I can write it and then after that I take it to.

• 15:09Craig S. Smith?

  And then run it, yeah. I mean, is, you know, these are probabilistic models and they, even with all the safeguards and engineering around them, they can still make mistakes. So how do you, and if you're working on a massive codebase and you're doing something like Cascade, you're going back into the codebase and changing. Uh, different elements. Uh, it, how do you guard against um screwing it up? You know, making mistake and then not knowing where uh that mistake was made. And suddenly you've uh you've got a bug in the code.

• 15:53Speaker 00

  Yeah, I think the way we like to do it is like, look, the AI is like almost treated like a person that can make mistakes. Um, you know, this is like a big part of the product. Their UI is also important. We give people like easy ways to go out and actually review the changes that are actually ultimately made it. And after reviewing it, we want to see if if uh people feel that uh it it makes sense, right? Like you have to review the software and like uh the same kinds of bugs that humans make, AI is also going to make them. And uh I I don't think this gets away from it. In fact, like, think about it in general, like even with and without agents, like, people were copying Stack Overflow code and stuff into their code base, like that didn't eliminate the need to review software. So I think I think reviewing software is still incredibly important. I think AI can actually be very helpful in the review process. It can kind of give you explanations on like what this code does so that you can actually like kind of find out like,

• 16:38Speaker 00

  hey, what are the pieces that are probably most important from like most important to least important for me to read? Right? And it doesn't mean you don't read the least important thing. It just means that if you want to get a general sense of where the most bugs are, probably start with the part with the with the most highest likelihood of bugs. Right? So it makes the process of reviewing software much more efficient. And uh, but no, this doesn't eliminate the need to review software. In fact, it probably like increases the need to review software.

• 17:00Craig S. Smith?

  And and that review, as you said, is uh AI aided, but you still need people that can look at uh at chunks of code and and read it as and ensure that it's it's.

• 17:14Speaker 00

  That's right. Yeah, the review process is not going away.

• 17:16Craig S. Smith?

  And and how uh how the does someone use uh Codium to write applications from scratch or models?

• 17:25Speaker 00

  Same process. You can actually just in an empty directory, just say like, I want to build app X and it'll go out and systematically build it. It might ask you some questions about what maybe some more requirements it has and and you know, you can kind of see where it goes.

• 17:37Craig S. Smith?

  Yeah, and and that's with natural language that you're you're.

• 17:41Speaker 00

  That's with natural language. That's right.

• 17:42Craig S. Smith?

  It's just amazing how this is progressive.

• 17:45Speaker 00

  Go out and download it and try it out. Like, I think you'll be uh like excited about what you could actually build in within five minutes.

• 17:51Craig S. Smith?

  So uh this is a wildly successful.

• 17:56Craig S. Smith?

  People are using it all over the world. Uh, where do you go from here? Because you you said that there's a certain amount of autonomy. Um, but, um, you know, agents are the hot new things, society of agents or networks of agents. Uh, are are are you looking at that at at, um, uh, introducing agents so that you users can direct them and they'll go do the work uh without a coder actually having to type the code?

• 18:30Speaker 00

  Um, I think I think the place where this stuff is moving is, um, you know, when I think about the process of software development, it's like there are many stages, right? There's like, uh, writing software, reviewing software, designing software, testing software, deploying software, debugging software, right? There's a lot of different pieces. I think what's going to fundamentally happen is it's going to be like chopping wood from every single one of these places with time. Like very clearly, just writing software fast doesn't suddenly like make the time to review software faster unless you like actually make a concerted effort there. And you're going to be chopping wood from every single one of these places. And as you sort of chop more and more wood, um, what I think that will enable developers to do is spend more time doing the things they actually want to do. I don't think like wasting time deploying software, uh, that is actually, uh, you know, kind of like, uh, like a pretty deterministic thing

• 19:13Speaker 00

  should be something that developers spend a vast majority of their time doing, right? I would love to get them back to actually designing software. Solving the pain of and solving the problems that they actually want to solve, right? Like, what's the purpose of software? The purpose of software is to fundamentally solve a problem. It's not to write software, right? So I think I think, um, I think it's going to fundamentally give people a lot of time back. And I think more people can then participate in the software creation process than than was possible before.

• 19:38Craig S. Smith?

  That's what interests me because I'm I'm not a coder, but I'm very interested in software. Uh, and I've talked to a lot of people about various no code. Platforms or or you know, code completion. I've talked to Amazon about her AWS about code whisper. At the so so your your selling point is mostly ease of use. Is that right?

• 20:03Speaker 00

  I don't think I would say that. I think I think maybe the right way to look at it is is um, in fact, I think this is enabling people to do things that they could never do in the past. I'll give you a couple of examples here. Um, with a system that now deeply understands a piece of software, let's just take one example here. People that are new to a team or new to the company can now onboard onto a very complex system a lot quicker, right? Because they can actually ask deep questions about the code base, right? That's one. Maybe a more important one that I think is important is a lot of languages are actually languages that are not very common, right? Like, let's take a company that's writing in a language like Cobol, that was mostly written in the seventies, but is largely speaking needs to be maintained because some applications still do run on mainframes, right? Um, I think generative AI makes it so that it's easier, right? It's easier to introspect these applications and update them.

• 20:45Speaker 00

  So I think I think the right, maybe the right mentality is not, actually, this is easier. I think this is the whole point is actually to make things that were not possible in the past now possible. Because easier is one thing, you can you can throw more people at that, but but smarter and more capable actually means that the business could do things that it never could have dreamt it could do in the past.

• 21:02Craig S. Smith?

  That's interesting, the comment about Cobol. It it it what? How many languages can Codium handle?

• 21:11Speaker 00

  Yeah, so I think the crazy thing is it can actually handle uh sort of the entire gamut of sort of languages in the in the in the in the software development lifecycle. It can handle like every language out there.

• 21:20Craig S. Smith?

  Is that because it was trained to do that? Or or it it can just uh on first glance uh recognize uh the patterns in in the code?

• 21:33Speaker 00

  I think it was because it was it was it we do actually over sample on some languages, right? Like some languages like Verilog RTL that are more that are more uncommon. But I think it's also because we, you know, the base model understands a lot about different languages. And then after that, we're able to use forms of context awareness that should read different parts of the code base to then provide better suggestions.

• 21:52Craig S. Smith?

  And and what are some uh some of the the the biggest? Enterprises using Codium, maybe you don't have to name them, but.

• 22:02Speaker 00

  Yeah, so very recently, we just kind of like won the JP Morgan Chase Hall of Innovation Award. So a lot of like, you know, I think like we've been great vendors to one of the largest software, sort of software companies in the world, right? They they actually work with the most number of software companies out there. And and I think I think sort of some of the largest companies that you can think of are currently using Codium.

• 22:23Craig S. Smith?

  To build this, you you started with GPU optimization. Did you did you just, you know, have excess development muscle and say?

• 22:38Speaker 00

  No, yeah, I think we we just we decided that this was more important and and more more likely to actually succeed and be a be a durable company. And because of that, we pivoted almost overnight.

• 22:51Craig S. Smith?

  Wow, and and you had to build the system overnight.

• 22:55Speaker 00

  Yeah, we needed it took time and we didn't. There was a period of time when we had, we needed to work on something when we had nothing. And we we probably prodded along.

• 23:04Craig S. Smith?

  How does Codeium relate to things like like Devon, which got so much press when they released their first marketing video?

• 23:14Speaker 00

  Yeah, I think the the I think I think it's hard for me to know what that product actually does. Given it's not accessible, I think like maybe one of the things is we're generally accessible. So you can use the product now. That's like, that's that's one nice differentiator. Right, I think, but but but beyond that, I think I can't really tell.

• 23:33Craig S. Smith?

  But in terms of of that, the the being able to ask the the system.

• 23:41Speaker 00

  I think for what for what it's worth, if if they are capable of doing what they what they said they were, which is replacing a developer, which is effectively what they said, right? AI software engineer, it's like, I guess that's way more powerful. But also at the same time, like, I've not heard anything about them doing that in the last like six months.