#224 Tariq Shaukat: How Safe Is AI-Assisted Coding?

This episode is sponsored by Oracle. Oracle Cloud Infrastructure, or OCI is a blazing fast and secure platform for your infrastructure, database, application development, plus all your AI and machine learning workloads. OCI costs 50% less for compute and 80% less for networking. So you're saving a pile of money. Thousands of businesses have already upgraded to OCI, including MGM Resorts, Specialized Bikes, and Fireworks AI. Cut your current cloud bill in HALF if you move to OCI now: https://oracle.com/eyeonai In this episode of the Eye on AI podcast, Tariq Shaukat, CEO of Sonar, joins Craig Smith to explore the future of code quality, security, and AI's role in software development. Tariq shares his journey from leading roles at Google Cloud and Bumble to helming Sonar, a company disrupting code assurance for developers worldwide. With over 7 million users and support for 30+ programming languages, Sonar has become a critical tool in ensuring clean, maintainable, and secure code. We dive into Sonar's innovative AI Code Assurance Workflow, which integrates seamlessly with generative AI tools like Copilot and Codium. Tariq discusses how Sonar addresses the challenges of AI-generated code, tackling issues like security vulnerabilities, maintainability problems, and the accountability crisis in today's coding landscape. Tariq also unpacks the importance of hybrid deterministic and AI-driven approaches, the role of design and architecture in modern software development, and how Sonar is helping companies manage tech debt across billions of lines of code. With Sonar's recent enterprise-grade SaaS launch and commitment to reducing developer toil, this episode offers valuable insights for developers, tech leaders, and anyone interested in the evolving intersection of AI and software engineering. Don't forget to like, subscribe, and hit the notification bell for more discussions on AI, technology, and innovation! Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A. I. Twitter: https://twitter.com/EyeOn_AI (00:00) Introduction to Tariq Shaukat and Sonar (01:23) Overview of SonarQube (03:03) Deterministic Systems and AI Integration (07:36) Challenges of AI-Generated Code (10:12) Early Issue Detection in Development (12:33) Accountability in AI Code Generation (16:20) Importance of Rigorous Code Reviews (19:34) Managing Tech Debt with Continuous Improvement (22:16) Why Sonar Focuses on Integration (25:08) Reviewing Billion-Line Code Bases with Sonar (29:37) Tailoring Sonar for Specific Codebases and Workflows (32:40) Avoiding Overwhelming Developers with Noise (37:49) Governance and Managing Complex Codebases (40:50) Addressing Tech Debt in Legacy Systems (45:07) Sonar's Open-Source Model and Philosophy (48:11) What's Next for Sonar

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Maybe Gen AI can help with design and architecture work, but fundamentally, somebody has to sit there and think about how is the health insurance system going to work? What are the design and architecture rules that you're going to put in place? How are you going to construct this billion line code base? How are you going to to maintain and migrate it? How, when you're going to move part of it into the cloud? How do you take pieces of it and do it intelligently? Right? Like that? I don't want to say AI can never help with that because there's already AI tools that try to help with that. But these are real hard problems that I think require a deep understanding and focused effort.

• 0:35Craig S. Smith?

  AI might be the most important new computer technology ever. It's storming every industry and literally billions of dollars are being invested. So buckle up. The problem is that AI needs a lot of speed and processing power. So how do you compete without costs spiraling out of control? It's time to upgrade to the next generation of the cloud. Oracle Cloud Infrastructure, or OCI. OCI is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds, offers one consistent price

• 1:16Craig S. Smith?

  instead of variable regional pricing, and of course, nobody does data better than Oracle. So now you can train your AI models at twice the speed and less than half the cost of other clouds. If you want to do more and spend less, like Uber, Eight by Eight, and Databricks, Mosaic, take a free test drive of OCI at oracle.com/eyeonai. That's E Y E O N A I, all run together. Oracle.com/eyeonai. That's oracle.com/eyeonai.

• 1:57Craig S. Smith?

  Why don't you go ahead and introduce yourself and tell us how you got to Sonar?

• 2:04Speaker 00

  Sure. Well, I'm Tarek Shauq and I'm CEO here at Sonar. Pleasure to be on the podcast, so thank you for having me. I joined Sonar about a year and a quarter ago as co-CEO with our founder and then more recently as CEOs, he stepped more into the founder role as opposed to the co-CEO role. Yeah, I, prior to being here, I've got a long history inside of various parts of the tech world, both as a user of technology on the enterprise side as well as as a, as somebody working in the tech space.

• 2:43Speaker 00

  I have was, prior to here, a president of Bumble and helped take it public the last couple of years. And prior to that, was president of the Cloud Group at Google for a number of years, helping to get that really off the ground and scaled to a substantial scale that it's at now.

• 3:03Craig S. Smith?

  Great. And, and tell us what Sonar does? And the, the, the URL of Sonar Source, right?

• 3:13Speaker 00

  Sonarsource.com, exactly right. So, Sonar is, we believe, the leader in code quality and security. We started off really 15, almost 16 years ago now, as an open source project, building a product called Sonar Cube that really inspects code as developers are writing that code. And it inspects it for, really what we refer to as, as quality in general, to make sure it's clean. So what that means is, we look for bugs, we look for maintainability issues, we look for stylistic issues, we look for really anything that could be wrong with the code as you're writing it,

• 3:57Speaker 00

  including security issues as well. And this has, we, we do this in 30 different programming languages, we do this globally, and it's turned into not just the open source project, which is still available and a lot, we have 400,000 organizations who use our product, over 7 million developers use our products. So, we've got Both that open source, but also commercial editions where we've got some of the largest enterprises in the world, all the way down to two-person startups using our products to help improve the quality and security of the code they're writing. So that their software is better and more maintainable and has less tech debt.

• 4:38Craig S. Smith?

  Is it a deterministic, rule-based system?

• 4:42Speaker 00

  There's still a good part of the product that is deterministic and rules-based. And we actually think that that is a real plus in a Gen AI world because one thing that that Gen AI and all non-deterministic systems have is by definition, you know, you can't quite predict the outcome that you're going to get exactly. For things like quality control, it is helpful to be able to have repeatability and certainty as you are doing the verification and various analyses. So the history of the company has been in deterministic, rules-based analysis of code. We have been supplementing that recently with a couple of different things.

• 5:26Speaker 00

  One is looking at how AI in general, Gen AI in particular, can be used in some of these gray areas. There are some types of problems that deterministic systems are really good at. And there's others that are a little bit more gray, where you need a little bit more judgment. You need to take into account more context. And we're starting to look at how can we use AI to really supplement what we've historically been doing? So that's certainly a key piece. And then the second is, our historical business has been in finding issues in code, and then it's up to the developer to fix the issues. And what we are using is Gen AI to help suggest fixes for the issues that we uncover.

• 6:09Craig S. Smith?

  Yeah. And you run alongside, I mean, so many people are using Copilot or CodeWhisperer.

• 6:18Speaker 00

  Yes.

• 6:19Craig S. Smith?

  There's another one that I was introduced to yesterday.

• 6:24Craig S. Smith?

  Yeah, do you run alongside them? Because, yeah.

• 6:29Speaker 00

  Yeah, so Sonar Cube can run in the IDE. We have an IDE plugin for a number of the most popular IDEs, the development environments that are out there. And then we run either on premise or in the cloud as you're starting to do your pull requests. And really, the source of the code doesn't matter to us, right? What we look at is the code itself. And so we have, I would say, the majority of our customers now, anyone who's using Copilot as an example, or Codium, or Code Whisperer, you know, more likely than not, they're actually taking the output of Copilot and running it through Sonar Cube. We've actually recently launched

• 7:08Speaker 00

  a specific workflow for this that we call AI Code Assurance, right? That is meant to, we don't see ourselves as a code generator. This is not the business that we're in. We're happy to work with Google and Microsoft and Amazon and Codium and Cody and all of these other players that are out there. What we really see ourselves in is providing the workflow and the assurance that the code that you get out is high-quality code, that it's not going to cause you maintainability issues down the line, that it's not riddled with bugs, and that it's not riddled with security issues. And so we really work in tandem with any of the code generators that are out there.

• 7:49Craig S. Smith?

  Yeah, that's interesting. So you, if someone's using Sonar Cube alongside Codium, for example, which now has its own IDE, you would copy the code from the IDE, put it into Sonar, and Sonar would check it and make fixes, and then you would copy it back into the IDE? Or how does that work?

• 8:19Speaker 00

  Yeah, it's a much more integrated workflow than that. We still, we fundamentally view ourselves as being a developer assist tool, right? For lack of a better way of saying it. So meaning, we want to work really with developers to, to improve their workflow, improve their productivity, you know, cut down on a lot of the nuisance that they have, a lot of the non-value-added work that they do. And so, you know, everyone I've seen is using some level of code review for the code that is being generated by any of the code generators, including Codium. You know, what we see is the industry sort of norm at the moment is

• 9:01Speaker 00

  only 30% of suggestions coming from a Copilot as an example are actually put into, are actually accepted by the developer, right? And so it's, it's clear, I think most people recognize that there are issues in the code that comes out, just like there would be with, with a human, right? And there are different types of issues. In some cases, you don't have as many stylistic issues with, you know, GenAI is pretty good at the spell check types of things, you know, you know, normally put a semicolon where you should have put a colon and that sort of thing. But you do have some more complex issues that sometimes it makes up variable names, sometimes it, it doesn't operate in the context of your codebase correctly, things like that. And so that's where we really, as you say,

• 9:45Speaker 00

  we sort of take the output, run it through our analyses. We do that in the IDE and then a more complex analysis as you're issuing the pull requests themselves. But it's not quite as manual as a copy, paste, copy, paste. We've actually done a lot of work with the GitHubs and GitLabs and, and others of the world to really integrate into the workflows that they have and be kind of seamless for the developer, or as seamless as you can be for the developer.

• 10:10Craig S. Smith?

  So you can run Sonar Cube and Copilot in the same IDE? They don't conflict?

• 10:19Speaker 00

  Correct. They're just different plugins inside of the same IDE. That's right.

• 10:23Craig S. Smith?

  And then as you're writing code or accepting suggestions from Copilot or, or Codium or any of these, the Sonar Cube scans that code immediately? Or do you have to activate it?

• 10:43Speaker 00

  It depends on the type of analysis that you're running, right? Inside of the IDE, we have a product that we have essentially an IDE extension for Stoner Cube that will do, for all intents and purposes, continuous inspection of the code. Now, there are some types of analyses that require just more computation, more time, and more sophisticated analyses of the code, so that doesn't happen, you know, right as you're writing the code, but a lot of the analyses do, right? And our philosophy is, as much as possible, not even, excuse me, there's this expression that's used of shifting left, we actually try and talk about how we start left, right?

• 11:25Speaker 00

  We want to start as early as possible with the developer and catch issues while they're being created. So there is a lot of the analysis that is being run fairly continuously in the background. We do a lot of work to make sure it doesn't slow down, the workflow that it's not distracting to the developer. I try and optimize the number of true positives and minimize the number of false positives, things like that in the IDE. But then really where the sort of next level of analysis happens is when you're issuing the pull request, the merge request, right? And you want to actually put the code into, you know, into your main code base. And that's where a lot of people would do code reviews, and we essentially do the automated

• 12:07Speaker 00

  version of that, so we identify all the issues that we think the code has. We'll tell you how severe we think the issues are. If you click into the issue, we'll try and explain the issue to you as much as possible, like why we think this is an issue. And so there's an education component to it also. And then with our AI code fix for certain issues where we have a high confidence, we will suggest, oh, and if you accepted this piece of code to replace that piece of code, that would fix the issue, right? But it really happens, we try and have it happen as early as possible in the development process in the IDE and at that pull request phase.

• 12:45Craig S. Smith?

  Yeah, and then when you accept a piece of code or a fix, does then it trigger a new analysis to ensure that that hasn't introduced a bug or something?

• 13:00Speaker 00

  Yeah, at the moment, we are, we're doing a lot of pre-training, if you will, of the model. So that we are, the acceptance rates we're seeing are about double. The acceptance rates of your standard code generator, because we have a lot more context than a typical effort to create new code has, right? We know what the issue is, we know what was causing it, we can kind of use that to create the prompt and put it into the L L M's in a specific way. So it's not pre-training exactly, but it really is a higher fidelity way of generating the response. You can absolutely have the option of running the analysis. Again, what we find is that most people at that point then subject to the human code

• 13:45Speaker 00

  review as well, because, you know, all good code reviews have an automated component and a, a kind of expertise component to it. And rather than getting stuck in this do loop that goes over and over and over again, we find that developers just want to see the fix. They look at it, they agree to, to accept it or not, and then they go on with their new process.

• 14:06Craig S. Smith?

  Yeah.

• 14:06Speaker 00

  So you can, but it's not the typical way.

• 14:08Craig S. Smith?

  You know, I'm not a coder, but I've, I've tried coding simple applications with, and it, from my experience, you end up in this endless rabbit hole where, you know, the code doesn't work and then it's the, the, the G N A I says, ah, I see the issue and then it fixes the issue and then it doesn't work. And it's, well, let me try a different approach and it just goes on and on and on.

• 14:46Craig S. Smith?

  Is, is, and certainly with a coder, with a copilot or Codium, they would have the skills and the context of not accept code when they see that it's going to be problematic. But has G N A I, there's so much written about how much code is now being generated by code assistants, but not as much as written about how much bad code is being written by code assistants and whether that.

• 15:27Craig S. Smith?

  counteracts some of the productivity gains of having a coding assistant. I mean, how do you guys view that?

• 15:38Speaker 00

  Yeah, I I think there's actually, there's a couple of different aspects to this. One of them is, you're absolutely right. Gen AI code has bugs, it has stylistic issues, it's got maintainability issues. It it has security issues that are getting introduced. And in many cases, it's actually introducing new vectors on which you can have a problem. Right? There are new ways of of potentially causing security issues inside of your code base using Gen AI that really didn't exist as much. I mean, they kind of exist, but they didn't exist as much in the more traditional world.

• 16:19Speaker 00

  So you do have, Gen AI code is not perfect, and it's not for English language search things either, right? This weekend, I was trying to write something for our board and I said, oh, ChatGPT, give me a bunch of sources. And it turned out it made up three of the sources, right? And thank, thank goodness, I double checked the URLs that it gave me, right? So, you know, and it's a good lesson learned that, you know, the, the, the way the math works on the Gen AI systems is it is going to have hallucinations. They're a feature, not a bug, right? In many cases, and that does have real world implications on your code base, right? And on your security and on the reliability of the software that you're shipping.

• 17:02Speaker 00

  But I'd say that's a known problem. It's mainly not a fully appreciated problem, but it's a known problem. You can look at the research. There's a lot of research that's been published recently that talks about this issue. The part that we think is as big a problem and maybe a bigger problem is that while people kind of, if you push them on it, they understand that there's issues, there is this accountability crisis. And I really think crisis is, is starting to be the real word here of who's actually responsible for the AI generated code, right? And it's Very easy to say, oh, the developer that's using the code generator is, but it's almost human nature.

• 17:42Speaker 00

  You don't hire a software developer. They didn't go to school. They didn't learn how to write code to become a copy editor for AI.

• 17:48Craig S. Smith?

  Right.

• 17:49Speaker 00

  And what, what one of my customers, large financial institution, told us a story. A couple of months ago now, they were starting to have a minor outage a week because of AI code generated. And it's not because the systems are flawed. The systems are actually quite good. They're just not perfect. Right. And they, they have this one, one example of they figured out there was this line of code that was buggy. They traced it back to the developer who checked it in. They went to the developer and said, hey, so what happened here? Let's do the postmortem. And the response was, well, that's not my code. What do you mean you checked it in? Well, yeah, but the AI gave me the code and I just accepted it.

• 18:30Speaker 00

  Right. And, and I'm not at all saying that developers are being lazy, not doing their job, et cetera. It's that we're asking them to do a job that really is not what they signed up for, when that, what they're trained to do. And, and humans, unless this is your skill set, you're not trained to sit there and look at hundreds of lines of code that are being written by AI and double checking it. Right? And so this is why we think having some automated checks tools like Sonar Cube, this AI assurance workflow we've put in place is so important because you have this accountability crisis. And the solution to that is to make sure you've got really robust workflows and you're doing rigorous code reviews. Right. And by the way, most developers who are listening would say, duh, of course, this is what you should do.

• 19:15Speaker 00

  This is like best practice in software development in general. And that's true, but I think there's been this misperception in a way of this will be easy in the AI world. And I actually am finding that it is substantially harder for the reasons that I said just a minute ago.

• 19:31Craig S. Smith?

  Yeah. And, and in that case, that code would have gone through a code review after that developer

• 19:40Craig S. Smith?

  submitted it. Right? So it was also missed by the code review.

• 19:45Speaker 00

  Yeah, and code reviews will miss things as well because, you know, we're only human and I guess at this point we're all also only Gen AI, right? And and there's going to be bugs in the system, if you will. I think part of the question is, how do you have? We we find that organizations that employ rigorous quality gates, which is something our product offers, which means that you just can't go forward until you resolve these issues, have substantially better reliability and security and developer productivity, actually, than people who don't. And, you know, because there's a lot of judgment, it comes into code reviews. Not not only is this an issue, but is it an issue that needs to stop production, right? I mean, this is the key thing that you're always, there's always a trade off.

• 20:30Speaker 00

  There's always a time pressure. Part of the reason we want to shift left or start left as much as possible is because when you're at the point where the product's supposed to ship, you know, in two hours and you just find a bug, now you're the person holding this up, right? This is not a position anyone wants to be in, and you cut corners knowingly or not knowingly. I don't think anyone would describe it as cutting corners, but you do, because that's just life, right? That is the way, that's the way time pressures and things like that operate. So the earlier you can detect these issues, the more likely you are to fix them and the cheaper it is to fix them as well.

• 21:05Craig S. Smith?

  Yeah, and and that anecdote also points to something else that I've been talking to people about, which is automation bias, that if a coder is using a code assistant, coding assistant, and and it's it's generating good code for, you know, fifty fifty times, and you just get lazy and just start accepting it without without worrying about it. And that's when when bad code slips through. So you're adding a Gen AI component, but it sounds like.

• 21:52Craig S. Smith?

  What you really want in this kind of a product is a, you know, deterministic system. You don't want to introduce another layer of probability.

• 22:06Speaker 00

  Well, I, I think it, you know, there are certain types of problems where the deterministic model works really well. And if, if it ain't broke, don't fix it, right? And there's other areas where it doesn't work that well. And these are problems that have not been addressable in the past. And so I think those are areas where new techniques are really good. And so our approach is very pragmatic, almost transactional, you know, if this problem can be solved deterministically, we like the predictability of it, we like the cost of it, we like the efficiency of it. There's a lot to like about that, right? If it, if the quality can be improved, or if it can't be solved with deterministic models, then how do we supplement those models with other approaches?

• 22:49Speaker 00

  And it could be GNI, it could be other, you know, more probabilistic analysis, analytical approaches as well. I think fundamentally, I believe that a lot of these systems are going to end up being a hybrid of deterministic and non-deterministic approaches that are brought together, you know, where the use determines which, where the, the problem itself determines what type of solution is being used.

• 23:16Craig S. Smith?

  Yeah. And, and actually that, why not then build out a coding assistant on top of Sonar Cube? Or why wouldn't Copilot or Codium integrate something like Sonar Cube into their system so that they're, they're not two different systems? It's, it's all one thing.

• 23:45Speaker 00

  Yeah, I mean, so to tackle the second part first, and then I'll talk about the first part. You know, we view GitHub and GitLab and Bitbucket and all of these different players as really good partners. We do a lot to integrate with them, right? There are certain things that we're really good at. And we've got, you know, over a decade of experience with, we've got deep IP and we, you know, understand the nuances of this sort of code analysis, particularly for quality and security, we think as well or better than anybody out there, right? And so to us, the question is, absolutely, if we can integrate this so

• 24:25Speaker 00

  it is completely seamless for a Codium user, or for a Copilot user, or a Gemini user, or whatever it is, that we would love to do that. And that is what we are doing. So that is very much a part of the strategy. We think that there's some pieces of IP and context that we have that's pretty unique, that helps you, as I mentioned, solve some of these problems early. So you can throw brute force analytics at certain problems, right? But in some cases, just really understanding exactly what's causing the issue and having that sort of full set of know-how helps you get to the answer faster than just the brute

• 25:06Speaker 00

  force piece, right? And so that's why we find that our code fix suggestions have a 2X higher acceptance rate than your sort of generic code-generated fixes, right? It's because we can supply it with that much more details of the analytics of your code base and things like that. So, and, you know, by the way, the same thing holds true on the first part, you know, there is very little inside of Sonar that makes us

• 25:37Speaker 00

  Likely to be the best at generating code from scratch, right? We're not good at financial, at foundation models, right? And none of these players are building their own foundation models, to my understanding. Maybe Microsoft is, right? But we're not good at foundation models. We don't have proprietary data sets, you know, for code model training. Most of them are trained on the same code bases of open source and competitive coding, you know, competitions and things like that. So it really comes down to, it's, I think, a different problem than what we're trying to solve. What we believe we're really good at is taking a billion lines of code,

• 26:20Speaker 00

  which is the size of a lot of, the code base of a lot of our customers, and really decomposing it, deconstructing it, a better word, right? And figuring out, how do we make sense of it so we can find these problems? And that's a different, we think equally valuable, but different problem than creating code from scratch.

• 26:37Craig S. Smith?

  Yeah, and so you, you not only have kind of a live code review as a coder is, a programmer is writing code. You can apply this to an entire code base retroactively, is that right?

• 26:57Speaker 00

  Yeah, you can, this is not just for your new code that's being written, right? Most of our customers apply this, they kind of manage their code using SonarCube, right? And we've got some, some customers who do this on two billion lines of code, right? And then there's others who do it on 10 million lines of code or 100,000 lines of code. So it really depends on the size of your code base. But as you think about the challenge a code generation has, you know, fitting into the code base of a

• 27:32Speaker 00

  Large health insurer, something like that. You need to not only understand how does Java work, right? And how do you solve this one line problem that you're trying to solve? You have to figure out, okay, what's the structure of the code? What's the architecture been? What's the, what's the syntax you use? What's the style that you use? Things like this, right? That are, that are kind of unique to every customer. And so that's one of the challenges that I think a lot of the code generation companies are trying to get at as they launch fine tuning and things like that. So you can fine tune on your code base and make it that much more tailored for you. We think that's super important because otherwise you're going to end up with a hobbyist tool for the, or an autocomplete, right? Like, you know, you'll end up with autocomplete or a hobbyist tool.

• 28:17Speaker 00

  Tailoring to your specific code base is really important. And then analyzing the code that's generated in the context of your specific code base is really important. Right? So in part to analyze the new code well, we have to understand all of your code base, right? Because that's what determines if it's a good fit. It's not just a circle, but it's the right size circle to fit into your code base, right? And, and you can use it to reduce your tech debt that you've got because it'll show you, here's all the issues that you have. And you can slowly work your way through those issues as well.

• 28:49Craig S. Smith?

  Yeah. How long does it take to review a billion lines of code? This is something I've never really understood and

• 29:00Craig S. Smith?

  maybe for, for you guys, it seems obvious, but do you, do you give, well, anyway, tell me how you do it.

• 29:11Speaker 00

  I mean, it's, without getting into all of the sort of math behind it, you know, you've got your, your repos that holds your code, right? You've got your different branches and sub branches and all of that. And essentially to make it fast, if you just tried brute force to get your way through a billion lines of code, it's going to take you a long time, right? And so part of what, part of what we do, part of what others in the space do is, is think about, how do you deconstruct the code, right? Into pieces that are analyzable, then how do you add it all back up together? For lack of a better way of saying it. And that makes the analytical problem much more.

• 29:49Speaker 00

  trackedable, right? In most cases, it's not, by the way, one, the right way to say it's not one piece of software that has a billion lines of code. It's in, you know, thousands of different software programs in, you know, you have your, you have your online banking system, then you have your core banking system, then you've got your risk management system or whatever. So there's kind of natural structure to the code base that we use and, and you analyze it, you know, both in parallel and sequentially in different ways. You know, one of our offerings is we would call it data center edition and part of the value of the data center edition is

• 30:30Speaker 00

  high availability because you have multiple instances of Sonar Cube working in tandem, but it's also high performance because you can actually break down your code base into, you can essentially apply more and more instances, right? So analyze more in parallel, which makes things faster and higher performance.

• 30:50Craig S. Smith?

  Yeah. And, and so

• 30:58Craig S. Smith?

  in, in that review, when it, when it lists all these various issues, does Sonar Cube offer, as you were saying with the kind of live coding, does it offer fixes for every issue or is it just flagging them?

• 31:19Speaker 00

  So it is certainly flagging them and it will provide you with metadata around them in most cases, right? So the simplest example is do we think it's high medium or low severity, right? So what's a criticality? We launched this thing we call the Clean Code Taxonomy, gosh, nine months ago, maybe 10 months ago, which is really a taxonomy for describing code problems, right? So is this a responsibility problem, right? And what does responsible coding mean? It would be things like, you know, is it, do we think it's inefficient from a power consumption standpoint, things like that? Or is it a maintainability issue? Whatever. So we provide a lot of Metadata around this, so you can categorize the issues.

• 32:03Speaker 00

  There are some companies we've worked with who've said, no issues should move forward. Right. And they take a very, very strict approach. There's others who just say, kind of understandably, no high severity issues can move forward, but low and medium severity is fine, right? And this is where that workflow element of what we do is so important because you can define your quality profile. What is good for you as a company? And by the way, it may vary, like to the example I was using earlier, your wire transfer system, if you're a bank, that thing can't go down. You have to notify regulators if it goes down. You lose millions of dollars if it goes down. You're probably going to have pretty strict quality gates for that, right?

• 32:41Speaker 00

  For your mobile app that takes care of your internal knowledge base for employees, probably okay if it goes down every once in a while. At least the regulators don't get involved, right? Things like that. So you may set your quality profiles and quality gates differently, right? So, so part of it is we are giving you a huge amount of data about what is the type of problems that you have. How would we help you prioritize it essentially? And then for certain types of problems where we think a fix can be generated, and this is like a really important point for us. We will suggest the fix, but we have to believe

• 33:26Speaker 00

  that we're not just giving noise to the developer, right? We pride ourselves on high signal, low noise ratios, right? And there's a lot of tools out there that are just, will suggest a fix for everything, right? And imagine, to use a sort of non-developer example, imagine if you were in Google Docs and it kept suggesting to you a whole bunch of grammar fixes that are not actual grammar fixes, right? That are actual mistakes, pretty soon you would tune it out, right? The developer would go, oh, this is an overwhelming list of things I need to look at. Everything has the red squiggle underneath it. I don't want to deal with it, right? So we put a huge amount of effort, and we're not perfect by any means, right? Put a huge amount of effort into how do we try and give confidence

• 34:11Speaker 00

  that if we identify an issue, you're going to think it's a real issue, right? Otherwise, and we see this happen in a lot of cases in our industry. There's a long list of stuff that gets identified and nobody ever deals with it.

• 34:24Craig S. Smith?

  Right.

• 34:24Speaker 00

  It just sits there, right? Because you tune it out and developers think this is useless. And that's the negative spiral we're trying to avoid.

• 34:32Craig S. Smith?

  That's, that's really interesting. And, and one of the things that fascinates me about as software piles up or eats the world is Marc Andreessen famously said that it's be, and particularly with, with generated code in the code bases that we're reaching a point, or, or maybe we reached a long time ago where nobody really understands an entire code base.

• 35:08Craig S. Smith?

  It's just too much and too big.

• 35:12Craig S. Smith?

  And so you need tools to explain the code base to you and, and identify, in this case,

• 35:22Craig S. Smith?

  bugs or, or errors.

• 35:25Speaker 00

  Yeah.

• 35:26Craig S. Smith?

  But, but you also have to trust that system that it's, that it's finding everything or explaining everything. I mean, are we getting to the point where we're

• 35:39Craig S. Smith?

  kind of beholden to AI

• 35:44Craig S. Smith?

  to, to, to manage our, our software infrastructure?

• 35:49Speaker 00

  I, I think the future is sort of

• 35:53Speaker 00

  a, a mosaic of different approaches, right? Is, is right. I don't think this is a world in which Gen AI is going to be the be all and end all, right? And I'll use the example that you gave. Maybe Gen AI can help with design and architecture work, but fundamentally, somebody has to sit there and think about how is the health insurance system going to work? What are the design and architecture rules that you're going to put in place? Well, how are you going to construct this billion line code base? How are you going to, to maintain and migrate it? How, when you're going to move part of it into the cloud? How do you take pieces of it and do it intelligently, right? Like that, I don't want to say AI can never help with that because there's already AI tools that try to help with that. But these are real hard problems that I think require.

• 36:40Speaker 00

  We've required deep understanding and focus effort, right? And I think those are areas where there's judgment calls that are going to need to be made, not just the analytics, right? And I think this is an area where we see that, you know, human developers are going to spend a lot more time obsessing about design and architecture of their software system because only when you've got really well thought through design and architecture can the Gen AI be really effective at building the code that you need for that, right? Otherwise, you end up with a spaghetti mess, right? And it's just now we even we acquired a company recently called Structure 101 that helps us do the type of analyses that we have historically done, but on design and architecture points as well. So you can define this is what your architecture looks like.

• 37:23Speaker 00

  And we can tell you when is this part of the code base violating your architecture, things like this, right? I think that there's software development is made up of thousands and thousands of problems, right? In order to create a code base and simply, I don't want to trivialize it, but simply writing the code, right, is a pretty small part of it. Again, you think about writing a novel, you have to have your storyline, you need to know who the characters are, et cetera. You can't just put in, write me a book, right? And I mean, I'm sure to write a book, it probably will not send the wire transfer that you need to have sent, right? If it does that. And so I think a lot of the work that's going to be required here is do the design

• 38:07Speaker 00

  work, do the architecture, really be a partner, have the AI be a partner with you in constructing this all together. And you're exactly right. The the size and complexity of most, let's call it global 2000 code bases is extraordinary. And it's only going to get more complex because Gen AI is going to write more code and humans are going to write more code and companies are going to run more and more on software. And so really, this is why we think thinking through these workflows is so important, right? Because you have to have the governance and governance sounds really boring. Most people don't want to talk about governance, et cetera.

• 38:47Speaker 00

  But you really have to have really well documented governance and controls in place so this doesn't get out of control.

• 38:56Craig S. Smith?

  Yeah. How much is out of control, do you think, in the global 2000 code base? I did a wrote a piece for.

• 39:06Craig S. Smith?

  Somebody somewhere in the last five or six years about Cobal about uh and it's kind of like there's there there's there are these uh code bases that are massive that are running and working and people kind of tinker around the edges, but no one really

• 39:27Craig S. Smith?

  understands the entire code base, but since it's working, you just leave it alone. I mean, I I wonder how much of that exists out there.

• 39:38Speaker 00

  And it was, yeah, we Cobal is one of the languages we support actually up to thirty um and and there's a decent number of companies who really care about Cobal because, you know, it is harder and harder to find really good Cobal developers whereas they have to make changes to that code base. They um they they need, you know, the quality assurance and and um things like that that that we provide. Um, you know, this is, I I think this is an evergreen area, right? It, you know, code bases, not to sound too metaphysical about it, but are living systems, right? I mean, you're you're changing your code base. The stat that I think I've seen is that twenty percent of any code base changes every year, right? Um, so, you know, you think about that if you're

• 40:21Speaker 00

  if you're a large bank, you may have a billion lines of code, but you've got twenty thousand developers, thirty thousand developers, right? So they're actually changing that code base. And I think the twenty percent sounds probably about right, but inside of that code base, there's a twenty percent that changes frequently. And then there is your code that, hey, it's working, nobody's ever touched it. And it's the it doesn't pencil uh from an ROI standpoint to try and refactor it again to all these conversations about why don't you rewrite your Cobal and and Java or something like that? And the answer is in part what I said earlier, like if it ain't broke, don't fix it, right? Um, so I think every large

• 41:01Speaker 00

  Global 2000 company has got a large amount of tech debt, right? And part of the approach that we advocate for is to make sure you've got a rigorous process for remediating tech debt as you're creating new code, right? Because I don't think you need to do a, you know, gut renovation of your code base, right? You don't need to say, all right, stop everything. I've got all this tech debt. I'm going to go and clean it all up and throw it all out. Rewrite it. It's just not practical for people to do that. However, saying, hey, we're going to make adjustments as we see the opportunity to do so. I'm touching this piece of code. Oh, I can make this better, this better and this better in addition to the new thing I'm writing. That is something we think people should be doing.

• 41:46Speaker 00

  So continuous maintenance of your code base. And it's one of the use cases you've got for Sonar Cube is that, you know, as you are, as you are working on a particular piece of your code, it'll show you not just the issues, as you mentioned earlier, with your new code. It'll show you the issues with your existing code as well, right? And it gives you the opportunity to kind of continuously improve your code base. But you're right, it is, it is kind of, it's a mess of organic growth and evolution over, over many, many years. I don't think this was an intelligent design problem. I think in most cases, this was an evolutionary process with all the mess that goes with that.

• 42:29Craig S. Smith?

  Yeah. If, if you point Sonar at any Global 2000 code base, let's say a billion lines of code, is there, are there metrics around how many issues are typically found? And how many of those issues are critical or, or, you know, some, some high urgency?

• 43:01Speaker 00

  You know, I don't have, most of the work, because most of what we do is on premise, right? So we give you the software, you run it yourself. We don't have a lot of the telemetry like that. The best proxy we have for this is on our cloud offering. So Sonar Cube, our SaaS offering for Sonar Cube,

• 43:24Speaker 00

  which you can find at sonarcloud.io. We actually let you analyze open source projects for free, basically, right? So anyone can put their open source project in there and we will analyze it and show you all the issues. And it's a fascinating thing to look at, you know, packages or open source projects that people use every day. We can show you, are these an A-grade package from a reliability and a security standpoint? Or do they have different issues? And it really is, it's kind of all over the map. I wouldn't tell you there's like a typical pattern that you see as you look at these open source packages. And I think it gets to developers and developer teams are different. Some of them have really cared about stylistic consistency, right?

• 44:06Speaker 00

  And I know a couple of companies that are just militant. You join the company and you will go through a month of training on their style guide, right? Maybe a month is too much, but you know, you will be rigorously indoctrinated in how they write code, right? There's others where it's the Wild West, right? And as long as you get your, as long as it works, it's fine and we'll worry about it later. And so you really see the personality, if you will, of different companies in their code bases. But as a result, there's no one type of problem, which actually means that you need to really understand the character of your development organization.

• 44:46Craig S. Smith?

  Yeah, and presumably you guys use Sonar Cube on your own code base.

• 44:53Speaker 00

  We do, and we find issues with our own code base as well, as you'd imagine, right? And so we, well, we use the most stringent version of Sonar Cube, of the workflow I talked about, we call it clean-as-you-code, right? Where we sort of insist that all issues are resolved before you go through with the pull request, right? So this is, I'll call it a highly opinionated workflow where we've got this super strict quality gate we've set up, and that's what all of our developers have to go through.

• 45:25Speaker 00

  It's good. It makes them understand, you know, you kind of want to eat your own dog food is the expression, right? It lets us do that. And I think historically, our developers, because they've been users, they've had a lot of empathy for

• 45:41Speaker 00

  the people we're building for, whether those are open source developers in the open source community, or whether that is a developer at a large financial institution, a retailer, or whatever.

• 45:50Craig S. Smith?

  Yeah, and when you say our developers, I mean, Sonar Cube is... What part of Sonar is open source?

• 46:02Speaker 00

  Um, so the major, so we have what we call a community edition

• 46:10Speaker 00

  version of Sonar Cube. It's fully functional, you can download it, you can use it, it only tackles certain languages and certain types of problems, right? But it is, it is widely used, right? Mostly focuses on, call it stylistic and maintainability and those sorts of issues, doesn't get into security as much as an example. So, and that is open source.

• 46:36Speaker 00

  Despite the fact it's open source, it's sort of a strange open source model that the majority, actually, the entirety of the contributor base, almost, we have one or two small exceptions to this, are Sonar developers, right? So we don't have a lot of outside contributors. For us, open source is very much a, it gets back to what you were saying earlier. We want you to understand how the machine works, right? We want you to understand what the rules are, because you need to have trust in the system, right? And so for us, open source has not been a development philosophy, although if people want to contribute, we are...

• 47:11Speaker 00

  Excited to have them, right? It's just not been the motion that's been established. For us, open source is about providing transparency, visibility, and hopefully engendering confidence within the user base that they understand how the machine is working. Right? And, and this is something we're very committed to. Um, moving forward is that we are going to be providing that level of visibility and kind of clarity as much as possible for our community.

• 47:40Craig S. Smith?

  Yeah. Uh, the, um,

• 47:46Craig S. Smith?

  we're coming down to the last 10 minutes or so. Um, what have, what's new, what's on the horizon? And is there anything I haven't touched on that you want to talk about?

• 48:00Speaker 00

  So first of all, it's been a great conversation and we've covered lots of different topics. I really appreciate, um, the curiosity built into the dialogue here. You know, our historical, the kind of legacy of the company has been on this issue identification. We think we're really good at it. We think we're the de facto standard at the quality analysis of code or code quality analysis. And this is an area we're going to keep investing in to be the leader. We've got this orchestration, for lack of a better word, this workflow element that more and more companies find to be really valuable because you can control not just for quality but for things like unit test coverage and anything that you want to put in place

• 48:45Speaker 00

  as either guidelines or, or prescriptive measures for your development team. You can kind of do in a very organic way, right? And so that's really the, the kind of history of Sonar. And it's an area that we're going to keep investing in. How do we get better at security? How do we get better at new languages as they come out? Things like this. The part that we think is additive to that and really exciting. And we're just in the first innings of is the, how do you really fix the issues that are being identified? You know, every Every CISO I talk to right now is overwhelmed by lists of vulnerabilities, lists of issues that are coming out, right?

• 49:27Speaker 00

  And the question that they're all asking is how can we help them triage these lists better? To my earlier point, not everything's a real issue and not everything has to be addressed. But today, they really don't know how to do it, how to triage well. They don't have a lot of tools to help them with that. And then where we can help with fixes, let's help with fixes, right? I think Stripe put out a survey a couple of years ago that estimated that 50% of a developer's time was on stuff that they would describe as toil, right? Just like, no one really, you know, it's debugging, it's documentation. It's necessary. I don't want to say it's not necessary. You can't just stop doing it, but it is not what anyone enjoys. It's not, quote unquote, productive, right?

• 50:12Speaker 00

  It's in the lean sense, it's sort of like waste in a way. And I think the more we can start helping developers and be more effective by cutting out the toil, the better. A lot of it gets into that debugging area, the code context, issues that you were talking about, things like that. So we think there's a big role for us to play with AI as that continues to evolve.

• 50:35Craig S. Smith?

  Oh, I was gonna ask about the cloud offering. So what's the, how much of your business is on-prem? And how much is through the cloud? And what's sort of the profile of people that are using the cloud, the SaaS offering?

• 50:54Speaker 00

  If you went back three, four years ago, I think a lot of people still manage their code bases on-prem or, you know, it's very sensitive data and people were just a little hesitant on this. I'm not exactly sure. Like, having come from Google Cloud, I found it a little surprising, to be honest, when I arrived, how little of our customers, how few of our customers were actually asking for a cloud solution. Now, this is changing and we just launched our enterprise-grade SaaS offering. And this was in August of this year, right? So pretty new. The demand has actually been through the roof now, which is, which is great to see. I think there's a lot of people who want to be out of the managing your own software business.

• 51:38Speaker 00

  And, you know, it sounds like we're having this conversation in a way in 2018, right? So this is not new news, but for us, this is something that people had not. Had their developer tool chain, I think, operating this way now there's more and more demand for it. So for our new business, we see it at, you know, 30% and growing, moving forward. But historically, it's not been the dominant use case for us.

• 52:03Craig S. Smith?

  AI might be the most important new computer technology ever. It's storming every industry and literally billions of dollars are being invested. So buckle up. The problem is that AI needs a lot of speed and processing power. So how do you compete without costs spiraling out of control? It's time to upgrade to the next generation of the cloud, Oracle Cloud Infrastructure, or OCI. OCI is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds,

• 52:42Craig S. Smith?

  offers one consistent price instead of variable regional pricing, and of course, nobody does data better than Oracle. So now you can train your AI models at twice the speed and less than half the cost of other clouds. If you want to do more and spend less, like Uber, Eight by Eight, and Databricks, Mosaic, take a free test drive of OCI at oracle.com/eyeonai. That's E Y E O N A I, all run together, oracle.com/eyeonai. That's oracle.com/eyeonai.