Pod Insights Training Data Snowflake CEO Sridhar Ramaswamy on Using Data to Create Simple, Reliable AI for Businesses About this episode All of us as consumers have felt the magic of ChatGPT—but also the occasional errors and hallucinations that make off-the-shelf language models problematic for business use cases with no tolerance for errors. Case in point: A model deployed to help create a summary for this episode stated that Sridhar Ramaswamy previously led PyTorch at Meta. He did not. He spent years running Google’s ads business and now serves as CEO of Snowflake, which he describes as the data cloud for the AI era.
Ramaswamy discusses how smart systems design helped Snowflake create reliable "talk-to-your-data" applications with over 90% accuracy, compared to around 45% for out-of-the-box solutions using off the shelf LLMs. He describes Snowflake's commitment to making reliable AI simple for their customers, turning complex software engineering projects into straightforward tasks.
Finally, he stresses that even as frontier models progress, there is significant value to be unlocked from current models by applying them more effectively across various domains.
Hosted by: Sonya Huang and Pat Grady, Sequoia Capital
Mentioned in this episode:
Cortex Analyst: Snowflake’s talk-to-your-data API
Document AI: Snowflake feature that extracts in structured information from documents
Transcript 10,137 words · 3 speakers
Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 The product that makes even the people that go, I have GPT-4, I have an army of software engineers. The thing that even they struggle with is things like a reliable talk to your data application. Because even with GPT-4 out of the box, you end up getting forty-five odd percent reliability, meaning it gets half the questions wrong when it tries to answer it. We are well in the nineties and we are racing to get to like ninety-nine percent reliability on talk to your data applications. Obviously, we restrict the domain and turn this into more of a software engineering problem than just like a pure AI model problem. But that's the thing that makes every Snowflake customer perk up and go like, I want that.
0:44 Speaker 00 Because even the people with the money and the resources to spend on software engineering teams very quickly realize that this is a wall that they are likely not going to break through.
1:11 Pat Grady? Today, we're excited to welcome Sridhar Ramaswamy, CEO of Snowflake. Snowflake is one of the most important enterprise companies in the public markets. It's the default cloud data platform. But today, the question of what role does Snowflake have to play in the world of AI looms large. Sridhar is somebody we've known for a couple decades. He actually started on the very same day as our partner, Bill Corn, at Google back in April of 2003. We backed Sridhar in his own startup, Neva, which was an AI-driven search engine. Snowflake acquired Neva, which is how Sridhar became the successor to Frank Slutman. Rarely have we encountered somebody who is as in the weeds on the technology, but also
1:56 Pat Grady? as commercially savvy as Sridhar. And he will join us today to talk about what AI means for Snowflake, the importance of safety nets, the open source community, the competitive landscape, and the practical applications of AI that he's seeing in the enterprise through his lens as CEO of Snowflake. We hope you enjoy.
2:17 Pat Grady? All right, Shridhar, we're excited to have you here with us today. You're a technologist by trade. You've spent a lot of time in the consumer world and you are now at the helm of one of the most important enterprise companies of our generation. So before we jump in, we have a lot that we want to know about enterprise AI, what Snowflake is up to, some of your predictions on the world of AI. Before we jump in, though, just a level set. Can you give us a couple of words on your personal background? And then just for people who aren't familiar, which is probably not a lot, but just for fun, for people who aren't familiar, what's Snowflake? So who's Shridhar, what's Snowflake? Let's start there.
2:52 Speaker 00 That's great. Pat Sonia, super excited to be here at iconic Sequoia, home to many, many legends I admire. Yeah, I'm a computer scientist by training. Early career as an academic. I joke to people that I'm a reformed academic because I was like, I wanted to do things with more impact. Super lucky to be an early part of Google, where I joined one of the greatest businesses ever invented by humanity, which is the search ads business. I ran that for close to a decade, all of ads and commerce at Google. For five years, helped grow that business from a billion and a half to over $120 billion in revenue.
3:36 Speaker 00 And then funded by Sequoia, did an ambitious startup called Neva, which wanted to modestly rethink what search meant before getting acquired by Snowflake and becoming its CEO. And Snowflake is the AI data cloud. Our core thesis is that a cloud computing platform that puts data at its center is going to be way better for enterprise customers to act on data than a generic cloud.
4:08 Speaker 00 And AI, of course, we think of as a transformational technology that is going to change every aspect of how data is stored, how it gets moved around, and of course, how it's accessed. We have over 10,000 customers, made 2.6 billion last year, but at the center of everything, enterprise and data. That's a super quick blurb.
4:30 Pat Grady? Perfect, thank you. And so you have 10,000 or so customers. I know you've met at least 100, probably hundreds of them since you took over.
4:37 Speaker 00 I've met hundreds of them by now.
4:38 Pat Grady? So there you go. So I'm guessing you have a pretty decent read on what's going on in the world of enterprise AI. So maybe we'll just start there. What's going on in the world of enterprise AI? What are you seeing at your customers?
4:49 Speaker 00 First of all, people get that this is going to be transformational. You know, lots of technologies have skeptics. I'm sure you've run into folks who are all like, ah, mobile, it's not going to be a thing. This browser, like, so lame. It takes a while for people to absorb. I think what's different about AI, first and foremost, is people are like, I get what this can do. I think some of the power is just like honestly looking at the magic that ChatGPT is. Anyone that like has interacted with it, asked it to write a poem, asked it to create an image, knows like, wow, this is something that's very special. So the level of awareness is incredibly high. And we have thousands of customers
5:33 Speaker 00 that are in various stages of implementing AI solutions. They span the gamut from people like Bayer that are very excited by the idea of giving business users access to business data without going through like an elaborate, you need an analyst, you need a BI tool, you need blah, blah, blah. You need a week before a change can be made. They're like, I just want to put data into the hands of people that need it right now. But we also have dozens of people that are using AI as a transformation engine. So for example, if you have unstructured data, whether it's an image or let's say like a transcript, previously you had to run a software engineering project to figure it out. What's this image about?
6:15 Speaker 00 Now you feed it into a model, ask it a question, and you get the answer. And so people are super excited by things like that. We have a product called Document AI, which extracts structured information from documents, say like contracts. All of us have contracts sitting around in our company folders that have all kinds of magic numbers that ideally you want to do analysis on. So there's a wide variety of cases that people are implementing and sending into production. But I would say stuff at the bottom, which is how do you transform data more effectively, more flexibly? And stuff at the top, which is how do you make data easily available to all kinds of business users in new ways, in interactive ways? I would say that's the sandwich in terms of what are people wanting to do with data.
7:00 Pat Grady? Uh huh. And can you say a couple of words on Snowflake's right to win? So some of the things you mentioned, like data transformation, for example, feels like that is very close to the core business of Snowflake. But then there are some things that are maybe a bit further afield. You know, if somebody wants to deploy an enterprise agent of some sort, they can use Snowflake to do it. But what's Snowflake's right to win in that situation? Can you just say a couple of words about how Snowflake fits into this overall landscape and sort of the right to win?
7:25 Speaker 00 Um, so first and foremost, the basic approach that we took to AI, sort of enabling or infusing AI into Snowflake, is it should be an axelren for everything that you do with Snowflake. That's what Cortex AI is. It's a model garden, but it's more than that. Snowflake prides itself on super tight integration of its various product features. And this is not another service that's part of Snowflake. It's built into Snowflake. This means that any analyst that has access to SQL has access to AI. And so it's a massive democratizing mechanism. And then the early applications that we have built,
8:10 Speaker 00 like Document AI, are a very natural next in the progression of what people want to do, which is, hey, I want to act on the data that is within Snowflake's purview. By both expanding the data that Snowflake has. Has access to via things like Iceberg, which is basically an interoperable storage format for cloud storage. But then providing things like Document AI, we just make a whole bunch of AI applications that previously used to be software engineering projects into two commands that an analyst can issue. And so our first lens very much is that
8:48 Speaker 00 AI should become easy, trivial for data that is sitting in Snowflake. 100%. There are going to be applications that are, you know, cutting edge or going to involve many, many different services. But the angle that we bring to all of those customers is we make reliable AI. And this is a topic that we can get into. So for example, I tell people you have no business believing the raw output of a language model for anything. You can't actually do any business with that because it's ungrounded. It doesn't understand truth from falsehood, doesn't understand authority. So we make things like, you know, creating a grounded chatbot. Again, as I said, two commands, not a software engineering project.
9:32 Speaker 00 Similarly, with Cortex Analyst, which is our Talk to Your Data API, we bring the full power off. We know everything about the schema, all the queries that have been run on the schema, the semantic context on the schema, we can produce a reliable, you know, application that others are going to struggle to create. So we are leveraging our strengths in data to make AI products better. Are there going to be like specialist applications that can only be done with GPT photo and a custom integration with a bunch of other stuff? Absolutely. But that's not what we are after. The bulk of our customers want to get work done. They're not in the business of doing research with AI.
10:13 Speaker 02 And are you seeing customers bring net new data that maybe didn't sit inside Snowflake historically into Snowflake because of your AI services? And how do you think about your right to win as it comes to the data that's not in Snowflake yet?
10:24 Speaker 00 This is a broader question. I think one of the things that I've actually been a good part of is in expanding the lens of data that Snowflake should play in. Snowflake, as you know, is first of all, it's closed source software for the most part. The code engine is closed source, just like search. But we also had a proprietary storage format where data was ingested into Snowflake and kept in this format. But what we consistently heard from customers, and I'm sure like you hear all the time, is there is a hundred or a thousand times more data sitting in cloud storage than there is inside a specialized player like Snowflake.
11:06 Speaker 00 And more and more, industry trends have been towards interoperable data. People want their data to be accessible from multiple places. So for example, if they want to write their own bespoke applications, most people don't want to do that, but the biggest ones do. They want their data to sit in cloud storage, where yes, Snowflake perhaps can write it and read it, but other applications should also be able to read it. So we made a big push around iceberg, which is the interoperable format. We also announced a cloud catalog recently. The idea is that in 10 years, data is going to be sitting mostly in the cloud, mostly in cloud storage, which is very cheap, mostly in interoperable formats, accessible via open catalogs.
11:51 Speaker 00 And this is the place where we see there being so much more access to data from Snowflake. So everything from data engineering and AI now comes into our purview. We have customers that, for example, are doing things like, oh, let's run a video model using Snowflake's container services on data that is sitting in S3, extract transcripts and stick it into Snowflake. So it's just a very different world we are playing in.
12:16 Speaker 02 Makes sense. And then sort of, let's say for the data that's currently sitting in one of the hyperscalers, for example. You started the conversation by saying, you know, the core tenet of the company is that when you build your infrastructure kind of all around the processing of data, you can do better things. What are some of the ways that you're able to kind of offer better AI services around the data that doesn't currently sit in Snowflake, but that you're hoping customers will bring in versus, you know, what the hyperscalers are doing already?
12:41 Pat Grady? Yeah, and can I add onto that real quick? Because one of the things that we have heard from customers is at either end of the spectrum, you've got, at one end of the spectrum, work directly with OpenAI, send your data, you know, into their cloud, and maybe have some nervousness around whether that data is going to leak into the model, or whether they have the right security and privacy sort of governance around it. At the other end of the spectrum, you can just do everything yourself, grab a model off the hugging face, build it internally, you know, super safe, super secure, but pretty painful to do all that. And then the middle ground, you've got Amazon Bedrock, or you've got a Snowflake. And they both kind of have a value prop of best of both worlds.
13:23 Pat Grady? We're going to make it easy for you, but it's also safe and trusted and secure and all that good stuff. And so I think my angle on Sonia's question is like, for somebody who's making a practical decision about sort of what should I build in Snowflake versus what should I build on Bedrock or a comparable cloud service? What leans people in the direction of Snowflake?
13:43 Speaker 00 Um, it's the fact that everything that you want, whether it is data security, data governance, ease of use, all come out of the box. The incredible power that comes with Core Snowflake's platform, including things like collaboration, other third-party applications. We make AI simple. 100% there are those people that will say, I want to take data that's sitting in cloud storage or even in another application. I want to bring it into cloud storage. I want to recreate Apple's, you know, Access Control list. And then I want to create a vector index using a bespoke, you know, vector indexing solution.
14:29 Speaker 00 And then I will stitch together. I'll figure out which model that I want to use, whether it's an API or something that I host myself.
14:38 Speaker 00 And then I will use LangChain and write like custom routing logic for my application. I can assure you that, you know, 99.9% of our customers want no part of this. You know, that's just the reality. All those poor people wanted was a chatbot to run on 100,000 docs that they have so that they can replace the annoying search box for FAQs on their site with, you know, here's a solution that just works. And our take is, yes, whatever governance you've had before works out of the box. And your data does not go anywhere else.
15:19 Speaker 00 You have the same rock-solid guarantee that Snowflake will never use your data to train any like cross-customer model.
15:28 Speaker 00 And, and we will be very efficient and cost-effective from just like overall cost of running the solution. But Snowflake's magic, honestly, is we make the hard simple. And it's things like total cost of ownership. Many of our customers, you know, are are banks. They are healthcare institutions. They are finance, you know, or other kinds, like we play a lot in the media space as well. Most of our customers want to solve problems, not solve technology for the sake of for the sake of technology. You know, we have a foundation model team. They're very focused on things like, how do we, you know, get models that are better grounded generation? How do we get them to follow directions well?
16:12 Speaker 00 How do we get them to say no to questions that they should not be answering? When it comes to, let's say, like, talk to your data. You know, so we focus on specialized areas like that. But the biggest reason to use Snowflake for a lot of our customers is 10% software engineering project with a whole lot of risk about data and security and what else can happen. Turns into six hours of work for an analyst. We are good at that. We are proud of that.
16:40 Pat Grady? So it sounds like the one liner might be it's kind of the the level or the layer at which you're intersecting these products. If you're working with one of the public clouds, you're still very much at the infrastructure layer, building a lot that yourself. Snowflake, you're at the platform layer. A lot of the hard work's been done for you.
16:55 Speaker 00 And our long term bet, Pat and Sonia, is that ecosystems move upstream. There was a time not so long ago where, I don't know, our parents, our grandparents, knew every part of a car.
17:07 Pat Grady? Yeah.
17:08 Speaker 00 They're like, oh, so manly to change a carburetor and get oil in between your nails.
17:13 Pat Grady? I gotta be honest with you, I'm still impressed every time my dad knows exactly what is wrong with the car.
17:17 Speaker 00 Yes, you know, while I'm willing to go to, you know, go to strength training every day, getting oil in between my fingers, you know, with my carb, hmm, does not sound so attractive anymore. And so 100%, you can work with CSPs and you can be like, I have a model garden here. I have a caching service there. I have a database here. I will stitch all of this together. As I said, everything turns into a software engineering project. For us, you're like, hmm, no, that's just a little data pipeline that you set up. And here is a beautiful UI that you get if you want a chatbot. Obviously, you can do more, but you don't have to.
17:54 Pat Grady? Yeah.
17:55 Speaker 02 Whether your customer is building on Snowflake? And are there certain types of AI applications that are better suited to be built on Snowflake than others?
18:02 Speaker 00 As I said, the categories of AI applications come naturally from the kind of data that are already there. I would say the broadest, broadest use case is really using Cortex AI via SQL in either interactive queries and dashboards or in jobs that people are running. And so these span the gamut from, oh, let's do sentiment detection with a small model. It doesn't really have to even be that expensive. So that's just like, literally, it's one function call. Or let's do other kinds of data extraction. Where, as I said, you have things like a transcript or maybe clinician notes.
18:43 Speaker 00 You take that out and you get structured data from it. Or the other thing that I talked about, Document AI, which is you extract structured data from things like receipts, from contracts, so on and so forth. That's kind of our sweet spot. But I have to say, the product that makes even the people that go, I have GPT-4, I have an army of software engineers. The thing that even they struggle with is things like a reliable talk to your data application. Because even with GPT-4 out of the box, you end up getting forty-five odd percent reliability, meaning it gets half the questions wrong when it tries to answer it.
19:25 Speaker 00 We are well in the nineties and we are racing to get to like ninety-nine percent reliability on talk to your data applications. Obviously, we restrict the domain and turn this into more of a software engineering problem than just like a pure AI model problem. But that's the thing that makes every Snowflake customer perk up and go like, I want that. Because even the people with the money and the resources to spend on software engineering teams very quickly realize that this is a wall that they are likely not going to break through.
19:57 Speaker 02 And how do you accomplish that? Like, maybe peel back for us, how you're able to get to the 90s percent? Are you training your own models? Are you? Just tell us about, you know, how this all becomes possible.
20:08 Speaker 00 It's systems design.
20:09 Speaker 02 Okay.
20:10 Speaker 00 Just like the magic of how you make a coding agent or an, in fact, actually less a coding agent, more an effective co-pilot work and practice. It's not always the, like, the giant models. It is carefully breaking problems down so that you present the right context to the model. It's in deciding things like, oh, I see. The problem of answering a question, whether to answer a question is different from how to answer the question. So you can specialize and have different models for these different subtasks. And also, what's the, um, basically, the, I call this a, like, a problem definition, a product structure question.
20:54 Speaker 00 We structure the product of Cortex analysts so that it is more restricted than a free-flow domain. What I mean by that is schemas are weird things. People do random stuff. They have horrible column names that mean completely the opposite. Every company has its own definition for revenue. And if you, like, take the best model on the planet and let it loose on an arbitrary schema, the likelihood that it's actually going to understand the nuance of what's in there, um, close to zero. Like, now, our big deployments, for example, our customers have 200,000 tables. And you can bet that there are several tens of thousands of tables with the word revenue in it. They just don't have the same meaning. So it's really like problem definition.
21:38 Speaker 00 To me, by the way, this goes back to the magic of product. I think of, like, any amazing founder, any amazing product manager, as someone that can visualize what's like the right trade-off to be making in order to create something that has broad applicability. And that's the thing that we have done here. We constrain the problem. But as I said, we also explicitly train for things like when to refuse questions, as opposed to trying to pretend that you can answer every question. But obviously, there's a precision recall trade-off there. You can get 100% precision by answering no question. That's not the goal. You want to be useful but still be precise. But it's a lot of software engineering.
22:17 Pat Grady? I want to go in a slightly different direction. Sure. Okay, which that that reminded me of this and I don't know why. But you guys, you seem, the product velocity at Snowflake seems to have inflected to the positive. Yeah. Even in the last six months or so. And we've worked with a lot of founders where, you know, the bigger the company gets, the slower and slower the velocity becomes. And so I guess I'm curious, what have you guys done to positively inflect product velocity? Because that's hard to do when you're dealing with an organization at the scale of Snowflake.
22:51 Speaker 00 I've done this many times before. And the formula is always roughly the same, which is first and foremost, you make sure that you have a safety net that you believe in.
23:04 Speaker 00 Which is you have like regression tests, so you don't blow up big functionality. But if you're pushing hard enough, you will make mistakes. And so you have to distinguish between different kinds of mistakes. For a database company, there are catastrophic mistakes.
23:21 Pat Grady? Yeah.
23:22 Speaker 00 Like if you write data badly, it's going to take you months to get out of that. So, you know, so you need to understand like what is risk? And then you build a safety net for things like, as I said, to detect problems before they happen. But in case you do have problems, how you get out quickly. At Google, for example, we built auto experiment scaling frameworks. Basically, you would, you know, come up with a new experiment. All new change, all changes went through this experiment framework. And this thing would automatically say, I'm going to run this on a machine, watch it for 15 minutes, make sure that the machine doesn't crash. And then it'd roll it out to 0.1%, 1%, 10% with measurement all along the way. All of a sudden, you have velocity because someone can design,
24:06 Speaker 00 people can design a whole bunch of experiments, they're sort of now pushed out. So as I said, the first part is the safety network. And so we spent a lot of time on on that. The second part is the inner loop productivity, which is how quickly can you get like a single change in quickly? Because ultimately, it ends up being the decider for how many changes are you going to, are you are you going to get through? Another system design, Snowflake actually went through a process that predates me, starting about two years ago. Of how to make the system extensible. As I said, at Snowflake, we are very proud of the single unified product. But that can become like, you know, something that gets in the way of speed.
24:49 Pat Grady? Yeah.
24:49 Speaker 00 And so you have to design carefully for how do you make things extensible? So things like AI basically took advantage of that, of that framework. And then to a certain extent, to be honest with you, it is also the focus that leadership needs to bring on what is important. How do you drive clarity? At all times with all teams, there is an infinity of work to be done.
25:13 Pat Grady? Yeah.
25:14 Speaker 00 And driving that clarity, driving a sense of accountability with AI team. For example, I force every team to make promises for, yes, over three months, but also what are you going to do the next two weeks? And calibrate yourself on, did you deliver on the things that you were doing? That you said you were going to be doing. It's pretty much, in my mind, if you want to get better and better, life boils down to say what you're going to do and do what you said you, you know, you would do.
25:40 Pat Grady? Yeah.
25:41 Speaker 00 And, and examine and make things, make things better.
25:46 Speaker 00 And so it's a bunch of things that have been there, that have been building up at Snowflake. But certainly, I bring this sense of quality and speed are both requirements in, in what we do. It's a, it's a change, but people like the idea of just getting more things done.
26:08 Pat Grady? Yeah.
26:08 Speaker 00 You know, like you and I have never met a software engineer that says like, yep, I want to release that day after tomorrow. It's like, no, you want to get it done today. And so that itself builds momentum. When you release a bunch of products and you have a lot of customers that are using it, that becomes positive energy for the team to build on the good behavior that kind of got you there. And so I would say the team has responded very, very well. And I told them, hey, listen, this is the world of AI. Stuff changes every week and you need to build with that with that speed. I I'm I'm very happy with how the team has responded.
26:42 Pat Grady? Is there anything in particular that you're most proud of in terms of what you guys have done in AI thus far?
26:47 Speaker 00 I'd say Cortex Analyst is probably the hardest product that we have designed and launched.
26:55 Speaker 00 Things like Cortex AI, which is like our platform layer, I'm I'm proud of it. But, you know, it is it is sort of predictable infrastructure work, even though there's a lot underneath in terms of, hey, should you use V L V L L M or something else? How do you optimize for inference? How do you get capacity in like this annoyingly crazy world where it's very hard to get your hands on GPUs? There's a bunch of stuff. But to me, that is a unique that things like that, things like Document AI are a unique combination of our strengths being applied to new areas in ways that can make a big difference to our customers.
27:35 Pat Grady? Yeah.
27:35 Speaker 00 And, you know, but you also know, Pat, that this is a little bit of like, you know, who's your favorite child? So I can't really do that. And so there's a there's a bunch of stuff, like even if you take Polaris, which is our cloud catalog, you know, done in a matter of three months. And so I think there's a lot of energy within within the team because,
27:57 Speaker 00 you know, it's a slow message, but it's getting through that you can have speed and quality. They're just different aspects of the same problem. And my firm belief all through my life is that virtuosity trumps strategy all day long.
28:14 Pat Grady? What does that mean?
28:16 Speaker 00 Your speed of execution, your speed of reacting to situations is going to trump strategy very, very quickly. Yes, you need strategy, but life is never about fixed strategy because we live in a very, very dynamic world. It's hard to predict which product is going to be wildly successful, what your competitor is going to do. Like we're going to talk about like G P T five. It's like it's a big unknown, whether it's going to come out and what impact that's going to have. So I place a huge amount of emphasis on you just need to be really, really quick at at what you do. And I would say like, that's the message that I'm trying to convey to the team.
28:58 Pat Grady? That's very, I see nice continuity from the Slutman era into the Shridar era, because I know I've heard Frank say at least a few times, the General Patton quotes, "A good plan executed violently today is better than a perfect plan tomorrow."
29:12 Speaker 00 100%, 100%. And, and as I said, that adaptability, Napoleon has a famous quote, which roughly, I mean, it's not his, it predates him, it roughly translates into, you know, "I commit and I adapt," which is, you go into an important area, knowing that you're not going to know everything, and then you're adaptive to the situation that actually presents itself.
29:39 Pat Grady? Yeah.
29:40 Speaker 02 Are there any misconceptions about Snowflake and AI that you want to debunk?
29:43 Speaker 00 We are a real player. It used to be that Snowflake used to be thought of as somebody that didn't really get AI. And, you know, but like early on, we relied on things like more of a partnership-oriented strategy for AI. But my big sort of observation, realization is that AI is a, like, it's a platform change in the sense that it is a new way in which you and I and everybody else in the world is going to get to software, is going to get to applications. And, and so, once we had that realization, out came a bunch of product consequences,
30:27 Speaker 00 which is, AI needs to be central to Snowflake. We need to make it super easy to both build applications, but also build the most important applications ourselves. Cortex analysts, for example, is a direct-to-business-user application. We have never really done things like that before. It is driven by a strong belief that AI is going to disrupt how information is going to be consumed very, very broadly. And, you know, I am proud of having a world-class team from bottom to top, from foundation models to inference experts, to product engineers that integrate the AI, plus also
31:08 Speaker 00 the product engineers that are creating applications on top of, on top of AI. That, combined with things like broad data access, which is Polaris and, you know, Iceberg, I think puts us in a very, very good position.
31:25 Speaker 02 Can we zoom out and ask a little bit about your, I guess, your hypotheses and your hot takes on the future of AI?
31:30 Speaker 00 Absolutely.
31:31 Speaker 02 I just think you are so well positioned. You probably built one of the first, if not the first, kind of LLM-native consumer applications at Neva. And now, obviously, from your seat at Snowflake, you see so much. Maybe first on the LLM kind of race to scale.
31:46 Speaker 00 Yep.
31:46 Speaker 02 Like, what do you think about all that? Are we reaching the limits of scale? Like, what's next for those guys?
31:52 Speaker 00 I mean, obviously, this can go in a couple of different directions. I talked to a lot of experts. And, you know, there is a collective belief that there is a GPT-5 in the horizon.
32:06 Speaker 00 What I don't think anyone has a
32:11 Speaker 00 like a clear bar for is what that's going to represent.
32:15 Pat Grady? Yeah.
32:16 Speaker 00 GPT-4O was very cool. Much faster. It also integrated multimodality natively in a way that's pretty, that's pretty amazing. But when you think about reasoning capabilities, the ability to come up with plans for how to execute stuff, it didn't feel like it represented a step change.
32:38 Speaker 00 And while agents are, you know, very hot, similar to Cortex, you know, until Cortex analysts came along, people didn't really believe that you could build reliable talk-to-your-data applications. They were always kind of hidden. And remember, the bar is very high. If you're giving data to a business user, like 75% accuracy is like one out of four wrong. And so I think the big unknown is whether these models are going to represent a big step forward in things like multi-step reasoning. And if they can, they're going to unleash like a whole new class of applications that you and I just cannot imagine right now.
33:23 Speaker 00 You know, on the other hand, I think when it comes to driving broad adoption. There is a lot that can be done with existing models. So many things that are useful for you and me every single day. Whether it's a piece of mail that we are looking at or looking through a PDF. Just think about all the tedium that all of us have to go through. And so I think there is huge impact to be had simply in AI technology just permeating software as we know it, especially the the user input part of of software. So unlike other technologies, I think like there is enough that AI has already delivered that is going to have a meaningfully large impact on society.
34:06 Speaker 00 It's just going to take a while to to run out.
34:11 Speaker 00 You know, I sincerely hope we don't get to a phase where you need a billion dollars to train a great new model. I actually think that while what that model can do is cool. I think it also reduces the number of people that can have models like that to a very small number. And I think competition is just overall healthy. So but it's very hard to make a call.
34:35 Pat Grady? You've mentioned this a little bit, but I'm curious to get your take on it a bit more.
34:40 Pat Grady? You know, if GPT five is delayed or not a big step up or whatever the case might be, or if you just imagine a world in which the current capabilities of the foundation models, that's what we've got. And it comes down to how do we implement those? How do we optimize those? How do we tune those? One of the things that we hear from a lot of people building in AI. The first couple of weeks are like magic. Everything is amazing. This is great.
35:04 Speaker 00 Yep.
35:04 Pat Grady? And then the next few months are pretty painful. Oh, shoot. It can't do this corner case. It can't do that corner case. It's not quite accurate enough. And people get really frustrated. And sometimes they can engineer their way out of it. Sometimes they can't. But sometimes it leaves people feeling kind of disillusioned. Like, yeah, this stuff's not as good as I thought it was. You know, maybe the time's not right. And so I'd love to get your take. If we froze the capabilities of the foundation models today, what sort of changes will we see in the enterprise landscape over the next handful of years? What sort of stuff will we not see? Because we're just not ready for it yet.
35:38 Speaker 00 To me, this is this is honestly the magic of software engineering. Part of.
35:45 Speaker 00 What I feel they have implicitly accepted with chat G P T.
35:52 Speaker 00 Is it's sort of like, is it's omniscience. They're like, it can do everything. They don't say it. In fact, they are, they go to, they take pains to not say it. But just like Google search never tells you, that's a dumb query.
36:09 Speaker 00 Think about it, right?
36:10 Pat Grady? It's kind of fun if it did.
36:11 Speaker 00 If it, right. But there are lots of dumb queries that people type into it. Google's like, oh.
36:15 Pat Grady? I typed lots of dumb queries.
36:16 Speaker 00 Yeah, they're like, oh, here are a hundred million pages on the web. And here are the best pages for you, Pat, for your dumb query.
36:25 Speaker 00 And so I think it's like, it's, some of it is good old fashioned. You know, A I enthusiasm. It can do everything. But some of it is just also plain dumb. You should not be doing that. To me, this is where things like, okay. Let's actually make grounded chatbots the norm for, you know, like interacting with information.
36:53 Pat Grady? Yeah.
36:54 Speaker 00 The model is there, you know, this application should tell you where it got, where it got the information from. It should be very easy for you to verify said piece of, of information and feel good.
37:07 Pat Grady? Yeah.
37:07 Speaker 00 That you're actually getting something. Similarly, you need a test framework. You know, like Harrison talked about an observability framework to do this on an ongoing basis. But I think sometimes when it comes to things like chatbots, people forget, wait, like there is such a thing as a set of regression tests. There is such a thing as acceptance criteria for software. Everything that we have, like, if somebody were to build a new application, like one of your founders, your expectation is that, you know, they got their clue together. And are actually testing stuff before they give it to customers.
37:41 Pat Grady? Yeah.
37:41 Speaker 00 And someone in the world of A I, we're like, no, no, no, no, no, it doesn't matter. And these models, you know, react pretty violently to the addition of a period in a prompt. And so I think there needs to be this this idea that like you need good old fashioned software engineering and you need to measure the performance of these things. And and so I think this is where it goes, you know, away from these are hobby projects that can be hit or miss to, you know, here's somebody that can actually software engineer this for you. And we think of that as a core strength of what we bring to the table, which is like, you should be able to have a predictable way to say, you know, this chatbot is going to work.
38:23 Pat Grady? Yeah.
38:24 Speaker 00 Or this agent like application, this is the success rate that it's going to have. Or this is what Cortex analysts is going to do for you in your domain so that you're like, okay, I feel good about about deploying it. So even if GPT five did not happen, I think there is a lot of magic to be done, but it's also just work.
38:47 Pat Grady? Yeah, yeah, yeah, yeah, well put. Well, what's the, I forget who said, there's a quote that we use every now and then. People miss most great opportunities because they tend to be wearing coveralls and they look like work. You know, I think this is one of those where, like anything else, if you want it to be great, you gotta work pretty hard on it.
39:07 Speaker 00 You got you got to you got to sweat it out. And to me, this is also the place where the thinking of recall as something that you should tune, thinking of recall as an important part of how you think about these applications. Any ML engineer worth their salt will promptly come and tell you, it's like, okay, I have an AUC curve for you. What are they trying to say? They're basically trying to say there is a trade off between how much you squeeze the model to do and how good it is. There's no perfect answer. That's really what the AUC curve represents. And the more we think of AI applications as also having this AUC curve.
39:48 Speaker 00 There are trade offs to be made between reliability and ability to respond. And that's a very conscious factor in how you should think about things. I think the better off we are going to be in terms of where can they deliver value. Yeah.
40:03 Speaker 02 I want to go back to the point you said a little bit earlier about reasoning and kind of that delivering the next big leap, hopefully for GPT-5 and Claude, et cetera. It seems like the approach that most folks are taking is kind of bringing in search at inference time and a lot of more inference time compute and kind of this AlphaGo style search stuff. I'm curious, just given you are, you know, one of the best people in the world at search, like, do you think that is the path to the promised land on the research side for bringing reasoning into these general models?
40:35 Speaker 00 Give me a little bit more context. Like, I can certainly see how search plays a role in how these models operate. But can you just tell me a little bit more?
40:42 Speaker 02 Yeah, so I mean, if you take the example of, you know, if you take AlphaGo and you're trying to decide what move to do next. If you can, if you can kind of create a branching tree of here are all the possible moves from here and do a search kind of over that of like, here's what move I should do next. I think people are trying to bring that logic into, out of the gaming world and into domains like, I don't know if you saw Devin's cognition, where they're effectively searching over different things that you can do when you're coding as well. And so just like at inference time, just giving the model kind of the ability to like search possible paths to decide what to do.
41:20 Speaker 00 Yeah, there have been a number of papers, you know, on this. I think even NeurIPS had a bunch of papers about searching over domains as you, as you come up with a plan. What I don't have, to me, it's important to understand. I'm forgetting the name of the NeurIPS paper, but it also had the same problem. They were doing tree search. Is that they fundamentally rely on a model, typically a neural network, being able to do things like grade a particular point in a like a state space. Yeah.
42:00 Speaker 00 Basically, like AlphaGo, for example, you know, has pretty solid ideas about what is an advantageous position versus what is not, and the search is guided by, you know, by that. What isn't clear in sort of, you know, very open-ended questions is as you come up with alternatives for the search space, you know, can you actually grade them effectively? If it's like an open-ended plan. Certainly, a number of these techniques work well for games that have structure, in which you can actually learn what does optimal mean, and you can begin to optimize towards it. What I don't have as good a feel for is, let's take like, you know, something as simple as cooking.
42:46 Speaker 00 You would think it's, you know, it's simple, but if you take, I don't know, ten ingredients and twenty steps that you can take along the way, and various things that you can do in each of these twenty steps, and the steps themselves can be short, they can be long, you quickly end up with like this crazy combinatorial explosion of different ways of doing things, and yet there is just one perfect recipe, or two, or three. That's the part, honestly, I don't have a good feel for in terms of, like, how do you even begin to measure the jump in terms of cognitive ability? It's easy in structured environments, but like out in the real world, where you're trying to do some pretty complex things, I think it becomes trickier.
43:29 Speaker 00 We've built prototypes for basically like agent analysts, but it's again a structured space. Yeah. So what we do, one thing, I've done numbers like pretty much all my life. I used to do whatever, household finances for my dad when I was ten, same, like we did in a notebook. And over the past twenty years, every day I get like this email that tells me how my company did the previous day. It used to be called Bean Counters at Google. Every day you got, you got a report card. Every few weeks, something would go wrong. Like, you know, you made less money somewhere. And we would like start this predictable problem, like predictable exercise of some poor analyst would like go drill down into a bunch of different things, blah, blah, blah, blah, blah. Look at sliced stuff.
44:14 Speaker 00 And then they would come back with like, oh, Sridhar, it was like Easter in Germany and Ascension Day in Brazil. And that's why our numbers were off. And it took like a decade to model all of these complex things in the world into like a prediction model. So you're like, okay, I can, I can begin to predict. But if you think about it, the analysis that they do is constrained. It's pretty much if a metric is wrong, go slice it by 10 different dimensions. Go look at the results, see where likely the problem is.
44:42 Speaker 02 Yeah.
44:43 Speaker 00 Certainly, we have built prototypes of this AI analyst that can remove 60, 70% of the work that is needed in actually diagnosing problems. It's pretty free form, but you can make a, you can make a language, if you can tell a language model, these are my attributes. Oh, go call Cortex analysts with all of these parameters, get the output, take a look at it and then tell me what I should do next. So you can begin to automate some of it so that this is actually useful. So you can do things like that, but a much more open ended problem of here are 100 different things, incomparable things you can do. And how do you judge and how do you prune? I think that's the part I honestly don't have good intuition for.
45:22 Speaker 02 Totally. I want to ask about search in a different sense, if that's okay. You obviously have a incredible point of view on search, given your time at Google and at Niva. And it seems like right now, the, you know, the consumer world is watching excitedly and nervously about, you know, is, is there going to be a new kind of search king crowned? I'm curious your take on, on the whole kind of AI search space right now.
45:48 Pat Grady? How about a hot take on perplexity? Do you have a hot take on perplexity?
45:52 Speaker 00 Like, look, I'm happy for perplexity. And it reminds you again that right time, you know, right time, right place matters a lot. At Neva, you know, which converged on to a view of what search should be, that was very similar to perplexity. We were just two, three years early. And timing ends up being everything. You can think of perplexity as, as like a consumer manifestation of how we want to deal with information.
46:23 Pat Grady? Hmm.
46:24 Speaker 00 Like, it's just, um, let's face it. I want to look through an eight-page doc to find the two lines that I really care about. Said no one. But that's search. And so in that sense, it's absolutely the right place. I think the more important question is whether the business of search, which is carefully preserved with business contracts, not with consumer choice. Consumer choice is fiction in a whole bunch of things that, that we do. We eat what's put in front of us and we will search with the, the default search engine that came in our browsers.
47:06 Speaker 00 Okay, we might resist it, but on aggregate with humanity, that's the reality of, of the world. And so I, um, you know, I, I would say that that is the bigger challenge. Because search is mostly locked up by a few players that control the entry points. But I think that's the fundamental problem, which is it is very difficult to break into the business of search. Consumers don't like doing stuff.
47:39 Pat Grady? And this also gets to one of the kind of broader questions in the world of AI right now, which is incumbents versus startups. And historically, the battle is can the incumbents with distribution build cool products?
47:51 Speaker 00 Yep.
47:52 Pat Grady? Before the startups with cool products build distribution. And I think search is a great example of that. You might have the coolest products in the world. It's awfully hard to change consumer behavior.
48:01 Speaker 00 That's right.
48:01 Pat Grady? Um. AI is an interesting test case for this because so much of the coolness of the products is available through the open source world or through third-party models. And so it feels like it might be a scenario in which incumbents are advantaged versus the startups. But do you have a point of view on that?
48:21 Speaker 00 I would take two different lens to this one. One is what you said about models, open source models plus players like Meta that basically have infinite budgets.
48:34 Pat Grady? Yeah.
48:35 Speaker 00 Under willing to open source models. I think the world of creating models from scratch, unless you have an attached hyperscaler, an attached business, looks very, very hard.
48:48 Pat Grady? Yeah.
48:50 Speaker 00 And, you know, so I think, as I said, I hope this doesn't go to like an ergo three GPT five class models that the world has because I think that's a bad ending for for the world. So I would definitely say that foundation model companies without a strong business to accompany them, it can be a product, like I think OpenAI has created a pretty solid product. It's not just a foundation model.
49:18 Pat Grady? Yeah.
49:19 Speaker 00 I think that's one thing to keep in mind. I'd answer your second question of sort of disruption slash innovation from a historical lens. I think of every generation of Silicon Valley companies as learning from the previous ones. They are smarter. They know the ways in which things can be disrupted and they lean in pretty heavily. You know, we all know, for example, the IBM to the mid-range computer sort of disruption.
49:59 Speaker 00 And then the DEX and SGIs of the world then getting disrupted by the Microsofts of the of the world. And then the web coming along, leading to the rise of companies like Google or mobile.
50:14 Speaker 00 I would say that in each and every one of these transitions, powerful incumbents with very large pockets have shown an ability to lean in sooner, lean in faster. At Google, for example, when I was there, we leaned in very heavily into the home assistants. Because Alexa was going to take over the world. That was going to be the way in which you and I and everybody else searched. We were terrified. And we put a pile of money into it. And nothing came of it. And it didn't matter. Why? Because the cost of a disruption is way higher than the amount of investment that you have to make.
51:00 Speaker 00 I would say now this is generation five, or something to that effect. I'd say all the incumbents are very aware of what can be disrupted. And they lean into it. Like, there's a bunch of strategic thinkers, as I told you. I think of AI as basically shuffling the tiles on enterprise software. And a part of me goes like, you know, no way. Snowflake is going to be leading the charge when it comes to AI, not waiting for it to develop. But I think you see every enterprise AI company lean in the same way. And so this, to me, would be the question about how much disruption is AI going to drive in consumer software? Certainly, there'll be new categories. To me, if I were a startup,
51:43 Speaker 00 I'd feel a lot more comfortable that I'm creating a new category. Image creation, like, done in a mass scale, clearly amazing. But the same goes for videos, same goes for voice. There's a bunch of specialization that you can do here, adapt them to marketing. New things feel like a much safer bet in the AI world than, you know, take your pick. I can do X, Y, Z faster because I am AI-enabled.
52:10 Speaker 00 I don't think of that as having a whole lot of legs. Yeah.
52:13 Speaker 02 Do you think ChatGPT has a chance of becoming the next Google? And to your point on consumer choice being a mirage and like, you know, business deals are where this stuff gets locked down. Like, I'm curious what you think of the Apple ChatGPT deal.
52:25 Speaker 00 I think ChatGPT, I mean, the phone is a pretty interesting place. To me, the phone, because it's a controlled environment, actually offers enormous potential for consumers.
52:41 Speaker 00 I tell people, something as ridiculous as copying, I don't know, an address like from your calendar or a piece of email over to Uber. So dumb, so hard.
52:54 Speaker 00 You know, like, you would think Citi would like, you know, do this? Copy, copy the address from this email from Pat and stick it into, stick it into Uber so I can get an Uber. So to me, I think like there's a huge amount of potential again in like mundane applications. And because the mobile ecosystem is a, is a pretty closed one, where Apple can mandate things like you must have APIs that make it possible to access your functionality using language models, or else you might not get any traffic. That sounds like a pretty good incentive for everybody to kind of get in line. So I think there's a huge amount of potential there. I honestly wish there was more innovation in this space.
53:38 Speaker 00 Because again, all of this is super doable technology. You and I can argue about should this be done in the cloud? What can be done on the phone? But like, as a consumer, do you care? Like, we have great connections. I'm kind of like, if this thing works only when I'm connected to the internet, I'll take it. And so to me, those are, those are sort of, you know, those, those are details. I actually think ChatGPT is an amazing product. There's underlying technology, but in so many different ways, they've actually created a stunningly beautiful product experience. Yeah. That spans the gamut from, you know, they've turned pretty much like visually illiterate people like me into budding artists.
54:22 Speaker 00 I tell people, it's like, I'm good with words. I can talk all day long, I can write all day long. And the magic that I can do with ChatGPT is truly amazing.
54:32 Speaker 00 That or even even things like I, you know, for example, like I'm on this language kick. I'm learning Hindi and at some point I was like, oh, I'm struggling with these numbers. But off comes a prompt that says, hey, I want a CSV that translates numbers, just a string of numbers to Hindi. And can you do that? Can you just give me a CSV file that I can import into Quizlet? That literally is faster for me to type than to describe to you. I type it in, out comes a CSV file in 10 seconds. I download it into Quizlet. I have a quiz.
55:04 Speaker 00 And so, and pretty much everything that I used to do with Python scripts on structured data, I just do like with with English. You just upload the CSV file and you're like, oh, add these two columns, do this other thing, format it into this nice table and get it out for me. It's it's magic. So I think there's absolutely a there there in terms of like, is it a great product and a great business? But, you know, being the king of search is like a few more zeros. They don't come easy to people.
55:35 Pat Grady? Yeah. All right. Should we close with a couple of quickfire questions? Rapid fire questions?
55:41 Speaker 00 Yeah, let's do it.
55:42 Pat Grady? Okay. Who do you admire most in the world of AI?
55:45 Speaker 00 Who do I admire most in the world of AI?
55:53 Speaker 00 I admire the people that are, you know, like, working on things like foundation models that are able to do it on on on the cheap without the infinity of resources. So for example, people like Arthur or Danny. I think they've gotten Danny Yagotama from Rekha. I think they've gotten just like a remarkable amount of things done. Or from our own team, folks like Samyam and Yushang. To me, they represent so much creativity because I go and tell them, ah, limited budget and what can you, you know, what what can you do? I think there are there are a set of just like amazing earnest people
56:36 Speaker 00 that are driving research under tight constraints. So there's there's obviously lots and lots of people. But it's the it's it's the doers that are doing the work, imagining our future that I'm a huge fan of.
56:51 Speaker 02 What's your favorite AI application?
56:53 Speaker 00 ChatGPT by far.
56:55 Pat Grady? Easy one.
56:56 Speaker 00 Easy one. Just the utility that I get from it day in and day out is just truly remarkable.
57:01 Speaker 02 Okay, follow up then. What's an AI app that you wish existed?
57:04 Speaker 00 Like, an actual talk to your phone? That can actually mediate between apps? That would be, that'd be super cool. Because remember, as I said, like just flipping between applications, doing very, you know, little things, such, such, such a pain.
57:17 Speaker 02 Yeah.
57:18 Pat Grady? All right, we're gonna end on an optimistic question. What is the best thing that can happen in the world of AI over the next five or 10 years? What would you be most excited to see coming out of the world of AI?
57:30 Speaker 00 Software, which you can think of as encoding our thinking,
57:38 Speaker 00 capturing our ability to think, and act in real world situations, clearly has been transformational for the past 50 plus, you know, years. To me, AI as a, as an enabler of access, both to the act of creating software, but using software to all of the people in the world, would be a significant step up. And as I said, I don't think it's like lots of fancy new technology that you need. The newer technology can certainly help newer classes of applications. You know, I was very proud of the fact that we put Google Search, thanks to things like Android, into the hands of pretty much every human being on the planet.
58:24 Speaker 00 It's, it is a, you know, you can be cynical about technology, but it's a genuine step forward for humanity. To me, just like AI models as like the new layer between humans and software, and software and software, is actually a significant step forward just in having this functionality be vastly more accessible to lots more people. As I said, both in the creation aspects, but also in the consumption aspect. I think that's a pretty cool thing to look forward.
58:54 Pat Grady? Awesome. Thank you, Shridhar. Thanks for doing this.
58:56 Speaker 00 Thank you, Pat. Thank you, Sonya.
58:58 Speaker 02 Thank you.
Previous OpenAI's Noam Brown, Ilge Akkaya and Hunter Lightman on o1 and Teaching LLMs to Reason Better Next Google NotebookLM’s Raiza Martin and Jason Spielman on Creating Delightful AI Podcast Hosts and the Potential for Source-Grounded AI Page updated October 12, 2026 · transcript and summary are machine-generated
