Kumo’s Hema Raghavan: Turning Graph AI into ROI

Hema Raghavan is co-founder of Kumo, a company that makes graph neural networks accessible to enterprises by connecting to their relational data stored in Snowflake and Databricks. Hema talks about how running GNNs on GPUs has led to breakthroughs in performance as well as the query language Kumo developed to help companies predict future data points. Although approachable for non-technical users, the product provides full control for data scientists who use Kumo to automate time-consuming feature engineering pipelines. Mentioned in this episode: Graph Neural Networks: Learning mechanism for data in graph format, the basis of the Kumo product Graph RAG: Popular extension of retrieval-augmented generation using GNNs LiGNN: Graph Neural Networks at LinkedIn paper KDD: Knowledge Discovery and Data Mining Conference Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital

• 0:00Speaker 00

  If you have your data laid out as relational tables, a Qumu just sucks it in. So we just, you just specify and, you know, through connectors, tell Qumu what your schema is. And then you can just start writing predictive queries. So the graph is abstracted away. But if you have someone like a data scientist who loves tweaking the neural network parameters. In case Constantine is. Yes, decides to. I would be interested. I'm not guilty. Exactly. You can look under the hood. And the analogy we always use is we'll give you the self-driving car. But if you want to look under the hood, or if you want a drive stick, we'll let you drive stick.

• 1:00Speaker 01

  We have a brilliant guest today on Training Data. Welcome Hema Raghavan, co-founder and head of engineering at Qumu AI. Hema brings decades of experience leading AI initiatives at LinkedIn. She came up with the People You May Know technology and other core features that leverage the power of graph learning. Her journey in AI predates many of the technologies we all take for granted today. She was working on NLP before BERT was even a thing. With Qumu, Hema and her team are revolutionizing how companies harness AI by making advanced graph neural networks. These neural networks let you do AutoML, automated machine learning, on any platform, from Snowflake to Databricks.

• 1:46Speaker 01

  Qumu's innovative approach allows companies to leverage their existing data warehouses in order to build sophisticated AI models faster, cheaper, easier. You don't require the deep expertise in graph learning or maintaining complex features. You can just go straight to business value. Welcome, Hema, to Training Data.

• 2:07Speaker 01

  Today, we have the amazing Hema Raghavan. You are building Kumo AI, which is AutoML on the data warehouse using advanced neural networks and graph neural networks. AutoML was incredibly promising a few years ago. It was a major trend in the last wave of AI, five, six years ago. It went through a little bit of a trough of disillusionment. A lot of the AutoML players receded from the forefront and companies started to store their features in future databases and the like. Why are you focusing on AutoML? What's different about Kumo?

• 2:43Speaker 00

  Okay, so there's AutoML and then there's AutoML on GPUs. And I think that's the big difference for Kumho.ai. And let me give you a little bit of an example from my own career. So I started in NLP and when we would build systems back in the early 2000s to answer a question like, when did Marco Polo land in Asia? We would be encoding features like Marco Polo is the subject of the sentence and it's going to be the subject of the answer and all of that. So we had to know a lot about language, about linguistic structure and so on. And then the GPU revolution came that enabled neural networks to come at the forefront of this technology.

• 3:28Speaker 00

  We don't write features like that anymore. Those intermediate layers in a neural network really learn the parts of speech, the named entities, all of those properties of language. It's the same in other classes of problems. So in the class, the AutoML that was happening maybe a decade ago, we were looking at CPU based models. So think of logistic regression, think of XGBoost, SVMs and so on. And all AutoML did then was parallelize what a data scientist would have done, which was a lot of hand-computed features.

• 4:08Speaker 00

  And that required you to be, you had to write code to think like a data scientist. So you were trying to get. The machines to think like humans.

• 4:20Speaker 01

  Hmm.

• 4:21Speaker 00

  Whereas here, what we're doing is we've, uh, we use graph neural networks. So it's a neural network technology. And you can think of a G N N as a superset of a C N N, which is used for images or a sequence model which is used for languages. G N Ns, you know, allow for arbitrary structure. And the G N Ns are learning all of the features that you would normally use for prediction, predictive problems. So Kumo sits in the space of predictive A I. And we're really bringing transformer technology to predictive A I problems.

• 4:57Speaker 01

  Hmm.

• 4:57Speaker 02

  Can you say, you mentioned graph neural networks and you gave a great explanation. Can you explain to me like I'm five years old? Because that might be where my level of understanding is. Like, are graph neural networks good for any class of problem? Is it good for, you know, you came from LinkedIn where you were working on, you know, the, the social graph of LinkedIn? Is it, is it good for specific types of, of domains?

• 5:18Speaker 00

  That's a great question, Sonia. So let's say you're going to put this podcast episode out and, you know, it's going to be on some, uh, video streaming site. And we want, you know, uh, to recommend the, the relevant podcasts for, you know, users of that video streaming site.

• 5:37Speaker 01

  YouTube.

• 5:38Speaker 00

  YouTube.

• 5:38Speaker 01

  To be explicit.

• 5:38Speaker 00

  YouTube.

• 5:39Speaker 01

  Someone's watching this on YouTube. You want to recommend someone to watch it or not?

• 5:42Speaker 00

  Exactly. So user logs in and, uh, you, you not only have the content of the, this podcast episode, but you also have what you might have watched in the past. So you can think of that, the records of what you watched in the past is sitting in a views table.

• 6:02Speaker 01

  Collaborative filtering era.

• 6:04Speaker 00

  Exactly. Exactly. But the difference with collaborative filtering is it's just looking at views.

• 6:11Speaker 01

  Yep.

• 6:12Speaker 00

  How can we take? You know, the view data. So the view data is a network. So coming to Sonia's question, right? There's, there's a podcast episode. There's all the users who are watching it. So you got it in terms of, you know, you have a bidirectional graph, the users and the podcasts. But then you have the organization. You have Sequoia Capital. You have the channels from Sequoia Capital. You have other metadata that you may have in. So all of that can lend itself naturally to a graph and start thinking about links across these, you know, nodes of a graph. And effectively, what a graph neural network is learning is, let's look at what Sonia watched

• 6:56Speaker 00

  in the past. It seems like she really likes AI.

• 7:01Speaker 02

  AI and Baby Shark videos at this moment.

• 7:03Speaker 00

  Okay, so AI and Baby Shark. But, and then the neural network also learns that Konstantin likes AI. And what would it be for you?

• 7:15Speaker 01

  Probably AI. That's, that might be, yeah, like history.

• 7:20Speaker 00

  That's great. So there's AI and history, right? So the neural network can learn that there's an overlap between both of you on the AI pieces of content. You both engage a lot with Sequoia content and it's learning across this network, right? But the next time Sonia watches a Baby Shark video, we don't want to be recommending that to Konstantin, right? So how do you take that content that you engage with, the view data, the click data that you're engaging with, and learn across all of these edges? Think of clicks, views, every, all your behavioral signal that you engage with, with entities in this world as a graph? And how do we learn across that graph?

• 8:06Speaker 01

  Hmm.

• 8:06Speaker 00

  Yeah. So you don't have to be a social network to have a graph. Everyone.

• 8:12Speaker 00

  Almost every enterprise I know has a graph. Fintech has graphs because they have customers, they have transactions, they have related data. Think of, you know, one of your delivery services. They have the inventory, the suppliers, the, the, the means of transportation. You have, so they're all sitting as tables, they're all sitting as entities and they're all linked across each other.

• 8:40Speaker 01

  So that's.

• 8:41Speaker 00

  Graph learning lets you learn across that.

• 8:42Speaker 01

  That's a pretty key insight, the tables.

• 8:45Speaker 00

  Yeah.

• 8:45Speaker 01

  Before we go there, that was a very smart five year old. I think that you have a five year old.

• 8:49Speaker 00

  I have an eight year old.

• 8:51Speaker 01

  An eight year old.

• 8:51Speaker 00

  And a 12 year old.

• 8:52Speaker 01

  A 12 year old, okay, well, they're very, very smart if they understood that explanation. Like what would be to Sonia's point, if you were, if you were five and you were going to say graph learning versus any other type of machine learning, what's the difference?

• 9:05Speaker 00

  Ah, graph learning versus machine learning.

• 9:10Speaker 00

  Easy.

• 9:12Speaker 00

  Fast. I think those would be the two things, just, you know, low code. I think that would be the key about Kumo.

• 9:21Speaker 01

  It, it learns all the weights, it learns all the features and discovers them over time. Is that fair to say?

• 9:26Speaker 00

  Exactly. And my eight year old has no machine learning, but if you were going to write, uh, uh, the, uh, you know, the classifier, the old school way.

• 9:36Speaker 01

  Yes.

• 9:37Speaker 00

  You'd be writing, uh, features that say, okay, users in this platform, uh, uh, we need to look at click through rate data for the last three months and six months and eight months for every single video. And we discover that Sonya has a preference for data that's, for videos that are evergreen. So six month windows really matter for Sonya. So imagine all of that code being written as features. Graph neural networks eliminate all of that code.

• 10:07Speaker 02

  Do you think that means feature, feature engineering goes away as a discipline or what, what happens to it?

• 10:13Speaker 00

  I think feature engineering goes away and that's not a bad thing as such, because, uh, prior to, uh, Kumo, I was at LinkedIn for, uh, almost seven, uh, close to eight years. And data scientists love.

• 10:31Speaker 00

  Finding opportunities for the business to make value, right? And it doesn't mean that feature engineering is the place where you spent that, you know, that that's the the time well spent. You'd much rather try out N different models on N different parts of the app or whatever your business is and and drive value. So trying out models in different parts of your application is where a data scientist needs to spend time.

• 11:02Speaker 02

  We started this episode, you you said, you know, AutoML on GPUs is different from AutoML.

• 11:07Speaker 00

  Yes.

• 11:07Speaker 02

  And so what about GPUs specifically makes what you're describing possible? Like, was it even possible to do this on a CPU? Or is it just, is it faster now? Or what's different now that you're doing on GPUs?

• 11:18Speaker 00

  Yeah, that that's a great question. So it's definitely possible. It's much slower, right? So it's very similar to what neural networks brought to the text and image image spaces in that we can scale these models to, you know, large amounts of data. And while these models existed before the GPU revolution, it's we can actually take an entire enterprise, it's like fintech data and learn graph neural networks for them.

• 11:49Speaker 01

  Yeah, in the previous era of AutoML, so much of the juice in the performance came out of ensembles. So you do these logistic regressions or you do these SVMs or what have you, and then you'd ensemble them together.

• 12:02Speaker 00

  Yeah.

• 12:02Speaker 01

  Frankly, in the Kaggle era, which was how I first met your co-founder Yuri and the data science era of Kaggle and the like, always the ensembles won. Even in the Netflix Prize back in the day, it was the ensembles that won. And there was something to the fact that these ensembles are just tons of little algorithms chained together. And what is a neural network but tons of little algorithms chained together? I mean, you could consider it billions of sigmoids or billions of logistic regressions. And really, the way I see graph neural networks is you're able to discover the the features and the ensemble that you chain together to actually optimize towards the solution. So it's the, to me, graphs are the most general data type.

• 12:42Speaker 00

  Yeah.

• 12:43Speaker 01

  And a graph neural network is the most general. You kind of, you mentioned it's a generalization where even a transformer is a subset of this generalization. The most general type of algorithm that can do some learning.

• 12:55Speaker 00

  Yeah, absolutely. And As you mentioned ensembles, something that struck me was try maintaining that in production. You have N different feature generation pipelines and an ensemble. And I've seen a world where you'd have one front end engineer change how we were logging the view data.

• 13:19Speaker 01

  Yeah.

• 13:19Speaker 00

  And everything either needed to change or something, you know, one pipeline.

• 13:26Speaker 01

  Breaks and it's all done.

• 13:27Speaker 00

  And it's it's a mess to debug. So you so graphs give you a simple elegant framework to get get at the same outcome.

• 13:39Speaker 01

  It it also reminds me a lot more of our brain.

• 13:41Speaker 00

  Yes.

• 13:41Speaker 01

  Right, our brain kind we think operates like a like a graph. And is forming and pruning connections more like a graph, even more so than a more structured neural network.

• 13:53Speaker 00

  Yeah.

• 13:54Speaker 01

  And so have you ever have you guys experimented or thought about that as an analogy? And any ideas of the pros and cons of that analogy?

• 14:03Speaker 00

  I think it's very similar to the way I think about it is let's go back to that video watching example, right? And if I think of Sonya as a node in in a graph. And what these neural network algorithms are really good at is learning these embedding representations, right? And on this big graph, which has Sonya with her preference for baby shark and AI. Or her household's preference. Exactly.

• 14:36Speaker 01

  Makes more sense, but that that checks out.

• 14:40Speaker 00

  Your embedding vector would be pretty close to both AI. So you're close to Constantine, but you're also close in Euclidean space or, you know, in some big N dimensional space to. Uh, the all the baby shark loving folks, right? And we're basically learning these representations. So people or these and all the entities in the graph, like even Sequoia Capital in that case, becomes a, you know, a representation. So in that sense, it's the idea is very similar, but what GNNs do and is allow for arbitrary structure.

• 15:21Speaker 00

  And that's where I think it's a lot closer to to the human brain, but I, because I don't think the human brain is wired as a linear sequence or as a grid as an images. Yeah.

• 15:32Speaker 02

  Could you say a word about how it works under the hood? Like, how are you able to, let's say you go and work with, I don't know, a food delivery service. How how does it actually work for you to go and kind of, you know, automatically learn this graph representation? And how are you training models on that?

• 15:49Speaker 01

  Or pick YouTube, given people might be watching it there. And we're talking about AI baby shark in history already.

• 15:54Speaker 00

  Exactly. So, um, so there's two pieces to come. Historically, graph learning has been restricted to, I want to say PhDs in graph learning. Because it was, it's not easy to view the world as a graph. People think in terms of relational data. That's the most common data layout in companies. And that's largely because of the analytics revolution that preceded the AI revolution. So everyone thinks in terms of relational data.

• 16:35Speaker 00

  But really relational data and graphs are have a one to one mapping because you have data laid out in tables, usually an entity is a primary key in a table, and then you have all these relationships, primary key, foreign key relationships, which encode the edges in a graph.

• 16:54Speaker 01

  Yep.

• 16:55Speaker 00

  So that that automatic construction from a table layout to a graph layout is one of the innovations inside Co. The other bit is we've invented a language called predictive query language. And the language is allows you to specify any machine learning problem in a few lines. That looks very much like SQL. So think of SQL with the predict clause. So we've created this very simple abstraction layer on top of relational data warehouses. There's already a universe of people who's who are writing SQL queries.

• 17:37Speaker 00

  And we've created a language that appeals or, you know, is one on that resonates with them in some sense. So that's one of the innovations of Kumo. The other one is running these graph neural networks. So once you go from relational to graph, just running graph neural networks at scale. And that again is something that has not been easy to do. There are a few companies in the world that can do it. And it usually takes a huge infrastructure team to build that out. And because graphs inherently, like unlike databases where you can think of some logical partitioning. Graphs, it's all entangled in.

• 18:21Speaker 00

  So how do you split it across different machines with limited memory and so on? So that's, so all of these bits coming together makes Kumo easy to use. But that's it. So when we go to a company, like, you know, a YouTube like company. We'll often talk to a data science team that is looking to get faster. Our return on investment in AI.

• 18:47Speaker 01

  Yep.

• 18:48Speaker 00

  And but then Kumo becomes really easy to do. Because if you have your data laid out as relational tables. A Kumo just sucks it in. So we just, you just specify and, you know, through connectors, uh, tell Kumo what your schema is and then you can just start writing predictive queries. So the graph is abstracted away. But if you have someone like a data scientist who loves tweaking the neural network parameters.

• 19:18Speaker 02

  In case Constantine is.

• 19:19Speaker 00

  Yes, decides to.

• 19:21Speaker 01

  I would be interested.

• 19:21Speaker 00

  Exactly.

• 19:22Speaker 01

  Guilty.

• 19:22Speaker 00

  Exactly. You can look under the hood and the analogy we always use is we'll give you the self-driving car, but if you wanna look under the hood or if you wanna drive stick, we'll let you drive stick.

• 19:36Speaker 01

  So, so concretely in the YouTube example.

• 19:39Speaker 00

  Yeah.

• 19:39Speaker 01

  Historically, if I was an analytics at YouTube and I'm watching this video, I can look and say, hey, query all AI. There would be some, some tagging or some system to understand all AI historically. Let's see what the trends are over time. That's querying the past.

• 19:53Speaker 00

  Yes.

• 19:53Speaker 01

  What you're saying is, once you have this in this database, in this structure, you're able to predict how many people are gonna watch AI videos in the next several weeks.

• 20:01Speaker 00

  Yes.

• 20:01Speaker 01

  How many are gonna watch Baby Shark videos?

• 20:03Speaker 00

  Yeah.

• 20:04Speaker 01

  Uh, how much are they going to spend? What is gonna be their monetization? What are their ads? What else can you do with this?

• 20:10Speaker 00

  So you can say, uh, is this user going to churn, for example?

• 20:14Speaker 01

  Yep.

• 20:14Speaker 00

  Right? And then you can say, what's the most relevant video that I wanna show this user in order to retain them on my platform, right? So I wanna drive value for my business.

• 20:26Speaker 01

  Hmm.

• 20:27Speaker 00

  Uh, given the past videos that they've watched, what's the next, uh, video to watch? And so on. And we can also do demand forecasting. So, uh, we have customers in, uh, in fact, we have, uh, in, in the healthcare sector and they use Kumo to forecast, uh, demand so that they're, they're well stocked on their emergency room.

• 20:51Speaker 01

  Hmm.

• 20:52Speaker 00

  Right? So the applications of using Kumo, uh, go from consumer to healthcare to fintech, where fintech, we see, uh, applications in fraud, for example.

• 21:05Speaker 01

  Mm-hmm.

• 21:06Speaker 00

  Uh, just, uh, is this user's behavior suspicious? Should we flag the user? So on and so forth. So think of any question which says, how much? I love the use of query the future. How much? Uh?

• 21:24Speaker 00

  Is this event going to happen? Uh, is, uh, uh, what's the next best action for this user from an action space? Those are all the kinds of questions that Kumo can help answer. And I love that you, you, you said analyst because Kumo aims to be as auto ML as you want it to be. And you know, we, uh, but we also have a Python interface. So you know, you, you want to be a neural network expert, you can go all in.

• 21:58Speaker 01

  Cool. So it's the brain. It's the brain, it's analytical brain.

• 22:01Speaker 02

  Out of the applications you've discussed just now, I would imagine, you know, there's such classical ML problems that you discuss. Each of them probably has a five person fraud team and a 15 person demand forecasting team. What do those ML people think when, you know, when Kumo is pitching the company? Like, uh, uh, walk me through that spiritual journey. And, and are you actually able to, to, to get results out of the box that are better than a 15 person team maintaining it can, can do?

• 22:30Speaker 01

  I'm okay with it as long as they don't have VC prediction.

• 22:35Speaker 00

  So I, uh, for a lot of the companies we work with, uh, the data scientist is excited about Kumo. As I mentioned, writing those feature engineering pipelines comes with maintenance jobs to maintain those pipelines. And that's not where they want to spend their time. Data scientists in most companies are incentivized with direct business impact. So did I push that ad CTR model out this quarter? Did it drive X percent revenue? So a lot of our customers will come to us and say, you know what? I signed up for X percent revenue. But I'm only one third of the way there.

• 23:20Speaker 00

  Can you guys, you know, uh, help us accelerate? And we do a four-week P O C. So you, you know, we'll, and within four weeks, we'll, uh, almost always, I'm trying to think of a case when we've not shown value, but, and I can't remember one, but we've, uh, we've always shown value within those four weeks.

• 23:43Speaker 02

  Wow.

• 23:43Speaker 00

  Yeah.

• 23:44Speaker 02

  So you convert them into, into believers?

• 23:46Speaker 00

  Into believers, yeah. And it's about where you wanna spend your time, right? And so I think, once they get hands-on product,

• 23:55Speaker 00

  many times, people will come in and say, oh, but feature engineering is where I spend all my time. Right? And how can you say that I don't have to do it manually anymore? But, uh, we'll remind them, we'll remind them of the N L P journey. And then we'll also remind that, uh, once they get ha- hands-on keyboard with the product, and they realize that the journey in Qumu, it's not completely automated away, right? Because we say, a data scientist knows their business well. So if you're going to, uh, define churn prediction for your business, maybe on YouTube, uh, activity around, in the last thirty days is a good predictor of churn.

• 24:38Speaker 00

  So, you know, you wanna bring your events table with a thirty-day window.

• 24:42Speaker 01

  The schema, the actual structure that you expose it to, yeah.

• 24:45Speaker 00

  The window, right? So because these are all queries, and these are all parameters in the queries. Or you could play with ninety-day or three sixty-five-day activities. So these are all queries. You can write five of these queries and say, oh, really? On my system, the best predictor of churn is behavior in a three sixty-five-day window. And I didn't even know that because of spending all my time looking somewhere else. So the data scientist spends lot more time. Finding the relevant tables in their organization that are going to, you know, bring value and then finding those, the right query formulation or the right business formulation. In this case, you know, for example, churn.

• 25:29Speaker 00

  What's the right definition of churn for my business? And once they see that, actually, they really, they realize that this is a lot more fun than what I was doing before.

• 25:38Speaker 02

  Totally.

• 25:39Speaker 00

  Yeah.

• 25:39Speaker 02

  You mentioned tables, structured data, schema. That naturally leads me to think about Snowflake and Databricks. A lot of companies have spent the last five years heavily investing in their data warehouses. How do you work with the data warehouses?

• 25:55Speaker 00

  Okay, that's a great question. So at the outset, we started as a purely SaaS company, you know, emulating a lot of, you know, the principles from the Snowflake architecture, looking at their success stories. One thing we realized is that data scientists, though, they need to see value on their own problem because they're so KPI or business impact focused. Showing them value on a CAGL dataset doesn't really count. So the easiest way to show value is, of course, when they can connect to their own data.

• 26:38Speaker 00

  But connecting to your own data on a SaaS product means you go through a huge security review through the company, which can, in many organizations, can take, you know, a couple of months. So that, we wanted to reduce that friction. And we started partnering with the warehouses to think about deployment models where compute can be closer to the data. And we have a deployment with Snowflake, which is part, we use a combination of what is called Snowpark container services. And really, Qumu can deploy as a container in Snowflake's compute pool. So from a data scientist's point of view, we're also a native app in Snowflake.

• 27:22Speaker 00

  So you, a data scientist in an organization, let's say YouTube, can go in and click install Snowflake. So it's like an app on your iPhone. It gets installed and then they can start writing those predictive queries and looking for value. And oftentimes, the security team is completely okay with it because there's no data leaving the ecosystem. We have a very similar deployment model with Databricks, though in that case, we manage the GPU compute and, but data residency stays completely inside Databricks. And that also led us, so we started that from the point of view of, you know, get, getting,

• 28:05Speaker 00

  letting data scientists get hands on keyboard with, you know, with Qumu quickly. But we also realized that freed us up a lot to not have to think about security, compliance, governance, and let the data warehouses, as they're already building all of the tools and technology for, you know, for management of data, let it stay there, let it be managed there, but Qumu just, you know, talks to data directly sitting inside the warehouse.

• 28:38Speaker 01

  So you talked about relational versus graph data and relational data is kind of how many of our brains have been taught to think.

• 28:44Speaker 00

  Yes.

• 28:46Speaker 01

  We think about things in spreadsheets oftentimes. We might go down and say, if we have a series of AI videos, you have them as rows and then you have some descriptors of them as columns. But really, when you start to see things as graphs, which I did frankly back in the day around your time as a, as a professor.

• 29:07Speaker 01

  I think everything starts, you can start to see everything as a graph.

• 29:10Speaker 00

  Yes.

• 29:10Speaker 01

  It's the most general data type. And when you start to see things as graphs, it's actually kind of how our brain thinks.

• 29:15Speaker 00

  Yes.

• 29:15Speaker 01

  Hey, here's a video and that has a pointed characteristic that's some other part of the graph, which is connected to another part. How do you ingest all of this relational data, which is the way that the world has been run for, the way computers have been run for 50 years, and put them into a graph structure?

• 29:32Speaker 00

  Mm-hmm.

• 29:32Speaker 01

  It sounds like a very heavy lift. And doing that inside of Snowflake and Databricks is probably pretty hard.

• 29:39Speaker 00

  Uh, I want to say that's part of the magic of Kumo, right? And that's, that was the friction that prevented graph learning from taking off and it staying within, and, you know.

• 29:55Speaker 01

  The big companies.

• 29:55Speaker 00

  Yes, exactly. The, the few people who could hire these individuals. But really, it is a question of, we have a unified schema that, you know, that's the graph schema. And of looking at the relational schema, we're able to identify what the entities are. So in that YouTube example, it's a video ID, it's a user ID, it may be a channel ID and so on. And often those are primary keys. And then it's a lot of, I want to say SQL-like code that runs under the hood and it could,

• 30:35Speaker 00

  or I want to say Spark-like code that runs under the hood that converts this data to the graph format.

• 30:42Speaker 01

  I see, I see, it makes sense.

• 30:44Speaker 00

  And, and beyond that, once we get to the graph, there's an edge index. So we store all of the edges in a very, you know, a proprietary and compressed format.

• 30:56Speaker 01

  Hmm.

• 30:57Speaker 00

  And then we distribute out the nodes.

• 31:01Speaker 00

  Okay, because now, because we realized that

• 31:05Speaker 01

  Two GPUs, distribute them to GPUs or to?

• 31:08Speaker 00

  To CPUs, because we wanted to keep costs low. So we keep the, we only reserve the GPUs for training, right? So when we are doing the learning. But we store the edges in what we call the graph engine. And then we have a column store where we store the features. So we can bring in arbitrary features that represent the users, right? So everything about Sonya that we can infer, we're not constrained by memory. That just horizontally scales. Everything on the CPU machine horizontally scales. And we're only using the GPUs for message passing.

• 31:49Speaker 01

  Cool.

• 31:50Speaker 00

  Yeah.

• 31:50Speaker 01

  Amazing, in fact.

• 31:52Speaker 00

  Yeah, so that, that, that has also reduced costs for our customers. And they're often surprised that we can run graph learning at the scale that we do, at the costs that we do.

• 32:07Speaker 02

  You've made the comparison to large language models a couple of times.

• 32:10Speaker 00

  Yeah.

• 32:10Speaker 02

  I'd be remiss not to ask, like, what are the connections between your graph world and the LLM world? And are there, you know, are there synergies between the two?

• 32:19Speaker 00

  Absolutely. What a great question. And so many synergies. So let's take the example of this podcast, which will get generated. It's going to get transcribed by an LLM. It's going to, it's, so you have all of the summaries. You have all of the semantic information that will come from the large language models, right? A graph neural network can actually take all of those features that

• 32:53Speaker 00

  You know, the semantic representation that is inferred for this particular video as a node feature. And what the GNN is learning is it's learning across all of the interactions that one may. Now, let me give you another example, and this, we have a demo of this up on our website or on our LinkedIn channel. But an example would be, a lot of people think of the LLM revolution as creating chatbots. Okay, so let's say you come to a clothing store and you are searching for yellow summer dresses. So you search yellow summer dresses.

• 33:37Speaker 01

  All the time.

• 33:38Speaker 00

  Yes, and you're not a logged in user. And the LLM is going to probably get you a really good set of things that look like yellow summer dresses. But if you were a logged in user, and we knew all of that information about the kind of interactions that you'd had in the past, we can actually use Kumo's predictions to

• 34:07Speaker 00

  inform the LLM. So think of RAG and think of Kumo predictions as feeding a RAG algorithm to ground its truth to be closer to what is personalized. So you can do that as well. So that is the bringing in features, but then there is also, they're complementary, because Kumo brings you all of that personalization based on all that behavioral data that the app has, which the LLM doesn't take into consideration.

• 34:41Speaker 01

  You mentioned RAG, and we've talked about graphs. Graph RAG is having a moment.

• 34:44Speaker 00

  Yes.

• 34:45Speaker 01

  In AI right now in general.

• 34:47Speaker 00

  Yes.

• 34:48Speaker 01

  Thoughts on Graph RAG, which is different from our, from our approach at Kumo, but thoughts on Graph RAG, and then also how it differs from using a graph neural network to do certain inferences tied to some sort of RAG?

• 35:01Speaker 00

  Yeah, so Graph RAG is a lot closer to what we just talked about.

• 35:06Speaker 01

  Yep.

• 35:07Speaker 00

  But many organizations may have knowledge graphs, and that's another entire field of study in graphs. Think medical domains, for example. You have all of your.

• 35:22Speaker 00

  Insurance codes and how the insurance codes connect with each other. You have symptoms, you have all of that. There are, there's a lot of knowledge bases sitting out there with interconnected nodes. Graphrag allows you to ground your LLM output in the, the answers that are answered from these kind of knowledge graphs, right? So instead of going to a search and index. So you can think of RAG as going to a search index, a knowledge graph, a recommender system like Qumu and bring, making the LLM output more, or hallucinate less.

• 36:07Speaker 02

  I want to ask about explainable AI. One of the things we've been discussing in prior episodes of the show is, you know, these LLMs, will we ever be able to understand how they think? And I remember the anthropic results were really interesting. How do you think about explainability as it comes to Qumu's models?

• 36:25Speaker 00

  That's such a great question, Sonia. Because for the kinds of problems we work with and the customers that we work with, this was another area we had to actually develop a solution. Because we have customers in insurance and healthcare and they often need to understand why a recommended output was recommended to them.

• 36:47Speaker 02

  Yep.

• 36:47Speaker 00

  You want to know that we didn't, the model didn't over rotate on, you know, race, color, ethnicity, and so on and so forth, right? And so it became table stakes for us to actually solve this problem. And at Qumu, we've innovated by actually developing an algorithm, which after the, the training part of the algorithm, looks at the graph and looks at the gradient algorithm and can come down at the table level to say, these were the tables that were used. These are the columns that were used.

• 37:30Speaker 00

  And we have some early results that show that we can even come at an instance level and predict, you know, here's an instance, the score, why Sonia was recommended that video is high because of these specific features.

• 37:47Speaker 01

  Wow.

• 37:48Speaker 00

  So it was stable stakes, just given the domain we were going into. Yeah. Yeah.

• 37:54Speaker 01

  So we talked a lot about AI and graph learning. You have been in the AI space for a long time, Hema. Can you tell us a little bit about what you developed at LinkedIn? And specifically, what AI growth was at LinkedIn? Maybe if you can, how graph neural networks helped there?

• 38:10Speaker 00

  Yeah.

• 38:10Speaker 01

  And the types of challenges that you dealt with really, really large scale operationalizing AI?

• 38:15Speaker 00

  Yeah, that's a great question. So I joined LinkedIn just a couple of years after the IPO. And AI was making its way into various products.

• 38:29Speaker 00

  I joined the the growth team and the team I led first led was the People You May Know team. And People You May Know is all about graphs. It's about large scale graphs and about.

• 38:47Speaker 00

  And the amazing thing about LinkedIn was how closely tied People You May Know, because it's a social network, was to our core consumer metrics. So I was, I had come to LinkedIn as an AI researcher and I suddenly found myself responsible for one of its core KPIs, which was sessions and monthly active users. And by then I'd also started owning notifications, which was a huge part of the growth ecosystem at LinkedIn.

• 39:27Speaker 00

  And along the way, we had to start operationalizing AI. And before ML Ops became a word, we were actually thinking about, hey, how do you?

• 39:43Speaker 00

  From when the, from the time when you deploy a model, how do you measure? How do you A B test? And then how do you maintain a model in production?

• 39:51Speaker 01

  Mm hmm.

• 39:51Speaker 00

  We would see models degrade in production. We would see our, our.

• 39:57Speaker 01

  Why is that, by the way? Why did you see that so frequently? If the, if the graph wasn't losing nodes or edges, why would it degrade over time?

• 40:07Speaker 00

  Because depending on your business problem and the kind of.

• 40:11Speaker 01

  Behavior change.

• 40:12Speaker 00

  Yeah, behavior will change, right? Uh, people behave on LinkedIn in the new year very differently uh from summer break, right? So creating those pipelines which do auto-training.

• 40:26Speaker 01

  Yeah.

• 40:27Speaker 00

  Those all, those all became very important. And then when you talk about scale, that was an interesting problem as well because I started at LinkedIn when we were, I think, about four hundred million members and then it, it was rapidly growing. So that's when we had to start thinking about infrastructure and the fact that uh the CPU-based algorithms, uh, we can't just keep horizontally scaling them. So what would be more efficient ways to run AI models in production?

• 41:04Speaker 00

  Graph neural networks now at LinkedIn, and I want to say there's been an amazing team that took it forward after I left as well. It took them about four to five years to build and many, many, many engineers and, but now it powers everything from the ads to the feed to uh, uh, jobs and so on. There's, they, they published a paper, yeah, recently about it.

• 41:26Speaker 01

  So you guys, the founders, there's three founders at Kumo, you were senior AI leads at, at LinkedIn, at Airbnb, at Pinterest. All those are massive scale, also really sophisticated.

• 41:39Speaker 00

  Yeah.

• 41:40Speaker 01

  Uh, that can be, I think, intimidating for smaller companies that have problems and say, wait, this is a champagne problem. This is what the hyperscalers with hundreds of millions of users have, and we don't have nearly the same problem. Is that true?

• 41:53Speaker 00

  Mm hmm.

• 41:53Speaker 01

  And what kinds of companies are not a good fit for graph learning?

• 41:57Speaker 00

  That's a great question. So um, I think.

• 42:01Speaker 00

  There are two things. The first question is the reason why we ended up inventing predictive query language. Because what we needed to do was create a platform that was super easy to use, right? And many of the other companies had few data scientists and large number of potential avenues where they wanted to bring AI. So oftentimes, what we would get is, hey, we would love to be a LinkedIn, an Airbnb, or a Pinterest, but we can't put so many people to it. So giving them that easy-to-use interface and giving them that managed infrastructure at scale actually lets us get in.

• 42:47Speaker 00

  But that's it. When is Qumu not a fit? Qumu's not a fit if you're so early on that you haven't figured out your data landscape. So sometimes we'll talk to customers who are super excited about Qumu, but they haven't figured out how to measure the value of AI. Or sometimes we'll talk to customers and they're still in spreadsheets and they're moving to one of the warehouses. So we'll say, you know, get the data layout settled. It's like building a city, right? You've got to have your roads in the foundation first and then the vehicles come on it. So we'll wait and customers come back, you know, in a year or so.

• 43:31Speaker 01

  But there's no category or type of problem. It's more a data sophistication.

• 43:35Speaker 00

  Exactly.

• 43:35Speaker 01

  I see.

• 43:36Speaker 00

  But you don't have to be so far along the sophistication curve. You don't have to be an Airbnb, LinkedIn.

• 43:41Speaker 01

  Yeah, it's table stakes. You know certain KPIs that can be optimized by an algorithm.

• 43:46Speaker 00

  Yeah.

• 43:47Speaker 01

  You can quantify certain things, A. And then, B, you have access to that data and something that you can plug into like a data warehouse.

• 43:54Speaker 00

  Yeah. And the way I look at the evolution of an organization in data is often that they'll

• 44:02Speaker 00

  You of course have to know what your product market fit is. After that, you start figuring out your data ecosystem. You build your data ecosystem for analytics because now you've built a product. You've got to start measuring what that product is doing, what it's studying, what the behavior is. And that's when leaders usually start thinking about AI. Which is, okay, now I know how to query the past. But I now need to start bringing in AI to instrument the change that I need in the ecosystem.

• 44:39Speaker 02

  I'd love to close with some questions about your vision for the future. Maybe, you know, you've been in AI for a long time. You mentioned you were in NLP before Bert was a thing. What are you most excited about in AI most broadly?

• 44:53Speaker 00

  Oh, I think I'm most excited very broadly about the productivity gains it's giving all of us. I mean, Kumo is one part of it, but just how we write documents or how we. Or think about health, right? Like if health improves, productivity improves. For example, if you're just using one of those health apps that do monitoring, but not you to for behavioral change. That is better health is better productivity. So what I'm most excited about is how we're going to evolve as a human race with all of this productivity gains.

• 45:36Speaker 01

  What about technically? What what features or approaches or algorithms or venues you think are going to be most interesting?

• 45:44Speaker 00

  I've always found the big innovations come at the intersection of hardware and software. And I think while GPUs were invented for graphics. There's probably something more that has to happen on the, you know, processor side. So that you can scale these graph neural networks or neural networks further. Make models maybe less expensive.

• 46:16Speaker 00

  So I'm looking forward to that technically.

• 46:20Speaker 02

  What about your vision for the future of Kumo? Like, what can we expect to come out of the product in the in the future?

• 46:26Speaker 00

  So in terms of Kumo's vision, I'm actually really excited about the kinds of apps people are going to build with Kumo. We're starting to see people plumb Kumo with Langchain and Pinecone and put together apps like the one we talked about, like the, you know, a chat agent that recommends for you the the yellow summer dresses, right? So I'm very excited about the the top layer of applications that are going to get built on top of Kumo and what that's going to power.

• 47:04Speaker 01

  So, Hema, one of the star features about you, you're incredibly technically deep, but you also are really good at culture. If you talk to anyone at Kumo, there's basically no regrettable turn ever. And you guys hire some of the best PhDs in the world in ML and certainly in GraphML. How do you do that? What have you done to make the Kumo culture exceptional and to have so much retention within your team?

• 47:26Speaker 00

  I was at a leadership training once and we had to think about what our true north was. And I realized that, and the true north concept defines your true north, which is a value as who you are, but what, you know, the kinds of problems you solve either in your work and what you bring to the table. And for me, it's always about empowering people to do more than what they think they can. So it's common to empower people to do what they, to get to full potential. But it's those aha moments, like, wow, I built this. So you hire a smart team, you get them.

• 48:07Speaker 00

  I think good managers step away, but have their eye on how the team is operating. And you get people to innovate, get people to own what they're building. So just see that vision. So you want to be able to hire people, whether it was LinkedIn, where the value was economic opportunity, or at Kumo, where the value is about

• 48:33Speaker 00

  building an AI platform that makes AI so easy to use. It's about when you bring smart people together that rally together on the same value, that's when magic happens. And my job is to just let the magic happen.

• 48:48Speaker 01

  Hey Mo, why is that passion around relational data? You could have, you could have done, we've talked about this sometimes in the past. You could have used graph, graph learning as a different type of architecture to do language models, or you could have done graph learning to do any sort of, any sort of AI. Right? Once you've figured it out at scale, how to do the generalization, why can't you do the specifics? By taking this big marble block and carving away all the nodes and edges until you get to a superior architecture? But you decided to do it on relational data. Why is that? Relational data usually is not the most exciting thing in the world for most people.

• 49:22Speaker 00

  Yeah.

• 49:23Speaker 01

  But it is your life passion.

• 49:25Speaker 00

  Because nobody else was doing it. And there's so much data in relational format. And that was such a pain in our past jobs. So I, I feel like there's magic happening in the core area of NLP, where, you know, I'm happy to see that revolution and all the investment that's happening there, people, money, and so on. But there's this whole workload that's out there, a whole set of data scientists that work on those workloads. How do we bring that magic to them? So it's really about the opportunity and the past pain that each of us saw in our previous jobs.

• 50:11Speaker 02

  Okay, I have one last question. For the young Constantines out there who are watching this episode on YouTube, what advice do you have for, you know, aspiring AI engineers who want to really make a dent in the field in the future?

• 50:24Speaker 01

  She has a specific yellow dress recommendation.

• 50:28Speaker 00

  I would actually say tools come and go, languages come and go. I know there's a lot about learning Python and, you know, taking the class on the latest deep learning. But I would say don't skip your probability and linear algebra classes. Because when it, whatever method has been there in the last several decades, it's always come down to core linear algebra and probability. So don't skip those classes.

• 50:57Speaker 01

  Okay. And mine is, there's a lot of graph enthusiasts out there.

• 51:02Speaker 00

  Yeah.

• 51:02Speaker 01

  When do graph neural networks come main stage in the AI revolution? Best guess for timeline?

• 51:09Speaker 00

  I think we're getting there. I was at a conference called KDD recently. It's one of the biggest data mining conferences, a lot of academics, a lot of industry folks. And more than half the papers were graph neural networks. So I think we're sitting at that explosion. It's, it's going to happen. Thank you, Hema. This was fantastic. Thank you, Sonia, and thank you, Konstantin. It was lovely being here.