Operationalizing ML/AI with MemSQL

A lot of effort is put into the training of AI models, but, for those of us that actually want to run AI models in production, performance and scaling quickly become blockers. Nikita from MemSQL joins us to talk about how people are integrating ML/AI inference at scale into existing SQL-based workflows. He also touches on how model features and raw files can be managed and integrated with distributed databases. Sponsors: DigitalOcean https://do.co/changelog – DigitalOcean’s developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99% uptime SLA, and 24/7/365 world-class support to back that up. Get your $100 credit at do.co/changelog https://do.co/changelog. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Featuring: Nikita Shamgunov – Website https://neon.tech/, X https://x.com/nikitabaseDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: MemSQL https://www.memsql.com/MemSQL’s ML/AI capabilities https://www.memsql.com/solutions/predictive-ml-ai/MemSQL’s recent AI/ML e-book https://www.memsql.com/resources/memsql-achieving-successful-operation-of-ml-and-aiContact tracing case study with MemSQL and True Digital https://www.memsql.com/blog/case-study-true-digital-group-helps-to-flatten-the-curve-with-memsql/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at ChangeLaw because of Rollbar. Check them out at rollbar.com. And we're hosted on Linode cloud servers. Head to linode.com/changelog.

• 0:17Speaker 00

  This episode is brought to you by Digital Ocean. Digital Ocean's developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, TM accounts, worldwide availability with a 99.99 uptime SLA and 24/7 365 world-class support to back that up. Digital Ocean makes it easy to deploy, scale, store, secure, and monitor your cloud environments. Head to do.co/changelog to get started with a $100 credit. Again, do.co/changelog.

• 0:58Speaker 01

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.

• 1:24Daniel Whitenack?

  Welcome to Practical AI. This is Daniel Whitenack. I'm a data scientist with S I L International. Normally I'd be joined by my co-host Chris Benson, who is a Principal AI Strategist at Lockheed Martin, but he's in the midst of some family health-related things. So he's taking the time that he needs, but we're definitely excited to chat about a really interesting topic today. Actually, in our Slack channel, I remember some conversation a couple of weeks ago where we were discussing the issue of, hey, I trained my model. It works great on my data. I evaluate and it all seems good. But then when I try to integrate this into code, the performance is actually really terrible.

• 2:08Daniel Whitenack?

  And it's kind of a mismatch between production things. And I think that we're going to be able to get into some of those things today. Today we have as our guest Nikita Shamgunov, who's the CTO of MemSQL. Really excited to talk to you today. Nikita, welcome.

• 2:24Speaker 03

  Happy to be here. Like Daniel said, my name is Nikita. Actually, I'm co-CEO and founder of MemSQL. I don't mind the confusion. I started as the CTO and I took over as CEO in 2017.

• 2:37Daniel Whitenack?

  Gotcha.

• 2:38Speaker 03

  And recently, about a year ago, brought a co-CEO Raj Verma with the thinking that we're going to take the company public.

• 2:46Daniel Whitenack?

  Ah, gotcha. On that note, why don't you give a little bit of maybe the background first of yourself? And then we can get into maybe a little bit of the background of MemSQL. I think that would be great context.

• 2:58Speaker 03

  Definitely. So I've spent my career in data management and databases specifically. I came to the States, to the United States after finishing my grad school at St. Petersburg, in St. Petersburg, Russia, and joined the SQL Server Engine team. So went from kind of very research-oriented life and work to basically system engineering, right? When you build databases, it's actually a very different cadence versus using databases. When you use databases, you think about things like performance, you think about SQL as the API to the database, and then you think about reliability and uptime. And when you build a database, you think about quality.

• 3:41Speaker 03

  You think about the life of somebody who is using the database and how you make that lives easier. And obviously, you think about performance and scalability and how the database user or developer can achieve that performance and scalability.

• 3:56Daniel Whitenack?

  It sounds like an interesting transition from the sort of academic world to the systems engineering world. Was it a hard shift for you or was that sort of focus on the user and reliability? Was that something that you were already kind of passionate about going into that work?

• 4:12Speaker 03

  I was very passionate about engineering in general. What I loved about building databases is that the product, the database engine, is like a computer science in a box. It has algorithms, it has data structures, it has system engineering, you interface with networking, I O, CPU caches, you need to be aware of the computer architecture in order to build, you know, world-class software. So that certainly resonated a lot. And that was the core premise of why I wanted to start working on, you know, world-class industry products. And then from there, the passion to the user came in and over time, just that curiosity about, you know, building new things and breaking ground and entrepreneurship

• 4:57Speaker 03

  came through the years while, you know, working at SQL Server. Mind you, during that time, Microsoft was going through a cloud transition. Everything that we're, you know, we're seeing today at scale, at that time, all of that stuff was being born, being conceived and major architectural choices were made. Some of them were right, some of them were not right. So that's my kind of a big company background. And then I switched and joined Facebook. And in fact, one of the premises of me joining Facebook was not to make a lot of money or because, you know, that was a pre-IPO, you know, 2010, right? Actually moved to the Silicon Valley and meet the kind of people I will later start a company with.

• 5:41Speaker 03

  And what happened is that as I walked into Facebook on day one, I met my future co-founder, Eric. And relatively shortly after, just within six to eight months, we started Meme SQL and I left Facebook.

• 5:56Daniel Whitenack?

  Oh, that's a wild ride, I guess. Moving to a total new place, experiencing, you know, Facebook and that culture, especially at that sort of stage and then founding something. So maybe describe a bit how like that happened so fast. The idea from Meme SQL and the sort of motivation that this was something that was really needed. How did that occur?

• 6:16Speaker 03

  Yeah, so distributed systems at SQL Server, we always knew that was the future. Right? Especially as you go into the cloud transition. And the time, you know, you know, back in like 2008, 2010, Microsoft had a flagship product, SQL Server, which is a single node database system, very, very good, very, very powerful. Really proud I worked on this one. The main competitor, Oracle, had distributed systems at its disposal. Oracle Exadata and Oracle RAC. And the way the database market is structured is that the top tier workloads that have high performance requirements, high availability requirements, do require distributed systems and Microsoft didn't have that.

• 6:59Speaker 03

  And so that top of the market, Microsoft was losing to Oracle then. Actually, I think they mostly caught up right now. But then architecturally, single node databases are very hard to change and may turn them into a distributed system. That was some of the, you know, kind of moonshot projects that my co-founder and CTO Adam worked on at Microsoft. And that moonshot project didn't succeed. When I walked into Facebook, the need for distributed systems became apparent. Because every Facebook workload is that high level, high end workload. And sometimes it's from the reliability standpoint.

• 7:41Speaker 03

  Sometimes it's from the scale standpoint, most of the time the scale standpoint. Because, you know, back in 2010, I think Facebook was on the march to cross a billion active users. So that was on everyone's mind, that was everyone's goal. How can we cross a billion active users? And obviously, you know, history shows that Facebook had blown through that goal quite successfully.

• 8:02Daniel Whitenack?

  Were they trying to architect something internally to deal with that? Or was it sort of an open problem when you were there?

• 8:10Speaker 03

  It's more than one system, right? For something that is so big, right? As Facebook, turns out that all the data workloads are split into categories. So some of them are data lakes and, you know, Hadoop basically got a lot of advancements at Facebook. Some of them are operational, you know, powering Facebook.com and multiple data management technologies are on the critical path between, you know, typing Facebook.com and actually seeing the news feed. There's a separate data management solution for messaging, separate news feed, and this goes on. And within that, also there's a whole bunch of kind of point solutions for various analytical workloads, right? One is for time series.

• 8:54Speaker 03

  In fact, there's a startup called Signal Effects, who took some of the ideas and then the folks from Facebook left, started that company that was recently acquired by Splunk.

• 9:03Daniel Whitenack?

  I got you.

• 9:04Speaker 03

  And then there was a system called Scuba that gives you real time analytics and a lot of ideas there influence and sequel roadmap as well. So, long story short. Lots and lots of data management systems and data management workloads inside Facebook. But each and every one is a distributed system. And so that pervasiveness of distributed system, we're captivating. It like really validated the thinking that the future of database systems are is distributed. And that's how we started MemSQL.

• 9:35Daniel Whitenack?

  Awesome.

• 9:36Speaker 03

  And we started this as a memory, hence the name.

• 9:38Daniel Whitenack?

  Yeah.

• 9:39Speaker 03

  Now, in fact, the name is kind of limiting because MemSQL has evolved way past being in memory and memory only. It's the version one that was in memory and single node. But very quickly, we expanded to distributed system, built tiered architecture from memory to disk. And now we expanded it to S3 or, you know, other object stores.

• 10:01Daniel Whitenack?

  Yeah, I'd be interested to hear a little bit about, I mean, you kind of gave a sense of the initial founding and some of the initial ideas. I'd be curious, as far as right now with MemSQL, could you just give a sort of high level view of the sorts of things that people are turning to MemSQL for? Kind of the consistent things that you see really people getting value out of? And then maybe some of the newer things that are enabling maybe new sorts of workloads that you didn't even anticipate in those early days?

• 10:29Speaker 03

  Definitely. So first of all, databases are very long game.

• 10:34Daniel Whitenack?

  Yeah.

• 10:35Speaker 03

  And the most successful database products on the planets, which are Postgres, MySQL, SQL Server and Oracle, are all 30 plus years old.

• 10:44Daniel Whitenack?

  Yeah.

• 10:44Speaker 03

  And we still use it today, which is basically if you turn into any other piece of technology, that's not the case.

• 10:51Daniel Whitenack?

  Yeah.

• 10:51Speaker 03

  Right? Technology is very transient, right? We're building something and something new comes in and completely disrupts what's there before. But database seems to stay for a long time.

• 11:02Daniel Whitenack?

  Yeah, I think in my experience from working at the different places, I've encountered Postgres a lot. I've encountered, you know, right now I'm working on a team that's using SQL Server for certain things. Of course, I've encountered certain things like, you know, Mongo or other databases that.

• 11:21Daniel Whitenack?

  Or you know, like the NoSQL or those sorts of databases. But I think like you were talking about the user experience. It always seemed like to me the the natural user experience. And you gain a lot of power with that SQL interface to the database.

• 11:35Speaker 03

  The relational, yeah, for sure.

• 11:37Daniel Whitenack?

  Yeah.

• 11:37Speaker 03

  And so, yeah, so the vision is a single pane of glass to all your data and all your workloads. But when you start peeling the layers and understanding what would it take to deliver on that vision? You start understanding how you scale storage, how you scale compute, and how you scale both storage and compute for your low latency operational workloads. Think about powering your apps and, you know, loading your webpage. And, you know, the results need to come back to you and ideally sub hundred milliseconds to running what is called big expensive analytical queries that scan large volumes of data to give you insights.

• 12:17Speaker 03

  Those insights could be reports. Those insights could be information, analytical information, which is also called decision support, right? You need to make a decision. So you need to know what works, what doesn't. You need to know, you know, how your sales are doing in this state versus the other state, this product versus that product. And so that is a continuous process of evaluating and looking at data and understanding, driving insights out of data. So the interesting piece about what I just described is that for your operational needs, you need a SQL database, right? Like Postgres, like MySQL, like SQL Server. I mean, you don't need it.

• 13:01Speaker 03

  I mean, you can use a MongoDB, you can use a NoSQL database. But you need an operational database, right? Let's just put it this way. And for the majority of workload today, people are using relational databases that that speaks SQL. And for a smaller part of the market, they use NoSQL databases. Which is more preference and user experience and whatnot. And then for an analytical system, uh, people use data warehouses, Teradata, Snowflake, BigQuery. And the interface to those databases is also SQL.

• 13:34Daniel Whitenack?

  Yeah.

• 13:35Speaker 03

  And what you just started the podcast with is like, oh, I trained my model against the data that sits in the data lake or a data warehouse. Uh, and now I need to put it in production and my, you know, I have data quality, data consistency issues, my performance is not the same. A lot of that comes from the underlying data management. And if you like really peel the layers, it comes from the fact that you run this very same model on top of data and data management systems that are different.

• 14:04Daniel Whitenack?

  Yeah.

• 14:04Speaker 03

  And one can argue that, well, they're different, there's they got their reasons for them to be different. But there's a more contrarian viewpoint here that is, we live in the world of clouds where things are abstracted away from you. And that gives an opportunity to build ideally a serverless interface that speaks SQL. And that gives you access to all your data and gives you access to all your data for reporting capabilities, for low latency capabilities, for operational workloads. And that would allow you to never leave your data universe as you go and move from one workload to another.

• 14:48Speaker 03

  And that can be huge, right? That can be huge. Because whatever data you trained, for example, your example, whatever data you use to train the model lives in that ocean of data and that data is easily accessible to you. And then you train the model. So now you need to convert new data that is incoming or, you know, marry new data to an old data and convert it into pixels, which is your app or a website or whatever. And that can be done right there off of the same data set you've been operating on. Which is certainly not the say the case today. Today you have a data lake, a data warehouse and a number of operational databases

• 15:28Speaker 03

  that can be integrated by a third piece of software like ETL tools or integration tools, which just generates a lot of complexity. And a lot of that can be simplified if you imagine a world of having a serverless SQL low latency API to all your data.

• 15:48Speaker 03

  That's the vision where we driving towards. And this is a multi-year, probably multi-decade. MySQL is 90 years old today. So it's going to be a multi-decade kind of life's work. But the workloads that we see emerging and the new workloads that are enabled by a system like this are real-time analytics and real-time decision support. When you need to kind of go back and look at the history of what was happening to make a real-time decision and do it at scale as well. So that is something that we see a lot in financial markets. You know, MySQL is a give or take $40 million run rate company with 70% growth. We just had an article on TechCrunch where revealed our numbers.

• 16:31Speaker 03

  And a good amount of that revenue is coming from financial markets. And if you think about it, like, that's what happens there, right? There's constant stream of information that's coming in modifying that data state that you have, and you need to make decisions about, you know, buy, sell. You need to make decision of, you know, in wealth management. You need to make decisions in portfolio management and trading. But also you need to make decisions in various systems that, for example, monitor, you know, something that's very, very large, right? So in Morgan Stanley, for example, it's a, it's a trading system. And we monitor this trading system, providing decision support to, oh, should we provide

• 17:16Speaker 03

  some sort of maintenance? Should we reroute our trades? You know, all of those things. That's what MySQL is used for today. And that's new. Like, we didn't have a system on the market that had those capabilities before.

• 17:38Speaker 01

  I'm Jared Santo, JS Party's producer and one of nine regular voices you'll hear on the show. We are a party-themed podcast, so fun is at the heart of every episode. One way we keep things fun is by mixing it up and trying new things. We play games like JS Jeopardy.

• 17:53Speaker 04

  This gives you access to an outer function scope from inside an inner function.

• 17:58Speaker 01

  Oh, I think that.

• 17:59Speaker 05

  Global scope?

• 18:02Speaker 04

  Incorrect.

• 18:03Speaker 03

  Yeah, I didn't think so.

• 18:05Speaker 01

  Debate hot topics like should websites work without JS?

• 18:08Daniel Whitenack?

  I'm going to appeal to authority and read some quotes at this time. Okay.

• 18:14Speaker 01

  I've lost complete control of this panel. Go ahead, Ross.

• 18:17Daniel Whitenack?

  The first quote, "No code is faster than code."

• 18:20Speaker 01

  Discuss and analyze the news.

• 18:22Speaker 06

  Yeah, this reminds me of when you're playing Pokémon and you have like, you know, an electric Pokémon versus a water Pokémon and you try like an attack.

• 18:30Speaker 01

  Share wisdom we've collected over the years.

• 18:33Speaker 05

  To be honest, a lot of what we rely on is pretty garbage.

• 18:41Speaker 05

  And like, I mean, I wrote some of it, so it's okay. Like, I can say this.

• 18:45Speaker 01

  Interview amazing devs like John Rezeg and Emilio Wattenberger and a whole lot more. Oh, and did I mention we record the show live? We do. You can be part of the hijinks each and every Thursday at changeodd.com/live. This is JS Party. Please listen to a recent episode that piques your interest and subscribe today. We'd love to have you with us.

• 19:12Daniel Whitenack?

  I definitely liked where you're going in terms of describing the sort of single window to all of your data via the SQL interface. And I know that we talked a little bit, so we kind of touched on the AI and machine learning elements of this and how they fit in. You're talking about going on this journey to create the single window of, single interface to all of your data. How did AI and machine learning workloads start to cross your path at MemSQL and start to be something that you felt like needed to be part of the strategy of how you were building out the system?

• 19:50Speaker 03

  Yeah, this is a great question. When we did our analysis, we discovered that about 20 to 30% of all the workloads that MemSQL support have a, some sort of machine learning or AI angle to this.

• 20:06Daniel Whitenack?

  Oh, wow.

• 20:07Speaker 03

  So this is a very large number. And when we looked at it, you know, we always wanted to have AI, like dedicated AI capabilities in the system. And we certainly use AI internally to make certain decisions around workload management, query optimization. But the fact that the modern workloads, and obviously people put modern workloads on MemSQL.

• 20:28Speaker 03

  Have a lot of AI and ML capabilities was eye opening to us.

• 20:34Daniel Whitenack?

  Yeah, in those cases that you noticed, was it like people that were like you were saying, using MemSQL to do kind of large queries to prepare their training data for an AI model?

• 20:45Speaker 03

  Actually, both.

• 20:46Daniel Whitenack?

  Both? Okay.

• 20:47Speaker 03

  And like two specific examples. Well, we have a great integration with Spark.

• 20:53Daniel Whitenack?

  Uh huh.

• 20:53Speaker 03

  So we have a Spark, MemSQL Spark connector that allows, gives you very fast data exchange between MemSQL and Spark. You know, fast mean multi cluster to cluster. So like a multi-channel bus between the two. We noticed that people put all their data into MemSQL, grab it through Spark, store model somewhere else. So we don't take part in hosting models. So this is the first part. And what people like about MemSQL is that two-way path for data exchange between MemSQL and Spark. If you have something in Spark, you can persist it in MemSQL. You have something in MemSQL, you can pull it into Spark. And MemSQL is a world-class query processing engine. So you can send SQL query to it

• 21:37Speaker 03

  to do the kind of the first pass and slice and dice data before it gets fed into training algorithms. Which MemSQL itself doesn't support. It's just the backbone for that data. And the second use case that we started to see being pronounced is, well, people build apps on top of MemSQL. And those apps have, you know, models, evaluate models real time. And they need to, usually there's some sort of an SLA for an app either displaying this information to the end user, or the app is completely kind of back office and they just, you know, crunching data. And for that, they need to pull data from somewhere, run this data against a model.

• 22:21Speaker 03

  And based on the results that you see from their model, you know, do something. Typical examples, fraud.

• 22:29Speaker 03

  And we do in transaction fraud detection for some of the major banks. Where the SLA is forty milliseconds to make a decision if that particular transaction is fraudulent or not. And in order to make that decision, you need to go, you know, you have a model, that's model is already trained. Then you need to grab some data for that specific account, go back and look at the previous, you know, thousand transactions. Feed those transactions against the model. And then the model will tell you it's a fraudulent transaction or not. So MemSQL is supporting use cases like this.

• 23:05Daniel Whitenack?

  Yeah, that's really interesting.

• 23:07Speaker 03

  Right? So again, like both sides of the spectrum, right? Both just providing basically a data lake or data warehouse capabilities. Put all your data in one place, let data scientists play with that data and use whatever data science tools, the tools du jour, right? Should it be Spark?

• 23:24Daniel Whitenack?

  Most talks SQL.

• 23:25Speaker 03

  Yeah, yeah, yeah. Should it be Spark? Should it be Pandas? Should it be TensorFlow or PyTorch, whatever? We provide very, very fast data exchange to whatever frameworks you use. And the second one is, oh, I want to put my model into production. So I'm going to register that model somewhere in my, you know, either in Kubernetes, SageMaker, you know, there are tools for that now. And it's a rapidly evolving space. But it all starts with data anyway. So you need to have a data backbone and you need to have data management system with system of record capabilities. In order to provide, you know, uptime, low latency, all of those things. And where it's going is we're thinking to keep building world-class integrations.

• 24:09Speaker 03

  With systems that both data scientists use for training and engineers use for putting models into production. To enable that exchange from a push-button standpoint. Given you have a model, put that model somewhere, tell MemSQL about that model. And you'll be able to consume that model either from SQL, through user-defined functions. Or through an application, query the model, query the data, you know, and the application provides the glue.

• 24:37Daniel Whitenack?

  Okay, so yeah, I was curious about that piece. So it sounds like right now the sort of workflow is you have an application like a Python application or whatever it is, you load your serialized model into memory. And then when it's time to fulfill a user request, then you make a SQL query against MemSQL, get the data you need, run it through your model and respond to the user. Is that about right?

• 25:03Speaker 03

  Yeah, that's how it works today. And where it's heading is this will still be a probably 50% of the use cases because certain things you still want to control and write very, very custom logic. But we want to make MemSQL aware of models that are stored in a particular repository and being able to through SQL to run data through those models and return results back into MemSQL.

• 25:28Daniel Whitenack?

  Yeah, that's really interesting.

• 25:29Speaker 03

  Yeah, the reason that's useful is that sometimes you want to run that model against a very large volume of data. And so if your application row by row pulls data from a database, rounds it against the model, gets the results, potentially stores it back in the database, that is an extremely inefficient way. But what you can do is you can establish a similarly to Spark Connector, a multi-channel bus with optimized data format. You know, we're thinking Apache Arrow or something like this, where running a model against a billion records should be, you know, one or two second proposition.

• 26:11Daniel Whitenack?

  Yeah, that's awesome. I'm thinking of like the use, like a facial recognition use case or something like that where like you may want to run like compare the embedded representation of this image against like, you know, thousands and thousands and thousands, or maybe even millions of records that you have in your database that are reference, you know, faces from your facial recognition or something like that. Am I following the right sort of path here?

• 26:40Speaker 03

  You are, and we have use cases like this where people store feature vectors in the database in a way people run this use case in MemSQL from the do it yourself kind of way. Meme SQL supports vector tensor operations.

• 26:55Daniel Whitenack?

  Okay.

• 26:55Speaker 03

  As a built in. And obviously, if you, uh, facial recognition models, not all the time, but often is represented as, you know, kind of a tensor flow. DAG, I would say that evaluates and, and the individual nodes in the DAG are vector math. Right? They're not something that's spectacularly complex. It's a vector dot product, uh, also known, uh, as a scalar vector multiplication, right? So Meme SQL does that. And we have customers that in production do facial recognition over millions of faces. To enable things like, you know, somebody walks into a supermarket and they want to custom tailor the experience for that person.

• 27:39Speaker 03

  Or security systems in airports. And what happens is there's a camera, the camera looks at the next face. The feature, custom logic extracts a feature vector out of the new face. And then you run a query against Meme SQL that says, give me all the records where vector dot product of feature vectors stored in the database. Multiplied by the vector you just received is, uh, you know, between point nine and one.

• 28:07Daniel Whitenack?

  Yeah.

• 28:07Speaker 03

  And that gives you all the similar faces. And because Meme SQL is a distributed system, even though it's, you know, it's a brute force way of doing it, there's no index. You just go literally run the dot product against, you know, millions of faces stored in the database. But because everything is so tightly optimized, you can still run this within, you know, fifty to a hundred milliseconds.

• 28:29Daniel Whitenack?

  Yeah, that's crazy.

• 28:30Speaker 03

  And that's running in production, right? It's running in production. And like I said, for both kind of government, security. Use cases as well as, um, you know, things like walking into a grocery store. And the system suggests, oh, grapes that they usually buy are not in the system right now, but go buy this something else.

• 28:47Daniel Whitenack?

  Gotcha. Yeah. So I was curious as we were talking and we were talking about, I guess that's a computer vision use case. And I'm thinking about like the types of data that are involved in machine learning and AI workloads. And we've got, you know, of course imagery and video. We've got a lot of natural language processing going on these days. And, you know, some of these types of data I've dealt with in SQL databases before. Of course, numbers and strings and that sort of thing. But I wouldn't typically think of like, oh, I'm going to store this image or video or like an audio file or something in, in a database. So are you thinking that in the longer term, the, a good workflow around this is that you're storing the sort of feature vectors

• 29:31Daniel Whitenack?

  or embedded representations of maybe text or audio or maybe like spectrograms of audio via like the tensor built in or those sorts of things? Or are there other ways around that?

• 29:43Speaker 03

  This is a great question. To me, it's what it is now and what is it going to be as we go. And I will give you a very kind of product centric answer to this question. You know, like what would a product manager think? And they always start with a user. The user in this particular case is again, data scientist from the training standpoint and an engineer from building an app standpoint. I think today, data scientists and with the tools that the data scientists use, it's a lot more natural to store this data in a data lake, right? Basically in S3.

• 30:17Daniel Whitenack?

  Yeah, just.

• 30:18Speaker 03

  It's bottomless, it's files, it's cheap and all the toolsets work out of the box. And the reason to put that data into a database is only when you get some sort of additional benefits to that. When you put structured data, the benefits are obvious. The aggregations, so it enables low latency access to that data and enables very fast aggregations and reporting. So you can slice and dice that data in the database before pulling the data out and use your custom tools to provide reporting. For unstructured data, the only benefits that I see are governance. Database can provide that unified access layer to all your data,

• 31:02Speaker 03

  but it doesn't give you any compute benefits over that, right? You get a kind of, kind of a wash. So that's the way we think about it right now as well as exploring. I think. What's gonna happen in the future? That databases, just like MemSQL, will give you an option to access that data that's stored in the data lake and in the file system through the database API, with the benefit of marrying that data and like really understanding metadata, potentially building a full text like index against that data. So you can marry that data with the rest of your enterprise data, which is usually relational.

• 31:41Daniel Whitenack?

  Gotcha.

• 31:42Speaker 03

  But do not yank the direct access to the file system, because that's what data scientists do every day, and they would be confused.

• 31:53Daniel Whitenack?

  Uh huh.

• 31:54Speaker 03

  If you remove that access pattern from them.

• 31:56Daniel Whitenack?

  Yeah, I guess on that side of things, we kind of talked a lot about the operationalizing of models. On the training side, now we're kind of talking about access to files and all of those things. And you're saying you have the one interface or the integration with Spark. You know, for me, a lot of times I store everything in S3. Like you're saying, it's very natural for me. I just like say, I want this file and I'm gonna use it. But there's definitely issues that come up very quickly on that front, too. I know even like this morning, it's like trying to like deal with, you know, like 200 gigabytes of of audio data and like the I was just sitting around for a while and making coffee. It's not very productive or or fun to deal with those sorts of things.

• 32:42Daniel Whitenack?

  When you say like, I guess on the training side of things, you have maybe people that are used to the Spark interface can do that. Are there other ways with MemSQL that like? Like if I want to access my audio files in S3, is there a way to do that with MemSQL outside of Spark? Are there other sorts of interfaces I can use?

• 33:02Speaker 03

  Not at the moment.

• 33:03Daniel Whitenack?

  Okay.

• 33:04Speaker 03

  But I will share some of the thinking, right?

• 33:06Daniel Whitenack?

  Yeah.

• 33:06Speaker 03

  So right now, there's a lot of technology we're building around.

• 33:12Speaker 03

  Just relational data and providing that single pane of glass window into all your relational data. That's where we're kind of the strongest. When we think about S3, we think how we can offload all the data that's not currently touched by the system into S3. We call this thing bottomless and making databases bottomless. Like, if you think about Postgres, Postgres is not bottomless, right? It's bound to the amount of hard drive that you run Postgres on. But we want to make it completely bottomless and very, very cheap. Uh, and S3 is, is, is probably one of the cheapest way to store data in the cloud. And we have things like MinIO that is one of the cheapest way to store data on premises.

• 33:54Daniel Whitenack?

  Yeah.

• 33:55Speaker 03

  When specifically around that pattern that you described, right? They have an audio file and it's 200 gigabytes and it's a pain to go and transfer that file from one device to another. And it's a pain to download this from S3 to your local storage and all those things. So the thinking there is again is through integrations. If MemSQL is aware that here's the file in that particular format stored in S3 and then you want to somehow either bring computation to data or you want to, uh, access to a subset of that file. And only that you want to bring into your training environment, also either running on the cloud or somewhere else. So we want to enable those things. That's where it stops so far.

• 34:40Daniel Whitenack?

  Yeah.

• 34:41Speaker 03

  That's where our thinking stops so far. We're certainly aware of the scenarios and we're aware of some of the pains that people go through. The place where we think MemSQL can add value is versioning. Because you oftentimes need to run and rerun experiments. And the model, it's not just the model, it's the model and the data that's been trained on. That's really the unit that is consistent. And if the data changed, the model might be rendered obsolete, might not be. So it just versioning makes a ton of sense from the ability to run experiments, verify experiments, share and exchange the models and data across data scientists.

• 35:22Speaker 03

  So I think that's where we can provide a nonlinear amount of value over time.

• 35:39Speaker 01

  Changelog News is the best way to keep up with the ever-changing world of software. We track, blog, and contextualize the coolest projects, the best practices, and the biggest stories each and every week. Make Changelog.com your daily destination, or hit the snooze button and subscribe to our weekly newsletter that hits inboxes on Sunday mornings. Join more than 15,000 enthusiastic readers. It'll cost you exactly $0, and you can subscribe right now at changelog.com/weekly.

• 36:27Daniel Whitenack?

  All right, well, turning now a bit, I think, from the sort of AI and ML integrations, maybe to more sort of analytical workloads. I know that when we were talking before the show and in conversations leading up to the show, it sounds like that, you know, there's some pretty interesting things going on in terms of MemSQL being used during the COVID-19 pandemic. And of course, there's interesting, you know, tracing work going on and all of those things that I've heard about, but I haven't really heard about how some of those things are being enabled. So I'd be curious to hear a little bit more about that.

• 37:04Speaker 03

  Definitely. So let's step back for a second and think about what's different parts of the world and different companies and governments, what do they fundamentally want to accomplish as we go through the pandemic? The first one is, you know, simple, how do we stop the spread of the virus? And, okay, well, maybe we cannot really stop it, or let's say we put our actions and efforts to do that, but since it's spreading, it's a matter of fact. What else can we do and how we can drive our decisions based on data? What kind of decisions? Well, could be capacity planning for ventilators. All

• 37:43Speaker 03

  right, we know that, you know, there's an outbreak there, and we will likely have our healthcare system overrun, and we need to provide extra capacity to the healthcare system, but how much capacity? So all of those questions require answers, and the answers are in data. That's where data science comes in. And that's where just starting from collecting the data, putting it in one place, organizing the data and feeding this information to people who have the levers of power. Second one is like, who owns the data? We have obviously Apple and Google who own the data because they have a device.

• 38:24Speaker 03

  Every individual on this planet, not every, but most of them now on a smartphone. So you can tap into that stream of data and getting information about who is at which location at any point in time. And then marry that location with migration patterns and marry that location with like individual tracing. You know, given that we know that this person has COVID-19, you know, who are all the people that this person came across in the past two weeks? So we can go reach out to them and say, hey, you probably want to be tested. The second entity that has that data, obviously maybe government, but I don't know about that, but certainly telcos.

• 39:07Speaker 03

  Telcos have this information, maybe not as accurate because they don't have a GPS on the device. Well, actually they do GPS on the device, but they may not be able to tap into the GPS. But they can triangulate their location based on cell towers. So we're working with some of the largest telecommunications operators here in the outer space as well as around the world. And I think the one that's public is True Digital, one of the largest telcos in Southeast Asia. And we do both. We do the migration patterns where, you know, if you go back to like, you know, March, February timeframe, we already knew that there was an outbreak in China and there was an outbreak in Italy. And we already knew how bad that was.

• 39:50Speaker 03

  And looking at the flights from Italy and like tracing individuals that land and then starting to see this pattern of people getting sick emerge, you can start driving decisions off of this. And you can start putting policies in place that can stop the spread. You can start do capacity planning. You can start manufacture masks and ventilators and distribute them into places based on the patterns that we're observing. And so that's how data management solutions are helpful to companies that have the data. And also the insights that those systems generate are useful for people with the levers of power to drive policy and to drive decisions. Yeah, Google and Apple, well especially Google, has the technology, but you know,

• 40:35Speaker 03

  telcos don't. And that's where we partner and give them those abilities.

• 40:39Daniel Whitenack?

  Yeah, it strikes me that, you know, the things you're discussing, there's definitely a lot of potential and value there. And earlier on in the episode about facial recognition and a lot of things that are possible there on a sort of large scale. And I think that as people are now in this pandemic and kind of layered on top of that, all of the climate that's in our country and around the world, around, you know, injustice and policing, there's a lot of people asking really good questions about actually data management and security and privacy. And I'm curious, you know, with you being in a position to have so many conversations with different types of entities around like how they view

• 41:23Daniel Whitenack?

  data management, how that's changing as we think about these powerful applications of of kind of large scale analytics, but also the potential concern with privacy and tracking and all of those things. I'm just curious to get some of your thoughts on how large organizations are starting to view data management and security, maybe now a little bit different than than they might have in the past, given that all of the things that are going on in our in our world.

• 41:54Speaker 03

  Definitely. It's a multifaceted question. It starts with data management to highlight everything that's going on and the big problems that we face and big issues that we face as a nation. How can data management help here? And I think one of the answers to that of many, right? There's so many things that would go into solving this big issues that you raised. But one of those things where data management can actually help is with data sharing and data consumption, right? Imagine the police data was given by the government to the whole world in the easiest way from the consumption standpoint. And

• 42:37Speaker 03

  it's completely real time. So if you have an arrest and that arrest by regulation has to be a part of a public record, that is in the system in 10 seconds after that arrest happened. And so that information is just live, real time for everyone's consumption. And with our vision of a single pane of glass towards all your data and all workloads, we will be able to enable those things, enable anybody to log in into our cloud service and consume that data, assuming the provider is willing to publish that data. Imagine that climate change data is available to anybody in real time and it's live and it's easy to consume.

• 43:20Speaker 03

  So where we live today is a lot of data sets are public and a lot of data sets are public and there's regulation that forces them to be public, but they are published in a non-standard, obscure way.

• 43:33Daniel Whitenack?

  Yeah, they're not discoverable.

• 43:35Speaker 03

  They're not discoverable, so to consume that data set is, it's a project. It's like going to a library or going to a court and like asking for permission and they will like bring this papers and put it on the table. I'm inspired by rewatching rather Spotlight where they got access to some sensitive data that had to by law be public and had to, they had to jump through hoops. But imagine all that data is discoverable. It is at your fingertips and that data is up to date. So you don't have to think about it like, oh, like I downloaded this from last month, what changed between last month and today? So it's just there. That can make a lot of things easier, more transparent and we'll be living in a better world.

• 44:18Speaker 03

  We need to think about the implications of that from the like, you know, what if bad guys had access to this data? But that's a policy question. That's not a data management question. I think data management should enable us to live in a world like this and the technology is already there.

• 44:35Daniel Whitenack?

  Yeah, and I imagine that if you have this sort of single way to interact with data that's centered around SQL and people are familiar with that, they're able to use it. There's also, in addition to the sharing of data, there's sort of the sharing of methodologies that can happen. For example, even in our last episode that we recorded, we talked about some tooling that's out there around fairness and bias and other things. It's a little bit like you have to read a good amount of documentation. You have to figure out how to use these things. I wouldn't say it's like seamless and, you know, easily integrated into your workflow at this point. But I could imagine, for example, if a suite of tooling that is easily accessible via certain SQL workloads

• 45:21Daniel Whitenack?

  that look for bias in your data on certain, you know, features or highlight certain things in your data set and all of those things. And whether you're using, like you say, whether you're using TensorFlow or PyTorch or Spark or whatever, you could potentially have access to those things in terms of people sharing their methodologies because things are centralized in terms of the SQL language. Do you see that? I'm wondering, you know, what's the MemSQL community like? I guess in terms of, you know, people working on projects built on top of MemSQL. What's that community like? And do they share certain things like that? Or certain things available that are maybe open source that are built on top of MemSQL that people can work on in a collaborative way?

• 46:06Speaker 03

  The community is on the forum.memsql.com. And then there's a community of mostly enterprise developers, actually, because that's has been our focus so far, that are sharing through MemSQL events and conferences. Where we're going is, you know, now that we've gotten here and we're opening up the platform more and more to the community, we're thinking a lot in terms of free and how we can make a lot of the things that got us here, you know, to got us to the 40 million run rate with 70% growth. How can we take some of that and open them up? And by opening up, providing certain set of features and capabilities to the world for free.

• 46:48Speaker 03

  So on our dime, you go in the cloud, you log in and there's this free tier of stuff that you can do. That's our current thinking so far and I'm actually going to be personally overseeing that effort here at MemSQL.

• 47:00Daniel Whitenack?

  Yeah, that's really exciting. I'll be excited to kind of dig in and play around with those things. One other thing that I guess is COVID related and also related to our changing world is I guess people's just workflow and productivity during this time. I'm just curious with MemSQL growing so fast and obviously a lot changing, a lot happening. How has that been for MemSQL and how do you see kind of, I guess, tech work from home and productivity sort of stuff moving forward from your perspective as a CEO?

• 47:35Speaker 03

  Yeah, so first of all, we are in the uncharted territory, right? MemSQL wasn't the company that was born remote first, right? Even though we're global and we have office in San Francisco, Seattle, Lisbon, Kiev, Ukraine, India, I think it's Bangalore, and sales offices all over the place. There's still concentration in each location and usually a particular

• 48:01Speaker 03

  concentration of like a component that people work on within an individual location. We weren't impacted from our performance standpoint. It's like it's been one quarter of COVID. That was basically just finished our COVID quarter. We demonstrate tremendous results. We're very happy and excited about the future. And we obviously shifted all our workflows into working from home workflows. Now, the worry that I have, and I'm being paid to be paranoid, is that it works fine so far because we are tapping into the social capital that we've built over the years, right? And a quarter of COVID is we spending that social capitals and all this like social links

• 48:46Speaker 03

  are established between people and they fill them while working, you know, at a particular location and looking into people in the eye, to their friends and colleagues. So that's gone, right? So every meeting is a formal meeting, if you think about it. We're missing out on hallway conversations.

• 49:04Daniel Whitenack?

  Yeah, I guess I hadn't thought about it that way, but it's true.

• 49:07Speaker 03

  Yeah. We're missing out on hallway conversations. We're missing out on, you know, uh, grabbing coffee together and having this like nice positive experiences, brainstorming while walking towards a, a, a nice coffee shop and, and grabbing a latte. So I want those things to be back.

• 49:28Speaker 03

  So hopefully this will, will happen relatively soon and we'll have a dent in the social capital that we've built and then we'll kind of fill up that dent by getting back together.

• 49:39Daniel Whitenack?

  Yeah.

• 49:40Speaker 03

  So that's my hope. Obviously we can't control that. The situation controls us a little bit.

• 49:45Daniel Whitenack?

  Yeah, it's interesting. I mean, so I've been working remote, previous to COVID, I've been working remote for I think maybe about three or four years now. And I definitely get what you're saying. I, I, I've had to intentionally over time, like develop relationships with local like data scientists or technical people that are, you know, maybe not, so they're not working at the same organization that I am, but it's a chance for me to like get together with those people and just talk about things. Cause sometimes I wonder, just sitting at my computer, I brainstorm a lot of things. And sometimes I wonder if I'm crazy because I'm just, I'm not talking about those things to anyone, except in when I'm like presenting them to my supervisor, I'm presenting

• 50:27Daniel Whitenack?

  them to a group and I'm supposed to, you know, sound like I know what I'm talking about, hopefully a little bit. So that, yeah, it's not that sort of informal, um, environment. And that's very interesting observation. I hope that some of that can come back.

• 50:41Speaker 03

  Definitely.

• 50:42Daniel Whitenack?

  Yeah. As we wrap up here, uh, love to give you a chance to just let people know. Obviously there's, uh, memsql.com. We'll have the links in the show notes, but as like a data scientist or AI sort of person, are there ways that people can kind of play around with memsql and get a little hands on and see what it feels like and how to do certain things? Where would you recommend that they start getting on boarded?

• 51:09Speaker 03

  Definitely. So if you want free forever, we have our software and we give our software to up to four servers, like I said, it's a cluster software to install whenever you want and run forever. So we call this our software free tier. It grew three times over the past year from the active user standpoint. It's basically one of the best column stores on the planet. So data is highly compressed, stored on disk, very fast reporting, everything is updatable, transactional system of record. And so where other companies like, I don't know, that run on premises, you know, the vertices, the green plums, they want to charge you for that, you know, you get it free and you can put billions and billions of data

• 51:52Speaker 03

  points in the system and get very, very fast SQL response from it. In the cloud, our free tier is time based. So I encourage people to log in and you know, you can play around with a system that would allow you to not use any software and consume everything as a service. But because we're running it on our infrastructure, we're limiting access to free for a period of time. We'll be announcing more changes there. We'll give the system for you for free forever for a limited usage, but that's hasn't come out yet. So that's something we're working on. So that would be probably the best places to start. And of course, go to forum.memsql.com to learn about the system.

• 52:32Daniel Whitenack?

  Awesome. Yeah, we'll have those links in the show notes. Really appreciate you chatting about everything today. I think our listeners will really enjoy the content and hopefully check some of these things out. And yeah, thank you so much for joining us and hope to have one of those hallway chats with you some point when when things are actually opened up.

• 52:52Speaker 03

  Well, if you're in Silicon Valley or I'm there, I will make sure to paint you and we'll hopefully make that happen.

• 52:58Daniel Whitenack?

  Yeah, definitely. Yeah, thank you so much.

• 53:00Speaker 03

  Bye, Daniel.

• 53:01Daniel Whitenack?

  Bye.

• 53:05Speaker 01

  Thank you for listening to this episode of Practical AI. People ask us all the time. They say, hey, how can I support your work? One easy way is to leave a five star review on Apple Podcasts. Tell folks why you listen and why they should too. It only takes about 30 seconds and believe it or not, those ratings and reviews really do help us rank higher in AI-related search results. Practical AI is hosted by Daniel White Neck and Chris Benson. It's produced by Jared Santo, that's me. And our music is brought to you by the one and only Breakmaster Cylinder. We are sponsored by amazing people at companies who get it. Thanks again to Facilit, Linode, and Rollbar. Did you know we have a master feat of all Change Log podcasts? We do. It's your one stop shop for everything we produce.

• 53:47Speaker 01

  If you like this show, you'll love the Change Log, Brain Science, and GoTime. Check it out at changelog.com/master or search for Change Log Master in your favorite podcast app. You'll find us. That's it for now. We'll talk to you again next week.