The fastest way to build ML-powered apps

Tuhin Srivastava tells Daniel and Chris why BaseTen https://www.baseten.co is the application development toolkit for data scientists. BaseTen’s goal is to make it simple to serve machine learning models, write custom business logic around them, and expose those through API endpoints without configuring any infrastructure. Sponsors: O'Reilly Media https://www.oreilly.com/changelog – Learn by doing — Python, data, AI, machine learning, Kubernetes, Docker, and more. Just open your browser and dive in. Learn more and keep your teams’ skills sharp at oreilly.com/changelog https://www.oreilly.com/changelogSnowplow Analytics https://snowplowanalytics.com/ – The behavioral data management platform powering your data journey. Capture and process high-quality behavioral data from all your platforms and products and deliver that data to your cloud destination of choice. Get started and experience Snowplow data for yourself at snowplowanalytics.com https://snowplowanalytics.comFastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: Tuhin Srivastava – GitHub https://github.com/tuhins, X https://x.com/tuhinoneChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: BaseTen https://www.baseten.coBaseTen | LinkedIn https://www.linkedin.com/company/basetenBaseTen | Twitter https://twitter.com/basetencoBaseTen | Blog https://www.baseten.co/blogUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The thing that I'm seeing more and more is that the thing that is most valued is people who are able to take some data, write some sequel, create this simple data set to train a model. And then if you can get it behind a fast API, working decently, performantly, not amazingly performantly, honestly, you're in the top 1% of data scientists, I think. For me, the trend that I'm seeing is that people are valuing those people more. I think like five years ago, people were looking to hire data scientists and more and more, it's like, hey, we want machine learning engineers. There's a difference there and your data scientist is, in my mind, kind of a flawed title. In the sense that it just represents anything from someone doing BI as an analyst to people building models to ML engineers. And we're kind of separating those things out, which I think is great. But I think the trend that I'm seeing is that more people value those machine learning engineers.

• 0:45Speaker 00

  Unfortunately, it's very, very difficult to find those folks. I think it'll take a bit for everyone to catch up to that.

• 0:55Speaker 01

  Big thanks to our partners, Leno, Fastly and Launch Darkly. We love Leno, they keep it fast and simple. Check them out at leno.com/changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by Launch Darkly. Get a demo at launchdarkly.com.

• 1:13Speaker 01

  This episode is brought to you by our friends at O'Reilly. Many of you know O'Reilly for their animal textbooks and their conferences, but you may not know they have an online learning platform as well. The platform has all their books, all their videos and all their conference talks. Plus you can learn by doing with live online training courses and virtual conferences, certification, practice exams, and interactive sandboxes and scenarios to practice coding alongside what you're learning. They cover a ton of technology topics, machine learning, AI, programming languages, DevOps, data science, cloud, containers, security, and even soft skills like business management and presentation skills. You name it, it is all in there.

• 1:54Speaker 01

  If you need to keep your team or yourself up to speed on their tech skills, then check out O'Reilly's online learning platform. Learn more and keep your team's skills sharp at O'Reilly.com/changelog. Again, O'Reilly.com/changelog.

• 2:15Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We are at Practical AI FM.

• 2:41Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whiteknecht. I am a data scientist with SIL International, and I'm joined, as always, by my co-host, Chris Benson, who is a Principal Emerging Technologies Strategist at Lockheed Martin. How are you doing, Chris?

• 2:58Chris Benson?

  I'm doing great. How's it going today, Daniel?

• 3:00Daniel Whitenack?

  It's going wonderful. You got any circada broods, brooding down by your parts?

• 3:06Chris Benson?

  So for any listeners that have listened to us for a while, they know that I do these weird wildlife things. And so, yeah, I'm getting tons of calls. It are like totally outside the AI thing. Our nonprofit that does the animal stuff, we're getting all sorts of calls about copperheads and all that, and people are afraid. And really, it's probably too late by the time people listen, but it's not an issue. There are no more copperheads than there ever are. So I know that has nothing to do with AI and ML, but you know, there you go. There's my public service announcement for, for the circada.

• 3:36Daniel Whitenack?

  Yeah, I mean, it is sort of a fascinating thing that I hear people talking like, you know, of course, like with climate changing and other things, how the broods are sort of a bit off. But some people, you know, it's hard to know a little bit because also more people are reporting data via all sorts of apps and channels and all sorts of things. And now, I mean, you're probably familiar with these apps that-

• 3:59Chris Benson?

  Yep, there's several of them.

• 4:01Daniel Whitenack?

  I think it's iNaturalist and other things where people report different species and that sort of thing. And those are fed into models and all sorts of goodness. So, I mean, it is interesting.

• 4:10Chris Benson?

  I love how you just tied the wildlife thing in with the whole AI ML thing. That was really good. That was good.

• 4:15Daniel Whitenack?

  You know, it's just one stream of consciousness for me.

• 4:20Daniel Whitenack?

  I'm really excited. Well, Chris, you know that we're very practical on this show. At least we try to be and relevant to our listeners. Indeed, we are. And I think that this is definitely going to be a very practical discussion because we're going to talk a little bit more about some new AI and ML developer tooling. Excellent. And deployment tools and all of that stuff. Today, we've got Tuhin Shrivastava with us, who is co-founder and CEO of Base10. Welcome, Tuhin.

• 4:48Speaker 00

  Hi, thank you. Thanks for having me, Chris and Daniel.

• 4:51Daniel Whitenack?

  Yeah, it was great to meet you. We had a chat last week and I was learning a little bit about some of the things that you're doing and had to get you on the show right away. So really excited to hear about that stuff. But before we dive into all that, could you just give us a little bit of info about your background? How you got into doing what you're doing now?

• 5:10Speaker 00

  So my background is actually electrical engineering. I studied electrical engineering in college and after college, I decided that the right thing to do was to go work in finance in New York for a couple of years. It had nothing to do-

• 5:22Daniel Whitenack?

  That's the rumor in academia that I heard was like, you can sell your soul and go make a ton of money in finance. Not that you have to sell your soul to go into finance.

• 5:31Speaker 00

  Yeah, well, I got to work on the fun problem of privatizing toll roads.

• 5:35Daniel Whitenack?

  Ah, there you go.

• 5:36Speaker 00

  Which is fascinating in its own way, but after a couple of years of it, I decided that this wasn't for me and I decided to go back to engineering. So I actually moved to Boston from New York to work at a weird academic lab at Beth Israel Medical Center, which is part of Harvard Medical School, where there was a professor there who'd won a prize for coming up with a non-invasive way of tracking the progression of ALS. And he was spinning off a startup as part of that that was very, very data-driven. This is like 2011. And, you know, I didn't know anything about machine learning or stats. Well, you know, I'd done a bunch of electrical engineering and information signal processing in college.

• 6:14Speaker 00

  I was like, you know, I could probably convince him, just take me on and, you know, Boston startups are a weird breed in that, you know, they're kind of very research-focused. They get a lot of grants and, you know, frankly, they're quite cheap. And so I was like, you know, I'll go and tell him to pay me hourly and I'll go work there. So I went there and I got to work with, you know, this guy and these three MIT PhDs, basically figuring out if they can predict the prognosis of neuromuscular disease.

• 6:38Chris Benson?

  Do you remember their names? I'm just curious because I've read up on ALS lately.

• 6:41Speaker 00

  Yeah, so the guy's name is Stuart Rotkove. And the technology that he created was called EIM, which is electrical impedance biography. And this is really interesting and it makes me excited. So I'll tell you really quickly what it is. We use EMG today to track how muscle health is progressing when folks have neuromuscular disease. It's a pretty horrible thing to go into every three months and have a bunch of needles poked into you. So his idea was rather than poke needles in and track the electrophysiological properties to shoot a small current, a microcurrent, that no one can feel across the outside of the muscle, measure the impedance drop and see if that correlates with muscle health. And, you know, it does. He'd been working on it for 20 years. I happened to catch him at a time when he was commercializing technology. You know, it was a really fun time.

• 7:26Speaker 00

  You know, I got to work on all sorts of things. I learned statistics again. I really go into machine learning. And so really, you know, the first thing we did was at the time we were applying support vector machines. This is like 2011, 2012, when support vector machines were all the rage. Support vector machines to see if we could separate out healthy muscle from diseased muscle. And that was really fun. Published a couple of papers and kind of really got, I was like, okay, this is pretty cool. You know, it seems like there's not that many people doing this. I could focus on this for a few years. And so I did that for a couple of years. And my wife was, my now wife, then girlfriend, I guess, was a PhD student at Berkeley. I was like, I need to move out to California. So moved to San Francisco and started working at this company called Gumroad. You know, Gumroad is a lot bigger now than it was then. And I was one of the first few employees.

• 8:11Speaker 00

  It was an open-ended e-commerce platform, so anyone can sign up and start charging people, anyone can get paid out. Great idea for creators, you know, really, really horrible from like a payment security and fraud perspective. As, you know, especially in the early days, you know, we used to get hammered with fraud. And so we just started first applying heuristics and then we built out an entire kind of like ML pipeline there to not only catch fraud, but also the tooling. To be able to deal with it once something had been flagged. So did that until 2015 and then one kind of had the entrepreneurship bug and founded our first company, which went through a bunch of twists and turns, ended up being an operational analytics company where we would

• 8:52Speaker 00

  capture data from a bunch of different operational tools, created BI layer on top of it, not really MR related, more data related, sold that company in 2018. And then, you know, after working at the acquirer for a year, decided that, hey, I really wanted to go back and work on ML tooling. I'd built a bunch of this stuff. I think a lot of ML engineers and data scientists end up building their own tooling even today. Um, and you know, it seems like the beginning of something a lot bigger and, you know, don't know where it's going to take us, but really wanted to work on this idea of how do we get more people doing more machine learning and the tooling that will enable that.

• 9:26Daniel Whitenack?

  So I'm curious, as you were sort of in that progression, you know, you were sort of thrown into the fire, working with statistical tools and machine learning at various stages in different companies. Throughout that history, was it mostly sort of you as a like data scientist or ML type person working with a bunch of developers? Or was it sort of a bunch of developers trying to kind of pull in machine learning tooling where they needed it? For like the fraud detection and stuff? What sort of patterns did you see as you were working at those various places?

• 9:58Speaker 00

  Yeah, that's a really good question. I didn't actually start as a engineer. I started as a data scientist and machine learning engineer. And when I was in Boston, working at the healthcare company, it was okay because it was very academic. I mean, a lot of it was like in MATLAB and we could get a lot of the stuff done in MATLAB and throw it over the fence to someone who would happily take it out of that. When I joined Gumroad, it wasn't like that. I was a data scientist and engineer, it was an early stage company. You know, ended up hiring another data scientist and engineer who was a really dear friend of mine from Australia. You know, we didn't have the resources to help us productize the stuff we were building our product. So we just had to learn it ourselves. So it was very much coming at it from a data scientist perspective. But I don't think I was ever lucky enough to have, you know, a team of devs ready to go being like, hey, we'll happily productize this or productionize it.

• 10:44Speaker 00

  It was very much, I was expected as a data scientist or machine learning engineer to do that myself or be resourceful about that.

• 10:51Daniel Whitenack?

  So do you feel like in that process, where was your time spent in that process? Like as opposed to like before, when you were at the company where most of your time was developing this sort of prototypes in MATLAB? Like, and now you were having to bring in this engineering side of things. Like, how did that sort of split up your time?

• 11:11Speaker 00

  Yeah, that's interesting. I think initially it was obviously in the model development part of it. What I realized really quickly was, like a lot of other data scientists, you end up being perceived as somewhat of a research function as opposed to a product function. You know, I found that really demoralizing. And so what I found was that it was in my best interest and the company's best interest if I, you know, worked on getting my stuff out of the Jupiter notebook and in front of other folks as opposed to keep optimizing my model. Cause you can do that forever, right? The model is never going to be a hundred percent. What I realized pretty quickly was that for me and Phil, who was the other day at Santos I was working for all that, hey, we'd already got the highest order bits out of the model. What was the most important right now was to make this practical for the company and have it plugging in back into the business process.

• 11:55Speaker 00

  And really have other folks, whether that be, you know, the CEO, whether it be other engineers, you know, customer support. I really seeing the fruit of what this model can enable for the company for them.

• 12:06Chris Benson?

  It's a bit of a timeless problem that you're describing there. And that I know in the organizations that I've been part of over the years, I've seen that same set of problems recur over and over again, as well as those same biases and perceptions that people have. You talked about people assuming that the data scientist just has a research function. Whereas the three of us are talking here today, you know, we understand the value of applying data science into the daily operations of business, and yet so many people have failed to do that. It's a fairly wide chasm to get all of the things together to be able to not only do the data science, but to be able to communicate that out to the audience in an effective way and in a way that is accessible and usable to them.

• 12:49Chris Benson?

  How did you approach that problem? Because I mean, that's a massive problem that many people

• 12:55Speaker 00

  Yeah.

• 12:55Chris Benson?

  have tried to tackle, and I don't envy you that. So how did you set about going about that?

• 13:00Speaker 00

  Yeah, I think I was lucky in that I've talked to a lot of customers over the last like year and a half. And one thing we realized is that a lot of engineering function, and this is just a kind of a nature of the beast of where a lot of data science and machine learning engineers are coming from. A lot of engineers don't want them writing code. They're like, hey, like, you know, it'd be best if you didn't do that. And I was lucky enough to be in a place where it was encouraged to write code. And so honestly, like for three, six months, like I just became a Rails developer. I was like, the company's monorepo wasn't Rails. I was like, I'm going to figure out how to do this. Like I'd already done a little bit of Django programming on the side. I was like, I'll learn Rails, I'll understand kind of their language and then I'll be able to bridge that gap. I think that was really good.

• 13:42Speaker 00

  Like it's been really great for me because, you know, as I've gone through and, you know, built products over the last five or six years, being able to be an engineer has been great. But at the same time, I don't know if that's attainable. I don't think, you know, a lot of great, really intelligent data scientists who have a great intuition when it comes to models want to be engineers. I don't think they want to learn about unit testing. Man, I don't know if the best use of their time, but that's how I went about it. I think I've heard this story enough that there's a lot of leverage to be gained if you can somehow give data scientist engineering skills and, you know, and there's not enough folks who have both of those skill sets.

• 14:18Chris Benson?

  Yeah.

• 14:19Speaker 00

  Has that been your experience as well?

• 14:20Chris Benson?

  It has. I mean, I know that like Daniel and I both came into the AI world with lots of programming experience and different backgrounds. We actually did not meet each other in the AI ML world. We met each other in the Go programming language world and kind of

• 14:33Speaker 00

  Awesome.

• 14:34Chris Benson?

  transitioned over. And so I think my own perspective on that is that no one can master everything, but it makes sense to have some areas of strength and then extend your tentacles in other directions so that you can collaborate with other people who also have multiple areas, but maybe their strengths are in a slightly different area. And so that's how I've kind of tackled it as I've worked with different groups over time. I'm always super excited to hear about how people are addressing it. And really the thing that you're doing that I'm really impressed with is being able to scale that out to many people. Whereas I'm struggling to do it on my own and to work with different teams and to make those skills overlap. And so I'm pretty excited to see, you know, kind of what you came up with in your solution here.

• 15:19Speaker 00

  Yeah.

• 15:19Chris Benson?

  To enable that for other people, such as myself.

• 15:22Daniel Whitenack?

  Yeah, I think that I do see this changing and maybe this is what you're referring to, Heen, with sort of things changing in the landscape. Before, when I saw many tutorials for people getting into AI and other things, really just focused on data prep, training, and evaluation.

• 15:39Speaker 00

  Yeah.

• 15:39Daniel Whitenack?

  And I think that like, there's just this sort of common perception that like, okay, I'm evaluated, like, end of story, like, I've done my job. And some people I talked to, just even this concept that like, you can save a model, like, you can serialize it to a file and then use it later. Like, that's something they're sort of not quite used to yet. And I find that very, like, yeah, I mean, even if someone's not going to be like, you know, a very engineering-focused person, like, having some level of understanding of the integration points.

• 16:14Speaker 00

  Yep.

• 16:15Daniel Whitenack?

  And how like, our work feeds into other things, that integration point is very important. Was that what you were referring to when you're sort of talking about like, trends you were seeing or other things?

• 16:26Speaker 00

  Yeah, totally. I think the thing that I'm seeing more and more is that the thing that is most valued is people who are able to, you know, take some data, write some SQL, create this simple dataset to train a model, start to train their model. And then, you know, if you can get it behind a Flask API, working decently, performantly, not amazingly performantly, honestly, you're in the top 1% of data scientists, I think. For me, the trend that I'm seeing is that people are valuing those people more. I think like, five years ago, people were looking to hire data scientists, and more and more, it's like, hey, we want machine learning engineers. There's a difference there, I think. And your data scientist is, in my mind, kind of a flawed title, in the sense that it just represents anything from, you know, someone doing BI as an analyst, to people

• 17:09Speaker 00

  building models, to ML engineers, and we're kind of separating those things out, which I think is great. But I think the trend that I'm seeing is that more people value those machine learning engineers. Unfortunately... It's very, very difficult to find those folks. I think it'll take a bit for everyone to catch up to that. And so, you know, that's kind of like the trend we're seeing. And that's one of the trends that we kind of latched on to as well.

• 17:29Chris Benson?

  So I'm very curious. We've led up to it without talking, without diving into it yet, but I would love to hear the backstory about how the idea for Base10 came about, how you moved into it, how you arrived at the start of that process.

• 17:44Speaker 00

  Absolutely. So, you know, I think we've kind of gone through the context here, which is, hey, you know, it's pretty hard to get a model working. The hardest part of that, or maybe the most important part of getting a model working from my perspective, and you know, quite frankly, from the name of this podcast's perspective, is making that model practical, is making it plugged into some process or in front of real users who can give feedback on that. Again, like as I mentioned, we noticed this, we learnt that skillset. So our two co-founders, one is Phil, who I've worked with a bunch, and our third co-founder was kind of the head of engineering at Gumroad, who went to work at a healthcare startup, which was very, very data-driven. And from his experience as well, was that there's so much tooling that needs to go in once you have that model. So like, you know, whether that be the deployment,

• 18:29Speaker 00

  monitoring, tooling, but more importantly, like the stuff that happened afterwards, which is, you know, how do you build, how do you integrate that into existing system or, you know, build an application out of that model? And, you know, what happens today? That we noticed over and over again was people were kind of building these [inaudible 00:01:25] sorry. I don't know.

• 18:48Chris Benson?

  It's okay. Keep going.

• 18:49Daniel Whitenack?

  There's a lot of them out there. I've seen them with my own eyes.

• 18:52Speaker 00

  Yeah. So, you know, like fast gaps that they're definitely not going to scale. It's pretty hard to get them deployed. You know, everyone starts by pickling their models, but really quickly, as soon as their model gets any significant size, you can't fit it in memory anymore. So you can't keep pickling and unpickling things. You know, we used to do this stuff ourselves. Like, I'm not hating on it at all. It's just like, you know, we built these really bad systems where we took a model, and especially at Gumroad, we then deployed it. So we built a fast API. We were pickling and unpickling it on the fly. Not great. But then we were building all the application logic that sat on top of that model. So, hey, model gets run. So we compute the features, model gets run, and then the decisioning part of it, if the model exceeds the threshold,

• 19:32Speaker 00

  do X, Y, and Z. Otherwise, you know, put it in front of a human to go and review that transaction. What we did was we went and built all those things. So we deployed the model, we built the backend service to support that model. And then we also learnt from engineering to go and build that kind of a queue-like interface for folks to do that. And you know, our, my co-founder had a very, very similar experience at a healthcare company where they were trying to predict diagnoses from a medical record. And, you know, they wanted to put that in front of the doctors so the doctors could get feedback on it. The way they did this was they ran the predictions in Bash, they piped into some data warehouse, pulled it out, put it into Excel. I'm not joking, they sent the Excel file with like the patient ID and the predicted diagnosis

• 20:15Speaker 00

  to the doctor and ask for feedback. Obviously, the doctor's not gonna like act on that, right? They're gonna be like, this is too much work. And so I've kind of led up to this where I'm saying that, you know, if you are that top 1% of data scientists or machine learning engineer who can kind of do all those things that, you know, we had to learn how to do, this is great, you can do that. But for the rest of them, can we give them the leverage to become that data scientist? And what does that leverage mean? And so what we went and built was a way to lower the barrier to use a product as machine learning. And so today with Base10, you can deploy machine learning models with a couple lines of code. And so what that means is that you go import Base10 in your Python library, you Base10 deploy with your model binary, and we take care of everything. So

• 20:59Speaker 00

  we Dockerize, we create a Docker container, you know, we load it all up, you get an API that's ready to go, that's nice and scalable. And really, like, don't worry about Flask, don't worry about the infrastructure, the servers you'll need behind this to make it work. Do you need GPUs? It's literally, you know, an argument or a check of a box in our UI to put it behind, power it with a GPU. And don't worry about versioning. So you can keep deploying over this and we can do like version roll out and you can switch back. What's most important, though, is like, that to us is the integration cost. Like, this is, this is really exciting that you can do that with a couple lines of code, but that's not really where we see our value. Our value is the stuff that comes after it. Like, can we make it so as a data scientist, a machine learning engineer, you can also write all the business logic on top of that model. So we kind of give you a serverless

• 21:42Speaker 00

  framework to be able to write logic on top of that model. And so like, you can build APIs, you can run things from Kron, you can even, you know, have forks, so you can have decision points within that business logic. But a data scientist or machine learning engineer can do all these things without learning any infrastructure. All they really have to worry about is their Python code. And, you know, that complexity of the infrastructure doesn't vanish, it just shifts. And we think we can take care of that. And all you need to worry about is your code, your Python and your model. Lastly, Something that we loved, you know, at times we're kind of told that we're trying to boil the ocean here, but that's fun to us. The last thing we do that we love is, you know, we always wanted to put a face on our model and interface on that model so that folks could either give feedback on it

• 22:27Speaker 00

  or operate on the output of the model. So if you're building a recommendations model, can you really quickly assemble a UI so someone can input something, run that model that you deployed, see the recommendations and give feedback on each of those recommendations? So the data scientist can take that and really get that iteration cycles going. So you can build that in a drag and drop way without learning any JavaScript, HTML or CSS. And that's really where we arrived for now. What we again hope to be able to do is just give more machine learning and data scientists that leverage so they can kind of ship full stack applications. And hopefully as a result of that, what you'll see is that people see results of machine learning faster. And as a result, they will invest in machine learning more because they'll see what it can do for them. Sorry, I just spoke for a long time.

• 23:11Daniel Whitenack?

  No, it's good. No, that's great. Yeah, really good to get that whole context. One of the things that I find interesting is like I see sort of a trend on my end in terms of ML AI. In that world, there's sort of this trend to kind of no code solutions and that sort of thing where things plug together. What I find interesting about sort of what you just talked through is it's not no code. So the data scientist is still sort of working in their like Python world, but then it lowers the barrier to these other things. So there's like, you know, a data scientist, I think, if we assume they're working in Python, it's not that far, like you say, to maybe have them think about like

• 23:53Daniel Whitenack?

  a flask app or something like that. But then as soon as they try to start getting that out into the world, and they're like, oh, how do we scale this up and other things? And then they find out, oh, there's these like various layers of abstraction that I need to think about, like, you know, containers. And then how does that scale up? Well, now I need multiple instances of my service behind a load balancer and where are all those containers going to run? Well, they're going to run in Kubernetes or some container orchestrator. It's like, it starts to develop in their mind this sort of like inception. Dream within a dream within a dream like they just sort of keep going down that thing and things start becoming just less clear and really strange. So I think like it's cool that you could still let them operate.

• 24:36Daniel Whitenack?

  From my experience, data scientists, you know, and people working in this area, they love opening their Python editor.

• 24:43Speaker 00

  Hundred percent.

• 24:44Daniel Whitenack?

  Writing out their code, really having kind of fine grain control over the preprocessing and fine grain control over how they build their model architecture and all those things. That's sort of where they want to live. So not like saying, sort of taking that away from them and saying, no, don't do that in code anymore. But you know, sort of letting them still enjoy that part of their code, but then adding on these other things, I think is a really interesting way of thinking about it.

• 25:12Speaker 00

  This guy has become a friend of sorts. His name's Peter Stitch. He had this great tweet over the weekend, which was, you know, what I want even more than no code is yes, code, but literally you never have to think about infra at all. And I think that is from our perspective, like the data science, like great articulation of the data scientist. If you're like Python is a really, really valuable tool, you know, you take pandas away from me. I'm a third of the person I was. I want to write Python. I want to, you know, be able to use SQL as well, you know, but I don't want to think about Docker Kubernetes. I don't want to think about, you know, if I deploy a model, how do I switch versions? How do I A B test things? How do I monitor things? How do I deal with downtime? You know, code is great. Infra is hard.

• 25:55Chris Benson?

  Yeah. Before I ask the next thing, I gotta say you have some really good one liners in there that I'm going to steal for later on.

• 26:01Speaker 00

  Please do.

• 26:01Chris Benson?

  The boil the ocean. Well, that's what I like to do is an excellent one because I hear that all the time.

• 26:07Daniel Whitenack?

  Every day of my life. Yeah.

• 26:09Chris Benson?

  Yeah, you've had several there. So I'm stealing your stuff, dude.

• 26:12Speaker 00

  I appreciate it.

• 26:13Chris Benson?

  I wanted to ask, so one of the things that it sounds like you've done a really good job on is by. Taking those levels of abstraction, as Daniel referred to it, the dream within the dream, you can only handle so many layers of abstraction. You can't do it all. And it sounds like by taking control of some of that, that somebody can use base 10 to integrate in with an existing environment by using the APIs. And so they get a big payout and they don't have to worry about all of the things, whether you know, from containers, you name it, all the things that Daniel just talked about, and yet can get the benefit that way, which is quite beautiful, because I know that it's kind of where people have been trying to drive to. And it sounds like you've gotten a very nice elegant solution

• 26:56Chris Benson?

  for achieving that. Is that fair? Is that kind of how you would think of using it?

• 27:00Speaker 00

  Yeah, I think we're very early on and early users will tell you that there's a lot more elegance to be added, but

• 27:06Chris Benson?

  There always is.

• 27:07Speaker 00

  That is, from like an abstraction perspective, absolutely. You know, like I think one of the things that we did early on was like, we kind of defined kind of the principles of the architecture that we wanted to build. And I think like two of them that really stand out here are like the principle of least astonishment, you know, the like, take it away, but don't make it magic. I think that's like, you know, it's like hide it from me, but still give me details of what you're doing.

• 27:30Chris Benson?

  You have a blog post that includes that information, I noticed, which you could talk a little bit about here, because I was going to bring that up as well.

• 27:35Speaker 00

  Yeah. So the principle of least astonishment is really important to us, just because, again, like data scientists and machine learning engineers, we're kind of like these pseudo engineers, right? We're somewhere in between. We know what's going on. Maybe we don't know kind of all the intricacies. So all the abstraction, we've tried to keep quite similar to, you know, how firstly, like what data science and machine learning engineers understand. And two, you know, even like building an application for end user, like there's a backend, there's a frontend, and you'll see these concepts in base 10. And we separate them out and we allow the interplay of them, but we're not hiding them all together. I think, you know, the second thing, which I think you've kind of talking about is like, this is something that we saw from one of our early engineers, was like, you know, easy things are easy and hard things are possible. And like, to me, that's such a great line.

• 28:19Chris Benson?

  It is.

• 28:19Speaker 00

  Which is like, you know, we want to 80 or 90% of the things to be simple. And you know, you don't have to think about it and they just happen. But that shouldn't come at the expense of control and visibility. And I think that's one place where A lot of kind of like lower code or no code abstractions get dinged with that they hide things away and you're kind of like, well, how did that happen? That kind of scares me. We are exposing these things and so you can go deeper if you want with each of our concepts because you know, like we are built on Docker, K native, Postgres. You know, you can write Postgres queries, you can kind of inspect those technologies going deeper if necessary.

• 28:59Speaker 01

  This episode is brought to you by Snowplow Analytics. Snowplow is the behavioral data management platform for data teams. Maximize the value of your behavioral data using Snowplow Insights. A managed data platform that's built on leading open source tech leveraged by tens of thousands of users. Capture and process high quality behavioral data from all your platforms and your products and deliver that data to your cloud destination of choice. When marketing needs to make data informed decisions, when product needs next level understanding and when analytics needs rich and accurate data, Snowplow is the solution for data teams who want to manage the collection, processing and warehousing of data across all their platforms and products.

• 29:40Speaker 01

  Get started and experience Snowplow data for yourself at snowplowanalytics.com. Again, snowplowanalytics.com.

• 30:02Daniel Whitenack?

  So with your solution, we've talked about several different things. So you talked about the sort of, you know, importing a few lines of code, deploying your model and also this sort of UI builder. Could you kind of describe like for someone maybe that's listening to this that doesn't have a webpage up, could you sort of, you know, just describe a bit like the developer workflow in terms of what they would need to do with Base10 versus in their code to take their model from local model to an API and then also maybe to build in one of these UI apps around it? What does that workflow practically look like at this point?

• 30:39Speaker 00

  It's a really good question. So you start the workflow from where you do your work. So from a Jupiter notebook. I mentioned earlier, you can import base10 and base10 dot deploy your model. That's it. And so.

• 30:49Daniel Whitenack?

  And that would be the REST API sort of framework?

• 30:52Speaker 00

  Yeah, exactly. So like basically, you know, we work with PyTorch, scikit-learn and TensorFlow models right now. You can also write a custom model. So if you just create a class with a load and predict method, basically every time the model gets run, that predict method will get called. The load is kind of the deployment step. Just again, easy things, easy, hard things, possible. Like if you want to do more, you can do those things. But once you deploy it with that single line of code or a couple of lines of code, what you end up with is like a REST API. You'll go to base10.co, you'll be able to see the status of your model all the times it's been called. But then you get instructions on how to call it from your own services.

• 31:32Speaker 00

  Straight away, once you're there, you can easily move from the realm of the model to the realm of the application. And that's what we're really excited about. And so, you know, for that model, you can then write all the preprocessing code and post-processing code within base10 in the browser around it. So think of it kind of like a, you know, we're still iterating on this and figuring out the exact right experience. But right now, it kind of represents DAG that you're used to in Airflow. So you can write the code within base10, it's representing a graph-like structure. Again, there is a shared state in context between each of those nodes in that DAG. So, you know, you can reference things from the preprocessing node and the post-processing node, but you can do other things as well. You know, if you want to call the Twitter API, it's a Python-like environment, so you can bring in whatever libraries you want.

• 32:17Speaker 00

  Once you've done that, now that entire thing, we already gave you the REST API for that model. That entire thing is callable by an API. You can also trigger it from a cron job. So we've got built-in support for cron all in line. Or you can call it from a streaming data source. Like if you have a Kafka queue, we've built those integrations as well. So, you know, you've really gone from. Just a model in your notebook to something that was deployed with an API really quickly. And then in, you know, just a little bit more, you can start to write the kind of pre-processing and post-processing code and logic within base 10. Put that behind an API. And so really what we're seeing is stuff that would take, you know, three to five days of kind of wrangling and getting it set up without really even thinking about how to get it on AWS with an API.

• 33:02Speaker 00

  I don't like exaggerating, but it really is like within, you know, 30 to 40 minutes, we've had live services running with machine learning models. I think the last thing now, as I mentioned earlier, is that now you have kind of like, so you have this model, now you have these API endpoints. Then you can start to build UI within base 10 and link actions within that UI to those API endpoints.

• 33:24Daniel Whitenack?

  So with this deploy method, one of the things I'm thinking about just from your perspective in terms of developing this infrastructure is all of the crazy stuff that like varied frameworks and architectures and everything that people are using. So they have all sorts of crazy dependencies. Some people are using, like you're saying, TensorFlow. Some people are using PyTorch. Right now, how, how do you approach that sort of dependency stuff? So you mentioned supporting like TensorFlow PyTorch models, scikit-learn models. What about the sort of other dependency stuff? Is it a matter of pulling that from a person's like virtual environment or how do you go about handling that side of things?

• 34:04Speaker 00

  Yeah. Again, as I said, like with the PyTorch and TensorFlow and scikit-learn stuff, like you don't have to really worry about that. With the custom model stuff, we do need that. We do need to know like what requirements you need. So really, when you deploy that custom model, you also pass us a requirements.txt. And then we basically set up the environment with that ready to go.

• 34:23Daniel Whitenack?

  Cool.

• 34:23Speaker 00

  And yeah.

• 34:24Daniel Whitenack?

  Yeah. And I, I'm trying to, so at the beginning of this conversation, I think this conversation has clarified a lot for me. And sort of where I'm distilling things down is that it seems like if you look at this sort of landscape of developer tooling for AI people, there's sort of some things kind of over in one area around ML Ops and like experiment management and like logging experiments and all of the thing. You can run your Jupiter notebook and log your experiments and put your job in queues for GPUs or whatever. And then you've sort of got maybe on the other side of things like. Widget and prototype type builders, you got sort of like Streamlit or Gradio or something.

• 35:07Daniel Whitenack?

  In the middle, maybe you have like, maybe some serving frameworks that people use. Maybe it's like TensorFlow serving or, you know, something like that. But this sort of like path from like left, you know, from one side of that to the other, just like involves so many different like jumps between different, different sort of systems and spinning up all sorts of different kind of infrastructure and all of that. And I think all of those tools have obviously some really interesting features. But it almost seems like you're sort of, not all of it, but you're gluing many parts of that together in sort of a consistent workflow for people that really could, you know, sort of accelerate them

• 35:50Daniel Whitenack?

  through that process from experimentation to deployment. Am I characterizing that right? I'm trying to think of like how people might sort of view this with reference to like other things in the landscape they might be familiar with.

• 36:03Speaker 00

  Yeah, I think that's a really good characterization of it. I think, you know, we talked a bit about this last week on our calls. Like, you know, this is pretty wild, like how like the ML Ops tooling has evolved over the last couple of years. You know, there's so many great point solutions out there. And, you know, that's kind of the ML Ops side on like the Streamlit Hex side. You know, there's some really great kind of notebook-based application builders that are out there. You know, putting all these things together is quite difficult. And, you know, also like having to jump between three or four different tools to kind of have that end-to-end thing working is pretty hard and, you know,

• 36:38Daniel Whitenack?

  It's complex.

• 36:39Speaker 00

  It takes, you know, time, patience, budget, you know, and most of these tools, including us, you know, we're in beta still, like you can't just sign up and you're ready to go, you know? So it is difficult to do. What we do want to be able to do is, again, when you're a RUBA and you have, you know, literally millions of requests per second going through, you know, base ten is not the right tool for you. You know, you need something specialized. When you're working on something that, you know, in its infancy, you know, it's critical, but, you know, it doesn't need that type of latency or doesn't need that sort of specialization. What we hope is that base ten can kind of fit the bill of that end-to-end solution. So you have your model. And rather than waiting for DevOps to come and deploy your model, waiting for a front-end engineer, product engineer to come and build the things that make that model relevant for like 80 or 90% of

• 37:23Speaker 00

  the use cases, we think Base10 can be that end-to-end solution to, one, get your prototypes and toys out the door, and two, in the end, build those full-stack mission-critical applications as well.

• 37:35Chris Benson?

  As you're looking now, you know, you're in beta, you're approaching release, what do you have in mind? What needs to be there that may not be there yet that you're envisioning for release? Are there any new features, or is it solidifying the things that you already have?

• 37:48Speaker 00

  Yeah.

• 37:49Chris Benson?

  In Base10, how are you seeing that 1.0 release in the short term?

• 37:53Speaker 00

  Yeah, totally. That's a really good question. So we did like a launch last week, which was well received, and you know, we got a ton of inbound from that, which has been great. But really, what we're trying to figure out in this next phase is usability. You know, again, there's like the user and the use cases, the intricate and the complex, and you're figuring out, hey, how can I get you to value? How quickly can you have your aha moment and be using this in something real? We want to solve that, and we want to solve that world before we, you know, open this up to kind of the world and say, go wild, because we know that, you know, a lot of those decisions are easy to iterate on and change, you know, once you are working with a small, core group of users who, where you know the values to be added.

• 38:30Daniel Whitenack?

  As you're having all of these discussions with users, obviously, right now, you're talking to a lot of people, you're getting feedback, so you're sort of seeing a cross-section of the industry to some degree. As you're having that perspective and maybe looking at the industry more generally, as we close up here, is there anything in terms of the AI ML industry, or maybe the developer tooling industry, that just sort of generally excites you in terms of where the industry is headed, or maybe capabilities that are being developed, or other things? What's on your mind?

• 39:01Speaker 00

  Yeah.

• 39:01Daniel Whitenack?

  I mean, you're obviously focused a lot on Base10 right now, but sort of, what are those things that you're thinking about towards the future?

• 39:09Speaker 00

  Yeah, so there's two things that I keep thinking about, which make me really, really excited. So the first one is, One of our investors has a really good campus recruiting program. And so we're able to spend a lot of time with new grads. What's really exciting to me is that, you know, nine out of 10 software engineers, not data scientists, coming out of, you know, the best schools in the country or honestly, any computer science degree have exposure to AI and ML. They're taking a couple of courses. At the very least, they can phrase, they can frame a problem in terms of how machine learning can solve it. So, and that to me is amazing because that means that the amount of evangelization that needs to be done within a company is is lower because people already appreciate what can happen with that.

• 39:50Speaker 00

  And then the second thing, which should be considered in line with this or like together with this is the prevalence of pre-trained models is going up. You know, hugging face has done so, so freaking well over the last couple of years. And, you know, every company, almost every company that we talk to who's starting to some NLP initiative is starting with hugging face, maybe with some fine tuning step. But to me, that's really exciting because you have two things going on, right? So you have one thing going on, which is more and more people know what ML can do. And two is that there's all these ways to have that model ready to go. Like you don't need that, you know, two or three month effort to get that first model up and running. Put those things together and I think you've got something really special, right? So you have, you know, people understand what the tool can do and more powerful tools

• 40:34Speaker 00

  in the form of, you know, pre-trained models. And I think, you know, that's one thing that we're kind of leaning into as well. Even with our user base is that we're not only considering data scientists, machine learning engineers as our user base. But are you an engineer who knows what ML can do for you? Cause if you are, we hook up all those pre-trained models and give you a model zoo. So if you want to have a zero-shot classifier, like, you know, you don't really have to think about anything. You can have it embedded in the workflow in an application with base 10, as long as you know how to frame the problem. And so, you know, like I've tied it back to ourselves, but you know, from like a high level, they're the two things that I'm really, really excited about.

• 41:09Daniel Whitenack?

  That's awesome. Yeah, those are really exciting. It is really cool to see like most conversations now that I'm having with teams and companies. Like you don't have to sort of get over the hurdle of like, we expect AI ML to give us some value. It's more like entering into the follow-up discussion to that, which is where is their value? Where should we focus and that sort of thing? And so technical teams are really thinking about this a lot, which is cool. So thank you so much, Tuheen, for joining us. This is really exciting. Congratulations on your launch. It's awesome and we really hope that our listeners, please check out what Base10 is doing. We'll have links in our show notes, so check them out. Talk to us about some of your opinions about this in our Slack channel.

• 41:52Daniel Whitenack?

  You can always find us in one of our communities on LinkedIn or go to changelog.com/community. Join our Slack channel and happy to chat with you about Base10 and other things. So yeah, thank you so much, Tuheen, and we'll let you get back to work.

• 42:06Speaker 00

  Thank you guys so much. You guys are really nice and it was really nice to be on the show.

• 42:13Speaker 02

  Thank you for listening to Practical AI. We appreciate your time and your attention. If you enjoyed this episode, help us out by spreading the word. Think of a friend, think of a colleague, somebody who would benefit from listening to it and send them a link. We'd really appreciate it. Practical AI is hosted by Chris Benson and Daniel Whitenack. It's produced by Jared Santo with music by Brake Master Cylinder. Thanks again to our sponsors, Fastly, Linode and Launch Darkly. That's our show. We hope you enjoyed it and we'll talk to you again next week.