Putting AI in a box at MachineBox

Mat Ryer and David Hernandez joined Daniel and Chris to talk about MachineBox, building a company around AI, and democratizing AI. Sponsors: Hired https://hired.com/practicalai – Salary and benefits upfront? Yes please. Our listeners get a double hiring bonus of $600! Or, refer a friend and get a check for $1,337 when they accept a job. On Hired companies send you offers with salary, benefits, and even equity upfront. You are in full control of the process. Learn more at hired.com/practicalai https://hired.com/practicalai. Rollbar https://rollbar.com/changelog – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Linode https://linode.com/changelog – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog https://linode.com/changelogFeaturing: Mat Ryer – Website http://matryer.com/, GitHub https://github.com/matryer, LinkedIn https://www.linkedin.com/in/matryer, X https://x.com/matryerDavid Hernandez – GitHub https://github.com/dahernan, X https://x.com/dahernanChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: MachineBox https://machinebox.io/Books “Go Programming Blueprints” by Mat Ryer https://www.amazon.com/dp/1786468948Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Baiting for change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at change law because of Rollbar. Check them out at rollbar.com and we're hosted on Lino servers. Head to linode.com slash change log. This episode of Practical AI is brought to you by hired. One thing people hate doing is searching for a new job. It's so painful to search through open positions on every job board under the sun. The process to find a new job is such a mess. If only there was an easier way. Well, I'm here to tell you there is. Our friends at hired have made it so that companies send you offers with salary, benefits, and even equity upfront. All you have to do is answer a few questions to showcase who you are and what type of job you're looking for.

• 0:45Speaker 00

  They work with more than 6,000 companies from startups to large publicly traded companies in 14 major tech hubs in North America and Europe. You get to see all of your interview requests. You can accept, reject, or make changes to their offer even before you talk with anyone. And it's totally free. This isn't going to cost you anything. It's not like you have to go there and spend money to get this opportunity. And if you get a job through hire, they're even going to give you a bonus. Normally it's $300, but because you're a listener of Practical AI, it's $600 instead. Even if you're not looking for a job, you can refer a friend and hire will send you a check for $1,337 when they accept the job. As you can see, hired makes it too easy. Get started at hired.com slash Practical AI.

• 1:38Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slag with us around various topics of the show at change.com slash community. Follow us on Twitter, we're at Practical AI FM. And now onto the show.

• 2:03Daniel Whitenack?

  Welcome to Matt and David from Machine Box. It's great to have you here on Practical AI. I know that Chris and I, when we started thinking about guests for Practical AI, and I was thinking about our slogan or our mantra at Practical AI, which is making artificial intelligence practical, productive, and accessible to everyone. I know the first people that came to my mind were Matt and David from Machine Box. So, welcome, guys. Thank you very much. It's great to be here.

• 2:33Speaker 02

  Yeah, thank you.

• 2:34Daniel Whitenack?

  Yeah. Let's start out maybe with just a kind of brief, personal intro from both of you guys. So, David, why don't you start by giving us a little personal intro?

• 2:44Speaker 02

  Yeah, I'm David Hernandez. So, my background is in computer science and engineering. So, I was studying back more than 10 years ago in the uni, machine learning and artificial intelligence. Mostly, I never used it till recently, the last three years that the machine learning was bombing. And I started to get through, implement, refresh my knowledge, and I started to implement things. And probably that's how we started Machine Box in some way. And professionally, I've been developing since I finished my degree. It's almost also 10 years doing distributed systems, website.

• 3:29Speaker 02

  My highlights probably, I've been, I worked at the BBC in 2012 for the Olympics. So, we were delivering the real-time system to all the stats and all the video player data for the Olympics, basically. It was a real nice project. So, yeah, that's it.

• 3:53Daniel Whitenack?

  Sounds great. And Matt, why don't you give us a little bit of an idea of where you're coming from?

• 3:58Speaker 03

  Yeah, sure. So hi, my name's Mat Ryer. I've been doing computer science all my career in various forms. I spend a lot of time in the Go community at the moment. I kind of fell in love with Go as a language before it was released as version one. There's a little experimental tag in Google App Engine and I wanted to build something on App Engine and anything that's got a little experimental tag is going to grab my attention. It always has. So I jumped into the language kind of quite early and I've just been kind of using Go since then, really, wherever I can. And it turns out you can use it everywhere. I speak at conferences about Go mainly and also I have a book, Go Programming Blueprints,

• 4:44Speaker 03

  which is nice because you build real projects in that book. So it's not a book where you learn the language or you just learn theory. You actually build real things. So it's very practical and very pragmatic. And that's why I quite like the way you guys are approaching this podcast. Because, you know, complicated things can be extremely powerful, but they're very difficult for people to kind of marshal and get, you know, get into a shape that they can put into production. And that's really our philosophy at Machinebox is to give people a head start on that and get them into production much quicker.

• 5:20Chris Benson?

  And hey, Matt, while we're at it, what's the name of your book before we go on?

• 5:24Speaker 03

  Oh, it's called, thank you. It's called Go Programming Blueprints: Second Edition.

• 5:30Chris Benson?

  Okay, thank you very much.

• 5:31Speaker 03

  Yeah, you don't have to say it in that accent, but it helps.

• 5:34Chris Benson?

  It sounds much better.

• 5:35Daniel Whitenack?

  I think it does help.

• 5:37Chris Benson?

  So how did you guys originally meet and how did you start thinking about forming a company together that's focusing on AI?

• 5:46Speaker 02

  Well, that's interesting story. So, um, back a few years ago, uh, I was one of the organizers of Gollan UK, the first one, now GopherCon UK, or just GopherCon, I don't know.

• 6:00Speaker 03

  GopherCon UK.

• 6:01Speaker 02

  GopherCon UK. So, uh, Matt was one of the speakers, so I met actually Matt in that conference. He was in, in another company, uh, before, and I was looking kind of for a job. I was, I was contractor at the time, so I, I joined the same company that Matt was, and we, we met there, basically. Uh, we, we worked there for, for a few years.

• 6:26Speaker 03

  Yeah, David has a, a really kind of unique ability to, uh, think very clearly a- a- about big problems that are, um, otherwise very complicated. And that's, that's a key skill for any team to have. If you can bring somebody in that can look at these kind of big, broad problems, like massive kind of scale, planet scale sort of problems. Like you said, like David mentioned earlier, he, he was part of the team that, that delivered the software that ran the Olympics. And you can't, there's no dry runs of that. You can't say to everyone, guys, can we just, like a week before, can we just have another Olympics? Just so we can test out all the, uh, all the software. Um, they'd probably say no.

• 7:09Speaker 03

  Uh, so having somebody like that on a team is, is invaluable. And it was very natural when it came to looking at machine, machine learning, um, and David's expertise in it. It was kind of, um, his eureka moment where he said, you know, we could actually kind of containerize this and deliver it in a way that makes it very trivial for everybody to use machine learning capabilities, rather than having to learn TensorFlow and work in these kind of abstract mathematical models. We could, we could tell some stories differently.

• 7:48Speaker 03

  We could give people an API that, that just makes it very clear and sort of changes the way you think about the problem a little bit. It focuses really on the problem, what you, that you're trying to solve, rather than, um, technical sort of low-level machine learning components that you might use to solve it.

• 8:08Daniel Whitenack?

  Yeah, that's, that's great. And again, I think, you know, I've been a machine box user for quite a while. I think, you know, very soon after you guys launched, I was super excited about it just because of the, the practicality of the, of the project. So in terms of like what machine box is actually, you know, as, as a developer, like if I was wanting to use machine box to do something, what might that something be? And what would be the interface to, to doing that?

• 8:37Speaker 02

  Yeah, sure. So, so machine box is basically we deliver machine learning models in, in Docker containers. So what you basically need is Docker install in your computer that is available

• 8:53Speaker 02

  in any major platform, Windows, Mac and Linux. You just Docker pull one of our images. You have a nice API in, in our images. So you only need to know about HTTP, HTTP APIs to, to, to get started and do, for example, face recognition. That is one of most, our most famous boxes. So you can add face recognition to your stack in, in just minutes. So that's basic tools. Docker, a little nodeless to, to do HTTP APIs as a programmer that probably

• 9:33Speaker 02

  every programmer should, should learn that skill nowadays. And that's basically, you don't need any, any, any other knowledge.

• 9:43Daniel Whitenack?

  So just to, just to clarify, I mean, really, like if I was a, you know, a data scientist or a developer or whatever I am, you know, there's a lot of APIs out there, both, you know, from the cloud platforms, like with, you know, machine learning, but also other things, like if I want to send an email or something programmatically, there's, there's like a REST API for, for that, which uses HTTP and, and JSON. And so you're saying kind of one of your goals is to really make the interface to doing something complicated. Maybe like facial recognition or something as easy as it is to, you know, send an email via via one of those APIs. Is that is that kind of a good?

• 10:25Speaker 03

  Yeah, that's that's exactly right. Yeah. So essentially, the machine learning that that's going on inside the boxes is very complicated. And sometimes we mix different kinds of technologies in different ways where, and if we tried to explain how to do that, it would be very complex. And I think the the deployment would be difficult. And even just managing the dependencies would be a bit of a nightmare. So we take on all that pain and provide APIs that tell different stories. So for example, you mentioned facial recognition. Facebook is a Docker container. You you download it, you run it, you then have HTTP access.

• 11:08Speaker 03

  And you you the operations you can do are things like, here's an image. Tell me all the faces in that image and give me, you know, the the coordinates of the faces. Not only that, if you recognize these people and who who the face belongs to, tell me who that person is as well. And then there's another API call to teach. And we we support one-shot teaching, which is also pretty kind of rare still, which is but it just means that with one image. So, Daniel, I could take an image of your face and teach Facebook with one example image. And then if we took a big photo, big photograph at a conference and you were in it, Facebook would be able to find you and identify you.

• 11:53Speaker 03

  You know, so you get that facial recognition capability. And it's only a couple of API endpoints you have to learn. It's basically teach this face and here's an image. Who do you who do you see in there? And then, yeah, it's all JSON because we wanted to just feel really familiar and just fit into what people already had. And you know, HTTP and JSON APIs still dominate. They're the simplest to use. You can use them like they're nice because you can just use them in the browser. And when you run one of our boxes, we actually host inside the box, a little private website, which you access through local host 8080. And that website contains all the API documentation, but also lets you interact with the box without even writing any code.

• 12:36Speaker 03

  Because it's very important on our mission to make, first of all, communicate what's possible in a very simple way, and then make that easy to play with and get to use so that people can see the power of it. And then, once they've sold on that, then it's just a question of making that, making the integration easy and operations. And so we're, we're really focusing on that whole, that whole flow end to end. In, in particular, we care about people without any kind of machine learning experience, being able to, to use these powerful technologies.

• 13:10Chris Benson?

  So it sounds like Machine Box is, is, is been, you've taken the machine learning part and abstracted that and put it in a little black box for your end users. Who specifically are you targeting as, as your customer for this?

• 13:25Speaker 03

  Well, we've, we've, we have already paying customers. And so I say already, because although Daniel started playing with Machine Box way before we really launched anything. And one of the nice things about the fact, the way we approach our developer community is we give them the technology for free early and let them just play with it. And that process, what happens is, first of all, any bugs are immediately found and squashed. Luckily, it doesn't happen very often. We do a lot of testing and test-driven development and other techniques which help us when it comes to kind of code quality. But beyond that, we get to validate the way we've told a story and also, you know,

• 14:07Speaker 03

  if the APIs really make sense for the particular way in which their system expects to use a technology like this. So we've had, we see customers of all kinds. We, we, we really only, it's a developer tool. So this is for developers to integrate into their platform. So by and large, all of our audience are developers. But the people that really kind of have so far found it to be useful are people who they understand machine learning in broad terms, some of them. But they know that it's a lot of effort to go to, to build your own things yourself. And then, you know, if you care about the data not leaving your own network, whether that's on prem or your own cloud.

• 14:53Speaker 03

  Because we're just Docker containers, you can spin them up anywhere and scale them anywhere. It's, you know, you keep control of all that data. So it's people who they understand, they have already a need, which is great. They've got a problem that they want to use machine learning to solve. And then they use our APIs to to solve that problem. So they're basically developers of all levels, usually. I mean, some, some of them are just JavaScript developers, some of them are Ruby. We do have a Go SDK, so we have a lot of Gophers, we have a lot of Go people that are using it. So it's really that, that's, that's who we target is basically, anyone's a potential target. But specifically, we've seen traction in in developers who

• 15:38Speaker 03

  don't want to have to do all the heavy lifting of machine learning, you just want to get something and get going.

• 15:43Speaker 02

  Yeah, my favorite users are, it's kind of a personal opinion. And it doesn't necessarily mean that that is right. So my favorite users are DevOps, or people doing DevOps, basically. Because they, they basically love it because they usually don't have time or willing to learn any kind of data science, they want to solve a specific problems. And and they find Machine Box and our API is really good and really productive for that. So we, we, we get a lot of love from, from DevOps. The best comments that we, we hear is from people doing DevOps, like, oh, I have this problem. I want to solve it quickly, I want to deploy it quickly.

• 16:27Speaker 02

  And it, it is just the, the perfect tool for, for, for that kind of people. And, and yeah, pretty much.

• 16:39Daniel Whitenack?

  Yeah, that's great. I know personally, I can attest to, you know, just the quality of the models. I know I actually kind of got into a little bit of trouble at a conference because I was showing Facebook and kind of one-shot updating of the of the model and and people didn't believe me that it actually worked worked that well. So that made for

• 17:04Speaker 03

  Yeah, that's happened to us as well. I think in a demo, we've had it where people just think we've spoofed it. Yeah, I know it's surprising because, you know, we're told again and again, for machine learning to be any good, you need massive amounts of training data. So that's why it's and really the solution. I mean, it's kind of a bit secret of what we do, but it's which is just a clever use of technology inside the box, which allows us to provide that. But the thing is, we don't want people to have to worry about how it works. We just want them to know that it works and and integrate it, you know, and and get to MVP really quickly. That's really another one of our goals.

• 17:49Chris Benson?

  You know, a few weeks ago, I was in San Jose at Nvidia's annual GPU Technology Conference. And through my employer, I had I was in a small group meeting with the Nvidia CEO Jensen Huang. And he noted something that I see you guys kind of going toward and he that we're really at a junction where software developers are becoming the targets of machine learning rather than just data scientists. And it will continue to be both. But he noted that that was that was a big strategic initiative on them was to target the software development community, which is somewhat new to these technologies. And it seems that you guys have really centered your strategy around that approach.

• 18:29Speaker 03

  Yes, I mean, that's right. I mean, really, what happened, if I'm being completely honest, is we just built something that we needed to use. We wanted to use some of these technologies, and it's hard, and we had constraints, and, you know, some of them at scale, some of the prices of the machine learning APIs at scale, really, it's really becomes prohibitive. I mean, it's just, it's still quite expensive, and it's still, it's quite valuable, I guess, so that's why. But we weren't, we weren't really kind of too strategic about it in the beginning. We just thought, let's just build, let's build it how we think it should be built,

• 19:11Speaker 03

  and how we would want to use it. And from there, we've then started to see traction, and, you know, some great feedback on our developer experience.

• 19:22Daniel Whitenack?

  Yeah, definitely. And I kind of want to follow up a little bit on those, you know, that idea, kind of that we mentioned around, around the conference talks, is, you know, you kind of use this machine box to do something, and it's doing something complicated under the hood, and it's giving you great, you know, great results. But to some degree, you know, even though you might know generally what's happening in the box, it still is a black box. And there's kind of a lot of back and forth in industry right now, at least in the circles that I kind of frequent around, you know, is treating machine learning and AI models as kind of a black box a good thing or a bad thing? And, you know, AI, you know,

• 20:06Daniel Whitenack?

  like, I can, I can download pre-trained models and that sort of thing that I don't really understand, right, from the TensorFlow repo and other things. And often, really, there, you know, I don't get the kind of results that are, that are, you know, either published results or the kind of quality that's promised from these pre-trained models. Now, the models that you're putting out are definitely, I get, really good quality, but I still don't really know all of what's going on on the inside. So in this case, like, we're treating machine learning and AI models kind of like a black box. Why do you think, at least in certain cases, you know, treating models like this like a black box

• 20:47Daniel Whitenack?

  can be, can be a really good thing? Or maybe what, what are, what are some downsides or cases in which maybe you wouldn't want to treat them like that?

• 20:55Speaker 03

  Yeah, sure.

• 20:57Speaker 02

  Yeah, all the machine box models are kind of a black box. So in that case, we don't have any explainability for any of the models. But also, most of the models are based in neural networks. So nobody has that answer yet in the research. There have some research about it, but nobody knows what is happening inside.

• 21:21Daniel Whitenack?

  So you just mean in terms of the complexity of the models?

• 21:24Speaker 02

  Uh, yeah. But also for use cases. So I mean, for example, if you're gonna deny or accept a credit or an insurance, it's quite important to understand what a model is predicting. And saying, oh, if my income is less than this quantity, the model is gonna say, oh, you're not gonna get the insurance or you're not gonna get the credit. But for example, facial recognition, you care less about why the model is predicting that this is matching a face rather than not matching this other identity.

• 22:07Speaker 02

  So you are more worried about the value that you can extract for that matching rather than the value that you can get explaining why the model is doing. So it's quite a balance and it really depends the use cases. Mostly, our use cases doesn't really matter the explainability. In most of the boxes, we have, for example, classification box that allows you to build any kind of classifier, given text or images. So it may matter most for that kind of model. But in general sense, we're more focused on getting value for the models rather to explain what the models do.

• 22:51Daniel Whitenack?

  Yeah, that's a great point. And I mean, to your guys' point, I think, you know, if if you're not able to put your model into production and get any value out of it via a useful interface, then, you know, really what we're talking about is just, you know, AI research that isn't really applicable in a business setting. So you have to be able to get things into production. And I think that's where this sort of black box treatment, in my opinion, is a really good thing in terms of, you know, providing a unified interface for developers and DevOps people and infrastructure people to interact with a model.

• 23:32Speaker 02

  But yeah, but anyway, it should be, I believe that the research is gonna come through. And someday, we can explain how a neural network do the reasoning and why a prediction is that prediction. So we probably try to keep up with the research. And if that comes through, we, it's a possibility to add it to the boxes.

• 23:56Speaker 03

  Yeah, but those kinds of, those sorts of things, and a lot of the arguments against black boxing are, it's really, I think, people who are deep in machine learning, they know about it, they want to, you know, they want to invest time and resources into kind of building expertise and things like that. Lots of people aren't in a position where they can do that. So we, you know, we, we give them a capability, it's a solution, it's, they are models inside. Sometimes there are multiple ones inside each box. But there's also other things going on in there. So really, it is a solution that, you know, we, the

• 24:39Speaker 03

  only reason really that Machine Box isn't just completely an open source project is that it's just so complicated that it wouldn't be, I don't think, you know, it's not like, it's just kind of a trivial little, little package that would be sensible to open source and everyone can get use out of, to use, to be able to contribute to the Machine Box codebase. I think would be more difficult than other projects. And so that's one of the reservations I have against open sourcing is that. But yeah, so it's really an audience question, I think. If people care deeply and know a lot about machine learning, then maybe they're going to want to pick up TensorFlow and tackle it themselves. If you're an app developer and you want to quickly, you know,

• 25:23Speaker 03

  make your make your software smarter, slotting MachineBox in is just the quickest way to do that.

• 25:31Daniel Whitenack?

  Yeah, and I think it's like not inconsistent with other trends. We're seeing like TensorFlow estimators and that sort of thing, right? Which which is intending to kind of give these modules to people that that will let them practically integrate things.

• 25:44Speaker 02

  Yeah, exactly. Yeah, it's kind of overlapping. They are catching up with with MachineBox.

• 25:52Chris Benson?

  And that was a great transition when you were talking about about the tooling. And under the hood, I assume you're talking about TensorFlow there. What other tooling are you using? Where are you using Go, if any? Love to know how you guys are putting the pieces together.

• 26:09Speaker 02

  Yeah, so the basic stack is in Go. So we basically probably more than 80% of the code is Go, because more than 80% of the code is just APIs and network calls and and these kind of things. And the machine learning models, the training is done in Python. And our favorite frameworks are Keras and TensorFlow. That's mostly what we use for deep learning. We use other ones like more traditional machine learning things like Boop and Bobbit is a really old C library that I quite like.

• 26:54Speaker 02

  But but basically, that's it. This is not not so much machine learning code. We serve all the models in Go and train all the models in Python on even even scripts.

• 27:08Chris Benson?

  And and just out of curiosity and maybe for the audience, why Go for 80% of the stack? What is it about Go? Because so many people in the AI space are doing Python. They're doing C++. You don't hear Go as often. So I'd love to know why that for your selection.

• 27:25Speaker 03

  Yeah, so Go has a deliberately limited language feature set. I once was speaking to a group and I said, "You can't do that many things with Go." And it got a laugh because I realize how it sounds, but what I meant was, the actual language itself doesn't have that many features, which forces the code to therefore be simpler. You know, in some of the more modern languages with OO, you have big type inheritance, you've got all these language features that allow you to build really quite complicated, very clever and complicated things. The Go philosophy is around simplicity, which mirrors exactly what we're doing at Machinebox, so it fits brilliantly.

• 28:06Speaker 03

  Essentially, all of our code is, Go code all kind of looks the same, so it's all familiar. And you get such kind of benefits at development time, but actually more as you maintain the project, you know. So that's why Go wins, I think, from our point of view. Plus, we're fanboys of Go, there's no denying that. We met at a Go conference.

• 28:27Speaker 02

  Yeah. But also, some people are really surprised when they ask, and they may hear about Machinebox in a blog post or at a conference, they contact us and say, "Oh yeah, I like your product." Just out of curiosity, how many people are you? Well, it's just my development. We have some business site with our own, but it's just pretty much three people company right now. And the people get quite surprised, like, "Oh, you did so much. You have so many boxes, so many products in like two people developing and one business developing."

• 29:09Speaker 03

  Yeah, and the answer isn't that we're awesome, although David is. The answer is that we're very selective about what we do. So we deliberately don't do as much as you could do. There's loads of possible things we could... Push into Facebook, for example, and some of them tell you where the eyebrows are. I haven't yet seen a good use case for why you need to know in an image where the eyebrows are. But maybe there is one, but until that, you know, until then, we're not gonna, we're not gonna invest all that time and effort and also, you know, add, add that kind of complexity to the API. So yeah, it's because we pick, we're very selective about what we do.

• 29:51Speaker 03

  We pick the things that we think are just the, the gold from all this potential kind of complexity. And, and we just sort of focus around telling that story and solving that problem. So that's how we're able to do so much, it seems, I think.

• 30:05Speaker 02

  Go is the perfect tool for our, our philosophies. Yes, it fits really well into that mantra, into that mindset. So, so it's, it's the perfect tool for us.

• 30:18Daniel Whitenack?

  I, I think both of you guys are awesome, just to set the record straight.

• 30:22Speaker 03

  Thank you. I was, I was fishing for that.

• 30:24Daniel Whitenack?

  That's why I said it.

• 30:25Speaker 03

  Yeah, I'm glad you picked up on it.

• 30:26Daniel Whitenack?

  I, I figured you were.

• 30:30Daniel Whitenack?

  And not only that, but you've given me my next blog post idea, which is around eyebrow, eyebrow based analysis. Very important stuff.

• 30:39Speaker 03

  Yeah, you can, you can detect sarcasm with it. That's the only use, I think.

• 30:43Daniel Whitenack?

  Yeah, with you. Or maybe anger. Matt, with you, if you had that sarcasm detector, wouldn't it be pegged most of the time?

• 30:52Speaker 03

  Yeah, it would, you can basically just return true.

• 30:55Daniel Whitenack?

  Okay, fair.

• 30:56Speaker 02

  Yeah, that, that could be 99.9 accuracy.

• 31:00Speaker 03

  There was one time where I said something serious and wasn't being sarcastic, but I forget what it was now.

• 31:09Daniel Whitenack?

  So you, you've talked a lot about kind of your, your technology stack, why, why you've chosen Go. One thing I'm curious about, I mean, so I think everybody should use Machine Box in one way or another. But there's a lot of people out there, maybe that are working on data science teams or data engineering teams or whatever it is and are, you know, maybe using TensorFlow to develop and train models that are getting deployed internally into their own. sorts of services and products. I'm curious, kind of, you know, because you consistently produce such high value models that are integrated into your products. Do you have any advice around kind of that progression from training, training your model to

• 31:52Daniel Whitenack?

  kind of getting it deployed within some type of service? Whether that be kind of, you mentioned testing, you know, testing might look differently for machine learning models or AI models than than in other cases. But do you have any kind of advice and and insights around that process from, you know, training your model to actually integrating it into a service? Whether that's integrating Machine Box into your service, or maybe that's integrating your own model into your own internal service?

• 32:19Speaker 02

  Yeah, so I don't really know. So most of the problems are just technology that usually technology you just get it solved with one way or another. So there are a lot of tools coming up these days that solve that problem, well, including Machine Box, but also in TensorFlow, the deployment is getting better. So but I think most important is people. So how this machine learning thing is transforming the way that people see software? Especially talking with customers. Now we have, well, you know, in machine learning, we have a lot of false positives, false negatives. Once you have something in production, they come up with with questions.

• 33:04Speaker 02

  Sometimes the most, the question that most of the customers, ah, so we have this problem. Well, that's not actually a problem. It's just a false positive and there are ways to deal with false positives and false negatives. And changing the mindset to accept that a thing is not a bug. It's a it's a false positive in a machine learning model. It changed the way that you interact with people. It's like, oh, you're not gonna have a machine learning that is 100% accurate. So you have to deal with these situations. And that situation is just you, the way that we are mostly struggling or just trying to get the right conversations with people. And I think that is gonna come up.

• 33:50Speaker 02

  In any software development in the next couple of years like.

• 33:54Speaker 03

  Yeah, our job, one of our big challenges is communicating what's actually going on. Like, you know, we thought we're just gonna deliver face recognition APIs, that's it. Or image recognition, image classification or personalization APIs. And we found that quite quickly, we did, we did actually have to get into the conversation a bit more about, look, this, we don't expect this to, to get everything right a hundred percent of the time. We, we expect it to do a much better job automatically than, than you're doing. Hopefully you can get it to the point where, you know, the exceptions that you have to deal with, if there are any in the workflow, get smaller and smaller.

• 34:35Speaker 03

  But yeah, that's definitely been something we've had to focus on is, is communicating that this is a kind of, unlike other software where you do something and you get a result you don't like, that's a bug. And we've had some bugs opened where it says, I put this image in and it didn't find the face. You know, and of course the image, the face is like turned to the side or it's got a weird shadow on it or just something is weird about it. And then we kind of get into that conversation. It's, well, it isn't really a bug. I mean, uh, you know, it's kind of part of the expected workflow. The question is, how do we then tackle that going forward? From a data scientist's point of view, someone did actually ask if they could put their models into our boxes.

• 35:20Speaker 03

  Because they knew the, the building the models bit, they were good at that. But they had no idea about getting things into production and running them at scale. One of the things, one of the very early kind of rules that we gave ourselves, and this comes, this is kind of common sense now, I think, a little bit, but comes from David's experience building at massive scale for the Olympics in particular, was that we had, you know, we, we had to be able to horizontally scale the boxes just by adding more of them. You know, because scale is, you know, it's fine if you get this awesome technology and it works nice and slow on one machine. But to, to, to, to really get the value from it, in most cases, you wanna run this thing at scale so that it can really, you know.

• 36:05Speaker 03

  Get through work that it needs to get through. And so we did, we spent a lot of time also, which you don't really see apart from the fact it just works. But we spent a lot of time in making sure that this, that these boxes could horizontally scale in a kind of Kubernetes environment where it was just elastic up and down as you needed. And of course, you have to think about what's the state inside the box? How does that work? And various other sort of, you know, will, will just load balancing across the boxes be enough in a, you know, to, to, to get what you want? Or is there more that we need to do? And where does that happen? And all those kinds of things. So, yeah, it's been a great, it's been a great sort of experience building it. And it's even, it's more fun when people start integrating it and paying for it.

• 36:47Speaker 03

  That's, that's when you really feel like you've created something valuable.

• 36:51Daniel Whitenack?

  Yeah, that's, that's great. And I can definitely resonate with, with some of the things you said around kind of exceptions in, in models and that sort of thing. I, I think people too often, in my personal opinion, you know, think about an end to end machine learning or AI model that does everything all the time correctly. And I think that's, you know, to some degree, the wrong thought in, in a lot of cases. Because, you know, when machine learning models fail, it's the same, you know, we, we have an opportunity to refactor them, right? Which is, is in the end, a good thing, right? So just to kind of, you know, getting, getting close to the end here, I was wondering, you know, again, what, what you guys are doing is kind of setting some, some standards as far as interacting with machine learning models.

• 37:36Daniel Whitenack?

  And so I'd love to get more, more advice from you guys. In terms of the, like the skills that data engineers or just kind of developers who don't really consider themselves data scientists or AI researchers, what sort of skills would you kind of recommend them, you know, looking into? Or what kind of skills do they need to begin to start integrating machine learning into their, into their applications?

• 38:03Speaker 03

  I think that, I, I don't think you need that many. It depends how deep you want to go into it. The trajectory that I would recommend to somebody who didn't have any kind of idea about it would Be to start by consuming APIs and and because if if those if if that's good enough, if that works for your case, then you don't have to do anything more. And that's what we've found so far. A lot of our customers have said, you know, we're just gonna, we we're just gonna kind of try this because then we can build MVP quickly and then later we might change it. And then that later never happens because the, you know, the boxes are doing just such a good job that they don't need to then change it. So definitely like any kind of API skill around consuming APIs.

• 38:48Speaker 03

  You know, most people already have those already. And then I think beyond that, it's really just a question of, I think, understanding a little bit more about just the kind of high level concepts, I would say, would be useful like, you know,

• 39:04Speaker 03

  with with with classification box. With classification box, you can create your own classifier with with training, a training set. Now with classification box, you do need a good amount of examples for each class. So, you know, if you, when some people start using it, they have just a couple of images, a couple of examples, and you can't really get a model that's that's useful from from that. So learning things like the sort of softer skills around machine learning, I guess, which is, you know, the kinds of data, the kinds of problems that machine learning is good at, first of all, then what kind of training data are you gonna have? Because machine learning is only as bad as its training data.

• 39:48Speaker 03

  So I think those sorts of things would be the useful for everyone to have. And then if if you're getting into more machine learning technical stuff, then then I don't know.

• 39:58Speaker 02

  Yeah, so in my opinion, you should focus in one type of problem. So the machine learning is quite broad. So if you want to get started, there are many different subfields. So probably just focus in a problem that you have or you want to solve like, I don't know, sentiment analysis or classifying text or something more or less straightforward or in in machine learning work. More or less easy and learning by doing it instead of focusing in maths or or things like that. You you can get easily losing in that sense. So try to solve, try to learn by doing.

• 40:39Speaker 02

  Solve a problem that you have and and see how it goes. And once you have that working, you you have that boost of of energy just or have something that is more or less working. Maybe it's not, it's not the state of the art, it's not very accurate, but it's better than random. So it's it's the machine, the machine is actually learning and and that's just a good feeling. And and probably just, just that is is good to to get started and and get more curiosity and learn about more more things.

• 41:08Chris Benson?

  That sounds great. So let me ask one last question for you as we wind up. So many of the listeners that we have are are trying to figure out how to get into machine learning themselves. And they might be software developers, they might be business people who are intrigued by by what's possible here. And so as as two entrepreneurs who have gone down this road and you have created a business based on on making AI technologies available and and recognizing there's so many people that may want to to either supplement their own business that they have or or create a new business. What what advice do you have for other entrepreneurs that that might be interested in in taking the same adventure

• 41:49Chris Benson?

  that you guys are are now, you know, a couple of years down? What what would you say to them?

• 41:55Speaker 03

  Yeah, I would always say, solve a specific problem. Make sure you're solving a real problem. This goes for any kind of software, actually, but it's too, especially machine learning, because it's all cool and sexy and and hard. Like, machine learning is hard. So anyone that, like David said, if you make some ground, you really you get really kind of big rewards for doing that, like just emotional rewards you get. So, yeah, it's kind of difficult to make sure that you're building something that has some true value, you know, because if you're just building cool tech, then you there's no guarantee that's ever going to be anything. And often.

• 42:37Speaker 03

  Often you can build, you end up building something that technically is brilliant, but actually doesn't quite fit the problem. And then you have to basically move or change what you're doing so that it does solve a real problem. And that can be quite a painful transition. Usually involves adding loads of complexity because it didn't quite, you weren't really thinking about those things from the beginning. So of course, you want to be able to evolve and learn and move, you know, a project along. But I would say, start with a real problem that you understand. And the problem shouldn't be anything to do with machine learning, but machine learning might be part of the solution.

• 43:17Daniel Whitenack?

  Great. Yeah, that's, that's wonderful advice. And we'll include links, of course, to Machinebox and other things that we've talked about, you know, TensorFlow and Keras and Docker and Kubernetes. If you're not familiar with those technologies, we'll include some, some good links to getting started with those and learning more. And just want to, want to thank David and Matt one more time for joining us. It's been, been great to have you here and really excited about what's going on with Machinebox.

• 43:44Speaker 03

  Thank you very much. Yeah. And good luck with the podcast. I think it's awesome. I can't wait for future episodes. I'm sorry to everyone who had to listen to our voices for this episode, but future ones I'm sure will be even more interesting. Yeah, thank you very much. Thank you.

• 44:01Daniel Whitenack?

  Thank you. Appreciate it very much.

• 44:05Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes and give us a rating, go in your podcast app and favorite it. If you are on Twitter or a social network, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for changelog is provided by Fastly. Learn more at fastly.com. And we catch our errors before our users do here at changelog because of Rollbar. Check them out at rollbar.com/changelog. And we're hosted on Linode Cloud servers. Head to linode.com/changelog. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in your email address. Get our weekly

• 44:50Speaker 00

  email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.