Answering recent AI questions from Quora

An amazing panel of AI innovators joined us at the O’Reilly AI conference to answer the most pressing AI questions from Quora. We also discussed trends in the industry and some exciting new advances in FPGA hardware. Sponsors: DigitalOcean https://do.co/changelog – DigitalOcean is simplicity at scale. Whether your business is running one virtual machine or ten thousand, DigitalOcean gets out of your way so your team can build, deploy, and scale faster and more efficiently. New accounts get $100 in credit to use in your first 60 days. Rollbar https://rollbar.com/changelog – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Linode https://linode.com/changelog – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog https://linode.com/changelogFeaturing: Wes Jensen – Website https://www.linkedin.com/in/wejensen/Peter Ma – Website https://www.linkedin.com/in/nyceane/David Ojika – X https://x.com/DaveOjikaVinay Rao – Website https://rocketml.net, X https://x.com/vrstartupOsamah Mohammed Ali – Website https://www.linkedin.com/in/osamah-mohammed-ali-34a85436/Daniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Intel AI Academy https://software.intel.com/en-us/ai-academyIntel DevCloud https://software.intel.com/en-us/ai-academy/devcloudMovidius Neural Compute Stick https://developer.movidius.com/Cloud AutoML https://cloud.google.com/automl/Google’s AutoML: Cutting Through the Hype http://www.fast.ai/2018/07/23/auto-ml-3/MLPerf https://mlperf.org/OpenVino, OpenCV SDK https://software.intel.com/en-us/openvino-toolkitUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Beta for change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at change law because of roll bar. Check them out at rollbar.com and we're hosted on Lino servers. Head to linode.com slash change log. This episode is brought to you by Digital Ocean. They now have CPU optimized droplets with dedicated hyper threads from best in class. Intel CPUs for all your machine learning and batch processing needs. You can easily spin up their one click machine learning and AI application image. This gives you immediate access to Python three, R, Jupiter notebook, TensorFlow, side kit and PyTorch. Use our special link to get $100 credit for Digital Ocean and try it today for free.

• 0:45Speaker 00

  Hit the do.co slash change log. Once again, do.co slash change log.

• 1:03Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence, practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show at change.com slash community, follow us on Twitter, we're at Practical AI FM. And now into the show.

• 1:29Daniel Whitenack?

  All right, I'm really excited to be here at O'Reilly AI with some of the Intel innovators in AI and ML. Really excited to ask these, these guys some questions today and hear from them. But before we do that, let's kind of go around and introduce who's in the room. So I'll start, start with you, Vinay.

• 1:46Speaker 02

  Thank you, Daniel. I'm a huge fan of Practical AI, your podcast. I've listened to every one of your episodes. My name is Vinay Rao. I'm the CEO of Rocket ML. We make distributed machine learning platform to help data scientists do experimentation much faster.

• 2:03Daniel Whitenack?

  Awesome. And maybe everyone could share something that they're excited about in the world of AI too. What about, what about you, Vinay?

• 2:10Speaker 02

  I think the market opportunity for AI, whether it's doing machine learning or putting out products, AI products is very, very large. Everybody's underestimating the market size. My own prediction is that I'm going to go on a limb here. It's going to be a trillion dollar market size.

• 2:28Daniel Whitenack?

  Awesome. So let's go on to Peter.

• 2:30Speaker 03

  So thanks for having me. Daniel, my name is Peter Ma. I'm working on a project called Clean Water AI. It uses a computer vision, artificial intelligence to detect the bacterias and the harmful particles in the water.

• 2:40Daniel Whitenack?

  Awesome. That's really exciting.

• 2:42Speaker 03

  So what I'm really excited about AI is that what it can do and how it can apply into the current world right now from self-driving down to pretty much, you know, repetitive healthcare tasks. And I think in, you know, in the very near future, we're pretty much gonna have using it to reduce all the costs on the healthcare that a lot of doctors, pretty much they're just for simple screening, at least in the United States, that's gonna be very helpful.

• 3:07Daniel Whitenack?

  Yeah, yeah. AI is gonna make a large difference for good, as well as all the, you know, malicious things that we see kind of advertised in industry. Yeah, it's great to get that perspective. What about you, Dave?

• 3:17Speaker 04

  Hey, thanks for having me. So I'm Dave. This is actually my first time on Practical AI, so it's very exciting to be here. So I'm a research assistant or research affiliate. It's a new position I just got at the University of Florida. I recently just finished my PhD, so thank you. So on this side, I'm also looking at other projects. I'm one of the Intel innovators. We had a demo at Intel booth today on Applied AI in retail. What am I excited about? I'm gonna say FPGAs. So FPGAs are essentially like GPUs for acceleration. And I do know that I'm heavily into GPUs, right, with Rocket ML. I'm more into FPGAs for inferencing. So I do see a very large market for inferencing.

• 3:57Speaker 04

  My personal thing three years ago was when scientists were really, like, really going hard to develop ML models. And we're in the era right now, we're trying to take those ML models and put them into production. So there's a much larger market space in that area, for sure.

• 4:10Daniel Whitenack?

  Awesome. Yeah. Ali?

• 4:12Speaker 05

  Yeah, this is Ali with Netrolux. It's my first time with Practical AI. Netrolux, we're excited to use AI in the network, network field. We control the traffic engineering of our core utilizing deep learning. And I agree, GPU and both, because we're pushing FPGA also to the edge of the network. We're working on pushing the AI to the edge of the network.

• 4:36Speaker 06

  So thanks for having me, Dan. I'm awesome. With Netrolux and I saw Lee to join because he's much smarter than I am and the more I learn about this, the more I realize I don't know. And that being said, I like what we're doing because we're, we're kind of the underlying technology to support all of the AI initiatives, which is the network. And optimizing the network and creating a neural behavior and self-correcting behavior in the network, which is 90% Internet. And even more so as we grow into IoT and other things like that. What excites me about AI is actually pretty simple, the adoption of it. How quickly people are going to start trusting it and actually not being scared of a Skynet or something ridiculous. But actually putting faith in it like, like we do here, the group of us. So thanks.

• 5:14Daniel Whitenack?

  Awesome. Yeah. And Ali and Wes, just to kind of clarify, because we do talk about networks a lot, but you guys are talking about networks in the sense of the Internet and interconnected infrastructure, right? Not necessarily neural networks, although you're using neural networks on the network, right?

• 5:31Speaker 06

  That's right. So, so it's funny. I say kind of tongue in cheek to people that we've harnessed the Internet, right? But what that really means is that we're leveraging about 20,000 sensors, plus a footprint of about 68 data centers globally, only 22 of which are in North America. And essentially we just collect data, performance data, latency, jitter, packet loss, throughput, availability, and hundreds of other metrics that we as humans will not even think to account for that impact network performance over the Internet. We take that same intelligence and apply that into security as well. So our security posture equally is strong powered by AI as well, all over the Internet as an OTT software. So a little bit disruptive to the service provider or equipment space, but it's fun.

• 6:08Daniel Whitenack?

  Yeah, that's awesome. So I have something maybe a little bit special for you guys today. Normally we just kind of do a standard interview, but since this is a panel and you guys are all experts in the field, I thought it would be fun to just pull some of the recent questions from Quora about artificial intelligence and machine learning. Get your guys' perspective to hear from some of the experts about how they would answer some of these questions that the people are really asking as they're diving into the field. So does that sound okay for you guys for the challenge?

• 6:39Speaker 06

  Yeah.

• 6:40Daniel Whitenack?

  Awesome. So the first of these questions is, what do I do next when I've achieved a machine learning program at 90? 7% accuracy and good fit. And I think really you guys are laughing a little bit. Stop.

• 6:55Speaker 04

  This was the point I was alluding to, right? So a few years ago, researchers were, you know, working so hard to develop models that would meet 97%, 98%. But what's next after that, right? And so we're in the era right now, we're trying to productionize machine learning models. And it's very exciting to see startups that are coming up almost every day with a workflow to let you take machine learning models, put it into production in a more scalable fashion. So I really think that's the next phase of ML and AI. It's how do you put it into production at the edge, right? Or some IoT device.

• 7:26Daniel Whitenack?

  Yeah, so what's different, maybe to follow up on that, what's different about production or utilizing a model after you've kind of trained it to get a particular evaluation metric that you might be after?

• 7:39Speaker 02

  I think about your original question a little bit differently. 97% might be at particular level of problem statement. But as humans, we know how to split the problem into minute pieces. So something underneath that may not be 97%. As we collect more data, as companies collect more data, the new problems will start to emerge. Again, accuracy within that would be like 70%. So you'd have to increase it. For example, if you take translation as a problem, you know, 96% accuracy, whatever. But maybe some languages it's not. So the problem will shift as the AI practitioners will be chasing unsolved problems all the time.

• 8:23Speaker 06

  I agree. I think we just need to get out of the way. I think we learned so much as soon as we got out of the way and let that machine learning and those algorithms do what they do and what we designed it to do. And we keep getting away and we put a number like 97%. Who am I to define that? In my own mind, I can't comprehend or factualize what the 90% looks like or 97%, right? And you said other languages. We can't assume that it will be that. It has, if you get to a certain level, like we say, what, 85%? We get to 85%, we're super excited. And then the rest, we kind of manage that 15%, that variable, right? But you have to always view it as a variable, not a static, I reached 97%, you know?

• 9:01Speaker 02

  Just to add on that, for example, self-driving car, if you take self-driving car, the accuracy of whatever the little functionality within self-driving car could be 90 plus in daytime, for example. But in nighttime, it may not be. So there will always be challenges in the area.

• 9:18Daniel Whitenack?

  Yeah, I think something I always tell people as well is to think about these accuracy or other evaluation metrics, it's really on a case by case basis because like you said, Wes, it's not always 97%. If I'm in an actual real world scenario and all we need is 80%, I would get fired if I spent, you know, six months trying to get anywhere past that, right? So you definitely have to take it case by case.

• 9:41Speaker 04

  Just to add a little bit to that too, I think it's also led to us, the community, AI community to sort of define what a standard is. So there are efforts like ML Perf where folks from Google, Facebook, Amazon, IBM actually come together to define what a set of metric is, like what is the accuracy, what is the throughput and latency. So I think such efforts actually could help propel a standard benchmarking in ML.

• 10:05Daniel Whitenack?

  Awesome. Well, let's go into the next one of these. The next one is if I can't afford to buy a GPU, like me maybe, for deep learning, what does it mean, does that mean that I can't do any serious neural network training? What do you guys think? Peter?

• 10:22Speaker 03

  So I started the entire AI without a GPU. I actually just got my entire GPU recently. There is a lot of resources online, including Intel's AI Academy, where it gives you, you know, using CPUs on servers that you can train. And also, there's a lot of schools that you can leverage their GPU servers, but you know, you have to wait a little bit and, you know, online that they're generally available, the resources available for you to go train. And for the most part, when you're starting with the AI, I don't even recommend you go training. There's thousands and thousands of pre-trained models that you can actually start practicing. And a lot of times, those are models are like, like some of the models I used right in the beginning, I still have not beaten their benchmark, you

• 11:07Speaker 03

  know. So, so that's the way I see, if you want to get into AI, deep learning, just use pre-existing models first. And then, once you cross that threshold, you have a really, you know, you can go through AI Academy. And then, one, the Intel's AI Academy, once you go through that, you can, you can get funding. I mean, if you're already an AI expert by then.

• 11:25Speaker 05

  Yeah, I agree. And the new development in FPGA and in neural network cores, including that core support inside the new CPUs, making it a lot different. You will be able to see a lot of neural network applications existing today on your smartphone. Android and iOS already support, the SDK is already in there. And you're actually training on a small set, of course, small data set. You're training on the CPU, doing inverse quickly without worrying about the CPU power. So probably fifty, sixty dollar Raspberry Pi-like product will help a lot. And there's a lot out there in the market. Intel Movidius is another good example.

• 12:09Speaker 05

  It's a neural computer stick that you can actually get for under seventy dollar. And using the resources on the AI Academy, Intel AI Academy, you can actually start with a pre-trained model, as you basically mentioned. But you still don't have to go GPU. GPU, unless you already figured out the model size and what you're doing, you're working on a larger scale of data. That's when you need GPU. So it's not ready to start with. It's very advanced stage with AI.

• 12:40Speaker 02

  I'd like to add a point there. So we specialize in distributing machine learning load for training. That's what we specialize in. And we're working with Intel team to show, actually, we have benchmark data that it's already there. But I'm letting the cat out of the bag here a little bit prematurely. We're co-writing a paper with Intel that one can do training faster than GPU, cheaper than GPU for sure, just with commodity CPU. That rocket and Intel combination paper is coming out. So far, you know, there is inferencing and there is training

• 13:24Speaker 02

  where you consume a lot of the hardware compute resources. So far CPUs outperform GPUs in the inferencing. But that wasn't the case in the Training space, but the reason for that was that most of the software people use on top of the hardware is bloated. For example, Apache Spark is great software for some other purpose for training. It's a bloated software. Comes with a lot of baggage that's not required for machine learning training. So we have built a system that can overcome that barrier. And now we are hitting a benchmark below the GPU speeds. So it's much faster than GPU speed.

• 14:07Speaker 02

  So that's an interesting thing that we found out.

• 14:10Daniel Whitenack?

  Awesome. Yeah, that's exciting to hear about. I'll look forward to seeing that paper.

• 14:14Speaker 02

  Thank you.

• 14:15Daniel Whitenack?

  Yeah, I appreciate all of your guys' perspective on that. I think that a lot of people see that as a barrier when it doesn't have to be. And I think, you know, I would emphasize as well that, you know, training isn't the only bit of the AI workflow in general, right? Even if you you're thinking of training only, there's a lot, there's a lot to learn about the fundamentals, about pre-processing, about inference, like you said, Peter. So there's a lot you can learn and we'll post some of those resources in the show notes. Okay, so this next one's kind of interesting. Here you guys' perspective. This question is, why is there a sudden craze of programmers with little math background jumping onto machine learning, which requires a much different skill set than traditional programming? What do you guys think?

• 14:58Speaker 05

  I mean.

• 14:59Daniel Whitenack?

  Have you seen this?

• 15:00Speaker 05

  SDK tools, SDK tools that we see out today made it a lot easier to consume and build and utilize training models compared to start doing everything on your own using Python, for example. I mean, this why, yeah, I mean, if there is a ready to use machine learning model tools that I can go to AWS, for example, and just pull it and immediately start using it and have prediction ready to go. Yeah, you will see a lot of people without mathematical background trying to get into that, but you will face issues. You will fail. You'll need that mathematical background.

• 15:36Speaker 03

  Okay, so as a hacker, you know, I basically built prototypes a lot. So that's one of the trend is that you follow where the entrepreneurs generally follow what the trend is. And the latest trend is basically, you know, blockchain and AI. Blockchain is still really difficult to program, but people gets into it because there's one or two really good applications. Like Bitcoin is, you know, they see Bitcoin as one of the things that's going to change the world. All the programmers get into study blockchain. And they see self-driving car, they start seeing this as being deployed in the real world. They're like, okay, there, this is a time where you have a emerging technology that's being applied in the real world now. And this is a time to, for me to invest my time to then to, to, to understand and to execute and to find opportunities within this space.

• 16:19Speaker 03

  This happened back in mobile, when mobile start getting to people's hand, all of a sudden, all the programmers start jumping on mobile. Because I jumped in before that and I remember just, just being one of the independent Android developers. I was invited to, you know, to give a talk at Ted Global. That was, you know, really rare because, you know, people didn't get into it. But as soon as that, that, that tipping point has been passed through, like we have current applications that are being deployed in the real world now. That's when the programmers all want to jump in, then this is where opportunity is.

• 16:49Speaker 06

  That's such a great point, because if you think about it, you know, if they started from square one, when they found out what was hot on the market right now, like blockchain or AI, they would never catch up. The ideas or where they think the application would be, would be two years out by the time they caught up in calculus, right? And then understanding derivatives or, you know, that's, that's messy. So what they do is, you're right, they go in and advance, they say, hey, I'm just gonna, and then I'll backtrack. And then I'll, I'll try to figure out that, that actual knowledge down the road. And to Ali's point, you know, that can be dangerous, absolutely. So I think to the, I would caution those, don't, don't not innovate and don't not jump into it. Absolutely, because I will say something that will spawn the rest of your minds here, much better than anything I could come up with on my own,

• 17:34Speaker 06

  regardless of what my mathematical history is, right? And I think that's what we need to spawn is innovation, but be careful, I, I totally agree with that, they're, they're following a trend.

• 17:42Daniel Whitenack?

  Yeah, I think there's definitely a balance to be had there. I think, you know, we've had guests previously on the show that don't have a math background, but are still, you know, innovating in a really great way in AI. And so I think that, yeah, I would love to encourage people to get in the field, get their hands dirty, like Peter said, get some pre-trained models, try out some things. This is a new layer in the software stack that people can experiment with. But then, you know, once you start getting into it more, you're going to naturally be drawn into these ideas around the theory behind neural networks and other things. And, you know, you don't have to have a math, I don't know, do any of you guys have a math PhD or something like this? Far from it.

• 18:24Daniel Whitenack?

  Yeah, so I definitely, I definitely don't. And so I think the math is important, like you guys have said, and people will get into it. But you don't have to feel like if you don't have a math PhD, then you can't, can't get into AI and start, start innovating.

• 18:41Speaker 04

  One quick one I want to make is also a question you have to ask yourself as a developer, when the software stack do you see yourself, right? You know, the machine learning software stack is pretty complex and deep. You can look at the analogy of, say, an assembly language developer or a Java developer, like where in that stack do you want to be? Do you want to be writing drivers? You want to be writing assembly language programming? Because very soon AI is really going to be a box, essentially, you're just going to be talking to APIs, right? So do you want to be writing applications for this box or you want to actually build the box, right?

• 19:12Daniel Whitenack?

  Yeah.

• 19:13Speaker 04

  So that's one question the developer has to answer himself.

• 19:14Daniel Whitenack?

  Great point.

• 19:15Speaker 02

  That is so true, very good point.

• 19:16Daniel Whitenack?

  Awesome. So this next one is probably somewhat controversial. So I talked actually to Wochek Zeremba yesterday about, about general intelligence. What is your guys' take on how intelligent is AI now? So the question is, how intelligent is AI now? This was in July, but, you know, now, now we're a little bit later, so maybe it's more intelligent now, but what do you guys think? Very dumb.

• 19:45Speaker 03

  So AI right now, as we, because everything is called AI now because of the buzz like I explained earlier. But right now, it mostly consists of inferencing. That's what I consider deep learning. That's what spawned the entire industry. It becomes a sensor rather than just, rather than intelligence itself, right? Because a computer doesn't know what a cat is before 2012. And the whole point that right now, computers are, this is a cat and this is a dog. That's what the intelligence. And this is all it can do. But with all the other programming of human intelligence that we have combined with what's so-called deep learning, artificial intelligence. This is where the creating the real intelligences. And that can go into some danger field, right?

• 20:27Speaker 03

  Like if you think about not just how it can benefit a human being, but what technology you can do. It gets into very, very dark sides of humans as well as dark sides of a machine itself. For example, you can easily target people using drop link bombs. That's just, that's just something. So as we develop AI, I think this is a thing we have to consider is that right now, it is, everything is dictated by human intelligence. So we have to decide whether we want to like build something that can just drop bomb because they sees human. Because at the end of the day, if you train that model, just detecting humans, you have to figure, you have to have, you know, you have to have something to back it. So you cannot use this to drop bombs.

• 21:09Speaker 06

  Can I summarize your answer? The answer is yes and no, good and bad, smart and not so smart. And as an example, if I apply AI in a retail architecture and I want to do RFID tracking to push advertisement or just to track foot traffic generally in a store or motion tracking for, you know, for my AC. I'm saving power. Simple things like that, right? AI is super smart. AI is brilliant and I see an ROI immediately. But if you take it to a more complex model like what you're saying where it can get really scary, you think about simple things like an autonomous vehicle. Everybody talks about autonomous vehicles. But let's say I'm in a car and let's say I'm in an autonomous vehicle that tracks vitals now because it's self-driving. Maybe it's tracking vitals. Maybe I have a heart attack. Maybe the car needs to.

• 21:53Speaker 06

  Pull over. Does the car understand, which we just taught it what a road is? Does the car even understand the density of the gravel on the side of the road? How far to pull over? How to look over your shoulder and see how much traffic you're gonna cause by pulling over here versus 10 feet ahead? It's that consciousness, and so it requires so much more. And so in that instance, it's not very smart, not yet. Instead, I would argue that it's not autonomous at all in vehicles, but rather automated, programmed, based on a set of parameters, not autonomous at all. And I don't think we're there yet, so.

• 22:24Speaker 02

  Yeah, there are two ways to think about it. One is, it's very simple statement is, easy is hard, hard is easy. What is hard for humans is easy for machines. Like, if you want, ask someone, a data scientist or, you know, whatever, take Larry Page also, and say that, hey, what does the recommendation engine look like? He won't be able to answer that question, but whereas machine learning can do that much better than humans. On the other hand, what's easy for humans, that is, you know, cat is a cat, or whatever it is, that's a lot easier for humans, but for machines, it's harder, but it's getting better and better. Andrew Ng, one of the key guys in the AI segment, we almost worship him.

• 23:08Speaker 02

  He says that anything, I can't remember, I think he said one second, for humans to do an inferencing, this is a badge, or whatever, this is a human being, or this is a mic. If it takes one second, machine can do it at the moment. But I think that that threshold is increasing very rapidly. We increase the compute capacity, data quality, and of course, there is algorithms, the math behind it, as people invent more new math, ways of representing the real world. It's all getting faster, rapidly, at an exponential rate.

• 23:44Speaker 02

  And I think there is some threat because of that. And also, at the same time, you could solve significant problems. For example, in cardiology or in radiology, medical spaces, you can literally make a huge difference to the human beings.

• 23:59Daniel Whitenack?

  Awesome, yeah, thanks for your perspective on that. So, this one probably built on that one, and maybe this is a point of confusion for a lot of people. So, the question is, is a neural network actually a good model of how the brain works? So, I, we-

• 24:15Speaker 05

  I mean, if you look at a perceptron as an example, I mean, the way your brain actually get signal from sensor in your, in your fingers and the way you react to it. Just building that same model in a neural network, it, it may look easy and it may work for a lot of application. As, as we said, we can actually have machine learning as models. But putting these models together, the sensors together, to actually make an intelligent decision based on all these, I don't think this is how the brain work. I mean, the, the way, the, the, the way you may sometime think about a, like a smell or a, or a, you hear a music, it remind you of something, you did not, you don't actually how that worked.

• 24:59Speaker 05

  You don't have any idea how it worked. With the machine, the consciousness is not there. You don't expect it to, to be able to do that or come up with new ideas on some. This is hard.

• 25:11Speaker 02

  I would just add on that, I think there is a symbiotic relationship between neuroscience and neural machine, neural network, people who are trying to hack, biohack, you know, what they think, the, the way the brain works into machines. I've heard neuroscientists say that, you know, by reading the papers of mathematicians writing and about neural network, they learn a lot about the brain. So, there's still a large areas of how the brain works we, we don't understand, even though neuroscience is improving quite a bit. So, this is an interesting, definitely a very intriguing area to keep an eye on.

• 25:54Speaker 03

  So I just want to add something that's just like, I think the neural network is just one part of the brain, right? We, as a part of the brain, we have a lot of things. We have memory, which computer have, you know, built a hard drive that's specifically just simulating our memory. And neural network is basically just classification, object detection, and that is part of that. And as we do more computer science and more AI in the future, we're going to uncover different parts of the brain that's, you know, some of them is going to react to smell that currently where it is hardwired, but is that, is it human hardwired, is that learned? And these are things that we're going to come, like, is going to be very interesting next few years.

• 26:30Daniel Whitenack?

  So kind of building on what we were talking about, about what a neural network is, I'll jump into this next question. The question is, in a neural network, each neuron is a hidden layer, in a hidden layer is said to focus on a certain feature. Take an eye for an example, but how does it deduce that an eye is an eye? Or since it can contain a combination of things like an eyebrow and, you know, other parts of the face and the pupil and all of those things. So any one of you guys want to take a stab at describing that?

• 27:02Speaker 02

  I think it depends on the, how much data we provide the machines. You know, with enough data, a machine is able to parse out those similarities. Everything that machine does is just trying to match a pattern. And every layer helps match one little pattern at a time. So that's how the neural nets, deep neural net, keep adding layers and layers. And each layer has quite interesting set of information that it can make use of for some other purpose, too.

• 27:35Daniel Whitenack?

  Yeah, so maybe a follow-up on that. I mean, there's a lot of talk about deep neural networks now. Is the purpose of all of these hidden layers in the deep neural networks to detect these more complex patterns? Or why have the deep neural networks kind of advanced so far?

• 27:51Speaker 05

  It's basically like in your brain, the way your brain have a receptor field. Like when you think about the way your brain react to caffeine, for example, or to nicotine. I mean, it bind with certain perceptron and neurons in your brain to actually feel that. And now, the same thing, I mean, with these layers, you got a lot of filtering. You got a lot of control that happening in there. And part of that is basically having multiple receptor fields. So you're actually going faster because if the receptor field doesn't match, you're skipping, you're actually not going in that neural network, completely going through it. And remember, you're doing that thousands and thousands and thousands of time during the training level.

• 28:37Speaker 05

  So basically, adding the receptor fields in there, it changed the way we see it. It changed the way we control it. But it's still, it's the way neural network will work is gonna go more about distribution than just as a single node network that work together. You're gonna have multiple nodes making decision together. That's just what makes sense, basically. I mean, distributing the decision making instead of deciding on one type of neurons. For example, with certain sigmoid control, you're gonna have multiple. Based on the decision of the first one, which one to trigger, which one to attack. That's how the eye work, actually. There is multiple layers of filtration that happen and sensitivity that change with the lights around you.

• 29:22Speaker 05

  We're far away from that, but cameras will have these technologies slowly. As we understand more, we'll build more into the neural network.

• 29:31Speaker 04

  You know, there's that constant debate if machine learning is actually science or arts. You know, so I think it's really hard for data scientists or a non-machine learning expert to kind of figure out what layer should I put where? How many neurons should a single layer have, right? But I think efforts like AutoML are actually trying to solve that problem. So you can install AutoML or such tool and you're ready to go. But time will tell, really see how far we get close to getting accurate models based on such tools.

• 29:58Daniel Whitenack?

  Yeah, and just describe AutoML a little bit because people might not be familiar. I think Wes was gonna say something about it.

• 30:07Speaker 06

  I was actually just gonna say, these questions are so presumptuous, right? They're, again, they're human questions about a non-human platform, and they're presumptuous, right? Peter, right, you mentioned about, you know, how neurons are only perceptrons and how those impact the whole model, right? And now we're only focusing on maybe two elements of our brains, storage, right? And potentially some reasoning, right, yes, and decision making. But the fact is, maybe AI will evolve even past that to where it will define that maybe our brain isn't connected the way it's most optimal in a compute environment. And so the questions are presumptuous. So when you answer some of these questions, we have to answer based on unknowns. It's the easiest analogy I like to use is when you say, oh, that planet can't support life.

• 30:50Speaker 06

  Who are you to define what life is and what another life form requires to exist, right? How would you even perceive to know that, right? And so some of these questions, they have to be open-ended, and you have to keep your mind open to what is a much larger picture than what we can even think can exist.

• 31:05Daniel Whitenack?

  I'm glad that we've covered the full gamut between neural networks and aliens. So I'm happy that we made it there.

• 31:11Speaker 06

  I did that for you.

• 31:13Daniel Whitenack?

  So yeah, just briefly before we go on, I'll circle back to Dave and let him mention what AutoML is so people can.

• 31:20Speaker 04

  Yes, typically when you're training a machine or a model, you're tuning a bunch of parameters like, or hyperparameters, like, you know, back size, learning rate, iteration, and whatnot. So AutoML lets you sort of tune those knobs here and there and then gives you like a way to accelerate the process, essentially. The other tools to like AutoML, but that's one of my favorite.

• 31:40Speaker 02

  Awesome. Yeah, just to add on that, you know, great models are built through iterations. Iterations are very tedious and taxing for data scientists to do it. AutoML reduces that extra effort quite a bit. It's a machine learning on machine learning.

• 32:02Daniel Whitenack?

  Makes sense. And we'll put some links in the show notes, of course, to some of those resources. So the next question, if deep learning is the future, which maybe everyone says it is, what is the need for machine learning methods like SVM, decision trees, Bayesian methods, Markov chains, etc.?

• 32:20Speaker 05

  I mean, each accelerate in a different field. I mean, deep learning, deep learning will not work for a lot of mathematical equations that we deal with in the network field today because the data type, the way it behave, it changed with a lot of factors that is not fitted in a table. I cannot just fit it in a table and have parameters define it. So there is a lot of, a lot of fields in AI where deep learning may be good to actually predict a model based on tree size, information that you can fit easily in tables. And also like pictures, if you look at pictures, it will taking pictures and actually turning that into a model that's basically looking at the pixels inside. It's same idea.

• 33:04Speaker 05

  So yeah, the other machine learning models still exist and it actually a lot faster in performance and in behavior, especially supervised behavior when you, when you control that, deep learning doesn't have all of that built into it. But again, it accelerate into pattern recognition a lot, a lot faster.

• 33:23Speaker 02

  Yeah, not everything is a nail. You need a hammer for a nail, but if you have a screw, you need a screwdriver. All methods have value and a good fit in different areas. Deep learning is quite, turns out to be quite versatile, but it has its own weaknesses too.

• 33:41Daniel Whitenack?

  Yeah, on that note, for people kind of getting into the field, they, they probably hear a lot about deep learning, but there are so many methods to, to learn about. What would you guys recommend as far as a person getting into the field? What sorts of things might they want to learn about before deep learning? And, and how quickly should they kind of make that leap to, to learning about, about these deep neural networks?

• 34:06Speaker 05

  I mean, you start with machine learning basics before you go to deep learning. What is machine learning? What are you trying to achieve? And you understand basically how you're taking parameters, what training mean. You actually get familiar with the frameworks. There's a lot of frameworks you will get introduced to. Each of them got a powerful side that you can use in certain systems. Yet, before all of this, in my opinion, distributed systems. You need to be able to, you know, run around, know your way around as a distributed system engineer. Because AI need a lot of that. You're gonna have models sitting in the cloud. You're gonna have inverse happening on the edge. You're gonna have training maybe happening on the edge in some places, so.

• 34:49Speaker 03

  So, to get into this, I would recommend somebody just first, just this is how I got into it. Just basically first figure out what is a cat and what is a dog. I'm sure all of us have went through that. And once you go through that, you know, because it's the goal that's pushing you to keep learning, right? First, first like, okay, how do I determine elephant from dog and cat? And from there, I was like, okay, I need photos of elephant. So I have to basically Google a bunch of photos of elephant. It was like, okay, from there, I basically said, okay, what framework to use? And I actually select a framework, I select a couple of them. I'm not gonna say which one. I basically choose the easiest route that I was able to train my model. And this is, but this is, it's a limitation of a person's imagination is what will keep their curiosity going. Will keep them Googling for like, you know, Stack Overflow for informations.

• 35:33Speaker 03

  And when they get stuck and then, you know, each step you learn a little more. And eventually, pretty much how your own way would work. That's a good way for learning, getting into deep learning.

• 35:44Speaker 02

  That's a very good advice. I agree with them that, you know, it's a complex topic. There's math involved, there's domain knowledge involved. And there is ability to define a good problem is very critical too. And then you have the tools like TensorFlow one day, PyTorch the other day. There is so much. I would agree with Peter in saying that just get in there, start with a simple problem. You don't have to worry about defining a problem and you're trying to make the machine identify cat versus dog. So you kind of minimize the buttons you have to push and start small. And as you gain confidence and curiosity kicks in and your goals change.

• 36:28Speaker 02

  And you learn new methods, new complicated approaches. That's the best way to do it.

• 36:33Speaker 04

  The other quick thing I want to add is it also depends on how deep you want to go. So if you're in the academia, then you probably want to do statistics and probability. Or if you're not, if you're in the industry, you probably want to build prototypes as fast as possible. Then you want to use frameworks like TensorFlow, Keras especially is one of my favorites. It's very easy to go, four lines of code, you have your model trained and then it's doing inference. So it really depends on the user, the developer, where you see yourself being in that machine learning stack.

• 37:01Daniel Whitenack?

  Awesome, yeah. One of the things that we've emphasized before is kind of in line with what you guys are saying. Find a problem that you're passionate about, find some data that you're interested in and just try to start answering those questions. And that will lead you to the right frameworks and the right methods and the right things to learn about.

• 37:18Speaker 06

  It's a great point, like just start where you want to start, find interest in it. Because if you're trying to do static tasks, A, B, and C to get to the end result, I think you're following a set of steps. I think you have to have a passion about what it is, what do you want to solve for? What do you want to do? And then the excitement and the motivation behind that pushes you further. But I think, yeah, as soon as, if you're just trying to give someone a guideline, do this, do that, do this, do that. That's boring. That sends me back to school when you just had to get your homework done. That's not fun, right? But I think everybody at this table is super excited about what we do and we follow our own path. That's why we're all doing different things, right? So.

• 37:50Daniel Whitenack?

  Awesome. So I don't know if all of you guys primarily work in Python. I think we could extend this next question to whatever language you guys work in. But apparently the internet wants to know, what is the coolest thing that you've done with Python? Slash whatever language you're interested in. So maybe if you guys just want to highlight one of the cool things that you've done to give a little bit of inspiration to the audience.

• 38:13Speaker 04

  So my kind of projects, just going back again to what excites me is FPGA. So field programmable get a race. So FPGs are pretty low level and they don't really operate at a level of Python. But as you know, the frameworks operate at a level of Python. So what I typically would do is create wrappers around the FPGA low level APIs, right? So I would work with C C plus plus and create wrappers for Python. So you can easily plug in the device to such frameworks.

• 38:39Daniel Whitenack?

  Awesome. Are any of those kind of wrappers and that tooling, is that available to the public?

• 38:43Speaker 04

  Typically, no, because those are very low level driver API stuff. Right, exactly. I don't even think the users would be interested in knowing what goes on at that level, right? So Python is the way to go, I would say.

• 38:56Daniel Whitenack?

  Awesome, yeah. And is any of that stuff that you've done public to where people could take a look?

• 39:01Speaker 04

  Clearly not public.

• 39:02Daniel Whitenack?

  Yeah, awesome. Where I know that Intel, for example, is putting out a lot around FPGAs. Are there resources out there or webinars or anything?

• 39:11Speaker 04

  So yes, Intel has done quite a lot in terms of enabling FPGAs for data scientists and machine learning. There's actually a tool called TBSDK. It was recently released not too long ago. So TBSDK would allow you to take your train model and run it through what is called a model optimizer. It's actually a Python script. And then from that you would get an intermediate representative, which you can then use to port against pretty much any hardware that Intel makes today. You know, movies, FPGAs, CPUs. So OpenVINO, it used to be OpenVINO, I say OpenCVSDK? It's the same thing.

• 39:46Speaker 05

  It's integrated together.

• 39:47Speaker 04

  Right, right. So TBSDK is what it's mostly called today. That's the tool that you can use if you're using Intel's hardware. It's pretty easy to use. It's Python based.

• 39:57Daniel Whitenack?

  Awesome, yeah, I appreciate that. I definitely want to dive in more myself on that.

• 40:00Speaker 05

  Yeah, and the most exciting thing, I mean, FPGAs is a very exciting thing for me right now. But I believe by taking control of SDR and FPGA together, that's what's going to change the way we push AI to the edge. Because imagine, I mean, you don't have the limit of a certain technology that already sit on this device. Just because the technology changed in a few weeks or we found a silicon level issue with the design itself. Since FPGA, it's easy to rearrange. It's just a framework that you can actually push. Make it easier and make it a lot easier to control. SDR also, pushing SDR, which is software defined radio, basically.

• 40:42Speaker 05

  That's going to make 5G technology, LTE technology and IoT technologies without being stuck in a limit.

• 40:49Speaker 05

  And imagine if you actually let the machine decide the control of the waves and the control of which which band to actually use to make sure like a swarm of drones, for example, stay always connected. So AI gonna get pushed to that. And I I believe FPGA, I mean, as an industry, uh, we we've been in the network industry for a while. It started with FPGA, especially on the on the network side, because it's easier than building a complete ASIC. Right. Go to the market. But with FPGA pricing going extremely down in the last five, six years, until introduction of a new type of FPGAs that actually built for intermediate

• 41:30Speaker 05

  devices. It's not just basic input and output features. You have more cores from the technology that's available to you. I believe for me, seeing Python being able to actually utilize to control SDR and FPGA, and we did some research on that, on the edge of the network. It's extremely amazing. It can push AI capabilities a lot more to the edge and and use more sensors capabilities, utilizing that FPGA and sensor fusion. You don't need a lot of sensors. You can use less channels by fusing your data together and having like, we we refer to it as a the edge of the network, which is, we refer to it as MAC today, multi-access edge compute. But we also believe there's another layer. So it's not everything gonna be in

• 42:14Speaker 05

  the cloud. It's in the edge, but also furthermore, on the device itself. There will be some decision making happening there to eliminate the amount of bandwidth we use. Otherwise, these all connected devices will send so much data, no matter what technology we have, it's not gonna be enough to process it.

• 42:30Daniel Whitenack?

  Awesome. Well, any other, any other thoughts around interesting things you've done with Python or or other languages that you want to highlight?

• 42:37Speaker 02

  Uh, we we work at the infrastructure level, just bare metal up silicon up level. So we kind of uh try to be hardware agnostic, be it CPUs or FPGAs, whatever it is. So we work C++ level, uh, not so much on the Python.

• 42:54Daniel Whitenack?

  Yeah, well, what have you been doing recently in the at the C++ layer? And also, maybe I'd love to hear you speak towards the the role of C and C++ in ML and AI. A lot of people see Python as kind of the the only player, but I think that's that's kind of a facade in some ways. So maybe you could speak to that.

• 43:14Speaker 02

  At the infrastructure level, you need compiler language like C++. And then at the user level, we put a wrapper and allow people to use our product with Python interface. The Jupyter Notebook, they can call our libraries and make use of it. So all languages have value there, but I think a lot of the models will get built in Python only. That's that's the fact. But when it comes to distributing to consume compute, you need something else underneath.

• 43:47Daniel Whitenack?

  Gotcha. So kind of on that on that note, you know, maybe the last thing that we can end with is there's a lot of people kind of getting into, like you said, training with with Python. And maybe they're they're hitting some blockers as they're they're trying to scale things or they're trying to build up an AI team and maybe not not knowing how to productionize models and that sort of thing. What general recommendations would you guys have around kind of the team that you build up around that you're trying to put together to build AI models? And kind of some of the the tooling that that you might need to consider, the methodologies that you might need to consider as you're actually trying to scale AI?

• 44:27Speaker 04

  I would add by saying it depends on if you want to be a machine learning developer or an infrastructure engineer, right? Or you want to be both, which is great because I'm sort of somewhere in between. A little bit of this, a little bit of that. Two that I would recommend are Dockers, maybe Kubernetes, to then manage Docker containers that are running these ML models across multiple nodes. But again, like I said, it really depends on what you what you really want to do.

• 44:51Daniel Whitenack?

  Yeah, and on that note, do you think like as far as teams that people are trying to build up? Maybe you're trying to start an AI effort in your in your company. Do you need to kind of have have both of these components? Or do you think there's people out there that can fulfill both of the roles themselves? Or?

• 45:06Speaker 04

  Right, so it turns out there are actually really nice dev tools you could use. I would say maybe just a team of one person should be able to manage the infrastructure that runs this ML models. And really, you can deploy Docker containers and Kubernetes, um, Mesos, for example, to kind of manage these models. It's a different thing if you're running it at scale. Then you may need a large engineering team to kind of make sure that business is running up and going, yeah.

• 45:32Speaker 02

  The most time-consuming part about building an AI, great AI product, uh, isn't necessarily the deployment piece. It's thinking right about the domain, about the problem you're trying to solve. Especially if the team is being built in a bigger company. The problem definition itself requires a lot of multiple levels of agreements and consensus. Uh, that is a challenging thing. So you need people who can build consensus. You need people who can do critical thinking. And then you need great data scientists who can translate those, uh, problem statements into some kind of a model. Okay. Uh, it's a classification problem or a clustering problem, whatever. Somebody who is familiar with that kind of, uh, semantics,

• 46:17Speaker 02

  be able to translate that and hack together and experiment, put out a prototype. You should be able to get to a prototype really fast. Uh, that should be the goal. And that requires a multidisciplinary team.

• 46:29Speaker 06

  I think that's a good point. I, I'd like to say you need, you need translators. You need people because the folks that are doing the dirty work and getting their hands dirty and the programmers and developers, those aren't the people with the cash a lot of times. And those aren't the people making the business decisions. And so you kind of need someone in between that's, that's savvy enough to get the idea, be able to communicate with those that are really putting the rubber to the road, but also communicate what the why is, right, Vina? What the end result is going to be, and even bigger, what's, what's the ROI? What's the return on, what, what, how many sales am I gonna get? I mean, you, you almost have to dumb it down. And it's almost insulting. That's why, you know, you hear the, the joking tagline, don't let the engineer make the product, right? Because you'll keep fixing it. You'll keep getting better and better. And that doesn't scale for enterprises or for a product to go to market, right?

• 47:14Speaker 06

  And so while we constantly want to do better things. And we can never deliver something because it's never done. It will always be better. I think you gotta have that buffer, someone in the middle that can translate from the tech to the business side. Because if you don't have it, there's a huge gap between the business guy and the people like folks at the same like you guys are actually doing it, right?

• 47:31Daniel Whitenack?

  Awesome. Well, just to kind of end things here, I want to thank you guys for for taking time out of the conference. I know there's a lot of great talks going on, so thank you guys for taking time to talk. It was really some great perspective and and thank you to Intel AI for for helping arrange a range of this panel. Really appreciate it and thank you guys so much.

• 47:50Speaker 04

  Thanks for having us.

• 47:51Daniel Whitenack?

  Thank you, Daniel. Thank you for having us.

• 47:55Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes and give us a rating. Go in your podcast app and favorite it. If you are on Twitter or a social network, share a link with a friend, whatever you gotta do. Share the show with a friend if you enjoyed it. And bandwidth for change log is provided by Fastly. Learn more at fastly.com and we catch our ears before our users do here at change law because of roll bar. Check them out at rollbar.com slash change log. And we're hosted on Lino cloud servers. Head to linode.com slash change log, check them out, support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com.

• 48:37Speaker 00

  When you go there, pop in your email address, get our weekly email, keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.