Celebrating episode 50 and the neural net!

Woo hoo! As we celebrate reaching episode 50, we come full circle to discuss the basics of neural networks. If you are just jumping into AI, then this is a great primer discussion with which to take that leap. Our commitment to making artificial intelligence practical, productive, and accessible to everyone has never been stronger, so we invite you to join us for the next 50 episodes! Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Learn more about neural networks with the following learning resources. Books “Deep Learning” by Ian Goodfellow, Yoshua Bengio, and Aaron Courville http://www.deeplearningbook.org“Data Science from Scratch” by Joel Grus https://learning.oreilly.com/library/view/data-science-from/9781492041122/There are literally too many others to name…Online Courses fast.ai MOOC https://course.fast.ai/Deep Learning course by Google on Udacity (free) https://eu.udacity.com/course/deep-learning--ud730Practical Machine Learning with TensorFlow 2.0 Alpha (free Udacity course) https://blog.udacity.com/2019/03/learn-tensorflow-for-deep-learning-applications.htmldeeplearning.ai Deep Learning Specialization https://www.deeplearning.ai/deep-learning-specializationIntroduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning (free Coursera course for TensorFlow 2) https://www.coursera.org/learn/introduction-tensorflowIntro to TensorFlow for Deep Learning by TensorFlow (free Udacity course for TensorFlow 2) https://www.udacity.com/course/intro-to-tensorflow-for-deep-learning--ud187Websites Microsoft AI School https://aischool.microsoft.comTensorFlow 2 Docs https://www.tensorflow.org/betaAmazon Machine Learning Training https://aws.amazon.com/training/learning-paths/machine-learningPyTorch https://pytorch.orgFacebook AI https://ai.facebook.comGoogle AI https://ai.googleUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at ChangeLaw because of Rollbar. Check them out at rollbar.com. And we're hosted on Linode Cloud servers. Head to linode.com/changelog.

• 0:16Speaker 00

  This episode is brought to you by Digital Ocean, the simplest cloud platform out there. And we're excited to share they now offer dedicated virtual droplets. And unlike standard droplets, which use shared virtual CPU threads, their two performance plans, general purpose and CPU optimized. They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, CICD, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers, or active front-end web servers that need to be full-duty CPU all day, every day, then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive, starting at 40 bucks a month.

• 1:01Speaker 00

  Learn more and get started for free with a $100 credit at do.co/changelog. Again, do.co/changelog.

• 1:21Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slay with us around various topics of the show at changelog.com/community. Follow us on Twitter, we're at Practical AIFM. And now into the show.

• 1:46Daniel Whitenack?

  Welcome to another fully connected episode of Practical AI, where my co-host Chris and I keep you fully connected with everything that's happening in the AI community. We're going to take some time to discuss the latest AI news and we'll dig into some learning resources to help you level up your machine learning game. So I'm joined today by my co-host, Chris Benson, who's the Chief Strategist for AI High Performance Computing and AI Ethics at Lockheed Martin. And I'm Daniel Whitenack. I'm a data scientist with SIL International. How's it going, Chris?

• 2:22Chris Benson?

  It's going great. I'm excited today.

• 2:24Daniel Whitenack?

  Yeah, me too. How's the week been for you?

• 2:28Chris Benson?

  The week's been good. Just lots of, lots of stuff at work. Was traveling about a week ago, was up in Boston at LiveWorks. And been doing lots of interesting stuff in high-performance computing and AI ethics and artificial intelligence as a field just keeps getting more and more interesting in terms of what we're doing and just at large and, and all the places you can go in it. So, great time to be in it.

• 2:52Daniel Whitenack?

  Yeah. Well, hopefully, hopefully I'll survive. I'm working on a little bit of a jet lag right now. As you know, last week I was in India, which was, was great. Was in Bangalore. But India was great, but getting back from India was quite a chore. So it turns out I was on an Air India flight, which they don't fly through Pakistani airspace because of obvious reasons.

• 3:20Chris Benson?

  Yeah.

• 3:21Daniel Whitenack?

  But then while I was in India, this tension happened between the US and Iran and the US put restrictions on planes coming to and fro over Iranian airspace, which is the reroute that Air India does, you know, as it kind of goes around Pakistan and then over the Arctic back to Chicago. So finding a route back to where I needed to get turned out to be rather interesting. So I got back a lot later than expected and yeah, it's I think 2 a.m., about 2 a.m. now in Bangalore. So hopefully bear with me if I start going off on a tangent.

• 4:04Chris Benson?

  Not a problem. I will wake you up though, because this is a special episode for us. It's our 50th episode.

• 4:09Daniel Whitenack?

  It is. Congratulations on 50th episode. Pretty crazy.

• 4:15Chris Benson?

  It does. So it's gone by so fast over the past year and, you know, just the idea that we've put out that much content and that we actually have people that still want to listen to us after doing that. I'm amazed by that every day.

• 4:29Daniel Whitenack?

  Yeah, definitely. Thank you to all the listeners. It's been great to kind of gradually get more and more connected to the listeners and listeners engaging on our Slack channel, on LinkedIn and other places. It's just really great to hear that you're appreciating some of the content, but also great to hear some of your ideas that we've been able to filter into the show. So keep those coming. Thank you so much for listening. We really appreciate all of you and really want to engage with all of you in our community. So make sure and check that out at changelog.com/community.

• 5:05Chris Benson?

  I just gotta say, when we get messages from people out there or people engaging us in the communities and stuff, it is just enormously exciting because it's kind of the reason that we're doing it and the fact that people are out there, not only are they listening, but they're saying, "Hey, what about this? I'd love to hear that." And, "Hey, here's a suggestion," or, "Hey, I know somebody who would be great on your show." It's just, it makes the whole thing wonderful. I know that sounds a little corny, but it's true.

• 5:32Daniel Whitenack?

  Yeah, it is super encouraging. So keep that coming. We're really excited about episode 50. This is kind of a celebration for us. And so we were talking before the show of like, how, what should we do to celebrate episode number 50? And what we came up with was kind of to loop all the way back to kind of where things started, you know, with AI and with Practical AI. And that's to devote this kind of celebratory episode to one of our favorite things, which is the neural net.

• 6:10Chris Benson?

  Absolutely.

• 6:11Daniel Whitenack?

  Yeah, so we thought we would, so we've talked about a A lot of neural nets on the show, obviously, and many advanced architectures and applications and all of that. But we've never actually just talked about the neural net itself, where it came from, and just kind of in brief, and from scratch, what a neural net is, what makes it a neural net? And we thought this would be a great episode to kind of circle back to that starting point.

• 6:40Chris Benson?

  Yep, absolutely. One of the common comments that we get back, that I've had conversations with several people about, including the young man that is at the Chinese restaurant two miles from my house, because he actually listens to the podcast, but he's not a data scientist. And he made some comments to me a while back. He said, you know, you guys are really good, so long as you don't do jargon. And we took that as a point to be very careful about that. But he said that sometimes we get a little bit out there for where he's at, and he's very interested in the topic. But we've never really done a true intro to neural net type of show. And it occurred to me that for those people out there who are trying to jump in and may find it a little bit intimidating,

• 7:24Chris Benson?

  I can't think of a better way to celebrate kind of a milestone episode.

• 7:28Daniel Whitenack?

  Yep, sounds great. So why don't we start with giving just a little bit of history about the neural net itself? So neural nets are not new. They've actually been around for quite some time. Do you know when sort of neural nets came onto the scene, Chris?

• 7:45Chris Benson?

  Sometime around World War II, if I recall correctly. Do you have the specifics?

• 7:51Daniel Whitenack?

  Yeah, so if you just, you can just search Google for like neural net history. And you know, there's several lists that come up that have varying, you know, variations of the various dates and facts and all of that. But generally, people include kind of a date around the mid 1940s, when the first kind of computational model for neural networks came out. It's a guy named, I'm sorry if I'm mispronouncing this, I don't really hear this name too much, but Warren McCulloch and Walter Pitts created a

• 8:29Daniel Whitenack?

  These computational models, really that paved the way for both sort of modeling biological processes, like actually neurons in our brain or neural networks in our brain, and then kind of more practical applications of neural networks.

• 8:47Chris Benson?

  Yeah, and then I think there was a, kind of the next major step was when the perceptron was invented, and that was by a guy named Frank Rosenblatt in 1958. So, we're getting, for me, we're getting a little bit closer to my year of birth, not quite there yet, not quite that old. But that really set off kind of, you know, one of the early waves of research in this area.

• 9:11Daniel Whitenack?

  Yeah, people are sometimes surprised because there's been a lot of talk about neural networks recently, but maybe they didn't hear it a while back. So, these sorts of things have been around for quite some time in research. And like you were saying, moving up through like the 60s and 70s, they were a topic of research, but I think that a big shift happened in the 1980s, kind of up to the mid 90s. And this is where things like deep learning and back propagation, so these kind of larger neural networks, and applications to different types of data, came around. So, up until this point, people were researching

• 9:55Daniel Whitenack?

  neural networks, but they hadn't really figured out a way to kind of make them bigger and learn more complicated patterns. So, before that, they were pretty limited towards like linear divisions between class, you know, linear class boundaries and these different things. But as they saw that they needed to model more complicated relationships, they saw that kind of the size of the networks needed to increase, but they didn't really have a good way of training those sorts of neural networks. And that kind of changed in the 80s.

• 10:27Chris Benson?

  That's true. And that's, I have a special affinity for that time period, because in 1992 is actually when I first became aware of neural networks, and that's before like the The name deep learning was applied to it and it was before anyone was calling them deep neural networks necessarily. My father worked for Lockheed Martin, just like I do. He would have been shocked that I do at this point, but he worked there and there was an event that really affected me in a very personal way. And that was that there was a fighter plane called the F-22. At the time, it was the YF-22, which is still kind of the world's top air superiority fighter. In other words, for dogfighting, you might

• 11:08Chris Benson?

  say. And there were two prototypes, and one of those prototypes was doing some testing at Edwards Air Force Base, and they were testing avionics on it, and there was a malfunction. And fortunately, it was close to the ground, and the plane slammed into the ground and went skidding in a fiery ball down the runway for quite a ways. And the pilot got out, the test pilot got out and got away safely. But as the aftermath of that, both of my parents were on the F-22 team, the core team that built the avionics for the plane. And my father was assigned to help solve that avionics thing. And as part of that, he was using neural networks of the day. And he started with feed forward and back propagation, which

• 11:53Chris Benson?

  we'll talk about in a few minutes, and moved on to other architectures. But that was really special because he would come home, and there's all sorts of classified stuff he would not talk about. But in terms of the actual science, we'd come home and he introduced me to neural networks. And this was our dinner table talk for a while. And as he made progress into different areas, and he would explain it to me at night, and I would ask questions. But, you know, before, so that's going back to when I was a college student at the time. And so it was a really interesting way for me to kind of get into it, and a very practical problem. And obviously, the problem was solved, and the F-22 is in service today. And so, anyway, but that's how I originally came to be aware of them.

• 12:35Daniel Whitenack?

  Yeah. And that's where you first started getting the ideas for this great podcast, Practical AI, I'm sure. I don't know if podcasting was not a thing.

• 12:47Daniel Whitenack?

  But yeah, this time was really, really where some interesting things came on the scene. That was recognized actually this year with this year's Turing Award, which went to La Coon, Hinton and Bengio for things like back propagation and the sort of ideas around deep learning. And that was big news recently. But there was this kind of time period of the 1980s and kind of up to the mid 1990s where things were getting really exciting. And then there was a sort of die off of interest in these sorts of methods. Some people call this the AI winter.

• 13:25Chris Benson?

  Yeah.

• 13:26Daniel Whitenack?

  And that, yeah, that kind of, that kind of led up almost to the, to the mid 2000s. So this was a time when kind of these, these methodologies were, were known. But the problem was that the kind of, as these networks got larger and larger, of course, they had more parameters that needed to be fitted. And that required more data and more compute. And so there was kind of this lag of the actual data and compute that was needed and along with that, the adoption that we've seen recently. So that really kicked into gear maybe in the mid 2000s and on where people really had access to a lot of compute, a lot of data, and really were able to plug that into these advanced methods.

• 14:11Chris Benson?

  Yeah, it's, it really got kicked off by a guy who had been in the field for a while, kind of coming out of this AI winter by Geoffrey Hinton. And he is, he's kind of one of the legends in this field. And he started research and at first people, as he kind of, and I think he, he continued through the, through that AI winter, but while, while everybody else was turning to other things, and I would, I would argue that it was really some of his initial and kind of in this latest wave since the mid 2000s that kind of kicked it off. And, and he is really, I credit him with coming out of the AI winter and kind of being at the moment that we're at now.

• 14:53Daniel Whitenack?

  Yeah, and you know, recently, of course, Google has switched kind of from a mobile first to AI first approach to to their business in general. And that's kind of sparked a lot of interest from a lot of other industry leaders as well. So pretty much all the big tech companies now, along with, you know, a host of startups and smaller companies have really switched to a focus on AI in terms of research and development and the methodologies that are powering their their products. So AI has kind of, at this point, become a new layer in the software stack that's enabling new sorts of functionalities

• 15:37Daniel Whitenack?

  in in applications. And, you know, at the core of most all of those AI systems are neural networks, these things that started back in the 40s that were kind of envisioned and built up over time. But the the core idea is there, it is the the neural network now. Not a lot of people will kind of argue about what AI encompasses and the sorts of methods that are AI and aren't AI. And there's certainly a lot of methods that aren't just kind of simple neural networks. There's non neural network methodologies. There's a lot of other machine learning type of methodologies.

• 16:16Daniel Whitenack?

  But really, the neural network is kind of the core piece that's that's powering a bunch of things in industry now. And really is the focus of a lot of the the AI research that's that's going on, which is which is why we're focusing on on them.

• 16:31Chris Benson?

  That's true, and that was, I have to say, that was very well said. Because the reality is when you put when you put different people in this field, data scientists and deep learning engineers, and you ask them what AI is, you're going to get a lot of different answers. And I was actually at an event where that it almost comically, you know, demonstrated itself in that way. It was an Adobe event, which which was a live broadcast on Facebook, and I was one of 10 people that Came and participated and there was a lot of stuff we agreed on, but the one thing that all of us had different viewpoints on was exactly what constituted

• 17:12Chris Benson?

  artificial intelligence today. And without, without delving any further, I just found that fascinating that they introduced us as experts, whether, whether we were or not, but that we were, we were positioned in that way. And yet, none of us could agree on the, the basic definition of the field.

• 17:38Speaker 03

  The Data Engineering Podcast is a weekly deep dive on modern data management with the engineers and entrepreneurs who are shaping the industry. Go behind the scenes on the tools, techniques, and difficulties of data engineering, so you can learn and keep up with the knowledge to make you and your business successful.

• 17:54Speaker 04

  Can you give a bit of an outline about the motivation for choosing Jupiter notebooks in particular as the core interface for your data teams?

• 18:02Speaker 05

  Yeah. And actually, when I first joined Netflix, it was a sort of toss at me and I was definitely like, are we crazy? And the answer was like, we might be a little crazy.

• 18:11Speaker 03

  Go to dataengineeringpodcast.com to listen, subscribe, and share it with your friends and colleagues.

• 18:29Daniel Whitenack?

  Okay, so we've talked about kind of a little bit of the history of neural networks and we've talked about, you know, how they came onto the scene and really that they're powering a lot of these big tech innovations. But before we jump into kind of the very, very specifics of a neural network and, and what it is, I think it would be useful to kind of just give a real broad definition of supervised, the supervised learning. And there's, you know, a lot of different types of machine learning models out there, some of which are kind of unsupervised and semi-supervised, but the bulk of models that people kind of get into when they're first getting into AI and machine

• 19:13Daniel Whitenack?

  Machine learning or supervised learning models. And I think that would be a good framework for us to talk about neural networks within.

• 19:22Chris Benson?

  Yeah, I was just gonna say, when I'm talking about neural networks in an introductory thing, I pretty much, I may allude to some other things that are out there, but supervised learning is definitely the place to start. It's kind of the basics and you learn the basics and then you can build on it in a lot of different directions.

• 19:38Daniel Whitenack?

  Yep, so let's kind of, when I'm teaching classes, I normally try to introduce some type of kind of model problem that people can have in the back of their minds. When I'm thinking about supervised learning, you might think about like, let's try to model the number of, or let's say, let's try to model our sales for the month based on the number of users on our website or something like that. Now, one way you could do that is by creating a sort of function, like a function in code that would take in your number of users and output your sales. And most often, that would include some type of like model definition and some parameters.

• 20:22Daniel Whitenack?

  So it's like, you might input a number of users and then multiply that by a parameter or a coefficient and out comes your sales. So that's a model definition with a parameter. Now, the big thing that separates kind of machine learning functions versus kind of regular code functions is that regular code functions, that definition and parameters are kind of set by domain knowledge and someone coding them in. Whereas in a machine learning context, I like to think about those parameters being set by kind of trial and error or an iterative process of looking at a bunch of examples and trying to make predictions for all of those examples

• 21:04Daniel Whitenack?

  and then fitting or setting those parameters based on this sort of iterative process. Overall, that's kind of how I have the picture in my mind. Does that kind of make sense, Chris? Or do you have a sort of different view?

• 21:16Chris Benson?

  No, I think I have, I think I would see it the same way. I think one easy way to think about it is if you Look at solving problems programmatically up until you get to deep learning. In other words, just using kind of more traditional programming, you explicitly are going and giving the program commands on what you're going to do. And you might think of it in a very simplistic way as lots of if-then type statements, lots of case statements, and you're having to think of all the things. Whereas this way of doing it, where machine learning, the model is learning what it needs to do, is sort of implicit. It's figuring it out for you.

• 21:58Chris Benson?

  And it's kind of a different programming paradigm in large in computer science beyond just what deep learning is. So if you think of it as the job of training the model is now to go figure out what it needs as opposed to being told what it needs, it kind of puts you in the right frame for learning this.

• 22:18Daniel Whitenack?

  I would say, you know, some people, when we use the words like, you know, "the computer figures it out" or "the computer learns", they kind of have this view of like, "Oh, I'm gonna go put my laptop like in the corner of my office and then kind of sprinkle some special fairy dust on it, and it's gonna kind of spontaneously start learning things about the world." Really, at the...

• 22:39Chris Benson?

  Fairy dust?

• 22:40Daniel Whitenack?

  Yeah. At the... You know, you don't have some of that laying around in your kitchen or something?

• 22:46Chris Benson?

  Yeah, I'll borrow it from my seven-year-old daughter.

• 22:48Daniel Whitenack?

  Yeah, yeah, maybe. So in reality, what there is, is there's always a sort of model definition. Remember thinking of our kind of users to sales? There's some definition and there's some number of parameters that parameterize that model definition. It might be the coefficients multiplication or what we call bias, which is kind of a number that we add on to the definition. But there's some model definition in those parameters. And what we mean by "learning" or "training" isn't just that it kind of... Our computer has, it's at the right temperature and the right conditions and year of the month and the stars align and it starts learning. It's that these parameters are set through an iterative process of looking at a bunch

• 23:32Daniel Whitenack?

  of training examples, of examples of what input is and should be and what output should be. So there's a bunch of examples of, there's this input and this should be the output. There's this input and this should be the output. And there's a training process, which is just another function written in code that iteratively looks over all of these examples and fits these parameters such that the model can then make predictions on new examples that it hasn't seen yet. So it isn't that this, there's kind of the spontaneous learning that happens. It's really kind of something much more benign. It's that there's a bunch of examples and computers are good at repetitive tasks, right?

• 24:17Daniel Whitenack?

  And so we just have the computer look at these examples over and over again and tweak these parameters until we, we get a good set of parameters to parameterize this model definition and then we can make a new, new prediction. So that first process is called the training process. And then when we make new predictions, that's called the inference or prediction process.

• 24:39Chris Benson?

  So in the training process, we've talked about making the little tweaks. And that's called error correction. And so as Daniel was talking about, we, we already, when we're in training, we already know what the ground truth is for any given example. And, and the model is essentially trying to find that with, with where it's trained to up to that point. And then it actually, it says, okay, I have a result in this training cycle. And then I have the, the ground truth and there's a difference in the two. And I'm going to use an error correction algorithm to say, what should I do? What tweaking should I do in this case? When my result isn't what I know to be the ground truth in the training set.

• 25:24Chris Benson?

  So it is, it is an algorithm that is driving that tweaking, but it is able to, to use that algorithm based on the data that has come a point on that particular cycle.

• 25:35Daniel Whitenack?

  So let's maybe make this a little bit more concrete now. So we've talked about, you know, supervised learning in general, and that there's this definition and parameters that are set. So what does that look like for a neural network? So in a neural network, there's these kind of subunits. I have an overall definition, and then I have a bunch of kind of subdefinitions within that, or you could think about it if you're a programmer, like a function that calls a bunch of kind of subfunctions underneath it. And these subunits or subdefinitions are called neurons. And so each of these neurons kind of have its own inputs and outputs with its own definition and its own set of parameters.

• 26:18Daniel Whitenack?

  And these parameters for the neuron are often called weights and biases. So again, you can kind of think of my overall definition of my model containing a bunch of these subdefinitions of neurons that are linked together in some way. And that together, that assembly of neurons make up what's called a neural network architecture. So that architecture just means there's a bunch of these kind of subunits. Each of them have a definition and some parameters that can be set. Now, there's a lot of different ways that you can set up those neurons. So maybe we should look at, you know, a kind of common way to set up one of these neurons.

• 27:03Chris Benson?

  Yeah, like a fully connected feed forward is a good starting point.

• 27:06Daniel Whitenack?

  Yep. So maybe, do you want to start there, Chris?

• 27:09Chris Benson?

  Sure. So Daniel was just talking about these units of neurons. And if you want to paint a picture in your mind as you listen, you could think of each one of those, the way they're usually depicted graphically, is as a little circle. And you could think of it as a little circle that has some stuff inside it, which we'll talk about in a moment. But you take each of those circles. And you line a few circles up into a row. And so you have a row of circles. And then at that point, you line up a second row and maybe a third row. And so there's some number of rows, in number of rows that you have there. And there's some special relationships between each of those layers. So if you take for every neuron in that first layer,

• 27:52Chris Benson?

  it is connected to each of the neurons in the second layer, but to none of the neurons in its own layer. And so in that second layer, you recreate that. And so each neuron in a given layer is connected to all the neurons in the previous layer and all the neurons in the next layer, but none of the neurons in its own layer. And so you can kind of envision this mesh of rows of little circles in that way. And you start from one side to go in as an input, and then you come out the other side. And that is the basic image in your mind of how you might think about a fully connected feed-forward network. And I'll note, one other thing is these shows where Daniel and I talk about topics on our own without a guest,

• 28:35Chris Benson?

  you may have noticed that they're called "Fully Connected Episodes." This is what we're referring to. It was named after this.

• 28:40Daniel Whitenack?

  How clever. You mentioned that each of these nodes or neurons is fully connected in this sort of network. And each one has its own inputs and outputs. Now, if we dig into one of these neurons, to think about kind of what's inside of that bubble. And again, you can think about that visually like a bubble or a node. Or if you're kind of a programmer, you might think about it as one of these kind of sub-functions under a big function. But it has its own inputs and outputs. And if we think about it maybe as just having a couple inputs, let's say x1 and 2, now what happens inside of that circle or inside of that neuron?

• 29:24Daniel Whitenack?

  There's some kind of simple things that happen often. So one way we could think about processing these inputs in the neuron is to just add them up. Right? And so in a kind of linear regression sort of way, we could multiply each of my inputs, X1 and X2 by a couple coefficients. Let's say W1 and W2. Those are often called weights. So I just add up the two things after I multiply them by coefficients. And then I might add in like a intercept or a constant. And you know, so just X1, X2 plus something. And that's often called a bias. So in this case, there I would have like three parameters that I that I parameterize

• 30:08Daniel Whitenack?

  the way I'm combining these inputs. And so each of my X1 and X2 come in, I combine them together in this way. And that's all good and fine, except you know, most relationships in our world aren't linear. So it might be good to introduce some non-linearity into this combination. And that's where a thing called an activation function comes in, which is just a nonlinear function that's kind of applied to this combination of inputs to give it some non-linearity. And common functions that are used are like sigmoid or ReLU or other functions, hyperbolic tangent that are applied to this combination of inputs. So when my inputs come into that node or that circle,

• 30:53Daniel Whitenack?

  they're just kind of added up in a special way and then output out the other end. And all of my neurons in my network kind of do similar sorts of simple operations that are parameterized in a similar way. So each neuron has a certain number of inputs. They're combined together using some parameters and then output is a number. And that's kind of what each what each neuron does.

• 31:18Chris Benson?

  That was a very good explanation there. And so as you, as you, as these inputs start flowing through these layers, and they're doing this concurrently, so the inputs come in, it hits all the neurons in that first layer. Simultaneously, everything that Daniel was just talking about happens in each of those neurons in that first layer. And as they go through their transfer function that adds the non-linearity, and then they go out, and the output of each one of those neurons in that first layer goes to all of the neurons in the second layer and is combined. Remember, since they're fully connected, there's lots of inputs potentially coming in, and they're all summed up again in each neuron, just the way Daniel described.

• 31:59Chris Benson?

  And so this happens concurrently, in concurrency, at each layer, and it goes through layer by layer by layer till you get to an output. And then you either, you discover at that point that if you're, while you're going through this training process, that you have some values coming out, and you compare that against what you know to be the ground truth that's in your training data set. You know what the result is while you're trying to train. And that's when your error correction comes in, where you have to say, okay, well, I've ended up with an output, and it's not quite what I was hoping it would be, so I need to change, I need to change the values throughout the architecture. There's the, the initial thing that most people learn and is most widely used is called

• 32:44Chris Benson?

  backpropagation. And that's where you work your way back through the layers through a set of algorithms that make little tweaks all the way through your layers. And then, hey, guess what? You've done one full cycle, and it's time to go to the next row of your data to train that. And you do that whole process over and over again.

• 33:02Daniel Whitenack?

  Yep. It's, it's kind of like, you might think about if you're trying to set these weights and biases manually, as, as a human, what we would do is just kind of try to make, you know, make an initial choice for them, try to make some predictions, and then see if our predictions were good or bad, and kind of adjust the parameters accordingly, and then just do that over and over. So that's what the computer is doing, is essentially a bunch of trial and error. It's making some predictions, and of course, there's More sophisticated ways of updating the the weights and biases rather than just kind of randomly making choices for updates. And that's where kind of this gradient descent comes in. But essentially, we're just making those corrections. Now, I think an kind of

• 33:46Daniel Whitenack?

  interesting thing to to add in here is we're always talking about models and like what, you know, we have a neural network model, we have this type of model. So here, we've talked about the kind of definition of the neural network. We've talked about all of the parameters that need to be fit for this neural network. We've talked about the training process that trains or fits all of these parameters. And then we've talked about the inference or prediction phase in which we use all of that to make predictions. So I'm kind of curious, in your mind, Chris, what do you consider the model? Like, what is the model in your mind amongst all of that?

• 34:25Chris Benson?

  So the way I would think of a model as I think of it, when you start out with these layers of neurons lined up, and and we're talking about the simplest use case, obviously, and there are, you can, you can add a lot of different complexity to this over time to achieve different architectures. And there are many, many architectures out there. When someone talks about a model, though, I typically think of a trained architecture. If you think of a fully connected feed forward architecture as being something you're training, when it gets done, it has a purpose. It's trying to create, its its purpose is to make inferences about a particular set of inputs to give you an output. And that's what I

• 35:06Chris Benson?

  would call a trained model. It's it's the architecture at work that is deployable. One thing that we did mention briefly is that is is when you're training a model, how do you know when you've gotten there? And I just wanted to note that it's it's it's arbitrary based on your use case in that we've been talking about the fact that when you get to each, the end of each cycle in training, you have some sort of delta between what you have and what you know to be the truth. And so that is an error. That you have there. There's a, it's a, it's a, a degree of error and you have to decide for your use case, how much error can you tolerate?

• 35:48Chris Benson?

  If you can tolerate more error because it's not a very critical need and you know, and if it happened to be wrong, it might not, it might not be a terrible thing. Then you can probably achieve training quicker and deploy because it's not, if it's a life and death thing and it has to be extremely accurate. Then you need a very small error in your final product and therefore you may take quite a bit more training to achieve that. And I just wanted to note, that's how you know when your training is over is that you've achieved an acceptable level of error for your use case.

• 36:18Daniel Whitenack?

  Chris, you were just talking about kind of the acceptable level of, of error with a neural network. And I think something that, you know, needs to be understood here is that these nodes or these neurons can be assembled in all sorts of, from simple to very complicated ways. And you could have sort of layer after layer of these that might be fully connected, might not be fully connected. But as soon as you start adding these things up or putting them, assembling them in all sorts of complicated ways, which is really what's done in deep learning.

• 36:53Daniel Whitenack?

  Then you start accumulating a ton of parameters. So in some of these, you know, in some of these recent models, let's say like transformer models that have come out for language. There's millions, in fact, hundreds of millions of parameters that need to be set. And so when you're thinking about the compute and the data that's needed to actually train these models or fit all of those parameters. Now you can kind of understand why a lot of data and a lot of compute is needed because you can't have like 300 million parameters. And then like 2,000 training examples and call it good and say that's going to set all of your parameters.

• 37:32Daniel Whitenack?

  You have to have a significant amount of data for you to be able to kind of learn the complicated patterns and fit all of these parameters. So a ton of compute and a ton of, a ton of data is needed.

• 37:45Chris Benson?

  Absolutely. It's, I think in calling out the scale that you're talking about. About there is important because it is, it is a distinguishing factor between this particular tool in data science and other tools that we've all worked with previously. I think it's, you know, and people say, okay, I understand that. And there they shortly upon coming into the field, you learn that there is special hardware used for the computation and people have often asked me, why is that? I've heard GPU and stuff like that. And, and that is the fact that to do these computations, which are actually, they're, they're not, they're not complex, but it is a field of linear algebra. It's called matrix multiplication.

• 38:28Chris Benson?

  And as Daniel just pointed out, when he was talking about the scale of the parameters and, and you might have very large architectures with many, many neurons in the, that are all concurrently doing these mathematical operations, you, it, it lends, it lends to efficiency to have hardware that is able to do this type of computation much faster than, than the hardware that came before. And that's why you hear about GPUs and TPUs versus something that we may have all grown up with, which was the CPU, you know, driving our laptop and stuff. And that is that this hardware enables these math, enables the mathematical operations that have to happen at such scale.

• 39:09Chris Benson?

  And that, that, it, that, the fact that you have that relationship there is, really distinguishes this particular data science toolbox from others.

• 39:20Daniel Whitenack?

  And makes it expensive in some cases.

• 39:23Chris Benson?

  Oh, boy.

• 39:25Daniel Whitenack?

  Yeah, so we, we've talked about the neural or neurons. We've talked about architectures or combinations of these neurons. We've talked about what it takes to fit all of these parameters of the neurons. But we haven't actually got to maybe what's the most important point, which is why do neural networks work? So if you think about kind of what we've done, it's, it's somewhat arbitrary in some ways in the sense that we've just put a bunch of functions all together in a, in a row that. Combine things over and over. That's that's kind of simplifying things, but it's it's really what we're doing.

• 40:05Daniel Whitenack?

  There's inputs and those are fed through a bunch of things that combine them over and over and then output something that combines that output over and over. And, you know, why, why does that sort of thing work? And the way I kind of like to think about it, I'm curious about, you know, how you think about it. And I know there's more formalisms we can put around it, but the way I like to think about it is, you know, if I have a relationship, let's say between some input and output, and I'm thinking of like, again, the users and sales example. That might be a fairly, you know, simple relationship. It might just be a proportional one that I can, that I can model via one or two parameters. And I just put that in and, you know, there's a simple relationship there.

• 40:50Daniel Whitenack?

  But there's a lot more complicated relationships in our world, like, you know, if I'm trying to detect a face in an image, there's a lot of important things there from, from color to edges to certain features of the face. And it's really hard for me to write down a definition using my own domain knowledge that kind of is the definition of a model of a face. And so the way I think about neural networks is kind of just saying, well, okay, I'm not even going to try to write down this sort of domain knowledge definition. What I'm going to do is make my model definition as complicated as it needs to be, such that whatever the relationships are between my input and output, whatever those happen to be,

• 41:35Daniel Whitenack?

  I'm able to account for those complexities because my model is is parameterized in such a complex way. And so this takes some of the, some of the burden off of the programmer, the domain expert, and really puts it on the computer in terms of computation and data. Because I, all the assumption I'm making is that there is a relationship between my input and output. And if my definition is complicated enough, I'm going to be able to parameterize it to to model that.

• 42:06Chris Benson?

  That is actually a great explanation. I really like how you said that. And I think it differentiates from a number of other approaches one might take. And so, you know, when we are using neural networks to solve really complex problems, we'll try, there's a balancing act that we're trying to do. The bigger the architecture, the more computation you're introducing into it by default. But you need it, and you can actually have, there's actually mathematically the ability, you could have a feed-forward network with a single hidden layer. And since we haven't specifically mentioned the word hidden,

• 42:49Chris Benson?

  think about this neural network architecture that we talked about. And you had that input layer of neurons. And then the second layer only takes the output from the first layer, and then it passes its output to a third layer, which is your output. So you have a hidden, I'm sorry, an input, a hidden, and an output. And there is a mathematical equation called the universal approximation theorem. So you can go look it up on Wikipedia. And it notes that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions. And that sounds like, you know, not a very impressive statement to make, but I think it's pretty amazing in that it's saying you can approximate all sorts of different functions out there.

• 43:34Chris Benson?

  And I think that's really important because it lends itself to why this is so powerful. And going back to what you said a moment ago, Daniel, you mentioned the fact that we'll add complexity. Because if you have a really complex function, that one hidden layer might eventually get there, but it may be unreasonable in terms of the time it takes to train it to get there, to get within what is an allowable error for you. And so the way we get around that is we either add more neurons, or we add more layers to it. And so we deliberately add complexity before we know what the solution is. And in doing that, it gives this matrix multiplication a lot more options on finding all those little things.

• 44:14Chris Benson?

  You know, here's a line, and here's a line with another line that creates a shape. And lo and behold, it turns into a part of the face or something eventually. And so by having these layers... That complexity allows you to pick apart pieces of it and do it. And so you're balancing how big of a neural, how big of a network do I want for computational expense versus what does my problem require? And so it's, it's, when you get into this field, you learn that you have to balance that and then you, and then obviously have various architectures that lend themselves to solving particular problems.

• 44:49Daniel Whitenack?

  Speaking of getting into this field, I think maybe, you know, with the few minutes that we have left here, Chris, it might be good to just talk about, you know, if you're getting into this field, or if you really, like, you've done some tutorials with neural networks, but you don't kind of have this fundamental understanding of how they operate. How can you get some of that intuition about how neural networks operate? I know one of the things that, that I did in the past that was really, really helpful for me was implementing kind of a simple feed-forward neural network from scratch. And I did that just for the iris classification problem, which is kind of a very well-defined

• 45:32Daniel Whitenack?

  classic problem in machine learning where you're trying to classify types of iris flowers based on the measurements of their petals. And I did this in, in the Go language because I was also interested in, in that and using that. But I think whatever language you use, it, it doesn't really matter, but just kind of introducing each of these components, like the neuron, the activation function, this loop of training, is really, really useful to gain a fundamental understanding. And one, one way, if, if that's kind of intimidating to you, I might recommend the great book from Joel Gruse called Data Science from Scratch. He just released a second edition of that book.

• 46:15Daniel Whitenack?

  And he added in a bunch of things about neural networks, deep learning, recurrent neural networks. But in that book, he kind of walks you through some implementations of neural networks from scratch using Python. And so I think that's a, it's a really great way to gain this fundamental intuition and something that I think would be even good for, for me to do occasionally in different languages or in different ways to kind of help me keep that, that intuition.

• 46:46Chris Benson?

  Yeah, not only that, there are so many, there's so many approaches. I really think it's such a great time to get into this field right now. It's, it's, it has, I won't call it mature, but it has matured a lot in the last few years. And back when you and I were first looking at it, incidentally, that's, that's what I did as well. And same programming language, I, I created a toy neural network in Go. Just to make sure that I understood where, where I wanted to start from and, you know, all the pieces made sense to me. It was, it was more of a, a science, you know, experiment kind of thing. And, and before moving into frameworks, which is where the real action is. But there's a lot of learnings where if you're into books, there's all sorts of different books.

• 47:31Chris Benson?

  There's the, there's the deep learning textbook, which is, which was written by several of the luminaries in the field. You gotta love your math if you want to jump into that one. If you're very comfortable with your linear algebra and your calculus, then that's a great place to go. If you're not so, then it's a good reference to try to work toward, but you might want to find some, some books that cater toward whatever your knowledge level is. And also there's a whole bunch of, of really fantastic courses online. Coursera has them, Microsoft, Google, there's a bunch out there. And so whatever your approach to learning is, however you consume new information best.

• 48:12Chris Benson?

  I can almost guarantee there's, there's a high value way of doing that, that you can cater it around yourself. I know that didn't really exist when we were doing ours originally, but the last two, three years, it's just exploded.

• 48:25Daniel Whitenack?

  Yeah, there's, there's great online resources. I really like the machine learning crash course from Google. There's of course the fast.ai material that's all online that people love. So it's a great time to get into the field and this is, you know, hopefully this has given you a sense of what neural networks are, or given you a refresher in that, to really encourage you that like we, we can, you know, get some intuition about what's going on under the hood here and, and that's not too far away from you. Within reach. So if you have a passion about this stuff, you know, get involved, dive into some resources. Let us know if you need help finding those resources. And yeah, I'm, I'm just excited

• 49:08Daniel Whitenack?

  about the next 50 episodes that we get to dive into more about AI. Chris.

• 49:13Chris Benson?

  I am too. I hope, I hope people listening out there will join us in the various communities. We're on Slack, we're on LinkedIn, we're on Twitter, and, and we really do have a lot of great conversations. And as we look toward the next 50 episodes, we really want your input. What do you want to hear about? Who do you want to hear from? What topics are of interest to you? And, and we really want to build the next 50 episodes around you.

• 49:38Daniel Whitenack?

  Yep. And congrats again, Chris. Great, great to be doing this with you and looking forward to, to the future episodes. We'll see you next week.

• 49:48Chris Benson?

  See you next week. Thank you.

• 49:52Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or a social network, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for change log is provided by Fastly. Learn more at fastly.com. And we catch our airs before our users do here at change law because of rollbar. Check them out at rollbar.com slash change log. And we're hosted on Lino cloud servers. Head to lino.com slash change law. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in your email address, get our

• 50:35Speaker 00

  weekly email, keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.