The mathematics of machine learning

Tivadar Danka is an educator and content creator in the machine learning space, and he is writing a book to help practitioners go from high school mathematics to mathematics of neural networks. His explanations are lucid and easy to understand. You have never had such a fun and interesting conversation about calculus, linear algebra, and probability theory before! Sponsors: RudderStack https://rudderstack.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog – Smart customer data pipeline made for developers. RudderStack is the smart customer data pipeline. Connect your whole customer data stack. Warehouse-first, open source Segment alternative. Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipLaunchDarkly https://launchdarkly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Ship fast. Rest easy. Deploy code at any time, even if a feature isn’t ready to be released to your users. Wrap code in feature flags to get the safety to test new features and infrastructure in prod without impacting the wrong end users. Featuring: Tivadar Danka – Website https://www.tivadardanka.com, GitHub https://github.com/cosmic-cortex, X https://x.com/TivadarDankaChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Mathematics of Machine Learning - Roadmap Graphic | Twitter https://twitter.com/TivadarDanka/status/1426158532311896067Books “Mathematics of Machine Learning” by Tivadar Danka https://tivadar.gumroad.com/l/mathematics-of-machine-learningUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  There is a pattern I see in mathematics very frequently that old ideas influence new ideas. As a researcher myself, when I was working on one of the problems which I solved in my PhD, one of the key components of the proof I found in a paper which was written during the Second World War. It's very odd. As a machine learning engineer, when would you use papers that are more than 20 years old? Never. It never happens because the technology moves so fast, but mathematics is kind of timeless because it's about the language. It's the language of algorithms.

• 0:34Speaker 01

  Big thanks to our partners, Leno, Fastly, and Launch Darkly. We love Leno, they keep it fast and simple. Check them out at leno.com/changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com. And get your feature flags powered by Launch Darkly. Get a demo at launchdarkly.com. This episode is brought to you by our friends at RudderStack. And we're calling all data engineers to check out RudderStack Cloud and start building smart customer data pipelines. RudderStack is warehouse-first, no more silos. RudderStack builds your customer data lake on your data warehouse, not theirs, enabling all functionality of a CDP with more security and retaining full ownership of your data. It's open source and API-first. RudderStack can be easily integrated into your existing development processes.

• 1:17Speaker 01

  And because they're open source, you can see all their code, so you don't have to worry about vendor lock-in or black boxes. And best of all, they have transparent pricing. Stop paying your CDP a premium to store your data. RudderStack is free up to 500,000 events and pricing scales transparently from there. Learn more and get started at rudderstack.com. Again, rudderstack.com. That's R U D D E R S T A C K dot com.

• 1:52Speaker 01

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We are at Practical AI FM.

• 2:17Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm a data scientist with SIL International. And I'm joined as always by Chris Benson, who is a strategist at Lockheed Martin. How are you doing, Chris?

• 2:31Chris Benson?

  I am doing very well, Daniel. Enjoying nice cooler weather and looking forward to some cool topics on today's episode.

• 2:38Daniel Whitenack?

  Yeah, for sure. Are you brushed up on your math?

• 2:41Chris Benson?

  Oh, my God, no, I'm not. I'm not as good at math as I wish I was. So I am super jealous when we find somebody to talk to that is that is really, really good at math, like I think we're about to.

• 2:54Daniel Whitenack?

  Yeah, well, I think we're both lucky in that in that respect because we've got Tivadar Danka with us, who is an educator in the machine learning space. He's writing a book, which he talks about as going from high school mathematics to neural networks, which is a pretty cool idea. Welcome, Tivadar.

• 3:11Speaker 00

  Hi, everyone.

• 3:12Daniel Whitenack?

  Yeah, it's great to have you here.

• 3:14Speaker 00

  It's great to be here.

• 3:15Daniel Whitenack?

  Yeah. So first of all, I guess, how did you start thinking about like, hey, there's a need out there to help people along the pathway from like high school mathematics to neural networks?

• 3:28Speaker 00

  So I'm a mathematician by training. All of my higher education in the mathematics field, like a bachelor's, master's and PhD. So it was like a kind of a given to me. I'm an expert in mathematics. And I just started to post on Twitter about mathematics and machine learning. Because after my PhD in pure mathematics, I started working in machine learning, more specifically bioinformatics. And then I just started to create content on the topic. And it got traction as I created more and more content. I got more and more feedback. I realized that there is actually like a need from people. They want to understand mathematics.

• 4:13Speaker 00

  But often the learning curve is so steep that many people fall off of the cliff, which I think is quite unfortunate because mathematics is not that complicated. You know, mathematicians make it complicated. It doesn't have to be. If you can properly explain and motivate concepts, it's like what we use in machine learning are quite intuitive. And I really enjoy teaching and explaining stuff. I realized this like a few years ago. And ever since then, I'm putting, you know, more and more energy into teaching, educating, creating educational content in the topics of mathematics and machine learning.

• 4:50Chris Benson?

  Where were you when I needed you so desperately a few years ago as I was getting into this field? What you just said described my entry into deep learning because I was coming from a software programmatic perspective, writing code and wanting to get into it. And I had gotten up through some basic calculus. I think, you know, what we would describe it in our university level as Calc II, integrals and stuff. And that's as far as I made it. And it wasn't far enough to do the stuff. And so I think that that is an experience many, many people have is the frustration of trying to level up in mathematics to be able to do this kind of, you know, really cool work that we're all talking about and working in today. So it seems like something that's desperately neat.

• 5:33Chris Benson?

  I think, I don't think that's changed at all since I came through.

• 5:37Speaker 00

  Usually, usually how they teach mathematics, that classical mathematics courses are very boring, which is something I try to change.

• 5:44Daniel Whitenack?

  Yeah, that's unfortunate that it's that case because I do, like, I have had math classes in the past with, and I'm sure it's just because of the instructor, but ones that I have really enjoyed. But also I have had those that are just like really tough, not even from a content perspective, but just from a like grinding through the content perspective. I'm curious from, as you're starting to work with more and more people and talk to them specifically about like mathematics in the context of machine learning, from your perspective, like as a practitioner, like I can download, you know, TensorFlow, run through some tutorials, like download Hugging Face or something.

• 6:26Daniel Whitenack?

  And like do some pretty like state of the art stuff without having much of an understanding of what's going on under the hood. From your perspective, like what as a practitioner, are some of the benefits of like spending time digging into some of the theoretical and mathematical elements of like what's going on under the hood?

• 6:49Speaker 00

  It really depends on the stage where you at in your machine learning journey. I strongly believe that when you start out, you don't really need higher mathematics. High school mathematics is often enough. There is this awesome machine learning course from Fast AI, which explicitly states that all mathematics you need to start is basically taught in high schools. So you don't really need more. And as you go on and on in your journey, you pick up pieces of mathematics here and there. And this is how I think machine learning practitioners should start about learning mathematics. But there comes a point when using built in solutions, like you just mentioned, like downloading like a pre-trained model from Hugging Face,

• 7:34Speaker 00

  won't work. And how I see mathematics is that in the aspect of machine learning, this is like really the language of machine learning. Mathematics is the language in which algorithms are written. And if you want to effectively talk about those algorithms, you need to learn this language. You don't need to be like a master writer in this language, but you need to speak it properly to exchange ideas with other people, your colleagues and whatnot. And also, the level of mathematics you need depends on what will be your job or what is your job. For instance, if you are like a machine learning ops engineer, I don't think you really need that much mathematics. If you're a data scientist,

• 8:18Speaker 00

  you need a lot of probability theory and statistics, for instance.

• 8:22Chris Benson?

  So for me, this is, as I kind of mentioned, a particularly kind of a fascinating thing because, and I'm still very questionable in my mathematics. It was kind of learning as I could, but I was at the time coming through the teens, you know, on or around 2015, when I was struggling with this and coming from feeling a strong deficit in my mathematical knowledge. How, I'm really fascinated when you talk about the Fast AI course, talking about you only need it in high school level. Like, that's something I wish I had known then. How would you reset me? I'm kind of curious. If I can take you right now and I'm going to transport you back in time, I guess six, seven years when I'm struggling through that, what would that look like? Point, what would you advise me so because I did waste a ton of time.

• 9:07Chris Benson?

  I struggled through stuff I may not have needed. I probably missed some stuff I did need. How would you lay it out for a beginner like me who's coming from another technical skill, but isn't a mathematician?

• 9:17Speaker 00

  I had the same feeling, actually, just in other other field in mathematics. I would basically always suggest people to find a problem which they are interested in, and then just start building stuff. Once they start, you know, working out the solution, they will stumble upon these smaller obstacles. And sometimes these obstacles are related to mathematics. And then if they know that they have some kind of difficulty of understanding, I don't know, matrix operations, then they should do like a focused research on that topic. Because if you are a machine learning practitioner, you you pick up a bunch of mathematics on on your way. Right?

• 9:55Chris Benson?

  I found it hard to connect. I was trying to go back and learn the matrix multiplication. And then simultaneously, coming from a software background, we tend to to go and try to build something anyway. And so I was trying to do that with the tools of the day, which was are different from the tools of today. But trying to connect across those two to where I'm attempting to learn the mathematics of machine learning based on what I'm reading about and I'm supposed to know. And then trying to use the tools of the day, connecting those was agonizing at times. How do you do that? If you're taking someone that is just getting into it, it's today, there are someone's coming out, maybe coming out of university, they're a software developer. They took one high level machine learning class and they want to dive into it.

• 10:40Chris Benson?

  How do you help connect them with that practical side of it, of the using the tools and and the code, with the theory to understand it, to actually not just be doing it rote, but to be having a sense of what you're trying to accomplish?

• 10:54Speaker 00

  I usually try to motivate every concept in mathematics with machine learning applications, potential machine learning applications. So for instance, if we are talking about matrix multiplications, if you are in a classical linear algebra class, they just, you know, like, give you the formula for matrix multiplication and you have to have to understand it and have to understand it as is. But if you actually take a look at matrix multiplication from a different perspective. So if you go back to machine learning, for instance, if you think about neural networks, if you are a practitioner. It probably works with fully connected layers, right? So you know what they are. And then I would tell you that, hey, fully connected layers are actually defined by matrices. And why do we need to understand what matrices are?

• 11:39Speaker 00

  Because we want to understand what our neural network is doing in this small layer. So then this would serve as like an initial motivation to raise your interest in the subject. And then when I talk about matrix multiplication, for instance, I always introduce data transformations. So from an abstract viewpoint, a neural network is nothing else than something which takes the data and transforms it step by step, eventually giving you a representation from which you can read out class labels or whatever you want. And those transformations are essentially made from linear transformations, which are given by matrices. So linear transformations are essential in machine learning. And then, you know, we introduce the concept of linear transformations.

• 12:24Speaker 00

  Then I would take you through what is the structure of linear transformations? How can you actually describe them? And this is how you stumble upon the very definition of matrix, because a matrix is nothing else but the image of unit vectors under a linear transformation. And then, once you understand that, I can just tell you that if you compose two transformations, then you get basically a third transformation. And if you describe the matrix of these composed transformations, then you can calculate by hand how two matrices are multiplied. So composition of linear transformations give the matrix multiplication definition. I think, for instance, this way is much more interesting

• 13:05Speaker 00

  and much more understandable for somebody who is coming from an application perspective.

• 13:10Chris Benson?

  It's very coherent. I got it and I didn't get other explanations that before I met you in my learning days, yeah, I was struggling. That's a great way of putting it.

• 13:19Speaker 00

  Awesome.

• 13:19Daniel Whitenack?

  Yeah, it's probably also a bit, I understand your plight because it's probably a bit difficult also in audio format on a podcast to describe some of these things without visual aids or something like that. But I was wondering, you mentioned sort of getting a lot of Interest on Twitter and people starting to interact with you enough that you were like considering creating some of this content which you have and are creating. I'm wondering, like, was it mostly current practitioners that were sort of interested in this and like deepening their level of understanding? Or was it mostly like those trying to get into the

• 14:00Daniel Whitenack?

  machine learning or AI space and having trouble like understanding jargon and understanding the theory? What sort of mix of people was it that you were interacting with and are interacting with?

• 14:10Speaker 00

  It's a mixture of both. Twitter is huge. Every group is properly represented in terms of machine learning. So I interacted with both of these groups, yeah. So once you are, you reach like a enough experience in your machine learning career, you have some kind of curiosity about understanding the underlying so-called magic, which is not magic, but you want to understand what is behind the curtains. Then you're basically happy to consume any kind of content which can basically give you that. But I also also meet with young students who they are taking their first mathematics classes and they want to to understand what's this about. They want to have like more intuitive explanations.

• 14:54Speaker 00

  And they are not that sure how to connect it with machine learning at that point because usually in traditional courses, you are learning mathematics and you are learning computer science. Usually it's like a two different tracks. And mathematics courses are usually taught by mathematicians. And I think this is a huge problem because even though I'm a mathematician, I know that this is not ideal. Because there is this quote from Google that mathematicians are like French people. They see something and they translate it to their language. And from that point, it means something completely different. So this is why it's suboptimal that you are basically taught mathematics by mathematicians.

• 15:44Speaker 04

  ChangeLog++ is the best way for you to directly support Practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal, and helps sustain our production of Practical AI into the future. Simply follow the ChangeLog++ link in your show notes, or point your favorite web browser to changelog.com/plusplus. Once again, that's changelog.com/plusplus.

• 16:30Daniel Whitenack?

  Well, where I sort of came across your work initially was on Twitter, and I think a post that probably got a lot of attention, where you broke out and had a diagram of sort of all of these different areas of mathematics that you sort of categorized as having something to do with the mathematics of machine learning, and how you broke it down, which I thought was, first of all, like a really well-made diagram, and easy to understand. But I think also people, maybe where Chris was talking about, where they sort of have all of these bits and pieces of what they think is maybe useful in machine learning and neural networks, but to have it broken down on sort of a single page, and like, hey, what's on the radar?

• 17:14Daniel Whitenack?

  What should I be thinking about is maybe useful? Is that sort of breakdown of mathematics? And we can go into the details of what's included there. How did that sort of come about? And how did you end up doing this sort of categorization? Was it part of the creation of the book?

• 17:31Speaker 00

  It was before the creation of the book. Oh, okay. I did it to serve as some kind of blueprint. I tried to min-max this roadmap. So to be more specific, I tried to leave out every field or sub-field which is not of interest. But I wanted to include everything that might be eventually interesting. And I worked from backwards. So I started with neural networks, like what you need. Like basically you need optimization techniques and you need linear algebra to describe models. And you need probability theory to fit models to data. So from that point, I started to work backwards. If you want optimization methods, what do you need? You need calculus to do that.

• 18:15Speaker 00

  If you want to describe your models, you need linear algebra and you need a bit of calculus. And you also need probability theory to fit models to data and interpret results. And after I got those three, essentially broke down into small pieces. Basically, I highlighted a few key milestones, which we will encounter later. As I said, I tried to be as minimal as possible.

• 18:36Chris Benson?

  And I think right now, this is one of those moments going back to what Daniel said is, you created the diagram, so you know it really well. And Daniel and I are able to look at it visually right this second. But just for listeners who are driving in their car and listening to this, I just thought I'd take a two-second describe the central line. And then before to do that, so that people can get the visual. And you kind of did that. You start with fundamentals and you hit calculus. And with for each of these things, you hit all those things that go in. Calculus and linear algebra lead to multivariate calculus, lead to probability theory. And then as we get into the neural network section, basics of optimization and mathematical statistics are together and then neural networks down there.

• 19:17Chris Benson?

  And all of these have this amazing branching into all of the knowledge areas. I mean, this is, I wish I'd had this so bad a few years ago. This would have really, really helped. And so I just wanted to kind of call out the visual aspects of it as you continue kind of describing how you got there.

• 19:34Daniel Whitenack?

  Yeah, definitely. It's always helpful for me to see something in a visual form like this. And we talked a little bit, you went into a little bit of detail about how you connect. Matrix operations with neural networks and all of that. I'm wondering if you could sort of do that with a couple of these other ones too. When you first started talking about like how you came up with these categories, you talked about optimization along with linear algebra. So maybe for those like getting into neural networks, maybe they're not familiar with like what you mean by optimization in a mathematical sense. Could you give just a little bit of an intro to like, what do you mean when you say optimization?

• 20:17Daniel Whitenack?

  And where does that fit within the framework of neural networks and machine learning?

• 20:21Speaker 00

  Sure, so let's just talk about neural networks because they are the best.

• 20:26Daniel Whitenack?

  I think our listeners, we don't have to convince them too hard on that.

• 20:29Speaker 00

  Yeah, okay, cool then. We have a common ground there. So again, from a mathematical perspective, a neural network is a function. You have an input, an output, and you also have a bunch of parameters, like basically define this function. And these parameters are in form of, you know, real numbers. And your job as a data scientist or machine learning engineer, whatever, is to find a good, good set of parameters that fits well to your data. Right, so how would you do that? You need to have some kind of search in the parameter space. And one very simple, although useful, method is called gradient descent. Essentially, you measure the quality of the fit.

• 21:14Speaker 00

  Like this is what you do with plugging in an error function. You know how well this given set of parameters fits. And then you basically want to minimize this loss.

• 21:24Daniel Whitenack?

  And that measurement of error is based on those like training examples, right? So people probably have heard the jargon like, hey, here's my training set, right? So that's where that comes in, yeah?

• 21:36Speaker 00

  Yeah, and then if you visualize this loss function given in terms of the parameters, you see some kind of landscape, right? Just like you would go on a hike and you pull out a map and you see that this is a hill, this is a valley, this is how you can imagine such a landscape. And what you want to do is we want to climb to the bottom of this landscape because this is where your loss is minimal. And if you are at a given point and your goal is to climb to the lowest point of the landscape, how will you do this? You look around yourself, you try to figure out which way is the steepest descent, and then you start going there. You take a step in that direction. And once you took a step, then you repeat this whole procedure, right?

• 22:18Speaker 00

  So again, you look around yourself and see, determine the direction of the steepest descent, and then go another step. And this is high-level description of the method. And if you want to implement this mathematically, you need to introduce the definition of gradients. So if you have a landscape, how do you define where to go? And this is where calculus comes in. This is where gradients come in. Because the gradient is actually the direction of the steepest increase, but in gradient descent, you actually move to the opposite direction of the gradient. So this is how you realize that you have to understand differentiation and gradients, because this is what you do if you want to optimize. It's quite interesting that it is an extremely simple algorithm.

• 23:02Speaker 00

  It was known a few hundred years ago, but still useful, and it still makes possible to train neural networks. Although modern versions are, of course, like, how to say, like, a supercharged version of the classical gradient descent. But this is what happens, basically.

• 23:18Daniel Whitenack?

  This is a safe space, Chris.

• 23:20Speaker 00

  Yeah. I think it's more user-friendly if you start with single-variable calculus, because multi-variable calculus is slightly more complicated and not that much abstract, but you have a lot more notation. And it's very easy to get lost in those, you know, huge matrices and functions if you don't understand the core concepts. So this is why I usually recommend taking single-variable calculus first, because then you can understand the concepts of derivative very easily. Derivative is just basically the speed of an object. It's easy to understand, but if you if you try to generalize this concept into higher dimensions, you include possible complications. For instance, I don't know which direction do you measure the rate of change?

• 24:03Speaker 05

  Okay.

• 24:04Speaker 00

  It's not that clear why would you define partial derivatives?

• 24:07Speaker 05

  Gotcha.

• 24:08Daniel Whitenack?

  And one of the other big areas that you have on this map is probability theory. You already mentioned that sort of data scientists oftentimes are very concerned with probability theory. Maybe there's like certain types of models that they use that are like, maybe they're thinking of like naive Bayes type models and there's like these things out there. But when we talk about neural networks, where is probability theory kind of intersecting with neural network based models?

• 24:37Speaker 00

  At the very foundations, I would say. For instance, let's talk about mean squared error, which you encounter. Or even better, let's talk about cross entropy. It's more common for classification problems. So when you want to train a classification model, you often use the cross entropy error. This is the concept which is introduced by probability theory. If you want to understand intuitively what it means, then you need to understand, for instance, the fundamentals of random variables. You want to understand entropy itself, expected value. So what, once you can think in terms of expected value and entropy, cross entropy becomes much simpler for for you.

• 25:19Chris Benson?

  So to go on about the graphic, because I'm in love with the graphic, because it's super helpful. I know this is stuff that you guys know, and I've learned parts of it, but it also shows me a lot of things that I need to ramp up on even today. So that begs the question, it doesn't matter if this was when I was first learning or whether it's now and I'm and I'm trying to continue to level up in my own skills. How can, this maps out all of these knowledge areas and it helps me, I can look at each one and say, what do I know about that? What have I learned about that? Do I feel comfortable with that? How can I take this mapping and translate that into a practical learning plan to help myself move, you know, on up and develop the mathematics that I

• 26:01Chris Benson?

  need to do the work that I love to do? How do you translate that into that kind of linear progress mode from the the map, you know, that kind of covers so much here?

• 26:11Speaker 00

  So I'm actually writing a book about this topic. So maybe now it's the perfect time to talk about this because

• 26:16Chris Benson?

  Yeah.

• 26:17Speaker 00

  As I mentioned, I kind of created this roadmap as a blueprint for my book. So the roadmap itself, this diagram which we are talking about is just like a rough learning plan. And then this book is where I put this roadmap into action. So I organize the chapters in order so that it would take you through from top to bottom.

• 26:38Chris Benson?

  I can't wait.

• 26:39Daniel Whitenack?

  Yeah, so the top, for those listening who aren't looking at the graph, was sort of at the top is calculus and linear algebra, then going through multivariate calculus, probability theory, basics of optimization, mathematical statistics and neural networks. In the book, what sorts of decisions are you making in terms of like, because you mentioned notation, jargon being like one of the main, main sort of blockers for people when they encounter these subjects. It's like, oh, there's all this new jargon. Not every book or course uses the same jargon or notation, depending on like your

• 27:21Speaker 00

  This would be a problem.

• 27:21Daniel Whitenack?

  Yeah, your field of study. Like I know in physics, oftentimes I would encounter a lot of much different jargon than I would if I was reading like, even like a sub-discipline, like mathematical physics types of like books or articles or something. So I guess my question is like, how are you deciding as you're writing the book? Like what jargon and notation to use and like how much to pull in code into that? Or like algorithmic thinking into that, I guess?

• 27:49Speaker 00

  I always, always put machine learning first. When I decide whether or not I should include the topic, I always think it through from the perspective of machine learning. So can I motivate this with an example from data science or machine learning? Where will this be used? If this won't be used anywhere, I won't include that in the book. I don't want to basically cause math overload, which all of us experience at some point in our studies.

• 28:16Chris Benson?

  I still do on a regular basis.

• 28:18Speaker 00

  Yeah, trust me, I also experience that. Guiding principles, number one, no unnecessary stuff. Guiding principle, number two. I always, always introduce every concept as visual as possible. Use as much geometry and then visual as possible. Because this is how you think. If you work in mathematics for long enough, you learn to think in terms of formulas. But I think this is kind of like a side effect of being a mathematician. Nobody should, should be expected to think in terms of formulas about, you know, concepts such as gradient descent. And this is what is basically guiding me all, all the time. Even with simple concepts such as matrix multiplication or matrix determinants.

• 29:00Speaker 00

  Determinants for matrices, I mean, they are tough to understand if you don't know the geometry behind them. But once you understand the geometry, it's like easy peasy.

• 29:10Chris Benson?

  You know what? The thing in my head is you're telling me about this and I'm so thankful that you're taking the approach that you are. This is humane machine learning mathematics. I felt like when I was starting my journey along this path, it was not humane at all. And I always felt out of depth, but I love the, the what your approach on that. So I mean, are you really targeting it's, I mean, it feels in the conversation, it feels like you're targeting me for this.

• 29:34Daniel Whitenack?

  I don't know if there's going to be a dedication to you in the book, Chris. I, I, I don't, I don't think he's promising that.

• 29:42Chris Benson?

  To help Chris get out of machine learning purgatory. There's your dedication right there.

• 29:46Speaker 00

  That can be arranged. Essentially, basically it's for every computer scientist who is interested in machine learning. And even those who are not interested in machine learning.

• 29:55Chris Benson?

  That sounds great.

• 29:56Daniel Whitenack?

  Yeah.

• 29:57Speaker 00

  It's important that it is not for mathematicians.

• 29:59Daniel Whitenack?

  I know myself in just thinking about like the things I'm exploring in machine learning and AI. They're like changing so rapidly. And there's new things being introduced all the time. From your perspective, as you're sort of have, you're monitoring the trends within the industry and like talking to different people. How stable are these like sorts of fundamentals as compared to like what people are exploring? And what's becoming more mainstream? Are there like new, new areas of mathematics that are starting to like impact? Machine learning and AI that like maybe people should be aware of, or is it basically just like?

• 30:42Daniel Whitenack?

  Everything is built on these building blocks and like that's pretty much ninety nine point nine percent of things.

• 30:49Speaker 00

  I would say the latter. I mean,

• 30:50Daniel Whitenack?

  Okay.

• 30:51Speaker 00

  Essentially, new mathematics fields influencing machine learning is like science fiction for me.

• 30:57Daniel Whitenack?

  Yeah.

• 30:57Speaker 00

  At the moment, because mathematics is always hundreds of years in front of applications, because this is how mathematicians operate. I mean, they always try to generalize things to to prove even more and more abstract theorems and so on. So it's like an intellectual sport, which is like a after a while it loses touch with with reality. And I say this as someone who who wrote his PhD in mathematics. I have research papers, so I know this firsthand that most research papers have absolutely nothing to do with with applications. Even though mathematicians may try to deny this, to get grants and whatnot, but sadly, this is the case. So I see this, as I said, mathematics as the language which we speak.

• 31:41Speaker 00

  Even though you can write new novels or poems in this language, the language itself remains the same. And it's it's it evolves slower than, for instance, literature.

• 31:51Daniel Whitenack?

  Yeah.

• 31:51Speaker 00

  Although they are they are interconnected, they influence each other. Modern mathematics is something like a like a abstract art. You won't understand it unless you are you are an abstract artist yourself.

• 32:02Chris Benson?

  That's a really interesting dichotomy there when you talk about, you know, we think of machine learning as just racing along in terms of its impact and the development of it, you know, and you can barely keep up with it because it's it's going a thousand miles an hour. And yet it's built on something that, you know, to your point just a moment ago, is hundreds of years ahead of applications. You know, and is a long, long in the development. It's not something I had ever considered before. It's fascinating.

• 32:29Speaker 00

  Just to give you give you an example, gradient descent, hundreds of years old algorithm. Base CRM and even nowadays you can still uh discover new uses of this.

• 32:40Daniel Whitenack?

  Yeah, I think um in my mind, not being a mathematician, I have no idea about like that sort of abstract art world that you've described. Like, I totally don't understand or even have can fathom what's going on there. But I think about things like, uh, I think what I had in my mind is like, when I'm seeing trends in the AI field that are like graph neural networks, bringing in like different structured data and specific ways of processing that. Or I also see like more like mention of differential equations, partial differential equations in relation to like AI.

• 33:19Daniel Whitenack?

  I didn't do a search on your your diagram, but like, are there things that you're seeing that maybe are old in terms of like the math world? But like people are bringing in to the AI world in sort of ways that they didn't before? I guess maybe that's a better way to phrase it.

• 33:38Speaker 00

  There is a pattern I see in mathematics very frequently that old ideas influence new ideas. So mathematics is kind of timeless, even now, hundreds of years old results can basically implant new ideas. As a researcher myself, when I was working on one of the problems which I solved in my PhD, one of the key components of the proof I found in a paper, which was written during the Second World War. It's very old. As a machine learning engineer, machine learning researcher, when would you use papers that are more than twenty years old? Never, it never happens because it moves, technology moves so fast, but mathematics is kind of timeless because,

• 34:20Speaker 00

  as I said, it's about the language, it's the language of of of algorithms. And this is how I think it can basically influence for the new ideas like PD with neural networks. What are the technical term for those?

• 34:34Daniel Whitenack?

  Yeah, I I see the the mention quite a bit. Yeah, people are exploring that more. Yeah, along with other things like graph theory and other other things like that. Yeah.

• 34:45Speaker 00

  Graph neural networks, I'm not very familiar with, unfortunately.

• 34:49Daniel Whitenack?

  Yeah.

• 34:50Speaker 00

  Neither with neural networks that use partial differential equations.

• 34:53Daniel Whitenack?

  Well, I appreciate you taking your perspective on like how those interactions between the math world and the machine learning world happen and trends in those. I definitely appreciate you taking time to chat about all of these things on the show. It's been, it's been really useful and interesting. If you're listening to this, we're going to include a link to Tevadar's book and website and a link to this sort of diagram that we've been talking about in the show notes. So make sure and check that out. If I understand correctly, the book is out for, for early release, right?

• 35:29Speaker 00

  Yeah, it's an early access, which means the book is out early yet. And I publish chapters just as I write them. It's usually one chapter per week. And one of the big advantages of, of this early access program is that you essentially, as a reader, you have direct access to me all the time. So we have a closed Discord server where you can just ask me up with any question you might have. And if you have some feedback, I'll just correct it in the book. And next week, you'll get an updated version with better explanations. So I chose this format because, as I mentioned, I want to write this book for readers, not for mathematicians. So this is why I kind of love this early access solution.

• 36:12Daniel Whitenack?

  Awesome. Well, I appreciate you being also vulnerable during the writing process as well and getting that out to people and letting them, you know, get feedback to you. I think that will produce a really valuable resource. So yeah, we'll, we'll be watching and I'll be excited to have you back on the show to educate us a little bit more on machine learning math. So thanks, Tevadar.

• 36:35Speaker 00

  Awesome. Thank you very much for the opportunity.

• 36:40Speaker 04

  Thank you for listening to Practical AI. We have a bundle of awesome podcasts for you at changelog.com, including our brand new show, Ship It with Gerhard and the Zoo, a podcast about getting your best ideas into the world and seeing what happens. It's about the code, the ops, the infra, and the people that make it happen. Yes, we focus on the people because everything else is in implementation detail. Subscribe now at changelog.com/shipit or simply search for Ship It in your favorite podcast app, you'll find it. Of course, the galaxy brain move is to subscribe to our master feed. It's all changelog podcasts, including Practical AI and Ship It in one place. Search changelog master feed or head to changelog.com/master and subscribe today. Practical AI is hosted

• 37:25Speaker 04

  by Daniel Whitenack and Chris Benson with music by Breakmaster Cylinder. We're brought to you by Fastly, Von Starkly, and Linode. That's all for now. We'll talk to you again next week.