Visualizing and understanding RNNs

Andreas Madsen, a freelance ML/AI engineer and Distill.pub author, joins us to discuss his work visualizing neural networks and recurrent neural units. Andreas discusses various neural unites, RNNs in general, and the “why” of neural network visualization. He also gives us his perspective on ML/AI freelancing and moving from web development to AI research. Sponsors: DigitalOcean https://do.co/changelog – Check out DigitalOcean’s dedicated vCPU Droplets with dedicated vCPU threads. Get started for free with a $50 credit. Learn more at do.co/changelog https://do.co/changelog. Discover.bot https://bit.ly/2OWMyw2 – A digital space for bot developers and enthusiasts of all skill levels to learn from one another, share stories, and move the bot conversation forward. Want to learn more about building bots? Get started with their Guide to Bot Building Frameworks https://bit.ly/2Vxk7qO. strongDM https://www.strongdm.com/?utm_source=changelog – Manage access to any database, server, and environment. strongDM makes it easy for DevOps to enforce the controls InfoSec teams require. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: Andreas Madsen – GitHub https://github.com/AndreasMadsen, X https://x.com/andreas_madsenChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Distill.pub by Andreas https://distill.pub/2019/memorization-in-rnns/Taylor approximation and other mathematics related to neural networks https://medium.com/@rajarshi.banerjee47/the-mathematics-of-neural-networks-5af71963e538Books “Programming Collective Intelligence” by Toby Segaran https://www.amazon.com/dp/0596529325Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Change Law because of Rollbar. Check them out at rollbar.com. And we're hosted on Leno cloud servers. Head to leno.com slash change log.

• 0:16Speaker 00

  This episode is brought to you by Digital Ocean, the simplest cloud platform out there. And we're excited to share they now offer dedicated virtual droplets. And unlike standard droplets, which use shared virtual CPU threads, their two performance plans, general purpose and CPU optimized. They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, C I C D, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers, or active front end web servers that need to be full duty CPU all day, every day. Then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive, starting at 40 bucks a month.

• 1:01Speaker 00

  Learn more and get started for free with a $100 credit at do.co slash change log. Again, do.co slash change log.

• 1:21Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slide with us around various topics of the show at change.com slash community. Follow us on Twitter. We're at Practical AI FM. And now into the show.

• 1:47Daniel Whitenack?

  Welcome to another episode of Practical AI. I'm Daniel Whitenack, a data scientist with S I L International. And I'm joined by my co-host, Chris Benson, who's a chief AI strategist at Lockheed Martin. How are you doing, Chris?

• 2:03Chris Benson?

  Doing great today. How's it going, Daniel?

• 2:04Daniel Whitenack?

  It's going good. It was a long weekend here in the, in the U.S., a holiday weekend. So I know you're, I know you're a vegan, but I don't know if you, you barbecued any, anything, but that's kind of the tradition in the U.S. here on this weekend.

• 2:21Chris Benson?

  Well, I didn't barbecue, but we ate far too much. So definitely feeling that. Need to, need to get out there and walk or run a bit to get to work off the calories.

• 2:30Daniel Whitenack?

  Yeah, definitely. Well, we'll, we'll jump right into it. I'm really excited. Last, you know, I don't know when it was. It was a little bit ago. I usually like to watch this publication called Distill. So you can go to distill.pub and it's a really great resource to kind of learn about various topics in machine learning and they have really great articles that give explanations of various topics and kind of state of the art research and all sorts of things. And recently I saw a great article on there about visualizing

• 3:07Daniel Whitenack?

  memorization in recurrent neural networks. And the author of that publication of that article is with us here today, Andreas Madsen. Welcome, Andreas.

• 3:18Speaker 03

  Hello.

• 3:19Daniel Whitenack?

  Yeah, so we're really excited to have you and really excited to dig a little bit into RNNs and visualizing neural networks and all of those things. But if you could just give us a little bit of background about how you ended up in AI and, and what you're doing, what you're doing now.

• 3:40Speaker 03

  Oh, so like eight years ago, I think, like many others, I read the book Programming Collective Intelligence. And that was before university. So at that point, I sort of knew what I wanted to do. So I just took like mathematical engineering at a university. And that was sort of before machine learning came a thing. And now like, yeah, after five years of education, I had my master's degree in that. And now I've been doing freelancing and machine learning for like two years. So I like to do like more

• 4:18Speaker 03

  like heavy science in, in AR, deep learning. But sort of it's a little difficult for me right now. So what I'm doing right now is just like do freelance some months of the year and then the rest I just do like volunteer research with the university.

• 4:35Daniel Whitenack?

  Yeah, that's great. I know, you know, it's uh sometimes it's hard to navigate the the roadmap into the specific area of AI that you're interested in, but it's awesome to see, you know, that you're you're contributing in a great way to the community, but also doing freelancing and all of that sort of stuff. How how has freelancing as a machine learning and AI person, how how has that been? What what sort of insight can you give those out there that might be be thinking about about doing that sort of thing?

• 5:12Speaker 03

  I don't know if I recommend it. And the thing about freelancing is, you know, your clients, they they don't pay the hours, they pay the product, right? And deep learning is very much a research field right now, so it's very hard to guarantee anything. And they don't like that. And so the typical solution you sort of end up with a solutions where you have a very good idea that this is gonna work. And that is sort of more statistical solutions or like or red X, like this kind of like a cycle learn solutions or, yeah. Those more like traditional solutions rather than deep learning.

• 5:55Speaker 03

  This is not that exciting for me, so that's why I like to do the research, but I mean, it's good for the money.

• 6:02Chris Benson?

  So I actually have a quick follow up question about that before we dive, you know, fully into the questions. And that is, you you mentioned that it was as a freelancer, and I've done freelancing in in terms of being like a programmer in the past, but never freelancing in this particular industry. And I was just curious, you mentioned paying by the project, is that do they do you do like a fixed fee for a project, or do people pay you by the hour, kind of in more of a traditional freelance way?

• 6:28Speaker 03

  I'm I'm paid by the day, but they get really upset if they don't get like a product, right? If you come out with this and say, okay, after three months, like this was the contract, and I made this amazing deep learning system, unfortunately doesn't work because it's deep learning and it's research, right? And but I learned a lot, and I think like next time it's gonna be better.

• 6:52Chris Benson?

  I totally understand what you're saying.

• 6:54Speaker 03

  This doesn't work, right? Because I learned something, they don't feel that has any value at all.

• 7:00Chris Benson?

  Totally, totally get it.

• 7:02Daniel Whitenack?

  It's probably hard to manage expectations to when, you know, a client comes and they say, oh, you know, we've heard deep learning solves everything. So I'm assuming you'll be able to solve this problem with deep learning. And in the back of your mind, you're thinking either, you know, I'm not sure if it could be solved, or maybe like just random forest with scikit-learn is probably better for these people. It's always hard to kind of crush those expectations.

• 7:27Speaker 03

  Exactly. Yeah, yeah, yeah.

• 7:29Chris Benson?

  And that's whether you're a freelancer or a permanent employee at a company. It's that's the same same situation. So like to dive in, I just wanted to know kind of how you originally got interested in the visualizing aspect of deep net of deep networks and and, you know, neural units specifically. What what what drew you into that particular subfield?

• 7:51Speaker 03

  I always sort of been been interested in sort of design aspects and that kind of thing. Like before I started university, I did a lot of JavaScript and web development, that kind of thing. But what sort of really kicked my interest was that I was like a conference some years ago, and there was this bank here in Denmark, and they said, okay, we have this amazing machine learning model, and probably should take that was a grain of salt. But they have this amazing machine learning model for predicting whether or not a client or a customer, I should say, could pay back a loan, right? And they were super happy about this model. It outperformed the typical advisors who had like

• 8:34Speaker 03

  a master or bachelor's degree in finance. So that's great. But the really big challenge was actually that they couldn't communicate it. So in the end, it's the advisor who had to make the decision should this customer get a loan or not. And this advisor have his own education that contradicts the machine learning model.

• 8:55Speaker 03

  And so they didn't trust it. And actually, sometimes they just say, okay, I'm not gonna listen to a machine learning model. And that in itself is a problem, but also even if you choose to listen to the machine learning model, how do you explain to the customer that they cannot get the loan? Like, that's really upsetting for the customer, and certainly I don't want to live in this future where as AI become increasingly, increasingly more integrated into our lives, uh, into our lives, uh, maybe we get to the point where also these consequences happen and we cannot really fight against it, it's just sort of this machine learning model that end up controlling our life, right?

• 9:40Speaker 03

  Not because, uh, you know, some dystopian Skynet future, but just because we have learned that we should trust the machine learning model, and when we don't agree with it, that's just how it is.

• 9:52Chris Benson?

  So you you kind of hit, uh, right off the bat, as you were diving into this, kind of the the the issue around trust and issues around explainability. Um, and and then as a consequence of that, the the necessity to communicate the value right off the bat. So so that kind of drove it you into this uh, this focus on visualizing so that people could kind of get it?

• 10:13Speaker 03

  Yeah, I I I talked to him later. So, okay, how do you solve this issue? And amazingly, like, they didn't take it seriously at all. They just said, um, we do some quantization with some colors and some arrows and that's all of it.

• 10:28Daniel Whitenack?

  Yeah. Do you think, I like, I I definitely see it from from that perspective that you guys are talking about in terms of like, uh, you know, communicating the value and giving an interpretation? Do you think that there is value as well on the like AI practitioner side as far as understanding the the types of things that we're doing when we're when we're training a neural network, um, you know, in addition to kind of communicating those results? What do you think are the benefits on the on the practitioner side?

• 10:59Speaker 03

  Oh, absolutely. I think this is actually more what what my uh distill publication is about. Uh because we we create all these uh models here, right? And then we benchmark them on the same dataset and then we clap our hands when we get zero point one percent better performance, right? But we don't really have a a very good understanding about what changed. And so, for example, in my publication, right, I show that I have one model that is and another model and they get pretty much the same performance. But actually, one model is really good at a long term contextual understanding and one model is really good at short term contextual understanding.

• 11:39Speaker 03

  And if you just always look at the accuracy scores or cross entropy loss or whatever. Um you don't get into that, right? So I think it's really fundamentally necessary to do like good science. Uh to look into these things and not just say, okay, this time I worked for half a year, we got the one percent better performance, that's it. I think you need to look into like where did this one percent better performance come from? Is it the place that is actually relevant for the task at hand?

• 12:14Chris Benson?

  So how do you how do you accomplish that in the sense of what tools are you using to visualize uh neural networks uh in the way that you're doing? And obviously, there there are things that we're familiar with, you know, like TensorBoard and such, um, you know. And kind of as an addendum to that, do you, how does design and and understand user interactivity play into your ability to do that?

• 12:37Speaker 03

  Um, yeah, I think for for me, like, uh, TensorBoard is a lot more about debugging. Um, i- i- and it's really difficult to answer the other question, right? Because fundamentally, it's about how do we understand how our deep learning model works. And that is something that's really complicated because and it's like about you take a brain, you slice it up and you look at the individual learns. And somehow you have to ask this person, what is your favorite color? Right, just from looking at the neurons. Hmm. And I mean, I don't think we can imagine that technology in the next ten years when it comes to humans.

• 13:17Speaker 03

  And it's the same kind of complexity that we have when working with these kind of deep learning models. So you somehow have to like aggregate all this information in some smart way. And and that in itself is not just enough. So I think there's like three components to make to really understand something. The one is you need a data set that you can understand. So for example, if you talk about like in natural language processing, like sometimes we just use Chinese poetry generation, right? That's really hard to understand how well it creates Chinese poetry, right? And. I certainly couldn't understand that.

• 14:01Speaker 03

  Oh, I mean, I think the overlap between Chinese poetry writers and machine learning enthusiasts is probably quite small.

• 14:11Speaker 03

  That's true.

• 14:12Daniel Whitenack?

  We need, we need more of them.

• 14:14Speaker 03

  Yeah.

• 14:15Daniel Whitenack?

  So, so like maybe just to follow up on that, like when you're, like you're talking about kind of the, as you put together your data set, as you use these models, there's the need to kind of visualize and understand what's going on at a lower level. Um, how did, like, where, where the workflow of being an AI practitioner, like if I am, if I'm approaching a problem, where, where do you think I need to be thinking about these things on a little bit deeper level? Is it like just at the learning time, like when I learn about R N Ns, then I need to kind of learn about these things and then move on?

• 14:54Daniel Whitenack?

  Or is there an ongoing need to kind of run these sorts of visualizations or maybe more controlled experiments to understand that a, at a deeper level or at a more intuitive level, what's, what's happening?

• 15:08Speaker 03

  Yeah, I think like on once you have your model and you think that it works okay, like look into what does it actually do? Um, it's really hard as a debugging tool to use this kind of visualization, because if you have a model that doesn't really work that well, um, you sort of just get nonsense out of your visualization, right? And so you need like in the end, you have a nice model, it works, and you want to show maybe it's it's better than this other model. Like, okay, compare them up against the same kind of visualization. Does it show what you would expect?

• 15:45Chris Benson?

  So a- as you're going through this process, you kind of mentioned earlier that you, prior to getting into the field, that you had used things like web development tools and you were using JavaScript and such. Is are those skills that you developed ahead of time playing in to kind of like when you actually are producing a visualization, are are using those kinds of tools, like which ones? Are there are there things you could point to somebody else who is kind of getting into, they're interested in this, the subfield of visualizing? And, you know, you would say, hey, go use tools A, B, and C, and and and then this is the workflow through it. Is there something you can point to in that way?

• 16:22Speaker 03

  Um, no, I mean, there's some tools like out there like LSTM this, I think, is is a tool that I've seen some use. Um, I all the tools you're seeing in the this article, I just programmed them from scratch. Um, I use these free for visualization, which basically gives you nothing but a few axes and some, like data management tools. Um, but like, like I said, like, there's these three points, like one is the data set, and then the other part is like having a a good visualization. But then the third really important part is having sort of the feedback loop, where like you can very quickly ask a question, or like in this case, you hover over a character,

• 17:07Speaker 03

  and then you get some sort of visual feedback. And because this is like an interactive part, um, You get this sort of feedback loop and that is really what generates your intuitive understanding. And I mean, you don't need to use web development tools for this. I'm sure you can use it in, do it in pure Python. You can do some pretty funny things in MapLip. Yeah, absolutely. Yeah, yeah. But you definitely need this kind of interactive piece here in order to get this feedback loop, which is really what's fundamentally creating your intuitive understanding, I think.

• 17:45Speaker 00

  This episode is brought to you by discover dot bot. Learn everything there is to know about bots at discover dot bot slash practical AI. Discover that bot was built by Amazon Registry Services as an online community for bot creators and makers of all skill levels to learn from one another, to share stories, and they regularly publish guides and resources to answer questions like how to set up payments to your bot, how to stop shopping cart abandonment, what KPIs are worth measuring, how to write an engaging chat bot dialogue. You can even register dot bot domains there. Learn more and explore this huge library of bot resources at discover dot bot slash practical AI. Again, discover dot bot slash practical AI.

• 18:42Daniel Whitenack?

  So Andreas, you, you mentioned right before, uh, when, when you were talking about the type of tooling that you're using, that, that interactivity is really a key piece of this puzzle in kind of, uh, developing good visualizations of, of complicated things like, like neural networks. I know that that sort of interactivity and kind of, uh, uh, interactive, uh, visual forward, um, sort of idea is behind this distill publication that, that we've been talking about. So could you describe a little bit kind of how you got interested in this distill publication? Maybe what it is, maybe orient people to how it's different from kind of a normal academic type journal article?

• 19:26Speaker 03

  Right. Um, so no academic journals, you know, it's a PDF. Why is it a PDF? It's because we used to print everything. And so I think this still had this interesting thought that.

• 19:42Speaker 03

  We really want to explain and visualize things better. And we actually, what about that we don't take the PDF format, but we use the sort of the interactive capabilities of the web platform in order to visualize and explain on a completely different level than what you can normally do in a in a PDF document. I think that's really like the core philosophy of of this still.

• 20:08Chris Benson?

  So if you're if you're out there doing some work and and that feels like, what does it take to get published on that? How do you get into the process? And and is it, you know, how does it differ from, say, doing academic publishing?

• 20:22Speaker 03

  Well, it a volunteer organization, right? So I think you have to be a patient person. This one took like almost a year from, I will say, eight months from I finished writing it to it getting published. And that part is just a lot of feedback that you get from Ludwig and Chris, who runs Distill. And they do this on a volunteer time.

• 20:52Speaker 03

  So you get amazing feedback from them, like almost on a mental level. But it takes a long time and it can go months where you don't get anything and you have no idea if you're gonna get published or not.

• 21:04Chris Benson?

  Do you, considering how fast the field is moving, does that cause any kind of concern for you? Just going that length of time, considering that somebody else could may may publish something similar to what you're working on in that in that timeframe?

• 21:16Speaker 03

  Not in that field, no. Like visualization, no, not a lot of people do that.

• 21:23Speaker 03

  I know it's something they want to improve. They want to get it down to three months. And I think they're gonna get there with some more help. So, yeah, it's not something I would worry so much about. In terms of how you get into it, like you just go to the website under publication and you send in your thing.

• 21:41Speaker 03

  But, I mean, just know, like, they don't do anything. All these amazing visualizations that you see is something you create yourself.

• 21:49Daniel Whitenack?

  Yeah, so I mean, it sounds like there's, there's a real, maybe there's a need for, for more people doing this if, if, you know, more people doing this and also developing good, good tools around it, you know, since there is a fairly small group of people doing this. I know that you're, so your Distill publication was specifically focused on recurrent neural networks and recurrent units, so LSTMs and, and other things. I was wondering, you know, in, on the podcast, we've talked about a lot of different type architectures, neural network architectures.

• 22:28Daniel Whitenack?

  So maybe you could just give us kind of a, a brief crash course in recurrent neural networks, just a brief description of what those are. Cause I don't think we've actually done that on this, on this show yet. So in, in relation to kind of, uh, maybe what we would consider like a quote unquote, "basic neural network," so like a fully connected neural network where inputs come in and then they're, they're added together and an activation function is, uh, is applied and then they're passed off to a next layer where the, kind of the same thing happens and they eventually get to the other side of the neural network as, as output. How do recurrent neural networks and the, the units that are used in those neural networks,

• 23:12Daniel Whitenack?

  how do they differ, uh, differ, um, from, from the kind of basic, fully connected, uh, situation?

• 23:20Speaker 03

  Yeah, so like you said, like in, in a typical neural network, you have some sort of like fixed input, right? So you have like maybe 10 input elements or maybe a picture of some fixed size and you pass this through your network and you get an output. That doesn't really apply that well to text or audio, for example, because, uh, sentences, they have a different number of characters, a different number of words. So you don't have the, the same number of input values really every time. So instead you do this as a sequence of input vectors and, uh, you just start from like the, the first part of the sequence.

• 24:05Speaker 03

  So it could be the first word and you put this, you describe that with some input, like maybe you have a, a really big dictionary where you just assign one number to every single word. And then you pass this through your network and then you go to the next word in your sequence and you pass this through. But how do you then combine that with the previous word? So you just take every sort of intermediate output that you have in your network. And you concatenate that with every intermediate output in for this next word here. So by concatenating like what you got from from the previous word to with this word here.

• 24:48Speaker 03

  Um, you can really sort of in theory memorize something from the entire sequence.

• 24:55Daniel Whitenack?

  So it's like, it's like, uh, let's say if we're, we have the example of words, one word. Uh, so word one comes in, but then when we process word two, we process word two along with uh, something that was was output the first time. Is that kind of the the basic flow or?

• 25:14Speaker 03

  Yeah, that's the basic flow, right? And and then sort of your intuitive idea might be, okay, as I go through the sequence of characters, I would be likely to forget something from the very beginning. And so this is sort of what's called the the vanishing gradient problem. And that is solved through this thing called LSTM, for example, with uh, sort of trends.

• 25:41Daniel Whitenack?

  And as we've mentioned LSTM several times in the podcast, but for those who aren't familiar, can you just quickly define that is just a a quick one off and then continue your thought?

• 25:51Speaker 03

  Yeah, uh, so LSTM is basically just trying to simulate a memory cell in a computer. So it has like the capacity to memorize something for a long period of time.

• 26:02Speaker 03

  And that's how I would describe it. And similarly, you have the Gru, which is just a different variation of that, but doesn't use, it uses less memory, let's say.

• 26:13Speaker 03

  Like physical memory.

• 26:15Daniel Whitenack?

  Yeah, and so those long, short-term memories, so the LSTM and the gated recurrent units or groups, you talk about those a lot in, I mean, that's kind of part of the focus of your of your publication, right?

• 26:28Speaker 03

  Yeah, because they really solve the same issue, just in different ways. They solve this vanishing gradient issue for getting in long sequences, right?

• 26:38Chris Benson?

  And so just to position it, those are kind of, those are are specific types of recurrent neural networks, right?

• 26:44Speaker 03

  There's specific components in, yeah, in recurrent neural networks, I would say.

• 26:49Chris Benson?

  Yep, fair enough.

• 26:51Daniel Whitenack?

  Yeah, specific units of it. So when you're saying the the vanishing gradient problem, and you also mentioned like memorization in recurrent neural networks, are those, are we talking about the same thing? Are those different?

• 27:07Speaker 03

  Um, well, they're sort of like part of the same coin, as you say.

• 27:13Speaker 03

  Like, if you have a vanishing gradient problem, that certainly means that you cannot memorize. But you can have issues with memorization, even if you theoretically don't have a vanishing gradient problem.

• 27:26Speaker 03

  And so, for example, what we see in this publication I written, right, is basically just autocomplete, like you have on your phone. And so as you type, it sort of catches on from the previous word that you put in and then tries to guess the next word. Or if you're in this word here, you put in a few characters, then it might just use the last few characters instead of the previous words to guess what it is that you want to type.

• 27:59Speaker 03

  And so here we sort of have two different concepts. One is long-term memorization, where we use previous words, and another is short-term memorization, where we just use the previous characters from the same word.

• 28:13Speaker 03

  And what I sort of seen is, and I think this is very specific to sort of your dataset, but at least for this application and dataset, the LSTM shows that it's better at short-term memorization, where the GRO shows it's better at long-term memorization. But if you look into like the theory, there's no reason for that. They both solve the vanishing gradient problem in their own ways.

• 28:37Chris Benson?

  So where did that leave you? You know, it sounds like that's kind of the summary of what you took away from that. And so recognizing that you had the LSTM architecture was great for that short-term, character-by-character approach, whereas the GRU was better at word-by-word. And presumably in an architecture where you do some combinations and stuff, I mean, what learnings did you take away that you would use in things going forward with that?

• 29:05Speaker 03

  Right. I think the wrong learning is to take that, okay, Gru is better at long-term memorization, because I don't think that's the case.

• 29:14Chris Benson?

  Okay.

• 29:15Speaker 03

  But if you just look at, for example, the accuracy or the course entropy, you get almost the same value, right? But we see such a huge difference in how they actually behave. And that is like the LSTM is really good at the short-term local characters, right? So it gets its score from that, and the Gru gets its score from the long-term.

• 29:45Chris Benson?

  So what kind of practical thing? I mean, how would you use that? If you were going to go into your next project?

• 29:52Speaker 03

  Yeah.

• 29:53Chris Benson?

  What kind? How would that influence the next thing that you're going to do in the space?

• 29:58Speaker 03

  Right. So like, let's say I have a customer, right? And they want an autocomplete function or whatever. I mean, it might be worth to ask them, okay, I mean, what is important for you? Is it that it understands the full context? Or is it that it understands the locality? Of course, it would be nice to get the best of both worlds, right? And probably you can do that if you do some more advanced stuff. But it's not always that you have the data in order to make that choice.

• 30:28Chris Benson?

  And maybe, you know, maybe it's like a...

• 30:34Daniel Whitenack?

  You know, I don't want to put words in your mouth, so give me your thought on this. But it seems like recently, like people like Alan AI Institute, or Spacey, or even like OpenAI coming out with GPT-2 and all of these kind of text models. It seems like there is a trend to show visually, like, you know, things like co-reference and like visual examples of how like BERT, for example, is embedding words and like all of these different kind of visual ways. It seems like people are really seeking after kind of visual tools that they can utilize, like on, for example, on their next BERT project where they're utilizing BERT.

• 31:18Daniel Whitenack?

  They want to have a tool in their toolset to kind of visualize how, visualize the embeddings, or like in this case, maybe you're using an RNN model, maybe for that particular dataset, you do try a bunch of different models, but then you kind of have this visualization tool in your, in your toolbox to be able to understand for this particular problem, what are the memorization implications of these different architectures that I'm, that I'm trying? Is that, would that be a good way to kind of utilize, maybe leverage some of the stuff that you've done and say, you know, this is a tool that I can use to visualize these sorts of memory issues for my particular problem?

• 32:00Speaker 03

  Yeah, I think what was really important for me in this publication was to create a tool that is so general that you can really compare many different kind of architectures, like, does it have attention, or is it bidirectional, right? And that was really what was important for me here, because that creates like a really strong scientific tool for comparing different models.

• 32:25Speaker 03

  If you look at, for example, in like in some cases, you just look at an attention mechanism in a, in a network and just, that is a different way of visualizing, or you look at embeddings as you talked about. So this is what I would call internal visualization, where what I've done is what I would call external visualization, because the sort of visualization strategy I have only looks at the output with respect to the input. It doesn't look at anything in, in the middle. And I think there's space for both things.

• 33:00Speaker 03

  Yeah.

• 33:02Speaker 03

  I would think the sort of the the input-output part, the not intermediate part is perhaps mostly for for arguing or explaining your model. And the sort of intermediate part is really for like validating your model that it did come up with something meaningful.

• 33:30Speaker 00

  This episode is brought to you by StrongDM. StrongDM makes it easy for DevOps to enforce the controls InfoSec teams require, manage access to any database server in any environment. And in this segment, we're talking to Jim Mordeko, VP of Engineering at Hearst. He's sharing how they're using StrongDM within their team of 90-plus engineers. It now takes them just 60 seconds to offboard a team member from a data source.

• 33:54Speaker 04

  We have an engineering team of somewhere in the area of 80 or 90 engineers. Because we've got so many services and many databases and so many developers, we need a reasonable way to manage access to them. It was a somewhat painful and, you know, labor-intensive process. Our DevOps team would literally have to manage every one of the permissions for everybody who wanted access. So StrongDM has been a real godsend in that area for us. Requests for access to specific databases were pretty much manual. Now we've adopted StrongDM. It's something that you don't even know is there. Once it's installed, it just works. It's very simple. We've set up a multitude of data sources so that when somebody's on-boarded, we just give them access to StrongDM. It's pretty simple.

• 34:37Speaker 04

  Our DevOps team, they have a very minimal effort required to enable each data source to be connected to StrongDM. And then installing the client software is very, very simple and straightforward. You can use whatever client you want to to talk to the database. So there's really no training necessary.

• 34:52Speaker 00

  All right, if your team can benefit from nearly instant onboarding and offboarding that's fully sought-to compliant, head to StrongDM.com to learn more and request a free demo. Again, StrongDM.com.

• 35:18Daniel Whitenack?

  So Andreas, how did you get interested into diving into unique neural units? I know you've studied some different types of neural units such as sparse max and NALU and things like that. Could you kind of tell us how you got into it?

• 35:34Speaker 03

  I mean, I just got into it through my university. I like sort of these kind of like very fundamental things that where we can actually understand them. Like if you have a 30-layer network, it can be really hard to understand how it works. But if you focus on like a specific unit and try to really understand that, that can give you a lot. And for example, the sparse max, so most of you are probably familiar with the softmax. The sparse max is just a version of that is capable of predicting zero probability, one probability. That can also be a great tool for understanding something because

• 36:16Speaker 03

  you get a much higher contrast in your visualizations.

• 36:21Daniel Whitenack?

  Awesome. So what do you think in light of these kind of fundamental things that you like exploring, what are you wanting to explore next? Do you have any interesting new neural units or anything that you're trying to visualize right now?

• 36:39Speaker 03

  I don't know so much about visualization, but I've done a lot of work on this NALU that is really just trying to do mathematical operations like addition and multiplication and learn that. And they have this gating mechanism in NALU that can either choose addition or multiplication and it doesn't work at all. But I think it's a really interesting concept and I think that's something I would like to look more into, like these fundamental gating mechanisms that we have in LSTM and GRU and also these most specialized things such as NALU. Like what is it really that drives them? What is it really that makes them choose either this or that?

• 37:21Daniel Whitenack?

  Yeah, so could you, could you maybe dig into that a little bit in terms of what you mean by this sort of gating mechanism in terms of maybe, so you were saying this NALO, which is the neural arithmetic logic unit, I think if I have that right. It kind of has a, it has a gate between the, like you were saying, addition or multiplication. What is, what does that gating kind of mean and why, why would it be useful?

• 37:55Speaker 03

  I mean, so the sort of idea is that you have some kind of maybe physical model, but you don't know what the physical model is. And physical models typically are composed of additions and multiplications. And so you would like to learn what is sort of the appropriate order of addition and multiplications that needs to be performed.

• 38:18Daniel Whitenack?

  Ah, gotcha. So it's like in a, it's like in a, in a, the most simple of neural networks that we're thinking about, maybe you have one operation that's repeated over and over in the different, different nodes of the network. Here you're saying, well, I don't know exactly what combination of operations is the best combination, so I'm going to have a mechanism to switch between, between them. Is that right?

• 38:43Speaker 03

  Yeah, I mean, I mean, in theory, you could go crazy, right? You could also have something that could switch between LSTM and GRU, right? It's a bit crazy perhaps, but like a gating mechanism is really just something that can switch either completely or partially between two very different things.

• 39:04Speaker 03

  It's sort of what I done in the last couple of months of my research here is I sort of discovered that in order to do this kind of switch, they need actually to be quite similar. Like in LSTM, the different parts you switch between are very similar. But in NALO, one part is addition and the other part is multiplication, which have completely different behaviors. And that actually makes it a really big challenge.

• 39:29Daniel Whitenack?

  So if you were, if you're listening to this and you're getting interested in you in in unique neural units and you want to kind of dive into it yourself or or maybe even get ambitious and design some of your own or something, do you have any tips for people on how to explore this particular area?

• 39:44Speaker 03

  Um, I think there's always like a few things to consider when designing like a custom neural unit, right? One is the gradients, like you can maybe come up with something, but then the the gradients are zero in many cases, that's not great. And the other part is how are you gonna initialize the weights? And so those are like two quite big challenges and something that probably wants that you should think about when designing these kind of units. Beyond that, I mean, do whatever you like.

• 40:22Daniel Whitenack?

  So what would have would have been useful ways for you to kind of understand kind of some of the best ways to initialize weights and and those sorts of things? Has it been mostly trial and error or is there any sort of kind of, you know, systematic way you can go about exploring the the certain ways to initialize and and best ways to do that? I know that can be a particularly challenging element of this.

• 40:48Speaker 03

  About 10 pages of probability, I would say. Like, that's really the strategy. Like, sometimes you're stuck and you don't really know how to sort of calculate the expectation or the variance of this particular structure. In that case, like, Taylor approximations can be really nice. You can go on Wikipedia and search for like Taylor approximation of moments. So that that can be quite helpful to at least come up with some reasonable initialization. Um, yeah. Beyond, yeah, like just run your model quite a lot of times. See, does this utilization scheme, does it like converge consistently?

• 41:33Daniel Whitenack?

  So I, I kind of have a follow up from that and I, I really, um, really like what you're saying. And I also find it encouraging that, you know, the, you've kind of, um, you've found your way into this and, and are publishing in like really great places like Distill. Um, you know, and you kind of developed this passion while you were doing sort of web development and, and other things. Um, I imagine that there's some listeners out there that are maybe doing like web development things right now and have a really big interest in, in AI, but maybe are somewhat frightened by like Taylor approximation of moments and other things, uh, like this.

• 42:13Daniel Whitenack?

  Um, as, as you know, I think it's encouraging, first of all, that like just to see the, the great things that you're doing and that you came kind of from that background and you have that passion and that's really driven you to, to do these things. Um, I'm just wondering, kind of, uh, is there any encouragement that you can give to, to those sorts of people or resources that have been helpful for you along the way and kind of going from sort of a different sort of engineering, um, and getting interested in AI to, to doing some of this more, uh, more detailed work?

• 42:45Speaker 03

  I mean, so between the web development and now I had like five years of mathematical education, right? And so there's certainly a journey that you have to take. I think if you want encouragement, like, um, you don't need that kind of mathematical education just to do, um, some neural networks. But if you really want to, to get into that kind of like really difficult stuff and publish, it's a long journey and just be patient, I think.

• 43:16Daniel Whitenack?

  Yeah, and I think that you obviously have a passion for these things and that can really, really drive you forward. I think that, yeah, the certain mathematical pieces aren't out of people's grasp, but it does take some work to put the time in. And I know continually as things like new topics come out and all of those things, I have a whole list of things that I need to brush up on and learn more about. And I imagine if you want to get into AI, it's kind of like a lifelong thing of learning about all of these sorts of random things. I

• 43:56Daniel Whitenack?

  don't know if you can sympathize with that, Chris, but that's certainly how I felt.

• 44:02Chris Benson?

  Every time I knock one thing off my list, by the time I look up again, there's three more to jump into. So I think for me at least, the question is trying to figure out how to shortcut as much as I can to get where it is that I want to go in particular.

• 44:18Daniel Whitenack?

  Right. Yeah. So, well, thank you so much for, you know, for taking the time to walk us through some of these things. We're definitely going to post the link to your distill publication in our show notes, so everyone can read the great article there. Any other ways that we can connect people to you on the web or other things you'd like to point to?

• 44:48Speaker 03

  I mean, you can find me on Twitter. I still have my, what is it called, private messages open. So Andreas underscore Masson and I'm also on GitHub as Andreas Masson.

• 45:01Daniel Whitenack?

  Awesome. Well, thank you so much. This has been, this has been really a great conversation. I know I've learned a lot and really appreciate you taking time. Thanks for joining us.

• 45:12Speaker 03

  Yeah, thank you. It was great.

• 45:16Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes and give us a rating. Go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you gotta do. Share the show with a friend if you enjoyed it. And bandwidth for changelog is provided by Fastly. Learn more at fastly.com. And we catch our airs before users do here at changelog because of Rollbar. Check them out at rollbar.com/changelog. And we're hosted on Linode Cloud servers. Head to linode.com/changelog. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in your email address.

• 45:59Speaker 00

  Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.