One algorithm to rule them all?

From MIT researchers who have an AI system that rapidly predicts how two proteins will attach, to Facebook’s first high-performance self-supervised algorithm that works for speech, vision, and text, Daniel and Chris survey the AI landscape for notable milestones in the application of AI in industry and research. Sponsors: Fastly https://fastly.com/ – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipChangelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: The first high-performance self-supervised algorithm that works for speech, vision, and text https://ai.facebook.com/blog/the-first-high-performance-self-supervised-algorithm-that-works-for-speech-vision-and-textDeepMind says its new AI coding engine is as good as an average human programmer https://www.theverge.com/2022/2/2/22914085/alphacode-ai-coding-program-automatic-deepmind-codeforceArtificial intelligence system rapidly predicts how two proteins will attach https://news.mit.edu/2022/ai-predicts-protein-docking-0201Seeing Theory https://seeing-theory.brown.eduBooks “The Art of Doing Science and Engineering” by Richard W. Hamming https://www.amazon.com/dp/1732265178“Patterns, Predictions, and Actions” by Moritz Hardt and Benjamin Recht https://www.amazon.com/dp/069123373XUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  With current technology, the way it is more or less right now in 2022, you could have thousands, tens of thousands, hundreds of thousands of tasks that are trained in a very narrow task-oriented model. So could you have fully automated brain surgery? Maybe, but that's what I think about a lot more in terms of the AI ethical concerns going into the future is all of the models and handing our life over to all the models rather than one super knowledgeable model that rules them all, which is a fiction.

• 0:36Speaker 01

  Big thanks to our partners, Leno, Fastly, and Launch Darkly. We love Leno. They keep it fast and simple. Check them out at leno.com/changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by Launch Darkly. Get a demo at launchdarkly.com.

• 0:53Speaker 02

  Hey, Jared here. One of the things we can count on in the software industry is change. The state of the art changes so fast, in fact, that keeping up can feel like a whole other job on top of your actual job. That's why we created Changelog Weekly. It's our totally free newsletter that we drop in your inbox each and every Sunday. We link to the latest news, the best articles, and the most interesting projects that you should be aware of. We also add a little commentary from us saying why something's important, pointing you to other instances of a trend, or just making a dorky joke to keep it lively. So if you haven't yet, I recommend subscribing to Changelog Weekly and help us help you keep up with the latest.

• 1:36Speaker 02

  Head to changelog.com/weekly and sign up today. Again, it's totally free and we never spam you. Yuck. One last time, that's changelog.com/weekly.

• 1:59Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at change.com/community and follow us on Twitter. We are at Practical AI.

• 2:25Daniel Whitenack?

  Well, welcome to another fully connected episode of Practical AI. In these episodes, Chris and I keep you fully connected with everything that's happening in the AI community. I'll take some time to discuss the latest AI news and dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitenack. I'm a data scientist with SIL International, and I'm joined, as always, by my co-host Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

• 2:56Chris Benson?

  I'm doing very well, Daniel. How are you doing? I know you had been out with COVID like almost everybody else on the planet. Are you better?

• 3:03Daniel Whitenack?

  Yes, much better. Yeah, I mean, for listeners that are maybe just jumping in at this episode, I've been out for a couple of recordings, had the COVID. Overall, though, I count myself very blessed because I didn't have the severe symptoms that some people had. Definitely had some, but not near as bad, of course, is what a lot of people are dealing with. So I count myself blessed in that regard. And yeah, it's good to be back in action.

• 3:32Chris Benson?

  Welcome back, buddy.

• 3:33Daniel Whitenack?

  Yeah, good to be back. It was awesome to have Natalie Pastunovic fill in in a couple of recordings for me.

• 3:39Chris Benson?

  She did fantastic job, very good.

• 3:42Daniel Whitenack?

  Yeah, she's awesome. Love her insights on everything. And you know, it's kind of fun to be able to listen to my favorite AI podcast without my voice on it.

• 3:55Daniel Whitenack?

  You know, I don't know if that's if that's okay for me to call this my favorite AI podcast.

• 4:00Chris Benson?

  It is. It has to be. I'm gonna be worried if it isn't. Okay.

• 4:05Daniel Whitenack?

  Yeah. So, yeah, I mean, it's cool to, I do listen to some of our episodes where I am on it. And, you know, obviously, I'm part of the conversation, so I remember what's going on. But I gain a lot out of these podcast episodes. And actually, I just had a conversation with someone there asking me, with doing the podcast and kind of interacting with all sorts of AI people across industry, how has it shifted some of how you think about AI or the industry or something? I thought that was a really interesting question. I don't know how you'd answer that, Chris. I think I said something, you know, like,

• 4:46Daniel Whitenack?

  it's encouraging to learn from so many people that have really deep knowledge in different areas. But it sort of also reinforces that idea that everyone, even though they might seem like an expert in all areas, then there are some people like that. But most people have a sort of very deep knowledge in a certain area.

• 5:07Chris Benson?

  Yes.

• 5:08Daniel Whitenack?

  So that's kind of what breeds this kind of imposter syndrome thing. And so to be able to, I think, over time, talk with so many kind of luminaries and experts in different areas, it's cool for one to learn from them, but also to just talk to them and feel like they sort of accept you and are excited to talk to you about what they're doing and don't view, you know, because we're not experts in those areas. But a lot of the people we have on are very gracious. And so, yeah, maybe that's my insight is that there's just a lot of very gracious people in the industry. I'm sure that there's many that aren't, but being able to learn from them and... Yeah, that's kind of what came to mind. I don't know if anything comes to mind on your own.

• 5:53Chris Benson?

  I think that's well said. I wasn't expecting you to go here, but you know, us talking about kind of that. We've been doing this for about three years, getting close to three years now. And having done that on a more or less weekly basis over that time, we've gotten to talk to so many really interesting people. They come on. We haven't. Maybe we've just lucked out and we've only found the gracious ones, but they're so human and it's that human touch around the technology that makes it special. And so that's really, and then the other thing that I would notice, and I think it would be the same whether you're a listener or whether you're the host like us, is that you're privy to this conversation and it kind of keeps you out of some very narrow

• 6:34Chris Benson?

  bubbles. You know, we're still kind of in the AI bubble as an industry, but I know that, you know, if I go to my employer, there's a particular way we think at any given employer and it gives you perspective and you hear really smart, amazing people saying things that you can learn from. I think that enriches us all.

• 6:52Daniel Whitenack?

  I've seen this with other podcasts as well and other just like, I mean, if you read books, this is a, this is something that happens to where you sort of, you don't remember everything, but you sort of have this low level remembrance of certain ideas and technologies and products or whatever it is. And so when you're in conversations in your job and someone brings up like, oh, well, we need to like label things in this way, or oh, we're having this problem. I'm not sure like how, then it's sort of like, all of a sudden something comes out of that like soup of, of things you've, you've sort of stored away and is really useful. So I, yeah, I found that as a, as a really interesting benefit.

• 7:36Chris Benson?

  Yeah, it definitely expands the mind. I know as my own day job, not in the podcast, has evolved over time and I've moved from very focused on the deep learning stuff to I'm also now looking forward a lot and you know, kind of the what's next, it may not be here yet and all that.

• 7:52Daniel Whitenack?

  And ethics things.

• 7:54Chris Benson?

  Ethics things, which are huge. You know, these are big, big parts of not only the industry, but it affects everybody now. So it's a very cool time. To be alive and participating in things. I know a lot of people focus on all the negatives in the world, and there are, there are many. We are in the middle, maybe approaching the end, we'll have to see, of a global pandemic. We've had all sorts of crazy politics in recent years, lots of, of challenges, but I still, my, my daughter was asking me the other day, if you, what historical period would you choose to be a part of? And I said, this is it. You know, we're living it. This is a cool moment. And you know, someday, this is going to evolve. We'll have, we'll have things like quantum AI eventually. I don't think we're there yet.

• 8:38Daniel Whitenack?

  Yeah.

• 8:38Chris Benson?

  But it will happen someday. So yeah, pretty cool.

• 8:41Daniel Whitenack?

  Yeah, yeah. Well, I mean, I guess on that note, with the sort of theme of, of health things and trending news and all that, I know, just to get into some of the AI news things that happened in recent times, both you and I kind of sent each other the same link, I know.

• 9:01Chris Benson?

  Yeah, we did.

• 9:02Daniel Whitenack?

  Which was this work about proteins attaching.

• 9:07Chris Benson?

  Yes.

• 9:07Daniel Whitenack?

  Which people might, might not know why that's, yeah, we found two different references to the same thing.

• 9:14Chris Benson?

  It was the same thing written by two different online things, but clearly it was something that mattered to both of us. I didn't mean to cut you off. Go ahead.

• 9:21Daniel Whitenack?

  No, no. I was just thinking, like, people might be wondering why, like, proteins attaching, it seems like sort of a random, random thing, especially for like, myself, I am not, I don't have a lot of biology knowledge, or like, that sort of, I'm not familiar with that side of, of things. But the idea here is that, you know, there are things like viruses, which we're familiar with, like the virus that causes COVID-19. It's like, something that it attacks, but you can attach things to it, right?

• 9:58Chris Benson?

  Yes.

• 9:58Daniel Whitenack?

  Like the, these virus spikes. So there's like antibodies that can bind with the virus spike proteins to prevent the virus from entering a human cell. So it's almost like, one way to fight this sort of virus, and I'm speaking way above, beyond what I know about biology. But, you know, I think we've all learned a bit about viruses and, and vaccines in recent years.

• 10:23Chris Benson?

  Yeah.

• 10:24Daniel Whitenack?

  But yeah, one of the ways to sort of think about this vaccine or fighting a virus is attaching things to it, to the virus.

• 10:33Daniel Whitenack?

  Like proteins to prevent it from entering our cells. So that's kind of the context of this.

• 10:39Chris Benson?

  And this article, it uses COVID kind of as the example of why it's important because it's such a relevant thing right now. Antibodies are proteins themselves. And so this ability for these complex molecules, in this case, proteins, to be able to to do attachment and to figure out what it's going to be, it's a very difficult thing. AI, as it's done in so many other kind of pattern matching use cases, is doing an amazing job. But I think the thing for me, this one struck a personal note, which I'm going to share because lately, on several occasions, and as recently as this morning, as we record this, I was talking to a friend who has a PhD from Harvard University in chemistry.

• 11:22Chris Benson?

  This person really, really knows chemistry well. And we were talking more or less about this topic, not in terms of COVID, but we were talking in terms of the fact that I'm vegan, he's not, and he loves teasing me about all the all the challenges vegan have, vegans have getting nutrition. And I said, well, you know what? You need to help me with your chemistry PhD, find a way for me as a vegan to get all the things that I need. And we got into a, I am not a chemist. I know just enough to have a conversation and we got into a conversation about that. And then I saw this article and I was like, okay, we, we really want to talk about that today, or at least I do. And I was glad you did.

• 12:03Daniel Whitenack?

  So yeah, maybe a side outcome of this work is, is nutrients for vegan.

• 12:09Chris Benson?

  There's so many things in medicine and nutrition and stuff that this could help.

• 12:13Daniel Whitenack?

  Yeah, it's very interesting that how computers doing this sort of task are working with expert, you know, biologists and chemists and drug developers to produce things much, much quicker than they could otherwise, which brings up like all sorts of like, I think it is a really good example of like humans and machines working together very closely with, with a lot of benefit. Cause no one would want like your machine to sort of like combine a bunch of things together virtually and say like, here's the vaccine and like. You know, then they just go out and produce it and that's that like there has to be a

• 12:58Daniel Whitenack?

  human element in this process still. And my understanding is the sort of machine learning model is doing a lot of this brute force stuff that was very time consuming before with previous software products. So it kind of directly predicts complexes that will form when two proteins bind together.

• 13:18Chris Benson?

  Yep.

• 13:19Daniel Whitenack?

  And so it's talking about it doing it, you know, 80 to 500 times faster and potentially more, more accurately. So you just sort of think about like, this is probably, and you know, again, I'm speaking outside my expertise here, but this is probably like in past time stages of the process that were really like the humans didn't even enjoy this bit, I would imagine, like, just like searching through structures and kind of combining.

• 13:48Chris Benson?

  Oh, God, you haven't met my friend.

• 13:50Daniel Whitenack?

  And yeah, I don't know. Like, it seems like the, like, yeah, manual grid search sort of like thing.

• 13:58Chris Benson?

  As an analogy, you know how you always talk about how you love to clean data, you know, all the things that the rest of us complain about and, you know, grumble about, and you love that. There are people that love to do this.

• 14:09Daniel Whitenack?

  Love just like looking at structures.

• 14:11Chris Benson?

  Yeah. And there's a lot of variability and, you know, there can be so many combinations. I'm not going, you know, I'm assuming millions. I'm not going to speculate. One of the co-authors, and I'm going to butcher his name, Octavian Eugen Gania, maybe, I hope. And if I have mispronounced your name, sir, I sincerely apologize on that. But he made the comment, "Deep learning is very good at capturing interactions between different proteins that are otherwise difficult for chemists or biologists to write experimentally. And some of those interactions are very complicated, and people have not found good ways to express them. So the deep learning model can learn these types of interactions from the data." So kind of a classic deep learning task, just not, you know, one that those of us not in chemistry might not have thought about previously.

• 15:16Speaker 04

  I'm Karhal Azou, host of Shipit, a show with weekly episodes about getting your best ideas into the world and seeing what happens. We talk about code, ops, infrastructure, and the people that make it happen like charity majors from Honeycomb.

• 15:30Speaker 05

  We act like great engineers make great teams. It's exactly the opposite, in fact. It is great teams that make great engineers.

• 15:39Speaker 04

  And they farly were the founders of continuous delivery.

• 15:43Speaker 06

  Start off assuming that we're wrong rather than assuming that we're right. Test our ideas, try and falsify our ideas. Those are better ways of doing work. And it doesn't really matter what work it is that you're doing. That stuff just works better.

• 15:55Speaker 04

  We even experiment on our own open source podcasting platform so that you can see how we implement specific tools and services within changelog.com, what works and what fails.

• 16:07Speaker 02

  It's like there's a brand new hammer and we grab hold of it and everyone gathers around, we put our hand out and we, we strike it right on our thumb. And then everybody knows that hammer really hurts when you strike on your thumb. I'm glad those guys did it. I've learned something instead. Yeah.

• 16:22Speaker 04

  I think that's a very interesting perspective, but I don't see it that way.

• 16:27Speaker 02

  Okay.

• 16:27Speaker 04

  It's an amazing analogy, but I'm not sure that applies here. Listen to an episode that seems interesting or helpful. And if you like it, subscribe today. We'd love to have you with us.

• 16:52Daniel Whitenack?

  I also noticed, Chris, that some of what was done with this protein attaching model thing.

• 17:02Daniel Whitenack?

  Had to do with sort of very interesting data structures and how they're kind of processed into the neural network, where here we have sort of 3D structures, these 3D protein structures. And from what I'm understanding here, they sort of convert those 3D protein structures into 3D graphs, which are processed by the neural network. And of course, that also fits a theme that we've been seeing in really the past sort of year and a half or more, where people are really exploring graph-structured data and putting in graph-structured data into neural networks in very interesting ways.

• 17:44Daniel Whitenack?

  So like a sort of different type of data was used here, which is pretty interesting. That, I guess, kind of leads me to one other thing that I wanted to bring up, which is related to the types of data that are, that we process with neural networks, which is this data-to-vec project from Meta AI. Okay. So people might have heard of sort of word-to-vec, or maybe certain types of image-to-vec things, or image feature extraction, or word embedding, or, you know, feature representation type of things. All of these sorts of things fit into kind of some, some type of category where you

• 18:26Daniel Whitenack?

  have a, you have a neural network model, and the sort of pre-training of that model helps you pre-train vectors, or embeddings, or representations of some type of data that you put in, that then can be maybe fine-tuned later on for downstream tasks. So this has been like part of the driver for a lot of the innovations in NLP recently, but also in other fields like image processing. A lot of sort of large image processing neural networks have, most of the complexity of the network is involved in the sort of feature extraction phase, which kind of takes an image and converts it to a vector representation.

• 19:09Daniel Whitenack?

  What's interesting, I think, about this work is that, for the most part, all of these techniques that people have been doing By and large are a single modality. So they take, you know, text input and convert it into some vector representation, right? Or they take image input and convert it into some vector representation, or speech input into some vector representation, or a spectrogram or something. And I think this is really interesting because they're taking the perspective of, well, whatever data we put in, and I guess in this case, whatever is either image, image, speech, or text.

• 19:48Chris Benson?

  Okay.

• 19:48Daniel Whitenack?

  But whatever data we put in, we're going to create a model that that is trained to represent that in a learn vector space. So that's sort of hints the name data to vec. So moving from thinking about a sort of single modality of data input to like a model that can take like one, or two, or three, or a combination of kind of data inputs and represent that in a vector space. So that's why I kind of thought this was pretty interesting.

• 20:20Chris Benson?

  It does sound interesting. Did you get any sense of why the modality was no longer kind of a critical part of the pattern? I mean, or any sense of how they've overcome that approach in this?

• 20:33Daniel Whitenack?

  Yeah. So, I mean, they talk a little bit about, you know, and there's a paper, we'll link in our show notes if people want the specifics, but essentially they have a sort of non-learned representation of each of these types of data that they can input to the, to a similar series of layers. And what they do is they sort of use this trick of masking, which is used kind of both in image processing, speech processing, and text processing as a sort of way to, in a self-supervised way, learn the vector representation. So if you think about an image, you can mask off a piece of that image and try to fill

• 21:18Daniel Whitenack?

  in the missing pieces. Or if you have speech, you can mask off a bit of that speech and try to fill in the missing pieces. Or text, you can mask off certain words and try to... Fill them in and that kind of principle of that masking is applicable across these different modalities in a way that I guess is similar enough to where they can use a single set of models to in a self-supervised way, learn the representations and they use this also student teacher method that sometimes we've seen in in recent years as well.

• 21:52Chris Benson?

  Very cool. I noticed that that was done by Facebook's AI team.

• 21:56Daniel Whitenack?

  Yeah, or Meta or whatever.

• 21:58Chris Benson?

  Yeah, Meta now, I guess.

• 22:00Daniel Whitenack?

  We're in the metaverse now.

• 22:01Chris Benson?

  Did they have any mention of use cases for it or was that not addressed? Was it just still too much in the research area?

• 22:07Daniel Whitenack?

  Yeah, I think they provide some motivation in that like there is an overhead and big differences in the way that algorithms are put together for images and speech and text and other modalities. And so bringing them together provides both that flexibility and and more simplicity. But I think there's also this element of something about working with multiple modalities of of data, at least in certain cases, can provide you a boost in performance across all modalities. They show certain benchmarks where they beat other

• 22:49Daniel Whitenack?

  models or other image to vector, word to vector sort of models in the different modalities. So like on ImageNet and at least similar or or better performance than sort of models like Roberta, which is a single modality. So you get sort of similar performance, similar or better performance in a single modality, but it's flexible to use it across different modalities, which allows you to have a sort of one model that operates in multiple spaces, which is kind of cool.

• 23:23Chris Benson?

  That is pretty cool. One of the articles that I came across, I've been, actually several articles are in the general area, is I've been very interested lately in AI and its interface with robotics. And once upon a time at a previous employer, I was kind of doing that exclusively for a while. And so I ran across this article, it was in Science Daily, called Robot Performs First Laperscopic Surgery Without Human Help. And it's actually referencing a paper by Johns Hopkins University that was very recent, January 26, which had a similarly named, "A robot has performed laparoscopic," I'm probably butchering the name, "surgery on soft tissue of a pig without the guiding hand of a human, a significant

• 24:06Chris Benson?

  step in robotics toward fully automated surgery on humans." That's pretty cool, because this is when you, you know, when you think of what's called keyhole surgery, where they make a very small incision and it goes in, the laparoscopic surgery is on the gut. So when you think of all of the various ailments that humans, and for that matter, animals, you know, get, all mammals, you know, in their gut, anything from cancer to various digestive problems. And, you know, we've come far enough in such a short amount of time, so that they can put the robot to work, and it makes the cut, it goes in, it does the work it has to do, mends, sews back together, things like that,

• 24:45Chris Benson?

  and pulls out. And we've been talking in recent years about human surgeons doing these surgeries with the assistance of robots and the assistance of AI. But now we're, we're looking at this, making this next jump, where you're taking that human surgeon out of the loop, at least in this test case, and the AI combined with the robot are successfully doing it on their own. What do you think of that?

• 25:12Daniel Whitenack?

  It's interesting, and it sort of gives, I think, people mixed feelings, probably.

• 25:18Chris Benson?

  Probably. We did mention AI ethics.

• 25:21Daniel Whitenack?

  Yeah, I think you're right in that the sort of next question comes, well, is it always better to have a human with the machine?

• 25:30Chris Benson?

  Yes.

• 25:30Daniel Whitenack?

  That's one interesting question I would have, is sort of, the automated robotic

• 25:36Daniel Whitenack?

  Procedure, it would not surprise me if that was sort of more or had a higher performance than a human alone. What would be interesting to know, and maybe they do mention this in the paper that I don't see at a first reading, but if the human plus machine sort of combination, if there's a way for that to outperform either the robot or the human itself, that's an interesting question.

• 26:01Chris Benson?

  I think that's an evolving question over time. I know in the industry I'm in doing kind of defense stuff, the term for that is mum tea, which is manned unmanned teaming.

• 26:12Daniel Whitenack?

  Right.

• 26:12Chris Benson?

  Where you have an automation, which may be an AI, may include robotics, may not, working with a human to achieve some sort of task. And the value proposition in a, in a broad sense on that is exactly what you said, that by combining the two, you're doing better. I think we're entering a period now where that is called into question and may change for various tasks over time as we see the fully automated version able to outperform. And this, this goes back to another example that I've, I have talked about in earlier episodes. There was a DARPA program, been, I don't know, a year and a half, probably, maybe two years. I think it was before COVID fully set in called Alpha Dogfight. And in

• 26:53Chris Benson?

  that they had, it was publicly broadcast and they, on YouTube, and they had a fully automated F-16 in a simulator, is using an F-16 simulator, the fully automated pilots completely outperformed the human ones, completely outperformed. And now that's in a simulation world and there, and there's all sorts of caveats you can apply, but it surprised everybody just how well the AI did in those contexts. So here we are in a different area in medicine, and now we're seeing the first ones coming about and they'll start to evaluate performance over many of these tasks. It's a really interesting time that we live in as we see the relationship of whether the human's in the loop or not in some capacity is evaluated.

• 27:39Daniel Whitenack?

  Yeah, and I mean, this is obvious, I think, to practitioners, but it's something that I that I really noticed in talking with like just friends or people that aren't practitioners. It's not obvious to them. And that's the like, when you say the sort of AI quote is doing the surgery, there's this sort of perception that like, like the AI could like go off and do something so it could decide to do a different surgery than you told it to do.

• 28:11Chris Benson?

  That's a great point.

• 28:12Daniel Whitenack?

  The sort of idea of control and like the sort of distribution of inputs and outputs is not something that's obvious to like general people out there. And I don't know how to better communicate that as practitioners, but like the things that we're doing are for the most part, very task specific.

• 28:33Chris Benson?

  Very narrow.

• 28:34Daniel Whitenack?

  Very narrow. So like this process is not like a robot that can do all surgeries.

• 28:40Chris Benson?

  Yeah.

• 28:41Daniel Whitenack?

  I'm guessing that it's a single sort of like operation and it's very specialized. It's not gonna like, you thought it was going to like, you know, remove a tumor and instead it like took out your appendix or like something.

• 28:57Chris Benson?

  Since we're talking kind of along the AI ethical thing, I think that raises an interesting point, which I don't think non-practitioners generally understand. And it's extending what you're saying right there. And that is the fact that because these models address very narrow tasking, it's not intelligence, I think, the way most of us define it. It's it's mathematically, you know, learning a set of patterns and going through. That's a little bit loose in my language, I know, but being able to go and do that. So I think when people focus on the intelligent side, they are missing the boat on that. I think it's important that they start to understand. I think what's a lot more likely to happen

• 29:41Chris Benson?

  is that with current technology, the way it is more or less right now in 2022, you could have thousands. Tens of thousands, hundreds of thousands of tasks that are trained in a very narrow task oriented model. And all of those tasks are out there. So could you have fully automated brain surgery? Maybe at some point here in the future, fully automated heart surgery, maybe so. And we're just talking about surgeries right there. But that's what I think about a lot more in terms of the AI ethical concerns going into the future is all of the models and handing our life over to all the models rather than one super knowledgeable model that rules them all, which is a fiction.

• 30:24Daniel Whitenack?

  It's interesting, Chris, on that last point you made, you kind of made this comment about what we perceive as intelligence or what the sort of general population perceives as intelligence. I was reading a book by Hamming called Learning to Learn. I think it's the art of doing science and engineering. Really interesting and in certain cases humorous book. But yeah, he talks a little bit about, I think he has a few chapters on AI and limits of AI. He brings up the point that sort of we keep kind of moving the goal marker of intelligence. So like, you know, 15 years ago or something, we said,

• 31:10Daniel Whitenack?

  well, if a robot could perform a surgery, would we consider that artificial intelligence? Or would we consider that intelligence? Likely you'd get a different distribution of answers than after the event has happened.

• 31:24Chris Benson?

  Yes.

• 31:25Daniel Whitenack?

  Like after the event has happened, it's very easy for us to look at that event and say, well, it wasn't really intelligence because X, Y, Z, right? So then we move the goal marker, you know, however far. And I'm not saying it is or isn't sort of intelligence. Cause that, I mean, you can define that in so many different ways.

• 31:44Chris Benson?

  That's where I was gonna go next. Keep going.

• 31:47Daniel Whitenack?

  Yeah, I think what's weird is like somehow there's the mix of using the terminology, artificial intelligence to describe what's going on here, which triggers the wrong thing. And people's mind in terms of how they define intelligence. And so, yeah, I think it's just a lot of confusion, which probably feeds into that sort of narrow versus wide perception that you brought up earlier.

• 32:13Chris Benson?

  It's very imprecise, the phrase. So, except for the name of our podcast, which is perfect. The term artificial intelligence is otherwise a very, very nebulous and imprecise way of describing it. And as we see, as you just pointed out, it's changing constantly. And I think, you know, as practitioners, we're not spending our time thinking AI in this big nebula. We're looking today at specific deep learning algorithms and approaches to training and implementation. And it's very narrow. And I don't think most of us spend much time thinking about it the same way that untrained people in the general population probably perceive it as. And so,

• 32:54Chris Benson?

  it is very practical. It is very pragmatic what we're doing. But I think a lot of people miss that. And we've just demonstrated, you know, going to how to define intelligence, that now that the surgery is done, neither one of us, we're both in agreement that that isn't what we think of as intelligence. And I think that's because people conflate several ideas together, which is consciousness, intelligence, self-awareness.

• 33:18Daniel Whitenack?

  Yeah, robustness, generalization.

• 33:22Chris Benson?

  Yeah, absolutely. So I think that in this maybe narrow form of intelligence that we're describing, I think we'll see a lot of that in years to come. But I still think that I'm looking forward to the day when the general population kind of understands it, maybe along the same lines that we do, who address these things on a daily basis.

• 33:41Daniel Whitenack?

  Yeah. And I guess on that note, Chris, did you see that now AI coders are as good as the average human coder? Did you see that article?

• 33:52Chris Benson?

  I did not see that article, but I am not at all surprised about that.

• 33:57Daniel Whitenack?

  So DeepMind, well, the article that ran across my feed is DeepMind says its new AI coding engine is as good as an average human programmer. And so DeepMind, so if you remember, we've talked about OpenAI's Codex in the past, right? Which is this sort of code, it's like a language model for code and can do sort of like code. It's like your AI pair programmer is how it's being used within like the GitHub Copilot product, which could be integrated into VS Code and such. Well, DeepMind has this AI system called Alpha Code.

• 34:40Chris Benson?

  Everything is alpha. Have you noticed that?

• 34:42Daniel Whitenack?

  Yeah, yeah. I mean, I can't wait for Beta Code. That's gonna be, that's gonna be pretty killer.

• 34:47Chris Benson?

  Yeah.

• 34:48Daniel Whitenack?

  But yeah, the quote is sort of, it writes computer programs at a competitive level. And this was done with data from a set of challenges that were curated as a, as competitive coding challenges on a platform in which there is competitive coding that, that happens. This Codeforces competitive coding platform, which one kind of side note, I really do not want to get into competitive coding.

• 35:21Chris Benson?

  I don't either.

• 35:21Daniel Whitenack?

  Because I really think I would be really bad at it.

• 35:24Chris Benson?

  I would too. I love to code, and I know you do too, but I'm too slow. Maybe it's a function of age, but I, I think it would be, it would take the fun out of it at the stress level on that. I'd rather just enjoy my coding.

• 35:36Daniel Whitenack?

  Yeah, it's like those people that are like competitive professional video game players.

• 35:40Chris Benson?

  Yeah.

• 35:41Daniel Whitenack?

  Like esports. And my understanding is that many of them like no longer actually enjoy like the game that they're

• 35:49Chris Benson?

  They're too busy competing.

• 35:50Daniel Whitenack?

  Competitive and because it's competitive, right? It's like a, anyway, that's like a total rabbit trail. But despite that, there's this sort of set of tasks and they ran Alpha Code through its paces on these competitive coding tasks.

• 36:05Daniel Whitenack?

  And determined that I don't think they're saying that it is vastly better than the, you know, human coders, but I think they're saying like it's as good. Like that's kind of their conclusion as good in some cases better. And you can go and look at the example challenges that they have. And they're definitely not, you know, I would not consider them easy challenges.

• 36:32Chris Benson?

  No, I was looking at those on this article that you've asked me. They are not, you know, it really, as we record this in early 2022, you raise that is it is good, but it probably will be going forward and surpass. And it really, it really draws out the fact that if you can take a task and break it down into kind of a repetitive pattern, there's a pretty good chance that there is a narrow deep learning algorithm, which can learn it eventually better than a human expert can. It's an interesting world that we're moving into, Daniel Whiteknight.

• 37:09Daniel Whitenack?

  Yeah, yeah, for sure. And it's sort of that line I view it as like people have known this for a long time, right? That there are repetitive coding tasks, which is why code generation.

• 37:20Chris Benson?

  Of course.

• 37:21Daniel Whitenack?

  Which is kind of that rule based approach to generating code for XYZ thing. So like you have, you know, generating a client for based on some proto buff or something like that, or you're generating like the some backend code for your web application based on some config that you've created or, you know, something like that, that all of these problems, they're like very repeatable and can follow patterns. And this is sort of like, there's probably, I guess, in my view, if you look at all code that's written on like the one side, one third of it is probably almost repetitive enough to just be like rule based generation.

• 38:06Chris Benson?

  Yeah.

• 38:06Daniel Whitenack?

  There's like a middle third of it, which is. Kind of like, it's not quite like rule-based generation. Like, it goes beyond that. There's levels of decision-making and such, but maybe it could be attacked by this sort of algorithm. And then there's probably still a third. It's like very hard code that is solving very unique problems that aren't represented well in other spheres or, you know.

• 38:32Chris Benson?

  Tiny fraction of all code.

• 38:34Daniel Whitenack?

  Yeah.

• 38:34Chris Benson?

  A tiny fraction in that last category.

• 38:37Daniel Whitenack?

  You think I am over-generous in my thirds?

• 38:41Chris Benson?

  I do. I think that you're way over-generous because, you know, aside from this AI that we love, you and I both came from programming before that, and we were, we were doing a lot of programming in various languages. And most of the programming across many projects, even across time.

• 38:59Daniel Whitenack?

  Copy, paste?

• 39:00Chris Benson?

  It really is. It's copy, paste, or it's basically doing the same thing in a different language that you did before. Maybe a little bit of a different architectural paradigm. But I'm going to postulate that 95 to 99% of all the code out there has a lot of commonality across projects. And there's a little fraction that's multiple. And if this is something that we think of, if you go back just a few years before we were doing this podcast, and we programmers were thinking, wow, I'm out there making amazing things and all that. This is the challenge of automation. This is, I think, the question of our time. Certainly one of the top questions, if not the only one. It's, it's one of several that we need to contend with, is that automation doesn't have

• 39:44Chris Benson?

  to be all-knowing, all-intelligent, super, you know, the, the thing that people think of in the Hollywood movies to be able to do this.

• 39:52Daniel Whitenack?

  And now we come to the great battle of our time.

• 39:55Chris Benson?

  Indeed, indeed. This is the AI version of Lord of the Rings right here. You're in it, folks.

• 40:00Daniel Whitenack?

  Yeah. As soon as we can generate a next reasonable script for the Lord of the Rings, then

• 40:07Chris Benson?

  Oh, boy.

• 40:08Daniel Whitenack?

  we can probably do a pretty good job right now, but still, I think, lacking. But, uh, yeah.

• 40:13Chris Benson?

  I agree. Narrow AI still can't quite match J. R. R. Tolkien, if I'm saying the name right.

• 40:19Daniel Whitenack?

  Yeah.

• 40:19Chris Benson?

  Yeah. That's a tall order. I agree. Okay.

• 40:22Daniel Whitenack?

  So maybe that's my definition of intelligence there. I'll say that. Yeah. No, I'm just kidding. So as we wrap up in these fully connected episodes, we do always like to share a couple of learning resources. Chris, I noted a couple over the past week, so I'll share those. One that I found, which I'm not totally sure if I've shared before, but I thought it was really cool is there's this seeingtheory.brown.edu site, which is a visual introduction to probability and statistics. And it probably like, some people don't learn this way, right? Some people don't learn visually, but I love to have sort of a visual component

• 41:07Daniel Whitenack?

  that I can have in mind for a concept. And especially for, I think, statistics and probability, a lot of people view that like they have trouble with the intuition around certain concepts within probability and statistics, maybe more so than like calculus or algebra or linear algebra sorts of things. And so this goes through and actually kind of walks you through various ideas like expectation, values and other things. And you can like see a visualization paired with that. And it's a very cool sort of way of approaching probability and statistics and an intro to certain ideas. So I would definitely recommend going through that.

• 41:48Chris Benson?

  It's a very nice website. It's elegantly done, pretty. And I like the visuals in it. It's easy on the eyes.

• 41:54Daniel Whitenack?

  Yeah, yeah, for sure. The other one that I'll just mention, people might want to take a look at is there's this book, which is posted online, which I came across on Twitter, Patterns, Predictions and Actions: A Story About Machine Learning. And I thought that was intriguing, so I clicked on it. I think it's intriguing that they sort of, they sort of frame it as a story. But if you look at Kind of the things that are covered, they might cover them in a sort of different narrative approach, but and maybe from a from their own perspective, but it is a very seemingly comprehensive introduction to various topics, going through supervised

• 42:39Daniel Whitenack?

  learning, representations, optimization, deep learning, causality and causal inference, really a lot of relevant topics. And they kind of start, I think, from the idea of fundamentals of prediction and the classification problem. And that's kind of where they're starting and they they lead from there in their story to these other areas that people are very interested in now. So it's a cool approach from Maurice Hart and Benjamin Recht, A Story About Machine Learning Patterns, Predictions and Actions. I'll check it out. We'll share those links along with the other links that we talked about in our

• 43:19Daniel Whitenack?

  show notes. So definitely people check those out and let us know your thoughts on these things and the various rabbit holes that we went through. As a reminder, you can connect with us directly on our Slack channel or LinkedIn. If you go to changelog.com/community, you can join our Slack channel there and connect with us on Twitter or LinkedIn. We'd love to hear about what you're interested in. But yeah, good to good to be on with you again, Chris. Another good conversation. Thank you so much. Uh huh. I'll talk to you soon. Bye. Bye.

• 43:53Speaker 02

  All right. That's Practical AI for this week. Thanks for listening. If this is your first time with us, subscribe now at practicalai.fm or simply search for Practical AI in your favorite podcast app. We're in there. And if you're a longtime listener, do us a solid by recommending the show to a friend. Word of mouth is still the number one way people find new podcasts they love. Special thanks to our partners for supporting our work, Fastly, Launch Darkly, and Linode. We appreciate it. And to the mysterious Breakmaster Cylinder for cranking out new beats for us all the time. That's all for now. We'll talk to you again next week.

• 44:53Speaker 07

  Game on!