AI in healthcare, synthesizing dance moves, hardware acceleration

Chris and Daniel discuss new advances in AI research (including a creepy dancing video), how AI is creating opportunity for new chip startups, and uses of deep learning in healthcare. They also share some great learning resources, including one of Chris’s favorite online courses. Sponsors: DigitalOcean https://do.co/changelog – Enjoy CPU optimized droplets with dedicated hyper-threads from best in class Intel CPUs for all your machine learning and batch processing needs. Easily spin up a one-click Machine Learning and AI application image and get immediate access to Python3, R, Jupyter Notebook, TensorFlow, SciKit, and PyTorch. Our listeners get $100 in credit! Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Linode https://linode.com/changelog – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog https://linode.com/changelogRollbar https://rollbar.com/changelog – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: News/Updates: New seq2seq architecture without explicit encoder/decoder https://twitter.com/PyTorch/status/1033881797841575936Dancing video synthesis https://twitter.com/hardmaru/status/1032762806796312576O’Reilly AI SF https://conferences.oreilly.com/artificial-intelligence/ai-caHow AI is decommoditizing the chip industry https://venturebeat.com/2018/08/16/how-ai-is-decommoditizing-the-chip-industryJohns Hopkins researchers use deep learning to combat pancreatic cancer https://www.healthcareitnews.com/news/johns-hopkins-researchers-use-deep-learning-combat-pancreatic-cancerArtificial intelligence system designs drugs from scratch https://phys.org/news/2018-07-artificial-intelligence-drugs.htmlLearning Resources: Hands on ML books dataviz https://anvaka.github.io/greview/hands-on-ml/1/Udemy - Complete Guide to TensorFlow for Deep Learning with Python https://www.udemy.com/complete-guide-to-tensorflow-for-deep-learning-with-pythonBooks “Hands-On Machine Learning” by Aurélien Géron https://www.amazon.com/dp/1492032646Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Bane with for change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at change law because of roll bar. Check them out at rollbar.com and we're hosted on Lino servers. Head to linode.com slash change log. This episode is brought to you by Digital Ocean. They now have CPU optimized droplets with dedicated hyper threads from best in class. Intel CPUs for all your machine learning and batch processing needs. You can easily spin up their one click machine learning and AI application image. This gives you immediate access to Python three, R, Jupiter notebook, TensorFlow, side kit and PyTorch. Use our special link to get a hundred dollar credit for Digital Ocean and try it today for free.

• 0:45Speaker 00

  At the do.co slash change log. Once again, do.co slash change log.

• 1:03Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence, practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show at change.com slash community, follows on Twitter, we're at Practical AI FM. And now into the show.

• 1:29Daniel Whitenack?

  Hey Chris, how's it going this week?

• 1:30Chris Benson?

  Doing great. I am super excited about the next few weeks. Got some cool things coming up, which we'll talk about.

• 1:37Daniel Whitenack?

  Yeah, definitely. So I'm here joined by my co-host, my co-host Chris, and we're going to bring you some news and updates from the AI world today, and then also some learning resources to help you level up your, your AI skills in a, in a practical sort of way. But yeah, there's a, there's a lot coming up, Chris. I know I'm traveling over the next three or four weeks, quite a few times and we're, I know you've got some travel too. So there's a lot of exciting stuff coming up.

• 2:09Chris Benson?

  There is, and there's so much happening in the news right now. I am going to just leap on into the first one, if you don't mind. Sounds good. Okay, so I was, I came across something that I thought was really defined what I've been noticing in the, in the industry, and that is, it was an article from VentureBeat called "How AI is Decommoditizing the Chip Industry." And it was a cool read, because it really was pointing out something. It seems like, you know, for years, as you had, you know, with, before the, the era of AI and, and chips and, and the commoditization of different things coming out in computing in general, and now AI has kind of reversed that trend to some degree, and that processing devices,

• 2:51Chris Benson?

  CPUs and the like, GPUs, etc., are really becoming more and more specialized, and that is creating all sorts of entrepreneurial opportunities for different companies. And so we're seeing lots of chip startups, instead of just software startups. And some of these companies are like Nirvana, whom some people may know from, from Intel at this point, Graphcore, Cerebrus, Vassif, there's a whole slew of them, and they're really challenging the, the big incumbents, which, which certainly in the AI era has been Nvidia kind of leading the way, but you know, Intel's come, come in hard, come back from the CPU world into, into the new AI-oriented chip world. Microsoft has, has theirs, AMD, Qualcomm, Google TPUs,

• 3:35Chris Benson?

  and IBM. All these big players are getting huge challenges. So Nvidia really came in with early dominance with the GPU, as they moved from consumer gaming into AI, and, and you know, they had been kind of the poster child for, for the AI world. Even they, at this point, are, are having to watch some of the, the new risers coming in because of what's called ASICs, which is application-specific integrated circuits. And that is chips that are designed specifically for a particular application. They're completely optimized for that. And, and that's just fascinating when you think about it, because even though Nvidia has their Volta architecture, and Intel has Nirvana, Google's TPUs. At the end of the day, this article is suggesting that the future lies in ASICs rather than commodity

• 4:20Chris Benson?

  hardware. What do, what do you think about that? Do you think they're right?

• 4:23Daniel Whitenack?

  Well, I first of all, think that you did an amazing job saying decommoditizing correctly on the first try, which I think is maybe one of the more impressive things that I've heard recently. But, but in all seriousness, I don't have as much exposure to, you know, the application specific type circuit world. I think it'll be interesting to see that develop. What I do have exposure to is definitely the kind of resurgence of specialized hardware in the context of things like, you know, another company that Intel owns now, Movius, has, you know, things like these neural compute sticks.

• 5:07Daniel Whitenack?

  And one of the things that I really like about them is they've kind of found a new niche that they're that they're filling, but it's also, you know, enabling really interesting new types of of applications. So they've got these little, you know, VPU visual processing unit, like USB sticks that you can plug into a Raspberry Pi or to a drone or other things and, you know, run your neural networks on this specialized architecture, you know, in a very kind of, you know, low power, at the edge sort of scenario. And, you know, I met I met GopherCon this week, actually, and just had a conversation before I walked over to record this with someone, said, oh, it's so cool.

• 5:48Daniel Whitenack?

  Now I have, you know, I've been doing all this drone stuff and now I can just plug in these sticks into my drone and have them follow around, you know, specific people based on their, you know, facial recognition and all that, you know, stuff. So people are really excited about these things.

• 6:05Chris Benson?

  You know, you have a great point there and I've used a Movius and I know so many software developers who are not data scientists, they're not coming from the traditional backgrounds leading into AI. There's software developers that have started in other areas, they might have been web developers and such, and and they've moved into this and having something like a Movius stick or or similar devices has really opened up the space for them. So since I was halfway thinking about I'm suggesting you and I go create a startup where we create an AI chip that's designed around AI for good. I say tongue in cheek.

• 6:40Daniel Whitenack?

  Yeah, I would be happy to do that. You provide the funding. A couple of things that I found this week actually had to do with more on the research side, which was kind of, I don't know, kind of different for me because I'm usually more on the non-research side. But I really found these interesting. The first is this new paper that came out on the archive from some people at Grenoble University in France. And there's also a PyTorch implementation of this network. And what it is, is it's a kind of new type of sequence to sequence prediction. So if you're not familiar, that's where, you know, a very common type of neural network

• 7:23Daniel Whitenack?

  that's used in things like machine translation of text and, you know, taking sequences of something to other sequences. And normally what happens in those is there's kind of an encoder and a decoder stage of those networks. And this paper showed that they could kind of combine those two things into a single two-dimensional convolutional layer, which I think is really, you know, it seems after you see it, it's like, oh, that's a really great idea. But, you know, it took someone to, you know, kind of come up with that and a natural step. So it's cool to see, even in things that are utilized in production so often, there can be this sort of innovation and refining happening.

• 8:06Chris Benson?

  You know, it's funny because when you think about sequences, you know, most people automatically turn to RNNs. But I know over the last year or so, I've seen so many CNN applications for sequential applications. And it's interesting to see how versatile different architectures in the larger convolutional neural network space have been, going beyond, you know, just the visual thing that we tend to associate with them normally.

• 8:34Daniel Whitenack?

  Yeah, definitely. And like I say, there's an implementation of this already on GitHub and PyTorch, which I love working with PyTorch. And so I would love to try out some other examples and you guys can as well.

• 8:46Chris Benson?

  Gotcha. Well, I am going to turn briefly toward the medical world where it intersects with AI. There is a couple of articles that I've run across. One is called "John Hopkins Researchers Use Deep Learning to Combat Pancreatic Cancer." And that one really struck a nerve with me because, incidentally, I'm on my second marriage, but in my first marriage, I lost my mother-in-law to pancreatic cancer. And we watched as she went downhill very, very rapidly. And in doing that, it made me very aware of how bad pancreatic cancer is in terms of only 7% of patients that are diagnosed make it another five years. It

• 9:29Chris Benson?

  has the lowest survival rate of any form of cancer out there. And so, this really caught my eye. And they're basically saying that early detection could lead nearly a third of all diagnosis to be made four to 12 months earlier, which could save a lot of lives or extend a lot of lives out there. So, in our ongoing theme of AI for good, I really am encouraged by that. They use deep learning in combination with a CAT scan to look for minute textural changes to the tissue. So, that was pretty amazing. And then the other thing I saw, which was on the pharma side, was a system that they call "RELEASE",

• 10:12Chris Benson?

  which stands for Reinforcement Learning for Structural Evolution, which uses these two neural networks, one that's kind of a teacher neural network and one which is kind of a student. And they say that the teacher knows 1.7 million active molecules in great detail, and the student's able to learn from that and then actually create new molecules and evaluate those new molecules with properties that researchers specify. And so, this is where you're seeing deep learning being applied to pharma to create designer drugs much more rapidly. And between the diagnosis of cancer and the life-saving aspects of that and being able to get to new life-saving drugs sooner, I'm just really impressed with how deep learning is revolutionizing medicine in general.

• 10:58Daniel Whitenack?

  Yeah, definitely. I think that especially coming from like a background where I was exposed to like computational chemistry and those sorts of methods, I think people don't, you know, when you say all of those molecules and those sorts of things, you know, maybe people don't fully realize that, you know, for even a single molecule, a small molecule like oxygen, you know, there's, if you ignore the protons and neutrons, you still have a bunch of electrons, six, I believe, if I haven't forgotten everything. And each of those are in a three dimensional space and, you know, there's a time element and there's potentially external fields and there's just a lot of variables that happen

• 11:40Daniel Whitenack?

  in these sort of computational chemistry scenarios. And if there's anything we know about deep learning, it's, it's pretty useful in high dimensional spaces sometimes. So I think that's, that's really interesting to see more of those methods come out.

• 11:56Chris Benson?

  What else have you seen this past week?

• 11:57Daniel Whitenack?

  Well, I saw this super creepy video, which I shared with our

• 12:02Chris Benson?

  I saw it.

• 12:03Daniel Whitenack?

  I shared with our users on, not our users, our listeners on, in our Slack channel. It's this new work from Berkeley. And essentially the video that I saw, and maybe there's multiple out there, I'm not sure, is like a guy dancing, you know, it's like a Bruno Mars song. And what they did is they kind of taped two individuals moving around in some space to kind of train on their movements and then, and their body structure. And then they generated video of these two individuals dancing in the same way as in, as in the Bruno Mars and then a ballet video. And it's, it's just amazing. And I think you mentioned when, when

• 12:47Daniel Whitenack?

  you first showed, you know, some people, they didn't even realize that it was generated, you know, videos of people. They thought they were actually dancing synchronized in that way.

• 12:57Chris Benson?

  Yeah, it was actually my six-year-old daughter Athena, we were, after you posted it in Slack, I saw it. And she heard the music on it. And she comes running up to my laptop and we were looking at it and, you know, I'm marveling because they showed in the video, which people can see in the show notes, they showed the video of the of the source dancer. In one case, it was kind of a funk type dance, and then there was a ballet dancer. And then these two people that they were using to superimpose the motion on. And they didn't always line up. And so you'd have these these brief moments, these subtle moments where the body was doing things the body couldn't do. And it was enough for me to, you know, I kind of knew what I was looking at. But my six year old daughter never realized that it was generated.

• 13:41Chris Benson?

  She's grown up in a world where this is normal. You know, AI doesn't even faze her at her age because she's seen it from me. And just like mobile technologies and everything else, it's normal. But later on, I said, you realize those people weren't actually moving like that. The computer made them do that. And she goes, no, I had no idea.

• 14:00Chris Benson?

  And I'm just thinking, just two or three years down the road, where is this going to go? You'll be unable to distinguish generated motion from from real life.

• 14:08Daniel Whitenack?

  Yeah, I think, you know, not to give away all of our startup ideas. And I guess you can scoop us if you like. But another startup we should create is, you know, the computer generation of music videos where we kind of make obsolete all of the music video dancers and just get their training data. And then we can reuse the same dancers in any sort of video and any sort of background and make it seem realistic. That's my my second startup proposition.

• 14:36Chris Benson?

  I'm all over it, but just so long as it doesn't involve, you know, the two of us out there dancing, you know, so long as we're not the models used in any of the stuff.

• 14:44Daniel Whitenack?

  Yeah, well, I'm sure you could do better than those dancers in real life.

• 14:48Chris Benson?

  I don't know about that. My wife would tell you not so much.

• 14:53Daniel Whitenack?

  So the last thing that I wanted to share with everybody is that Chris and I are going to be at O'Reilly AI in San Francisco, the O'Reilly AI Conference, which is, I believe, going on when this airs, it will be going on this week. So if you are at O'Reilly AI in San Francisco, come find us. We'll be walking around, doing some interviews. We'll have stickers, we'll have some nice swag and all sorts of stuff. So come and meet us. We'd love to hear from you. We'd love to hear about your ideas and discuss whatever topics you'd like. So so come find us there. I'm really excited to to attend.

• 15:32Chris Benson?

  And I second that. We are just there to meet everybody and to have conversations, to do a bit of recording. So don't be shy. We're looking forward to meeting lots of people in San Francisco.

• 15:45Daniel Whitenack?

  Definitely. Well, let's go ahead and move on to some learning resources. This is the part of these type of shows where we just kind of share a couple of things that we've run across that have been useful in terms of learning new things within the AI ecosystem or new frameworks or new techniques or whatever it is. The first one that I found this week, which I don't think is totally new, but it was new to me, is the site that kind of creates a data visualization, a map of all of these different data science and machine learning books. So it's called Hands-on Machine Learning.

• 16:25Daniel Whitenack?

  And it's kind of like a little roadmap of all of these different books that have been released on different subjects like deep learning and Python in general, and beginner books and expert books. And it kind of guides you to the different sections that you might be interested in. So if you don't know where to start or what books to look at, that might be a good place to just kind of explore what's out there and maybe avoid an expert book if you're looking to begin and start out in AI or maybe you're looking for a book specifically about deep learning or something like that.

• 17:02Chris Benson?

  I think this is great. I had not seen this before. So I'm looking at the link at this point. And after we're done recording, I'm going to go snoop through it and see what I need to go get.

• 17:12Daniel Whitenack?

  Sounds good. What do you got for us this week, Chris?

• 17:14Chris Benson?

  So I have, there is a Udemy course that I decided to put that's pretty good. It is a paid course and the price typically, this is one of those things where I've never seen it at full price. It's always at some kind of discount. Currently it's $10 to get it, but it's called Complete Guide to TensorFlow for Deep Learning with Python. And it is quite lengthy, actually, and you can kind of pick and choose, but it has 14 hours of on-demand video. And so if video is your thing in terms of learning, it kind of takes you through everything from the beginning is kind of what is machine learning to an introduction to neural networks, TensorFlow basics, CNNs, RNNs,

• 17:56Chris Benson?

  various other topics that they that are kind of ancillary auto encoders, reinforcement learning, and even generative adversarial network scans. And so that was it had such a breadth of topics that it was covering that for 10 bucks, I thought it was a pretty good, pretty good investment to get people into. And so if you like to use Udemy as a platform for learning, then I recommend this one. It has 4.5 stars out of 6,500 ratings as I, as I read this on their website right now. So that's what I found. I thought it was a good thing for a beginner to get into.

• 18:31Daniel Whitenack?

  Awesome. And yeah, I should mention too, that if anyone out there is looking for books on specific subjects or looking for specific types of courses, or maybe just GitHub repo that has some relevant examples, go ahead and, and jump over into our Slack channel. You can join the Practical AI and Changelog Slack team by going to changelog.com/community and there's a Practical AI channel in there. And just send us your question and we'll do our best to point you to whatever resources we know about and maybe some other resources that our other listeners know about. So make sure and leverage that and otherwise we'll, we'll reconvene next week.

• 19:13Daniel Whitenack?

  We're talking with Susan Etlinger about AI ethics, which I'm really excited about. I know you are, Chris, because our listeners have expressed to us a lot that they want to hear about this topic and I know I want to hear about it as well. And Susan's an expert will also be talking at O'Reilly AI about the same subject. So make sure and, and join us again next week.

• 19:34Chris Benson?

  Yep. She is super impressive. I can't wait for that conversation. And as this gets released, I will be seeing you within a couple of days at O'Reilly AI and we'll meet a whole bunch of our listeners and do some recording as we go. So I'm looking forward to this coming week.

• 19:50Daniel Whitenack?

  All right. See you there.

• 19:51Chris Benson?

  See you there. Talk to you later.

• 19:55Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes and give us a rating. Go in your podcast app and favorite it. If you are on Twitter or a social network, share a link with a friend, whatever you gotta do. Share the show with a friend if you enjoyed it. And bandwidth for change log is provided by Fastly. Learn more at fastly.com. And we catch our airs before users do hear change law because of rollbar. Check them out at rollbar.com slash change log. And we're hosted on Leno cloud servers. Head to leno.com slash change log. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in

• 20:38Speaker 00

  your email address. Get our weekly email, keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.