Learning about (Deep) Learning

In anticipation of the upcoming NVIDIA GPU Technology Conference (GTC), Will Ramey joins Daniel and Chris to talk about education for artificial intelligence practitioners, and specifically the role that the NVIDIA Deep Learning Institute plays in the industry. Will’s insights from long experience are shaping how we all stay on top of AI, so don’t miss this ‘must learn’ episode. Sponsors: DigitalOcean https://do.co/changelog – DigitalOcean’s developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99% uptime SLA, and 24/7/365 world-class support to back that up. Get your $100 credit at do.co/changelog https://do.co/changelog. Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Featuring: Will Ramey – LinkedIn https://www.linkedin.com/in/wramey, X https://x.com/wrameyChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: NVIDIA GPU Technology Conference https://www.nvidia.com/en-us/gtc, October 5-9, 202020% off for our listeners! Use code CMINFDW20 by 9/25 for additional 20% offNVIDIA Deep Learning Institute https://www.nvidia.com/en-us/deep-learning-ai/educationNVIDIA to Acquire Arm for $40 Billion https://nvidianews.nvidia.com/news/nvidia-to-acquire-arm-for-40-billion-creating-worlds-premier-computing-company-for-the-age-of-aiNVIDIA Ampere Architecture https://www.nvidia.com/en-us/data-center/nvidia-ampere-gpu-architectureGeForce RTX 30 Series Graphics Cards https://www.nvidia.com/en-us/geforce/news/introducing-rtx-30-series-graphics-cardsPractical AI Episode #15: Artificial intelligence at NVIDIA with Chief Scientist Bill Dally https://changelog.com/practicalai/15Practical AI Episode #36: Growing up to become a world-class AI expert with Anima Anandkumar of NVIDIA and CalTech https://changelog.com/practicalai/36Practical AI Episode #90: Fully-Connected with Chris and Daniel - Exploring NVIDIA’s Ampere & the A100 GPU https://changelog.com/practicalai/90Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  After about a year, maybe two years of that, a few of the people that we had been working with internally at Nvidia are subject matter experts who are constantly going out and engaging with customers and helping them solve their challenging problems. Came to us and said, hey, there's this thing called deep learning. It's kind of interesting and we built these little like Jupiter notebooks that kind of show people how to do it. Could we just maybe host that on your training platform and see if people want to learn how to do this thing? Within a very

• 0:26Daniel Whitenack?

  Oh, how far we've come, right?

• 0:29Speaker 00

  Yeah, within a very short amount of time, the number of people who, you know, signed up for an account and logged in and experienced this deep learning training completely eclipsed Anything else we had done before and we realized that we had a tiger by the tail and that, you know, we'd been experimenting, we'd been trying things, something worked. That's when we decided to double down.

• 0:51Speaker 02

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Change Law because of Rollbar. Check them out at rollbar.com. And we're hosted on Leno cloud servers. Head to leno.com/changelog. This episode is brought to you by Digital Ocean, droplets, managed Kubernetes, managed databases, spaces, object storage, volume block storage, advanced networking like virtual private clouds and cloud firewalls, developer tooling to robust API and CLI. To make sure you can interact with your infrastructure the way you want to. Digital Ocean is designed for developers and built for businesses.

• 1:31Speaker 02

  Join over 150,000 businesses that develop, manage and scale their applications with Digital Ocean. Head to do.co/changelog to get started with a $100 credit. Again, do.co/changelog.

• 2:02Speaker 03

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.

• 2:29Chris Benson?

  Welcome to another episode of the Practical AI Podcast. We are the show that tries to make AI practical, productive, and accessible to everyone. My name is Chris Benson. I'm a principal emerging tech strategist at Lockheed Martin. And with me as always is Daniel Whitenack, who is a data scientist with S I L International. How's it going today, Daniel?

• 2:49Daniel Whitenack?

  It's going really well, Chris. It's been a busy week so far, but fairly productive. How about yourself?

• 2:55Chris Benson?

  It's been a fun week for me. I just hit my 50th birthday and

• 3:01Daniel Whitenack?

  Happy birthday!

• 3:02Chris Benson?

  Thank you. It was a big one. I was pretty happy with it.

• 3:04Daniel Whitenack?

  Yeah.

• 3:05Chris Benson?

  I don't get into the old, the whole crying over your birthday thing. So I'm having fun.

• 3:10Daniel Whitenack?

  That's good.

• 3:10Chris Benson?

  I'm trying to extend it for the entire month in every possible way. So good times.

• 3:15Daniel Whitenack?

  Right. Yeah, that's good. You have a party or, you know, any fun over the weekend?

• 3:20Chris Benson?

  We are scheduling all sorts of fun stuff. We're just trying to do some good family stuff, stay safe with the, you know, with COVID, the way it is and good times.

• 3:28Daniel Whitenack?

  Yeah, awesome. And it's been a pretty eventful week as well on the AI front. I'm sure you saw, as I did, a pretty large acquisition that's looking like it's going to happen between Nvidia and ARM. So that's pretty cool stuff.

• 3:41Chris Benson?

  Yeah, if it happen, if it gets through $40 billion acquisition, which is obviously big news, but I mean, really, anything Nvidia does is big news as we have proven time and time again, we're always talking about Nvidia. And speaking of Nvidia, we have a guest that I know both of us have really wanted on the show for a while now. With us today is Will Ramey, who is the Global Head of Developer Programs at Nvidia. Welcome to the show, Will.

• 4:08Speaker 00

  Thank you for having me.

• 4:09Chris Benson?

  So I'm wondering, I know you and I have met previously at GTC and had conversations, and you have an interesting background in terms of how, I know you run the Deep Learning Institute. I was wondering if, before we dive into all the cool stuff, if you could just tell us kind of how you got to where you are now at Nvidia and in the particular position. And you guys do some amazing work, so I'm really looking forward to hearing this.

• 4:33Speaker 00

  Wow, yeah, let me, uh, let's see if I can summarize the short version of it. I got an undergraduate degree in computer science and worked as a software engineer in Silicon Valley for many years for, uh, starting with Silicon Graphics and then a number of startups, uh, including a game studio. And during that time, I transitioned from working as a software engineer to a product manager and a producer and a general manager in the games industry. And then some friends of mine invited me down to join this scrappy little company called Nvidia that was just, you know, really ferociously competing in the video game industry back then.

• 5:15Speaker 00

  And, uh, I accepted their offer, and that was 17 years ago.

• 5:19Chris Benson?

  Holy cow.

• 5:21Daniel Whitenack?

  A little bit of change happening during that time at Nvidia, I would imagine.

• 5:25Speaker 00

  It's been a wild ride, both, you know, for me personally and for the company over the last, coming up on two decades here, where when I joined the company, we were viewed as a chip company, right? We just, we designed chips, that's kind of our business. And I had the opportunity to, uh, work as the product manager for some of our developer tools and then do some program management of large software projects we were working on with Microsoft and others, and work a little bit in our embedded business. And slowly but surely, the company started to climb up the value chain from chips to boards and embedded systems, kind of system-level products.

• 6:09Speaker 00

  And then, uh, in 2009, we introduced this new technology called CUDA.

• 6:15Speaker 00

  And CUDA, as you know, many people know and love it today, is a parallel computing platform that gives developers access to the parallel computing power, the accelerated parallelism of Nvidia's GPU processors. And that was exciting, right? Scientists, researchers, were starting to figure out new and innovative ways to take advantage of this computing capability. And I moved into a role as the product manager for CUDA. So introducing this new technology, talking with people who were trying to make sense of it and apply it to some of their most challenging problems. And that gave us the opportunity to go and really start working with people who we had never worked with before.

• 6:59Speaker 00

  You know, working with Fortran programmers, working on like climate and weather simulations, working with people in the energy industry, trying to figure out what does the subsurface of the Earth look like? People who are working at the very smallest scales of physics, trying to figure out molecular dynamics and how things work at the quantum levels. Really big sophisticated simulations.

• 7:23Daniel Whitenack?

  Yeah, in the beginning days of CUDA there, it sounds like the focus was a lot on like scientific computing, like materials modeling, more on the like scientific computing side. Was AI even in the mix at that point?

• 7:38Speaker 00

  Not at all. In fact, one of the most common questions we got was, wait, why should I use gaming technology to do real science?

• 7:47Daniel Whitenack?

  Yeah.

• 7:47Speaker 00

  And it seems kind of like it's just a strange question these days where some of the most important scientific challenges of our time are being addressed using not only this gaming technology, but what has grown into a more general purpose parallel computing platform. The application of this parallel computing technology to AI happened several years later. And it really came again from the research sector, where there were a small number of researchers who had somehow managed to maintain funding and were exploring this area of deep neural networks, and what we have come to call deep learning. And they were building what at the time

• 8:31Speaker 00

  seemed to be really large, complicated. Neural networks that had very simple amounts of processing in each node. And this was a very different approach from what had been done previously in other machine learning techniques. And using really, really large amounts of data to train and fine tune these neural network models to perform various tasks initially in computer vision. So things like image classification and object detection and segmentation and things like that. And what they discovered was somewhat surprising in that these GPU parallel processors were

• 9:12Speaker 00

  almost ideally suited for accelerating the work of training these neural network models to perform a wide variety of tasks. And that was really the genesis of the deep learning AI revolution that we're all witnessing today.

• 9:33Daniel Whitenack?

  And I'm kind of curious, you know, because it sounds like, you know, this realization came from the research community initially when they started thinking more about accelerating these AI workflows on the GPUs or the graphics cards. So at what point did you sort of begin to see a shift into like sort of real world industry applications where industry people started, you know, really expressing this desire for GPU accelerated machines and that sort of thing, you know, out of research and into the industry world?

• 10:12Speaker 00

  Yeah, that's a great question. It happened in a couple of phases. And in some ways, we're all very fortunate that those initial researchers decided to publish not only their results in research papers, but they published the software that they used to train the deep neural network models, the things that we call deep learning frameworks today. You know, one of the early ones, there was CAFE and Torch, now PyTorch, TensorFlow, MXNet, these, you know, there are some called PaddlePaddle. And these frameworks being available in the open source community, but really just being available at all, allowed All of us to start to experiment with them.

• 10:57Speaker 00

  And what we observed happening was that very early on in the process, the cloud service providers who had adopted GPUs into their offerings very quickly recognized that this deep learning technology was going to be not only valuable to their own businesses, but a valuable new type of workload that allowed them to offer more compute services to their customers and grow their cloud computing business. And so you saw new types of instances, new types of servers being offered by the cloud service providers and prepackaged virtual machine images, basically the operating system and all the software

• 11:40Speaker 00

  that you run on these servers on the cloud service provider platforms, prepackaged and ready to go. All you had to do was click a few buttons and you had a server with GPUs and all the software you needed, ready to start running and training your models on your data. So that lit a fire, really, especially in the startup community, where up until then, the cost of buying your own data center and getting everything set up was pretty prohibitive to small startups. Now they could just very quickly rent all of the compute capability they needed and most of the software was already set up for them to start exploring all the innovative ideas

• 12:23Speaker 00

  that startups come up with every time a new technology is released. And we also, in that timeframe, saw a number of, let's say, forward-thinking enterprise organizations and government agencies began to take advantage of these capabilities, where they saw there was a good business opportunity, either to improve their own internal business operations and practices or to build new and enhanced products that allowed them to provide better experiences, better services to their customers. And now what we're seeing is really beyond to sort of cross the chasm, if you will,

• 13:07Speaker 00

  to the point where the the leading, the early adopters have already proven out the technology and everyone is just racing to figure out how to apply it practically to improving their products, their services, their business operations and and just other other aspects of their challenging problems that this technology can address.

• 13:26Chris Benson?

  So before we dive fully kind of into the the AI, you know, powerhouse that Nvidia is today, I I'm kind of curious, you know, referring back to those early days when you started and, you know, we really thought of Nvidia as is that graphics company, graphics cards and, you know, anytime in gaming and anytime we were thinking along those lines, that was the name. And you made this remarkable transition as an organization into being, you know, the one of the dominant AI companies and on the hardware side, the dominant AI company. And that's a huge cultural shift. And I'm just curious, as someone who lived through that, that very successful shift that you made, how did the company do that when so many other organizations that do something similar tend to fall down on their face?

• 14:10Speaker 00

  Well, you know, I I have to give our our leadership a lot of credit here for having a very strong vision and a and a conviction that we were headed in a direction to be more more valuable to the world. But at the same time, taking measured steps, right? Small, measured steps each year, each quarter that allowed us to experiment and to learn quickly and to test what worked and what the various markets, what our customers were were ready for. And through that process of continuous experimentation and innovation

• 14:50Speaker 00

  and proving out our hypotheses as we went, we were able to find our way and into providing.

• 14:58Speaker 00

  Not just the technology, but the the business models, the training, the support and resources that developers, researchers, data scientists, this emerging class of DevOps engineers who are proving to be so crucial in connecting the work that happens in the engineering and development teams with the IT deployment and and operational teams. Just learning as we go and really investing in this this ecosystem of developers and people who can use the technologies that we're creating to solve their problems is what my experience was kind of along the way.

• 15:41Daniel Whitenack?

  Yeah, and and maybe to just kind of set the stage for where we're at now. So you you mentioned a lot of different, you know, stakeholders there and and involved parties from developers to DevOps and all of that. And I know, of course, people might think of Nvidia as like the actual GPU cards and servers and and these other things. But could you just kind of give us a very, you know, I know it's hard to to not leave anything out, but just kind of a brief sketch of the different, you know, types of things that Nvidia is offering the AI community. So there's the hardware side of things, but then there's other things as well. Like, one of the things I use fairly frequently is the NGC containers that are fairly

• 16:26Speaker 00

  Hmm, yeah, yeah.

• 16:27Daniel Whitenack?

  Like, really easy to use, like optimized. So maybe you could just give a sketch of some of these other things that that people might not be as aware of that that Nvidia is kind of offering the community.

• 16:39Speaker 00

  Sure, sure. So it's a really long list, but maybe I can just pick a couple of the highlights.

• 16:43Daniel Whitenack?

  Yeah, yeah, I just highlight a few. There's there's a bunch, but.

• 16:49Speaker 00

  So in in addition to the, you know, the GPU hardware itself and now the the networking hardware with the acquisition of Mellanox and and Cumulus, or bringing them into the Nvidia family. And then the systems, the the HGX system designs that we've designed and worked with lots of our OEM partners to build and put in into the market. And of course, the DGX product line, the the workstations and the servers and the the pods, kind of data center scale computing solutions that we offer. There's a really, really rich.

• 17:27Speaker 00

  Collection of software, some of which we contribute to as open source projects. So whether it's PyTorch or TensorFlow or any of the other leading deep learning frameworks, we have teams of engineers who are working on contributing to those projects to help them take best advantage of GPU acceleration directly and then also take advantage of things like our TensorRT. This is the deep learning model compiler and runtime environment so that once your deep learning model is trained, you know, in the same way that we as humans, we kind of put a lot of energy into training, you know, training ourselves and learning a new thing. But once we've learned it, we kind of have like the mental shortcut. You know, let's say you're learning to juggle.

• 18:12Speaker 00

  It takes a lot more effort to learn how to juggle than it does to actually juggle after you've learned how to do it. And it's the same thing for playing tennis or playing piano or, or anything like that. Same thing for deep learning. Once you've trained the neural network model, it turns out that it can be optimized and run at a much faster performance in an inference deployment environment or at lower energy profile depending on what your needs are. And this TensorRT runtime is able to help with that. But there are also higher level tools and resources like NGC. The NGC catalog provides both containers with all kinds of accelerated computing and

• 18:56Speaker 00

  deep learning and data science software environments that are pre-tested, pre-configured, ready to go, can be deployed in your own servers and your embedded Jetson-based platforms or on cloud service provider platforms. And also includes things like model recipes. So maybe you don't want the entire environment, you just want some deep learning models that are known to be good and a recipe for how to use that to train it with data and things like that. So NGC provides a number of those things as well. And then there are even higher level services on top of that that work with our EGX

• 19:36Speaker 00

  solutions that are kind of combined cloud or on premises with Internet of Things. So say you have, say you're a city and you want to build a smart city type environment to be able to keep track of, you know, how many of which types of vehicles are running on your road. So you can do predictive maintenance before potholes show up or figure out where the safe and less safe traffic patterns are. So you can put in better crosswalks or change the timing of your stoplights and things like that. You might have hundreds or thousands of cameras and you need to be able to do analytics on those video feeds that are coming into the system. This is the kind of thing that these EGX scalable systems are able to handle.

• 20:37Speaker 03

  Change log plus plus is the best way for you to directly support practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal, and help sustain our production of practical AI into the future. Simply follow the change log plus plus link in your show notes or point your favorite web browser to changelog.com/ plus plus. Once again, that's changelog.com/ plus plus.

• 21:22Chris Benson?

  So given that we are just about to head into GTC, the GPU Technology Conference, I know there's all sorts of announcements and stuff and you guys are going to release new stuff coming up. Obviously, the thing on everyone's mind is ARM. I was wondering if you could tell us a little bit about what GTC is for those who may not have had the opportunity to attend already. And also, I imagine it's a little bit early, but if there is anything that you can address with the ARM acquisition, we'd love to hear it.

• 21:50Speaker 00

  Sure, okay, let's take those one at a time. Sure. We can start with GTC. You know, the GPU Technology Conference, or GTC, is something that's really near and dear to my heart. We organized the first one a little over ten years ago. And it has grown and evolved with our ecosystem of developers and researchers and data scientists and everyone else who's learning and exploring how to how to apply the new technology. This year, with the COVID-19 virus, we transformed GTC into an online experience. And over sixty thousand people joined us earlier this year for the very first one.

• 22:34Speaker 00

  In just a couple of weeks, we're going to be doing what we call GTC Fall, the next installment of GTC. And it'll have all of the the usual elements that people have become familiar with in the GTC conference. We'll have a keynote by Jensen Huang, our president and co-founder. He'll be kicking things off on Monday, Monday morning, October 5th. But we'll also have just hundreds and hundreds, I think it's something like over six hundred live and pre-recorded sessions available this year, talking about all of the amazing work that is happening out in the ecosystem.

• 23:16Speaker 00

  You know, some of the talks are by Nvidia engineers and experts and product managers to help people learn how to use these technologies. But a lot of them are by people who are using it to solve their own problems and then sharing back with the community the types of solutions that they're building in graphics and ray tracing, in artificial intelligence with deep learning. They'll be talking about hybrid cloud computing. They'll be talking about the work that they're doing in healthcare, in public sector and government applications. You know, just a really, really amazingly broad range of topics. And one of the most I think valuable things about GTC is it's one of the very few places where

• 24:00Speaker 00

  there's an opportunity for people from different disciplines, people working on completely different application domains, to kind of cross pollinate their best ideas and inspire innovation in a large number of fields. Unlike most other events where, you know, people who are working on the same kind of thing get together and there can be, you know, a little bit of an echo chamber sometimes. So some of the things that we're doing to help facilitate that kind of, you know, innovative cross pollination, even in a virtual environment, is hosting networking events for the attendees. We have this thing that we call Dinner with Strangers. There's basically, you know, a topic that could be an industry-related topic or a

• 24:43Speaker 00

  technology-related topic, and people who are interested in that technology and want to talk about it with people maybe they haven't met before are invited to get together in kind of small groups. We'll have someone from Nvidia with some knowledge on the topic kind of host the gathering. But it's not a presentation, it's just a way for people with common interests to get together, maybe bring some dinner to their desk, and eat and have some social interaction, talking with other people who have common interests.

• 25:11Daniel Whitenack?

  Yeah, this is so great. And before we go too much further, I just want to mention as well that, you know, this is coming up, it's October 5th through the 9th. As you can already tell, we haven't even gotten into everything, but it's so much interesting stuff here. And Nvidia has provided us with a 20% off discount for our listeners for GTC this year. And additionally, there's an early bird price that lasts until September 25th. So I know, I'm registering a code we're going to list in our show notes. The code is CMINFDW20. It might be hard to remember, I'll definitely put it in our show notes, but this is a great deal, it's not very expensive, lots of great material, it sounds like, to come.

• 25:55Daniel Whitenack?

  I'm curious, as you sort of tried this out earlier in the year, and now it's this fall edition, what were some of the, maybe the surprising elements that you noticed from having it online? For the first time, completely online for the first time, of course, there's like, sounds like a huge attendance for one. But what were some of the maybe surprising things that came out of the fact that you were able to have it online that maybe were unexpected to you?

• 26:22Speaker 00

  Well, I mentioned earlier how the kind of the the evolution of Nvidia over time has been supported by lots of small experiments. And one of the experiments that we did at GTC in the spring of this year was to offer our hands-on Deep Learning Institute training in a virtual environment. We had never done that before, and we'd certainly never done it at the scale of of trying to train thousands of people at the same time as part of a GTC in these virtual classrooms. I'm just incredibly proud of the work that our Deep Learning Institute team did to pull that together, to work out all the kinks, to cross-train all of the instructors

• 27:05Speaker 00

  and teaching assistants so that thousands of people who attended those trainings in March could have a great hands-on learning experience. And I'm really excited that this upcoming GTC is a place where we're going to offer 16 of these workshops, timed so that several of them are available in the North American time zone, several are available in the European time zone, and several are going to be available in the Asian time zones. And we'll be offering our brand-new Fundamentals of Deep Learning course for everybody who just kind of wants to get started and understand what the basics are, get their hands dirty, training their first neural networks.

• 27:50Speaker 00

  We have a completely all-new, updated natural language processing course that's based on the state-of-the-art tensors-based approaches to natural language processing. And we also have a brand-new recommender systems. Course that will help people learn how to build recommendation systems and integrate them into their applications. And as always, you know, the other courses, including the multi GPU course have been updated. There's a new course on CUDA. It's Deep Learning Institute program has helped us to train over 250,000 people worldwide in the last several years.

• 28:29Speaker 00

  And the demand for access to this kind of, you know, hands-on learning by doing style of training is just continuing to increase. And we're really excited about being able to offer that in an even larger way at GTC this October.

• 28:46Chris Benson?

  Yeah, those are great classes. Having taken several of them myself, I love taking them and definitely recommend them. And so I guess wanted to just follow up for a second. I imagine that your CEO Jensen Huang is probably going to address the acquisition of ARM. Is there anything you can tell us? Or do you really have to wait until after he's talked at the keynote?

• 29:09Speaker 00

  You know, that that news is hot off the presses just maybe a day and a half ago. We're still counting it in hours. And Jensen is really going to be our spokesperson on that. But but what I can say is we're all just really excited about it. I mean, there's so, so many opportunities for ways that we can serve the ecosystem of developers and data scientists and researchers that the companies and organizations that are they're trying to solve really, really hard problems. And we're just delighted that we have the opportunity to help them with the platforms and the technologies and the training that they need.

• 29:47Chris Benson?

  Totally get it.

• 29:48Speaker 00

  To make progress on those things.

• 29:49Chris Benson?

  Yeah, that I totally understand that. And I knew it was so new, but I felt like I had to ask.

• 29:55Speaker 00

  It's exciting. And, you know.

• 29:57Daniel Whitenack?

  Good, good timing for this episode, for sure. And, you know, tune in here in a couple of weeks to find out more, for sure. It's gonna be gonna be exciting. Willie, you mentioned one thing, kind of as you were talking about GTC and, you know, Chris mentioned that you'd taken some of these classes and you talked a little bit about the Deep Learning Institute, some of its activities at GTC. I was wondering if you could just give us, kind of step back a bit and tell us a bit about sort of the origins of the Deep Learning Institute and kind of its current state and what it is now.

• 30:29Speaker 00

  Oh, sure, yeah, actually, that's a pretty fun story. I think I mentioned at the beginning that for a period of time I was the product manager for Cuda. And when we were first starting to bridge from working in the academic research segment into commercial enterprises, there were a lot of customers who said, "Yeah, yeah, that looks promising and we love the demos, but can you come show us how to do it?" Right? We have smart people here where they can learn, but we need someone to come show us how. And so, you know, I dusted off my frequent flyer mileage card and started traveling the world with literally a pallet of laptops.

• 31:09Speaker 00

  We got some gamer laptops that had GPUs in them and a colleague and I started just like flying all over the world, shipping pallets of laptops. And we had this challenge that, you know, stuff breaks, it gets stuck in customs, you have to like re-image the darn things after every training to make sure they're going to work for the next people. And it was a big hassle. And after doing that for a couple of years, Amazon, AWS, announced that they were going to have thousands of GPUs available in their cloud. And we immediately said, "That, that is what we need." And so, we updated our training materials and figured out how to get all of that hosted in the cloud.

• 31:54Speaker 00

  And now, all we had to do was show up.

• 31:59Speaker 00

  And all of the people who wanted to learn how to, at the time, develop applications using CUDA could just connect to these cloud servers, pre-configured with all the software they would need from their own laptops. In fact, we had a few people just like do it from their iPads and it was fantastic. And then we realized, you know what? We could probably develop some self-paced content so that anybody in the world could just log in and learn how without having to wait for me to show up on their doorstep. And so that started taking off too. And after about a year, maybe two years of that, a few of the people that we had been working with internally at Nvidia, our subject matter experts who are constantly going out and engaging with customers

• 32:43Speaker 00

  and helping them solve their challenging problems, came to us and said, hey, there's this thing called deep learning. It's kind of interesting and we built, we built these little like Jupiter notebooks that kind of show people how to do it. Could we just maybe host that on your training platform? And, and see if people want to learn how to do this thing?

• 33:03Speaker 00

  And uh, How far we've come, right? Yeah, within a very short amount of time, the number of people who, you know, signed up for an account and logged in and experienced this deep learning training completely eclipsed anything else we had done before. And we realized that we had a tiger by the tail on that. So that's, you know, we'd been experimenting, we'd been trying things. Something worked, that's when we decided to double down. And we gave this initiative a name. We decided to call it the Deep Learning Institute. We hired a team and began building out a rich catalog of both self-paced and instructor-led content. We started an instructor certification program.

• 33:46Speaker 00

  So now there are over 600 instructors all over the world who are certified to deliver our training to their audiences.

• 33:54Speaker 00

  Many of them are, you know, well-qualified professors and academic researchers working in higher ed. Many of them are independent, like, training service providers who've now been able to make this part of their business practices. Some of them are even companies who have decided that they need so many of their own employees to be trained, that they're getting their internal employee instructors certified to deliver this training and scaling it up within their organizations as part of workforce transformation projects to become AI companies. We're happy to work with all of them. That's kind of how we've come to where we are today. And as I mentioned earlier, the latest new thing in the evolution of the Deep Learning

• 34:39Speaker 00

  Institute is this introduction of our online virtual classroom format that allows us to deliver training anywhere in the world and even to aggregate demand across many different customers or many different sites within same customer. Because, you know, it doesn't really make sense to send an instructor to train two or three people. That's a lot of effort and expense for everyone. But if you can get fifteen, twenty, twenty-five people together from multiple different sites, whether they're from the same company or from several different companies or organizations, then you can put together a really good learning experience for them to quickly learn how

• 35:21Speaker 00

  to use these new technologies through hands-on exercises and assessments and end up with a certification at the end of the day, proving that they have actually developed competence in applying these technologies to solve worthwhile problems.

• 36:00Chris Benson?

  So Will, got a question about, you know, you guys have all these great classes and they're very well run and they're very well integrated, but we're in this really fast-moving world of deep learning and AI in general. And how do you choose what should be part of your curriculum? What classes? There's so many topics out there and they're evolving so fast. That has to be a bit of a challenge to figure out what to provide for the people that you're serving out there.

• 36:26Speaker 00

  You know, it really is, but it's one of those problems that it's a good problem to have. It's like having an embarrassment of riches. There are so many opportunities to develop training curriculum around these new and emerging and quickly evolving technologies. But we do have to be a little bit careful which ones we choose to invest in and get a small army of instructors trained and certified to deliver. And so what we've tried to do is keep up with what is the state of the art and keep tabs on all the new research that's

• 37:03Speaker 00

  coming out. And as that research kind of evolves and new practices begin to start getting adopted, that's really the sweet spot for developing training. You know, you think training is really a way to teach people how to use and apply things that have already been proven. And so if you were to ask me like at this time last year, I would have said, you know, there's a lot of work in recommender systems, but the state of the art is still evolving so rapidly that if we were to go take a couple of months to distill that down into best practices and build a training course around it, by the

• 37:48Speaker 00

  time we actually got that out into the world, the state of the art would have moved on so quickly that people would be maybe a little frustrated or disappointing in the training. And so at this point, the best thing we should be doing is connecting them with the research papers and the open source projects and things like that. So that people who are more comfortable living on the bleeding edge can adopt that technology while it is still in its infancy. But this year,

• 38:13Speaker 00

  Things have evolved to the point where many of the recommender system development and design approaches, th- those best practices are established. In fact, they're really well established. And so we built a training course around that and are starting to offer it to everyone who needs to build, build these kinds of recommender systems. And it can kind of happen in waves. So this happened a few years ago for natural language processing. We built a course around it that was really incredibly popular. It was one of our more difficult courses, but it was really popular. And then last year, or over the last year, a completely new approach to natural language processing has emerged that is based on transformers

• 38:57Speaker 00

  and models like BERT and NGPT and so forth. And so it was time, very, very clearly time for us to go back and completely update and replace that older course with something that is teaching people how to use the latest state of the art techniques now that they've been boiled down to practice. On the DLI content team, we talk about this as it's kind of like painting the Golden Gate Bridge. By the time you get to the end, you know, you paint it from south to north. By the time you get to the north side, it's time to go back and start painting it all over again.

• 39:29Daniel Whitenack?

  Yeah. And so you mentioned the interactions with open source and the research community. Of course, it was really interesting when you were talking about the kind of story of the Deep Learning Institute and how kind of Jupiter entered the mix and like these different projects. From your perspective, like, you know, i- it seemed like part of the reason why Chris, Chris and I started this podcast was that very much focus on the, the practical side of AI when it seemed like to us that there were a lot of, you know, it was kind of like the Wild West. There were a lot of things out there that just weren't extremely practical. But, you know, of course, that, that has definitely

• 40:10Daniel Whitenack?

  changed and evolved over time. What is, maybe in the Deep Learning Institute, or maybe in your work more broadly at Nvidia, how do you think about engaging with different open source projects and, and contributing to those? I know Nvidia is involved with With a lot of those, but like you mentioned, for example, NLP and Transformers, of course, we've had Hugging Face on the show. And I know I've seen pictures on their Twitter of like them getting boxes of Titan RTX GPUs in the mail for training their models. So how do you keep your pulse on not only the new techniques, but how the tooling and the

• 40:52Daniel Whitenack?

  open source side of things is evolving? And is there a lot of freedom within Nvidia to contribute to open source, or an encouragement around that? And how do you choose where to put your effort and all of that sort of thing on the open source side?

• 41:05Speaker 00

  Wow, that's a really good question. If I could even distill it down to a simple list or a recipe, there's a lot of people within the company who'd really appreciate that. I think the best way to really explain it is that we look at each opportunity at each situation based on its own merits. And we recognize that there are so very many different types of open source projects, some of which, many of which, hundreds of which, in fact, Nvidia engineers have created and continue to maintain out in the world and invite others to participate in. Others of which we actually have adopted and integrated

• 41:49Speaker 00

  into some of our products and continue to support and make contributions and so forth, too. There isn't really like a single one size fits all recipe. I think it really depends on what do our customers need, what do our developers need, and what's the best ways that we can find to support them? And then at the same time, what are the best implementations? What are the best projects out there that we might be able to use and contribute to? To, you know, just be... Positive contributing members to the open source ecosystem. I'm not sure if that's a great succinct answer to your question.

• 42:25Daniel Whitenack?

  It's a big question. Yeah, it's a it'd be hard to distill that down, but I think it's a yeah, definitely good thoughts on that subject.

• 42:32Chris Benson?

  So I guess, you know, as we're starting to to get toward the end, I'm curious, being in the center of AI education and doing and accomplished all having accomplished all that you've done on this, kind of where do you see the future of AI education going with the Deep Learning Institute and even in a in a broader sense at large within the the larger AI community? Obviously, we've had changes lately where we've we've moved from a lot of in-person training to more and more remote training because of of COVID and its impact on the world. But what do you see over the horizons that maybe the rest of us haven't thought about and, you know, where you think things might go?

• 43:10Speaker 00

  Wow, that's a great question. Off the top of my head, I think there's really kind of three really important areas. One is in the training of AI practitioners, people who are going to be building and evaluating AI-based functionality to be integrated into these applications and helping them to understand just critical importance of analyzing the data that is used to train the neural networks and testing the outputs of those neural networks to ensure that they're really functioning as expected in all of the different scenarios where we need them to work. These are things like just making sure you have enough examples

• 43:52Speaker 00

  of all the different types of data that you need in order to have your neural network be effective in a real world environment. You know, for example, if you train a neural network to be able to distinguish between all the different types of flowers or dogs or cats in the world, and then later you show it a picture of a raccoon or a giraffe, it's not going to know what to do, right? So you need to train it to be able to handle all the variations, the variety of the real world that it's actually going to be exposed to. So that's one thing. The second thing is that for people who are not AI practitioners, who or who are, let's say, not yet AI practitioners, let's say they're earlier in their career or schooling. So much of our world and interaction

• 44:38Speaker 00

  With it are going to be influenced by these powerful AI tools. That I think it's really important that, you know, even school age kids have a basic understanding of what AI is and how it works. And, you know, when I go to a website and someone is recommending that I buy this pair of sneakers, or this shirt, or this, you know, whatever product, that I have a sense for, well, what might that be based on? Why are they recommending it to me? Is it because other people like me have also purchased this product and enjoyed it? And if so, how do they know who else is like me? You know, just being really thoughtful

• 45:19Speaker 00

  about how how does that world work? And and taking a peek under the covers, I think is going to be a really important thing for really everyone to be comfortable with the capabilities and the limitations of this powerful technology.

• 45:35Daniel Whitenack?

  It's a great point. I was reminded of, I bought, uh, my nieces and nephews, there's like this series of books called, uh, I think it's like Calculus and Other Things for Babies or Kids or something like that.

• 45:47Speaker 00

  Oh.

• 45:48Daniel Whitenack?

  I was like, oh, we need some like AI for babies books, uh, to, you know, like, you can put in along with the other children's books. Anyway, that my mind was going there while you were talking.

• 45:59Chris Benson?

  What was your third point there?

• 46:01Speaker 00

  Oh, the third, the third area, you know, these are just off, off the top of my head here, is the impact that AI is going to have on education itself. You know, as, as we transition to more online learning and you have things like Khan Academy and other services that have, that have really packaged up some very high quality lectures on and and learning materials on all of the core subjects. The ability of computers using AI and and other data science techniques to observe how students are learning and to understand what is the

• 46:42Speaker 00

  learning style, right? Because we all have different types and and different extents of different flavors of learning styles. Some people like to learn more visually, some people to learn more by listening. I happen to be someone who learns, maybe you can tell by talking.

• 46:59Speaker 00

  And kind of repeating back what I have understood in order to get confirmation that I've understood it correctly. These different learning styles are things that can be observed and then the students can be offered different forms or different formats of the material that helps them learn more effectively. And so that can, that can take the shape of, you know, think of it as a almost like a choose your own adventure type of storybook, if you, if you remember those. Where you're offered different kinds of learning experiences to learn the same materials based on how you have learned best in the past. And there's always going to be some experimentation and

• 47:41Speaker 00

  you know, we're going to have to make some smart decisions around, you know, maybe some students are better auditory learners. They learn by listening. But it's really important that they develop their visual analytical and comprehensive skills as well. And, and so we'll need to, to balance that out in the types and formats of learning materials that are offered to them. This is an area that our deep learning institute team is really interested in, right? They want to get all meta and apply deep learning to deep learning education.

• 48:12Daniel Whitenack?

  Yeah, that's so cool. And I, I, I just get the sense from, um, you know, how you described that and, and your passion around it. You seem fairly optimistic, um, in terms of AI's benefit to, to certain areas, like in terms of AI for good in, in education and other areas. It's something definitely Chris and I are, are very passionate about. As you look to the future, is that true? Are you sort of generally optimistic about, you know, AI's uh, ability to, to impact our, our world, world for the better and different areas like education or, or health and that sort of thing? Maybe what's a one other area that, that you're particularly, as, as we kind of close out, what's a, what's another area where you're particularly excited about the potential?

• 48:57Daniel Whitenack?

  Um, you know, impacts, uh, for AI over the next couple of years?

• 49:01Speaker 00

  You know, I, in general, yes, I am, I'm an optimist. I'm, I'm optimistic that we as humans.

• 49:09Speaker 00

  Will learn how to use AI technologies because it's just a tool, right? It doesn't actually think for itself. It it's still a computer that does what we design it to do. Now, we may be designing it at a little further level of abstraction than we have with with previous technologies. But it's it doesn't have its own agency to decide what to do. And so I am optimistic that we as humans will continue to learn and apply AI in in really positive ways. And also learn how to mitigate some of the potentially less positive ways in which this this tool, this technology could be applied. One of the areas that I'm most excited about really is in healthcare, right?

• 49:53Speaker 00

  If you look at the the work that is being done, even just around COVID nineteen research and the speed at which the computational simulations are allowing scientists and researchers to very rapidly explore the problem space, to test out their hypotheses in effectively zero risk scenarios before they go start testing these these drug molecules and treatments in a a lab environment. It's just really, really incredible. And I think that what we learn through this incredibly focused period of time on COVID nineteen

• 50:34Speaker 00

  is going to turbocharge the entire kind of healthcare drug discovery area as a result. So I have a lot of hope, a lot of optimism around the effective application of this deep learning AI approach in in healthcare and and a number of other fields.

• 50:57Chris Benson?

  Fantastic, very inspiring. Um, as we finish up, I actually want to note to listeners that we have had several episodes devoted entirely to Nvidia and obviously Nvidia comes up in many episodes as part of our casual conversation. Episode fifteen was artificial intelligence at Nvidia with Bill Dally, Nvidia's chief scientist.

• 51:19Chris Benson?

  Episode 36, we had Anima Anankumar, who is the Director of ML Research at Nvidia. And then more recently, on episode 90, Daniel and I were just alone, but we talked all about exploring Nvidia's AMPIER architecture and the A100 GPU. Just wanted to let listeners know that. You may want, if you're interested in this conversation and haven't heard one of those, you probably want to go back and listen to get a lot more about Nvidia. Wanted to remind listeners that we have a code. If you go to our show notes, which you can get to on whatever device you happen to be listening to, there is a discount code for the GPU Technology Conference. And with that, Will, thank you so much for coming on the show. This was a great conversation. We really enjoyed it.

• 52:03Speaker 00

  Well, thank you both for having me. This has been a lot of fun and I'd look forward to seeing you again in person someday or, you know, at least virtually like this sometime soon.

• 52:12Chris Benson?

  Absolutely, will do. Thank you.

• 52:18Speaker 03

  Comment on this and every episode of Practical AI on changelog.com. There's a discussion link in your show notes for easy click-ins. We would love to hear from you. We also have a free Slack community where these discussions take place. Join today at changelog.com/community. And don't forget to follow the show on Twitter. We are at Practical AI FM. Practical AI is hosted by Daniel Whitenack and Chris Benson. It's produced by me, Jared, and our music is provided by The Beat Freak, Breakmaster Cylinder. Shout out to all of our longtime sponsors. Thanks again to Fastly, Linode, and Rollbar for their continued support. That's all for now. We'll talk to you again next week.