Low code, no code, accelerated code, & failing code

In this Fully-Connected episode, Chris and Daniel discuss low code / no code development, GPU jargon, plus more data leakage issues. They also share some really cool new learning opportunities for leveling up your AI/ML game! Sponsors: Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! The Brave Browser https://brave.com/cha913 – Browse the web up to 8x faster than Chrome and Safari, block ads and trackers by default, and reward your favorite creators with the built-in Basic Attention Token https://basicattentiontoken.org. Download Brave https://brave.com/cha913 for free and give tipping a try right here on changelog.com https://changelog.com. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Linode https://linode.com/changelog – Get $100 in free credit to get started on Linode – Linode is our cloud of choice and the home of Changelog.com. Head to linode.com/changelog https://linode.com/changelog OR text CHANGELOG to 474747 to get instant access to that $100 in free credit. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Follow up content from Rajiv Shah: Running code and failing models https://www.datarobot.com/blog/running-code-and-failing-models/Rajiv’s previous episode https://changelog.com/practicalai/109Lambda Lab’s GPU benchmarks https://lambdalabs.com/gpu-benchmarksMachine Learning in Microsoft Excel https://venturebeat.com/2020/12/30/you-dont-code-do-machine-learning-straight-from-microsoft-excelDeep Learning at the Speed of Light https://spectrum.ieee.org/computing/software/deep-learning-at-the-speed-of-lightMLCommons and MLCube: Previous episode about MLCommons https://changelog.com/practicalai/119MLCube project https://github.com/mlcommons/mlcubeLearning Resources: Yann LeCun’s Deep Learning Course Is Now Free & Fully Online https://analyticsindiamag.com/yann-lecuns-deep-learning-course-is-now-free-fully-onlineTensorFlow Everywhere https://www.tensorflow.org/community/tf-everywhereUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Daniel Whitenack?

  You may be able to get more cards that are not linked with NVSwitch, but are like the later gen architectures.

• 0:09Chris Benson?

  Right.

• 0:09Daniel Whitenack?

  That will actually perform faster than last generation top tier cards that are connected with NVLink or NVSwitch.

• 0:18Chris Benson?

  That's interesting.

• 0:19Daniel Whitenack?

  Yeah, so it's not just a like, if you're doing multi GPUs, you need this special interlink infrastructure. You kind of have to also weigh in the generation of cards that you're looking at. Cause like, if you go down to one GPU on the Lambda benchmarks, you can see like that the 3090, which is a consumer card and is very cheap as far as car, you know, GPU cards go. Gets you halfway to the training on that throughput as the A100 that's listed there.

• 0:51Chris Benson?

  Really?

• 0:52Daniel Whitenack?

  And so like, that's pretty incredible, actually, right?

• 0:56Chris Benson?

  It is.

• 0:58Speaker 02

  Bama for Change Log is provided by Fastly. Learn more at fastly.com. Our feature flags are powered by LaunchDarkly. Check them out at launchdarkly.com. And we're hosted on Leno cloud servers. Get $100 in hosting credit at leno.com/changelog.

• 1:15Speaker 03

  Change Log++ is the best way for you to directly support Practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal and helps sustain our production of Practical AI into the future. Simply follow the Change Log++ link in your show notes or point your favorite web browser to changelog.com/++. Once again, that's changelog.com/++.

• 1:57Speaker 03

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.

• 2:24Daniel Whitenack?

  Welcome to another fully connected episode of Practical AI. This is where Chris and I keep you fully connected with everything that's happening in the AI community. We're going to take some time to discuss some of the latest AI news that we've run across, and we'll dig into some learning resources to help you level up your machine learning game. I'm joined, as always, by my co-host, Chris Benson, who is a Principal Emerging Technology Strategist at Lockheed Martin, and I'm Daniel Whiteknight, a data scientist at S I L International. How are you doing today, Chris?

• 2:58Chris Benson?

  I am doing very well. We are kind of surviving the last bits, I think, of winter as a winter storm blows across the nation.

• 3:06Daniel Whitenack?

  Yeah, it's a crazy time here in the US weather-wise. So for those, maybe, international listeners, you may have heard about it, may not have. There's been a lot of winter storms in the US lately. So for those listening in the future, we're in February of 2021. And so I know further south, some places in the US, they're getting like weird, you know, freezing rain and that sort of things.

• 3:35Chris Benson?

  Texas is having like the worst weather in decades right now.

• 3:39Daniel Whitenack?

  Yeah, so they don't have, and I mean, I get it, they don't have sort of any infrastructure in terms of like plow trucks and big stockpiles of ice or salt to put on the ice on the roads. So it's, yeah, it's just a giant disaster in there. It's so odd for that area that nothing is prepped, you know. People further up north, I mean, we got, so where I'm at, we got probably between a foot and a half of snow, which is a lot for here, but we also, you know, there's always snow, so we have snow plows and, you know, they were working long hours, but it wasn't like they didn't know, you know, it didn't have that stuff in place and. And all that.

• 4:21Chris Benson?

  And here in the usually warm south, we do not have snow on the ground, but we actually did see flurries.

• 4:28Daniel Whitenack?

  Wow.

• 4:28Chris Benson?

  There you go. And, and speaking of snowplows, you know, that's such a rare, I think I'm more likely to see a, a living dinosaur rumbling by than a snowplow. I think we can count the number of snowplows we have in our state on, on my two hands. So.

• 4:40Daniel Whitenack?

  Yeah.

• 4:41Chris Benson?

  Yeah. Not many.

• 4:42Daniel Whitenack?

  Yeah. And, uh, you know, it's, uh, I've got a few aches and pains right now from, from shoveling myself, but that was a good, uh, good workout, I guess.

• 4:53Chris Benson?

  Yeah. You know, you got the cap on, you know, I know listeners can't see, but got the winter cap on everything. You're looking, you're looking good and rugged there.

• 5:00Daniel Whitenack?

  Well, that's because I have, right now I have my recording set up in like the back area of our house, which is the furthest from the furnace. It's also really cold here. And so the hot air like generally doesn't make it to this side of the house.

• 5:14Chris Benson?

  Gotcha.

• 5:15Daniel Whitenack?

  So, yeah, it's one of, one of those things that's, uh, just a practicality.

• 5:20Chris Benson?

  Yeah. We, we're at least up in the forties, which while, while being fairly cold for us is, is downright tropical compared to the rest of the country.

• 5:29Daniel Whitenack?

  Sure, sure. Yeah, it is. Yeah. It's, uh, one interesting thing after another, but, um, things have been going pretty well work wise. Got a lot of interesting things to work on. And as a principal emerging technology strategist, which, you know, is always a great title to say.

• 5:47Chris Benson?

  It's a long title.

• 5:49Daniel Whitenack?

  It is a long title. I think we talked once that, uh, your title could be acronym P E T. Yeah.

• 5:57Chris Benson?

  That's right. And that, yeah, which is very fitting.

• 6:00Daniel Whitenack?

  Yeah. Yeah. Very fitting. So what emerging technologies are on your mind as of late?

• 6:05Chris Benson?

  Well, you know, there are actually a lot of really cool, you know, with, with the fact, you know, noting for people who may listen to this down the road, that we are still in the middle of a global pandemic.

• 6:15Daniel Whitenack?

  Right.

• 6:16Chris Benson?

  Though maybe on the tail end as vaccines are, are beginning to roll out, you know, there has been quite a lot of interesting research, uh, in the, in the medical space. And there's a lot of AI being used, uh, at this point, you know, we talked early on. About how there were some uses for AI for data sets regarding pandemic, but we haven't really addressed it in many, many months at this point.

• 6:38Daniel Whitenack?

  Yeah, that's true.

• 6:39Chris Benson?

  And since then, I've seen, I've seen quite a lot of deep learning being applied to various medical and health concerns, some pandemic related and some not. So it's really turned the corner in that space.

• 6:51Daniel Whitenack?

  Yeah, that's super interesting. Obviously, you know, there's going to be a lot of, a lot of applications in healthcare, both, you know, for new processes, new technologies, new applications of AI spurred on by this pandemic, which is sort of shown in a lot of ways how unprepared we are for, for certain, uh, certain scenarios. I'm sure that will prompt, that pain will prompt a lot of people to innovate. And it's always an interesting one for AI in the healthcare space because of course there's all sorts of privacy concerns with data and, um, you know, interesting scenarios

• 7:32Daniel Whitenack?

  where, uh, you have ideas about how to apply AI maybe, um, you know, to detecting, uh, certain patterns in medical imagery in lungs to, you know, detect COVID related, uh, outcomes or something. But how are you going to get that imagery to, to develop those models?

• 7:55Chris Benson?

  That's a good point. And we had an episode very recently having to do with radiology and imagery. Um, so I would refer people back to that. I'll get the specific episode number, uh, and put it in the show notes. So it's only been a couple back. And then today, as we address some of the things, um, for those who have not listened to Fully Connected, it's Daniel and I, we don't have a guest. We're just talking about things that we've noticed in the news, kind of catching up on this. So it's a, it's a very organic conversation.

• 8:21Daniel Whitenack?

  Yeah, yeah.

• 8:21Chris Benson?

  And one of the things I was going to mention was I, I saw an article having to do with, um, with CAT scans, um, and the fact that there are, without getting deeply into the medical side of it, there are different grades of CAT scans, if you will, with different, uh, different utilities associated with that and different costs. I noticed that there was an article about using deep learning models to take certain types of images and increase their fidelity so that they're more useful in that thing without, you know, kind of diving right into the topic here.

• 8:52Daniel Whitenack?

  So basically, you're saying you take the cheap CAT scan and make it a expensive CAT scan?

• 8:57Chris Benson?

  To some degree, yeah.

• 8:58Daniel Whitenack?

  Sort of is the idea.

• 9:00Chris Benson?

  Yeah, I think that is more or less along the right ways. I'm going to pull it up. They actually took ResNet and they're transforming a, and I am not a medical person. So I'm just kind of reading this little blurb off, but a clinical, single-spectrum CT image to VM counterparts. And as I understand it, the VM part, hopefully, I know we have some, some doctors in the audience and they can, they can offer some, some comments after the fact. But the VM refers to virtual monoenergetic.

• 9:30Daniel Whitenack?

  Yeah, I have no idea what that means.

• 9:31Chris Benson?

  I don't, as I read the article, I was just kind of like, you know, apparently they are able to do some enhancement with this at different energy levels and stuff. And, and I don't know that that's revolutionary in today's, because we've, we've talked about imagery a lot. But I think it's really notable that when, you know, a while back when we were talking about these things, as we first saw them, they were very cutting edge. And now it's becoming almost commonplace. And I think that's the thing going into this episode that I was thinking about was, was just how incredibly commonplace deep learning model deployments have become.

• 10:07Daniel Whitenack?

  There is sort of a general trend of utilizing lots of low fidelity or lower resolution or noise-filled data. So using lots of that data to get really good results versus using small amounts of really high resolution, high fidelity data. I was just hearing on the radio, one of the shows I was listening to, I think, I think the, the company or the technology they were talking about was persistent surveillance, which this is a more creepy sort of side of this, but essentially they're just saying, well, you know, we're not going to surveil

• 10:49Daniel Whitenack?

  like certain areas in front of a store or somewhere that are maybe like where crime is going to happen. Because basically if you put cameras around that have good resolution, you're kind of expecting to, you know, you're going to put them in the best spots. Right? Because that equipment is expensive. So they're on the whole other side of things where they're basically saying, well, let's just fly a drone around constantly.

• 11:15Daniel Whitenack?

  In like over some city and basically have a whole bunch of low resolution video of essentially everywhere. So they're not capturing things with enough resolution. And I think what they're saying is they're not capturing things with enough resolution to like identify people in, you know, the video. So they can't like say this is you or whatever, although they probably can. And like in certain ways, there's always ways.

• 11:44Chris Benson?

  I have no comment.

• 11:45Daniel Whitenack?

  But, uh, you know, that's what they're saying. There's a lot of noise. It's low resolution. So they're saying, you know, we're not like tracking your face or like your identity. But if something were to happen, like, let's say there was a shooting in a certain area or something, they could basically replay to that specific area and like see the little blobs of people moving around and like, you know, say, oh, no, this guy did come out of this door and then this happened and whatever. Of course, that's like alarming and interesting in so many different ways that we probably don't have time to talk about it. But it's also like an example, like where the mindset was typically like, I just need to get better and better resolution surveillance cameras or something

• 12:30Daniel Whitenack?

  to be able to really capture everything, every detail in my surrounding. They're sort of like flipping that and saying, no, just get really low resolution imagery, but a whole bunch of it.

• 12:43Chris Benson?

  It's the many imperfect considerations that go into the, the entire field now of AI ethics.

• 12:50Daniel Whitenack?

  Yeah.

• 12:50Chris Benson?

  And it's funny, I've seen so many articles and for a while, as listeners may know, I was working directly in that field and I no longer am in a direct way, but I still follow. And it's so complicated in terms of there are so many use cases and you're trying to figure out and, you know, people will look at something that's a little bit more clear cut, like the Chinese system.

• 13:09Daniel Whitenack?

  The social credit system.

• 13:11Chris Benson?

  The social credit system where they're actively monitoring citizens or anyone for that matter. And, and actively recognizing who they are and all that. And so we, you know, it is, it is a level of surveillance that most people in the West are not at all comfortable with. And nor, nor would I be, nor would you be from previous conversations. And, and, um, but, you know, there's clearly benefits to doing such surveillance. So, you know, how much is enough and too much?

• 13:40Daniel Whitenack?

  Yeah, I think that's the thing, like the. Benefit that could be derived from the technology is not the only motivating factor. Just because something can provide beneficial results doesn't mean that it is beneficial as a whole.

• 13:57Chris Benson?

  Exactly.

• 13:58Daniel Whitenack?

  I think that's the hard thing to wrestle with, right?

• 14:01Chris Benson?

  It's a really hard problem and as companies are trying to wrestle with it, not just surveillance, but you know, use of AI across the board, use of deep learning, I should say, across the board. The question is who's responsible for what? What is it due for liability? You know, we've, we've had whole episodes talking about some of this and it is not going to be solved anytime soon.

• 14:19Daniel Whitenack?

  Yeah.

• 14:20Chris Benson?

  There are no right and wrong answers explicitly, except on the extremes. It's something people will contend with it. It's, and as periodically it makes for a news story that we get to talk about when something goes awry. Um, and I think we're going to see more and more of those over the years.

• 14:35Daniel Whitenack?

  And it's not only sort of privacy and security, that sort of thing. I mean, automation is another big one that obviously has benefits, right?

• 14:46Chris Benson?

  Sure.

• 14:46Daniel Whitenack?

  But has a lot of costs as well. Um, so thinking about, and we've talked about this balance before on our show of, you know, automating away jobs versus sort of creating new types of jobs or, you know, other things like that. We have a couple of episodes on, on manufacturing things and, and all that.

• 15:07Chris Benson?

  Yeah, we have. I think I've said this before on the show. I'm sure that I've said it to you between us. And that is the fact that I think as you look at all of the various types of industrial revolution that has happened over time, you know, and moving from the, the horse and buggy into the automobile and, you know, the airplane and all the various things along the way. Each one of those has taken away some opportunities. It has also created new opportunities that nobody had envisioned to handle the new thing. I think the unanswered thing on automation at this point is with it being cognitive in nature and being combined with robotics that can do things that do not require great cognitive overload that, you know,

• 15:49Chris Benson?

  the question is how many new things are created for those that are taken away? As we really work into that over time, I think that's one of the great challenges of this century going forward. It's one of those fundamental things that we're going to have to do some restructuring to work our way through.

• 16:16Speaker 03

  We deserve a better internet and the Brave team has the recipe for bringing it to us. Start with Google Chrome, keep the extensions, the dev tools and the rendering engine that make Chrome great. Rip out the Google bits, we don't need them. Mix in ad and tracker blocking by default, quick access to the Tor network for true private browsing, and an opt-in reward system so you can get paid to view privacy respecting ads. Then turn around and use those rewards to support your favorite web creators like us. Download Brave today using the link in the show notes and give tipping a try on change.org.

• 16:57Daniel Whitenack?

  So Chris, I had an interesting email the other day. Actually, our friend Rajiv Shah, who was a guest on the episode 109. He came on the show and talked a lot about data leakage, which was a really great show just discussing like scenarios that can go wrong in your experiments where some of your target data leaks back into your training and you know, things behave much better than you think they might. It's often hard to prevent this in a lot of scenarios is basically what he's saying. You have to be extremely careful about this and you know, check yourself constantly.

• 17:38Daniel Whitenack?

  So he emailed me, he wrote this new article on Data Robot's website blog called Running Code and Failing Models. And one interesting thing that I think he's drawing out is that just because you release your source code and people are able to rerun it, doesn't mean that you kind of have a successful or validated model, which is interesting because a lot of the code that I try to run that people release, I can't get to run. So that's like a first thing because I think there's this temptation.

• 18:15Chris Benson?

  That happens, I think, to all of us.

• 18:17Daniel Whitenack?

  Yeah, they're like, I was thinking about this while I was reading the article. Like, there's this temptation, like, you know, there's eight different implementations of this thing I want to do. And I start trying out one, the first one doesn't work, the code doesn't run. The second one, it seems weird and it doesn't run. And the third one, and I eventually I find that one and it's like, this runs all the way to the end. You know, I can get my model out. I get that, like, evaluation number, you know, maybe I have to tweak a few things and whatever. But I'm so happy that basically I'm like, this ran and it produced a result. Like, I win.

• 18:57Daniel Whitenack?

  Sometimes that's how I feel because it's such a process to like, get any, like, state of the art model implementation to like, run sometimes, which is great. One of the reasons why it's great that like, things like transformers from Hugging Face or like, other like, packages out there have such a great like, model zoo or other things that are validated and will run. Because sometimes it's just so hard to get these implementations to run.

• 19:24Chris Benson?

  So as an aside, so I'm going to pull you for a second out of this particular article. And I just, what do you think is really caused, I mean, I've run into that too. I think everybody runs into that problem. What do you think is, is it just sloppiness on putting, you know, the solution together?

• 19:40Daniel Whitenack?

  No, no, I don't think it's intentional in a lot of cases. I think it's a combination of things. One of the things is people version control the code they use to do something for their own internal purposes or their own research project. And they actually don't intend to release that thinking that people are going to try to reproduce what they're doing. And so they just sort of, as a result of that, they make certain assumptions. Like, oh, yeah, like, I know I have that always installed, or, you know, I'm running on this type of environment, or whatever. And that's just sort of assumed knowledge on their part. And it's not like malicious, like, trying to keep people from reproducing what they're

• 20:24Daniel Whitenack?

  doing. It's just they're released that because that's how they're managing their code. It's open on GitHub because they have a personal account, or they want it out there for whatever reason. So I think that's thing number one. Thing number two is, it's just, you know, really hard with all of this technology constantly updating and being interconnected in so many weird and different ways. People will have like a requirements file or something like that, that has certain packages or something like that with versions. But they sort of miss a certain dependency, or they don't provide that version number or something. And, you know, that thing makes all the difference and breaks things in weird ways. And it's

• 21:09Daniel Whitenack?

  just hard. When you're trying to make things reproducible, it's actually very difficult.

• 21:14Chris Benson?

  So this is something, though, that in the software world, we have tackled and done quite a good job in that, you know, it is now typical to pass work around like that through containers. Is there an argument to be made that data scientists and deep learning engineers and other practitioners in the field maybe should ramp up on containers and pass these around? Because that way you're passing, you know, a unit of work, if you will, that has all the dependencies built in?

• 21:42Daniel Whitenack?

  Yeah, I mean, all of our projects internally that I work on, we containerize everything with Docker, and that's how we run things. Even our

• 21:50Chris Benson?

  Me too.

• 21:51Daniel Whitenack?

  Jupiter servers, or whatever we're running, it's all run within Docker based off of, you know, also based off of supported, oftentimes supported images from Nvidia or the TensorFlow team. Or, you know, images that are supported and have documentation. You can get these from Docker Hub under the official and verified, you know, links of TensorFlow, TensorFlow, for example, or you could get some from like NGC from Nvidia has a lot of really great optimized containers. So yeah, we use that. It's probably not the only way, though. I certainly know that there's a lot of data science people, AI people, researchers that,

• 22:34Daniel Whitenack?

  you know, Docker, they just don't like it. For whatever reason, they can't get the workflow that they like. So there's other efforts too. I know we had a conversation about ML comments recently, and they've got this project ML Cube, which we'll link in the show notes. And that's part of the goal of that project as well as is solving some of these issues.

• 22:56Chris Benson?

  Gotcha. And I had pulled you away from Rajiv's thing. And I was just wondering if what the thesis was from his perspective. Yeah. Sorry for pulling you away from that. That was a longer tangent than I expected.

• 23:05Daniel Whitenack?

  Yeah, no, but the, I mean, his conclusion, he says, you know, sharing code isn't enough to validate models. So if you are kind of lucky in one of those cases, and let's say they've provided the Docker image or something, and you can reproduce their exact training run and do the exact thing that they did. Basically, his comment then is that, well, that doesn't mean that the model is validated that comes out of that process, because there's this whole other thing called data that is at play here. And what he shows, interestingly enough, in the article is that one thing could happen is that the code that's part of what just ran,

• 23:46Daniel Whitenack?

  you got to run, actually creates some data leakage. So he found certain examples of this. And again, I don't think these are honest, you know, mistakes. And like, I'm sure there's a lot of these around. He just found some like in some data science or AI textbook or

• 24:05Daniel Whitenack?

  Books and examples, but he also, I think the one that struck me more was this Sarkos dataset, which is a shared task dataset based on predicting the movements of a robotic arm. And he said that he basically found that the holdout dataset was straight up built out of the training dataset. So there was overlap of like, you know, 4,500 examples in the test set that are also present in the training set. And so yeah, you're gonna get some really interesting behavior of your models there. So like, it just, I think, goes to show like, you can obviously get your data from a trusted source. You're not going to be able to reproduce everything. You know, you're always going to be standing

• 24:50Daniel Whitenack?

  on the shoulders of other people, but you've also got to assume that those shoulders that you're standing on, they're also fallible people, right? So there's definitely stuff that could be bubbling up from those things that you're building your project on that have a little bit of issue in them as well, whether that be a shared task dataset, whether that be a library that you're using, or an implementation that you're referencing, or whatever it is.

• 25:18Chris Benson?

  Gotcha. Oh, that's interesting. One of the things is you talk about kind of the fallibility of, you know, people not doing that. There's a lot of manual stuff unless you're going all the way to containers or some other tool that kind of helps catch you up. There's been a couple of things that I've seen that are almost the other end of the spectrum on that. And that is trying to do machine learning without code, you know, to where you don't have all that.

• 25:43Daniel Whitenack?

  Yeah, there's definitely this like jargon of low no code.

• 25:47Chris Benson?

  Yes.

• 25:47Daniel Whitenack?

  Sort of deal, right?

• 25:48Chris Benson?

  And that's really taken off lately, I've noticed. I've seen a whole bunch of different things, both on the software side and now people are, I've seen multiple articles about Excel for doing machine learning and stuff.

• 26:00Daniel Whitenack?

  Things come full circle.

• 26:02Chris Benson?

  It does. And I will, I will reveal my bias to say, you know, the first time or two, I was seeing the Excel articles, I was kind of shrugging it off entirely, but they seem to be, depending on how you want to define. Find, you know, machine learning and what level and such as that. It's catching on a little bit. There's a strong interest in, just as there is in the software development field. I had a conversation just a couple of days ago with a good friend who was talking about all the no-code approaches. Now, and this is a programmer, someone who programs in many languages, talking about all the no-code solutions and that some of them are quite good. So having seen some of these Excel, promoting Excel as machine learning tools, what do you think?

• 26:43Chris Benson?

  Is that something that would ever Daniel Whitenack enter into your mind to get some work done?

• 26:49Daniel Whitenack?

  Well, first of all, I don't have access to Excel at all. So I probably wouldn't use Excel just because I don't pay for it.

• 27:00Chris Benson?

  Yep, got it.

• 27:01Daniel Whitenack?

  I mean, I don't know if Google Sheets or whatever, maybe there's some of those things. There was probably some of those things there as well. But it's interesting that you bring this up because even just last week, I had a Slack message from one of my coworkers who is not a, who's a product manager, not a programmer, developer type person.

• 27:24Chris Benson?

  Sure.

• 27:25Daniel Whitenack?

  But tries to keep up with various things and she sent me a message just asking, hey, what is this like low-code, no-code thing? And how does it relate to what we're doing? And to be honest, I didn't know how to answer because I don't know if I fully explored the low-code, no-code things. The one thing that I had exposure to is Alterix Self-Service Analytics, Data Science and something. Yeah. So they, they say, I'm looking at their website right now. They talk about simply powerful, extraordinarily easy automated analytics of every kind,

• 28:09Daniel Whitenack?

  data science and AI without coding, data quality in minutes. So I will say that I worked, I did some advising and consulting with. A team last year and for some of their data pipelining, they use this tool. And I thought it was really cool, actually.

• 28:25Chris Benson?

  Is it open source or is it commercial?

• 28:26Daniel Whitenack?

  No, it's commercial and I don't think it's cheap by any means, but it seemed to be very powerful for the things that I think it fits a certain group, right? A group that is doing sort of this data pipelining, wanting to use reusable components that other people have developed to do tasks that they can define the tasks, they can define the business logic, but they don't know how to build those components or manage them in a robust way. And so they were sort of able to drag in like, oh, here I'm gonna like do this. So like my data comes in and then they drag in a little box that does some type of preprocessing on that data. And then they drag in a little box that creates a random forest model or something like that

• 29:10Daniel Whitenack?

  and does some classification, predicts some labels. And then they drag in another box that like egresses that data to some database or something. So is all like for at that level, it seemed quite useful. I can't speak to like the flexibility of it. Like, what if I have a set of data that's like very different from what they normally deal with?

• 29:34Chris Benson?

  Sure.

• 29:34Daniel Whitenack?

  Like, how many of those reusable components can I use? How can I customize them? And I think there are ways to customize them and bring in your own Python code and all of that. But then the question becomes, well, if I'm doing that customization

• 29:50Daniel Whitenack?

  and not being able to reuse the reusable components, am I getting the full value out of the thing? So I don't know. That's a whole ramble of all, basically all I know about low code, no code, which isn't much.

• 30:03Chris Benson?

  So I'm going to make a bold prediction, my friend. I am predicting that deep learning eventually has many options in the low code, no code arena, and that some of them become quite good and that there are both commercial and open source. And there probably are already some that we're not aware of, that people are yelling at us as they listen to this, as they're running or whatever they're doing today.

• 30:27Daniel Whitenack?

  There's a lot. Yeah.

• 30:28Chris Benson?

  And so I think that we'll go that way because I think one of the interesting things that helps in this way is that we are also seeing, if you listen to the show a couple of years ago, we were talking about totally new form, you know, new algorithmic approaches to deep learning all the time. And right now there is a conversation within the community about, well, we were seeing kind of the same classifications of models that are used or architectures that were used over and over again. And I'm waiting for the next really big breakthrough. I'm looking forward to that, but it's been a little while. We're seeing a lot of evolution. And we can talk about it in a negative, but I will say it helps the tooling catch up to some degree so that people, as it gets commoditized, because that's what we are seeing, it was being a lot of productivity

• 31:11Chris Benson?

  on very diverse and massive commoditization of deep learning architectures into many, many use cases. And so the tooling will catch up. So the way I'm measuring that, I have my software development workflow, which is, I think, fairly mature, as is for other software developers. And then we're seeing our, our deep learning workflows have been maturing steadily as new things have come out the last couple of years. I'm looking forward to the day when those two are able to merge in the way that I want it to merge, meaning it is a sensible, comprehensive approach. So I'm, I'm excited about that. And if there is some level of no code, low code, built into that workflow, I'm okay with that. I'm not going to push it away.

• 31:51Daniel Whitenack?

  Well, Chris, I recently have been working on specking out and getting some quotes for new server for some of the research that's going on in our organization. And I was looking through all sorts of different options and I, you know, I talked to all sorts of different people. Thank you to, if there's anyone out there that helped me through that, you know, thank you. But after all of our conversations on the podcast, I thought like, yeah, I kind of like know enough to like pretty instantly figure out what we need.

• 32:29Daniel Whitenack?

  And then like I started looking at these different options and the thing that got me, so I knew a lot of the things that we needed. But then I had this thought of envisioning the multi GPU jobs that we are going to need to run. And looking at the different systems that were available. I started sort of questioning like, what sort of like machine do you actually need to run a GP, multi GPU job? Because there's all of these different like acronyms and things that are promoted, like the NVLink, NVSwitch from Nvidia. Sure.

• 33:09Daniel Whitenack?

  And there's these little NVLink bridges that bridge directly between GPU cards. So I was unclear, like I just kind of hit a roadblock and I was like, like, what do I actually need for the type of multi GPU jobs that we're, we're going to want to run, which are mainly like training, speech related models? I ran across this, so in doing that, Lambda Labs, which builds and sells GPU workstations and servers and pods, they have this really interesting set of benchmarks that they've done, which we'll link in the show notes. And they break out for both PyTorch and TensorFlow.

• 33:51Daniel Whitenack?

  And you can sort of, they have these interactive graphs, so you can have it a chart or a table. You can look at different metrics, different precision, different number of GPUs, all sorts of different models, like transformer models, ResNet, WaveGlow, Tachatron, BERT, LargeSquad, all sorts of different models. And see, for all of these different combinations of GPUs, the throughput numbers essentially was, is what I came down to. And one interesting thing, like if you go to four GPUs, you all of a sudden see some, some really interesting things, like for example, if I'm looking at, let's say, four

• 34:35Daniel Whitenack?

  GPUs, and I look at all models, I can see, okay, well, there's the A1, 4X A100, and one has PCIe, one has SX M4. The difference there being the way that the GPUs interconnect. And it, essentially, if you start looking through these, like if I go down to two then, and look at the numbers, some of them like... Say NVLink, some of them say PCIe explicitly. And after doing some exploring, I started to realize a lot of these numbers didn't have the NVLink or NVSwitch interconnect, which made me start to question like, well,

• 35:19Daniel Whitenack?

  do you really need NVSwitch to do multi GPU jobs? Like if I get a bunch of cards and then like they're not linked with NVLink, NVSwitch, can I just do nothing? Is it going to be absolutely terrible? And so what I ended up finding was, yeah, like you do get a slight boost with this NVLink, NVSwitch. But in a lot of cases, and even for some of these models, it wasn't significant at all. And, you know, in some cases it was significant. And yeah, I found that, I found that sort of interesting that like, yes, you do get a sort of performance hit, but, or, you know, if you don't have the interconnect. But in some cases, it's not that much because, you know,

• 36:03Daniel Whitenack?

  you're only updating those parameters between GPUs when you update your gradients, which isn't happening all the time. Right? So I just thought I'd share that while we were discussing things because I hadn't fully like got through that train of thought before I explored these things.

• 36:21Chris Benson?

  So clearly at the different levels that you can buy in in terms of hardware, you know, from, from Nvidia or other vendors, you know, clearly they're pushing NVLink and those interconnect technologies. What you're saying, if I'm understanding you is, is it really depends on the architecture that you're deploying and, and the data that you're using, the data set?

• 36:44Daniel Whitenack?

  Yeah, I think it does. So here's one way to put it. You may be able to get more cards that are not linked with NVSwitch, but are like the later gen architectures. That will actually perform faster than.

• 37:02Daniel Whitenack?

  Last generation, like top-tier cards that are connected with NVLink or NVSwitch. That's interesting. Yeah, so it's not just a like, if you're doing multi-GPUs, you need this special interlink infrastructure. You kind of have to also weigh in the generation of cards that you're looking at, because like if you go down to one, you know, one GPU on the Lambda benchmarks, you can see like that the 3090, which is a consumer card and is very cheap, like as far as car, you know, GPU cards go, gets you halfway to the training on, on that throughput

• 37:45Daniel Whitenack?

  in, as the A100 that's listed there. Really? And so like, that's pretty incredible, actually, right? It is. So if you then go to like, you know, two GPUs on the benchmark, for example, you could have two 3090s that are going to be more throughput on their, on their benchmarks than two RTX 8000s connected with NVLink. And those are pretty hefty cards, right? And they're connected with NVLink, but two 3090s that are just connected with PCIe,

• 38:24Daniel Whitenack?

  no NVLink, are going to outperform it in terms of this particular benchmark for cheaper, right? Using this, did you arrive at a spec for your own purposes? So we have a couple of different ones that, that we've specced out. The 3090s, they're consumer cards, right? So if you're getting like an enterprise server, they're likely not going to be specced out with an enterprise server. But yeah, we did come up with a solution, and I think our solution was to get one server that had

• 38:58Daniel Whitenack?

  A bunch of cards that weren't NVLink, didn't have the fancy NVSwitch because we were going to do a lot of single GPU jobs and get the later gen cards because they're going to be so fast for that stuff. And then just get, you know, a second server with maybe two of the later gen cards that are linked.

• 39:21Chris Benson?

  Gotcha. You split the difference sort of.

• 39:23Daniel Whitenack?

  So you kind of split the difference. Yeah. So

• 39:25Chris Benson?

  By use case.

• 39:25Daniel Whitenack?

  Because it's going to depend. It basically depends. Yeah, I found this all very interesting. And I don't know, maybe like going into all those details, no one, no one's listening is like, why is he talking about this? But it was very.

• 39:38Chris Benson?

  We had a whole episode about this not too far back.

• 39:41Daniel Whitenack?

  Yeah, it was, it was very surprising to me how much I thought I knew going into that versus what it actually sort of took to really understand what was the optimal setup for our team in terms of the generation of the card versus the linking and all of that stuff.

• 40:01Chris Benson?

  And we'll link it in the show notes, but I know, I don't have the episode number off the top of my head, but we did an episode where you, where you talked about your process of putting together your first server and all the decisions that you made. And so if this conversation is of any interest to someone listening, then they should definitely, if they haven't already listened to it, go back to that episode. It was excellent. And we'll put it in the show notes to give you a quick link to it. As we look forward, I ran into a IEEE Spectrum is a publication of IEEE and they had an article here called "Deep Learning at the Speed of Light," where they were talking about LightMatter, which is a company who is putting together, you know, photon-based computing, you know, using

• 40:46Chris Benson?

  light and its chips to do accelerated deep learning. And so kind of an interesting thought there. I, you know, I don't know where, where the competition is. I don't know where Nvidia is on that, but definitely was an interesting idea about the idea of, of doing that. Any, any thoughts of you, if you had an affordable chip that you could use for your purposes and there was advantage to it, would you consider doing a, a light-based solution for your computation?

• 41:12Daniel Whitenack?

  Yeah, of course. I mean, I could so nerd out with that. With my physics background, it's like, you know, oh, you're using Nvidia 3090. I'm using a Max Zender inpherometer.

• 41:27Chris Benson?

  I mean, who needs electricity when you got light?

• 41:29Daniel Whitenack?

  It just has a different ring to it.

• 41:30Chris Benson?

  Yeah, exactly. Totally. It'll be interesting to see if the market hits a point where that becomes a commonplace thing and not a novelty. I just thought it asked since I wasn't even going to bring that up, except that we were talking about about chips here.

• 41:46Daniel Whitenack?

  Infrastructure, yeah. Cool. Well, as we kind of get to the end here, we always like to share a share a couple of learning resources with people. And I know that you found one deep learning course that's online. Now, do you want to talk a little bit about that one?

• 42:01Chris Benson?

  Yeah. So one of the, I assume you're talking about Jan LaCoon's course?

• 42:06Daniel Whitenack?

  Yeah.

• 42:06Chris Benson?

  Yeah. Jan LaCoon, who is at Facebook, I forget his title, Chief AI or something. I don't have it in front of me. But he is also one of the legends of the field. So, you know, he is one of the top minds in the field. And he released a deep learning course that is free. And it's really had quite an uptake on it. It's called Deep Learning DSGA 1008. It's part of the New York University Center for Data Science, and it's accessible by all. And so by signing up for this course, which is released in the spring, so I think spring 2021, we'll be signing up shortly, if not already. But you get to learn from one of

• 42:50Chris Benson?

  the great minds of the field. And so we've seen a few other major figures in the field release their own courses. And we've taken some, but it definitely, the depth of somebody who is one of the leading thinkers on where the field is going, not just not just as a practitioner, but as a researcher. I don't know, it adds it adds a little zest to it. It's a course that I'm definitely going to look into myself, just to see what's on Jan's mind.

• 43:15Daniel Whitenack?

  Yeah, that's awesome. The one learning resource that I wanted to share was this sort of new format that the TensorFlow team is doing for their... In the past, they sort of did this TensorFlow Dev Summit, which is a great event. It was, you know, I also watched it livestreamed. I'm not sure how that'll all play out in what formats they'll eventually do, but this year, they're doing this thing called TensorFlow Everywhere. Yep, I noticed that. Yeah, we'll link it in the show notes. I think this is really cool, actually. So they essentially have TensorFlow Everywhere, which is a series of global events that TensorFlow is hosting with all sorts of different content. And I think it's really cool because they

• 43:58Daniel Whitenack?

  have events that are, you know, geared towards certain geographies. So the time zones match up well for people. It's not like people have to livestream at, you know, midnight or 1:00 a.m. or something like that, but also in a number of languages. So I see events in Bahasa Indonesian, Chinese, Korean, Vietnamese, Turkish, different languages. And so people are actually getting that content in a language they value most, at least a little bit more than they were before. You know, there's 7,000 languages in the world. There's only a handful here, but I think it's a great step in that direction as well. So I thought

• 44:39Daniel Whitenack?

  it was really cool, the way that TensorFlow is doing this. And you can go to the website, figure out which event is sort of in your geography and works for your, you know, timing. They're kind of going on, actually, right now. There's some that have already gone on, but then they're stretching out through March as well. And the ones in North America, which is where we are, is the 27th of February. So I'll probably be tuning into that one. But yeah, definitely check this out.

• 45:12Chris Benson?

  That sounds really good. You know, just as a, as kind of a closing comment on that, there are so many, obviously, very, very enormous negatives having to do with the pandemic. But one thing that that might be a small positive is the fact that there has been some interesting innovation with the whole world kept away from one another and us still having the need to have, you know, to, to have commerce, to conduct business, to see each other and to talk and communicate and learn. There's been quite a bit of innovation in terms of how we've approached it. So I'm curious, as we potentially are at the beginning of the end of the pandemic in terms of vaccine rollouts and, and maybe, maybe at some point in the months ahead, having the ability

• 45:54Chris Benson?

  for, for large numbers of people to return to, to normal life, how do you think that'll change? What do you think? You know, are we going back to, are we all just going to fall back into our old habits and go to the conferences? Do you think more things will stay online?

• 46:08Daniel Whitenack?

  I think there'll be quite a few people that will be eager to do that, but I'm also hoping that people sort of have found at least some ways to, you know, save on, on travel and those sorts of things. But also, I'm really hopeful that it, you know, ends up that there are lower cost and more inclusive ways of involving people, especially from, from Africa, from Asia, from Latin America, from other places in the events that, you know, were primarily sort of North American and Western European based. That's the part that I really hope survives is like enabling that, that access for people and also enabling those people to contribute,

• 46:53Daniel Whitenack?

  not just sort of be consumers, but actually realizing that they have valuable things to add into these events and giving them a way to do that. Those are the things that, that I'm hopeful will survive.

• 47:05Chris Benson?

  I'm 100% in agreement with you. And you cannot possibly have a better note to go out on. So I think we should stop right there.

• 47:12Daniel Whitenack?

  Yeah, sounds good. This has been fun, Chris. Thanks.

• 47:15Chris Benson?

  It has been a fun conversation.

• 47:20Speaker 03

  Thank you for listening to Practical AI. If this is your first time, make sure you subscribe so you don't miss a thing. Head to practicalai.fm to subscribe or find us in Apple Podcasts, Spotify, or wherever you listen to podcasts. And if you get value from the show, please do share it with a friend or a colleague. We appreciate you spreading the word. Practical AI is hosted by Daniel Whiteneck and Chris Benson. It's produced by Jared Santo and our music is provided by Breakmaster Cylinder. We are brought to you by some awesome sponsors. Shout out to Fastly, Linode, and Launch Darkly. That is our show. We hope you enjoyed it, and we'll talk to you again next week.