It's been a BIG week in AI news 🗞

This last week has been a big week for AI news. BigScience is training a huge language model (while the world watches), and NVIDIA announced their latest “Hopper” GPUs. Chris and Daniel discuss these and other topics on this fully connected episode! Sponsors: Fastly https://fastly.com/ – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipChangelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: BigScience Language Model Training Model Details https://bigscience.notion.site/BigScience-176B-Model-ad073ca07cdf479398d5f95d88e218c4Live model logs and training curves https://huggingface.co/bigscience/tr11-176B-ml-logs/tensorboard?tab=scalars&tagFilter=lm%20loss#tensorboardH100 from NVIDIA [White paper](https://www.nvidia.com/en-us/data-center/h100/ https://cf-store.widencdn.net/nvdam/8/4/a/84ae26c1-6774-4083-a41d-4f3d82a6a449.pdf)Omniverse platform https://developer.nvidia.com/nvidia-omniverse-platformEarth-2 Supercomputer https://blogs.nvidia.com/blog/2021/11/12/earth-2-supercomputer/Stanford’s AI Index 2022 https://aiindex.stanford.edu/report/ Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  AI-driven simulation is a huge thing and maybe not for the reasons that everyone is most often associating. It's it's less about leisure and recreation and such as that and more about having digital twins for manufacturing, design and things like that so that you're that you're able to try and experiment on things that would otherwise be too costly in the real world. That's the context is looking at Omniverse in the context of using GPUs so that we we can try some otherwise really hard things to do.

• 0:41Speaker 01

  Hello friends, Jared here to tell you about Change Log++ our membership program for those of you who want to directly support our work. Your++ membership gets you closer to the metal with extended episodes, makes the ads disappear, and takes our audio to the next level with higher bitrate MP3s. You can join today at changelog.com/plusplus.

• 1:14Speaker 01

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join us at practicalai.fm/community and follow the show on Twitter, we're at Practical AI FM. Thank you to our partners at Fastly for shipping our pods super fast all around the world. Check them out at fastly.com.

• 1:47Chris Benson?

  Welcome to another fully connected episode of the Practical AI Podcast. In these episodes, Chris and I keep you fully connected.

• 1:57Daniel Whitenack?

  With everything that's happening in the AI community, we'll take some time to discuss the latest AI news and we'll dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitek. I'm a data scientist with SIL International, and I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

• 2:19Chris Benson?

  I'm doing great, Daniel. Just been going through the news. There's lots of interesting stuff happening right now.

• 2:24Daniel Whitenack?

  You know, we have to keep doing these FC, fully connected, episodes because there is just a lot going on. There's a few big things that we'll talk about today, but yeah, it just seems like a lot going on. Lots of cool stuff to talk about. The other big thing in my life is allergy season is here. And so I've been like in Zoom calls, like muting myself and like trying to quickly turn off my video before I sneeze, so I don't look like a fool.

• 2:55Chris Benson?

  It's okay. It's springtime, man. All things are coming out and blooming and birthing and all sorts of stuff.

• 3:02Daniel Whitenack?

  Yeah, including new large language models.

• 3:06Chris Benson?

  Indeed.

• 3:07Daniel Whitenack?

  So, Chris, if you go to the Hugging Face website, just go to the main Hugging Face page, and then what you'll see is a bunch of things, but up in the top right-hand corner, you'll see a little line, and it's sort of going down and to the right. And then there's a thing that says "training loss" and a thing that says "time to completion."

• 3:34Chris Benson?

  Okay.

• 3:35Daniel Whitenack?

  And this is pretty cool because this is the actual training logs of the Big Science Large Language Model that they're training, the 176 billion parameter model, which the Big Science Research Workshop is training. So, I don't know if you remember, I think we talked about this a while back, but there's this group of researchers called the Big Science Workshop or Big Science Research Workshop. It's hundreds of researchers from around the world, and it's kind of the same philosophy of a You know, like CERN in Switzerland, the big accelerator is a collaboration between

• 4:18Daniel Whitenack?

  a whole bunch of different researchers, a whole bunch of different institutions and and countries. And the reason that happens is because it's become increasingly hard to do high-energy physics research because of the size of the experiments, right? And so Big Science was like, well, it's becoming increasingly hard to train these large language models, or maybe what some people would call foundation models. We talked about that last time.

• 4:43Chris Benson?

  We did.

• 4:43Daniel Whitenack?

  So it's becoming increasingly hard for any single researcher, or even a team of a small team of researchers to train these large language models. You might think of things previously like GPT-3 or things like that. So this Big Science workshop said, hey, let's get together this like international collaboration, very collaborative and transparent thing, and do the next training of the kind of next big multilingual language model. And so what you're seeing on that homepage of Hugging Face, at least now in March of 2022, is the actual live training of that, which is is a pretty cool thing to see.

• 5:25Chris Benson?

  It is indeed. And as we are recording here and I'm looking at their page, they are already, what, a dozen days into the current model training?

• 5:33Daniel Whitenack?

  Right. But it's only getting started because they estimate time to completion four months. So now note, if there's people interested in the specifics of this, this model is training, so it will be training for the next four months, and it's training on 416 A100 GPUs. So a single model training on over 400 A100 GPUs for greater than four months.

• 6:05Chris Benson?

  That's some serious model training right there.

• 6:08Daniel Whitenack?

  If there was some serious model training, that is definitely it.

• 6:12Chris Benson?

  Right.

• 6:13Daniel Whitenack?

  So we'll link to this in our in our show notes, but there is a bunch of, one of the things I really appreciate about this whole process is just the transparency and collaborative nature of this project. So you can go and look at the notes about what the model is. It has 160 or 176 billion parameters. And then you can look at the GPT-like architecture. And then it's trained in 46 languages on 341 or 342-ish billion tokens of text, which for those maybe that aren't

• 6:54Daniel Whitenack?

  familiar with the terminology, tokens here, well, depending on how you set up the tokenization, it might be a word or a like subword of text. So there's over 340 billion of those words with a vocabulary size of 250 plus thousand.

• 7:14Chris Benson?

  And the only thing that doesn't seem shocking on the big end of this is actually it's only one and a half terabytes of text data.

• 7:22Daniel Whitenack?

  Yeah, I mean, text data is small in general.

• 7:25Chris Benson?

  Yeah, I know.

• 7:25Daniel Whitenack?

  That's the one advantage of working with text data.

• 7:28Chris Benson?

  I know. I'm just saying, we're so, we're so busy talking about these massive numbers that terabytes are big, but I guess, you know, it says something about the model when it makes me think of 1.5 terabytes as a small thing.

• 7:39Daniel Whitenack?

  Yeah, yeah. Well, I mean, yeah, if you're doing speech or images or video or even dare say 3D data, I don't know if you ever watched Silicon Valley, but they, they try to compress like a 3D video file. That's like the thing that trips up their compression algorithm. That's why I always think of that. But uh.

• 8:01Chris Benson?

  Throwback there.

• 8:02Daniel Whitenack?

  Yeah. Now, I mean, the other thing like, this is incredibly impressive, just the whole process of putting the data set together. You can read more about that in the notes. But there, you know, it still only includes 46 languages of, you know, 7,000 plus spoken on earth. So that is, I mean, a drop in the bucket as far as languages. But if you consider this sort of like scale that they're, that they're approaching, the size of the data and the number of languages, it's still a hugely amazing progress in terms of multilingual models because GPT-3 or something like that, we're talking about, you know, just supporting English.

• 8:44Daniel Whitenack?

  Or other versions that might be trained in like a single large language like Mandarin or something like that. So 46 languages, pretty cool to see that them work on a model at this scale with more languages.

• 8:58Chris Benson?

  I have a sort of an aside off the main topic, but you you made me wonder.

• 9:02Daniel Whitenack?

  I will permit it.

• 9:03Chris Benson?

  I appreciate that. As someone looking at that long tail of 7,000 languages, do you think that these kind of mega models that we keep looking at, that they ever will eventually include? Or do you think they'll always be outside and there'll be other models that that are are catching the long tail? How do you think that will be parsed?

• 9:24Daniel Whitenack?

  Yeah, I guess in a very unsatisfying way, it's probably like a both and. I mean, I certainly hope that like as data sets become more available and we work with local language communities to have representation of their language within NLP data sets and other things. Certainly that will get better in terms of what's included in this sort of multilingual model. And I guess the other thing to keep in mind is if you have kind of representation of languages across language families, which this, you know, does have definitely more representation across language families, then it's easier to sort of fine

• 10:04Daniel Whitenack?

  tune or adapt that sort of model to something maybe in the same language family or a related language. So it's still, it's still helpful for even languages, maybe that aren't even included potentially. But yeah, I think that there will also be like a continued need for language specific models or maybe language family specific models. I think that that will always, always be a need that that will be there. In fact, I just saw in my recent Twitter feed, there's a new paper at ACL this year from Graham Newbig and company.

• 10:45Daniel Whitenack?

  And they're talking about the title of the paper is Expanding Pre-trained Models to Thousands More Languages Via Lexicon-Based Adaption. So I think that the work that their lab is doing is pretty extraordinary in in this field where. They are doing a lot of creative things that are maybe hybridized, like they're leveraging lexicons or dictionaries to adapt models or to align sort of cross-lingual embeddings or other things like that. And those sort of creative solutions are still going to be needed. So yeah, it'll be both and there'll be big models that will include more languages and they'll be more sort of like trickery

• 11:28Daniel Whitenack?

  in terms of adapting those models to local language scenarios. At least that's my prediction of the future, which is sure to be wrong.

• 11:37Chris Benson?

  So not one model to rule them all, maybe a group of of models that are...

• 11:42Daniel Whitenack?

  Yeah, probably that that's true. And it'll be a hybridized sort of world I think we'll live in. Now, I'm I'm kind of excited, Chris, because and I'm proud of the organization that I work for, S I L International, because they had different working groups that you could join as part of this process. And we were able to contribute some of the multilingual data that's actually in the in the training data set that's being trained right now. So it's just really cool to kind of come full circle where like, you know, at the time in that working group, it seems sort of just like a very, well, it is kind of a small piece of a much larger puzzle, but all of those pieces are necessary to make this happen,

• 12:24Daniel Whitenack?

  including those that worked on the architecture of the model and the scaling and the compute and the data governance and, you know, data storage and all of those things. You know, there are a lot of people that made it happen.

• 12:39Chris Benson?

  You're in danger of not just analyzing the news here, but becoming the news yourself, a little bit of it.

• 12:46Daniel Whitenack?

  Well, yeah, I don't know about that. Certainly, we're we're excited to be a part of this. And yeah, I'm just kind of feeling giddy and excited to watch it go down. You can actually, you know, if you're on a Twitter person, they have a Twitter bot where you can get updated with the progress of the training and other things. They have that set up, so, you know, follow it and see what happens.

• 13:09Chris Benson?

  So at the end of the workday... Is your wife, who is a wonderful person?

• 13:13Daniel Whitenack?

  I agree.

• 13:14Chris Benson?

  I know. Is she having to patiently tell you to put down your phone and stop looking at the Twitter feeds? And, you know, actually have dinner with her and things?

• 13:22Daniel Whitenack?

  Yeah, you know, it, there've been a lot of fun things to be a part of recently, including this that have come across our team. And yeah, it's, it's a fun time to be an AI person. But also, you know, you still need to have dinner with your wife.

• 13:37Chris Benson?

  You still need, you know what? That, that's like, yeah, that is the comment right there of the whole show. You still need to have dinner with your wife or your husband.

• 13:45Daniel Whitenack?

  Yeah, still need to maintain positive, positive vibes like the hugging face emoji would indicate.

• 13:51Chris Benson?

  Gotcha.

• 13:52Daniel Whitenack?

  Okay, Chris, well, we just talked about this huge training that's going on on over 400 A100 GPUs.

• 14:03Chris Benson?

  You know, that's exactly what I was thinking too, don't you?

• 14:06Daniel Whitenack?

  Yeah, yeah. So the A there stands for Ampere, which is a certain generation of the Nvidia platform of GPUs. And big news right now, I mean, we're in the midst of G, uh, GTC, wait, is that the right? Yeah, GTC season, um, which is, uh, when Nvidia announces things and there are some things.

• 14:32Chris Benson?

  There are indeed some things. So they have just announced their new Hopper architecture, which is, uh, based on the H100. We, so we've jumped from A to H for Hopper. The H100 Tensor Core GPU. And the thing that was jumping out, you know, while you were saying that, is that they're saying that it, you know, for large language models, it's, uh, it's 30 times faster for training than the previous architecture, the A100. So yes, I, that's, you know?

• 15:03Daniel Whitenack?

  Yeah, the big science training just started a little too soon.

• 15:07Chris Benson?

  It just started a little too soon. They needed to wait. They just needed to stand outside Nvidia headquarters with their hands open, asking for just a mere few dozen H100s, you know, instead of the hundreds that they that they otherwise have.

• 15:20Daniel Whitenack?

  You know, like every year, the architectures come out and it's always like 10x or 30x better or whatever it is, right? So like by the time they started this year with the H100, we'd be having the same conversation.

• 15:36Chris Benson?

  I have lots of friends at Nvidia and I'm just going to say some of them do actually listen to our podcast and I'm just going to say.

• 15:42Daniel Whitenack?

  Hello out there. Great work.

• 15:44Chris Benson?

  Yeah, great work, but there's a little bit of fatigue. Like you're just, last year's amazing thing, you're obliterating it 30 times. Come on, man.

• 15:52Daniel Whitenack?

  I already, I already bought the A100s. Like, do I get a refund now?

• 15:58Chris Benson?

  I need some relief. We need the A100 upgrade to the H architecture.

• 16:03Daniel Whitenack?

  Yeah.

• 16:03Chris Benson?

  Just to keep sane at this point.

• 16:06Daniel Whitenack?

  Yeah, I will say like the A100s as far as usage and our team uses those. We have the great privilege of having some to use. And I will say that they have been really wonderful in terms of our workflow. And one of the things that I was really, actually, if you want to know, I went to the H100 page. So I also listened to some of the keynote, GTC keynote announcement stuff. But I went to the H100 page and the very first thing I looked for was the letters MIG, MIG. Because that, for those that maybe aren't familiar with the most recent, the most recent architectures from Nvidia,

• 16:48Daniel Whitenack?

  they have this what's called MIG technology, which essentially lets you take your big GPU and then split it into smaller virtual GPUs, which can be used like in independent run, like training runs or whatever you're doing.

• 17:07Daniel Whitenack?

  And I have to say, we have actually liked loved that because we can have the big card, right? But most of our jobs don't require like every bit of the memory or every GPU we have. And so we have a lot of small jobs that we need to run, maybe like fine tuning language models. That's a typical thing we do, right? And that doesn't take like a long time. Our data is smaller because we're doing fine tuning or transfer learning. And so that's perfectly fine to run on like half of the A100 or, you know, maybe less of the A100, a third of the A100. And so I was really hoping when I went to the page, basically, I just wanted to see that

• 17:50Daniel Whitenack?

  that was still an option and they weren't going away from that, which it is. It seems like that's caught on. I don't know if you've you've used that at all, but.

• 17:58Chris Benson?

  Oh, we have, without going into specifics. And that was definitely, if I recall, that was introduced with the AMPYR architecture.

• 18:05Daniel Whitenack?

  Uh, yeah, it was not available before. I don't.

• 18:07Chris Benson?

  Yeah, it was introduced last year and that it's been one of those amazing utility things in terms of making what you buy a whole lot more useful for actual, you know, real life jobs for most of us. So I agree. Because at the end of the day, not everybody is producing the largest language model in the world. It doesn't happen for most of us.

• 18:28Daniel Whitenack?

  Yeah, yeah.

• 18:28Chris Benson?

  So, yeah, it was, it's been very useful in that way.

• 18:31Daniel Whitenack?

  Yeah, speak for yourself. I didn't, I didn't hit the train button and probably most people in the workshop don't even know I exist, but my data is there. I've got that.

• 18:42Chris Benson?

  Your data is in the, yes, there you go. It's training right now, man.

• 18:46Daniel Whitenack?

  So the other kind of interesting thing, flavor they added on to this was with the MEG technology, I think they've seen that catch on and they want to make use of it more in kind of the cloud enterprise scenario where you might want more isolation between the virtual GPUs. And so they're making this to where you can actually have it. You know, secure end-to-end and multi-tenant usage of the virtual GPUs. Now, for us, really, it's just about distributing our jobs. We don't have a lot of security concerns between our different users accessing the different queues we're running on. But I think that makes a lot of sense in terms of flexibility.

• 19:31Daniel Whitenack?

  And if you were lucky enough to have a whole bunch of these, utilizing them well and managing isolation of users and that sort of thing.

• 19:40Chris Benson?

  I mean, that's a great point about the fact that all of the cloud providers are buying a whole lot of these and they're trying to use them effectively within their architecture. And it's certainly a key use case for that technology. For our friends at Nvidia, I wasn't actually picking at you. I was kind of envious at the fact that you always pull out the superpowers and do these amazing jumps.

• 20:04Daniel Whitenack?

  Yeah, so if you still want to send us, yeah, if you happen to have a couple spare H100s around, you know, we're here.

• 20:13Chris Benson?

  I thought that was implied in what I was saying. I thought that was just there. And by the way, Daniel and I have two different addresses and we could provide both.

• 20:19Daniel Whitenack?

  Yes, yes. We have not been compensated at all, Nvidia, for this conversation, just to be clear.

• 20:25Chris Benson?

  Not at all.

• 20:26Daniel Whitenack?

  Yeah, no, I think that that will be true going forward. Like you mentioned, the only thing I'm wondering is, in the cloud, like when I'm selecting my GPU now, how will they be labeled? It's like, why use fractions, like half of an H100? Or are they just going to come up with new things? I don't know.

• 20:48Chris Benson?

  I'm sure that the cloud providers will be very happy to charge you for a full H100, though, regardless of whether you use a slice, complements of the multi-instance technology that they've now embedded here.

• 21:02Daniel Whitenack?

  It is interesting, as you mentioned, Chris, this focus on large language models with the new architecture. So in my understanding, they actually have a new set of tensor cores that are part of the processor that are specifically geared towards transformer architecture models. I mean, it makes sense that it would be a lot faster for transformers if you built something specific for transformers, right? That sort of makes sense.

• 21:34Chris Benson?

  Yep. They call it an engine. The new transformer engine.

• 21:38Daniel Whitenack?

  Vroom, vroom. I have like mixed feelings about this, I think. So first of all, it's great because I use a lot of transformer models. Like, I'm not complaining. But then I do wonder what the implications are in terms of like the research world in general. If, if the GPUs we're using have their own sort of bias towards a certain architecture type, does it sort of penalize people that are exploring new, different, crazy architecture types that might be the next big thing that comes along? It seems like, well, let's just keep exploring transformer and not do as much of that, which might be sort of...

• 22:19Daniel Whitenack?

  So there's less flexibility and there's a reason for it, but I see like, uh, that kind of have that concern a little bit.

• 22:25Chris Benson?

  My answer to you is gonna depress you.

• 22:27Daniel Whitenack?

  Oh, okay.

• 22:28Chris Benson?

  Yeah, my answer is, of course it does. I mean, there's, there's clearly, but that's also the marketplace. The number of true cutting-edge researchers compared to the 99.999% of people.

• 22:42Daniel Whitenack?

  That are just cranking inferences out of a...

• 22:44Chris Benson?

  That's right. Cranking inferences out all over the place, but not truly doing cutting-edge research. Yes, it's making, it's making life easier for the, uh, the 99.999%. They're not helping you if you happen to be in the pure research, which is funny because I, you know, to your point, that's what I want to see more of. I, I would rather, I would love to see, just while I'm doing my wishlist, since we're talking about it, we wandered into this, blundered into this. I would love to see some new architectures out there in the deep learning space. It's been a little while since we've had a major, major shift and I'm, I'm waiting for that. So, uh, that, that's my, it's on my Christmas list right there.

• 23:25Daniel Whitenack?

  Yeah. And if I'm just looking, so I'm, this is a lot of interpretation and sorry for whoever designed the page if this wasn't how it was supposed to be read. But I see kind of on the H100 page, in order of priority, kind of at the top is the transformer engine thing. And then right after that is real-time deep learning inference. So the speed-ups in inference, which like you're talking about, and I think has been mentioned by multiple guests on this show, right? The bulk of operations in kind of the enterprise AI world are inferences, right? Because you train a model and then you do it.

• 24:03Daniel Whitenack?

  You use it billions and billions of times. And so that I think is really cool that you can have more throughput on the inference side and be more efficient, maybe with less, you know, less cards. So yeah, I can see why that side of things is prioritized for sure.

• 24:23Chris Benson?

  Sure. Yeah, it's hitting most of their customers.

• 24:26Daniel Whitenack?

  Now, they also, they bring in the Omniverse to this. So I don't know if you saw that reference, but it's like, so obviously there's the Metaverse, which is for those, I assume a lot of people know what that is out there, but it's like the virtual reality world that Facebook or Meta is designing and running. I don't know. Have you been in the Metaverse, Chris?

• 24:51Chris Benson?

  I have not been in the Metaverse, but I am quite familiar with Omniverse.

• 24:55Daniel Whitenack?

  Yeah, so I've been in the Metaverse and I think it's cool that like there's this effort to develop like these 3D sort of simulations, 3D worlds, VR, all of that's cool. And maybe since you're more familiar with the Omniverse, you can talk about that. But I am not a computer graphics person, but obviously these are graphical processing units. So to support those 3D worlds, I think that is one of the things they're featuring as something that this could enable.

• 25:27Chris Benson?

  So it's funny, we're taking a little bit of a tangent here, but AI-driven simulation is a huge thing and maybe, maybe not for the reasons that everyone is most often associating. It's less about leisure and recreation and such as that, and more about having digital twins for manufacturing, design and things like that. So that you're, that you're able to, to try and experiment on things that would otherwise be too costly in the real world to experiment on. And so, so anyway, that's, yeah, that's the context is looking at Omniverse in the context of using GPUs so that we can, we can try some otherwise really hard things to do.

• 26:11Daniel Whitenack?

  Yeah, looking forward to, to seeing more of that cool 3D stuff, especially where it intersects AI and simulation.

• 26:20Daniel Whitenack?

  One more thing that just related to all of this Nvidia stuff that's happening. I don't know if you also saw as part of what was talked about this supercomputer that Nvidia is working to build Earth 2 supercomputer. Now, I don't think there was an Earth 1 supercomputer. I think it's Earth 2 supercomputer because it's

• 26:44Chris Benson?

  We're Earth 1.

• 26:45Daniel Whitenack?

  Literally, it's running a clone of Earth. It like there is a virtual Earth 2 that's in the computer. Is that how I'm understanding that?

• 26:56Chris Benson?

  I'm actually not super familiar with that, but I'm assuming so. I mean, it seems that that would be the natural, you know, thing to be working towards. So maybe this is it. That was the one thing I was not, I had not dived into before the show.

• 27:08Daniel Whitenack?

  Yeah, I think in my understanding, they've built this supercomputer, there is sort of a clone of Earth virtual world and they're doing weather and climate simulations that are not sort of like a random side topic. My first job out of college was at NCAR in Boulder, which is the National Center for Atmospheric Research, where they do have big supercomputers to do climate modeling. That's what I did that summer is benchmarking some of that software. But those climate models kind of operate around like fluid dynamics and other types of physics simulations

• 27:52Daniel Whitenack?

  and that sort of thing, physics and chemistry simulations. This, in my understanding, these weather predictions that they're doing in this Earth 2 supercomputer are driven by, at least partially by AI predictions of weather conditions and climate, which is kind of an interesting, it might be a subtle point, but it is very different than sort of how climate models have operated for quite some time.

• 28:19Chris Benson?

  So I'm actually somewhat familiar with that, at least lightly. It still is computational fluid dynamics. CFD is the is the short is the lingo that people use in the industry. And we are seeing and have been for the last few years and are continuing to see a very large shift from CPU driven CFD simulations into AI driven with GPUs. And and really the holdup there is usually the software being used as written for older CPU things. And so in general, industry is kind of trying to catch up with with the with the technology at this point in terms of updating their software to do that.

• 28:59Chris Benson?

  But yeah, I think that is the future of computational fluid dynamics is is this kind of simulation that's GPU driven.

• 29:06Daniel Whitenack?

  Yeah, and they specifically reference in the article physics informed neural networks, which I think that would be a great show in and of itself. So if you are out there and you're working on such things, reach out to us and we'd love to talk about it on the show.

• 29:22Chris Benson?

  That would be very interesting.

• 29:24Daniel Whitenack?

  Yeah, I would. I would be super interested to to hear more about that. Yeah, any any other things that stood out to you with this with this Nvidia stuff, Chris?

• 29:34Chris Benson?

  No, but I actually have, believe it or not, I actually have a CFD, a computational fluid dynamics project that I would like to try. So and it's relating to a real world thing. So the two second backstory.

• 29:47Daniel Whitenack?

  Give me the pitch. Why Combinator pitch?

• 29:49Chris Benson?

  As longtime listeners may know, I'm a private pilot and I fly in the Great Smoky Mountains on many weekends. And that's one of the few places where you have mountains that are high enough to create mountain waves, which are giant waves of atmosphere that hit the mountains and they go up and then they crash down. I actually had a very dangerous moment where I got hit by an invisible mountain wave, flipped my plane upside down and I had to recover over the peaks.

• 30:18Daniel Whitenack?

  I didn't even hear about this. You about died.

• 30:21Chris Benson?

  Guy, it was very close or ripping the wings off the plane kind of thing. So this happened. This happened earlier this year, early in 2022, but it. Made me keenly interested in the fluid dynamics of mountain waves. And so if anybody is looking for a CFD toy project and you want to look at that in the Smoky Mountains, you may save my life later. So that's, that's the pitch. If you like the podcast and you want to see it perpetuated, then you got to save my life so that I don't get driven into the ground by a mountain wave. How's that? Is that exciting enough for you, Daniel?

• 30:52Daniel Whitenack?

  If you could inference fast and you did get the lightbulb blinking, the warning that you were about to be overtaken by a mountain wave, would you have time to respond?

• 31:03Chris Benson?

  I don't know. I guess it depends on how fast the GPU can do this inference. I'm, I'm going to have to really dig into the hopper architecture to see if it's going to save my life or not.

• 31:13Daniel Whitenack?

  Yeah, maybe you can create a digital twin of yourself in the plane in the omniverse and run the simulation.

• 31:19Chris Benson?

  Oh, I'm frightened though, that if I do that and it's really, really good, I might just not need to fly anymore. And that, I don't know that that's a nonstarter right there. Did you see how I tied all those things together?

• 31:30Daniel Whitenack?

  I saw that.

• 31:31Chris Benson?

  Don't normally go together. There we go. Having fun today.

• 31:34Daniel Whitenack?

  Yeah, exactly. Speaking of tying all things together, we actually not that long ago in the, I think, fall of last year, we talked about a few things related to the AI index report, which comes out of Stanford Measures Trends and Artificial Intelligence. Actually, the next version of that is out. So because we talked about it recently, we won't spend a whole episode on it, but maybe it's worth bringing up here as we close out our Fully Connected episode, just to kind of skim the top takeaways. And we'll definitely include it in our show notes so people can look at it. I noticed that one of the top takeaways on theme is language models are more capable

• 32:18Daniel Whitenack?

  than ever, but are also more biased. That's one of their key takeaways. So big science, language models are going to be more capable, but. Maybe it's worth exploring the bias. I think that Research Workshop has done an amazing job at exploring the data and the bias. They even created their own tool to explore such things. But yeah, it's still a problem, right?

• 32:41Chris Benson?

  It's interesting. It is a problem. And I think that exploring the bias actually ties into another thing that we're seeing there, just to pull two together, and that's the rise of AI ethics everywhere. And we're also seeing they have a note on data, data, data. And maybe sometimes we need to, we need to go back and address some of that because bigger is not, bigger may be better in many cases, but not if it's bigger bias.

• 33:06Daniel Whitenack?

  Yeah. Also, notice a shout out to our friends over at ML Commons, David, who has been on the show a couple of times. They mentioned that ML perf task and benchmark related to AI performance in terms of training time and compute. Their big takeaway is AI becomes more affordable and higher performing. So I think and being the key word there. So they give this statistic that since 2018, the cost to train an image classification system has decreased by around 63%, while training times have improved by 94%. So, pretty

• 33:49Daniel Whitenack?

  cool. That's sort of like combination of more affordable and higher performing, which is, you know, they talk about driving widespread commercial adoption and blah, blah.

• 34:01Chris Benson?

  I mean, in all of this, it's just, the field is exploding in terms, you know, exploding in a positive way, not a negative explosion. In terms of, of that, there's so much money flowing into it. They are noting, you know, kind of driving that process, that private investment is soaring, but that investment is also becoming more and more concentrated as well. So you're seeing winners and losers also kind of coming into play there. And I frankly think a lot of these topics are interrelated. I think that comes back to things like, you know, the ethics of it and where you're going to place investment and what you're getting out of it. I really find this an interesting report.

• 34:42Chris Benson?

  I know that they always are, but I think the thing that I noticed this time as we've looked through it is we're seeing a lot of direct, kind of like one topic directly relates to another one that they have there, rather than being all à la carte topics. We definitely have some challenges in this space. I mean, it's like, it's overall really good news, you know, cheaper training, cheaper inference, more productive, it's going everywhere, but we're also seeing bias, we're seeing concentration of power. We're seeing, you know, the challenge of how to tackle AI. There's a whole industry around AI ethics now, which I'm supportive of, but also a little bit, I don't think they all always capture it very well.

• 35:25Chris Benson?

  In terms of getting to the, it's a business still. So nothing against business, just, just saying that we have some, we have some challenging things in front of us to solve.

• 35:33Daniel Whitenack?

  Yeah, no, definitely challenging things, but apparently one of the challenges is that's disappearing is access to robotic arms. I don't know if I was surprised, but it was like, oh, that made a top takeaway. Robotic arms becoming cheaper. Newsflash.

• 35:52Chris Benson?

  Literally the next thing I was going to bring up, and that is, is they are becoming cheaper. We're going to be seeing more and more, you know, for a while, our exposure to AI has been through kind of existing mechanisms, our laptops and, you know, things that have access to cloud providers and stuff like that. But if the robotics side is coming down in price so much, then I think more of us will start interacting with AI in new and novel ways that were not previously widely distributed. That will be another interesting change in how we live our lives. So it's really going to be pervasive.

• 36:29Daniel Whitenack?

  So where do you think I'm going to start seeing robot arms? Where am I going to start interacting with them, do you think?

• 36:35Chris Benson?

  Well, I just know that I am positive. They don't have arms on them, but we have these little robotic vacuum cleaners running around.

• 36:42Daniel Whitenack?

  Right, sure.

• 36:43Chris Benson?

  And we're starting to move into the thing where they're gonna have infertis.

• 36:46Daniel Whitenack?

  They're gonna be able to like, reach over and pick up my Coke can and toss it into the trash.

• 36:51Chris Benson?

  Yeah, I mean, I'm really thinking, was it Fred Flintstone that had the robot maid that moved around? Am I thinking the right cartoon?

• 36:58Daniel Whitenack?

  I don't know.

• 36:59Chris Benson?

  There was one, it had the old fashioned like 1950s-ish looking robot and she was running Rosie the Robot or something like that. And I'm just saying, you know, you know, it's taken another century almost, but maybe we're getting awfully close to, we're moving from the little disc robot vacuum cleaner scooting around our living rooms and maybe Rosie the Robot is right there. And I apologize for the implied sexism. I'm, I'm, I'm calling up a character I saw in the past in a more sexist TV world. It's not my own thing.

• 37:32Daniel Whitenack?

  Yeah.

• 37:32Chris Benson?

  That little moment of awkwardness right there as I realize.

• 37:35Daniel Whitenack?

  Right, as you realize that your language is biased, which is gonna be recorded and put in a language model, which will then be biased.

• 37:42Chris Benson?

  God, yeah. I was merely trying to recall a cartoon from my childhood and look what happened.

• 37:48Daniel Whitenack?

  Well, I mean, I'm sure that that cartoon is transcribed somewhere and probably sucked into Common Crawl from some crawling bot that is producing that in our data sets and.

• 38:02Chris Benson?

  It probably is. We're gonna have to just delete Rosie the Robot from the collective, you know, storage of, of all data out there. It's gone.

• 38:11Daniel Whitenack?

  Yeah, well, I would recommend, we're not gonna go over everything in this report for sure. We will link it in our show notes, the AI index report. It's always a good thing to look at. It includes a lot more information than we talked about, so take a look at it, you know, keep up with the latest trends. That's what we try to do here in a practical way. And speaking of practical things, we've highlighted quite a bit of Nvidia things this episode. So just remind people in terms of a learning resource, you can watch a lot of the GTC content online. So obviously, it is produced by Nvidia. So you're going to see the Nvidia perspective, but Nvidia is also dominant in the market. So you're

• 38:56Daniel Whitenack?

  going to hear a lot of really great things and lots of really good things to learn there. But they also have what's called the Nvidia Deep Learning Institute, which can help you with online courses in terms of using the latest GPUs and acceleration on GPUs and a whole bunch of things. So in terms of learning resources, I know I've mentioned that one before, but because it's GTC week, seems relevant to mention it again.

• 39:24Chris Benson?

  That's a good call out. And those Deep Learning Institute courses are quite good. I've, I've taken several of them myself and always enjoyed them. Once again, I'm not trying to promote their stuff, but I've just said I have found it useful.

• 39:34Daniel Whitenack?

  Yeah.

• 39:35Chris Benson?

  And they hit a wider range of topics of interest.

• 39:39Daniel Whitenack?

  It's okay to mention good things.

• 39:40Chris Benson?

  There you go.

• 39:41Daniel Whitenack?

  So I appreciated this conversation, Chris. It's, it's been fun. We'll, we'll link to everything in our show notes. Please find us online. You can engage with us if you go to changelog.com/community. You can find us in Slack or LinkedIn or Twitter. Engage with us. Let us know your thoughts on this various news stuff. And yeah, we'd love to hear from you. So thanks for the conversation, Chris. We'll talk to you next time.

• 40:10Chris Benson?

  Talk to you next time.

• 40:15Speaker 01

  All right. That is Practical AI for this week. If this is your first time listening, subscribe now at practicalai.fm or just search for Practical AI in your favorite podcast app. We're in there. And if you're a long time listener, please do share the show with your friends. It is the best way you can help Practical AI succeed. Thanks again to Fastly for shipping our shows super fast all around the world, to Brake Master Cylinder for the Beats, and to you for listening. We appreciate you. That's all for this week. We'll talk to you again next time.