CMU's AI pilot lands in the news 🗞

Daniel and Chris cover the AI news of the day in this wide-ranging discussion. They start with Truss from Baseten while addressing how to categorize AI infrastructure and tools. Then they move on to transformers (again!), and somehow arrive at an AI pilot model from CMU that can navigate crowded airspace (much to Chris’s delight). Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Truss on GitHub https://github.com/basetenlabs/trussCMU: AI Pilot Can Navigate Crowded Airspace https://www.cs.cmu.edu/news/2022/ai-pilotEmbarrassingly Parallel Training of Expert Language Models https://arxiv.org/abs/2208.03306Toucan: Learn español without even trying https://jointoucan.com3D Vision with Transformers: A Survey https://arxiv.org/pdf/2208.04309v1.pdfSphere: Natural Language Processing with Transformers https://www.getsphere.com/ml-engineering/natural-language-processing-with-transformers?source=Instructor-Socials-LinkedIn-80822-announcement-postSkyJack on GitHub https://github.com/samyk/skyjackNVIDIA AI Demos https://www.nvidia.com/en-us/research/ai-demosUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  I will say kind of obliquely in the military space, we're doing autonomous stuff as is reported in the general news all the time in terms of aircraft. And I think that's fairly well understood. In the civilian space, though, if you imagine in the future being on an autonomous airliner, they say that this model can safely avoid collisions, predict the intent of other aircraft, track those aircraft and coordinate with those aircraft's actions, communicate over the radio. It uses natural language processing to do that. It has a vision system that uses six cameras. So it's a pretty cool problem to solve. I know people shudder when I say this, but I think that it is not so far out that all of us will be getting on airliners

• 0:45Chris Benson?

  that are almost entirely automated. And so this is one of those big steps toward trying to do that.

• 1:02Speaker 01

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive and accessible to everyone. Subscribe now if you haven't already, head to practicalai.fm for all the ways. Special thanks to our partners at Fastly for delivering our shows super fast to wherever you listen. Check them out at fastly.com and to our friends at fly.io. We deploy our app servers close to our users and you can too. Learn more at fly.io.

• 1:38Daniel Whitenack?

  Welcome to another fully connected episode of Practical AI. This is where Chris and I keep you fully connected with everything that's happening in the AI community. We'll discuss a few things that are in the AI news and dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitenack. I'm a data scientist with SIL International. And I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris? Doing great, Daniel. How are you today? I am doing great. I don't know if our listeners listened to a couple of episodes ago.

• 2:17Daniel Whitenack?

  We had a bird family setting up camp on our deck at our apartment and having two eggs. So one of those eggs unfortunately didn't make it, but one turned into a bird. And that bird flew the coop and now they have decided to start another family on our porch. Because apparently it's a great place to start a family of doves, I guess.

• 2:43Chris Benson?

  It's the coop, the great hatchery.

• 2:45Daniel Whitenack?

  Yeah, so I have another chance. I never got to get out my like computer vision kit and cameras and monitoring last time around. So maybe I'll have another chance here because I suspect that the same pattern is repeating.

• 3:01Chris Benson?

  Gotcha. Looking forward to that. We need to post pictures of it, you know, or put some video up or something.

• 3:07Daniel Whitenack?

  If anybody has any suggestions out there for alerting or monitoring based on activities of doves in a nest, let me know and I can set up the Raspberry Pi and all that stuff on my deck.

• 3:21Chris Benson?

  Excellent.

• 3:24Daniel Whitenack?

  Chris, every once in a while, we get to do one of these shows where we bring out an assortment of topics that have caught our attention over the past couple of weeks. And I think it's a good time to do that because, you know, there's a lot, continues to be a lot coming out related to infrastructure and new models and new products and all sorts of things. So yeah, good, good time to do that. One of the first things I wanted to highlight, which came out from actually a company that was a guest on the podcast a while back, Base10, released a new open source project called Truss. So if you go to GitHub under Base10 Labs and then Truss,

• 4:09Daniel Whitenack?

  you'll find this project, which they kind of market as a seamless bridge from model development to model delivery and an open source standard for packaging models, built in any framework for sharing and deployment in any environment, local or production.

• 4:29Daniel Whitenack?

  So what are your thoughts when you see this, Chris? What comes to mind?

• 4:32Chris Benson?

  I love it. I think it's very much needed. I've been putting a lot of thought lately into the need to make all of this stuff that we talk about much easier for people to get into. And so I think trust is a fantastic way of kind of getting that going. Moving from environments that are already in, like Jupiter notebooks, out into production without having to go back to a web framework and do all that work and stuff. So that's good stuff.

• 4:59Daniel Whitenack?

  Yeah, and you know, they include emojis on their readme. So you know it's friendly and accessible.

• 5:06Chris Benson?

  Exactly.

• 5:08Daniel Whitenack?

  Yeah, I've actually, since we talked with Base10, I've used their product a little bit. And I know that I, just from looking at this, I'm assuming that they're sort of eating their own dog food. Because, you know, some of the convenience that's built in their product is, you know, not all of it, but some of it is released in this package, which is pretty cool. And kind of allows, so there's certainly a lot of frameworks out there to do model serving or deployment or like model registry sort of things. Some of them kind of assume that you have a bit of infrastructure chops.

• 5:50Daniel Whitenack?

  I think to start with, like, you know, maybe figuring out how to run something on Kubernetes or something like that. And that's a big step for, for a lot of people. So I think this, that it's really targeting, hey, you're running maybe something in a Jupiter notebook. How do I export this model and deliver it? I also think that, you know, some of these things around model deployment miss some key, key aspects. So like, trust talks about bundling secret management into the, and API keys into their deployments, which I think is really, really important.

• 6:32Daniel Whitenack?

  So like, it's not that difficult for people to figure out how to build like a Flask app with their model. But then figuring out like testing and deployment and API keys and securely managing like the API, that's a whole nother ballgame. Yeah, I'm really impressed that they've kind of built a lot of this, a lot of this in.

• 6:53Chris Benson?

  They've put a fantastic capability out in trust in terms of of being able to address that. It's something I've been thinking about. I've, right now, is a slight sideline on this that that is relevant, I think. I'm doing something, I'm learning something new through Beginner's Eyes, because, you know, both you and I program in several languages. And we came together originally in the in the Go programming community, which is how we got to know each other. And right now, for a different thing, I'm learning Rust. And so I've been diving into Rust, but it's forced me back into that beginner mindset. And I've brought that back into these other things that we talk about a lot lately. And I've been looking at a lot of the AI and deep learning kind of through that beginner's mindset.

• 7:36Chris Benson?

  And there's such a need. We're still leaving out a lot of people on these capabilities. And things like Trust are amazing solutions. I think I think we need others as well, just so that people with different levels of skill, different skill levels and such can can find it accessible. So Trust is one of, was one part of that solution, it looks like.

• 7:55Daniel Whitenack?

  Yeah. And just to give people like a, since you're listening, a sort of visual picture of what this might look like. If you're in sort of Python and you create a model, you can import the Trust package and then sort of use this make Trust command or method. Point to the directory where your, you know, where your model or where your code is. And then that will sort of serialize and package the model and freeze like dependencies within a Docker image and all of that, which can be complicated in and of itself. And then you can call that and or deploy it via a variety of ways.

• 8:39Daniel Whitenack?

  I mean, in in clouds and like really simple ways to run things like ECS or GCP Cloud Run. Of course, you could run it in Base10 as well in their own infrastructure. But it has that because it's sort of freezing all of these dependencies and your model package in Docker image, then you have the ability to kind of run this all sorts of places, whether that's local or in these cloud solutions or in Base10 or wherever. So yeah, I think it's pretty cool. And it's, yeah, an approachable way to get into this model delivery stuff for those that maybe are, you know, hitting that pain of, hey,

• 9:20Daniel Whitenack?

  I've got my model in a Jupiter notebook, but I don't know what to do next sort of situation.

• 9:26Chris Benson?

  So I have a random, I have a random question for you.

• 9:29Daniel Whitenack?

  Sure.

• 9:30Chris Benson?

  As we're looking at things like trust and recognizing that most of our community here is mainly Python oriented, you know, for the development stuff, do you think that anytime soon we will start expanding some of the development and then, you know, deployment and packaging tools into other languages? Or do you think that's likely? Or do you think we have a way to go before we get to something where we're starting to look at kind of a multi-language community rather than the Jupiter focus we've had for so many years?

• 9:58Daniel Whitenack?

  Yeah, it's an interesting question. I think there's certain sets of tooling, like I know that there's tooling now in Go where you can import like hugging face transformers models and such. And so there's, there's more interoperability. There's certainly a lot of ways to run models in various other systems, whether those be like, or languages like JavaScript or Go or Rust or other things. But that kind of like model development workflow.

• 10:30Chris Benson?

  I think so.

• 10:31Daniel Whitenack?

  In my mind, still seems very much Python focused. I don't see a lot of motion away from that. I do see certain, certain trends happening, like language things seem to be more focused on Python now in terms of the model development side and maybe like interoperability on the inference side. Are you seeing similar things?

• 10:57Chris Benson?

  Yeah, that seems that seems accurate to me as well. Yeah, I am. It's just as like going back to my my Rust learning experience as I'm having to delve into, you know, out of something I know well and into something that that that puts me back into first grade, so to speak. I've been thinking about the fact that we're still leaving behind communities of people. And I'm really curious to see what what other options, you know, some some different organizations or companies or just inventive individuals come up with to let us be more inclusive with people that are maybe not traditionally, you know, have this space accessible to them.

• 11:36Daniel Whitenack?

  Yeah, just out of curiosity, Chris, what has your experience been like with Rust?

• 11:42Chris Benson?

  Well, without without taking us too far off the main line of our of our topic area, just as a new thing, they take a different approach. I can it's it's one of those languages that always wins the most loved, you know, when people are rating their languages and stuff, and I can see why, but it is a substantial learning curve. And it has made me very empathetic to people who are having to deal with other learning curves, such as this one that we're talking about in general, because it unlike Go, which tends to be fairly small by design and kind of have one way of doing everything and does some stuff for you that's pretty nice. Rust kind of takes the opposite approach and gives you every possible option out there

• 12:25Chris Benson?

  to optimize what you're doing, which is important for certain use cases, including one that I'm working on. And so it's it's been interesting. It's put me back in the I'm going to take a big thing and learn it new. And it made me think about if you're not a Pythonista and super savvy in in the juxtaposition of where Python intersects with the deep learning world in general, then this is still we're still in a moment where where this really huge important field that's revolutionizing technology is still quite inaccessible to a lot of people, I think. And so it's just a reminder. That's why it's on the top of my mind today.

• 13:27Daniel Whitenack?

  Well, thinking about like this model delivery side of things, it gets me thinking also about how, you know, we've seen an increasing number of things in the news and in conversations about ML Ops and Git Ops and CICD impacting kind of AI and the ML world and people thinking more about this operations side of things. I wonder, you know, I still encounter a lot of AI practitioners or data scientists who are really kind of trying to get a grip on what is CICD and how does that side of automation

• 14:13Daniel Whitenack?

  or how should it or could impact their development workflows? Might be worth talking for a second about, you know, CICD and what that exactly means and how it might impact practitioners workflows. What do you think?

• 14:29Chris Benson?

  I totally agree with you. And we've actually talked a little bit about this on previous episodes about the general feel that there has to be a convergence between CICD and ML Ops instead of them being kind of thought of as separate subfields, if you will, because we're going into a future where they're not two totally separate things that are always on their own tracks and on their own infrastructures. Models are going to be in everything we do going forward. And so the idea of software and models being completely separate with their own infrastructures is a little bit, I don't know, it seems antiquated to me. I think there's a movement that we're seeing right now where they're starting to integrate,

• 15:12Chris Benson?

  where ML Ops and CICD in general are starting to come together as people realize that, yes, I'm going to be deploying software and yes, I'm going to be deploying models. Most often, they will be happening together and at the same time. And so therefore, I must have something that works for all of the above. And I think that that's a bit of a challenge. For no other reason, there's some cultural differences in how we approach and what the priorities are and stuff. And so there's kind of two worlds smashing together, trying to find something that works for all.

• 15:42Daniel Whitenack?

  So Chris, I want to maybe describe some of what I've been doing recently with like this intersection of automation and CICD and machine learning or AI models. And I'd love to get your critique of that and help me know how I can, how I can do better, or maybe just initial thoughts. So a lot of what we've been doing recently, we're always kind of tweaking this workflow is we have an ML Ops solution and we're really thinking of that ML Ops solution as our experimentation and model training platform. So like, this is where a lot of jobs will necessarily fail because we're trying like weird thing,

• 16:25Daniel Whitenack?

  weird and crazy things. And we like have queues of GPUs where we can queue up experiments and train new models or do preprocessing of new data sets. Eventually, we get to a state where we sort of figure out what we're doing and we know there's a certain type of model that we are training successfully and would like to integrate into some system or service that we support. And so our ML Ops solution kind of provides as an output of these training jobs, like a hash of a bundle that is like the model bundle output that we have trained,

• 17:05Daniel Whitenack?

  right? So by hash, I just mean like a series of numbers, letters that, that points to a unique bundle that we've trained. And so we can look up in our ML Ops system like this model bundle, you know, whatever the hash is, was trained, you know, on this date with this model and this is how the task went and all that. And it connects also back to our Git repo where we have the training code with the exact commit ID that trained that model. So we've got like the code that trained it. The output of the model and the hash of that model. But then like that's the trust project was just talking about. That's not like model delivery, right?

• 17:48Daniel Whitenack?

  So in terms of connecting more to the C I C D things at that point, our model is really just like an artifact is used in software, right? Is used in various software functions. So we also have a usually a separate GitHub repo. Maybe it's an API we're supporting or an application or something like that. The thing that's integrating our model. And then we use hub actions, but other people use like Jenkins or Travis or something for C I C D. But what happens is we have GitHub actions, which for those of you that don't know is like a, it's integrated into GitHub, but it's a continuous integration,

• 18:31Daniel Whitenack?

  continuous delivery system similar to Travis or Jenkins or whatever. And so when we push a change into that repo, GitHub automatically runs a series of tests that we specify in GitHub actions. So these are like unit tests for our Python code. And then deploys the updated version of the application. Let's say it's an API. It deploys a updated version of the API. Now, what's interesting, I think where this connects with machine learning and the model bundle is like if we update that API to use a new model, what I kind of recommend our team do, and we don't have it integrated everywhere

• 19:16Daniel Whitenack?

  because you know, we, we have limited time. But in an ideal scenario, what we'd have is a sort of minimum functionality test for this updated model. So like if it's a sentiment analysis model, I would have a series of like a table of tests that would say like, you know, one sample is like, this is a really great thing and it's so awesome and that should be rated as positive sentiment always. Regardless of what model I update, it should always get that right. And so that way, if I update a model and I say, I point my API to the new model bundle in C I C D, it'll run. That minimum functionality test against the functions that are calling my model.

• 19:59Daniel Whitenack?

  That way, if I accidentally point it to a really bad model that can't even pass like minimum functionality, then it fails the build and it won't deploy with the new model, right? And so it's almost like a table-driven test that's used in APIs and such, except it's really a test against the functionality of the model versus the functionality of the actual application. So have you seen it approached in other ways? I'm always curious to kind of learn, learn what people are doing in this respect.

• 20:33Chris Benson?

  So I'm going to say nice things about what you're saying and I'm going to do this despite the fact that you're my host, my co-host on this. So I would say this to anybody. No, I think that you have the benefit of having been a software developer as long, you know, as you've been a data scientist. And so you're able to see both sides of that. I think that's often absent, that perspective. So what you've described, you know, you've picked your, you've picked some specific technologies that you want to use to support CICD efforts, which are fine. And I think that there's a whole bunch of different options there that are all more or less equally good, you know, with some, you know, pros and cons to each one as normal.

• 21:14Chris Benson?

  But you've integrated it so that you're, you're not only testing the software, but you're testing your data by testing how that data is running through the model and inferencing. So that is a very, very cohesive system. Unfortunately, here's the bad news. I think, I think that your, that your approach is a little bit more of the exception than the rule in the broader industry out there. I think that, you know, this is, I've seen this at a number of organizations where the skillsets of understanding that are still kept in kind of in separate groups, separate departments, maybe even whole separate organizations. And I think the benefit of being, working for a relatively, you know, small organization,

• 21:57Chris Benson?

  you know, not, not a giant Fortune 500 thing is that you're able to keep everything close enough together and your expertise. Is able to intertwine to solve that well. I think that that is a it's a good guideline for others to look at. If you ever find, I know you have all the spare time on your hands, but should you ever find it, I think maybe, maybe actually publishing a little thing on that would be would be a useful tool for people to kind of see how you've approached it. Other than just listening to the podcast here, because you're kind of hitting, you're hitting the software best practices and you're hitting the data science best practices together and treating the model as an artifact that needs to work per testing in that software deployment process and delivery. So anyway, yeah, I, I love what you're doing there.

• 22:41Daniel Whitenack?

  Yeah, I think if there was a takeaway for people that might get them thinking, like you don't have to do things exactly that way. And I'm sure there's better ways to do it. But I think one thing I've learned over time is like, if your model is being used in software and you can update your model without that software being retested in some way, there's like a huge risk and problem, right? Cause you could just run another like training run and there your model gets updated and all of a sudden like your software product breaks. But the software team or the other people working on it are all transparent to them. So like that, that sort of,

• 23:24Daniel Whitenack?

  that step, whatever it is, whether that that could even be manual, right? Like, you don't update the model in one S3 bucket until you run this script to test it and then you updated it. Like it could be manual in that way if if nothing else, right? But there needs to be some process there.

• 23:42Chris Benson?

  Yeah, that can be brutally hard to debug something like that because if you only have that that insight, if you're, you know, going to your example, if you're the software team and you haven't made any changes and yet now you're deployed and delivered software has just broken and you don't have insight into the fact that the model changed. You can waste weeks of time trying to figure out what happened on that. So it's a huge productivity hit not to have that, that, that point of integration and to apply those best standards. In the scheme of things, it is, it's still complicated more, more. So that it should be, but it's cheap in the sense of the, the things that you need to run and store and keep up with it in that versioned manner is not hard today.

• 24:24Chris Benson?

  That is a capability that anyone can afford and to not have the discipline to do that can result in some real challenges that, that waste a lot of time there. So, so I'm, I'm with you on that. I think that having those integrated and having the discipline to do them together and make sure that it runs at the end is, is pretty vital to, to, to moving as fast as we can.

• 25:09Daniel Whitenack?

  Well, Chris, I, I sort of went down the infrastructure rabbit hole as I often do. My team will tell you that I often go down that rabbit hole, but there is a lot going on in else in the AI world that has sort of hit our desks over the past weeks. You forwarded something to me related to some of what you've been following in the aerospace kind of industry or your own interest in, in piloting and that sort of thing. You want to describe that a little bit?

• 25:42Chris Benson?

  Yeah, I'd be happy to. I ran across something Carnegie Mellon University, uh, which is by any measure, one of the top AI schools, often described as the top AI school, uh, in the world. Definitely in that top half dozen without question, released a paper on something that they had been doing in their robotics, uh, organization, which, uh, entitled AI pilot can navigate crowded airspace. And of course, this appealed to me both from the AI perspective, uh, the fact that I am a pilot and the fact that I work for an aerospace company. So it hit me on a bunch of fronts. And, and so in this one, what they did was they put together a model and trained it.

• 26:23Chris Benson?

  And have been testing it in simulation that enables an autonomous aircraft to navigate crowded airspace. And so, and for those who don't fly as pilots, you know, airspace around airports gets very crowded. And you really have to work hard to maintain separation and keep things safe and such. So this is a non-trivial problem. People that don't pilot will, will look up and go, well, you got the whole sky there. You know, how bad can it be? But you're also in fast-moving vehicles and you're all moving on in the same patterns. And so you can have a problem very quickly. So this is a pretty important challenge to overcome. And it's one that we know that the industry is pushing for.

• 27:05Chris Benson?

  So I will say kind of obliquely in the military space where you're not necessarily in air patterns and stuff. We're doing autonomous stuff as, as is reported in the general news all the time in terms of, of aircraft. And, and I think that's fairly well understood. In the civilian space though, if you imagine in the future being on an autonomous airliner. And you and 200 of your best friends are flying around and, and you have AI models that are driving this. It's not as hard to move between busy airspaces as you're moving across the, maybe across the countryside or something. You're kind of out there by yourself. There's not as much to do, but on the start of that journey and on the end of that journey.

• 27:48Chris Benson?

  There are a lot of other aircraft in close proximity to you. So the ability to do this is, is pretty important. They say that this model can safely avoid collisions, predict the intent of other aircraft. Track those aircraft and coordinate with those aircraft's actions. Communicate over the radio. It uses a natural language processing to do that. It has a vision system that uses six cameras to visually track. And, and one other distinction in general with flying, there are two kinds of systems for flying. One is instrument flight rules, which is what you would think of with airliners. And one is kind of what us private pilots, the, the little guys, so to speak, do, which is visual flight rules. And you tend to have visual flight rules lower down to the

• 28:33Chris Benson?

  Ground. And so this system, we've had flight, automatic flight control systems and airliners for decades that fly, but those tend to be high up in the sky and you're kind of alone traffic wise. This is designed to do visual, can work with instruments, can work with radios, can work with cameras to do the visual stuff and make all the decision making in real time, right there to keep everybody in the sky safe. So it's a pretty cool problem to solve and it's one that eventually, I know people shudder when I say this, but I think that it is not so far out that all of us will be getting on airliners that are almost entirely automated. They might have a human in the cockpit because it makes us feel better, but eventually that that just won't be

• 29:17Chris Benson?

  really needed. And so this is one of those big steps toward trying to do that. And they're combining, going back to a theme that we've been talking about lately, they are combining natural language processing models with visual processing models, they're integrating those, and they're being able to use that system across multiple domains to affect a real world solution here. So I think this is very much in line with with the kinds of innovations that we've been looking at over this past year.

• 29:47Daniel Whitenack?

  Yeah, so I have a couple of follow up questions on on this, which is really interesting. One is, I just want to maybe get your perspective, since you're more plugged into the space and have interest in the area. I know that it's been talked about a while, for quite a while, that the sort of short or last mile kind of trips, like in a city, in a large city, you could have like air taxis, right, which are basically like humans in a big drone, right, and flying around the city. My understanding is any reasonable person would say, well, there needs to be computer systems within that that would coordinate and manage the

• 30:30Daniel Whitenack?

  safety of all the routes. And if there's a bunch of things flying around in the air, it gets very complicated in a crowded space. So maybe this gets us closer to that. You have any thoughts?

• 30:41Chris Benson?

  It does. And so, I will comment, I'll both answer your question and I'll make a reference that not even you know about me. Not just the listeners. The specific issue there is you're talking about massively scaling up the number of platforms, as I would say in my industry, that are in that space, which is my specialty. Is if you were to say, instead of having 10 things in a given closed, you know, space, it could be airspace, it could be on the ground. What if there were 10,000 in the same space? Maybe not all big, but maybe many are very small autonomous. How do you manage that? That's specifically where my, my current focus and expertise lies. And so

• 31:24Daniel Whitenack?

  Oh, cool.

• 31:25Chris Benson?

  This is a control system for aircraft that can enable things like autonomous taxis and package delivery and all these other things. It doesn't solve the whole problem. Kind of solves the how do you make decisions from one platform? It doesn't necessarily solve all the integration things when you have a massively scaled situation there. But it's an important step. It's really, really crucial to enable this future of civil aviation, which includes all of these low-flying drones, package deliveries, kind of like when we look at a Hollywood movie, you know, these futuristic science fiction movies, maybe Star Wars, and they have all, the city is just full of flying things everywhere at every level.

• 32:09Chris Benson?

  We're heading that way, but we need to get some of these technologies in place. And this is a key crucial Lego in that pile of Legos to build that.

• 32:17Daniel Whitenack?

  Yeah, my, my second follow-up question to that, which is, it's funny that this came up today when we're chatting, because, so I, I always listen to podcasts when I'm in the shower, which is a weird way to start this subject, but one of the ones I really like is, is Darknet Diaries. I love Darknet Diaries and, you know, just the stories there. But I listened to one this morning, it's actually from a while ago with a guy named Sammy, and he, he created this sort of proof of concept, which you can look at on GitHub, where it's called Skyjack. And essentially what he showed at the time, this This is a while ago. I'm sure things have gotten better now, but he he showed that he could put a drone up in the air

• 33:00Daniel Whitenack?

  with a antenna on it and a raspberry pi and basically wherever he would fly it, he would hijack the other drones around because the the signals he would intercept and he could actually take control of them. And so it's one of these things like anything that's connected is hackable. Right? And so also, as you kind of increase the number of things in an area, I think certain people might think, oh, that's dangerous because a computer is automating the flying. I think the dangerous part is not the computers automating the flying, but humans hacking into the computers automating the flying, maybe. So I don't know from your perspective how like security impacts these sorts of systems, but as you kind of automate,

• 33:44Daniel Whitenack?

  yeah, as you automate things in this area, the computer can obviously do, well, maybe not obviously, but I buy into the fact that a computer can do a better job at this sort of flight control than a human. But it makes it sort of hackable as well, right?

• 34:01Chris Benson?

  It does. And if anyone doubts that computer and model together are better at this point, you can go back, you can Google DARPA Alpha Dogfight. Uh, and and a couple of years ago, there was a public, it's on YouTube, there was a public demo where, and we've talked about this briefly on the show before, where they had automated a bunch of companies, brought their autopilots, they put them on an F-16 simulator and they competed against each other and then they had to compete against an air force instructor. And that air force instructor was the equivalent of what we would think of as a Navy Top Gun instructor. It was a weapons school instructor.

• 34:42Chris Benson?

  And the top one that went against the weapon, the human absolutely demolished him in a dogfight. I mean, demolished him five times in a row. It was, I mean, just, it was like stunning to watch. And so, yes, computers can currently do this better than humans can, even if you're the best human in the world. So that's already a done deal and it's just being improved upon since. So when people are worried about computers flying these things, I'm like, I would much rather, I'm that one person who would much rather for this technology to be flying the airliner I'm in than the human, because I know what the difference is in capability.

• 35:22Chris Benson?

  So it doesn't solve everything and it doesn't solve what happens at massive scale when you're struggling to get to handle all of the things in the airspace together. But it's, it's pretty crucial. And I, and I think your point about the cybersecurity there, it's cybersecurity is huge when it comes to aviation and autopiloting, because it's a natural thing to hack. And in my space, which is defense oriented, you would assume that your adversary is always going to actively try to do exactly that. So it's just built in to the equation. We're automatically handling that as we solution for it. And that will obviously roll out into the civilian space as well. And air taxis and package delivery, all of that has to have that cybersecurity capability.

• 36:07Daniel Whitenack?

  Super interesting. I'm glad we went down that, that rabbit hole.

• 36:11Chris Benson?

  It's fun, isn't it? I, I love talking about that.

• 36:14Daniel Whitenack?

  Yeah. One thing I wanted to highlight, maybe just somewhat quickly before we get on to learning resources is I took a look recently, it popped up in my Twitter feed, one of the things, but I looked through the whole set of demos and I would recommend people go and check out some of the Nvidia AI demos that are coming out. We'll, we'll link to that on the, in our show notes. I was kind of like, I don't know if I just hadn't looked at it in a while, but I looked there, I was like, oh wow, like, and then, oh wow, there's another, oh wow. That's sort of like all sorts of things I wasn't, I mean, I sort of peripherally knew were going on, but really powerful kind of demos.

• 36:56Daniel Whitenack?

  One being kind of this, this way of taking like sketches and turning them into photorealistic images. This, this is kind of related to some of the like image stuff that we've been seeing recently. But another really cool one I thought was This vid to vid cameo thing, which you could sort of synthesize a talking head based on an image of yourself that you could use in, for example, like Zoom calls. So you could just have like your talking head synthesized based on imagery versus like your actual video on the Zoom call, which I thought was really, really interesting. And I don't know, something I kind of want to try.

• 37:40Chris Benson?

  No, I agree with you. I'm so glad that this podcast is audio only because if it wasn't, we would absolutely, definitely need to implement that to make us presentable. So, um, yeah, we, we have faces that are made for radio, so to speak, uh, of our podcasting in this case. Uh, but yeah, that, that's, you know, we're already kind of seeing that. I mean, using Zoom, a lot of folks are using Zoom and other platforms like that at work. And I, I always have alterations that I'm making to make it more interesting and stuff. And some of them actually do do the facial fixes in real time. So, uh, yeah, that sounds like something that I definitely need. Uh, my, my, my wife would, I'm sure, she's always telling me, oh gosh, you need to look better than that.

• 38:26Daniel Whitenack?

  No comment from my end. Well, let's maybe hit a few learning resources as we close out here. I think going straight off of the Nvidia stuff, which is mostly vision and 3D things. I wanted to highlight this paper that I saw trending on papers with code. It's a survey paper, so it's a little bit maybe more approachable in some ways. 3D vision with transformers, a survey. Um, this is from Jean Lahoud, Lahoud et al. And, uh, they go through and talk about all of these sort of 3D representations and using transformers on 3D data for vision. Um, it's really interesting. And if you're kind of wanting

• 39:06Daniel Whitenack?

  to get an overall picture of some of the things going on in this space with 3D data, I think that's a, that's a really interesting place to get some of that information all in one sort of shot. Uh, the other thing I was going to Mention, which is not not related to vision, but back in my sort of world of of NLP is it's going to be a natural language processing with transformers course. It's going to be in September through October of 2023. And there's some people from Hugging Face that are teaching that. It looks pretty awesome. So it's like live teaching sort of thing.

• 39:49Daniel Whitenack?

  So yeah, I would definitely recommend check that out if you're interested in. It is paid, but if you're interested in that sort of paid live learning opportunity, then it seems like a really good, good one to to learn some of the latest stuff.

• 40:05Chris Benson?

  Yeah, good people teaching it there in terms of if you're going to spend money on it, spending it for people that are that are at the top of their field.

• 40:11Daniel Whitenack?

  That are legit. Yeah, exactly. So cool, Chris. Well, that's all I had for today. I enjoyed the the various rabbit holes. We went down and learned a little bit about aviation along the way. So yeah, I appreciate the conversation.

• 40:27Chris Benson?

  Yep, absolutely. Keep flying. Hi, Daniel. I'll talk to you next week.

• 40:30Daniel Whitenack?

  All right. Bye bye.

• 40:32Chris Benson?

  Bye bye.

• 40:41Speaker 01

  All right, that is our show for this week. If you dig it, don't forget to subscribe. Head to practical ai.fm for all the ways. And if practical AI has benefited your life, pay it forward by sharing the show with a friend or a colleague. Word of mouth is the number one way people find shows like ours. Thanks again to Fastly for fronting our static assets, to fly.io for backing our dynamic requests, to Breakmaster Cylinder for the beats, and to you for listening. We appreciate you. That's all for now. We'll talk to you again on the next one.