Leading the charge on AI in National Security

Chris & Daniel explore AI in national security with Lt. General Jack Shanahan (USAF, Ret.). The conversation reflects Jack’s unique background as the only senior U. S. military officer responsible for standing up and leading two organizations in the United States Department of Defense (DoD) dedicated to fielding artificial intelligence capabilities: Project Maven and the DoD Joint AI Center (JAIC). Together, Jack, Daniel & Chris dive into the fascinating details of Jack’s recent written testimony to the U. S. Senate’s AI Insight Forum on National Security, in which he provides the U. S. government with thoughtful guidance on how to achieve the best path forward with artificial intelligence. Sponsors: Shopify https://www.shopify.com/practicalai – Sign up for a $1/month trial period at shopify.com/practicalai https://www.shopify.com/practicalaiFly.io https://fly.io/changelog – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog https://fly.io/changelog and check out the speedrun in their docs https://fly.io/docs/speedrun/. Featuring: Jack Shanahan – LinkedIn https://www.linkedin.com/in/jackntshanahanChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Gen AI Master class https://lu.ma/nbi7pz6yWritten Testimony of Lieutenant General John (Jack) N. T. Shanahan (USAF, Ret.) AI Insight Forum: National Security https://www.schumer.senate.gov/imo/media/doc/Jack%20Shanahan%20-%20Statement.pdfSoftware Defined Warfare: Architecting the DOD’s Transition to the Digital Age https://csis-website-prod.s3.amazonaws.com/s3fs-public/publication/220907-Mulchandani-SoftwareDefined-Warfare.pdfArtificial Intelligence and Geopolitics:vHitching the Disruptive Technology Cart to the Geopolitics Horse | LinkedIn https://www.linkedin.com/posts/jackntshanahan_ai-and-geopolitics-activity-7143953788532375552-XzwrJoint Artificial Intelligence Center (JAIC) | Wikipedia https://en.wikipedia.org/wiki/Joint_Artificial_Intelligence_CenterUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:06Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related tech is changing the world, this is the show for you. Thank you to our partners at fly.io, the home of changelog.com. Fly transforms containers into micro VMs that run on their hardware in 30-plus regions on six continents, so you can launch your app near your users. Learn more at fly.io.

• 0:43Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm CEO and founder at Prediction Guard, and I'm joined, as always, by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

• 0:58Chris Benson?

  Doing good. It's a beautiful, almost spring day here in Atlanta. I walked my dog before the show.

• 1:05Daniel Whitenack?

  Yeah, the sun is out.

• 1:06Chris Benson?

  It's getting nicer out. Have to start with the weather, of course.

• 1:09Daniel Whitenack?

  Yeah, and spring is breathing sort of new life into various ideas around this podcast as well. And before we hop into our really interesting guest interview today, I wanted to highlight something new that we're trying at Practical AI. We want the show to be practical and useful, and part of that is actually helping you all get live, hands-on help with the latest technologies. And so, March 14 at 1:00 PM Eastern, we're going to have our first Gen AI Mastery Class, or webinar, or tutorial, or whatever you want to call it. We're going to dive into all things text-to-sequel,

• 1:50Daniel Whitenack?

  data analytics, questions with large language models. And we're going to be joined by Chung Shi, who is from Lance DB, the CEO there, who is a recent guest on the show. So it's going to be a lot of fun. Find out more at tinyurl.com/genai-mastery1.

• 2:10Daniel Whitenack?

  And we'll include the link in the show notes as well. I'm pretty excited about it, Chris.

• 2:13Chris Benson?

  I am too. Looking forward to that. It's a new feature for us to do and we hope folks in the audience enjoy that. I guess if it's okay, I'm going to dive right into our guest.

• 2:24Daniel Whitenack?

  Go for it.

• 2:25Chris Benson?

  I will say ahead of time, I have been looking forward for years for having this guest come on the show.

• 2:31Daniel Whitenack?

  It's a long time coming.

• 2:32Chris Benson?

  A long time coming because I'm at Lockheed Martin and we have a lot of don't cross the streams mentality. I had to wait for this guest to retire from his previous position so that there could be no crossing the streams concerns. And so I'd like to introduce Jack Shanahan. He is the only senior military officer who has been responsible for standing up and leading two organizations within the United States Department of Defense dedicated to fielding AI capabilities. One of those is was Project Maven, which was also known as the Algorithmic Warfare Cross Functional Team. And the other is the Department of Defense Joint Artificial Intelligence Center. And he was the founding and inaugural director of that. Jack, welcome to the show.

• 3:16Speaker 03

  Chris and Daniel, thank you so much for having me on. I just want to, I'll, I'll begin by just saying how much I love the title Practical AI, because I am a practitioner. This is not about research and development. There is lots of wonderful things happening in the AI R and D world, but that's not my world. My world is how do you take this esoteric technology and turn it into real product for the government, or for my case, for the Department of Defense, and a little bit for the national intelligence community, because Project Maven, you talk about crossing streams, we cross streams. It was both what we say the IC, the intel community and the Department of Defense. But I'm really happy to join both of you for this podcast. Thank you.

• 3:55Chris Benson?

  No problem. It's, you know, I will note, I follow you on LinkedIn. I encourage our audience to follow you on LinkedIn. You are out there reading the scientific papers as they come out and you often, when you're sharing posts, will put a spin on the various, you know, papers that are coming out from that AI and national security context, because those papers obviously aren't necessarily in that way. Very insightful, would encourage. People do that. Wanted to start out as a military professional. How did you get into this particular? I mean, this is, you know, you were the right person at the right place at the right time in the Department of Defense. How did you end up in this? What caught your attention? And how did you get going on this story?

• 4:36Speaker 03

  Yeah, I guess what I would begin by saying is I spent 95% of my career not working on this because I started in actually fighter aviation for the first 15 years of my military career. I spent 36 years in uniform, retired in the summer of 2020. And then from the flying piece, I was charged to go run an intelligence organization. And then I did a command and control wing and then a flying wing out of Nebraska. And then did a command down in San Antonio, which was about intelligence, surveillance, reconnaissance, and policy positions and other positions in the Pentagon. So this was not my destiny, my path, as we say from the beginning. However, when we got to that point where I came back as a three star into the Pentagon

• 5:20Speaker 03

  in something we call the undersecretary of defense for intelligence. I didn't walk into that job with AI is on my plate, as we say. I left that job with AI dominating everything I did, every single waking moment. And most of my non-awake moments, nightmares, dreams, whatever you want to call it. As I say, I cut a long story very, very short by saying we had an intractable problem. And we had intelligence analysts looking at full motion video coming from drone. And there was more drone video than at any point in history, hands down. The intel analysts just couldn't do it. They couldn't spend enough time in a day. As I used to joke about at the time, they would have some sort of energy drink next to them

• 6:01Speaker 03

  and probably some chewing tobacco, trying to just this mind numbing work of looking through this video. It was just, first of all, we're going to run out of people. But two, we were going to get more and more collection, not just full motion video, but from other assets, as we say, to include unclassified, open source information. And it was a success catastrophe, more collection from more places at more classification levels than at any other point in history, period. And so what could we do? And we could not find a technology that was ready to be fielded in the Department of Defense. Again, wonderful research and development work that was going on and goes on as we speak today. So we turned to commercial industry. In fact, we turned to Silicon Valley and they said, yeah, we've got something that could probably help you. And it's called computer

• 6:45Speaker 03

  vision, natural language processing. And that's where the journey began, to be honest with you. And we formally stood up in April of 2017. It's hard to believe it's been seven years now on that journey. And then we were off to the races. And I did that for just about 18 months. And then I was asked to stand up the joint AI center, or the JAKE, which was to take on everything that we weren't doing in Project Maven. Project Maven was about intelligence, but the entire department needed to start bringing in AI and everything else the department does. And there was no mechanism for doing that. Each of the military services had something going on, but I would call them pilot projects, small scale pilots that could not cross that valley of death. So the whole point of this new organization

• 7:29Speaker 03

  was let's get going. Let's get going at scale. Fast would be very good. Fast and at scale would be superior. And that's what this was all about. It was speed and scale. So that is how I ended up at the JAKE after having done Project Maven. And we had to stand up in our, as I say, I don't, I don't, you know, I've decided it's no longer tongue in cheek. I actually was the CEO of two AI startups in the Pentagon. But that doesn't make sense to a lot of people, like the Pentagon does startups. Yes, we did two startups and all the challenges that you would expect from startups, I lived firsthand with my team in Project Maven and the JAKE. So that's a good starting point to tell you the journey that we took to get to the final 2020 where I walked out the door. But I haven't walked

• 8:12Speaker 03

  out the door because I'm still very connected to what's going on in government.

• 8:16Daniel Whitenack?

  You mentioned in part of that discussion, the words at scale a couple of times. And I know as even now, I'm building my own company, you encounter problems at each level of scale that you you try to achieve. But when you're talking about at scale at the government level or worldwide, where all of these organizations are operating, I'm sure there are things that go beyond the technology element. So choosing saying, oh, this is a great model for this application, right? Whether it's a computer vision or natural language processing or Gen AI now. I'm mostly curious because I don't really have a good window into what does it mean to

• 9:00Daniel Whitenack?

  actually scale one of these applications, AI applications in the national security space versus in an industry context where you might be scaling to, you know, this many users or something within your company. What's different and what's the same and what's kind of unique of that at scale component in the national security world?

• 9:24Speaker 03

  Really, all germane points, Daniel, and there's so much to talk about just on that alone. I think I've now spent enough time, now that I'm out of uniform, I'm out of the Department of Defense, working with venture capital companies, Insight Partners in particular. And I get a chance to go and spend time with CEOs of everything from small startups to pretty big companies that are still in that venture capital business somehow, whatever seed round of funding that they're getting. And the journey is remarkably the same, actually surprising to me. I would sit there and cry in my beer, so to speak, and say, oh, nobody understands my problems. And then one day, I'm driving into the Pentagon, just despondent about the, how I'll never get this thing called the Jake Bill, because I have no people,

• 10:07Speaker 03

  I have no money. How are we going to get there? And I'm listening to Guy Raz, how I built this. And it was listening to the CEO, the founder of Belkin routers. It was a fascinating story. It was exactly my story. He says, I built this thing in my parents' garage. And there were days when I was just ready to throw in the towel and give up. And there were other days, 24 hours later, where I would suddenly, some technological breakthrough or some big contract that we got, and all of a sudden, it looked bright again. I lived that. I said, well, if he could turn it into that company, then hell, I can do the same thing in the Department of Defense. And I want to come back to something you said, Daniel, because it's so incredibly important to this discussion. The technology, of course, was fundamentally at the center of what we were trying to do. But everything else was

• 10:49Speaker 03

  even more important to how do you get to scale? So I would say maybe the differences are, well, they're used for military operations. But I would say that if you look at any big commercial company that was not built as a digital company, it was built as a hardware company in the industrial age. We have the same experiences. I've talked to enough people now and say, oh yeah, it's the same thing we're going through. It's getting that pilot project scaled and built in a way that you can then put it in one, you know, for a company. So let's say for a medical or financial business, it's for that company. But for a lot of other places, it's how do I get this thing scaled to the rest of the world? But in terms of industries, very similar to what we were going through.

• 11:30Speaker 03

  And I talk about this all the time. I have eight things and I don't have to go through all of them, but they have nothing to do with technology. It's, you know, mandate, vision, alignment, obstacle clears, bureaucratic enablers, resources, authorities, and then sort of talent management. All of those are not technology focused, but they have to exist or you can't get there. We all want to focus on, I have this cool new widget, gadget, and this thing is going to change the world. Well, it might, but if you don't have all those other pieces in place, which if you understand the bureaucracy known as the federal government, you have to have somebody that understands how to navigate in that world. Whatever I didn't know about technology, and that's a fairly long list of things at the time I started all this journey.

• 12:14Speaker 03

  I sure knew a lot about how to work in the, in the federal government. I had commanded six different levels in the United States Air Force. I had been, you know, in all these places where I had to do this day in, day out. So that part I was very comfortable with. I wasn't nearly as comfortable with technology piece, so I had to, it was a hockey stick learning curve. No if, ands or buts about it. I still feel like I'm a novice today, by the way. And that's seven years in the past where we started. It's actually eight. We got started in 2016. So I think there's more similarities than there are differences, at least for the non-digital companies. Because they have the same problems that a CEO is experiencing today, saying, is this AI thing real? Should I really invest millions of dollars of this company's money? What is the return on investment?

• 12:59Speaker 03

  That return on investment question. Is an unsettled discussion and I had lots of those with people very skeptical about what we were trying to do in the Joint AI Center.

• 13:08Chris Benson?

  That's actually a point I'd like to extend a little bit. You talked about the federal bureaucracy and the challenge and the US military is not just a single organization. It's a lot of large organizations with their own structures and their own cultures in each of the services. In a sense, that's almost a much harder thing, I would argue, than a lot of like, you know, Fortune 500 companies, not only from size, but just the, you have so many different things at play. When you're taking something as new and as hyped as AI is, so you know, having both the potential of the technologies that are being developed, but with a lot of hype thrown in on top of that, and those skeptical people in different organizations within

• 13:50Chris Benson?

  the larger, how do you navigate that to make, you know, you talked early on about the velocity being so important, velocity at scale. How do you navigate that in a large organization? What are some of the lessons that you learned on that? Because normally, we see such slow change, you know, the American people see such slow change in other nations too, in their governments, at least that's the perception. You did a lot in a very short amount of time. How did you navigate that?

• 14:16Speaker 03

  Very carefully. It was hard. It was hard to do it. But Chris, I'm going to come back to the word that you said, and to me, I would put this at the core of everything I was trying to do, and that's culture. You're trying to change the culture of an organization. And the thing about the Department of Defense, I'm not going to say it's completely unlike a lot of big companies, but I think there are big differences because when you talk about the Department of Defense, there are cultures within cultures within cultures. There are military unit cultures, there are service cultures, there's an Office of the Secretary of Defense culture, there's a Pentagon culture, foreign as that culture is, it is a culture unto itself when you walk into that building every morning and leave at

• 14:56Speaker 03

  the end of the day, dark on both ends. So this culture piece is difficult to figure out how to change that. So what you have to do is just persist for days, months, and years at a time and say, join me, come on this journey with us. And there'll be a lot of resistance to that because, not surprisingly, when you talk about warfighting, the Department of Defense is still largely a risk averse organization. It has to be because lives are at stake. We're trying to take a culture and change it in a way that people are just not comfortable with because of the hype, because they don't understand what AI really is. To them, they keep hearing miracle whip, miracle, miracle whip, this thing that we're just gonna take

• 15:39Speaker 03

  out of the jar and spread on a couple of pieces of bread and all will be well with the Department of Defense. It just doesn't work that way. So this idea of changing culture from a bottom layer, top layer, middle management layer, it all has to be done simultaneously. And part of this culture is, and this is why when I talked about what you need is not just the bureaucratic enables, but the people who are obstacle clearers. And I put them in three categories. The classic disruptors, which the Marine Corps Colonel that ran Project Maven, Drew Kokor, was absolutely positively one of those. You also need the people that clean up the broken glass that comes from the disruptors. And we had a lot of people that were capable of doing that. And then I have the networkers. These are the people

• 16:23Speaker 03

  that a lot of times are what we would consider middle management. They get disparaged, but to me, they're the key to success. There are the people that know how things get done in the federal government. They go have a cup of coffee with the budget person. They go have a cup of coffee with a service general officer, and they work the networks that they've established over years. And if you find somebody that can do that effectively, you're going to do much more than people that just think sheer force of will alone will change culture. You really need some ratio of all those three types of people. And the ratio will change over time, which I found in Project Maven, we need the disruptor, period. We had to force feed this down the throat of the Department of Defense. Hard,

• 17:06Speaker 03

  hard, hard to do. But over time, as I got into the Jake, I needed more of the people that could do the networking and that would, you know, do a little bit of disruption, but more it was about cleaning up some broken glass, but really moving faster and faster. So culture eats strategy for breakfast. It always has, it always will. You have to put culture at the center of any technology project.

• 17:27Daniel Whitenack?

  Well, Jack, I really love how we're getting into a lot of these culture, strategy, talent sort of subjects, which is really key and practical across a lot of organizations that are trying to adopt this technology. One of the things that you mentioned, which kind of caught my ear is, you know, there's people involved in particularly in the context that you've been working in that do have a true valid concern around risk of these technologies. Like, hey, people's lives are on the line and we care about that, which hopefully they they should care about that, right? So after working in these environments for a while and bringing new technology to sort

• 18:08Daniel Whitenack?

  of risky situations, for a lot of those people out there that are also navigating, maybe maybe it's not people's lives, but maybe it is like real impact to people's lives and integrating AI systems around automation or really sensitive subjects like finance and healthcare and other things. Do you have any learnings or thoughts from your experience of AI in risky situations or with sensitive data that you'd like to highlight and how that played out in your situations, what you learned over time?

• 18:41Speaker 03

  Yeah, Daniel, I would say this is something we talked about pretty much every day. Now, maybe it was a little different because we knew the problem from the beginning. That problem could not be solved in any other way, so we knew we were going. But when I got into the Jake, the conversations we had were generally along this line. We're going to start with lower risk, lower consequence use cases, solve those, learn from them, and then slowly move up the ladder, maybe even fast up the ladder, depending on what we were talking about, to get into those higher risk use cases. There is a lot of talk, of course, about the dangers or the risks of so-called killer drones. I'll tell you the thing I wasn't working on in all five years, last five years of my career was killer drones. I had nothing to do with those because they were too high risk, too high consequences.

• 19:26Speaker 03

  And we had to understand how to actually do AI before it started getting to that. So we started with things like predictive maintenance. Or there were some medical initiatives that we were taking on. There were some things that we could put on individual intelligence platforms or sensors. And so we learned from that. And then based on those lessons learned, began to sort of edge our way into more consequential use cases. And to me, there would be a clear analogy with any business that's out there. Because again, a CEO, the risk will be different. It might not be life or death, but it could be tens of millions of dollars. And that's very risky for a CEO. So the idea of proving success at something. And the reason we stood up the Jake is because we showed enough success in Maven.

• 20:10Speaker 03

  As difficult as it was, we did show real success and put AI models out in combat operations within a year of standing up the organization. Okay, let's go try some other things. And so what you'll see now is, okay, now that we've learned all these lessons, they're going to a little bit more consequential use cases. Uh, maybe AI enabled autonomous drones and maybe not lethal drones, but autonomous drones. And then you'll start seeing this. In fact, you can kind of see some of this playing out in Ukraine. As you look at the headlines, what they're doing with drones on both sides right now. So to me, it's like starting with something that's manageable. Get your arms around it. Learn what it means to build a data management pipeline.

• 20:48Speaker 03

  Because what I've found in the DOD, hands down, what stopped people cold when trying to start AI projects was data. Much better now than it was, uh, seven years ago, much, much better. But it's still hard. So if you can't get over that data hurdle, then people get despondent. They can't reconcile what they're hearing about the hype with the challenges of doing it for real. And I tell you, I've been around long enough now to know when somebody has never done a real AI project and just talks about AI. Because there's a big difference in those two things.

• 21:18Chris Benson?

  One of the things I wanted to share with the audience is that you have recently submitted a paper with a lot of guidance to the US Senate. They had the AI Insight Forum on National Security and you submitted your document, and this was very recently on Wednesday, December 6 of this past year. And it's great in that it gives a fantastic kind of oversight into AI in a national security context. And you offer some recommendations which I'd love to go through in terms of that. But you mentioned Ukraine and you also mentioned that in the document. I was wondering, as a transitional question, kind of what have

• 21:58Chris Benson?

  we learned that you can share from Ukraine? So as you're racing through getting AI gradually integrated into our armed services and we're watching and supporting the Ukrainians against the Russians, and we've had this real time, real life learning process that's come out of that and seeing what they've done with drones and stuff, is there anything that stands out in your mind that was either, you know, great learning or something unexpected that came from that real life application that's being developed right now?

• 22:28Speaker 03

  Technology is at the center of this fight that's going on, this war that's going on between Ukraine and Russia. And what you've seen in Ukraine in the first year of the war was how quickly they adapted. It's an amazing story. It's actually an amazing story that what you have is some people that might have been born in Ukraine, came to the United States, got educated, went out to Silicon Valley, but when the war started, went back over to Ukraine and then focused on how much faster they can bring technology to the fight. Things that we've been talking about doing in the US Department of Defense for many years, they did instantly. Like, people don't maybe understand this part of it, moving their entire government to the cloud. If they had not done that, it would have been a disaster. They would have lost everything.

• 23:11Speaker 03

  But then just on the military technology pieces, how quickly they were able to bring in, what I say is this really example of what we've been talking about for a while is software-driven warfare is moving that fast. And it's not to imply that technology, of course, is no longer relevant. It's as relevant as ever, but technology that is software-defined is a different kind of technology. And that's how Ukraine has been gaining an advantage is by moving faster than the Russians and adapting much faster. And the drones are the best example of this. It's crazy to see what they've done. The idea of bringing in 100,000 drones, some of which are first-person view, and you're watching somebody wearing these goggles, drive a drone with explosives on it into

• 23:53Speaker 03

  a Russian tank and blow it up. So there are a lot of lessons. Now, no two conflicts are exactly the same. So we got to be careful about the fungibility of lessons learned. But I think there are so many lessons that will apply to conflict for the next 10 to 20 years. And this idea of being smaller, smarter, cheaper, attritable, networked, and even swarming, it's playing out in Ukraine as we speak. Now, is that going to make the difference between winning or losing the war? Well, I think it's made the difference in not losing the war. It's much harder to win the war because you're up against Russia. It's a very difficult fight that they're up against. But what we're seeing is technology used in such imaginative ways. And it's this weird juxtaposition of World War I-like

• 24:35Speaker 03

  trench warfare with AI-enabled systems, Maven-like capabilities that are being used to find targets, spot targets, and send artillery against targets. So you're seeing what a lot of us thought was coming, and it did happen, it is happening, about how quickly you can adapt to the changing conditions of the battlefield, or I would say battlespace, because cyber is part of this as well. Electronic warfare is really... They're really important right now because it's killing Ukrainian drones. The Russians are very good electronic warfare. So now the Ukrainians are trying to adapt based on what's going on. So we have to listen to those lessons and apply them to the US Department of Defense. And I, by the way, and I'll stop here is I think what you're seeing is the replicator initiative,

• 25:19Speaker 03

  which is buying thousands of drones of various sizes and capabilities is in a large part, I think, based on the lessons that they're absorbing from what we see in Ukraine.

• 25:29Chris Benson?

  I wanted to go back for a moment to software-defined warfare. There is a, the paper that you just referenced that you wrote with Mr. Mulchandani, if I'm getting his last name pronounced correctly, who is your CTO at The Jake? And I believe he's the CIA CTO at this point.

• 25:46Speaker 03

  He is.

• 25:47Chris Benson?

  That was quite a landmark paper. Ironically, for those of us who are in industry and maybe not military related at all in the audience, you're used to Daniel and I always talking about, you know, AI is still part of the software. It's all, it's all bound together. You can't do AI without the hardware and the software and the systems written together to make it practical AI that's usable. And you really went there in software-defined war. It's called software-defined warfare, architecting the DoD's transition to the digital age. And it was quite a landmark paper for those of us in that industry because it really laid out the future of how software needs to be integrated. Do you have any, anything that you wanted to comment on about that?

• 26:30Chris Benson?

  Just given, I thought it was very important to anybody concerned with AI in the DoD.

• 26:37Speaker 03

  And it was really written by Nan Mulchandani after his experiences in Department of Defense with The Jake. But then he left and before he took the CIA, he'd had all these ideas germinating. And he said, you know what? I'm going to write about this and we're co-authors, but. He's the author, I'm the editor, and I placed the operational imprimatur on that report. And it's so important because it represents all the commercial software industry best practices that need to be brought into the Department of Defense. And what's sad about this, Chris, is, to your point, is if you're in commercial tech industry and you were to read this report, you would be flabbergasted that the department's not already doing all these commercial best practices, which are just the way of doing

• 27:19Speaker 03

  business. You know, microservices, platform as a service, software as a service, it's just so foreign to the Department of Defense. Getting there, it's come a long way, but that's why he wrote that paper. It is a blueprint and say, we can talk about, this is what Nond and I used to have these conversations toward the end of my time there, is that we can talk about AI till we're blue in the face and we will, it's a wonderful conversation. But unless you make the Department of Defense modern, digitally modernized, which includes data best practices and do all the things that's in that software-defined warfare report, AI will be meaningless. You'll never get there, at least not at scale. And by the way, I am now a member of the Atlantic Council's Commission on Software-Defined Warfare. So they've taken that mantle and are making it the centerpiece of all these

• 28:03Speaker 03

  recommendations that they're going to have. So it's a really important piece, and I thank Nond because, as I'll say, for as long as I talk about this, Nond changed that organization the day he showed up, because he understood there was a different way that we had to be doing business than the standard, traditional Department of Defense way.

• 28:21Daniel Whitenack?

  One of the interesting things that I think, I haven't seen the specific paper you're referencing, Chris, but I like the ideas that are being discussed here, because one of the things that I've seen, and I'm kind of trying to parse through and figure out how to put words to, is when I'm on customer sites or interacting with people, it's something about this AI technology that seems to disconnect people's mind from the fact that there's still... A need to like have error checking or like it's not just that you have this AI and you send something over and it kind of automates things and then you move on. There's still the idea of a software application and there's still a call at the minimum to an API

• 29:05Daniel Whitenack?

  that maybe you need like retries around or some sort of health checks or backups or this sort of thing. Does that perception hold water in terms of what you've seen, Jack, as well? And you were talking a lot about culture. Any recommendations for those kind of dealing with this fact of like swapping software for AI versus embedding AI in software?

• 29:31Speaker 03

  Yeah, completely resonates. It absolutely resonates because this is, I even say, maybe this is going a little bit too far, but I don't think it is. It's the, and Ukraine's validating this, by the way. The next conflict is going to be an API-driven conflict. If you get that piece right and you can update faster than your adversary on this idea of software-driven warfare, you will have a competitive advantage. Because the battles, let's face it, you put out an AI model and you never update it, then you might as well have never done it in the first place. The idea of the battlefield is going to change, the battle space will change, but also models will drift and all the other things that happen. They get exposed to different data, new data, whatever. You're going to have to do a continuous integration, continuous delivery or deployment.

• 30:13Speaker 03

  That's part of this, but you can't do that in the traditional way of doing it, which is you have an entire weapon system that you have to pull apart, completely pull apart, rebuild and put back out again. That doesn't work. You're going to have to break all this apart and just focus on those little bits and pieces that have to be updated. You mentioned API calls. There is so much more that can and should be done in the department of getting those status updates and two-way fees. You know, one, feeding those updates of higher headquarters down, but two, putting all the things that the end user is seeing, feeding that back uphill. That's what we have to be thinking a lot more about to be able to handle what I think is going to be a very, very chaotic environment. This AI, however great it may eventually become, does not change the fact, as I said

• 30:57Speaker 03

  in my testimony, that warfare is a very chaotic, nonlinear, dirty, ugly place, and it's horrible that we have to go to war. But this technology could provide that competitive advantage that makes a difference in a future fight.

• 31:31Speaker 04

  You know, when we started podcasting back in 2009, and online stores was just the furthest thing from our minds. Now we have merch changelog.com and you can go there right now and order some t-shirts and that's all powered by Shopify. It's so easy, all because Shopify is amazing. Shopify is the global commerce platform that helps you sell at every stage of your business. From the launch your online shop stage to the first real life store stage, all the way to the did we just hit a million dollar stage? Shopify is there to help you grow. Whether you're selling security systems or marketing memory modules, Shopify helps you sell everywhere from their all-in-one ecommerce platform

• 32:12Speaker 04

  to their in-person POS system, wherever and whatever you're selling, Shopify has got you covered. Shopify helps you turn browsers into buyers with the internet's best converting checkout, up to 36% better compared to other leading commerce platforms, and sell more with less effort thanks to Shopify magic, your AI-powered all-star. You know, nothing gets me and Jared more excited than when our guests get that coupon code in their email when their show ships or to everyone out there who loves changelog podcasts and can go to merch.changelog.com and get your favorite threads to support our podcasts. It is just the best thing ever. From stickers to threads, all that is at merch.changelog.com.

• 32:52Speaker 04

  And did you know that Shopify powers 10% of all ecommerce in the US? And Shopify is the global force behind Allbirds, Rothy's, and Brooklinen.

• 33:03Speaker 04

  And millions of other entrepreneurs of every size across 175 countries. Plus Shopify's extensive help resources are there to support you and your success every step of the way. Because businesses that grow, grow with Shopify. Sign up for a $1 per month trial period at Shopify.com/PracticalAI. All lowercase. Go to Shopify.com/PracticalAI now to grow your business no matter what stage you're in. Again, Shopify.com/PracticalAI.

• 33:54Chris Benson?

  To extend what we were just talking about right before the break. You kind of finished talking about the study of warfare being asymmetric technology advantage. You talk about that in your paper and just the challenges. And you've talked about integrating in these latest greatest technologies. One of the things that you address a fair amount is you talk about teaming between AI and a human being in the context of national security. And adjacent with that, which I know Daniel asked you a little bit of a question about that earlier, kind of having to do with safety and the security of the AIs and stuff. Can you talk to us a little bit about, you know, that's obviously a question that many people have in their minds that have nothing to do with this space, you know,

• 34:36Chris Benson?

  the military or the industry around it, just concerned about what does teaming mean? How far in your thinking does autonomy go? Is there a point, just to infer something, that as the pace of warfare speeds up? You know, we have drones and we have different types of autonomous technologies and stuff. But as the pace of warfare speeds up and you look into the future versus kind of what we're looking at in the near term, how does that change the human AI teaming equation as you're trying to keep up? You know, and maybe having humans in the loop can become a problem or a serious logistical issue. How are you looking at the future in terms of where we're at today and where that's going?

• 35:19Chris Benson?

  And what are some of the challenges that we need to overcome to get there?

• 35:22Speaker 03

  I think so so strongly about this idea of human machine teaming or human systems integration that if we don't get this right, we're in trouble. And what I mean by that is there are people have been writing about human machine teaming in some form for 50 years, no different in terms of the concept of how do you get the most out of human machines. But to me, we're at a period where it is changing and it's changing because the machines are going to be so much smarter than we're used to talking about that it will be a different view of human machine teaming than we're used to. So I think there's a whole new research field. There are already people working on this and it's what I'm amazed at some of the work that's being done, but we just don't know how good some of these technologies are going to be.

• 36:03Speaker 03

  Let's just take the example, very briefly example of sort of chat GPT or the equivalent. There's this idea of prompt engineering. I look at that as almost like the elevator operator in the 30s or 40s, whenever it was, and it goes away eventually because people get comfortable with it, but they're not as comfortable with it today. You need help figuring out what that interface looks like. But the other thing I say, and this is where I come back to, I really do believe there will be situations, I call this the bell curve of military operations. Some cases, you really do want that human making that final decision. Maybe to launch nuclear weapons is the president of the United States and nobody but the president of the United States. On the other end, you need the machine to do what the machine does best. There is no time to get in the way.

• 36:44Speaker 03

  Everything else in between, I don't know what that is, 80%, 90% is human machine optimized for both. That Centaur idea, and that what we need a lot more work working through. And here's the example I would give from from my time in the flying world. There were a lot of times the machine did not operate as intended. The human was there to make sure that everything went fine with the mission. Oh, that broke again. We're going to have to pull that circuit breaker. I started in the F4, circuit breakers popped all the time. You'd piss out hydraulic fluid out the back end or whatever. So the human would have to make up for a lot of the machine's mistakes. I'm not sure that's going to be a luxury in the future, that you're going to have to let the machine do what the machine does and the human do what the machine does very well. What do humans do well?

• 37:29Speaker 03

  They do reasoning. They do inductive reasoning. They do deductive reasoning, do abductive reasoning. They put things in context. They deal with other human emotions. Machines don't deal with emotions. They don't understand emotions and I'm not sure, despite what some people claim, they ever will. So what does that look like? I think there's a lot that has to be done in this area in experimentation. Play around with it, see what works, because if you don't get it right, and there's a couple of examples where it's went dramatically badly, and I think the 737 MAX is one of those examples. Well, we're going to put this software in. Trust us, you don't need to be retrained as a pilot on this. Just listen to what it says in the cockpit. That was wrong. And it's proven that it was dramatically the wrong thing to do. So I think this is an area that needs a lot more explanation.

• 38:14Speaker 03

  There are people writing about this and then really good writing in various places, but we have to figure out how to get this part right, because the machines are going to be smart enough that they have to be allowed to sort of run when they should run.

• 38:27Chris Benson?

  We're seeing both modern models are getting so impressive in certain areas, in certain capabilities. There's a lot of hype around them. They can't do everything, but what they do, they tend to do quite well, recognizing that there are hallucinations and other technical issues to be worked through over time. But we're seeing kind of a rapid increase in capability in a lot of these areas. So one of the things that I get asked all the time, just kind of on the side with people, is as those capabilities go to some level of increase in the future, whatever that is, and whatever the tasking is, and you've kind of alluded to, it kind of changes the balance in human teaming.

• 39:07Chris Benson?

  Is there a set of metrics or some guidelines that you have in your mind on, as you see the technology progress, maybe not today, maybe not this year, maybe not next year, but maybe five, 10 years out, and you see these capabilities in specific areas increasing far beyond what a human can do for a given task? How do you rebalance? How do you... Assess that. You talk about assessments in your paper as well, so that you say, this is one of those moments where you let the model do what the model does really well, because it does it faster. It does orders of magnitude better than a human could do in the same amount of time.

• 39:46Chris Benson?

  How do you make those assessments? Because I think that for us, humans with emotions around technology and warfare, I think that causes a lot of concern. It seems to be the foundation of many questions that I get asked. How do you make that metric judgment on those adjustments to the culture of how we interact in that way? You know, kind of the way we think about it.

• 40:07Speaker 03

  Yeah, a few thoughts on that. And while I remain an AGI skeptic, I do believe these machines, these AI-enabled or smart machines will get so good. The human interfering with the machine could be a worse outcome than human-machine together. And the example that I've heard, I think it was Gilman-Louis raised this about AlphaGo and the Move 37. If a human was there to override the machine, it would have lost the game. Machine said, no, you really do want to make this move. Leave it alone. And it won. So that's an example of humans are going to get to the point where they've got to be more comfortable with this technology. And they're not necessarily today. Why? Because just like when I use ChatGPT, whatever version of a large language model, it does get some things still very wrong.

• 40:51Speaker 03

  Maybe the ratio is only 10% versus 20% just a year ago. But if that 10% is dangerously wrong, then do you really want that in military operations? Not yet, you don't. So how do you do that? This is the core of your question. To me, it is test and evaluation. It's experiments. It's putting it into the hands of users as an MVP, a minimal viable product, which again is a little bit different than, or a lot different than the way the Department of Defense has fielded systems in the past. Where you fielded a system when it was as close to perfect as you were going to get, which may have not been perfect, but close enough for government work, as we say. But in this case, you need to put things in the hands of users sooner rather than later. This is why I'm a big fan, despite my reluctance to say we should not be using large language models for.

• 41:37Speaker 03

  Putting items in the presidential daily intelligence briefing, you should have all sorts of experimentation being allowed in all these federal government organizations and agencies. In fact, the White House executive order on AI says, go try this out. It does, it basically says, go do it. So I'm a fan of the experimentation, what we'd call sandbox experiments. Put it in users' hands because we don't know yet. But part of that is core test and evaluation. We do not want a short circuit test and evaluation. And why is this so important? Because I think that's where we're going to see the biggest risk, at least initially. Some risks are going to be hard to figure out until you use them in an operational world. We just know that, just like in commercial business, some things are going to surprise you. But that's why you then update the models, because you learn.

• 42:21Speaker 03

  But in that early stages of design and development, before you get to the fielding part, there is this thing that we have done really well in the government for many, many years, especially on the hardware side, it's called test and evaluation. That does not go away when we talk about AI. In fact, AI is still novel enough. I think it's more important than ever to get, spend a lot of time on the D and E. Doesn't mean you're going to go slowly relative to sort of putting out other things. But I am a little bit cautious about moving too quickly on some of this because we just don't know what those risks are yet. So there's a lot of work going on, including some things I've been, I've been working on, on a risk management framework for AI systems in the military. Because this risk, it is a hierarchy, right? There's one end, there's AI-enabled, their weapons are really, really bad.

• 43:06Speaker 03

  On the other end, there's process automation for a finance system that touches nothing having to do with warfighting. Negligible risk, move really fast on that one. And then a lot of things in the middle, you've got to do test and evaluation, determine how many risks there are, and then come up with risk mitigation strategies.

• 43:21Daniel Whitenack?

  I'd love to weave a couple of things together from what you mentioned in your testimony. One of those having to do with a concept which I thought was really interesting around techno-economic net assessments. And I think there's a lot of people, I'm sorry, ask you to maybe explain what you mean by that. There's probably a lot of people that see news articles about, oh, this country is ahead of this country. And This type of AI, whatever. And of course, you know, you see articles about, oh, China, whatever company in China bought up this many GPUs or like other things like that. And depending on where you're at, you get concerned, but it's all sort of very amorphous.

• 44:03Daniel Whitenack?

  And it's hard to really grasp where countries are in terms of the AI stack and their their capabilities. And the other thing that you mentioned in the testimony is encouraging the DOD to take bigger bets. And I imagine that that's also connected with sort of making sure that our techno economic net assessment is sort of on the on the upward trend. Could you talk a little bit about those ideas and maybe how they're connected? And how you would love to see those kind of big bets going forward?

• 44:35Speaker 03

  Yeah, if anybody was not in the government and skimmed through my testimony, probably would have went past that paragraph and not paid any attention to it. There's a reason I put it first. It's that important. It really is the center of attention. And that is, let me put it in terms of, say, one commercial company competing against another commercial company in the same general business space. CEO of one is always looking at CEO of two, saying, how much faster are they moving? What special sauce are they bringing into their product that threatens our market dominance? This is going on every day in commercial industry. Well, it's also going on between states, in this case, in AI. So the problem is with states is you can get a lot with industrial espionage, but it's a little harder to do when you have a nation state like China.

• 45:20Speaker 03

  And both sides, the United States and China, are talking about how much they're each doing in AI. But how much is reality and how much, again, is the hype? Well, you need a lot of intelligence assessments. And I found this out from my earliest days in Project Maven, where I would ask these questions about, okay, and I'm gonna stay very unclassified here, the People's Liberation Army, what does their AI stack look like? The intel community is not spending any time collecting on, or at least they weren't at the time, because nobody told them to. What is a GPU? Okay, well, we know where we have to start this conversation. That is, what does their compute look like? What do their models look like? Are they using open source or who's building them for China? What does their talent base look like? So this idea of not just technology, but then equally important on both sides,

• 46:05Speaker 03

  US-China or US-Russia, whatever you want to say, is what are they doing with the technology? What are they building new operational concepts? Are they actually reorganizing? When you start seeing bureaucratic reorganization, they're far along. We don't see that yet. We don't see it in the Department of Defense because we haven't figured out exactly what this new technology is going to do for us yet. So that's what I mean by this. And by the way, the US government used to have this thing called the Office of Technology Assessment. I think Newt Gingrich managed to kill that as part of the revolution in government, whatever. Now there's some serious work to bring something like that back. Now, I'm just talking about within the Department of Defense and the intelligence community. On the classified side, how do I bring in all this information, both unclassified and classified information,

• 46:49Speaker 03

  to give us a relative net assessment, a net assessment, us versus them? Where do we stand? And it turns out that's pretty difficult to do because technology is not so easy to take a picture from a satellite of a GPU and decide how much farther they are ahead. So to me, we've got to do better at understanding, vis a vis, you know, the United States, vis a vis China or Russia or anywhere else around the world, what they're doing in this area. And so it's such a big concept to get right, and it's hard because it's just not like collecting against tanks or nuclear weapons or something. I can't see it anymore. And as I say, there's no fluid coming out of a building somewhere to tell me that they're working on a particular project there.

• 47:28Chris Benson?

  Well, Jack, we have covered so much material here and time that we've been talking has flown by. I think we're gonna, if you're willing, we're gonna have to have you back on the show because there's a lot that we haven't been able to address yet. As we finish up, I'm kind of thinking in the background about culture and what you were talking about and just the massive organizational and even national change that we're doing here in the United States in government and military and intelligence. It's changing the way that we are looking at the future of warfighting. You mentioned in your paper, the joint warfighting concept, which I'm very familiar with. And I believe there is an unclassified version out there that we'll add into the show notes. But it's changing the way we think about all that we do with the military and intelligence.

• 48:13Chris Benson?

  And I think AI is a big part of that. As we wind up, do you have any? We often ask guests, To kind of take the last question to be whatever you want it to be, paint a picture of the future with whatever you want to be. Can you give us a sense of kind of what you're excited about going forward, how you think some of this may evolve and what that means to the United States military and intelligence community so that the listeners of this mostly not involved in that directly have a sense of where things are going? Any thoughts you want to finish with today?

• 48:46Speaker 03

  Yeah, thanks, Chris. And thanks, Daniel, for allowing me this time with both of you today. And it's a big thought, and it's something I've been thinking a lot about. And when I retired, I went back and got another master's degree, thanks to the GI Bill. And I did get a chance to think, you know, big thoughts about technology over the course of human history. I mean, really, back to the very beginning, I took some courses about looking at what we would call, maybe not technology today, but it was back then, say, sugar or D, and how it sort of diffused globally. What I say in my testimony, I do believe in my core that we are going to be in the middle of, at some point, a third revolution, you know, sort of agrarian revolution, industrial revolution. This is different. It's not the fourth industrial revolution. It's some kind of digital revolution.

• 49:29Speaker 03

  We don't know what it's going to look like because we just haven't been there yet. I say in my testimony, the future is, to a large extent, both unknowable and unpredictable. Why? Because it's not determinism, it is not technological determinism. It will be dependent upon the decisions by many, many, many, many thousands of people, from leaders to citizens of countries, deciding they like or don't like the technology. It's going to take maybe 50 years to 100 years to play out. And historians will be the ones that look back and define when this revolution began. But to me, it is fundamentally different, which means warfare will be different. The character of warfare is going to change. I will not say the nature of war is changing,

• 50:09Speaker 03

  because the nature of war is a human-centered decision, like why we fight and why one country fights another country and so on. So I don't believe the nature of war, but the character of warfare is going to change dramatically, as we're seeing in Ukraine. And we have a chance to be on the right side of that, as I say in my testimony, asymmetry equation. If we're on the wrong side of it, we risk losing. And we're not used to losing. And this is a very serious risk. So this idea of it's playing out as we speak, get involved in it, don't wait for it to catch up. You've just sort of got to dive in and start working these big projects in the government, wherever you are, or anywhere else in the industry.

• 50:48Chris Benson?

  That's a great call to action to finish up with. Jack Shanahan, thank you so much for joining us on the Practical AI Podcast. Really, really interesting conversation. Thank you for your insights and hopefully we can get you back on the show to cover things going forward. Really appreciate your time. Thanks, Chris. Thanks, Daniel. And of course, I'll be glad to come back.

• 51:15Speaker 00

  All right, that is Practical AI for this week. Subscribe now if you haven't already. Head to practicalai.fm for all the ways and join our free Slack team where you can hang out with Daniel, Chris, and the entire change log community. Sign up today at practicalai.fm/community. Thanks again to our partners at fly.io, to our beat freaking residents, Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us. That's all for now. We'll talk to you again next time.