How the U.S. military thinks about AI

Chris and Daniel talk with Greg Allen, Chief of Strategy and Communications at the U. S. Department of Defense (DoD) Joint Artificial Intelligence Center (JAIC). The mission of the JAIC is “to seize upon the transformative potential of artificial intelligence technology for the benefit of America’s national security… The JAIC is the official focal point of the DoD AI Strategy.” So if you want to understand how the U. S. military thinks about artificial intelligence, then this is the episode for you! Sponsors: DigitalOcean https://do.co/changelog – The simplest cloud platform for developers and teams Whether you’re running one virtual machine or ten thousand, makes managing your infrastructure too easy. Get started for free with a $50 credit. Learn more at do.co/changelog https://do.co/changelog. The Brave Browser https://brave.com/cha913 – Browse the web up to 8x faster than Chrome and Safari, block ads and trackers by default, and reward your favorite creators with the built-in Basic Attention Token https://basicattentiontoken.org. Download Brave https://brave.com/cha913 for free and give tipping a try right here on changelog.com https://changelog.com. Featuring: Greg Allen – LinkedIn https://www.linkedin.com/in/gregallen2, X https://x.com/Gregory_C_AllenChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: U. S. Department of Defense Joint Artificial Intelligence Center https://www.ai.milSummary of the 2018 Department of Defense Artificial Intelligence Strategy https://media.defense.gov/2019/Feb/12/2002088963/-1/-1/1/SUMMARY-OF-DOD-AI-STRATEGY. PDFFact Sheet: 2018 DoD Artificial Intelligence Strategy https://media.defense.gov/2019/Feb/12/2002088964/-1/-1/1/DOD-AI-STRATEGY-FACT-SHEET. PDFWhite House AI Website https://www.whitehouse.gov/aiPractical AI – Episode #47 - GANs, RL, and transfer learning oh my! https://changelog.com/practicalai/47AlphaGo https://deepmind.com/research/case-studies/alphago-the-story-so-farDefense Advanced Research Projects Agency - DARPA https://www.darpa.milIntelligence Advanced Research Projects Activity - IARPA https://www.iarpa.govLaw of War https://en.wikipedia.org/wiki/Law_of_warUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at ChangeLaw because of Rollbar. Check them out at rollbar.com. And we're hosted on Linode cloud servers. Head to linode.com/changelog.

• 0:18Speaker 01

  This episode is brought to you by Digital Ocean. Digital Ocean provides worry-free database hosting with their managed databases. If you need to get data in and out of Postgres, MySQL, or Redis, call on the world-class support teams at Digital Ocean and stop wasting time on setup, backup, and maintenance. Get simple, predictable pricing. Get detailed documentation. Get up and running in minutes so you can get on with your business. What are you waiting for? Head to do.co/changelog. Again, that's do.co/changelog.

• 1:03Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slake with us around various topics of the show at changelog.com/community. Follow us on Twitter. We're at Practical AI FM. And now into the show.

• 1:28Chris Benson?

  Welcome to another episode of the Practical AI Podcast. I am Chris Benson. I am a Principal AI Strategist at Lockheed Martin. And with me as always is my co-host, Daniel Whitenack, who is a data scientist with S I L International. How's it going today, Daniel?

• 1:43Daniel Whitenack?

  It's going pretty good. Got a bit of a cold, but not too bad. And doing some, some pretty interesting stuff with text to speech this week. So that's pretty fun.

• 1:52Chris Benson?

  Really? Anything worth sharing or are you gonna wait till later?

• 1:55Daniel Whitenack?

  Well, I think there'll be some things revealed at the Project Voice Conference, which will both be out in Januaries.

• 2:02Chris Benson?

  Yep.

• 2:03Daniel Whitenack?

  So I'll leave you in anticipation until then, but it involves lots of speech data and some text-to-speech in local languages, so.

• 2:11Chris Benson?

  And for any listeners who happen to be in the Chattanooga, Tennessee area, north of Atlanta, and wanna catch Daniel and I, we're gonna give a talk at the Project Voice Conference, and we're also gonna be recording some episodes there. So we would love to see you there. If you happen to listen to us, come up and say hi. Well, I am pretty excited about today. We have a guest that I already have quite a bit of knowledge about. We have some work between our company and his organization. With us today is Greg Allen, who is the Chief of Strategy and Communications at the U.S. Department of Defense's relatively new Joint Artificial Intelligence Center. Welcome to the show, Greg.

• 2:52Speaker 04

  Hey, thanks for having me.

• 2:53Chris Benson?

  Yeah, I'm pretty excited about it. You guys do some super cool stuff, and so I'm really happy to share that with our listeners. I've been looking forward to doing this for a while. I guess if you could just start us off, kinda telling us a bit about yourself and your background, kinda how you got to this point, and then after that, we'll dive into what the Joint Artificial Intelligence Center does.

• 3:17Speaker 04

  Sure. Well, as you said, I'm the Chief of Strategy and Communications here at the Joint AI Center. And I came here from a variety of backgrounds, really. My primary professional experience has been in corporate strategy work, and I did that both as a management consultant and then also working in the corporate strategy offices of a variety of commercial technology companies. But after a while, I was working on some AI strategy projects and ended up doing a piece of work for the Intelligence Advanced Research Projects Activity. That organization is called IARPa. If you're familiar with DARPA, this is the sister agency that covers the intelligence community of DARPA. And my report there was titled "Artificial Intelligence and National Security," and it was essentially, you know,

• 4:01Speaker 04

  going through the different aspects of AI technology, how it was changing over time, and elaborating on what the likely implications of that would be for different areas of national security.

• 4:12Daniel Whitenack?

  What year was this around in terms of kind of how things were developing?

• 4:16Speaker 04

  Yeah, this report came out in 2017 and I was working on it for the entire preceding year.

• 4:22Chris Benson?

  Gotcha.

• 4:23Speaker 04

  So if you're familiar with the Obama administration White House's reports on artificial intelligence that came out in October 2016, sort of the origin of this was that there was an additional report that they talked about writing on AI and national security, but ended up punting on it and deciding, you know, oh, somebody will get to that later. We should focus more on the workforce and the economic and the research and development impacts of AI. And so IARPa, the head of IARPa at the time, Jason Matheny, asked me, well, you know, rather than wait, why don't we have you as sort of an outsider, take a first stab at this report that was not written? And so it was on behalf of IARPa, but it was ultimately published through the Harvard Belfer Center for Science and International Affairs.

• 5:08Speaker 04

  And after that report came out, there was a, you know, a great deal of interest from the US national security community, and I ended up joining the Center for a New American Security, which is a think tank in the Washington DC area, and doing a lot of analysis and sort of pro bono advisory work to the Department of Defense. And so when they ended up standing up the Joint Artificial Intelligence Center, the individuals who were standing up that organization asked me if I would be willing to help out. And of course, I jumped at the chance. It was a very exciting opportunity.

• 5:37Chris Benson?

  Very cool. So I guess, and just as, to call out a couple of acronyms, I know you described IARPa. You also referenced that by talking about kind of the intelligence side from DARPA. Listeners who are not familiar with it may remember that DARPA is the Agency of Defense to Advance Research Projects Agency, if I got that right, that started the internet. So if you're not familiar with this already, that's probably where you would have heard of DARPA and IARPa, its sister agency for defense. So I guess at this point, if you could kind of, you know, I know the JAKE, and I should say, JAKE refers to where the center that you're at, the Joint AI Center at the Department of Defense, and we call it the JAKE for short. So I know it is relatively new.

• 6:19Chris Benson?

  You guys just came into existence in the, I think, middle to late part of the year in 2018. If you could kind of start off by telling us, what is the JAKE? And, you know, kind of describe its mission and its budget and, you know, why is it there?

• 6:35Speaker 04

  Sure, so the Joint AI Center was established as the Department of Defense's Center of Excellence for AI Technology. It was officially stood up in parallel with the release of the Department of Defense's AI Strategy. So that document was released in the middle of 2018, and it was released publicly in the unclassified summary in February of 2019. So a bit of a gap between when the strategy was finished and when it was released publicly, but ultimately it's out there, you can read it on the internet. And so the DoD AI Strategy that came out in the summer of 2018, what it says is that the Department of Defense recognizes the strategic importance of AI technology, and it also says that, you know, the Department of Defense wants to pursue advances in AI

• 7:20Speaker 04

  technology basically for military advantage, so the basic reason that we're interested in most technologies. And that document said that we were going to create a new organization called the Joint AI Center, which would be the focal point of the implementation and execution of the DoD's AI Strategy. So maybe I'll start by talking a bit about the DoD AI Strategy, and then I can get into the specifics about what the JAKE is doing to make good on that strategy. That sounds fine. The DoD AI Strategy has five pillars. These are: Deliver AI capabilities for mission impact. Scale AI's impact through a common foundation. Cultivate a leading AI workforce.

• 8:02Speaker 04

  Engage commercial, industry, academia, and international allies and partners. And lead in military ethics and AI safety. What I just described, those are the sort of five key pillars of the DoD AI Strategy. The JAKE is the focal point for each of those pillars, but we're especially interested in, at least in the near term, we have been especially interested in delivering AI capabilities for mission impact. So and this is a little bit of how we are structured as an organization. It's quite common in commercial industry for AI organizations to separate into, you know, who are the people who are actually developing and implementing and executing AI capabilities? And then who are the folks who are building the infrastructure, the platforms, the tools

• 8:47Speaker 04

  to enable that other group? So there's a separation between your data scientists and your data engineers. And similarly, a separation between your sort of AI capability developers and your AI infrastructure and platform developers. And that is a distinction that is recognized in the DOD AI strategy and is also recognized in the organizational structure of the JAKE. So we have our mission initiatives, our national mission initiatives. These are specific projects that we're going after because we believe that they have a lot of the features that allow for success in AI development. So some of these projects are, I'll just list off a few. Humanitarian assistance and disaster relief, predictive maintenance,

• 9:32Speaker 04

  cybersecurity, intelligent business automation, warfighter health, and joint warfighting. And the sort of common approach to how we identify projects and select them to go after them are, you know, what are the features that we believe lend themselves to success in developing AI capabilities? So this is areas such as, is there a relevant data set that you can use for training your machine learning algorithms? And do you have access to that data? Is it of sufficient quality? The second thing that we're looking for is, is there mature AI technology in commercial industry or academia that could be used against this problem? The third, and perhaps most importantly, is, would there be mission impact?

• 10:17Speaker 04

  If you did succeed at building this thing that you are setting out to build, would anybody actually care? And then the fourth criteria is, do we have access to end user partner organizations who would be willing to test our capabilities as they are being developed in an iterative basis? User feedback is Critical for any software development effort. But we believe that iterative user feedback is especially critical for success in developing AI programs. So if you have, you know, sort of those four criteria as a recipe for success, then you know, you have a shot at being one of our mission initiatives. And those are the ones that we're going after right now. The sort of second big chunk of our organization

• 10:59Speaker 04

  is related to, as I mentioned before, infrastructure and platforms. And so our infrastructure and platform team is developing what we're calling the Joint Common Foundation. And this is an environment that lowers the barriers to entry to develop machine learning and AI capabilities in the Department of Defense. As you can imagine, the Department of Defense is a pretty significant target for hacking and adversarial intent, just in general. So when we're developing software, it's critical that we do that in a secure environment. But at the same time, you know, a lot of these security processes can sort of slow down the development of software. So that's why we're developing the Joint Common Foundation.

• 11:42Speaker 04

  This is an infrastructure environment that has pre-cyber hardened tools that are the same sort of machine learning development frameworks that you might want to use in commercial industry. But they're adapted to be compatible with DOD cybersecurity policies. They're containerized so that they run in cloud environments at reasonably high performance levels. And that combination allows you to develop software quickly and get it into the hands of users and testers in a reasonable timeframe. But also do so in a way that takes into account the very significant cybersecurity risks that any significant DOD undertaking faces.

• 12:23Daniel Whitenack?

  Yeah, I have a question there around that sort of flow from what frameworks and software is used in industry to what you have to develop on. Are you involved very closely in, for example, the TensorFlow PyTorch communities? And kind of you have your own versions of those things that you run internally? Or is it more taking frameworks like that and wrapping them around like middleware and other things that give the protection that's needed?

• 12:50Speaker 04

  Yeah, so I don't want to get into too much of the secret sauce for security reasons, but the basic thinking there is that open source tools are popular for a reason, right? They work well, they have been tested by users who have pretty significant requirements and needs. And we want to make sure that when you're developing machine learning software in the DOD environment, we don't want you to have to use, you know, baby software or software that has been largely disabled because, you know, so many of the features were not approved. So the goal very much is to give our communities of developers access to the same types of things that they would be using if they were doing so in a commercial industry environment,

• 13:34Speaker 04

  but doing so in a way that gives us confidence about the security of that operation. So there are a few, in the commercial world, a lot of what we talk about in software development is what's called DevOps, development operations, and the sort of seamless figure eight loop between those two, where you're constantly, you know, adding features, deploying those features, gaining feedback from your users on how those features are used, which informs how you want to modify them, or come to with a new batch of features, or overhaul the software in more significant ways. In the Department of Defense, we have sort of adapted that paradigm to DevSecOps, which sort of recognizes the different security requirements that you might have at each stage, the development and operations process.

• 14:18Speaker 04

  And so for the machine learning world, some of this is relatively uncharted territory, which makes it exciting, but we're finding that a lot of the value that we can add is I'm just taking, you know, commercial and academic and open source tools and adapting them to the, to the national security use case.

• 14:38Chris Benson?

  Gotcha. One of the things I'm kind of curious about is, you know, the very first word is joint in the, the organization's title. And you end up, as you described already, working with all these different partners. Some are from industry, like the company I work for, some are academic, various universities that are doing great work in AI. Obviously, you work with the various Branches of the military and they obviously have their own initiatives in AI. Could you talk a little bit about what those different types of interactions look like? Both with with industry, how you support it, what you're asking from it as well with the academic?

• 15:19Chris Benson?

  And also, what is the division of responsibility that the Jake has with the various, you know, service specific laboratories that do work in AI in their own specific missions? What do all those different relationships look like to you?

• 15:34Speaker 04

  Sure. So, starting with the last part of your question, which is how we interact with the service laboratories. And I'll add to the service laboratories also, you know, an organization like DARPA. In general, these laboratories, you know, the Naval Research Laboratory, the Air Force Research Laboratory, the Army Research Laboratory and so on. They are primarily focused more on advancing the state of the art in AI and dealing with situations where the sort of existing state of the art is not a good fit for military requirements. So that's a very important job, but it's a bit different from the from the job of the Jake.

• 16:13Speaker 04

  We are interested in problems that we can go after where technology that is available in commercial industry or academia today. So sort of state of the art as it currently exists is a reasonably good fit for for military needs. DARPA and the service laboratories are focused more on areas where some additional research and development is needed in order to get the technology to a level of maturity where it can be useful for for DOD needs. So the way we sort of view it is if your problem is the kind of problem that can be solved, you know, in in zero to five years, you're probably a better fit for the Jake.

• 16:54Speaker 04

  If your problem is the kind of problem where it might require five to 20 years of research and development to be solved, then it's probably a better fit for the for the service laboratories in DARPA. And that's not a that's not sort of a perfect summary. Of course, each of the service labs in DARPA. Also, you know, work on some, some near-term projects as needed, you know, by necessity, or some specific competencies and skills they might have. But to sort of a first approximation, that kind of characterizes the division of labor between the two organizations. And, and, and the fact is DARPA and the service laboratories have been doing amazing work and continue to do amazing work. But the JAKE was, uh, kind of stood up to solve a different problem.

• 17:36Speaker 04

  So not the problem of advancing the state of the art, but the problem of adopting the state of the art, um, as it exists in, in commercial industry and academia. Which gets to your, the second part of your question, right? How do we engage with commercial industry and academia? And the obvious answer is early and often. We have, I would say, a sort of unusually aggressive, you know, outreach, uh, program. We have one staff member who is co-located with the Defense Innovation Unit in the San Francisco Bay area and full-time. Uh, hi- his job is to sort of go out, meet with companies, meet with venture capitalists, identify, you know, what is going on in the tech space that might be relevant, uh, to the types of problems that the military has and the types of problems that the JAKE is

• 18:21Speaker 04

  either trying to tackle or, or might be interested in trying to tackle in the future. Uh, and the organization that he's co-located with that I mentioned, the Defense Innovation Unit, or DIU, is, uh, similarly, that's an organization whose primary priority is improving the Department of Defense's relationship with commercial industry and the commercial technology industry especially. So there's a variety of sort of different contracting mechanisms that are designed to make it a little bit easier to do business with the Department of Defense. Folks who have not been paying close attention would probably, uh, remember that it's, it's often very, uh, lengthy and process-intensive and there's a lot of bureaucracy to do business with the Department of Defense.

• 19:02Speaker 04

  And that's very much something that the DOD has been working on quite intensely to reform. And so DIU in particular has pioneered the use of some, uh, not new, but comparatively unfamiliar contracting and acquisition techniques such as other transaction agreements.

• 19:20Speaker 04

  And these allow you to sort of get on contract quicker, get money flowing quicker, and actually start doing work quicker. And to that, the Jake has also been an early adopter of commercial solution openings, which is sort of another contracting mechanism. And the important feature of these mechanisms is, again, it makes it easier to do business with commercial technology companies, and it makes it easier to do business with sort of smaller companies who can't always afford the overhead required to do business with the Department of Defense for our bidding process and our proposal writing process.

• 19:57Daniel Whitenack?

  I'm coming at this conversation as someone who is not deeply involved in the defense world, but one of the things that was kind of coming to my mind as we entered into the conversation was I was wondering how much of the strategy that you're putting into this, that the DOD is doing, is driven by what other maybe potential adversaries are doing in AI? And like, what is the landscape of AI and defense look like around the world? And how has that impacted the priority we put onto it? And then like, how we go about, you know, developing that technology?

• 20:36Speaker 04

  Sure. I mean, I would start by saying that the two countries are specifically named by the National Defense Strategy, which came out in 2018. Two countries are specifically named as strategic competitors, and that is Russia and China. And so these are countries who have interests that are identified as being, you know, contrary and in contradiction to, in many cases, the interests of the United States, and also who have oriented their national security establishments in competition with the United States. And that's not a very surprising statement, I would say, to most. I mean, when the National Defense Strategy came out, it was sort of putting on paper

• 21:20Speaker 04

  the sorts of things that a lot of United States leaders had been saying, and frankly, that, you know, a lot of leaders in the two countries I just mentioned were also saying. So that's sort of... Sort of basic backdrop of strategic competition. Into artificial intelligence, of course, this is the national security world we're talking about and the military we're talking about. So we remain quite interested in what is going on around the world. And we would be silly not to be paying attention to that. I think speaking about China and Russia each in turn. China's AI strategy, which came out in two thousand seventeen, specifically identifies that they see AI as a transformative technology in many different areas, including in national security.

• 22:02Speaker 04

  And it also identifies AI as a leapfrog technology. The term leapfrog is interesting in this use case because it is described elsewhere by Chinese military thinkers and strategists as really sort of describing their belief about what AI technology will enable their military compared to the United States military. So if you can think about the example of cellular telecommunications infrastructure in developing countries, notably, you know, many developing countries in Africa. This is the canonical example of a leapfrog technology. There were many developing countries in Africa who did not have well-built out landline telephone infrastructure.

• 22:44Speaker 04

  And yet, this was no disadvantage whatsoever in adopting cellular telephone infrastructure. They just skipped the development step of landline telephones and went straight to cell phones. And that skipping is referred to as leapfrogging. And in our competition with China in military technology, there are many things that we are quite good at that they have a very hard time with in a technological sense. Things like jet engines, things like aircraft carriers. These are really tough technologies, really complicated technologies, that we are, you know, as a country, tend to be quite good at. And that China as a country has historically had a lot of difficulty with. And so when they write about AI technology, they're saying, well, if we could really develop an interesting advantage in AI,

• 23:28Speaker 04

  perhaps we could leapfrog the United States, which is to say, perhaps we would not have to catch up to them in aircraft carriers or catch up to them in jet engines because we will shift the basis of. Of competition and they write, you know, Chinese military thinkers often write quite optimistically about China's opportunity to to compete with the United States technologically in these terms. So, you know, we would be remiss if if we were not paying attention to that. The second thing I will say is that, as pointed out by the Secretary of Defense at the National Security Commission on AI uh in his speech at that commission, you know, there are many Chinese weapons manufacturers who are currently selling on international markets

• 24:09Speaker 04

  weapon systems advertised as being autonomous, meaning they can sort of make their own decisions and act independently. And also, you know, having that full combat autonomy, meaning they can, you know, actually be responsible for the use of lethal force. And then so that's what China is sort of up to today, at least in terms of what they're advertising on the international market. Russia is similarly, you know, very interested in AI technology. One quote that everybody really paid attention to was in September of 2017 when Vladimir Putin said that, you know, whoever leads in AI technology will be the ruler of the world. And I think Russia does not have a very clear path to leading in AI technology. Whereas the United States and China, you know, regularly top the lists

• 24:54Speaker 04

  of who is publishing the most AI research papers annually and who is publishing the best AI research papers annually. And similarly lead in measurements about who is attracting the most, you know, venture capital for AI companies. You know, Russia is pretty low on all of the rankings that I just mentioned. So I don't think that Russia has a clear path to leading in AI technology. Unfortunately, they do have a reasonably clear path to leading in the weaponization of AI technology. I think this is a pretty similar story to the internet. Russia was not a leader in any of the foundational technologies for computer networking or the internet.

• 25:32Speaker 04

  And yet nevertheless, you know, Russia developed a very advanced and broad and deep cyber capability, military cyber capability. And so similarly, I think, you know, Russia is looking to be a leader in the weaponization of AI, just as they were a leader in the weaponization of the internet. In terms of what they're doing. It's a lot of what you would expect. The social media disinformation campaigns and influence operations that Russia has been in the news a lot for lately. They are also interested in bringing more advanced machine learning and AI capabilities to these operations. And then secondarily, combat robotics is an area that Russia has devoted a lot of investment and has shown a lot of interest.

• 26:13Speaker 04

  And they're experimenting with a lot of their military robotic systems operationally, literally, using some of these systems in Syria. So both Russia and China are moving out aggressively to incorporate AI capabilities into their military. I would say, in terms of the United States' response, our intent is to lead the world in the military use of AI for the benefit of United States national security. I don't think we're so much obsessing over each individual capability that comes out of China or Russia so much as we are sort of looking at the broader landscape. I think AI is a general-purpose technology, the way electricity is a general-purpose technology.

• 26:56Speaker 04

  It's useful for things like radio, but it's also useful for light bulbs. Similarly, computers are useful for just about everything, and we similarly think that AI ultimately has a lot of application-specific opportunities for us. So we're interested in leading in AI in the broad sense.

• 27:14Daniel Whitenack?

  And would I be correct in saying, then, that of course there's certain things that, even me not being involved in the defense industry at all, like I'm aware of, for example, China using facial recognition technology and Russia interfering in elections and doing behavioral type of stuff, like you talked about. And so would I be correct in saying that there's kind of, for every different type of AI model you can think of, whether that's natural language processing, computer vision, object recognition, trend analysis, forecasting, there's probably all sorts, because this is general-purpose, like you say, there's probably a way to weaponize all sorts of

• 27:58Daniel Whitenack?

  those types of models. It's just not that facial recognition is an issue. It's a whole varied range of things that could be at play.

• 28:09Speaker 04

  Yeah, I think machine learning is just a new way to create software. You know, traditionally, software required, you know, every line of code to be typed out by human hands. But suddenly, there's been a vast improvement in this this field of artificial intelligence known as machine learning, from which, as I'm sure most of your listeners already know, you know, from which we can program, or to a certain extent, the system can program itself, based on what it learns from data. And so that general truth about the rise in in machine learning software, well, that's useful for most of the places where software is useful, which is to say absolutely everything, whether that everything is a missile guidance system, or sort of, you know, back office

• 28:53Speaker 04

  applications like word processing. I think over the long term, and I do want to emphasize long term, because while some of this transformation will be quick, some of it won't. Over the long term, we expect, you know, machine learning AI to be useful for just about everything, right? From the from the most backwater parts of the back office to the front lines of the battlefield.

• 29:14Chris Benson?

  So one of the things that you were talking about a moment ago was kind of the the our adversaries' perspective on, you know, they are kind of advertising AI in a lethal context and stuff. I'm assuming that the US Department of Defense has a policy out there regarding how we think about AI being incorporated into lethal force scenarios. Can you talk about that? What is the American viewpoint on that question?

• 29:43Speaker 04

  Well, sure. I think I think the first thing I would point out is that the Department of Defense, you know, abides by the law of war. And for folks who are not familiar, you know, with the national security context, the law of war might not be familiar to them. These are the, you know, literally legally binding principles that are enshrined in documents such as the Geneva Convention, but also have flow down requirements to DoD policy, DoD training, rules of engagement in different scenarios. So the overarching thing which guides all, you know, US conduct in military operations is the principles underlying the law of war. And these principles really get to, you know, when is the use of force appropriate?

• 30:28Speaker 04

  When is it acceptable? And and let me just, you know, sort of list some of the key principles underlying that. Military necessity. Right? Was your use of force the only way that you could achieve the objective that you had to achieve? Proportionality. If somebody slapped you in the face, you know, did you respond with a nuclear weapon? Well, then that would not be a proportional response. Distinction, right? Did you make every effort to prevent harming civilians and only harming enemy military combatants? You know, humanity, did you use any means, you know, that violate the principle of humanity? And of course, you know, abiding by the military principle of honor. So those ethical principles, you know, the Department of Defense has been guided by

• 31:12Speaker 04

  those, and these are actually legally binding. You know, you could be court martialed for failing to abide by these principles, but those have been guiding the Department of Defense for decades now. To that, there are two, two things that are comparatively new. The first is the Department of Defense Directive 3000.09, which relates to the use of autonomy in weapon systems. And the second, and that was a policy that was released in 2012 and was widely praised at the time. And that policy was renewed and essentially made permanent in 2017. You know, basically, you know, neither Russia, you know, nor China have any kind of policy

• 31:53Speaker 04

  comparable to 3000.09, which I think says a lot, right? And then there's also our, what came out just recently, which is the Defense Innovation Board's Principles for the Ethical Use of AI.

• 32:14Speaker 01

  This episode is brought to you by Brave. We deserve a better internet. That's why the team behind Brave reimagined what a browser could be. Brave is like Chrome, the good parts. Even your extensions will just work. It has built-in ad and tracker blocking, easy anonymization with the Tor network, earn tokens while you browse, and use them to tip your favorite creators. And did I mention it's lightning fast? Turns out the web is super fast when you remove all the cruff. Download Brave today using the link in the show notes and give tipping a try on chainsaw.com.

• 32:55Chris Benson?

  So Greg, I know one of the really hot and important topics these days in AI is kind of the ethical concerns, ethical principles of how you apply AI and what that means. And I know both in the defense industry as well as in lots of different commercial industries, that has become a big thing. And you know, a while back, we saw Google and Microsoft and other large organizations releasing kind of public principles and frameworks about how they thought about this. And I know in the defense industry, there is an organization made up of a lot of experts from outside the defense establishment itself. It's called the Defense Innovation Board, and they had a fairly substantial conversation

• 33:40Chris Benson?

  around AI ethics and principles, and recently released a document that covered a lot of that. And I am told that you were pretty involved in that process, as was the Jake at large, and that, and I was wondering if you could kind of talk a little bit about your involvement, what it means to you, and actually go through what those principles that you guys worked on are, and how that relates back to the Jake and what the DOD will be doing going forward.

• 34:07Speaker 04

  Sure. So the Defense Innovation Board is a federally appointed advisory committee. So while they are not actually part of the Department of Defense, the federal government has sort of given them an official relationship whereby they can advise the Department of Defense. And when we released the DOD AI strategy in the summer of 2018, it was immediately obvious to us that the ethical considerations of AI technology was something that we wanted to take very seriously and that we wanted to understand very thoroughly. So the Secretary of Defense asked the Defense Innovation Board to run a study about the ethical implications of AI and recommendations for principles for the Department of Defense's use of AI

• 34:46Speaker 04

  technology. They ran a very robust process. They were working on this for 15 months. They held three, you know, public forums at leading universities in the United States. They ran more Multiple, you know, rounds of open calls for comment on the subject from expert communities and also just the general public. And they ended up talking to hundreds of, you know, not just leading AI researchers and leading AI industry types, but also ethicists, lawyers, and folks with perspective from a lot of different industries and walks of life. So this was a lengthy, robust process.

• 35:25Speaker 04

  And the report that they released on October 31st, 2019, you know, reflects their best judgment at the intersection of, you know, ethical obligations, the requirements of the domain of national security and the mission of the Department of Defense, and then the nuances of AI technology. And so they are an independent organization, helping out the Department of Defense in these issues. So I was not involved in the actual writing of these principles. My only involvement was giving advice on how they could structure this to maximize the benefit from the Department of Defense's perspective as a consumer. So helping them craft actionable recommendations and thinking through, you know,

• 36:08Speaker 04

  how to write their recommendations in a way that would actually be compatible with the DOD bureaucracy and the DOD process. But what came back ended up being really quite substantive and something that, you know, we take seriously here in the department as now it falls to us to implement them. And also something that our allies, whether that be, you know, in Europe or in Asia or elsewhere, our allies have shown a great deal of interest in these principles as helping think through, how do we recognize the legitimate ethical concerns that the rise of AI technology raises, while also being able to move forward and use this technology, you know, in the mission of national security?

• 36:49Speaker 04

  So maybe I'll just sort of go through each of the principles that the DIB is recommending that the Department of Defense adopt, and we can sort of think about how the DOD thinks about these issues.

• 36:58Chris Benson?

  Yeah, if you can kind of hit the six and then kind of dive in however you want to the dozen recommendations afterwards and just kind of name what the six principles are and then talk about what those recommendations that came out of that, if that's okay with you.

• 37:12Speaker 04

  Absolutely. So the principles start with responsible. So this is that human beings should exercise appropriate levels of judgment and remain responsible for the development, deployment, and outcomes of AI systems. Second is equitable, that DoD should take deliberate steps to avoid unintended bias in development and deployment of combat or non-combat AI systems that would inadvertently cause harm to persons. Third is traceable, that DoD's AI engineering discipline should be sufficiently advanced, that technical experts possess an appropriate understanding of the technology, the development process, and operational methods of AI systems, including transparent and auditable methodologies, data sources, and design procedure and documentation.

• 37:57Speaker 04

  Fourth is reliable, DoD AI systems should have an explicit, well-defined domain of use, and the safety, security, and robustness of such systems should be tested and assured across their entire lifecycle within that domain of use. And fifth, and finally, governable. DoD AI systems should be designed and engineered to fulfill their intended function while possessing the ability to detect and avoid unintended harm or disruption, and for human or automated disengagement or deactivation of deployed systems that demonstrate unintended escalatory or other behavior.

• 38:29Daniel Whitenack?

  Yeah, there's a lot of jargon there. I would be interested in hearing kind of how those play out to real sorts of scenarios, or maybe the sorts of scenarios that were in people's mind when they were thinking of those.

• 38:41Speaker 04

  Sure. I think the overarching two things that I would say about these principles are, one, that the Department of Defense's ethical principles related to the use of force are those enshrined in the law of war, right? Military necessity, proportionality, distinction, humanity, and honor. That really gets to the ethical question of what is the ethical use of force? So the DIBs principles are answering a different question, which is, you know, assuming that you are abiding by the ethical principles that govern the use of force, how do you have confidence that your technology will be able to implement your desires? Right?

• 39:26Speaker 04

  You know, today, it is ethical to operate a commercial airline. In 1903, it would not be ethical to operate a commercial airline for passengers because aircraft technology, I'm thinking specifically of, you know, fixed wing propeller aircraft, was not ready. It was not safe enough for you to responsibly offer that on the commercial market. So part of the, sort of, you know, the ethics of the technology relate to matching its current use to the maturity of that technology and your processes for understanding that technology and ensuring that it is both robust and safe. And so that, that sort of gets to the DIBs approach to ethics and how it complements

• 40:09Speaker 04

  the ethical principles that are enshrined in the law of war. If you say that you're going to have AI systems that abide by the law of war, how do you know that? And how do you prove that? And how do you have, you know, the relevant processes to ensure that? That's sort of what the, with the DIB AI principles are going after is, you know, not what is the right or wrong way to use force? That question is best answered by the principles enshrined in the law of war. But if you are trying to do the right thing, how do you have confidence that your technology is going to allow you to do the right thing?

• 40:48Speaker 04

  And so that's about having the kinds of testing, evaluation, verification, and validation procedures so that we can rigorously test our systems to show that they are going to perform as intended. That gets to the training of your operators so that they know that this is an appropriate time to use this technology and this is an inappropriate time to use this technology, keeping with the aircraft analogy before. If your aircraft is only rated to fly at 40,000 feet, do not fly it at 80,000 feet, right? And so that gets to sort of what is the appropriate domain or mission use case for a technology and only using technologies for applications for which they have been appropriately and rigorously tested.

• 41:33Speaker 04

  And I think this is really valuable. We see AI ethics and AI safety as intimately interconnected. And that's reflected in our approach. I think one of the other things that the DIB study very helpfully pointed out is that the Department of Defense is not starting from scratch when it comes to reliability of incredibly complicated technologies operating in circumstances with life and death consequences. The Department of Defense stuff that uses technology in applications that are incredibly complicated, incredibly difficult. And not only that, there's potentially an adversary out there who is trying to make you fail

• 42:17Speaker 04

  in addition to the inherent difficulties of the technology.

• 42:21Chris Benson?

  That's a good point.

• 42:22Speaker 04

  And we have a lot of experience doing really extraordinary work in that regard. One example that I like to point out is a system called GCAS. This is something that exists in the fighter aircraft that the United States uses. And if you know anything about what it's like to be a fighter pilot, you know that they undergo some really intense acceleration periods. And those acceleration periods can cause the pilot to blackout or to red out. And part of the autopilot of this system, which is a software system mixed in with hardware developed here in the United States, can automatically detect if the pilot has blacked out and then fly the plane level and straight so they don't crash into the ground,

• 43:05Speaker 04

  fly the plane level until the pilot regains consciousness and then transfer control back to the human. I mean, so it's not as though the Department of Defense is starting from scratch here. A lot of the techniques and processes and policies that we have rolled out related to traditional software will serve us very well in an Era of AI powered software. And I think with the with the dibs sort of long form report. This is a 70 page report goes into quite some depth. A lot of what it's helpful for is pointing out, you know, here is where your existing processes kind of already do what you need to be done. And here are some areas where AI really does change the game. And you need to go through the hard work of improving upon these policies and processes.

• 43:49Chris Benson?

  Absolutely. It just is a comment to finish that up. I want to note that for the F 16, that system you described is actually a Lockheed Martin system. I end up talking about that with with folks quite a lot. I really appreciate you kind of diving into that because that's a question that so many people have in terms of the ethical and talking about lethal force policy and how you guys think about it. Explaining how it fits into the law of war is very helpful as a framework. I guess as we kind of close out the show, I was wanting to just kind of understand as the Jake is going forward over the next few years. And we are in this this industry that is moving so fast with all of the the research and the new algorithms and new techniques and the compute that's racing along.

• 44:33Chris Benson?

  How does the Jake look at AI related technologies going forward? What kinds of things draw your attention and make you sit up and go? That's something that has use cases that we particularly care about. What does that look like as you're seeing new things come across your screen?

• 44:52Speaker 04

  Sure, I think the the sort of first answer is it's much of what you would expect. There have been some really exciting developments in computer vision and natural language processing. You know, these are two areas in which the performance of machine learning systems today is just orders of magnitude better than where we were a decade ago. And I think we still have not actually finished, you know, harvesting those gains. There are many parts of the Department of Defense where a lot of people's time and energy is devoted to, you know, applications where machine learning technology is is already ready to start helping them do their jobs. And so I think

• 45:34Speaker 04

  The sort of next couple of years will continue to be focused on adopting the technology that already exists and is already pretty capable and powerful. I think over the longer term, there are some really exciting things coming down the pike. Of course, I think the first folks to get to play around with these technologies will be in DARPA and the service laboratories, who, as I said, their mandate is a bit more focused on advancing the state of the art in AI technology. But these are things like the increasing relevance of transfer learning. So, ML systems, if they learn something in one domain, can that help them not start from scratch as they look at related problems in perhaps different data sets?

• 46:17Speaker 04

  So, for example, I think all of your listeners are probably familiar with the AlphaGo example. So, the AlphaGo system was literally better than the world's best Go player, but that system not only could not play chess, it could not play Go on a different sized Go board. So, some Go boards are 16 by 16, some are 32 by 32, etc. And so, it really had no ability to apply what it had learned about Go and make that a better player for playing chess. And so, that transfer learning capability was not present when AlphaGo. There have been some really interesting research results that indicate that researchers are making more and more progress

• 47:02Speaker 04

  at tackling the transfer learning problem. And so, that could be really useful to DOD. If we're thinking about analyzing satellite images from a desert environment, might that actually make our algorithms perform better if they are also looking at images in a seaborne environment, for example? And so, that ability to sort of combine what the machine learning system learns from different domains is something that I think is just going to have a ton of benefit for us.

• 47:32Chris Benson?

  All right. Well, Greg, this has been a truly fascinating conversation. Thank you so much for coming on to the Practical AI Podcast and kind of taking us through how AI integrates into the world of the DoD and defense at large. Really appreciate it. Thank you for coming on.

• 47:50Speaker 04

  Well, thanks very much for the opportunity. It was great to talk with you.

• 47:54Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes, give us a rating, go on your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for changelog is provided by Fastly. Learn more at Fastly.com. And we catch our errors before users do here at changelog because of Rollbar. Check them out at rollbar.com/changelog. And we're hosted on Linode Cloud servers. Head to linode.com/changelog. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in your email address,

• 48:37Speaker 00

  get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.