941: Is Responsible AI Possible? with Dr. Sarah Bird of Microsoft

Scott heads to Microsoft’s campus for the VS Code Insider Summit to sit down with Dr. Sarah Bird and explore what “Responsible AI” really means for developers. From protecting user privacy to keeping humans in the loop, they dig into how everyday coders can play a role in shaping AI’s future. Show Notes 00:00 Welcome to Syntax! 01:27 Brought to you by Sentry.io. 03:13 The path the machine learning. 04:44 How do you get to ‘Responsible AI’? 06:43 Is there such a thing as ‘Responsible AI’? 07:34 Does the average developer have a part to play? 09:12 How can AI tools protect inexperienced users? 11:55 Let’s talk about user and company privacy. 13:57 Are local tools and services becoming more viable? 15:06 Are people right to be skeptical? 16:58 The software developer role is fundamentally changing. 17:43 Human in the loop. 19:37 The career path to Responsible AI. 21:21 Sick Picks. Sick Picks Sarah: Japanese pottery Hit us up on Socials! Syntax: X Instagram Tiktok LinkedIn Threads Wes: X Instagram Tiktok LinkedIn Threads Scott: X Instagram Tiktok LinkedIn Threads Randy: X Instagram YouTube Threads

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Scott Tolinski?

  Hey what's up? This is Scott with Syntax and just a quick message before this episode gets started. This is going to be an interview that I did with Dr. Sarah Byrd on Microsoft's campus. I was invited to do a VS Code Insiders Summit and had the extreme pleasure of being able to sit down and chat with Sarah. Sarah is the Chief Product Officer of Responsible AI at Microsoft. But she's not just the Chief Product Officer of Responsible AI at Microsoft. Her credentials really speak for themselves. She's been doing an extreme amount of work in the open source community for a long time. And she is responsible for many projects which you may have heard of. In fact, I'm going to list off some of them right here. She co-founded Onyx, FairLearn, and OpenDP's Smart Noise,

• 0:44Scott Tolinski?

  and was a leader in PyTorch 1.0 and Interpret NL projects. She was an early member of the Machine Learning Systems Research Community. She co-founded the ML-CIS Research Conference and the Learning Systems Workshop. She has a PhD from UC Berkeley. She's been epically involved in the machine learning and AI world for a long, long time, longer than most of you have even heard about this stuff. So being able to sit down with Dr. Byrd and pick her brain about all kinds of stuff, from like, what is our role as individual developers with responsive AI? To just her general thoughts, and we even get a sick pick out of her. So before we kick this interview off, just a quick message. This podcast is brought to you by Sentry.

• 1:29Scott Tolinski?

  And in regards of AI and responsible AI, man, Sentry is doing some great stuff where it's letting you know of potential issues in your application. And using a new tool called Sear allows you to fix those right away. I had a situation just the other day where I had this really mysterious bug. You know, the CloudFlare worker was giving me some obtuse error, and then this was not surfacing correctly. I clicked the Sear button, and Sear was like... Hey, I think you might want to check this env variable. And that env variable I could not find in the error. I have no idea how Seer found the cause of this error. But sure enough, I went to CloudFlare's dashboard and my env variable that I was so confident was there

• 2:14Scott Tolinski?

  had a very slight typo in it. And I changed that, fixed it, was able to mark that as complete. It closed the GitHub issue. It fixed my problem and I was on my way. Shout out to Seer from Sentry. It's an awesome tool. All right, without further ado, here is the interview with Dr. Sarah Bird.

• 2:38Scott Tolinski?

  Welcome to Syntax. Today, we have a really special interview. I'm joined by Sarah Bird. She's the Chief Product Officer of Responsible AI at Microsoft. It's a tremendous pleasure to be able to sit down and chat and really pick your brain about some of the things about Responsible AI, especially from the perspective of our audience, which is primarily web developers, right? So Sarah, welcome to Syntax.

• 3:07Speaker 01

  Yeah, thank you for having me. I'm so excited to be here.

• 3:10Scott Tolinski?

  Yeah, so with your background, you've been working machine learning quite a while. How did that end up happening for you? Like, how did you go down that path into machine learning?

• 3:23Speaker 01

  Yeah, I actually started my career in systems. So I actually originally in college worked in chip design at IBM. And I worked on the design of the original Xbox 360 processor. And so I started working in my PhD on operating systems. It always kind of felt wrong to me that systems were just kind of dumb and always did the same thing. And so my thesis was in using kind of

• 3:58Speaker 01

  Dynamic optimization techniques, convex optimization, which is a key thing in machine learning, to automatically adapt resource allocation in the operating system. And so I was using machine learning to make systems smarter. And it turned out to be the same time that we started to see systems is really important for machine learning. We wouldn't have had all of the breakthroughs that we've had in AI today if we hadn't been able to scale up the training algorithms, if we hadn't been able to scale up the data delivery and everything. And so I really just got started at the right time, at the intersection of systems and machine learning. And I think that it is so amazing to put these different types of technologies together and achieve something that we just really couldn't do before.

• 4:43Scott Tolinski?

  Yeah. So how does that take you down to the path of responsible AI? Because it seems like the machine learning space has evolved considerably in the past. I mean, just consistently, right? So how do you get to responsible AI from there?

• 5:00Speaker 01

  So I came to Microsoft for my postdoc in Microsoft Research in New York City, where I still am. And I was working on reinforcement learning, which was a really exciting new technology right when the deep learning boom started happening. And we were trying to figure out what were the most important problems we needed to solve to make this technology usable by more people. And so I was talking to various customers, potential early adopters, saying, what do you want to do with the technology? And we were calling it a decision technology because the way that reinforcement learning works is it tries different things.

• 5:46Speaker 01

  For example, showing you a sports article and then showing you a cooking article to see which one you care about more. And it learns as it goes. And at that time, customers said, oh, can we use it to automatically interview people? And that was the moment where I'm like, well, maybe you could, but should you?

• 6:07Scott Tolinski?

  Yeah.

• 6:07Speaker 01

  And I was lucky to be sitting with amazing, what are now leaders in the responsible AI space, like Hannah Wallach and Kate Crawford. And we came together and formed the first kind of responsible AI research group, which is called FATE at Microsoft at that time. And that was really how we got started in it. And I've stayed in it because I think it's one of the most challenging and interesting problems to make the technology work really well in practice in every single use case in a way that people can be confident in it and actually trust it.

• 6:43Scott Tolinski?

  Yeah, I guess that leads me to my next question, which is, is there such a thing as responsible AI?

• 6:51Speaker 01

  Yeah, the term is maybe a little bit problematic in that sense where what it's about is about humans being responsible with the technology and the way that we develop the technology and the way we deploy the technology and the way people use the technology. And so it's really not about the AI being responsible. It's about responsible development of AI. And people sometimes get that confused. I have a lot of people ask me about how do we make AI ethical? And it's just another technology. We shouldn't be making it ethical. We should be developing it in a way that aligns with the principles we're trying to achieve.

• 7:33Scott Tolinski?

  Yeah, that makes sense. I think a lot of, especially in the web developer space, it feels like, okay, these are tools, they've been given to me by my company, or we're subscribing to them, whatever. I have no part in this equation. Do you think that the individual developer in the web space or otherwise has a part to play here?

• 7:58Speaker 01

  I completely understand the feeling and the concern, but absolutely they have a part to play. A lot of responsibility is making sure that what you're developing really works well in practice. And so developers today absolutely have a responsibility to make sure that they're making code that is secure, for example, or that handles data appropriately and follows privacy policies. So in the same way, there's going to be things that you need to do in your specific use of AI that only you as the developer are going to understand. If you're designing an agent that, for example, is going to take very consequential actions,

• 8:43Speaker 01

  then you might need to design into your system having it go back and get human approval if there's a risk that it might be wrong. And so you need to do that tailoring to your specific application. And so, yeah, there's a lot that the developer needs to do. And the organization, anything that they do is part of the story too, but it's going to be a one-size-fits-all. It's not going to be necessarily what makes sense for your particular unique thing you're developing.

• 9:11Scott Tolinski?

  Yeah. Yeah, there is a sense that like people maybe just get overwhelmed with this technology and now these newfound powers. And it's like, okay, I guess maybe how I'm using these things isn't necessarily that important, but then you have security, privacy impacts, all those things. We're seeing apps that are being vibe-coded and pushed out now that have massive security holes where buckets are left open and this or that happens. How do we like bake security into these processes and safeguard ourselves from creating these things if the people who are pushing the software out don't know any better?

• 9:53Speaker 01

  Yeah, I love this question because...

• 9:57Speaker 01

  What I like to do as much as possible and where I see my role is to figure out what are the new risks we see emerging with AI. How do we actually address those risks? And then how do we make it easy for everyone to do that? And so, of course, as much as possible, we want to just bake it into the AI system. So if the AI model produces vulnerability in code, then we should try to train the model to do less of that. Or if you need to run your different sort of security checks when you check in a PR, the more that we can just integrate that directly into GitHub so it just runs, so you don't even have to think about it, the better.

• 10:39Speaker 01

  And so, as much as possible, we try to just bake it into the platform or system. But, you know, that doesn't work for everything. And part of what makes this actually work is people do need to understand their responsibility and their role in it. We do a lot today to educate developers on what they need to do to build production code and what the bar is to, you know, check something in. And we have, you know, checks and balances on that, right? Many people work in an organization where you have to have someone else review your PR before it's allowed to be approved. And so, we're still going to use all those same safeguards. There's not just, oh, okay, we'll go solve the problem for you. And so, education is part of that, too.

• 11:20Speaker 01

  And, you know, that's part of why I'm here, because it's so important that people like you are helping expose everyone to these concepts and, you know, what their responsibility are, because awareness is definitely part of the story.

• 11:33Scott Tolinski?

  Yeah, I never, I never thought about that too much with the role GitHub could play. But in the past, yeah, GitHub introduced things like, you know, warning you if you've committed secrets or something like that. Like, to me, I guess this falls right in line, right? Because at the end of the day, the end result's going on GitHub. It can use these systems to analyze your code. What about like? Privacy, I guess. And what role does that play? Because I think a lot of people do get a little weirded out with sending their code to a service somewhere. In that same regard, we have local models, things running locally. Like, what is the responsible choice there with your code as a developer in keeping

• 12:18Scott Tolinski?

  things private, I guess?

• 12:20Speaker 01

  Yeah. Privacy is something that's really important, both like a fundamental human right and then so very important in practice for businesses with their confidential data, their confidential IP. And one thing that's really great, actually, about this generation of AI technology that some people don't realize in terms of how it works is the foundation models themselves are trained usually in a big pre-training run. And then they're set and you can use them. And what they're really great at is actually reasoning over data that you are giving to it. So it doesn't need to be trained on the data to use the data effectively.

• 13:04Speaker 01

  And so we have a really nice potential privacy story there where you can send your data to the model. It's not learning from it at all, but you're getting very personalized response back. For example, a completion for your specific code. And so we've designed a lot of the systems that we've been developing at Microsoft and in GitHub to take advantage of that. But every kind of company and service is different, and so different ones have different privacy policies. And so it's important for your organization to actually look at the specific tools you're using, understand those privacy implications, and determine whether or not that's appropriate for your application.

• 13:45Speaker 01

  We see that healthcare data is going to have potentially a different standard than a sort of consumer entertainment information. And so you have to think about kind of all of those dimensions.

• 13:57Scott Tolinski?

  Yeah. Do you see the usage of local tools and local models, local services becoming?

• 14:04Scott Tolinski?

  More viable?

• 14:05Speaker 01

  I definitely see us going in that direction. So we released, for example, Foundry Local. Foundry is our AI platform and that enables you to take a lot of the models and run them locally. And there's just increasing demand for that because it's also good from like a latency point of view. It's good from a connectivity point of view. You can run offline. And so we want to enable AI to work as in many places as possible. But there is also cases where it's a really big model and you want the maximum power. And so running in the cloud is going to make sense. So I think we'll just see a diversity of these different options growing.

• 14:42Scott Tolinski?

  Yeah, yeah, I guess running locally too. I was on the plane here. There was no Wi-Fi in the plane and all of the AI tools. I'm like, ah, I should have, I should have set something up locally before I left here to at least have something. Yeah.

• 14:55Speaker 01

  Exactly. Airplane travel for all of us that travel a lot is like the number one use case. We're like, I know why we still need disconnected, even though, you know, plane Wi-Fi is getting better. But, you know, maybe not good enough yet.

• 15:05Scott Tolinski?

  Yeah, totally. I, we get a lot of comments on, on our channel specifically from like AI skeptics who see this tech as being scary, scary to their livelihood, scary to their code base possibly. Do you hear from a lot of people about being frightened about utilizing this stuff in software?

• 15:27Speaker 01

  Yeah, I think that we hear actually several kind of different types of concerns. And there's a taxonomy that came out in the International Safety Report that I quite like to think about this. And it says there's kind of three categories of concern or risk with AI. One is malfunctions. And so saying that, look, it's not good enough. It, you know, produces insecure code. Or it hallucinates. Those are examples that we're trying to work on to address like the malfunction in the system. Then you hear people say, OK, you know, I'm concerned that people are going to misuse it. They're going to use it to like automate hacking and, you know, cause more risk.

• 16:12Speaker 01

  And those are, you know, issues we have to address as well. And then the last one, as you said, around people's livelihood is systemic risk and concerns around what is the impact of this technology going to be on kind of the broader system? Is it going to change how we all have to work? And I think the answer is yes, it is going to change. And yes, we do need to be concerned about all of these types of risks. But part of the goal of responsible AI is to break down each of these and figure out what are the right ways to make progress on them. And then, at the end of the day, like, this is a tool and you need to find the right ways to put it to use. And it's not, you know, useful in every setting. And so it still really puts the developer in charge to be thoughtful about when it makes sense for them and when it maybe doesn't.

• 16:57Scott Tolinski?

  Yeah. Do you think it's inevitable that the role of a software developer is inherently going to be completely different in a short amount of time here?

• 17:08Speaker 01

  I think it's inevitable that the role of many, many jobs is going to be completely different in a short time here. And that's probably one of the things that keeps me up most at night. And not because I don't think we're going to get to a great place on the other side in terms of how much better jobs can be with tools like this. But the sort of upskilling and the redesigning of these jobs and doing kind of all of them at once, that's a significant challenge for us as a society and for each corporate leader. And so we've got a lot of work to do with this. And we need more people to contribute to that.

• 17:44Scott Tolinski?

  Yeah, and we were having a conversation, not you and I, we were having a conversation amongst the group yesterday about

• 17:51Scott Tolinski?

  Human in the loop and in general like what is what does that role look like and is there a future of of us not looking at the code as an interface at all, right? The code exists, but we're not actually manipulating the code or having a firm eye on it. And I it that's a it's a scary thought that that that human in the loop might eventually be gone there. Is that something that you think needs to stay as a requirement?

• 18:19Speaker 01

  So I think that we will always have human accountability like it's not going to work if we don't. And so that necessitates some concept of human in the loop. But what we will see is the way we put the human in the loop is going to be different. So if we go farther down, kind of the the future that you were talking about where people aren't looking at code anymore, it might be that we really need humans to put the investment in designing the acceptance test and signing off on the acceptance test. And so if you're sure that about how you want the system to behave and you are sure that the test really get that component to behave the way it's supposed to, why do you need to inspect inside of it?

• 19:00Speaker 01

  And so it's about designing the right mechanisms for human oversight and human in the loop. But those will absolutely change as what the technology change does, you know, and so it's going to be different over time.

• 19:14Scott Tolinski?

  Yeah, it's definitely one of those things you can get lost in a just a whole thinking for a long time. Yeah, I know certainly as somebody who likes to create things, there's this newfound superpower of just how many like personal software things you can create with low effort and you get into it and these are like low stakes. And once you get into higher stake stuff, it definitely always feels a bit heavier. For some people who are like kind of skeptics of AI, I do wonder like if learning more about responsible usage of AI would would benefit them. Is there like a a career path for getting deeper into like more responsible AI technologies and topics?

• 19:56Speaker 01

  Yeah, so first of all, I love the statement here because that's like exactly right in terms of the whole whole point of responsible AI. We also, you know, call it trustworthy AI. It's about Making the technology worthy of people's trust. And so I talked to many customers and developers and people who are skeptical, where we then walk them through, these are the safeguards we've put in place. And this is the testing we have. And this is the defense in-depth approach we use. And like, oh, I feel different about this technology now. That doesn't mean it's still it's suitable for every use case, right? But understanding how we put these pieces together to address the risk and how we're going to do that going forward is part of helping people

• 20:41Speaker 01

  understand why the technology can be worthy of trust in some use cases. And is it a career path? Absolutely. You know, before, 10 years ago, responsible AI wasn't a job. And now I've talked to people, actually even yesterday, they were saying, oh, I did my master's degree in responsible AI. And so it's gone from nothing to a thing where there are degree programs and where it is a job. And it's a very rapidly growing field, which has just been a joy for me to see.

• 21:12Scott Tolinski?

  Yeah. And I wonder if that's a better career path than just sticking your head in the sand and pretending this will all go away at some point here. Right? Yeah. So on this show, the last thing we do is ask for a sick pick, which is like anything in your life that is bringing you joy right now. Like anything that you're liking. We've done all kinds of things from podcasts to socks to anything. Do you have a sick pick or anything in your life that you're like, I love this right now?

• 21:40Speaker 01

  I absolutely love Japanese pottery right now. Maybe it's because AI and we're talking all about machines. But it's these beautiful handmade pieces that are designed to show the imperfection. But that's part of, in essence, what makes it perfect. And so I'm really liking the very visceral sort of. Physical element of that and the beauty that comes from that.

• 22:06Scott Tolinski?

  Man, I love that. There's a really nice Japanese pottery pop-up that is down the street from me and every time I walk by it, I'm just like, "Oh, I would love to go in there and buy everything."

• 22:17Speaker 01

  Yes, I would also feel the same.

• 22:20Scott Tolinski?

  Yeah. Well, thank you so much, Sarah. This has been incredible. I think a lot of developers are going to get a lot of out of this. So, thank you so much for joining me and just truly a great chance to talk to you here.

• 22:33Speaker 01

  Yeah, thank you so much for having me. It's such an important topic and near and dear to my heart, so I appreciate everyone taking the time to learn more.

• 22:40Scott Tolinski?

  Well, thank you.