From research to product at Azure AI

Bharat Sandhu, Director of Azure AI and Mixed Reality at Microsoft, joins Chris and Daniel to talk about how Microsoft is making AI accessible and productive for users, and how AI solutions can address real world challenges that customers face. He also shares Microsoft’s research-to-product process, along with the advances they have made in computer vision, image captioning, and how researchers were able to make AI that can describe images as well as people do. Sponsors: Linode https://linode.com/changelog – Get $100 in free credit to get started on Linode – our cloud of choice and the home of Changelog.com. Head to linode.com/changelog https://linode.com/changelog OR text CHANGELOG to 474747 to get instant access to that $100 in free credit. Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. LaunchDarkly https://launchdarkly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Power experimentation at any scale. Fast and reliable feature management for the modern enterprise. Featuring: Bharat Sandu – LinkedIn https://www.linkedin.com/in/bharatsandhuChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: What’s that? Microsoft’s latest breakthrough, now in Azure AI, describes images as well as people do https://blogs.microsoft.com/ai/azure-image-captioningFrom search to translation, AI research is improving Microsoft products https://blogs.microsoft.com/ai/mona-lisa-translation-research-productsAzure AI: Build mission-critical AI apps with new Cognitive Services capabilities https://azure.microsoft.com/en-us/blog/azure-ai-build-missioncritical-ai-apps-with-new-cognitive-services-capabilitiesUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Mixed reality or virtual reality or augmented reality is really good examples of applied AI. So when you're doing hand tracking, for example, that we do in HoloLens, you can imagine this is literally looking at your hands and saying, okay, this is finger is pointing one particular way, it's pointing that way, or it's I'm grasping my hand here. And to even create a data set, of course, you can capture a lot of hand angles, but you can also create different 3D orientations of a hand and have the labeled data set. So you can train your hand detection models and so forth. And even all these devices in HoloLens, you can speak to it. So there's already onboard speech recognition and so forth. So there is, one is, of course, is a very good example of applied AI for us. But the second one is for our customers and our developers. It is actually the simplifying and accelerating the development experience. And really, at the end of the day,

• 0:45Speaker 00

  helping customers realize more value out of these technologies.

• 0:49Speaker 01

  Being with for change log is provided by Fastly. Learn more at fastly.com. Our feature flags are powered by Launch Darkly. Check them out at launchdarkly.com. And we're hosted on Lino cloud servers. Get $100 in hosting credit at linode.com/changelog. What up, friends? You might not be aware, but we've been partnering with Lino since 2016. That's a long time ago. Way back when we first launched our open source platform that you now see at changelog.com, Lino was there to help us. And we are so grateful. Fast forward several years now, and Lino is still in our corner, behind the scenes, helping us to ensure we're running on the very best cloud infrastructure out there. We trust Lino.

• 1:32Speaker 01

  They keep it fast and they keep it simple. Get $100 in free credit at linode.com/changelog. Again, $100 in free credit at linode.com/changelog.

• 2:00Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at change.com/community and follow us on Twitter. We're at Practical AI FM.

• 2:26Chris Benson?

  Welcome to another edition of the Practical AI Podcast. My name is Chris Benson. I'm a Principal Emerging Tech Strategist at Lockheed Martin. And with me as always is Daniel Whiteknack, a data scientist with S I L International. How's it going today, Daniel?

• 2:39Daniel Whitenack?

  It's going great. I think we discussed when we recorded last week, which was pre-US Thanksgiving, our plans for Tofurkey for Thanksgiving, which I had. I'm not sure if you partook as well.

• 2:53Chris Benson?

  I did. So anyone who listens to us regularly hears Daniel and I talk about our plant-based meals. And I'll take a two-second divergence. I had an interesting entry into Thanksgiving dinner that has nothing to do with AI ML whatsoever. We were just about five, 10 minutes from sitting down, and I got a call from a neighbor saying, I have a large black snake in my house. Can you come help me? And so anyone that's listened to us knows I'm the animal.

• 3:18Daniel Whitenack?

  Yeah, and you're on call for that sort of thing, I think.

• 3:21Chris Benson?

  I am. So I like totally separate from all this, what we're talking about today. I do a bunch of animal stuff, and I actually run a snake hotline. How weird is that? Do you know anyone else that runs a snake hotline? So I get the call, I go, and I have a neighbor with a large black king snake in their basement. And they were very upset because they were not very snake savvy people. And so I calmed them down. We got the screams to stop. And I picked the king snake up and carried it delicately outside, released it, educated them along the way as the screaming stopped.

• 3:54Daniel Whitenack?

  You made an unexpected house visit on Thanksgiving.

• 3:57Chris Benson?

  It was. It was. And then I dashed home just in time to sit down for Thanksgiving dinner. So it was a memorable Thanksgiving dinner in that way. Yes, a little odd story that goes with it.

• 4:09Daniel Whitenack?

  Yeah, that's awesome. Yeah, well, we have some interesting things to discuss today and including, you know, if you were to have taken a picture of that experience, you might have wanted to come up with a caption for it. And that might have been somewhat difficult. That's one of the tasks that we're going to discuss today.

• 4:28Chris Benson?

  Indeed.

• 4:29Daniel Whitenack?

  Yeah. And we're really excited to have with us Bharat Sandhu, who is the Director of Azure AI and Mixed Reality at Microsoft. Welcome, Bharat.

• 4:39Speaker 00

  Thank you. Thank you for having me.

• 4:41Daniel Whitenack?

  Yeah. If you could just give us a little bit of an idea about your background and how you got into AI and what you're doing now.

• 4:48Speaker 00

  Yeah, sure. By the way, just for Thanksgiving, even we had a plant-based Thanksgiving this last week. So we joined, we joined you.

• 4:59Daniel Whitenack?

  Ah, good.

• 5:00Speaker 00

  No snakes, though.

• 5:00Daniel Whitenack?

  You have any, any recommendations on that front for our listeners? We always try to throw in an occasional recommendation.

• 5:07Speaker 00

  You know, I, we just tried like, so we tried to reduce the amount of meat we eat, my wife and I. So we tried it this Thanksgiving and we just cooked up some new recipes. The green beans recipe was, it turned out extremely well. It was spicy and, you know, it's just like, normally, it's a different kind of Thanksgiving food, a little bit of spice in it. But yeah, I don't know the, we just copied some recipe off on, on the web. So I can, I can, I can share that with you. And yeah.

• 5:34Daniel Whitenack?

  It's good. I don't know that mine ever look quite as tasty as the picture on the recipe, but sometimes, you know, just finding those recipes online, that it works out pretty good.

• 5:44Chris Benson?

  You know, before long, because this is not a new topic, we have had other folks on the show talking about this on the side before we got in the main topic. Pretty soon, we're going to need our own conference. It'll be like the Plant-Based AI ML and Data Science Practitioner Conference.

• 6:01Daniel Whitenack?

  Yeah.

• 6:01Chris Benson?

  Just saying.

• 6:03Speaker 00

  Yeah, it'll describe to you what you cooked and what you could have cooked. And actually, cooking is kind of like a machine learning, right? It's like not just the ingredients, it's how you shape them and the work that goes in it.

• 6:13Daniel Whitenack?

  I'm glad you bring that up, actually. It's one of my favorite examples that I give when I'm teaching workshops is a lot of industrial AI applications are really more like, you know, using a recipe and bringing your ingredients to it rather than sort of designing the recipe itself, right? And, you know, maybe that gets into some of how people use AI and other things which we can talk about. But, um, but yeah, before we get into that, I'd love to hear a little bit about your background. And yeah, how you ended up where you're at right now.

• 6:45Speaker 00

  Yeah, so, you know, I've been at Microsoft for a bit over 10 years, close to 11 years now. But, you know, I'm a computer engineer by training. My first job was a computer engineer writing code. And, you know, as before we got on the call, we were talking with, you know, Chris was Lockheed Martin. So my first job was this company at National Instruments. And, uh, you know, I started as an engineer, but then I moved out to work with customers in our field engineering and sales division back then.

• 7:10Daniel Whitenack?

  Are they the ones that make LabVIEW? Is that the?

• 7:13Speaker 00

  Yes, that's right.

• 7:14Daniel Whitenack?

  Yeah, okay. Yeah, that's my experience with National Instruments. For those that don't know, LabVIEW is like software that helps you interface with all sorts of different instruments and circuits and design various things. And yeah, anyway.

• 7:27Speaker 00

  Yeah. And some of the work, you know, like, so if you're, if you're an engineer by training, you know, and you, and you start working with customers like Sikorsky, making the Black Hawk helicopter. United Technologies, the fuel cell, and Pratt and Whitney, making the, you know, the engines for, I think, uh, a lot of fighters, including the jet, uh, F-35 now.

• 7:45Daniel Whitenack?

  Correct.

• 7:46Speaker 00

  It was quite fascinating because back in the day, not that many companies were collecting a lot of data, but these companies were collecting a ton of data and kind of building, you know, hardware in the loop, uh, testing and those kind of things. So it was a fascinating time and, you know, so that was my kind of like early career learnings, working with folks, building literally jet engines and working with tons and tons of data and trying to make sense of them. We actually focus a lot on hardware in the loop testing, which is before you actually build something and you want to. You know, instrument it all up, you want to maybe do some simulation. So you know, MATLAB and Simulink were common players we worked with, competed with also back in the day. And you know, and nevertheless, I did that and got my MBA, then I joined Microsoft. And in Microsoft, my career has mostly been commercializing incubation businesses.

• 8:31Speaker 00

  So first was unified communications, which is funny we're talking about over Zoom right now. But 11 years ago, you know, at least for Microsoft, it was a new entry back then. And then, you know, I joined our IOT, Azure IOT team. We were launching our Azure IOT offering. And then the big data space in the last three years, we lead the AI space and very recently the mixed reality. And the common trend has been commercializing these very emerging new technologies. And you know, making it a viable business value proposition for companies, for our partners and for our customers. Making it easy for them to actually benefit from these technologies.

• 9:08Daniel Whitenack?

  You know, it's pretty interesting. I think a lot of companies that are getting into AI, like whatever AI based system they're creating might be one of those things that they're incubating within their own organization. So do you have any insights from your experience kind of productizing these different, you know, incubated ideas into actual offerings? Do you have any insights for those out there that are maybe their AI based system? Is there internal incubation project? Any suggestions for them or advice?

• 9:40Speaker 00

  Yeah, you know, common, like first of all, you know, technologies come and go, you know. Deep neural networks are fancy today, will be quite passe in a few years' time, right? So like, just like reinforcement learning is kind of coming into vogue in some areas. But these are just technologies, so never please get fascinated by technology. And and at least for those of us who work in technology companies, have to be very careful of that. Because we get super excited about new techniques. It really is actually, at the end of the day, business has not changed. You know, we have customers and we have to serve them better. And we have competition and we have to provide better offerings than the competition. So. Being super grounded, whether a project's going to help dramatically reduce

• 10:22Speaker 00

  cost, improve operations, or increase revenue is the most fundamental starting point. And, you know, I have seen, you know, I think we've all have a lot of organizations have incubation or innovation offices, which is great, but you know, you know, as we work with customers, it's always like, okay, what is the core business need? And more importantly, have the key stakeholders bought into it upfront and then decide up for maybe a year-long project, or maybe sometimes longer, to prove out whether this kind of venture, in this case, AI for this particular application, is actually going to provide value for the business or not. So just setting up the project in a very methodical way with key stakeholders bought

• 11:05Speaker 00

  in all the way upfront, and then having regular meetings and making sure we have the KPIs identified or improving them as we go through it is super important. You know, so that'll be the one thing we'll say, like at Microsoft, we kind of make sure we stay true to that. At the same time, you know, for us, we have the luxury, like with some other companies like Google and Apple and Amazon, to do a lot of just pure research. But a lot of companies in the world do not have the, you know, the luxury sometimes. So especially if their core competency is not technology or AI, machine learning. So for these companies, it's our, I think, duty to help them kind of walk through how they can evaluate projects, not just technologies.

• 11:44Chris Benson?

  Gotcha.

• 11:45Speaker 00

  So I'll just go back to kind of going back to business basics.

• 11:47Chris Benson?

  When I was introducing you and I was mentioning your title, it's one thing I was wanting to follow up on is, as part of your title is mixed reality. And we hear about all sorts of different types of, you know, something reality, virtual reality. Augmented reality. Augmented, but what is mixed reality, or at least how does, how does Microsoft define that?

• 12:06Speaker 00

  Yeah, sure. So mixed reality for us, you know, and I think X-reality is a new term that's also started kicking around. But basically, like if you look at our, the way we interface with our phones or laptops, it's a 2D screen. And then you have things like Oculus Rift or Windows Virtual Reality headsets, and I got an Oculus a few weeks back, I love it, and Quest, sorry, Oculus, it's a virtual reality. So there, you're in a virtual reality environment. But you're in a separate environment, you're totally removed from the physical space. And then you've augmented reality, which HoloLens is an example, Magic Leap is another example, where you can actually see through and you can see holograms. So you can have presence of the world around you as well as virtual elements in it.

• 12:48Speaker 00

  Now, mixed reality is, you know, is a term that spans augmented and virtual reality. Because what you'll see happening more and more, and even with Quest, I was quite impressed to see, you can actually see through and see the augmented reality. So from a development platform perspective, you should not have two separate set of tools and development environments and, you know, Unity for this and Unreal for that. You really want to have a consistent development platform, a consistent ecosystem. And really for our customers, you use these technologies. Some, like, well, of course, have a better experience in virtual reality, like gaming. But for a lot of like industrial and commercial use cases, augmented reality is much more effective. But they also have a seamless experience through these different devices.

• 13:30Speaker 00

  So mixed reality, as a term, really is meant to span the world of virtual reality and augmented reality. And really say it's a common development platform and a common user experience. It's really more of a term at this point, but it is meant to bridge the two different worlds right now.

• 13:45Chris Benson?

  It sounds like kind of that that element of common has a lot to do with it in terms of spanning, so.

• 13:49Speaker 00

  Yes.

• 13:50Chris Benson?

  Would it be fair to almost say it's kind of taking augmented reality and applying tooling, common tooling across so that your your productivity is improved without some of the the natural distinctions on the two sides of it?

• 14:03Speaker 00

  Absolutely, yes. So if you're a developer, you don't have to learn two separate types of development stacks and so so forth, right? Some, there of course be some nuances, right? But more and more, as much as we can make it easier for customers to go from an iOS or a phone-based augmented reality system to really like holographic, like HoloLens can experience to even virtual reality experience. And the developers don't have to learn a different stack every single time, different set of tools every single time. I think that's added a value to the ecosystem. So that's one thing. As an end customer also, we're not there yet, but we have a consistent experience, they'll have a consistent experience through the different devices and so so forth.

• 14:45Speaker 00

  But yeah, mixed reality is basically trying to say is a bridging of these virtual reality and augmented reality worlds.

• 14:52Daniel Whitenack?

  And is that where the kind of common thread with the other things you're involved with in terms of the Azure AI platform, I know there's, you know, just from my own experience, so vastly different number of environments that people use for for developing AI. Are part of the goals with the Azure AI platform similar in the sense of bridging some of those and creating a unified development experience?

• 15:17Speaker 00

  Yeah, also like AI is, I would say, a much more mature place than mixed reality or AR so far. But yes, I think there's a common, like, you know, basically just simplifying the developer experience is a common thread for sure. The other thing is also, by the way, mixed reality or virtual reality or augmented reality is really good examples of applied AI. So when you're doing hand tracking, for example, that we do in HoloLens, like, you can imagine this is literally looking at your hands and saying, okay, this is finger is pointing one particular way, pointing that way, or I'm grasping my hand here. And to even create a data set, you know, of course, you can capture a lot of hand angles, but you can also, in a 3D studio like Maya or something, create different 3D orientations of a hand and have the labeled data set.

• 16:00Speaker 00

  So you can train your hand detection models and so forth. And even all these devices in HoloLens, you can speak to it. So there's already onboard speech recognition and so forth. So there is, one is, of course, is a very good example of applied AI for us. But the second one is for our customers and our developers, it is actually the simplifying and accelerating the developer development experience. And really, at the end of the day, helping customers realize more value out of these technologies.

• 16:26Daniel Whitenack?

  So as you describe it that way, and would it be fair to say if you take, you know, applied AI, you know, meaning deep learning, reinforcement learning, things like that, and enabling the mixed reality environment or experience with these technologies, and that becomes kind of the use case, if you will, for the applied AI as an enablement technology, is that a fair way of of talking about the relationship between them?

• 16:49Speaker 00

  Yeah, yeah, absolutely. And of course, not every single application needs to have. Have the customer, the partner build an AI app element to it too. But like, you know, we, we had this example in, uh, I believe it was Disney where they actually had a depth sensor sensing people walk up to a screen and I think they were doing the stranger things. Yes, they were doing the stranger thing enactment and then the monster, the stranger thing will come out and say, who are you? You'll say your name, it will reckon, they will use, uh, speech to text or speech on, uh, recognition to, you know, to understand your name, but then do modulation of the speech and speak to text your speech to speak back to you and to understand you and so forth. So there you actually have blending of AR and AI from a customer perspective too. So, you know, right now, these are two distinct

• 17:32Speaker 00

  businesses for us. Uh, you know, some places they overlap when it comes to customer use cases, but a lot of the way we take these technologies to market has a lot of commonalities of kind of going from super early incubation into commercial businesses for us. So there's a business connectivity tissue of us.

• 17:59Speaker 02

  Change log plus plus is the best way for you to directly support practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal and help sustain our production of practical AI into the future. Simply follow the change log plus plus link in your show notes or point your favorite web browser to changelog.com slash plus plus. Once again, that's changelog.com slash plus plus.

• 18:50Daniel Whitenack?

  So Brad, I'm kind of interested, you know, while we're talking about the Azure AI platform, to dig in a bit to what developing AI in the cloud looks like now. So in its very sort of raw form, I imagine people might think, oh, I could spin up a cloud instance or a virtual machine, maybe attach a GPU to that and do some AI things. But of course, each cloud provider has developed a whole suite of different things that, you know, aid AI development. What does that picture look like in Azure AI right now?

• 19:27Speaker 00

  Yeah. So you know, our AI stack, do you mind if I just explain what our

• 19:32Daniel Whitenack?

  Yeah, please do.

• 19:33Speaker 00

  How we take AI into the market, what Azure is.

• 19:34Daniel Whitenack?

  Absolutely.

• 19:35Speaker 00

  So first of all, you know, like, you know, there's a ton of innovation in AI in the open source world, which is awesomeness. So, you know, we, our goal is to make sure we can provide that to our customers in a more packaged fashion. We also have a, you know, ton of research centers around the world. We have eight research centers and the work they do is really fine tuning and developing algorithms, you know, and their charter has really been to kind of like say, okay, let's go and build the best text to speech or speech to text or image captioning that we'll cover later on, techniques. Now, these are researchers and PhDs in machine learning, you know, we have more of those now than we used to have maybe many years back. But they specialize in really deep like machine learning space. We take all the work they do and then we bring it to

• 20:18Speaker 00

  Azure AI. And our goal on Azure AI is to use all the open source technologies available, all the work we're doing in that space too, and taking all the research milestones and baking them on a platform that we can give to developers and data scientists. So then they can use it from a simple, no cost, free API call to kind of doing highly high scale reinforcement learning techniques. We also, by the way, test this stuff in our own products and we have the, I think, we're just lucky to have many different types of endpoints from Teams. Which if you were to join a team session now, you can do a transcription of all the calls. They can do recording and transcription and translation, right? So all that's powered by Azure AI. Within PowerPoint, if you've used it recently, there's something called PowerPoint Designer

• 21:04Speaker 00

  that looks at your PowerPoint bullet points and says, okay, hey, maybe you want to actually have a better representation of the slide. And it's really using Azure Machine Learning under the covers to do all the inference thing and give you those recommendations. And even Xbox, you know, the holiday season is here, but if you have an Xbox, if you log in, you're given a personalized screen. So if you log in Daniel or Chris, you've got a different dashboard. And that's powered by Azure AI and a service called Personalizer that uses reinforcement learning on behind the scenes to do very personalized recommendations for folks. And the same technology is available to developers. So our approach to bringing AI to developers and data scientists is we take things in the open source world, we of course make do our own research and then we package it

• 21:47Speaker 00

  and really goal is to make it usable for customers, whether they're very professional data scientists, so give them access to, you know, the best collaborated notebook environments we can with highly scalable compute, so they can do the really highest end training work they need to do, all the way to simple APIs that any developer can call using multiple any of the language of the choice and add AI to the application in a very simple fashion. And then what we have also done recently is we started creating what we call scenario specific app services, and we can get into that. A good example would be a metrics advisor where we have an anomaly detection models running behind the scenes, but they also have a bunch of other logic around them to ingest

• 22:29Speaker 00

  data from multiple time series sources of data, to do root cause analysis and to have composable UX elements that a developer can embed in the application. So if they want to do more of metrics monitoring. We don't just give them a model, but we give them a higher level, still a past service that does a lot of the other work and the other business logic for that thing. So that really becomes a platform. So at the base, we have Azure Machine Learning, which is used to train machine learning models. On top of that, we have customizable AI models. We call them cognitive services. Those are the speech, vision, languages, and so forth. And then on top of that, we have a set of services that are really tuned for separate kind of scenarios, like form recognition to metrics, monitoring, and so forth.

• 23:13Speaker 00

  And that's really a stack for Azure AI. We do go one step beyond that, and we have a platform called Power Platform. I'm not sure if you are aware of that. It is basically a platform for business users. It's a no-code, low-code environment and really allows citizen developers really to go ahead and design applications. So all the Azure AI tech that I described is provided in a low-code, no-code way in this environment for, you know, anybody with a citizen developer to to build AI into their applications without having to write code.

• 23:46Chris Benson?

  That's fantastic. So first of all, I want to say regarding my next question, you've actually answered it in a fantastic format in kind of going through a lot of the benefits. But I also want to kind of distill it a little bit because so much of our audience is our practitioners.

• 24:01Speaker 00

  That's right.

• 24:01Chris Benson?

  That are engaged in AI. And it is very, very typical for them to be in an environment where they're going to engage in a project and they may have access not only to Microsoft, but competitors as well. And they're trying to justify that. If a practitioner has just listened to this episode, and we have more to go, obviously, but they're like, I want to do Microsoft AI on Azure. How would you differentiate it? Or how would you recommend they differentiate it for their manager or whoever they are having to report back to to say, I definitely don't want to go with the other guys. I want to go with this one. How do you position that?

• 24:35Speaker 00

  First of all, I must say it's better for the AI field if the customers go with anybody and they do an amazing AI project, right?

• 24:42Chris Benson?

  Understood.

• 24:43Speaker 00

  I just want to acknowledge that, like, look, they have great competition, and that's great for the industry. And we're excited to play a role in this thing. But what guides us is our core three main investments, which You know, we've had the fortune of working with multiple enterprises over many, many decades as a company. So we really hone into kind of making things enterprise great. So really breaks onto the three main things. First is making sure we can give the tools to developers and bring all the heritage of ID tools and so so forth on their terms, which means they can use the tools and frameworks of the choice. Whether the professional data scientists that love Jupiter notebooks, right? And then they wanna use TensorFlow, PyTorch, great, but then have it optimized for the in high scale training,

• 25:26Speaker 00

  the kind of work we do with OpenAI, for example, and give them really fleets of very high powerful you know, infrastructure, computer infrastructure, which is just no GPUs, but also the networking layer for them to be able to do training very cost effectively at the end of the day and highly reliably, right? But but that's for what I would call the most more professional practitioners, right? Who are really looking for tooling to do ML in a scalable, reproducible fashion. And that's really the Azure machine learning layer, where our investments are making that really robust machine learning platform on the cloud, where the class layer, you know, is really resonating with the customers. But the second one would be for customers who are not deep machine learning experts.

• 26:07Speaker 00

  And for them, like, it's easy to say, hey, you can have an API call and you can have an AI model in your application. But the AI model has to be really good. Otherwise, it will not, you know, it will work, but it will work in a bad way. So our commitment to quality, whether it's speech to text or text to speech, or you're getting Turing models, which are very large scale models, right? But available through an API call. So your language or text analytics is very high fidelity. And then also the high end services. So it's just not having models, but ability to kind of like get to market faster for form recognition and other scenarios like we do indexing and so so forth. You know, we just have the services that, you know, a lot of times our competitors sometimes don't have it at this level we do.

• 26:51Speaker 00

  And be able to kind of provide this level of different various various level of access points for developers to work on, but work on one one environment together. So that's one thing. Other two things I was just saying, which I think we sometimes take for granted, but it's super valuable for our customers is really baking them and making them enterprise grade. Like, it might sound boring, but things like VNet and, you know, having ability to run these in like, you know, environments that are highly regulated and having the certifications, having all the services and models be certified, you know, and so forth. This goes a long way. And last one, which, you know, has surprised us too, is our commitment to responsible AI is proving

• 27:35Speaker 00

  us to kind of like allow customers to take AI and use it in their applications much more effectively. So if you have long, not just principles, but even like tooling that we provide to our customers and guidance that we provide to apply AI in a responsible fashion. So so really, those are the three things. One is like allowing customers to kind of get access to AI on their terms, whether the professional data scientists or just developers who don't have machine learning expertise at all, but want to get access to AI. Second, giving them an enterprise grade fashion so they can deploy with confidence. And third one, also giving them responsible AI, not just guidance, but also tools and capabilities to do it. And having it all built into our into our products, not just you can go to this doc here,

• 28:16Speaker 00

  you can go to this GitHub repo there, it's all available as part of our platform.

• 28:21Chris Benson?

  That was a great explanation. You mentioned along the way, OpenAI, and I know that Microsoft has an exclusive relationship on GPT-3, which obviously the whole industry is excited about with OpenAI. I'm assuming that at some point that becomes pretty normally accessible via Azure. Anything you can share on that along where that's going to go or what we should expect?

• 28:43Speaker 00

  Yeah, you know, we're still working with them on a lot of these things right now. Like right now, what we want to make sure is we can enable OpenAI to do really breakthrough AI research and to give them amazing cloud resources. The things we spoke about, I think we have now the fifth largest supercomputer, maybe probably the first one in the cloud that they access to build these models. You know, these are about 300,000 CPU cores, 10,000 GPUs and the networking layer that goes with it, you know. And then also it allows us to kind of develop our optimization. So you might have heard of something of Onyx runtime. Uh, which is very used to be for interesting, high speed, interesting, but we've also tuned in to do high speed training. But all those optimizations kind of came in also with the work that we've been doing with OpenAI and even internally and all that stuff.

• 29:25Speaker 00

  So yeah, more to come on where some of this work shows up. But you know, uh, we also, I think, you know, our goal is to allow OpenAI to do amazing work like G P T three and give them the ability to do it from tooling and all that. And all that also shows up for our customers, even if they don't get, let's say, access to G P T three model today, right, uh, which they can get from OpenAI. But all the work that went into enabling OpenAI to do the work is available to our customers also.

• 29:52Daniel Whitenack?

  Yeah, just before we leave the subject of the Azure AI platform, where are some good, good resources for people to dig in and get started, maybe specifically for data scientists or software engineers who are wanting to get their hands dirty, you know, uh, working a little bit with the Azure AI platform? Where can, uh, where can they get started? Any recommendations for them as they're sort of onboarding?

• 30:16Speaker 00

  Yeah, you know, so I'll say they're like, depending on where they are, three main ways I'll recommend them. If they're already using machine, they're already know machine learning, you know, the machine learning practitioners, you know, just, just jump into Azure machine learning. So, you know, the documentation, we have a bunch of courses on our Microsoft Learn, we have them on Udacity too now and other courses, you can take courses. But if you know machine learning, the best way is to kind of just dive into the product. And again, the product's really designed for M L as a team sport. So, uh, for, especially for folks that have AI teams or data science teams, that data engineers, you know, scientists, and they want to work in a collaborative environment, they'll really benefit from A M L and having it to do it in a, with, with really modular M L pipelines. So it'd be great to jump into the product and learn about, you know, what an M L platform

• 31:01Speaker 00

  in the cloud provides like Azure machine learning. Uh, so just go to Azure machine learning, M L dot Azure dot com. And if you don't have Azure subscription, you'll be able to go sign it up. It's free and you'll get access to it. Uh, and that's a great place you could learn. Documentation is always a really good place and easy place for people to access. If folks who are wanting to learn and same with other rest of the portfolio. Uh, you can also go to www.azur.com slash AI. That will take you to all the different offerings we have. Uh, but if folks who are interested in learning machine learning, who might not be experts, you know, we recently partnered with Udacity to launch a couple of courses and they're really good in-depth courses that have been developed by us in partnership with them. So that'll be the other second piece I'll recommend them. We also have great training on our Microsoft learning website.

• 31:47Speaker 00

  And there can go deep in different areas. So those will be the three main areas. The product itself, you know, and please give us feedback for what you don't like so we can improve it. And the learning courses that we just mentioned.

• 32:18Daniel Whitenack?

  So Bharat, in our conversations prior to the, the show, we had talked a little bit about some of the work that Microsoft is doing research wise and productizing research to enable computer vision to be used to help people who are blind or have, have low vision. And we talked a little bit about the seeing AI app and other things. But I was wondering if you could kind of just give us a little bit of a briefing on Microsoft's work in this area. And I know that they had a pretty interesting milestone in terms of image capturing. So maybe you can go into that a little bit.

• 32:55Speaker 00

  Yeah, you know, as I mentioned earlier, like a big focus is kind of doing good research, but then putting it to products. And things we've done in the responsible AI work for model interpretability, fairness detection and all that, which is available in Azure machine learning. We did the same thing very recently for what the cognitive services family of products, which is the customizable AI models. In particular, we did the image captioning milestone. So you might know this is basically novel image captioning benchmark. And basically, you know, if you look at image captioning, it is a technique where you show the computer an image and it says it's an image of blah. Normally, the way these things have been trained is by actually giving them, you know, images and complete sentences of what these images are. So what the research team had did was something different.

• 33:41Speaker 00

  So they actually kind of went about how children will learn vocabulary. So when we're learning, we basically say, hey, there's an apple. And there's a picture of an apple, and this is an apple. There's a picture of an orange, here's a, this is an orange. We never said this is an apple sitting on a table.

• 33:58Speaker 00

  So, so that's kind of like the first process that he went through, which is building a visual vocabulary by really training object and what the object was. Then that was how they did the pre-training of the model. Then for the fine tuning, they actually then started giving full sentences to the image, to the model. And so through the inference time, what it first does is it does a visual vocabulary. So it's an image, it contains A, B, C. Then, based on this fine tuning step, it is able to put together sentences together. So we've had image captions for a while. But, you know, we really dramatically improved it. So some couple of examples was, it was a picture of, and I'm happy, I'm happy to share these before and after.

• 34:41Speaker 00

  It's easy to see it and then explain it. But imagine a picture of a doctor with a stethoscope. Before, it said a woman looking at a camera. Now it says a woman wearing a stethoscope around her neck. So very much more accurate. Before this picture for a satellite looking up in the sky, it said a yellow sunset. Because, yeah, there was a sunset. But now it says a satellite dish on a dirt road. So, so basically, it was because of the ability to kind of the way they trained the model. And really, it was actually also done, and there was a benchmark that they were able to beat. And really, really achieve human parity, which really means, you know, the computer's describing images in the same way that you or I would describe images, or sometimes even better. So that was kind of the research milestone. But, you know, these research milestones are very fascinating.

• 35:27Speaker 00

  Organizations have them around the world all the time. What are, you know, what we want to do was bring them to our developers. So in record time, we brought them to Cognitive Services. It's a feature of Computer Vision API that's available as part of Cognitive Services. There's a free tier, people can try it. We've also, by the way, embedded it into our applications. So in PowerPoint and Word, if you insert an image, you can do Alt Text to describe the image. And, you know, that's models there now. Cine AI is the application that was developed. To really help people with blindness to see the world around them. And you can only imagine the kind of impact this kind of technology can have in their world. So that's a quick, super quick overview of the research that went in.

• 36:09Speaker 00

  And then really how we're bringing it to developers and customers.

• 36:13Daniel Whitenack?

  I think this is amazing. You know, oftentimes on the show, we talk about various things that, you know, like object recognition or computer vision or other things that are either seen by a lot of people as, you know, competitions to be a part of or, you know, benchmarks to break. But really seeing how that can actually connect to benefit people in a really positive way. So impacting people in terms of the Seeing AI app or, you know, helping accessibility in other ways. That's, that's a really amazing place where AI can have a contribution.

• 36:51Daniel Whitenack?

  So I really appreciate you, you know, sharing that and Microsoft for really emphasizing that because it is really amazing work. How long has the Seeing AI app been around? And have there been impressions of the app from the blind or low vision community?

• 37:08Speaker 00

  Yeah, it's been there for many years. I'd love to go see like how many exactly is, but at least the last five years, I believe it's been out there, five, four years. I believe it runs on award two about from in the industry, you know, for disabilities. But, you know, like those things are great, but really like at the end of the day, I think it's still a ability to provide something to the world and improves people lives. It really goes to our mission of empowering people, all kind of people to do more with what they have. So it's been there for a while. It has good adoption. Shaqib, who created it is blind himself. He works at Microsoft. He's, you know, and, and we're just super honored to have this as one example of us kind of helping customers do more.

• 37:51Speaker 00

  And the other example you might not know is something we have called Immersive Reader, which is also, by the way, available as a cognitive service that, you know, developers can put in their applications. But you can just, if you download Edge Browser, you know, you can right click on the. And it can says read aloud or immersive reading. So it does a lot of different things. It allows you to blow up the text, allow you to focus on separate parts of the text. Of course, read it, read the text to you and so forth. But that's another like, you know, thing we've been doing to use AI to help people with disabilities just do their job fairly well.

• 38:24Chris Benson?

  Love hearing that. We, we, we talk about these kinds of use cases pretty regularly on the show. And, you know, the general classification of AI for good in different ways and making the world a better place. So those, those are pretty fantastic. As you were going through that a moment ago, it made me wonder, you were talking about research and stuff. And within Microsoft as new research is coming out either internally or external to the organization and everyone's consuming it and trying to understand the latest advancements that are going on. How do you get from that point where you read a scientific paper? And at some point, somebody realizes that there is a service in Azure that you can, you know, utilize that and it becomes something that people like us will then use through Azure Cloud.

• 39:08Chris Benson?

  What does that process at a high level kind of look like to get from cool new thing that we're reading about to something that you're now offering the public?

• 39:16Speaker 00

  Yeah, look, some of this work is what I call loosely coupled. Cause if you try to like orchestrate it too finely, then things break down. Cause all our research is exploratory and it is kind of like, should not be tied to commercial goals directly, right? And, and the idea of just pure research and pure learning. So that we have a dedicated research arm there, but this totally connected to our product side. So an AI is a great example and quantum computing is another example. There's some spaces where really like cutting edge research has a direct line into our engineering teams where we actually always looking to see, you know, especially when we look at vision, speech, language, and what we started calling decision category,

• 39:58Speaker 00

  which is where the computer is making a decision or AI algorithms making decision where something is not just classifying it, but actually making a decision. And, you know, recommended something on behalf automatically, not just classifying it again. These are kind of four lesser domains of research also that we really invest in. So our research teams stay super plugged in with our engineering teams or rather our engineering teams stay super plugged in with their research teams. And as these things happen, you know, we're able to kind of, we have established mechanism to take. Kind of literally raw research and have pipelines built to kind of put them into production and processes to do that. And one other thing we've done a lot is have a responsible AI threat throughout this area. Because you know, it's easy to say we have

• 40:41Speaker 00

  principles of responsible AI and we of course do. It's also like, hey, was the data collected with consent? Was the data was collected for this kind of research breakthrough or before we productize it? What other ways we need to augment the data set to make sure it's not biased or does have any biases and all that stuff? And there's a pretty very big ethics committee that works through the from research into kind of releasing it as a product to the world where we augment anything that might have been missed in the research phase. Because that's really not the focus of oftentimes, as you can imagine. It's loosely coupled, but it's connected. And then we have set of processes that walk us through that enables us to kind of bring these things to market very quickly, but do it in a responsible fashion too.

• 41:21Daniel Whitenack?

  I think it's pretty interesting that you've had the chance to work for many years on these incubated projects within Microsoft. And it strikes me that this might give you a unique perspective in terms of the future trajectory of some AI related technologies and maybe things on the horizon. So I'm curious to know a little bit about what excites you about the future of practical AI or AI platform or the mixed reality space and some crossover there. What are what are some of the things like when you're, you know, when you're falling to sleep at night, what are what are those things that are kind of running through your mind that you're dreaming about?

• 42:05Daniel Whitenack?

  This is my favorite question when we ask people this because we get some really cool answers coming back. So no pressure, but. Yeah, we've never had anyone pull out their dream journal and, you know, read something off, but.

• 42:21Speaker 00

  Look, AI is a fascinating space because I think it's not just an appearance creation space now, it's getting more and more mainstream in many other technologies. And even like when it compare other technologies is gaining mainstream adoption faster. And it's really happening, I just believe because you know, like they're like what we call digital native companies especially that have been built on the foundation, they've been built in the cloud and have used AI and ML as a way to really separate themselves. So the things that can be excited to be more precise is we're seeing companies that you would not imagine using machine learning and doing it not just for descriptive analysis, but actually running ML pipelines, building thousands of models. We have a company called AGI, AGI in Australia, they make windmills.

• 43:05Speaker 00

  And now they're building a machine learning model per windmill, right? Not one generic, not just one generic machine learning model, one. Yeah, but Nestle is another very commonly known name, right? Like, but it's a, it's not known as, it's not a born in the cloud company. It's a very old company that has really adopted technology to really help them out innovate other, their competition. And they're also, by the way, using Azure machine learning in this case to do phishing, you know, the email phishing attacks to detect them because they have a huge employee base. And they're using very sophisticated ML pipelines to not only train the model once, but to actually look at all the emails that come in and to kind of then stack rank them in the risk factor and then to run more sophisticated machine learning models to really reduce the amount of things that are happening.

• 43:50Speaker 00

  Scannadave and Aliance is another one. You know, they use machine learning to do fraudulent fraud detection activities. And when they want to say, hey, Daniel, Chris, you're a loyalty member, but you did something bad, they need to be very sure before they ask you that question. And they're using, again, machine learning and a responsible machine learning capabilities to make sure the models are built. You know, I have super high confidence, they're using model interpretability and all that thing. So what really gets me excited is, A, we have many more mainstream companies using AI, not just in the lab, but literally doing thousands of models in production and, you know, really revolutionizing the businesses now, you know? So it's really not in the hands of, let's say, Google, Microsoft, Apple.

• 44:34Speaker 00

  Only it's really, you know, what I would call, you know, I'm sure Lockheed Martin, uh, you know, also does a bunch of machine learning, although I don't work with them anymore. A little bit, right? But this mainstream company is benefiting from this a lot. That gets me excited every single day. The second thing is the space is innovating at a super fast pace, right? So you've got, you have all these new techniques coming in. And it almost seems like what was cool two years ago is quite passe now. You know, like whoever, like we all use sidekick learn, but like nobody really talks about it that much in this sense, right? Now we're all about putting, uh, deep learning models in production at high scale and just base of getting new AI research into the world. It's just super fascinating and

• 45:16Speaker 00

  what keeps me excited is like, you know, like I think everybody in this field is AI should never be in the hands of, hands of a few. And increasingly it's not. Now, as it does get to reach more people, you know, you have to do it in a very responsible fashion and that word cannot be used loosely. In the sense that, yeah, anybody can build a model using automated machine learning. But unless you give them the ability to kind of look under the hood to understand, you know, did they either have any issues, were there any biases built into the data, whether they have high cardinality issues? You know, just because the model's ninety nine percent accurate, if it has a, you know, if it's classifying the wrong thing and it's missing nine of the ten cancer detection, it's a pretty damn bad model. And but being able to now

• 45:58Speaker 00

  bringing the sophistication of being a data scientist to people who know data science is not easy. And we should never trivialize, say, you know, using API or automated machine learning, you have a model and you're lucky and you know, you're, you're happy and go and deploy it around the world. Now the fun part is, okay, now that we've simplified some aspects of machine learning, how do we make sure it's applied in a in a way where people are applying it fully understand the implications? And that's a super exciting space for me and with the, what the industry is doing, what Microsoft's doing. Not just simplifying it, but putting it, making it practical at the end of the day. So more people can benefit from it and, you know, it's a never ending cycle. New research comes in. Taking to market and doing in a way that most people can benefit from it,

• 46:41Speaker 00

  controlling the hype around this topic, but really driving the, you know, benefits for our customers. So it's a long-winded answer, but it is kind of making sure AI is applicable to a large sets of customers, but in a practical fashion, not in a buzzwordy fashion, not just saying one click and boom, you have the model. And the second one is just the pace of innovation and new techniques coming in and the opportunity it offers to customers even who don't have data to do machine learning, like things like reinforcement learning techniques when applied correctly can solve some of these kind of issues. So it's really making this not a special topic, but a really, a really widely useful topic is what excites me and keeps me up.

• 47:26Daniel Whitenack?

  Awesome. Well, I don't know about you, Chris, but I'm, I'm inspired.

• 47:29Chris Benson?

  I'm psyched right now, if you can't say.

• 47:32Daniel Whitenack?

  Brad, a really great way to end our conversation here. I think our listeners will be equally as, as inspired. I hope that they get out and try some of these things. And as your cloud, we'll make sure and include links in our show notes to all of those things that we discussed. So make sure you, you get out there and try some things, get hands on. And yeah, thank you so much for joining us, Brad.

• 47:53Speaker 00

  Thank you for having me and thanks for having a great podcast.

• 47:57Daniel Whitenack?

  Thank you.

• 48:00Speaker 02

  Come hang out with Daniel, Chris, and hundreds of other AI practitioners in our community Slack. It's a cool place to be. Not a lot of noise, some great signal, and best of all, it's totally free. Check it out at changelog.com/community. And don't forget to follow the show on Twitter for AI news and links, highlights from past episodes and more. We are at Practical AI FM. We love to have you following along. Thanks to Daniel and Chris for hosting Practical AI week in and week out. To the mysterious Breakmaster Cylinder for the excellent beats you hear on all Changelog podcasts. To our sponsors who have our back, Fastly, Linode, and Launch Darkly. And to you for listening. We appreciate your time and attention. That's all for now. On the next episode of Practical AI,

• 48:44Speaker 02

  we are running Daniel's panel at the R Conference talking AI for good. So stay tuned for that one hitting your feed next week.