TensorFlow in the cloud

Craig Wiley, from Google Cloud, joins us to discuss various pieces of the TensorFlow ecosystem along with TensorFlow Enterprise. He sheds light on how enterprises are utilizing AI and supporting AI-driven applications in the Cloud. He also clarifies Google’s relationship to TensorFlow and explains how TensorFlow development is impacting Google Cloud Platform. Sponsors: DigitalOcean https://do.co/changelog – DigitalOcean’s developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99% uptime SLA, and 24/7/365 world-class support to back that up. Get your $100 credit at do.co/changelog https://do.co/changelog. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Featuring: Craig Wiley – X https://x.com/craigfinding42Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Google’s “Rules of ML” https://developers.google.com/machine-learning/guides/rules-of-mlTensorflow https://www.tensorflow.org/TensorFlow Hub https://www.tensorflow.org/hubUnity https://unity.com/TensorFlow enterprise https://cloud.google.com/tensorflow-enterpriseGoogle’s ML crash course https://developers.google.com/machine-learning/crash-course10 week Kaggle course https://www.kaggle.com/getting-started/87793Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Change Law because of Rollbar. Check them out at rollbar.com. And we're hosted on Leno cloud servers. Head to leno.com slash change log.

• 0:17Speaker 00

  This episode is brought to you by Digital Ocean. Digital Ocean's developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, TM accounts, worldwide availability with a 99.99 uptime SLA and 24/7 365 world-class support to back that up. Digital Ocean makes it easy to deploy, scale, store, secure, and monitor your cloud environments. Head to do.co slash change log to get started with a $100 credit. Again, do.co slash change log.

• 0:59Speaker 01

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slack with us around various topics of the show at changelog.com slash community and follow us on Twitter. We are at Practical AI FM. Okay, here's Daniel and Chris.

• 1:24Daniel Whitenack?

  Well, welcome to another episode of Practical AI. I'm Daniel Whitehead, a data scientist with S I L International, and I'm joined as always by my co-host, Chris Benson, who is a principal AI strategist at Lockheed Martin. How are you doing, Chris?

• 1:41Chris Benson?

  Doing great. How's it going today, Daniel?

• 1:42Daniel Whitenack?

  It's going well. Coming off of vacation, it only took me just about till 2:00 PM on Monday to catch up on emails, so not too bad. Had to jump in early and catch up from things, but all is good. How about you?

• 1:56Chris Benson?

  Just slaving away while you're on vacation. My wife and daughter actually went to Disney World without me.

• 2:00Daniel Whitenack?

  Oh, no.

• 2:01Chris Benson?

  So they left me at home with the pups and doing work.

• 2:04Daniel Whitenack?

  Uh, gotcha. Didn't go to see the new Star Wars deal, Star Wars World, or I don't, I don't know what it's called, actually.

• 2:10Chris Benson?

  No, I think they saw Harry Potter, but not Star Wars this time. Yeah, Universal.

• 2:14Daniel Whitenack?

  Cool. Well, I'm excited to be back from vacation. I'm excited to have with us today an awesome guest. Today we have with us Craig Wiley, who is Director of Product Management for Google Cloud AI Platform. Welcome Craig.

• 2:31Speaker 04

  Thank you. Very excited to be here.

• 2:33Daniel Whitenack?

  Yeah, excited to talk to you today about a lot of TensorFlow related things and a lot of Google Cloud related things. But before we get into that, could you just give us a little bit of your background and how you got into AI and eventually ended up at Google?

• 2:50Speaker 04

  Sure. So, you know, as you said, I run product management for our AI platform here at Google. Previous to that, I spent a couple of years at AWS, building Amazon SageMaker. And then previous to that, I spent a number of years in Amazon's supply chain group, doing kind of optimizations and, you know, starting with pivot tables and moving on to classic econometric regression and then moving on to kind of, you know, more and more unsupervised and more and more, you know, deeper and deeper learning as we tried to solve some of these kind of unbelievably complex problems. And that the ever-present goal of trying to make all of this go faster and yield stronger results got me just super interested in the tooling space.

• 3:34Speaker 04

  And so since SageMaker and now at Google, I've been really focused on how it is that we can unlock the power of data within the enterprise and give companies and enterprises the ability to, you know, gain full benefit from the data sets they're collecting.

• 3:50Daniel Whitenack?

  So then is your background more on the, did you do like software engineering before that? Or is your background more in science in terms of how you got into working on supply chain stuff at Amazon and all that?

• 4:03Speaker 04

  Yeah, so I was a philosophy major who believes deeply that the world can be regress. You know, if you had all the information, then I think you could kind of predict anything. Right. And so you kind of followed that sense. And as I joined Amazon, I found myself, you know, at a data-driven company increasingly trying to answer the questions deeply with data. And that, you know, next thing I knew, I was leading a team of machine learning engineers trying to do this at a scale that I had never imagined.

• 4:32Daniel Whitenack?

  So does your philosophy major, do you feel like that gives you a unique perspective on AI-related things? Or do you feel like you're, you're pretty much in the weeds of, of the technical stuff? And you're not thinking about philosophy too much?

• 4:48Speaker 04

  You know, it's, it's a little of both. There's times when I'm pretty deep in the details here and there's other times when I think that ability can lend some context around some of the work we're doing in responsible AI and under just, I mean fundamentally, you know, if we can think about these problems as, you know, here is the input and from this input, we can determine an output. You know, when we break the problem down into something that's that simple, I really think that we can broaden the customer group who we reach out to with regards to building these types of models.

• 5:21Daniel Whitenack?

  I know one of the things that I really appreciated for quite a while is, and you probably know more of the background on this, but I remember that someone in Google came out with this sort of, I think it's, it was called a machine learning guidelines or rules for engineers or something like that. I remember seeing it as a PDF originally and I think now it's actually got a nice website and everything, but it was very practical set of guidelines for engineers getting into machine learning problems and all of that. And I found that really useful. Maybe we can link that in the show notes, but I definitely think that, you know, your background probably has helped you dig into some of those problems. And I really appreciate a lot of what, what Google has done in that area.

• 6:03Speaker 04

  As a leader in this space, you know, with kind of with great power comes great responsibility. And so we find ourselves, you know, building things like our AI principles in an effort to try and ensure that we're proud of the work we're doing and that our customers are doing and that others in this industry are doing. So it's a deep area of passion for many involved here.

• 6:23Chris Benson?

  Yeah, I'm not surprised. It's I know that kind of going to following up on what Daniel was saying a moment ago. As Google has published various principles in terms of how to do machine learning, how to do ethics, which is a hot topic now that we talk about quite a bit. All these things are certainly very much leading edge. And the rest of us in the industry are constantly looking to see what you guys are putting out there, which kind of, you know, leads to transitions us to another topic, which is TensorFlow, which obviously being, you know, certainly one of the two dominant, maybe the dominant tool out there, TensorFlow, to be able to do machine learning in this space. Probably the vast majority of our listeners are quite aware of it by this time. I'm kind of curious, though, what is the relationship? Could you kind of tell us what TensorFlow is?

• 7:07Chris Benson?

  And what is the relationship between TensorFlow and Google at large? And, you know, how do those communities interact? And kind of maybe give us a little bit about the origins of TensorFlow at Google? How it got kicked off, you know, how people opted into that? And how the ecosystem started developing on that?

• 7:25Speaker 04

  Yeah, so, you know, TensorFlow is, for those who aren't as aware, is, you know, kind of the premier deep learning framework for building these neural networks that have become so transformational in the kind of journey of artificial intelligence and machine learning. And, you know, TensorFlow is, it's an open source project, you know, kind of started and led by Google, but with contributors from all over the world and from many different, you know, companies and corporations and organizations. And, you know, at the end of the day, I think when I think about TensorFlow, I think about the ability for people to do increasingly complex tasks easily, so that they can get to the point of value creation faster.

• 8:10Speaker 04

  Now, you know, you asked about kind of how we sit organizationally and things. TensorFlow is a part of our research group. And, you know, internally, kind of has a charter or mandate to ensure that Googlers have the best tools they need in order to build the kinds of assets, the kinds of machine learning tools and assets that they want. Having said that, in cloud, we work hand in hand. With this group, to be honest, probably of folks who are not in my direct kind of, you know, org. This is probably the org I am closest with, meeting with them kind of on a daily basis. To figure out how we can provide more value to TensorFlow users and how TensorFlow can provide increasing value to Google and Google's cloud customers when need be.

• 8:55Daniel Whitenack?

  Yeah, and I know, I mean, even though I guess a lot of the decisions around TensorFlow and maybe the roadmap are made by people within Google. I know that, for example, TensorFlow 2.0, there was a lot of community involvement in that and community feedback that led to more usability and that sort of thing. TensorFlow 2.0 was a pretty big deal, I think. Maybe you could kind of remind us what happened with TensorFlow 2.0 and how that was driven by a lot of this user feedback and that sort of thing.

• 9:26Speaker 04

  Yeah, you know, I think you're right, that TensorFlow continues to be an active open source program and is one that, you know, the TensorFlow, both the Google TensorFlow community and the open source community are kind of constantly seeking a wide variety of, you know, kind of input and contributions. You know, one of the things that I was most excited about within this same development time period, just as TensorFlow 2.0 was starting to really take shape and get to a point where it would be nearing release, is at that same moment that we started to come together and bring TensorFlow Enterprise to our customers, which is, you know, our basically offering

• 10:09Speaker 04

  for enterprises to be able to have a rich TensorFlow experience with the requirements that they need kind of front and center in a cloud environment.

• 10:19Daniel Whitenack?

  And what are those, could you just kind of briefly describe what are some of those requirements that are unique to like a production enterprise setting? That maybe are harder to deal with in an open source setting?

• 10:32Speaker 04

  Yeah, I mean, so you know, TensorFlow Enterprise, you know, it's really designed to accelerate the software development experience and improve the reliability for AI applications at the enterprise. And, you know, it starts really first with enterprise-grade support. And when I say enterprise-grade support, really what I mean there is a lengthening of the support window for previous versions, right? We know that folks have developed, you know, exciting models and models that create a lot of value for their organizations on, you know, 1.14 or TensorFlow 1.15 or something like that. And, you know, when, you know, TensorFlow as an open-source project, they only support older releases for one year.

• 11:15Speaker 04

  And what we've done in cloud is we've come through and said, we will extend that by an additional two years so that you get a full three years of support, so that these models that you've built in, you know, TensorFlow 1.14, you know, you're not feeling pressure to move them out of production and replace them with newer models. You can keep them in their environment, producing value, knowing that security patches will still be deployed and kind of, you know, fundamental functionality patches or bugs will still be fixed if you should incur them even after that one-year window has elapsed. We also, we focus deeply on cloud-scale performance and ensuring that the TensorFlow Enterprise binaries are highly

• 11:57Speaker 04

  optimized for Google's cloud environment and as well as for running on Kubernetes should folks want to run it in a hybrid or on-prem installation. And then finally, we believe there is a ecosystem of apps that are going to be critical for customers to gain the most value out of TensorFlow. And so, you know, for instance, we've launched a public-facing, you know, TensorBoard experience. And, you know, we've been open about the fact that we're working on an enterprise version of that TensorBoard experience as well. So, you know, it's really not kind of one thing, but it's kind of a series of pieces that together we find we can kind of piece together and build a comprehensive, you know, addition

• 12:41Speaker 04

  of TensorFlow that can really speak to the, to the needs and requirements of these enterprises.

• 12:46Chris Benson?

  So you have customers that are engaging in TensorFlow in the enterprise model and recognizing that the core TensorFlow project is open source led by your internal Google team. From a customer's perspective, if they're opting into the enterprise version, what is the relationship like for the customer between engaging you with the additional support that you just talked about in the enterprise model versus that open source community? How do those two separate sides of the TensorFlow world at large interact with each other for that customer who has access to both?

• 13:23Speaker 04

  Sure. So I mean, it really comes down to, you know, TensorFlow is an open source project. Anyone can go and download TensorFlow, run it on their laptop, run it on their own data center, or run it on their cloud of choice. You know, TensorFlow Enterprise, having been built and highly optimized for Google Cloud, is designed, you know, so we have a series of products such as our deep learning virtual machines, which are basically, you know, VMs within Google Cloud that are pre-configured to run TensorFlow Enterprise on them. So all you have to do is come in, pick an instance size, you know, do you want a small CPU or do you want a giant GPU and, you know, say which version of TensorFlow Enterprise you want installed and we'll, we will outfit you with a machine. So you'll

• 14:06Speaker 04

  be up and running instantly with everything you need pre-installed. And then we have containers for folks who want to use our, any of our managed services, our Kubernetes managed services, or if they're running their own clusters, you know, using Kubeflow or other Kubernetes frameworks for managing AI. It gives them the opportunity to get closer to their app if that's what they need to do, or move into an on-prem environment if that's a priority for them. So really, that, you know, the way they can engage in TensorFlow Enterprise is by, it's kind of the default engagement when you work with Google Cloud.

• 14:40Daniel Whitenack?

  So I'm curious, we've talked about open source TensorFlow, we've talked about the various elements of TensorFlow Enterprise. I was wondering also if you could mention a bit about what TensorFlow Hub is, since a lot of people, I think, part of their first interaction with TensorFlow and maybe the problems that they're working on can be solved by pre-trained models. So I was wondering if you could kind of give us a glimpse into that world and, and what's available and how that works?

• 15:08Speaker 04

  Yeah, so you know, I mean TensorFlow Hub is a library for the publication and discovery and consumption of reusable parts of machine learning models, right? So, you know, users can create modules which are kind of a self-contained TensorFlow graph along with its weights and assets that can be reused across different tasks and, you know, or different applications. And so really the idea here, and you know, I think this is, we're increasingly seeing this both within TF Hub as well as other locations, is this idea of kind of composable AI where folks can build small pieces that others can then take those pieces, leverage them, and make them, you know, more broadly available.

• 15:52Speaker 04

  And so TensorFlow Hub is really a collection of those modules to help accelerate the machine learning process. I'll tell you, as someone who has worked for a number of years and continues to work in cloud machine learning, a huge percentage of, you know, of my kind of thinking is spent on how can we, you know, improve the cycle time of building machine learning models and getting them into production? Because for so many companies, it comes down to a, hey, I have a team of data scientists. That team of data scientists today can get me, you know, 10 models a year. And if I can come through and with the tooling that we put together, we can help them double,

• 16:32Speaker 04

  triple, you know, 10x their velocity, then the kind of the ROI of machine learning in that organization, you know, increases substantively. And now the, the problems that they're willing to attack with machine learning, you know, gets much broader. And so I think any, you know, I'm super excited about all of these technologies that give customers the ability to save time and effort and thought, you know, really kind of like time of thinking and thought energy on these problems that have been solved by someone else and you don't need to reinvent the wheel. You just need to go get their wheel, drop it in and use it in your case. And I think that's what TF Hub has shown, you know, such a strong ability to do.

• 17:26Speaker 01

  If you like this show and you aren't listening to the Change Log, hey, let's fix that bug. The Change Log is our flagship show and we've been doing it for over a decade. Adam and I seek out and interview the people who are pushing the world forward with software. We dive deep into the hacks, the innovations, and the leadership required to do what these amazing people do. One recent example is our conversation with Anders Damsgaard, a climate scientist from Denmark who gave us a peek inside his work and how he scratched a common itch he has when gathering academic research from around the web. Here's a dorky moment from that episode.

• 18:01Speaker 00

  Are you trying to be right or are you trying to solve the world's problems?

• 18:04Speaker 05

  Exactly.

• 18:05Speaker 00

  If you're a scientist trying to be right, well then, your right may not actually be the right.

• 18:09Speaker 05

  Yeah, exactly. There's another saying, "All models are wrong, but some are useful."

• 18:15Speaker 01

  I like that one. There's another saying, "All models are wrong, except for mine." Mine's correct.

• 18:23Speaker 01

  We had a lot of fun with Anders. He's a fascinating guy. Continue listening at changelog.com/podcast/378 or search for The Change Log on your favorite podcast app and find the episode called "Open Source Meets Climate Science."

• 18:54Chris Benson?

  So Craig, if you are a practitioner out there and you're using Google's Cloud AI services in the enterprise, what are some of the things that you're seeing pop up over and over again in terms of use cases, in terms of applications, that are kind of clearly leaning toward what your services are able to provide? Are there certain use cases that you're seeing come up over and over again, are fairly common to the platform, where people are able to bring value fairly quickly to AI, using TensorFlow in this context of Cloud AI? Or is it just everything is different? What kind of trends are you seeing?

• 19:34Speaker 04

  Yeah, you know, this is one of these questions that gets harder every month for me to answer. You know, like, you know, I mean, if you had asked me this question a couple of years ago, I would have kind of said, oh, you know, people build great recommendation systems and, you know, and people build great forecasts or great, you know, vision models or something like that. And the kind of, you know, application of models was still relatively narrow. But today with, you know, if I look at company like Unity. So Unity uses Google Cloud and TensorFlow Enterprise to quickly test, build and scale out ML models at a massive scale, right? Allowing them to serve up kind of the most relevant ads and drive revenue for game developers. And so, you know, we don't think about,

• 20:18Speaker 04

  you know, when we kind of turn on the game on our phone or, or, you know, light up the most recent, you know, kind of game that we're excited about. You know, we don't think about the fact that there is tremendous technology going on behind the scenes to target those ads to pay for that game so that it can continue to be free for us. And, you know, this is something that is very important to Unity and its developers and customers. And they have found tremendous success using TensorFlow, using TensorFlow Enterprise and using Google Cloud to, you know, kind of be able to scale this problem that, you know, requires them to kick off kind of hundreds, thousands or tens of thousands of models, kind of, you know, in an effort to kind of highly target their advertising and what have you. I take that as one example. And then on the other end,

• 21:03Speaker 04

  I can call out a company like GM Cruise, right? So GM Cruise is, you know, a strong leader in autonomous vehicles and is a TensorFlow Enterprise customer. And, you know, the deep collaborations between Google Cloud, TensorFlow and GM Cruise have given us the ability to reduce their training times from, you know, I think when they arrived, it was something along the order of four days and now it's less than a day. And so, you know, when you're able to achieve, you know, a 4X reduction in cycle time, now all of a sudden. They can potentially move four times faster in trying to get to their, the solution to their problem. And as the father of a young child, I have to say, the idea that he'll never drive

• 21:48Speaker 04

  a car is a, a very exciting idea to me. And so I'm, I'm more than willing to help all of these folks get as far down that path as they can as quickly as possible.

• 21:59Chris Benson?

  It's funny you say that. By the way, I was just gonna say, cause I also have a young child and I, I try to tell her the same thing and other people and, and they struggle to believe it. I just found it very curious that you jumped in on that.

• 22:10Speaker 04

  My son has even said, you know, at seventeen, I'm not sure I'll get my driver's license because I'm not sure those will still exist, you know, and things like this. But, but you know, those, those two problems are, you know, about as different from one another as could be. And certainly we see, you know, problems across industries, whether it's fintech, retail, manufacturing, you know, I, I continue to be humbled by the creativity that folks have in finding ways to utilize this technology to create value for their companies.

• 22:45Daniel Whitenack?

  So I'm kind of interested, since you see so many use cases in your day to day and, and have seen so much variety, I'm kind of curious to get your perspective on actually how enterprises are leveraging AI. And what I mean by that is kind of in my mind, how I see various companies approaching AI is you've got on one end of the spectrum, you've got companies that are like really investing in intense research and development and maybe they're actually, you know, have their own AI team and they're developing their own model architectures and very unique models, very unique combinations of models and doing those sorts of things. You've got other groups that are what I would consider more kind

• 23:28Daniel Whitenack?

  of like doing AI cooking in the sense that they get like a recipe, they get a, an existing neural network architecture or something like that. They plug in their own data to create their own model and then go from there. And then there's other end of the spectrum, definitely a lot of people that are just scaling. Up inference, so they might be using a pre-trained model or a combination of pre-trained models or modules. Or maybe they're just doing a bit of transfer learning or something like that, based on what someone else has has previously done. And so they're not even contributing a ton of data to the situation in terms of training. I was wondering, from your perspective, where are you seeing the biggest investment from enterprises these days? Or is there a shift one direction or the other on on that spectrum?

• 24:12Speaker 04

  You know, it's a really good question. And, you know, it's one that we spend a lot of time thinking about and working with companies on how they're thinking about this problem. You know, I'll say the companies who I've been most impressed when I hear their strategy around these things. It really comes down to where they choose to buy versus build. And, you know, from that perspective, you know, one of the things that I think deeply about and that I think others do as well is this idea that, you know, if you can buy a solution, right? You know, whether it's a, you know, contact center AI, our kind of intelligent, you know, virtual assistant or contact agent in your call center or our document understanding.

• 24:56Speaker 04

  These technologies that are powered by AI, you know, like these make a ton of sense for companies to buy. Because for most folks, you know, your contact center isn't the primary point of differentiation for you. And now, you know, as you move kind of further down the stack, you know, I I think the question becomes, you know, where are you comfortable using AI to accelerate your business and gain efficiencies in your business? And where does that do those efficiencies? Where are they tied to your competitive advantage as a company? And fundamentally, the idea of buying your AI makes a ton of sense.

• 25:38Speaker 04

  If it's something that you're very comfortable with your competitor buying as well. But when it comes to what you should build, it really comes down to where are the the unique areas that you think you could kind of, you know, express some differentiation in that industry. And, you know, so if I take retail as an example, we'll see retailers who are, you know, happy to buy a recommendation system, but they want to optimize the back end and their supply chain deeply because that's how they believe they can stand out, or maybe they'll want to uh highly optimize their promotional capabilities or something along, or their forecasting capabilities. Whereas others may say, you know, listen,

• 26:21Speaker 04

  you know, forecasting and supply chain, I'm happy to kind of work with a partner and and get the best in class that that is easily available to me, but I really want to stand out on our usability or our recommendations and things of that nature. And that's how I plan on setting us apart from others. And, you know, I think that that's a conversation that companies have been having for a long time with regards to the software they're building. And, you know, if I if I think over the last decade of the number of industries where the leader in that industry has almost become a, you know, hybrid software company, right? Whether it's a retailer who's become more or less a software company or a healthcare provider who has a giant software investment, you know,

• 27:05Speaker 04

  those same decisions that have powered that kind of investment over the last decade, I think now we're seeing that same set of decisions being applied to how they'll invest in intelligent computing and where they'll choose to build versus buy. So I'm not seeing kind of a wholesale like, hey, people are, you know, giving up notebooks and are only going to go buy from now on or vice versa. It's really around how companies will choose where they, you know, are willing to build versus where they're willing to buy and how that aligns to their strategy.

• 27:35Chris Benson?

  So I love your perspective there, you know, kind of the business take on where you're going to choose to invest in adding value and how you create competitive advantage using these types of AI tools for your own organization. And so I found myself, as you were saying that, I found myself wondering, you know, what do you think some of the big challenges that you see people trying to create competitive advantage for themselves, where it's not the run-of-the-mill stuff, where they're just, you know, taking their data and doing, you know, doing a little bit of transfer learning and creating their own version of the same model, but the things where people are saying, this is where our organization wants to make a mark. Are there any examples that have particularly surprised you or caught

• 28:17Chris Benson?

  your attention in terms of big challenges that organizations are staking themselves on?

• 28:22Speaker 04

  You know, it's an interesting one. You know, my life is spent trying to make it easier for them, right? And so, you know, and I'll say, I continue to believe that one of the biggest challenges in this space is actually far to the left of machine learning. And, you know, when I say left, I kind of mean in a workflow, you know, if I start with kind of data acquisition at the left-hand side and I end up with a model in production at the right-hand side, you know, that kind of data acquisition, that data cleansing, you know, I'll tell you, the number of organizations who are, you know, where I'll talk to data scientists and they'll say, yeah, the data is decently clean and I know where to get it, but I can't get it because there's organizational silos and this other organization owns the

• 29:06Speaker 04

  data and I can't do machine learning on it or something like that.

• 29:09Chris Benson?

  Really common problem, too.

• 29:10Speaker 04

  Yeah. And these are the things that just break my heart. And, you know, one of the exciting things for me about working with Google Cloud is that the tight integration between our AI team and our analytics team results in some opportunities for customers to much more easily do these things. So a great example would be BigQuery. So BigQuery is Google's hyperscale data warehouse product that I'll say, if you haven't played with it, you should, because it is, you know, an order of magnitude, kind of faster and more scalable than anything I have gotten an opportunity to play with. And I came into machine learning by running data warehouses in many ways. And so, you know, it's close to home for me. And so, but, you know, today,

• 29:56Speaker 04

  You can build machine learning models in BigQuery using SQL as your programming language. And so like you can train an XGBoost model or train a TensorFlow model all from kind of the SQL UI that data analysts would be accustomed to. And so, you know, it's integrations like that that I hope can help us break down some of these barriers that are, you know, I'll say often they're blamed on tech, but often the problems are as much policy as they are tech. And, and, you know, I think the key is can we simplify the governance for folks, not remove it by any means, but simplify it so that it becomes much easier and much less

• 30:39Speaker 04

  scary so that they can get to the point of extracting value from their data even more quickly.

• 30:45Daniel Whitenack?

  I know one of the things that I've hit occasionally is, you know, because you're working in the cloud and people, you know, in large enterprises have traditionally had their data in their local data centers and maybe there's restrictions like governance things like you were talking about around moving that data around. Do you still see that popping up in terms of people being hesitant to utilize cloud resources for AI just because they have to move the data around? Or do you, do you see any progress on that front in terms of people being more willing to invest in, in moving their, their training data set to the cloud and managing it there?

• 31:22Speaker 04

  Absolutely. You know, customers are getting more comfortable. I think this problem is being solved in two directions. One is, you know, the clouds in general and Google in specific with Google's Anthos, which is our ability to run a Kubernetes cluster across a data center and cloud as if it were kind of one Kubernetes cluster.

• 31:43Daniel Whitenack?

  Sort of like a hybrid.

• 31:44Speaker 04

  Right. You know, technologies like that start to, you know, diminish the damage of saying, hey, I, I can only have my data here and I can't move it elsewhere kind of thing. Right? And so, you know, in one direction, there are strategies like that. And then, you know, in the other direction, as cloud continues to grow and, you know, I will routinely meet companies who will say, you know, well, my industry. And you know, you can fill in with whatever industry doesn't allow me to put data in the cloud and you know, we'll talk to them about how many references we have in their industry who are putting data in the cloud and it and you know, it very quickly, they realize that it's not their industry, it's their company or even their department that has

• 32:28Speaker 04

  built these rules probably years ago before it was understood the benefits of things like cloud security and reliability and resiliency. And so, you know, I think those kinds of blockers are kind of falling by the wayside either because companies are realizing that the cloud is not as scary as maybe they imagined. And then secondly, because the cloud is coming to the customers in a, in a big way.

• 32:51Chris Benson?

  So, you know, one of the things as we kind of talk about Google's cloud AI and using TensorFlow in the cloud, I know for me, using Colab has been a real game changer. Um, it is, uh, I will just go ahead and say it is my favorite cloud environment to work in. Uh, and I know a lot of other people who feel the same way. And so I've kind of stuck, stuck my own bias out there. You know, what kind of insights have you had into the future of how we're going to use these tools in the cloud, where Colab might be going specifically? And what kinds of, you know, integrations will we continue to see and accelerators will we continue to see in these types of environments?

• 33:27Speaker 04

  Yeah, you know, whether it's Colab or whether it's notebooks on Kaggle, or whether it's uh, Google's own Google Clouds, you know, Jupiter as a service or Jupiter Notebook service, you know, I continue to be excited and I'll be honest, slightly disappointed, and I'll get into that in a moment. I continue to be excited by the evolution of tooling for machine learning. You know, I say slightly disappointed, I wish it were moving faster. And you know, I think you're starting to see that with more and more kind of uh, opinionated workflows, right, around, you know, whether it's AutoML type of, of workflows or, you know, you know, kind of rich templates to your point of the cooking recipes, right,

• 34:10Speaker 04

  that can kind of get people 90% of the way there and then let them adjust the part that matters to them. I, you know, I continue to, to believe that. The way we're doing machine learning today probably isn't the way we'll be doing machine learning a decade from now. And you know, I often think about software development. And you know, if we were to rewind the clock on software development by, you know, 20 years, you know, concepts like regression testing and unit testing and these kinds of things, you know, weren't a part of the everyday software development lifecycle. Today, you know, if you went to go deploy something and you hadn't done, you hadn't passed it through a regression testing suite, you know, we would kind of think you're irresponsible.

• 34:50Speaker 04

  And you know, we don't have those same concepts yet for machine learning. And you know, to me, one of the reasons why it's so exciting to work in this space is that I feel like we are getting to develop the kind of, you know, next generation of standards around how people will, will develop machine learning. And you know, whether it's, you know, things like, you know, pipeline technology to improve CICD experiences, you know, whether it's specific TensorFlow modules built into TF Hub, or notebooks, you know, notebook examples on Colab or on Kaggle. This is an area that you can feel is, you know, very likely going to evolve very quickly over the next few years.

• 35:31Daniel Whitenack?

  Yeah, I was just gonna mention before we get off of Colab and some of the things there. There's certainly, as you mentioned, there's standards that need to be developed and rigorous things that need to be developed. And I know guests like Joel Grus and others on the podcast have talked about that. But on the other side of things, like I've been working with this Masakane collaboration recently, which is trying to involve research groups and people on the African continent into developing machine translation technology for the languages that they care about. And a central piece of that is the fact that they're able to very quickly onboard participants into their working group because they have

• 36:11Daniel Whitenack?

  a set of Google Colab notebooks that they're able to spin up, they're able to run in their browser, they're able to have access to a GPU to get them started on training baselines. And they don't have to have access to their own GPU. They don't have to even have a lot of experience and a Local environment set up with a IDE that's all set up to do certain things and all of that. So that's really, I think, been crucial to that collaboration is the fact that they are able to onboard quickly into that. Now, obviously, as you productize things, as you're mentioning, as you work with enterprises that have certain concerns,

• 36:49Daniel Whitenack?

  there's definitely a matter, matters of integrity and that sort of thing and robustness that need to be dealt with. But I think some of those concerns are being addressed, but also on the other side, there's a lot of tooling that's being developed out there that's really helping make this technology more accessible to more groups of people, which is, is really exciting for me.

• 37:11Speaker 04

  Absolutely, and I couldn't be more excited to be a part of the, kind of, the work that is going into this democratization process and widening the field of machine learning developers and creators. And then, you know, fundamentally though, I'll say, I always worry about whether or not those models are going to get into production. Right? Because one of the things we know is that the vast majority of machine learning that gets built never makes it into the kind of, you know, the area where the inferences can now start creating value, right? All too often, it gets built, gets built on someone's, you know, laptop or maybe in the cloud. And then, you know, they're never able to actually integrate it into the application or integrate it into that point where it could create value.

• 37:56Speaker 04

  And, you know, this is an area where we invest deeply to try and ensure that like, hey, once I've built that model, whether it's in Colab or, you know, wherever I built it, that I can easily put it into a place where I can call it for inference and kind of gain the full value of the model. As data scientists, I think it's very easy to become attached to the training as the, you know, the training and kind of the AUC curves or what have you as the point of, of success. But, you know, for me, really, the success comes when those inferences are creating the value that we all set out to, or the goal we all set out to achieve when we first started trying to build the model.

• 38:36Chris Benson?

  That's a great point that you're making. We do tend to get caught up in our own training and thinking of as the training is the thing versus the value that you're creating with the model after you deploy it. I would feel remiss, I'm kind of curious, I just wanted to go back a little bit to your personal bio for a minute and definitely not wanting to get into the politics of competition and stuff, but coming from AWS, you came into Google, you had a hand, obviously, in creating SageMaker and obviously, like you would with any job moving to another, you learn and you get better and you gain knowledge as you do that. Is there anything that comes to mind that you learned from that previous experience and, you know, and definitely not putting down SageMaker, but just like, you know, you went through that process and you gained expertise and then you came to

• 39:21Chris Benson?

  Google and you started working on Google AI. Is there anything that really helped you as you came across there?

• 39:27Speaker 04

  You know, I was unprepared for the benefit of sitting side by side with the Google's research groups. You know, Google is clearly a leader, if not the leader, in AI and machine learning. And to get to, you know, sit shoulder to shoulder with Jeff Dean and others as they solve some of the most complex machine learning problems in the world and then taking the components with the kind of fundamentals or the, you know, the primitives that they've had to use, you know, extracting those out of some of these world-class problems, you know, and putting them into an environment where enterprises of all levels can interact

• 40:12Speaker 04

  with them is something that, you know, I was just unprepared for. To, you know, to give an example, you know, we're using, you know, if you look at DeepMind and what DeepMind has done with AlphaGo and with StarCraft, and then, you know, we use that same technology for optimizing manufacturing facilities, right? And, you know, optimizing the control systems in manufacturing facilities. And, you know, that kind of research, the tight connection between kind of research and the practical applications and practical tools we're building is just something I don't think you could get anywhere in the world like you can get it at Google.

• 40:53Daniel Whitenack?

  I know one of the things that I feel a lot as someone who isn't at Google, but keeps a close eye on what's going on in AI. It often seems like there's just so much happening so quickly. And obviously, Google is doing a lot. Other groups are doing a huge amount of research and it just seems like, oh, you know, like recently, I remember Facebook released this like multi-language question answering dataset. I was like, oh, this is really cool. This is like something I wanted to see for a while. And so I started playing with it. And before I even got done, like, you know, figuring out some of what they had done and how it worked and the format, there were already like, I think Google came out with

• 41:35Daniel Whitenack?

  another one and there was like another, there's been like three or four non-English question answering data sets and models that have come out after that. As someone who is kind of helping shape the future of Google Cloud AI and in those discussions, how do you parse through all of the advances that are just happening so quick and what people are wanting to do? How do you keep from getting left behind? I guess that tight integration is part of it, but it still seems like a hard thing to keep up with.

• 42:09Speaker 04

  Yeah, herding ducks, I think, would probably be the right idea here, right? I mean, you know, it is amazing because you'll sit there and think, you know, my goodness, I couldn't be any deeper on this topic than I am today. And then you'll find, you know, a team will announce that they've done something that's, you know, two orders of magnitude deeper than where we were. And it's kind of, you know, you just sit there and are like, wow. And so, you know, really it's, you know, getting to know all those folks and keeping close with them so that we can look at not what they're doing today, but where are they going? And where are they going to be six months from now, 18 months from now and 36 months from now? And how do we ensure that we're building towards that fact?

• 42:52Speaker 04

  I mean, and you know, in kind of cloud where I sit, you know, I look at things like GANs or things like, you know, reinforcement learning and say, you know, well, listen, most enterprises are aren't running machine learning at the, you know, most enterprises, if they're running machine learning, they're, you know, they're running psych it models at this point. And so like, you know, what is the value of GANs to me right now outside of an academic environment? But then, you know, what we find is very quickly, you know, far more quickly than I would have imagined, companies start jumping on some of these technologies and we find new and exciting ways. And, you know, so for example, we are running a Kaggle competition, a reinforcement learning Kaggle competition right now, first of its kind within Kaggle. And, you know, seeing some really interesting ways in which users and

• 43:37Speaker 04

  developers who have never done reinforcement learning before are starting to interact with this. And by watching the conversations they're having and the challenges they're having, I think it allows us to build better products, you know, as we come out of that.

• 43:49Chris Benson?

  So I guess, you know, as we kind of work our way to the end here, people are listening to this, they're, they've gotten a little bit of, of knowledge about cloud AI if they haven't already and, and TensorFlow. What would you recommend is the best way for people to get their hands started in TensorFlow and in TensorFlow in the context of, of cloud AI as an enterprise customer? You know, obviously there's the crash course that Google has, which kind of gets you started there. But could you take us a little deeper into how people can steer into all these tools and get started with you?

• 44:21Speaker 04

  Yeah, I mean, so, you know, I think if you're getting started fresh and, you know, haven't touched this technology before, then I would certainly recommend heading towards Kaggle and some of their educational resources. I think, you know, they have things like, you know, 10 weeks to, you know, coding in Python, coding in Python in 10 weeks and, you know, other things around, you know, learning a number of the basics and, and starting to get engaged with kind of machine learning. And so, you know, I think that's a great place to start if you're newer. If this is an area where you have kind of skills and expertise, then absolutely, you know, continuing to, you know, utilize Kaggle and, and their competitions and their wide variety of code samples, Colab as well. But then, you

• 45:01Speaker 04

  know, going to cloud.google.com and our, heading to our AI tab and then, and then, you know, honestly, going to our deep learning environments and spinning up a VM with

• 45:13Speaker 04

  Deep learning on it. You know, I remember years ago when I was kind of working in supply chain, I I had a really strong kind of econometrician on the team. And I said, hey, why don't you go install a deep learning package? I can't remember if at that point, it was, you know, Theano or TensorFlow or, you know, this was kind of earlier days. And, you know, install a package, get it going and, you know, let me know what you think. And, you know, the next week I met with him and he said, you know, I worked at it for a number of hours, couldn't get it running. And so, you know, I went back to the stuff I know because I knew I could create value there. Well, with our deep learning environments, our deep learning VMs and what have you, all you have to do is click on the thing. And now you have a TensorFlow experience or, you know, open a notebook with a GPU as a backend and you have everything you

• 45:58Speaker 04

  need to run, you know, strong and run fast there. And so I would just tell people dive in, find some code samples, try and break them, try and mess them up and go from there.

• 46:08Daniel Whitenack?

  Awesome. Well, appreciate that perspective and we appreciate you taking time from all of your work to come and chat with us. It's been been a great time. I know I've learned a lot and excited to continue exploring what's what's coming out of Google. And thank you so much for for joining us. We'll we'll mention all the things that you just did in our show notes and hope to stay in contact and see you at a conference or something soon. Yep. Thanks a lot.

• 46:33Speaker 04

  Absolutely. Thank you.

• 46:38Speaker 01

  Thank you for listening to Practical AI. If you're not following Practical AI FM on Twitter, you're missing out on clips and highlights from past episodes, links and repos from around the AI and data science community and more. Follow us, Practical AI FM. You won't regret it. Practical AI is hosted by Daniel White neck and Chris Benson. It's produced by me, Jared Santo, and our music is brought to you by the beat freak, Breakmaster Cylinder. We have awesome sponsors, support them, they support the show. Special thanks to Fastly, Linode, and Rollbar for helping us do what we do. If you aren't receiving Change Log Weekly every Sunday, you are missing out. It's our take on this week in the world of software, what's interesting and why. Head to changelog.com/weekly to subscribe. Get it for the price of a free

• 47:23Speaker 01

  cheeseburger. Thanks again for listening. We'll talk to you next week.