#246 Will Granis: How Google Cloud is Powering the Future of Agentic AI

This episode is sponsored by Thuma. Thuma is a modern design company that specializes in timeless home essentials that are mindfully made with premium materials and intentional details. To get $100 towards your first bed purchase, go to http://thuma.co/eyeonai What happens when AI agents start negotiating, automating workflows, and rewriting how the enterprise world operates? In this episode of the Eye on AI podcast, Will Grannis, CTO of Google Cloud, reveals how Google is leading the charge into the next frontier of artificial intelligence: agentic AI. From multi-agent systems that can file your expenses to futuristic R2-D2-style assistants in real-time race strategy, this episode dives deep into how AI is no longer just about models—it's about autonomous action. In this episode, we explore: How AgentSpace is transforming how enterprises build AI agents The evolution from rule-based workflows to intelligent orchestration Real-world use cases: expense automation, content creation, code generation Trust, sovereignty, and securing agentic systems at scale The future of multi-agent ecosystems and AI-driven scientific discovery How large enterprises can match startup agility using their data advantage Whether you're a founder, engineer, or enterprise leader—this episode will shift how you think about deploying AI in the real world. Subscribe for more deep dives with tech leaders and AI visionaries. Drop a comment with your thoughts on where agentic AI is headed! (00:00) Preview and Intro (02:34) Will Grannis' Role at Google Cloud (05:14) Origins of Agentic Workflows at Google (09:10) How Generative AI Changed the Agent Game (12:29) Agents, Tool Access & Trust Infrastructure (14:01) What is Agent Space? (16:30) Creative & Marketing Agents in Action (23:29) Core Components of Building Agents (25:29) Introducing the Agent Garden (28:06) The "Cloud of Connected Agents" Concept (33:53) Solving Agent Quality & Self-Evaluation (37:19) The Future of Autonomous Finance Agents (40:55) How Enterprises Choose Cloud Partners for Agents (43:50) Google Cloud's Principles in Practice (46:27) Gemini's Context Power in Cybersecurity (49:50) Robotics and R2D2-Inspired AI Projects (52:39) How to Try Agent Space Yourself

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  I want to go build an agent. What's the first thing software developers do when they need to go write code? They go look for some code that already exists that kind of does exactly what they need to do. And as a friend of mine and someone I really admire a lot, Kelsey Hightower used to say, "Good engineers copy, great engineers paste." One of the things that we're going to enable inside of Agent Space is an agent garden. It's just like we have model garden today in Vertex. We're going to enable the same thing for agents. So if you want to build a customer service agent, if you want to build a marketing agent, if you want to build a data science agent, you shouldn't have to start from zero. So there will be a kind of a starter pack, if you will, of agents inside of Agent Space to jumpstart the development process.

• 0:38Craig S. Smith?

  Create an oasis with Thuma, a modern design company that specializes in furniture and home goods. By stripping away everything but the essential, Thuma makes elevated beds with premium materials and intentional details. I'm in the process of reorganizing my house and I'm giving Thuma a serious look for help in renovating and redesigning. Thuma combines the perfect balance of form, craftsmanship, and functionality. With over 17,000 five-star reviews, the Thuma bed collection is proof that simplicity is the truest form of sophistication.

• 1:23Craig S. Smith?

  Using the technique of Japanese joinery, pieces are crafted from solid wood and precision cut for a silent, stable foundation. With clean lines, subtle curves, and minimalist style, the Thuma bed collection is available in four signature finishes to match any design aesthetic. Headboard upgrades are available for customization as desired. To get $100 toward your first bed purchase, go to thuma, that's T-H-U-M-A dot C-O slash IONAI. IONAI all run together, E-Y-E-O-N-A-I.

• 2:10Craig S. Smith?

  So for a hundred dollars off your first purchase, go to thuma.co slash ion AI. That's T H U M A dot C O slash ion AI to receive a hundred dollars off your first Ben purchase.

• 2:31Speaker 00

  Well, Craig, thanks for having me. Thrilled for the conversation today. What do I do at Google? So my current role is CTO of Google Cloud, which covers all of our technology, all of our customers, all of our industries. You know, kind of a little bit of everything. And, you know, I came to Google via enterprise technology. So my background is, you know, building things, companies, technology teams, all the way from being a founder of, you know, a one person company, you know, bootstrapped with a couple of Mac Minis daisy chained in a, in a shared office space. You know, living the dream all the way up to, you know, Fortune 10 companies and trying to build technology that

• 3:12Speaker 00

  helps customers, helps their customers. And, you know, from a background perspective, mostly in industrials, aerospace, manufacturing, the public sector. And then when I got to Google about 10 years ago, really spread out to cover all industries. So, you know, retail, financial services, banking, media entertainment, and all layers of the stack from silicon all the way up to, to SaaS. And, you know, current responsibilities, our team does two things. We work with the top roughly 200 customers around the world of Google Cloud. And the goal is to make sure that, you know, we, we build a lot of things and we have a lot of advanced technology coming out of the machinery here. But how it turns into value for our customers is not always obvious at the beginning.

• 3:54Speaker 00

  And so we help our customers get the most out of this advanced technology. And this, you know, this could have been big query back when I started. You know, GKE, Google Container Engine to AutoML to these days, it's all about generative AI agents. And eventually it'll be even quantum, which we work on.

• 4:11Craig S. Smith?

  Yeah, yeah.

• 4:13Speaker 00

  The second bit is, you can imagine working with these top customers all around the world. We also learn a lot and we feel that the subtle tremors of what's to come. And so our team also incubates engineering projects to try to prove the viability of a certain technology direction. And then we hand that off to the product team and say, hey, Yeah, we burned down a little bit of risk. We've got some subtle signals here. Here are some things you might be interested in, then we partner to try to make some of those into reality.

• 4:36Craig S. Smith?

  Yeah, that's fascinating because you really have an overview of what's going on. You know, I have to.

• 4:43Speaker 00

  We'd like to think so. You know, it's, it is a dynamic environment, Craig. You know, one day you think that, you know, we're building on a platform for very, you know, domain-specific models and task-specific models in AI. And then the next day, we're working on kind of this orchestrator, you know, generative AI top-level, you know, engagement kind of mechanism. Where now a singular kind of model experience is orchestrating a whole bunch of those sub-modes. You know, and a shift like that happens in less than a year. So like the old school capital planning and all that has no relevance in a world like this.

• 5:15Craig S. Smith?

  Yeah, that's remarkable. On agents, which is the flavor of the month or the flavor of the week, I don't know how long it'll last. When did you guys start

• 5:31Craig S. Smith?

  building agents with customers? When did you start seeing that as a trend?

• 5:36Speaker 00

  Well, I don't want to claim, you know, that we saw it so definitively that we mobilized everything we had against it. But I think back to the origins of even in 2017. My team was working with engineering and our internal incubator at the time is called Area 120. And we, we were working with a customer Verizon. And Verizon has a really tough problem of they get a lot of inbound customer support requests every single day. Millions. And out of those millions of inbound calls, what they're trying to figure out is how they can take the intent behind all of those calls. And kind of, you know, pull, cluster them into a smaller set of kind of overall intents and then be more proactive in the engagement when a customer enters their funnel.

• 6:22Speaker 00

  It's actually a really tough problem. You know, clustering is a, is a well-known method in data science. But actually, you know, proactively predicting, you know, kind of the top trends or the top intents of your customer base is actually a very difficult problem to get right. And so at the time, we actually with Verizon took a little bit of this kind of intent clustering technology that we had built in Area 120. Plus dialogue flow, which was kind of our, you know, agent assist, you know, kind of predefined agent workflow tool in Google Cloud. And we smashed them together and we created the capability for Verizon to decompose all of those, you know, hundreds of thousands of intents every day into like the top five to 10.

• 7:05Speaker 00

  And so, once you've got five or 10, you can actually proactively engage with your customers and say, well, you know, what about this? And certain words and certain phrases convey intent. So it's not just like strict semantics anymore. And, um, yeah, that was really for me, you know, thinking back on my 10 years now at Google, that was, that was the first time that we really orchestrated an agent-based workflow. We weren't calling them, we weren't calling them agentic workflows at that point. We were calling them agents. But I think it's taken on a much more expansive, you know, meaning in today's lexicon, you know, when you say agent.

• 7:37Speaker 00

  So, you know, like you fast forward to today and, you know, an agent could mean, you want to negotiate with a supplier and you send a procurement agent on your behalf to negotiate with an agent that represents that supplier. You actually are negotiating between agents in real time, you know, potential contractual terms or pricing. And you're embedding in these models, you know, like game theory or other mechanisms so that they can actually have dynamic negotiations with each other. That is, you know, that's much more expansive than what we were thinking about in 2017 for sure.

• 8:09Craig S. Smith?

  Yeah, actually, that's a fascinating example. I mean, uh, what's certainly changed from, uh, from the mid 2010s or late 2010s is, is generative AI, the transformer architecture.

• 8:26Craig S. Smith?

  And these agents are, are now, uh, they have conversational interfaces, uh, that make them a lot easier to interact with. But they're also now, as you just said, talking to other agents. I, I have to confess, uh, I didn't realize that, uh, Google, uh, cloud was so much in the solutions business. I thought it was primarily a resource provider. So how can you talk about how agents have developed under, you know, since generative AI became the dominant technology?

• 9:10Speaker 00

  Yeah, absolutely. There are a couple of different directions that we should look at when you talk about like the evolution of agents. You've already highlighted in my mind what is the most profound change today, which is we now have a natural language interface into every data store that you can imagine. And for those of us that have been in data and AI for a while, like I have, I mean, one of my startups, I was doing unsupervised learning on cyber events to try to like cluster them and figure out like where the most severe threats were. I mean, having and being able to express in human language and intent and then be able to actually get like business intelligence back.

• 9:49Speaker 00

  Short circuit so much of the data engineering problem that is, you know, for a data scientist in the data team, eighty percent of their toil. That is the most profound change. And you could think about, and I know we'll probably talk about agent space at some point, which is our platform for building agents. But you know, the vision behind something like agent space is that it enables human language as the new programming language for AI. And so that by far is the most profound shift is this conversational interface to all of data and instructing AI using conversations. The second really big shift is the ability to now orchestrate an agentic workflow

• 10:30Speaker 00

  using an intent and actually being able to wrap like APIs into tools. So now that AI can go and and access, you know, other functionality in a very seamless way. Give me an example. We are working with a customer right now. It's confidential. You know, my team works on a lot of very early stage stuff, but we're working with a very large customer that everybody would know. And one of the problems, this is like typical of an enterprise, you know, so there are there are really cool problems we're working on, like finding planets, new planets, and then there are other problems we're working on, like submitting your receipts to an expense system. So we're, you know, so like the problem we're working on right now is if you've ever worked in a large company, you know that you have to submit receipts if you travel or if you eat out or whatever.

• 11:13Speaker 00

  And it turns out that OCR, optical character recognition is a, has been a really tough problem for a long time. You have to train models. It doesn't usually get it right. And you've got all sorts of, you know, sadness. As someone who travels a lot, I've lived this firsthand and we've been working with this really large company to say, well, what if we moved from typical OCR, which maybe captures the total on a receipt when you take a picture, gets all the rest of the stuff wrong. You have to have a human intervene anyway. And you end up doing like twice the work to an agentic system that says it takes a picture. And now it's first, it's going to, you know, try to recognize all of the characters. But then it's also, once it's got all the characters, it's going to go to your

• 11:55Speaker 00

  audit system and say, is there anything out of tolerance based on what it just has read? You know, is there anything that's out of compliance? And then it'll also go to like a financial system and it'll say, you know, is this within the bounds of, or is this, what type of currency is this? You know, what type of currencies you express something? So you have like five or six different, you know, API calls and tools happening underneath the scene. And all the user experiences is just a button press, an automatic cataloging of all this information and a submit button. And so that is, in my mind, you know, the second big area of change is that it's no longer just a singular task executed with a very rigid kind of rules-based workflow.

• 12:37Speaker 00

  Now you're accessing other systems, gathering intelligence and wrapping it all in an outcome that a user doesn't have to like, and candidly, a lot of like analysts and data people don't have to like orchestrate all the bits and pieces too finitely. And the last bit is around trust.

• 12:55Speaker 00

  And you can imagine when you start getting, when you start imbuing agents with the ability to act on your behalf, it requires a whole lot of trust. And so, you know, we just talked a little bit about grounding. So, you know, we're grounding these workflows in, uh, cause one of the, one of the knocks against generative AI is that you gotta be careful because it might not, it might hallucinate. Well, with grounding, you take that hallucination probability way down because now you're accessing authoritative systems, search, um, company data, and you're reducing the likelihood of hallucinations. But you're also a lot, you can now put guardrails on what these agents are allowed to do and they can operate within a certain kind of rules of engagement, if you will.

• 13:38Speaker 00

  And then finally, you can secure this agentic workflow because now, you know, happening in cloud, you get to take advantage of all the cloud patterns, like virtual private clouds and, and keep these systems from accessing the internet or leaking. And so, um, you know, you've got this trust component. So you've got this conversational interface, you've got this like rich tool and reasoning flow, and then you also have this trust foundation. But Agent Space is the, think about Agent Space as the singular platform that when you just want to build an agent, you want to index your data to make, and tools to make it available to that agent, you want to provide the context. That's like the out of the box experience. And it also wraps a couple of really important capabilities because we've always real, we've

• 14:20Speaker 00

  always believed, and this is one of the things that brought me to Google, is the way to succeed as a platform, uh, business, which, you know, I would contend that Google and Google Cloud are, you know, there's a series of platforms and ecosystems. You think about Android, if you think about Chrome, if you think about YouTube, if now cloud, um, you know, these are platform businesses. And so as a platform, you have to be open. And so Agent Space is this nice kind of, you know, guided experience, right out of the box, you get all these rich experiences. But if you've got another framework for building agents that you want to use, like Langraph or something, you know, you can use that and you can bring it to, you know, our Vertex AI Agent Engine, and we'll take care of like setting up servers for you and managing all of like the identity access stuff that, you know, when you're trying to experiment.

• 15:04Speaker 00

  You don't want to have to deal with a lot of those things. And so it's easy for us using the native capabilities of cloud to spin up some servers, get you off and running. So you can use third-party frameworks. You can use ours. You can use our proprietary models. You can use open models like Gemma, which we just, I think we just reached 100 million downloads of our open Gemma model in just the last year. But that openness is what builds affinity to a platform. And so whatever state and enterprise is in, maybe they're just getting started with agents, or maybe they're really savvy and have been building a whole bunch of agents and they want to deploy them effectively, efficiently and kind of build enhanced functionality.

• 15:46Speaker 00

  There's a spectrum of uses of agent space and the underlying technology.

• 15:51Craig S. Smith?

  Yeah. The example of the negotiating agent, negotiating with another agent. You were talking about trust. Is that how autonomous would that be in a typical implementation? Or does it then report, each agent reports back to its human minder? That this is the deal we've come up with? Or are there cases maybe not that high dollar value or that critical where the agents can can actually sign a contract or conclude a deal?

• 16:30Speaker 00

  Yeah, I mean, we're still on the frontier of autonomous, multiple autonomous agents, you know, acting autonomously on more of the kind of high value, high risk, you know, use cases. Most of the use cases deployed right now are ones that you would find, you know, very straightforward. So for example, you know, build me creative for marketing. You know, we're working with WPP, you know, world-class agency. And one of the things that they're doing is they're using Gemini to be able to express an intent for a campaign. And then within three minutes, land on brand safe. Totally generative content that can be pushed out into social channels.

• 17:12Speaker 00

  So that type of, you know, the agents like creating content, refining it, building it into a post and being able to deploy quickly with brand guidelines built in, by the way, like you can take the this is a great part about multimodality in these models is you can take brand guidelines, you can just add it to the model as context without having to do like fine tuning. And it can adhere to those guidelines in the content that it creates. That's an example of, you know, what I would call like, you know, very current, modern agentic workflow. We will see more and more of these autonomous multi-agent interactions because if you think about it, you know, on a value curve and on a risk curve, you're going to take care of the low hanging fruit first. So, you know, content creation and creative agents is one.

• 17:56Speaker 00

  Customer service agents is another. The other really popular one right now is code agents. You know, roughly 25% of new code at Google is AI generated. And we're finding this to be the case more and more across our customers is that it's also, you know, these, these agents for code assist completion, understanding. You know, those are things that can, that can happen, you know, right now. But we'll see more and more of these multi-agent, multi-mode, um, multi-agent interactions on kind of the newer use cases. We'll see more and more of that over the next few years. One of the really critical enablers for that, you know, are, is fast and secure agent to agent communication.

• 18:39Speaker 00

  And so there's a lot of work going on right now across frameworks and across protocols to make sure that, um, you know, they're passing the right information, they're authenticated correctly. And I would say like quite a bit of work right now is going into those platform level kind of identity authorization. And, you know, I mentioned the rules of engagement, kind of guidance to agents. That's, and that's where we're spending a bunch of time right now to enable the multi-agent interactions.

• 19:03Craig S. Smith?

  Yeah, that's, that's interesting. I had a call yesterday with a guy talking about, um,

• 19:10Craig S. Smith?

  Self sovereign identities for agents. Is that an area that you guys are working on at all? I mean, so that when agents, when you send an agent out into the world to do things, the counterparty knows that it represents who it's supposed, claims to represent, that sort of thing.

• 19:33Speaker 00

  Yeah, absolutely. And if you think about sovereignty and trusted execution environments, I mean, this is an area where Google has a lot of strength. We were just at Mobile World Congress, talking about trusted execution environments. You know, that some of our mobile partners are utilizing to create new experiences on mobile devices. So it'll range all the way from, you know, agents acting on device, all the way out to kind of these enterprise agents that are acting across systems of record. But that's absolutely, you know, our heritage and in my opinion, a real strength of Google is, you know, verifying the trustability, if you will, of everything, starting from the hardware that these agents are running on, all the way up through the interactions that they're having

• 20:15Speaker 00

  and that they're taking and carrying that trusted identity. You know, this also dovetails into, you know, another thing that we see in the future, at least, and it's happening a little bit faster than even we had anticipated, is sovereign AI and sovereign compute more generally, which is, you know, we've now, we've now figured out how to provide our customers kind of shells of compute and shells of AI in, with the controls, the data controls, the software controls and the operational controls that, you know, best fits them. So like in Europe, you know, there are very specific regulatory and compliance regimes for data locality, for example. Well, when you're dealing with AI,

• 21:00Speaker 00

  you know, you have to ensure that as these agents are accessing tools and accessing data sources, that it falls within those sovereign controls and those regional controls. And so, you know, we have been deploying things like Google Distributed Cloud and Trusted Private Cloud to customers in Europe for the last few years, and that really sets the stage. So now the government can trust, you know, the data, they can trust the compute, and they can trust the AI that's layered on top of it.

• 21:23Craig S. Smith?

  Yeah, you've been a proponent of multi-cloud systems where we're.

• 21:30Craig S. Smith?

  Google Cloud is, you build something and it's interacting with other clouds, non-Google clouds. How do you ensure in those cases that the the trust and compliance extends to the other cloud provider?

• 21:46Speaker 00

  Well, a lot of this is in standards work, a lot of this is in, you know, so like we put out a secure AI framework as Google, you know, but it wasn't just Google that was participating in this. This is a multi-vendor, multi-company effort. I think what you're going to see along frameworks and along protocols, you're going to see a lot more of this work be multi-vendor, multi-party. It may originate, think back to, like Kubernetes is a good example to me. You know, Kubernetes was a technology that we developed at Google for managing a bunch of containers and compute effectively. But we open sourced it and we had, there was like this foundation, the Cloud Native Computing Foundation that was incubating

• 22:28Speaker 00

  this approach to managing containers. And eventually, you know, all of the large hyperscalers utilize Kubernetes and manage Kubernetes services to manage, you know, fleets of containers. And it's just as popular, you know, across all three hyperscalers, major hyperscalers. It's going to be very similar here. And so one of the questions that we're asking, for example, in our team right now is like, what's the equivalent of the CNCF? For, you know, the technology that will enable agents and agentic workflows in the future. And again, you know, protocols, frameworks, you know, there, there will be a lot of work done to ensure consistency across the board and governments, you know, and our, you know, public policy teams and support.

• 23:14Speaker 00

  You know, we're looking forward to working with these organizations to define this future because it's not there yet. You know, we, I think we just had figured out AI. And now we have these kind of autonomous, you know, agents as an extension of this AI. So something we all need to work on.

• 23:28Craig S. Smith?

  Yeah. For building agents with agents based on Google Cloud, I mean, you're, I mean, sort of the basic components are a reasoning model for the planning. APIs and, and access to tools for, you know,

• 23:54Craig S. Smith?

  Making contacting other systems and making them them happen. And then maybe vector databases or other kinds of knowledge.

• 24:07Craig S. Smith?

  Knowledge basis that that give some grounding. Is there anything else? I mean, are those the basic components or what what else is involved in building an agent?

• 24:19Speaker 00

  Well, so you highlighted pretty much like the top level architecture. I would say there are a couple nuanced things under the under the hood that are really important. So number one is if you want to enable great AI data is really important. So one of the key things in agent space is enabling Google quality search on enterprise data. Because when this, when this, you know, AI goes and looks for, you know, grounding or references, the quality of the data sources that it's calling will matter a lot. One of the emerging things that we've been doing in in search and at Google for a long time, but our customers are now, you know, the most sophisticated customers are utilizing

• 25:05Speaker 00

  is, you know, like the combination of knowledge graphs and AI workflows. You know, the relationships between things as as informative as the things themselves or the records themselves. So that's, you know, again, enabling this Google quality search over data is and these indexes. That's critical for this agentic workflow. Two, think about it from the point of a user.

• 25:33Speaker 00

  I want to go build an agent. What's the first thing software developers do when they need to go write code? They go look for some code that already exists that kind of does exactly what they need to do. And as a friend of mine and and, you know, someone I really admire a lot, Kelsey Hightower used to say, you know, good engineers copy, great engineers paste, you know? And so you're, one of the things that we're going to enable inside of agent space is an agent garden. You know, just like we have model garden today in Vertex that I'm sure, you know, Nishad mentioned to you. We're going to enable the same thing for agents. So if you want to build a customer service agent, if you want to build a marketing agent, if you want to build a data science agent, you shouldn't have to start from zero. So there will be a, you know, kind of a starter pack,

• 26:17Speaker 00

  if you will, of agents inside of agent space to jumpstart the development process.

• 26:23Speaker 00

  And then from there, you know, it can get into connectors. So it's really important that, you know, data sources could exist. It should be, could be everything from SharePoint to SAP to, you know, BigQuery. And so enabling the set of rich connectors, this is actually one of the things, having been here for a while, you know, BigQuery, one of the, one of the kind of breakout moments for BigQuery was when we were creating connectors to other data sources that could loop in and out of BigQuery more seamlessly and allowed people to do, you know, to use BigQuery for the analytics scale that it was really good for, but also loop in data more seamlessly in and out. And we're going to see the same kind of path in agent space,

• 27:06Speaker 00

  which is the connectors and the availability of those connectors is going to drive a ton of the value in the platform. It's just going to simplify the, the actual work of getting agents going.

• 27:15Craig S. Smith?

  Yeah. And I'm sort of looking forward to where this is going. You know, I've spoken to Microsoft, they, they use the term society of agents. I'm not sure what Google Cloud's term is, but these vast multi-agent systems that will take over whole layers of an enterprise's operations, whether it's, you know, maintenance, tracking maintenance, or predicting maintenance calls and things like that, you know, factory or, or something else.

• 27:55Craig S. Smith?

  How big do you imagine these agentic networks can, will get it? Or how big have you seen today?

• 28:06Speaker 00

  Yeah, it's interesting. I was just, I was just at a large customer of ours this week at their kind of annual tech kickoff, speaking with their CTO, CDO. This is a Fortune 5 company. And the way I described it to them and the way that they're architecting it is that. It will be a cloud of connected agents, if you think about it that way. So it can scale up and it can scale down, just like the cloud. Could also be, you know, the extreme scale, but it can also be, you know, a job that just runs for as long as it needs to and is more ephemeral. So I like to think of it as a cloud of connected agents. And that's really the way they're thinking about is there are going to be some agents that are long running, you know, kind of always on and very active. And there are going to be a lot of agents that they will accomplish their goal and then they

• 28:51Speaker 00

  will be, you know, will just knock the resource away and free it up for the next thing that we need to do and construct another agent on its behalf. And that's why I keep mentioning these protocols and frameworks and like the lattice work underneath, because it's all about scale. And, you know, as a company, I think about another analogy is maybe like RPC and GRPC at Google, like machine to machine communication, the way for us to manage all of these, you know, this vast fleet of machines and computer science operating in real time is you have to strip away as much of the fluff as you possibly can, and you have to make these things as efficient as possible. So for example, like we care a lot about the packet size that's traveling over the network, right?

• 29:36Speaker 00

  Because today, a workflow for customer service is a very like it pops up, it happens for a little bit and then it goes away. Now imagine if you're running 24/7 continuous agent driven ops for customer service, your network could become very chatty. And so we're also thinking about how to optimize, you know, transaction sizes, payload sizes, you know, all the little stuff that accretes over, you know, thousands, if not millions of agents. And that's the same way our largest customers are thinking about it because they're these diverse multinational, many function, you know, many vertical lines of business even within one company. That's the way they're thinking about it because that represents, you know, the way their business actually operates. So a cloud of connected agents is how I view it.

• 30:21Craig S. Smith?

  Wow, that's fascinating. And there's a lot of experimentation going on. Do you see these Fortune 5 or Fortune 10 or Fortune 100 enterprises really implementing these into their, their operations into these multi-agent systems? Or is it still? People are talking about and experimenting, but the trust level is not there.

• 30:54Speaker 00

  I think the state of the art for the large enterprise right now is, you know, I mentioned the three common, most common, code, customer service, and creative. Those I see in most of the large companies and they're already executing even very complex workflows, agentic workflows against those three.

• 31:14Speaker 00

  What we will see more and more of, and I actually think this is one area where large enterprises may have parity with startups. Startups have the ability to move quickly in a very specific domain and like an agent for X. And you just fill in the blank with X of like, you know, a business function or like, uh, what's it, an example might be, you're trying to generate a document that, that, uh, in the pharmaceutical industry, that has all of the knowledge and all of the state of prior attempts to get like FDA approved. And what you're trying to do is have the agent construct for you while the R and D is happening, like a preemptive document that says, you know, this could be our FDA approval release document before the thing's even done.

• 31:59Speaker 00

  But like based on all the things we've tried before, based on the nature. So the startup can get the, the wiring done really quickly. But the large companies have the data. And so the large companies, the more and more they leverage, you know, modern platforms and I, you know, contend agent space is one of those. They can reach parity with these startups because they can get quickly to the agent building. But they have the data in the context and all the documents of their FDA releases that they can feed to the agent, give it the context that it needs to create a better document outcome. And so I think you're going to see a lot of these companies that are, um, you know, whether it's,

• 32:38Speaker 00

  if we're just talking about pharmaceuticals, it could be media and entertainment. Yeah. What if a large sports organization wanted to create a real time media editor that's agent powered? Well, they've got all the videos. Right? So when it comes to, you know, yes, large enterprises don't always have the processes and procedures and ability to move as quickly to necessarily, um, you know, layer in the technology all the time, but they have the data and the data and the data quality will end up creating agents of higher quality. And that's really, um, you know, Craig, one of the big things that we bump into right now in terms of, you know, kind

• 33:17Speaker 00

  of what will be of unlocking agents is evals and quality, because it's really quite straightforward today to build an agent of low quality. And a lot of our benchmarks are built around scientific benchmarks, which are great, especially when you're dealing with science problems. But when you're dealing with an agent that can create a document of sufficient quality to pass an FDA submission, you know, that has to be really high quality and it takes some time and some data refinement to get agents to generate documents of that high quality that can pass the muster of like a regulatory regime. Um, and so evals, solving evals would allow us to solve auto raiders, which is another thing my team's

• 34:01Speaker 00

  working on, because instead of having to have humans evaluate the quality of everything, if you can create a baseline, now you can have AI actually evaluate itself. And think about how much velocity that adds to a workflow rather than stuff piling up on a human's desk. Imagine like you've got a million agents running around and they're creating all of this stuff that humans need to review. The next problem is going to be humans have too much to review. So they're going to need some help and the help will be in the evals and the operator side.

• 34:29Craig S. Smith?

  Yeah, you know, there's, uh, Yuval Hariri, uh, had a book out recently and he was on all the talk shows and, and talking about, you know, the day will come when the sort of base bureaucracy of society is going to be run by networks of agents and, uh, it will...

• 34:55Craig S. Smith?

  His warning is that it'll get so complicated uh that we'll be beholden to the agents because it will have surpassed the ability of humans to to manage the uh the workflows. Do you think about that at all?

• 35:13Speaker 00

  Uh, I I am definitely a technologist, a pragmatist, and someone who comes from a world where, uh, so for example, like flight safety, you know, uh,

• 35:24Speaker 00

  that's a problem that requires like AI assisted um workflows, but human decision making. And I think that's the balance we're going to see more and more, which is prompting from AI systems. There there's like two, there's like, there are more classes than this, but for like time and simplicity sake, because we don't have all day, you know, we could, it'd be great, but we don't have all day. Uh, let's put them into two buckets. There's like the general class of workloads where toil can be removed and it's a win for everybody. And they're very low risk. Then there's like the second bucket, which is

• 36:03Speaker 00

  toil can be removed, but you have to do it very carefully because the decisions are kind of, um, you know, as you other leaders have framed this, you know, one-way doors. And they're very important missions. So like airspace deconfliction. You really, really want to remove some of the data processing, um, you know, burden and toil that humans today are having to munge through. But you probably want to keep the humans in the loop on the decision side and prompt them in ways, allow them to make more informed decisions. That is, you know, what I see as the, you know,

• 36:39Speaker 00

  as the potential for these complex AI and agentic systems is to is to cue humans in better and more holistic ways to make better decisions. There are going to be a class of problems where it's literally people just moving papers or documents from one place to another. And those, you know, like You know, there I don't know that a human, you know, wants joins a company so they can drag and drop, you know, a document from one place to another. Um, there's a whole class of problems that, uh, you know, are probably in need of some retrofitting and some acceleration via AI.

• 37:19Craig S. Smith?

  Yeah. I mean, you were talking about the expense receipt, uh, problem. Uh, and you can imagine that, that there's the solution that you were referencing where, uh, you know, the, the, uh, employee uploads a scan of his, um, receipts and the agent processes them and, and then sends, uh, instructions to, uh, you know, a dispersment agent who then disperses the funds, uh, and then

• 38:01Craig S. Smith?

  notifies the, uh, ledger agent to, to balance the books. And, you know, there's this whole series of agents that, that take care of that whole, uh, that whole process. And then on accounting and budgeting, presumably agents could, uh, do, do that pretty efficiently, autonomously. Um, so yeah, it's

• 38:29Speaker 00

  Very much, very much so, Craig. In one of the, in that example that you just gave, like, let's combine the concepts, you know? Uh, you just mentioned like, you know, over, overloading humans and, you know, the potential of, you know, like them escaping, you know, plausible, um, solutions via humans. I've been doing this a long time and it turns out that, uh, at least my experience is most of the time you, you're kind of like you incrementally work and you're, and you're chipping away at pieces of the problem. And so in the workflow that you just described, and we were talking about earlier, the expense one. You know, what you really want to avoid is is kind of over automating things that have really high consequence without really understanding what you're getting into. So one of the ways

• 39:14Speaker 00

  that an organization can learn how much to automate with agents in terms of decision or budget allocation is to start with a small dollar items, right? And to work your way up. So if someone's submitting a expense report and the AI says, you know, a receipt for 40 bucks, you can get some of those wrong and it's not going to bring the company down. Candidly, a lot of companies get them wrong today and the way they solve it is by hiring people to like overwatch it, which then adds fixed cost onto a variable cost problem. But you can also like learn where your thresholds might be. And a business might say it really doesn't make sense from a risk perspective to automate the really expensive things that come through an expense system. And it's still worth having some of those like human, more human in the loop

• 39:59Speaker 00

  systems. But for anything under a certain dollar guy, we just like, we just take care of it. It's just gone. And those economic trade offs, those really, you know, pragmatic economic trade offs. That's what's happening behind the scenes in all of these, you know, large enterprises is they're working their way towards the really big things. And then they can reason better because they've got more experience from doing the smaller things.

• 40:21Craig S. Smith?

  Yeah. How does one of the top companies globally, how do they work with you? I mean, they're already presumably Google Cloud customers and Azure customers and AWS customers.

• 40:43Craig S. Smith?

  Would, how, how does that happen? They want to build this agentic layer or agentic workflow. They've got options at all of these plus they've got, you know, McKinsey and, you know, Boston Consulting Group and, and these other consultants and then, you know, more specialized consultants, everyone knocking at their door saying, we can do this for you. How does it happen that, that Google Cloud gets involved in building or you?

• 41:22Craig S. Smith?

  Advising and and they're doing it on their end sort of using some of your advice, some of Microsoft's advice and, you know, I mean, I've always wondered that how how much of a lock do you have on these uh on these problems that you're solving?

• 41:46Speaker 00

  Well, it's it's an interesting question because um I mean, that's what started my function. When I showed up at Google 10 years ago, I wasn't, you know, we weren't, we didn't have a CTO function. Uh, you know, I was just some random guy with a laptop and a dream trying to contribute to the future of cloud, you know, at the time, our cloud business was a little smaller and our engineering leadership could fit in like a conference room. And um, it's interesting because the function actually arose out of a need that we so Google has always had really great consumer technology sensibilities. I mean, that's the core of, you know, the business. And as we were building out cloud, one of the things that, um, you know, the company realized is that enterprise innovation is

• 42:31Speaker 00

  a long-term partnership and it's a two-way street, you know, we will build things and and customers will, you know, try to rationalize how to use them. But then customers will have problems that should impact what we're building over time. And if you go back to the what I described as the, you know, the two core functions of my team, you know, we decided to set off, you know, this group and start this group, we were very small at the beginning. And the hypothesis was to build a little bit stronger bridge between the decision makers at our of large companies and fast-moving companies, fast-growth companies, and the core of engineering and research and development at Google Cloud. And, you know, over time, Google Cloud has scaled out, you know, our go-to-market team, our

• 43:14Speaker 00

  professional services organization, our partnerships, um, you know, the the amount of just high caliber people across, you know, engineering, sales, marketing is, you know, exploded, has been fantastic. And

• 43:28Speaker 00

  The senior people at, you know, for our largest customers and our largest prospects, they still want to have a direct access into, you know, kind of the where we're headed, why we're headed there, so they can balance that against what they hear from everybody else that they deal with. And so I encourage, you know, I'll always encourage our customers to get many points of view because I've been in their seat. I've been a public company CTO who deals with many, many vendors. And it's, you know, people who, one, acknowledge that this is, you know, a multi-vendor world. And I would argue that Google and Google Cloud has been on the forefront of multi-cloud and truly OpenAI, you know, all along. This has not been something we just came to. We have been, these have been embedded in our

• 44:12Speaker 00

  principles forever. And that builds a lot of trust with our customers because they know that those principles are what we build our technology on. Yeah. Second is, you know, our technology actually reflects our principles. And so when they say, hey, we, you know, we're really interested in multi-cloud analytics, we build a version of BigQuery that can return results from other clouds without having to move the source data. You know, that's a very complicated, unusual architectural pattern, unique to us, that we built because our customers asked for it. You know, just like with AgentSpace, you can start from a UI and a very opinionated flow of building agents, you know, using, you know, our core first-party models, using our

• 44:55Speaker 00

  frameworks, or you can come from the other end and you can deploy a model that you really like and using frameworks that you really like on top of our infrastructure. And we can help you spin up the underlying stuff. And it's that trust, you know, why do, why do people come to Google Cloud? The feedback I get, and I've, I've been at this a while, a long time now. I've been doing the CTO gig for four or five years and started the team, you know, nine years ago. It's because we, you know, trust accretes over time. And they see that we act in line with our principles, our software and what we build is in line with our principles. And it truly is, reflects the need to be open. You know, we mentioned

• 45:38Speaker 00

  trusted before. And by the way, our roadmap, I would be remiss if I didn't mention that most of the top customers that I work with, one of the primary reasons they choose Google and Google Cloud is because of our roadmap. You know Gemini is a state of the art multimodal reasoning model that is also on the frontier of efficiency for cost. And that gets, that gets attention because multimodality is like the killer app of generative AI, in my opinion, at least right now in this class of things, you know, as the reasoning capabilities get built out and they evolve. Being able to drag a document to a model, being able to, I gave this example to a customer of mine recently.

• 46:22Speaker 00

  Have you heard of the critical or the common vulnerabilities and exploitations database? So in cybersecurity, there's this website that, you know, the government and a federally funded research organization maintains and it's all the common vulnerabilities found out in the world, the, you know, the malware, the hacks, and it's a database, but it's all in JSON and it's kind of cumbersome to work with. So I was meeting with a very senior customer and their CISO and they were like, well, how is all this AI really, you know, what, what does AI really mean to me? And I showed them that you can go to the website, you can download a folder of files natively, you can drop it in the context window in Gemini and it will interpret what the most common vulnerabilities have been over the period of time of the files that

• 47:06Speaker 00

  you dragged in, it'll give you like recommended remediation steps. And that's without any transformation whatsoever. There's a whole, there have been whole companies built around just taking JSON file threat data and turning it into, you know, human readable, understandable synthesized outputs. And so that, when, when they, when they have the long relationship, they see we're acting on our principles and we can give them a little bit of this magic. That, you know, that makes our relationships, you know, really, really strong and enduring.

• 47:40Craig S. Smith?

  Yeah. And this is an aside, but I have to ask, how big is the context window now for Gemini?

• 47:48Speaker 00

  And, and Yeah, so a million tokens, but we've also been very public in that, you know, we've been operating at multiple millions of tokens and it's not.

• 47:59Speaker 00

  It's an architectural trade off, you know, because there's also now we've enabled things like context caching, where you can take in kind of a less intense way, you can make sure that the context that these models need throughout a workflow are available to it, and you can just use caching as a mechanism now too. So we've also added functionality that once you reach a certain point of the context window, you're creating a trade off between, you know, like rapid memory immediately available and memory available with like a little bit of a delay. It just like storage, like archival storage versus like always on storage. It becomes more of a how much do you want and for what use case and for how much money? So, you know, just as we always have with cloud, we're offering a range of options architecturally.

• 48:43Craig S. Smith?

  Yeah, and doesn't the accuracy decline as the context window fills up?

• 48:51Speaker 00

  Not necessarily. Again, it really depends on the implementation. I'm one of the, actually, one of the projects that we took on a few years ago within my team was, we looked at whether you could just solve all the problems with the context window or whether you are, we're always going to need like a secondary retrieval augmented generation source. And it turns out for a lot of use cases, having both yields the best results. So, you know, bringing it into the context window, but also, you know, bringing in some other sources and, you know, that context is what, you know, the model acts on. And in our team, we've coined a phrase of context engineering versus prompt engineering, because I think that is more reflective of the way people actually look at solving these problems with these models.

• 49:36Speaker 00

  It's through context, not through prompts.

• 49:38Craig S. Smith?

  Yeah, you know, DeepMind recently came out with some work on robotic.

• 49:49Craig S. Smith?

  Speaking, you know, robots that that can interact with humans using language.

• 49:59Craig S. Smith?

  How in in these networks of agents, how much are you employing robotics? In them, I'm thinking like a warehouse, network of agents in a warehouse that can track inventory and then restock. And there's a certain amount of robotics involved in that.

• 50:23Speaker 00

  We have a number of customers who are very interested and active in robotics, especially in use cases like what you described around inventory management warehousing. You'll imagine, you can imagine this also apply to automotive on the floor. You know, because that's a place where machines and robots have already been very, very productive is in the assembly of, you know, advanced electronics and in in machines like automobiles. Interestingly enough, one, one project that we're working on right now with a racing league is creating like a, like an R two D two.

• 51:02Speaker 00

  Because in software, you know, we can already take data off of a race car, we can already take voice inputs out of, you know, both the crew and the driver. We can take real time video feeds. And since we have multimodality and we have, you know, really fast response times and cloud that can scale up and scale down. We're now look, we're now able to modify race strategies in race based on real time inputs coming in from, you know, how someone's braking, how they're accelerating, what the response has been on the machine, what the video is seeing and interpreting, like the condition of the roads degrading over the period of a race or

• 51:45Speaker 00

  the crew and, you know, like the, even just like the tone in people's voices. So, you know, in terms of, you know, like I, I guess I'm kind of a Star Wars nerd, but you know. R2D2, you know, having instantiating an R2D2 is essentially enabling control or like a conceptual interaction with any type of physical robot. Plus, you don't want to just have the machine itself. You want to have all the context that surrounds it. So again, going back to the context engineering, because then they'll have more functionality. And yeah, that's, it's actually a project that we're working on right now.

• 52:21Craig S. Smith?

  Fascinating. If, if somebody wants to use some of these tools and they're not a Google Cloud customer, not a big Google Cloud customer, is it as simple as logging on to agent space?

• 52:39Speaker 00

  And yeah, that's what I, well, that's what I do. I go to cloud.google.com. I go to the console and you can try out, you know, all of our different capabilities. I'm also very excited. We have NEXT coming up, our big annual conference in a few weeks. We'll be rolling out much more on agent space, its capabilities. And so I also encourage folks that they're looking to get, you know, like, just like I O at Google is kind of our consumer side, single point of like all the latest and greatest cool stuff all in a short period of time. If you really want to get the high calorie condensed version of all the stuff we're working on, you know, logging in virtually to NEXT, or if you can, you know, make it, you know, head to Vegas, just be careful, stay hydrated.

• 53:24Speaker 00

  You know, I think you, your listeners could get as much as they can stand on what's the latest and greatest in technology from Google and how to use it. With this year, we've also really bolstered the central developer, like everything's oriented around people who want to build. So on like the floor, all of the experiences and all the like real world sharing mechanisms are right in the center, easy to find.

• 53:47Craig S. Smith?

  Okay. Well, that's a good way to, to end the podcast, but I'm going to ask one more question. What, you, you say you, you, you sort of sense the tremors of what's coming. What do you see beyond?

• 54:01Craig S. Smith?

  What we've talked about that that's that's going to change things.

• 54:05Speaker 00

  Uh, there are a couple of topics, uh, you know, right now that I'm really interested in and our team is, you know, actively exploring and I think, you know, will be really, really fascinating. One is how AI is going to change the world of science forever, whether it's talking about materials, molecules, minerals. You know, we'll be talking about this a little bit more next too, but, um, you know, AI for science is the the outcomes and the magnitude of the outcomes are are mind blowing. One of the first projects I worked on when I got to Google was with NASA and it turns out that if you can give people a GPU and a dropdown window, you know, attached to a virtual machine that they can find planets.

• 54:51Speaker 00

  Interns can find planets, new planets. Um, just imagine what we can do, you know, with modern, with modern AI. And so that that area in particular is of high interest to me. Um, but we'll also see the evolution of these multi agents, this kind of cloud of connected agents will become a cloud of autonomous agents. And that has a lot of implications in workflows and how businesses operate, whether you're a startup and you know, you've got three people and you can leverage the superpowers of an entire industry and all the world's knowledge immediately, or you're an organization who has deep customer experience and deep customer context and wants to, you know, provide like a concierge, like front end to customers instead of making them

• 55:34Speaker 00

  interact with your app or your surface the way they have to today. It's kind of like bit by bit. Imagine if you had like a an AI concierge that guides you through an experience. It's intuitive without you having to provide a lot of data. Um, you know, those are the types of things that are on the horizon that I'm extremely excited about.

• 55:50Craig S. Smith?

  Create an oasis with Thuma, a modern design company that specializes in furniture and home goods. By stripping away everything but the essential, Thuma makes elevated beds with premium materials and intentional details. I'm in the process of reorganizing.

• 56:10Craig S. Smith?

  My house and I'm giving Thuma a serious look for help in renovating and redesigning. Thuma combines the perfect balance of form, craftsmanship, and functionality. With over seventeen thousand five-star reviews, the Thuma bed collection is proof that simplicity is the truest form of sophistication. Using the technique of Japanese joinery, pieces are crafted from solid wood and precision cut for a silent, stable foundation. With clean lines, subtle curves, and minimalist style,

• 56:52Craig S. Smith?

  the Thuma bed collection is available in four signature finishes to match any design aesthetic. Headboard upgrades are available for customization as desired. To get a hundred dollars toward your first bed purchase, go to Thuma, that's T-H-U-M-A dot C-O slash I on A I. I on A I all run together, E-Y-E-O-N-A-I. So for a hundred dollars off your first purchase, go to Thuma dot C-O slash I on A I.

• 57:31Craig S. Smith?

  That's T-H-U-M-A dot C-O slash I on A I to receive a hundred dollars off your first bed purchase.