#252 Jeffrey Hammond: How to Build Scalable GenAI Products (AWS Strategy)

This episode is sponsored by Netsuite by Oracle, the number one cloud financial system, streamlining accounting, financial management, inventory, HR, and more. NetSuite is offering a one-of-a-kind flexible financing program. Head to https://netsuite.com/EYEONAI to know more. AWS partnered with Forrester Research to understand how software providers (ISVs), in particular, plan to drive profitable growth with generative AI, how they are uniquely approaching generative AI development, and the key challenges they're facing. In this conversation with Jeffrey Hammond, Global ISV Product Strategist at AWS, he dives into the findings of the research and discusses how — particularly with AWS's help — ISVs can drive profitable growth and succeed in the gen AI gold rush. Jeffrey helps software product management leaders leverage AWS cloud services to accelerate product delivery, create new revenue streams, reduce technical debt, and optimize operational costs. You'll learn: Why "toil reduction" is the fastest path to GenAI ROI How AWS's GenAI Innovation Center helps companies cut costs and ship faster What most ISVs get wrong about trust, security, and customer communication The secret to scalable AI product pricing—and what Canva got right Why agentic workflows and federated models are the next frontier in software Whether you're building on AWS or just exploring GenAI adoption, this conversation is packed with frameworks, examples, and strategy. Stay Updated: Craig Smith on X:https://x.com/craigss Eye on A. I. on X: https://x.com/EyeOn_AI (00:00) The Future of Work with Generative AI (03:20) Inside AWS: How Jeffrey Supports AI Innovation (06:00) What the Forrester Survey Reveals About AI Adoption (09:15) From Hype to Value: Building Real GenAI Use Cases (13:45) How ISVs Are Reducing Toil and Driving Efficiency (17:10) Balancing Innovation with Trust and Security (22:00) AWS Programs That Help ISVs Win with AI (28:00) GenAI Product Strategy: Accuracy, Cost & Pricing Models (34:30) Overcoming Infrastructure Challenges in GenAI (39:45) The Rise of Agentic Workflows and Interoperability (46:00) The Biggest Tech Disruption in Decades?

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Executive coaching is something that has traditionally been done, you know, kind of person to person. It's very high touch. As a result, it usually, it usually only happens at the executive level. There's a very, very small number of people that, that organizations are willing to, to support. There's all these kind of low value added activities that, that even though I like our expense reporting system, it's actually a lot better than, you know, anything else that I've used in the past, there's still opportunities for improvement. Net, net, they were able to, to get to a point where they think they can get a 70% improvement in their product team execution. That's pretty significant.

• 0:39Craig S. Smith?

  There's a growing expense eating into your company's profits. It's your cloud computing bill. You may have gotten a deal to start, but now the spend is sky high and increasing every year. What if you could cut your cloud bill in half and improve performance at the same time? Well, if you act by May 31st, Oracle Cloud Infrastructure can help you do just that. OCI is the next generation cloud designed for every workload where you can run any application, including any AI projects faster and more securely for less. In fact, Oracle

• 1:18Craig S. Smith?

  has a special promotion where you can cut your cloud bill in half when you switch to OCI. The savings are real. On average, OCI costs 50% less for compute, 70% less for storage, and 80% less for networking. Join Skydance Animation and today's innovative AI tech companies who upgraded to OCI and save. Offer only for new US customers with a minimum financial commitment. See if you qualify for half off at oracle.com/eyeonai. That's oracle.com/eyeonai. E Y E O N A I.

• 2:03Craig S. Smith?

  All run together. Oracle.com/eyeonai. So go ahead and introduce yourself, Jeffrey.

• 2:11Speaker 00

  Okay. Thanks, Craig. So I'm Jeffrey Hammond, and I am a Global ISV Product Strategist at AWS.

• 2:20Speaker 00

  I work primarily with software company customers and you know I've had a long career in in the software development space. I started out as a developer for a number of years and then spent more than a decade building software development products primarily in product management, product marketing roles from from small startups to large, high growth and then acquisition in my case by IBM. After that, spent a number of different years working with a lot of different software companies as they built their products and worked through multiple technology disruptions. Mobile technology is an example, cloud technology, and now here we are again with disruption

• 3:05Speaker 00

  in the AI space. And so that's really what I do as a product strategist. I work with our software company customers to help them understand the opportunities that they have and then how Amazon can can help them take advantage of those opportunities and accelerate the way that they they they seize them to to drive profitable growth.

• 3:25Craig S. Smith?

  Right, yeah. I mean, you work with software vendors, not not with people who are integrating AWS or building on AWS.

• 3:35Speaker 00

  Yeah, yeah. Amazon has five different segments that we that that we organize customers by. There's there's enterprise, makes sense, small, medium business, startup, digital native businesses, and then the fifth one is ISV. And the reason that we do that is because, as you're probably well aware, ISV software companies have particular challenges when it comes to building products. They live and die by the success of the software that they build. And because they are serving their own customers, those customers often have specific requirements on them that that that create demands on the products that they build. So with respect to AI, you're not just building a service that you're deploying

• 4:18Speaker 00

  internally inside the firewall to your own employees or maybe to consumers. You're building products that you might have to deploy to thousands of businesses that all have their own unique demands with respect to what they want to do with those those AI capabilities. And so you've got to be able to prepare for that when you design your products.

• 4:41Craig S. Smith?

  Yeah, and and these are independent software vendors that are building products on AWS or with AWS using Redlock or SageMaker or whatever the

• 4:55Speaker 00

  That's right.

• 4:56Craig S. Smith?

  Yeah.

• 4:56Speaker 00

  Or even considering it, you know, they may not be building on on AWS yet, but they think that there's an advantage that they can get by doing that.

• 5:05Craig S. Smith?

  Yeah, you did a survey with Forrester Research of about how successful ISVs are in in working with generative AI. Can you talk a little bit about that?

• 5:26Speaker 00

  Yeah, yeah, happy to. One of the things that that we do when we're when we're thinking about strategy and serving our customers here is we like to dive deep and data is one of the ways that you can dive deep into understanding what's going on in an industry segment. And so we commissioned this survey in the the fourth quarter of of 2024. And we went out and Forrester designed the survey, ran it, got over 650 responses around the world, spread across three different regions, North America, EMEA and APAC. So

• 6:06Speaker 00

  it gives us a good perspective on what's happening around the world, not just here in in North America. And what we asked the Forrester researchers to do was to ask about implementation challenges, plans, future goals, what what these software companies were trying to accomplish. It gives us insight into how we should we should help them. But we also wanted to make sure that that data was more broadly available because you know, it helps informed decision making with our customers as well. So it was a lot of fun to take a look at the the details. You know, I'm often say with with surveys, the data is the data and and it's interpreting the data that

• 6:49Speaker 00

  is is what I like to do and then help our customers work through that.

• 6:54Craig S. Smith?

  Yeah, uh, and there's, there's been this rush to implement generative AI. Um, a lot of experimentation, a lot of pilot programs. Um, but they've, a lot of people have ended up with premature solutions. Uh, and unproven business models, high development costs. Um, so they need to prioritize value-driven use cases.

• 7:28Speaker 00

  Right.

• 7:29Craig S. Smith?

  To succeed. Can you talk about that, about, about how they do that prioritize prioritization?

• 7:39Speaker 00

  Yeah, yeah. So first of all, I think it's important to, to, to understand that, that that sort of rush to, uh, to, to build isn't necessarily unique to generative AI. I've seen it in, in lots of other major technology disruptions. I think about, um, when the iPhone came out in 2008. We had lots and lots of organizations building apps as quickly as they could without a clear understanding of how they were going to drive value. And then, you know, they kind of learned, uh, what worked and, and what didn't. Um, and so I think that we're working through that process. Uh, it's one of the, the reasons that, that I, I think it is so important with respect to, to, to generative opportunities to work backwards

• 8:22Speaker 00

  from the customer. And that's just something that's like second nature at, at AWS. Um, you know, uh, there's a quote from, from, from Jeff Bezos in our, in our 2016 shareholder letter that talks about the fact that customers are always beautifully, wonderfully dissatisfied, even when they are, re- report being happy. Uh, and their business is great. And so if you go out and talk to the, to a customer and you ask them a question like, where, where do you have to do things today in this product? And it's still a pain, a pain wherever you want to call about it. And I'll, I'll use expense reporting as a, as an example.

• 9:00Craig S. Smith?

  Yeah.

• 9:00Speaker 00

  You know, I've been doing expense reports for over 30 years. Uh, and I think about how I used to do expense reports when I had to tape receipts, uh, to pieces of paper. And then I had to scan those pieces of paper and then I had to mail them in. That was a high toil process. But even today, you know, there's still a lot of toil involved in how I do expense reports. Anything that's over fifty dollars, I've got to have a receipt for. And sometimes that means I've got to go search my email for the receipt that was sent to me by the hotel. And so there's all these kind of low value added activities that that even though I like our expense reporting system, it's actually a lot better than, you know, anything else that I've used in the past. There's still opportunities for improvement.

• 9:41Speaker 00

  And that's really the the starting point of of being successful and driving business value in in AIs, working backwards from the things that customers want. So toil reduction is one of those. There's lots of good examples of of very successful use cases in that world. Accounting is a really good example of a business where there is high toil. And there's also a shortage of labor. If if you if you look at the research, there are just not enough accountants to fulfill the demand. And so you're seeing practically every accounting software company that that we work with,

• 10:20Speaker 00

  making large investments in generative AI to reduce the toilsome activities and give accountants more leverage. Healthcare is another example of that. I was talking with a a company last week that's a customer of ours. And one of the things that they said is whenever a lab runs a a blood test, you know, there's a whole bunch of of figures that come back. And yeah, a doctor or a clinician can take a look at those and spend, you know, seven or eight minutes kind of looking through the lab and trying to extract that information. But what if we can automatically extract the most important information, give them the summary at the top and reduce the amount of time that they spend scanning through that report from eight minutes to to to three or two?

• 11:05Speaker 00

  And what happens if we can do that dozens of times a day? It's a small example of toil reduction that adds up in the long running, in the long run. So, so very useful use case. Retail. I worked with a, a company a couple of months ago where they had a case where, you know, third-party selling is, is a really big deal in retail now. You think of all the third parties that sell on Amazon. And one of the things you have to do to enable that is you have to map products from one catalog to another. And that can be a really, really tedious process if you've got a thousand products to map. So, one of the things that they said was, well, if we could get to 90% accuracy with, with, with a, a, you know, a generative process, we could automate this.

• 11:51Speaker 00

  And so, taking that, that identification of toil and then starting to work backward from that was one of the things that they did. There are other examples, content generation, that's what we all know of. Executable artifact generation. To me, the perfect example of that, you know, code is, is what we all think of. Code is an executable artifact that computers can, can run. But there are so many other things, like an, a calendar appointment. The ability to automatically create that, send it out and synchronize it. A data pipeline. Snap Logic is a really good example of a company that has a, a strong meta-model underneath its product. And so, you can either understand those pipelines that exist and summarize them so that, so that they can be explained, or you can create them.

• 12:35Speaker 00

  And there are just dozens of areas in the software space where there are these executable artifacts that are, that are ready for, for a higher level of, of abstraction. So, these recurring use cases, and there are different types of them, are the basis for creating a product strategy. But it starts with understanding what's giving the customer heartburn.

• 12:58Craig S. Smith?

  Yeah, yeah. And you, you were talking about the time savings for doctors reviewing medical tests. I've talked to a lot of people, a lot of enterprises about, you know, if, if you use generative AI to focus on time-saving, on productivity, it doesn't.

• 13:25Craig S. Smith?

  Across a large organization, it certainly is great for the user, for the person in, you know, in an accounting function or something that can use Gen I to summarize and it speeds those saves them a few minutes. But those incremental time savings don't necessarily flow to the bottom line. You know, if I'm an employee, I've saved five minutes. I'm going to check my personal email or I'm going to do something else. And there's something in the in the notes here that were sent over by your team.

• 14:04Craig S. Smith?

  Where you talk about finding real value driven use cases that are that are innovative, not not simply time saving. Can you can you talk about about how you do that? How you identify those deeper value situations?

• 14:29Speaker 00

  Yeah, yeah, I think one way to do that is to start by asking the question, where are there opportunities to serve customers or to do things that we have not been able to profitably do before? And I'll give you a good example.

• 14:48Craig S. Smith?

  And just for listeners, when you say customers, you're talking about AWS customers.

• 14:54Speaker 00

  No, I'm talking about the ISVs customers. So again, I I'm always working backward from the ISV and thinking about their customers. Because if they can serve their customers well, they're going to drive profitable growth. So here's a good example.

• 15:12Speaker 00

  Executive coaching. Executive coaching is something that has traditionally been been done, you know, kind of person to person. It's very high touch. As a result, it usually it usually only happens at the executive level. There's a very, very small number of people that that organizations are willing to to support with that model.

• 15:34Speaker 00

  Well, what if we could open up, you know, that sort of career development capability to a much broader set of employees in the organization because we could do it at a much lower cost? That's an example of where an innovation, being able to take best practices, being able to take the identified competencies that an organization values and expose them through a generative approach, maybe an agentic approach, could really unlock opportunity that to this point in time is just underserved. If you look at what's happening right now in marketing around hyper-personalization or clienteling, you know, the reality is, is clienteling was only, you know, really done

• 16:19Speaker 00

  with, with your top-tier customers because it was too expensive to do it for your, your more casual customers. Well, if we can change that equation, then you can use it to drive additional value and expand the market. So, so that's where some of the big ideas lie, you know, and, and I'm not saying you do those just, just in, instead of the little things, because the little things can add up. I think code generation is, is a, is a great example. I, I shared the stage at Reinvent this year with, um, the group vice president from New Relics, Raj Krishnan, and one of the things he talked about was that they've been able

• 17:00Speaker 00

  to drive a 15% increase in developer productivity by deploying, you know, essentially code, co-pilot capability. When you're talking about a couple hundred developers, 15% is nothing to sneeze at from, from, from an expense perspective. So you've got to do both.

• 17:17Craig S. Smith?

  Yeah. You, you also want to be careful that you, you're doing, implementing these AI strategies in a careful way so that you don't lose trust of your customer. Can you, can you talk about that?

• 17:37Speaker 00

  Yeah, yeah, for sure.

• 17:42Speaker 00

  This is actually, I think, one of the takeaways that was a little bit surprising from the survey that, you know, I thought that we would see a little bit higher priority placed on customer trust and on data security and privacy in the survey results. And I think that that was a little bit of a miss from some of the ISVs because we do see that as important. It's one of the reasons when we built Bedrock by design, the way that Bedrock is deployed, it's designed so that the data stays private, that it never leaves, you know, the private, you know, environment that the customer set up.

• 18:26Speaker 00

  Because, you know, that's a huge concern from our customers' customers. And, you know, again, I don't think it's necessarily unique to generative AI. I remember 10 or 15 years ago, when we first had the opportunity to put source code up on the cloud with something like GitHub. And there were a lot of organizations that were like, well, I don't want to put my code up on the cloud. What if I lose control over it? That's my core IP. And then over time, you know, we learned to trust it. We learned that it was, you know, it could be implemented in a private way, that my code wouldn't get exposed. And that trust level got to the point where essentially storing source code in the cloud is just the major way that most companies do it now.

• 19:07Speaker 00

  We have to get to that same level of comfort with the data that customers are using. Because one of the things that we see over and over again is one of the best ways to improve the accuracy of these generative use cases is to include data through, you know, retrievable augmented generation or cache augmented generation. And so making sure that that data is safe, critically important. So, yeah, I think there's probably some more work to be done there.

• 19:36Craig S. Smith?

  Yeah. And is the issue that...

• 19:42Craig S. Smith?

  ISVs adopt AI solutions in their software stack and overlook privacy issues or security issues, or is it that they're not communicating them adequately to their customers so that customers trust the this new AI powered solution?

• 20:12Speaker 00

  Yeah, I think it's part of the learning experience. What I'm seeing is like when we work with a customer and we go through a POC, so much of the focus is on just seeing if we can get to the right economies of scale, you know, where the costs are low enough, the accuracy is high enough, the performance is fast enough. And then they start to worry about the additional illities. Or sometimes what they'll do is is is they'll put this out as a as a tech preview. And then the enterprise customers start to ask these questions. And then it's like, okay, you know, we better make sure that this is, you know, that that that we've got these capabilities as well. So I think it's it's mainly that it's it's a a priority that

• 20:57Speaker 00

  we need to be more proactive about. So we design it in from the start.

• 21:02Craig S. Smith?

  Yeah, and AWS, when you're working with a software vendor,

• 21:10Craig S. Smith?

  are you advising them of this or you're recognizing through the survey that this these are things that that they need to be paying attention to?

• 21:21Speaker 00

  Yeah, when when we work with a software vendor, we definitely call this out. And it's it's, you know, part of the the value that, you know, getting some of our subject matter experts involved in in in the customer when they're building provides. So as an example, if we're doing an engagement with a software company in our generative AI Innovation Center, it's just part of the discussion of working through the process of the use case and then building it out.

• 21:51Craig S. Smith?

  Excuse me.

• 21:52Speaker 00

  Yeah. Bless you.

• 21:56Craig S. Smith?

  Excuse me, comes in threes usually.

• 22:00Speaker 00

  Yeah, yeah, and I should also say it's one of those things that that we build in to the product strategy as well. You know, the idea of secure by design. So a great example of that is is beyond the design of Bedrock itself and how it treats customer data and information is the capabilities that we layer on top of it. Something like Bedrock guardrails as an example, specifically designed to reduce the instance of hallucinations, to reduce the possibility of objectionable content sneaking through. You know, some of the work that our partners do to be able to potentially identify issues like data loss that might be flowing through the prompts that are coming in to an LLM.

• 22:46Speaker 00

  So, so, so, yeah, that's where the value of using a cloud platform and and helping to accelerate the release can pay big dividends because otherwise you got to solve the problem yourself if you're doing it all on prem.

• 23:00Craig S. Smith?

  Right. Can you talk then about how do you work with an ISV? I mean, all these guys are, unless they're Gen AI native, they're they're trying to figure out, you know, how to use generative AI in their products.

• 23:19Speaker 00

  Right.

• 23:19Craig S. Smith?

  These presumably are people that have already built products on AWS platforms.

• 23:27Craig S. Smith?

  Do they come to you? Do you reach out to them? If they come to you, is there a process that you go through with them? You know, a checklist or yeah, how do you, how does that work?

• 23:40Speaker 00

  Yeah, yeah, absolutely. So first of all, it it starts at the account team level. I just came back from a week and a half in in Australia, New Zealand, meeting with with customers. And I was talking to some of our account teams out there. And one of the things they mentioned to me was that every single AM that that's responsible for working with ISVs is currently going through the certification process to be an AI practitioner. So it's the same external certification that that are that our customers get. So so they're the ones that that have those discussions. When the customer has questions, then they bring in the subject matter experts that are aligned with the particular services. They also then will identify opportunities to help accelerate those.

• 24:26Speaker 00

  And so I already mentioned the generative AI Innovation Center. That's one way that we would take an opportunity and

• 24:34Speaker 00

  define it and then move to a proof of concept. There are other things that we do. There's a program that I actually co-run. It's called the Roadmap Acceleration Program. And we do workshops with an ISV where we'll say, let's take a particular use case that you think is high value. And we'll go through the five questions that we use internally at Amazon, which are part of our working backward process. From those five questions, we'll go ahead and draft a PR FAQ, Press Release, Frequently Asked Questions document. This is the way that we build our products. As part of that, then we also have reinforced that that standard working backward process with

• 25:16Speaker 00

  some specific questions around building generative AI functionality. Like as an example, why is generative AI the best way to meet this customer's need? Because it isn't all the time. And I think that that's one of the things that that the Forrester survey kind of showed that there was a little bit of hype here. And it's like, I need Gen AI. It's my solution. Let's go find the problem. What you really want to do is say, what's the problem? What's the toil? What's the dark data? What's the, what's the, the content? What's the area that we've never served before? And then is Gen AI the right solution? And what specifically in Gen AI? Is it generating content? Is it automating tasks?

• 25:58Speaker 00

  Is it, you know, being able to use natural language so that a larger user set can, can, can use the product? And then, you know, that, that question about data is critically important because one of the things that we see from a pricing perspective, if you want to drive business value as a software company, if you've got unique data that is hard to acquire, if you've got control of the customer through a user interface or a hero product, if you are

• 26:31Speaker 00

  Making a creative user or a user that's in short supply customer, if you're making them more effective, you're giving them leverage, those things are more valuable and allow you to to look at user-based pricing models instead of consumption-based models. They let you look at potentially outcome-based models. They let you charge a higher price premium. So a great example of that, one of my my favorite software companies out there is Canva. If you look at their hero product, there are just dozens of little places inside their product where they've implemented generative capabilities, which collectively speeds the whole user user experience for the creative.

• 27:16Speaker 00

  And the net result of that is that they've been able to steadily increase the what they charge for their product. Because the value that it provides is apparent.

• 27:27Craig S. Smith?

  Yeah, yeah, it's the pricing models. That's something I've seen are also changing. I know in customer service, the old model is you charge by headcount.

• 27:43Craig S. Smith?

  And with the integration of of generative chatbots and seamless handoff and all that stuff. These companies are starting to charge by outcome.

• 27:56Speaker 00

  Hundred percent.

• 27:57Craig S. Smith?

  Yeah, which is wonderful for the for their customers.

• 28:02Speaker 00

  Yeah.

• 28:02Craig S. Smith?

  Rather than paying a fixed headcount cost.

• 28:07Craig S. Smith?

  Can you talk about, you know, these as these companies are shifting to hopefully sustainable growth and profitable, sustainable growth. Are there steps that they should, is there a checklist that you go down? And and also, before we get to that, you mentioned a certification. What certification was that that you were talking about?

• 28:34Speaker 00

  It's the AWS Certified AI Professional.

• 28:38Craig S. Smith?

  I see. Is that for ISVs or is that for?

• 28:42Speaker 00

  It's general. It's for anybody. Um, you know, it's similar to what we have as our certified cloud practitioner, which, you know, basically every uh, Amazon employee when I joined a number of years ago that was customer facing, uh, was required to get, but it's the same thing that our, our customers, uh, uh, you know, take as well from a training perspective.

• 29:03Craig S. Smith?

  And, and this training is required or it's just, uh, the service that Amazon offers customers?

• 29:12Speaker 00

  No, it's a service that we offer our customers to help them get up to speed, to develop their own skills. But you know, it's, it's kind of like physician heal thyself. The best way to be able to advise, you know, customers is to make sure that all of our folks have that same level of expertise. And that's actually something that, uh, I, I, I've actually found really, um, really, um, um, invigorating here at, at Amazon that, that, you know, the expectation that, that, that, that the sellers are going to have the same level of skill and competency that we would normally expect out of software architects or technical, you know, uh, uh, sales support. Um, is one of the ways that we, that we help identify those opportunities so that we can progress them.

• 29:58Craig S. Smith?

  Yeah. You, you mentioned, uh, the generative AI innovation center. Can you explain a little more what that is?

• 30:06Speaker 00

  Yeah, that is an invest, a set of investment, uh, investments that we made at a corporate level to put together a set of specialists that are designed to accelerate the big ideas, uh, that our customers have. So it's not unique, uh, to ISV, but we have a number of ISVs that have taken full advantage of that. Um, I'll give you one example, uh, Deputy, which is a, uh, an up and coming ISV out of Australia that, uh, you know, our team has, has worked with extensively. In their case, the use case that they wanted to focus on was what I would call an optimization use case. They wanted to get more effective in how they built their own products, which are for small businesses. Workforce management, that sort of thing.

• 30:50Speaker 00

  And so, you know, the the Gen AI Innovation Center worked with them to really push hard on toil reduction for their development teams and their product teams. So code generation, but also collaboration within the product teams, how they prototype. And net net, they were able to to to get to a point where they think they can get a seventy percent improvement in their product team execution. That's pretty significant. From an external perspective, BMC is a is a company that that that worked with the Gen AI Innovation Center specifically around the price and performance of the models that they were working with and the use cases that they had developed. Because, you know, they wanted to get to this point where the cost of goods, the the

• 31:33Speaker 00

  COGS for delivering this inference was low enough that they could, you know, make money off of it. And so in that case, the the the the engagement with the Innovation Center was was really focused on performance and and and cost reduction. And in that case, I think they reduced the costs on an order of of forty percent or more, I think. So so, yeah, so, you know, those are just examples of of the types of engagements that happen at at that level. In my case, with, you know, the work that I do with with our individual AI ISVs, there's a kind of a stepwise process that that that we go through.

• 32:16Speaker 00

  We start with this idea of of the the high priority use case. In some cases, it's an internal use case. They want to increase and improve their sales effectiveness. I did one where churn reduction, identifying churn reduction early on in the renewal process was critically important because the customer had a churn issue with their existing product. And so we worked on that. In other cases, it's product embedded. So we want to, you know, focus on one of these toil reduction use cases. I I think of a case around.

• 32:51Speaker 00

  Environmental health and safety. And there's a lot of opportunity there through processing video as an example, where you can identify safety risks. Or where you can look at the result of an inspection and start to look at failure trends and identify, you know, what might need to be done to reduce the number of failed inspections around a particular class of asset. So you can start with that use case and then start to build out what has to be done to make that use case successful. And I think I've a couple of times I've mentioned the idea of accuracy and performance and cost. Those are kind of like the modern iron triangle for generative AI, because you have to start with that accuracy thing.

• 33:34Speaker 00

  How accurate do we need to be for this use case to be successful? That's, that's kind of, whatever level that is, is kind of like non-negotiable. And once you've got that accuracy goal, then you can say, okay, well, how fast does it need to be? And how much are we willing to pay to hit the performance and accuracy targets that we need to hit? What you may find at that state is you simply can't do it cost effectively today because the inference is too expensive or the models take too long because we have to use a reasoning model to get to the level of accuracy that we want. And if that, if that, if you, if that's what happens and you, you put it aside and say, okay, we can't do this today. But, you know, in the last two and a half years, we've already seen the inference costs come down something like on the order of 90%.

• 34:16Speaker 00

  So while we can't do it today, maybe we can do it 12 months from now so we can revisit at that point. But if you pass that, that accuracy, performance, cost triangle, then you can move on to the next step. And, you know, in that, that product mapping example I used earlier, they set a goal of 90% accuracy and it took them three different tries with different models and different approaches to using that model to hit that accuracy target. But once they did, then they could see what their cost was and, and say, can we, can we make this work in our product? And if you're embedding it into your hero product, which they were, you said, well, the cost is relatively minor because this only happens every so often. And so we'll just put

• 35:01Speaker 00

  it in the cost of what we do and it'll be an additional differentiation and we'll, you know, we'll push it to production. You do have to ask that that pricing power question. That's really step four and and figure out the right pricing model. And and they're probably the best work that I've seen on figuring out the right pricing model is is what BCG has done. Guys by the name of John Pineda and Jacob Konikoff. What they have have observed is that if you ask questions like, what's the unique data that we bring to the use case? And how defensible is that? If we can expose that value in our existing hero products

• 35:45Speaker 00

  instead of creating new separate standalone products like co-pilots or that sort of thing, which we then go have to go out and have to sell as new products or additional products? If you can create use cases that that create customer leverage as opposed to cost reduction, you can generally charge a higher price, a more premium price. So when I work with with with our our software company customers, I'm always asking these questions because that helps us get to the point where we say, yeah, we can do this and we can use it to drive profitable growth. So it's not rocket science, you know, but but it but it but it works.

• 36:26Craig S. Smith?

  Yeah, this, all of this, the generative AI innovation center, is that available to all customers or even prospective customers? Or do you have to have a certain level of billing at AWS?

• 36:43Speaker 00

  Yeah, yeah. You do need to work through an account manager to start that process and they'll work with the request. And, you know, obviously, there's finite capacity. So we we prioritize requests based on on on the opportunity. If it's a huge, big idea, the differentiation is apparent, the opportunity is is massive, you know, they're going to work with with that that software company.

• 37:11Craig S. Smith?

  Yeah, yeah. What is, or maybe you covered it and just didn't name it. What's the roadmap acceleration program?

• 37:20Speaker 00

  Yeah, that is an ISV unique program that we run. And there, the only requirement is that the product decision maker, CPO, VP of Product is willing to spend the time to go through the use cases with us. So very low bar of entry. You know, we start with that, that two to three hour intake process. And then we go through the PR fact writing process. We can, we can involve them in that. That usually is something that's a combination of email and meetings. So maybe another hour or two. And then from there, we go into a couple day POC process. So as an example, you know, in Australia,

• 38:02Speaker 00

  we use the Melbourne Builders Lab, which is, you know, we also have one of those in New York. We bring their developers together with our prototyping teams and we use that PR fact to start to, to drive the idea and we see how far we can get. We test the validity. So that's an example of a, of a, of a program that's kind of in between something like the Gen AI Innovation Center and something that's, that's more tailored around an individual customer, like an immersion day that, that is, that, that the Bedrock team would come in and spend time or, or a subject matter expert. So we really try to meet the customer where they want to, where they want to meet us.

• 38:38Craig S. Smith?

  Yeah, ISVs, when talking about deploying general AI infrastructure over the next four to 12 months, or I shouldn't say ISVs, maybe it's all respondents to the Forrester survey that 46% mentioned leveraging cloud-based Gen AI platforms and only 30% expressed interest in, in building infrastructure, Gen AI infrastructure. That is.

• 39:11Craig S. Smith?

  Inference, that's basically inference, right? You're talking about leveraging API inference.

• 39:20Speaker 00

  Well, there's also training as well. So first of all, that that stat is all software, it's software company only. The the survey that the Forrester folks did only looked at companies that said, our business model is building software products for businesses, so ISVs.

• 39:37Speaker 00

  So, so, you know, we have a lot of software company customers that choose to train their own models on services like SageMaker. You know, inference is is obviously where they make the money after they've done that. So, you know, that's, that's probably the the the primary focus. And I think a lot of it has to do with just the desire to to keep up. I mean, I think of, what was it, second or third week of January, where in the course of a couple of days, we had DeepSeek and then we had Quen and the entire financial market evaporated for a couple of days. And then by the end of that week, we had DeepSeek support in inside Bedrock. And, you know, the other cloud vendors, you know, had very rapid uptake, too.

• 40:23Speaker 00

  That that matters because, you know, this this idea of I've got a model and now I'm done and I can move forward with it. You know, that that's, that's not reality. You know, you see the software companies already that are pushing hard on Claude 3.7 as an example. And and we're talking about end of lifing, I think Claude 3.0 and it's only what, a year old? You know, that that level of speed is just

• 40:52Speaker 00

  is hard, especially, you know, when, you know, it can take a little bit of wow, a little while when you're a software company to validate a new model and assess how well it's going to work for you. And so with the rate of speed of all these models coming out, model evaluation has become one of those factors that can really accelerate product strategy. So things like, you know, prompt evaluation or or prompt optimization, the ability to, you know, use FMEVal and SageMaker to very, very quickly test new models is one of those things that that ISVs value highly because

• 41:32Speaker 00

  You know, a couple percentage point reduction in your COGS or a couple percentage points improvement in your accuracy could be the difference between we can't make this use case work now and now we've crossed the barrier. So let's go ahead and implement it because we know we can do it profitably.

• 41:49Craig S. Smith?

  Yeah, yeah. And when I was saying only thirty, thirty percent expressed interest in building gen AI infrastructure. I assume that meant taking an open source model and training it. What does that mean?

• 42:05Speaker 00

  No, I think that means on their own premises, you know, in their own environments, in their own data centers.

• 42:10Craig S. Smith?

  Right, I see. Yeah, yeah. But in terms of fine tuning existing models. I mean, you mentioned deep seeks available on Bedrock now, right?

• 42:22Speaker 00

  Yep.

• 42:23Craig S. Smith?

  If someone wants to use deep seek and they want to

• 42:31Craig S. Smith?

  fine tune it, do they do all of that on Bedrock? And do you guys offer support for that?

• 42:38Speaker 00

  Yeah, on the models that we do support, fine tuning for some models. I'm not sure if we actually support deep seek fine tuning or not. I'd have to, I'd have to look at that and give you the update on that. But yes, where we can, we support that, that, that capability. And actually, that's one of the things that, that software companies are often interested in doing because they do have proprietary domains, proprietary data, you know, domain specific terminology. And that, that fine tuning can yield a few extra points of accuracy and it's worth the additional cost to be able to support that. You know, one of the, the issues with, with, with tuning is that you then have a, a model that is, is somewhat unique.

• 43:22Speaker 00

  And so you need to be able to support that. And so, you know, often that'll get tied to provision throughput in Bedrock to be able to, to execute that. So you've just got to make sure that, that, that, that, that accuracy, performance, price triangle is, is reflected when you're, when you're doing that evaluation.

• 43:41Craig S. Smith?

  Yeah. From where you sit, you can see enterprises and ISVs in particular, in this case, adopting AI, as I said at the beginning, I know there is all kinds of experimentation and pilot programs, but there have not been. I mean, the the diffusion through the product space, there are a lot of new products, but has not been as quick as one would expect. I mean, everybody now has a little chatbot on their on their product, but it the products themselves haven't necessarily evolved. Do you, where do you see

• 44:24Craig S. Smith?

  that process? Do you think that things were changing and still are changing so fast that a lot of people started experimenting and then just backed off and said, well, let's wait and see how this settles out before I dive in again?

• 44:40Speaker 00

  Um, I actually think it's a little bit of a different challenge. Uh, and and I think there was some interesting data in the Forrester survey when we looked at some of the issues around data challenges, because as we see data as one of those things that that really dramatically improves accuracy. I think some of these early experiments um, exposed some of the the the challenges uh in their existing infrastructure. So for example, among the the respondents that had implemented at least one um, a generative AI use case or or product. Silo data came back as a real challenge uh to uh to their efforts.

• 45:21Speaker 00

  Now, when you look at the data from those that had done more than that and were expanding uh the the use cases that they were building. So they gotten kind of they they they, you know, had had had to are a little bit more aggressive. Silo data wasn't as big a challenge, but um integrating with existing systems then became a a challenge. And so I think part of the reason that you're seeing all this focus right now on agentic frameworks and uh agentic technology is because of that latter issue. So, you know, you've got to solve these challenges before you can take the next steps at unlocking uh the value uh of of of toil reduction. You know, you don't get toil

• 46:03Speaker 00

  reduced without having tools that agents can use to actually act on the user's behalf. Uh, and so, you know, that that that blocking and tackling has to be done uh to unlock that next level of value.

• 46:18Craig S. Smith?

  Yeah, yeah, and it's being done. I mean, they're they're sure. Yeah, there's I mean, the AWS is is where are you guys in in agents and agentic workflows?

• 46:33Speaker 00

  Yeah, so we've got Bedrock Agents, which has support for multi-agentic communication. Boy, it's hard to it's hard to believe that that, you know, MCP Model Contact Protocol has only been around since November. It hasn't even been six months, right?

• 46:50Craig S. Smith?

  Yeah, yeah, no one, no one really was paying attention to it until you know, January or so. It took a little while for people to understand what it was. Yeah.

• 47:02Speaker 00

  Yeah, you know, when I started paying attention to it, when I saw the the the the Groll team on on on the Java VM implementing an MCP server and it's like, OK, so now we're opening this up to Java programs. And when you know, you started to see MCP servers for for databases. And it's like, OK, this is, you know, I know people describe it as USB C for AI. My my kind of early description was it's kind of like ODBC for for for generative AI. And getting access to the data and being able to use it is is is critical. And so, you know, when I started to see, you know, GitHub repositories that showed how Nova was integrating with MCP, it was like, OK, you know, this is this is becoming

• 47:47Speaker 00

  what what, you know, when I was an analyst, we would have called a de facto standard. But I think we're getting there very, very quickly. There's still more work to be done. And and, you know, that's actually one of the concerns that I have. It feels like that there are so many agentic frameworks out there. There's a risk of what I would call islands of automation. So I've got an agent that's working in Agent Force. And I've got an agent that's working, you know, that maybe is is part of Boomi or HubSpot. How do I start to put these things together? It seems like as an industry, we have to do a little bit more work around making sure that that these agents can work with each other regardless of where they might be deployed.

• 48:28Craig S. Smith?

  Yeah, yeah, yeah, that's that's fascinating. And I guess that's one of the challenges is which of those frameworks do you commit to, right?

• 48:39Speaker 00

  Yeah, I think the big, you know, the big thing from our perspective is is is

• 48:47Speaker 00

  Making the the the the calculation that there's probably not going to be a single framework that an enterprise is going to use. They will be you. They will end up with multiples. And federation is probably an approach that that benefits everyone. And so, you know, I think that's that's, you know, the the strategy that that that makes a lot of sense going forward, because otherwise, I don't see how you you you rein in the fragmentation that is already out there.

• 49:17Craig S. Smith?

  Yeah. And and the the link between the federated states would be M P M C P? Or or is something?

• 49:29Speaker 00

  I think there's I think there's a part of of of certainly a part that it plays. I think what Google's done with A two A agent to agent is is pretty interesting as well. Certainly, some, you know, there's certainly a lot of of of opportunity for innovation here still.

• 49:50Craig S. Smith?

  Yeah, yeah. So have I? Is there anything that that we haven't talked about that you're doing that you'd like listeners to hear about?

• 50:02Speaker 00

  Uh, no, it's amazing. We've we've done what we time has gone by so fast. I wouldn't be respectful of your time. I guess maybe what I would just say is is is

• 50:17Speaker 00

  this this um, this disruption, I think, at least in my opinion, again, I've been doing this from for more than 30 years, is potentially the biggest disruption to how humans work with computers that that I've seen. I mean, if I think about the the last 50 years of history from the mainframe on, we've always had to expect the the user to respond and adapt to the system, you know, because we threw up a form, or we we give them a command line, and they've got to know what to poke the system with to get it to do something. And you know, to me, the the power of what's happening right now is is the evolution of intentional

• 51:02Speaker 00

  interfaces that I express what I want the system to do for me with the expectation that the system is going to respond and adapt to me. And you already see some some beginnings of the power of that, like one of the things that Drava reported when they when they started to implement code generation co-pilots and and you know, we're we're automating documentation. Is it reduce the onboarding time for new developers by 30%?

• 51:28Craig S. Smith?

  Yeah.

• 51:28Speaker 00

  That's pretty cool. I was working with a company in the healthcare space that was talking about their own own onboarding experience. And one of the things that they said was, instead of having to throw up a bunch of forms that the user has to fill out, we can ask the the new customer, what do you want to accomplish? Do you want to run a marathon? Yeah, I want to run a marathon. I want to lose 10 pounds. I want to keep my my, you know, my my lipids under control. And then the system can react with personalized capabilities based on what the customer wants.

• 51:59Craig S. Smith?

  Yeah.

• 51:59Speaker 00

  It makes me so excited for for what what we can do with this technology, especially as we continue to drive the cost down to make these sorts of use cases possible.

• 52:08Craig S. Smith?

  That's right.

• 52:10Craig S. Smith?

  Yeah, it's an incredibly exciting time. Okay, I'm gonna end it there. There's a growing expense eating into your company's profits. It's your cloud computing bill. You may have gotten the deal to start, but now the spend is sky high and increasing every year. What if you could cut your cloud bill in half and improve performance at the same time? Well, if you act by May 31st, Oracle Cloud Infrastructure can help you do just that. OCI is the next generation cloud designed for every workload, where you can run any application, including any AI projects, faster and more securely for less.

• 52:53Craig S. Smith?

  In fact, Oracle has a special promotion where you can cut your cloud bill in half when you switch to OCI. The savings are real. On average, OCI costs 50% less for

• 53:06Craig S. Smith?

  Compute 70% less for storage and 80% less for networking. Join Skydance Animation and today's innovative AI tech companies who upgraded to OCI and save. Offer only for new US customers with a minimum financial commitment. See if you qualify for half off at oracle.com/eyeonai. That's oracle.com/eyeonai. E Y E O M A I, all run together. Oracle.com/eyeonai.