#308 Christopher Bergey: How Arm Enables AI to Run Directly on Devices

Try OCI for free at http://oracle.com/eyeonai This episode is sponsored by Oracle. OCI is the next-generation cloud designed for every workload – where you can run any application, including any AI projects, faster and more securely for less. On average, OCI costs 50% less for compute, 70% less for storage, and 80% less for networking. Join Modal, Skydance Animation, and today's innovative AI tech companies who upgraded to OCI…and saved. Why is AI moving from the cloud to our devices, and what makes on device intelligence finally practical at scale? In this episode of Eye on AI, host Craig Smith speaks with Christopher Bergey, Executive Vice President of Arm's Edge AI Business Unit, about how edge AI is reshaping computing across smartphones, PCs, wearables, cars, and everyday devices. We explore how Arm v9 enables AI inference at the edge, why heterogeneous computing across CPUs, GPUs, and NPUs matters, and how developers can balance performance, power, memory, and latency. Learn why memory bandwidth has become the biggest bottleneck for AI, how Arm approaches scalable matrix extensions, and what trade offs exist between accelerators and traditional CPU based AI workloads. You will also hear real world examples of edge AI in action, from smart cameras and hearing aids to XR devices, robotics, and in car systems. The conversation looks ahead to a future where intelligence is embedded into everything you use, where AI becomes the default interface, and why reliable, low latency, on device AI is essential for creating experiences users actually trust. Stay Updated: Craig Smith on X: https://x.com/craigss Eye on A. I. on X: https://x.com/EyeOn_AI

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The ARM architecture has existed now for almost 30 years. And that started from early investments from companies like Apple, early adopters of the ARM architecture that made it the stalwart that it is. Where companies like Nintendo, companies like Nokia back in the 80s and 90s as these smartphone revolution, all that kind of stuff really started around ARM. And that's what's driven us to be where we are today. We have big CPUs and little CPUs. And we're actually moving the workloads back and forth because certain times you need the performance, certain times you don't. And so that's really the way these devices work. Where to your doable example, you're looking for maybe some motion, you're looking for something, and then once you trigger that event, okay, now let's fire up some more computing elements.

• 0:43Craig S. Smith?

  In business, they say you can have better, cheaper, or faster. But you only get to pick two. What if you could have all three at the same time? That's exactly what Cohere, Thomson Reuters, and Specialized Bikes have. Since they upgraded to the next generation of the cloud, Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs. Where you can run any workload in a high availability, consistently high performance environment, and spend less than you would with other clouds.

• 1:27Craig S. Smith?

  How is it faster? OCI's block storage gives you more operations per second. Cheaper? OCI costs up to 50% less for compute, 70% less for storage, and 80% less for networking. Better? In test after test, OCI customers report lower latency and higher bandwidth versus other clouds. This is a cloud built for AI and all your biggest workloads. Right now, with zero commitment, try OCI for free.

• 2:08Craig S. Smith?

  Head to oracle.com/eyeonai. Eye on AI all run together E Y E O N A I. That's oracle.com/eyeonai.

• 2:23Speaker 00

  So it's great to be here, Craig. Thanks for inviting me. So my name is Chris Bergy. I'm the Senior Vice President, General Manager of the Client Line of Business at ARM. And that means I very much focus on all of the ridge, rich edge devices that ARM is so prominent in. So things like smartphones, but also we are obviously making quite a bit of inroads into things like PCs. We participate all parts of your house, whether that's your TVs, your smart speakers, all kind of rich endpoints that that you have powered and and are quickly becoming AI-enabled. And I think that's what we're going to talk about today, Craig. Just a little bit about my background.

• 3:03Speaker 00

  I've spent almost 30 years in semiconductors, various big companies, started out of school at AMD and and moved to Broadcom for almost a decade and also did some startups in between. So along, I can't believe it's gone by this this quickly, but I guess, you know, semiconductors have never been this cool as it seems like governments and everyone really cares about semiconductors. So I guess I was very fortunate in my career choice.

• 3:30Craig S. Smith?

  Yeah, that's kind of funny, isn't it? How things happen that way. Yeah, and it's in particularly semiconductors on the edge. That's like the where everything's at right now. So ARM and we're talking about chips for AI inference. So ARM's ARM version nine edge AI platform has a couple of new.

• 4:00Craig S. Smith?

  Processors designed to enable complex AI models.

• 4:07Craig S. Smith?

  Can you talk about those or is there something even more recent that you want to talk about?

• 4:13Speaker 00

  Well, I think that's a good starting point, Craig, and you know, and we've actually been on quite a journey. You know, in the edge evolution, not just actually at the, I'm sorry, in the AI evolution, not just at the edge, but also we're actually a big part of the infrastructure build out as well and data centers that, you know, really leverages a lot of our business model and capabilities. And so we're doing quite well there as well. But, but I'd like to spoke probably most of mostly about the edge today. And so we, we did create, we did a Lumex, our Lumex platform, which is really around kind of high end smartphones, high end multimedia experiences at the edge.

• 4:54Speaker 00

  We launched that back in September and already there's several products, chipsets that have been launched and now phones that are being launched by different leading phone manufacturers based on Lumex. And what's cool about it is it is the first, it's, it's addition to the platforms of V9, as you mentioned, that starts rolling out SME, which is the matrix extensions to the V9 architecture. And, and starts pulling that into the ecosystem to allow people to

• 5:31Speaker 00

  really take advantage of AI at the edge in these devices and do so with the traditional CPU programming model. So obviously there's a lot of discussions around accelerators, accelerators are great and, and are quite dominant in data center. And are also finding their home in the edge. But it's, it is always this balance of, you know, accelerators have gate great metrics around maybe tops per watt. But they definitely are a bit more challenging to program and versus let's say a CPU. And so you really got to get, the reality is we, we talk about heterogeneous computing because I think the answer is.

• 6:14Speaker 00

  Everything, right? I think, you know, in the AI world, it's about CPU capabilities. It's about GPU capabilities. It may be about dedicated accelerator capabilities. And then it's about memory bandwidth, you know, quite frankly, and really AI is putting stress on that whole system.

• 6:32Craig S. Smith?

  Okay, and let's back up a little bit for listeners that are not deep in the chip space. Uh, uh, and I'd like you to talk a little bit about V9 and what that is. Uh, but when, when you talk about heterogeneous computing, what you're and, and, and the programming language.

• 6:56Craig S. Smith?

  Nvidia, one of the reasons it has such a strong position is very early on, it developed a very user-friendly programming language called CUDA. And, and everyone has adopted that for programming AI on to GPU processors. And that has become the standard. And so a lot of new chips are arriving, but you need to learn a new programming language. Uh, and that's a barrier, but you, in this heterogeneous, particularly at the edge. Uh, what I understand is that you still use a, a

• 7:41Craig S. Smith?

  traditional processor with a programming language that you're familiar with, and then there's some sort of a conversion that sends some of those workloads to the edge uh, chip, uh, and it, it could all be packaged together, but is, is that right?

• 7:59Speaker 00

  Yeah, I think so. You, you, you, you got a lot wrapped up into there. So let, let me just first start with V9. So, um, the ARM architecture, um, has existed now for almost 30 years. Um, and that started, um, from early investments from companies like Apple. Um, early, uh, early adopters of the ARM architecture that made it, um, the, the stalwart that it is where companies like Nintendo. companies like Nokia back in the you know eighties and nineties is as kind of the these like you know the smartphone revolution all that kind of stuff really started around around arm and then that's what's driven us to be to be where we are today. So V nine is is is an architecture that we launched about five or six years ago now.

• 8:44Speaker 00

  Which was the next generation of architecture. Which was focused on a couple of different things. One was security. Another was performance and third was AI and you know we are very much at the forefront of looking at what these next generation systems are going to require. And so that's really what V nine is about. And at this point in time. You know, a large percentage of both iOS and Android handsets ship with V nine. CPUs and you'll see that continue to accelerate. Over the next couple of years and so and now V nine also is getting perforated across.

• 9:23Speaker 00

  The other markets that that arm participates in whether that's data center, whether that's automotive and then kind of AI IOT as well. So that's that's that's V nine.

• 9:36Speaker 00

  So let's talk about the the programming comment that you made there. So you're right. Cuda is an amazing language. I've actually had the pleasure of working very closely with Ian Buck, who is the I guess, you know, who basically was the Stanford student that really kind of started this and is obviously now, you know, has been a very important part of the Nvidia story and where they're at today and still still is there today. You know, but and and it is a great program for, you know, basically programming GPUs, especially taking care, taking advantage of some of the capabilities in the accelerators there. But you know, it is a accelerator language, right, versus a CPU language and

• 10:21Speaker 00

  you know, what people do is that, you know, if you basically have to start making things like driver calls and you have to move your workload off of the CPU to that. Now, it makes a ton of sense if you're going to get You know, I think, you know, Ian and I actually used to work on on quite a bit of high performance, HPC stuff for high performance computing, big government labs. And we would have these kind of rules where it would almost want to have a hundred X uplift when you kind of move that workload over. Maybe at least you'd have to have 10x, but you'd really want to have that versus the convenience and and and keeping it on the CPU because you really need to load that and then you stream that workload on an accelerator. And so that's kind of as you think about this heterogeneity of you're really moving

• 11:04Speaker 00

  these workloads around based upon, you know, what's required. Is it latency? Is it, you know, is it high performance? Is it lower power? All those kinds of things. And so this is, you know, what I would say the ecosystem is doing, what ARM is focused on is we're really focused on making all of those pieces, the CPU be as performant as possible on AI workloads and be super developer friendly, which it is. Also moving some of those workloads to GPUs. ARM is actually, many people don't know this, but ARM is actually the highest volume GPU out there. We've shipped over nine billion GPU cores if you, because of the mobile handsets and so much in the mobile industry that's based on ARM GPUs.

• 11:46Speaker 00

  And then also in support of accelerators through ARM's extension of our architecture and some of those accelerators hang off of things like Chai buses or that are kind of part of the ARM architecture.

• 12:00Craig S. Smith?

  Yeah. And and when you say ARM ships GPUs, where does the NPU sit in, in fit into all that?

• 12:12Speaker 00

  Yeah. So so in a mobile handset, let's talk about that since that's when we started. You know, there are, you know, traditionally there was two large computing elements or, you know, there was the CPU and the GPU, right? And their main function CPU was to run the OS and then eventually apps. The GPU function was to display and play games and do all those kinds of things. Well, as these other workloads become important, whether it was, you know, camera imaging is obviously become super important and the amount of video that you take. Well, then we start putting little accelerators that are maybe doing some of the compression for the different video codecs and and those kinds of things.

• 12:56Speaker 00

  And so MPUs have kind of evolved. As a way to efficiently do matrix multiplications or CNNs and different kinds of models in as an accelerator. And so the system can choose to send that. But what really happens in the real world, and this is the heterogeneous, is you actually end up kicking off the job usually on the CPU. You may actually run it to the GPU, you may send it to the MPU, and then it comes back to the CPU usually to kind of conclude. And so that's tends to be how these workloads actually work in a system. It's all obviously not seen by the user. But it is the developer has to make some of those decisions as a software developers thinking about,

• 13:41Speaker 00

  you know, one, what kind of capability is in the handset? They're also making the decision of do they want to do it on the cloud or the edge? And we should have that conversation as well today. So these workloads move around and there's a whole set of reasons that they move around. But we're making the the CPU at the edge be as AI friendly, as AI performant and power with with a good power envelope as possible.

• 14:05Craig S. Smith?

  Yeah. And are these all packaged together, the CPU, GPU and NPU? If you're using the NPU? Or are they?

• 14:15Speaker 00

  It depends on the system. In today's cellphone chips, yes, they are largely all in a single SOC. So all these computing elements exist in SOC. As you get into, you know, things that have, let's say, larger, you know, larger battery windows, you know, in PCs. Many of the NPUs are integrated in the SOC. We are seeing trends around people adding accelerators. Obviously, people are also using the GPU that's in a in a in a laptop or something like that. And again, that may be integrated, it may be discreet. I would say there's a trend towards integration in many of these markets.

• 14:57Speaker 00

  And that is because of the memory pressure and the memory size requirements for for AI. And so I think if you look at some of the latest You know, I'm going to say, put it on your desk, kind of computing platforms. We're especially proud of our partnership with Nvidia on the GB 10, the product they've actually just started shipping last week. It's puts, you know, I think Jensen likes to say it puts puts a supercomputer on your desk, right? Where they're actually providing, I think it's a petaflop of AI performance on your desk. And it is because of, you know, you've got these ARM CPUs, again, you know, 20 of them coupled tightly to this accelerator

• 15:42Speaker 00

  with a single memory system that offers up to 128 gigabytes of DRAM and very high bandwidths. And so you've got this whole system that you can put together or that developers can use. And and we're seeing this as a trend, whether you look at, you know, the latest V9 M5 that Apple just announced this week as well. Or I think it was last week. Again, you see amazing AI performance, leveraging the ARM CPUs, GPUs and other accelerators, and then a tremendous memory system for that. So that's really what we're seeing happening and those are integrated. But some of them, as they get a little bit bigger, they get discrete.

• 16:22Craig S. Smith?

  Yeah.

• 16:22Speaker 00

  But the problem with discrete is then you have to split the memory system and it gets it.

• 16:27Craig S. Smith?

  Yeah.

• 16:27Speaker 00

  AI is so memory heavy that it just, it's a, it's a whole balancing act.

• 16:32Craig S. Smith?

  Yeah. And again, a mindful of listeners who are not deep in the chip space. So see a system on a chip and that's where you you combine different kinds of chips into one. From the consumer's point of view, it's one looks like one chip, right? It's all compressed in there with a cover. And the, the reason this is important is that increasingly, I mean, right now, I have an app that I built for myself. When I drive around, it talks to me about the history of the places I'm in.

• 17:13Craig S. Smith?

  And and it the voice drops out a lot of time because it's got to, you know, read my GPS and send the data to the cloud and then, you know, do a lookup in in the model or whatever model I'm using. You know, get the the text from the model, convert it to speech, send it back to the the phone. Maybe the the text of speech is on the phone. I'm not sure, but but that whole chain gets there's a million ways that it can get interrupted.

• 17:54Craig S. Smith?

  And if you have that all happening on device, you don't have those problems with connectivity and and things like that. Can you talk about how you see,

• 18:08Craig S. Smith?

  you know, V nine or or these these SoCs changing the way we we interact with AI?

• 18:17Speaker 00

  Yeah, absolutely, Craig. And and you're you're at the forefront here. It,

• 18:24Speaker 00

  you know, it's it's it's my job's super fun because I get to talk to many of the industry leaders and and visionaries, whether that's both on, you know, companies like Google that we work so closely with and all the chipset companies and and then all the the CE companies, the consumer electronics companies that are actually putting these things in your hand. And, you know, it's it's it's fun because, you know, I always I'm an I'm an edge guy. I think, you know, I want edge computing. That's kind of that's the business I run. And so, you know, I, you know, of course, I want edge computing to happen. And so, but I I do often go to these partners and say, hey, you know, okay, why can't you run it in the cloud? You know, why? Why can't we do it? You know, it seems to work just fine. People love chat

• 19:10Speaker 00

  GPT. They love Gemini. You know, they're starting to really utilize these services. And, you know, they they basically said, you know, what what gives me reassurance is all these companies are like, no, no, we need to put these in devices. And and and one of the reasons is what you just said, Craig, which is You know, we, the what, the goal of AI is that, you know, we're seeing some of the early cool use cases, real time translation, um, you know, some, some smart, some agents that are starting to do some things, but we're, we're in the early, early innings here. And, you know, one of the analogies that I like to use is, um, touch. And if you take a child, let's say, less than 10 years

• 19:55Speaker 00

  old, and you give them a screen, they just start touching it because they don't know anything that's not a touchscreen. Um, and, and, you know, whereas, you know, you and I, you know, like, the mouse was a big thing. Once, once we, you know, we thought that was cool. Um, but, and I, and I reuse that analogy because that is how AI is going to be. If something doesn't have AI and you can't interact with it and it can't start figuring out what you're trying to do, it's going to be that, like that child that says, this thing's not going to have a touchscreen. I don't know how to use it. I don't want to use it. Right? And so, so first off, you need to just believe that that is, that is how essential this is going to be. It's, it's the, you know,

• 20:36Speaker 00

  it's how annoying it is when it takes a long time for the app to boot up. And it's how annoying it is if you have an app die, or, you know, those kinds of things that we, we work out the edges and they don't really happen as much anymore, but as, as technology is new. And so, it's the same thing with AI where, you know, in talking to one of these partners, you know, they said, hey, Chris, we, you know, okay, yes, maybe the AI in the cloud works great, you know, 90% of the time. But, you know, you're driving up 101, I'm, I'm here in, in Silicon Valley in San Jose, you're driving up the highway, there's a dead spot. And, you know, and basically, you're going to get a bunch of latency that, you know, it's not going to be

• 21:19Speaker 00

  conversational, you're going to be waiting. Um, and the reality is, is that people aren't going to complain about to their carrier saying, hey, you got this dead spot on 101, when are you going to fix it? They're going to say, hey, Craig's app, I don't like that experience. It's frustrating, it doesn't work, you know, three times a day, I'm trying to use it, you know, it's, it's like doing these video calls, right? When it doesn't take long for somebody to have a shaky connection, you're kind of like, hey, let's just talk next week, or let's talk in the office, right? It's just too frustrating. That's just one example. There's privacy, there's performance, there's all kinds of other things that is, that really is going to drive this to the edge.

• 22:01Speaker 00

  The, the counter side is the models have to get smaller. There is a real cost because of the computing and the memory computing pressure it puts on it. And so it's a balance, but it is happening. And again, training and there'll still be quite a bit of inference that'll happen in the cloud, but as much as we can move to the edge, I think it's pretty unilateral. Whether you're a hyperscaler, whether you're an app developer, whether you're a device manufacturer, all are quite incentivized to try to make it one very, very well on the edge, in the device.

• 22:37Craig S. Smith?

  But there are challenges and there are trade offs, as you said. One is that there's a power issue and a heat issue is a result of that. How do you manage that? Because, and that was, I mentioned this company, Brainship, I had a conversation with that's using neuromorphic chips that only fire when the, when there's enough activity to wake them up.

• 23:11Craig S. Smith?

  So for example, in a, in a doorbell camera, you don't want the doorbell absorbing the, the video, sending it to the cloud, computing and sending it back when, when there's nothing happening in the scene, but that's in fact what happens. How do you manage that?

• 23:35Speaker 00

  Yeah, so I, I think that, I mean, we, we, we have these management techniques that get used in, you know, in almost every aspect of a, a computing device, right? I mean, we get, we have very clever engineers who figure out how to kind of fool you that the device is running full power, yet we're, we're obviously very aggressively changing things in the background. And that, that was actually one of the ARM innovations, over 10 years ago, we came up With this big little concept around, we have big CPUs and little CPUs and we're actually moving the workloads back and forth because certain times you need the performance, certain times you don't. And so that's really the way these devices work where you, to your adorable example,

• 24:17Speaker 00

  you're looking for maybe some motion, you're looking for something and then once you trigger that event, okay, now let's fire up some more computing elements. Let's go figure out. Oh, it's a face. Okay. Is that a face? We know. If we don't, let's, let's trigger the cloud or whatever. And that's the way these systems work. So, you know, I think it is about really these more intelligent computing platforms and getting smarter on how you do these accelerators. And that's, you know, that's for example, our, our, the SME I mentioned, that's part of this SME two, that's part of V9. It is this tightly coupled accelerator, but we do it in a very, very efficient manner.

• 24:56Speaker 00

  Where it's actually shared across multiple CPU cores. So you're also being very area efficient, which really drives cost. So, I mean, those are things, but the other thing is there's tons of innovation. I, again, don't want to get too technical, but your, your, your listeners maybe have heard about HBM, right? HBM memory is stacked DRAM with a very high bandwidth interface. And it's become a huge enabler for AI in the cloud, right? Because we have to have all this memory bandwidth and we put these HBM stacks next to these accelerators. We're looking at very similar things at the edge relative to, you know, if, where, where is the power, right?

• 25:41Speaker 00

  Is it in the computing element? Is it in the joules per bit of memory transfer? All of these things are opportunities for innovation and there's a ton of research and a ton of investments going into them. So I, I don't worry. About the power, I actually think the power is fairly manageable relative to inference. I think the bigger pressure we have right now is actually memory, memory size. So the model size and and how how big it, how do we make it small enough? Because at the end of the day, that drives, it drives costs, it drives power and other kinds of things. So it's happening and the good news is we're in this innovation cycle where,

• 26:24Speaker 00

  you know, there's two things you can do, you can shrink the model and get the same performance. And and that seems to be shrinking, you know, almost, you know, 50% a year, if not faster than that. And on the other side, you can say, hey, I want a three gig, you know, gig, three billion parameter model. And it's just getting more and more intelligent every, every six months, every year, right? And so there's different ways to solve the problem, but this is, that's really the enabler, I think.

• 26:51Craig S. Smith?

  Yeah, and again, I'm just thinking about listeners so that so we don't lose them. SME is scalable matrix extension, and that's a way to optimize the the math operation.

• 27:06Speaker 00

  Correct.

• 27:07Craig S. Smith?

  That's used. And are you, you know, on the models, are you guys exploring state space models that optimize memory usage? I mean, that have a different way of handling memory?

• 27:28Speaker 00

  Um, so I'm not as familiar personally with state space. I think I understand the general concept of it. I would say that, you know, we provide this platform that many of these innovations sit on top. So, you know, many of the, what we've basically done with, for example, our our matrix extension engine that you just talked about, we basically built what we call CLEADY framework on top of it, which is these libraries that now developers can basically leverage. So it just does the right thing in the hardware relative to taking advantage of which, you know, You have the latest RMV nine, et cetera. So a lot of those how the model works and states fates and what,

• 28:12Speaker 00

  you know, are they using a KB cache? Or how are they doing the different updates? Those actually sit at a little bit higher level in the stack where we're kind of more of a, I guess, a plumbing enabler is kind of the way that I would say it. So we, we support all of that and we're making it super easy for developers to, to explore that and to build their innovations on top. But there's nothing unique, at least at this point in time, that we're necessarily doing from a state state point of view in our CPUs.

• 28:42Craig S. Smith?

  And can you talk us about some of the, the current applications?

• 28:49Craig S. Smith?

  And where you see this going, this move to the edge, I mean, obviously,

• 28:57Craig S. Smith?

  you know, self-driving cars is, is one where you, you can't afford to risk connectivity and latency by sending data to the cloud. It's got to be computed on, on the platform or on the, in the car. What are some of the other applications that you're currently, putting chips into and or current devices, I should say. And, and where do you see that going? I mean, how, another, the form factor of these chips, how small can they be? So that they're, they're sitting, I mean, I was talking to a guy, the, I wear hearing aids.

• 29:40Craig S. Smith?

  That, you know, you'll be able to have these chips in the hearing aid.

• 29:46Craig S. Smith?

  You know, filtering or, or isolating sound and that sort of thing. So can you talk about that a little bit?

• 29:54Speaker 00

  Yeah, I mean, that's a great question. I, I mean, I think the scalability is almost everywhere, right? I mean, your, your hearing aid example is a, a good example. And, and this is one of the reasons why ARM builds the, the huge set of portfolio that we do, because we are in many of those, those hearing aids and those kinds of things. And we've, we've actually announced AI across, I mentioned Lumix, but we have our Edge AI platforms that we've also announced back in February. So we are enabling that and it, it, it just comes down to if you know the task you are trying to do. You can make things quite small and efficient, right? So in a hearing aid, right?

• 30:39Speaker 00

  In a hearing aid, you're probably not trying to run a large language model, not yet at least. You're doing things like trying to do, you know, reduce noise or picking out, you know, amplifying only the interesting, you know, what are you trying to hear? Now, obviously adding translation and those kinds of things are probably going to be possible soon. Now, if I know what language I'm translating to, then I can, you know, that gonna make my model smaller. But yeah, everything, you know, one of the things that I like to think about is how much, you know, we use apps today to configure things. Like a good example is like a security camera, right? So today, you probably, you know, when you try to install your security camera,

• 31:23Speaker 00

  in the past, you would have went to your laptop and connected it. Now probably use your smartphone to do that. Well, you know, that is a use model. We've gotten used to it, but but why can't you just do it with your voice? Like, why can't it talk to you and have the camera have a large language model? And you say, hey, it asked for what is your SSID? You're the ones I see and you know, now again, that may or may not be a better user experience, but we just see the applicability across the board. Again, I use that touch analogy because touch has become so prevalent. AI is going to be way more prevalent than even touches today in changing the use model, how you interact with these things.

• 32:04Speaker 00

  You know, a great example I'll give you, Craig, is we worked very closely with Meta, who is really doing a great job, I think, with with the, you know, some of the advances in their glasses, right? So the XR glasses and their and the product they just announced two weeks ago, three weeks ago, they now have a wristband that goes with that product. And that actually uses one of our Ethos NPU's, which is

• 32:33Speaker 00

  Again, super small, super low power that you can have this wristband that basically, you know, has a huge battery life. I forget how many days or weeks you can wear it. And it literally just, you know, you manipulate things just by moving your fingers like this, and it senses in your wrist the changes and what's happening, you know, below your skin using AI to basically figure out that, oh, you just did your second finger. So that's going to do this, or you just did this. So that's, or you know, you're doing this. So basically, that is the new UI of a wristband that has AI in it that has, you know, long, long battery life and has a tiny, tiny battery. That's, that's how small we can make AI

• 33:18Speaker 00

  for very specific use cases.

• 33:22Craig S. Smith?

  Yeah. And, and what's the focus with, with ARM right now? Is it a reducing size? I mean, obviously, it's going to be all these things. But, but where do you see the next breakthrough? Is it reducing size, reducing power consumption, increasing the model size that can, can be

• 33:50Craig S. Smith?

  on these chips? Yeah. Where, where do you see it going?

• 33:55Speaker 00

  Um, it's a great question, Craig, and it is a, it is a, a very open design space right now, to be honest with you. Um, you know, definitely there has been a focus on power in the past, and that's kind of where our ethos product line comes from. And, but a lot of that was around CNN networks and some of the early stuff. Now you're starting to see the move to transformer networks. What's next, right? I don't think that transformers are actually the end goal. And so we also need to allow flexibility as these models change because, you know, back to kind of educating your users, I mean, it takes two years almost at a minimum to design silicon and get it in a shipping product. And you can see how fast AI is moving.

• 34:40Craig S. Smith?

  Yeah.

• 34:41Speaker 00

  So there's also this flexibility piece, but I, I think most of it is performance. It's really memory performance and it's, and it's tops and CPU performance. And then, and, and trying to shrink that down as, as, as the best we can for, for whatever the power envelope is to, you know, how much computing you can get. And I think that's obviously happened in the data center as well, right? Where it's okay, we're trying to get, we're now building gigawatt data centers. But how much, how, you know, how many tokens can you create? How much performance can you get in that? So almost everywhere in the space, it's like, okay, tell me what your power envelope is. It's a gigawatt here.

• 35:24Speaker 00

  It's, you know, seven, six watts in a phone. It's, you know, in my wrist example, it's literally, you know, a couple hundred milliwatts. You know, how much AI performance can you get? And so it, it really is spanning a, a large space.

• 35:40Craig S. Smith?

  Yeah. And you guys don't, you, you're, you're producing IP, right? You're not building, you don't have a fab. You're not sending your designs out to a fab to make your chips. You're licensing this to other chipmakers.

• 36:03Speaker 00

  That's correct. Yes, our business model is we provide IP to, you know, the, a large portion of the semiconductor ecosystem. And, and then our partners build on top of it and create, create silicon solutions that can span, like we talked about, anything from, you know, a wristband or, or your hearing aid to, you know, your next self-driving car.

• 36:30Craig S. Smith?

  Yeah, yeah. And where is most of this? Obviously, most of it's going into the US. But do you have partners that are licensing your IP? What are the other markets that you're operating in?

• 36:50Speaker 00

  Well, you know, I would say that traditionally, semiconductors have been quite global, right? You know, that the the cost of developing a semiconductor is tremendous, right? Obviously, the foundry costs, and I'm not even saying, like, put aside the foundry costs, but literally building a chip, you know, if your example, if you're using one of the latest nodes, just the mass cost alone of when you're done your design and getting that manufacturer is in the tens of millions of dollars. The development costs upfront can often be in the hundred million dollar plus range. So you're talking about tremendous amount of cost to get to the first unit.

• 37:35Speaker 00

  But what's made semiconductor so great is that you can scale that. And because we have these amazing manufacturing capabilities, etc. And so they tend to be very large markets. And so generally, it's been a global market for us. And yes, you know, obviously, there's many great IC or semiconductor companies in the US that do designs. We work very closely with with European semiconductor companies in Taiwan, in Korea as well. And we also have had a Arm China, a JV where semiconductors get built in China based on our IP. And so we get, it's a pretty global market.

• 38:17Craig S. Smith?

  Yeah, I would imagine the Chinese relationship has been strained by the sanctions. How does that affect you guys?

• 38:28Speaker 00

  Well, yeah, I mean, we always make sure we're following the local laws and obviously where we're relocated.

• 38:37Speaker 00

  Most of the constraints have been around the manufacturing process and not necessarily some of the IP, there is some around IP enablement, but you know, it's something that we always are very careful about. Make sure we're complying to what is required and we'll see how things evolve.

• 38:57Craig S. Smith?

  In that you guys are primarily selling IP, do you work with a large developer community and what's that relationship like? Is how much of this is open source, for example? And then if you could talk about where you see, I mean, you mentioned wearables, where wearables are going to be a hot thing, where you see the edge really exploding the public

• 39:30Craig S. Smith?

  interaction or experience with AI?

• 39:33Speaker 00

  Okay, yeah, so I mean, let's start about developers. So, you know, ARM is, you know, developers are super important to ARM. And in fact, we now have the world's largest developer ecosystem. We believe we have over 22 million software developers that are developing software on ARM. And that's just because of our footprint. That spans all the way from, you know, iOS to now Windows on ARM. And then, of course, things like Chrome, Android and Linux. You know, I've been a longtime supporter of, so, you know, really, the world's largest software ecosystems are quite,

• 40:14Speaker 00

  you know, work closely with ARM and we support them. And we really focused on that developer experience and how do we improve that developer experience? I mentioned Clidi before in this, in this conversation. That's really about, again, trying to make it seamless for developers to use AI. Because, you know, you talked about CUDA, AI is not simple from many of the programming elements. And I would say we're still seeing. Not necessarily the hardware abstraction that we've gotten other software ecosystems to. We still see a fairly tight coupling to today's models to actually the hardware that they run on.

• 40:57Speaker 00

  Whether that's the operators they support, whether that's, you know, the way they kind of assume the model is going to propagate and those kinds of things. So, um, you know, but we are, we're clearly focused on the supporting those developers and, uh, and, and seeing what they can build, right? And I think that kind of goes to your second question of, you know, what's kind of on, on the forefront. Uh, and that's why it would makes my job so fun because I get to, to, to, to interact with so many of these super innovative folks. Both startups and established, established companies on kind of what is next or what, what's the art of possible? And, and if I was just to use a general concept, it would just be intelligence.

• 41:40Speaker 00

  The idea of something is capable of interacting with you in a way that exceeds your current expectations. Right? And maybe I'll, I'll use another product example. I'm a huge Amazon fan, um, but one of the things I've noticed is that before I upgraded to Alexa plus, you know, because of the way I was using a chatbot or chat GPT or Gemini, I started talking to it in a, in a more conversational manner because that's what I could do. But then when I would come home and use Alexa, it wouldn't necessarily, it was expecting more of kind of a search, kind of give me,

• 42:22Speaker 00

  you know, really break it down pretty, you know, I'll give you your three words and try to contextualize those three words. You know, and so, and I think, by the way, Amazon's done a great job with Alexa plus, and I think they're, they're on the right path here. But that's just a change in expectations and I, and I think that's going to only go, you know, just through the roof relative to, you know, the idea of, you know, the things that we think are how you interact with a device for it just to be smart. You know, one of the things people ask me about is like, what's a Gentic AI or how's that going to feel? Right? And and one of the things I like to use, and I think I think Microsoft has done an amazing job of really integrating AI into their products as large as a company they are.

• 43:08Speaker 00

  You know, the way they push copilot, all those kinds of things. But what I like is a is a silly example of settings. So when you use Windows and Windows settings. We've all done it and you have to figure out, okay, I want to add a screen or I don't like this resolution screen. How many like Windows do you have to click in to kind of say? Figure out how to make this change, right? Well, they've changed it to be more of an agentic experience in Windows 11, where you just say, what do you want to do? I want to add a screen. I want to reduce this. And it just says, okay, well, here's how you do it, or I'll do that for you. And so, you know, to me, that's a touchy feely thing of how many times?

• 43:51Speaker 00

  Do we, you know, click through three apps? Oh, I'm going to cut this. Now I have to go to my flight app, then I have to go to my rental car app. And I like, you know, so many of these things that we're just trained to think like, oh, yeah, it works good enough. You know, but like this idea of things are just so smart that, you know? When you call an Uber, it of course knows where you're going to go. And it's already told people when you're going to show up and and all these kinds of things. So, you know, those are, and of course, those are consumery examples. There's huge amount of enterprise examples of, you know, teams that are not sharing data or people don't understand. If they're looking for information they can't find. And so I think it's just going to be the general idea of just how do we make things just super smart?

• 44:35Speaker 00

  And really kind of exceed expectations of what technology limitations are today.

• 44:41Craig S. Smith?

  Yeah, yeah, and robotics is is another one. I'm not a great believer in humanoid robotics, I think.

• 44:50Craig S. Smith?

  We're a long way from that, but uh what you're describing, I mean, uh it's um. You know, the day will come when uh our children or grandchildren uh will will look back and can you believe that? You know, any object that you use daily uh was a dumb object back then instead of being able to talk to it and and have it like configure itself or or whatever it is. And that's that's pretty exciting. What about robotics? Uh I mean, obviously you guys uh are involved in that.

• 45:29Speaker 00

  Yeah, I mean, it's pretty amazing the what's what's happening there um and the ability to train the robots in virtual worlds as well as real world. Um and we are a huge believer in that and and and we're quite involved with many of the ecosystems. Um, you know, I think it is really taking the advantage of, you know, you know, the camera sensor has been an amazing enabler, right? If you just look at what our cell phones are able to do in capturing images. But really, the the camera sensor has now unleashed this idea of physical AI and how do you, you know, because we can use vision systems and AI is so good

• 46:08Speaker 00

  with vision systems, right? Whether that's looking at an MRI and or, you know, maybe and and augmenting a doctor's reading of that or in a system and trying to understand what's going on. So it is it is a huge area. I think we're going to see a lot of um innovation. I think, you know, to your robotics example, you know, I think, I don't know, go back to the Jetsons or whatever. I mean, people said they're going to have flying cars and they were going to be able to do all these autonomy things. Uh it's taken a long time, but you know, another great arm powered device, my Tesla, you know, it is gotten pretty darn good relative to self driving and and and I really enjoy having that augment me in in many, many scenarios. And so

• 46:52Speaker 00

  it is, like you said, it's going to just be an expectation and you're just going to be disappointed when it doesn't do something that it clearly can't do today. But these things are hard. I mean, you know, I think that that blend of physical um and and. You know, is going to be a hard thing, but like anything, it's probably, you know, I go back to, I was involved with Bluetooth in the early days and, and, you know, oh, it's going to be amazing. You're going to have all these Bluetooth devices and just, and it, it didn't work great in the beginning. Experience wasn't great, but just look at fundamental Bluetooth is today. Again, when I walk up to my Tesla, it just unlocks, it just like unlocks it and it's all over Bluetooth and augmented by UWB or, you know, so it's, it's just these technologies,

• 47:37Speaker 00

  you know, sometimes the hype cycle gets a little bit ahead of expectations and that's, that's our fault as technologists. But at the end of the day, the technology delivers and, and it's going to be an exciting future for our children and grandchildren.

• 47:51Craig S. Smith?

  Yeah, absolutely. And yeah, with moving into physical, the challenges aren't really the AI, it's the, it's the mechanics, the actuators and, and the, the wear and tear and the dust and the grease and all that stuff that people kind of forget about. That, that's, that's, it's still got a long ways to go. Yeah. Okay. So, so is ARM on the verge of announcing anything new? Or is this, is it really the, the V9 that you're?

• 48:34Speaker 00

  Well, we, we've got quite a broad product, set of products, right? So yes, I, you know, I have my responsibilities, but we have a whole automotive and robotics group. We have a whole group that focuses on data center. Same thing with Edge AI and additional intelligence. So I think you'll keep seeing quite a bit from us. Lots of great stuff going on. And, and again, we're just super excited about the partnerships that we get to partner with and how much developers, and hopefully some of your listeners are inspired to build things on top of the ARM architecture and, and help make the future what it's, what's, what's, show us what's possible.

• 49:14Craig S. Smith?

  Yeah, and and on that note, we'll end on that. Developers, if they're interested, do you have a portal, developer portal?

• 49:25Speaker 00

  We do, developer-arm.com. Great, great, great resource to to find out more, figure out how to, you know, build and and and explore many of these new technologies. Obviously, many of the makers are familiar with Raspberry Pi. That is a also arm powered and something that many, you know, kids and educators start, start their journey and and those things are getting super excited now with robotics and beyond.

• 49:56Craig S. Smith?

  In business, they say you can have better, cheaper, faster. But you only get to pick two. What if you could have all three at the same time? That's exactly what Cohere, Thomson Reuters and Specialized Bikes have. Since they upgraded to the next generation of the cloud, Oracle Cloud Infrastructure. OCI is the blazing fast platform for your infrastructure, database, application development, and AI needs, where you can run any workload in a high availability, consistently high performance environment and spend less than you would with other clouds. How is it faster?

• 50:42Craig S. Smith?

  OCI's block storage gives you more operations per second. Cheaper? OCI costs up to 50% less for compute, 70% less for storage, and 80% less for networking. Better? In test after test, OCI customers report lower latency and higher bandwidth versus other clouds. This is a cloud built for AI and all your biggest workloads. Right now, with zero commitment, try OCI for free. Head to oracle.com/eyeonai.

• 51:25Craig S. Smith?

  Eye on AI, all run together, E Y E O N A I. That's oracle.com/eyeonai.