#222 Andrew Feldman: How Cerebras Systems Is Disrupting AI Inference

This episode is sponsored by Shopify. Shopify is a commerce platform that allows anyone to set up an online store and sell their products. Whether you're selling online, on social media, or in person, Shopify has you covered on every base. With Shopify you can sell physical and digital products. You can sell services, memberships, ticketed events, rentals and even classes and lessons. Sign up for a $1 per month trial period at http://shopify.com/eyeonai In this episode of the Eye on AI podcast, Andrew D. Feldman, Co-Founder and CEO of Cerebras Systems, unveils how Cerebras is disrupting AI inference and high-performance computing. Andrew joins Craig Smith to discuss the groundbreaking wafer-scale engine, Cerebras' record-breaking inference speeds, and the future of AI in enterprise workflows. From designing the fastest inference platform to simplifying AI deployment with an API-driven cloud service, Cerebras is setting new standards in AI hardware innovation. We explore the shift from GPUs to custom architectures, the rise of large language models like Llama and GPT, and how AI is driving enterprise transformation. Andrew also dives into the debate over open-source vs. proprietary models, AI's role in climate mitigation, and Cerebras' partnerships with global supercomputing centers and industry leaders. Discover how Cerebras is shaping the future of AI inference and why speed and scalability are redefining what's possible in computing. Don't miss this deep dive into AI's next frontier with Andrew Feldman. Like, subscribe, and hit the notification bell for more episodes! Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A. I. Twitter: https://twitter.com/EyeOn_AI (00:00) Intro to Andrew Feldman & Cerebras Systems (00:43) The rise of AI inference (03:16) Cerebras' API-powered cloud (04:48) Competing with NVIDIA's CUDA (06:52) The rise of Llama and LLMs (07:40) OpenAI's hardware strategy (10:06) Shifting focus from training to inference (13:28) Open-source vs proprietary AI (15:00) AI's role in enterprise workflows (17:42) Edge computing vs cloud AI (19:08) Edge AI for consumer apps (20:51) Machine-to-machine AI inference (24:20) Managing uncertainty with models (27:24) Impact of U. S.–China export rules (30:29) U. S. innovation policy challenges (33:31) Developing wafer-scale engines (34:45) Cerebras' fast inference service (37:40) Global partnerships in AI (38:14) AI in climate & energy solutions (39:58) Training and inference cycles (41:33) AI training market competition

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Training is how you make AI and inference is how you use AI. And for between 2014 and the end of 23, we were making AI. It was a novelty. It wasn't important in any action, the type of AI we were talking about now. But starting in 2024, it left novelty, it left cool, interesting to move into productive. And people began to use AI. And you saw this extraordinary boom in inference. And I think that's why companies like us and others are now announcing leading products. That's why we invested behind this idea of why we announced the fastest AI inference in the world.

• 0:43Speaker 00

  We're in the dial-up era of inference. Yeah. It's slow. You can see the spinning ball of death we used to get. You could almost hear that modem noise. Yeah. Going back and forth. Now, we have a near-memory compute architecture. It's a data flow architecture on a wafer-scale chip. So all our memory is on the chip and it's S4. And it's close. Each compute core has its own dedicated memory.

• 1:22Speaker 00

  And what this means is we have 7,000 times more memory bandwidth. Right? The path that data moves to get to compute. It's closer and we have more bandwidth.

• 1:39Speaker 00

  And the result is we are vastly faster at inference. And that's a

• 1:49Speaker 00

  sort of unover- overcomeable advantage for the GPU that we have there. This is GPU impossible speeds. Correct. Because we're running faster than their memory bandwidth can allow.

• 2:01Craig S. Smith?

  Yeah.

• 2:02Speaker 00

  So the HBM connected to a GPU has a certain upper bound that you can move data across.

• 2:10Craig S. Smith?

  Yeah.

• 2:10Speaker 00

  And we run faster than that.

• 2:11Craig S. Smith?

  Right, and it's and that's constrained by the length of the connection, the the material of the connection.

• 2:18Speaker 00

  Yeah, a- and the technology and the memory.

• 2:20Craig S. Smith?

  Yeah.

• 2:21Speaker 00

  And we are faster than that. And so what we announced was the fastest inference in the world bar none for Llama eight B, three point one, and Llama seventy B. And so we're really proud of these results. They are blisteringly fast. As one of our customers said, speed and capacity changes everything.

• 2:48Craig S. Smith?

  Yeah. Now, the question is,

• 2:54Craig S. Smith?

  in order to to to get people to use Cerebras inference, and this is a cloud-based service you're providing, you have to convince the clouds to install Cerebras wafer engines.

• 3:10Speaker 00

  Right.

• 3:11Craig S. Smith?

  Uh, wafer scale engines. Are you, you're starting with your own cloud?

• 3:16Speaker 00

  Correct, correct. I I think wa- waiting for other people to deploy your equipment may not be the best strategy. So we stood up our our own cloud, and we have X of flops of capacity available. And we made it dead simple to move. It's less than thirty seconds. It's API-based. There's a chat engine. You can move by a single code snippet just directing to the Cerebras API. It it is really extraordinarily simple. And we had tens of thousands of developers sign up in the first ten days.

• 4:01Craig S. Smith?

  Yeah. But but for this really to go.

• 4:07Craig S. Smith?

  To to dominate the market, which presumably at some point you would like to do.

• 4:12Speaker 00

  I would like that, yes.

• 4:13Craig S. Smith?

  Yeah, it has to move into the big cloud providers and replace GPUs, at least for for certain kinds of inference. How big a barrier is that? I know that Nvidia had this moat for developers because of CUDA. People were comfortable with CUDA, with cerebrus.

• 4:35Speaker 00

  Sure, sure.

• 4:36Craig S. Smith?

  In this case, it's an API call for people building applications. How do you get the the big cloud providers to provide it at scale?

• 4:48Speaker 00

  So I I think there that's really good question. There are a couple of points there.

• 4:54Speaker 00

  We we can provide today hundreds of millions of dollars a year of of revenue for us in our cloud.

• 5:07Speaker 00

  But we want more, obviously. I think the dynamics of the hyperscale world has changed with the rise of

• 5:18Speaker 00

  people like CoreWeave and the rise of Lambda and the rise of sovereign clouds, all looking to do training and inference. So the relative importance of the traditional three or four US-based hyperscalers has reduced. We would still love to be a part, but there are other vehicles and paths to market as well. I think the CUDA moat today is vastly overestimated.

• 5:52Speaker 00

  CUDA, over the past eighteen years, has had approximately forty million downloads. In the last eighteen months, Llama has had three hundred and fifty million downloads.

• 6:06Speaker 00

  All right, what's happened is.

• 6:11Speaker 00

  Fifteen years ago, CUDA was it. Then people moved up into TensorFlow. And then they moved up into PyTorch. And right now, they've moved up into into Llama. And application developers don't want to bother with C-like languages. They want to bolt on to to LLMs.

• 6:38Craig S. Smith?

  Yeah.

• 6:39Speaker 00

  And so, um. You know, when you have, uh, you know, nine X more downloads in the 12th time. Right? Something's happening there.

• 6:52Craig S. Smith?

  Yeah.

• 6:52Speaker 00

  And you have to take that very, very seriously. And so that that's what we see and that we our view is that. The building block of the future is going to be an LLM, not some C-level language that nobody likes to program in.

• 7:09Craig S. Smith?

  Right. The the Llama has done great. Mistral has done great.

• 7:15Speaker 00

  Done great.

• 7:17Craig S. Smith?

  But OpenAI still dominates.

• 7:20Speaker 00

  They do.

• 7:21Craig S. Smith?

  And.

• 7:23Craig S. Smith?

  Why is OpenAI so married to Nvidia chips, GPUs? Have you talked to them? It seems to me this would be natural to move. Uh, some of their their big models, foundation models onto a cerebral wafer.

• 7:40Speaker 00

  Okay, I think OpenAI has a complex, uh, large.

• 7:48Speaker 00

  Relationship with AWS. Excuse me, with Azure. Sorry about that, with Azure. Um, and that included for a period of time delivering all their compute. And so, uh, I I think the compute that that Azure had were were GPUs. Uh, as you saw in Nvidia's earnings announcement, they have four customers that are approximately half of their business.

• 8:14Craig S. Smith?

  Right.

• 8:15Speaker 00

  Right? Extraordinarily consolidation for customer consolidation for a company that big.

• 8:21Speaker 00

  Um. So I I think OpenAI has done extraordinary work with what was available to them. And I I think based on what what Sam has said publicly, uh, they would very much like uh uh other hardware options. Now, at the same time, they've built optimizations and uh around the uh around what they had.

• 8:47Craig S. Smith?

  Yeah.

• 8:48Speaker 00

  And so I I think that's the the race.

• 8:51Craig S. Smith?

  Yeah, it's but but the the W S C, the wafer scale engine that that you're putting Llama three point one on, and I don't know if you have four oh five B on it yet.

• 9:01Speaker 00

  Soon to come, soon to come.

• 9:03Craig S. Smith?

  Uh.

• 9:05Craig S. Smith?

  You could put

• 9:07Speaker 00

  Anything.

• 9:08Craig S. Smith?

  G P T four oh.

• 9:10Speaker 00

  G P T four, G P T five.

• 9:12Craig S. Smith?

  Yeah.

• 9:12Speaker 00

  Uh, we we can put any model on it.

• 9:15Craig S. Smith?

  Yeah.

• 9:16Speaker 00

  Um, and so I I think uh they are now so big that uh it it it it involves a wide ranging partnership.

• 9:28Craig S. Smith?

  Right.

• 9:29Speaker 00

  And and they're not easy to do.

• 9:31Craig S. Smith?

  Yeah. Have you done any pilots with them? Look, let's just see.

• 9:35Speaker 00

  We we uh we we don't generally talk about people we're doing pilots with. We we we will be announcing a whole range of really interesting customers uh in the weeks and months ahead.

• 9:47Craig S. Smith?

  Yeah, uh, the uh wha- what's happened generally, I mentioned to you that I interviewed Rodrigo from San Benova, uh, a company frankly that I I really wasn't aware of. Uh, and they're hitting bait benchmarks, they they claim.

• 10:07Speaker 00

  Yeah, some good work.

• 10:09Craig S. Smith?

  Uh. What's happening with inference? I mean, there was the in the hardware market was really focused on uh training. And Nvidia had that locked up.

• 10:21Speaker 00

  Sure.

• 10:21Craig S. Smith?

  And there were some other players. Uh. As you said, the the these these foundation models now people are building on top of. They don't.

• 10:33Craig S. Smith?

  Need to build their own foundation model. It's so is the market shifting to toward inference?

• 10:41Speaker 00

  Look, the way I think about it is training is how you make AI and inference is how you use AI.

• 10:47Craig S. Smith?

  Right.

• 10:47Speaker 00

  And for, you know, between two thousand fourteen and the end of twenty three, we were making AI.

• 10:58Craig S. Smith?

  Right.

• 10:59Speaker 00

  Right? It it it was a novelty. It it wasn't important in in any action.

• 11:06Craig S. Smith?

  Right.

• 11:07Speaker 00

  Right? It was the type of AI we were talking about now.

• 11:11Speaker 00

  But starting in two thousand twenty four, it left novelty, left cool, interesting to move into productive.

• 11:20Craig S. Smith?

  Yeah.

• 11:21Speaker 00

  And and people began to use AI. And you saw this extraordinary boom in inference.

• 11:28Craig S. Smith?

  Yeah.

• 11:29Speaker 00

  And I I think that's why companies like us and others

• 11:34Speaker 00

  are now announcing leading products. That's why we invested behind this idea. Why we announced the fastest AI inference in the world.

• 11:47Speaker 00

  Be because what we saw was this movement from making it to using it.

• 11:53Craig S. Smith?

  Right.

• 11:54Speaker 00

  And what we know about hardware is speed makes multi-billion dollar markets.

• 12:02Craig S. Smith?

  Yeah.

• 12:03Speaker 00

  And we know this, right? If you remember, and and you've got a little bit of gray hair like I do, right? You remember when Netflix was sending you DVDs and envelopes?

• 12:11Craig S. Smith?

  Yes, the guy started in the town that I live in, Chappaqua.

• 12:15Speaker 00

  Yeah, right? That's what happens when your internet is slow.

• 12:19Craig S. Smith?

  Yeah.

• 12:19Speaker 00

  And when your internet is fast, right? You have a multi-billion dollar streaming business. And when your internet is really fast, my eleven year old granddaughter is a creator.

• 12:32Craig S. Smith?

  Yeah.

• 12:33Speaker 00

  Right? She's not just the recipient of video. She's a creator of video and sending it to her friends.

• 12:41Craig S. Smith?

  Yeah.

• 12:41Speaker 00

  It's become a two way medium. Each one of those, each step, as the internet got faster, created enormous new markets. We believe the same happens with inference, by making it instant.

• 12:55Craig S. Smith?

  Yeah.

• 12:56Speaker 00

  All right, we make whole new markets possible.

• 12:59Craig S. Smith?

  The, again, on, on OpenAI, they're about to come out with, I don't know what they're gonna call it, GPT-5.

• 13:08Speaker 00

  Cool new stuff.

• 13:09Craig S. Smith?

  Yeah, something new, there's reasoning agents.

• 13:12Speaker 00

  Right.

• 13:13Craig S. Smith?

  Uh, do you think Meta will follow quickly after? Uh, I mean, this is, goes to the proprietary open source debate. Is, is, is open source going to be able to keep up, is going to be able to take the lead?

• 13:28Speaker 00

  I think that's an extraordinarily interesting battle.

• 13:32Craig S. Smith?

  Yeah.

• 13:33Speaker 00

  Right, that, uh, uh, is unfolding before us. I think, uh, the strategy that Meta has undertaken is interesting in that they undertook a similar strategy in hardware.

• 13:47Craig S. Smith?

  Yeah.

• 13:48Speaker 00

  Right, if you remember, they pioneered OCP, the Open Compute Projects, where, uh, they, they open sourced model, uh, open sourced chassis and various hardware building blocks for the community. And that was very successful for them. I, I think, uh, they've been a, a company that has pioneered openness.

• 14:12Speaker 00

  I think it is, uh, TBD, wh- wh- whether, uh, it will win, uh, i- in the big model space. We're rooting for it, we, we think openness is, uh, extremely valuable. I think their licenses are fairly permissive, so that many large enterprises can build on, uh, their open models.

• 14:36Craig S. Smith?

  You're talking about Meta?

• 14:37Speaker 00

  I'm talking about Meta's model, yeah, Meta's license for, for Llama. I, I think they have seen tremendous benefit to the company, uh, for, for their open source work. And so we, we hope that there continues to be a flourishing, open, uh, model creation ecosystem.

• 15:00Craig S. Smith?

  Yeah, and, and you mentioned that, uh, inference were kind of at the beginning, or its beginning, I mean.

• 15:09Craig S. Smith?

  Generative AI, there's a lot of pilot programs, there's very little in enterprise production yet, other than maybe customer services starting.

• 15:19Speaker 00

  Correct.

• 15:19Craig S. Smith?

  Uh, so, how big is this market gonna get, do you think?

• 15:25Speaker 00

  Uh, more than half a computer.

• 15:28Craig S. Smith?

  Yeah.

• 15:29Speaker 00

  Yeah, I, I think this will be, uh, it will be two or three ways that compute is done. And one of them will be, uh, the way we do it for AI. I think AI will be foundational in most, or maybe not most, many uh, uh, of corporate workflows. And I, I think if your organization isn't thinking hard about these right now, uh, you're gonna be left in the dust.

• 16:01Craig S. Smith?

  Yeah, and the advantage here again is that it's an API call.

• 16:06Speaker 00

  It's an API call.

• 16:07Craig S. Smith?

  Yeah.

• 16:07Speaker 00

  That's right.

• 16:07Craig S. Smith?

  Yeah.

• 16:08Speaker 00

  I, I think, uh, you know, what, what, um,

• 16:14Speaker 00

  what Python brought to the world was a, uh, interpreted,

• 16:23Speaker 00

  simplified way to get code written. Um, and I think what large language models will become are these building blocks that are called upon, uh, to do tasks.

• 16:38Craig S. Smith?

  Mm-hmm.

• 16:39Speaker 00

  And they, they will be underneath, uh, many, many of your corporate workflows.

• 16:48Craig S. Smith?

  Yeah, the, uh, uh,

• 16:54Craig S. Smith?

  the, you know, 5G technology, 6G, I mean, the, the, the transfer of data, uh, wireless transfer of data is, is getting better, is.

• 17:09Craig S. Smith?

  I'm trying to ask about the edge is how does this can this reach the edge? Or are we really going to talk about, uh, which is what it is today? A different class of, uh, chips, a different class of models that can operate at the edge? Or, or do you think that the day will come where those transmission speeds and the bandwidth is high enough? That you can, uh, reach the edge from the cloud?

• 17:42Speaker 00

  Well, I, I think, uh, many,

• 17:48Speaker 00

  if not most, consumer applications will eventually be, uh, delivered by the edge.

• 17:56Speaker 00

  And, and I, I think that is our history.

• 17:59Craig S. Smith?

  Yeah.

• 17:59Speaker 00

  It, it is that consumer applications get pushed to the edge. I think enterprise applications are much heavier. And generally benefit from, uh, a different class of compute. And so a- as we see, uh, models improve algorithmically. As our ability to prune them, to use techniques like sparsity.

• 18:28Craig S. Smith?

  Right.

• 18:28Speaker 00

  To make them smaller. As our ability to build better edge chips.

• 18:36Speaker 00

  I think you'll see a great deal of work done at the edge. But it's a fallacy to think that the edge subtracts from the core.

• 18:47Craig S. Smith?

  Yeah.

• 18:48Speaker 00

  I, I think, uh, the more work done on the edge, the more work the core needs to do as well. I think the more AI diffuses through our ecosystem, the more AI is done, the more training that needs to be done, the more value accrued to, to advantages in training, the more heavyweight inference you need to.

• 19:06Craig S. Smith?

  Is cerebrus working on any hardware for the edge?

• 19:10Speaker 00

  We are not.

• 19:11Craig S. Smith?

  Yeah, it's just, just not, not your business.

• 19:14Speaker 00

  No, I, yeah, that, that's not, uh.

• 19:19Speaker 00

  That's a, that's a different business. It's differently challenging.

• 19:24Craig S. Smith?

  Yeah.

• 19:26Speaker 00

  And I think the returns to Apple and others who build the edge are, are different. It's been a place, it's very challenging for startups to enter.

• 19:41Craig S. Smith?

  Yeah, but with the speed of, with the speed of inference from the cerebrus in the cloud and with wide enough bandwidth, how much compute actually has to take place at the edge other than critical, safety critical applications?

• 20:01Speaker 00

  I, I think it's TBD. I think you certainly want your, your car doing inference.

• 20:06Craig S. Smith?

  Yeah.

• 20:06Speaker 00

  You don't want that going back to, to cloud. I think you, you'd like your phone doing some basic inference. But I think

• 20:19Speaker 00

  just like search on your phone.

• 20:22Craig S. Smith?

  Yeah.

• 20:24Speaker 00

  There, there's some, some hooks that, that use the phone CPU and some work done in the cloud. And I, I think that's sort of a model you'll see for inference.

• 20:35Craig S. Smith?

  Yeah.

• 20:35Speaker 00

  And, uh, you know, when you're as fast as, as we are, what's left really is transport latency. And where, where that matters a lot, your car.

• 20:47Craig S. Smith?

  Yeah.

• 20:48Speaker 00

  You definitely don't want to do it.

• 20:49Craig S. Smith?

  Right.

• 20:51Speaker 00

  But I think a lot of inference, and this is, I think, a mistake people make right now. Is thinking of inference as

• 21:03Speaker 00

  human, human readable inference. So it generates as free syntax or images.

• 21:09Speaker 00

  I, I think increasingly that the customer for most inference is other machines. Yeah. Right. And so the consumer for uh the results of inference is going to be a another chunk of software. Yeah. And you're going to have collections of these uh where uh the input of one produces an output which becomes the input of of the next. Right. And so blistering speed matters an enormous amount. Otherwise, you have a stacking of latencies. Yeah. Um. And so I I think that's the way enterprise workflows are are going to unfold.

• 21:53Speaker 00

  You have sort of collections of of inference. You see this already with agentic workflows. You see this already with uh train of thought.

• 22:05Speaker 00

  So I I think tha- that's more complicated inference, and I I think that's a direction we're going.

• 22:11Craig S. Smith?

  Did you ever think why some businesses are all over the place? And you're familiar with their brands and you interact with them seamlessly. When you think about those businesses whose sales are rocketing like Feastables or Mr. Beast or Thrive Cosmetics, you think about an innovative product, a progressive brand and button-down marketing. But an overlooked secret is actually the business behind the business making selling and for shoppers buying, simple. For millions of businesses, that business is Shopify. Nobody does selling better than Shopify.

• 22:54Craig S. Smith?

  And the not-so-secret secret with Shop Pay that boosts conversions up to fifty percent, meaning way less virtual shopping carts going abandoned and way more sales going. So if you're growing your business, your commerce platform better be ready to sell whatever your customers are scrolling or strolling on the web, in your store, in their feed, and everywhere in between.

• 23:25Craig S. Smith?

  Businesses that sell more sell on Shopify. Upgrade your business and get the same checkout that Feastables uses. Sign up for your dollar per month trial at Shopify.com/eyeonAI. That's all lowercase. Shopify, S-H-O-P-I-F-Y.com/eyeonAI. E-Y-E-O-N-A-I. All run together. Go to Shopify.com/eyeonAI to upgrade your selling today. That's Shopify.com/eyeonAI. Yeah.

• 24:08Craig S. Smith?

  And mix your ensemble of models where you

• 24:13Speaker 00

  That's right.

• 24:15Craig S. Smith?

  You want an average of several models.

• 24:20Speaker 00

  Yeah, we have a very interesting case study where a customer has multiple different models

• 24:33Speaker 00

  stacked on the same wiper. And the same stimulus, the same signal comes in. And the models are completely different classes of models.

• 24:48Speaker 00

  And they use the results, the correlation between the results as a measure of certainty.

• 24:54Craig S. Smith?

  Yeah.

• 24:55Speaker 00

  Right, so we have four different types of models agree. All right. On their their result, then they are more certain.

• 25:06Craig S. Smith?

  Yeah.

• 25:06Speaker 00

  Of the result. That's an extremely interesting approach to manage uncertainty.

• 25:11Craig S. Smith?

  Yeah.

• 25:12Speaker 00

  So we've seen some very, very creative work.

• 25:15Craig S. Smith?

  Yeah, and that'll eventually reach consumer.

• 25:19Speaker 00

  Of course.

• 25:20Craig S. Smith?

  Yeah.

• 25:23Craig S. Smith?

  Solve the hallucination.

• 25:24Speaker 00

  That's right. That's exactly right.

• 25:26Craig S. Smith?

  Yeah. How many players do you think there will be in this inference hardware market? I mean, right now, it's. I don't know. Four, five?

• 25:40Speaker 00

  Yeah. I think.

• 25:44Speaker 00

  There are very few healthy markets that have only two players.

• 25:51Craig S. Smith?

  Yeah.

• 25:52Speaker 00

  Right, so my expectation is there will be a handful.

• 25:56Speaker 00

  Three to five.

• 25:58Craig S. Smith?

  Yeah.

• 25:58Speaker 00

  I, I think, you know, we're, we're proud to, to, to be

• 26:04Speaker 00

  among the leaders in the challenger category.

• 26:08Speaker 00

  And so I, we intend to be here.

• 26:12Craig S. Smith?

  And you have the, the production pipeline to, to meet capacity, which is always an issue. It's an issue with, within Nvidia.

• 26:22Speaker 00

  Well, and it is. We, we increased manufacturing capacity 5X last year, this year, and, and we, we intend to, to do something similar next year. And so capacity is, is a challenge, but we're up to the task.

• 26:40Craig S. Smith?

  Yeah, and this is coming out of TSMC.

• 26:43Speaker 00

  TSMC makes the, the wafer. But remember, we build a system. And, and so

• 26:51Speaker 00

  in, in fact, that was a, a good decision, right?

• 26:56Speaker 00

  When we made that decision, in, in Nvidia hadn't doubled down on a building boxes. AMD hadn't acquired a company to help them build boxes. And several of our chip competitors were thinking they could build PCI cards.

• 27:13Craig S. Smith?

  Yeah.

• 27:13Speaker 00

  And we bet on a system because that was clearly, in our view, the, the way to deliver this extraordinary technology. That's proven out, eh, to play in this market, you have to build a system.

• 27:24Craig S. Smith?

  Yeah. You guys are restricted from, from doing business in China with the current export controls.

• 27:34Craig S. Smith?

  What do you think's gonna happen? I mean,

• 27:38Craig S. Smith?

  I have a conversation tomorrow with a

• 27:42Craig S. Smith?

  I'm not gonna remember his name, but he, he wrote the book on.

• 27:46Speaker 00

  Oh, Chip, Chip Lars.

• 27:48Craig S. Smith?

  Yeah.

• 27:48Speaker 00

  Cr- Chris.

• 27:49Craig S. Smith?

  Chris.

• 27:50Speaker 00

  Yeah, yeah, yeah. A good book, read the book.

• 27:52Craig S. Smith?

  Yeah.

• 27:53Speaker 00

  Extremely.

• 27:56Craig S. Smith?

  Where do you think that's going to go? Is that, is, is, I mean, my view, you know, I spent a lot of my life in China. I see where, uh, things have gone with Xi Jinping and, and the reproachment with Russia. I just don't see us coming back from that. And, and it's to China's detriment because they're ten years behind in ship production, in ship manufacturing. What, what, do you, you think that this is a fork in the road?

• 28:30Speaker 00

  First, I, I think, uh,

• 28:35Speaker 00

  because your CEO doesn't give you great insight into the geopolitical matters. No, I, I think it's, it's somehow you're, you, you, you're good at building teams and product and people ask you geopolitical questions.

• 28:46Craig S. Smith?

  Right.

• 28:47Speaker 00

  And, um, I, I think most CEOs don't, including myself, aren't experts in that.

• 28:53Craig S. Smith?

  Yeah.

• 28:54Speaker 00

  I, I think what we can say is, um, the following.

• 29:01Speaker 00

  We have, over the course of a thirty year career, built product together with Chinese companies.

• 29:07Craig S. Smith?

  Yeah.

• 29:07Speaker 00

  And, and found really good people.

• 29:09Craig S. Smith?

  Yeah.

• 29:09Speaker 00

  Like-minded engineers.

• 29:10Craig S. Smith?

  Yeah.

• 29:11Speaker 00

  That, uh, governments are different and the behavior of governments are, uh, are difficult to fathom. I think, um,

• 29:24Speaker 00

  we are, uh, I think obligated as good citizens to, to, to, to abide by not just the letter of the law, but the spirit of, of the rules put down by Department of Commerce and, and by the executive orders. And, and so we, we've been following that diligently.

• 29:45Speaker 00

  I, I, I think that, uh,

• 29:50Speaker 00

  Everybody loses.

• 29:51Craig S. Smith?

  Yeah.

• 29:52Speaker 00

  I in uh if the two nations take adversarial positions, both.

• 29:59Craig S. Smith?

  Yeah.

• 29:59Speaker 00

  Everybody loses, I I think, and and so.

• 30:05Speaker 00

  Uh I I'm no expert in this domain.

• 30:10Speaker 00

  But it I I it it we're not in a good spot, clearly.

• 30:15Craig S. Smith?

  Right.

• 30:16Speaker 00

  And and you know, tha- tha- that's sort of the the observation um tha- that we have.

• 30:22Craig S. Smith?

  Yeah. And and China is is, I mean, I I frankly agree, I I think that.

• 30:32Craig S. Smith?

  That China was happy to let, you know, the Dutch do the

• 30:39Craig S. Smith?

  ultraviolet etching machines and let TMC do the wafers and everybody was happy.

• 30:50Craig S. Smith?

  But they're gonna they're gonna build their own ultraviolet etching uh machines, it'll take a while, but but the Chinese, you know, they've got a lot of incredibly talented people and a lot of money. So it's, I think the US kind of mis- misplayed that because it it triggered that uh that response in China and and they'll be a competitor in the chip market.

• 31:16Speaker 00

  I

• 31:18Speaker 00

  a- a- as I said, uh thi- thi- this is some

• 31:24Speaker 00

  Sunday, Sunday sort of couch quarterbacking, right, so they have to do quarterbacking from guys who barely played football in in high school, right? I- it's easy to talk about what's going on on TV.

• 31:39Speaker 00

  But I I think uh right now, uh there are returns to industrial policy, strategic industrial policy. And I I think um we are in the US

• 31:55Speaker 00

  sometimes challenged in making long-term policy.

• 32:00Speaker 00

  Our budgeting process, our the way we allocate, the way we approve funding. And I I think that's a challenge for us. And large capital investments that pay dividends over decades have proven challenging in the last ten or fifteen years to to get appropriated.

• 32:24Craig S. Smith?

  Yeah.

• 32:25Speaker 00

  And I I think a- as we look forward, things like artificial intelligence, continued investment in STEM. It being sure that that our colleges and universities continue to produce world-class engineers. That there are

• 32:44Speaker 00

  that we produce electrical engineers who who while at university can do cool things, can build chips, can work on interesting projects, that investment needs to be made. And I I think the Chips Act is a tiny little start. Right, forty billion, I mean, it sounds like a big number, it's a tiny little number.

• 33:04Craig S. Smith?

  Yeah.

• 33:05Speaker 00

  Right, i- i- Nvidia did more than that, i- is chasing that in a quarter.

• 33:10Craig S. Smith?

  Right.

• 33:11Speaker 00

  Right, so, you know, this is for a nation, for for economic policy and industrial policy for a nation.

• 33:19Craig S. Smith?

  Yeah.

• 33:19Speaker 00

  And so it's a tiny little bit, it's a start. And so I think it's a good start, but I I think we need to think much, much bigger.

• 33:26Craig S. Smith?

  Yeah, have you heard of anyone in China doing wave for scale?

• 33:31Speaker 00

  I've seen papers that they've published written about it. Uh, but we've not seen anybody who who's done it. It turns out to be pretty hard, right?

• 33:40Craig S. Smith?

  Yeah.

• 33:40Speaker 00

  I mean, it's no joke.

• 33:41Craig S. Smith?

  Sure.

• 33:42Speaker 00

  Um, and, you know, we we were the first to succeed at it in the seventy-five year history of the computer industry. Gene Amdahl failed failed at it and no dope was he, right?

• 33:55Craig S. Smith?

  Interestingly, he he was, uh, lived down the street from where I live. Uh, in Chappaqua, I always tell my kids, Gene Amdahl lived in that house, they have no idea.

• 34:09Speaker 00

  So my neighbor was William Shockley.

• 34:11Craig S. Smith?

  Oh, is that right?

• 34:12Speaker 00

  Growing up, he invented the transistor and we, we, we were young and had no idea that this was seminal in, in sort of invention in the industry and all we knew was that his wife gave full-size candy bars at Halloween. So we loved stopping there for Halloween.

• 34:29Craig S. Smith?

  Yeah, yeah. Uh, okay. I'm running out of questions. Is there something I haven't touched on that, that you wanna talk about? We didn't really talk about, uh, the, the inference service.

• 34:45Speaker 00

  The inference service is blisteringly fast. Uh, there are three tiers. Uh, and this is our, our, our cloud-based service. There's a free tier. You can jump on. It's a playground. Come and play. Uh, I, I think, uh, if you read some of the, uh, the, uh, the comments from early users, I mean, their, their, their eyelids were blown back at how fast it was. There's a pay-as-you-go, uh, tier where you, you buy a million tokens. Uh, and then there's a, a dedicated throughput tier and enterprise class tier. We have seen extraordinary things made already.

• 35:24Speaker 00

  And it's as infrastructure builders, it's a, it's a pleasure when people build cool stuff on your stuff.

• 35:29Craig S. Smith?

  Yeah.

• 35:30Speaker 00

  Right? That's one of the great joys of being an infrastructure builder.

• 35:34Craig S. Smith?

  Good.

• 35:34Speaker 00

  And we, we've seen take up in our enterprise customers too. And, uh, both immediately in the cloud and for deployment on, on their premise. And so, uh, it is just creating an explosion of opportunity for us.

• 35:50Craig S. Smith?

  Yeah. What, what about the government? I mean, I, I had, uh, the head of the institute at Lawrence Livermore on the podcast recently. Fascinating, brilliant guy.

• 36:04Speaker 00

  Bronis?

• 36:05Craig S. Smith?

  No, it was, uh,

• 36:09Speaker 00

  Never mind.

• 36:10Craig S. Smith?

  Yeah, but uh, uh, it'll come to me in a minute.

• 36:16Craig S. Smith?

  Uh, he was telling me about the models that they're building at the labs.

• 36:21Speaker 00

  Right.

• 36:21Craig S. Smith?

  That these are government models. I was asking him, why, why doesn't the government build its own? Foundation model and they are.

• 36:31Speaker 00

  They are. You've seen large foundation models under the AI for science that's headed out of Argon. You've seen the Trilabs, which is Lawrence Livermore, Sandia, Los Alamos. We have a large program with them in which models are being built and our equipment is deployed.

• 36:50Speaker 00

  Uh, you see work being done out of Oak Ridge, uh, in which, uh, foundation models are being run on Frontier. You're seeing work at Argon where foundation models are being run on Aurora. These are, I think there's a great deal of, of, of work being done, both in the Trilabs and in the scientific DOE labs.

• 37:13Craig S. Smith?

  And, and how difficult is it to get, um, get them to adopt a way for scale engines?

• 37:21Speaker 00

  Lawrence Livermore is a customer, Argon is a customer, Sandia is a customer. National Center for Supercomputing is a customer, Pittsburgh Center for Supercomputing is a customer, European Parallel Computing Center is a customer. LRZ, Bavaria is a customer. We, we sort of ran the table on large computing sites.

• 37:39Craig S. Smith?

  And these are on-prem.

• 37:41Speaker 00

  These are on-prem deployments.

• 37:43Craig S. Smith?

  Yeah. And you mentioned Aramco and when.

• 37:45Speaker 00

  We had, uh, well, thank you for bringing that up.

• 37:47Craig S. Smith?

  Yeah.

• 37:48Speaker 00

  Uh, we signed an MOU with Aramco, uh, here at the show where, where, uh, we've done, uh, a fair bit of work with Total Energies. We, we now made ourselves knowledgeable on uh, various simulation techniques and AI models for oil exploration and for simulation and reservoir modeling. And so we look forward to a, a, a long and successful partnership with Aramco.

• 38:14Craig S. Smith?

  Yeah, yeah. Biggest company in the world.

• 38:17Speaker 00

  Yeah, about 10% of the world's oil.

• 38:19Craig S. Smith?

  Yeah.

• 38:19Speaker 00

  It's an amazing organization.

• 38:22Craig S. Smith?

  Yeah. Uh, uh, the, uh, on the other side of the, the fossil fuel coin, are you doing much with, uh?

• 38:33Craig S. Smith?

  For simulation and climate mitigation.

• 38:36Speaker 00

  We are. We actually have large models that customers have built for for climate.

• 38:43Craig S. Smith?

  I mean, Lorenz Livermore is doing this, yeah.

• 38:46Speaker 00

  But there are also models being done and used for carbon sequestration, their their models, and their techniques that are being used for carbon capture. There are all sorts of interesting companies and ideas. We've seen Total be a pioneer in this, where they in inject into the empty oil well

• 39:11Speaker 00

  carbon gas, all sorts of other ways of sequestering carbon.

• 39:17Speaker 00

  Excuse me.

• 39:20Speaker 00

  So we we've done all sorts of work in this domain.

• 39:25Speaker 00

  And interestingly, the the nations that you might not expect it, the oil-producing nations, Emirates, Saudi, have taken a leadership position in thinking about climate and environmental impact, and have really been aggressive.

• 39:44Craig S. Smith?

  Yeah, yeah, certainly. The impact is most extreme in this part of the world.

• 39:52Speaker 00

  And so they're doing really interesting work in mitigation and in clean energy.

• 39:57Craig S. Smith?

  Yeah.

• 39:58Speaker 00

  They also have good sun, right? They have opportunity for solar, they use solar for desalination of water. They've already u used clean energy. They're building clean data centers in the UAE and here as well. I know NEOM is doing that here. So, well, I think we should talk just a sec about training. I think we are, because inference has exploded, it hasn't.

• 40:26Speaker 00

  Reduced the demand for training. I think this is a myth that there's some sort of zero sum game.

• 40:32Craig S. Smith?

  Right.

• 40:33Speaker 00

  I think quite the contrary. I I think there's a virtuous cycle that is created that as people use AI, they want better AI. The way you make better AIs with training. And and so what we have is vast amounts of new data coming in by virtue of the inference. This puts pressure on model builders and on the computer infrastructure to produce better models, to produce more accurate models. And so we're seeing a concomitant sort of explosion or or acceleration in the demand for for training. I I think I I hear this all the time. Is it going to be inference or training? And I I think you you've misunderstood the problem.

• 41:16Craig S. Smith?

  Yeah.

• 41:16Speaker 00

  Right? You you you can't use, which is inference unless you make, which is training. And the more you use, the more you want to make.

• 41:25Craig S. Smith?

  Right, except that Nvidia really has the training market locked up.

• 41:32Speaker 00

  I I I think that's, uh, certainly they have a great deal of market share. But I I think a great deal of work is being done right now on TPUs at Google. I think real work is being done on our equipment around the world. I mean, in this region, for example, we trained on our equipment in combination with our our partner G42, the leading Arabic model.

• 41:57Speaker 00

  There are other models that are trying to compete with it, but for nearly a year, we we have had the the premier Arabic LLM.

• 42:06Speaker 00

  I think there are plenty of examples. I know the guys in Anthropic are are doing training on non-GPUs. The guys at Stability are doing training on non-GPUs. And so I I think there's, I I think it's a mistake to say it's game over.