IMEC Says Today’s AI Will Look Ancient in 10 Years

AI's next bottleneck? It might not be GPUs. Memory prices have risen 700% this year, while power, cooling and data transfer are all becoming major constraints on the AI infrastructure buildout. At the same time, the semiconductor industry is approaching the physical limits of simply making transistors smaller. I sat down with Steven Latré, VP of AI at imec and Thema Board Member, and Adam Chambers of Strike Capital, to talk about what comes after the GPU and where the next wave of value in AI infrastructure could be created. We get into › Why memory is already one of AI's biggest bottlenecks › The power, cooling and data transfer constraints behind inference › Why photonics could become a critical technology for the next generation of chips › What happens as Moore's Law approaches the limits of physics › Why Steven thinks we'll laugh at today's AI in 5–10 years Steven describes imec as the “hidden gem” of semiconductors. The organization works years ahead of the commercial market, with the world's biggest chip companies coming through its facilities to develop technologies that may not reach the market for another 7–10 years. His hottest take is that today's LLMs will eventually look like dial-up internet. “I think we're gonna laugh 5-10 years from now at how old-fashioned AI was today.” Recorded at the Strike x Sourcery Summit in the South of France. Steven Latre: https://x.com/slatre Adam Chambers: https://www.linkedin.com/in/adam-chambers-0b2009274 Molly O’Shea: https://x.com/MollySOShea Sourcery: ⁠https://x.com/sourceryy 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊 YouTube: https://youtu.be/6-q_8T_m5GM 𝐒𝐏𝐎𝐍𝐒𝐎𝐑𝐒 • Brex https://brex.com/sourcery—The modern finance platform, combining the world’s smartest corporate card with integrated expense management, banking, bill pay, & travel. https://brex.com/sourcery • Zone https://zonefrontier.com/—develops next-generation data center campuses, partnering with AI companies, site developers and technology leaders to bring compute online faster and at scale. Visit: https://zonefrontier.com https://zonefrontier.com/ • Turing https://turing.com/sourcery—Turing delivers top-tier talent, data, and tools to help AI labs improve model performance—and enables enterprises to turn those models into powerful, production-ready systems. https://turing.com/sourcery • VCX https://fundrise.com/vcx—VCX is the public ticker for private tech, allowing investors of all sizes to invest in venture capital. View The Portfolio at http://GetVCX.com http://getvcx.com • Deel https://www.deel.com/sourcery—Deel is the global people platform that helps startups hire, manage, pay, and equip anyone, anywhere. Trusted by more than 35,000 fast-growing companies, Deel is the people platform that just works, so teams can scale without the chaos. Visit: https://www.deel.com/sourcery • Public http://public.com/sourcery–Investing platform Public just launched Generated Assets https://public.com/generated-asset, which lets you turn any idea into an investable index with AI. With Generated Assets, you can build, backtest, refine, and invest in any thesis with AI. Gone are the days of one-size-fits-all ETFs. https://public.com/sourcery Follow Sourcery for the latest updates! https://www.sourcery.vc https://www.sourcery.vc/ Disclosure Paid Endorsement. Brokerage services by Open to the Public Investing Inc, member FINRA & SIPC. Advisory services by Public Advisors LLC, SEC-registered adviser. Crypto trading provided by Zero Hash LLC, licensed by the NYSDFS. Generated Assets is an interactive analysis tool by Public Advisors. Output is for informational purposes only and is not an investment recommendation or advice. See disclosures at public.com/disclosures/ga. Matched funds must remain in your account for at least 5 years. Match rate and other terms are subject to change at any time.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Molly O'Shea?

  Welcome to Sorcery.

• 0:10Speaker 01

  The potential of photonics.

• 0:12Speaker 02

  The technology to interconnect these GPUs.

• 0:14Speaker 03

  Global payroll and HR platform.

• 0:16Speaker 04

  Eight billion operational data points.

• 0:18Speaker 01

  Scale-ups over startups. You'd rather have a million of something than one of anything. Building the intelligence layer for defense.

• 0:24Speaker 05

  There isn't enough power. There's not enough memory chips.

• 0:27Speaker 02

  We're at the end of physics of what we can still do.

• 0:41Molly O'Shea?

  Okay, amazing. So we're on our final talk for today. Today, we have Steven and we have Adam. Steven, you come from IMEC. Adam, you're from Strike. So we're going to talk a little bit about what comes after the GPU. So let's start with you. Could you explain what IMEC is?

• 0:57Speaker 02

  Sure. I think we're a little bit the most unknown hidden gem in semiconductors. At least unknown for the average Joe. Definitely not unknown for the semiconductor world itself. We're the worldwide leader in chips, but in actually the early stages of building a chip. So hardware is not like software. It's not that you prompt it and suddenly 30 seconds later, something comes out of it. No, it's a process of five to 10 years. And we're at a very early stage of that process, where we have a unique clean room facility that nobody else in the world has. And so all the biggest chip providers in the world actually first come to us, first in Belgium, to kind of design that next generation chip that you'll see maybe in seven, eight to 10 years from now.

• 1:40Speaker 02

  And going through a whole phase of designing that chip, we hand it over then to them to further commercialize it. But in that sense, there's almost no chip in the world today that has not been touched by IMEC in some way. We already exist for 40 years. And we've been driven more or less that chip roadmap for the last 40 years.

• 2:00Molly O'Shea?

  So I should have asked you the secret question, because you know all the secrets on chips.

• 2:04Speaker 02

  And some of them I'm allowed to answer.

• 2:06Molly O'Shea?

  What can you answer?

• 2:08Speaker 02

  Well, I think we're definitely going to talk more about that. But indeed, that we're actually seeing a huge amount of disruptions going on. I think we're now kind of plateauing in what we call Moore's Law. What does that mean? Chips have been becoming bigger and bigger or faster and faster in the last years. We always did that through scaling. So what does that mean? It means that we made the transistor smaller and smaller. And we're at the level that a transistor has about the depth of one nanometer. What is one nanometer? It's about 100,000 times smaller than a human hair. That means that we're at the end of physics of what we can still do. And I think we're gonna come into an era where what we do in hardware is gonna be, it needs to be way more inventive than what we've been doing in the last 10 to 20 years

• 2:53Speaker 02

  to keep on scaling that roadmap of more to make sure that chips become faster and faster, as they already did the last four decades, basically.

• 3:04Molly O'Shea?

  Adam, as you look into the chip industry, how do you make sense of it? How do you map it out?

• 3:10Speaker 05

  Yeah, I think right now GPUs are very good at the training part. And we're now looking at the bottleneck that sits in the inference layer. And the bottlenecks that I look at the most and try and look at companies that attack those bottlenecks sit in power and energy, memory, cooling, and data transfer. So when we look at the inference side of things, memory, the memory shortage in general, and increasing the bandwidth and lowering the latency of getting the useful memory to a GPU as quick as possible is a really interesting and exciting place that needs to be built within. Cooling as well, and where that's going.

• 3:52Speaker 05

  I think bringing the liquid that takes heat away from the chip closer and closer to the GPU itself is how it's going to progress over the next five to 10 years. And I think a lot of the value will go to the manufacturing side of that.

• 4:09Speaker 05

  And yeah, I mean, power, everyone knows how much of a constraint in a bottleneck that is. And increasing the capacity there is huge.

• 4:16Molly O'Shea?

  I recently interviewed Tony Kim of BlackRock at the Raze AI Summit. It's actually in Paris. And we're talking about hardware. It's like the hardware era, Renaissance, everything's shifting to hardware. Software is getting...

• 4:32Molly O'Shea?

  It's taking a bit of a beating, but the margins... People don't understand, the margins for chips are fantastic. And for hardware versus software, which is getting eaten by AI. And again, like the token spend and that kind of thing. So how do you make sense of the shift in value? And how like the pyramid kind of flipped?

• 4:52Speaker 05

  Well, I think the demand is just so high for the thing that is less fruitful out there, right? So there aren't, there isn't enough power. There's not enough memory chips. The shortage in memory, I mean, if we look at, there's only three companies in the world that produce memory, which is 95% of the market. Micron, SK Hynix, and Samsung. Two of them are in Korea. And a lot of their shift has been filled out by hyperscalers taking over their whole inventory. So you've seen prices rise by 700% this year just for the memory part of the chip.

• 5:34Speaker 05

  And that will happen across the layer because it's the thing that's in the most demand. The training side and the software side of things is continually getting better. But the physical side is the thing that's harder to get better overnight.

• 5:50Speaker 02

  Yeah, to be honest, I don't really think it's necessarily flipped. I think it's more the immaturity of the software market right now. Might seem weird that I say immaturity of the software market, but I think what we've been doing at AI the last four or five years has of course in software been tremendous. But if you look at it from, okay, how complex it is to actually, in terms of computations, do whatever, I mean, large language models can do, it's still very much a brute force approach. And as a result, the value that you can get out of it is pretty limited. And I think if you're going to get further, we're going to see new types of software evolutions as well, which are tied way more closer to hardware. And because of that, I think the value will increase way bigger. And I think so, we're kind of

• 6:34Speaker 02

  filling the base of that pyramid already in hardware of creating a lot of value there. I think software will follow in that sense.

• 6:41Molly O'Shea?

  We're seeing more and more chip companies founded every year, probably than ever before. And venture dollars are definitely helping fund that. Is there a capacity to how many chips can exist in the market?

• 6:55Speaker 02

  I think right now, there's kind of endless demand for that. And I think it's still going to be the question of how that can continue. I think it's rather, the bottleneck is manufacturing. Manufacturing the right type of components in different regions. And I think, for example, what Tema Foundry is doing there is super important. But having the right type of components for that is, I think, the real bottleneck. Not necessarily the number of companies that do this, because every company can come with a completely different, unique design. And in that sense, can also kind of revolutionize what we're gonna have as well. And probably in the software world, we're gonna see way more different types of algorithms.

• 7:36Speaker 02

  And as a result, we will also need more diversification in the type of hardware. So more than just GPUs as well.

• 7:44Molly O'Shea?

  As models get bigger and bigger, what breaks first on the chip layer?

• 7:49Speaker 02

  Well, yeah, I think for sure right now, memory. Memory is absolutely the number one component that is already broken today. But I don't really think that models will become necessarily bigger and bigger. If you just think about it, the latest open source models like KIMI K3, they already have more parameters than what the human brain has. But on the other hand, that K3 chip, or the chip that we need to fuel that K3 model is about a million times more or less energy efficient than what the human brain is. And so I think we're gonna see a different type of paradigm.

• 8:30Speaker 02

  We kind of went into an era the last five years of building bigger and bigger and bigger models. We went from 175 billion parameters to kind of 10 trillion parameters right now. That's gonna stop. And there's a lot of companies right now taking that shift as we speak, abandoning what we call the scaling hypothesis of building bigger and bigger models. And therefore trying to solve those memory bottlenecks as well.

• 8:58Molly O'Shea?

  Copper has kind of met its days. I'm curious, how do you think about photonics and how that changes the layer?

• 9:05Speaker 02

  Yeah. I mean, feel free to...

• 9:08Speaker 05

  Yeah, I mean, I think there are so many benefits to using photonics instead of copper. And copper is easy because it's all in electrical, it's all digital. And the bottleneck with photonics is changing between optical and the electrical layer. But transferring data more quickly and more efficiently and not having that latent heat aspect that electrons in copper wires do will only continue to scale the efficiency of data transfer. And then I think we'll see the architecture continually shift for where photonics will be within data centers. At the moment, it's between the clusters and we're using linear pluggable optics.

• 9:53Speaker 05

  But I think in the next five years, we will see these optical components get closer and closer to the actual GPU. And we will go towards a state of co-packaged optics.

• 10:03Speaker 02

  So for me, I mean, every day what I do actually at IMEC is thinking about what could be AI in the future. So how is AI going to evolve in like five to 10 years from now? And I think we're seeing kind of from a software perspective that very much diversifying path of a lot of different approaches, whether it's world models or more reinforcement learning based approaches. But what is 100% clear is that whatever scenario in the end will become the dominating model of the of the future. Moving data as fast as possible in the quickest way with the huge amounts of data that we need to shift is always going to be what we need because again towards the analogy of the human brain.

• 10:48Speaker 02

  That's what our human brain is also unbelievably good at. We have an unbelievable 3D structure that is able to do this. And so photonics will play, I think for me, especially in the next years, it's going to be duck killer technology to enable the chips of the future.

• 11:04Speaker 05

  I think also on photonics, it's attacking many parts of the bottlenecks that I touched on at the start. It's attacking energy because it can transfer data quicker. It's attacking cooling because you don't need as much cooling around these GPUs if you're transferring through light. And then it's attacking data transfer at the core. So I think having spanning most of the bottlenecks that we're seeing in the AI for build out right now. It's like the key technology that is likely to win.

• 11:34Molly O'Shea?

  As we close out, and this is the last talk of the day, so thank you all very much. What is your hottest take right now?

• 11:44Speaker 02

  I think we're gonna, five to 10 years from now, we're gonna laugh at how old fashioned AI was today. That's a bit my hottest take in the sense that, of course, we went through this explosion of what AI was bringing us, but I fundamentally don't think that today's technology, that this is what we're going to be talking about in 10 years from now. I think an actual also in software revolution is coming and

• 12:10Speaker 02

  for those who are old enough, we still might remember the the internet days of the dial up connection where we had all these bleeping sounds to make connections to the internet and we kind of laugh at how that's how we did it in the back of those days. I think in 10 years from now, we're going to talk about large language models, which are very much a brute force approach in exactly the same way.

• 12:30Speaker 05

  I might be a little bit biased on this one, but I would say it's a very good idea to recruit or invest in people that have had a academic and technical background in Europe and had the wherewithal and business sense in the US and combining those two mindsets. A lot of the best founders that I've backed and I've seen have that dichotomy between the two. So I'd say both recruiting and investing those two traits.

• 13:04Molly O'Shea?

  Well, Stephen, Adam, thank you very much. I think Louis is going to come up next to say something.

• 13:16Molly O'Shea?

  Hey, it's Molly. If you enjoy our interviews, check out our newsletter sorcery.vc where we deliver a once a week top deals and tech headlines email and also go deeper on our podcast interviews. Subscribe to Sorcery today and don't forget to subscribe to the podcast on YouTube, Spotify, Apple or wherever you listen. Link in description to sign up.