The AI Gold Rush Is Out of Gold | Here's Where VCs Are Still Investing

Data center economics are coming under increasing scrutiny. Learn why power constraints could signal a major market correction and why the future of artificial intelligence may depend less on software and more on power grids, data centers, semiconductor chips, and the physical infrastructure powering AI. In this episode, TDK Ventures Investment Director Ankur Saxena joins Geoff Nielson to talk about the economics driving today's AI race. From Nvidia GPUs and hyperscaler spending to semiconductor shortages, AI infrastructure, robotics, edge AI, and quantum computing, Ankur explains why the biggest challenges facing AI aren't just technical, they're economic, physical, and increasingly difficult to ignore. This conversation examines whether today's AI investment boom is sustainable, what could trigger the next wave of market corrections, and where the greatest opportunities lie for investors, startups, and technology leaders. If you're wondering where AI is really headed and what it means for business leaders, investors, CIOs, and the future of technology, this episode is for you. In this episode: 00:00 Intro 00:20 AI's biggest problem isn't software, it's infrastructure 01:37 Ankur Saxena on the future of AI & why he's optimistic 03:20 Is the AI investment boom becoming a bubble? 07:40 AI prototyping for enterprise leaders 08:50 The AI Gold Rush: Are investors chasing hype? 10:17 Why AI infrastructure economics matter more than ever 12:09 The hidden bottleneck: Power, data centers & AI growth 15:45 AI data centers explained 19:53 Supply, demand & what happens when AI gets expensive 21:03 Why AI demand isn't slowing down anytime soon 26:33 Should every enterprise care about AI infrastructure? 27:15 Why most companies aren't ready for AI compute 32:08 The next billion-dollar opportunity: AI infrastructure orchestration 34:22 Where smart investors are looking beyond AI applications 37:35 Edge AI explained: Why AI is moving onto devices 40:43 Why AI chips need hardware & software co-design 44:18 The race to build the operating system for Edge AI 48:33 Self-driving cars, robots & the future of physical AI 52:08 Quantum computing: Beyond the hype 55:25 Quantum computing investment opportunities & "picks and shovels" 58:44 Practical AI advice for CIOs, business leaders & enterprises 1:02:02 Will AI replace humans? The future of human-AI collaboration 1:04:42 Final thoughts Connect with Ankur: LinkedIn: https://www.linkedin.com/in/saxenaankur/ Our links: Visit our website: https://www.infotech.com/?utm_source=youtube&utm_medium=social&utm_campaign=podcast Follow us on YouTube: https://www.youtube.com/@InfoTechRG

• 0:01Speaker 00

  I think the economics of it is not very clear at this point. So where the dust settles is to be seen. But that is the challenge that will probably start a domino effect of correction over time. As the power acquisition stalls, as the many states in the US and other countries globally push back on these mega projects to deploy data centers, hyperscalers will push out these $200 billion cap cycles out versus deploying all of that money each year. And that will then have a trickle down effect on their revenue, their bookings and their quarterly performance, which will then start the stock correction.

• 0:42Speaker 01

  This is a show about the future of tech and the future of work. I'm Jeff Nielsen, and today we're diving into the physical infrastructure shaping AI. From compute and microchips to power to robotics, edge AI and industrial systems. My guest today is Ankur Saxena, Investment Director at TDK Ventures. Ankur has well over a decade of experience in venture capital investing in AI, including a key role at Cisco Investments and has a deep understanding of the entire AI ecosystem. I've found that the layers of AI beyond software are completely underexplored. So I want Ankur to take us on a deeper dive through the whole stack. How should we be thinking about the technology holistically?

• 1:24Speaker 01

  What are the implications on our current AI boom or AI bubble? And does it even matter at all for people living outside of big tech? Let's find out.

• 1:36Speaker 01

  Ankur, thanks so much for joining today. Really excited to have you. I'm curious, just before we kind of dive into things, from your perspective, you know, I wanted to hear a little bit about your outlook on the future of AI, the future of technology over the next handful of years. Where do you see this space evolving to? And what has you most excited and concerned about the current environment here?

• 1:58Speaker 00

  Yeah, Jeff, happy to be here. First of all, thank you for having me. Uh, future of AI is pretty bright. I've been monitoring this market, actively investing in this market for more than 10 years now. And we are what seems to be like the second innings of AI. The first one was more around the chips and infrastructure. Now we are seeing a way more secular adoption and the conversation has shifted from just startups talking about AI and the impact of AI to governments, to enterprise customers, to consumers like you and me actively using AI in day to day life.

• 2:35Speaker 00

  Having said that, the current investment cycle seems to be a little inflationary, a hype dot kind of cycle. So we can, we can talk about that. And like a lot of other technologies, humans tend to overestimate the impact of a technology in the short run, but underestimate it in the long run. So we can maybe unpack both sides, but I feel like it's an exciting time to be investing in AI and also to be as a consumer of AI. It's a very, very exciting time. I think we're seeing a significant step up in productivity gains leveraging AI. So there's a lot of promise and I'm particularly very excited about this space.

• 3:17Speaker 01

  I'm, I'm interested in, you know, the fact that you're describing the investment environment as sort of inflationary here and, you know, that we're maybe even over investing in it. Not because I disagree with you. I think that's probably right because it's interesting to me that even on the investment side, being as close to it as you are, that you're noticing this. And so, you know, what are the ramifications of that? Where does the investment environment go from here? Do you believe we're sort of over invested in the space? And, you know, what does that mean in terms of what you look for and the investments you're looking to make?

• 3:51Speaker 00

  Yeah, I think on the positive side, it's actually good that a lot of investment is going in this space because we do need all the kind of capital support that we can provide to entrepreneurs who are really pioneering innovation in key building blocks of AI, whether it's software, hardware or both. But there is a lot of crowding that is happening, right? A lot of VCs who have never really invested in deep tech areas, hard tech areas. They are now investing in AI because they don't want to miss out on what is the next gold rush. Right? They want to play an active part. They want to have an investment exposure.

• 4:32Speaker 00

  So this kind of uninformed or less informed capital that's in getting invested in the market, that is the thing I have issue with, right? More capital is one thing, but what I'm also seeing is in a king, kingmanship kind of, you know, effort that's going on from a lot of VCs where typically they do not invest in AI or at least in the AI infrastructure layer. But now they are deploying hundreds of millions of dollars of capital in companies at a very room stage. And the round sizes have just gotten out of hand and it's become really big.

• 5:13Speaker 00

  And a lot of companies are actually raising significant capital without actual product clarity, let alone productivity. And the valuations are also just getting out of hand, right? People think that this AI investment super cycle will lead to significant exit outcomes. We're not talking about billions or tens of billions anymore. We're talking about hundreds of billions or trillions of dollars of exit thesis being underwritten by, by a lot of investors. So they are looking at these as compounding platforms, your sort of next generation Googles or Metas or Amazons in the making.

• 5:54Speaker 00

  And with that kind of lens in mind, they are investing a lot of money at very early stages, which is actually, I would say, doing the contrary work of instead of supporting the ecosystem, it's actually crowding out a lot of the other players, other, other startups that may not have access to the same capital, but might still be doing fundamentally, you know, reasonable work and, and might be actually innovating. So this, this VC strategy of concentrating capital in a handful of companies just because, you know, they look promising or maybe it's their, you know, second venture or third venture. That is not great for the overall ecosystem. So I'm a little concerned about that.

• 6:38Speaker 00

  And this inflationary environment again, has ramifications in terms of how many such companies can actually exit, you know, exit at tens of billions or hundreds of billions of valuations. Right? Is the public market, is the acquisition market really, you know, prepared for those kinds of liquid events? Probably not. So over the next few years, there will be some sort of correction in the market. It may start in the public markets, right? Which is another, another, you know, kind of discussion topic in itself, right? Public markets have also seen the AI stocks, the memory stocks have, have, you know, risen a lot over the last 12, 18 months, right?

• 7:21Speaker 00

  On the hoax of this multi-decade AI super cycle. Is that all really justified in the next few years? Or is there a lot of euphoria and hype that's kind of driving these valuations? That's, that's another kind of point that's, you know, debatable.

• 7:40Speaker 01

  If you're a technology leader, Infotech is a name you need to know for getting ahead with AI. Infotech helps leaders deliver on top technology initiatives. And one of our most popular programs right now is AI prototyping. We've helped hundreds of organizations design and build agentic AI prototypes using our step-by-step methodology. Infotech's road-tested approach gives you a trusted blueprint to execute faster and avoid costly mistakes. And what you walk away with is a prototype tied to real business outcomes, integrated with your systems and built with the right guardrails from day one. What sets Infotech apart is how we deliver. Unlike traditional researcher AI tools, we don't stop at guidance.

• 8:25Speaker 01

  Infotech works with your team to build the prototype. You can do this through a focused one-week workshop, ongoing analyst support, or by using our full methodology and toolkit to get it done yourself. The outcome is a working prototype and a team that's ready to scale it. To learn more about our agentic AI prototyping, check out the link below and don't forget to like and subscribe.

• 8:50Speaker 01

  You know, Ankur, from my perspective, one of the trends that we're seeing, as you mentioned, is there's this sort of gold rush around this sense that this is a, you know, once in a generation opportunity to, you know, shift the environment and create something of value. And to your point about the public markets and this sort of, you know, for lack of a better word, gold rush mentality, it feels like there's there's sort of this greed here as investors, you know, unsophisticated investors look for a place to park their money where they think they're going to get outsized returns. And one of the themes I've seen is it seems like as this market and these technologies mature,

• 9:32Speaker 01

  it's becoming obvious that some of the usual suspects are no longer in a position where we can really expect them to have, you know, 100x growth. And investors are starting to, you know, very quickly and, you know, almost with minimal due diligence, turn over rocks and ask like, well, what is the next big, you know, scaler here? And whether that's looking in, you know, semiconductors or microchips, hardware, software, just kind of this frantic search for, you know, crazy returns. Is that your read as well? And how do you see this environment playing out? Is there necessarily going to be a correction here as investors realize, okay, that there's probably not a silver bullet? Or do you still believe there are silver bullets to be found?

• 10:17Speaker 00

  Yeah, no, there are no silver bullets, first of all, to just kind of address that point right away. Fundamentally, AI market is a really big growth area, but are the valuations justified in terms of actual revenue delivery over the next three to five years? That is questionable, right? The kind of massive infrastructure rollouts, the CapEx, you know, expenditure of these, some of these big hyperscalers that are spending close to $200 billion each year, building data centers all over the world, and particularly in the US. Is all of that going to generate economic returns for those hyperscalers?

• 10:56Speaker 00

  That is questionable. Having said that, they're also in this prisoner's dilemma of, you know, they have to actively invest in the AI infrastructure. If they lag, if they fall behind, then they just basically, you know, fall behind significantly. And then they lose the market share, then they lose the investment dollars that are going in the public stock, right? So there's this prisoner's dilemma. Everyone has to keep on betting more and more capital expenditure money into AI infrastructure, which is supporting all these venture capital investments, because if Google, Meta, Amazon, and others are going

• 11:38Speaker 00

  to continue to deploy $200 billion of CapEx, where is that CapEx going? That is going in buying power infrastructure, semiconductor chips, data center networking, and the whole sort of fiber rollout, and then the software layer and GUP. All of that is getting funded from these approximately $2 billion per hyperscaler kind of annual CapEx, right? So that is kind of the the whole trickle down effect of how this hyperscaler investment cycle is driving the venture investment cycle in this whole gold rush theme. Now, where the disconnect is, the physics of it, right, or the real estate of it, right? The

• 12:16Speaker 00

  physics aspect is, we only think about ChatGPT, Claude, all the AI investments or AI innovations happening in software. But where the real bottleneck is, is the physics of it, right? The current GPU infrastructure is not the most power efficient, energy efficient, cost efficient infrastructure to be deployed at scale, right? It is being deployed right now because it is the only chip that is available in the market at scale. But it is not the most economically efficient infrastructure or compute infrastructure, right? So the physics is the bottleneck. And where that physical constraint will come into picture

• 13:00Speaker 00

  is the power acquisition, the power delivery, right? These hyperscalers, wherever they're building these big data centers, they need access to cheap and sustainable power. And a lot of regions in the US have started opposing the use of their natural resources, their power grid to power data centers. I think the economics of it is is not very clear at this point, where hyperscalers are claiming that it will lead to economic boom, more jobs in those regions, but it'll likely come at a cost of, you know, more pollution of natural resources, water and air, and also, you know, increase in power costs for

• 13:40Speaker 00

  the the local consumers there, right? So where the dust settles is is to be seen. But that is the challenge that will probably start a domino effect of correction over time, right? As the power acquisition stalls, as the, you know, the many states in the US and other countries globally push back on these mega projects to deploy data centers, you know, Nvidia's GPU chips will, you know, the the demand for it will reduce, right? Hyperscalers will push out these $200 billion CapEx cycles out versus deploying all of that one each year. And that will then have a trickle down effect on their revenues,

• 14:20Speaker 00

  their bookings, and their quarterly performance, which will then start the the stock correction. The other thing is the chip supply, the memory supply, that is also severely constrained, right? Memory prices have more than quadrupled in the market. You know, Apple and others have have decided to pass on some of those increased memory costs to consumers in the form of, you know, price increases for iPhones and tablets and other products. I think that is not very sustainable, right? The consumers will only absorb so much cost, right? So I think that the chip supply, the memory supply, all of all of these physical products, you know, their supply is constrained, their actual delivery is constrained, their actual

• 15:02Speaker 00

  deployment is constrained, that will bottleneck the the hyped up growth plans of hyperscalers, right? And that is how the whole connection cycle will begin at some point.

• 15:14Speaker 01

  You mentioned, you know, at some point feels like it feels like a given these days. And you mentioned that, you know, you're not exactly sure where the dust is going to settle. But, you know, can we maybe speculate a little bit about what are some of the scenarios about how this plane plays out, regardless of exact timeline? What could this look like in terms of the the impact of that constraint and just kind of the downstream impacts in this entire, you know, technology and investment ecosystem?

• 15:45Speaker 00

  Yeah. So think of AI infrastructure or these data centers as AI token factories, right? These are modern factories where raw inputs go in and inference, you know, intelligent inference comes out, right? And like I said earlier, they are bound by the physics of the chips, the power, the thermals, they interconnect, right? Deploying data centers at scale, running these ChatGPT Claude models at scale, where you're powering millions of inference queries every second, is sustainable in the long run only if the economics of this factory works, right? And these factories are not running at at the right kind of optimal

• 16:29Speaker 00

  level right now. The the hundreds and thousands of GPUs that are deployed by the hyperscalers are, you know, getting powered on, but they're not actually getting used optimally. Now, I've read that utilization rates of a lot of these chips that are deployed in data centers is below 30%. So for the remaining 70%, they're consuming a lot of power, they're getting heated up, but they're not really optimally driving any inference. So it's like you're running a factory at only 30% utilization, right? Over time, investors, whether it's public market investors or in private markets, you know, VCs, will start demanding that there be a sustainable way where, you know, people will start talking about what are

• 17:13Speaker 00

  the margins on these data centers, right? Just the the money going in and deploying these data centers is not enough. What is the payback, right? What are the margins of operating this at scale? These conversations will start to permeate the public market and the private markets more and more. And that is when I think this brute force scaling of just deploy more GPUs at whatever cost, that brute force scaling will have to stop. And people will think about the real economics of running these factories, right? This is like the steel mills of 100 years ago, right? Steel mills were started near coal mines and rivers because they wanted access to water, to coal, and railroads. Data centers

• 17:55Speaker 00

  are also getting started around the power sources and and around, you know, chip availability, right? But how do you actually operate it at scale? There was only one US steel or, you know, few steel companies that survived, right? Out of maybe 100 others that were started a century ago. Economics was the the factor that basically was forcing the survival of the fittest, right? And that economics is not something that's actively debated right now, actively analyzed right now. I've started to think about the economics of this token factory in the form of five Ps. These are performance, power, price,

• 18:40Speaker 00

  privacy, and programmability. People have, so far, these hyperscalers, everyone in the chip industry is mostly focused on performance, right? I can drive so much inference per second or per watt. Power is the next gating item that people have started to realize. And it's a boardroom conversation right now in a lot of public companies that are actively deploying AI infrastructure, right? How do we get access to power cheaply, affordably, and sustainably? But what is also not talked about is at what price? And are we impacting the privacy? And programmability refers to how do we actually orchestrate this this whole AI token factory so that we drive up the economics, the margins, right? So the

• 19:24Speaker 00

  other three Ps will will get talked about more and more. And those are the factors that will be used to scrutinize the operations of these big infrastructure vendors at scale over time. And I think once they start to falter on some of these, that's when I think people will start to settle or the markets will start to settle into more sort of rational view on valuation, long-term growth prospects, and long-term CapEx cycles.

• 19:52Speaker 01

  I really like that model and I like the I like the kind of the economic lens on all of this as well. And so keeping with the theme of economics and we can keep it kind of 101 here, you know, we've got a supply constraint and we've got a significant amount of demand. I'm curious, Ankur, from your perspective, you know, clearly we've got something that's unsustainable from a pricing level here, right? Because there's price subsidization going on. You know, there's performance increases, but there's, you know, a limited amount of supply. In terms of like the next stages here, is is demand in your mind going to stay steady or increase to a degree where it forces an increase in the amount of supply?

• 20:35Speaker 01

  Is demand going to soften? And, you know, one of the interesting things about supply here, and you know a lot more about this than I do, I'm sure, is, you know, it's it's not that easy to just spin up a new AI infrastructure, right? To come up with something. The the lag time on that and the the upfront investment is significant. So using that economics lens, how do you see that playing out?

• 21:01Speaker 00

  Yeah.

• 21:03Speaker 00

  Yeah, first of all, I agree that it's not easy to spin up a data center. Right? That is why a lot of these forward looking investments are happening by hyperscalers because, again, back to the prisoner's dilemma point, they don't want to be the ones whose, you know, inference latency is several milliseconds more than their competitors, right? They want to serve all these inference queries at the best possible throughput and and other criteria. That's why they're investing ahead of the curve. Now, talking about the curve, the demand curve, I feel like demand curve will continue to grow for the time, you know, for the foreseeable time

• 21:42Speaker 00

  because right now it is not constrained by costs as much, right? Because the costs are being subsidized by the vendors because they are in a land grab situation, right? OpenAI, now we've talked about charge, sorry, about infrastructure vendors. Now, let's talk about the model vendors, right? Model vendors are in a land grab situation themselves. They cannot differentiate purely based on the frontier models that they're building. They are going to sustain their growth, their scale through retention and through the strong value proposition of their their models, which will really come with

• 22:24Speaker 00

  metrics like, can this serve these models at the lowest latency of the cheapest cost, which also has infrastructure ramifications, but then they're also doing a lot of the application stack themselves, right? So far, I've started to see enterprise customers spend millions of dollars each month on running inference workloads. That, I think, is already significant expenditure for a lot of enterprise customers, but we've barely touched the tip of the iceberg here, right? I think there is so much more potential, first in the digital realm, and then over time in the physical world, the physical realm of how AI can actually impact real physical world interactions.

• 23:09Speaker 00

  So I feel like that is the optimism that's driving all these investments by both public market companies as well as venture capital firms, right? The demand is what's keeping all of this going, right? People are seeing companies built like never before, right? Companies in the coding space, they have attained billions of dollars of ARR when it took seven to 10 years for the the the poster children of the SaaS era to attain billion dollar ARR in seven to 10 years, right? Cursor-like companies have attained it in less than a year. So there is something substantially different in this AI cycle compared to the SaaS cycle of a decade ago.

• 23:53Speaker 00

  That is what is prompting these

• 23:58Speaker 00

  investments, right? These super investments in companies where hundred million dollar rounds are are so common these days, when it was so uncommon in the in the SaaS era, it was a special thing. Nowadays, almost every day of the week, you hear about 100 or 200 million, maybe even 500 million dollar fundraises.

• 24:19Speaker 01

  Do you think, are we at a pace here where you see that continuing, you know, indefinitely into the future, at least for the medium term? Or are we coming close to a reckoning point? And I guess the, you know, getting back to the underlying sort of financial physics of that, are those valuations too rich just based on, you know, investors wanting a taste of this gold rush?

• 24:44Speaker 00

  I think at some point, the valuations will revert to more reasonable SaaS-like valuations. But again, at some point is is indecisive. So I would say, at least in the next few years, I expect this to happen, right? Not 10 years away, next few years. Having said that, like Claude, Anthropic and OpenAI, their scale that they've attained in the last two, three years is just completely unimaginable, right? From the SaaS era, right? Nobody was growing from 2 billion to 10 billion in a year, right? Very few companies attained even 10 billion scale after 10 years, right? But these companies have attained it in two years or three years

• 25:29Speaker 00

  of operation.

• 25:32Speaker 00

  So they do deserve more, I mean, a rich valuation. But over time, the underlying physics and economics will bring their growth rates down. There will be extra scrutiny on not just the growth rates, but on the margins. Are you actually delivering these astronomical growth rates at sustainable margins? Or are you actually just funding your growth rate through these losses and you're just raising money or external investors? I think that that will be the the reality check that these companies will get in the next couple of years.

• 26:07Speaker 01

  I I'm inclined to agree, and it's going to be very interesting to see how that plays out and how, you know, sustainable or not sustainable that is. And yeah, I think a lot of enterprises are in for a rude awakening. Once, you know, the token subsidization, you know, if that starts to go away, they're really going to have to think hard about, you know, their inference costs and their inference usage. I'm curious, though, Ankur, with that in mind, we've talked a little bit about this demand. I want to unpack a little bit more. I guess the shape of this ecosystem. So specifically thinking about the AI infrastructure that we've discussed so far. You've you've talked about the idea of enterprises using inference, being concerned about power.

• 26:53Speaker 01

  The invest, the infrastructure sides behind the scenes, is the demand really just coming from a handful of hyperscalers? Or is this something that most enterprises need to have on their radar in a more fulsome way?

• 27:09Speaker 00

  You mean demand for infrastructure, the the data center, physical infrastructure, right? Yeah, so this is in short contrast to the pre-cloud era, when everything was run on on-premise servers, right? All the big companies had these big closets of servers that were running in their campus somewhere or in their enterprise buildings, managed by IT, right? Then the cloud basically took a big portion of that and, you know, basically made it available as a public utility for anybody to rent that capacity versus managing all these big closets of compute themselves. That was still manageable because

• 27:52Speaker 00

  the underlying chips in those servers were all of the same architecture, the x36 architecture that Intel and Dell and others built. But now, over the last couple of years, we have entered a heterogeneous computing era where it's not just CPUs, there's GPUs and there is Grok and other type of chips and accelerators. And frankly, any enterprise IT is not prepared. They just don't know how to manage these different chips, let alone the whole sort of cooling infrastructure, the power infrastructure.

• 28:26Speaker 00

  You know, some of these data centers that run GPUs, their temperatures spike up into several hundreds. It's like a furnace in there. How can you run that furnace in the bottom, the basement of a building? You cannot, right? The kind of cooling infrastructure that is needed, you can't do that out of a basement in in a big city, right? So it's just not possible to do big hyperscale kind of data centers in enterprise campuses. Having said that, even the hyperscalers are not fully adept. They are gradually acquiring the right kind of talent and the technology to run these data centers at scale. But like I said before, their utilization rates of these massive rollouts are less than 30%.

• 29:10Speaker 00

  So they've just deployed massive data centers, but they're not even optimally using it themselves. And I've not even talked about the software inference optimization that's needed on top of these these physical chips, right? You need an air traffic controller kind of software orchestrator to think about, based on the right workload, what is the right rack or right data center to run this specific workload on, right? Like an air traffic controller decides amongst a couple of flights that are all trying to land in the next five minutes, how do I prioritize who lands at which runway and at what time? That is the kind of role that software needs to play and that software stack is in the works. The hyperscalers are also figuring it out.

• 29:56Speaker 00

  What they built in the cloud era using Kubernetes is not applicable for AI stack. Everything has to be built from from scratch. Almost nothing from the cloud era can be borrowed for the AI infrastructure era. You need specific chips, specific cooling, specific interconnects. Then you need its specific tailored software on top to do all the optimization and drive that less than 30% utilization to more than 70% utilization, while achieving the kind of margins that you were able to achieve during the cloud era. If you fail to achieve that over the next five years as a hyperscaler, you know, your stock will suffer, right? So that's that's kind of why I think that

• 30:41Speaker 00

  most of the demand for AI infrastructure would be

• 30:48Speaker 00

  Concentrated in hyperscaler market. But then there are also these Neo clouds, right? Like Lambda and Core V and others that were started in the AI era. Again, because they did not have the baggage, if you will, of the hyperscaler cloud era, right? They had no cloud before, these Neo clouds. They built these GPU clouds and the software stack on top of it to exclusively serve the AI market. These Neo clouds don't serve your other non-AI workstations. They don't, you can't rent capacity for non-AI workstations. You use them for AI work tools. That's a big emerging market.

• 31:26Speaker 00

  And then there are these massive capitalized AI native companies like OpenAI, Anthropic and other companies that are also building their own data centers. Because they realize that they are spending so much money on their, you know, solid hyperscaler bills that they might as well just do this themselves. So they are in market for chips, for talent, for power, for someone who can run this whole operation at scale, right? And they're doing this themselves. And a lot of the chip companies are now working directly with OpenAI or Anthropic versus working with Meta or Amazon or Nvidia or other companies.

• 32:04Speaker 01

  So you know, you've described fairly clearly that there's this huge opportunity for optimization here, right? You call it air traffic control, sort of a, you know, if I can call it an infrastructure orchestration layer, I guess, between the chips and the actual inference requests here.

• 32:24Speaker 01

  What are the implications there? Who's best positioned to take advantage of that? Is it going to be the chip companies? Is it going to be companies, you know, is it Neo clouds? Is it organizations like OpenAI? Or, you know, in your view, is that field sort of wide open for disruption by, you know, new entrants?

• 32:43Speaker 00

  Yeah, I feel it's the latter. The part that, you know, I feel that if this kind of inference control or inference optimization software were to come from one of the hyperscalers, they would naturally be biased towards their own chips and their own cloud. We need an unbiased Switzerland of orchestrator that can orchestrate workloads across different clouds, different chips, different data centers and racks, right? Basically, whenever there's a big architectural shift, there is a new control plane that emerges in the software layer, right? So there was Kubernetes for the cloud era that virtualized the underlying cloud environments or the on-demand environments.

• 33:27Speaker 00

  There was the VMware hypervisor, right? That virtualized the bare metal. There was also the software-defined networking that virtualized the underlying networking from Cisco and Juniper and others. Now, in this AI infrastructure era, we need the next VMware to emerge, right? We want a unbiased inference control vendor to emerge in the software layer that's able to support a variety of workloads, variety of chips, and all the underlying hyperscalers and data center racks, right? So it's a play. Nvidia house a software like this. But again, Nvidia is a biased party in this.

• 34:07Speaker 00

  And I think right now, for the lack of a better alternative, their software works and their software is being used. But over time, we will see the next VMware in this space emerge, a big company in this space, independently.

• 34:21Speaker 01

  So with that in mind, I want to come back to something you said earlier, which is that you believe there's too much investment potentially being concentrated in a few big players here. And there's, you know, quite a large number of startups that are, you know, could be very well positioned to disrupt the market and add real value, but that, you know, investors aren't necessarily seeing that. So with that framing in mind, where do you see the investment opportunities? And what do you look for when you're looking for, you know, the next AI startup to invest in?

• 34:54Speaker 00

  So each investor has its own set of constraints, fund size, timelines, risk appetite and all that. We are early-stage deep tech investors. We invest when the company are just starting out with their product idea, the teams are getting together. They have a vision, they have a plan. Hopefully, they have some validation or key industry insight that sets them apart, right? And they have to be deep tech, which means, you know, we're not investing purely in AI application companies. We're investing in the underlying physics, the underlying material science innovation companies. And whenever there's a golden rush, we invest in the picks and shovels that help people, you know, filter the sand out of the water and, you know, find the gold that gets there, right?

• 35:39Speaker 00

  So basically, we have invested in the whole building, you know, set of building blocks of the modern data center. And it's the next generation power delivery, generation cooling, next generation set of chips and different architectures again there, which are more power efficient than Nvidia GPUs. We've invested in the fabric or the railroads of the steel mills, right? The connectivity fabric companies.

• 36:03Speaker 00

  So basically, these kind of companies versus a big sort of data center developer or Neo cloud, which is a very capital-intensive play and something that we are not comfortable under the business as an investment opportunity. So that's what we have done on the data center side. Now, the world of AI is broader, right? There is the physical AI component as well, which is how you actually embody this AI in the real world, whether it's in the form of robots or self-driving cars or drones or augmented reality or other forms of physical interaction. That's another key area of focus for us. We've invested in several robotics companies. We've invested in enablers of robotics in the form of compute companies

• 36:47Speaker 00

  whose chips can actually go in these robots and other physical AI machines and help them do inference on the device itself versus depend on connectivity to the cloud.

• 36:59Speaker 00

  And we have invested in also the enabling manufacturing and supply chain companies in the material science side that will enable physical AI and the AI infrastructure to grow.

• 37:12Speaker 00

  So all of these are deep tech investments fundamentally rooted in scientific innovation, started by teams with deep domain experience. These are not college dropout kind of teams, right? This deep tech space needs people who have significant experience, the connections and the insights to bring these hard technologies to market.

• 37:35Speaker 01

  So under the banner of deep tech, we've talked about, you know, that kind of hard AI infrastructure for most of this conversation. But we've also now sort of backed into another category of deep tech, which I don't know if you would describe it as edge AI or edge computing. But, you know, you described the, you know, the computation or the AI calculations happening on the device itself. You know, for listeners who don't necessarily have a good understanding of that space, can you talk a little more about what that looks like, what the advantages are and how advanced that space is right now?

• 38:10Speaker 00

  Yeah. So edge AI is basically running AI inference on the device itself. And those devices could be smartphones, watches, satellites, robots, industrial machinery. A lot of times, these physical devices have to take decisions in real time. And interaction with cloud, that round trip of signal going to the cloud for computation in the cloud and then coming back with a decision that round trip is, you know, very latency-heavy. And real-time decisioning is not possible if you're depending on the cloud. So you need to act in real time on the edge itself. And that's why all the AI-related compute has to happen on the device itself.

• 38:55Speaker 00

  Now, you can't put a big Nvidia GPU, let alone a server on a watch or a phone. So you need a toned-down version of the compute that is exclusively tailor-made, size-wise, power-wise, price-wise to fit into a watch or a phone or a similar device, right? And that's a market that is a specialty market where you do have incumbents like Nvidia, Qualcomm and other companies. But these companies have not kept up with the innovations in the edge AI space. And that is creating a very clear wedge for startups to build very focused set of chips for the edge AI market, right?

• 39:38Speaker 00

  And the space is actually heating up a lot with the advent of physical AI and how you actually enable machines to interact with humans and also do work in real time without depending on cloud. So that space is heating up and edge AI compute is actually one of the key enablers of physical AI.

• 39:59Speaker 01

  I was just sort of smiling there at the phrasing of heating up because where my mind went was you described on the infrastructure side, these challenges where, you know, doing these AI calculations can actually physically, you know, heat up these data centers to hundreds of degrees. And I have to imagine that, I mean, obviously, nobody wants that on their wrist. But that some of the challenges you have with that infrastructure are even more pressing if we're talking about, you know, local computation there. So the need for optimization is probably more important than ever. How does that sort of play out in practice? And, you know, where do you see the space going?

• 40:43Speaker 00

  Yeah. So edge computing as a space has existed for more than a decade, right? People have talked about digital twins, edge computing, both on the telecom side and on the industrial side, and also on the the wearables and devices side. But edge computing as a space has never taken off. But AI at the edge is the killer application that has now, you know, driven a lot of interest, both from enterprise customers, but also from investors to invest in this market. Now, where a startup can stand out in this space is the deep software hardware integration of the code design.

• 41:25Speaker 00

  Edge is an area which is really fragmented. There are so many different device types, right? You have dozens of wearable brands, dozens of smartphone brands, buries of robotics companies, and even more number of industrial machinery companies. You cannot provide the same chip to all these companies, right? You need a chip or a family of chips that is tightly controlled by software to be exclusively tailored and just do the kind of functions that are needed for that particular device. Right? And there is a privacy element of this as well, right? You cannot run some of these computations in the cloud or offload some of these computations per cloud.

• 42:08Speaker 00

  How do you actually do minimization of the latency of the compute? Minimization of the power draw, because most of these devices have battery power too, right? That's the other thing. You don't plug your phone and your watch continuously to power, right? Otherwise, how do you use them?

• 42:24Speaker 00

  They're supposed to be worn and be used on the go, right? So power constraints are really critical here, right? You're drawing power from batteries. So there's a certain envelope of power you can operate in. You cannot go beyond that. Otherwise, you drain the battery and make the device pretty useless, right?

• 42:45Speaker 00

  So software is the one that does a lot of this optimization in the form of when I'm not using the compute power down the chip. When I'm using the chip for a certain workload, only light up certain areas of the chip so that the power draw is optimized versus lighting up the whole chip and drawing more power, right? And then there's other optimizations that are done in terms of making the really complex AI models tinier in their footprint so that they can be run on a lighter footprint chip. You can't run a 60 billion parameter model, LLM model on a watch, right? You need to tone it down to less than 10 billion parameters to be useful and to be run in a power efficient way on a chip,

• 43:31Speaker 00

  on a watch, or a phone, or a robot, right? So all those optimizations are done in software. But what I've seen in the market is a lot of chip companies, they don't think about hardware software design from day one. To them, given their legacy in semiconductor chip manufacturing and their hardware legacy, they focus on building the chip first and software is like an aftercard. But the real differentiator is designing the software and hardware together from day one. And that is part of our investment thesis. We support companies that think about software and hardware innovations and code designs together from day one. And that actually has the potential to unify this fragmented landscape through software again.

• 44:17Speaker 01

  It's really interesting. And to me, I can imagine at least a few different competing design philosophies there. One of them is, you know, you described earlier with your five Ps, like programmability and just creating chips that are a little bit more flexible. The other one is continuing the fragmentation, but just having these really narrow use cases on on types of devices or types of workloads. Do you envision a world where there is more, you know, general usage and programmability of these chips and of this compute? Or do you think it's going to be just more sort of, I guess, like verticalized fragments based on use case?

• 44:59Speaker 00

  I think the real value unlock would be the programmability aspect. If if a edge AI compute company can deliver a really rich software platform for for running models on their chip and maybe even support other other chips, they have the real potential to become the operating system for the edge AI, right? So imagine the company can can let a developer and app developer who's building some sort of personalization app for your watch. If the developer can actually go down to the lower level of optimizing the chip and not just this company's chip, but maybe Nvidia chips or Qualcomm chips or some other chips, then that software ecosystem will be really rich.

• 45:43Speaker 00

  And that is actually the power of the CUDA ecosystem for Nvidia, right? They invested in the Nvidia CUDA ecosystem 25 years ago, probably not thinking that that could be one of their modes. But it is actually the most important mode for Nvidia right now. People can trump their chips, but they cannot trump the huge ecosystem of developers and the applications and scientific libraries that are built on top of CUDA. That is missing from the edge AI environment, right? There is no CUDA in edge AI right now. And there is a potential for a startup to build something like that. And that will be, I think, a significant value unlock if anybody can do that.

• 46:25Speaker 00

  It's an area where Nvidia is not actively focused. It's an area where Qualcomm is not actively focused, or Intel, or other big chip companies, mostly because of the fragmentation. But it's also a chicken and egg problem. You only build a developer platform if there are developers, right? We haven't seen a lot of developers focused on edge AI. So if you wait for developers to come, you don't build the developer ecosystem, the SDK and the platform. And then there's not a good set of, you know, libraries. So it's like a chicken and egg. But I think if if there is a visionary founder or founding team that can build something here, this is a key investment area.

• 47:08Speaker 00

  We have a company in this space called Edge Quartics that actually has the software hardware code design methodology in their ethos. They were actually a software company from day one, and then they built their own edge AI chip. And they are trying to be that operating system for edge AI compute. But it's a massive undertaking. We believe in them and wish them, you know, all the good luck, but it's the same big undertaking.

• 47:36Speaker 01

  I'm just, I'm going to dramatically oversimplify so much of what you said here by just wrapping it under the banner of a lot of this, especially in the context of edge AI, sounds really hard, right? Like, these are just, these are big meaty problems, and you have to get a handful of things right for it to work well.

• 47:56Speaker 01

  From a, I guess, from a device or a robotics manufacturer side,

• 48:03Speaker 01

  if you can avoid edge AI, you may try to, to just say, okay, well, we're going to try to avoid that because it's too hard to get right. So we're going to use more of the traditional models. Where do you see the use cases where edge AI is really going to be the biggest differentiator from, I guess, like a product or a device perspective? Like, what can we unlock? And if we get that right, and what are the manufacturers that are most likely to going to be looking at that?

• 48:32Speaker 00

  Yeah. So self-driving cars, they cannot depend on cloud connectivity, right? The round trip to the cloud to make a decision of, you know, whether you want to yield to the traffic or actually go into the traffic, that is, you can't depend on the cloud, right? You have to make that decision in the car itself. So these self-driving cars have Nvidia GPUs because they obviously have much bigger batteries and they can power the GPUs and all that. But if you take that same kind of immediacy of, or the real-time aspect of decisioning, there are several other use cases where you cannot depend on a wired connection to a big server or a, you know, a wireless connection to the cloud.

• 49:16Speaker 00

  Things like drones, things like robots that are operating in environments where they cannot be tethered to a server. Right? So when training a robot, a lot of the companies actually tether them to a server and while they're loading the inference model on the robot. But when it comes to inference, which is the use of that model in real-time decisioning, they, you know, these robotics companies have to depend on the, on more chip to make the decision. Because if the robot is tethered, well, it can't really do much, right? If it can't move around much, can't really do a lot of work. And a lot of times, these environments are very, you know, hazardous. So you can't really operate a server and the wiring and all that.

• 49:59Speaker 00

  So, yeah, I mean, and think about satellites. There is no concept of wires, can't be tethered to anything, you're a satellite. So satellites, spaceships, drones, robots, self-driving cars, wearables, right? Again, you need to do inference on the watch itself. Like, next generation of unlock for Apple Watches will not come from fitting more sensors. But how do you actually make sense of the data that that sensor is, you know, measuring and emitting as data stream? You have to embed a local model that a chip will run on the watch itself to give you real-time measure of a bunch of, you know, bio variables and things like that, right?

• 50:44Speaker 00

  So I think the unlock will come from embedding compute next to the sensors. And for that, there will be innovation, both on the software side, as models become tinier and tinier. And they basically filter out what is not necessary for a core inference operation to run. And they basically, there is a method called distillation. So you start with a much bigger model, 20 billion, 30 billion parameter model. You distill it down to 5 billion or 1 billion parameters by stripping away things or variables that are not important for inference, right? So there is software-related optimization that will be done in the model layer. Then there will be software-level optimization done in terms of how you actually run a model on the chip.

• 51:29Speaker 00

  And there will be chip-level optimization done in terms of how do you actually light up only a certain portion of the chip? How do you miniaturize the chip? How do you kind of densify the compute with more transistors and other optimizations on memory? So there are a bunch of innovations happening here. And I think it's inevitable for physical AI, for edge AI to really work. The compute part has to be done on the device itself.

• 51:56Speaker 01

  Got it. It's exciting. And that gives me a nice sort of framework for understanding, you know, what that might look like and the types of devices. I want to shift gears a little bit, and I know you talked about deep tech. I understand one of the other areas, you know, you look in and you're planning for, investment-wise, is quantum computing, which has been sort of a hot topic for a number of years. And I guess just to set the table a little bit, I'm most used to hearing about the impact of quantum, from a cybersecurity perspective, and post-quantum cryptography, and what happens when quantum suddenly breaks all your, you know, passwords and codes and, you know, you're living in a non-secured world.

• 52:37Speaker 01

  So maybe I can start by asking, what do you see as being the biggest impacts that quantum will have, you know, at its onset?

• 52:47Speaker 00

  Yeah. I think I would like to first clarify that quantum computing is only starting to see some real traction now in terms of actual deployments and, you know, actual applications. Security is definitely one area where national governments of several countries have made it a national policy to mandate the use of post-quantum cryptography or quantum key distribution, QKD in China, for example, and EQC in US, Lurk in Japan, as an example, as kind of protective measures for banks and other government institutions to deploy before we hit the Q day. Q day is when, you know, it's like the D day, right?

• 53:31Speaker 00

  When quantum will be able to break these existing encryption algorithms, right?

• 53:38Speaker 00

  So, so far, we've seen a lot of these deployments mandated by governments are being proactively taken as, you know, protective steps by banks and other government institutions, like defense institutions. Other than quantum security, quantum sensing is another area where we've seen active deployments of quantum sensors. And I would also like to clarify that people think quantum will basically completely replace AI and the classical compute, that's not going to happen. Quantum is just another modality, another compute paradigm in the overall computing to do blocks. It's not going to replace, it's going to work in tandem with your classical compute of AI,

• 54:21Speaker 00

  GPUs, CPUs and other, you know, chips that are like you talked about. And everyone, all these computing modalities will coexist. Quantum will only get deployed for certain scientific discovery, chemistry, weather forecasting and some other security-related problems or applications. It's not going to take over the world, it's not going to replace everything.

• 54:44Speaker 01

  I think that's a helpful framing device and a helpful reminder. So, you know, I tend generally to shy a little bit away from quantum because, as you said, it's a little bit more niche and it's a little bit of a crystal ball exercise to try and figure out exactly when Q day is going to happen. And, you know, when organizations will actually be dealing with this in a practical way. But I've heard one of the phrases you've used in this space is enabling technologies and starting to look at enabling technologies around quantum and, you know, some preparation there. What in your mind does that look like? And what sorts of organizations should be starting to investigate and adopt enabling technologies?

• 55:25Speaker 00

  Yeah. Whenever there is a big architectural paradigm shift, there is an unbundling of the tech stack. That's what we saw in the cloud era. We are seeing now in the AI infrastructure era where startups are starting, they are funding, other investors are also funding specific building blocks of the infrastructure. In quantum, we are kind of entering the second innings of quantum computing, where the first innings was all about heavily capitalized, full-stack plays, where IBM, Google, Microsoft and some of the heavily capitalized startups were doing everything in-house, whether it's building the actual quantum computer, the cooling infrastructure,

• 56:06Speaker 00

  the software to control there, like error qubit correction, error correction and all that. Everything was done in-house. But now the market is starting to grow and that gives enough fill-in to startups to start and focus on specific tooling or specific enablers of the quantum computing ecosystem. So, again, like the AI infrastructure gold rush, at some point, there will be a gold rush in quantum computing, where selling the building blocks or the picks and shovels of the gold rush movement would be a viable investment strategy. So I have started to become really excited about specific enabling technologies like cryogenic,

• 56:51Speaker 00

  like the coolers that actually, you know, help these quantum computer companies operate at these really low Kelvin temperatures, right? So that's one area. Another area is the whole control circuits, like data center. Quantum computers will also need these control circuits and control electronics. And a lot of the AI infrastructure control electronics can actually work with quantum computing at home temperature, but if they have to operate at really low Kelvin temperatures, then there is a lot of engineering problems there to solve for. So that's some, there's some innovation there that startups can actually focus on and derive value from. Then there is the whole error correction piece, right?

• 57:33Speaker 00

  The initial innings of quantum computing was generating qubits. But there is a lot of errors in those qubits, right? How do you correct for those errors to make quantum computing actually reliable and be reliably deployed at scale? That is the next gear. So we are kind of moving from the scientific era of quantum computing to engineering era of quantum computing and making these lab projects be deployable at scale. So now engineers come into the picture, right? And you have these specific tools and technologies that these quantum computing companies will need, which will be a good target market for all these picks and shovels companies. So that's where I'm focusing my time going forward in quantum computing.

• 58:18Speaker 00

  I'll be looking at these fiction shovels and investments.

• 58:22Speaker 01

  It's very cool and it's fascinating that, you know, to follow the picks and shovels analogy. They're very different picks and shovels even from the, you know, the AI era. It seems like it's completely different and somehow feels even more, you know, science fiction than anything that's come before us. I do want to switch gears a little bit, though, Ankur, and, you know, put on my, you know, business leader or technology leader hat. So for leaders in kind of the long tail of organizations, not in a hyperscaler, not in the, you know, frontier labs where probably developing and, you know, actually engineering the quantum tools themselves.

• 59:04Speaker 01

  In the entire space of what you described, deep tech, what do they need to know? What should they be focused on? And what, if anything, should they be, you know, doing differently or looking at adopting that's going to help them get ahead?

• 59:20Speaker 00

  Yeah, I think first thing is to to stop worrying about AI replacing them, right? That's probably not going to happen for most jobs, at least for the foreseeable future. I think the best way to to get ahead of this AI revolution is to start using these AI tools actively, right? Whether they are mandated by your your organization or not, I think being curious, being active in using these AI tools is a worthwhile exercise to kind of stay updated on how these technologies work. Um, and then figuring out ways to actually use them to to amplify your productivity. I think that's that's really important. Uh, I think that has already started in the the software world where a lot of programming

• 1:00:04Speaker 00

  and the documentation and the QA work is not AI driven, right? And there are other pockets of other functions like legal work or or resume scanning or, you know, things like that, like content writing, a lot of that is starting to leverage AI. But when you talk about, uh, again, I'm kind of shifting gears from digital AI to physically AI. When you talk about a plant operator, he or she who's running a plant, manufacturing or some other, you know, kind of physical operation, how should they think about AI? I think to them, again, it should be outcomes based. Can AI help me get to my outcomes in a more efficient manner?

• 1:00:44Speaker 00

  And by efficient, I mean, I mean, I've invested in companies that cut down analytics times for plant managers from three months to three days, right? If you can, if you can only only analyze one gigabyte of data in an Excel file, that is severely limited. These plants, these refineries, the manufacturing operations generate gigabytes of data every second, right? If you can use cloud and AI tools to actually analyze and correlate data from different sensors and manufacturing lines, and that helps you become more efficient as a plant, why not go for it, right? Like, do it. Um, it's not going to replace anybody's job. It's actually going to make the plant more efficient and more worthwhile as an investment

• 1:01:27Speaker 00

  for the overall organization to continue investing in your plant, right? So, um, yeah, I think using AI, diving into AI, I think that's that's the kind of advice I can give at this point. And then obviously AI will have different different kind of timelines of impact on different departments, different key role functions.

• 1:01:49Speaker 01

  That's great. Just, uh, just, Ankur, before we move to wrap up, is there anything we didn't cover in this conversation that you were hoping that we could talk about? Or are you good to start the wrap up piece?

• 1:02:02Speaker 00

  The role of humans or basically the the interaction of humans with AI, I think that is maybe not talked about enough in general. I feel like humans feel threatened. There's all this hyperbole about, you know, jobs are going to be replaced by AI, you know?

• 1:02:21Speaker 00

  Uh, you know, uh, graduating classes from Stanford and MIT and other kind of places found difficult to to land jobs in computer science. People are skipping off computer science or programming because they think everything can be automated. We don't need these into the undergrad people. I think that's just a lot of proportion, I think. Um, AI, like any other technology paradigm, will disrupt the human computer interaction. It's not going to, uh, remove the role of humans. It's going to change the role of humans, right? Instead of humans completely out of the loop, it'll be humans on the loop, right? Ultimately, AI, you know, the the starting points and the endpoints of AI operation

• 1:03:06Speaker 00

  will end with an interaction with a human, right? Whether it's physically AI, whether it is digital, anything of that sort, you have to have a human input and a human output or an output to a human, right? And then also there is, there is this thing where, uh, you know, people think that we can completely automate all the processes, whether it's in factories or even in digital, like enterprise processes, like HR, marketing, sales and all that. It does not factor into, uh, uh, the scene, the accountability of it, right? A lot of times when AI fails, AI is not perfect, right? AI is only as good as the, you know, the data it's trained on. It's not perfect, it cannot cover everything.

• 1:03:50Speaker 00

  But the accountability still rests with somebody human. You cannot jail AI, right? You cannot choke AI's neck. You still need a human to be accountable for it. So, so the role of humans will evolve, it will not completely, uh, go away. It'll, it'll, it'll be more of a supervisory role, right? They will still be accountable for everything that's going on in the AI realm. Um, so I think if folks can kind of figure out a way to to not just work with AI, but kind of figure out how their roles will evolve with AI, I think they need to start doing that and start preparing towards that kind of eventuality where they'll be on the loop, not in the loop. They will be not not completely off the loop either. They'll be in a supervisory role and they'll still be accountable.

• 1:04:34Speaker 00

  They may have to go to jail if AI now functions.

• 1:04:38Speaker 01

  I think that's well said, um, and an interesting and thoughtful note to end on. Ankur, I want to say a big thank you for coming on today. It's been a really interesting and insightful conversation.

• 1:04:48Speaker 00

  Thank you so much for having me. It was great to have this chat. Thank you.

• 1:04:53Speaker 01

  Most viewers don't know this, but Digital Disruption is developed by Infotech Research Group, a leading advisor to technology leaders around the world. If that's not you, you don't need to care. So skip ahead and enjoy our content. But if you are a technology leader, Infotech helps IT teams get projects done faster, better, and at a lower cost. Infotech provides unlimited access to practical tools and expert guidance you need to execute at a fraction of the cost of traditional consulting. No matter the project, from AI strategy to cybersecurity to vendor negotiation, Infotech has you covered. Check it out at the link below and don't forget to like and subscribe.