How 3 CEOs Use AI to Run $10B in Companies | This Week in AI

This Week in AI, JCal sits down with three CEOs building the infrastructure, intelligence, and interfaces for the next era of AI: Jeremy Fraenkel (CEO, Fundamental), Victor Riparbelli (CEO, Synthesia), and Nick Harris (CEO, Lightmatter). We break down what's actually happening beneath the AI hype: the data modality LLMs completely missed, why copper is the real bottleneck in AI data centers, OpenAI shutting down Sora, the build vs. buy debate for AI tools, and how close we really are to AGI. AI's Biggest Blind Spot, Tabular Data: LLMs transformed text, images, and code, but 70-80% of enterprise data lives in rows and columns. Copper Can't Keep Up: Nick explains why AI data centers are hitting a wall. GPUs compute faster than they can communicate. Lightmatter's photonic chips push 1.6 terabits per fiber and can 3x training speed. Why OpenAI Killed Sora & Anthropic's Focus is Winning: Victor breaks down why even OpenAI had to learn the lesson of focus, and why Claude Code has every founder talking. Vibe Coding Your Own CRM vs. Buying Salesforce: Jeremy reveals Fundamental built their own internal CRM using vibe coding. The panel debates when building beats buying and when it's a distraction. The Omnipresent CEO: Jason shares how he's using AI agents for root access to Slack, Gmail, and Notion, resurrecting former employees as AI personas, automating SDR workflows, and summarizing employee inboxes while they're on vacation. Are We Already at AGI?: Nick says the rate of progress is a double exponential. Jeremy argues AGI is a moving goalpost. Victor warns of "Future Shock" and societal disruption. 🔗 Learn more about Fundamental: https://fundamental.tech🔗 Learn more about Synthesia: https://www.synthesia.io🔗 Learn more about Lightmatter: https://lightmatter.coThis Week In AI is made possible by:PayPal Open - One Platform for all Business: paypalopen.comTimestamps:00:00 Welcome & intro to Jeremy Fraenkel, Victor Riparbelli, and Nick Harris01:47 What is Fundamental? Large tabular models explained07:01 Victor Riparbelli on Synthesia & why OpenAI killed Sora11:09 Claude Code dominance & the Lightspeed founder retreat12:08 Nick Harris on Lightmatter, photonics & the new Moore's Law14:38 Copper vs. fiber: why AI data centers are hitting a wall18:44 Reinventing video: interactive, real-time, personalized21:32 The economics of a custom AI movie23:55 Why Amazon, Google & Meta are building their own chips28:06 Tables have a bandwidth problem too32:27 When will compute be as cheap as storage?36:10 The future of software: every company gets a custom stack38:06 Vibe coding your own CRM vs. buying Salesforce45:57 Jason's quest for root access to Slack50:18 The omnipresent CEO: Doctor Manhattan meets Jesus CEO52:13 Resurrecting former employees as AI personas53:25 Victor's executive changelog for a 650-person company55:07 Whisper Flow & the Plaud Pin1:00:03 AGI: is it already here?1:03:37 Jeremy: we've only solved half the brain1:06:30 70% of Americans fear AI will impact jobs1:08:47 Future Shock & keeping the rope tight*Mentioned in the show:* Wisper Flow: https://wisperflow.ai Plaud Pin: https://www.plaud.ai Athena Executive Assistants: https://www.athenawow.com WHOOP: https://www.whoop.com "Future Shock" by Alvin Toffler: https://www.amazon.com/Future-ShockAlvin-Toffler/dp/0394425863 Victor on TWiST, E1776: https://youtu.be/jxET4fq_2eA Nick on TWiST, E1787: https://youtu.be/FPW2nnEqfMsSubscribe to This Week in AI on Apple: https://thisweekinai.ai/spotifySubscribe to This Week in AI on Spotify: https://thisweekinai.ai/appleThanks for watching!🤖 If you want to stay ahead of the curve on all things AI, make sure to join our community across all platforms:📩 Get the Weekly Newsletter: https://thisweekinai.ai/📺 Subscribe on YouTube: https://www.youtube.com/@ThisWeekinAIPodcast📸 Instagram: https://www.instagram.com/thisweekinaipodcast

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Hey, it's Oliver from This Week in AI, the brand new podcast from the team at Twist. We're dropping a sneak peek right here in your feed to show you what we've been building. If you enjoy it, join the community at thisweekinai.ai or find us on Spotify, Apple Podcasts, or YouTube.

• 0:14Speaker 01

  I was talking to a friend of mine, she's an accountant, and she told me accounting is never going to be replaced by automation. I'm like, what are you talking about? It's the first time we're really automating cognition as opposed to just automating the physical part of the job.

• 0:27Jason Calacanis?

  Seventy percent of them think they'll have a decrease in job opportunities. Only thirty percent of Americans are worried in the same poll about themselves, so they all think it's happening to somebody else.

• 0:36Speaker 03

  I do think that humans will want to play status games. I think we'll find other jobs. I think we'll probably be doing less numerical and logical jobs.

• 0:43Speaker 04

  It feels like something very big is coming. The world doesn't appreciate that it's happening because most people are not very good at asking questions. You can taste the singularity at this point. I can't even imagine the end of this year is going to be shocking.

• 0:58Jason Calacanis?

  Thanks to our friends at PayPal, the exclusive sponsor for This Week in AI. Try the payment and growth platform that's trusted by millions of customers worldwide. PayPal Open. Start growing today at PayPalopen.com. All right, everybody, welcome back to Not This Week in Startups, Not All In. This is a new roundtable I'm doing. It's called This Week in AI. It's in the name, folks. Every week, three amazing CEOs, just like on All In or the VC roundtable we do over at Twist. Three amazing CEOs who are actually building the future and me, an investor in the space and an entrepreneur, talk about the week's issues and sometimes the bigger picture issues. You can find out more about the podcast thisweekinai.ai.

• 1:40Jason Calacanis?

  Or if you want to see the YouTube channel, thisweekinai.ai/youtube. And this is our seventh episode. It's March 31st, 2026. Three amazing guests with us today. Jeremy Frankel is here. He's the CEO and co-founder of Fundamental. They're building large tabular models for enterprises. They emerged from stealth as a unicorn just sixteen months after founding, 255 million, Series A, led by Oak, with participation from Valor Battery, Salesforce, and more. Welcome to the program, Jeremy Frankel. So Jeremy, explain what your company is doing and how it's going so far.

• 2:21Speaker 01

  Things are going great. So what we are doing is we built a foundation model for tabular data. So what does that mean? So when you, you know, when people think about, you know, the AI boom or the AI revolution, everyone is thinking about, you know, LLMs and, you know, for good reason, right? Like, you know, chat GPT, like, um, you know, a crater breakthrough, like, you know, you can now, you pre-train one model on the entire internet and you understand language and you can use it to power thousands of use cases. But what's less appreciated is that LLMs really mostly solves unstructured data issues, such as, you know, text, audio, video, images, coding, but they really didn't impact structured data. And structured data means everything that comes in rows and columns.

• 3:05Speaker 01

  So think about spreadsheets, databases, CRMs, ERPs. It's all rows and columns and that's the vast majority of useful data for enterprises. And that part of the enterprise AI has never had its chat GPT moment. And so this is the modality we're going after. And for a variety of reasons, it's the one modality that acts very differently than others. And what we're bidding is really the chat GPT moment for tabular data.

• 3:35Jason Calacanis?

  So if I can repeat it back to you to make sure I understand this vision, is what I always like to do with founders, is see if I can repeat it back. You have large language models. Those are built, as we all know, like guess the next word and transformers. It's based on massive corpuses of text-based data, generally speaking. You're building an LLM to focus specifically on tables and tabular data structures that we all know as a, you know, might experience an Excel sheet or a database, a Notion database, a, you know, a SQL database. Am I correct?

• 4:13Speaker 01

  So it's not, it's not an LLM, it's a large tableau model. So it has a very different architecture than LLMs. Basically, if you look at the LLMs, as you said, they're built on, you know, being next token predictors, right? It's an auto-aggressive model. The problem with that is that if you look at the way transformers, for example, are applied to LLMs, they have a positional encoding as part of it. So like the order of the sentence matter, right? And if you change the orders of your, of your, of your sentence in Claude or chat GPT, you can get a different output. But with tables, you actually don't want that. And the reason why is like, imagine if you have a table with a million patients and you're trying to predict which one of them have cancer. And your first column has the weight of the patient and your second column has the heart rate of the patients.

• 4:57Speaker 01

  If you switch the order of those columns and you first leave the heart rate and then the weight, you shouldn't expect or you wouldn't want to expect a different output. But with LLMs, if you change the order of the data, you get a different output. And that's fine when you're writing an email. It's not fine when you look at the deterministic outputs. And so that's what we've focused on.

• 5:15Jason Calacanis?

  Got it. So large tabular model and LTM is how they're first. Are you the first people to do this? Or is this like a known alternative to an LLM?

• 5:26Speaker 01

  It's very nascent. There are few smaller companies working on it as well, more in an academic environment, an academic setting, but we are the first like large company doing that at an enterprise scale.

• 5:36Jason Calacanis?

  What is the benefit here? What's the, is it because you'll have a better fidelity, a better results, more trustable results than the problem we have with hallucinations in large language models? Would that be the reason to do this?

• 5:51Speaker 01

  It's very different use cases. So if you'd look at, you know, everything in the economy, for example, on everything you swipe your credit card, one of the credit card providers has to make, you know, a split second decision of whether a transaction is fraudulent or not. When you work with retail, for example, forecasting demand, all of those, or like, for example, if you order an Uber, Uber has to make a prediction on the ETA of your driver. Each one of those tasks is tabular by nature and it's predictive. But when you look at the way those predictions are being made, they still rely on traditional machine learning algorithms that predate LLMs. And those algorithms still do better than most LLMs at making those predictions. And so what we've built is a model that can essentially unify all of those use cases

• 6:36Speaker 01

  into one model to allow you to make much more accurate predictions than what you would otherwise be able to make.

• 6:41Jason Calacanis?

  Genius. So the company is named Fundamental and your flagship model, the LTM, large tabular model is called Nexus. Am I correct there?

• 6:51Speaker 01

  Correct.

• 6:52Jason Calacanis?

  All right. And we just had Perplexity CEO Aravind, who's really crushing it. He's one of your angels, huh?

• 6:59Speaker 01

  Correct.

• 7:00Jason Calacanis?

  Awesome. Victor Riparbelli, I got your name correct, I hope.

• 7:03Speaker 01

  You did.

• 7:03Jason Calacanis?

  Riparbelli, welcome back to the program. You're with Synesthesia.

• 7:08Speaker 03

  Synthesia.

• 7:09Jason Calacanis?

  Synthesia, S Y N T H E S I A. You're an AI video platform for business. Ninety percent of Fortune one hundred companies are your, are customers already. Over a hundred million in ARR. And you've raised over five hundred million, four billion dollar valuation. And we're seeing an incredible demo here. Explain to us. How you're different. Nano Banana, the free services out there, ChatGPT's image generation. And why you exist as a company dedicated to just, you know, working on video models.

• 7:43Speaker 03

  So I think it goes, we started the company in twenty seventeen. Way before any AI video tech actually worked. So we've been like quite a bunch of like different companies all the way up to twenty twenty six. And what we decided on like five years ago was we're kind of looking at the early iterations of of AI video. Right. Which is the model that we're seeing right now, like V O, Sora, which just been discontinued. Our models, obviously very high fidelity. Fairly inexpensive to run, assuming you're just like creating single videos. And really getting to the point where you actually can't tell the difference, which unlocks a whole bunch of new use cases. But what we figured out five years ago was that the first iteration of this technology was not ready for prime time. It could definitely not make a Hollywood film. It would definitely not make performance, marketing ads.

• 8:26Speaker 03

  But there was a very real use case in taking all the world's PowerPoint users. And enabling them to communicate in video as opposed to slide decks or documents. Which in twenty, you know, twenty twenty two, when we launched the first product. Was what everybody wants, right? People want to watch and listen to their content. They don't want to read that much anymore. And we essentially provide a way for PowerPoint creators to very easily switch to making video instead. And that has worked really, really well. And today, you know, we both have our own models. We build voice models, we build video models, we have interactive models, so avatar, we can actually talk to in real time. That's launching very soon. And then we also use a mix of like the, the big models from some of the bigger providers to, to, to solve some work for our customers.

• 9:10Jason Calacanis?

  And for folks who want to hear from you, three years ago, we had you on This Week in Startups as part of our next Unicorn series. Before you were Unicorn, episode seventeen seventy six. What a great episode number to have. Why did? ChatGPT OpenAI shut down Sora and why is Elon doubling down on video? He's been very vocal just this week, talking about how video is the most important thing. Take us through your take on that. Obviously, you were in video years before ChatGPT was even launched.

• 9:45Speaker 03

  I mean, I think it's kind of interesting that even a company like OpenAI found by like some of the smartest people in the valley, right? Like some of those accomplished people still had to learn the lesson of like focus. You know, it's kind of like, I feel like it's almost like unteachable lessons, but they had to learn the hard way. I think it's very obvious to anyone looking at the way that Anthropic is ripping right now, that CodeGen is probably the most valuable near term use case for all these technologies. And I think OpenAI probably had a little bit of a flying too close to the sun moment, but it decided to do like absolutely everything all at once, right? Which often in the PowerPoint, it sounds doable. But in reality, I think anyone who's run a business knows that doing too many things at once is like rarely a really good idea.

• 10:28Speaker 03

  And Anthropic focused on no voice models, no video models, just like CodeGen, B2B, no freemium. And that has still paid off really well for them. So I think my take on what's going on in OpenAI is that they're like, let's cut all the side quests and focus on the market that's really, really going to matter. And I think that's going to be probably screwing more towards B2B and it's going to be heavy on CodeGen, empowering just this like campy explosion of products that we're seeing being built with the vibe going right now.

• 10:56Jason Calacanis?

  I guess Claude's got people shaken. They've done, or specifically, it's got OpenAI shaken. They've added so much revenue. They've become such a darling. Jeremy, I see you smiling about this. This has become notable in the industry. Yeah, Jeremy?

• 11:11Speaker 01

  No, correct. And I mean, it's funny, I was at the founder retreat a few weeks ago, and everyone was just talking about Claude Code. No one was mentioning anything else but Claude Code, so.

• 11:20Jason Calacanis?

  What founder retreat was this?

• 11:22Speaker 01

  It was a Lightspeed event.

• 11:24Jason Calacanis?

  Ah, Lightspeed, okay. The venture capital firm. And that's fascinating. Nick Harris is back in the This Week in Family. You were on This Week in Startups. I remember this discussion, August of 2023 as well, episode 1787. And so we got it right, having you guys on early. And we were talking about, and you were predicting just how important, you know, these data centers were. And that photonics, using light instead of electricity to connect AI chips.

• 11:57Jason Calacanis?

  Would be critically important, and you explained to me back on that program, Nick, that energy and data centers were going to be a major, major issue. And here we are three years later. Energy is the bottleneck, isn't it, Nick?

• 12:10Speaker 04

  Yeah, it's exactly the bottleneck. You know, as a company, we've been focused on driving the future of computing. The central challenge that we're solving at LightMatter is around how do you create a new roadmap for Moore's Law, for Darnard's scaling? These are rules that drove computing progress for, you know, our entire lives. I think about being a kid in the 90s, and every 18 months, you get an incredible new chip, more performance, all these things. That's over now. And there's only two ways that computers get better at this point. One is big computer chips. You put more chips in a package. Computer chips are getting to the point where, you know, Nvidia sells nearly $100,000 chips. So those chips are getting really big, and the size of the chip is going to keep growing.

• 12:52Speaker 04

  And the other way is that at any given time, there's the biggest chip you can build. So networking them together is the other piece. So big chips network together. This is the future of computing. It's the new Moore's Law. And we power both of these with our product passage. And we also, since we spoke last, started building lasers, which I never thought we would get into. But when you look at the photonics revolution, what's interesting is that you've got this device that powers all of the communication for these AI supercomputers. And it relies on the laser. It's kind of like batteries for electric vehicles. It's a really fundamental, huge part of the bomb, and it drives all the progress in how these computers are going to connect.

• 13:34Speaker 04

  You know, one of the cool things to tie into the software piece of this is with photonic technology, we've shown, you know, in research and so on, that you can actually 3X time to train. If you guys are watching Anthropic and their incredible tear, I'm hearing about the new model Mythos that's coming out. We can actually 3X, we 3X faster time to train. Imagine if you have PE to the RT, where you've got R times 3 now. So the rate of takeoff is going to go up like crazy. The first companies that adopt this photonic technology for linking up GPUs and AI data centers, the foundation companies, they're going to have an enormous advantage.

• 14:13Jason Calacanis?

  Got it. And for the audience, again, my typical technique of repeating back, so we all understand what you're doing, Nick, at LightMatter. Ethernet cables, that's how we connect computers typically. Obviously, people are consumers are using Wi-Fi. You would never use that because it's very limited bandwidth. But Ethernet, which is typically copper wrapped in plastic versus photonics, which would be made of glass, I'm assuming here, and fiber optics. Yes, and the throughput is radically different. Maybe you could explain and give us a bit of a primer and then you could sportscast what we are seeing on the screen.

• 14:49Speaker 04

  Right now, the way that AI supercomputers are built, you think about NVL 72 from Nvidia. They take 72 GPUs and they link them all together in a very high bandwidth domain. So here we're talking about petabit per second bandwidths within a rack. That's all linked together in copper. What's really interesting about copper is it can't go very far. The cables have to be quite short. And so what you're seeing is the racks are getting packed as tight as possible. Now, people are building racks that are a megawatt. So you have a rack that's a megawatt. You have to reinforce the concrete below it because it's so heavy that it's actually a load on the infrastructure. And you're building these custom racks that are just for delivering the cooling to these systems. That's kind of where people are at today. And the reason they're there is you have to bring the density

• 15:34Speaker 04

  because the copper can't reach very far. So just to give you an example of the delta in what we do, we just announced a chip with Qualcomm where with each glass fiber, we're packing 16 wavelengths of light and we're pushing 1.6 terabits over a single optical fiber. That bandwidth is crazy. It's like 1,600 houses worth of internet. You know, a normal house has one gig internet, so 1,600 houses. So copper really doesn't have very much reach. And the reason it matters is when you're building these AI supercomputers, if you want to have great performance, you want to link as many GPUs as you can tightly together. If you have optics like what we do, you don't need to put it all in one rack. You can separate it by a kilometer.

• 16:20Speaker 04

  It travels at the speed of light. There's very little loss in the optical fiber. And you can build giant systems that act like a single brain rather than a bunch of mini brains with 72 GPUs talking in parallel. You could have thousands of GPUs working together on a workload. It drives interactivity, drives time to train. Both inference and training get a huge benefit out of switching from copper to optics. And we're kind of the leader in performance in the space.

• 16:46Jason Calacanis?

  And just so people can conceive of a petabit. You're talking about thousands of 4K movies from, you know, coming from Netflix every second. So this and probably 100 million high-res photos from your camera, your library per second. So if you had 100, if we as consumers had 100 million photos somehow in our photo libraries, Victor, you could be sending just, you know, hundreds of people's photo libraries. The entire, you could send the entire corpus of Netflix movies in 10 seconds. Yes, Nick?

• 17:26Speaker 04

  Yeah, exactly. And there's a kind of a cool analogy here. We have the chip M1000 that we announced last year. That chip is 114 terabit per second. So that's 114,000 gigabit per second. That's 114,000 houses worth of bandwidth. And a more interesting comparison is that that is about the bandwidth of the cables that connect North America to Europe for the internet, the undersea optical cables. So we're building chips that have just an obscene amount of bandwidth. And it's all needed to drive AI scaling.

• 17:57Jason Calacanis?

  Jeremy or Victor, have you, in terms of your data usage, when you hear about light matter and the impact it could have, what goes through your mind, Victor? You know, because obviously you're working in video and your data centers. I'm not sure what your standard platform is and where you host. But maybe thinking ahead to the future, how do you think about what Nick is building?

• 18:20Speaker 03

  I think it's super exciting. So there's kind of two big ideas we found in the thesis. The first one was like, as AI increasingly can generate data, the marginal cost of creating video, audio, all the other content has gone drop to zero, right? Both in dollars, but also much in time required and skills required. I think we're in the middle of that right now. But this is still video as we know it today. It's a broadcast medium. You make one video, you put it on YouTube, and everybody watches exactly the same version of it. The second part of our thesis was always around, you know, when we invent new technologies as humans, we always invent new media formats that are native to those technologies. Like, you have a podcast or TikTok video wouldn't exist without modern technology. And for us, the big question is like, what does video look like if you were to reinvent it in 2026

• 19:04Speaker 03

  with all the new primes we have around us, right? We have LLMs with Essentially, intelligence on tap. We have offline video models that can create extremely high quality content. We have real time video models that you can interact with. Advertisements you can talk to, canvases that can be drawn in real time. And a whole bunch of other like cool technologies around it. And so what we're building for, what we're actually launching, it's in private beta right now, launching in a couple of months, is real time video, which is the idea that if you, if to take one of our use cases, if you're a salesperson and you're doing a bunch of training to understand like the competitive landscape and your product that you're launching, instead of just like receiving a video that you sit down and then you watch it and then, you know, you hope you understand it.

• 19:46Speaker 03

  It's got to be an interactive experience, you know, it's got to be, maybe first you consume some content. You go into an agent, you role play with it. It pretends to be a customer, you have to answer questions, overcome objections. Then you go to another thing where we actually in real time draw a diagram of a customer's potential tech stack. How you're going to, you know, work with this, how you're going to integrate it. That's a very different type of video, which is almost like closer to maybe like a game or like a website, something like that. But one of the bottlenecks here, right, is of course that if we're actually going to do this with video and we're going to do like avatar models, we're going to draw things in real time. That's going to take up a lot more bandwidth. It's also going to have much higher inference costs. And so the more we can, we can build, we can, we can reduce these, right? The more accessible this becomes.

• 20:28Speaker 03

  So I think in the next couple of years, we'll see this becoming a new type of interface that's going to emerge. But for it to really take off and just be, you know, every interaction we have with the computer could be done with technology like this. We need the cost of serving that content to drop like very significantly. And I think that's, that that's the core of what the problem Nick's working on. So I think it's, it's very exciting.

• 20:47Jason Calacanis?

  Nick, when we have this ready for Victor to experience it and like it's, we could probably do a deal right now that he could be one of your beta customers because it would be amazing for Disney. I mean, I'm thinking in a consumer mind frame to sort of help the audience follow along here. But imagine, you know, Disney releases Mandalorian and they had done a deal with Sora to try to get the IP to work. Now imagine with Light Matter being able to enable Victor's company to be able to make a short film with Grogu and the Mandalorian and you're talking to them in real time and it's making that in real time. That's just.

• 21:26Speaker 03

  Yeah, that's absolutely. Economics on that, right? I think, you know, if you were to do that today, let's say you were to like personalize like a one hour movie for a kid from Disney. Like that would cost you like a lot of money, right? If you say like an eight second clip with like a state of art video model costs like one or two dollars today. You can add that up for like an hour of eight second clips, right? That's not going to be sustainable within like a fifteen dollars per month.

• 21:50Jason Calacanis?

  It'll be, yeah, if it was six dollars a minute, if we just made it like six bucks a minute, hundred twenty minute, you're talking about seven hundred dollars for a custom movie that you could live in.

• 21:59Speaker 03

  Exactly. And we're not, we're not that many years ahead of like, I remember I was a kid, right? And I had to like call my dad and ask him if I could download like a ten megabyte file because it was like ADSL and you were like, me, I don't know how much you would download. The idea of doing a video call for an hour with someone across the world, like that's an after the ludicrous idea, right? But probably in like X amount of years, this is gonna be like completely normal. We're gonna be just generating content in real time in front of people and we're gonna be able to offer that at like, you know, within the subscriptions that these services charge today.

• 22:28Jason Calacanis?

  Nick, you were gonna add to this.

• 22:30Speaker 04

  Yeah, we're actually busy building chips for a bunch of companies. We typically work with hyperscalers to build their own chips. Think about like the Google, Amazon, Microsoft, Meta type companies who are building their own hardware to do both training and inference. And then we also work with semiconductor companies, both GPU companies as well as networking companies. So those are the people we build for. We're building a ton of chips right now. So I would say in the next, you know, year and two years, you're gonna start running on light matter hardware. These will be in the new data centers. Think about like the Texas stuff.

• 23:03Jason Calacanis?

  Yeah, CoreWeave, what's the one, not Starbase, Stargate, another great film, speaking of films.

• 23:11Speaker 04

  Yes, excellent film.

• 23:12Jason Calacanis?

  Yeah, and so there's a picture of, I think that's Stargate. And what you see in the middle is that plus, I think, is, I think I was talking to Jensen or the CEO of CoreWeave about this. Somebody on my team will tell me, I believe this is. Uh, CoreWeave's data center, you have the cooling there, that bottom line that looks like memory chips in a motherboard. These are starting, these data centers are starting to look like giant motherboards, but I think those are the cooling apparatus where the contained water system, uh, what do they call it? Closed loop water system?

• 23:43Speaker 04

  Closed loop.

• 23:44Jason Calacanis?

  Yeah, closed loop water system. So that was Crusoe's data center. I remember the CEO was walking me through it on a previous episode. That's the closed loop water system and the data exchange. Uh, pretty compelling stuff. Uh, Jeremy, when you look at all this, oh, by the way, Nick, Amazon making their own chips and I don't know if they're a customer or if they were, you could say so, but you gave us the sort of like Amazons are called Trainium for training AI models and Inferentia for running inference models. Uh, somebody at Amazon needs to go to branding school. That's a little too on the nose. Trainium and Inferentia. I mean, what did they do? They asked ChatGPT to come up with names.

• 24:28Jason Calacanis?

  Um, but these are going to be dedicated chips and I think Broadcom generally builds people their chips. Is that typically what happens? So if we explain it to the audience, you have Nvidia, they work with TSMC, they're making their own chipsets. They're the leader of the pack. Every single other hyperscaler, uh, tensors from Amazon, tensors from Google.

• 24:52Speaker 04

  From Google. Yeah.

• 24:52Jason Calacanis?

  Then you have these Inferentia, Trainium.

• 24:56Speaker 04

  And MTIA from Meta.

• 24:58Jason Calacanis?

  MTIA from Meta. So explain to the audience why people are doing two different supply chains, Nick, and then we'll get to you, Jeremy, on your thoughts on this next wave.

• 25:10Speaker 04

  Well, I mean.

• 25:10Jason Calacanis?

  And how that'll affect your business. Yeah.

• 25:12Speaker 04

  If you look at the incredible spend, I mean, they're over a hundred billion a year. They're like 180 billion a year, I think is what Google announced they spent. I think Amazon was over 200 billion. For the year, when you're spending that kind of money, developing your own custom silicon is a little bit of a rounding error. So I think that they're they're really looking at these costs. They're trying to figure out how they can optimize costs and and they think they can build their own solutions. Now, building a chip is one thing, but building all the software and the ecosystem around it is another. And that's where Nvidia has had, you know, decades of experience, building out the moat there with CUDA and everything. But everyone's trying to build these chips. And the reason is that it's a race on the infrastructure point.

• 25:53Speaker 04

  People are trying to get power. They're investing in these micro, you know, nuclear reactors to go power the data centers. Hundred megawatts each. So you get 10 of those and you've got a gigawatt data center. They're working on that power delivery. They're working on building the chips. They do their own. You know, these these hyperscalers are becoming like very heavy duty infrastructure players. From cement to to energy, all the way to chips and obviously the software stuff on top. There's just so much money in this space that they're all kind of making the bet on doing it themselves. And it's all in service of powering, you know, technologies like Jeremy and Victor's. It's really about the apps that run on top of this and getting their costs to the right point. Because the AI models are incredible, but we've got to keep driving down the token cost and driving up the inference rate.

• 26:39Speaker 04

  And then we'll be able to keep unlocking incredible things like, you know, custom movies. And you're going to need blazingly fast interconnect for that.

• 26:46Jason Calacanis?

  Jeremy, I'm assuming you're building on CUDA, which is the proprietary layer for coding and sending jobs to Nvidia hardware, correct?

• 26:57Speaker 01

  Correct.

• 26:58Jason Calacanis?

  And if you were to consider other platforms, other hardware platforms that were non Nvidia, is there, have you considered that? And is there a path for you? Or would you have to maintain, you know, CUDA plus some other open source software? I guess there's some abstraction layers for CUDA now to get on AMD processors. So as the CEO, how do you think about where to spend your energy? Is it just too much to even consider other platforms? Or are you like Amazon, Meta and Google saying, hey, we need to have two swings at bat?

• 27:34Speaker 01

  I very much think that, you know, we we are in the process of exploring different different chips as well. You mentioned Trainium, one of the chips we're in the process of of exploring. Um. And the idea here is that, you know, we we don't want to just be dependent on one, you know, hardware or one type of chip. Of course, it comes with, you know, like, CUDA gives you a lot of advantages. Uh. And, you know, it's not easy to switch away from CUDA, but um it's definitely something that's when the process of exploring and. You know, what Nick is doing is is really uh is really exciting, right? Because like, you know, the funny thing about everyone knows, but you know, with video, like, you you know about that. The amount of data that is being moved from one place to another. But people also don't realize that you have the same problem with tables.

• 28:16Speaker 01

  Like, if you think about, like, a table with ten million rows and hundred columns, which is not that big of a table, right? It's like, what? You have, like, a billion cells? That's um orders of magnitude more than the context the largest L L M can even take in, right? The largest L L M can maybe take hundred thousand rows, but when you're working, for example, with uh banks on fraud detection, you're working in billions of rows. So like, you just need and, you know, and they're like, you know, um millisecond matters, right? Like, you make a this, like, a decision, like, when you start your credit card, you don't want to be waiting for ten minutes before you get an answer. You just want to get an answer right away. And so the amount of data out there in tables is just massive. Like, we um we've been, you know, talking to a few companies where, like, they, like, think about every I O T sensor.

• 28:59Speaker 01

  Every time, like, you know, you get some data, that's that comes in some so some form of structured form. And like, the amount of data that's uh that they uh they are dealing with is like petabytes of data. And so being able to um to move data much faster and having a lower latency, as Nick said, you know, also lower cost, will really be essential.

• 29:19Jason Calacanis?

  Yeah, I mean, if you were to think about, I have these air things, I don't know if you guys care about air quality in your homes. But it turns out, like, in your office, in your home, like, C O two and radon, all this stuff, very important for health, very important for, like, cognitive function, especially C O two. So I have these air things, and it's taking recordings in six rooms in three different houses for me. The amount of data that just one person consumes, or let alone your whoop or your flip bit. Like, how many heartbeats is whoop? And shout out to Whoop, they just raised money at a ten billion dollar valuation, like. What's the data processing there when they have to do my sleep and my recovery and my run and my heartbeat? I mean, my lord, it's huge amount of data.

• 30:02Speaker 00

  Exactly. It's unfathomable amounts of data. Join the community at thisweekinai.ai or find us on Spotify, Apple Podcasts, or YouTube.