Pushing compute to the limits of physics

Dr. Maxwell Ramstead grills Guillaume Verdon (AKA “Beff Jezos”) who's the founder of Thermodynamic computing startup Extropic. Guillaume shares his unique path – from dreaming about space travel as a kid to becoming a physicist, then working on quantum computing at Google, to developing a radically new form of computing hardware for machine learning. He explains how he hit roadblocks with traditional physics and computing, leading him to start his company – building "thermodynamic computers." These are based on a new design for super-efficient chips that use the natural chaos of electrons (think noise and heat) to power AI tasks, which promises to speed up AND lower the costs of modern probabilistic techniques like sampling. He is driven by the pursuit of building computers that work more like your brain, which (by the way) runs on a banana and a glass of water! Guillaume talks about his alter ego, Beff Jezos, and the "Effective Accelerationism" (e/acc) movement that he initiated. Its objective is to speed up tech progress in order to “grow civilization” (as measured by energy use and innovation), rather than “slowing down out of fear”. Guillaume argues we need to embrace variance, exploration, and optimism to avoid getting stuck or outpaced by competitors like China. He and Maxwell discuss big ideas like merging humans with AI, decentralizing intelligence, and why boundless growth (with smart constraints) is “key to humanity's future”. REFS: 1. John Archibald Wheeler - "It From Bit" Concept 00:04:45 - Foundational work proposing that physical reality emerges from information at the quantum level Learn more: https://cqi.inf.usi.ch/qic/wheeler.pdf 2. AdS/CFT Correspondence (Holographic Principle) 00:05:15 - Theoretical physics duality connecting quantum gravity in Anti-de Sitter space with conformal field theory https://en.wikipedia.org/wiki/Holographic_principle 3. Renormalization Group Theory 00:06:15 - Mathematical framework for analyzing physical systems across different length scales https://www.damtp.cam.ac.uk/user/dbs26/AQFT/Wilsonchap.pdf 4. Maxwell's Demon and Information Theory 00:21:15 - Thought experiment linking information processing to thermodynamics and entropy https://plato.stanford.edu/entries/information-entropy/ 5. Landauer's Principle 00:29:45 - Fundamental limit establishing minimum energy required for information erasure https://en.wikipedia.org/wiki/Landauer%27s_principle 6. Free Energy Principle and Active Inference 01:03:00 - Mathematical framework for understanding self-organizing systems and perception-action loops https://www.nature.com/articles/nrn2787 7. Max Tegmark - Information Bottleneck Principle 01:07:00 - Connections between information theory and renormalization in machine learning https://arxiv.org/abs/1907.07331 8. Fisher's Fundamental Theorem of Natural Selection 01:11:45 - Mathematical relationship between genetic variance and evolutionary fitness https://en.wikipedia.org/wiki/Fisher%27s_fundamental_theorem_of_natural_selection 9. Tensor Networks in Quantum Systems 00:06:45 - Computational framework for simulating many-body quantum systems https://arxiv.org/abs/1912.10049 10. Quantum Neural Networks 00:09:30 - Hybrid quantum-classical models for machine learning applications https://en.wikipedia.org/wiki/Quantum_neural_network 11. Energy-Based Models (EBMs) 00:40:00 - Probabilistic framework for unsupervised learning based on energy functions https://www.researchgate.net/publication/200744586_A_tutorial_on_energy-based_learning 12. Markov Chain Monte Carlo (MCMC) 00:20:00 - Sampling algorithm fundamental to modern AI and statistical physics https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo 13. Metropolis-Hastings Algorithm 00:23:00 - Core sampling method for probability distributions https://arxiv.org/abs/1504.01896 ***SPONSOR MESSAGE*** Google Gemini 2.5 Flash is a state-of-the-art language model in the Gemini app. Sign up at https://gemini.google.com

• 0:00Speaker 00

  That was the most technical podcast I've ever done. I think that was one of my favorite conversations, for sure. Hey, I'm Guillaume Verdon. Growing up, really, I was pursuing theories of everything. I want to understand the universe. Was a big fan of Feynman, Stephen Hawking, growing up. And I was seven, I was like, talking about subatomic particles and- As all seven-year-olds do, right? I guess I got swept up in the school of thought that was, you know, a generalization of Wheeler's it from bit, which was the it from qubits program, seeking to unify theoretical physics through quantum information theory. So viewing everything in the universe as one big quantum computer running a certain program or self-simulation. We have proof of existence of a really kick-ass AI supercomputer that we're both using right now to talk to each other.

• 0:45Speaker 00

  It's our brains. That, I would argue, is a thermodynamic computer, right? Because there's a, there's master equations describing the chemical reaction networks in your brain of neurotransmitters hopping around. And so, you know, if we're doing very similar physics, but with electrons sloshing around a circuit, then, you know, there's a very, much stronger chance that we could run a very similar program in a very similar fashion, arguably with even more energy-efficient components, because electrons are much lighter than big neurotransmitters.

• 1:17Speaker 01

  This podcast is supported by Google. Hi, folks. Paige Bailey here from the Google DeepMind DevRel team. For our developers out there, we know there's a constant trade-off between model intelligence, speed, and cost. Gemini 2.5 Flash aims right at that challenge. It's got the speed you expect from Flash, but with upgraded reasoning power. And crucially, we've added controls, like setting thinking budgets, so you can decide how much reasoning to apply, optimizing for latency and costs. So try out Gemini 2.5 Flash at aistudio.google.com and let us know what you built.

• 1:48Speaker 00

  Hey, I'm Guillaume Verdon. I'm the founder of Xtropic, a company pioneering thermodynamic computing, a new form of computing for probabilistic inference, using exotic stochastic physics of electrons. Formerly, I was working on quantum computing and machine learning at Alphabet. And I also happen to be the founder of a philosophical movement called Effective Accelerationism, under the pseudonym Beth Jayzos, online.

• 2:19Speaker 00

  Happy to be here at MLST.

• 2:21Speaker 02

  Hello, everyone. Welcome to Machine Learning Street Talk. I'm your host for today, Maxwell Ramstead. Very excited to be here. I'm subbing in for Tim Scarfe, who unfortunately is down with a little bit of COVID, but I'm sure we'll have a really interesting conversation. It promises to be a super interesting conversation because we have a really awesome guest for today. Very excited to have Guillaume Gilverdon, also known as Bef Jesus on X and just generally online in the meme space. Very excited to have you, Gil. How are you doing?

• 3:01Speaker 00

  Yeah, thanks for having me. Thanks to MLST for hosting. And you know, it's great to talk to you again, Maxwell. I think this collaboration, at least for this conversation, has been a long time coming. So it's a great platform to do it. And yeah, let's get to it.

• 3:19Speaker 02

  So you're a very interesting character, Gil. Like you're a fascinating researcher. I think you're a big presence like in the meme space as well. Do you want to tell us a little bit about yourself? Like, tell the audience a little bit about your trajectory. How did you wind up being, you know, the CEO of a thermodynamic hardware company and also, you know, basically like a well-known, you know, person in the meme space as well?

• 3:49Speaker 00

  Yeah, I mean, it's been a long journey. I guess I've lived many lives. Growing up, really, I wanted to, really, I was pursuing theories of everything. I want to understand the universe, as one does, and eventually leverage that knowledge to expand civilization to the stars. Originally, my plan was to become a theoretical physicist and work on quantum gravity and figure out some, you know, exotic form of propulsion that would obviously, you know, I thought that was the bottleneck for the expansion of civilization. It was the speed of propulsion. And so I went down that path, you know, was a big fan of Feynman, Stephen Hawking,

• 4:33Speaker 00

  growing up and of-

• 4:36Speaker 02

  So this is like a childhood project, like since you were a wee boy-

• 4:40Speaker 00

  Yeah.

• 4:40Speaker 02

  You've dreamed of-

• 4:41Speaker 00

  Yeah, no, I was seven, I was like, you know, talking about subatomic particles and-

• 4:46Speaker 02

  As all seven year olds.

• 4:47Speaker 00

  Yeah, and I wanted to be a physicist, and I want to be, I would, I used to call it astrophysicist, I didn't know what theoretical physicist was, but I want to be an astrophysicist, I want to work on, you know, FTL travel and stuff like that. And over time, you know, I did, I did the career path for theoretical physics, you know, I did math and physics in undergrad at McGill, and then I went to Waterloo, the Primater Institute for Theoretical Physics, you know, met some of the greatest minds on Earth, you know, they're really strong caliber. But what I, what I realized going through theoretical physics was that the reductionist approach to physics was failing us, right? You know, originally I was like, okay, well-

• 5:29Speaker 02

  Do you want to, just for the benefit of our audience, what do you mean by the reductionist approach?

• 5:32Speaker 00

  Yeah, yeah, yeah.

• 5:33Speaker 02

  And what do you think were the problems?

• 5:35Speaker 00

  Yeah, and, and, and I guess, you know, these two sides to my life now came from my, my reaction to realizing this failure. And the reductionist approach is kind of the traditional way we've done physics, it's kind of the very sort of rational, old school way to do things, which is like, oh, I have, I want to get a, I want to have a model, maybe it's an equation, it's some analytic model, and it has a few parameters, and I've reduced all of physics to a very simple model with few parameters, and ideally the least amount of parameters possible, you know, Occam's razor principle, and, you know, these equations with these few parameters allow me to have predictive power over the world, and with this predictive power, I can steer the world, I can do things, right? I could predict and control it. And

• 6:19Speaker 00

  that's the goal of physics, is to have better- Better models of the world. And what we realized was that, you know, as part of this, I guess I got swept up in the school of thought that was, you know, a generalization of Wheeler's IT from BIT, which was the IT from Qubit program, which, you know, was seeking to unify theoretical physics through quantum information theory, right? So viewing everything in the universe as one big quantum computer running a certain program or self-simulation. And that framework's actually a really useful kind of unifying framework to understand all sorts of systems. And people were studying sort of systems that,

• 7:03Speaker 00

  you know, have some connections between quantum gravity and regular quantum mechanics in the context of holography. So, you know, ADS-CFT for those that are familiar. But really, what I realized there was that even whether, you know, whether or not ADS-CFT was going to work, it was clear that the complexism approach to physics was the way forward. The universe is very complex, you know, even if you, even beyond just quantum gravity, just looking at condensed matter systems, you can have equations describing the microscopics, but you can't predict the emergent properties, right? You actually have to apply an amount of computation that is similar to that of the universe

• 7:45Speaker 00

  to have a prediction of what happens at a larger scale, right?

• 7:51Speaker 00

  You know, not all equations you can renormalize, you know, analytically. Renormalize means get an effective physics at a larger scale, at a more coarse-grained scale, right? That's how we go from different types of physics, from, you know, quantum field theory to quantum theory to, you know, statistical mechanics, and then eventually classical Newtonian mechanics, and then, and then even beyond that, we get to general relativity and so on.

• 8:16Speaker 00

  And so clearly, it was, the way forward was to understand everything as a complex system. But, you know, that was a sort of ego death, because in a way... You know, the human can't necessarily be the hero of the story. The model is no longer interpretable, right? If you have a, if you have something like a deep learning system, back then, we were looking at tensor networks, which are a different sort of parametric complex system that we use to to model, let's say, condensed matter systems or quantum gravity systems. If you, if you look at at these systems, they're no longer interpretable, right? I could, you know, I have these big networks and they're just as opaque as the system that I was trying to predict.

• 9:00Speaker 00

  But with numerics and with a lot of compute, they actually, you're kind of, you know, not deferring agency, but you're, you're leveraging a programmable parametric complex system to grok a complex system of nature for you. And in a way, it was like, OK, well, maybe, maybe I can't be the hero of the story. I won't be the one to figure out, you know, the grand unifying theory of physics. But maybe I could build a computer, computer software that, that can understand the universe or chunks of the universe for us. And actually, you know, the more I dug into it, it was clear that this was going to be the way forward. Right? And

• 9:41Speaker 02

  Basically, building a digital brain to overcome the limitations of our fleshy brains.

• 9:45Speaker 00

  Yeah, but even, that's correct. But then, you know, at the time we were studying, you know, I came at it trying to understand quantum mechanical systems. So you had systems that have quantum complexity. And there you can show that actually classical representations will struggle to capture quantum correlations. So already, you know, I came at it from a non-anthropomorphic form of intelligence. You know, to me, intelligence was just, I'm trying to, to compress, learn compressed representations of systems through parameterized distributions. Or in my case, it was parameterized wave functions and density matrices, right? Which is the more general form.

• 10:28Speaker 00

  So, so that actually got me into a field. You know, initially, it was mostly numerics with tensor networks. But there was another institute I was part of at the University of Waterloo, which was the Institute for Quantum Computing. Right? And there is very interesting because we had these controllable quantum systems. We had these control parameters, right? And

• 10:51Speaker 00

  It was, it was, it became clear to me that there was maybe a way to, to run these representations. We were trying to learn of quantum systems on a, that are parametric, on a parametric programmable quantum system, right? And so now we could fight fire with fire, right? We can have parametric quantum complexity that's tunable to understand quantum complexity of our world, right? And that's, that, that was actually my entry into artificial intelligence. And, um, you know, I, I wrote up some of the first algorithms for, uh, quantum neural networks. So that's what we called them. They have nothing to do with actual, uh, neurons. Uh, they're just parameterized quantum programs.

• 11:32Speaker 00

  But, um, essentially, um, you know, one of us was one of the first quantum computer programmers. It was the first user, Ragetti and so on, um, which was one of the first startups. But, uh, essentially that got me on the radar of, of, of Google, um, who then approached me, uh, and, and a team I built, Waterloo, uh, an open source, uh, team to, uh, to, to go work at Google and build a product known as TensorFlow Quantum, which was a product focused on, um, creating software that allows us to learn quantum mechanical representations of, of quantum mechanical systems in our world. And, um, and that was my entry into, into, uh, AI, which, uh, you know, given that this

• 12:17Speaker 00

  is a technical, uh, podcast, I'm kind of, uh, you know, going a bit deeper than I, than I usually do, uh, which is a really nice in the backstory here. But over time, what I realized was that, um, actually this sort of, you know, physics based approach, right? Like if, if we have, um, if, if we're trying to understand the physical world with representations, we want to have, you know, physics informed, uh, representations or physics inspired representations, you know, just like, um, to understand a quantum mechanical system, I use a quantum neural network. And then, and then the dual to that is if I want to run and learn and train these representations that are physics inspired or physics based, then I need a, a physics based accelerator, right?

• 13:02Speaker 00

  That, that, that where, where that type of program kind of fits natively and can be executed as, uh, physics. And, you know, in my case initially was I want to understand quantum mechanical systems. I want to learn, um, uh, quantum physics based programs and I run them on quantum physics based. Processors, right? And so.

• 13:21Speaker 02

  Then I have a clarification question just for the benefit of the audience.

• 13:24Speaker 00

  Yeah.

• 13:25Speaker 02

  Because we have a very technical audience, but let's just make sure everyone follows. So what do you mean precisely by physics-based computing? So the question I ask because it might confuse some of our listeners. In some sense, all computing is physics-based, right? Like, it's all shuffling around, you know, electrons shuffling around in circuits and so on. What do you mean specifically by physics-based computing? And how would that differ from, you know, the kind of commonsensical or classical notion?

• 13:54Speaker 00

  Yeah, I mean, a quantum computer is a computer that leverages quantum mechanical evolutions as a resource, I would say. It's, you know, there's more formal definitions of what a quantum computer is. But a physics-based computer, I would say, of course, like every physical thing is embedded in the physical universe, so everything is physics-based. But it is, it's true that it's kind of a, it's kind of a continuum, right? Because you can have physics-inspired computers that are doing digital emulations of physics-inspired algorithms. So I would say that's physics-inspired. I would say a physics-based computer, at least for quantum mechanical computer, is

• 14:39Speaker 00

  one where you have a sort of parameterized Schrödinger evolution that you control. And you can show that it has some unitarity. And then in our case, and we'll get to that, you know, we're building physics-based stochastic computers or stochastic thermodynamic computers. So there are parameterized stochastic evolutions that we control. But, you know, in principle, you could try to emulate a quantum mechanical computer, emulate a stochastic computer digitally. Of course, in the case of quantum mechanical computer, there's proven separations of complexity there. And there's experiments showing that you can't emulate them at scale. In our case, with stochastic computers, there's no complexity class separation. It's just kind of.

• 15:23Speaker 00

  Just orders of magnitude constant speed up or energy efficiency gain, which practically makes them intractable to emulate at scale.

• 15:32Speaker 02

  Right.

• 15:33Speaker 00

  Right. Not impossible, though, of course. But yeah, so this journey, and feel free, if you have a better, I think you have a better definition of physics-based computers. I don't know if you want to go into it.

• 15:47Speaker 02

  Well, no, it's consistent with what you were saying. I mean, I would, at a very high level, I would say that a physics-based computer is essentially a computer whose components exploit the actual physical properties of the components in order to perform computations more efficiently. So rather than like, you know, use digital computation to simulate what, for example, is like a quantum phenomenon or a thermodynamic phenomenon, you actually use the probability densities that these systems embody, for example, at equilibrium and exploit those properties in computation.

• 16:22Speaker 00

  Yep.

• 16:23Speaker 02

  Would you agree with that as a broad definition? I mean, I know I'm not.

• 16:26Speaker 00

  That's for a thermodynamic computer, right? But yeah, so there's different kinds of physics and different kinds of corresponding physics-based computers. Arguably, you can have one, you can have probabilistic or quantum, photonic computers. Of course, you can have, you know, you can have physics-based computers in all sorts of substrates. Technically, you could probably train a neural net from puddles of water, right, if you wanted.

• 16:52Speaker 02

  Right.

• 16:53Speaker 00

  And that would still be a physics-based computer. So I think, you know, it's kind of a very broad community. I would say physics-based computing, if we include quantum computing and alternative computing, and there's all these archipelagos that don't necessarily talk to each other. And I think there could be more work in trying to unify the community because there's a lot of tools that could cross-pollinate between these different substrates. But to get back to our, I guess, our main line here, which is, you know, how did I end up going from quantum to thermo? Well, you know, I think going from being one of the first programmers

• 17:33Speaker 00

  of AI on quantum computers and seeing the field kind of evolve over several years. I mean, I was. Almost eight years in quantum computing. It wasn't progressing as fast as I wanted. And I was seeing a sort of writing on the wall where actually to have a quantum mechanical computer, you try to keep the computer at perfect zero temperature, right? Zero entropy. So you're constantly pumping out entropy that is seeping into the system through noise, right? And that's quantum correction and fault tolerance. And it turns out that most of your computation, most of your energy is going to be sunk into that pumping. So it's like an algorithmic form of refrigeration.

• 18:17Speaker 00

  And to me, it's like, okay, well, you know, it's just like, you know, if you have a fridge and you're trying to keep your freezer really cold, it's going to use up a lot of energy if the rest of the room is at room temperature because that gradient of temperature is very difficult to maintain. And so it's like, okay, what if we had a hotter physics-based computer? Maybe it would be much easier to maintain, you know, just the first thought, right? And so what would that look like? Ah, well, you know, in the limit where, you know, you let the noise seep in, things become mostly stochastic and the quantumness kind of fades out. And we know this from sort of renormalization of physics.

• 18:59Speaker 00

  That's why we don't have to care so much about quantum physics day to day because it gets washed out at larger scales, right? And things become...

• 19:07Speaker 02

  Would you say that it's accurate to frame it this way? You know, in classical computers, the scale of the fluctuations is dwarfed by the scale of the components of the system that you're considering. So you can basically ignore the noise for the most part, you know, assuming that you cool the system appropriately. And in quantum computing, it's almost the other way around. What you want to do is basically to engineer the scale of the fluctuations that are so big, basically, that you can kind of ignore them relative to the component size. Then, and this is just a, you know, an attempt at framing what you're doing. Feel free to disagree with this.

• 19:48Speaker 02

  And then in the thermodynamic regime, what you have is fluctuations that kind of coexist at the same scale as the components that the hardware is made out of.

• 19:59Speaker 00

  Yeah, I mean, you know, we're in Wimbledon weekend right now in London and, you know, you don't have to understand the vibrations of the molecules. At the molecular level in the tennis ball to be able to predict its trajectory, right? You kind of get a mean field sort of prediction, right? So that's Newtonian. At the quantum scale, it's more subtle because it's no longer probabilistic fluctuations. You know, you're essentially, it becomes all purely quantum superpositions, right? And the ideal quantum computer has no probabilistic uncertainty.

• 20:39Speaker 00

  And in fact, when there is probabilistic uncertainty, you know, that's when the quantum computer loses its quantum coherence. It becomes non-quantum, right? But we have a very similar problem with classical computers that I realized, which was, you know, we use all this power to be, again, to have a signal that's so strong relative to the jitter of electrons and the amplitude of that jitter in order to maintain determinism, right? We want our transistors to be absolutely on or absolutely off because, you know, we have a lot of transistors and we want the computer to be in a determinate state that we have control over, right? Because, you know, humans like to be in control, right?

• 21:22Speaker 00

  When we're not in control, there's anxiety, right? But, you know, I think, I think, I think we got to learn to let go, you know? Just like I learned to, in a way, let go of control mentally by giving up the reductionist approach, which is like the human interpretable approach to physics. We have to sort of, you know, just like we did with deep learning, we let go of software 1.0, imperative programming, and let gradient descent be a better programmer than you. And that was very humbling for some. Some are still trying to hang on for control, right? With ML interpretability research and all this AI safety research and still want to

• 22:03Speaker 00

  feel like they're in control or that they understand the complex system instead of kind of letting it do its thing and letting it figure out what's best, right? And that's also kind of my main gripe with the whole AI safetyist field. Of course, there should be some research in interpretability, but I just don't think... They're gonna go that far, but to close the loop here, I think we should also literally sort of let go, loosen our grip on electrons and hardware. And that's what we do, right? Like essentially, we go from having a very tight grip and yanking these signals around to kind of loosening our grip, letting there be fuzz and kind of gently guiding the signals, right?

• 22:42Speaker 02

  And so we can- So in some sense, you're kind of switching teams, right? So, you know, in the more classical kind of way of thinking of this, what we're trying to do is to keep the noise at bay, right? To make our systems as non-stochastic as possible.

• 22:55Speaker 00

  Filter it, pump it out, pay the price.

• 22:57Speaker 02

  That's right. But exactly, what you're doing is saying, no, no, no, like harness it, right?

• 23:02Speaker 00

  Yeah, but as we know, like from Maxwell's demon, you know, the classic fable, I don't know if we want to go through it right now, but you know, there, people can look it up, Maxwell's demon shows you that knowledge comes at a cost, right? Like reducing-

• 23:19Speaker 02

  That's right.

• 23:20Speaker 00

  Reducing entropy in a system, keeping something in a deterministic state always costs you energy. And so if every clock cycle, we're trying to, you know, prevent the classical or quantum computer from decaying to a naturally probabilistic and relaxed, closer thermal state, then we have to pay the price. We have to pay energy to maintain determinism and reduce entropy. Whereas a thermodynamic computer is not always at equilibrium, but we're dancing much closer to equilibrium. And it's much cheaper energetically to sit in those states and maintain those states, right?

• 23:56Speaker 02

  Do you want to maybe just like walk us through, you know, how are you actually like designing these things?

• 24:03Speaker 00

  For a technical audience, I guess, there are Markov chain Monte Carlo accelerators, right? And we support discrete variables, continuous variables, mixtures of the two.

• 24:15Speaker 00

  And essentially, we've found a way to harness natural stochastic physics of electrons in order to accelerate Markov chain Monte Carlo.

• 24:26Speaker 02

  Right.

• 24:26Speaker 00

  And there's, there's some subtleties on how we do that mapping, but you could just imagine we're embedding into the stochastic dynamics that are parametric and that we have control over those parameters. We're embedding our continuous or discrete variable MCMC into the dynamics of the electrons on the device, right? So it is partly analog. It is stochastic, but it's actually, our most recent chip is a mixed signal chip. So we use digital classical components and sort of stochastic electronics. And the two have to

• 25:01Speaker 02

  Right.

• 25:02Speaker 00

  interact just like, you know, in a Metropolis Hastings algorithm, you have some components of the algorithm that have some entropy, some proposals, and then you have some non-random parts, right? Like computing, acceptance or rejection.

• 25:20Speaker 02

  But I guess this helps you address the kind of paradoxical situation where, because you know, I was just saying, in some sense, from the hardware's perspective, you're switching sides and joining, you know, the side of noise. But from the point of view of software development in the context of AI and machine learning, this has already happened, right? Like, there's a kind of paradox in our current architectures where, as you just described it, we spend inordinate amounts of, you know, energy and effort pumping the noise out of the system. But then, you know, with sampling-based methods, like, we then reintroduce it through the software.

• 25:56Speaker 00

  Yeah, and you could think of even a transformer, you know, you have a softmax layer at the end. A transformer is a big probabilistic computer already. So we're running probabilistic software on this stack that was made for determinism. That's highly inefficient. And then now with test time compute, right? Thinking at test time, you know, Monte Carlo tree search and, you know, all sorts of RL rollouts. That's a Monte Carlo algorithm, right? So that can be, yeah, tree of thoughts. It could be discrete diffusion. It could be now even for content, there's diffusion models. Those are also, it's also a Markov chain, right? You're reversing a Markov chain there.

• 26:38Speaker 00

  So algorithms, the biggest workloads that are eating the world and consuming a ton of energy are actually probabilistic workloads. They are probabilistic graphical models. And so it's kind of funny, like, there's just like a, I guess, I think it's just the community, there's kind of the new additions to machine learning that don't learn the fundamentals, it just goes straight to transformers, whatever's hot. And then I tell them, we're building an accelerator for probabilistic graphical models, and they look at me like, what are you talking about? It's like, actually, you're running probabilistic graphical models all day. And it's kind of funny that, you know, there's a gap there. But hopefully, I think, coming on this podcast, and hopefully, getting more in the machine learning community

• 27:21Speaker 00

  to talk to each other, you know, I think probabilistic N L, hopefully, will have a resurgence. And of course, we're trying to stimulate that, right? Because what, you know, there's a co-evolution. What if all together fits together, right? And there's an evolution of hardware and algorithms. And we have, as investors, two of the authors of the transformer paper. And, you know, they say, they kept telling us transformers are not sacred, right? They were what worked on the hardware we had at the time, which was Google TPUs. And, you know, the hope is that, you know, there's going to be new foundation models, or new models that run more natively on probabilistic hardware that are really efficient on our hardware. And we're not just importing current day

• 28:03Speaker 00

  models. But hopefully, you know, by, if you view the algorithmic landscape as having a certain fitness function of, you know, what, in terms of performance you get, in terms of costs, but also it's induced by the current day hardware that's available at scale, right? And if you change that hardware substrate to a different substrate that has, you know, different preferences in terms of the structure of the algorithm that you're running, then you're changing the fitness landscape. And if you have a sudden shift in the fitness landscape, you have a sort of Cambrian explosion, you have this sort of high temperature phase of search, right, in the landscape. And that's hopefully what we're gonna, we're gonna cause, right? And that's very disruptive. So, you know, incumbents have all the reasons to

• 28:47Speaker 00

  To be skeptical, but at the same time, it's funny that even the current incumbent algorithms are converging towards sampling and more probabilistic algorithms, which wasn't obvious when we started the company in 2022, but that was the prediction, right? That there would be a sort of, you know, my joke is that we're kind of interpolating between deterministic forward pass models and full EPMs, right? And you can view diffusion models as interpolation there.

• 29:19Speaker 02

  I mean, correct me if I'm wrong, but your view is that this is essentially an inevitable development of tech, right? Like, you've talked about the thermal danger zone and the inversion of this Moore's Law into what you call Moore's Wall. Do you want to run us through the logic there?

• 29:36Speaker 00

  Yeah, I mean, if you, you know, I came from quantum computing there. If you try to scale up your system, you have noise seep in. So again, you're kind of, you're going from the quantum zone of physics to the thermodynamic, you're kind of edging on it. And then if you're in the classical zone, but you try to get smaller, then you're getting in the thermal zone again, right? In terms of the physics and then the jitter of electrons, because your transistors are so small, the fact that there's very few electrons and the transistors are small, the jitter of all the electron, the electron population matters compared to the amplitude of the signal.

• 30:13Speaker 02

  Right.

• 30:14Speaker 00

  And then typically, we like to have our computers have an error rate of 10 to the minus 15, so that you can do many, many operations before there's an error. And then so that you don't actually have to run error correction, right? The only computers that usually run error correction are those that we send in space because of radiation, right? But the reason we can't scale down deterministic computers is because of this thermal danger zone and they're innovating in all sorts of ways. They're doing all sorts of weird fins and. All sorts of weird designs that are sort of hardware level error correction.

• 30:49Speaker 00

  But.

• 30:49Speaker 02

  Those are, those are always going to be ad hoc in some sense, right? Because what you, what you're going to run into is just the scale problem.

• 30:56Speaker 00

  Yeah, yeah.

• 30:57Speaker 02

  As you miniaturize at some point, like these wires are going to become.

• 31:01Speaker 00

  To use less power, you need to use less charge. That's not controversial. And when you get to very little charge, the fact that charge is discrete gives you noise, period. Right? And so, so you're, you're going to have to go thermodynamic at some point, right? And, you know, at, at the end of the day, we have proof of existence of a really kick ass AI supercomputer that we're both using right now to talk to each other. It's our brains. And, and that I would argue is a thermodynamic computer, right? Because there's a, there's master equations, right? You know, describing the chemical reaction networks in your brain of, of neurotransmitters hopping around and you could, you could just.

• 31:42Speaker 02

  Operating really close to the Landauer limit as well, right?

• 31:45Speaker 00

  Right. And so, and so, you know, if we're doing very similar physics, but with electrons sloshing around a circuit, then, you know, there's a very, much longer chance that we could run a very similar program in a very similar fashion, arguably with even more energy efficient components, because electrons are much lighter than big neurotransmitters.

• 32:05Speaker 00

  And so, you know.

• 32:06Speaker 02

  So basically what you have is like a programmable bowl, right? Well. You can think of these chips as like a kind of programmable bowl? Do you, do you want to maybe tell us how they're used for computation?

• 32:17Speaker 00

  I think, so, you know, I think we started, for context, we started off in, in superconductors and, and there, and, and the reason we started in superconductors was because there's this beautiful theory in, in quantum computing, actually, called circuit quantum electrodynamics, where you go from, here's my circuit and here's my Hamiltonian, right? Directly from the circuit. And, and, you know, I think that should win a Nobel Prize or something someday. But that, that's basically how we engineer quantum mechanical systems to have certain desired physics, right? And, and, and the point there is that a Hamiltonian gives you a notion of, of energy, right? In the, in the classical regime.

• 32:58Speaker 00

  And so we had a very direct way to go from, hey, I want this energy. Here's how I design my circuit. Right? And so that's where we started. Furthermore, it was the way to build the most macroscopic thermodynamic computer you can build. Any other companies with claims to build a much more macroscopic thermodynamic computer either did it digitally or emulated it, so it's not a real thermodynamic computer. But, you know, we had to super cool it because it is too big, right? And, you know, Boltzmann distribution is, as you go smaller, you have higher frequencies, you can go to higher temperature, and then we move to silicon. But to answer your question about energy-based models, yeah, you could view,

• 33:42Speaker 00

  you could view, you could view our chips as a time-dependent programmable energy function, right? And you have something akin to large wave dynamics, more generally diffusion in that landscape. And large wave dynamics is, happens to also be an MCMCL algorithm, right? So there's a, there's a perfect match there between the algorithm that you can use for Bayesian inference, for anything really, and the native physics of the chip, right? And so it's almost like, it's, it's, it's been there the whole time, right? And it's like, we've been doing algorithms that were physics that we could literally implement, right?

• 34:25Speaker 00

  It's a literal analogy, and we're instantiating as an analog computer. It's, that was, that was too beautiful to not build it. So we did build it.

• 34:35Speaker 00

  Hopefully, we could cut some B-roll. I brought some, some superconducting chips here. We'll do that later. But now, actually, I think the big, the big breakthrough and the big challenge was to

• 34:48Speaker 00

  That's it, eh? Move to silicon, yeah. And getting programmable stochastic physics in silicon was a whole kind of order of magnitude more difficulty, and we had to really innovate in terms of understanding, you know, stochastic mechanics of electrons in silicon. And we built our team for that. And, and essentially, we kind of reproduced some core primitives,

• 35:21Speaker 00

  namely, the one we're talking about for now is the probabilistic bit. And the probabilistic bit, you could think of as a You know, a double well system and you can you can tune the tilt and so on. So so if you have, if you consider bouncy balls in this landscape, they can you can control how much time the bouncy balls spend in in one well or another. And you can consider one well zero, the other well one. So essentially you have a signal that's essentially dancing between zero and one, right? And you can control how much time it spends in zero and one. And and and so that's like a fractional bit, right? So, you know, going back to what we were saying earlier about, you know, coming from

• 36:04Speaker 00

  it from qubit, you know, we we school of thought, we we basically made pubit from it. And it initially was superconducting materials, but now we've we've done pbits in silicon and achieved really, really high energy efficiencies. You know, our pbits can generate bits controllable bits of entropy with only a few hundreds of atodules. And we've actually, as of recently, submitted our our results for peer review. And we're gonna put them on archive, probably timed with some other announcements in the coming months. But it's really exciting time. That's so cool. And and again, this was a concept for a very long time, and it was a big risk for me. You know, initially, when I left theoretical physics and went all in on quantum machine

• 36:49Speaker 00

  learning, everybody told me like, really, you you're gonna be a quantum machine learning lead at Google? And everybody was doubting me. And then I became kind of well known. And then I eventually led a team. I led quantum machine learning at Alphabet X. And and then I quit all that. I quit quantum computing, quantum machine learning. And it's like, I'm gonna take even more risk. I'm gonna build a whole new paradigm of computing from scratch, from the concept up. And everybody thought that was crazy. And now we're here. Now we've made a lot of progress. And now we're we're scaling, right? And because there's nothing stopping us from scaling at this point, because we, you know, we've de-risked the manufacturing as well, which is usually not something academics think about.

• 37:31Speaker 00

  But if you're a startup and you have to to scale or die, eat the world or die, you have to kill every risk possible. So, you know, we are looking to scale to millions of degrees of freedom next year. And I think we're gonna we're gonna hit it, so.

• 37:47Speaker 02

  That's really exciting. So, so your first chip, right, had um three P bits.

• 37:53Speaker 00

  Yeah.

• 37:54Speaker 02

  And the latest has about 300 degrees of freedom.

• 37:57Speaker 00

  Yeah.

• 37:58Speaker 02

  Correct?

• 37:58Speaker 00

  They're not all P bits, but we'll, we'll, we'll get to that someday.

• 38:02Speaker 02

  Yeah. Right. And now, and millions of degrees of freedom.

• 38:06Speaker 00

  Yeah.

• 38:06Speaker 02

  Uh, next year.

• 38:07Speaker 00

  Yeah.

• 38:08Speaker 02

  That's very cool. So, help me understand the, the layout, uh, like the general space here. So, I mean, presumably, uh, you don't think that this is a, or, or do you? Do you think this is a wholesale replacement for the current stack? Or do you see like, cause I could see a world where like you interface, you, you, you use thermodynamic compute when it's relevant, but you mesh this with digital and quantum at the appropriate junctures so that you get, what you really get is like a multi-scale stack where each hardware bit is specialized for the kinds of computations that run natively on that kind of hardware.

• 38:44Speaker 00

  Yeah.

• 38:45Speaker 02

  Which out of the?

• 38:47Speaker 00

  Yeah, absolutely. I mean, you know, if you, if you're, I think there was some work from Max Tegmark on um correspondences between sort of um the information bottleneck principle and sort of down sampling and the hierarchy and, and machine learning and the renormalization group, right? So in physics, you go, you know, if, if you're trying to learn from quantum mechanics and you down sample, you get statistical mechanics, you down sample, you get to Newtonian mechanics. You can imagine if I had a, a god neural network that just uh compressed all the information in a certain region of space, which is actually the thought experiment that got me into, into quantum machine learning. I was, I was trying to understand black holes as a machine learning system. But let's see, you, you created that system, then you could imagine

• 39:32Speaker 00

  the first few layers are quantum and then some layers are probabilistic and later they're deterministic. Just like as you distill, distill the information, uh, you don't need that sort of, you know, initial, initially you need some, some quantum complexity, later you need some entropy, and then later you just need to do some classical, you know, coordinate transformations, right? And, and they all work together. But in a more practical, practical workflow, yeah, I, I mean, you know, very often. Whether you're doing simulations of stochastic differential equations, you're trying to simulate a physical system, whether you're doing some sort of discrete or continuous diffusion.

• 40:14Speaker 00

  You know, there's very often, there's a classical function, often, you know, differentiable program that determines sort of some probability distribution that you want to sample from, whether it's in latent space or, you know, for these transitions and the denoising. So the two work together. You don't necessarily need entropy everywhere in your graph all at once, right?

• 40:36Speaker 02

  Right.

• 40:37Speaker 00

  In principle, you could use a probabilistic computer for deterministic operations, but it's not going to be the best at that, right?

• 40:45Speaker 02

  Right.

• 40:46Speaker 00

  In the low-precision regime, right? You could think of P-bits as literal fractional bits. So you can get into the fractional bit precision work. And so that can be interesting, but not everything is well-suited for that low precision. And, you know, I do think quantum computers will have some applications, but again, they would be supplements for just quantum mechanical systems, to understand quantum mechanical systems as a supplement to probabilistic and classical computers. But I would say that most things are, you know, would be well-covered by probabilistic and deterministic representations running on probabilistic and/or thermodynamic and deterministic computers.

• 41:31Speaker 02

  So currently, the superconducting chips require a lot of cooling, right? You need to get them around 1 Kelvin. Is that the case?

• 41:37Speaker 00

  Yeah, I mean, depending on the material, you can be a few hundred milli Kelvin or you can get to a few Kelvin. A few Kelvin, usually using niobium. I think that you don't need as big of a fridge. For us, it was an interesting sort of experiment, and we have a bunch of results there. A paper coming in the coming months.

• 41:59Speaker 00

  Essentially, it's as efficient as we could imagine building a thermodynamic computer. And again, it was just to get people to imagine, hey, actually, or realize, rather, hey, actually, there's forms of computing that are far more energy efficient by like an infatible amount of orders of magnitude, far more efficient than than digital computers, right?

• 42:23Speaker 02

  Right, because I'm just doing some back of the napkin, because so you guys target 1,000 to 100,000 x energy efficiency gains at the chip level. And what I was wondering is, how does that discount against the cryogenic energy costs? Like, how does that factor into your efficiency calculation?

• 42:41Speaker 00

  Yeah, yeah, yeah, so, you know, any, any sort of, anything from that, those results were, you know, and we put the asterisk there, like, that's just the chip, you know?

• 42:50Speaker 02

  Right.

• 42:51Speaker 00

  And, and, and in general, you know, you'd have, the thought experiment was, if you scaled a very large network of superconducting chips to football field sized, and you had a ginormous dilution fridge, right, again, it's like, the, the big energy cost is maintaining that, that, that boundary, right, with the, the outside world. If you had a ginormous dilution fridge, and you had millions and millions of, of, of P-bits and superconductors, you'd have the most energy efficient probabilistic computer, right? Is that practical? Probably not. Should a startup be doing it? Probably not either. Could a government do it in some sort of crazy moonshot? Maybe. And so, we're kind of just going to put the idea out there for academia, national labs,

• 43:35Speaker 00

  and whatnot to, to pick it up. I think for us, it was also just a great learning platform. Superconductors are ironically pretty accessible, a bunch of universities have fabs where you can experiment with them. So, for us, it was kind of like, hey, we're going to want a community of people working on thermodynamic computing and experimenting with new primitives and, and showcasing new algorithms, so we're going to put out that work into, into academia. But for us, the product is, again, the, the silicon, the silicon chips, which, you know, we could have decided to run them in cryo, cryo CMOS, but we ended up deciding to go for room temperature. Of course, a thermodynamic computer...

• 44:15Speaker 00

  At lower temperature, it consumes less energy, right? Of course, you're assuming you're maintaining the bath at that temperature, but then you have the costs of the cooling, right? So, you know, but you know, if you had a if you had a von Neumann probe thermodynamic computer, maybe it could run much, much, much cooler, right? If it's going to go and enter in interstellar space, it's not going to encounter that much heat. And so maybe we'd modify the design for that. But that's, you know, much problem for in the far future.

• 44:53Speaker 02

  And then coming back to impacts on the AI industry. I mean, EBM's have been around, energy-based models, right? Have been around for a long time. Would you say that this is really the, you know, the key to unlocking their potential? Because up until now, they've been basically limited by their their inefficient sampling, right? So would you say that this is really like the technology that we need to really do the EBM thing seriously?

• 45:19Speaker 00

  I think, I think modern neural networks are just, you know, mean approximations of EBM's, right? Like neural networks came from EBM's. The 2024 Nobel Prize was, you know, and backprop came from looking at mean field of EBM's. And so to us, we're just, you know, yeah, creating the hardware for the ancestors to neural networks. And they're kind of a superset, right? You could just take averages and get deterministic operations. And so, yeah, our hope is, you know, people think of going more probabilistic with their algorithms now that sampling is far more energy efficient and far faster. But because we've been stuck with deterministic computers,

• 46:01Speaker 00

  people have just tried to avoid, you know, using primitives that have, you know, they would just represent distributions by their moments, right? And they would use, you know, exponential families where you don't need too many moments, such as Gaussians, right? Because you could just represent them as matrices and vectors. And matrices and vectors can, can, can fit on a GPU really well, right? And you could do some transformations there. So there's been a bias in, in algorithms towards what runs well on today's, today's hardware. And hopefully with thermodynamic computers, it changes that. And So we're trying to get people interested in the in the space and to start imagining what they would do with, you know, a multimillion.

• 46:45Speaker 02

  This is how you break free of the.

• 46:47Speaker 00

  Go ahead.

• 46:48Speaker 02

  Well, this is how you break free of the vicious cycle that we're stuck in.

• 46:51Speaker 00

  Yeah, we're stuck in a.

• 46:52Speaker 02

  Like a hardware stack that works in a certain way. We optimize our software for that. Then we build bigger hardware and, you know, so this is how we break free from.

• 46:59Speaker 00

  Yeah, but I think I think we're I think we're going to break free one way or another. Because frankly, even if you just do a back of the envelope calculation.

• 47:11Speaker 00

  Well, right now, we're going to run out of power trying to scale AI. We can't even. You know, if everybody were to use like an agentic, you know, big model, chat GPT or Grok. Model, you know, we'd run out of power in the United States. You can't deploy it to everyone. Can't have everyone using it like with high intensity right now. We're going to run out of power. And then that's not even getting into sort of video models, world models, which are going to be necessary for embodied intelligence. At that point, you know. If you try to scale to the planet, we're going to run out, even if we produce the power. Let's say there was some moonshot and. Let's say we even figured out nuclear fusion, right? Let's say we just did that and we scaled that very quickly.

• 47:56Speaker 00

  We'd run out of, or, you know, we're going to generate more, we're going to double or triple the amount of heat. Being radiated by the earth, right? So we're going to literally cook ourselves to death, right? We're literally cooked, okay? And so something has to change at the hardware layer. We can't scale with the current hardware. And so. And so people thinking we're just going to scale transformers and get to the moon and scale scale intelligence to the whole planet, they're just flat out wrong, right? It's like provably wrong.

• 48:23Speaker 02

  When people say, hey, you know, to scale up intelligence, we need to start building nuclear power plants. I always think, well, you know. Like I run on a glass of water and a banana, maybe a coffee in the morning, right? Like certainly this is not like a naturalistic way to think about how intelligence scales, right?

• 48:42Speaker 00

  Because you're a thermodynamic computer, right?

• 48:44Speaker 02

  That's right.

• 48:45Speaker 00

  That's the thesis. But, you know, you can imagine, right? Like just taking our current design and scaling it to a wafer. You would have about 1.5 billion P bits, which you could think of as as neurons and about 20. Twenty billion parameters per wafer and then you could do multi-layered programs of those. And that would run on not twenty kilowatts, which is what a current wafer scale system would run on. Or more, maybe a hundred. It's twenty watts. Right? Which is like our brain, right?

• 49:17Speaker 02

  That's crazy.

• 49:18Speaker 00

  And then if you have a hundred, or you know, you have a few hundred billion parameter model running on twenty watts. Then we're, you know, we're in the brain, we're in, we're in the same ballpark as the brain. Not quite, it might be like within ten X, but that's still much better than where we are, which is like a hundred million X. Right? And so that's where, that's where we're going. And, and again, it's not, it's not, to us, it's far less crazy than quantum computing. There is no like quantum computer in nature that has very, a very high quantum coherence. And we have exquisite control and very high quantum complexity. All, all systems decohere. We have, you know, multiple billion thermodynamic computers out there in the wild and they work pretty well.

• 50:00Speaker 00

  Um, so we're just trying to, to tap into the same physics. We're not obsessed with sort of biomimicry. We're not upset, we're not, we don't call ourselves neuromorphic computing. And we're just doing, again, probabilistic, approximate probabilistic inference as a service, but in hardware, in physics. Um, and, and to us, that's kind of the parent workload of, of most of AI. And actually, it's not just for AI, it's, it's for broader computing, right? There's a simulation, optimization. And yeah, you know, statistical inference, science at large can use this. So it's not just, it's not just for Jenny I, right? We're not just riding the current wave. Yeah.

• 50:38Speaker 02

  I have the, uh, you know, the, the pleasure of interviewing not one, but two people today.

• 50:43Speaker 00

  Sure. Yeah.

• 50:43Speaker 02

  I've got your, yeah, exactly. We've got your alter ego, uh, on the call as well. So I guess, you know, to, to kind of frame up the transition to discuss, cause, uh, you know, for, for the audience. Um, uh, Gil, uh, or rather AKA of Beth Jesus is the founder of a philosophical movement. Um, called effective accelerationism, which originated as a kind of counterpoint.

• 51:08Speaker 02

  To effective altruism, and I really want to get into, you know, the details there. But I guess to kind of set things up, like looking, you know, 10 to 20 years out, like, how do you how do you envision the future? And, you know, what role does acceleration play in your your vision for the future?

• 51:30Speaker 00

  Yeah, I mean, 10 to 20 years, I think I think we'll have embodied intelligence. I think, you know, thermodynamic computers will be pretty ubiquitous. They're going to be what runs embodied intelligence. We're going to have greater intelligence density in all our devices. We're going to have personalized, always on, online learning intelligence. And so, you know, my goal is for everyone to own and control the extension of our cognition. Because that is important to maintain.

• 52:01Speaker 02

  Democratizing intelligence.

• 52:03Speaker 00

  Not just democratizing, yeah, but yeah, truly, because because, you know, you know, democratizing access, it's like, hey, I give you access to the one god model that amortizes its learnings across the fleet. Now you're all mind merged with my Borg mind. Congratulations, you've been assimilated. That's not democracy, right? If you have control over its constitutional prompts, you're essentially controlling people by proxy and their thoughts and their will and their actions, right? If you control, it's not just, it used to be people control people's access to information and they steer them indirectly. But now it's going to be directly controlling the model.

• 52:46Speaker 00

  That's an extension of their cognition or their thought partner. And then that's much deeper and more subversive control. And so to me, I think that was the big existential risk that I was most worried about was a sort of precedent of people vying for control and vying for power, right? Over people.

• 53:02Speaker 02

  Right.

• 53:03Speaker 00

  And that's why I've been pushing for decentralized AI. And I've been pushing for sort of avoiding sort of overregulation of AI that would cause, you know, red tape inflation and would mostly serve the incumbents and would cause a centralization of. Of AI power. But yeah, I mean, you know, someday, I would imagine 10, 20 years out, hopefully, you know, we're wearing some Neuralink, you know, our whole skull is some Neuralink that is the thermodynamic computer that's an extension of our cognition. And it's as powerful, or it's not more powerful than your brain. And, and you have a full merge there. I think something more plausible on a 10 year time scale is a sort of soft merge, right?

• 53:47Speaker 00

  I mean, you're, you're the active inference expert here, but you know, if you have the same Markov blanket, right, you have the same perception and action states, right? So let's say I had an agent, let's say my glasses were very smart and they were always on, always listening, and I have an earpiece. I could have sort of subconscious thinking almost, it could just perceive things and suggest actions. So we're sharing perception and we're sharing actions through my body. We are the same agent, right? So that is, that is a form of merge. And you don't need a neural interface for that, right? And, and, and maybe we even have a way to communicate non-verbally or, or, you know, just like friends that hang out a lot, have a prior of each other's behavior, or their

• 54:32Speaker 00

  behavior conditioned on the world states. And then they don't need as many bits of information to, to, to adjust the new setting. And so that's, that's kind of a more plausible way. I think, I think, I think the soft merge is going to come first, but I think people are gonna, it's gonna keep escalating and then we're gonna go towards a, a harder merge, right? A hardware merge.

• 54:52Speaker 02

  That's a very compelling vision for the future. It's very techno-futurist, I love it.

• 54:58Speaker 02

  Do you want to give me the elevator pitch for EAC, Effective Accelerationism? So, you know, because I, I think these are all related. I think this sets you up nicely to kind of just, you know, present the, the kind of core idea, so.

• 55:11Speaker 00

  Yeah, I mean, really, EAC is a sort of meta culture. It's kind of, I call it like a cultural hyperparameter prescription, if you will. And it's one where we're trying to maximize growth of civilization as measured by our free energy at production and consumption, AKA the Cartership Scale. The Cartership Scale is a log scale tracking how much free energy is consumed and produced. And really, it comes from the sort of realization, as I was studying stochastic thermodynamics in preparation for founding Xtropic, I realized that there was this sort of generalization

• 55:54Speaker 00

  of Darwinian selection, which is thermodynamic selection, where each bit, you go from selfish genes to selfish memes to selfish bits. Every bit of information specifying configurations of matter is fighting for its existence in the future. And the selection pressure on those bits of information are whether or not they can further host organisms, an ability to understand their environment, predict the future, capture free energy, use it strategically, and grow. But really, the master metric is like how much, if I change this parameter, this value, and I let time evolve, how much free energy has the system dissipated through its trajectory

• 56:39Speaker 00

  over time? And the theorems of stochastic thermodynamics tell us that actually trajectories of the system that have basically consumed more free energy are exponentially more likely. And so that's how you have this sort of pruning of branches that consume less free energy. And so it's like, ah, OK, well, this is the golden metric of selection pressure on the space of bits. And so I will design the Highest fitness selfish meme that is EAC, which is figure out what is optimal for

• 57:24Speaker 00

  growth and do it. And so by construction, it should be the most viral thing and it will persist.

• 57:30Speaker 02

  Right.

• 57:31Speaker 00

  And it's surprising, you know.

• 57:33Speaker 02

  The assumption there is basically, the, the alternative is basically growth or death is what you're suggesting, right?

• 57:39Speaker 00

  Yeah, you know, we have the saying accelerate or die. And it sounds dramatic, but it's, it's, it's meant, it's meant to sound dramatic, but, but it's also, you know, reality. It's either you align yourself with growth and you're a part of it. Right? So whether you adopt, you adopt your culture, you adapt your culture towards growth and then you're a part of the growth and you benefit. Or you don't and then you get outgrown. And whether it's at a national level, right, with your policies, right? Again, EU versus US, US is obsessed with growth to some extent. And, and, and we've seen a sort of bifurcation, you know, in the GDPs and so on. So at a policy level, at a cultural level, even at

• 58:23Speaker 00

  an organization level, right? If a company is not obsessed with growth, eventually it just gets disrupted by one that outgrows them and then has more resources to, to outcompete them. Right? And so I think, I think the point is that essentially if people are open minded about new technologies and in general and, and, and lean in, they will be positive, they will be selected for. And the people that are skeptical, push them away, show them away, you know, want to go back to the cave, get selected out. And so even from an empathetic argument standpoint, trying to popularize acceleration, like awareness that actually this is how the world works. I'm sorry to break it to you.

• 59:07Speaker 00

  You either embrace the acceleration. Or you get selected out, it's also kind of like, hey, you know, if you care about yourself and your tribe, your company, whatever, you should lean in rather than lean out and don't listen to the people telling you, you know, fear mongering you, they're actually very often doing so out of self-interest and it's an actual form, I view spreading deceleration as a form of warfare that is psychological in fact, that's how I view it and that's why I've been such a warrior online against the decel mindset, I just don't see a scenario

• 59:48Speaker 00

  where being decel is beneficial to the host and in fact, it's just exploiting this gap in understanding of how the world works to destroy competition, right? And I mean, there's a lot of this in, you know, nature and in general and human dynamics, you know, people kind of sabotage each other to outcompete each other, you know?

• 1:00:16Speaker 02

  See, I've been thinking about this a lot, my good friend and colleague Axel Constant, actually, and I have a longstanding debate as to whether ethics, morality and this kind of thing can be reduced to free energy minimization at the end of the day. And I think it's a kind of difficult question because thermodynamics, right, describes all mesoscale objects, right? So basically, any configuration whatsoever will be, you know, will be minimizing free energy in this fashion, right? Like, you know, seeking to find pockets of free energy and dissipating it as efficiently as possible. I guess my worry is that, you know,

• 1:01:05Speaker 02

  Things that grow without boundary, without bounds, I mean, in the biological case, you have like cancer, for example. So I guess, like, what's the role of constraints in the maximization of entropy when we're kind of designing the social system? Um, it, it, cause sh- surely, you know, things like, you know, dictatorships, uh, you know, fascist autocracies, those also minimize free energy. And you might even think, um.

• 1:01:32Speaker 00

  They're local, they're a local optimum, right? They're not a global, right? Just like a cancer is, is like, it kills the hosts and that's suboptimal on a sufficient time scale. Again, it's not, it's not an instantaneous free energy dissipation, right? It's, it's, it's, it's, it's basically infinite time horizon, right? And so actually blowing up the planet does, uh, you know, exhaust, you know, burn up a bunch of free energy. But in the long term, you actually, it's, it's, you know, the same reason life exists. Uh, it's, it's much better to sort of conserve and strategically use free energy to secure more free energy. And, and keep going rather, and, and have some order rather than just like burn all and then go and have chaos, right?

• 1:02:16Speaker 00

  And thermalize in one go, right? Again, um, you know, that's why we have life, that's why we're not at equilibrium, right? Um, we're not just burning up all our fuel and just dying immediately. Um, uh, because, you know, as intelligent beings, we, we burn way more energy by being this sort of, um, you know, or partially somewhat coherent system that has predictive power of its environment. And we're kind of this energy seeking fire, right? And, um.

• 1:02:45Speaker 02

  Right.

• 1:02:45Speaker 00

  Um, yeah, I mean, but this is, you know, I, I would argue that, you know, EAC is also, it's kind of a call to kind of popularizing complexism and complex systems thinking and the free energy principle at large. And we have to think through this lens about all systems in society, from policymaking to, to technology, to innovation, to, to, to, to, to ba- basically everything, economics. Um, and, uh, and it's a, it's a different way of thinking, one that is somewhat scary. Um, you know, again, it's like, it's again, going from reductionism and rationalism to sort of,

• 1:03:26Speaker 00

  um, uh, complexism and, and post rationalism and, and it's, we don't have as much control and interpretability. We don't understand the world. As well, right? We kind of have to, you know, pardon my French, but fuck around and find out the FAFO algorithm, right? Like, but really, it's kind of like exploration and discovery, right? You kind of, a priority, don't know what's optimal, right? And in startups, you kind of learn this, the, you know, you have to actually go on the market and try stuff. Sometimes your prior, your model-based prior can't do that well because the system is too complex to have a model that has good predictive power. You actually have to be in a, in a sort of open loop.

• 1:04:09Speaker 00

  I would say that, that yeah, hopefully, hopefully there's kind of a renaissance. And, and these ideas become more popular in all sorts of fields of, of science. Clearly, it's eaten the software world and we're trying to make this sort of school of thought eat the hardware world. And I think we will, we will do it. But again, most fields of study could be revolutionized by sort of thinking through the lens of complex self-adaptive systems and the free energy principle.

• 1:04:39Speaker 02

  So, I mean, you've described EAC as a kind of hyperstitious meme. Do you wanna, do you wanna say a few words about what that means?

• 1:04:47Speaker 00

  Yeah, I mean, you know, because again, you're the active inference expert. You know, in, in, in active inference, you know, you have perception and action, right? You can update your, you know, based on your input sensory information. You can update your model of the world. So you're minimizing divergence of your model, of your, your own internal model to the statistics of the world. But then the dual of that is taking actions in the world to minimize divergence between the world and your predictive model of it. Right? And in a way, you know, we're just naturally biased towards, you know, in a way, the, the car goes where, where the eyes look, right? When you're driving.

• 1:05:31Speaker 02

  Right.

• 1:05:32Speaker 00

  And if we, if we look at very negative outcomes and we're obsessed with them, we will drive. Whatever whatever system we're thinking about towards those negative outcomes. An example of this.

• 1:05:45Speaker 02

  Right.

• 1:05:45Speaker 00

  An example, slightly controversial, is bioweapons research. And for example, COVID, right? I would, I would, you know, I would say that COVID was an accident from bioweapons research. It was probably defensive and we were trying to explore what would be a really bad scenario. What if we had, you know, this mutation and this mutation would combine and, and it would be a really bad virus? And then they start experimenting and designing in that neighborhood of virus subspaces that would have never occurred naturally. Right? Just from evolution. But because we were exploring that subspace of bad things, because we're obsessed with it, we made it happen.

• 1:06:31Speaker 00

  Right? And so, you know, to me, it's like, if we're optimistic about the future, we tend to steer things towards that optimistic outcome. As a startup founder, if you're not optimistic about your startup, statistically, you are, you are screwed. Right? Because.

• 1:06:47Speaker 02

  Oh, yeah. And you know, from the active inference point of view, every action begins with a false belief. Right? So first you, you believe that you're moving and then you reduce the prediction error in the direction of action. Right? So.

• 1:06:58Speaker 00

  Exactly.

• 1:06:59Speaker 02

  I find this extremely compelling. I mean, one of the things I find most inspiring about EAC is that you're trying to be like, you're trying to present a radically optimistic meme for the future. And I think, you know, there's so much, not just AI doomerism, but there's so much doom and gloom going around that just like, just from the point of view of neurobiology, we need these kind of like, almost seemingly like, delusionally optimistic beliefs to get off the ground anyway.

• 1:07:24Speaker 00

  Yeah. I mean, you could think of like the memetic sphere as like a meta cortex, right? It's a biological supercomputer. And EAC, we're just trying to do active inference towards better futures. Right? So we're spreading this meme of optimistic futures and it actually does steer the world. And, you know, it's been three years now and it has steered policies throughout the world towards. You know, going for these moonshots and resurgence of, you know, exploration of nuclear energy, decline the Carter-Shev scale, deregulation of AI, you know, widespread embrace of AI, companies being far more aggressive in their, their exploration of it. And, you know, overall-

• 1:08:08Speaker 02

  So how do we spread it?

• 1:08:09Speaker 00

  Yeah.

• 1:08:09Speaker 02

  How do we spread IAC? I guess my, my question is kind of motivate, I think most of, most of the very visible IAC figures, you know, Andreessen, Musk and company, they tend to be, you know, libertarian, right wing associated. If we want-

• 1:08:26Speaker 00

  Well, that happened after the fact, right? So I would say, you know, like Gary Tan is, is rather on the left and, you know, initially it was very political. I would consider myself still a centrist and kind of, you know, not hitting-

• 1:08:41Speaker 02

  Well, how do we get the abundance bros to join?

• 1:08:43Speaker 00

  I, I think, I think it's already happening.

• 1:08:45Speaker 02

  The Ezra Klein's and, okay, cool.

• 1:08:47Speaker 00

  I think, I think that was a reaction to, to IAC in kind of the techno optimist, you know, you kind of end up getting clustered with one party in the US. You know, in other countries, it might be different, but there, there needed to be a, like, actual techno progressive, let's say, versus techno aggressive, you know, cluster on the left. It's not clear that that faction is kind of the dominant faction of Democrats. I really hope it, it becomes that. You know, again, IAC is kind of like, it's kind of like an RL algorithm. It's not clear what the optimal policy is.

• 1:09:23Speaker 02

  Right.

• 1:09:25Speaker 00

  I think, I think a few years ago, I was seeing a sort of convergence towards a lot of top-down control, a lot of top-down power.

• 1:09:34Speaker 02

  Yeah.

• 1:09:35Speaker 00

  And I think, I think the optimal is always balanced. You need top-down, you need some top-down control, but you also need sort of bottom-up self-organization if you have-

• 1:09:44Speaker 02

  That's how the brain works, right?

• 1:09:45Speaker 00

  Exactly, right?

• 1:09:45Speaker 02

  You have some, a little bit of centralization, but not all that much centralization.

• 1:09:49Speaker 00

  Exactly, right? And, and, you know, I think like absolute total libertarianism-

• 1:09:57Speaker 00

  It can work, it's just in the era of unconventional warfare, you know, complex systems also get adversarially steered, right?

• 1:10:11Speaker 00

  Whether it's free markets, they get gamed, or memetic markets, right? There's psychological operations, and yeah, and so on and so forth, dating markets.

• 1:10:23Speaker 00

  And so, I think some balance of top-down and bottom-up is the ideal. Is it more like D U S, or is it more like China, or something in between? You know, I guess we have to figure out, but that should be-

• 1:10:37Speaker 02

  That segues perfectly into my next question. So, I mean, for you, this isn't just like a philosophical thing. This is like of geopolitical, geostrategic significance, right? Like, this is important for the future of like, you know, democracy as we understand it.

• 1:10:52Speaker 00

  Yeah, I would say that our failure to view the world as a complex self-adaptive system, and still thinking in kind of the classical view of like, "Hey, I have a first-order model of what's happening, and I'm gonna do a first-order correction," right? And it's easy to convince a crowd of that, and politicians get elected on such platforms, but then they don't think of the higher-order effects of their policies, right? Instead of thinking at it from a complex systems steering standpoint. And whereas, I think our adversaries of the West understand the complex systems approach, and, you know, they tend to steer us in directions that lead to our detriment.

• 1:11:33Speaker 00

  And to me, it was like, okay, how do you fight a multi-dimensional war, you know, where there's, okay, there's like invisible sabotage of every complex system that can be steered idiotically in nefarious directions? And if it's slow enough, then it's kind of above the infrared temporal cutoff that politicians give a shit about, which is four years, right? So if you're steering the United States or the West on a 20- to 40-year timescale, you could win a very long war, and we don't even find the pattern, because we're too busy. You know, again, you know, there's a sort of timescale separation here,

• 1:12:15Speaker 00

  similar to, you know, what happens in thermodynamic computing and thermodynamic physics, but essentially, we're so preoccupied with- The dynamics that occur on on small time scales, the day to day, that we kind of ignore the long trends and those get sort of hacked, right? We slowly boil the frog. It doesn't realize it. And so

• 1:12:37Speaker 02

  In listening to you talk, it occurs to me that like, there's a beautiful coherence to your approach generally, right? Because what you're doing, both at the hardware level and at the level of your philosophical project, is essentially kind of moving us away from hard, rigid programming from the outside

• 1:12:52Speaker 00

  Yes.

• 1:12:53Speaker 02

  Towards a kind of organic, adaptive, thermodynamic-driven, kind of, learning and exploration of possibility space.

• 1:13:01Speaker 00

  Yes, yes. I mean, you know, again, it came from my own journey, trying to understand physics and kind of discovering differentiable programming and seeing that as the way forward for for everything, right? And and really, you know, some of the prescriptions of Yak are are to maintain variance and constantly explore across any parameter space, whether it's culture, aesthetics, policy, technology, etc. And one of the reasons is that, you know, according to Fisher's theories on evolution, you know, the speed at which you can traverse a landscape, of course, it depends on the gradient, but also depends on on on the variance,

• 1:13:43Speaker 00

  right? Evolutionary search depends on variance, because the more you fuck around, the more you find out, right? And so your rate of learning is faster and your rate of adaptation is faster. And to me, that that seemed like the main advantage of the United States is that the United States is very high variance, right? They have high variance individuals and they have high variance outcomes. And so I view the United States as a sort of high-temperature search algorithm. And they're always first to, and I'm not American yet officially, but you know, someday I will, that's my goal. But, you know, they're first to figure things out because they're always searching in very high variance. I view it as a sort of, I view sort of innovation as a diffusion process in some landscape and

• 1:14:25Speaker 00

  they're in very high noise regime. So they don't get stuck in local optima, right? Whereas, you know, I would say China is more kind of like the the low-temperature sampler. They're the kind of The optimizer at the end, they're doing the gradient descent, right? Once there's, once there's consensus, right? Once there's a clear gradient of direction of improvement, right? It's easy to convince a committee at that point. And, and then you can just execute top down control with a lot of conviction. And so they, they beat us in, in the final stretch. When sort of, you know, you have, it's, it's kind of like Bayesian inference. You go from a unsharp prior to a sharper posterior and what's the optimal thing, right? It's, it's like annealing, right?

• 1:15:06Speaker 00

  And, and so essentially China beats us at, at squeezing the end again, because they, they're, they're fully kind of, they have a lot more top down control power and a lot more coherence there. So, so I see this sort of, we're, we're kind of like a parallel tempering algorithm between the United States, the high temperature search and discovering things first. And then you don't, you know, the, the optimizing and, and improving the technologies and scaling the manufacturing, they're not necessarily the best. And then China squeezes the last bit.

• 1:15:38Speaker 02

  How do we avoid catastrophic outcomes and that, that, because you've been, you've been pretty dismissive about like, you know, P doom and doomerism generally. So how do we, how do we, you know, allow the kind of, you know, meta search to happen? But in a way that doesn't lead to, you know, catastrophic, like technological outcomes or, you know? Because I, I, I can also see some kind of tension between, on the one hand, this kind of open project. And on the other hand, like, you know, the, the, the, the participation of like proprietary closed corporate groups.

• 1:16:18Speaker 00

  Yeah, I, I, I guess, you know, for me, P 1984 was higher than P doom from AI, right? And so I think, you know, to, to me, I think China is kind of. China is like the ultimate monopolistic company, right? There's basically one set of executives for all the companies. They're acting like a conglomerate and they throw their weight around to crush American smaller companies. And so, you know, I would say that one thing about EAC is that we're kind of anti-monopolistic because again monopolies tend to be suboptimal. Again, if you have choices of hyperparameters that are over-concentrated, you're not exploring anymore.

• 1:17:01Speaker 02

  We see that that sort of motivates my question and my worry is that, you know, like very authoritarian forms of government where we impose kind of like, you know, ethnic or cultural homogeneity, like those are very good at minimizing free energy, right? If you and I are exactly alike because there's like a top-down imposition of sameness, right? Then, you know, that that minimizes free energy super well.

• 1:17:27Speaker 00

  Locally, locally, right? It's like, well, you know, if your energy term has like, you know, if you want to agree with your neighbor and so on, have no frustration, then of course, like that's a local optimum. But as we know, like what we're arguing for is embracing embracing variance, right? Embracing different culture, having different bets, right? Like even even, you know, it could be, you know, culture, genetics, it could be ways to even train your ML models, right? I would say that actually we've had-

• 1:17:55Speaker 02

  Same response to the cancer thing, right? Like you're not worried about the the push towards, you know, boundless growth, leading to cancer type formations because it's not it's not a global optimum. It's a local optimum.

• 1:18:09Speaker 00

  Yeah, yeah, that's right. I want to go on a slight tangent here that, you know, I think like, for example, there's been there's been some centralization in AI research labs, you know, even though there's like four or five players that matter, they kind of churn. I mean, it's kind of been a meme recently that researchers, they kind of all slosh around and they all churn between each other. So they're all kind of equilibrating in terms of beliefs, right? Because you have exchange of particles. Of researchers between the labs and you could see that like the American research labs all converge to to similar amounts of performance at similar compute. And then, you know, China with DeepSeek was exploring a whole different region of hyperparameter space due to its export control constraints.

• 1:18:54Speaker 00

  And then showed us that over concentrating our bets in hyperparameter space has risks. Because we're stuck in a local optimum and there might be a better non-local global optimum, right? And that we're missing. And so, you know, again, it's just been our push for variance there. In order to to to always be exploring. I think it's I think it's really important. I think I think it's really. You know, I think I think basically there's also been a reaction to sort of.

• 1:19:26Speaker 00

  Yeah, it's hard. It's hard to explain. It's kind of like the same pattern that happens. You know, in in in Western governments and late-stage corporations. And in in large bureaucracies, there's a sort of

• 1:19:40Speaker 00

  cover your cover your ass sort of mentality and culture and decision. Decision by committee. And so, you know, it tends to be variance reducing, right? Or variance killing and taking sort of.

• 1:19:56Speaker 02

  That's sort of my worry. And in addition, you know, I guess we're sort of in the same kind of situation as like a gradient descent learner. We're like, how are we supposed to know that we're not stuck in a local optimum? Right? Like to us, like we we might think, hey, like this is actually, you know, this looks pretty optimal. But there's no kind of God's eye view that could tell us that actually we're just stuck in a local.

• 1:20:21Speaker 00

  But that but that's why, you know, as long as you don't kill variance completely, you keep some explorer, you're not just purely exploiting. And you keep open mindedness, then you're you're always kind of, you know, spending some resources still exploring and and and potentially finding a new better way to do things. Just like, you know, I think, you know, some people. You know, at first thought, our bet was ludicrous and, and you know, we shouldn't have raised VC funding. It's like, really? You know, there's trillions of dollars of capital riding on the current paradigm. And you don't want a startup to raise a few tens of millions to take a bet.

• 1:21:02Speaker 00

  That would be completely disruptive to every, to everything, to this whole multi-trillion dollar bet. Right? And, and now the, the, the tune is changing a lot because people that are planning these half a trillion dollar build outs. Want to know if a technology that's a thousand X better is around the corner, you know, and could render some of their build outs maybe. Miscalibrated, right? In terms of the ratio of energy to compute and, and density, right? Again, I, I don't, I don't think, um, you know, I, I think thermodynamic computing is gonna work in, in tandem with, with GPUs or TPUs or whatever.

• 1:21:41Speaker 00

  Neural processing units. Um, I think, um, you know, it's going to take quite a, quite a while for models to be fully ported to, to thermodynamic computers. So for the foreseeable future, I think these build outs are, are safe and you could just have thermodynamic computing as an add-on and.

• 1:21:56Speaker 02

  Gil, it's been a real pleasure to discuss these issues with you.

• 1:22:00Speaker 00

  Pleasure as always.

• 1:22:01Speaker 02

  So how do we, how do we, uh, you know, stay abreast of what is going on at X-Tropic?

• 1:22:06Speaker 00

  Yeah.

• 1:22:06Speaker 02

  Do you have any kind of, kind of closing, closing message for us?

• 1:22:10Speaker 00

  Yeah, well, so, you know, stay tuned in the coming weeks and months. For those that haven't, that are interested in trying out thermodynamic computing, uh, you know, in the coming weeks, uh, we're going to give access to the, the first users. To our, our systems. Um, so you could test it yourself. It's mostly going to be private alpha in, in the early days. Uh, again, we, we can't, uh, handle that many users. We don't have that many chips. Um, but you know, the technology is here, it's very real and you can kick the tires. And, uh, you know, stay tuned for some, some scientific papers, some open source, uh, uh, software that we're going to put out. And I would say, you know, start reading more probabilistic machine learning papers.

• 1:22:53Speaker 00

  If you're a grad student interested in the area, start reading about, yeah, EBM's and various textbooks. And, you know, stay tuned for more announcements from Xtropic towards the end of the summer and this fall.

• 1:23:08Speaker 02

  Well, for Machine Learning Street Talk, I'm Maxwell Ramstead. We're signing off, and thanks again, Gil. This was awesome.

• 1:23:17Speaker 00

  Thanks, Max.

• 1:23:18Speaker 02

  Yeah, and fascinating, like, really, really cool stuff.

• 1:23:22Speaker 00

  Awesome.