#326 Zuzanna Stamirowska: Inside Pathway's Post-Transformer Architecture Designed for Memory and On-the-Fly Learning

This episode is sponsored by tastytrade. Trade stocks, options, futures, and crypto in one platform with low commissions and zero commission on stocks and crypto. Built for traders who think in probabilities, tastytrade offers advanced analytics, risk tools, and an AI-powered Search feature. Learn more at https://tastytrade.com/ https://tastytrade.com/ This episode dives into why Pathway's Baby Dragon Hatchling (BDH) might mark the beginning of the post-transformer era in AI. Zuzanna Stamirowska, Pathway's CEO and co‑author of BDH, explains why today's transformer-based LLMs hit a wall on long-horizon reasoning, how memory and synaptic plasticity are built directly into BDH's architecture, and what that means for continual learning, hallucinations, and "generalization over time." The conversation ranges from complexity science and brain-inspired computation to practical implications for real-world, small-data, and safety‑critical applications. Stay Updated: Craig Smith on X: https://x.com/craigss Eye on A. I. on X: https://x.com/EyeOn_AI (00:00) The Core Problem: Why Today's AI Lacks Memory (03:16) Pathway's Mission to Bring Memory Into AI (04:53) Zuzanna's Background in Complexity Science (10:30) Why Transformers Reset Like "Groundhog Day" (14:34) The Brain-Inspired Dragon Hatchling Architecture (23:59) How the Network Learns and Builds Connections (37:38) Performance vs Transformers on Language Tasks (49:37) Productizing the Technology With NVIDIA and AWS (54:23) Can Memory Solve AI Hallucinations?

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  So it's a bit like having an intern that you hire on the first day of her job. And she may be brilliant, but she stays an intern forever on the first day of her job. Doesn't get more context, doesn't get better with experience, doesn't get better over time.

• 0:13Craig S. Smith?

  Do you think there will be creativity in this kind of a network? Will be able to think beyond the fixed parameters of an LLM and come up with new ideas? This episode is brought to you by Tasty Trade. On Eye on AI, we talk a lot about how artificial intelligence is changing how people analyze information, spot patterns, and make more informed decisions. Markets are no different. The edge increasingly comes from having the right tools, the right data, and the ability to understand risk clearly. That's one of the reasons I like what Tasty Trade is building.

• 0:55Craig S. Smith?

  With Tasty Trade, you can trade stocks, options, futures, and crypto, all in one platform with low commissions, including zero commissions on stocks and crypto, so you keep more of what you earn. The platform is packed with advanced charting tools, backtesting, strategy selection, and risk analysis tools that help you think in probabilities rather than guesses. They've also introduced an AI-powered search feature that can help you discover symbols aligned with your interests, which is a smart way to explore markets more intentionally.

• 1:37Craig S. Smith?

  For active traders, there are tools like Active Trader Mode, One-Click Trading, and Smart Order Tracking. And if you're still learning, Tasty Trade offers dozens of free educational courses plus live support from their trade desk reps during trading hours. If you're serious about trading in a world increasingly shaped by technology, check out Tasty Trade. Visit tastytrade.com to start your trading journey today. I'm going to myself. Tasty Trade Inc. is a registered broker dealer and member of FINRA, NFA, and SIPC.

• 2:23Craig S. Smith?

  I mean, I've done some reading.

• 2:27Craig S. Smith?

  I'm not sure I entirely understand everything yet, but I've been interested in self-improving AI, along with everybody else, and continual learning and those things. And post-Transformer architectures, I've spoken to a few people. Carl Friston, I don't know if you followed his work, but he's working with a startup called Versus. And then there's a company called Manifest AI in New York City. I don't know if they're at all related to what you're doing.

• 3:04Craig S. Smith?

  But in any case, I'm interested. And you guys are looking at this specifically for robotics applications or no?

• 3:16Speaker 00

  Not necessarily. So first of all, Craig, thank you so much for having me. It's a pleasure to be here. And I mean, you also had a chance to talk to wonderful guests. So we actually do follow the work of the guys at Manifest AI.

• 3:33Speaker 00

  So it was actually great to hear them on your podcast as well. We don't necessarily focus on robotics.

• 3:41Speaker 00

  In fact, what we focus on and the part of

• 3:45Speaker 00

  maybe some sort of obstacles that we see for AI that are pretty big that we are lifting and hoping to lift are linked to memory.

• 3:57Speaker 00

  So not just continual learning, not just self-adapted systems, but memory. And then memory leading to enhanced reasoning.

• 4:08Speaker 00

  So it's slightly more than just, you know, allowing for a larger state or the capacity of learning over time. It's part of it. But memory can also power reasoning and having, let's say, memory organized in a different way. We can get to representation. Let's say something that folks call that internal representation that may potentially be more interesting and easier.

• 4:37Craig S. Smith?

  Okay. Can you give us a little your background before we start talking about Baby Dragon Hatchling? Pretty creating name.

• 4:51Speaker 00

  Of course. So I would love to, actually, I would love to also maybe set the stage a little bit.

• 4:58Craig S. Smith?

  Yeah.

• 4:58Speaker 00

  Why we're looking at those problems before jumping into what we've done? Because I think the most, maybe the most interesting thing would be to establish some kind of common ground for what we see is happening. And I think many people across a number of labs by now also are seeing and some of your guests as well. So as for my background, I'm actually a complexity scientist. So I worked at the Institute of Complex Systems of Paris. I am specialized in studying emergent phenomena.

• 5:38Speaker 00

  So seeing how some sort of global order emerges from small local interactions and how locality matters in appearance of phenomena. So, and this may happen on different sorts of topologies. I mean, in complexity science, we usually look at graphs. So graphs is just, a graph is a network, right? You have like dots which represent the nodes and then you have edges that link to nodes. This can be your own network in a city, right? This can be your internet. This can be your social network. This can be your network connecting neurons. And something happens on those links, right? There are some sort of, I don't know, cars travel, mail is being sent, right?

• 6:22Speaker 00

  Water travels through pipes, whatever. But something, something happens. I mean, sometimes there is some operation being done on the link. Um. There's some sort of function that then also drives the evolution of this network. So it's a system that may be, you know, evolving, following the local interactions that happen on the network. So I know for roads, we would look at, okay, somebody built a road in a, in one place, but maybe they built a road between two places that uh where, where two types of objects were produced to exchange, right? Uh, but then there was a settlement somewhere. So, okay, then we built a road again. And since there was a road, people started producing things because it was easier to trade with that location.

• 7:08Speaker 00

  So you sort of start to have this loops, you know, uh, in a way that's well, that's, that's the network structure uh was happening, or is it uh, is it the other way around? Chicken and the problem thing is, it's an evolving system uh, where things happen locally, not, are not necessarily usually always planned by, you know, a super uh, super designer at the global level that says what should be happening, which function should be done where. Um, you rather have this organic emergence. Um, and this is, this is what I actually focused on uh, early on in my career. Before that I worked, and actually what led me to it was game theory and game theory played on graphs.

• 7:53Speaker 00

  So you're gonna have players, but players play on, not like one to one or many to many, but they play on on potentially complex topology and not, not against everybody at, at every step. Um, but I also was trained at the French school, uh, the School for French Politicians. Um, I, in my past, I also got a chance to teach macroeconomics, for example, in the same place.

• 8:19Speaker 00

  Yes, Young Spoh. Um, so yeah, this is the the very, I think, I think a very, very useful type of uh, education as I kind of see it right now being a CEO of a startup. Um, and then, you know, when you, when you look at AI and many implications uh, that it, it, it has and, and probably trying to project where we can get uh, with AI as a society, then also as uh, as, as countries and how this may impact maybe even the, the, the concepts of how states collaborate, et cetera. I mean.

• 8:57Speaker 00

  Perhaps I at least I have some anchoring to start to think about this. So this is this is my background and I have a chance right now at Pathway. I mean, I'm a co-founder and also like a co-author of the BDH Architecture. I have a privilege of leading a team of fantastic researchers in AI with sometimes various backgrounds. So our CTO Jan Horovski was the first person to apply attention to speech recognition. So that was in the pre-transformer era and he was at Google Brain.

• 9:38Speaker 00

  Our our CSO Adrian Kosovsky is a quantum physicist and a theoretical computer scientist at the same time. And he has his PhD at 20 and you know, there's there's an entire team built of algorithmic missions. People with background from physics as well. And I'm I'm just, I'm just, you know, extremely actually proud to be to be able to to meet them.

• 10:05Craig S. Smith?

  And and what were you doing? I mean, you know, you wrote this paper on dragon hatchlings. I'm actually looking, I don't remember when that was. But and and it was that was in September. Was that before or after you founded Pathway?

• 10:30Speaker 00

  No, we found this pathway way before. So we've been working on this on the problem of memory and intelligence for for a long time. And this is just, let's say, the first glimpse into into what we're doing internally. So if you look at LLMs, right? And this is this is something that I think is becoming more and more public, I'd say, or maybe more people are becoming aware of this and and in general public. Like all the LLMs are pretty much based on the transformer architecture.

• 11:09Speaker 00

  That was a huge unlock for AI when that enabled like all of us pretty much to to to have the imagination of what could be possible with AI. Um. The transformers are probably not, and I mean, as Pavlov would argue, are not the last architecture in this entire AI uh market shift. And specifically uh from from what you're seeing, um and this is something that we kind of, you know, so early on, they're missing memory. So they work, they work a bit like a grand

• 11:49Speaker 00

  rock day. I mean, they wake up every day with their their memory of their interactions of their of of of whatever problems they're resolving being completely wiped out. So it's a bit like having having, you know, an intern that you hire uh on the first day of of her job, and she may be brilliant, uh but she stays an intern forever on the first day of her job. So doesn't doesn't doesn't get more context, um doesn't get better with experience, doesn't get better over time. And then also isn't capable of staying focused on a task for

• 12:28Speaker 00

  a very long time without without falling into hallucinations. Um. And then, of course, I mean, with capacity of staying, let's say, focused and coherent on solving a task, uh the complexity and the value of tasks that AI can accomplish uh will be will be growing. And there is a lab that is called Meter that is measuring uh like the length of human tasks that AI can do right now successfully. And we are at a couple of hours uh right now. And of course, there are so many tasks that would in reasoning, for example, would take it way long, will take us way longer, potentially also with changing inputs, data coming from

• 13:13Speaker 00

  the environment, right? Not all the problems are closed form, kind of math math problems. Some are open, highly dependent on context, some may depend also on small data.

• 13:24Speaker 00

  Um, so yeah, so this memory bit, I mean, we see it also from the market perspective as, let's say, one of the vectors of the, of the, a market shift. And there are people who, of course, work on the topic of memory from the hardware side. Uh, and we are working on this from the algorithmic side.

• 13:43Craig S. Smith?

  Just to back up a little bit and, and pull out, um, uh, to a wider view. So you're, you're working, I mean, these are, your language models are, uh, neural networks. Uh, but the, the difference is instead of the, uh, information being coded, and I'm guessing now, or from my understanding, being coded in the weights of the nodes. Uh, the, it's the edges, the, the connections between nodes that, that strengthen over time and, and in effect, uh, create, uh,

• 14:27Craig S. Smith?

  memory. Is, is that, am I getting that at all right?

• 14:33Speaker 00

  It's a bit more, uh, in a sense, it's a bit more, uh, it's a bit more because of the learning mechanism that we have. And as we started this discussion with talking about locality, um, the concept of locality and how the system works is very key. So we have a graph structure, a graph kind of structure of connections between neurons. This is true. So what we, that, so maybe what we're doing right now, we're actually discussing the architecture, the post transformer architecture that we, uh, that we published in the paper, uh, in September, uh, which works a bit like the brain on, uh, on GPU.

• 15:20Speaker 00

  So it follows the rule of of Hebbian learning. This is a simple, brain-like model. And we managed to get it to GPU and it actually trains even better than than Transformer. When we compare it to the GPT two. So this is this is the Transformer architecture. We compare architectures, architectures, not yet models to models, right?

• 15:43Speaker 00

  At at one billion scale and has a number of great properties. We designed this, in fact, to have memory and then to align it with the way we see reasoning. Let's say unfolding. So we can, we can discuss it a bit, a bit later on. So to give you maybe an image of how, how in this system learning works.

• 16:06Speaker 00

  We have neurons, a bit like in the brain. Yep. Right? And then the neurons are linked with synapses. I mean, with, you can, you can imagine wires or you can imagine roads, right?

• 16:20Speaker 00

  In a traditional transformer, everybody is connected to everybody. So imagine a conference call where everybody is connected to everybody. And you kind of, if you, if you want to, you know, double the number of participants, you have to quadruple the complexity of everything that's happening.

• 16:42Speaker 00

  Well, here, not everybody is connected to everybody. We have, we have a graph structure of relevant connections that are trained actually with signal that comes into the system. So whenever a signal comes to the system,

• 16:58Speaker 00

  the neuron which was, let's say, triggered, sends over a message to its neighbors over those wires. And they're not connected to everybody. They're connected, let's say, just, just to their friends, like you and PayPal, for example. Right? They could, they, they send over their message, the message just to their friends.

• 17:18Speaker 00

  According to some rule of a threshold, the friend is interested or not. So fires up, potentially, receiving, receiving the message. And then in the following step, this, this connection between them becomes stronger because it's been proven to be relevant. And it helps to create certain shortcuts or kind of wider, wider roads. Yeah. Like fast, high-speed roads between certain, certain places in the network. Yeah. And in that way, during learning, you get optimization of this space.

• 17:59Speaker 00

  I'm not saying this very formally right now, so please don't hold me to it. But it's somehow more optimized kind of space of connecting, connecting things. And we, we find that this structure optimizes itself naturally during training for communicability. And this is in fact very often a feature of, of complex systems that are, that are grown organically. You need to, like, the system, the systems naturally like to balance communicability and, and let's say, the cost of maintaining the links.

• 18:38Speaker 00

  But it should be usually, you know, easy for you and quick to, to, to, to, to reach everybody who's important on a network quickly. And then, then, you know, the, in this structure, like, the links can, can appear according to, we don't control the rules according to which they appear. So it may be the information, you know, you were at the cafe with a friend and that friend told you something very interesting about neuroscience. That was my case, kind of, actually, when I started this research. And, and you will have a connection between the taste of that coffee and, you know, something that happened in the brain of a mice, of a mouse. And you may have this connection. It's not the very formal one. It's the one that you kind of have.

• 19:21Speaker 00

  And then it may help you to do some sort of informal thinking and just kind of exploring this space of, of, of. I don't know, related concepts according to whatever types of links that you could have could have found while learning.

• 19:38Craig S. Smith?

  Okay, let me just ask some dumb questions to try and triangulate where this sits in the graph of my knowledge. The, because I've, I've been, I had a conversation recently with David Ha at Sakana AI. And they're talking about, you know, evolutionary systems that, that can grow new neurons in effect

• 20:16Craig S. Smith?

  to add knowledge and, and, you know, avoid the catastrophic forgetting of fixed LLMs. In your baby dragon hatchling system, two questions, is that what happens as, as you learn new things? Do you, does the system add new nodes? Or is it all in, in the strength of the connections in the, along the edges? And, and how does, how does that strengthening or weakening?

• 20:56Craig S. Smith?

  And mathematically, is there, because in a standard LLM, there's a function in the node that, that encodes knowledge

• 21:12Craig S. Smith?

  and, and could be wiped out with, with, with retraining. But is there a function in the edge then is, you know, rather than just?

• 21:26Speaker 00

  This is, this is a very good question. This is, this is one that our, our CSO likes very much and I'd say it's mostly coming from him. So we have something and, and I mean, this will be, this will be perhaps like pretty technical, but an important distinction for, for intuitions when, when one is thinking about such systems. And I think it's not, not very often the case when we, when we talk about L L M's. It's a duality between operators and the state. The intuitions, the way it was actually apparently, I mean, I'm just repeating after our CSO, because this is, this is his part. This in fact comes from quantum physics.

• 22:09Speaker 00

  Just, I mean, historically in science, how, how it's been developed. And in this case, the state sits on the edge. And on, on, on the fast weight, as we call it. And then we have the equivalent of it, which is a slow weight, which, which is a parameter. And this is, this is kind of key. And what would happen is that neurons are computations, but really the, the state sits on the edges, on the, on, on the fast edges mostly. So this is, this is how we look at this.

• 22:48Speaker 00

  And what happens is that while with, you know, every step you actually also update, you use, you use the network topology and weights for message passing. But then you also, after you've done it, you update your, your fast weights and you update the, the, the slow weights, let's say at the slower frequency. And this is, yeah, this, this is exactly.

• 23:18Craig S. Smith?

  And if you're, if there is no connection between two nodes, but as you said, you know, maybe there's a node for a coffee shop, maybe there's a node for. For, I mean, this is obviously a not realistic analogy, but maybe there's a node for you and for the person that you're talking to. But the the person you're talking to is connected to the coffee shop and you're connected to that person. How does the system know to build a connection, an edge or whatever?

• 23:59Speaker 00

  Yeah, this is this is actually a very, this is a very good question. So I mean, we've been looking at it from from both both perspectives. Important to know that concepts, let's say, if you think about concepts, concepts sit more on the edges than on the nodes. So so so we're looking more at the synapses where where the concepts are are are sitting. And then we would say that, OK, actually, if a concept is important, they will have one node responsible for that concept, one node, one one synapse actually responsible for that concept. And this is what we find, what we demonstrate with one, like with an example in the paper. Now, we have, we have like the concept of currency and we find this this synapse firing firing up.

• 24:41Speaker 00

  And so, yeah, we, sorry, can you repeat the question?

• 24:46Craig S. Smith?

  Yeah, I mean, how do you build a connection between two nodes if there's no connection?

• 24:52Speaker 00

  Yeah, exactly. So the nodes, we, in practice, we we usually keep the N as fixed. We we keep the N as fixed and there is an initialization rule at the beginning. So this is usually when you start growing network, you need to kind of start with a have an initialization rule for the very kind of first basic structure to appear that then starts to iteratively kind of grow in and hands. So there is a, I I mean, by now, I think we know, we tested a great number of them. But there's, there's like a simple initialization rule, which is linked to, OK, how how many attempts you have of creating your your first links at the at the very beginning.

• 25:36Craig S. Smith?

  And then as as as the network, how does the network grow? I mean, this is what I was talking to David Ha about, how do you, or or it don't, doesn't grow? And the network.

• 25:50Speaker 00

  So our network, you may imagine it doesn't grow so much. I mean, the N, the, the, the number of, the size of network defined as a number of neurons in the brain. Uh, and in practice for us stays fixed. But since it's a graph, right, you can pack a lot, a lot of synapses.

• 26:11Speaker 00

  Between them and then you can also, and so in the brain, we're looking at hundreds of trillions of synapses. For billions of, of, uh, of neurons. So this gives us like a very, very, very large state. That we can use it, access and use efficiently at any time. Because we don't always need to use the entire state. We actually use it very locally, thanks to the local dynamics. We may have huge state and use very little of it at every step because of the local interactions, like the local dynamics. This is key, this is very different to everything else that happens, uh, in AI as far as I know.

• 26:52Speaker 00

  Um, because we are really focusing on the local dynamics and we found a way to kind of. Maquillage this, I mean, apply some makeup and make it work. Uh, this kind of sparse things to make them work on GPU and make them seem like dense matrices. But actually being, uh, kind of preserving mathematically the sparsity of, of, of, of interactions that we have. And I'm happy, happy to jump into this. Um, so yeah, so, so, so point is local, local interactions. And if you do it, you effectively have the context size, which is the size of your model. But you don't use always all of that. You use just the, the, the tiny bits that are, that are necessary for you.

• 27:34Speaker 00

  And, um, then in terms of catastrophic forgetting, well, one of the arguments is that.

• 27:41Speaker 00

  Once you learn new skills, you can have so much space to pack things that you don't impact, you don't necessarily impact the ones that were linked to to your previous task. Uh, I mean, it's too early for me, you know, to, to, to, to, to talk more about, uh, like the solving the curse of catastrophic forgetting and continue learning.

• 28:03Speaker 00

  But, but let's say that, yeah, we, we see these dimensions being very, very helpful.

• 28:08Craig S. Smith?

  Yeah, and, and the art, the structure of the, of the, you know, a traditional neural network is, is in a very, uh, it's very structured, there are layers and there's the width of each layer. This is, uh, much more organic, is that right? It's, it's not in layers, it's, it's in a graph, so there's, this, this, yeah.

• 28:37Speaker 00

  It's, exactly, so it's in the, it's the, in the ideal scenario, right? And that, and the, and the pure thing to really, to build your intuitions and to really explain how and why it works. It's exactly a graph with local dynamics that work like a rumor spreading on networks. Uh, for technical listeners, I mean, I would invite you to maybe have a look at John Kleinberg's and Avatar Dush's papers uh, about rumor spreading. Um, so, but the, so, so for example, like why, why certain neurons fire up? I mean, the rule is linked to thresholding. Um, so you need to reach a certain threshold, in fact, of, of, of relevance and information to, to, to activate the neighbor,

• 29:20Speaker 00

  um, et cetera, but it's really like spreading a rumor on, uh,

• 29:24Craig S. Smith?

  Right.

• 29:25Speaker 00

  on the internet, right? You cared enough to pass it over to your friends and, and, and, and, et cetera. Some people didn't care, or like a bit like epidemics. Um, so this is, this, this is the logic. Then when you bring it to GPU, and I get, get questions about this, because of course, you know, was being seen, uh, like with the paper and online on GitHub is just a small portion of, uh, of our work. Um, what you, what you see, uh, what do you see in the code and in, uh, in the, like, very GPU-friendly implementation that we have there?

• 29:59Speaker 00

  Are ultimately, you know, we have to talk about matrices and and and it looks it looks very much like a transformer, but there are there are like important differences that happen. Attention is linear. So when you look at sizes of, let's say, our matrices, the traditional transformer has this large square, right? And there will be squares and they're dense. Whereas for us, it looks the only dimension that we care about is the N.

• 30:32Speaker 00

  That's the number of neurons effectively. And the objects we work with look like very kind of narrow but very long snakes. Okay, we have those huge squares in transformers that grow quadratically. And we have this very long snail, which which effectively we only care about the length, which is N. And this is the number of our neurons. And then to make it sparse, because of course, we had to, as I said, we had to apply some makeup to make it work on GPU that really likes dense matrices. We actually multiply

• 31:09Speaker 00

  it by a positive sparse vector of activations. And this, once you actually multiply one for the other, you get back your graph.

• 31:25Craig S. Smith?

  Yes.

• 31:26Speaker 00

  Just so this is this is for the for the listeners who might be asking, okay, where is the sparsity hidden in the in the implementation that is GPU friendly? Well, actually, mathematically, it is there. So you get you get it back. And hence, you can you can actually do a lot of things like spot the spot the synapses, you know, responsible for for a specific concept, etc. I mean, sparsity is very much there, even though if you just look at the code like this, you might not spot it because it's hidden in one specific multiplication. And this, the fact that we have positive sparse positive vectors of activations, this is a very big, very big difference that I mean, our CSO

• 32:08Speaker 00

  loves to loves to explain as well. I mean, I think, once he used Tower of Macarons to explain the kind of the space of those vectors.

• 32:17Craig S. Smith?

  And uh the uh you talked about activi- uh you know, you need a certain amount of signal to to activate a neuron. That sounds like spiking neurons is is.

• 32:31Speaker 00

  It's an edit. This is not, uh, I mean, it's in intuitively, I mean, not not in like all the implementation details, but it's intuitively, it's not too far. We're getting to, so what actually um response to our paper uh was especially good from the neuroscience community. Because indeed, we we we kind of started to show the way of bridging a very non-organic AI with uh with with the models and the and the thinking of of of how neuroscientists see uh brain activity. Of course, we're not getting to the levels of of chemical interactions, right?

• 33:13Speaker 00

  Like reactions that happen in the brain. Um but we we are offering like a plausible brain-like explanation for for for for how reasoning may appear.

• 33:23Craig S. Smith?

  And then the the sparse um uh the the activation of wh- when the network is working, I yeah, I understand that it's it's it's it's it's very localized, you're not using the entire network, it's whatever uh the edges connect. Uh that sounds a little bit like mixture of experts. Am I wrong there?

• 33:53Speaker 00

  This isn't just at a very at uh infinitely small, I mean, not infinitely, but really tiny granular scale. So if you were, because mixture of experts, uh you know, is is more kind of defined from top down. Uh here, imagine that your expert can be even just, you know, one potentially could be just one edge, right? And you don't decide expert groups, they're being decided on their own. Kind of organically while training, we don't, this is this is important, this is also maybe coming really deeply from the complex systems intuitions. Uh we really don't impose any structure.

• 34:37Speaker 00

  On the system. Uh, the system does what the system wants to do. And that's the point, that's magic of it. This is also why um we believe this kind of ultimately the space that's created can be very interesting for reasoning.

• 34:54Speaker 00

  Because at the same, at the same time, you know, the concepts appear, the more important ones are strengthened. And it gives you a topology, potentially with shortcuts uh to explore. And if you believe that reasoning is a way, some sort of search in the space,

• 35:09Speaker 00

  then then this gives you an interesting topology to explore. And it's aligned with the method of learning itself.

• 35:18Craig S. Smith?

  And then as as the system learns, uh uh edges uh become stronger or connections become stronger or new connections are formed. Uh, so so the the the graph becomes denser, not in the number of nodes, but in the number of connections. And and

• 35:47Speaker 00

  Yes, plus

• 35:50Speaker 00

  plus some may fade over time.

• 35:52Craig S. Smith?

  Yeah.

• 35:52Speaker 00

  So, yeah, some some could be, if you don't use them, I mean, you have only positive activations, but but but you you may you may have fading as well. So in a way empirically, what we find is that we get um like almost a scale free type of um type of distribution of of degrees of of of nodes. Which gives, this is this, this is a type of structure that we've find in in in the kind of real world organic that works quite a lot. Because this is one that's kind of known to be like pretty, pretty resilient. Um resilient and with good communicability and also behaves similarly wherever you zoom in in the network.

• 36:34Speaker 00

  So it kind of, it has like similar properties irrespective of its size.

• 36:40Craig S. Smith?

  And uh the the how uh how uh you said you've you've built uh like a a billion parameter model uh using this architecture, is that right? How how is it's?

• 36:56Craig S. Smith?

  Not only how is its performance, but I understand memory and in transformers, the attention mechanism, but how the memory here is the strength of the connections. Right? So, so, uh, yeah, so, uh, but how, how, how does it perform?

• 37:21Craig S. Smith?

  For example, in, uh, is it's not autoregressive, right? Or is it? Uh, how, how does it perform on language? I mean, you, you built a language model with this, right?

• 37:38Speaker 00

  Um, yes, so this is, um, we train it on predominantly on language. So I mean, we're not talking about division models, for example, right? Like there is, there is an entire line of, of, of research and, and right now attracting a lot of the attention, um, which is actually linked to, uh, vision and, and word models per se. We actually believe that, um, we believe that there is maybe, maybe a way to bridge the two. Um, but yes, we're looking at the, uh, we're looking at the language model. And, uh, we compare it, uh, on traditional language tasks. So this is, these are the benchmarks that we, or the tasks that we're showing in the paper.

• 38:23Speaker 00

  Uh, we're looking at translation, we're looking at how, how it behaves at, at, at traditional language tasks, uh, a bit the way in which transformers were developed. So the question of scaling it further, in fact, and, and the features that we see, uh, more likely coming out of it, uh, out of this architecture scaled, uh, well scaled or actually developed into a model into a product is, um, let's say, the power of your output.

• 38:54Speaker 00

  With way less data. So with a way smaller model and a way less data, we should be able to get to the two same results. Uh, I mean, compared to G P T two. So I mean, comparing apples to apples, architectures or architectures on the same data sets, exactly same data. Uh, I mean, we're actually learning sometimes even like faster than transformers and scaling laws are preserved. Knowing however, that we don't need to grow this uh size of neurons. Uh, so much and we can be adding, uh. I mean, we have state, right?

• 39:33Speaker 00

  Which is huge and um and which is huge. Well, we can actually use it efficiently.

• 39:43Speaker 00

  So it's both packed efficiently because it's packed in the graph between the connections between finally not so many neurons. Um, and then because of the local dynamics, we only access a bit of it. Uh, at, at, at every, let's say, step or.

• 39:57Craig S. Smith?

  Yeah, uh, and again, some, some more dumb questions, so I can. Understand this, I mean, in, in the brain. Uh. These local. Uh, networks that are activated represent.

• 40:17Craig S. Smith?

  Concepts. Perceptions. All of that. Memories. Uh, and. In, in your system. But, but, but it's not, um.

• 40:35Craig S. Smith?

  Let me think, how am I going to say this? It's it, uh, the, the knowledge that's stored there. Is, uh, depends on, on the. Connections between these neurons. Uh. In the knowledge.

• 40:57Craig S. Smith?

  Kind of emerges from those connections, right? It's not, obviously not stored explicitly. And does, does that, is there some plasticity in that? Like if you cut a bunch of connections, you still have the knowledge. Maybe it's got to grow stronger connections again. But yeah, what can you talk about?

• 41:23Speaker 00

  So this is, this is, this is, this is a beautiful question. I would like to, your, your questions are actually very deep.

• 41:31Speaker 00

  So definitely then that would qualify, you know, as, as the dumb question. This is, this is, this is dramatically important.

• 41:40Speaker 00

  I'll take it in, in two ways. So the very, the very concept that we have here is synaptic plasticity. So this means that not only a message is passed through the network or through a connection, but then this connection becomes stronger because it was triggered. And this goes on perpetually somehow, right? As, as the system lives, I mean, we, we have, we have this effect. And then there's this kind of update to the, to the slower weights that, that just, I mean, change at slower frequency. But then, then for the plasticity of the network, I mean, this is, this is a brilliant topic, actually for complexity science and network resilience in general. And when came intuition that's, actually this one came from me.

• 42:25Speaker 00

  I mean, most of this research actually, you know, came from, from, from my colleagues and, and our team, but this one came from me is that function shapes the network. And you actually may have, and this is something that neuroscientists at that coffee shop told me. You may, you may sometimes swap the nerves. And the function of, of them though, like, let's say, the example he was giving me was the auditory with the visual nerve. But then the mouse will turn out just great.

• 43:03Speaker 00

  Because the stimulus that was given shaped the network, because the network's job is to transport it in a way.

• 43:14Speaker 00

  So it's getting these local functions of what the network, what the edges, what the nodes are doing, that then help you to potentially even compensate. So I don't think we've run deep enough studies of what's happening even in our model right now in terms of its resilience to deletion of nodes. But I've done a number of studies of how such systems work and how scale-free networks behave when you delete. So two intuitions is that the function shapes the network. There's a good chance that some functions will be overtaken

• 43:56Speaker 00

  by the locality, and this will also happen according to the rules of local connections, like let's say common neighbors between the nodes. And this is like some redistribution maybe of a function is likely to happen.

• 44:16Speaker 00

  And yeah, so this is a very deep topic, very, very interesting.

• 44:21Speaker 00

  And then as for, you know, for concepts appearing. I mean, yeah, there is a number of things that happen with the fact that we're just working with the dimension N, but this is maybe jumping across topics.

• 44:32Craig S. Smith?

  Well, actually, I was gonna jump.

• 44:39Craig S. Smith?

  But well, I will. So you're in complexity studies and you're studying emergent properties. And the obvious thought is that in consciousness studies, you know, a lot of people think consciousness is an emergent property of the complexity of neural interactions. Do you have any thoughts on that? I mean, I know that's in left field. But are there, or rather, in a less sort of unanswerable way,

• 45:24Craig S. Smith?

  is there, do you think there will be creativity in this kind of a network in that?

• 45:33Craig S. Smith?

  The network will be able to think beyond the fixed parameters of a of a of an L L M. And come up with new ideas. And this is something I was talking to David Habbard and with

• 45:52Craig S. Smith?

  evolutionary systems in general, they're not.

• 45:56Craig S. Smith?

  They're they're looking across a landscape of not simply, you know, following gradient descent to a. Yeah, to a to a local optimum or something. Uh, so so they can jump over across to find, um, uh, you know, ideas that that didn't wouldn't necessarily emerge from a fixed L L M. So, yeah.

• 46:28Speaker 00

  Like, yeah, absolutely. So I I don't think I feel nearly qualified enough to have a position of how consciousness uh, appears in the brain or how it can, you know, uh, appear ultimately in a lens. I mean, it just somehow completely by accident, I started reading about Heidegger uh, over Christmas. Um, and and I feel, I mean, I I don't even know the advance, the advances in philosophy lately, but uh, the definition of being and thinking maybe somehow uh, impacted by by by what's what's happening with the reasoning models. Uh, so I didn't search as my, you know, kind of

• 47:11Speaker 00

  cultural thoughts. Um, when it comes to generalization, of course, this is the main goal. This is why we're doing all of this.

• 47:20Speaker 00

  Um, because the, you know, the real innovation wouldn't be just recomposing things that exist, but pretty much seeing the loophole. And and and this is then when when you know there is something interesting that should be added. How do you know that there is something interesting that should be added? That you need to actually somehow modify what's uh, your modify your topology, right? Um, and yes, so we are. There are two ways to maybe generalize to generalize the the most imminent generalization that we are after is generalization over time. So this is the kind of capacity to maintain coherent reasoning

• 48:04Speaker 00

  over time without falling into hallucination and just doing something completely silly. And that's the intuition. And then generalization in terms of having really, I mean internally, we call it the model which is a real innovator, right? One that can come up with eureka moments. And these eureka moments, they don't result from very formal thinking. I mean, I believe that very few mathematicians really kind of start their proof and work. It's formally step by step, reaching the conclusion. I mean, usually they have in different, different bits of information of conviction, filling in dilemmas on the way. Sometimes they're even like strategic proofs.

• 48:50Speaker 00

  So I know mathematicians who actually design ways to, strategic ways to get to their proofs.

• 48:57Speaker 00

  So, so yes, and I think that this will go through having having some sort of like an internal representation that allows for it. I mean, our hope is that having this plasticity and the structure, which, you know, is not 2D, is not 3D, is actually somewhat efficient and emergent. Uh, will, will make it easier. And then there is of course a lot of work to be done for this.

• 49:26Craig S. Smith?

  So what are next steps? I mean, this is not productized yet. It's still in the research phase, is that right? And where are you guys headed?

• 49:37Speaker 00

  Correct, knowing that you're actually right now working on productization. Because I mean, naturally it unlocks a lot of, a lot of interesting properties. I mean, some of the first ones are infinite context windows.

• 49:50Speaker 00

  Enhanced reasoning in some use cases can be, you know, jump from zero to one. And so we actually partner with, with Nvidia and AWS. Something that we announced at Rainvent in December. Uh, and the moment our, you know, the first model will be ready, it will be immediately available to AWS customers. Um, and yeah, so, so this, this is, I'd say, this is being productized, productized as we speak, knowing that we're productizing along.

• 50:20Craig S. Smith?

  Yeah.

• 50:20Speaker 00

  The research.

• 50:21Craig S. Smith?

  And, and what kind of use cases do you think the first iteration would, would be applied to?

• 50:28Speaker 00

  Um, so the very good use cases for this are the ones that are linked to small. So we're looking at highly valuable, small data where, uh, where actually one to, we were doing, for example, research about, um, about, or you make new designs for, uh, anyway, we, we actually saw a use case like this for, for, uh, in, in the nuclear space, right? You don't have a lot of, uh, a lot of documents, but those that exist are actually very, very highly valuable and you, you, you would like to get some ideas for, um, for you, for the current engineers, right? And ideally, um, like you want, you want to learn it very, very effectively from, uh,

• 51:13Speaker 00

  because there's just not enough data. Uh, and then we're looking at, at, at cases where you have, uh, continual learning and, and data that kind of keeps on changing, um, where, where you want to deliver values. So, and, and then, and then also reasoning, which is slightly more kind of complex because you, which is highly personalized. So the use case that we look at is, uh, for example, healthcare claims resolution. Um, this is, this is because of, uh, because of the fact that, okay, it has to be very personalized. It is complex reasoning because you need to, within the context of, you know, what's, what's, what's linked to the reprocess of, of, uh, of, of, of the, of the claim resolution, all the information that was, that was given.

• 51:59Speaker 00

  You kind of need to, you need to resolve it and also you need to be able to explain why and how. And because we kind of see, we have, we have some advantages in terms of interpretability. Uh, because we kind of see the synapses that light up. Uh, and then also in reasoning, I mean, you could, you can have layers of, let's say, explaining of what happened. Um, we, we have some benefit, like big benefits for the regulated industries. So, but journey's begin, think about small data, uh, with time changing element.

• 52:37Craig S. Smith?

  Yeah. And.

• 52:40Speaker 00

  And potentially next best action, uh, suggestion, uh, yeah, which is, you know, or, or like a sort of reasoning.

• 52:48Craig S. Smith?

  And in terms of, of

• 52:52Craig S. Smith?

  the traditional, traditional, I mean, they've only been around for a few years, but L L M

• 53:00Craig S. Smith?

  weaknesses, hallucination, th- does this, uh, eliminate the, the problem of hallucination? Uh, because hallucination comes, happens during inference when I mean, from my simplistic understanding, when the probability distribution on the next token doesn't include the correct token. And, uh, and, and gives something that's in the distribution, but then that error is propagated, uh, as, as you go along.

• 53:44Craig S. Smith?

  Uh, is, is first of all, is that right? But second of all, uh, in that you're not, uh, predicting, uh, the next token from a probability distribution, you're, you're you're, I don't know if reading's the right word, but you're, you're looking at, uh, an existing, uh, the, the activation of a existing sub-network within the network or within the graph. Uh, the, yeah, how does that affect hallucination?

• 54:23Speaker 00

  Yeah, so it's actually one of the big motivations for this work was to limit hallucinations, right? And especially hallucinations over time. So important to note, because I think we didn't stress it at the beginning, all the dynamics that we talked about happen at inference.

• 54:43Speaker 00

  It all happens at inference.

• 54:47Speaker 00

  So indeed, this is the hope, that you, because you keep the memory, you can keep consistency and as you go on, you don't fall over very easily. I mean, that's like a structure to hold on to. I mean, it was too early for me to communicate results.

• 55:13Craig S. Smith?

  Yeah, and you said that it follows the same scaling laws or similar scaling laws to transformers, which was a huge advantage for transformers. And, you know, what they did is just build bigger and bigger and bigger networks, and it got better and better and better, at least to a certain point. Do you think the same will happen with BDH, with Baby Dragon?

• 55:50Speaker 00

  Yeah. Actually, the paper is Dragon Hatchlings. It's Dragon Hatchling and the acronym is BDH. And I think people are very, very just tempted to put B, explain the B. So it's Dragon Hatchling. We are not playing the scaling game, in the sense that we don't need to grow the N so much. And the ultimate goal, and this is, you know, this is the goal of almost everybody, for us, the path goes through memory. But the ultimate goal is to get to generalization, and generalization in reasoning.

• 56:30Speaker 00

  So yeah, we are focused on reasoning models and seeing actually not the scale per se, because the power won't be coming from the scale.

• 56:44Speaker 00

  So yeah, we don't need to, we're not playing the game of having our N, you know, being humongous. Because I mean, that's precisely not, not the reason why we're doing it. I mean, this is a way more efficient way of

• 56:59Speaker 00

  doing, of, of, of, of learning and then, and then evolving and storing memory. In hardware sense, it also sits on the chip.

• 57:09Speaker 00

  And that, that, this is a big advantage. And if you look at those, like, if, if it sits on the chip, you do less lookups and you know, you may divide your compute for reasoning 10 times

• 57:21Speaker 00

  for the same output token, right? And then plus you can probably do reasoning on cases which are, which are more complex. Because you have consistency, overall consistency, you have capacity to work for longer without falling off into, into hallucinations. So it, it may be a more sustainable, I mean, we believe it's a more sustainable way forward also in terms of, in terms of compute. And, and how it distributes. It's a distributed system. So there is a part of theoretical computer science, like a very big community actually, that, that deals with distributed systems. And it is it. So for, for those listeners who are here, it is a, is effectively a, like a distributed system.

• 58:08Craig S. Smith?

  The, is, is there, the, the reason you're not looking at scaling is because the number of connections between the nodes or the neurons in this graph can, can, you can pack enough in that you don't need to scale the number of parameters or or, I mean, it could be that if you scale, you'll, you'd be able to contain that much more information in.

• 58:44Speaker 00

  It's possible, it's possible. So I'm not saying, but we don't need, we don't need to reach this scale. Our success is not blocked by the scale, by the, by the size of N. I would say it this way. So far as focusing on, on, on, you know, on a, on a, on a, on a larger N, that's, it's probably fun, but I, I mean, our, our, our goal is in fact to get to this, you know, reaching in places where it wasn't possible before.

• 59:11Speaker 00

  And so, so I'd say we have a slightly different objective. And for this, as of now, we didn't see the scale as being, being the main bottleneck. And, and yes, this structure allows us to pack a lot in the connections, not in the, not in the, in, in the N, but in the, in, in the synapses. And then keep it apart enough because we activate only parts of it and, and such that when we touch one part, we don't necessarily touch the other part of the, of the model. So I mean, hence, hence some of the nice properties for, against catastrophic forgetting.

• 59:43Speaker 00

  And like the plasticity is very, very important in, in, in this entire work.

• 59:49Craig S. Smith?

  Yeah. Uh, you, you know, with transformer architectures, once they were validated and they exist, then there's this whole second layer of activity in combining models or combining, uh, different kinds of models into, into larger systems. Do you think this would work with traditional L L M's?

• 1:00:15Craig S. Smith?

  You know, maybe, uh, I don't, I don't know, maybe the, the, the, uh, the transformer architecture handles one thing and the, and the post transformer architecture handles something else.

• 1:00:31Speaker 00

  Oh, in that sense. So, I mean, in terms of, you know, of, of, of, of maybe. Putting models together, I mean, at the engineering level, I believe there will be different use cases for which different models will be used. Uh, and, you know, for, for your traditional kind of knowledge type of use cases where we have language models, I mean, they are doing phenomenally well. Um, and, and, and I believe that a lot of the right now chatbot types of, of, of use cases, I mean, they, they will stay with the L L M, so they will be doing great. So, you know, I, I believe that the, the models based on, on, on, on B D H, uh, and when she had like other architectures, I mean,

• 1:01:12Speaker 00

  they, they will be, they will be used in use cases that, for example, require, uh, consistent reasoning over a longer time, right? So, and with, with fin data. So we're definitely looking at enterprise use cases, uh, naturally, or, or, or deep innovative research. Uh, where you, where potentially with also changing, changing inputs, right, over, over time, because you may want to, uh, kind of maybe iterate along with the environment as, as, as, as, as, as things progress. Um, so I don't know, in the engineering sense, I imagine you can, you know, you can use different models for different bits and it's kind of a question of, of, of just, I

• 1:01:53Speaker 00

  mean, building the system in which you can do it. On the more, uh, fundamental way, can you, can you glue B D H with a transformer and have kind of one model that somehow works together? Because, I mean, this is, this is what some people who modify attention do. Um, this is not the case. Uh, this is not the case. So this is, this is actually, it's either transformer or, or, or, or B D H, because it's so different, so fundamentally different. Um, so it's not, let's say, a plug into a transformer, uh, or a plugin. Um, then there is however, another, just, I mean, anchoring maybe on the merging kind of type of, or, or, or gluing, uh, gluing angle. Uh, because we have just this

• 1:02:36Speaker 00

  dimension N, like just the neurons, right? It's, it's a graph and it distributes its system. You, you can totally see that you're taking those brains and you kind of train them separately and you glue them together. And then you maybe have a couple of runs of training together such that they form connections between them and then becomes even stronger. And in this sense, you can get, you know, not not a separate mathematician and a separate separately trained computer scientist, but all of the sudden you actually have a mathematician and computer scientist who combines the intuitions of the two disciplines in one. Or, you know, this can work for finance and law.

• 1:03:18Craig S. Smith?

  You're talking about training of separate graphs and then merging them, is that right?

• 1:03:26Speaker 00

  Yeah, then gluing them. And we glue them along this one dimension N. We actually show a small experiment in the paper for this, where we actually didn't do the kind of the round of training when they were glued together, but we actually on language, we do, we're going to glue the two, the two models, train in two, two different languages. And then all of a sudden they start to speak, you know, some sort of like Creole. I mean, like a language that makes sense, but mixes the words of, of, of, of the two languages. Point is, it's very, it's very simple with, with this architecture, because, because you effectively, you look at the graphs and you just

• 1:04:03Craig S. Smith?

  Yeah.

• 1:04:04Speaker 00

  put them together. So it look, it, it, it feels a bit like composable programs.

• 1:04:08Craig S. Smith?

  Yeah. And it sounds, you know, again, Sakana AI came up with this, I think they call it model merge, where they can put two models together. And that's part of their evolutionary strategy that they, they'll have a bunch of models, models, you know, generate output. They, they pick the best models and merge them into, presumably you could do that as well with, with BDA.

• 1:04:46Speaker 00

  I mean, I presumably you could. I mean, I haven't explored it in full honesty. But yes, you could technically, because you can emerge them. And if you want to train evolution, like

• 1:05:01Speaker 00

  do some sort of kind of evolutionary pruning of what's your best model? Then probably you could do it. I don't know if it has a purpose in our case. And but technically, yes, the merging is.

• 1:05:15Craig S. Smith?

  Oh, that's all fascinating, Susanna, really. And and yeah, because transformers, it seems, have kind of hit a plateau or are hitting a plateau. So it's fascinating to see new architectures emerging or new strategies emerging. Yeah, OK, well, let's leave it there. Is there anything I didn't ask that you'd like listeners to know, maybe where they can go and find find BDH or? Yeah.

• 1:05:55Speaker 00

  Yeah, thank you. Thank you so much for this conversation. It was great. And I mean, these were pretty deep questions. I really appreciate it. So I mean, you can you can reach out to us. I mean, for me, LinkedIn is probably the easiest also. Research at Pathway.com. This is for all the kind of research types of questions. And we're very, very happy to answer there. Please note that the paper is available on GitHub.

• 1:06:25Speaker 00

  Do they just it? It's pretty long. I mean, Craig, I think you've got it. It's very deep. It's very dense and very long. Because it has to call it covers a lot of intuitions, actually a lot of proofs as well. Because we're showing the link between transformer and the brain that goes through expressivity. So all of this goes pretty deep on neuroscience. Physical computer science, that is not always the thing that's being the most, let's say, studied in the AI community. So I can strongly encourage something that we've seen working a lot, very well with with many people. Actually, by now, L L N's are doing a phenomenal job of launching this paper.

• 1:07:04Speaker 00

  Especially, especially the reasoning, the the most advanced reasoning ones. Once we ask them to properly read the paper with, you know, with the proofs and then and then break it down, they they do an amazing job. So I I would strongly, strongly encourage you to use AI to kind of read about AI. Yeah.

• 1:07:23Craig S. Smith?

  I read it on archive. It's the same paper, right?

• 1:07:28Speaker 00

  It's the same paper and the paper, the paper is publicly available, of course. And then there, I mean, there is also, there are also other podcasts, you know, that are that way more technical with with other members of of our team. So I would also, I strongly invite you to to to listen to this if you if you have questions. This one was great. I mean, thank you. Thank you so much.