Memory and Continual Learning: Engram's Dan Biderman and Jessy Lin

Dan Biderman and Jessy Lin, co-founders of Engram, are building a neolab around memory and continual learning, which they call two sides of the same coin. Their contrarian premise: instead of stuffing ever-larger prompts into the context window or bolting on RAG, bake a team's knowledge directly into the model's weights, so it knows your company the way an employee of several years does. The payoff: matching or beating frontier models while consuming up to 100x fewer tokens. Working with partners like Microsoft, Notion, and Harvey, the team draws on roots in computational neuroscience and state-space architectures to attack what they see as the real bottleneck in AI — not raw intelligence, but memory and continual learning. In contrast to the frontier labs' race toward one ever-bigger model and AGI, Dan and Jessy imagine a world where everyone has their own model — privately trained, always learning, and good at the things you actually care about. The real ChatGPT moment for memory, they argue, is the day your model feels like an intern that genuinely got smarter overnight. Hosted by Sonya Huang and Shaun Maguire, Sequoia Capital

• 0:00Speaker 00

  What about pre-training or even post-training makes it possible for the models to generalize in these magical emergent ways and controlling that process? So that a company has a set of private data? How do we make the models learn that just as well as the models know like the capital of France? Or you know, like how to write Python? So I think it's a really fun problem to think about.

• 0:41Speaker 01

  Welcome to Training Data. We are delighted to have Don Biderman and Jesse Lin, co-founders of Ngram today. Ngram is a Neolab focused on memory and continual learning and two of the hottest topics in all of AI research today. And Sean and I are delighted to dig in on those topics with you today.

• 0:57Speaker 02

  Awesome. Happy to be here.

• 0:59Speaker 01

  Great. So maybe to kick off, the Ngram website says, "We don't see the world through the lens of pre-training or post-training. Our models are always training." What does that mean?

• 1:09Speaker 00

  So I think like models today, obviously know a lot of things. They're incredibly smart, but we kind of think the bottleneck for making these models more useful these days is not really raw intelligence, but understanding like new and evolving context. So whether it's like, you know, a new task that you're doing or a particular context for, you know, like a job or something like this. How do you bake that into the model weights? The same way that, you know, pre-training and post-training bake that into the model weights very deeply. And this is kind of why we think of ourselves as working on these fundamental problems of memory and continual learning, which are really two sides of the same coin. How do you make the models learn new things and bake them deeply into the weights of the model?

• 1:51Speaker 01

  And is your premise then that memory as a separate database or separate? You know, thing that you've shoved into the context window is not true memory and is not true continual learning?

• 2:01Speaker 00

  I think all of these tools will kind of come together. So these days, like the way that people are solving these problems is with context engineering. So you take like a huge prompt, maybe you like keep talking to the model over many, many turns and hours and you know, reorganize the context to better understand like what you're trying to do. And we think like these kinds of things, like tool use, context engineering will play a part. But I think an under-leveraged tool these days is using the same kind of training pipeline or framework or kind of workflow that the frontier labs are using to make these models really good at frontier math or code. But applying that to every kind of domain, every kind of context that you have, like let's say in a company.

• 2:40Speaker 02

  Yeah. And to me, it's like, as an individual, taking notes and having sticky notes is a very valuable thing. We should never discard this. But whenever we get back to business, the next day, we always have some sort of trace of memory in our brain, some new intuition about how things should be and where should we look? So these two things should come together. And current solutions are more kind of externalized memory. And this has two, two issues. One is that the amount of tokens we will all collectively individually generate is going to be in the tens of millions of tokens per day soon. So just keeping it and searching through it is going to be and rereading it's going to be pretty expensive, but it's going to also be pretty hard, pretty confusing for the models unless we have major, major breakthroughs.

• 3:23Speaker 01

  Tens of billions of tokens for Sean.

• 3:26Speaker 02

  That's good.

• 3:27Speaker 03

  Depends on the day.

• 3:29Speaker 01

  Could you maybe tell us a little bit about the N-gram architecture, the N-gram product and how it works?

• 3:35Speaker 00

  Yeah, I mean, at a high level, I think what we're trying to do is take any context, like there's all these different workspaces, let's say, so we're working with partners like Notion and Microsoft and Harvey that have these places where people are doing a lot of work over a long period of time. There's all this context. Both in terms of like, you know, documents that you've already written as a team, as well as like now people are interacting with these agents more and more in these products. We're having conversations, giving them feedback. And figuring out how to have a model that deeply understands that context. So not just reading the files at test time, but really understanding it the way that an employee that's worked at your company for years has. So you kind of understand at a high level, oh, these are the initiatives across the company.

• 4:19Speaker 00

  This is the way that we do things. You've studied like how to run the hiring pipeline, or how to, you know, do this kind of thing within the company, and can operate just as well as like any, anybody else can in the company. And so what we're doing is training per team models within these workspaces that deeply understand those contexts and can improve with time on the things that people care about. So the way that we do this at like a technical level maybe is training these into weights. So we do a lot of like adapter fine tuning. So adapters of many types, like I think people have looked into this for decades at this point. Like whether it's LoRAs or prefixes or, you know, sparse architectures. I think like all of these tools are at our disposal.

• 5:04Speaker 00

  And then figuring out what the right data is. So how do you turn any kind of raw like document or interaction into useful training signal for the model? So again, we have like a variety of tools now, like supervised fine tuning, you know, RL, you know, on policy distillation, like all of these things that, you know, the field can, has kind of developed. And trying to fit these pieces together into a model that learns continuously on the things that people care about.

• 5:31Speaker 02

  Yeah, and it's not a bet that tools are not there. Like our models always work under the assumption that some knowledge is externalized. Some tools are always there. But what you need to do is you need to figure out, and that's the hard task, is what needs to be internalized and what can be externalized. And for even for stuff that's externalized, many individuals and companies have their own bespoke tools and ways of doing things. Not everyone has the same, you know, BASH CLI tools that, you know, the frontier models are training on, and how to get the models to better understand your bespoke setup, I think, is its own interesting thing.

• 6:03Speaker 01

  And so is the premise then that my Notion agent will be a custom agent that is Laura fine tuned or you know, it's some way with an adapter tuned so that it's constantly learning on new content that's added into my Notion workspace? Is that the premise?

• 6:19Speaker 02

  Yeah, and they're working with many models and they're the early users of all the frontier models and they're probably gonna keep doing that.

• 6:26Speaker 01

  Does this approach work on the frontier models or are the closed frontier models?

• 6:30Speaker 02

  So we need, we need wide, you know, we need wide box access to the weights, right? So, you know, we can partner with companies that have, you know, closed source weights and do this with them, but it's easiest for us to do it with with open source models. But any model that's a transformer model, we can do our thing to it.

• 6:50Speaker 01

  And what's the trade off then when when people are comparing the before and before and after using you, is it that they're no longer sending so much context? And so the trade off is like you burn more compute upfront to learn your company's way of doing things into the weights and then you're sending less context to the model on every inference pass? Is that the rough trade off?

• 7:10Speaker 02

  Yeah, that's, that's one, that's one thing. The fact that you don't have to research things and the fact and re-read things and the fact that you don't have to write like monstrous system prompts, they're already, that can give you, you know, two orders of magnitude reduction in token inference consumption. It's not like, you know, 50% or it's, it can be 100x fewer tokens because many things, especially things that relate to people and teams and organization and priorities, these are things that you can't really find in one document unless like you really have it, really regimented and document everything. And these kinds of things, the model can kind of implicitly learn by training on some of the data and answer, you know, within 100 tokens, what, what the best frontier models would consume 100,000 tokens doing. So these kinds of examples are interesting.

• 7:54Speaker 02

  And also the the quality, you know, there are tasks that are, you know, not supernatural for the current generation of the models and we we're we kind of think there's going to be consistently this gap of like three to six months ahead where there's certain things that are bespoke that people are just exploring. The models are not fully great for them. The models will at some point be great for them, but if you can autonomously learn in in a very lightweight way, it will give value in that time in terms of capabilities.

• 8:19Speaker 01

  Why train on the workspace level versus the individual level, for example?

• 8:22Speaker 02

  Uh, either is is fine for us. It's just easier to start with, you know, it teams of people have, you know, are more, you know, disciplined in how they collect context and in the amount of context they have over years. And it's easy for us to start there, but every person's computer and every person's phone one day is a useful, you know, target for our technologies. In fact, it will be very interesting to to go there. We just think, you know, the big deposits of information are now in in teams of people collaborating and knowledge work.

• 8:51Speaker 01

  Is it a feature or a bug that there is so much fact memorization basically built into large language models? And there's a school of thoughts that, you know, the mem- the models just wrote memorizing the fact that the capital of France is Paris is actually a bad thing. And what we would prefer for the models to do is, you know, abstractly learn the concepts of countries and capital cities.

• 9:10Speaker 00

  Yeah.

• 9:11Speaker 01

  But not to memorize all these facts in the weights. And so I'm curious what you think about disentangling memorization versus learning how it's done in the models today and then how you're thinking of approaching it.

• 9:24Speaker 00

  Yeah, I think it's a really interesting question. Like, to some extent, you kind of need to remember stuff in order to like compose them into more complex concepts. I think the thing that's kind of missing is figuring out what's important to remember. And I think even now, when you think about like learning new knowledge, if you look at a lot of these academic benchmarks, it's like, how can we learn very specific facts, like, you know, the length of a bridge in this like African country? And that's not something that you really want the models to devote capacity for, and it's not something that we devote capacity to. Um, so I think if you look at human memory, I mean, you can say a lot more about this, but like it's lossy. Um, because part of the feature of intelligence is compressing what's important and separating that from what's not important.

• 10:07Speaker 00

  Um, and so I think like you can't really separate fact learning from like non fact learning or skill learning as some people would like to think. Um, like if you take a model and like, some people have done this with models where you like strip out, you know, like all the facts and just have it like the pure core or something like this. It's very unnatural as a model. It doesn't know basic things. Um, and you kind of need, need that, but I think.

• 10:33Speaker 01

  Why do you need that? Like, why can't you look up facts and then just have?

• 10:37Speaker 00

  I think if you look at like how the models think, if you need to recall basic facts in order to like, take the next step in your thinking, you can't get very far. Uh, maybe that's like a high level intuition, but it's part of like the reason why we think training is really important. In order to like, think more and more complex and deep thoughts about things, you kind of need to internalize some things so that you can compose them into more abstract concepts.

• 11:00Speaker 02

  Yeah, and, and there have been efforts before that were hard to scale to try and, you know, disentangle the two and pre-train the models in a way that's, you know, allows it to retrieve and search for things and not internalize them. It's just the recipe we know to hill climb on collectively right now is this, you know, fact pre-training step. And I think the, the magic of, or the, the mystery of this approach is that, you know, traditionally in CS, we would have, you know, databases as its own curriculum and we would have algorithms and the databases is like facts about the world and capitals of whatever, store them, query them. There's also algorithms of how do you efficiently manipulate information and get some answers uh, in, in a sample efficient way. And I think the magic of deep learning is that these two things are

• 11:46Speaker 02

  now mushed together and we need all these smart people and tropic interpretability to try and, and, and, and break them apart. And I think a lot of what we're seeing now in the adoption of AI into the economy is that these things are gradually separating again, where companies have their own context and they really

• 12:03Speaker 02

  handle them with care and engineer them with care. And there's a generic model that's completely a stranger through these contexts. And the, the model is operating on them, but it, for us, it's clear that there needs to be a certain convergence, uh, at least with, with some cadence. Uh, where, where the, the facts and the stories and the details are, are getting mixed into the model. It has disadvantages as well, because if you, you have to, you know, capitals of, of, of countries are, you know, they can change, but it's not very frequent, but there's many other facts that are changing all the time and just imprinting them into weights is a, is a challenging thing to do.

• 12:35Speaker 01

  I see. So you're saying it's a false dichotomy that's trying to separate. Algorithms from databases here. Um, what really matters is like how to distinguish what's important to remember.

• 12:45Speaker 00

  Yeah.

• 12:45Speaker 01

  Versus what's not important.

• 12:46Speaker 00

  Exactly.

• 12:47Speaker 02

  And it's an open question.

• 12:48Speaker 01

  How we dream and are you guys taking any inspiration from that in in terms of ranking?

• 12:52Speaker 00

  Very, very loosely, I think, just the idea that that's kind of a phase that's missing, maybe where you take a context and you deeply internalize it. Right now, it's like everything happens at test time. You look at the, you know, context that the user gives you and you do some like thinking on the fly. Um, but again, like you can't get very far, um, or you can get so far, maybe. And like you make mistakes along the way, like, how do you digest that back into the model so that next time you do it, you do it the right way? And make even more progress.

• 13:21Speaker 02

  Yeah. And what are dreams? Dreams are pretty crazy things to to say we want to build an AI that's like our dreams, it sounds a little bit like a nothing to do. There's not a lot of coherence there. But what's interesting there is like what happens in our dreams, we we we see things, we talk to ourselves. And we we experiment with the affordances of what can we do and can't we do in the world and social situations and then, you know, uh, any, any, it's heavily biased towards social. Uh, stuff, right? So for us too, with things we're building is, you know, you, we give the models. The time to then go back, retreat from the actual interaction and experiment with its affordances. What can it do? In an environment, what can it, what does it know? Uh, how fast can it, can it, you know, handle these kind of tail extreme?

• 14:05Speaker 02

  Uh, things that same ones that we dream about at night.

• 14:08Speaker 03

  You guys come from academic backgrounds, like, what's a canonical example that motivates this problem or, you know? Like, or that's a win so far?

• 14:19Speaker 02

  Yeah, I have one example, maybe Jesse can give another one. A hypothetical one, for example, imagine one of the AI labs, say, OpenAI has to win some math Olympiad in a week's time from now. Would they construct the catalog of all the math textbooks and really have? People annotate which chapters to get and which graphs to to see, or will they actually collect this, synthesize some training data, launch a training job, see where it lands in five, six days, start evaluating it and stuff like that. So it's it's obvious for anyone who's trained models that there's superior way to integrate across the ideas and capabilities and involves this kind of magic of training. And we are clear that this has to happen in those high stake domains of math and coding and cyber and stuff. We just think

• 15:03Speaker 02

  much of this magic can actually end up in the in the hands of of many more people in in interesting ways.

• 15:09Speaker 03

  Like, why isn't it just the foundation model labs that own the end product here? You like, how do you go between giants?

• 15:18Speaker 00

  Yeah, so I think like, the worldview that we have is a bit different from the frontier lab worldview, where it's like, we want one model that's bigger and bigger, that's more and more intelligent across a variety of domains. Instead, how we see it, like, we kind of imagine this world where everybody has their own model. A lot of the things that people want to learn are either private, like, things that will never see the light of day in a post-training data set, or even conflicting, like, oh, the way that I want to do the task is different from how another company or another individual wants to. And I think a lot of these things we're already seeing are hard to train into the models with the same tools that we have used for like decades in machine learning, which is like, you have really clean supervision, you have like ground truth reward signals,

• 16:02Speaker 00

  and you like create a nice environment and you like train the model to like use the tools to better accomplish this like coding task. And instead, a lot of the things that actually happen out in the world are very ambiguous, or like, it's hard to say like what makes something good. And so I think a lot of these things are very specific to individuals, and I think very kind of misaligned, or not very aligned with how the frontier labs think about the whole training pipeline and what kind of models will exist in the longer term.

• 16:33Speaker 02

  Yeah, and to add to it, I think, you know, what is the P zero for the frontier labs? And some of you here are are pretty close with them. It's getting to AGI, getting this one generic model that's extremely capable. In coding and math and then using it to automate the economy or to solve really hard, you know, long-term problems in cryptography and defense or whatever. And it's pretty clear what needs to happen to push this, you know, more pre-training, bigger models, more data, more RL, more inference time compute, that kind of stuff. That's P zero. That's where the majority of expenditure and talent goes. And definitely, all of them are thinking about memory and all of them are thinking about continual learning. It's just more of a product kind of effort right now.

• 17:16Speaker 02

  We think it deserves its own attention. We think breakthroughs need to happen there and Demis and the Sequoia event about a month ago said pretty clearly that we need new breakthroughs around these topics. And obviously, they're thinking about them. We're just focusing exclusively on this. And we think certain things around incentives of where the data is and who owns the model are pretty interesting. So if you could learn from many humans or organizations at scale, without necessarily sending someone to work with them shoulder to shoulder, that would be a pretty big unlock.

• 17:49Speaker 00

  And maybe another point on that is like, I think a lot of things need to look different in the world. So one is there needs to be new research breakthroughs. Two is new infrastructure for training, like, you know, small models for everybody, rather than like one big model, one big run. And then the third, I think, is a different way of kind of combining research and product. So right now, I think like, there's like researchers in these frontier labs. They kind of train the model, they throw it over the fence to the product team, who then like prompts or context engineers like new product surfaces on top of the core models. But in this world where the models are always training, I think the inputs that users provide are very intricately tied to what the models learn from, like what the training signal is.

• 18:31Speaker 00

  And so there needs to be a lot more of a kind of integrated loop between like research and product. And so like while we're focused on tackling a lot of the core research challenges, and that's our background, I think we're also very focused on like how to deploy this as quickly as possible to like learn from actual feedback in the real world.

• 18:47Speaker 01

  What motivated you to work on this problem?

• 18:50Speaker 00

  I think like it's obviously like one of the grand challenges in AI. I think everybody's talking about it these days because like the models are so smart. What what else is left, you know, it's I think learning like at the edges, like learning the remainders of what makes these models useful. It's not just about raw intelligence anymore. It's about like learning new things. And I think it also feels very fundamental because it kind of goes back to really understanding what makes the model so good. So right now, the models kind of incidentally know a lot of things from pre-training, and we don't really understand why. It's like the Internet was just, you know, this gift granted to us where there's like a diverse set of data that contains like all of these different examples of coding

• 19:33Speaker 00

  and like writing and all of these other things. And it just happened that way. And now to figure out how to crack this problem of continual learning, it's about figuring out what about pre-training or even post-training makes it possible for the models to generalize in these magical emergent ways and controlling that process so that, you know, a company has a set of private data. How do we make the models learn that just as well as the models know like the capital of France or, you know, like how to write Python? So I think it's a really fun problem to think about.

• 20:07Speaker 01

  And Don, you came from the neuroscience world, is that right?

• 20:10Speaker 02

  Yes, yes. So I was initially interested in in questions around, you know, consciousness and the human condition and things like that.

• 20:17Speaker 01

  Are the models conscious?

• 20:19Speaker 02

  I don't have any any advanced thoughts on this more than you and you and Reid. I don't think so, but it's important that smart people are thinking about it. I would say like I was interested in how humans think, how humans perceive. And as Amos Versky, the Israeli psychologist, used to say, like, he's not interested in artificial intelligence, he's interested in natural stupidity. So I would say like I started kind of similarly, trying to see how people and animals experience the world. Gradually, you know, my inclinations took me to the stats and AI domains. And there I figured that so many of the same problems of memory and continual learning are really, really urgent. And the kind of solutions we have in the current systems are pretty far from what we have in biology. And I'm not one of these people who would say that the the machine should be like, you know,

• 21:04Speaker 02

  like the animal or the the human brain. I don't think so. There's many things computers can do better than us. But human memory has these like very different uh things in it. It's, you know, if you want to store a whole code base or you you can you use a computer, you don't even need AI on the computer to store everything losslessly and just get it. But the human brain evolved to work in these constraints of of, you know, information capacity and to have these fuzzy representations that can then, you know, be abstracted and form connections and inform the next day. Current systems don't really have that beyond the generic pre-training step. And I was really interested in, you know, what are ways to to to build that in? What are ways to learn from that?

• 21:44Speaker 03

  This is more of a philosophical question. You know, you mentioned in the brain, there's a bunch of different, real estate, different co-processing units, whatever, modern computer architecture, there's CPUs, GPUs, you know, memory, there's different co-processors.

• 21:59Speaker 03

  With like the bitter lesson, do you think that what's happening is that like L L M's are, you know, converge to say like one co-processor that's just totally dominant? It's like everything, all compute is going to happen in, you know, the GPU equivalent of like a language model? Or do you think that these models are kind of building a bunch of co-processors? Like, you know, emergently inside the model? Like, like, you know, and take with memory, like, do you think that the models themselves will just build, you know, whatever part of the brain equivalent would be that's good at memory or or do you think there needs to be like

• 22:44Speaker 03

  another standalone architecture that?

• 22:47Speaker 01

  Yeah, like is is memory an emergent property?

• 22:49Speaker 03

  Exactly.

• 22:49Speaker 01

  Versus like

• 22:50Speaker 03

  Yeah. And almost everything, like is everything that we need in intelligence will just be emergent with better training data and more scaled compute.

• 22:58Speaker 02

  Yeah, I would say just on a more like a superficial perspective on the current deployment of AI. It's way more than than just GPUs and we're seeing all these, you know,

• 23:07Speaker 03

  Of course.

• 23:07Speaker 02

  sandboxes exploding and models operating on other computers trying things.

• 23:11Speaker 03

  I'm more mean on the model architecture level rather than on the

• 23:15Speaker 02

  So other experiment, either there have been many previous experiments, different architectures that we contributed to like the state space family and others to try and handle very, very long context more efficiently. Uh, the thing with all these methods, it ends up being a trade off, usually a trade off between memory and accuracy and memory, not in the behavioral cognitive sense, memory in the computer sense, right? Instead of having, you know, the the memory footprint of the transformer attention, which is quadratic in the sequence length, uh, these models have

• 23:44Speaker 03

  Some are claiming, you know, they have sub quadratic.

• 23:47Speaker 02

  Yeah, some are claiming some do have it, right? And some of the the best Chinese model have layers that are, you know, inspired by those state space architectures and are, you know, not quadratic in cost. Thing is, is that in in in our hands, we find that, you know, you always compromise accuracy for this memory. There's no free lunch. Uh, and what we're saying is like, look, if you're really bitterless and pilled, what do you want to do is you want to think, how can I burn more compute? And how can I burn it on, you know, new context that I have not seen before? So we're as bitter less and pilled as anyone else. Uh, and we are not betting that the overall direction of AGI is is gonna, you know, end anywhere soon. We just think there's more compute to scale. And if I truly want to understand Sean and Sean's work and Sean's context,

• 24:30Speaker 02

  just like rereading files is not going to make it, especially for us, a special person like you. We got to

• 24:37Speaker 03

  Specials are

• 24:38Speaker 02

  We got to train a hundred trillion parameter for this guy.

• 24:42Speaker 01

  Cosine, cosine. What are you finding that people care most about their models learning? Like, is it memorizing facts about the organization? Is it remembering like, ah, no, we do CI this way? Is it like, what what are people actually hoping to? And then maybe this feeds into how you do the ranking of memory slots and all that.

• 25:02Speaker 00

  Yeah, well, I think if you look at what people are spending their time in the app layer doing these days, it's a lot of just trying to make the model work well for your use case. Like, oh, I want the model to like, you know, let's say like, design my website with my brand style. Like, that's like a, you know, very common example these days. But there's many kinds of different tasks that people do with agents, um, like learning how to run a workflow, um, or, you know, kind of your particular way of like writing, let's say. Um, so there's many, many kinds of things. And honestly, like, I think when we think about these methods, kind of going back to this distinction between like facts and skills, there really is none. Um, I think the methods are kind of agnostic to that.

• 25:45Speaker 02

  Yeah, to me, it's like the, the natural thing, almost all the app layers are basically, you know, a frontier model wrapped in, in a loop with search tools and stuff. And what they're all interested in doing with us is finding ways to kind of interface with their data in a way that's, you know, faster, more efficient, and also is more contextual. So almost all of them, it's like, we want to have our, you know, our firm knowledge, you know, be encoded in something that's more efficient, that I don't have to research. We want to have the model know in a targeted way who's the person I should triage a thing to. And we're just showing them that with pretty lightweight training, these things can, can be instinctual to the models.

• 26:27Speaker 02

  They don't have to have these very involved, long, uh, ripple loops to, to solve them. So it's, in a sense, it's like, um, you know, it's, it's a rag killer kind of, um, kind of thing. Again, we can always do a rag and we can always retrieve, but that's the thing that people are interested in interfacing with very large data planes and automating very repetitive, uh, things this way.

• 26:46Speaker 01

  Yeah, and I want to double click on this rag killer thing, and I'm sorry to beat a dead horse. I just don't fully grok it yet.

• 26:52Speaker 02

  Yeah.

• 26:54Speaker 01

  Is the premise that there's some trade off between doing RAG versus updating your model weights? Is the idea that you should be doing both? Like, what types of things should be done in the weights versus what types of things should be externalized to RAG?

• 27:07Speaker 02

  I think it's a it's an unsolved problem. I don't think anyone has answered to it. We're all working on it. It's also the fundamental question of like biological memory, what should be internalized versus what not?

• 27:21Speaker 02

  I do think that things that are like, you know, do you need to internalize the room number in a hotel that you were in like a year ago? Probably no, not in your neural tissue. Probably that's good to write down, but do you need to internalize? Maybe the password to your home right now, probably it's useful for the next few years to have that imprinted somewhere. So, yeah, how does this translate into like knowledge, work and products? This is still something we figure out and we try to take the approach that we try to use as few heuristics as possible. It's easier on filters on the data and say like, I'm gonna keep this, discard that, train on this, train on that. But as humans, you know, we watch TikTok and we, you know, get exposed to a lot of garbage and still the brain is able to learn and not completely go off the rails and we think models should be the same as well.

• 28:05Speaker 00

  Yeah, maybe concretely in the short term, I think a lot of what people are worried about these days is the huge inference costs of running these agents like for days on end.

• 28:15Speaker 01

  High inference cost's a good thing.

• 28:18Speaker 00

  I mean, consuming tokens for what?

• 28:21Speaker 02

  Sonia works at Fireworks. She really loves Zion.

• 28:28Speaker 00

  Yeah, so I think it's like in the short term, I think that's the immediate pain point. Like, why are you reading the same files over and over again, you know, even in the same query? But like definitely, you know, across people in the same company, they're running the same queries on the same documents over and over again. And that should be something the model just knows, like in the same way you ask an employee, they don't, you know, type into the search box like what was I working on yesterday? They just know.

• 28:51Speaker 01

  But doesn't caching kind of solve that?

• 28:53Speaker 00

  I think to some extent, yeah, but I think going back to this like question of what should be internalized versus what's like something you retrieve at test time? I think again, like a lot of it is about building on your knowledge. So if you are always doing RAG, you can't make associations like, oh, you know, I see somebody, you know, on the team is doing this kind of research and I kind of Like recall at an abstract level, oh, there's this like related thing that you might want to know about. You didn't even ask about it, right? But I think like these kinds of associations can only happen in weights because they're not really about, you know, you asked me to search for this, I'm going to search for this.

• 29:31Speaker 02

  And also, I think the main limitation with retrieval systems in general and in AI specifically is like, the problem is not so much what to store and where to put it. It's the problem is like how to address it, like how to query the thing? Do you know what to look for even?

• 29:46Speaker 00

  Yeah.

• 29:46Speaker 02

  And this is involves some sort of intuition that sometimes the models don't have. Interestingly enough, they don't know where to look. And especially if you're, you know, limited to the current way of doing things, which is keyword search. It is just easier to scale in RL and least involved in terms of like infra for embeddings and stuff. So yeah, knowing what to search is something that's intuitive and can and can happen in the weights. And also about caching and inference, like much of this company started with us taking like a deep dive into like KV caches and caching. And this is a fascinating thing, right? KV cache is a monstrosity of the current way of doing things that, you know, think about it. A KV cache for a single like Wikipedia article for some, you know, Taylor Swift or something like this.

• 30:30Speaker 02

  It will be like 80 gigabytes of HBM memory on the GPU. And an entire Llama, it's for say a 70B Llama model. And the entire weights of the model would be about 100 gigabytes. And, you know, with with some distortion, they remember the entire internet. And how come this thing is so, one thing is so bit efficient? And we have this proof of existence that gradient descent can pack a lot of information in very few numbers. Whereas this KV cache thing, you take a few tens of kilobytes of article and it becomes those 80 gigabytes of of brain state. So you're sure you can cache this, you can load this, you'll have issues with disk to HBM stuff. People are working on it, it's pretty interesting.

• 31:14Speaker 02

  But what if we can take those 80 gigabytes? Spend some compute offline, maybe also in fireworks. And file, but then compress it and make it really, really small. So that the thing we load in cache is like a thousand X smaller. That would have tremendous implications for how we load things, how fast we can do things and what the fidelity of their representation is.

• 31:37Speaker 00

  Super interesting.

• 31:38Speaker 02

  Yeah.

• 31:38Speaker 03

  What are some of the things that could happen in the next year or two that would be like the chat GPT moment of memory? Or do you think that that's not how things will play out?

• 31:49Speaker 00

  It's a good question. I don't know. I think like the first proof of concept of the thing that people keep talking about with continual learning, which is you have an intern that you can teach things over time and it actually gets better. I think everybody's waiting to see that, you know, and no matter how sophisticated the context engineering approaches are these days, they're not getting there. So I think you need, you know, all of these tools at your disposal to make that happen. But I think it will be something like that where it's like the model's actually getting smarter, like whoa, it's different from yesterday.

• 32:19Speaker 02

  Yeah, and it's important to say that the chat GPT model was not anticipated. We've just, you know, read about all the That's why I asked for few, not for one. The product directions that some people had before chat GPT was different. I feel like to me, the example is like, look, if you, you know, resigned from your job today and your sole mission was to make a model that's better for you. And you would use OpenAI and topic and all these frontier models and you just 24/7 engineer the context, write skills. Your way to move the needle is very limited as an individual. You'll just be better off waiting for the next version of the model and you'll take it from there. And we would like to see a future where actually the more time you spend on the thing actually translates to the quality of performance, at least in the things and domains you you care about.

• 33:05Speaker 02

  And this is pretty hard to achieve. And the only reason it we we think it could be achieved is if you start scaling compute and training on these data without destroying them all importantly, which is pretty hard.

• 33:17Speaker 03

  Couple like this is just for fun like rapid fire questions going off.

• 33:22Speaker 02

  Yeah.

• 33:22Speaker 03

  Just memory. When's the last time you reached surprised about something in AI? In any area?

• 33:28Speaker 02

  When reading about fundraising. A lot of surprises every day. I would say all of us felt, you know, a little bit of a change around the capabilities of the coding agents.

• 33:38Speaker 00

  That's true.

• 33:38Speaker 02

  But we've been, you know, dabbling with these things and trying to make them work in, you know, in more effortful ways before. So it didn't come as a complete surprise. But yeah, I think to me, the main events were GitHub Copilot. That for me was just the main event and ChatGPT. And then seeing the agentic stuff, we all anticipated, I think, and different, different people had different expectations on how far it can go and how long the horizon it can go. But I feel, yeah, it's we're yet to see something fundamentally different and people are working on completely new ways of doing things now. But yeah, to me, it's, it's models actually changing in a way that's not harmful. And learning new things on the fly that are, you know, personally and economically viable.

• 34:24Speaker 02

  That's interesting.

• 34:25Speaker 01

  Right now, there's this idea of like, we're each kind of a token wallet that we're going to bring around to companies. Or to different apps. Different workspaces. Do you think that we're going to end up with like a memory bank, a memory wallet that we're gonna move around? To across the digital world as we go?

• 34:43Speaker 00

  I think it's an interesting question. I don't know if we've fully figured out what the right kind of like product form factor is in this sense. In a way, even with like ChatGPT memory, let's say, I kind of don't want it to remember across my like personal and work context.

• 34:58Speaker 01

  Oh, yeah.

• 34:59Speaker 00

  Like, it's like, oh, you know, you might like these sheets because you trained a model on a GPU last week. It's like, that's totally irrelevant. And to some extent, it's like because the memory is flawed, but also I think you do want memory in your, I guess, tools and the products that you use to be separated to have control over that. So I personally think like there needs to be some separation there, but I guess to be determined what that might look like.

• 35:24Speaker 02

  Yeah. And like, I think a holy grail is like. You go to work and you just burn through all these tokens and you create all this value. And somehow, you know, all the IP and stuff stays with the company, but somehow the skills you learned, the things you invented, your ways of doing things, some of them you can take with you as well. To your next job in a way that's, you know, sanitized and not, you know, harmful to any other company's IP. So I do think like carrying a set of skills uh will be interesting. We do it in our biology right now and we just, you know, sign NDAs and have like ethical rules around it, but I think doing it in the digital world would be pretty interesting. And pretty rewarding because it will force each of us to push the frontier and implement AI more deeply in our companies and our individual life and then be rewarded for it.

• 36:08Speaker 03

  I started a PhD in stats firm in two thousand and seven uh at Stanford and AI like AI was boring as hell at the time. It was all statistical learning. And there's basically two areas like computer vision and NLP. So like vision and and language were kind of the two areas and I think that's still true. In two thousand and twelve, Alex, that happened, like vision was dominating for six years or whatever. Are you guys surprised that language seems to be like the language approach seems to be like dominating over vision in progress? Question two, do you think vision has any chance of coming back? How do you think about this?

• 36:48Speaker 00

  Yeah, I think it is pretty surprising to me. I mean, some people maybe saw it coming, but I think I've always kind of been interested in language um as like, I don't know, I guess like a medium for communication and like so many kind of complex abstract things can be done in language. Um, I do think like, you know, I imagine like in the longer term, language and vision will kind of like combine in this more like unified system where, you know, we kind of like. Taking inputs from all of these different modalities and like, understand them in this abstract way, but um yeah.

• 37:23Speaker 02

  Yeah, to me, like, I've never been interested in language. It seemed to me such a, such an advanced capability that, you know, is, is very, the, the entire animal kingdom has very different forms of, of speech and language than what we, you know, and how we communicate with ourselves and, and writing. Uh, and I was always, uh, as many other leaders in AI had this thought that, you know, the natural thing is you have to experience the world, act in it, in vision, in action, that will be the, the key. But then I've, you know, like a, anyone else who's seen the, the chat GPT moment and went to, to do some work at Mosaic and stuff like that, to learn how the sausage is made on the, on the NLP side. And the thing that's striking is that like the, the language should be pretty hard, like each word

• 38:08Speaker 02

  has this, uh, one hot embedding vector that's as dissimilar to any other word, uh, than it is, you know, uh, to, you know, it's, it's a completely high dimensional space. And it's really artificial in a sense, and we, we learn it with models that are order of magnitude bigger than the best vision models. And still, you know, things work pretty well. I do think there's a lot of juice to be squeezed in, in, in image and video, and I think you guys doing are good, good investments in this space. But it's, uh, I think the two would, would keep being interesting in different ways.

• 38:41Speaker 03

  I mean, now I'll tell you my, that was my lead up. Now I'm gonna tell you the crackpot theory. I, like, and this, this podcast is not for me to pontificate, it's for you guys. But this is something I've been thinking a lot about, and I just, you're the right people to share this with. I, I was pretty shocked that language kind of surpassed vision and I underestimated what was happening with LLMs in like 2018, 2019, 2020, because I just had this bias towards vision. And when I look back on it now, like, I think what was basically happening is that in biology, like, vision has a massive fundamental advantage over language in biology.

• 39:23Speaker 03

  And maybe I'm wrong, but basically like the bitrate that your brain can process.

• 39:31Speaker 03

  Optical data through the eye is, and this is my, I'm not a biologist, this is just kind of my dumb assessment, seems many orders of magnitude greater. And there's a lot of like optical processing that happens, like even before you reach, you know, like electrons. And so it's just like the total bitrate that is of training data that's kind of being processed and then making it to your brain seems many orders of magnitude greater than the audio data where, you know, it's sound waves where sound waves are fundamentally like much slower bitrate than light.

• 40:10Speaker 00

  Yeah.

• 40:11Speaker 03

  And then there's almost like, like an upscaling from the acoustics to electronics, which make it into your brain, whereas like, it's like a downscaling from photons to electrons with vision. Whereas in computers today, everything is electronic. So it's kind of like you nerfed vision and you like promoted language where the, it's like all processing is on the same playing field. It's all electronic. And I just, I think this might, this is like my crazy ass dumb non-technical crackpot theory, but I think this might be part of why just like from an information theory perspective that

• 40:52Speaker 03

  like maybe language and vision are on a similar playing field. By the time you get to like L L Ms and then L L Ms are, were just a really, really smart architecture that's better suited for language than for vision.

• 41:08Speaker 03

  How dumb this is, especially to you, Don, the neuroscientist.

• 41:11Speaker 02

  Jesse also has some background in cognitive computational science, right? So I would say my, my point here is like, look, much of what we're doing in knowledge work, we haven't evolved to do. Right? We're sitting on these computers, reading these things, writing these memos, whatever. We are not evolved to do this. It's new to us. Our brains are not wired for this. Still, nevertheless, it's useful to have L L Ms to do this for us. And you know, as humans, we're heavily vision biased, you know, other rodents are more olfactory biased and I've worked on. These things myself before. So what's the uh real estate in the brain that's allocated to vision? And you know, exhibit lobes versus like language areas, temporal lobes, probably more vision. I'll have to check with chat GPT, but I think that's the situation.

• 41:56Speaker 02

  You don't know from memory? No, man, I'm externalizing. I'm a big rag believer in my personal lifestyle. I think. In the limit, we're all, it's all rag. I internalize just, you know, important things like my emotions to you. I was just kidding.

• 42:15Speaker 02

  Anyways, um, yeah, and vision is dominating. When people are training vision language models, they end up. Language ends up dominating the vision content there. Um, but yeah, it's, it's hard to say that because a certain brain is more, you know, biased towards a certain modality doesn't mean necessarily that we're going to more efficiently do it. I do think that efforts on like brain computer interfaces should take this into account. How do you then relay it back to the brain? That's where I think it's very important to think like, what real estate do we have there right now? But for knowledge work, it's equally fine if it's text, I think.

• 42:46Speaker 01

  Last question. If, if everything was right, what does the world look like in five, 10 years? And then what is Engram's role in it?

• 42:52Speaker 00

  I think I'm imagining like a world where everyone has their own model um that is really different from the other person's model and from the frontier model and all of these kind of serve different purposes. And to have a model that really, you know, I think people often talk about like knowing, knowing you, um but also like kind of like helping you in the ways that makes sense to you um personally. Whether it's like an individual or a team, I think there's an element of like having different kinds of intelligence everywhere.

• 43:24Speaker 02

  Yeah, and to me, actually, it's, it's a variant of the story where like, you know, in neuroscience, we know that memory and navigation are pretty closely related, same circuits in the brain that, you know, present landmarks in space are in charge of some, you know, elements of episodic memory and things like this. And for me, I think the company can be. You know, the actual LLM interface to the data plane for everyone. So sharing some similarities to great companies like, you know, Databricks and Oracle, where, you know, we form these memories that happen to be neural memories with models that happen to be personalized and happens to be there's hundreds of millions of them. But they're basically a neural interface to the data plane in a way that's, that's very different from what we know and it's more efficient, it's more associative.

• 44:07Speaker 02

  It's not representing the file system as it is, it's representing a brain state of that file system. So that's for me a vision.

• 44:14Speaker 01

  Beautiful vision to end on. Thank you guys so much for coming by to share what you're building.

• 44:19Speaker 02

  Awesome.

• 44:19Speaker 03

  Love it.

• 44:20Speaker 02

  Thank you guys.