The Benchmark With No Instructions — ARC-AGI-3 (winning team!)

Tim Scarfe travels to Zurich to sit down with the Tufa Labs ARC-AGI-3 team — founder Benjamin Crouzier, with Jeroen Cottaar, Dries Smit, Stefano Viel and Michal Tesnar — to work out what their leaderboard-topping system does and what the benchmark is really testing. The cut opens on the games: a walkthrough of the Locksmith game, where you read the rules of an unfamiliar world straight from raw frames. ARC-AGI-3 makes ARC interactive and agentic, so the model has to *discover* the goal rather than transduce a static grid. It stays easy for humans and breaks LLMs, and it runs through everything that follows. Dries traces his StochasticGoose preview win — brute force that only searched actions which changed the frame — and why it collapsed once the organisers added action-efficiency scoring and unseen games. Induction and transduction run through the middle of the conversation — how much of an answer is really priors leaking back the moment a model recognises a maze. The abstraction mountain, and Tim's case that LLMs reach the right answer through fractured, entangled representations — performance, not competence. Whether transformers plan at all or just fake it well enough. Why the score really measures action efficiency, not games solved, and why agents lock onto the wrong goal and cannot climb back out. Crouzier closes on the Tufa Labs thesis — a small lab against the giants, the bitter lesson against hand-built harnesses, and safety — and Tim ties it back to Kenneth Stanley, deep constraints, and creativity as competence. Disclosure: Tufa Labs sponsors MLST. ---TIMESTAMPS:00:00:00 Meet the Tufa team and what makes ARC-AGI-3 hard00:02:11 Locksmith game: reading the rules from raw frames00:03:10 Why build an independent research lab00:04:11 StochasticGoose: a preview win, then the hardened games00:07:58 Induction, transduction, and priors inside LLMs00:10:31 Curiosity, world models, and exploring by frame change00:14:32 Understanding debt and losing sight of your own code00:15:53 Requirements-based agents and human-AI co-creativity00:19:22 Why auto-research misses the big picture00:21:54 The abstraction mountain and fractured representations00:27:36 Constraints and making LLMs act as if they understand00:34:51 Human difficulty calibration, esports priors, and emergence00:41:35 Agency, goal acquisition, and two kinds of planning00:47:31 Harnesses, the 36% number, and wrong-goal loops00:52:33 Rewards, goals, and why ARC-AGI-3 resists brute force01:00:46 Would solving ARC-AGI-3 prove AGI?01:07:53 Stripping language away, then priors leak back01:14:06 Representation and whether language is necessary01:18:04 The bitter lesson versus specialised harnesses01:22:20 Capability research, safety, and the software singularity---REFERENCES:organization:[00:02:11] ARC-AGI-3https://arcprize.org/arc-agi/3[00:03:10] Tufa Labshttps://tufalabs.ai/team/[00:04:20] ARC-AGI-3 Preview Agent Competitionhttps://arcprize.org/competitions/arc-agi-3-preview-agentstool:[00:04:55] StochasticGoose ARC-AGI-3 solutionhttps://github.com/DriesSmit/ARC3-solution[00:07:42] ArcGenticahttps://github.com/symbolica-ai/arcgentica[00:07:49] RGB-Agenthttps://github.com/alexisfox7/RGB-Agent[00:14:38] Claude Codehttps://www.anthropic.com/claude-code[01:03:42] Qwen 3.6 27Bhttps://huggingface.co/Qwen/Qwen3.6-27Bpaper:[00:13:03] On the Measure of Intelligencehttps://arxiv.org/abs/1911.01547[00:27:42] DreamCoderhttps://arxiv.org/abs/2006.08381[00:43:55] On the Biology of a Large Language Modelhttps://transformer-circuits.pub/2025/attribution-graphs/biology.html[01:18:46] ImageNet Classification with Deep CNNs (AlexNet)https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdfother:[01:18:16] The Bitter Lessonhttp://www.incompleteideas.net/IncIdeas/BitterLesson.html---https://app.rescript.info/share/463d7f031349b4b9db428553eed88230 https://app.rescript.info/share/463d7f031349b4b9db428553eed88230?t=2795

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 0:31Speaker 01

  Low energy, unfocused, foggy? You might be dehydrated. Whether it's hot yoga, over 100 degree weather, or too much sun. Gator Light, with a specialized blend of five electrolytes, was scientifically designed to help move fluids into the body faster for rapid rehydration. Shop now at retailers nationwide. Gator Light hydrates faster than water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 0:59Tim Scarfe?

  But we have the benefit of millions of years of evolution, right? So it's almost a bit unfair that we're expecting AI algorithms to be able to do all of that.

• 1:08Speaker 03

  Exactly. I think it boils down to one of the big open questions in the field, like is language critical to intelligence? I will say, let's say, at least when I'm playing the games myself, and I think that goes for all of us, I'm using language. My name is Jerome Cotard. I have a background in physics and mathematics, and I've been working in the industry for about 15 years, with the last five years specializing more towards AI.

• 1:30Speaker 04

  It was immediate to him what the goal was. He recognized the pattern, and I showed him the game. Within three seconds, he finished the first level.

• 1:38Speaker 05

  Often the agents start thinking that reducing the energy bar to the minimum is the goal, or that stepping 10 times in a region is the goal, which for a human is kind of clear that it's not the actual goal. My name is Stefano. I study computer science, machine learning. I did some research in reinforcement learning at EPFL.

• 1:58Tim Scarfe?

  I guess the million dollar question though is do you think it's possible in principle to do really well on R-CGI3 and be no closer to EGI?

• 2:07Speaker 06

  I think it is possible. I don't know, I guess the main idea is a bunch of bright people in the room and do good research together.

• 2:17Speaker 03

  The transformers can't plan, but they they can do a very good job of pretending essentially. That is, in a sense, indistinguishable.

• 2:24Speaker 04

  Thirty-six percent might be misleading as a number if you don't look behind it. So what it really measures is action efficiency.

• 2:31Speaker 07

  My name is Dris Schmidt. I studied electrical and electronic engineering. I specialized in reinforcement learning for the past five or six years now. And I've been doing on the research side ARC AGI two and three and also some quant work in between. Yes, perhaps that tells something about the benchmark. There's some leakage of human priors into the games.

• 2:52Speaker 04

  Hello, I'm Michael. I am my I do masters in data science and I'm super excited about ARC. And I love solving it. In the example that we saw, we have a maze, right? There's no native primitive in a newborn that says this is a maze.

• 3:10Speaker 07

  Okay, so this is one of the official ARC AGI games. This is one of the first games that was released. It's called LS20. And here you can see a sixty-four by sixty-four frame, which is typically how the game observations is presented to the agent. So you get an initial frame and each of these pixels can take one of sixteen colors. And now you need to figure out what the goal is and even how the player moves, whether it's a specific player or object that's moving around or some other more abstract game. You need to figure this out.

• 3:42Speaker 03

  So very often our harnesses will break here. So they actually got the idea right. You need to move to the exit with the two things aligned. But then, but they they think they should win here, but actually you need to move up one more move. But it is quite common actually that they fail to beat the level from this point. Because they cannot think that you should, they think, no, but that's outside the maze, so you can't move there. And the human would just, whatever, just give it a go and see what happens.

• 4:06Speaker 07

  Yeah.

• 4:07Speaker 03

  But somehow we can't get our harnesses to also do that.

• 4:10Speaker 06

  The best recipe we have today to build intelligent systems is scaling up these large language models. I think Sam said you should definitely not be trying to to train LLMs yourself. Nah, I don't recommend it. Why not? I think the easiest is just go work at Anthropic.

• 4:28Speaker 06

  That's the easiest. I have a sense, I cannot really think that this is the time things start to get weird. I think he's right about that. Most fundamentally is I want to understand intelligence, how the brain works and how to build intelligent machines. It's a big mystery. I guess when I was younger, I was excited about both computers and brains. Computers, I feel now we have a clue how it works. And I did computer science studies, so that mystery is kind of gone, but the mystery about brains is very much alive. I have a strong sense that like most of societies, bottlenecked on cognitive label and insight and basically intelligence. And so if you have more intelligence, that would be a big unlock for many things.

• 5:11Tim Scarfe?

  You won the preview competition and I was using a different method.

• 5:14Speaker 07

  Yes.

• 5:14Tim Scarfe?

  And then you found that that didn't work very well when the main competition came out and you moved towards this.

• 5:19Speaker 07

  Yes, so for the agent preview competition last year, I actually tried something completely different. So the goal of that competition was basically to test whether there is obvious solutions that breaks the mold, breaks what they're actually trying to accomplish. And there was, and the solution was basically just to brute force actions. And so what that solution did was it basically searches through the large action space. For R K G R three, we have more than four thousand actions, which makes this difficult problem to do. But you could theoretically search through all that or just randomly search through actions. What I did for stochastic goose was basically to search through a large number of possibilities, but I tried to only search through actions and results in a frame change.

• 6:04Speaker 07

  So initially, if you took an action that did not do anything in the game, nothing would change, not even the timer bar. And that allows you to to model that behavior to see, okay, does this action change the time bar? If it does not, we downvote that action in the future for that given frame. And there was also only three games, so you have to be selective of how you improve things. You can't pre-train because you'll just overfit. Exactly what I did was I just used a action model that learns which frames are valid for a given transition. And then it was more about the engineering of being able to learn within hundred thousand actions, because that was kind of the max action limit we could get out in the time we had. So that seemed to have worked well.

• 6:47Speaker 07

  It got, it solved, I think it got eighteen levels completed out of the three games, which was, I think it solved two of the games as well in the time limit that was provided to us. But they then hardened the competition specifically against that. So the new competition, the games are much harder and the timer bar moves if you, even if you use an action that's valid, but doesn't really change anything in the game. And more importantly, they introduced action efficiency. So this makes it, yeah, very difficult to brute force. You have to be very direct with how you explore. And that's where L L M's come in. So you could do, even though it's slower, just being able to somehow guide the exploration helps a lot.

• 7:31Speaker 07

  It's like the possible combinations is too many to just brute force. And obviously your score goes to zero quite quickly. So they, they, they've made it so that if you go just, let's say, two X or three X above the human baseline, you're already close to zero. And yeah, that's, even though it's slow, it just, it just helps. It's, it's, it's always kind of untractable for us. We tried other methods such as directly predicting, like more of a transductive method. We just have the input frames as context into a long sequence model and you just predict the actions. But that doesn't also, it doesn't generalize well and it doesn't really make intuitive sense a lot because if you, you play the game, for example, the first game here, the maze level, you would intuitively play one or two actions and then you'll think about your

• 8:16Speaker 07

  path and you'll go to the end position. But if you have to think at every step with the same computational capability or same budget, then yeah, you might be misrepresenting where you should go at the start, but at the end, for straight lines, for example, you don't have to think that much. You can just batch those actions. So that's yeah, where the, the coding agent idea came from. And we also had a lot of two good literature results where they scaled using OPUS models. One was ArcGentica and the other one was the RGB agent, which also showed good results. Given no complete constraints, you could use closed models. And we took that as inspiration.

• 8:57Tim Scarfe?

  Yes. You mentioned transductive as well, which is quite interesting because, you know, roughly speaking, I think of transduction as you're making a prediction about the specific test instance. And it's quite an interesting discussion whether or not this is transduction. Because even though it's chain of thought, it it looks like a form of induction in the sense that it's a rationale that could be cross-applied in the future. So you could use the memory in the agent, you you could do some kind of library transfer and make it inductive. But at the moment, if if it's only for the sole purpose of this particular problem, would you call it a transductive method?

• 9:31Speaker 07

  So I would call our R K G R two solution more transductive and this slightly more inductive. Exactly as you mentioned, you can actually read the reasoning trace and understand when it's understanding the game and making progress and when it's not. Um, yeah, because it has this reasoning chain of thought, which is in English and you could reasonably understand it. So, uh, yeah, I would actually say it's more more inductive. Like previous attempts, uh, as we mentioned, was where the agent actually just directly predicts the actions. Um, that will be more transductive and that doesn't seem to work at the moment. Uh, we, obviously, there's a lot of ideas to try. I'm sure the community will come up with something interesting to make that work. But for now, this this seems to be the way for us.

• 10:13Speaker 04

  Yeah, I think the action efficiency make this makes this problem really interesting. Um, because there's so little time to learn and you really, if you want to get the 100%, which is the which is the grand prize, then you really, you you have to outcompete a human, which really can take a lot of time just to think. And time is not a in question for the solutions. So I think that's that's really interesting and the reflections and looking back at the past, uh, at your past experiences makes makes L L M's very flexible, uh, model for this, uh, for the solution. Um, yeah, and interestingly enough, also a lot of, even though there shouldn't be any priors, there's already a lot of the game priors are already included in the L L Ms.

• 10:58Speaker 04

  So for example, here we're looking at the maze. Um, and maze is something that every L L M, even the small ones, uh, will know and will recognize from beat images or ASCII grids. So even though there are not, uh, all the priors are stripped away, there's still enough like game priors that the L L M can lean on. And those are not encoded in any reinforcement learning model to, or like a pure, um, uh, goose solution wouldn't have that encoded anywhere in it. That the maze is a thing, uh, and that really helps to direct the reasoning and the actions of the model.

• 11:31Speaker 07

  So the, one of the main constraints was basically two weeks. I joined the competition late, uh, two weeks to, to do it. So there's a lot of things that can be improved, but I think it was a good initial solution. So the main constraint was we only had three games, three public games and three private games to evaluate on. So we had to make sure, uh, that to make sure that whatever your design is, is, is, you don't assume too much about the, the environments and the games because it, uh, on the private leaderboard or like the public leaderboard beforehand, you could actually see that I wasn't even close to the top because it was super easy to overfit. Um, so the basic idea was, um, we can't pre-train. Uh, we don't have enough environments. Let's try to do test time training like we did for R K G I two. But how do you do that?

• 12:14Speaker 07

  Like you can try reinforcement learning, which I have a lot of experience with. But, um, it's, it's not that obvious how you do reinforcement learning. Because every time, so you get a new game at test time, you have, only real reward you have is a level transition. But once you transition a level, you never go back. So it's not like you're, you're trying various, uh, uh, routes to try and optimize for the passing level. You just have to pass it once. So it didn't really make sense to use pure, um, reward based R L. Um, I did a lot of research in the past in curiosity of unsupervised R L. And I thought perhaps that's a better approach where we don't have explicit rewards. We're just optimizing for, um, yeah, curiosity, like exploring new areas that the agent hasn't explored before. Um, so I tried a bit of like world modeling.

• 12:58Speaker 07

  So the basic idea is the game's not deterministic. So you can use a world model to perhaps have a, um, the input frame, then the action and try predict the, the next frame. And then if you can't predict it well, you have the policy explore that more and that's the reward you're optimizing. So that was a promising approach. I couldn't get in it working in two weeks. So what I just then assumed is that any frame change is interesting. You want to explore things where the state changes. You don't want to explore things where the state stays the same. So that was the basic derivation of the, or like a path I followed towards getting the action model implementation for stochastic goose. And then it was down to basic reinforcement learning things like how do you get it to learn within a thousand time steps?

• 13:41Speaker 07

  Because you have a hundred thousand times of, you need to start taking useful actions in like a thousand time steps. So it was like using a replay buffer hashing the experience so it doesn't go over the memory a little bit. You have to do some prioritized experience replay and some basic engineering stuff to get work of that competition.

• 14:00Tim Scarfe?

  Yeah, you mentioned exploration as well. I guess there's a bit of an elephant in the room, which is that Sholay is talking about the acquisition and synthesis of abstractions. And when reinforcement learning folks talk about exploration, it seems to be in quite a surface superficial way. So in terms of like entropy or things changing. And do you think, is that in any way against this idea that we can acquire deep abstractions about the domain?

• 14:25Speaker 07

  Yes, definitely for stochastic goose. It was not at all what I guess what Franca at the end wants as a good solution for Arcadia three. But I can't speak to what he actually thinks is a good solution, but that was purely to see if I can benchmark maximize this competition within two weeks. I think the duck and things we are doing on that side is perhaps more in line with what that initial vision is. For example, you have this reasoning tokens, which has like abstract representation of objects, which we then manifest or write down in like an English language textual tokens.

• 15:04Speaker 07

  And so I would say that's more abstract, identify objects, you try and find out what the mechanics is, dynamics of the game, what the goal is. That might be more abstract and it's also kind of like a neural guided search. And also the programs, the Python programs it creates is like executable Python programs to extract. And also build a some sort of a simplified world model and search over that world model using actual like algorithms like Biff's search.

• 15:31Tim Scarfe?

  There is understanding debt, right, that you're that you're building this really, really complex thing. And after a while, have you noticed in Claude code, they don't even show you the code anymore? Right, you can expand it, but by default, a lot of people don't even look at the code. So do you think that you need to be familiar with the deep abstractions in your code base in order to kind of build mental models and evolve it and extend it, you know, do you get lost in no man's land?

• 15:55Speaker 07

  Yes, this is a, yeah, if you want, this is an active discussion within our team. We don't have a discrete answer yet, but I think like what we've found is like you need deep understanding of some important parts, some other parts, like let's say a web viewer. You can vibe code more easily because that's not really, if it breaks, it's it's fine. But like the core logic for like they say, the implementation and also evaluation is especially important. Like we have to make sure it's working, but it is a case that we're gradually, we're understanding less and less of our own code base. And we're struggling even with reviewing some of the changes is you you might use a codex to help review some of it because it's such a broad change.

• 16:38Speaker 07

  Or we need to split it up, but things are moving so fast that you can't just manually. Yeah, if you just manually write everything, you can't keep up with the rest of the team. So, yeah, I guess it's an active discussion, it's difficult.

• 16:53Speaker 03

  As for me, I'm, yeah, my career has been going on a bit longer and let's say most of it, for most of my career, there were no coding agents. Which means I also have been an opportunity to still make use of patterns that have been useful in the past. And I think one important one that as a team we are more and more learning to use properly is requirements based engineering. So we will formally write requirements, let's say really following the, let's say detailed prescriptions like numbered requirements and specify how they are tested and I mean, we might still have the coding agents helping there, but mostly handwritten. We will review that as a team. And from there, the coding agents do, we can much more confidently hand it to a coding agent to implement. Then if it's just a Viper, a single prompt.

• 17:33Tim Scarfe?

  Yeah, I love all of that, but I suppose there's always been this thing, right? That in software engineering, we have specifications, we have documentation, we have user stories, we have requirements and all these things. And there was waterfall versus agile. For me, the difference between waterfall and agile is this admission that there is epistemic uncertainty, right? So we need, you know, like, reality pushes back, we discover contours in reality that we didn't see before, and we need to adapt the requirements over time. And my worry is that when you kind of crystallize the process, I guess what I'm saying is there's more to tacit knowledge in developer teams than the requirements. There's the generative process that created the requirements in the first place. And when you kind of only write software for the purpose of satisfying the requirements,

• 18:18Tim Scarfe?

  then the ability to evolve the requirements gets kind of deteriorated.

• 18:23Speaker 03

  Yeah, so what is always the case, and that was the case in classical, let's say, pre-coding agent development as well as it is now, is that it is not you write requirements and those requirements are frozen in, yeah, carved in stone. And then you write code. It is an interplay. So when you start implementing, you might realize, wait a minute, this combination of requirements is not possible. You don't then just implement something that approximates the requirements. You go back and we also kind of tell our coding agents to do this. You go back and say, hey, these requirements don't work. They need to be adapted. And you make a conscious choice there. So we will also very often kind of go back with the coding agent and hey, can you now very carefully check that every requirement is satisfied? In fact, we have tried specific text per requirement, like this is how it is being satisfied in the code.

• 19:07Speaker 03

  We try to also really write the tests around the requirements rather than just having kind of random tests for every line of code. And then you get kind of that classical interplay that you also have in classical engineering, you can do with your coding agents.

• 19:19Tim Scarfe?

  This is a kind of AI-human co-creativity. So humans have the taste, they have the intention, they have something that the coding agents don't have. Because, you know, the bull case, wouldn't it be amazing if you could stick the requirements in and the agents themselves would kind of understand, oh, I see what they meant, and they could evolve the requirements?

• 19:38Speaker 07

  So I think in the short term, I can't say. In the long term, at least what we're trying to do is find the areas where we can see the model still failing, like high-level conceptual ideas about the problem. And then we try to make sure when they say we're reviewing code or creating code, that we ask specifically around that areas and make sure it's correct and understands us well.

• 19:58Speaker 07

  But yeah, I guess it's always going to be a game of cat and mouse, where we're going to perhaps hand over more to the AI assistants. But in some areas, it might just totally fail and we need to take responsibility there, make sure we understand what it's doing and it understands what we want. But yeah, longer term, I guess if your AGI pulled or super intelligent pull, I guess it's yeah, it's up for debate with.

• 20:22Speaker 04

  So it's certainly accelerating us in doing experiments, because something that would just sit in your scratch pad, you can just paste it in a prompt and see if it sticks. So you can definitely experiment overnight with a new idea. I think the, really the added value that I see of the lab right now is, and what the AIs are missing, what the, what the coding agents are missing right now, is like literally the, the, the attention to detail. So this task is very long, like if you have a long context, the execution of one game might take an hour just for it to solve one level. And just seeing, and, and if you, if you could in, in principle point all the research at this and say, hey, just improve the score, right?

• 21:06Speaker 04

  But that doesn't work because it doesn't really, it cannot really, we have tried that and that has not really led to improvements because it doesn't really understand where the agent fails. It's not a simply as Andrej Karpathy's auto research challenges, where you just minimize a loss and there's just parameters. You need more insight to this task. And I think that's where a lot of improvements came from, not from. Like it it came from reading deep into the program, into the logs, and understanding what the issues are and improving those one by one, rather than just like vibe coding another 2 million lines of code.

• 21:42Speaker 05

  With coding agent trying to optimize this task, we observe kind of similar failure mode that what we observe when they play games. Like for instance, if you give them an auto research task, they might over-focus on details and try kind of not see the big picture and not kind of zoom out and see things from far ball. And they might, I don't know, start optimizing some hyper parameters for like a 0.01% improvement, while there is like some other big improvement that could be done, which is kind of in another abstraction level. And we see very similar failure mode in games where if the agent at the beginning of the gameplay, they start, they get kind of locked in on the wrong hypothesis, it's extremely hard that they escape that.

• 22:27Speaker 05

  Like they they kind of convince themselves that that's the right path and the right hypothesis to go. And they're not able to escape that. So it's quite interesting that, yeah, across models like smaller and frontier one, we observe kind of similar failure mode of them kind of not being able to jump between an abstraction level and not being able to zoom in in the details and then seeing the higher level picture at the same time.

• 22:54Tim Scarfe?

  I just wondered like, how do you guys think about the abstraction mountain? Because for Shalay, it is bottom up. So you start with the core knowledge, you know, like agent spatial knowledge, stuff like that, and and you kind of synthesize upwards. Whereas L L Ms are kind of interesting because they they learn this, these fractured and tangled representations that are quite high level, but also quite generalizing. And then you can stitch them together and you can repurpose them and cannibalize them in different contexts. But it's always a little bit lossy. So if you're doing this library learning and transfer, you always have this issue of like, well, yeah, there's a subtraction over here and it kind of works, but it's not quite what I want, but I can still use it anyway. I mean, is this is this kind of reasoning something that should be first class? Or do you think it could just be implicit in the harness if we had a memory system?

• 23:37Speaker 03

  So you kind of, you were just describing how our L L Ms will have, indeed, maybe, let's say, the core knowledge prior as something somewhat vague, but not fully testable, but still usable.

• 23:50Speaker 03

  The thing is, if you replace LLM by human there, I think it's still true. We also don't have formal core knowledge priors that are, let's say, fully exact and that we apply to the world and describe the world fully exactly. We have kind of vague, trained over our life understandings that evolve over time. So at least if you want human intelligence, that does seem to be the paradigm you want there. And I think, yeah, the LLMs get kind of the bonus, that they also can reason very well and much more implicitly than the humans can encode. So if they want to get exact, they can. But I think if you want to understand the world, certainly beyond these, let's say, simple games, you will need more probably that, yeah, vibe understanding, basically.

• 24:32Tim Scarfe?

  I'm guilty of always, like, you know, suggesting what Sholay thinks. I don't really know what he thinks, but, you know, I've got a pretty good simulation of Sholay in my mind. But I, I, I think he's, he's influenced by this kind of nativism psychology type thing. And, and he thinks that a lot of the reasoning we do do is almost platonistic. You know, that, that, you know, somehow the laws of nature impute these primitives into our minds. And we compose these primitives, primitives together for certain classes of problems. So we can do abstract system two reasoning. And for those types of problem, we do compose these things together. But you're absolutely right, there's so many things in the world are actually really complicated. Right, you know, like navigating relationships or even, you know, pathfinding in a complex environment,

• 25:16Tim Scarfe?

  you know, on the tube network or something like that. So we do a little bit of both, but there's, there's at least a pocket of pure reasoning. I think that's what Sholay thinks.

• 25:23Speaker 07

  Would you consider LLMs pre-trained on the internet? And then if you apply it to a new pattern using like in-context learning, is that a form of, okay, let's say with reasoning as well, would that not be a form of, um, like perhaps skill acquisition on the fly? Because it learns, obviously, there's a lot of core knowledge as well, but it's also a different part where it has to adapt on the fly. So you can give some example that might not be on the internet and might adapt. Sometimes it fails and it is still a distribution. But at least you get some adaptability. And then, for example, when you started prompting it, think of like a chain of thought prompting, it started improving because it had more time to reason and adapt to what you're asking. And then if you train it using reasoning, you can actually do way more

• 26:06Speaker 07

  because now it has more time to actually reason and figure out what you're actually asking and form new extractions for solving a specific problem case as opposed to just regurgitating what it's seen on the internet.

• 26:19Tim Scarfe?

  L L M's trained to do reasoning, they they are intelligent. Because intelligence is just adaptation. So you know, it's it's I know this now and and I need to combine what I know. Several steps, you know, to solve a particular task. So in in the partial knowledge regime where we have a verifiable function and we can do hill climbing. Um. But what we see, though, is that they combine together fractured and tangled representations. So they get the right answer, but for the wrong reasons. So they they create a spaghetti monster. But what we do is is we take um a more valid path and we can acquire and reuse abstractions. The models themselves, they don't seem to work bottom up. They work. They they have an an understanding. They have representations, but they're very fractured and entangled.

• 27:02Tim Scarfe?

  And and that's still useful, but that's a kind of statistical low level intelligence. And I think you could, in principle, still do abstraction from the fractured and tangled representations. But we're kind of in statistical land. We're not. It's performance, not competence.

• 27:14Speaker 07

  I don't have strong views on this, but I'm I have a feeling that we are doing the same with our minds. Like, if you look at the evolutionary process that derived us, the kind of um where we are at at the moment is also we've been evolved to work well with spatial information with like that's one of the the core priors in our brain. And some level of reasoning, um that might also be why intuitively understanding quantum physics is so difficult. It's because we kind of overfit to our our world and how we've been evolved. Um. So, yeah, I wouldn't. I still not sure whether the human brain is also like just this core priors and like pure reasoning. There is some some guided search uh to find answers based on what we found

• 27:55Speaker 07

  that through evolutionary process to actually work well um to yeah to actually survive in the world and um yeah propagate.

• 28:02Tim Scarfe?

  It's the it's the million dollar question, isn't it? You know, because our our brains think symbolically, but our brains are actually just a bunch of neurons. So is it is it a post hoc confabulation or or is it actually like a causal basis function of our cognition? And and and if and if it is just kind of, you know, a confabulation and it's not the basis, then maybe A I doesn't need it to be the basis either. But but but Sholay, I mean, I don't know if you guys want to talk about what Sholay is doing, but he's he's building this. This approach where we actually acquire and synthesize program graphs using neural networks as guidance.

• 28:36Speaker 03

  It's a tricky one because for me, coming mainly, let's say, from a more classical machine learning Bayesian perspective, it feels like this should be the right way to go. And certainly when I learned about these things and kind of very broadly also kind of sweeping stuff like Dream Coder into that regime, although it's not quite the same. Thing is, it definitely feels like it's more natural to think of it in that way and perhaps more natural to the way we reason. At the same time, it is not competitive at this point with LLM-based approaches. Like, who knows, who knows what happens in this competition? But, but we don't see those approaches necessarily being the best on Arc AGI one and Arc AGI two. And we really don't see how we could use them on Arc AGI three.

• 29:18Speaker 03

  That could also simply be because they haven't had the same attention, the same resources put into them. But yeah, there is that mismatch. Why don't they work if they match our intelligence so well?

• 29:27Tim Scarfe?

  And I would even, I'm trying not to use the word symbolic AI in general because I, I think the, the more abstract way to think about this is constraints. And Shaleh's been using the language of constraints and we wrote an article about creativity. And I, I, I think that creation, whether it's intelligence, so in a known domain, or whether it's creativity in a transformative unknown domain, is all about just respecting constraints and acquiring constraints of a problem domain. And this is why harnesses and tools and code work so well because you're increasingly imbuing domain constraints into the, into the learning and acquisition environment. And so, so we find ourselves in this kind of halfway regime where we have a constrained coherent execution environment,

• 30:08Tim Scarfe?

  but it's only really steered and legible in, in the sense of these functional descriptions that we're talking about. So rather than having some low-level bottom-up competence, we have a kind of intermediate competence, which is as described in a similar way to the blind men and the elephant. So from many perspectives, many functional descriptions from many perspectives. And we're using that as the basis of constraint building and acquisition of, of, of knowledge.

• 30:31Speaker 04

  What I find really interesting about the core knowledge priors and what you said, like we have these, these representations and they're very fractured, but. Sometimes somehow they work. I think eh there's two levels of abstraction. So um what what two levels of intelligence that human has. Like there's the core knowledge prize and that then there's what we learn through life. And the the games that have been tested with humans have been tested with humans. That were not infants uh for art challenge. Um and there's a lot more that we accumulate over generations through education and so on. And those abstractions are way more powerful. In the example that we saw, we have a maze, right? There's no native uh primitive in a in a newborn that says this is a maze.

• 31:16Speaker 04

  But this has been over generations of problem solving through humans. Uh very much established concept. And if we can inject this into an L L M, we can skip many levels of abstraction. Um and I would say that's why when we play these games, some of them for the first time, they're also very difficult for humans. Because if this if this abstract, if this is done well and there's nothing like a maze in a game. And it really core boils boils down to the core knowledge priors. Then there's it's very difficult even for a grown up adult to to synthesize the correct rules. However, most often, um and that may be a failure of of uh arc a little bit in its design.

• 31:59Speaker 04

  Is that these games usually have some sort of, oh, you're shooting a ball or you're solving a maze. You're you have an enemy, right? And these these are like as as pure as they're trying to be, they're never going to be pure because they're made by humans. Um and that's why we can do so well with L L M's, I think. Because a lot of these like the maze is the canonical example. Or uh uh I think uh like those can be really well done with L L M's.

• 32:25Tim Scarfe?

  They have these fractured kind of, you know, um fractionated understanding. But isn't it interesting that when humans use L L M's, we can make them act as if they understand? Because we can say, well, um you can put a framing in in the prompt, you can say, think about this problem like it's a maze. Or think about this problem like it's um tic tac toe or something like that. And what you're doing in the prompt is you're basically setting constraints on its generation.

• 32:46Speaker 04

  Yeah.

• 32:47Tim Scarfe?

  And then. You know, like the more you work on a software engineering project, you know, like it doesn't quite understand, you put more constraints and you update the prompts. And it's like it converges. And after a while, you don't need to repeat yourself. It it it understands. And so it's almost as if when you get L L M's in the mood, when you make them track this perspective where it respects the constraints. Then it understands. But like the million dollar question is, how can we just make it understand autonomously? How can we give it a novel domain? And do what we do, which is like saying, ah, this is basically a maze. This is how you need to think about this and get it get it into the understanding.

• 33:24Speaker 05

  Good way to look at this is to look at the function that they learned during training, because like. Uh, L L M becomes very good at understanding languages and operating on anything that is kind of language related. And then all of the work that we do in the harness is trying to um. Put this novel domain into language, so like trying to kind of um. Represent the game or whatever domain you're trying to make the uh L L M act on. In a in a way that is kind of language related, so it's closer to its pre-training prior and it allows to act on it. If you're not able to do that, if since the L L M has this kind of prior from pre-training, it will never uh. It it's way harder to to make it act on game because it's such an out of distribution uh domain compared to what it was trained on.

• 34:09Speaker 03

  Yeah, I also want to comment a bit. So you mentioned indeed that what you often uh. I think this is easiest to to do with coding agents, that your analogy where you kind of have to say, yeah, but you need to. Prompt it a few times before it really gets what you want.

• 34:21Speaker 05

  Yeah.

• 34:21Speaker 03

  Um, but I don't see, let's say, having worked a lot also with with instructing human teams of software engineers, for example. I don't really see a huge difference there in the sense that you cannot expect to just tell someone in two cents, I want something like this.

• 34:36Speaker 03

  And then get the right thing. That's a failure on your part not to specify what you want. And we see indeed with this requirements engineering, requirements based engineering, which takes a lot of thought. We might spend days getting the right requirements. But then it is, I would say reasonably well capable. Of doing the right thing to begin with, basically, but yeah, it's also our job to specify it, right?

• 34:56Tim Scarfe?

  I call it perspectival blindness. So you hire a new person and you build an internal wiki and you specify all of your learnings. And you think, oh, I've distilled this, I've crystallized it, the new starter will just be up to speed within a couple of days. It doesn't work like that. They, you know, there's almost like, experience is this weird tacit, almost like a causal graph. That has a provenance and it's only when the new starter plays with all of your tools and tries different things and has counterfactual like, you know, representations and whatnot. So yeah, it's a very human thing as well. Yeah. I mean, like the general thesis is that, you know, knowledge is quite path dependent. It's not about where you end up. It's not about these abstract functional descriptions.

• 35:37Tim Scarfe?

  Sometimes it is if they're so incredibly idealized like calculus or something, that they're the basis of of evolution on their own, but so much tacit knowledge actually requires the path of how you got there in order for you to continue the path.

• 35:50Speaker 04

  Yeah, I think it's really, especially in ARC challenge, I think it's really interesting to consider and would be really nice to see what the, when they were testing the game, like when the humans failed, how did they fail? Cause at first the scoring was based on the second best human attempt for efficiency. And that is, that is very skewed towards

• 36:11Speaker 04

  towards like, towards a good solution, right? And then it was changed to a median, I think.

• 36:17Speaker 07

  It was second best per level as well. So it was almost impossible even for the best human to solve a game. Yeah.

• 36:23Speaker 04

  Yeah. So it was, so it was based on every level. And it was really difficult for a single, even a single human would like educated human would not score a hundred percent on this benchmark, right? So, and we don't know much what the failures are. So we have played these games. We have tested them ourselves, let's say, and sometimes we don't get them. So it's not.

• 36:52Speaker 04

  Like I I struggled with some of them, eventually I solved them, but uh sometimes I would spend like on some of them, I would spend an hour to solve them. So you cannot really say this is something that's immediate. Uh it's also difficult for humans and I think it's really uh like looking at these counterfactuals like what do human engineers miss versus software engineers, as you're run just said, or like what what actually humans fail on on these tasks rather than oh, human can solve it if you give it ten attempts or something.

• 37:21Tim Scarfe?

  Really interesting point, you know, because so this this is the um the the human difficulty calibration and on R V two, for example, you know, the the test was let in if if two, any two humans could pass the task.

• 37:31Speaker 04

  Yeah.

• 37:31Tim Scarfe?

  And this this is a bit of an ideological thing, because I think Chollet believes in this notion of general intelligence.

• 37:38Speaker 04

  Hmm.

• 37:38Tim Scarfe?

  So in his mind, someone with general intelligence, because they have the core knowledge, they could, you know, inside, let's say, a certain number of steps in the combination or closure of these core knowledge primitives, any human with a certain amount of general intelligence could solve any of these problems. The counter to that is that, as we've just been describing, maybe human intelligence is quite high level and fractionated and pattern based and it's quite specialized. So some humans can solve certain tasks because they have a certain perspective, they have certain experiences and and whatnot, and and they can just that they can combine those fractured representations together to come to an answer, but maybe they're just in possession of those representations and another human isn't.

• 38:16Speaker 04

  Uh I have a friend that uh is a professional esports player. I showed him one of the games and he completed the first level without spending an extra move that he didn't have to. It was immediate to him what the goal was. He recognized the pattern and I showed him the game within three seconds, he finished the first level with sub uh uh superhuman performance, let's say. And okay, he has been playing professionally games for five years, right, but and he's been trained on this, but like exactly as you say, like there's something we can bootstrap on and we do bootstrap on and it does work. So maybe replicating that with L L M's. Is enough.

• 38:57Tim Scarfe?

  Yeah, I think gaming is a really good example. A good friend of mine is called DDK and he does commentary on Counter-Strike and stuff like that. And it's when I when I watch him do commentary, it seems super situational. So like in Quake 3, you know, there's you've got to time the mega health and then there's the red armour and then there's a there's a position over here. And what you get is like the emergence of these complex situational phenomena in the game. And this is, this doesn't seem anything like what Sholay is talking about, you know, because there's two worlds, right? There's the sort of the emergent complexity world and then there's the the sort of reductionist, you know, kind of core knowledge world. I think in this world, we can still use intelligence and we can still like acquire abstractions and descriptions for these high-level phenomena.

• 39:38Tim Scarfe?

  But but what do we do? Do we analogize them in terms of low-level knowledge we already have or are they something new? It seems like a different modality.

• 39:47Speaker 04

  No, I don't really have a good answer to that. I just only I think I can base my answer on these examples and I completely agree with you that these that there are some abstractions like that, but I don't know. I don't know.

• 39:58Tim Scarfe?

  Like, but you know, like Conway's Game of Life. It is still path-dependent, right? You can still trace a path from the low-level description to the high-level phenomena. But the reason it's unintelligible and it's surprising is because the path is a long one. And it's computationally irreducible, so you have you have to perform every single intermediate computation computation to get there. And and we and we just, it's confusing for us because we can't actually analyze or understand the path between the causal stuff that made this high-level phenomena. So we just kind of see it on its own, like disconnected. And many phenomena in the world are like that.

• 40:32Speaker 04

  Yeah, some sort of like emergence you're talking about, like where where just you have to call it emergent because you just can't trace. You're not capable of like the computational graph of that is just not tractable for you to.

• 40:44Tim Scarfe?

  That's true, but another feature of emergence is is compression. So like with with the glider, for example, we almost don't need to know how it came about because the new description encompasses, it's a complete description of the new phenomena. So it's almost like it's the start of a new piece of knowledge. And and a lot of our knowledge is like that. It's such a good compression of what went underneath that we can start from it. So like, for example, if we could do like calculus, you know, for example, so calculus is within the closure of calculus or probability theory. We can do a lot of things. We don't need to know how it came about. So but then is that a new layer of knowledge? And we don't care how it came about. It's just kind of like saying, do we need to start from the bottom?

• 41:26Tim Scarfe?

  Or in many cases, you know, this has come about and we can just use it, you know, higher up the abstraction mountain.

• 41:33Speaker 04

  Yeah, I think ARC is an example for the case that you can do this, at least to some level, because we see the frontier LLM scoring quite well on these, and they wouldn't be able to if these core priors would be breaking it. But I think what is really nice about ARC is that there are some games that are so different and are still easy for humans, but they somehow like break this concept, right? You talk about disclosure of like, and I gave the example of maze. But like you can maybe change the core prior concepts and they would still make sense for a human, but they would somehow break adversarially this, this like, let's say, calculus like maze like representation that helps solve the game. You can like move them around a little bit, then you suddenly break it and you still

• 42:16Speaker 04

  have a valid game for a human, but not for the abstracted LLM intelligence. And that's what we see also with the, with some of the difficult games. But I think it's really difficult to not move the priors too much so that you can still make it human solvable and that sometimes ARC unfortunately fails at this.

• 42:34Tim Scarfe?

  I want to talk a little bit about R K G I three. So the, the first two versions of, of the ARC challenge, they were quite abstract because Shale was trying to idealize intelligence on its own in the most abstract kind of legible way. Now, R K G I three, it introduces, in my opinion, and we can talk about this, the concept of agency. So agency, in my, in my definition, is the ability of an agent to have goals, to plan and realize those goals. And the more ambitious the goals are, and the more you realize the goals, the more agency you have. So there's a kind of low level, no nonsense definition of agency, which is just a thing that can sense and act.

• 43:17Tim Scarfe?

  So that's like if I'm doing computer programming, that's what an agent is. But the more kind of cognitive science definition of an agent is one that sort of has future point in control. I have these big goals in the future and I can realize them and that makes me an agent. So I, you know, I, I think that R K G I three introduces agency, not just in terms of realizing goals, but acquiring goals.

• 43:39Speaker 04

  Um. Yes, agency and also like acquiring these goals over time through interaction and adapting them. Because not only do uh do the levels on their own to be solved require interaction, but all the levels change. So these goals dynamically change and you so that you why why you cannot uh learn them uh learn them directly. Um. And then on the on the concept of agency, so I would I would argue that one of the interesting things that we saw is that what I what I thought coming into this challenge and started working on this problem is that would be difficult for the agent to have any sort of idea what it should be doing at all when it when it solves the when it tries to solve the first level.

• 44:22Speaker 04

  But I think by now we got at least one solve on all the public games, at least the first level, made it be by trial and error, or maybe it be made it be by luck. But still, there is some somehow L L M's seem to be able to figure out what they need to do. There might be through the different biases that are already encoded for humans that have been building these games. But somehow it is possible to generate these hypotheses. They may not stand for long. They might be kind of random day. Uh. But but there are signs of life that L L M's can already do this, I would argue.

• 44:59Tim Scarfe?

  Certainly, like there's that anthropic biology paper and and and they kind of, you know, talk about all the capabilities of language models. But one one thing that they said language models can do is plan. And planning is required for agency, in my opinion. And we know that language models can't plan because pl- planning is a type of computer science algorithm that is not possible in the in the substrate of transformers. So what they do is a kind of simulated planning, right? So that they they can simulate planning up to a certain depth. Do you think that matters?

• 45:29Speaker 03

  Maybe one important thing here is to realize that there's two types of planning that are involved in in playing an arcade. So there's a um you need to at some point when you understand the rules and you understand what you need to do to win, you need to plan a path to get there. And that's kind of more of a traditional game theory planning algorithm. So you might use a search or Monte Carlo algorithm or one of many such things, which the model is perfectly capable of because it can write Python code and run the Python code. That's one level of planning, but the other level of planning, when you enter just a level and you see it for the first time, is you need to figure out, how am I going to figure out the rules here? Oh, that looks like it might be a player. Let me try to move in a direction. Ah, okay, pushing up, moves it up. And you need to balance that exploring and exploiting, for example, when do I start trying to win?

• 46:14Speaker 03

  When do I start trying to, um, yeah, how long do I start playing around, basically? That's a whole different level of planning, which you can't encapsulate in Python code. And let's say, we see our agents actually, I think it's fair to say, managing that quite effectively. So in a sense, yes, maybe transformers can't plan, but they can do a very good job of pretending essentially. That is, in a sense, indistinguishable from, yeah, what might be formal planning in a computer science sense.

• 46:44Tim Scarfe?

  And I had been quite incorporated that. You're absolutely right. So even if they, intrinsically, they can't explicitly plan, they can call tools. They can write code that can plan. And there's a bit of a galaxy brain side discussion there about whether they can effectively explore the space of programs to do planning, or whether it's enough that they can statistically generate programs that do enough planning for the circumstances they're used in. Maybe that's too galaxy brain for this, but they can basically plan in essence, that's what you're saying.

• 47:12Speaker 03

  Yeah, and so what it comes down to again, whatever is, let's say, under the, let's say, galaxy brain understanding, you need to understand what's going on. They seem to be able to do it to a degree that lets them play these games essentially, whatever it is they're doing and whatever it really means.

• 47:26Tim Scarfe?

  Okay, well, in which case, where's the gap then? So if they can plan, is it a representation problem? What's the big gap?

• 47:34Speaker 03

  So maybe it's first good to discuss what the actual gap is, because indeed, if we see, for example, what Ark is publicly communicating, which is from Shulei, for example, they show simply the benchmark where even frontier models are not able to play the games to any significant degree. So they score under 1%, which either means they're not beating any levels or they're being very inefficient about it, doing way too many actions. Um. However, if you look at publications, you can see that score can go up to, it's the best now thirty five percent, thirty six percent, if you give them kind of a a proper harness. So the point is, they need some guidance. They need to, they they, you need to tell them more like, okay, you're in a game, you're going to do this, you can write codes, you you. But with that guidance, they can actually do pretty well. And and that guidance, it it's not game

• 48:18Speaker 03

  specific. It's not like, oh, maybe this is amazing. You need to think about this. No, it still needs to, the model still needs to do that explore versus exploit balance. The model still needs to do all the move planning. But it gets some kind of guidance on the general thinking patterns that might get it there.

• 48:31Tim Scarfe?

  What we know is that these harnesses, we we can, with a little bit of work, make them do anything. It's a little bit like, for example, if I already know how to solve a problem, I can use GPT three point five and it'll give me the right answer because I'm giving it guidance. So it's a little bit like a heuristic search problem, where if you guide the search enough, if you place enough constraints in there, then statistically, like if you sample enough times, it's going to get the right answer.

• 48:53Speaker 03

  Yeah. So I think that very much is Cholé's look at this. So that's also why they don't, that does cause a one under one percent because they don't allow harnesses for that leaderboard. Because his point is, yeah, you shouldn't need a harness. A human doesn't need a harness. It's a tricky one. I, for myself, don't know if I really fully buy that argument in the sense that, like, the reason a human can do this, uh, without needing a specific harness, I mean, that's fine, but most of the people, especially the people who are capable of playing these games, have played a lot of games, be them computer game, video games, or games in real life. So they've already learned the thinking patterns that we are now encoding through a harness, uh, much more directly. I could also imagine, if if that hypothesis is true, that it works that way, that if you were to take those one percent scoring things, but you do kind of

• 49:34Speaker 03

  a a reinforcement learning loop, specifically to teach them gameplay, but without hard coding any harness, they might actually do a lot better. And does that count? Does that mean they can then play it? Well, I don't know, but.

• 49:47Tim Scarfe?

  Is is that the same as arguing that if you went to a deserted island where there was some tribes folk that have never used computers before, do you think they would be able to do R K G I three?

• 49:58Speaker 03

  I think not immediately, but like if you give them a hundred of these things and you give them one per day and you reward the people doing well, I would assume at some point they can. And I do think it's fair to say they probably need a lot less training resources in this amount of, I don't know, amount of compute required to learn it than right now a frontier model would use. And even when we say they can reach thirty-six percent, that is true, but it costs like a few thousand dollars. Um, which is a lot more, otherwise hard to, of course, a lot more compute than the human beating these games is spending. So yeah, does that really count as the old models can play games now if it takes them so much effort? I would say that is a real gap. And also, as I say, being able to do frontier models, of no, sorry, with small open source models, would demonstrate a lot more.

• 50:42Speaker 03

  It's indeed what we're doing in the Kaggle competition.

• 50:44Speaker 04

  Thirty-six percent might be misleading as a number if you don't look behind it. So what it really measures is action efficiency. So, uh, correct me if I'm wrong, but it's the ratio of the, of the human baseline divided by the number of actions. Uh, the AI has taken whatever model it is on the level or a human, whatever, whoever the player is, and then squared as well. So this plays really adversely to anything that's a little bit action inefficient. So thirty-six percent, actually in this case, doesn't mean we solve, that approach solves thirty-six percent of the games. It solves way more of the games out from the training set, at least this number is on. Uh, but it just solves them inefficiently.

• 51:28Speaker 04

  So I think this is really important to emphasize that like the current frontier models are able to solve like, what is it, something like half of, two-thirds of the training games actually till the end, but just not as efficiently.

• 51:40Tim Scarfe?

  What is the hardest thing in Arc AGI three? Is it the goal acquisition or is it just simply the action efficiency?

• 51:46Speaker 04

  From what we see on the training set, so the, the testing set, uh, the private set is set to be harder. We don't know anything about it. No, nobody outside of the organization has, their organization has seen it.

• 52:02Speaker 04

  So we don't know how hard it is, but at least on the training games, we see that the LLMs can acquire the correct goals and pursue them somewhat effectively.

• 52:17Speaker 04

  The question is whether this holds also for the withheld private test. But it seems that goal setting is not the bottleneck, rather the action efficiency and the accumulation of the knowledge over a very long context. Because you need millions, hundreds of thousands, if not millions of tokens to solve this. And keeping consistent knowledge of everything that has happened over such a context is a major engineering challenge at this moment.

• 52:44Speaker 05

  What is harder in ArcV3 compared to the other one is this interplay between exploration and solving the game. Because in ArcV1 and ArcV2, you would get all of the information in a static frame as you are given the puzzle. Instead, in ArcV3, you are given the game, but just from the first frame, you cannot understand what needs to be done. So you need to start interacting with the game. And through that interaction, you gather information of what the game is about and you start understanding how to solve it. And at the same time, so you need to try to understand what the game is about and try to solve it at the same time. And this interplay is very hard to kind of explain to the agent how they should do it

• 53:28Speaker 05

  in an effective way, such that it is general and generalized to across all games. So I think this is probably one of the key parts that is hard about ArcV3.

• 53:37Speaker 04

  What we want is abstraction-based exploration.

• 53:41Speaker 05

  Yes, and that's, I think, the very hard part. And I guess here is where the similarity with ArcV2 and 1 are, that you kind of get the right abstraction level at which to do this exploration and kind of understand that, oh, the thing moving around is probably an agent that I can kind of control. And it's not just kind of patches of pixel moving around. And if you're able then to get this right abstraction, then it's usually kind of trivial to solve the game. But uh yeah, making the agent acquire this this right obstruction level, yeah, it's it's the main challenge.

• 54:17Tim Scarfe?

  How difficult is it to acquire the goal? Are you finding that these twenty seven billion parameter models can just reasonably infer what the goal is, or is it much more complicated than that?

• 54:28Speaker 05

  It some in the good runs when they get the goal right at the first um at the first trial, it's it's very easy. They kind of can, if they find the right hypothesis for the game, then they would go on and solve a lot of levels. Um but if they maybe try once, make it wrong, and then have an give another hypothesis. Um and don't and and it's not the right hypothesis again, then it's very hard to get them out of of the loop. And we often find that they get stuck in very kind of uh not intelligent goals that it's very clear they're not right. For instance, often the agents start thinking that reducing the energy bar to the minimum is the goal, or that stepping ten times in a region is the goal.

• 55:14Speaker 05

  Which for a human is kind of clear that that it's not the actual goal, but it's very interesting that uh they're not able to to see that it's it there's no way that's the actual goal. Yeah.

• 55:25Tim Scarfe?

  Yes, and can we contrast rewards and goals?

• 55:28Speaker 07

  We can use various different things, like the most naive interpretation is just use level transitions and then the ARC AGI score for a specific level transition. So if you did offers, yeah, if you did worse, it took more actions than what a a human would have taken, your score would be lower, but you use that per per level.

• 55:47Speaker 07

  Yeah, and then you can add various others, yeah, types of rewards to improve your your objective.

• 55:52Tim Scarfe?

  And did you use reward shaping?

• 55:53Speaker 07

  Yes, so for our um RL training pipeline, we do reward reward shaping. We have twenty five games. We actually have a lot more games that we've generated ourselves. And we can, for example, train on them to perform, to to make the agent better, for example, exploration, finding goals, um achieving those goals, um yeah, through into an RL on, yeah, just a a bunch of games, basically. And the thing is, this is much more difficult to do than in ARC two. Arc two, this was the standard approach. You would pre-train on a lot of puzzles and then you would do test time training as well on the set of puzzles. But to do this in Arc AGI three, you need to train over, as Niko mentioned, like hundred thousand, two hundred thousand

• 56:36Speaker 07

  tokens, which is extremely difficult to do. So we try and improve this by, for example, adding reward shaping, training on shorter sequences, trying to generalize to longer sequences. And I guess what makes this so interesting for Tufulabs as a problem is that this is exactly the kind of research that is currently the set of the art, agentic, multi-turn, long context reasoning. And yeah, it simplifies the problem. We don't have to worry about environments or other complexities like safety. We can just focus on the core problems and try and see if we can make progress there. And that's also perhaps some area we could actually contribute some new research, which might, yeah, I guess, the open source

• 57:21Speaker 07

  at the end of the competition, or we'll, we'll see how we, what we want to do with that research.

• 57:26Speaker 07

  But yeah, basically on your reward shaping question, we add various rewards, level rewards, Arc AGI scores, whether the code gets executed or not, the, the length of the reasoning steps, because if it's reasons too long per, per step, it might take too long to actually solve it. And you only have a certain amount of tokens per game. So all of that we try and improve.

• 57:46Tim Scarfe?

  Yeah. You know, like the, the weird side effect of Arc V one and two is it left me with the impression that intelligence is overrated. And what I meant by that was, you know, yes, we could, we could solve problems. We could do hill climbing, you know, when we knew what the problem was. And maybe you would agree that in Arc AGI three, when we know what the goal is, it becomes a hill climbing problem. But it, it feels like the challenge is abstraction. And it's almost unfair that Sholay is doing this action efficiency thing. I mean, in my opinion, what he should do is, yeah, you, you do it the dumb way the first time, and then you compress that into reusable knowledge and it's in your library. And then the next problem that comes up, which should ostensibly be using what you just learned in the first game, then you become more efficient. So you kind of become more efficient over time because

• 58:31Tim Scarfe?

  Your compressing knowledge, that that kind of feels like the goal to me.

• 58:34Speaker 07

  Yeah, so there's actually been discussion in the community about, let's say for RKGI four, having that as one of the goals. You play a game, you solve it as in like a large compute or action budget. Then you have to play it again and you have to like a speed run through it to improve it. So I think that is important, but I also think action efficiency is a practical step to reduce just brute force solutions. It it counteracts that. You have to explore in a more intelligent way. And it kind of makes sense in real world environments as well, like computer use, coding agents. You can't just explore every possibility. Um. You have to do it in an intelligent way and improve, uh, yeah, using some heuristic.

• 59:12Speaker 07

  And I I think, yeah, I think it makes sense to, yeah, not always have like, uh, yeah, just all the actions available to you.

• 59:20Tim Scarfe?

  Okay, but but even on it on RKGI three, though, when when you do have the goal and it's a hill climbing problem. I I guess it you you still can't brute force it because you get you get dinged on on efficiency. But in principle, you could if you had enough computation.

• 59:35Speaker 07

  You could, but even that they made, so they they updated games to be more robust to that. So like level one, you could, um, in principle, brute force in a certain amount of actions. Level two, the computational power you normally need is enormous. So you can't really, uh, with a reasonable, uh, with a reasonable compute budget, even get past level two. And then you have level three, four, depending how long the game goes. So, yeah, in principle, you could brute force anything. Um, but practically with the compute budget we have, you you won't be able to do that.

• 1:00:05Tim Scarfe?

  And could you just describe what is it about RKGI three that makes it almost impossible to brute force?

• 1:00:10Speaker 07

  The main thing is, uh, the action space. So we have eight main actions, but there's a mouse click action which has around four thousand possible places you can click, sixty four by sixty four. And also the amount of actions you normally take is quite large. It might be anything from a few hundred to a thousand actions on all the games. So that branching factor is just enormous if you're not doing that in an intelligent way.

• 1:00:34Tim Scarfe?

  Is anything in RKGI three badly designed or gamable? Because, um, didn't the preview have some, it was a bit. Brute forcible, wasn't it? You were talking about that earlier.

• 1:00:43Speaker 07

  Yes, yes, exactly. So I guess that was the main point about the previous competition to wish to show whether there is something you could easily exploit. And indeed, there was like the stochastic guise algorithm and a few other algorithms. Actually, so I think the first four places was basically brute force algorithms that just search over a large space of actions, but do some basic form of filtering. And you can get a very good score. Like I could solve two games, almost solve the third game on the private set. The games are also too easy, so they upped the difficulty level. And for example, one other thing was the timing bar, which only changed when you actually executed a valid action. So you could easily learn what actions are valid or not.

• 1:01:24Speaker 07

  On the new set of games and R K G I setup, I don't know if any of you have noticed anything that is easily exploitable or badly designed. Yeah, I think it's implemented much better now and that's why the scores haven't shot up initially. It's still like 1%. So I think it's still challenging.

• 1:01:45Tim Scarfe?

  I guess the million dollar question, though, is do you think it's possible in principle to do really well on R K G I three and be no closer to A G I?

• 1:01:55Speaker 07

  Yes, I think it is possible. But I guess that's the point of the benchmark as well, is that it doesn't have to prove that once you solve R K G I, you actually have A G I. But I think as the art prizes mentioned over and over again, as you just this proves that something that can't solve this benchmark is not A G I. So yeah, I think it's definitely possible, but getting 100% on this benchmark would definitely require some major improvement, if just like a base model or something, but like we're nowhere close to getting 100%, which is the grand prize that is also made available at the end of this competition.

• 1:02:30Tim Scarfe?

  And do you think people will get near to 100% at any time in the near future?

• 1:02:35Speaker 07

  So I guess that's a debate. Some of us believe it might be possible. I don't think we're going to get that this year. But yeah, I don't know if there's any other opinions on whether we might.

• 1:02:47Speaker 03

  So there's, I mean, that's the, you can split the problem at least into like, will someone reach it in the GAGA competition? Where you're playing 110 games in nine hours on a single RTX 6000 versus is someone going to do it with a frontier model? 100% is essentially impossible because it means you cannot kind of mess up once if you kind of have the wrong hypothesis on one level.

• 1:03:10Speaker 03

  That's it, you're done. So I'd more like say, but no, no single human is likely to score 100%. So I'd more say like maybe 90%, you could kind of say, you know, this model is capable of doing this. I kind of feel that is actually possible, even within the constraints of the category competition. I do think it's going to take more research and probably simply better base models. Then will be achieved by November, when the competition ends.

• 1:03:33Tim Scarfe?

  Do you think there's some notion of a almost Kull Magorov optimal score? You know, like if you were the perfect intelligent agent, you would kind of converge on this score. Does that even make sense?

• 1:03:43Speaker 03

  Probably yes, and I do think that score would probably be 100%. Like the game should actually kind of be designed that that score is 100%. That you don't kind of, that you don't have to, you kind of, that score, if it's not 100%, it essentially means there is guesswork involved.

• 1:03:57Tim Scarfe?

  Didn't you say before though, you could only get 100% if you already know? So like if you were an agent that genuinely needed to acquire knowledge to solve the task. So you had to do like the minimum number of steps to require the knowledge. So do you think that would converge on a certain score?

• 1:04:13Speaker 03

  Yeah, so that's what I think would be like 90% or something. Simply because it is not reasonable to, to, like we just saw the Ares 20 game, there's two places you can move to. Yeah, you just can't guess that right every time. The thing is, you're being compared to humans who also had to do the same exploration and you get a median score. So kind of with them, if you're actually lucky, you're going to do it. But yeah, you have 110 games. You're going to get unlucky in one of them.

• 1:04:40Tim Scarfe?

  Yeah.

• 1:04:40Speaker 03

  And not get, I will mention, even with our simple, with Gwen, we can already get sometimes 100% on certain games. So that it is, let's say, certainly doable to beat those human baselines.

• 1:04:52Tim Scarfe?

  What we were saying earlier is that, you know, one view is that intelligence is this quite crystallized process. There's a concept of IQ, for example, and if you have a certain amount of IQ, then it's predictable how well you can generalize, how efficient you are. And I guess the data might just not back that up. The data might just say, actually, it's kind of not random, but very specialized. And there's, there's huge differences, individual differences in capability and machines and in humans. And would that make them kind of reassess their whole idea of what intelligence is?

• 1:05:25Speaker 07

  On Arc AGI three specifically, I think we have a clear benchmark, which we know humans, which is general in some sub domain, which we can say whatever that domain is, can score that score and that is the medium score. So I think even with just in Arc AGI three, if we do bad with our, let's say the final submission score is like five percent, I don't think that would update their views because like humans can achieve that.

• 1:05:48Tim Scarfe?

  Or maybe they would argue that if if it did converge and become more regular. So if there's a new class of algorithms that kind of consistently solve the problems in some predictable amount of time, maybe Shalay would think, oh, that's the algorithm of intelligence. It's not just guessing anymore or something.

• 1:06:05Speaker 04

  I think the goalpost is gonna move further. I think they will update their model. I think in some way they already have by changing the games because they fought some of the games that they made weren't brute forceable and they actually were. And then there were some simple models and they also changed the the metrics, updated it a couple of times, both to be more fair and to represent better human performance. So certainly they have been updating their model a little bit already. I it's it's tough to say what's gonna what's gonna happen, but I think I think the goalpost is like moving all the time. And if we create

• 1:06:50Speaker 04

  an agent that would be able to solve all of this, then we're gonna still discover new areas where where it's still missing and it is not targeted by the data set. I maybe it's rumors, but I think there's like other Arc AGI competitions coming up. They're buil- building Arc AGI four or something for next year, I think. So certainly they have more to add to the story than just Arc three.

• 1:07:16Tim Scarfe?

  Do you think that they won't release four until people are doing well on three or they will just like release it anyway?

• 1:07:21Speaker 03

  If you look so far, it does feel like they, I don't know if it's intentional or just the speed which they developed them, but it does seem pretty clear. Unless I don't think they knew what Arc AGI three would look like when they built Arc AGI one. You might have even thought, oh, it's going to be 10 years before anyone beats Arc AGI one.

• 1:07:37Tim Scarfe?

  I think you could.

• 1:07:37Speaker 07

  It was like five years in the end.

• 1:07:40Speaker 03

  Yeah, sure, but the point is, at that point, you think, okay, now they can do this. Does this really mean intelligence? Well, probably not, but I think also you need to see what happens with Arc AGI 3 and also the ways in which it gets solved to then also understand where the gaps still are. Because again, no one is claiming if a game can play, if you can do a 64x64 game, you're intelligent in every sense. It is just one more step and one more challenge that they've beaten. You can imagine, maybe, for example, Arc 4 has kind of games, like you're playing Doom, for example, but not Doom, but of course a new version of Doom, that is a whole. You need to kind of work with a much more vaguer representation of the world and it would be a completely new challenge again. But I imagine, yeah, it's going to kind of taking little time seeing what's happening here. I've also no idea what they're doing in the background right now and.

• 1:08:23Tim Scarfe?

  Yeah, I've no idea, actually. Yeah, Sholay was quite forthcoming about Arc AGI 3.

• 1:08:28Speaker 07

  I'm wondering if it's not the idea you discussed, because the community has discussed this a lot. And I think there's been some back and forth about having to play a game and then having to acquire the skills and the knowledge and then replaying it at, let's say, almost a maximum efficiency, like speedrunning it. So that's one intuition that's been going around in the field, but I guess only Greg and the team can tell us next year.

• 1:08:52Tim Scarfe?

  The thing I really liked about Arc V1 and 2 was that, I mean, first of all, it was a grassroots community and loads of people without lots of funding could just focus on the kernel of intelligence. And now it's just ridiculously complicated and it's almost de-intellectualized the field, which means that, you know, loads of folks with resources are just throwing reinforcement learning algorithms at it and almost going back into kind of blindfold mode, you know, which is, I think it's a fair description of a lot of machine learning, you know, like we just have really big models and we just train it to do next token prediction and we see what happens and we try and scale up. The thing that was so magical about the community was there were just so many great ideas and like, you know, just really, really deep thinking.

• 1:09:30Speaker 05

  It's not that simple, like even if we have a lot of compute, it's still hard to make a harness that solves these games and it's still, and only starting, like, and training is also hard. Like, it's not as simple as just throwing computer. There are still many details that you have to get right. But for sure, as you said, I think it brought up a lot, level needed to enter the competition. Like, we're seeing that many people are stuck in the kind of template solution. And there are, as far as we are aware, at least in the competition, not many trying kind of LLM approaches, because it's so computationally expensive. So it's definitely made it harder for um kind of the average person to enter the competition.

• 1:10:12Speaker 05

  But I wouldn't say that it's just a matter of throwing a lot of compute at it and brute forcing, yeah.

• 1:10:18Tim Scarfe?

  It it's not like most other benchmarks, right? It there's there's no language, there's no instructions. Does that make it quite distinct as well?

• 1:10:26Speaker 05

  I think I think that's the interesting part about ARC, that it kind of tries to remove as much as possible the prior that you get from um language or from human knowledge and kind of strip them to the minimum and really only test for intelligence. I think, yeah, that that's a cool feature.

• 1:10:43Tim Scarfe?

  Language is the basis of how we do a lot of thinking. And what you folks have done is you've you've gone to language and then gone back again. So we've we've got this kind of loop. So we go to language, we do some reasoning, and then we might use that to do active fine tuning or like, you know, do reinforcement learning or or whatever. And we've got this virtuous cycle. So he took the language away and we're kind of putting it back again.

• 1:11:09Speaker 07

  Yeah, I think the reason we're putting it back is because we're specifically focusing on language models, which has been extensively trained on language. I guess if there's other approaches, like um not like neural guided search, they might not use language at all. You don't need language for this, at least uh human textual language. But it's just, in our case, we're heavily leveraging reasoning models. So it's just natural to bring them back into the domain they're experts in and what they're doing, uh while we wanna actually just, our harness should shape that and encourage that uh behavior. Uh so, yeah, I guess that's the main reason we're moving back.

• 1:11:48Speaker 04

  I think it's also very difficult to, like, I think the the the reason, and I've seen, I said it before already, is that the reason why language succeeds so well, because there's like higher higher level priors that are encoded that is just so difficult to strip away from the from the data, may that be what uh what like, the humans have created these games and they have an idea of what a game is. And even though they tried to remove uh The the the bias is a little bit, they're they're still there. You might look at these games and usually the backgrounds are some sort of like dark color or or some or or white rather rather than green.

• 1:12:30Speaker 04

  And this is like not necessarily true for all of them, but like more or less it holds. And you can see that this was created by humans and what their idea is of games. And that is just why language holds. I think it's incredibly difficult for a human to create something that doesn't have this in it. It might be even impossible.

• 1:12:50Speaker 04

  So that's maybe why coming back to the language works so well.

• 1:12:53Speaker 07

  Yeah, just to quickly add on that as other like I've also seen exactly that. So if you so one thing you could do, you could easily permute the game. So you can permute the colors and also rotate the images. And when you do that, the game becomes significantly harder. So if you just remove that priors, which shouldn't actually be priors, you can actually see it becomes harder for humans to play. So yes, perhaps that tells something about the benchmark. There's some leakage of human priors into the the game.

• 1:13:20Tim Scarfe?

  Harder for the algorithms as well?

• 1:13:22Speaker 07

  Yes, yes, yes. So that's one thing we found of the harness. If you're not, so you can you can provide numbers as colors. It does worse than if you encode those numbers to specific colors that it knows, like black for background, gray for immovable areas. So if you remove those priors, it does perform worse, which is interesting.

• 1:13:40Tim Scarfe?

  That is fascinating.

• 1:13:42Speaker 05

  On why I think it's important to pass through language? Because like initially, we tried to just train a neural net with RL on a lot of games and kind of not use L L M altogether. And try to, yeah, go to get those prior in by just seeing a lot of games. And it's way, way easier to just use an L L M. Like you you get away, like you get the good performance much faster because.

• 1:14:08Speaker 05

  The prior of language is so general, it can be applied to so many different domains. And getting the same prior by just training takes, it would take like so many games. Instead, by starting from language and then fine tuning to the specific games, you can get there much faster.

• 1:14:25Tim Scarfe?

  You know, it's quite a common technique to transform something so that it falls into a representation that has more friction with our knowledge.

• 1:14:35Speaker 07

  The encoding from just numbers to specific, like you use charts, like let's say C for, or like let's say, yeah, I don't know, blue, you use a B for blue. And we can tell it like this is blue, this is this color. This automatically helps it a lot in language to reason about it. Because it knows that typically when humans play games, bright colors are objects you want to interact with or something's going to happen. More dull colors, background or walls or something like that. So that definitely helps a lot.

• 1:15:06Tim Scarfe?

  Because that's actually really interesting, because I suppose like one strategy is you could have multiple representations. Or from a language point of view as well, you could do something really galaxy brain, like turn it into a story where the colors become dogs or agents or people. Do you sort of mean like, is there some representation that will just make the language model come to life and it's significantly more powerful?

• 1:15:26Speaker 03

  As far as we've seen so far, the models are actually pretty decent at perception. And they, because remember the game in the end is not a six four by six four grid. It has concepts, it has a player that's moving around, it has a goal, it has the maze. And you're not going to play the game if you don't get those concepts. The thing is, the L L M is pretty good at getting at those concepts. The problem is that it takes a while and we have very little time to play the games. So one aspect in the duck that we haven't discussed yet is, for example, that we give it different views on the world. So we already mentioned L S K grid, but for example, we also have a simply, it's very simple. It's just kind of a segmentation based on connected components. By just having that and giving it that list, it works well on some games, not well on others.

• 1:16:09Speaker 03

  But it kind of allows the model to short circuit a lot of this perception part. And the perception, it will kind of, yeah. Look at Askikirid, and we zoom in on part of the board and make an Askikirid there. Oh, that looks like this, and do it on another one, and spend a lot of tokens that way.

• 1:16:21Tim Scarfe?

  What about from a vision point of view as well? Do you think language is actually necessary for reasoning? Like, do you think in principle, we could train a vision model to go directly to the output from the input?

• 1:16:33Speaker 03

  Since I think it boils down to one of the big open questions in the field, like, is language critical to intelligence? I will say, let's say, at least when I'm playing the games myself, and I think that goes for all of us. I'm using language. Like, these games are set up that indeed with the object prior, like, where the four-year-old could do it. But I don't think a four-year-old would be beating most games. And I'm actually thinking through a strategy, myself. I am thinking, let's say, out loud, or it's in my head, but there's a narration, and a narrative and language, basically. Okay, we have to get the key, but the key is blocked. How does that work? And, yeah, how you decouple all that, I'm not sure.

• 1:17:07Tim Scarfe?

  Language is almost like an emergent phenomenon. So it's still almost determined by its physical instantiation in the world that we live in. But it's something which has evolved over a very, very long time. So we could, in principle, train an end-to-end vision model. And if it saw an insane amount of data, like, almost all of the distribution, it could potentially acquire the same abstractions. But language is almost like a shortcut, because you can bootstrap those abstractions.

• 1:17:35Speaker 03

  Yeah, it's also tricky, because it could, but it might end up kind of internally developing its own language. Because you need stuff like reasoning chains. I mean, in our case, it needs to write code, which is a language itself.

• 1:17:46Speaker 07

  That is a good point. So I did some initial oral experimentation. And what I did for the first experiment to show that we can generalize, is I created a ARC environment that is procedurally generated with some new objects and new objectives. And in fact, you can, you get the behavior. It's just like a vision model and like a Mamba backbone. And you can get the behavior that it learns to figure out the objects and place them in the right order, just without using. Language is using a latent representation. So I think it is possible in principle, but what that experiment showed is I required about ten five thousand different permutations just to solve that one setup. On real R games is going to be more than that if you want to train from scratch.

• 1:18:30Speaker 07

  And so, yeah, that's why the prior helps a lot. You need less environments to actually get it to perform well. Or you could just do it with a harness. And if you train, the hope is that you need less environments to actually just improve on and generalize, which is the most important part.

• 1:18:44Tim Scarfe?

  Yeah, and it does lean on this more general point that we think we think humans are so intelligent. But we have the benefit of millions of years of evolution, right? So so we're kind of we're using all of that trial and error implicitly. And so it's almost a bit unfair that we're expecting AI algorithms to be able to do all of that.

• 1:19:02Speaker 07

  Exactly.

• 1:19:03Tim Scarfe?

  How do you square the circle between, you know, like you're releasing lots of pre-built tools that are quite specialized? And you also are bitter lesson peeled. Is that is that a contradiction? Like, what what does it mean to be bitter lesson peeled?

• 1:19:17Speaker 05

  As a lab, we subscribe to the bitter lesson. I am, I guess, the idea behind the bitter lesson is that

• 1:19:26Speaker 05

  often detailed and specific solution lose compared to just more data and more compute and kind of going large scale on a problem. I think looking at the past, it's even if the bitter lesson wins in the end, it's always useful to try first. The more kind of digital, specific approach, for instance, for AlexNet, which was the paper that won kind of ImageNet and in some way started the deep learning revolution. I think a lot of the intuition that brought them to use convolution neural network for image recognition were in some way motivated to all of the tricks that were done before to do image recognition.

• 1:20:08Speaker 05

  And so I think that even if it's true that now we're doing pure engineering on the harness, we're kind of understanding the details of the problem that then will allow us to do a more end to end kind of solution. And allows us to do something.

• 1:20:24Tim Scarfe?

  That journalize hopefully more. So I think this understanding of the details and specification is probably a necessary step.

• 1:20:31Speaker 03

  Yeah, I want to add that I may be a little bit less bitter lesson built than than most AI researchers, also mainly because maybe because I was doing it maybe before the bitter lesson became the the baseline. But I do think at least certainly with Arc AGI three is right now. I would kind of be willing to make a bet that the winning solution, be it also someone else's of this competition, will actually not be a bitter lesson solution. So if you look at Arc AGI two now, it's been pretty much bitter lessened and it's being won by creating huge amounts of training data. I don't think there's a clear path just yet for this competition and that what is needed now is really kind of really the, yeah, basic design. I do think it will involve training, but probably not to the degree that all you need to do is have a good training data set.

• 1:21:16Speaker 03

  And then go for it.

• 1:21:18Tim Scarfe?

  It's fair to say that Sholei does not want it to be bitter lesson build, like every single design step has been away from the bitter lesson and he would he would love it in a world where it wasn't the case that that it was bitter lessened.

• 1:21:29Speaker 03

  I do say I might be willing to make that bet for this year. If there's another iteration next year or one other one the year after that. I could imagine that at some point it is bitter lessened. But I don't think the models are there to be able to do that this year already, but.

• 1:21:45Tim Scarfe?

  Yeah, it's it's so amazing that even in twenty twenty six, we we have so much uncertainty about this. We we we still don't know beyond a reasonable doubt that it might not be bitter lesson build in two years' time.

• 1:21:56Speaker 04

  Important to bring things into distribution and what Art Prize has done for Arc AGI three is that they specifically say that the training problems or the public problems are not representative of the testing problems. So that makes it even harder to create a distribution on which you could meaningfully train because you still have to extrapolate in some way. So maybe that's also why and overfitting just doesn't doesn't help because you have to generalize to new problems. So as you say, they are specifically targeting this not to be just like mass trainable. But maybe if we can extrapolate the distribution well and guess, oh, guess whatever.

• 1:22:40Speaker 04

  Guess what's in the testing distribution, then this could be done, but this is just hypothesis.

• 1:22:45Speaker 03

  I do think maybe in a sense it is going to be better lessened. But the classical better lessening is just you make a training set so big that everything is in there and then just bake everything into your weights and then you're there. This is kind of what happened for AGI two. I think this one does need something like that, but it needs to have, and I think that is maybe much more what Sholay would also like to see. What you need to get into the model is the basic thinking patterns. So it needs to know the general ways in which to approach a game and how do you solve a game without having memorized every game in particular. And I think if we can manage to do that, yeah, we'll feel like we've really done something past just beat a bunch of games.

• 1:23:20Tim Scarfe?

  And how do you reconcile the fact that you're trying to build more powerful AI, but you're worried about it?

• 1:23:27Speaker 06

  So yeah, reconciliation is I think you cannot do AI safety research in a vacuum. You need to be including about the best techniques and being able to kind of train good AI systems to be able to study them at all. Yeah, the way I would say it is if you if you want to make bridges safe, it's going to help to know civil engineering and how to build bridges.

• 1:23:50Speaker 06

  And so I think similarly for AI, but that runs the risk into then you participate in the race. And you make that kind of racing situation worse. Because this more like the progress about AI capabilities runs ahead of progress towards AI safety. So I want to be very careful with this. Right, is I think now we I'm very happy with the team we currently have and the kind of level of technical discussion and common interests and actual work being done. So that's all very positive. And gone wrong is it just takes forever to get there somehow. Like it's takes longer, more expensive than what you think it will, so.

• 1:24:32Tim Scarfe?

  Yeah, yeah.

• 1:24:33Speaker 06

  And then I guess what's wrong is. Um, the broad AI progress is very fast and it feels a bit um a bit tough that can you can you ever catch up? I think the next year it might not look that different than than today, but I would say five, 10 years away may look completely different. Yeah. So I think I'm a believer in the software and singularity. I think software advance will keep being made about kind of agentic uh engineering, stronger uh like stronger AI system in the world of atoms generally. Uh, but I think eventually we replicate this success with things like um

• 1:25:17Speaker 06

  industrial robots, like self-driving cars. I think they, at some point, the morphic paradox kind of falls and then the we start seeing powerful AI capabilities everywhere.

• 1:25:33Speaker 01

  When you shop Kroger delivery, you can expect the savings you love and fresh groceries delivered right to your door. From delicious produce to everyday essentials, enjoy the convenience of delivery and the trusted selection you've come to know from Kroger. Because bringing you fresh, quality groceries is what we do best. Right now, enjoy free delivery on orders of fifty dollars or more with your digital coupon. Restrictions apply. See site for details. Kroger, fresh for everyone.

• 1:26:03Speaker 05

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked three hundred fifty thousand Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com, not authorized by any candidate or candidate's committee.