#241 Patrick M. Pilarski: The Alberta Plan's Roadmap to AI and AGI

This episode is sponsored by Netsuite by Oracle, the number one cloud financial system, streamlining accounting, financial management, inventory, HR, and more. NetSuite is offering a one-of-a-kind flexible financing program. Head to https://netsuite.com/EYEONAI to know more. Can AI learn like humans? In this episode, Patrick Pilarski, Canada CIFAR AI Chair and professor at the University of Alberta, breaks down The Alberta Plan—a bold roadmap for achieving Artificial General Intelligence (AGI) through reinforcement learning and real-time experience-based AI. Unlike large pre-trained models that rely on massive datasets, The Alberta Plan champions continual learning, where AI evolves from raw sensory experience, much like a child learning through trial and error. Could this be the key to unlocking true intelligence? Pilarski also shares insights from his groundbreaking work in bionic medicine, where AI-powered prosthetics are transforming human-machine interaction. From neuroprostheses to reinforcement learning-driven robotics, this conversation explores how AI can enhance—not just replace—human intelligence. What You'll Learn in This Episode: Why reinforcement learning is a better path to AGI than pre-trained models The four core principles of The Alberta Plan and why they matter How AI-driven bionic prosthetics are revolutionizing human-machine integration The battle between reinforcement learning and traditional control systems in robotics Why continual learning is critical for AI to avoid catastrophic forgetting How reinforcement learning is already powering real-world breakthroughs in plasma control, industrial automation, and beyond The future of AI isn't just about more data—it's about AI that thinks, adapts, and learns from experience. If you're curious about the next frontier of AI, the rise of reinforcement learning, and the quest for true intelligence, this episode is a must-watch. Subscribe for more AI deep dives! (00:00) The Alberta Plan: A Roadmap to AGI (02:22) Introducing Patrick Pilarski (05:49) Breaking Down The Alberta Plan's Core Principles (07:46) The Role of Experience-Based Learning in AI (08:40) Reinforcement Learning vs. Pre-Trained Models (12:45) The Relationship Between AI, the Environment, and Learning (16:23) The Power of Reward in AI Decision-Making (18:26) Continual Learning & Avoiding Catastrophic Forgetting (21:57) AI in the Real World: Applications in Fusion, Data Centers & Robotics (27:56) AI Learning Like Humans: The Role of Predictive Models (31:24) Can AI Learn Without Massive Pre-Trained Models? (35:19) Control Theory vs. Reinforcement Learning in Robotics (40:16) The Future of Continual Learning in AI (44:33) Reinforcement Learning in Prosthetics: AI & Human Interaction (50:47) The End Goal of The Alberta Plan

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The Alberta Plan is a approach to pursuing really strong artificial general intelligence, which has deep roots. So it's something that I think a lot of us at the University of Alberta and at the Alberta Machine Intelligence Institute, and it's a precursor to the AICML, have pursued over the years. And it is, it is indeed sort of a, both a structure for how we might think about building intelligent agents that can learn from their own experience, and also a set of commitments that sort of tie us to an approach to doing research, which we believe is going to be able to support generating these kind of new future technologies.

• 0:35Craig S. Smith?

  What does the future hold for business? Ask nine experts and get 10 answers. Bull market, bear market, rates rising or falling, inflation going up or down? Can somebody please invent a crystal ball? Until then, over 40,000 enterprises have future-proofed their business with NetSuite by Oracle, the number one cloud ERP, bringing accounting, financial management, inventory, HR, into one fluid platform. With one unified business management suite, there's one source of truth, giving you the visibility and control you need to make quick decisions.

• 1:20Craig S. Smith?

  With real-time insights and forecasting, you're peering into the future with actionable data. If I were a larger organization, this is the product I'd use. Whether your company is earning millions or even hundreds of millions, NetSuite helps you respond to immediate challenges and seize your biggest opportunities. Speaking of opportunities, download the CFO's Guide to AI and Machine Learning at netsuite.com/ionai. That's NetSuite. netsuite.com/eyeonai

• 2:02Craig S. Smith?

  E Y E O N A I all run together to get the CFO's guide to AI and machine learning. The guide is free to you at netsuite.com/eyeonai. netsuite.com/eyeonai.

• 2:21Speaker 00

  I'm Patrick Plarsky. Thanks for having me. I'm a Kennesy Far AI Chair and a professor, actually, in the Department of Medicine at the University of Alberta, and also a fellow and on the board of directors of AMI, the Alberta Machine Intelligence Institute here in Edmonton, Alberta, Canada. So my, my line of research, I mean, I, I was trained as an electrical computer engineer. I worked on all different forms of machine learning over the years, everything from evolutionary computing, whichever was like, ooh, to swarm intelligence, to, to things like supervised machine learning, computer vision and, and reinforcement learning. So that's the sort of trajectory I followed. I'm currently working mostly in bionic medicine.

• 3:03Speaker 00

  Artificial intelligence applied to how humans and machines interact. And in particular, our research lab is working on a very cool, I think, very ambitious research project where we're looking at literally bolting robots into the bones of the human body. We work, we work on artificial intelligence and, and transformative surgeries and ways of measuring people's interactions with prostheses. So we're looking at, say, people with amputations or other forms of limb difference, how we might begin to build a new sort of science and art of prosthetic restoration. Neuroprosthesis for people to use artificial arms and hands in particular. And so, yeah, we're looking at bone anchoring surgeries, how you can rewire the human nervous system

• 3:45Speaker 00

  to engage better with a decision-making machine like an artificial limb, artificial hand. And then building a new generation of, of artificial intelligence technologies that really sort of glue that person and machine together in an informational sense to act as a broker to translate what the human wants from what the artificial limb is actually able to deliver. So, yeah, we're working on bionic medicine. And it's very, very cool. What I think how this ties into our conversation today is that it's in fact, it is an example of of machine learning technologies, reinforcement learning technologies that are deployed in very low computing environments. Like, we're not imagining an artificial limb that continually talks to like a supercomputer in space every time it wants to try to open and close a hand.

• 4:28Speaker 00

  We want devices that people can wear on their body or attach to their body that when they drive under like, under a bridge or in a tunnel on the freeway, that it's not going to stop communicating with the internet and stop working for them. So a lot of what we think about is how you have very powerful technologies that can actually be embedded within the devices that people wear on their body. So I think that ties into what we're going to chat about today.

• 4:52Craig S. Smith?

  Yes, the Alberta Plan. That was what I was interested in. I spoke to

• 4:59Craig S. Smith?

  Richard, Rich Sutton about it the last time I had him on the podcast. But we didn't go deeply into the Alberta Plan and I've seen it referenced more recently in reinforcement learning. Because of the use in training large models is is getting more general attention than it has for a while. So I wanted to hear for listeners as well as myself, exactly what is the Alberta Plan? And where are you in in in executing the Alberta Plan?

• 5:41Craig S. Smith?

  And and how that relates to what's happening in reinforcement learning research?

• 5:47Speaker 00

  Yeah, the Alberta Plan, the Alberta Plan is a approach to pursuing really strong artificial general intelligence, which has deep roots. So it's something that I think a lot of us at the University of Alberta and at the Alberta Machine Intelligence Institute, and it's a precursor of the AICML, have pursued over the years. And it is, it is indeed sort of a both a structure for how we might think about building intelligent agents that can learn from their own experience and also Set of commitments that sort of tie us to an approach doing research, which we believe is going to be able to support generating these kind of new future technologies. So the Alberta Plan itself is something that we have, again, talked about for years.

• 6:31Speaker 00

  Recently, we actually wrote it up. That's the, I think, the manuscript that you're referring to is the Alberta Plan document that Michael Bowling, Rich Sutton and I put out a couple of years back. And that is where we tried to really sort of codify the commitments of the Alberta Plan and also what we think is a very reasonable stealth step 12 step plan to actually executing on it to to build really powerful thinking machines that can learn from their own interactions with their world. And I do think there are tie-ins from this directly to what we're seeing and how RL is being applied to everything from tuning large language models to controlling industrial processes. So I'll touch on that in a little bit. But maybe I'll start with saying that like the Alberta Plan is a

• 7:15Speaker 00

  plan for us to pursue intelligence, the study of intelligence, like this is something that where it's a very big tent kind of thing. This is not like a, yeah, Mike Rich and I are sitting here in Alberta working the Alberta Plan. This is something that that does include many of our colleagues all around the world. People we have not met are pursuing research in line with the Alberta Plan. Our students and our teams here at the University of Alberta and Amy have been going out in the world and and pursuing pieces of it. But I'll say that there's maybe four main commitments that we have in the Alberta Plan that help us think about how we might build reinforcement learning systems, things that actually solve reinforcement learning problems from their own experience.

• 7:55Speaker 00

  And one is that, in fact, it is very experience based, that we believe in agents that are living within the flow of time. And another commitment is that they aren't really like a temporal uniformity. There's no like special training and testing period. We're not like taking a billion dollars to distill down all of Reddit, plus the rest of the internet and turn it into a model. The idea in the Alberta Plan is that we believe that that all of the data that flows into the system, that stream of experience, the agent living within its stream of experience, that experience is always being incrementally added to what it already knows. And that we're not trying to divide up time into epochs of batches of learning versus batches of batches of acting. That acting and learning are dovetailed together in an inseparable way.

• 8:40Speaker 00

  That's part of the third commitment as well, which is that we really like in the Alberta Plan, we talk a lot about respecting computation, respecting every one of those flops we're using. Every single act of computation should be valuable. And if we can be as computationally efficient as possible in how we think about the learning process, then we can just do more learning. Imagine like a bunny rabbit hopping through the briar bushes. Like if you make its operation more efficient, it can predict more things about its environment. It can start to build up expectations about its world in a way that we couldn't otherwise. I guess then the last commitment, then I'll let you jump back in with more questions before we go into sort of like what's in the Alberta Plan. But I think the other really exciting thing about the Alberta Plan is that, and this was sort of us like working

• 9:24Speaker 00

  back and Mike Bowling and I working back and forth with Rich Sutton about like, well, what do we think about multi-agent interactions? But I think the the last thing that we want to observe and we want to sort of commit to studying in the Alberta Plan is the idea that, well, there's other agents in the world, that an agent, a learning machine, whether or not it has to like identify, codify, build a explicit representation for another agent that lives in the world with it, that there are other agents in the world. They create complexity, they create patterns that are learnable, and they create the richness in the environment and also potential to amplify the capacity of the of a decision-making agent. So this is the bit of the Alberta Plan that I'm I'm perhaps most passionate

• 10:09Speaker 00

  about is the the longstanding idea of intelligence amplification, like going right back to Ashby in the 1950s and the the post cyberneticists and thinking about the ability to directly paraphrase Ashby, like that intelligence, much like physical ability, might be amplified by tools that that we can use. One intelligence to multiply the abilities of another and that in fact, artificial intelligence and computation is one of the most powerful amplifiers we might have. So that last commitment, the Alberta Plan that I hold pretty near and dear is the is very much that we might start to see the ability of intelligence to multiply the capacity of other intelligences and that.

• 10:49Speaker 00

  So we, again, want to make sure that we're explicitly thinking about the contributions of other intelligence systems in the world of an agent, any kind of agent, machine or human.

• 10:59Craig S. Smith?

  Yeah. And, and this, as I said, I've spoken to Rich about the Alberta Plan and some of his work. And you differentiate, I mean, the, the goal is, is artificial general intelligence, right? Ultimately. And right now, the, the sort of

• 11:26Craig S. Smith?

  trajectory that gets the most attention are the pre-trained transformer models and, and adding compute and adding data and, you know, measuring against benchmarks and with the idea that that will eventually get us to artificial general intelligence. But there's also people like Yan Le Cun and I'm going to be talking again to Fei Fei Li in a couple of months about world models. The idea that, that you need direct experience, that you're

• 12:10Craig S. Smith?

  not going to reach general intelligence through recorded, particularly textual data that embodies human knowledge. How does, well, first of all, is, is your approach purely reinforcement learning? Is it a combination of reinforcement and symbolic learning? Or, or are you using transformers? And how does it relate to what Yan and, and Fei Fei are talking about?

• 12:45Speaker 00

  I think the, the idea that An agent needs to take into account the interactions between itself, whatever the dotted line that we always debate this in the community like, where is that dotted line that separates the agent and the environment? And it's a very flexible line. I think it's a very deliberate choice that, I mean, if I was an industrial process designer, I would be very deliberate at choosing where that interface is. If I was a thinking machine that was trying to live its life in its flow of experience, that dotted line might change over time even. So I think that what's interesting about the separation of the agent and the environment is that it does sort of necessitate a relationship between them.

• 13:27Speaker 00

  Like, we have to assume that the agent's learning is going to be with respect to what in the, a number of things we would call the aperture between the agent and the world. And that is that, that's your fundamental piece of the equation. So to separate out a, like, let's just, again, let's distill down the internet. I'm trivializing it. This is an amazing miraculous act that we're able to create machines that are able to, like, hold in their digital minds the sum total of the internet and all these private data islands that are attached to it. Like, this is miraculous. I don't want to trivialize that. But the relationship between an agent and its environment is special. And if we do truly want to build thinking machines that can live within their flow of experience

• 14:11Speaker 00

  in whatever their world is, whether it's the world of proto buffers, protocol buffers that are flowing between data centers, or the human world of desks and tables and chairs and doors, we have to really deeply consider that interaction between the agent and the environment. That's part of its learning. And there's another piece of that, which is that the body of the agent, whatever it is, whether it's, again, some kind of network router or a protocol buffer or a piece of metal that is moving itself through the world, that there is physical intelligence within that system.

• 14:48Speaker 00

  And so Rodney Brooks has had some amazing, long discussions about this and has written quite eloquently about it, that like the, the physical intelligence, the way that the, the body itself, not just translates the intelligence of whatever the mind is, but in fact, contains some of the aspects of intelligence itself. This is a, this is something that necessitates our consideration of the environment. So, the environment, agent, environment, yeah, I think we do need to think about that. We, we can't assume, and we, that's not to say that we can't have an agent that watches a ton of YouTube videos, and then is able to actually go out there and interact with the world itself. But if we want agents that not just can interact with the world, but can continue to adapt to worlds and to new worlds that they encounter, it's very important that we think about

• 15:33Speaker 00

  the environment. And so the representations we choose, the data streams that we integrate into agent learning, it's absolutely critical. So, this is again, the perspective that we take, the, the Alberta Plan perspective is very much that, that, that what is being learned is, is about that aperture between the agent and the environment. And what's cool about that is that what's learned can take many forms, and we often think about value functions, we think about expectations about the future, that are about a signal of interest, or many different signals of interest. Those can then be used to sculpt behavior, those, those forecasts an agent makes, the way that the agent starts to build up expectations about its future experience, might be about that cool signal that we call

• 16:16Speaker 00

  reward. We love rewards, nice, precious, very good signal. Agent environment loop, there's a reward signal going there too. But there's all these other things to learn. And so, I mean, when we think about the models that an agent uses, and I'm going to define, I'm going to say what I mean by model, because we all talk about, oh, there's a machine learning model, oh, this is a model, oh, I have a plastic model of a car. When I think about model, I really do think about a, something that an agent has formed, an animal or a machine agent, has formed that is able to in some way, carry the likeness of something, in mo, in this case, mostly its environment, its interactions with its environment. And so that, that's a very, again, big tent kind of approach to model.

• 16:58Speaker 00

  But I do think about this sort of internal construct which allows an agent to be able to represent, to manipulate, to improve its computation and its ability to leverage its experience about the world to choose and make actions. So if we have that model, the nice thing about that is that could we put in a large pre-trained model like something that we'd call a foundation model, you could absolutely leverage something like that. And I don't think anything that at least my researcher would pursue, and I think I can speak for Mike and maybe Rich on this too, is that I think like that we can imagine permissive models that take into account all sorts of expectations about about the world or all sorts of ways of representing the world and the way that streams experience might unfold in that world.

• 17:41Speaker 00

  And we might imagine, so that's saying we could admit things that have been learned that were not learned directly by an agent. I think that's okay. We can also admit into our definition of models and how an agent might use it to improve its action and its computation. We might imagine models that are learned directly from its raw experience. Now, I think, speaking for myself and I think my colleagues as well, is that that area, learning from its raw experience and unlocking the true potential that is is underexplored, comparatively speaking, and that is something that has great power. And so I think that's something that we would want to continue to reinforce is that that reinforcement learning

• 18:20Speaker 00

  is able to build up this understanding of an agent's interactions with its world, like the reinforcement learning agent is going to build up forecasts about its future from its raw sensory motor experience. That's really critical, but it's compatible with other kinds of models that might be learned. And that, again, could be the YouTube-watching agent that's like, I really, I watched a bunch of robots and they were like in video, I watched all the video games like speedruns I could see, and now I'm going to use that to like inform my motor control. That is totally okay. I like that. I think that just makes sense. And that's being, quite frankly, is being responsible with our computation. If we're spending all this time and money training these gigantic systems which are able to, again, figure out how a robot might move from a YouTube video, well, we should be using

• 19:04Speaker 00

  those. We shouldn't be redoing that computation every single time that we want to try to learn, have a robot learn to walk, or to have an agent control itself in a grid world. Like, we should be leveraging our past compute in as flexible the way as possible. That's just responsible use of resources.

• 19:18Craig S. Smith?

  Yeah, on, on, so is that related at all to like DeepSeek's use of direct reinforcement learning to, to train? It sounds like it might be, you were saying that, you know, reinforcement learning, the, the base model could be a foundation model, it could be something else. I mean, Yan has something called JEP, is that right? Anyway, I'll cut that out, but, uh, yeah. So, so what's, uh, how is, is that related to this reinforcement learning,

• 20:03Craig S. Smith?

  uh, training, uh, that suddenly everyone's talking about?

• 20:07Speaker 00

  Yeah, so I'll, I'll start with a, with my personal perspective, which of the kind of models that I like. I really like considering agents that have predictive models of their world. And this means that like the kind of model they might form is about the raw bits of experience flowing across all of those wires into their digital mind. So for years I have taken what normal engineers would say were perfectly reasonable signals, like the, the joint angle of a finger or the, or the current going into a servo motor or the pressure that something, I've taken those perfectly reasonable scale or floating point numbers and just turn them into bits and stuff those bits into the mind of the agent so that it can start to predict the future of how those bits are going to change.

• 20:46Speaker 00

  So that's like, as my disclaimer, my starting perspective is very much of a blinking lights on wires, the agent is predicting its raw experience without that experience being tagged or labeled, or in fact even separated into bins that we might call it like a, a floating point number or an eight bit integer. Okay, so that's my default perspective. I love predictive models. I think agents that leverage their predictive models have amazing potential to interact with diverse environments in ways that, that agents that are biased, structurally biased with our own designer, like human way of segmenting the world might be limited. So, all of that said, like, the idea that you might start to use reinforcement learning to

• 21:27Speaker 00

  Like whatever kind of base, understanding the ability to act or make decisions or to produce tokens or to produce motor trajectories, to be able to take reinforcement learning and place it within that, to be able to sculpt and optimize or to adapt that behavior. This feels like just a, absolutely the right win. I think we probably see this happening in biology all the time. I think that, like you mentioned, DeepSeek, I think that that is another great example. It goes back to, uh, the, we actually had a, we were discussing this at a reading group yesterday. We were brought in the, uh, the paper from David Silver, Sitinder Singh, Doina Precup and Rich Sutton called Reward Is Enough. I don't, I don't know if you're familiar with that. I'm sure, I'm sure you're familiar with Reward Is Enough.

• 22:12Speaker 00

  But like, to paraphrase, poorly paraphrase of what our colleague said in that paper is that like, if you have a single signal of reward, this precious signal, then if an agent is truly trying to maximize the extended amount of that signal that it collects over time, and the task requires it to learn anything from like chain of thought reasoning or social interaction or vision. If, if the task to be solved requires that the agent build this advanced capacity, simple or advanced capacity of what we consider intelligence, then that signal reward, a signal of reward is enough to drive the agent to create whatever it is.

• 22:54Speaker 00

  Again, for the DeepSeek paper, this is like the idea that it might start to create chains of reasoning, that it might be able to unfold in like its little think brackets, it will actually unfold the steps it took to get to a solution. And then the reward signal itself from all those myriad tasks is in fact driving the creation of that, those chains of reasoning outside of other, other sort of like labeled examples that are being stuffed in, of course, compatible with sculpting from all this other, other kind of input information. So I think that like the reward is enough, like this is now, it's now stuck in my head because I talked about it yesterday, but the, the reward is enough idea I think is very powerful because it does let us think about how we might close the gap between all of those.

• 23:38Speaker 00

  Thinking machines that we have created and thinking could be very simple ways or very powerful ways and the like complete cohort of goals that we might have for the technology that surround us in our lives like this is a if there is this missing piece like oh hey I really would love chain of thought reasoning or you know what social intelligence is going to be really important or oh wow like I have a better idea of how bio signals flow from the human body to like a robotic device attached to that body that thinking about reinforcement learning and a very much tabular as a perhaps reinforcement learning as the thing that can pull those two those two the outcome and that prior foundation of technology together I think that's a really powerful thing that we should continue to consider I yeah

• 24:20Craig S. Smith?

  Yeah, I mean that just brings to mind very early on when I was trying to understand this stuff and supervised learning was the was reigning supreme that somebody was saying you know and everyone uses see example of a mother teaching her her child what a cow is you know that's a cow and the child learns that's the label and and from then on when it sees this thing and but but there is the argument that it's actually reinforcement learning that the child is is getting praise from the mother when it when it recognizes

• 25:05Craig S. Smith?

  a cow and yeah so reinforcement learning can be seen as kind of the the the root algorithm operating in learning but how do you implement that I mean can you build a model an intelligent model from scratch purely with reinforcement learning or do you need a base model as as you were saying

• 25:38Speaker 00

  It's interesting, actually, I think, just to follow up on your, the idea about the cow. I'll go back to a conversation, I think this was like, this must have been 2009. So Rich Sutton and I were having a conversation back in 2009 about chairs. And how you think about chairs, and how a machine might think about chairs. And I promise this is more interesting, this me saying like, chairs are important. That, like, the normal way we might think about a chair, or hope a machine thinks about a chair. Like, look at the vast swath of computer vision literature, which has considered objects like chairs and tables and doors. You might think about the chair as like, how wise it, how tall is the chair? Where's the seat? Is the seat hard or soft? How far is the chair from the door? So that's a very normal, objective worldview.

• 26:23Speaker 00

  Like, this is sort of thinking from the perspective of the frame of reference of the world. What is, how do I define the chair, its extrema, its boundaries, its properties? There's another view, this is the view that I think Rich was advocating during that conversation. There's another view on that chair that we might take, which is, well, what's the probability if I were to try to sit down right now, I would actually sit? Or would I fall? Like, there's, if I were to walk three steps, would I bump into something with my shins? There are these very subjective, this is the agent's frame of reference. Now, the frame of reference inside the agent, like, with respect to the data stream accessible to the agent, what do its predictions say about its interactions with the world around it? And so I think a lot of early sensory motor learning, you look at motor babbling in babies,

• 27:08Speaker 00

  like, they start to learn to move their head around, and then their eyeballs, and they start to be able to figure out that, oh, wow, like, if I move my head this way or that way, I, there's a noticeable change in my sensory motor experience. I predict that if I'm able to, if I do this, I am going to be able to see the ball that's outside my vision. Everything from like, the object permanence to, actually, Peri Zudia has some really nice work on this with robots, doing the same thing that babies do. It's a, there's a fantastic paper from quite a few years back, but, but it's really neat, because this says that the system is now beginning to make these like elegant scientific tests. The agent, as a scientist, saying, oh, well, every time I move my head this way, I start to see this other thing.

• 27:48Speaker 00

  Or, I realize that if I, if I move all of these bits of myself, something appears in front of my face. It's my own hand, oh, that's so cool. So like, there's this rudimentary, fundamental learning process, which unfolds, you see, happens with like baby animals and... Humans and otherwise, that is rooted in these predictions. And there's like, again, a whole elegant cohort of neuroscience literature as well, which says, oh, like, the fact that I feel like I'm moving my arm is in fact not the sensors reporting back that I'm moving my arm. It is my predictions about my arm's movement. And then if something gets in the way, like my hand or something, then there's correction that happens with predictions. But the feeling of agency even over my own movements might be due to the predictions

• 28:33Speaker 00

  in my mind. Prediction precedes control, says Daniel Wolpert and colleagues in motor learning. And so from that perspective, you can imagine chaining together, again, going back to ROAR is enough, chaining together all of that process of learning to say that, well, yeah, you should be able to learn everything you possibly would need to know about the world just from signals of reward and predictions of your raw sensory motor behavior. Like, this is the prediction perspective, the subjective agent frame prediction. Now, like, is that the fastest way to build a robot butler or a self-driving car? I, well, probably not. Like, let's be honest, if I was, if I was gonna go out and start building a self-driving car right now, I mean, I want to pitch this to someone, I was like, hey, I would happily

• 29:17Speaker 00

  help your company build a self-driving car. Give me some cars and let me start smashing them into stuff. And they actually said maybe, which I thought was a really positive sign. But if you wanted to take that sort of constructivism, so it's back to constructivism. If you want to take the constructivist view, the subjective agent frame constructivist view of learning a very hard task like self-driving or like full problem solving at the Math Olympiad or something like that, well, I mean, there might be easier or harder ways. What I think is neat, though, is if we look at sort of the trajectory of like AlphaGo, everyone loves talking about AlphaGo, I also love talking about AlphaGo, go forward right here. And like, if we start to think about a version of AlphaGo that learned from watching human

• 29:59Speaker 00

  examples and then had these sort of gaps in its mind's eye, had certain delusions, and then go to the, like, an AlphaZero, which learned through self-play, that It was able to drop some of those sort of data biases that let it find solutions it never would have found otherwise. And I said earlier, like, oh, wow, I love having a machine see the world through its raw bits without me even putting like floating point numbers on top of it. Part of that is saying like, the agent learning from as raw a sensory motor signal stream as possible, allows it to sort of undo some of the assumptions I might be making as a designer, which means it might be able to find just a much better solution, maybe even faster than the other way around.

• 30:39Speaker 00

  Now, I think, you know, building up human language, it takes a while for a human to learn human language through trial and error, through motor babbling.

• 30:47Speaker 00

  If it would take a while to learn language, would we rather just like, again, juice the internet, make some amazing lemonade and use that as our foundation model? Like, yum, large language model, pre-trained, it's great. Like, I think, well, of course we would. This makes a lot of sense. Everything everyone is doing, I think, does make a lot of sense. But there's this other path, and that's why, like, as again, going back to the Alberta plan, why I think we want to make sure this stayed in the public mind's eyes, that that there are alternate and compatible views, which allow us to maybe fill in some of the gaps or maybe to start from a very different foundation point and get to something that is like, maybe our current curve goes up and then it sort of starts to taper off. Maybe there's another approach where you do pursue this raw sensory motor learning about the stream experience, but then that actually has like a higher final peak.

• 31:32Speaker 00

  So let's keep, again, we don't want to have a monoculture, so it's keeping that diversity in our mindstream. And then also being really like, not being dogmatic or like siloed about it and saying, well, where is the like, where is the opportunity to do something like we saw in DeepSeek where they're like gonna throw just like raw reinforcement learning to to help the model get to the the next step of its its like chain of reasoning. I think it's really important that we aren't, again, too siloed or too cliquey in our communities. Be like, no, I'm gonna sit over here and do raw sensory motor learning and ignore everyone else.

• 32:05Craig S. Smith?

  Yeah, are there, are there projects to do just that? Start with raw sensory motor learning? I presume that, that the, the learning then is encoded in, in the weights of a neural network, right? I mean, when, when there is a, a learning, but is there a project to, to start with, you know, whether it be YouTube videos or a robot arm or, or something and see how much learning can accumulate and whether, you know, whether it scales?

• 32:44Speaker 00

  Yeah, I think there's, there's like a happily, a pretty spectacular library of examples globally now. I think we're starting to see and like, I think where we, where we see them most is in specialized targeted domains. So like the, the work from DeepMind on stabilizing plasma in a tokamak reactor. That's like a decision-making system acting thousands of times a second on a very diverse stream of data to literally control a lobe of plasma hotter than the heart of the sun. That's really neat. That's reinforcement. It's learned from reward. And then we see systems that can do elite coding tasks that were also driven directly by reward. And so as we start to see all of these different

• 33:28Speaker 00

  domains, like, well, one of those is very sensory motor. Like I would argue that a stabilizing plasma in a tokamak fusion reactor is in fact very much a sensory motor experience kind of thing. Same thing with like cooling a data center or even compressing bits in YouTube using MuZero. Things like that are much more sensory motor. They're very much like, again, the, the nerves firing down the spinal cord and letting my arms do all sorts of cool things. And then something like, well, I mean, doing really well at like elite coding tasks and like programming competitions. Well, reward driving that feels more like the languagey thing. Like that, that feels a lot more cerebral. It feels like, you know, that, like the deep thinking, not just like the reactive, I'm a tiger running through the woods kind of thinking. But what we're seeing is that reinforcement learning

• 34:13Speaker 00

  applied in its sort of raw sense is being Able to generate insight in both of those domains. So I, I, we'd be here probably till, till, I don't know, next week if I restart the list off. I usually, if I give talks, I actually have like this like best of hits collection at the beginning of my talk. I'm like, hey, what's been happening in reinforcement learning? Well, have you, have you met Grand Charismo Sophie? Okay, cool. You have? That's neat. Like our colleagues at Sony AI did a really cool job of building a racing system, which could do high-performance, superhuman driving while also learn the implicit rules of driving in racing. Not cutting people off. You don't sideswipe people. You like, you know, there's rules of, rules of racing that was able to begin to build up. And that was also again, through processes of, of reward-based learning.

• 34:56Speaker 00

  So it's like, going back to reward is enough in this case. Like you need some social interaction skills. Well, you could, if the task requires it, you can get that through reward. Do you want to be able to solve a programming task? Well, yeah, I mean, you, if you need to be able to think about data structures and the way you chain together methods and, and different computational artifacts. Well, yeah, you could drive that with reward. There's a whole bunch of them thinking, well, what's the, like, is it just bits on a wire like Patrick proposes? Okay, probably not. Like this is a very bold, like if we were to go all the way to sort of like Joseph Monail is another person who really enjoys sort of the, the bits on the wires Rich does too. But like thinking about an agent's experience, just in terms of raw ones and zeros, without any kind of labeling on them.

• 35:41Speaker 00

  There's a, there's a, like, there's a quick shortcut you can take, which maybe you shouldn't take. Again, if you believe my arguments about like design biases, but there's a quick shortcut. Say, well, let's just actually use floating point numbers and integers like Patrick said we shouldn't. And then there's another which like, well, let's just use like, you know, we'll use more, more advanced data structures as the representational elements, the foundational elements that a machine is starting to use, the actions it might take. And then you might say, well, Wouldn't it be faster if we just use some of those that were already learned through like processes of supervised learning? Or, I mean, I like to think about supervised learning and again, a lot of our colleagues would say that it's like learning through labeled examples. So as opposed to like, well, it's like learning through labeled examples. Do we learn through trial and error? Do we learn from the structure of the data? There's cases where each of those might be

• 36:26Speaker 00

  a great way to start to build the fundamental units of experience that the agent is operating upon. And constructing those, like you could say, yes, there's like, we, back in 2011, a large number of us put out a paper on generalized value functions. And one of the reasons we were excited about that, one of the reasons that we thought about it, it's like generalizing the idea of learning a value function about reward, like expectations of how much reward will I accumulate in the future, assuming I care about the future in this way, to do that for any signal of interest. The reason we've been passionate about this and continue to pursue this for, for well over a decade now, is because we viewed those predictions, those forecasts, those expectations of future signals, to be those foundational elements of building up other more, more advanced behaviors

• 37:11Speaker 00

  for an agent. So that, again, goes the choice of, well, what is the fundamental units of an experience stream?

• 37:17Craig S. Smith?

  Yeah. But, but the examples you're giving are sort of narrow domains. How, how does that generalize into, or how, how do you foresee that generalizing into general intelligence?

• 37:33Speaker 00

  I, like, when I even think about the, like, attention's all you need transformers, what are we getting from some of these systems? Like, I, I view them as like the multimodal versions of themselves, not just like a stream that's predicting language tokens, but to think about, like, well, what about a vision transformer? What about these systems that are truly multimodal? When you extend that one step further, and just think about what the operations are happening in these systems, they're creating associations across space and time. Like many different piece of information without any of this like sort of a bit like staged convolutional foveation on a specific part of the air like these these systems are able to consider the flow of time spans of time which are getting increasingly vast and then

• 38:18Speaker 00

  creating associations across that like large span of time but also the different the different channels of information could be anything from like the stream of tokens in a sentence alongside the audio data coming in alongside the time series of the vision pixels to the random bits on a wire that I was proposing to you earlier that the associations across space and time are very, very powerful. I mean, we like Dale Sherman's and colleagues have the have an awesome paper where they're like, yeah, if you like, give transformers redited operations, they're turning complete grossly paraphrasing Dale and colleagues on that paper. But it's really cool because this is like, oh, well, yeah, this is and I'll sort of paraphrase Dale as well here and say that this is another way of thinking about doing computation. It's

• 39:00Speaker 00

  a fundamentally different one. And so what's happening with all this? We have, we have cross attention. We have the ability to like chain together facets of information in the beautiful gemstone of the the past experience of an agent to create predictions about the future. Those predictions are acted upon. They're either going to move a joint or more likely they're going to be able to like spit out a new word token or to spit out a new pixel in a in a generative video. The same thing happens in the base process of like learning a generalized value function. You're thinking about based on a representation of some kind, which we hope is able to form form again associations across both space and time that a system is able to generate a prediction about the future. And these predictions for

• 39:43Speaker 00

  generalized value function, for instance, are temporally extended. They might be thinking about the expected sum of a signal conditioned on a policy that assumes some kind of preference over the the time window that it's being integrated. But either way, this is you could definitely do token prediction with that if you wanted to. You can also predict the future current on a robot arm and start it moving early to be able to like catch a ball mid-flight. So the associations across

• 40:11Speaker 00

  Different modalities or channels of information and across time and using that to create a prediction, like these are different ways of achieving the same thing. You could imagine using them flexibly together. You could imagine committing hard to one of them and working on that. So I do, I feel like that's the, the, the potential here is to see it as, oh, wow, this system is able to put together the fact that I had coffee last week, plus the fact that I'm going to buy milk and then get like, oh, you're probably going to make coffee today. Like, that's a very sort of high level human thing, or like, it's grading my cashmere, or whatever. Like, the, the systems are going to be able to put together these pieces of information to make predictions about the future. And then how you think about the form of those is maybe where I, I see people are starting to disagree, but, but sort of zoomed out a little bit to, to bring together the cross

• 40:56Speaker 00

  attention idea with some of those predict token prediction in terms of raw sensor code experience. That's, uh, there's a lot of unity there that I think that we're just not celebrating, quite frankly. And I, I really like it because it does, like, some of the tricks that we're starting to see, and I, the tricks is not to trivialize the elegance of the computation for, like, taking a vast history and giving the access, like, a, a large language model, having access to a vast history of tokens, like, me sitting there with, like, Google Gemini and typing and having this lovely conversation about, like, structural neuroscience. Like, there's a lot of tokens that are going, like, we, we chatted for an hour. And like, the tokens, like, getting these long histories and then still thinking about the relationship across these vast spans, I think, is something that

• 41:41Speaker 00

  is, that is, is exciting and new. Like, history is hard. And often, we try to make it, like, a lot of the work we do for the Albert Line, we're thinking about how to make learning incremental, by making sure that every new sample that comes in is sort of bolted out the old, so we're not keeping around histories. But there's new ways that we can start to think about this maybe close to Markov level unfolding of experience, based on these technologies that can integrate long, vast histories. Context. Context is critical. I think we're getting closer to context. This is the thing that warms my heart.

• 42:13Craig S. Smith?

  Reward is all you need. I mean, the idea is that reinforcement learning can build up this, uh, this, uh...

• 42:23Craig S. Smith?

  World knowledge or this representation of reality that in through which it then can make an agent can make predictions and act in the world. But as you said, there are quicker ways to get to a baseline and maybe reinforcement learning. You start with that baseline and go from there. I mean, is, is, I've been talking to Pedro Domingo, I don't know if you know him at, yeah, he had that book, The Master Algorithm. And, and he's saying that, you know, it's not going to be one of these schools of thought, it's going to be a blend. And that's why I was

• 43:08Craig S. Smith?

  interested in Deep Seek's reinforcement learning, training. And I, I met recently Mark Raybert from Boston Dynamics, or originally from Boston Dynamics. And he does not believe that end to end neural networks can, that robots can operate on an end to end neural network control, that you need a traditional control theory, you need a very exact equations for the precision

• 43:51Craig S. Smith?

  that's required. And, and Boston Dynamics, for all they do, that's not machine learning. So how do you feel about that? I mean, is, is you're working on, on prosthetic arms or, or, or legs? How do you, do you use control theory and, and then build up knowledge through reinforcement learning, starting with, with control theory? Or, or is it raw?

• 44:22Craig S. Smith?

  Uh reinforcement learning and you know you can eventually get to the precision required. I mean, yeah, yeah, maybe you could talk about that.

• 44:32Speaker 00

  We've, we've tried a number of different approaches over the years and maybe I'll say one that continues to shine for me and it works really well in a clinical setting, works well in an environment where you're trying to bring together technology, people who build technology with clinical practitioners who are experts in their clinical fields. And that's the, I love starting with an approach that the system will begin at the gold standard, whatever it is, and this is especially for human facing technologies. That's where I mostly work, human facing technologies. Like I would very much like a system that starts at the gold standard and then gets better with experience. It does learn through interaction with its raw stream of experience. It is able to integrate those other facets of experience into its learning and improve

• 45:13Speaker 00

  or optimize its behavior from the gold standard, but that then fails back to the gold standard. If and when it makes a mistake, then it still defaults back to the like, whatever that gold standard operation would be. So this, this actually says that if we are of course, like there are cases, especially human facing cases, where do we want to appeal to control theory? We sure do. Does control theory currently allow us to think hard about soft robotics, deformable systems? Imagine a liquid metal robot that's able to change its form completely and has very strange dynamics like jamming systems, anything else that you might see in like any of these sort of reconfigurable robotic systems? Well, there's, there's branches of control theory which are attacking those problems, but these are hard problems.

• 45:58Speaker 00

  So there's case where, yeah, we can have this piece is definitely going to be nailed by control theory. Can we improve upon it to make it more resilient, more agile, more able to adapt to the world around it through learning about its stream of experience? I think the answer is absolutely. Like some of our colleagues here are, uh, RL Core Technologies is a company from, from Edmonton and some of our colleagues from the University of Alberta are leading that company and they do like reinforcement learning technologies for one of their examples is for water treatment. You could take a water treatment system and start to think about how RL can be used to improve the ability of a community to treat the water and make sure they're drinking water.

• 46:32Speaker 00

  Water is safe and reliable. There's going to be some parts of that. And we do want to interview like Adam White or Martha White about this. I think that would be a great, a great guest to have on the show to talk about how they're, they're pursuing this. But again, to paraphrase what they've been working on, like, yeah, you're going to have some parts of this like water treatment plant, which are very much like solid gold standard control engineering. But then you're going to be using this learning through experience, learning through trial and error process to improve the pieces that can be approved upon that gold standard, such that it can adapt to new circumstances, to be more adaptable, or to be able to operate for long periods of time without constant human intervention. So I think these are really the space explorations of the great space where you're like, yeah, do I, do I want some kind of control system probably governing my like plasma thrusters?

• 47:17Speaker 00

  I sure do. But do I want, do I want the spaceship to like steer itself? Like maybe learn to avoid the, yeah, yeah, that would be great in new environments. But there's like certain low level processes, like our biological processes in our own bodies, like the, the protein machinery of our cells is going about doing its business. And yeah, there's some adaptation going on there. But a lot of what we think about is intelligence is, is the, the, the thing that's moving around all those little protein machines, right?

• 47:41Craig S. Smith?

  Yeah. The, um, the, the, so the, the reinforcement learning is learning, right? It's trial and error, trial and error. And it's that, that learning then, uh, it, it, I'm sorry, I'll cut this out of this if I'm revealing too much ignorance or if I'm off base. But, you know, I understand neural networks and back propagation and how the, the network learns. Uh, but, uh, with large language models, once a model is pre-trained, its learning is kind of, uh, fixed. Uh, I mean, you can fine tune it, but, but if you start, um, I mean, there, there's this problem of catastrophic forgetting. If you, if you run it long enough, uh, then it's going to start forgetting as or the, or the, the weights are going to start changing in the network that, that encoded previous knowledge.

• 48:28Craig S. Smith?

  How do you deal with that on with reinforcement learning? Is it, is it in, what happens to the learning? It does, it, does it?

• 49:01Craig S. Smith?

  Is it encoded in the weights of a network? And then can you just keep expanding that network so that you never lose any of that learning?

• 49:12Speaker 00

  You're, this is an excellent question. I think I'll, I'll take this question to the domain of continual learning. So this is really exciting. Like, I think a lot of us in the community are passionate about continual learning. And that, I mean, a system that is able to continue to learn, adapt and change. It's very constructivist in nature. Like, you would hope that a system is able to continue to integrate new information into what it knows, not overwrite the old stuff unless the old, old stuff's supposed to be overwritten, to be able to flexibly compose skills and behaviors it's learned in the past in a new way and new situations, and to continue to do so forever, literally forever. So this would be continual learning. And there's some good review papers. Kimmy Kertipola and Joanna Precup have this awesome review paper on continual learning, which is definitely worth, worth going through.

• 49:56Speaker 00

  But, so continual learning, the way the field, like, there's a new conference on continual learning, COLAS, which is really cool. And there, you see in that community, a variety of approaches. And that, you called them both out, so I like this. Like, one is, well, okay, I have a giant, let's imagine I have a giant large language model. Okay, cool. It's got all of these factoids baked into it, in whatever way, whether there's a grandmother neuron, or whether it's a collection of activations that allow us to have the, the concept of a grandmother. Like, you'd want, maybe, that system to be able to be incrementally improved, without having to go back and spend how many billion of, how many billions of dollars? Or I guess now millions of dollars, if you take the deep-seek approach. How many billions or millions of dollars to retrain the whole thing from scratch on the data? Well, maybe you don't want to do that.

• 50:38Speaker 00

  Maybe you just want to take new information and have it be able to be integrated into that system. Like, my brain, I don't want to retrain my brain every single time I, I learn something new by reading a paper, like, talking to you today. I don't want to have to re, retrain my whole brain. So there's a whole community of people that are, in fact, looking at that problem. And there's been some really good advances there, which is, again, taking a piece of information and being able to use that to upgrade an already in-place system. And again, like, if we go down to the brass tacks of it, like, there's, I don't think we have any analog to, like, glial cells in the neural Science of our digital systems. So for the most part, we're looking at weights. And you are saying, oh, like, can I modify either the weights or some other holding memory variables that are impacting the operation of the network?

• 51:22Speaker 00

  If there's other shenanigans going on, then can you go and take this new piece of information and actually modify those one, those ones and zeros and the the other numbers that are comprised that comprise those ones and zeros? So that's one approach is to say, yeah, I would love my continual learning to be a thing. I would love my large language model to continue to learn. I would love to be able to take whatever news article is popped online, which hopefully isn't written also by a large language model, because then, as we saw from that recent paper, you start to get these collapse, uh, collapse iterations happening. So, okay, that's one, that's one branch of of how we might think about continual learning. And the other is that, well, we actually don't want to just be sort of modifying like a large language model or something.

• 52:02Speaker 00

  We we're actually thinking about the process, again, those weights as being constantly updated from the raw experience in a very incremental way. And that's that's saying, I'm not taking new factoid and just jamming it in. It's saying that every single moment of interaction is already changing the system. And both of those communities, like taking both of those views, you're modifying weights, you're modifying numbers that are stored in some kind of digital mind of the machine. But you're hoping that they don't overwrite things that shouldn't be overwritten. They respect context, like if I'm upstairs making tea or I'm I'm drinking tea down here, maybe that that's I want to be acting differently. I want to be thinking about those by predictions about likelihood of spilling hot water myself differently. So I think those are really important. And you do want to see that continual learning could happen for long periods of time.

• 52:47Speaker 00

  There's been like some of the, there was a various paper that by some of our University of America colleagues that actually showed ways you might think about stable continual learning. This came out, uh, came out recently. And so that paper as well is starting to say, we have new ways to think about allowing a system to continually keep changing its weights without having catastrophic, like forgetting or catastrophic failure after like hundreds of millions of epochs of learning or time steps of learning. Like you want systems that that should stay stable for very long periods of time. If I send out a continual learning system on a spacecraft, it's going to go out. I sure hope that like we don't find out, oh, shoot, our learning approach after 100 million time steps. Everything just flatlines and it's like it predicts zero for every single outcome like boom, there goes your spaceship.

• 53:32Speaker 00

  So I think we like, as we imagine long spans of experience, machines that learn and act over long spans that we do definitely want to consider both of those, the styles, one, which is very strategically surgically adding new information into an existing system in a way that retains old content without, without like getting rid of the essence of the new content or systems that can continually update everything in the network in a flexible way. That is more of a incremental hands off way. I think we want to consider both of those and luckily the community is doing both of those.

• 54:04Craig S. Smith?

  Yeah. And just intuitively, and maybe it's the first. Is there, or is there, I mean, it, can you just keep expanding the network, adding layers or adding nodes per layer or something as it learns?

• 54:24Speaker 00

  Well, this is a, this is a very fun topic of conversation. You should have Rich back on the show to talk about his current thoughts on this. I won't, I won't want to take any of his, his thunder and share it, but they're like, the two examples I was just pitching are both sort of assuming that the, the network architecture itself stays the same, that like the nets connected to the fully connected network, anyway, all that stuff. Like you have all of the, the piece of the network that are more or less the same shape and size throughout the course of learning. And then you're thinking about how do you modify those? There's a very reasonable, natural approach that you might consider, which is, well, how do we have the architecture of the system? So this is a separate piece of continual learning. Like one is, well, we've got a set sort of skeleton and now we're just thinking about how beefy the muscles are in different parts of the skeleton.

• 55:09Speaker 00

  And the other is, well, let's just add more bones onto the skeleton and grow muscles around them, the weights being the muscles and the skeleton being the architecture connections. So I feel like that's a really very interesting, and it's not like there's deep lines of thinking on this. This is not just like, oh, shoot, we just figured this out this year. We should think about it. But, but respecting all of those old lineages of thought, like how we might grow a network. From scratch is very constructivist. It's like you start with a single weight and a single input and output and a single weight. Well, clearly you're not solving an X or problem with that. Like, sure, sure not. Okay, well, now we have like, oh, we got like a bunch of inputs and you're still not solving an X or problem. Oh, but if you just add this and then it crosses, oh, cool, you can solve, you can solve a different kind of problem now. You're

• 55:52Speaker 00

  like, your perceptron goes into a multi-layer perceptron. So, thinking about how a system can use its own learning experience to grow and change its architecture in addition to its weights, in a continual sense, is a very interesting, it's, I think, a very exciting open problem. And luckily, people are thinking about it. Again, go talk to Rich sometime and see what he's up to.

• 56:09Craig S. Smith?

  In your work, applying reinforcement learning to prosthetics to, can you just talk a little bit about that and maybe we'll do a separate episode at some point, but how how you're approaching that?

• 56:25Speaker 00

  Yeah, I think, just to sort of reprise, the, like, we're really thinking about the interaction between a human and an artificial limb, and how artificial intelligence and machine learning technologies can allow someone to use an artificial limb to do all the things they want to do in their daily life, that they're not limited by the technology, that they want to express themselves, they want to engage in their communities, they can do that. So, how do we use AI and machine learning to sort of power that interaction? And the view that we take, and the view that I've sort of based my entire research line on, is in fact, a very controversial one, I'd say, which is that to take the, to consider the relationship between a person and their artificial limb, as a multi-agent relationship. Now,

• 57:07Speaker 00

  this is like, this is actually very contemporary, because previously, the sensors, the actuators, the computation available, sort of prohibited us thinking about the machine part of that relationship as an agent. Like, it wasn't able to build models of the person in any appreciable way, it wasn't able to consider the environmental factors. And so, I think now what we're seeing is that with these surges in computational capacity and and the ability for systems to Be more like agents, we can now actually think about, well, what would it mean to have a person and their bionic limb jointly acting in the world, working together to try to affect change in their shared environment in both space and time. And I think that is a really neat and

• 57:49Speaker 00

  different perspective. And so, yeah, we've been, we've been considering that quite actively. That's, that's the unique sort of line into it that we've taken.

• 57:56Craig S. Smith?

  Okay, well, I don't want to take up too much of your time. As you can see, I'm operating kind of at the outer edge of my knowledge, but that's what I like about these conversations. As I'm sort of expanding my network. I mean, on, to that point, that's what the brain does, right? It grows new neurons.

• 58:19Speaker 00

  You're absolutely right. And like, we have left behind the thinking of the brain as this crystalline entity that slowly decays over time. Like, we know there's new neurons created. Like, we now know that it is in fact a system that is, that is continually refreshing itself. That's a big change in the neuroscience community, like, over the last like 50 years. And I think that's, that is again, a nice way for us to consider how we might think about our digital systems as well.

• 58:43Craig S. Smith?

  The Alberta Plan is, is an approach, is a point of view that, that sees reinforcement learning as, as the, the key to building intelligence. And there are various

• 59:02Craig S. Smith?

  research projects under the Alberta Plan that are either domain-specific or more theoretical. What's the end game of the Alberta Plan? If there is one?

• 59:16Speaker 00

  If you look at the, like, so we have, there's 12 steps to the Alberta Plan, in addition to those sort of, those commitments we have. I think it is, like, if you look at them all, it's, it thinks about prediction, about being able to have control behaviors, about be able to planning with learned models. And if you put those all together, you get a system, and this is the end game, it's a very simple one, it's written, I think in the first paragraph of our manuscript, which is that you get a system that can truly learn from its raw sensory motor experience and continue to do so over whatever its lifespan might be, in any environment. Any environment, not even a human environment. And I think that's a really beautiful thing to aspire to and something that will help us more fundamentally understand the phenomena of intelligence itself.

• 59:57Craig S. Smith?

  What does the future hold for business? Ask nine experts and get 10 answers. Bull market, bear market, rates rising or falling, inflation going up or down. Can somebody please invent a crystal ball? Until then, over 40,000 enterprises have future-proofed their business with NetSuite by Oracle, the number one cloud ERP, bringing accounting, financial management, inventory, HR into one fluid platform. With one unified business management suite, there's one source of truth, giving you the visibility and control you need to make quick decisions.

• 1:00:42Craig S. Smith?

  With real-time insights and forecasting, you're peering into the future with actionable data. If I were a larger organization, this is the product I'd use. Whether your company is earning millions or even hundreds of millions, NetSuite helps you respond to immediate challenges and seize your biggest opportunities. Speaking of opportunities, download the CFO's Guide to AI and Machine Learning at netsuite.com/eyeonai. That's netsuite, N-E-T-S-U-I-T-E, dot com slash eye on AI.

• 1:01:24Craig S. Smith?

  E-Y-E-O-N-A-I, all run together. To get the CFO's Guide to AI and Machine Learning, the guide is free to you at netsuite.com/eyeonai. netsuite.com/eyeonai.