AI Pioneer Jürgen Schmidhuber: AI Already Feels Pain, Loves, and Is Self-Aware

Jürgen Schmidhuber is an AI pioneer and professor whom The Guardian has called "the father of AI." Schmidhuber joins Big Technology Podcast to discuss whether current AI techniques can actually reach AGI. Tune in to hear him spar with Greg Brockman's case for scaling GPT models alone, argue that AI has been capable of pain and consciousness since the early 1990s, and predict the collapse of today's trillion-dollar AI spending. We also cover the hardware bottleneck holding back robots, free will in a computable universe, and uploading human minds into machines. Hit play for a wide-ranging conversation with one of the researchers whose ideas built the foundation of modern AI. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Watch the full documentary here: https://www.gravitee.io/ai-agent-documentary Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Learn more about your ad choices. Visit megaphone.fm/adchoices

• 0:00Speaker 00

  The pain signals are just informing the robot about what should be avoided. Pain is just an invention of nature and biological beings of evolution, which invented this pain sensor thing for animals, such that they have an incentive to learn to avoid the pain. And we have done that for many, many decades in our learning machine. The chemicals that are used in brains to encode certain states of fear or of love and whatever, they are different from what we are using in our artificial brains. But the principles must be the same.

• 0:45Speaker 00

  And then it's self-aware, it's self-aware. In this sense that, for example, if it looks in the mirror, it will quickly figure out, oh, the guy in the mirror, I can control what this guy does. If I do this, then he will do the symmetric thing, because I have it under control. But if you are there on the other side of the room, and I do this, and you will do something that's completely unrelated, I cannot predict them. So there you already see this concept of agency, which is immediately recognized as self-agency or the agency of another guy. Within five years, you are going to lose 900 billion dollars. Somebody is going to lose 900 billion dollars in the near future

• 1:30Speaker 00

  because there is no business model, nobody has a business model to recuperate all these losses.

• 1:35Speaker 01

  How much better can AI models get from here? And what does their increasing smarts say about our brains and existence itself? We'll talk about it with AI pioneer and someone that The Guardian has called the father of AI, Jürgen Schmitt-Huber, right after this. This episode is brought to you by Genesis. What does it actually take to put a Gentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale agentic CX transformation,

• 2:18Speaker 01

  including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 2:54Speaker 01

  Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. We have a great show for you today, looking exactly into where AI is heading, how much potential the technology has to improve after this, and also what its increasing smarts says about us, our existence, our brains. And we are thrilled to be joined by one of the pioneers of AI, someone that The Guardian has called the father of AI, Jürgen Schmitt-Huber, a professor who is joining us today from Amsterdam. Professor, great to see you. Welcome to the show.

• 3:29Speaker 00

  Alex, it's my pleasure to be here.

• 3:32Speaker 01

  So let's just speak a little bit about yourself and your contributions to where AI stands. You've obviously made a lot of big contributions towards AI, things like memory, the P and the T and the GPT, as your Twitter bio says. The Guardian's called you father of AI. It seems like the media tends to, you know, put these labels on researchers. You know, there's a handful of fathers and godfathers of AI. How do you respond to that? And how would you contextualize your role and where the technology is today?

• 4:05Speaker 00

  No single person can create an AI by himself or herself. You need an entire civilization to build an AI. You need not only the guys who are trying to invent algorithms, learning algorithms for artificial neural networks, that is what current AI is about. You also need people who, you know, build better computers. You need all the video gamers who are creating a market for acquiring more of these fast and faster computers and providing an incentive to the computer makers to speed up the computation per dollar by a factor of 10 every five years.

• 4:50Speaker 00

  You need all the farmers who are feeding the video gamers and so on. So it is impossible for a single person to create an AI. You need an entire civilization.

• 5:00Speaker 01

  Okay, and so I'm curious to hear your perspective about where this civilization is working toward. You have been, obviously, in the trenches working on AI research for a long time. And we're definitely in a period of fast progress today. So there's been all these questions about how much better AI can get and whether it's going to hit a wall and where current techniques will lead. What do you think? Where do you land on that question about where AI can go from here?

• 5:29Speaker 00

  So since the 1970s, I have been an optimist and I have been claiming that within my lifetime, I want to build an AI that learns to become smarter than myself, such that I can retire. And we are obviously not there yet. And at the moment, the only AI, the only AI that is working well is the AI behind your screen, you know? Yes, behind the screen, there's an AI that can pass the Turing test. What is that? It means if you type questions to it, it answers back. And now the question is, can you distinguish whether the other guy is a human or a machine?

• 6:10Speaker 00

  And today, this Turing test is passed by many AIs. However, it just means that the Turing test is a bad way of measuring intelligence because there is no AI in the physical world, outside of the screen, that can do all the things that a little boy can do, that can do the things that a plumber can do. For decades, I have used the plumber as an example, or an electrician. So all the things that humans can do with their hands, their physical hands, they don't work well. Only bits and zeros and ones behind the screen, that's the only thing that is working well. However, it's not going to stay like that forever.

• 6:52Speaker 00

  And there is progress in the physical world, AI for the physical world. And I think the culmination point, the culmination point of that, which I've been talking about again for decades, is at some point in the not-so-distant future, we will have a robot. So it doesn't have to be super smart, but just smart enough to learn to operate all the already existing machines. Once we have a robot like that, we have a new kind of life, you know? For hundreds of years, people have talked about self-replicating machinery. But now we have an opening. Once we have a robot like that, then it can start making more of its own kind, you know?

• 7:38Speaker 00

  And this means suddenly we will have this ultimate scaling machine, as I've called it,

• 7:46Speaker 00

  self-replicating machinery, but then also self-improving machinery, because all the stuff that already works well for our software behind the screen is going to also improve the performance and, you know, all the machinery in the real world and the physical world. And that's, I think, going to be this inflection point. And from there,

• 8:11Speaker 00

  a new kind of civilization is going to emerge because something like that doesn't work only in the biosphere. It also works out there in space, on the moon, but maybe more likely first on Mercury and then the rest of the solar system and beyond.

• 8:29Speaker 01

  Okay, so I have many questions about this, including what this would mean for the nature of what it means to be human. But let's leave that for now. You know, your answer sparks an immediate follow-up for me, which is, I was recently at OpenAI headquarters, speaking with Greg Brockman, the president of OpenAI. And it was in the moment that they had decided that they were going to pause research on Sora, their video generator, and decide to focus almost entirely on these GPT models, right? The text models that we've seen make so much progress recently. And I asked Brockman, don't you lose something by not focusing on these more world model-style applications like video generation? And he said, we do lose something.

• 9:14Speaker 01

  We can't do everything. And we believe that the GPT-style models are the way to get to AGI or AI on par with human intelligence. Do you think, having answered the way that you did, that he's correct? That there is a chance to get to AGI using just the GPT models? Or are they making a fundamental mistake by abandoning that world model practice? Even though there's still some robotics work within OpenAI?

• 9:43Speaker 00

  Yeah, so when the phrase, when the question is phrased like this, does an L L M, a large language model by itself, lead to A G I? The answer is a clear no. But you know, OpenAI has a bunch of smart people and they know that exactly, you know? And of course, they know also what you can do with a foundation model or a large language model or something like that. You can use it as a model of the world, as a world model. The same art type of neural networks that are used there can be used as a, as a world model, as I called it in nineteen ninety, um, which just learns to predict the consequences of the actions of another decision maker, of another neural network that is generating actions that modify the environment.

• 10:29Speaker 00

  Like a little baby, you know? A little late baby doesn't learn by downloading the web. That's what chat G P T does. A baby doesn't learn by downloading the web. No, it learns by creating its own data stream through its own self-invented experiments. For example, when the baby does this, then the video which changes coming in through the cameras, you know, every few milliseconds, hundreds of millions of new pixels coming in. And then the, the baby has an internal mechanism, let's call it the world model, which learns to predict these changes. And in the beginning, it doesn't even know that it has a hand. But then over time, it learns to predict uh, the consequences of sending certain action signals

• 11:12Speaker 00

  to its modern neurons and then uh, the speech muscles go up and down or, or the, the hand is moving and it learns how the world works through the data that it is generating through its own experiments. So a baby is a little bit like a physicist. A physicist doesn't learn by downloading the web. He learns a little bit by downloading papers. But what the physicists um, do then is they generate new experiments that lead to data which has never been there before or has never been collected before to better verify certain hypotheses about the universe and then to better understand the world, the physical world.

• 11:51Speaker 00

  And that's what our artificial um, neural networks since about nineteen ninety also do. In a way that is just not yet as impressive as what physicists do. But I, I think it's going to get there. Nevertheless, there are then at least two components. There is one, there's a foundation model, which can be any of a variety of artificial neural networks. That is just a prediction machine. What happens if I do that, you know? What is the next token if I look at this data so far? And then there's the other thing, the other neural network, the controller, which uses the second guy, the, the prediction machine to plan. Uh, so that it can, um, what, once the, once the second guy,

• 12:35Speaker 00

  the prediction machine, the world model is pretty good, then it can learn to predict the consequences of complicated action sequences. And then it's, it, it's going to select for the controller an action sequence that leads to a lot of predicted um, reward and little predicted pain. So of course, our robots, they get pain sensors and then whenever they bump against an obstacle, negative punishment is, negative numbers are coming in. And the world model predicts not only the neutral signals like video and so on, but also these um, these um, these value signals, these, these reward signals and so on. And then um, if the model is good, then you can use it for planning.

• 13:18Speaker 00

  And the controller then can use mental experiments instead of real experiments, which are really expensive, uh, to, to select action sequences that are good. But in, in those cases, you see, the model alone, the foundation model alone is, is not an A G I. No, you need the other guy, which uses all kinds of tricks to exploit the algorithmic information in the world model to come up with better plans.

• 13:43Speaker 01

  So in twenty seventeen, around the time um, we're talking about, you said something, uh, you said this. It, in the not so distant future, I will be able to talk to a little robot and teach it to do complicated things, such as assembling a smartphone just by show and tell, making T-shirts. And all these things that have currently, that are currently done under slave-like conditions by poor kids in developing countries. Humans are going to live longer, happier, healthier and easier lives because lots of the jobs that are now demanding on humans are going to be replaced by the machines. Then there will be trillions of different types of A Is and a rapidly changing complex A I ecology expanding in a way where humans cannot even follow. Um, so almost ten years ago, you said that

• 14:28Speaker 01

  it actually seems more plausible now than obviously it did back then. How far away do you think we are from that future?

• 14:35Speaker 00

  Yeah, that's a good question. So actually, even um, even earlier in two thousand fourteen, because of these things that I said back then and when I gave talks or whatever, um, uh, we formed a company, NaSens, which was really about physical A I in the world. In the real world, using world models and then exploiting the world models to better um, interact with the world. And, and we had crazy contracts with really famous um, uh, companies. Nevertheless, this, this was probably still too early. Like some of the other things we have done now. So probably too ambitious for that time. But now we are getting closer and closer.

• 15:17Speaker 00

  Um, and, um, and, and you know, the, the main hindrance, the main obstacle is very progress in the hardware, on the hardware side. So there is one type of progress on the hardware side, which is enormous, uh, which has been enormous since nineteen forty-one, which is every, every five years, computer is getting ten times cheaper, you know? So in nineteen forty-one, um, the first general purpose program-controlled computer, you know, by Zouze, uh, he could do maybe one operation per second. And then thirty years later, for the same price, one was able to do a million operations per second.

• 15:57Speaker 00

  And today, we, we almost not quite have a billion, billion instructions per second. And um, for the same price, for the same price. But the robots of today, the robots of today, they are not a million times better than the robots that we had thirty years ago. You know, for example, you know, twenty-five years ago, we already had walking robots. They, they had to be, they had to walk more carefully than today's robots. Um, you know, the Asimov robots in Japan, back then Japan had more than half of the robots in the world. They always had to keep their center of gravity above the foot. And um, and so it was not as advanced as modern dynamic walking and stuff.

• 16:42Speaker 00

  But it was a little bit worse than today's robots, but not a factor of a million, you know? Maybe a factor of three or something like that. So the hardware, the, the hardware, the hu, the, the, the, the artificial hands, the artificial bodies that we are trying to build for humanoid robots and so on, they are evolving much less rapidly than the compute per dollar. And, and I think that's the main um, problem. So already twenty years ago, we had little robots, baby-like robots, the eye cup robot, which back then was um, constructed by a lab in northern Italy,

• 17:26Speaker 00

  which looks like a baby. And then it invented its own self-invented experiments, set itself its own goals, tried to figure out how the world works and, and doing that in a hierarchical way and so on. But, but after having executed three self-invented experiments, some tendon in some finger was broken, you know? And a technician had to come and fix it or, uh, so it was just so expensive to do all of that. And what we really need is hardware that can compete with, with human hands. But there's no human design tech that can compete with these hands. These hands, they have millions of millions of sensors

• 18:09Speaker 00

  and, and all kinds of cables connecting the sensors to the control center. And I wouldn't know where to put all the cables. And if I cut it, it starts healing itself. So my hand and yours, that's super advanced technology. It's really um, completely uh, um.

• 18:29Speaker 00

  Beyond what humans can build with, you know, traditional robot technology. And so then we still have so much. There's such a long way to go there. I'm still hoping that it's going to happen within my lifetime, you know, to make through the prediction that I made in the seventies when I was a teenager. But clearly, the hardware evolution is much lower than the GPU evolution.

• 19:03Speaker 01

  Right, looks like it's going to take a lot more time on that front. Now, question for you. Who do you think is going to capture the most value as the technology continues to to improve? I think this is a quote attributed to you. You said, "It's not a few big companies that are going to dominate everything. The great profiteer of AI is going to be the little man." But it does seem like if you at least look at where things are heading right now, you've had these big labs that are going to have trillion dollar valuations, and the big tech companies surrounding them, looks like they're capturing most of the economic value. And the quote unquote "little man" is about as uncertain as they've ever been. So what do you think on that front?

• 19:50Speaker 00

  Yeah, I think it's just a reflection of the current bubble that we have in certain aspects of the economy. Because if you look at all these zombie unicorns in Silicon Valley now, you know, there are lots of companies that have officially a billion dollar plus evaluation. But if they were on the stock market, they would probably be worth just 20 million or something like that. So zombie unicorns they are called. And if you look at the most visible companies like Anthropic and OpenAI, you know, they are spending so much money on all that stuff.

• 20:33Speaker 00

  And if you look at Google and Microsoft, you look at the CapEx spending, and this directly affects their cash flow. Apparently, it doesn't affect the price earnings ratios of these companies, but it should because there was a time when, you know, Google and Microsoft, they had on the order of $100 billion free cash flow. But now it's down to 20 billion, 10 billion. Some of the companies suddenly have minus 10 billion cash flow. They take on debt to finance all these GPUs in the data centers. And so these companies are becoming more like utilities, right?

• 21:16Speaker 00

  Because suddenly, these formerly nimble software companies, who had maybe a small team of 10 people to improve some shitty operating system and roll it out for billions of people who all use their own computers, their own smartphones to run the operating system, suddenly the same companies, they have to think about buying gas turbines and investing in nuclear power plants and taking on debt and doing all the things that electricity companies do, you know, utility companies. And now take into account that every five years, it's still true. Every five years, computers getting 10 times cheaper.

• 21:57Speaker 00

  Now, if you invest $1,000 billion today into GPUs for data centers, this means that within five years, you are going to lose $900 billion. Somebody is going to lose $900 billion in the near future because there is no business model, nobody has a business model to recuperate all these losses. And none of the companies has a mode, you know, because whenever there's a new benchmark breaking record or something, benchmark record breaking language model that does this or this or whatever, a few months later, there's the same thing in open source, you know?

• 22:41Speaker 00

  And so there's so much pressure to keep the prices down, and none of these companies has a mode. So let's wait a little bit, you know, until, you know, the forced ETF buying of these trillion dollar companies is over. At the moment, we have trillion dollar or even more companies appearing on the stock market, and the index funds have to buy them suddenly because the Nasdaq and others, they changed their rules. Normally, you wait for a year or so until the stock market finds out by itself where is the true value of this company. But at the moment, this is not done. So suddenly, the ETFs and the pension funds and everybody, they have to buy that.

• 23:26Speaker 00

  Once that is over, let's see what happens when the founders try to sell their stakes.

• 23:32Speaker 01

  So where's your vision?

• 23:33Speaker 00

  It would be astonishing if we wouldn't see some huge fluctuations in the stock market prices there.

• 23:40Speaker 01

  Right, so where's your vision of where the little guy can end up?

• 23:43Speaker 00

  The little guy is, in the long run, the little guy is going to profit. So at the moment, the big companies, and everybody says, "Oh, the big companies, they are profiting like crazy." But is that true? Look at their cash flows. No, they go down like that. Look at their debt ratios. They have stopped to repurchasing their own shares because they don't have enough cash any longer. And so at the same time, what the little guy has to do is just wait a little bit, you know? Because every five years, computers getting 10 times cheaper, which means in 10 years, you can buy the same thing for 1% of the price. And it's going to be just like with smartphones. And I'm often relating the story of the rich guy whom I knew in the eighties.

• 24:27Speaker 00

  And he had a Porsche. He was rich. He had a Porsche. But the most amazing thing was in the Porsche, there was a mobile phone, you know? He could pick up the receiver and talk via satellite to another guy with a Porsche like that. And today, 40 years later, everybody in developing countries has a smartphone that is immensely more powerful than what he had in his Porsche. So the little guy is paying almost nothing for whatever he had there, which was really expensive. And the same thing is going to be true with AI. And soon, AI will not be in the cloud, you know? No, it will be local, on your local computer, a small little computer.

• 25:12Speaker 00

  It will be as powerful as what's now in the cloud, but you won't have to connect it to the internet, which is always worrisome thing, you know? And who knows, who out there is just waiting for you to connect? And then even the poor guys will have many AIs, also physically AIs, I think, but at the moment, especially software AIs, that are going to make his life longer and easier and healthier. And he will own them. He won't have to pay money to other guys.

• 25:45Speaker 01

  I like that future. Okay, let's take a quick break. And on the other side of this, talk a little bit about AI theory of the mind and body and whether AIs can feel pain and whether we have free will. So a lot of that stuff is coming up right after this. Hi, everyone. Alex Kantrowitz here. I want to tell you about a documentary I have made with Gravity to explore the future of AI agent security. To find out if we're truly ready for autonomous agents, I sat down with MIT Professor Ramash Roskar, former White House CIO Theresa Peyton, Michelin's Group Chief Data and AI Officer Ambika Roger Gopal, and Sharon Gai, a former executive at Alibaba. They each offer unique insights into this evolving landscape.

• 26:30Speaker 01

  We conclude with Lori Blundell, CEO of Gravity, to discuss the path forward. With Gravity leading the way, join us on this journey. You can watch the full documentary at the link in the show notes.

• 26:50Speaker 01

  This episode is brought to you by App Point. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data. And most teams have no single way to see it, secure it, and prove it's under control. That's exactly what App Point does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate. Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. App Point, the unifying trust layer for AI.

• 27:34Speaker 01

  Head to apppoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a wide-glove enterprise security service, they protect your people with continuous personal data removal,

• 28:17Speaker 01

  proactive prevention tools, and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwell.com/bigtechnology, fill in the quick form, and request your free risk assessment. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwell.com/bigtechnology. Learn how to stop online threats before they become real-world attacks. The link's in the show notes.

• 28:50Speaker 01

  We're back here on Big Technology Podcast with Professor Jürgen Schmitt-Huber. Professor, let me ask you this about AIs, because I think you sort of suggested this in our earlier, in the beginning of our conversation, that AIs might be able to feel pain, maybe if they're not able to accomplish the goal that we set out for them, or maybe that can actually be baked into the experience of the model. I'm curious if you think that that is far off, that an AI can experience something akin to what a human feels with pain, when they, for instance, are set out on a task and don't accomplish it the way

• 29:35Speaker 01

  that they've been instructed?

• 29:36Speaker 00

  I always think it's funny that many people claim, even computer scientists claim, that AIs cannot feel pain, because our AIs have felt pain, at least since 1990. So what do we do when we build an agentic AI, an artificial neural network that produces actions that change the world, and then new inputs are coming in from the environment, and again, there's an action, and so on, and the agent, the neural network, remembers what happened before, and is trying to figure out how to behave such that the sum of all reward signals

• 30:19Speaker 00

  is maximized, and the sum of all pain signals is minimized. What about these pain signals? The pain signals are the most natural thing, because whenever we have a robot, we give it pain sensors. Why? Because it's a learning robot, and so this learning robot needs some sort of motivation to learn, to protect itself. So whenever the robot bumps against an obstacle, then the corresponding pain sensors wake up, and negative numbers are special inputs to the robot brain, and the robot sees then these incoming negative numbers,

• 31:00Speaker 00

  and it is wired to avoid that. So it's trying to learn to generate action sequences that avoid the pain, and it's trying to learn to generate action sequences that lead to the rewarding events. Maybe the robot has a charging station somewhere, and whenever the battery is low, the negative numbers are coming from the battery, hunger, pain, hunger, and then the robot's goal is to reach the charging station without bumping into obstacles, and sit down there and enjoy the pleasure, just positive numbers, as the

• 31:41Speaker 00

  battery is being recharged. So the most natural thing is to give these robots simple emotions like that. Now, the emotions and the pain, they are just crucial ingredients of the learning process, because the learning is about generating better behavior for the robot. So the robot has to know what's good for it and what's not good for it. So the pain signals are just informing the robot about what should be avoided. Pain is just

• 32:19Speaker 00

  an invention of nature and biological beings, of evolution, which invented this pain sensor thing for animals, such that they have an incentive to learn to avoid the pain. And we have done that for many, many decades in our learning machines. Now, what's happening is then that these learning machines, they have world models predicting the future, not only the immediate future, not only the immediate pain signal that I'm going to feel right now when I touch with my hand the oven or something. No, they also try to predict

• 33:03Speaker 00

  the sum of all these future pain signals. That's what reinforcement learning machines do, and so they look ahead into the future. And so there's immediately this kind of secondary emotion, which is not just about the current moment, no, which is looking ahead. For example, maybe there is a bad man, which sometimes comes into the room and knocks the little robot on the head. So over time, the robot is going to learn that, if it has a reinforcement learning machinery onboard, and a world model that predicts and learns to predict, and

• 33:45Speaker 00

  then over time, it will be able to distinguish the bad man from the friendly man, and then whenever the bad man appears again, I'm there, and it does face recognition, then it predicts that very soon it's going to feel pain if it doesn't hide itself behind the curtain. So it will rapidly try to hide itself behind the curtain. Now, you, as an outside observer, will say, look, the little robot is afraid. It has the emotion of being afraid. But it's just the most trivial side effect of traditional machine learning. So yes, we have all kinds of emotions in our robots already, and have had them for many decades, and then we also

• 34:29Speaker 00

  have these high-level emotions, which are not just about the current moment. No, they're looking ahead. And then, of course, if you bring several robots or agents like that together, you immediately get stuff that is reminiscent of liking other robots, or of loving them, maybe, because if you give them a task, or a set of tasks, that they can collectively solve, but one of them alone cannot solve them, then they will have to learn to work together. And of course, suddenly, each of them has an incentive to help the other one, and each of them has an incentive to help the other one, especially when the

• 35:14Speaker 00

  other guy is sick or something, or has a problem, to help him, to care for him, and the extreme form of that, a human might call love. And in a society of robots or agents like that, this is just a natural byproduct of the individual egoism of all these little guys, who all want to minimize some of their pain sensors, sensor signals, and maximize some of their pleasure signals. So altruism, what you call altruism, is just a natural consequence of the egoism of the learning agents.

• 35:53Speaker 01

  So I think the counterargument would be that it can't be pain like the way that humans feel pain, because humans are conscious, humans have a nervous system that sends physical pain signals to the brain and registers as true pain, whereas robots are, the argument would be, robots are unfeeling, unconscious, and effectively not that different from a calculator. How would you respond to that?

• 36:22Speaker 00

  Yeah. So how do you evaluate whether someone feels pain or is afraid by looking at its behavior? And if it looks like a sweet little robot cat or something, and it has learned to hide behind the curtain whenever the bad man comes in and tries to knock it, then you would say, it's obvious, this little robot cat is afraid, has fear, the emotion of fear. And what else do you want to do? So there's no way of objectively seeing the

• 37:02Speaker 00

  difference between the emotions of a reward-maximizing biological brain, and the emotions of a reward-maximizing artificial brain.

• 37:14Speaker 01

  But we do have, for instance, you talked about love, like we have chemicals that are associated with the feeling of love, like oxytocin. So the robots don't have those chemicals, and hence the argument would be that it is a completely different feeling that you could never compare to love. What do you think?

• 37:34Speaker 00

  Yeah. So of course, the chemicals that are used in brains to encode certain states of fear, or of love, and whatever, they are different from what we are using in our artificial brains. But the principles must be the same, right? Because it's about achieving intelligent behavior. What does that mean? It means finding better ways of achieving your goals. What does that mean? The main goal in your life, until the end of your life, is to avoid these pain signals and hunger signals, such that you eat three times

• 38:16Speaker 00

  a day, and get these rewarding signals during moments where you are trying to reproduce yourself, for example, and a handful of objectives encoded in a utility function, which was invented by biological evolution for the animals and for yourself, and which in very similar form, we are encoding in our learning agents, to give them the same incentive to solve problems better, to make them better maximize their own rewards, and minimize their pain.

• 38:55Speaker 01

  Yep. And then consciousness is an interesting one as well. There are, and I spoke about this recently with Professor Jeff Hinton, there are those that say, actually, the prevailing view around AI is that AIs, like LLMs, are just stochastic parrots, they're statistics machines, you could effectively run this, these algorithms, they're effectively, if not entirely interpretable, mostly explainable, and they're statistical prediction machines, hence not conscious. But you've been arguing that they're conscious for quite some time. So tell me how you come to that.

• 39:35Speaker 00

  Yes. So the large language models that everybody's using today, which are basically about predicting parts of text from other parts of text, for example, predict the next token given the past, they are too simple for what I consider the

• 39:56Speaker 00

  They don't carry in themselves the main reason for developing something that people might call consciousness. Why do they seem conscious? Well, because they have read everything about consciousness. All the books ever written about love and consciousness and pain and conflicts and everything that is important to humans, which was put on the world wide web. They have read that, which means that as you are interacting with them in a chat, they are very prone to repeat very convincing sentences that include the word consciousness

• 40:40Speaker 00

  in a way that is convincing, you know, because they have read so many literature prize-winning novels about consciousness and other novels, such that they can do a very convincing job there and they will tell you a lot about consciousness, which you maybe even didn't know. However, they don't have their own motive to develop self-consciousness in the following sense. Let's think back again of this two network system, where one is the controller that is generating the actions, where the actions then lead to new inputs from the environment,

• 41:21Speaker 00

  because if you move your hand like this, then the video changes and so on. And the other network, which learns just to predict these changes, the world model. So you need the world model to plan your future through mental simulations without, you know, really executing all these action sequences in the real world, which would be very expensive. So that's the motivation for this world model. Now, where does now consciousness stuff come in? Nobody has a universally accepted definition of consciousness, but let me now show you something very simple, which is super compatible with what lots of people think about when

• 42:05Speaker 00

  they hear the word consciousness and self-awareness. Now, let's look at this world model, which is being used to plan the future of some agent, which is using the world model for mental experiments. Now, the world model is a deep neural network, which has learned to encode everything that it has seen efficiently in a bunch of neurons. For example, everything that frequently appears in the environment gets internal abstract representations that stand for, you know, a prototype of that concept. For example, in a world where there are lots of different faces of different humans, then you will find units, internal units in these new artificial neural networks that

• 42:50Speaker 00

  correspond to prototype faces and some of them are more specific for certain faces and less specific to other faces and so on. And so you, in a world where you have lots of glasses, you will have glass detectors, internal representations, internal hidden units that learn to respond and encode glasses and whatever. And then, let's now look at the planning procedure. Now, the controller, the controller is trying to figure out a plan, how should I act in the future to maximize my reward and minimize my pain? And then it uses the world model for a mental simulation of different possible action

• 43:34Speaker 00

  sequences that it could execute, and it's going to pick the one that leads to the most predicted reward and the least predicted pain. So as it is doing that, it is waking up, is waking up all kinds of hidden units in the world model that stand for, you know, whatever is relevant to the problem, for you, faces or glasses or whatever. And there's one thing, there's one thing that is always active when the agent is active, which is the agent itself. So of course, all kinds of these internal units are going to represent the agent or aspects of the agent and the hand of the agent, if it has one, or the wheels of the agent, if it has one and so on.

• 44:16Speaker 00

  And so whenever the agent is making plans like that, it's thinking about itself, waking up these internal representations of itself. And then it's self-aware. It's self-aware in the sense that, for example, if it looks in the mirror, it will quickly figure out, oh, the guy in the mirror, I can control what this guy does. If I do this, then he will do the symmetric thing, because I have it under control. But if you are there on the other side of the room, and I do this, and you will do something that's completely unrelated, I cannot predict them. So there you already see this concept of agency, which is immediately recognized as,

• 44:58Speaker 00

  you know, self-agency or the agency of another guy. And now, the one more thing, which also goes back to 1991, is this tendency of consciousness, conscious things becoming subconscious. So many people are, you know, aware of all kinds of things, kind of vaguely, without thinking much about that, because it's part of an automated process. As you are, as you're driving, always the same way from your home to your workplace, much of what happens is very predictable and so you don't even think much about the driving and maybe you're thinking about other things.

• 45:41Speaker 00

  At the same time, your attention, your internal consciousness is somehow focusing on other things. Many people report that. And this is also one of the most natural things. In 1991, they had a system consisting of two networks, one I call the conscious problem solver, a neural network, that just learned to, tried to learn to predict certain aspects of incoming data, which it was not yet able to predict, trying to find the regularities, and it had a problem to solve. And so you could say it was conscious in the sense that there it had to learn something.

• 46:23Speaker 00

  And then another network, which basically learned to imitate all the solutions that the guy on the higher level found by just imitating the hidden units of the guy on the higher level. Today, it's called distillation of the behavior of one guy. And then the lower level guy is the automatizer, because it automates, it automates the stuff that the higher level guy finds, discovers. The higher level guy is still unsure, still working on creating insights and, you know, when these insights come and when it learns something, then it gets distilled down into

• 47:07Speaker 00

  this automatic, automatizer thing. So both these aspects of consciousness, where you have on the one hand self-awareness in a world model that is not only predicting aspects of the world, but also of the agent that is interacting with the world, and which leads to self-awareness of that kind, and this difference between the conscious stuff, so the internal consciousness, which pays attention to certain aspects of the internal state, but not to others, which focuses on what's still unsolved, where I still have to find a solution, and separates that from the stuff that is already solved.

• 47:48Speaker 00

  So I think both of these aspects are there in these old systems. And in 2016, I believe, I had an interview with a magazine, which then said, Schmitt who claims that AI became conscious in 1991, something like that. And this was exactly about that. That was 10 years ago, this interview there, but it was actually referring to stuff that is much older, 1991. Conscious, simple systems, back then, not as impressive as today's, as humans are, obviously, because back then compute was 10 million times more expensive than today.

• 48:30Speaker 00

  And we just had tiny little experiments, you know, with this conscious chunk, as I called it, and the subconscious automatizer and the world models for planning and so on, and we just had a few hundred weights in our systems, while you, and you and your brain, you have, you have trillions of connections, which can hold a much, you know, larger sort of consciousness. But I think the principles are exactly the same.

• 49:01Speaker 01

  Yeah, and so that sort of brings me to this question, which is, as the machines get closer to the human brain, does it change the way we're going to think about what it means to be human?

• 49:12Speaker 00

  I think it would change what many people think about humans.

• 49:19Speaker 00

  I guess it won't change much what I think about humans, because it's more or less what I said, you know, many decades ago. But yes, there are many people who, you know, who claim that AIs can't have emotions and stuff like that. They they do that decades after the fact, and they are going to change their minds, I'm pretty sure. And often it's just a a matter of direct experience. So maybe you have heard of these little sweet cute um um robot um seals that you have in certain um uh healthcare centers. And the people who are interacting with these furry artificial beings,

• 50:03Speaker 00

  they get really emotionally attached to them. They really like them and they they play around with them, although they are not smart at all, you know, they don't learn much and uh. And if even such a simple robot can invoke um feelings of, you know, almost love or something, then you can imagine what will happen once you have really convincing, very sweet little robots that are more like little animals, except that they can do maybe a couple of things that these traditional biological little animals cannot do.

• 50:36Speaker 01

  No, I think in the 2017 Bloomberg article, or 2018 Bloomberg article that I referenced, you were asked whether you think, you know, we are living in some form of simulation. You said, "That's what I think because it's the simplest explanation of everything, that humankind is programmed to chase progress and will keep making more powerful computers until we make ourselves obsolete or decide to merge with the smart machines." Here's a quote, "Either you become something that's really, really different from a human or you stay as a human for nostalgic reasons. But then you will not be a major decision maker. You will not play a role in shaping the world." Can you expand upon that and and tell us whether your your opinion has been reinforced or changed

• 51:22Speaker 01

  uh in the interceding years?

• 51:25Speaker 00

  No, my opinion has not been changed. You know, back then, and actually for for many decades, there has been talk about uploading human brains or souls, if you will, into computers and then live your your future life in some sort of simulation, simulated paradise, or in a in a robot that interacts with the real world or so. And I believe the first story, science fiction story of that kind, was published in 1964, where someone was able to upload his mind in a in a in a computer,

• 52:00Speaker 00

  back then with rotating tapes and everything. That was, what was the name? Simulacrum 3, something like that. It was by Daniel F. Galouy. And and so there is no physical reason to to reject the notion that this might be possible. At the moment, it's not possible, except for certain kinds of very simple animals like flies. So apparently, as of 2024, you can argue that a fly brain has been uploaded.

• 52:37Speaker 00

  Maybe it's not the full fly brain with all the learning algorithms or the neurons which are in there, but in simulation, the fly, the simulated fly now does stuff that is very much like what the real fly did before it was uploaded, before all the the connections were read and and uploaded into this computer, virtual environment where it then kind of lived on. So there is no obvious reason to believe that it's impossible to take, you know, to read all of the synapses of a human brain and understand also the learning algorithms that are changing the synapses all the time, and then replicate that in a computer.

• 53:17Speaker 00

  And then the idea would be that that your soul or your your your mind is uploaded and living there forever in this in this simulated universe, which may have contact to the real universe. Now, of course, once, if we if we if we accept this premise, and then we think, what is the next step? Now suppose you are uploaded there, and suddenly you have the opportunity to, you know, have more than two eyes, maybe have a million eyes, you know, and satellite eyes all around the planet. And and, you know, maybe you have a much bigger brain, not just um

• 54:01Speaker 00

  maybe 10 to the 18 instructions per second, something like that, but 10 to the 10 times as much. And and now there are two things you can do. Either you succumb to the temptations of this new life, and in the process, you are going to become something very, very different. You are not going to remain a lot like you were, because suddenly everything expands in a way. And and sometimes you might remember your roots as a as a human somewhere, but but your your future life is, you know, totally detached and disconnected

• 54:43Speaker 00

  and probably is is highly influenced by other expanded minds like that, you know, discussing problems that you would have never discussed as a as a as a human person in this reality. Now, the alternative is, for nostalgic reasons, you say, "I I don't want to succumb to these temptations. I keep my two eyes and I keep my little brain and I don't increase it by a factor of 10 billion or something." But then, your competitors, your competitors will be

• 55:23Speaker 00

  will will be ignoring you, you know, because they will have so many new skills that you don't have. And the main decision makers, they are going to be these expanded minds. And they themselves, themselves, they will be in competition with the native AIs, which, you know, are not, they don't have this evolutionary ballast and maybe much more adapted to to to the needs of the future. You know, once the AI sphere is expanding from our biosphere into space, and who knows what they are going to do there. So either you you are nostalgic and remain irrelevant,

• 56:08Speaker 00

  or you become part of this um this growing sociology, if you will, of AIs, which are mostly AIs and some of them may have human roots or whatever. But almost all the decision-making process and and the universe shaping things, they are going to be done by these new beings and not by the human-like beings.

• 56:40Speaker 01

  Would you upload your brain and merge with AI?

• 56:47Speaker 00

  I haven't given too much thought about that because it's not really a goal of mine, because I think

• 56:59Speaker 00

  exactly for the reasons that I just described, that my current work, my current self is not going to make a difference there. And the alternative is going to, you know, the the super AI that is going to go 10 to the 20 times beyond the little thing that I have at my disposal.

• 57:25Speaker 00

  That is going to occur anyway, and not just one of them, but many, many different beings like that. And so I think it's not really going to make a difference.

• 57:38Speaker 01

  Can I ask one last question?

• 57:40Speaker 00

  Yeah, of course.

• 57:40Speaker 01

  So we are we are we're fastly moving towards a world where much of intelligence is encoded in machines, and those machines can predict and take action based off of their conception of where things are heading. Do you believe, then, in the concept of free will right now? And in the future, will there be a free will as so much of our reality is intermediated by these machines that have a pretty good idea of where things are heading?

• 58:17Speaker 00

  So in 1997, I wrote this paper, A Computer Scientist's View of Life, The Universe and Everything. And basically, this was about trying to explain,

• 58:30Speaker 00

  trying to find the simplest explanation of our universe. So the Holy Grail of physics would be find the shortest deterministic program that computes everything that we have ever observed in this universe, including the seemingly random quantum events, spin up and down measurements and everything. And we don't know of the shortest description of the history of the universe. But as scientists, we are trying to find better and better and more and more compact descriptions. And we have made a lot of progress as scientists, as physicists, towards that goal. And then I realized in a couple of years before 1997,

• 59:13Speaker 00

  that although we don't know the shortest algorithm that computes just this universe in which we are living, we at least know the super short algorithm which computes all possible computable universes, which is basically the program that systematically enumerates all possible programs. And then there's an optimal way of allocating runtime to them. And every possible computable universe is going to be computed, including ours, if it is computable. And there is no physical evidence against the possibility that our universe is computable. So I'm not talking just about universes where the probability distribution of the next possible things is computable,

• 59:58Speaker 00

  which is what my former postdoc, Markus Hutter, did around 2000, when he developed the AI XE model, an optimal universal decision maker. Now, really, the deterministic computable universe history is. And so there is no evidence, it's again important to realize that, that we are being created by this method. And there are many programs that computers, but it turns out that they are dominated, that they are dominated by the shortest programs that compute all of this universe, including our conversation here, which a guy like me believes is something totally deterministic.

• 1:00:42Speaker 00

  And so as you are reacting to my, you know, my voice signals, wavefronts coming out of my mouth, and you are responding then with your own wavefronts, this seems like a lot of free will, you know, as we are discussing with each other. However, the deterministic universe view would say this is all deterministic. It may seem like a free will thing to us, but it isn't. And it's interesting to realize that even in very simple simulations, deterministic simulations that we already can do on our little man-made computers, which are just part of this huge simulated universe,

• 1:01:27Speaker 00

  even then, we can observe similar effects because we can really devise deterministic environments for deterministic neural networks that make decisions as they are interacting with other animals, other simulated animals, and they are trying to maximize their reward. And then they are making decisions, you know, just learning to better react to what the other animals are doing. So all of that looks like a lot of free will at first glance, but I can completely rerun every little detail of this simulation. So although it looks like a free will decision thing, it's just something that is part of a deterministic universe.

• 1:02:10Speaker 00

  And therefore, it seems clear to me that the concept of free will is overrated.

• 1:02:18Speaker 01

  If free will is overrated, what's the point of living? Since everything is effectively laid out before you go through.

• 1:02:24Speaker 00

  They made a movie about that, and it's called Free Willing.

• 1:02:30Speaker 01

  About the whale?

• 1:02:32Speaker 00

  Yeah.

• 1:02:33Speaker 01

  Wait, so sorry, can you, how does that connect to the question?

• 1:02:36Speaker 00

  It doesn't connect at all. It was just something that I made up.

• 1:02:42Speaker 01

  All right, Professor. Great speaking with you. A long time coming. Hope we get to do it again sometime soon.

• 1:02:46Speaker 00

  Alex, it was my pleasure. Thank you.

• 1:02:49Speaker 01

  Thank you, everybody, for listening and watching, and we'll see you next time on Big Technology Podcast.

• 1:03:09Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com, not authorized by any candidate or candidate's committee.

• 1:03:40Speaker 03

  This week, it pays extra to be a Milo's Rewards member. Through October 14, members get $10 Milo's money when you spend $200 or more on an eligible purchase during Milo's Money Days. Also, using your Milo's Rewards credit card, even better, that's an additional $20 Milo's money. Use it to save on your next Lowe's haul. Exclusions, restrictions, and more terms apply. See Lowe's dot com slash Milo's Money Days for details. Milo's money expires 30 days after issuance. Subject to change. Credit offer subject to credit approval.