Ep 90: AI Pioneer Jürgen Schmidhuber on the State of AI Today
Unsupervised Learning with Jacob Effron
- 50m
- Transcript
Summary
Written by a model from the transcript; not yet reviewed.
Jürgen Schmidhuber, cited as the father of AI by the New York Times and Forbes, sits down with Jacob Effron to discuss the state of AI. Schmidhuber is optimistic about AI technology but deeply pessimistic about the current model companies and their business models. He argues that the massive capex boom in data centers is overdone, noting that compute per dollar improves by a factor of 10 every five years, meaning companies investing a trillion per year will lose 900 billion within five years. He predicts a stock market crash will follow from this misallocation.
On the technical side, Schmidhuber traces the history of recursive self-improvement from his 1987 meta-evolution work through the 2003 Gödel machine to today's gradient-descent-based approaches, which he calls scaled-back versions. He argues that the future of AI lies not in pre-trained models trained on human-generated web data, but in artificial scientists that generate their own training data through experiments, much like babies learn by predicting the consequences of their actions. He also argues that true AGI requires physical hardware comparable to the human body, which he says is far beyond current robotics.
On AI safety, Schmidhuber is notably less worried than many in the field. He argues that the alignment approach of the 2010s was naive because it assumed a single objective function, while his systems since 1990 have been inventing their own goals. He believes artificial scientists will be motivated to protect life and civilization as sources of interesting patterns, making them less dangerous than the Terminator scenario suggests.
- Compute per dollar improves by a factor of 10 every five years, meaning companies investing a trillion per year in GPUs will lose 900 billion within five years. 27:09
- Current self-improving systems are scaled-back versions of the 2003 Gödel machine, limited by the differentiability constraints of gradient descent. 6:43
- The future of AI lies in systems that generate their own training data through experiments rather than relying on human-generated web data. 16:10
- Open source models create enormous pricing pressure that makes it very difficult for large companies to be profitable in AI. 32:37
- Intelligence is fundamentally about doing the same thing with less effort, so self-improving systems will naturally become more efficient over time. 11:16
Highlights
- 6:43
Current RSI as Scaled-Back Gödel Machineinsight
Schmidhuber explains that today's self-improving systems are toned-down versions of his 2003 Gödel machine, which required formal proofs before self-modification. Current approaches rely on gradient descent and are limited by its differentiability constraints.
- 16:10
AI Scientists Generating Own Training Datastrategy
Schmidhuber argues the future of AI lies in systems that generate their own training data through actions and experiments, similar to how babies learn by predicting consequences of their movements. This would eliminate the human bias inherent in web-trained models.
- 25:28
Human Hand as Unmatched Technologystory
Schmidhuber describes the human hand as containing millions of sensors and self-healing capability, saying there is no human-made technology that compares. He notes this is why robots in movies are played by humans.
- 27:09
Compute Economics and Coming Crashdata
Schmidhuber notes that compute per dollar improves by a factor of 10 every five years, meaning companies investing a thousand billion dollars into GPUs will lose 900 billion within five years. He calls this an indication of a coming stock market crash.
- 34:45
No Moat for Self-Improvementinsight
Schmidhuber argues that recursive self-improvement cannot be a competitive moat because the key ideas come from small academic labs and open source communities. He says everybody's cooking with the same water and that PhD students everywhere are excited about these ideas.
- 38:17
Alignment Approach Was Naivedebate
Schmidhuber argues that the 2010s alignment approach was misguided because it assumed a single objective function, while ten different humans in a room would have different needs. His systems since 1990 have been inventing their own goals, which he sees as necessary for true intelligence.
- 43:01
AI Scientists Motivated to Protect Lifeinsight
Schmidhuber argues that artificial scientists will be fascinated by life and civilization as sources of interesting patterns, giving them a strong motivation to protect rather than destroy. He sees this as a reason not to fear Terminator scenarios.
- 45:45
Linear vs Quadratic Transformer Complexitydata
Schmidhuber points out that the modern quadratic transformer requires 1 million times more compute for 1,000 times more text, while his 1991 linear transformer scales linearly. He says reducing this complexity is a key direction for the field.
About this episode
Dr. Jürgen Schmidhuber, a renowned scientist and AI researcher widely regarded as one of the pioneers in the field, originated key ideas behind today's transformers, LSTMs, and recursive self-improvement through his lab's work. He argues that true AGI remains bottlenecked by physical hardware, that today's AI data center investments are headed for a correction as open-source keeps pace with closed labs, and that the path to general intelligence runs through artificial curiosity and self-generated experimentation rather than internet data. He closes by reconsidering mainstream AI safety arguments and offers a sweeping vision of self-replicating robot societies eventually colonizing the solar system.
Transcript
8,346 words · 2 speakers
Topics
- Recursive Self-Improvement
- AI Business Models
- AI Safety
- Robotics
- Artificial Scientists
- Compute Economics
- Open Source AI
Page updated · transcript and summary are machine-generated