Skip to content

The OpenAI board member who fired Sam Altman says we still can't trust AI CEOs

Decoder

Hosted by Nilay Patel

1h 10m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

This episode of Decoder features Nilay Patel in conversation with Helen Toner, executive director of the Center for Security and Emerging Technology at Georgetown University and a former OpenAI board member. The discussion explores the realities of AI safety, the gap between consumer-facing tools and frontier lab ambitions, and the challenges of governing technologies that are rapidly evolving toward recursive self-improvement. Toner explains how the quest for superintelligence drives companies like OpenAI and Anthropic, why current alignment methods are struggling to keep up with autonomous risks, and how policymakers in Washington and abroad are grappling with these unprecedented capabilities.

  • Policymakers should focus specifically on frontier AI systems and risk management standards rather than attempting a sweeping, comprehensive regulatory push across all sectors of the economy. 1:09:56
  • Independent oversight committees inside AI labs carry genuine legal weight and utility only when structured around the rigorous governance requirements of public companies. 57:46
  • The United States and China are beginning to find common ground around the shared threat perceptions of major cyber attacks and potential losses of control. 1:07:07
  • Independent research organizations can effectively bridge the expertise gap by providing rigorous, actionable analysis directly to busy policymakers and congressional staff. 3:52

Chapters

  1. 0:00
    Podcast Introduction and Host Greeting

    The host opens the episode, introduces the show and the guest, and sets the context for a deep dive into AI governance.

  2. 0:32
    Introducing Helen Tonner and CSET

    The host welcomes Helen Tonner, explains her role at the Center for Security and Emerging Technology, and outlines the center’s mission and team.

  3. 1:59
    AI Safety Gap Between Policymakers and Industry

    The conversation turns to the widening gap between policymakers’ understanding of AI and the industry’s priorities, highlighting the need for better communication and trust.

  4. 3:01
    Nature of AI Technology and the Myth of Superintelligence

    Helen explains the technical realities of current AI models, the distinction between general intelligence and the hype around recursive self‑improvement, and the implications for safety.

  5. 5:40
    Regulatory Approaches to Bioweapons and Cyber Risks

    Helen outlines policy options for preventing AI‑generated bioweapons and autonomous cyber attacks, weighing the effectiveness of deterrence, oversight, and international cooperation.

  6. 7:00
    Slowing Down Development vs. Maintaining Momentum

    The discussion explores whether a pause or stricter safety bar is feasible, the potential unintended consequences, and the balance between rapid progress and risk mitigation.

  7. 38:01
    Interpretability and Technical Measures for Safer Models

    The final segment focuses on concrete technical tools—interpretability, evaluation, and fine‑tuning—that can help detect and mitigate risky behaviors before deployment.

  8. 41:12
    Exploring AI Alignment and Control Strategies

    The conversation dives into various experimental approaches to improve AI alignment and control, highlighting the breadth of possible training and simulation methods.

  9. 42:58
    Policy Proposal: Decelerate Vertical Growth, Accelerate Horizontal Integration

    The hosts discuss a policy that limits the development of ever-more powerful models while encouraging broader, more practical AI deployment across industries.

  10. 46:17
    Cybersecurity Landscape: Agentic AI Threats and Defensive Measures

    A cybersecurity expert explains how agentic AI increases attack speed and scale, and stresses the importance of comprehensive visibility across endpoints and cloud environments.

  11. 52:41
    Commercialization of AI Labs and the Shift Toward Profit

    The discussion examines how research-focused AI labs have become commercial entities, driven by the need for capital and revenue generation, altering their original mission.

  12. 55:50
    Trust, Governance, and the Future of AI Leadership

    The hosts reflect on trust in AI leaders, the importance of diverse governance structures, and how public market pressures might reshape AI company behavior.

Highlights

  • 3:15

    Defining CSET and Policy Adviceinsight

    The guest explains that CSET is a 50-person team at Georgetown University providing rigorous, actionable analysis on emerging technologies to busy national security and policy makers.

  • 15:59

    Recursive Self-Improvement and Intelligence Explosionsinsight

    The guest breaks down the history and current practice of recursive self-improvement, where AI labs progressively use models to accelerate their own R&D pipelines.

  • 18:06

    Moving Past the Term AGIinsight

    The guest argues that the term AGI has become an unhelpful fuzzy cloud because the industry has already passed certain milestones while remaining very far from others.

  • 32:35

    The Trust Imbalance in Cybersecurityinsight

    The guest highlights the inherent asymmetry in cyber defense versus offense, noting that attackers can easily deploy autonomous agents while defenders need much stronger permissions and assurances.

  • 48:47

    The AGI-Pilled Mindset in Labsinsight

    The guest discusses how the true believers who viewed AI as a fundamental transformation rather than just a B2B SaaS product have steadily risen to prominence across frontier labs.

  • 53:49

    The Commercial Shift of AI Labsstory

    The guest explains that research-focused labs accidentally found massive commercial success and realized their scaling roadmap required enormous capital and computing clusters.

  • 1:00:38

    Security Vulnerabilities and Algorithmic Secretsdata

    The guest notes that frontier lab security remains weak, citing recent incidents where external teams successfully breached repositories and communication channels holding valuable secrets.

Quotes

  • “When the board actually tried to pull that lever, that accountability mechanism broke, and here we are.”
    Nilay Patel · 1:18
  • “If our hopes for the future depend on any one person being trustworthy, we're in big trouble.”
    Nilay Patel · 1:53
  • “We have a big audience. What do you mean by we're going to do RSI and that will lead to superintelligence?”
    Nilay Patel · 13:46
  • “OpenAI has its model spec, different ways of training to try and get the, you know,”
    Unnamed speaker · 41:12
  • State Street Investment Management 0:10–0:24
  • Decoder 2:22–2:31
  • T-Mobile 22:49–23:28
  • DecoderPod 1:10:25–1:11:02

About this episode

My guest today is Helen Toner, the executive director of CSET. This episode is part two of our series on AI in real life. Helen is a well-known figure in the AI safety and policy community, though she is perhaps best known for having served on the board of OpenAI, where was one of four board members who voted to fire Sam Altman. Since then, AI safety has entered the mainstream, as industry leaders have made increasingly dire predictions about what might happen if we don’t proceed more carefully or even slow down. Helen has seen up close what happens when safety commitments run headlong into commercial incentives. As she put it to me, if our hopes for the future depend on any one person being trustworthy, we’re in big trouble. Read the full interview transcript on The Verge. Links: The AIs are already out of control | The Ezra Klein Show AI broke the cybersecurity world. What’s next? | Decoder Ex-OpenAI board member shares why the AI panic Is good news | Offline Inside the suddenly explosive world of AI safety | The Verge OpenAI hack of Hugging Face was “expected” | The Verge Why Sam Altman was booted from OpenAI, according to new testimony | The Verge Former OpenAI board member explains why they fired Sam Altman | The Verge Subscribe to The Verge to access the ad-free version of Decoder! Credits: Decoder is a production of The Verge and part of the Vox Media Podcast Network. Decoder is produced by Greg Ott, Kate Cox, and Nick Statt. The show is edited by Xander Adams and Ursa Wright. Our editorial director is Kevin McShane. The Decoder music is by Breakmaster Cylinder. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Transcript

0:00
1:10:50

15,578 words · 6 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • AI
  • AGI
  • OpenAI
  • China
  • Anthropic
  • ChatGPT
  • Claude
  • Google
  • Hugging Face
  • California

Page updated · transcript and summary are machine-generated