Skip to content

The Missing Layer for Personal AI | First Person Labs’ Ken Liu

a16z Podcast

Hosted by Erik Torenberg

33m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

This episode explores Second, a personal context vault built by First Person Labs that captures digital activity and preferences to share across AI agents. Co-founder Ken Liu joins hosts Martín Casado and Joel de la Garza to discuss how Second continuously analyzes computer streams, messages, and priorities in the background, serving as a trusted digital butler rather than just another chatbot. The discussion covers the heavy security architecture required to handle private data, the economics of running 24/7 AI token streams, and why portable context may soon replace siloed AI applications.

  • Users can maintain complete privacy by bringing their own tokens to analyze local laptop data. 12:02
  • Confidential inference uses hardware-level protection inside GPUs so that no provider can inspect user data. 12:39
  • Token prices are dropping rapidly, making continuous 24/7 background analysis increasingly affordable. 19:07
  • Specialized agents are the future, where Second acts as a secure vault while other apps handle specific tasks. 9:35

Chapters

  1. 0:00
    Podcast Intro and Host Greeting

    Marc Andreessen opens the episode, welcoming listeners and setting the stage for a discussion on personal AI.

  2. 0:35
    Defining First Person Labs’ Vision

    Ken Liu explains the core idea of creating a personal model that gives users control over their AI interactions.

  3. 2:08
    Integrating with Existing AI Tools

    The conversation turns to how First Person Labs’ Second can plug into platforms like Grokbot and provide portable context.

  4. 5:16
    Security and Privacy Architecture

    Ken details the security challenges of giving AI access to personal data and the confidential inference approach.

  5. 18:14
    Cost and Model Portability

    The hosts discuss the token costs of running 24/7 agents and how model choice affects performance and price.

  6. 23:48
    User Experience and API Design

    Ken explains how users install the app, grant permissions, and interact with Second via APIs and text.

  7. 28:30
    Future Interfaces and Embodied AI

    The dialogue explores potential future interfaces, from audio modes to embodied assistants like Alfred or Jarvis.

  8. 31:28
    Balancing Security and Usability

    Ken reflects on the tension between building a secure system and delivering a smooth user experience.

  9. 32:35
    Product Roadmap and Community Involvement

    The episode concludes with Ken inviting listeners to try Second, join the community, and stay tuned for future updates.

Highlights

  • 1:46

    Defining the Second Architectureinsight

    Ken Liu explains that First Person Labs focuses on creating a comprehensive model of a person's memories, priorities, and preferences rather than building just another conversational agent.

  • 5:16

    Continuous OS Event Analysisstrategy

    The system continuously processes the infinite stream of events on a computer, from app switches to typing patterns, to hierarchically build up understanding without user prompting.

  • 6:31

    Real-time Proactive Weather Alertsstory

    Joel de la Garza shares how Second proactively monitored an ongoing event conversation and updated him on weather forecasts without being asked.

  • 17:22

    Handling Prompt Injection Risksdebate

    When asked how to prevent an LLM from divulging sensitive information, Ken explains that while prompt injection is hard, reasoning capabilities scale with model intelligence.

  • 18:38

    The High Cost of 24/7 Inferencedata

    Running multiple agents 24/7 on frontier models like Opus can cost around $1,000 per day, though cheaper models and price drops are reducing this overhead.

  • 22:46

    Mixture of Models for Cost Efficiencystrategy

    Second uses different models for different tasks, employing cheaper models for routine tracking and advanced ones only when deep goal analysis is needed.

Quotes

  • “People are underestimating how much the computer already knows.”
    Unnamed speaker · 0:00
  • “You can just screenshot anything and just say fix. And then you would know what you're fixing.”
    Unnamed speaker · 0:06
  • “I think we want to have humans, you know, exercise their nuclear.”
    Unnamed speaker · 25:09
  • “In the long future, I imagine that every person will have, like essentially everyone will have a second.”
    Unnamed speaker · 28:11

About this episode

AI agents are getting smarter, but every new agent still has to learn who you are. What if your preferences, priorities, relationships, and digital context could move with you instead? First Person Labs co-founder Ken Liu joins a16z’s Martin Casado and Joel de la Garza to introduce Second, a personal context layer designed to understand what you’re doing across your digital life and make that context available to the agents you choose. They discuss how Second can proactively connect information across conversations and applications, why protecting such personal data requires a different security architecture, and the longer-term vision of AI that doesn’t just respond to prompts, but understands enough context to know when and how to help.

Transcript

0:00
33:57

7,065 words · 4 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • Second
  • Grokbot
  • Confidential inference
  • OpenAI
  • Apple
  • Jeff
  • iCloud
  • iPad
  • Zero data retention
  • DeepSeek Flash

Page updated · transcript and summary are machine-generated