Skip to content

Are You Planning 20,000 AI Agents? Capital One Has a Warning | Abhijit Bose

Eye On A.I.

Hosted by Craig S. Smith

38m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

Capital One has emerged as an early and advanced adopter of artificial intelligence in the banking sector, operating largely as a tech company rather than a traditional financial institution. In this episode, the guest explains how the bank transitioned its entire technical stack to AWS and consolidated its machine learning platform teams under one roof to scale both classic predictive models and modern generative AI. Rather than chasing easy wins, the engineering organization targeted high-complexity use cases like multi-model loss forecasting and real-time fraud prevention, ultimately deploying agentic systems that serve over 100 million customers and 20,000 call center agents.

The discussion covers the critical architectural prerequisites for enterprise AI, contrasting isolated pilot projects with unified, governed platforms. The guest details the development of the Macau framework, which breaks complex customer requests into specialized agent tasks validated through synthetic data simulations. By treating emerging protocols like the Model Context Protocol as structured platforms, Capital One balances rapid innovation in the generative AI era with stringent regulatory risk management.

  • Consolidate separate machine learning platform teams under a single organization early to prevent fragmented cultures and technical debt. 14:43
  • Target the most complex machine learning and AI use cases from the start rather than filling a platform with low-value simple tasks. 36:29
  • Design agentic frameworks that break complex requests into smaller specialized tasks handled by dedicated models and validation checks. 33:38

Chapters

  1. 0:00
    Capital One’s Early AI Leadership

    The episode opens with Capital One’s pioneering role in AI within banking and its focus on fraud prevention.

  2. 0:39
    Introducing the AI Platform Lead

    Craig asks the host to introduce himself and his career journey leading the MLX platform at Capital One.

  3. 3:38
    AI Governance and Risk Management

    Discussion on balancing rapid AI deployment with stringent governance and risk controls in a regulated bank.

  4. 6:28
    Centralized AI Platform Architecture

    Explains the multi-layered platform: compute, orchestration, authoring, and deployment, and its cloud-native foundation.

  5. 14:43
    Scaling AI Through Platform Consolidation

    Highlights the consolidation of teams and building a unified platform to accelerate AI adoption across the bank.

  6. 21:04
    Agentic AI and Future Scale Challenges

    Explores the complexities of deploying thousands of AI agents and the need for a platform mindset.

  7. 29:28
    Real-World AI Applications: Fraud and Customer Service

    Examples of AI-driven fraud modeling and chat concierge services that improve speed and accuracy.

  8. 35:22
    Platform Design Principles and Advice

    Advice for enterprises on foundational platform design, control planes, and choosing use cases to avoid technical debt.

  9. 37:47
    Career Opportunities and Closing Remarks

    The host invites listeners to explore Capital One’s tech blog and career opportunities in AI and engineering.

Highlights

  • 0:54

    Leading Enterprise AI Platformsinsight

    The guest explains their role leading the MLX platform engineering organization, which builds core AI platforms, applications, and coding assistants after spending years at Facebook AI Research and Google.

  • 4:08

    Balancing Risk and Innovationstrategy

    Operating within a heavily regulated bank requires balancing strict governance and risk management with the rapid pace of external generative AI development.

  • 9:13

    Early Bet on Cloud Infrastructurestory

    Capital One realized the need for cloud migration back in 2016 and 2017, completely rebuilding their technology stack primarily on AWS without traditional data centers.

  • 25:45

    Deploying Model Context Protocolsstrategy

    The engineering team approached the Model Context Protocol by building a governed platform where hundreds of MCP servers and services are registered and monitored.

  • 27:04

    Accelerating Loss Forecastingdata

    Consolidating ML teams enabled distributed parallel computing that reduced complex loss forecasting ensemble model training time down to about one week.

  • 27:54

    Scaling Chat Concierge Agentsdata

    Capital One deployed automated AI assistants across auto dealer websites to handle inventory lookups and scheduling using multi-agent coordination.

  • 29:34

    Empowering 20,000 Call Agentsdata

    A large-scale RAG application was successfully rolled out to all twenty thousand call center agents across the company to provide instant knowledge base answers.

  • 32:42

    The Macau Agentic Frameworkinsight

    The proprietary Macau framework breaks complex tasks down for specialized agents and runs synthetic data simulations to validate actions before execution.

Quotes

  • “Eighty, ninety percent of that is because of the way we use AI to fight fraud.”
    Unnamed speaker · 0:14
  • “We are not going to compromise on any of these, but we also need to move faster in the age of, you know, LLMs and how fast the Gen AI itself is moving.”
    Unnamed speaker · 4:24
  • “We have an incredibly high bar on governance, you know, risk management, quality of our decisions, you know, operations.”
    Unnamed speaker · 4:14
  • “We have been able to scale to thousands of, you know, users within the company. You know, really a big four-digit number of users actually use our platform on a monthly basis.”
    Unnamed speaker · 8:10
  • G2 3:18–3:23

About this episode

Most enterprises are still trying to figure out how to get a handful of AI agents into production. Capital One is running tens of thousands. Abhijit Bose, Senior VP and leader of the company's AI/ML platform engineering organization, joins Craig Smith to explain how a decade-old bet on the cloud - made when the economics didn't yet make obvious sense - became the foundation for everything Capital One is doing in AI today, from proprietary GPU clusters built in a single quarter to a platform that took model deployment from months down to about a week. The conversation gets concrete fast: a 24/7 AI concierge now running across car dealership websites, a generative AI system serving all 20,000-plus of Capital One's call center agents - adopted so successfully that agents started asking to be enrolled - and the Macau framework, Capital One's answer to the governance problem that's keeping most regulated industries from putting agentic AI into production. Bose's closing argument is the one that should worry every enterprise still in pilot mode: deploying five agents is manageable without a platform. Deploying 20,000 is a different problem entirely, and the companies that didn't start building the foundation years ago are going to find that out the hard way. Subscribe to Eye on A. I. for weekly conversations with the people building and deploying the future of AI.

Transcript

0:00
38:37

5,221 words · 2 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • Capital One
  • AWS
  • Llama
  • ChatConseil
  • G N A I platform
  • MLX
  • Facebook AI Research
  • Google
  • JP Morgan Chase
  • IBM Research

Page updated · transcript and summary are machine-generated