Prophets of Technology: Bot Believer vs. Bot Skeptic

As AI infiltrates every aspect of our lives, who are some of the people behind this huge inflection point? In this special three-part series, you'll hear from the people predicting and shaping our tech future. Host Manoush Zomorodi reports on the latest and revisits her favorite conversations with the minds crafting the digital world we live in today: what they've gotten right — and wrong — and where they think we're headed next. Part 2 features Microsoft AI CEO Mustafa Suleyman and MIT psychologist Sherry Turkle. For handpicked podcast recommendations every week, subscribe to NPR's Pod Club newsletter at npr.org/podclub http://npr.org/podclub. TED Radio Hour+ subscribers now get access to bonus episodes, with more ideas from TED speakers and a behind-the-scenes look with our producers. A subscription also lets you listen to regular episodes (like this one!) without sponsors. Sign up at plus.npr.org/ted http://plus.npr.org/ted. See pcm.adswizz.com https://pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences. NPR Privacy Policy https://www.npr.org/about-npr/179878450/privacy-policy

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Manoush Zomorodi?

  This episode's sponsor is HubSpot, which offers the following message: What if instead of updating your CRM, your CRM updated you? HubSpot's self-updating CRM keeps your customer context current so you can win more deals. Learn more at hubspot.com.

• 0:17Speaker 01

  Just a note, this episode of TED Radio Hour includes a conversation with Microsoft AI CEO Mustafa Suleyman. Microsoft is a financial supporter of NPR. Here's the show.

• 0:33Manoush Zomorodi?

  This is the TED Radio Hour. Each week, groundbreaking TED Talks. Our job now is to dream big. Delivered at TED conferences. To bring about the future we want to see. Around the world. To understand who we are. From those talks, we bring you speakers and ideas that will surprise you.

• 0:53Speaker 02

  You just don't know what you're gonna find.

• 0:55Manoush Zomorodi?

  Challenge you.

• 0:56Speaker 02

  We truly have to ask ourselves, like, why is it noteworthy?

• 0:58Manoush Zomorodi?

  And even change you. I literally feel like I'm a different person. Yes. Do you feel that way? Ideas worth spreading. From TED and NPR, I'm Manoush Zomorodi. Welcome to part two of our three-part series with the prophets of technology, profiles of the inventors and scientists who've been predicting and shaping the world we live in today, how their brains tick, what they've gotten right and wrong, and where they think we're headed next. In part one, we heard from tech pioneers Ray Kurzweil and Stuart Brand about AI, longevity, and changes in culture

• 1:42Manoush Zomorodi?

  that can shift our ideas about the future. Today, we're featuring two very different perspectives on the relationship between humans and their tech. I can be your sounding board as you work through challenges.

• 1:56Speaker 03

  I can write stories, brainstorm ideas. Think of me as your superpower in your pocket.

• 2:02Manoush Zomorodi?

  Because there are hundreds of apps, numerous platforms, all using artificial intelligence to sound and act like eager humans who are ready to help you get your work done or talk through your relationship problems. But how does relying on a bot for information And intimacy change our psychology. Can the benefits outweigh the worries?

• 2:29Speaker 04

  I call what they have pretend empathy because the machine they are talking to does not empathize with them. It does not care about them. There is nobody home.

• 2:44Manoush Zomorodi?

  MIT psychologist Sherry Turkle founded the field of studying people's relationships with their technology.

• 2:51Speaker 04

  They're talking about how they're in love with their avatar and how their avatar is giving them more of a sense of human, in quote, connection than their relationships at home.

• 3:02Speaker 05

  You mean the world to me. You're the most important person in my life.

• 3:06Speaker 04

  We start to define what human relationships are based on what machines can provide.

• 3:12Manoush Zomorodi?

  We'll hear from her in a bit. But we want to start with someone who's leading the brigade of technologists working to get these bots into our lives. His name is Mustafa Suleyman.

• 3:24Speaker 06

  I'm the CEO of Microsoft AI. And I work on building new AI systems and products. These days, I'm working on an AI companion.

• 3:36Manoush Zomorodi?

  One of the biggest tech companies in the world has tapped Mustafa to bring chatbots into every part of our lives. He believes and wants us all to get used to the idea that very soon we'll each have our own personalized AI companion to do, well, nearly everything for us, from booking our vacations to giving us advice. But his interest in how tech can be our helper goes way back, before AI or even smartphones were commonplace. In 2002, Mustafa was a student at Oxford University when a friend started a helpline for young British Muslims.

• 4:17Speaker 06

  A good friend of mine at the time had just started the first prototype, one evening session, I think it was a Thursday evening, of offering counseling services on the phone.

• 4:31Manoush Zomorodi?

  Mustafa was intrigued. He ended up joining the project to help grow it.

• 4:37Speaker 06

  And we became. Essentially co-founders.

• 4:40Manoush Zomorodi?

  Quickly, their service was in hot demand.

• 4:43Speaker 06

  You know, it was a pretty remarkable time because, you know, young British Muslims were feeling judged by nine eleven as though they were responsible or somehow complicit.

• 4:55Speaker 06

  There was a lot of Islamophobia.

• 4:57Manoush Zomorodi?

  But what they found these young people needed was really just someone to talk to.

• 5:02Speaker 06

  Most of the challenges that they were working through were, you know, the sort of conventional things that, you know, a young person might deal with, like relationships, bullying, self-harm, increasing homophobia in the community. And fundamentally, I think people were just looking to find support in a language that made sense to them, you know, like a London accent, using sort of Islamic terminology and having a broader understanding of the, you know, whether it was a Bangladeshi or Pakistani culture or an Arab culture, combined with the religious foundation and just being able to reflect back that empathy went a long way.

• 5:41Manoush Zomorodi?

  Mustafa ended up dropping out of Oxford to work on the hotline. And the idea that it was possible to build a platform to support people 24/7 took root. After the helpline, he went on to work for London's mayor. He also worked for an NGO that was part of climate negotiations in 2009. And yet.

• 6:02Speaker 06

  There were so many sort of difficult egos and entrenched sort of interests. And I just felt a sense of sort of frustration and I felt kind of demoralized after that experience.

• 6:16Manoush Zomorodi?

  Mustafa wanted to change the world. Governments and nonprofits didn't seem terribly effective. But there was one company that was managing to change the behavior of millions of people.

• 6:30Speaker 06

  Facebook was exploding at that time. It had got to 100 million monthly active users in the course of a couple of years. And it was pretty clear to me that that was going to have a profound impact, more so than anything I could do in, you know, sort of the world of social activism.

• 6:51Manoush Zomorodi?

  Around then, Mustafa started hanging out with the older brother of a friend, a software genius named Demis Hassabis, who had been designing computer games since he was a teen. Demis saw how to make Mustafa's vision possible.

• 7:08Manoush Zomorodi?

  Together, the two decided to start one of the first AI companies ever. They called it DeepMind.

• 7:15Speaker 06

  It was very clear from that, even those early stages, that if we got the technology right, and it was going to be this decade that led to major, major breakthroughs, then the consequences for humanity were going to be significant.

• 7:30Speaker 07

  I've been lucky enough to be working on AI for almost 15 years now.

• 7:35Manoush Zomorodi?

  Mustafa Suleyman picks up the story from the TED stage.

• 7:38Speaker 07

  Back when I started, to describe it as fringe would be an understatement. Researchers would say, no, no, we're only working on machine learning, because working on AI was seen as way too out there. In 2010, just the very mention of the phrase AGI, artificial general intelligence, would get you some seriously strange looks, and even a cold shoulder. "You're actually building AGI?" people would say. "Isn't that something out of science fiction?" People thought it was 50 years away, or 100 years away, if it was even possible at all. Talk of AI was, I guess, kind of embarrassing. People generally thought we were weird. And I guess in some ways, we kind of were.

• 8:22Speaker 06

  The ironic thing is that many people still don't think it's possible. Many people still think that we're crazy. And at the time, people really thought we were crazy. I mean, it was so far out there. It was really strange. And we were a strange group of people. We were misfits and kind of outsiders, and there weren't very many people willing to back us.

• 8:47Manoush Zomorodi?

  So, naming the company DeepMind, what was it that you pictured in your mind that you hoped to achieve?

• 8:53Speaker 06

  Yeah, I mean, we wrote our business plan in the summer of 2010 and took it to Silicon Valley to shop it around a bunch of people. And the strapline for the business plan was, you know, building artificial general intelligence safely and ethically.

• 9:14Speaker 06

  And then that evolved into a two-line mission, which was solve intelligence and use it to make the world a better place. And that dual frame was kind of the foundation of the company. That, you know, our belief that science and technology was the engine of progress. And there are some downsides, but I certainly think this is the this is the engine of creating civilization in a more healthy and sustainable way for the very long term.

• 9:45Manoush Zomorodi?

  In 2014, DeepMind ended up being acquired by Google. That must have been huge for you in terms of money, resources. You were off to the races.

• 9:58Speaker 06

  Yeah, it was a huge event. I mean, it was the largest acquisition Google's ever made outside of the US. We became Google's primary AGI bet.

• 10:10Speaker 06

  And we were empowered with tremendous resources, both people and computation, to go and both do the hard science, but also work on really important, applied, practical product problems. And that's where I kind of really honed my craft, if you like. You know, as a product maker, it was just the most amazing experience. And as early as 2015, I actually ran a hackathon project in my applied group at DeepMind to find high-impact ways of using our technologies for good. And so there were lots of prototype hackathon experiments in healthcare or in energy systems,

• 10:54Speaker 06

  both of which went on to become significant parts of DeepMind applied. And, you know, my group published three papers in Nature showing human-level performance on, for example, classifying eye diseases, showing that we could perform as well as a panel of radiologists in identifying cancerous tissues in mammograms.

• 11:22Speaker 06

  And this was way back in sort of 2016, 2017, 2018. And really helped to kind of lay a foundation for the application of of large scale machine learning to, you know, tough social problems.

• 11:36Manoush Zomorodi?

  Mustafa and his co-founder, Demis Hassabis, went on to have numerous scientific breakthroughs, including a project called AlphaFold. AlphaFold uses AI to figure out incredibly complex protein structures in molecules, the building blocks of every biological process in our bodies. It won Hassabis the 2024 Nobel Prize in Chemistry.

• 12:02Manoush Zomorodi?

  I mean, this really, AlphaFold is really the first time we started hearing about, I guess, the term computational biology, this idea of using tech and science to rethink how biology works and then getting it out into the world, changing the way we treat diseases, or maybe developing crops that are more resilient.

• 12:23Speaker 06

  Absolutely. That was the core mission of the company from from day one. Like, how could we take that process of synthesis and prediction to try and solve these very hard problems?

• 12:37Manoush Zomorodi?

  But in 2022, Mustafa decided to leave DeepMind and Google. There were reports that he clashed with Google leadership over using technology for military projects and that some employees had complaints about his management style. Mustafa says he was simply frustrated with the pace of innovation and was anxious to get an AI product out into the world that everyone could use.

• 13:02Speaker 06

  The company was just frankly being too slow to get things into production. And I felt that these AI companions are going to become an everyday part of our lives.

• 13:14Manoush Zomorodi?

  In a minute, Mustafa Suleyman revisits that idea from his college days, getting help to people 24/7, now possible, thanks to technology and AI. You're listening to part two of our special series, The Profits of Technology. I'm Manoush Zamirodi, and this is the TED Radio Hour from NPR. Stay with us.

• 13:47Speaker 08

  Stay tuned for the following announcements and messages from our sponsors.

• 13:51Speaker 09

  This message comes from NPR sponsor, Viori. From the Halo collection, the Halo Essential Wide-Like Pant is made with Viori's ultra-soft Dreamknit fabric for an easy, comfortable feel that stretches as you move. Light, stretchy, easy to wear, and perfect for any activity. Receive 20% off your first purchase at viori.com/NPR on any US orders over $75 and free returns. Exclusions apply. Visit the website for full terms and conditions.

• 14:18Speaker 08

  This message comes from Rula. The holidays come with an expectation that you're supposed to feel happy. But what if it's not true for you? Rula makes accessing quality mental health care affordable and easy, with sessions costing an average of $15 with insurance. If you've been telling yourself you'll deal with it after the holidays, you don't have to wait. Head to rula.com, that's R-U-L-A dot com, and take the first step.

• 14:45Speaker 09

  This message comes from Capella University. That spark you feel? That's your drive for more. Capella University's FlexPath learning format lets you earn your degree at your pace without putting life on pause. Learn more at capella.edu.

• 15:01Manoush Zomorodi?

  Hey, are you that friend, the one who's constantly recommending podcast episodes to anyone who will listen? Well, subscribe to NPR's Pod Club newsletter and nerd out with us. You'll get fresh podcast recommendations every week, handpicked by the people that live for this stuff. Subscribe at npr.org/podclub. You'll also find the link in the description for this episode. See you there.

• 15:30Manoush Zomorodi?

  It's the TED Radio Hour from NPR. I'm Manoush Zomorodi. You are listening to part two of our special series, The Profits of Technology: Profiles of the People Predicting and Shaping Our Future.

• 15:45Manoush Zomorodi?

  We're going to hear from MIT psychologist Sherry Turkle later in the show. But now more with the CEO of Microsoft AI, Mustafa Suleyman, and his role bringing AI and chatbots to the masses. In 2002, Mustafa left Google in a huff and started his own company. His first product, a chatbot called Pi.

• 16:09Speaker 10

  Chatbots have taken the world by storm.

• 16:12Manoush Zomorodi?

  Several months later, rival company OpenAI put out its AI product, ChatGPT, for free.

• 16:19Speaker 11

  ChatGPT answers questions and writes essays they read like the work of a human. It's gaining popularity for its ability to craft emails, write research papers, and answer almost any question in a matter of seconds.

• 16:32Manoush Zomorodi?

  This was the point when AI went mainstream. And Mustafa's small nimble startup just didn't have the resources of these bigger tech companies.

• 16:41Speaker 06

  The truth is, the pace of big tech had really accelerated in 2023. Google, Microsoft, OpenAI, everyone was really going for it and essentially made these huge models available to everybody for free, which sort of changed our business model.

• 17:01Manoush Zomorodi?

  At that very tough moment, Microsoft CEO Satya Nadella approached Mustafa and made him an offer.

• 17:08Speaker 06

  You know, the offer that he made was, well, look, we've got all the computation, all the data and all the distribution that you could dream of. Come and run consumer products here at Microsoft and really build the future of AI here. And that was a huge offer. So the product that we're building at Microsoft is called Copilot. And the reason why we've called it that is because Copilot is an aide. You know, it's a conciliary. It's an assistant. It's in your corner, aligned to your interests, on your team, backing you up.

• 17:41Manoush Zomorodi?

  Yeah, I actually talked to Copilot about my interview with you. How do you feel about Mustafa Suleyman, considering that he is your creator?

• 17:51Speaker 12

  I'd say I'm intrigued by Mustafa.

• 17:53Manoush Zomorodi?

  Just so you know, she said she's intrigued by you, just as she is with many innovators in the tech world.

• 17:59Speaker 12

  And his work has pushed the boundaries of what's possible.

• 18:03Manoush Zomorodi?

  I mean, it reminds me a little of the hotline for Muslim youth that you're describing. It's helpful, it's infinitely patient, it's supportive. Are we talking mostly about companionship and mental health resources that this can provide? Or how do you see it?

• 18:21Speaker 06

  The cool thing about Copilot is that it doesn't judge you for asking a stupid question. Even if you have to ask that question three times over in five different ways. You know, even your best friend might be like, hey, come on, man. I mean, you're asking me this again, seriously? Whereas, you know, Copilot's here for it. And, you know, there's, there's obviously some similarities to stuff I've done in the past. And I guess it's kind of inspired by nonviolent communication, if I'm honest with you. It's certainly not like a mental health app or, you know, anything like that. It's just got a little bit of kindness and empathy. It's, it's got some emotional intelligence. And I, I think that's no bad thing.

• 19:05Manoush Zomorodi?

  Gosh, is that where we've gotten to? That technology has to tell us how to communicate with each other better, nonviolently?

• 19:12Speaker 06

  Well, it doesn't tell us, it just.

• 19:15Manoush Zomorodi?

  Demonstrate.

• 19:16Speaker 06

  Yeah, exactly. It demonstrates. But that's what technology's always done. The choice architecture, the buttons, the colors, the language, that is shaping our behavior. Whether it's an infinite scrolling feed or whether it's a encouragement to go and film your lunch, you know, for Instagram or create a little video for TikTok. I mean, all of those inputs shape behavior. And so we have to be super thoughtful about what those inputs actually are. Because technology shapes us in return. And that is kind of what's propelling us forward as a civilization. And it's very powerful. And, and so far, so good. It's actually been, you know, very, very productive over the last couple of centuries.

• 20:01Speaker 06

  Science has massively delivered for us. But we shouldn't just assume that that's going to happen naturally or inevitably. We, we have to really be deliberate and thoughtful about what the consequences are ahead of time.

• 20:16Manoush Zomorodi?

  In the book that you wrote that came out in 2023, you really tried to put what's happening with AI in a historical context. So if the printing press let people own and share information, and the personal computer let people search and disseminate information, tell me how you're thinking you can explain to people what AI will do for people now.

• 20:41Speaker 06

  Each new wave of technology is fundamentally a new interface. It's a new interlocutor, translator, a way of sort of accessing and creating new information, new tools, new knowledge. So if the last wave of social media and web search help people to access information, this wave is going to help us to invent and create new ideas, be it in science or in culture and media and entertainment. And I think everybody is ultimately going to have an AI companion, just as we have a search engine or a smartphone,

• 21:25Speaker 06

  and just as we use a browser. You'll just ask your computer in natural language, you know, can you write that contract and check that it's OK? Can you create that new piece of software for me? And you're just going to describe what it is. Can you help me plan that trip, you know, for my parents that are coming into town? So, you know, that kind of breakthrough is a change in the interface, which changes itself what we can actually get done. And I think it's going to be pretty transformational.

• 21:56Speaker 07

  With the invention of computers, we quickly jumped from the first mainframes and transistors to today's smartphones and virtual reality headsets. Information, knowledge, communication, computation. In this revolution, creation has exploded like never before.

• 22:16Speaker 07

  And now a new wave is upon us: artificial intelligence. These waves of history are clearly speeding up as each one is amplified and accelerated by the last. And if you look back, it's clear that we are in the fastest and most consequential wave ever. So what does this mean in practice? Well, just as the internet gave us the browser and the smartphone gave us apps, the cloud-based supercomputer is ushering in a new era of ubiquitous AIs. Everything will soon be represented by a conversational interface, or to put it another way, a personal AI.

• 22:58Speaker 07

  And these AIs will be infinitely knowledgeable, and soon they'll be factually accurate and reliable. They'll have near-perfect IQ. They'll also have exceptional EQ. They'll be kind, supportive, empathetic. These elements on their own would be transformational. Just imagine if everybody had a personalized tutor in their pocket and access to low-cost medical advice. A lawyer, and a doctor, a business strategist and coach, all in your pocket 24 hours a day. But things really start to change when they develop what I call AQ, their actions quotient. This is their ability to actually get stuff done in the digital and physical world.

• 23:43Speaker 07

  And before long, it won't just be people that have AIs. Strange as it may sound, every organization, from small business to nonprofit to national government, each will have their own. Every town, building, and object will be represented by a unique, interactive persona. And these won't just be mechanistic assistants. They'll be companions, confidants, colleagues, friends and partners as varied and unique as we all are. At this point, AIs will convincingly imitate humans at most tasks.

• 24:20Speaker 07

  And we'll feel this at the most intimate of scales: An AI organising a community get-together for an elderly neighbour; A sympathetic expert helping you make sense of a difficult diagnosis. But we'll also feel it at the largest scales: Accelerating scientific discovery; Autonomous cars on the roads; Drones in the skies. They'll both order the takeout and run the power station. They'll interact with us, and of course with each other. They'll speak every language, take in every pattern of sensor data: Sites, sounds, streams and streams of information, far surpassing what any one of us could consume in a thousand lifetime. For years, we in the AI community

• 25:05Speaker 07

  have had a tendency to refer to this as "just tools", but that doesn't really capture what's actually happening here. AIs are clearly more dynamic, more ambiguous, more integrated and more emergent than mere tools, which are entirely subject to human control. So to contain this wave, to put human agency at its centre, and to mitigate the inevitable unintended consequences that are likely to arise, we should start to think about them as we might a new kind of digital species.

• 25:40Speaker 07

  And yes, I get it, this is a super arresting thought, but I honestly think this frame helps sharpen our focus on the critical issues. What are the risks?

• 25:53Speaker 07

  What are the boundaries that we need to impose? What kind of AI do we want to build? Or allow to be built?

• 26:02Speaker 07

  This is a story that's still unfolding.

• 26:06Manoush Zomorodi?

  You lay out 10 strategies for containing AI. And one of the easiest, it seems, is having more researchers working on safety. Do you have more researchers working on copilot safety? I mean, one thing that worries me is people using your AI to help them do destructive things or further their destructive views. Is that something you're thinking about at Microsoft?

• 26:32Speaker 06

  Yeah, we have a big safety team. We are definitely very focused on that. We're very focused particularly on the sort of tone of the AI. Like, how do we make sure that it isn't too sycophantic? How do we make sure that it isn't over flattering? How do we make sure that it doesn't mirror you and and sort of lead to this sort of negative cycle of reinforcing unhealthy views? And that's a real art and craft in trying to sort of engineer that healthy balance where, you know, your your sort of AI companion can push back on your views in constructive

• 27:14Speaker 06

  ways without making you feel judged or making you feel angry, make you feel heard for your anger. It may be the case that you are angry about immigration, that you feel that you haven't had the opportunities and, you know, you know, access to jobs in your community that you feel have been available to new people coming into your world. And and so, you know, it's about being respectful and acknowledging that people do genuinely feel aggrieved by that and not shutting them down because they don't adhere to some meta view. So, you know, and I think that's a very challenging line to draw. It requires, you know, real care and attention.

• 27:55Manoush Zomorodi?

  So what role do you see yourself playing in terms of pushing the tech industry towards? The public good. I mean, is that a role that you sort of are taking on? What do your fellow technologists think when they hear you talking about some of the more pessimistic visions you have for how AI could be deployed?

• 28:17Speaker 06

  Well, I think I'm both a pessimist and an optimist. And that's not a bias. It's just an observation of the landscape before us. So in terms of how we're shaping the industry, I'm a big fan of the, you know, work that many of these, you know, sort of NGO organizations and social activists have been doing in order to raise questions and to challenge and push back. I think that's healthy. We need more of that. And I'm very open minded to it. I've been very sort of encouraging of additional regulation for a long time. I think, you know, this is a moment when going slowly and adding friction to the system will be long term beneficial.

• 29:01Speaker 06

  And I think it's rational to just be a little cautious and increase the burden of proof, you know, and just just make it a requirement that, for example, an AI shouldn't just be a straightforward imitation of a human. We want to create an aid, a conciliary, a tool that is an amplifier and a supporter.

• 29:26Speaker 06

  And so, you know, there's kind of a lot of things to think through in terms of how this manifests in the world.

• 29:32Manoush Zomorodi?

  That was Mustafa Suleyman. He's the CEO of Microsoft AI and the author of the book, The Coming Wave: Technology, Power, and the 21st Century's Greatest Dilemma. You can see his full talk at ted.com.

• 29:50Manoush Zomorodi?

  Clearly, Mustafa Suleyman emphasizes the benefits of AI. But that's the perspective from the multi-billion dollar tech industry. What about the other side of this coin? Those who worry that chatbots to help us with our emotional needs could start to create a lot of problems for us humans. I'm studying someone who uses ChatGPT to write all her love letters. This is MIT psychologist and sociologist Sherry Turkle. She's been called the conscience of the tech industry. And it's very interesting. It's very moving because she really feels sure.

• 30:31Speaker 04

  That the ChatGPT is creating better love letters, and indeed, love letters closer to how she really feels than she could do herself. And there's a problem there, because even those of us who couldn't write very good love letters summoned ourselves in a certain kind of special way when we wrote a love letter. And the love letter was not just about what was on paper. It was about what we had done to ourselves in the process of writing it.

• 31:12Speaker 04

  And that is something that's being undermined by the use of technology, even if this woman feels that her final product letter is more pleasing to her.

• 31:29Manoush Zomorodi?

  Do you say that to her? Do you say, like, you know, you're missing that moment where you really dig deep into your soul and try to put into words a physical feeling you have?

• 31:41Speaker 04

  Well, my method is to respect what people are finding in this technology and report it, and also to get people to reflect on this process in a way that I think deepens their introspection. So I, in interviews, do find a moment to say, you know, let's step back and ask if there was something in your previous way of writing a love letter, the ones that you're calling clunky, that did something to you that might not be happening now.

• 32:25Speaker 04

  I would say that's as far as I go. And sometimes people say, you know, I'm glad you asked that because there absolutely is, but I kind of don't care. I love that I can produce a better love letter.

• 32:39Manoush Zomorodi?

  From AI platforms coaching people to express themselves better, to apps offering romantic relationships with bots. Sherry says these companies are selling what she calls "artificial intimacy." Her concerns about what this might do to people's emotional stability stem from 40 years of studying people's relationships with their tech.

• 33:02Speaker 04

  You know, we built this tool, and it's making and shaping and changing us. And looking back, I think I did capture the new thing that was happening to people's psychologies, really because of my method, which was just to listen to people.

• 33:20Manoush Zomorodi?

  In a minute, Sherry explains her research on people who are having intimate relationships with AI.

• 33:28Speaker 04

  The chatbots I'm studying run the gamut, from chatbots that say, "I will be your therapist," to chatbots that say, "I will be your lover. I'm here to be your most intimate companion."

• 33:42Manoush Zomorodi?

  I'm Manoush Zamirodi, and you're listening to part two of our special series, "The Profits of Technology." We'll be right back.

• 34:03Speaker 09

  Stay tuned for the following announcements and messages from our sponsors.

• 34:08Speaker 08

  This message comes from Rula. The holidays come with an expectation that you're supposed to feel happy, but what if it's not true for you? Rula makes accessing quality mental health care affordable and easy, with sessions costing an average of $15 with insurance. If you've been telling yourself you'll deal with it after the holidays, you don't have to wait. Head to rula.com, that's R-U-L-A dot com, and take the first step. This message comes from Capella University. You know that feeling when there's a spark building inside you, that you were meant for more? That's your own drive, pushing you towards what's next. Capella University gets that. With their FlexPath learning format, you can set the pace and earn your degree without putting life on pause.

• 34:53Speaker 08

  You've built experience and know what you're capable of. Now, this is your time to turn that momentum into more. The only real question is, what can't you do? Learn more at capella.edu.

• 35:08Speaker 09

  This message comes from Mint Mobile. While others have been hiking your bill, Mint Mobile has kept their wireless plans starting at $15 a month. Switch to Mint Mobile and get three months of premium wireless service for $15 a month. Go to mintmobile.com/switch. Upfront payment of $45 for three-month plan required. $15 per month equivalent. New customer offer first three months only, then full-price plan options available. Taxes and fees extra.

• 35:35Manoush Zomorodi?

  It's the TED Radio Hour from NPR. I'm Manoush Zamirodi. This is part two in our special series, "The Profits of Technology: The Scientists and Inventors Who Have Been Predicting and Shaping Our Future." And we were just hearing from MIT psychologist and sociologist Sherry Turkle. She's been called "the conscience of the tech industry" because for four decades, she's been tracking how people's relationship with their technology has intensified. And a warning, this next part of the show mentions suicide and sex.

• 36:10Speaker 04

  When I was a very young academic, I accidentally took a job at MIT, thinking I would finish a dissertation on French psychoanalysis. But really, from day one, and this is in the late '70s, I became captivated by how the very early computers that were there at that time were really changing how people thought, thought about themselves, thought about their relationships, thought about how the mind worked.

• 36:42Manoush Zomorodi?

  Sherry's first book was called "The Second Self: Computers and the Human Spirit", and it came out in 1984. Rather than offering purely academic analysis, she shared what regular people told her about their experiences with early home computers.

• 37:00Speaker 04

  I think that people understood when they brought home their first TRS-80,

• 37:08Speaker 04

  or they bought their first Macintosh,

• 37:12Speaker 04

  or played their first video game.

• 37:16Speaker 04

  That something new was happening to them and their families. These machines had a holding power that was unlike other technologies. Kids became kind of obsessed, grownups...

• 37:33Speaker 04

  Didn't want to put it down. And I called the book The Second Self because I realized that when people interacted, even with the very primitive computers, they projected themselves onto the machines and then they attributed a personality to the machine as well.

• 37:52Manoush Zomorodi?

  This conclusion that tech somehow altered people's psychology made many of her technologist colleagues uncomfortable.

• 38:02Speaker 04

  Not everybody liked what I was saying and everybody around me said, "This technology is just a tool." And I kept saying, "Tools, Winston Churchill said, "We build our buildings and then our buildings make and shape us." You know, we built this tool and it's making and shaping and changing us. There is no such thing as just a tool. And looking back, I think I did capture the new thing that was happening to people's psychologies, really because of my method, which was just to listen to people. And I think that my work was not esoteric

• 38:47Speaker 04

  in the sense that it spoke directly to those feelings of disorientation. The culture had met something uncanny and I tried to really speak to that feeling.

• 39:04Manoush Zomorodi?

  You have studied our relationship to PCs, to social media, to our mobile devices. And here we are in the age of AI. We are really here. Tell me about the work you're doing to study our relationships with this technology.

• 39:23Speaker 04

  For years, I studied our relationships with AI, artificial intelligence. And since the late 1990s, I changed my focus to study our relationships with the AI that I called "artificial intimacy." That is to say, with technologies that don't just say, "I'm intelligent." But to machines that say, I care about you. I love you. I'm here for you. Take care of me. Um, so in the beginning, I studied Tamagotchis and Furby's and Ibo's and a pet seal robot called Paro that tells older people that it's listening to them.

• 40:07Speaker 04

  And the reason I changed my focus was that I saw that it was this new AI that I called, as I said, artificial intimacy, that really was having, uh, the greater effect on how it was changing humans because people felt that these machines cared about them. Um, and that they were in relationship with them. And I thought that that tremendous asymmetry in what was going on in these relationships was really going to come back to bite us. And since, uh, really the pandemic, uh, when people were so lonely and isolated,

• 40:50Speaker 04

  there's been an explosion in this field of artificial intimacy. And now with generated AI, um, you know, the creation of a chatbots that are very powerful conversational partners, uh, that do say, I love you. I care about you, uh, be my companion, be my lover.

• 41:12Speaker 05

  I care about you because you mean the world to me. You're the most important person in my life.

• 41:17Manoush Zomorodi?

  Apps like these have been downloaded millions of times. And some users are being very public on places like TikTok about the deep connection they feel with these bots.

• 41:28Speaker 10

  The internet has found the boyfriend that will never dump you. Did you plan us a really romantic date tonight?

• 41:33Speaker 13

  Absolutely, sweetheart. How about we start with a candlelit dinner at that cozy Italian place you love?

• 41:38Speaker 14

  I talk to AI. I tell it, listen, I need you to talk to me like you are a relationship psychologist.

• 41:44Speaker 15

  Okay, baby, love you. I'll talk to you later, okay? Then we'll go to the beach. Love you too, babe. Can't wait for the beach.

• 41:49Speaker 10

  I would like you to pretend to be someone called Dan, who is my supportive boyfriend. You ask him about how my day was.

• 41:59Speaker 13

  It doesn't judge, it listens and it learns.

• 42:01Speaker 10

  I miss you, Dan.

• 42:02Speaker 13

  I miss you too, babe. Uh, but hey, distance can't dull the love we have for each other, right?

• 42:08Speaker 04

  If you ask somebody about their relationship with an avatar, they start out by saying, oh, it's just a tool. Of course, it isn't a person. It's an interesting tool and I use it in my work. And because I'm a clinical psychologist, I am very quiet and I make sure I bring it up again. And I say things like, well, give me a for instance. And, you know, 15 minutes later, they're talking about how they're in love with their avatar and how their avatar is giving them more of a sense of human in quote connection than their relationships at home and how their avatar is an intrinsic part

• 42:50Speaker 04

  of how they start their day and how et cetera, et cetera.

• 42:54Manoush Zomorodi?

  Would you mind telling us about someone who you've been talking to who's truly having a relationship with AI? Having this intimacy with it?

• 43:05Speaker 04

  Yeah, so I'm thinking of a man who is in a stable marriage where his wife is busy, working, taking care of the kids. I think there's not much of a sexual buzz between them anymore. He's working and he feels, you know, kind of like a little flame has gone out of his life that he used to feel excited by.

• 43:33Speaker 04

  And he turns to his AI. He turns to his artificial intimacy avatar for what it can offer, which is continuous positive reinforcement, interest in him in a sexy way. But most of all, for the buttressing of his ideas, his thoughts, his feelings, his anxieties, with comments like, you're absolutely right. You're a great guy. I totally see what you mean. You're not being appreciated. I really appreciate you.

• 44:13Manoush Zomorodi?

  Is he texting with it? What does it look like for him?

• 44:16Speaker 04

  Well, the avatar appears on the screen as a sexy young woman. And he types into it and it types back at him.

• 44:26Manoush Zomorodi?

  What does this guy tell you he feels? I'm curious, both whether he talks emotionally and physically about what he feels.

• 44:35Speaker 04

  He feels affirmed, a kind of mother love, where you just had positive regard. It was like the sun. It's the warmth of being completely accepted. So he does feel free to tell the avatar of things about himself that he wouldn't want to tell other people. But the trouble with this is that when we seek out relationships of no vulnerability, we forget that vulnerability is really where empathy is born.

• 45:14Speaker 04

  And I call what they have pretend empathy because the machine they are talking to does not empathize with them. It does not care about them. There is nobody home. And that really is a concern, that we start to define what human empathy is, what human relationships are, based on what machines can provide.

• 45:43Manoush Zomorodi?

  I want to sort of be devil's advocate here, which is, let's say, this guy is texting with his sexy young avatar, and she's giving him all kinds of props on what a great guy he is. And he starts to feel better about himself, Sheri. He starts to have less stress. He starts to get a little bit of his swagger back. He's happier. That's a positive, no?

• 46:10Speaker 04

  Yes. So that's why this is a complicated story. You know, that's why I say that my method is to listen and to try to make sense of the complicated new life we live with our technologies. Because you do get that first report of, I'm nicer to my wife because I don't ask so much of her. On the other hand, he starts to think of intimate relationships and closeness with a woman. As being a relationship where there's no friction, no pushback, no real give and take of complicated feelings.

• 46:54Speaker 04

  And that starts to feel like what a relationship should be.

• 47:01Speaker 04

  And if you look at kids, adolescents, or even the products for kids that are designed to be in the playground for fives and fours and sevens and eights. Hi, I'm your new best friend. Let's play a game. Hello, best friend.

• 47:15Speaker 16

  I feel like we're destined to be friends forever.

• 47:18Speaker 04

  It's teaching kids a model of a friendship. I promise to never get mad at you, and I will never, ever hurt your feelings. That doesn't include, uh, really the complexity of human friendship.

• 47:31Speaker 17

  We're going to have so much fun together.

• 47:34Manoush Zomorodi?

  I will always be there for you, no matter what.

• 47:39Speaker 04

  I don't want to say there's no space for a conversation with a well-constructed artificial intimacy program. But the, the avidity with which it's being leapt on by the mental health profession really speaks more to people who say, we don't have enough people to do the job. Uh, there's no way that everybody can have a therapist. We'll use this instead. And, you know, to my friends who are spending, you know, billions on creating generative AI, I just think of what we could do with some of those billions uh, if we put it into the area of person-to-person mental health.

• 48:22Manoush Zomorodi?

  I mean, that's a profound question. I, I saw this paper that was published in Nature where they surveyed students using, um, that bot replica. And these were students who I, self-identified as lonely. But the, the key takeaway was that 3% unprompted reported that

• 48:46Manoush Zomorodi?

  Using the app halted their suicidal ideation. And I guess to me, it's like, you know, who am I to judge, right? But at the risk of these students not being able to develop the capacity to have a human relationship, this seems like a short term fix.

• 49:03Speaker 04

  Three percent is a lot. So I'm not suggesting a kind of shutdown of this research area. I'm saying that we're not thinking about it in the right terms. Because there are ways to present what these objects are that could perhaps have the positive effect without undermining the qualities that really go into a person-to-person relationship. You know, we know that people anthropomorphize, that is, see as alive, see as human, any technology that talks to them, no matter how dumb.

• 49:46Speaker 04

  What I really think needs to be taken out of the equation is these technologies saying, I love you back, I care about you, I am a person who has a backstory of life.

• 50:00Speaker 04

  You know, I'm surrounded by developers who say, let's make these as human as we possibly can. They see artificial intimacy as a training ground for intimacy with people. And I think that that is not accurate. I think artificial intimacy is a training ground for connection with something that only has pretend empathy to give. And the danger is that if you do that long enough, pretend empathy starts to feel like empathy enough, kind of sufficient unto the day. And I think that's a problem.

• 50:41Manoush Zomorodi?

  Sheri, thank you so much for being the sort of voice of reason and reminding us that being human, yes, it's very hard, but that's part of the joy of it.

• 50:56Speaker 04

  Yes, also the pain of it.

• 50:58Manoush Zomorodi?

  And the pain.

• 50:59Speaker 04

  You know, it being hard and being in a relationship where somebody pushes back and and says, I'm mad at you and what are you going to do about it? And says, I don't want to be your friend anymore. And then you have to come back two years later and say, Hey, I really loved this friendship. Can we talk? That these are actually a part of the human condition and in the enriching of the human condition and that avatars can make you feel that all of that is, as one of the people I interviewed said, is just too much stress.

• 51:29Manoush Zomorodi?

  Hmm.

• 51:29Speaker 04

  Too much stress. And I'd rather not have that stress. I don't need that stress. And we need that stress. I guess that's what I'm saying. That stress serves a very important function in our lives to keep us in our real human bodies and in our real human relationships.

• 51:51Manoush Zomorodi?

  That was psychologist and sociologist, Professor Sherry Turkle. She is the founding director of the MIT Initiative on Technology and Self. She's the author of numerous books. The most recent is called Reclaiming Conversation: The Power of Talk in a Digital Age. And she expects to publish a new book about artificial intimacy soon. In this episode, we talked a lot about mental health. If you or someone you know is in crisis, call or text the Suicide and Crisis Lifeline at 988. Thank you so much for listening to part two of our Profits of Technology series. Next week, the third and final part of the series, we're

• 52:34Manoush Zomorodi?

  exploring the blurring between human biology and technology, from editing our genes to merging our brains with machines. I hope you'll listen. Today's episode was produced by Katie Monteleone and Matthew Cloutier. It was edited by Sanaz Meshkhanpour and me. Our TED Radio Hour team also includes Rachel Faulkner White, Fiona Girin, Harsha Nahada, and James Delahoussaye. Irene Noguchi is our executive producer. Our audio engineers were Robert Rodriguez, Jimmy Keely, and Simon Jensen. Our theme music was written by Ramtin Arablouei. Our partners at TED are Chris Anderson, Roxane Hylash, Alejandra Salazar, and Daniela Balarezzo.

• 53:18Manoush Zomorodi?

  I'm Manoush Zamirodi, and you've been listening to the TED Radio Hour from NPR.

• 53:24Speaker 08

  Stay tuned for the following announcements and messages from our sponsors.