How AI really works, and how the smart home broke

Today on the flagship podcast of open-source lightbulbs: David Pierce chats with Verge investigations editor Josh Dzieza about his story detailing how humans matter far more to AI development than we may have thought. Inside the AI Factory: the humans that make tech seem human Later, smart home reviewer Jennifer Pattison Tuohy explains why we're probably getting the idea of a “smart home” all wrong. Smart homes for smart people How microgrids and smart homes are shaping our energy-independent future Every device that works with Matter What is a smart home, and do you need one? How to pick a smart home platform From brilliant to basic, here are our smart home setups Email us at [email protected] or call us at 866-VERGE11, we love hearing from you. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02David Pierce?

  Welcome to The Vergecast, the flagship podcast of open source light bulbs. I'm your friend David Pierce and I am coming to you from my laundry room/furnace room/ I don't know what you call it, it's just that weird room in your house where you keep all your tools and toilet paper and laundry detergent and stuff. Anyway, I'm in here because, well, last week was Smart Home Week on theverge.com and I was inspired to try and connect all of my stuff a little better together. So I'm trying to get a water sensor rigged up to tell me if my hot water heater is leaking, which sounds simple and is frankly exactly as boring as it sounds, but it will make me sleep better at night. Anyway, we have much better things to talk about on the show today than water sensors. Josh Jezza is going

• 0:44David Pierce?

  to come on and tell us a very different kind of AI story than the ones we've been hearing over the last few months. And then Jen Patterson Tuohy is going to come on and we're going to talk about smart stuff and whether we've been approaching the whole idea of the smart home all wrong. Actually, come to think of it, we might end up talking about water sensors after all, so sorry in advance if that happens. All that is coming in just a sec, but first I have to see if I can shimmy behind the water heater over here to get this thing installed. So if you don't hear from me again, I'm probably stuck back here. This is The Vergecast. See you in a sec.

• 1:20Speaker 01

  This episode is brought to you by On Investing, an original podcast from Charles Schwab. Each week, hosts Lizanne Saunders, Schwab's chief investment strategist, and Colin Martin, head of fixed income research and strategy for the Schwab Center for Financial Research, bring you fresh insights on what's happening in the markets and why, and what the implications might be for your portfolio. Join Colin and Lizanne as they explore questions like, how do you evaluate corporate bonds that look interesting? And what sectors are on the move right now? Download the latest episode and subscribe at schwab.com/oninvesting, or wherever you get your podcasts.

• 2:05Speaker 02

  Real innovation is powered by real people. This is AMEX Corporate. Trusted experts, smarter technology, 24/7 dedicated support from day one to answer questions, help set up policies and approvals, and consult on spend capacity as your business needs evolve. Plus, on-demand resources help level up operations at every stage. That's the trusted service you've come to expect. Built to simplify, designed to scale. This is AMEX Corporate.

• 2:37David Pierce?

  Welcome back. I made it. Everything's okay. Obviously, AI is the biggest story in tech right now, both in the chat GPT, chatbot sense, and just in the sense that AI tech is suddenly showing up in every product and service we use to help us write, help us find things, help us make things, and lots more. And we kind of understand how all the underlying tech there works, right? You feed a bunch of data into a large language model. It automatically, black boxy, trains itself to understand all that data, and you're done. But of course, it's not that simple. And it turns out, it's not that automated either. The Verge's Josh Jezza has spent the last few months looking into one particularly misunderstood part of the process,

• 3:21David Pierce?

  where humans matter far more to AI development than you might think. He even got some firsthand experience in the AI process. So we brought him in to talk about it. Hey, Josh.

• 3:31Speaker 03

  Hey.

• 3:31David Pierce?

  So, there's a lot to get to in this story. And I want to spend a lot of time talking about the part where you became a data annotator, because I find that deeply fascinating. But I think, just to like, set the scene a little bit, we need to do a little bit of, I think, term defining here to make some of this make sense. We go back to like, 2007, which is kind of the earliest date in your story, at Princeton with a researcher named Fei-Fei Li. Tell me what's going on kind of there and then, and why this is sort of the moment that starts what leads to your story.

• 4:03Speaker 03

  Yeah, so this is really the moment where deep neural networks and machine learning really comes back. It was sort of a technique of machine learning that had been languishing a little bit. And then you have these fast processors and then Pfeiffer just realizes that you need way more data than anyone's been training anything on to get image recognition, which is really sort of the testbed at the time. So using millions of images instead of tens of thousands, but it was really impossible to label or categorize that many images. You need, you know, there's this unprecedented corpus of imagery now of the internet, but you don't no longer have grad students kind of like page through encyclopedias or whatever and scan images, but you still need someone to go through and say, you know, cat, airplane,

• 4:47Speaker 03

  automobile, whatever, and sort these images into something that, you know, this neural network can suss out the patterns in and begin to identify stuff. So she creates this database called ImageNet using Mechanical Turk, which was sort of recent at the time, the Amazon work platform where you can have people from around the world do micro tasks for pennies usually. And, you know, it worked and the machine learning revolution kicked off. People realized that this is actually super effective if you have enough data and enough annotated data to train these models on.

• 5:20David Pierce?

  The phrase annotated data is like the one that kept jumping out to me because I think the way that we understand machine learning so much is basically like develop a model, just throw the internet's data at it, whatever you can find, like scrape Google Images, scrape Reddit, whatever you can find, throw it at it, magical step three, sort of black box nonsense, and out of it comes AI. And I think the thing that really jumped out to me through your story is not just this realization that what we need is a huge amount of data, right? Which I think now, 16 years later, is relatively well-known, right? Like to make these things great, you need outrageous amounts of training data. What I was surprised by is the amount of human interaction that there kind of was

• 6:05David Pierce?

  and still is in this process, that not only is it like a handful of people at the beginning saying, this is what a cat looks like, this is what a dog looks like, this is what an airplane looks like, that this is like a massive, ongoing, never-ending process. And it seems like that was kind of the like insight that sent you down this road to some extent, that like this is a bigger and more human task than anyone realizes even now.

• 6:28Speaker 03

  Yeah, I think there's two levels to that. So there's the sort of model creation, training aspect that's super manual. And there is sort of a first round where you just kind of throw all your data in there and it finds its own patterns, the sort of unsupervised learning process. But then you have to go through and, you know, there's different methods, but you need humans to either sort of pick the best examples of what the model does and then sort of retrain it on that or, you know, write your own sort of like, here's the gold standard essay or, you know, Python program or something and then train it on that and then kind of do more rounds of feedback. And so that's sort of state of the art for the language models you see now.

• 7:12Speaker 03

  But then you have this other element, which is like, all these programs are super brittle. You know, just because they can do one thing well doesn't mean they can do something else that seems quite similar well at all. You know, whenever they encounter something that's not in their data set, they struggle. And so you have kind of this other side that I had not really realized existed of people who are just kind of, you know, minding automated programs that are out in the world. You know, they're like sorting through credit card data and saying like this came from 7-Eleven or whatever, their, you know, self-driving car gets stuck and they're going to say, oh, yeah, we really need to, you know, get some more bird annotations or something like the sort of, the world is changing and things are always encountering data that's not in their training data. And so they, they fall apart and you always need kind of a human there to catch them.

• 7:56David Pierce?

  Okay. So this thing, this, this sort of insight from 2007 birthed this like gigantic industry. And I want to talk about kind of the industry at a higher level and then sort of the specific job of what the people who work for these companies do. But the industry is totally fascinating to me because you uncovered this like gigantic billion dollar sort of sketchy undercover world of data training. I don't know, there were moments where it was like, oh, this is a big sort of booming industry. And then there were moments where it was like, this sounds like the way that sketchy billionaires try to hide their money by like setting up shell corporations all over the world. This industry does not want you to know about it, but you learned a lot. Like sort of paint me the picture of what this training industry looks like now.

• 8:39Speaker 03

  Yeah, it's really weird. So there's a, you know, a bunch of different, super opaque elements to it. One component, it looks sort of like an outsourcing company, like a call center. You go to some vendor, you know, usually in the global south, and they have a lot of people in an office labeling data. But then increasingly, you have these things that are kind of, I think of them as like the sort of next generation of Mechanical Turk. Like Mechanical Turk is sort of a lot of work to get anything out of. Like, you need to go in there and like post a task and sort through a lot of garbage. You have these other companies and sort of one of the ones I focus on is called Scale AI. And their whole thing is like, we have well-trained people. They're really experts in this stuff.

• 9:19Speaker 03

  And they are also being assessed by, you know, automated systems that are testing, you know, you're doing sort of quality assurance and things like that. And they're trained. And it's also a platform like Mechanical Turk that, you know, anyone can sign up for, but like, it's a job. You know, you, you'd spend hours sort of learning each new task and you can get fired or banned or whatever the algorithmic platform equivalent of fire is. And that's sort of what you have now. It's a huge numbers of people, like hundreds of thousands of people who are doing this work. And because you don't want your in-development AI system leaking out, there are sort of these nested layers of opacity, you know,

• 10:01Speaker 03

  code names, Scale's platform is called Remote Tasks. There's no sort of public disclosure that they're related. And then every sort of project in on Remote Tasks is also codenamed with sort of weird stuff like pillbox, bratwurst, or whatever. So no one knows what they're working on. It's all quite murky.

• 10:21David Pierce?

  We're going to come back to Remote Tasks, by the way. I'm very excited to talk about your, your life in Remote Tasks. But the thing that kept jumping out to me is that it seems odd to me that this industry is so secretive. Because on the one hand, it sounds a lot like kind of the way content moderation works, right? Where it's, the job is a lot of sort of menial drudgery in a lot of ways. It is, it is sort of repetitive tasks. And folks outsource that. But I think that's in part because content moderation is viewed by a lot of these companies as Not that important. It's like a thing you have to do, but they're happy to have it done sort of serviceably at the cheapest way possible. But in this moment of AI, like this is the game now. These companies are raising gigantic amounts of money. They're spending a ton to train these things.

• 11:04David Pierce?

  You would think that they would want to have as much quality control, as much, you know, privacy and secretive control over all of it as possible. So I guess it's weird to me that it's this big secretive industry that is so mission critical to what everybody says is the future. Like, why is it this third-party subcontracting world instead of like Google hiring a team of 10,000 people to do it for Google and Microsoft for Microsoft?

• 11:31Speaker 03

  I think this is changing a little bit, but I do think there still is this perception that it's not that important, that like, well, we just, you know, AI is improving so quickly, where we just need to get a bunch of data and then it's going to be able to do this and we can automate it and we won't need this anymore. So why put a bunch of people on your payroll permanently for this?

• 11:48David Pierce?

  So this view is like the human element is like a stop along the way where like,

• 11:52Speaker 03

  Yeah.

• 11:52David Pierce?

  We won't. Yeah, I think that's wrong and I want to talk more about that in a minute. But the job of these annotators, you talk to a bunch of people who actually like sit down at a computer and do this kind of training work for a living. Like, what does the job look like? What do you do all day when you do this job?

• 12:06Speaker 03

  There's a lot of variation. So the image recognition stuff is very like, this is, I think, the thing people are most familiar with. It's like the self-driving car stuff. You get like a bunch of capture looking things and you outline the fire hydrants or whatever, or like a bunch of LIDAR laser points and you sort of say like, this is the human and that's a, that's a human there. That is kind of the classic stuff. And then the chatbot stuff is more, here's like kind of weird sort of text edit kind of interface a lot of the time. And you have a bunch of things you're supposed to be reading for or something and you're given, I'll say it feels like a standardized test. It's like, you have like an essay and you have like two attempts to summarize it and you like critique which one is better.

• 12:52Speaker 03

  Or write your own version of it, or like, here's some marketing copy. Did this include all the information in the product? Or did it make up a bunch of stuff? Things like that. It's pretty repetitive, but it is also really hard.

• 13:07David Pierce?

  Yeah. What is hard about it? Like, I understand why it's repetitive. You tell a story of a guy who has to go, basically, like you're talking about the self-driving example, frame by frame, which just, it just sounds brutal. And it's like, the fire hydrant was here in the last frame, and it's here in this frame. But I have to do it because that's how you train the thing, and it has to learn all the edge cases and so on. But I did get the sense that this is a, like, genuinely sort of difficult job. What is it about it that people find difficult?

• 13:32Speaker 03

  You have to be really precise. So like, that kind of lidar task, you have to go through every frame from sort of every perception mode for a car, and label it and say, this is a vehicle. It's moving. It's this type of vehicle. And sort of do that for each thing. And so you have like a few seconds of footage that ends up taking a, you know, eight-hour day to fully annotate. I mean, you have to be really precise. Like, I kept failing a task where you have to outline pallets for like an autonomous forklift or something. And it was just like, the quality you're supposed to get was down to the pixel level. So it's like, if I got one pixel off, I would fail. And it's this blurry photo in a dark warehouse. And it's like, I got, I flunked it.

• 14:12David Pierce?

  Wow.

• 14:12Speaker 03

  The other element is just that the things that you're doing are really alien. Like, the way these machines process information is not human. And so you're having to do these things like, tag guys' clothing, but also label all the clothing in mirrors. Because to an AI system, it's the same thing. And then what is clothing? You know, are rollerblade shoes? Like, these sort of weird questions that you get into.

• 14:37David Pierce?

  Right.

• 14:37Speaker 03

  And like, the engineers, they don't even know that they need this to make these distinctions. But then they, you know, put the data in, and their model doesn't work. And they come back and say, the guy, yeah, don't, don't label rollerblades, but do label flippers, or something like that. So you just get these kind of incredible, you know, 40-page instruction manuals that you have to follow.

• 14:57David Pierce?

  Right. So yeah, and this is where we get to your specific adventures in this world. You, if I'm remembering correctly from your story, decided at one point in the reporting process to sign up for remote tasks, right? Tell me about the process. How did you get into this world?

• 15:10Speaker 03

  Yeah, so I was kind of beating my head against the wall. It was, it's a hard world to get into because it's so opaque. You know, you can't go to an office. You can't sort of figure out, people, people don't put it on their resume on LinkedIn very often. And so it was kind of the only way to figure out. Who was doing the work and what the work was like was to join the platform and it's totally open. So this seemed like the the obvious way to go. And one of the quirks of the system is you're only brought into a group on remote tasks of workers if you pass like a training program, like you need to pass the test and then you're brought into a Slack channel with people. And so it was like this video game where I kept having to pass these training programs. And I kept failing because they're really like, I don't

• 15:54Speaker 03

  know, you see a thing and it's like, leave all the clothes in the social media posts. And it seems self explanatory. And so I would sort of click through the instructions, fail completely, have to go back and be like, there's like dozens of pages of distinctions that you need to follow. You need to keep it up on like a second monitor and always be checking it because it's just incredibly precise and these sorts of weird.

• 16:17David Pierce?

  Give me an example. So like I'm imagining I can think of a social media photo and it would say circle all the clothes. I would see somebody's wearing pants and a shirt and a pair of shoes. I would circle all those things as exactly as I could and then just like move on with my life. That doesn't sound that hard. Like what, what were the problems you kept running into here?

• 16:34Speaker 03

  So first off was what is the definition of real here? Like the instructions are like, only circle the real clothes that can be worn by real people. And one of the first things I got was just like a magazine on a like a photo of a magazine with some clothes on. It's like, well, that's not real, but that's a magazine, not correct. Like those are real images of real people in a magazine. And so you circle them. And the same goes for mirrors or whatever else, because like the AI, it's all just pixels. It doesn't know the difference between a mirror and a magazine. And so that was kind of the first thing was sort of that you need to be looking at, you know, someone's in the background and you don't think it's the thing that's relevant, but you know, they're wearing a hat and you need to circle that. And then it was weird stuff where it would be like, don't label

• 17:19Speaker 03

  clothes on action figures, but do label them on mannequins or label costumes, but don't label armor. And so what is, you know, someone is dressed up in like a stormtrooper armor? Is that a costume or is that armor? Like these sort of weird things where people do these distinctions and you can't keep them all in your head. And so you just sort of look at through it. And you're like, I have no idea sort of what side of the line that falls on. And then you guess and then you fail out of it.

• 17:46David Pierce?

  So, and I guess the assumption then would be that this AI is trying to learn to make the same distinctions that you're learning how to make. And essentially, you're just trying to train it over and over. You're doing all of the thinking so that the machine can just learn what the answers are, basically.

• 18:03Speaker 03

  Yeah, I never really figured out what that one was about, but you know, I could guess. And there was a lot of e-commerce stuff on there, and it all sort of seemed geared around the same thing, to be able to, you know, comb Instagram and say, this kind of, like, this shirt is, you know, spiking in popularity or something like that. Or like, this person is posting about this sort of item of clothing, like, maybe we should market this to them. Like, that sort of, it seemed like it was geared toward being able to make those kinds of calls. And so you had to teach it, you know, how to recognize a shirt in a, in a sort of a social media environment, which turns out to be incredibly difficult.

• 18:37David Pierce?

  Yeah. So you eventually passed clothes. Congratulations.

• 18:40Speaker 03

  Thank you.

• 18:41David Pierce?

  What did you then get thrust into? What was your first big project after that?

• 18:45Speaker 03

  So, yeah, I did, I did a little bit of it. And then I, you know, failed pretty quickly. And I think it was, I started getting some people in mirrors, or like, in windows, you know, reflected in building windows or something like that. And that was something you were supposed to label, but not if it was too blurry. And so I flunked out of it. And then it kicked me to something, I think it was called crab generation. It was like, it's quite humiliating, you know, you just get this pop up and there's like a cartoon crab that's like, your low quality has, has booted you from this, but now you can train for this new task. And that was more of a chatbot thing. And that was sort of just like seeing whether it was paraphrasing news articles without making things up, basically.

• 19:27Speaker 03

  But, you know, it's quite eclectic. There's all kinds of stuff on there. You know, there was one where it was just sort of videos of people talking about their jobs. And I was supposed to rate whether they seemed like emotionally stable, I think, was the criteria, one of the criteria, and sort of how outgoing they were. No idea what that was about.

• 19:44David Pierce?

  Whoa.

• 19:45Speaker 03

  It's just all over the map.

• 19:46David Pierce?

  So, okay, this makes me think two things. One, this is an enormously hard job to get good at. And the kind of job that maybe isn't gonna like last forever. Like you, you train on this clothes thing, you do it for a minute. And then all of that knowledge is useless to you for the rest of your life, which seems like a wild way to use that kind of tool and time. Memorize these, you know, 40 pages of rules just to look at pictures for an hour. And then just kidding, we're gonna fail you anyway. And then on the other side, that this stuff is so, we're trying to teach computers how to think like humans, when maybe that's just impossible. That like, the idea of quantifying what outgoing is, like, I can't do that. You, I don't think you can do that.

• 20:31David Pierce?

  If you figure out how to quantify outgoing, that's congratulations. But so it just makes me think there's this like fundamental disconnect between what people are being asked to do, which is basically teach computers how to think like humans. But then in order to do that, you as a human have to think like a robot, thinking like a human. And it's just, I don't know how anybody would do this well. It seems insane.

• 20:51Speaker 03

  It is really kind of crazy making. And you start to question, you know, how, how do I know things? I've been making all these calls about, you know, what clothes are all my life. But do I really know? And you do have to kind of think like a robot. Like the trick is sort of being so literal, just sort of, you know, way more literal than any human could be and operate in the world. But in terms of the unpredictability and sort of the, the stop and go nature of it, that was the big complaint I heard from people. You know, there were complaints about the pay, like the pay can be quite low. But even more than that, people would complain about, you know, spending a day learning a task, doing a dozen of them, and then having it end and have to go learn a new task. Or there's a bunch of work and it pays pretty well. And you start doing it and then like a week later it's gone.

• 21:35Speaker 03

  And there's nothing for weeks and you don't know. Should I go find another job or what's happening here? It's quite stop and go, which, you know, talking to engineers and vendors, it has to do with the way machine learning development works. Like you need a ton of data to train your model the first time. And you want it really fast. And so you sort of get as many people working on it. You can, and then it's done for a little bit. And then you need to find, you know, something, something comes up. You need to sort of retrain it on something. You need like a smaller, more specialized group of people. It's very kind of peaky. And that means people are, you know, you always have to be on the lookout for like a new task to drop. And then you have to learn it. And then maybe that's going to be totally wasted because, uh, you know, work for a day and then the AI gets it.

• 22:20Speaker 03

  And then it's on to whatever else.

• 22:22David Pierce?

  Yeah, that's tough. What was the general sense you got from the folks you talked to about whether this is a good job? Obviously, like you're talking about the variability is tough. But did the folks you talked to, did they like the work? Was the, was it better than other things you could do? Kind of sitting in a computer, doing these kinds of tasks all day? Like, what, what was the vibe you got?

• 22:40Speaker 03

  It varied a lot. You know, I talked to a lot of people where a lot of this work gets done. And also where I happened to just sort of get placed in my remote task experience. And when remote task started out there in like 2018, 2019, it was a good job. Like it paid pretty well for the area. The people were making about 10 bucks an hour. But then, you know, more people joined the platform. They started adjusting pay based on regional cost of living. And then it dropped to like a dollar to $3 an hour. And, and even then, you know, people stopped thinking it was a career, but it was kind of a backup. Was my sense of it like this is better than nothing until I can find something else to do. But it was still so stop and go that they felt jerked around by it.

• 23:25Speaker 03

  It's a full-time job for a little bit, but like you can't depend on it. And then it's gone or then I'm banned for mysterious reasons. And then I did talk to people who, especially doing sort of the newer language model stuff, who enjoyed it even. Like a lot of this work is coming to the US because you need English fluent people and people with certain cultural references and what have you, or certain expertise. And, you know, they're making 20, 30, 40, $40 an hour. And a lot of them, it's like one of these, you know, work from home thing. They let, they enjoy it while they have it. No one knows how long it will last. I think the people who are getting paid the best, they wanted it to be a full-time job or dependable job. But like, they don't still know who they work for or, you know, when they're going to be done collecting data for it.

• 24:07Speaker 03

  So they're kind of just like, as long as the tasks are running, I'm going to, I'm going to do them.

• 24:12David Pierce?

  All right, we need to take a break and we will be back with more from Josh on humans and AI.

• 24:21Speaker 04

  Your heart can tell you a lot about your health. Apple Watch Series 12 measures your heart rate every five seconds with the most accurate heart rate sensing and wearable. So your vitals app, now with heart rate variability, can tell you when something is off. And your readiness score can let you know when to rest and when to push. Hear the story in every heartbeat with Apple Watch Series 12. The features described are for wellness purposes only and not for medical use. iPhone 11 or later required. Based on Apple conducted study of heart rate accuracy, August 2026. Visit apple.com slash Apple Watch Series 12.

• 24:52Speaker 01

  Ask yourself, what are your best people spending their time on right now? Expense reports, receipt chasing, month end close that takes weeks. You become what you spend on, and that's not what you're building toward. Brex is the intelligent finance platform that eliminates that work before it starts. AI agents that handle the manual stuff automatically so your team can spend their time on what actually compounds. It's time to get Brex AF. Learn more at brex.com slash AF.

• 25:32David Pierce?

  We're back. The expert level stuff was one of the things in your story that I really jumped out at me as something I don't really know anything about. Walk me through kind of what that looks like, because obviously, like you're saying, there's there's this huge amount of work that is teach a self-driving car what a fire hydrant looks like in every imaginable situation. Or, you know, does this chatbot response make any sense? That's the kind of thing that is hard work, but I think a lot of people could learn how to do it. But there's also, it seems like, this other side of the industry that is super specific and expert-driven and really different. How does that side of it work?

• 26:05Speaker 03

  Yeah, this is sort of where I think things are headed. You know, you say, oh, I would love you to see it, just sort of how many people think ChatGPT is giving them good information and, you know, submit legal.

• 26:18Speaker 03

  It's a attempt to submit like things before the court or something like that. But like, it's just very fluent at sounding accurate in a bunch of different fields. And so you need, like, you need an actual lawyer to go through it and see if it is making sense. Or you need, you know, an actual physicist to be, to see if it's just, you know, bullshitting you or whatever, or if it's actually being accurate. And so that has increased the demand for people who have specific backgrounds in whatever area you're trying to fine tune your system to do. And that can get quite technical, like, there's a lot of, you know, you need a lot of programming if you want it to program well. And so that is kind of the, the trajectory that you see, sort of as these things get better. You can foresee something similar happening to what happened with self-driving cars.

• 27:03Speaker 03

  So if you have a high stakes area, like driving a deadly vehicle around, you need to get everything exactly right. And so you have these fields where people are, you know, saying it's going to be great in medicine and health. You need that to stop hallucinating. And so you need medical professionals or people with some kind of medical training to go through and make sure that it's actually recognizing the cases it's supposed to be recognizing and not doing something weird.

• 27:29David Pierce?

  The thing that's the most, that really jumps out to me about that is, this is kind of a random aside, but it comes back around, I promise. I was doing a bunch of research on, do you remember Ask Jeeves, the search engine from, like, the early 2000s? Ask Jeeves was like the sort of natural language question answering service 20 years ago. We're, we're kind of back to the same idea now. And everybody thought Ask Jeeves was this sort of magic of technology, but what it actually was, was a bunch of human editors who worked at the company, going in and seeing the most popular queries and like literally just answering them. And so because a lot of people tend to search the same things, they were getting these very good answers. But what that doesn't do at all is scale, right? And so it was like, if you wanted to do one of the, let's say, 200 things that everybody

• 28:13David Pierce?

  else on Ask Jeeves wanted to do, you were in great shape. You were going to get incredibly helpful information. It was going to feel like magic because there was a human on the other side. But if you went down this kind of long tail of new stuff, and as we've seen on the internet, that long tail is infinite, it's just going to get Worse and worse over time. And part of me wonders, especially with the kind of stuff you're talking about, if we're going to land in the same place with a lot of these AI tools, because we're going to get to the point where like, if this question that you ask your chatbot has been seen and answered by a person who has actual expertise and knowledge in this space, you're likely to get a good answer because they've they've picked the right answer or responded, or even potentially like written some of the text that you might see. But even as you start to get one degree away from that,

• 28:57David Pierce?

  it seems like it's going to start to get worse. And it seems like there's this bet that if I just show you enough things, eventually we'll have the right answer for everything. And I'm just not sure that that reasoning holds up. Is this just like a forever problem that these companies are going to be chasing with folks like you're you're talking about forever?

• 29:15Speaker 03

  I ask people that and I could never get, obviously, they hope it isn't a forever problem. People working on these systems hope that at some point, you know, you add enough handwritten accurate examples and it and it sort of starts to learn something about accuracy or something like that. But, you know, I think I sort of as an outsider, from what I can see about how this stuff works, it seems like that's not going to happen right now. And it's kind of in the whack a mole, ask Jeeves phase where it's like, oh, someone asked it this thing and it did something totally weird. We need to have some people, you know, handwrite some examples to retrain it on so that now when you ask it this, it says something that is, you know, not dangerous or or not crazy or whatever.

• 29:59Speaker 03

  And and even if you do like a lot and a lot of that sort of work, because it still doesn't really reason in any kind of human way, there's always the risk that it's going to get stuff really accurate, seeming really authoritative, sort of get all the formatting right and the language right, but substitute, you know, wrong nouns or something, do something that just means it is like a little, it's it's it's off in some crucial way.

• 30:24David Pierce?

  Yeah, ChatGPT is very good at confidently lying to you, which is somewhere between delightfully fun and super dangerous, depending on the thing that you're asking at any given time. That was just the thing I came away from your story with is this feeling like what everybody wants to To happen is that we're just gonna train, train, train, use humans, use humans, use humans. And then one day, this switch will get flipped and it will work. And what I came out of this thinking is that actually, maybe this is going to be a more human process for longer than potentially anybody wants or has planned for. But that what's gonna happen is just that the people are gonna push it up ever so slightly every day as they keep working on this stuff, rather than feeding an engine that will eventually,

• 31:08David Pierce?

  as if by magic, figure out all the right answers.

• 31:11Speaker 03

  Yeah, that's my sense too, and it was kind of reinforced by seeing how much stuff that I had already assumed was super heavily automated is still relying on people saying, you know, this TikTok video is heartwarming, or this ad is too sexually provocative or something like it's not. Stuff that I had sort of assumed was sort of so, we had so much data on already, is still, you know, hand tool because norms change, language is confusing and ambiguous, and programs just haven't been able to sort of keep up with it. You always need kind of a human there, giving it new examples, making sure that it's adjusted. And so that's sort of what it looks like with language models. You know, it could be really high end stuff.

• 31:53Speaker 03

  You could have, I mean, you already have math PhDs and stuff working on annotations, but you're always gonna need some of that because it doesn't, it just doesn't think in a human way. There's always gonna be a little bit of a, an area that's not overlapping.

• 32:07David Pierce?

  Yeah, one of the questions you reckon with a bunch in the story is this debate over, is AI going to replace everybody's jobs? Or is AI gonna create a bunch of new kinds of jobs? Or some combination or something else entirely? Where do you land on, obviously, that's a huge question that we could spend many hours talking about. But do you have any kind of new thoughts on where you land after reporting this story?

• 32:33Speaker 03

  I love feeling like it's going to both automate and create new jobs in sort of the same area that, you know, one of the data vendor CEOs that I spoke with had a very convincing comparison to industrialization. You know, you have this task that used to be all done by one person, and then you break it down. Machines are quite good at one sort of, you know, whatever it is, sort of tool punching component of it. You automate that, but that means you need to change what everyone else is doing, and everyone becomes more specialized at their little task. And that's what you could see happening. I mean, it's what's already happened with remote tasks, like automated system tries to identify something, it can't, it goes to a human, they annotate the scene, that scene is then broken up into

• 33:16Speaker 03

  three things that go to other humans, they annotate it. You know, it's sort of this global assembly line with all of these little, little tasks that add up to doing some larger process, but it's hard to say what it is.

• 33:28David Pierce?

  Yeah, well, and I wonder how long that chaotic opacity stays useful to everybody, because on the one hand, people are going to get more sophisticated about how these kinds of systems work, right? But on the other hand, we're still in this era where I think a lot of people feel like the best strategy is just to get as much of this data as you can, throw it at the machine, and then kind of see what happens. And, you know, with the people sort of endlessly refining the rules about clothes is such an interesting one to me, because what that means is they're still learning how their system works. And what is happening is it is just this constant back and forth of trying something, learning new mistakes, and then trying something else, and then learning new mistakes. And at some

• 34:13David Pierce?

  point, we're going to hit a point where the stakes get high for this stuff. Right now, it's like, people don't really expect ChatGPT to be right, and when it's wrong, and you, you know, use it in court, like you were saying, it's like a hilarious joke. But at some point, this stuff is going to be the kind of thing that people rely on for really important, high-stakes stuff. And I wonder if that's going to be the moment where all of this stuff at every level of the process is going to have to start to be a lot more sort of transparent and understandable on all levels. Because otherwise, we're just running like, it's just black boxes all the way down, right?

• 34:48Speaker 03

  I think that's right. I think especially, you know, you could see in healthcare, you know, there's already heavily regulated some sort of, you know, supply chain transparency mandate. Like, you need to say, who is annotating what in this system, or something like that. Like, that would be one way you could see it unfolding, and maybe having that sort of with other... I mean, it's a supply chain, right? So it's like, you need to sort of say, what is your impact at these various places? What are people getting paid? What are they doing? What kind of, like, quality assurance do you have? I think something like that would probably be better for everybody. You would have to move more slowly, but the number of tasks I saw in my brief time on there, where it was clear that the engineers had no idea what was going on, and there was this sort of international game of telephone

• 35:31Speaker 03

  happening, you know, it happened kind of every time. I remember this task where I was supposed to be labeling, sort of, I think they're called, like, rare obstacles in the street for self-driving cars.

• 35:40David Pierce?

  Okay.

• 35:41Speaker 03

  And it would be, like, traffic cones, traffic control directors, wires, cables, like, potholes of a certain size, things like that. And sort of every couple minutes, I would be doing it, and then, like, log back in, and it would be like, oh no, don't label traffic control directors if they're not directing traffic, because I had just labeled someone, you know, on the sidewalk having lunch. Or like, okay, like, a truck that has a bunch of traffic cones in it, and it's like, you'd label it, and then come back and be like, no, only label traffic cones if they're blocking an obstacle in this street, or something. You know, it's this sort of, like, they don't realize sort of how their instruction being interpreted, and then what that does to the model. It's kind of this, this sort of horrible game of telephone that, that keeps happening, that you would think would be cleared up if, like, the people working on the model were doing

• 36:25Speaker 03

  the annotating, or were in the same room and communicating, so that people knew kind of what the intention was.

• 36:31David Pierce?

  What is the status of your remote tasks career? Are you, did you ever figure it out? Are you a good annotator? How are we doing now?

• 36:38Speaker 03

  Uh, I'm a terrible annotator, and then I got suspended, I think, because I was using a VPN, and now it just gives me this sort of sad face of a bridge I signed into my account. So my career is, is over for now.

• 36:50David Pierce?

  Which it sounds like is probably for the best on all parties, including those of us who want to use good AI. It's probably good that you're not annotating it.

• 36:56Speaker 03

  Yes. I think it's, I think it's good for me and good for the AI.

• 37:00David Pierce?

  Fair enough. All right, Josh, thank you. Really appreciate it. It was a really great story. Everybody should go read it.

• 37:05Speaker 03

  Thanks.

• 37:05David Pierce?

  We need to take a break, and then we're gonna call up Jen Tuwee and get deep in the weeds on how to build a smart home, and frankly, what that even means. We'll be right back.

• 37:19Speaker 02

  If you're looking to hire, you need Indeed. With Indeed Sponsored Jobs, you can spend less time searching and more time actually interviewing candidates who check all your boxes. Less stress, less time, more results. And listeners of this show will get a $75 sponsored job credit to help your job get the premium status it deserves at indeed.com/podcast. Just go to indeed.com/podcast right now and support our show by saying you heard about Indeed on this podcast. Indeed.com/podcast. Terms and conditions apply. Need to hire? This is a job for Indeed Sponsored Jobs.

• 38:03Speaker 05

  Support for the show comes from Pipe Drive. Sales teams spend up to 50% of their time on admin work rather than selling, building relationships, and closing deals. Pipe Drive gives them that time back. It's the intelligent AI-powered sales CRM loved by growing sales teams. Pipe Drive just launched meeting intelligence built right into the CRM. Before your meeting, it pulls deal history and past conversations into one brief. During the call, it records and takes notes. After, it auto-drafts the CRM updates, ready for you to review and approve. Head to pipedrive.com/fox to get started and get an exclusive 30 days free instead of the usual 14-day trial.

• 38:46Speaker 05

  No credit card or payment needed.

• 38:52David Pierce?

  Welcome back. Like I mentioned at the top of the show, last week was Smart Home Week on theverge.com, and we spent the week doing lots of stories about how to make your smart home work and the interesting stuff that people are doing with their smart homes and much more. It turns out, by the way, that The Verge staff has some wild ideas about smart home. There's a lot of good stuff on the site, go check it out. But I haven't been able to stop thinking about this one story that The Verge's Jen Patterson Tuohy wrote, in which she basically tried to start at the beginning and answer a simple question: What is a smart home and why do you need one? The story's great, we'll link it in the show notes, but it really got me thinking. Isn't it weird that that's a question you have to answer and such a complicated one? How did we get to the point where a smart home is even a thing?

• 39:37David Pierce?

  And should it be a thing? So, after a few days strangely deep in my feelings about all of this, I figured I'd just grab Jen and we'd talk it all through. Hi Jen.

• 39:47Speaker 06

  Hey David. Always a pleasure to be here.

• 39:49David Pierce?

  So I was reading your stories and I had this like weird existential crisis I did not expect to have in the middle of reading your story, which is that I've never really thought about the question, do you need a smart home and what is a smart home and what makes a home, a smart home, a thing. But the more I think about it, the more I kind of have come to the conclusion that we all talk about this exactly the wrong way, that we think about it as sort of this like big holistic system that is like you have to have a smart home and it's very complicated and involves buying a bunch of stuff and you have to pick a platform. And I kind of get to the end of this and I'm like, maybe the answer is just buy some things that connect to the internet, let

• 40:31David Pierce?

  chaos reign and try to sort of figure it out over time. So I almost wonder if like, do we talk about the smart home in sort of the wrong way at this moment? Like, should the smart home even be a thing as we think about it now? Or should you just have stuff in your house that's on the internet?

• 40:48Speaker 06

  So you are completely correct. We should not be calling it the smart home and we should not be talking about it like that. However, we are and you kind of can't go back. It's like the cat's out of the bag. In one of my pieces that we published last week, we had a big Verge Smart Home Week. In case you've missed it, go check it all out. I talked about this a little bit, that really the smart home is a misnomer because it's just the natural evolution of our home, right? Our homes are just getting better. Just like, you know, we went from candlelight to electric light. We're going from non-connected to connected. And it doesn't mean that you need to go out and make a smart home. You're right. Like, it's great if you can. I would love to, maybe not everyone. But if I had

• 41:33Speaker 06

  bought a brand new house that had just been built and it was all connected and all wired and, you know, wonderful, but- Ultimately, for most people, you know, that's not going to happen. So no, you don't need to go out one day and buy every single gadget and make sure you have everything working in your house. I think the most sort of natural approach to the smart home is adding devices as you need them to solve problems. And then, once you get to a sort of a point where you have something like critical mass, you may want to start making them all work together. And that could be after you've bought two light bulbs, or that could be after you've bought seven gadgets and suddenly you're like, oh, hang on, here I can do, you know, I have a smart garage door opener and I have smart lights. I can have my lights turn on when my garage door opens.

• 42:18Speaker 06

  So yeah, I think the problem with saying we all need a smart home is that that's a bit terrifying to people. It's like, oh, what am I doing wrong? What do I have to go out and do? No, you consider this natural, something breaks, you need something, consider a smart replacement, see if it's going to fit in your home. And I talked to a lot of companies about this and they all quite often say, yeah, I kind of wish we were, we didn't call it the smart home because that's causing us problems now. And it's not always about the internet too. You've got to remember the smart home isn't, doesn't mean you're necessarily connecting your home to the internet. There are ways that you can run a smart home locally. So yeah, but we're stuck with the label. It's a label that people recognize and, you know, we don't want to rebrand like, you know,

• 43:02Speaker 06

  what happened to Coca-Cola or Pepsi?

• 43:04David Pierce?

  True. I do appreciate that the same companies who, you know, five years ago were ramming the phrase smart home down everybody's throats at every available opportunity are now the ones being like, maybe we overshot and panicked customers about all of this. But that transition you're talking about, I think is super interesting because like, it makes me think of like, if years ago, when, you know, refrigerators suddenly got built in freezers, if everybody was like, I have a freezer home now, like, it just doesn't make any sense. But there does come a time where you go from, I have, you know, one thing that connects to the internet, like I have a garage door opener that I can control with my phone, to I have a home that can sort of be more than the sum of its parts and put some of this stuff together, like you're talking about. And I feel like the key question for a lot of people

• 43:49David Pierce?

  Is kind of where that moment happens and where you go from like, like my own experience is actually a good example of this, right? I have a couple of Philips Hue lights around that I just control with an app on my phone. I have a Honeywell thermostat that's connected to the internet that I control with a different app on my phone. That's all fine. I don't actually need more system than that, but I feel like I'm probably one or two things away from needing more system than that and also being able to do a bunch more stuff if I put all those things together. How do you identify that moment where it's like, okay, you go from just having a couple of things to like, you should build a system because it will actually make it better for you?

• 44:25Speaker 06

  So there are two answers to that. And in your case, I would say you're maybe one or two children away from wanting to do more. Because convenience is one of the big things. And for me, that was a tipping point, having children. Once my children were older and, you know, we had more need for routines in our home, the smart home is brilliant for routines. And I think that's where that can really help parenting or aging in place or elder care or any kind of specific life use case where you could really benefit from something being automated rather than you having to do things yourself, saving you time. Working from home is another great example. You know, setting up a home office and adding smart sort of automations to your home office.

• 45:10Speaker 06

  I know one of our editors, Dan Seifert, recently set up an ingenious automation that turns on a do not disturb light outside his door when he's on a Zoom call. You know, things like those kinds of solutions to problems that come up in your home. That's sort of one use case. And it's probably the primary reason to start looking at routines and automations because that's when you need the platform is when you want things to work together, when you want more than just one light to turn on, when you want your light to turn on, your thermostat to adjust, your shades to open, your door to unlock. That's when you start needing these platforms and ecosystems and more of a cohesive smart home. The second answer would be, and this was a piece, this is one of the pieces that we

• 45:52Speaker 06

  had last week was looking at smart homes with energy, smart energy management. So basically. Building homes that are better, like as we've gone from building simple homes without a lot of insulation or, you know, electrification. You know, now we've started to move towards better built homes that are better for the environment. They cost us lots of money to run. And now the next level is adding smart to that. We talked to people who lived in new smart community in California. That has been built from the ground up to use smart energy. It has like a smart electrical panel, has solar panels, but it's also been built to the point that insulation is so good that you don't actually need to use an awful lot of energy.

• 46:36Speaker 06

  So there's this kind of combination of looking after the environment, saving money and convenience kind of all comes together. And that's where I think the smart home sort of of the future has a lot of potential. And if you go back to the beginning of the current smart home, the DIY smart home. I know there've been smart homes for a long time, but you know, the start of where we kind of think of it today. The Nest thermostat was kind of the darling of the beginning of this current smart home. And that was all about saving energy, saving money. You could feel good because you're helping the planet and feel really good because you were saving money. So I feel like that kind of a use case is one where we're really going to start to

• 47:17Speaker 06

  see not only individual people wanting to make their home smart, you know, retrofitting with a smart electrical panel. But we're also going to see society as a whole pushing us towards making our homes better. And, you know, there's government subsidies, there's all sorts of things going on in this space that are really interesting. And then the other use case there, I think I mentioned, touched on it earlier, is aging in place. And again, this is another huge sort of societal shift as, you know, the baby boomers reach their golden years. And, you know, we've got a huge influx of elder care that's either going to fall on the shoulders of the next generation or is going to fall on society, you know, the, um, the state.

• 48:00Speaker 06

  And aging in place is a really compelling use case for the smart home. If you can keep someone in their home for longer and not have to move into a nursing home or have to go to assisted living, you're really, you know, helping everyone. It's a win win for, for everyone. So there's these kind of larger societal pictures that I think are often left out of the conversation about the latest smart doorbell or the best smart light. I mean, those gadgets are great and fun, but there is a much bigger picture here.

• 48:28David Pierce?

  Totally. And I think one of the challenges that I know you deal with in covering a lot of this stuff is figuring out kind of what is true now, what's going to be true in a year and what's going to be true in sort of 10 years, right? Because I think that idea of having all of this stuff set up in a way that just kind of magically works feels to me like it's just not where we are right now, right? Like it's we're still in the kind of like you have to build all the blocks yourself to make some of this stuff work. And it's getting a little easier and the platforms have made it easier to kind of build routines and stuff. But like you still have to say, when I get home, I want the thermostat to change. I want the garage door to open automatically. I want this light to turn on. I want this light to turn off and I want the oven to start preheating. The right way for all of this to happen is that that should just happen automatically,

• 49:13David Pierce?

  right? Or that some process of that should like learn how you live and adapt around you. And I feel like we're just at the beginning of that. And it's like, starting to happen in just enough really cool ways that it's like, I can see why that's going to be great, but it's not quite there yet. And so that's where I keep coming back to this idea of like, forget the idea of a smart home. Like, let's worry about a smart home in 2035, when AI makes all of this really great. For now, just like, buy the coolest, smartest stuff you can find that works for you. And don't worry about like, the big 30,000 foot system of it all, until you absolutely have to. And I think that's probably bad advice, because I think the thing that happens is you then kind of get out over your skis with too much stuff. And then you have to like retrofit a system to it. And that doesn't work that well either. But I don't know, we're in this like limbo position,

• 49:58David Pierce?

  especially with things like Matter, which will solve some of this stuff, that it feels like the less you worry about building a quote unquote, smart home for the moment, the saner your life will be.

• 50:09Speaker 06

  Yes, exactly. I think you're right. I think there are great use cases for individual or a few devices. And if you have an issue you want to solve, you want to save money on your HVAC bill, you buy a smart thermostat, you keep losing your front door key, you get a smart door lock, you know, and that doesn't mean you need a smart home just because you bought a smart door lock. I think you're right, AI could provide a huge shift in terms of that learning. Are homes actually starting to sort of think for themselves? And learning what we want and how our home should operate. I think ultimately our homes are going to become, you know, like our cars have become. They are going to be computers, but I think we're a long way from that still. I feel like, you know, for individual people today, you're right.

• 50:54Speaker 06

  Don't necessarily need to think the big picture. But if you have one of those use cases we discussed before, if you're scared of the whole process, because it is daunting, it is confusing. And one of the worst parts about it is once you get it all set up and it's all working great and your routine is, you know, kicking off exactly as you wanted and then it just stops. And then you're like, what happened? Like now you've become like a network sysadmin for your home and you spend your weekends troubleshooting why your smart sprinkler didn't go off and all your plants are dead or why you can't open your smart garage door right now, you know? And that's frustrating and that's where the smart home really starts to break apart

• 51:36Speaker 06

  when it causes more problems than it was designed to solve.

• 51:41David Pierce?

  I'm sure that's a purely hypothetical example for you and not something you deal with every single day, all the time, every hour.

• 51:48Speaker 06

  Yes, yes. My house is, yeah.

• 51:50David Pierce?

  What's the most recent smart home chaos you've had personally?

• 51:53Speaker 06

  Oh, well, my husband not being able to open the front door because I put a new smart door lock on and he couldn't figure it out and he was not happy.

• 52:01David Pierce?

  Yeah, this is why my wife just straight up doesn't allow most of these things. She's like, I'm not, I'm not worrying about this and when it breaks, it's not going to be my problem. Keys are great. It's going to be fine. And it's like, I'm like, yeah, you're right. But also what if I put a cool door lock on?

• 52:13Speaker 06

  But on that point though, and I should mention this, and it's not something we cover a lot, but there are integrators out there that will do all of this for you. And they have kind of mastered a lot of these problems and solved them for you if you're willing to pay the money.

• 52:28David Pierce?

  Right.

• 52:28Speaker 06

  Because, you know, it costs money to have these people come out and do them, although it's gotten a lot less expensive. They've obviously facing a lot of competition from the DIY smart home. So you can have some of these experiences without having to deal with all the troubleshooting yourself. It's just going to cost you a bit more money.

• 52:43David Pierce?

  Fair.

• 52:43Speaker 06

  So I think, you know, it doesn't have to be frustrating. And as I said, You know, one of the things that I really, I think my biggest piece of advice for people when they're looking to embark on the smart home is, as you say, start small. But when you scale, don't scale all at once. Scale small, because if you're like, oh, I love this, I'm gonna buy fifty smart switches and fifty smart bulbs and wire them all up and get it all set up. I'm like, make sure you realize that smart bulbs don't work with smart switches, because otherwise, you know. So add slowly, build slowly, you're less likely to run into some of those sort of more frustrating scenarios that we've talked about.

• 53:23David Pierce?

  Yeah, that's, that's very good advice. So let's talk about the platforms, though, because i- if, let's say, you've either hit the point where you kind of need to build a system or you just are the type of person who wants to kind of solve the system bit from the very beginning. You basically land on one of four platforms, right? There's Apple Home, Google Home, Samsung Smart Things and Amazon Alexa. Are those kind of the four? Is there any other one that even sort of belongs in that same category with those four?

• 53:47Speaker 06

  Well, there's Home Assistant is another one that is a very popular option for more advanced users.

• 53:53David Pierce?

  It's the open source one that like, it's, it's the one all the nerds really like. We can, it's just, it's just true.

• 53:58Speaker 06

  Yes.

• 53:59David Pierce?

  Okay.

• 54:00Speaker 06

  And, you know, and it also, it is more complicated. So, I, you know, I would generally recommend starting with one of the big four that you mentioned. I mean, there is a big sort of push and pull, you know, is it a platform? Is it an ecosystem? Is it a controller? But, you know, ultimately, those four are platforms that you can use to control your smart home. You can also tie those in to some of the other options out there, like Home Assistant or like Habitat or security systems like Aboad or Vivint, you know, there are other sort of platforms out there, but they can all also work with these four one of, or maybe more than one of these four big platforms. So these four big platforms do kind of,

• 54:39Speaker 06

  that they're gonna be present in every conversation about any kind of smart home setup. So you, especially if you want voice control.

• 54:46David Pierce?

  True. And it seems like maybe the simplest, kind of heuristic about which one to choose is just to sort of look at the devices you already own, right? That if you're a person who owns a lot of Apple stuff, there are good reasons to kind of go all in on Apple Home. And same if you have a bunch of, you know, echoes around your house. Is it kind of that simple that like the first question you should ask is what other devices do you already have?

• 55:06Speaker 06

  Yeah, I think it's what other devices you have on the smart home side and also just which which phone you use because these platforms are all pretty tightly tied to their smartphone. Although, you know, you, some of them are cross-platform, but you generally are going to have a better experience with Samsung SmartThings on a Samsung Galaxy device, with Google Home on a Google Pixel, and definitely with Apple Home on an iPhone because you can't use Apple Home on anything else. Amazon obviously does not have a smartphone, but that also makes it the broadest, so it works really well with any of of the smartphone options. Plus, if you have any kind of Fire TV or any kind of Fire tablets, there's really good integration there. So, yeah, I mean, ultimately, if you're like, I'm interested in the smart home, where do I start?

• 55:51Speaker 06

  If that's kind of your first question, then look at your phone and look at what you already have. If you already have an Echo Smart Speaker, if you already have a Google Chromecast, even if it's just one or two devices, if you have a Nest Thermostat, that's a good place to to start. You're not limited, in most cases, by that. So if you've already bought a device, it doesn't mean it rules out the other platforms. It's just a good place to start because you're already probably familiar with that platform to some extent.

• 56:17David Pierce?

  Totally. But, okay, so let's let's throw all of that out then, right? Let's say I don't own, I don't own any devices. I have never in my life bought a gadget. And I don't know why you're listening to the Vergecast, but welcome.

• 56:29Speaker 06

  You're a hermit and you've just moved in from...

• 56:31David Pierce?

  And you're trying to decide between these four platforms, are there sort of meaningful differences between them? It seems to me there are a lot of devices that work across two, three, even four of them. So the idea of like, I have to, you know, use this platform to use this light is less true than even it used to be. And it seems like in in most ways, they're very similar, these four platforms, in terms of like what they let you do and how they work. Are there big differentiators between them at this point?

• 56:57Speaker 06

  So originally, the biggest differentiator was which devices will work with which platform. And that has been a big issue, but that is what Matter was designed to solve because it is meant to bring in cross-platform compatibility. So the idea being any device that you buy, if it has the Matter logo, it will work with any of these platforms. We're not quite there yet. But that is the promise. So I would still recommend buy Matter devices if you're not sure which platform you want to stick with. There just aren't a lot to choose from yet. So that was a big differentiator. That was an issue, especially for people who chose Apple Home, because there weren't a lot of devices that worked with Apple Home and the ones that were, were more expensive. They had, you know, the Apple tax. So whereas

• 57:41Speaker 06

  everything worked with Amazon Alexa, like everything, although how it actually worked with Alexa would vary. So there was a bit more Wild West with Alexa than Apple Home was very tightly curated. But in terms of platforms, like features, each platform does have different features and different things that you might consider choosing it for. For example, Apple Home is very well known as its devices are for privacy. It keeps all your data local. It doesn't use it to sell ads to you. It also works locally if you have an Apple Home Hub. All processing will happen locally for things like video. So even though there is a cloud component to Apple Home, it does all work locally in

• 58:23Speaker 06

  your house, which a lot of people feel more comfortable with. It also means it's a little faster. Local control means that when you ask a light to turn on, it turns on almost as quickly as flipping a light switch. And you know, that's the ultimate sort of goal of the smart home. Let's make turning a light switch on just as fast as flipping the switch, but way cooler. And then, you know, Amazon has a great integration if you want to buy stuff, surprisingly enough. Very easy to buy yourself toilet paper when you're, you know, stuck in a crunch, assuming you have a smart speaker in your bathroom. Alexa is actually one of the more innovative platforms. There's a lot of really kind of fun, unique features that it offers. One in particular is hunches, which is an AI-generated kind

• 59:05Speaker 06

  of, it takes what it knows about how you use your smart home and makes suggestions. So for example, if every night you normally lock your door at nine PM, and then one night you forget to lock your door, and you say good night to your smart speaker, and it will say, oh, by the way, did you realize you didn't lock your door? And that's, it's hunch. And that, you know, that can be useful. But obviously, there are privacy implications there. You are Basically letting Amazon know everything you're doing in your house. So if you don't feel comfortable with that.

• 59:33David Pierce?

  Right, but that is in, as we were talking about with that kind of broader, your house learns how you work thing. If that is the goal, then Amazon is kind of one full step closer to that than just about anybody else.

• 59:43Speaker 06

  Very much so. And this is the real kind of push and pull with the smart home right now is privacy and data. You know, we want privacy in our homes completely. Everyone wants privacy in their homes. But if you want your home to be smarter, it needs data. It needs that context. That's the only way it's going to get smarter. So it's trying to figure out how we can provide data without providing intimate details of our life. And that's a real push and pull. And that's something that that Matter is looking at trying to help fix. But I think it's also something that AI could be really instrumental to also machine learning on the edge. So devices being able to process that kind of contextual data on device rather than having to send it to the cloud. So you're not sharing your data in the cloud.

• 1:00:28Speaker 06

  And then with Google Home and Samsung Smart Things, they both have really interesting sort of, I think one of the things with Smart Things, if you're interested in energy management, that's a really good platform right now. It's working really hard and innovating really well in that space. Also, if you have Smart Things, Samsung appliances, that's at the moment the only platform that really works well with integrating those appliances. It's, we're still a long way from smart appliances being super useful though. But you know, that's the future, I think, that we'll get more of that as we, as the smart home expands and develops. And then Google Home, there's been a lot of really good innovation recently there. It's taken a while. I think the biggest selling point of Google Home right now is its Nest products.

• 1:01:12Speaker 06

  The Nest hardware is good. It's some of the best in class. And it also has a lot of local component too. So it does the machine learning on the edge, a lot of AI processing locally rather than sending it to the cloud. But there is also, you know, Google is a data giant and you know, you do have to weigh that if you feel comfortable with with supplying Google with your data. And Google Home is also probably the better of the voice assistants.

• 1:01:39David Pierce?

  That's what I was gonna say, that's what it's been for me is the, the one that is most likely to hear me say, turn on the office light correctly is Google Assistant by a mile.

• 1:01:48Speaker 06

  Yeah.

• 1:01:49David Pierce?

  I don't know if it's just me, but Siri is, I mean Siri is a dumpster fire. Alexa's good, but Google Assistant, at least for me, in most of these cases, is sort of at least one order of magnitude better than than everybody else.

• 1:02:00Speaker 06

  I think it depends also what your main use case is, because the biggest problem I find with Google Assistant for smart home voice control is the delay. It doesn't have a local processing that Siri and actually Alexa does have on some of its devices. So I can find it can be quite slow. But Google does have a really neat, I love its presence sensing feature. I think its presence sensing is probably its best feature because it has any device, any Google Nest device in your home to determine if you're home, not just your phone. Most of the other platforms rely on phone geolocation, which does not work for me because I live in a cellphone dead zone. It, I once was lying in bed, I had Apple Home geolocation turned on, turn the lights

• 1:02:45Speaker 06

  on when I arrive home. The lights just kept turning on and off, on and off. And I was just sitting there with my phone and it's like Jenny arrived home, Jenny left home, Jenny arrived home. I'm like, nope, nope, still here. That was not fun. So whereas Google Nest can use like your Nest Protect smoke detectors, your Nest Hubs, your Nest Smart Speakers, your Nest Thermostat to tell if there's anyone in the home, not just you. That's the other thing, like if you have kids who don't have smartphones and you leave them at home alone because they're old enough, they may turn the heating off because it thinks you're gone. But, you know, so I like that presence sensing and that's one of the things that Google does a good job for. And we have actually had, we had four or five writers on The Verge go and write about their personal experiences with each of these platforms.

• 1:03:30Speaker 06

  So if you kind of want to dive more and find out what they like about them, we've got lots of great content on theverge.com.

• 1:03:37David Pierce?

  Totally. Yeah, it's, it's all very good. And I do think it's interesting to hear they've all very much sort of converged on how they think a lot of this stuff should work and like the apps all look increasingly the same, which I think is really interesting. But they do each still kind of have their own strengths and weaknesses.

• 1:03:52Speaker 06

  They do. And one of the nice things about Matter now, if, you know, is that we do have this multi-admin feature, which means that you can use any of the platforms with any of your devices. So if you have a Matter device, it will work with any of these platforms. So you don't have to commit. If you do choose a platform and you decide you don't like it, you should be able to move to a different platform relatively easily. As I mentioned, Matter is still in its infancy, but it is coming. We are going to have more devices that work with Matter and hopefully work out some of the bugs that we've had to this point. But yes, you can chop and change. You can try out different platforms. And if you get really good and really enjoy some of these platforms and you start to

• 1:04:33Speaker 06

  hit some of the limitations of the platforms, which are largely based on more complicated automations, like if you want your lights to dim only if it's after sunset and it's 30 degrees outside and your wife isn't at home. You know, if you want to add all these conditions and do some really cool, we've got some crazy neat examples of people's fun automations. We had a piece that our staff wrote about their favorite smart home setups. There's some really neat fun stuff you can do, like RealGeekout. And you might want to move to a different platform from one of these four platforms, sort of graduate to the next level, so to speak. And you can do that. Like, the beauty of the smart home is you really aren't completely locked in.

• 1:05:15Speaker 06

  There is ecosystem lock-in to an extent, but the walled gardens are beginning to sort of crumble and we're beginning to be able to choose what we want and control it the way we want. So that's what makes me excited about the smart home space, that we're getting to the point where you don't have to choose your Apple Home device to work with Apple Home. You can choose what you want.

• 1:05:36David Pierce?

  Yeah, no, it's good. And I appreciate that you come on The Vergecast at least once a quarter and remind everybody that Matter is great. It's going to be great. If everybody could just ship stuff with Matter in it, it would make everybody's life better. Can we all please make this happen immediately? Love The Vergecast.

• 1:05:52Speaker 06

  I do feel that way. There are a lot of people, and I'm with them, that think that Matter has not done its job yet. I completely agree, but I don't believe that that means it never will. I think it's been hampered very largely by the platforms and some of the manufacturers starting to shrink back a little bit and get a little nervous. And a lot of people are kind of doing, well, we're just going to wait and see if this is going to be good for our customers. Whereas the customers are going, we want Matter.

• 1:06:22David Pierce?

  Yeah, right. The answer is yes, everybody. That's it.

• 1:06:25Speaker 06

  But is it really, is it good for our customers or is it good for the bottom line? We're entering the political stage of Matter and I'm just hoping that we're going to get through it. Because otherwise, we can all enjoy the alternative.

• 1:06:37David Pierce?

  Which is pure chaos.

• 1:06:39Speaker 06

  Exactly.

• 1:06:42Speaker 06

  What is the alternative? Like, what else are we gonna do? Are we just gonna throw this in the bin? And then what? Go back to these walled gardens? So, yeah, we don't want to do that.

• 1:06:51David Pierce?

  Agreed. Awesome. All right, well, we gotta take a break. But Jen, thank you as always. We'll be back here to name and shame all of the people doing this wrong next time. Jen, thank you as always. We'll be back here to name and shame all of the people doing this wrong next time.

• 1:07:03Speaker 06

  All right. It's always a pleasure. Thanks so much, David.

• 1:07:11David Pierce?

  All right. Before we go, we have a hotline question. This one is less of a question, as you'll see, but just something I think is interesting, and I have some thoughts. So let's just play it back. This comes from DeeDee.

• 1:07:22Speaker 07

  Hey, it's DeeDee from Maplewood, New Jersey. Love your Verge cast. I was just listening to your assessment of the Apple Vision Pro. You were speculating about what to do with eventually augmented reality glasses, not being able to completely occlude your entire environment as the current Vision Pro can. So I just wanted to throw out there, well, what if the end game is simply contact lenses? Presumably, they would cover your entire retina, and then you could also presumably reproduce the entire VR experience. So maybe they'll do all encompassing goggles and make them smaller and smaller, and then eventually go directly to contact lenses. Just putting it out there to raise your thoughts. Thanks again.

• 1:08:01David Pierce?

  So the funny thing about this is this has come up for me a couple of times in talking to people. And I think it is kind of the long term trajectory of this that people talk about, right? You go from headsets to glasses. And then, OK, what's the next step after glasses? Obviously, it's contact lenses. And then you can have a debate about, is the thing after that, brain implants? Or, you know, what Neuralink is up to? And stuff like that. But I think contact lenses is a really interesting kind of far endpoint for a lot of this tech that would solve a lot of problems. And I guess I have two thoughts. One is that I do think there are a lot of people in the tech industry who believe that. I think there are, you know, whiteboards that have the word "contacts" circled on it as the long term goal.

• 1:08:43David Pierce?

  I truly cannot overstate how far away that is from a technical perspective. Obviously, things change quickly. Who knows what will have been invented a decade from now, but the number of essentially technical miracles that have to happen between now and then in power consumption, in how these things are produced, in the size of chips to make a contact lens possible is just staggering. Like, we are several orders of magnitude of technical invention away from something like that being possible. Not to say we'll never get there, I just am not spending any time holding my breath. But I do think that is an interesting endpoint of this. And I think as a product, becomes really fascinating. It also raises really

• 1:09:27David Pierce?

  complicated questions because they get harder to get off of your eye. That brings up different questions about like, are you more of a cyborg if you're wearing contact lenses versus if you're just wearing glasses that you can more easily take off? How do we power these things? How do they get software updates? How do they connect to the internet? A million open questions. And I think one of the things we're going to start to see is if you, to pardon upon, use contacts as a lens through which to look at a lot of technical advancement, you can start to see, are we getting closer to that as things get smaller, as things get more power efficient? Imagine a chip that gets hot that is sitting on your eye. Like, it's just totally incredibly implausible right now that that is going to happen anytime soon. But I do think there are a lot of people who are going to look at a pair of glasses

• 1:10:12David Pierce?

  and say, okay, we've done it. They look like Ray-Bans. What's next? So, DeeDee, I think you're right. I think that's a way we might be headed. I am unbelievably skeptical that we are even remotely close to being decades away from that. I think that is a, that is a really cool thing that still exists and pretty much only in science fiction. All right. That is it for the Vergecast today. Thanks to Jan and Josh for being here and thank you so much for listening. There's lots more on this conversation, a big story that we did in collaboration with New York Mag that Josh is talking about all of Smart Home Week on theverge.com was amazing. We'll put some links in the show notes. But as always, read theverge.com. Very good website. One of my favorites. If you have thoughts,

• 1:10:55David Pierce?

  questions, feelings, or crazy smart home hacks you want to send my way, you can always email us at [email protected] or call the hotline, 866-VERGE11. We love hearing from you. Send us all your thoughts and questions and ideas for what you want us to do on the show. We're going to have more hotline stuff every episode. Obviously, keep it coming. This show is produced by Andrew Moreno and Liam James. Brooke Minters is our editorial director of audio. The Vergecast is a Verge production and part of the Vox Media Podcast Network. Neelai, Alex and I will be back on Friday to talk about Pixel tablets, Spotify, sous vides, maybe Reddit, and lots more. We'll see you then. Rock and roll.

• 1:11:42Speaker 08

  Trying to locate news sources that reliably separate fact from fiction can seem like looking for a needle in a haystack. That's why The Guardian is launching Stateside with Kai Wright and Carter Sherman, a conversation with experts who slow down the news and wrestle with the questions we all have about what's actually happening in the world. Three times a week, hosts Kai and Carter utilize all the reporting resources The Guardian has to discuss the news, international affairs, climate, culture, sports, lifestyle, fashion and wellness. And The Guardian is not billionaire owned, meaning they're free to report the whole picture without interference. Go to theguardian.com/stateside to learn more. And listen wherever you get your podcasts or watch on YouTube.

• 1:12:27Speaker 08

  That's theguardian.com/stateside.

• 1:12:34Speaker 09

  There are sales teams out there who never seem to stop moving. Always a step ahead, always working with a level of precision that doesn't seem normal. It's no secret they're on Atio, the agentic CRM. Atio runs the work behind every win. It's built for humans and agents with the guardrails of human software and the scale of agentic infrastructure. This is the frontier of go-to-market. Teams like Parallel, TurboPuffer and Wordsmith are already using Atio to set the pace. Now it's your turn. Try Atio for free by going to atio.com/vox. That's A-T-T-I-O dot com slash vox.