ChatGPT's human editors

Up to a billion people use ChatGPT every week, for everything from the mundane to the weird to the deeply personal. It is commonly assumed that these messages are private – between the user and the LLM. It is commonly assumed that these messages are private – between the user and the LLM. But in some cases, human reviewers are reading these conversations and offering editorial feedback to help ChatGPT sound more human. Reporter Joseph Cox of 404 Media broke the story of Project Lily, Open AI's initiative to use human reviewers to train and improve ChatGPT. He breaks it down for us on this week's Endless Thread. Show notes: Inside ‘Project Lily’: The Humans Reading Your ChatGPT Chats (404 Media https://www.404media.co/inside-project-lily-the-humans-reading-your-chatgpt-chats/) This episode was produced by Grace Tatter and Frannie Monahan. It was co-hosted by Grace Tatter, Ben Brock Johnson, and Amory Sivertson, and edited by Dave Shaw. Mix and sound design by Paul Vaitkus.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Support for this podcast comes from Is Business Broken?, a podcast from the Marotra Institute at BU Questram School of Business. A recent episode looks at what the rise of the gig economy means for workers, businesses, and the future of work. Stick around until the end of this podcast to preview the episode.

• 0:19Speaker 01

  Flowing ad budget on metrics that look great till the CFO sees them? That's bull spend and marketers are calling it out in dashboard confessions.

• 0:29Speaker 02

  I remember telling my boss, it'll be good for the brand when leads were slow. Yeah, it wasn't.

• 0:36Speaker 01

  Cut the bull spend. LinkedIn lets you target by company, job title, and more. Advertise on LinkedIn. Spend $250 on your first campaign and get a $250 credit. Go to LinkedIn.com/campaign. Terms and conditions apply.

• 0:50Speaker 03

  Whenever fall comes back around, it always feels like a fresh start. This year, I've been layering in some styles from the new Lululemon fall drop because the weather is shifting and I find I have to be ready for the entire day and not just one part of it. Lululemon's fall line has some really versatile transitional pieces so I can just wake up, throw on a matching set, add a jacket or a fleece, and head out, knowing that I'll feel comfortable enough to go wherever the day takes me. Check out the new fall arrivals in store and online now at lululemon.com.

• 1:20Speaker 04

  WBUR Podcasts, Boston.

• 1:29Speaker 05

  Okay, Endless Thread co-host Ben Brock Johnson and Amory Sivertson. Today, I, producer Grace Tatter, am your boss. I'm the boss.

• 1:40Amory Sivertson?

  Ooh, okay.

• 1:43Ben Brock Johnson?

  Is this about the TPS reports? Because Amory was supposed to do this. I didn't. It wasn't me.

• 1:50Amory Sivertson?

  A likely, likely story, Ben.

• 1:55Speaker 05

  I don't even know what TPS is, but I have an assignment for you.

• 2:00Ben Brock Johnson?

  Okay.

• 2:00Amory Sivertson?

  All right.

• 2:01Speaker 05

  I'm gonna Slack you a ChatGPT exchange and Ben, I want you to read the prompt and Amory, I want you to read ChatGPT's response.

• 2:12Amory Sivertson?

  Okay.

• 2:13Ben Brock Johnson?

  Okay. The prompt is, "Write me a LinkedIn post promoting the podcast Endless Thread."

• 2:22Amory Sivertson?

  And the answer is, "Some of the internet's strangest stories start with a simple question. Wait, how did that happen? That's the magic of Endless Thread, a podcast that dives into the fascinating, surprising, and sometimes downright bizarre stories hiding across the internet. From Reddit rabbit holes to unexpected human stories, Endless Thread explores the communities, mysteries, and conversations that make the internet such a strange and compelling place. Headphone emoji. If you love great storytelling, curious questions, and discovering stories you didn't know you needed to hear, give Endless Thread a listen. You might start with one episode and end up three hours deep in an internet rabbit

• 3:06Amory Sivertson?

  hole. #Podcast, #EndlessThread, #Storytelling, #Podcasts, #InternetCulture, #Audio."

• 3:19Speaker 05

  So on a scale of one to seven, one being unusable and seven being the best, how are you going to grade this response?

• 3:29Ben Brock Johnson?

  Well, why does it have to be one to seven? That's like a weird.

• 3:32Speaker 05

  I, look, I'm the boss of you, but I'm not the boss of the boss.

• 3:38Ben Brock Johnson?

  Okay.

• 3:38Speaker 05

  I did not make the rules.

• 3:40Ben Brock Johnson?

  I would give it a four.

• 3:43Speaker 05

  Okay.

• 3:45Amory Sivertson?

  Yeah, I would give it a three because I'd say the information is fine, but this is not at all how Ben or I write.

• 3:54Ben Brock Johnson?

  Not voicey.

• 3:55Amory Sivertson?

  It's not voicey, you know? Whoever wrote this, whatever wrote this, it's not enough of a goober. You know what I mean?

• 4:03Ben Brock Johnson?

  Yeah, no spunk.

• 4:06Speaker 05

  So if you're grading this, that's what you'd put in the notes?

• 4:09Ben Brock Johnson?

  I'd put more spunk and then I'd underline it three times.

• 4:14Amory Sivertson?

  Watch, the plot twist is Grace wrote this.

• 4:16Ben Brock Johnson?

  Yeah, exactly. Don't care about the clankers. Oh, never mind. Grace is the clanker. Oh, well.

• 4:22Speaker 05

  So you are reading real ChatGPT conversations and grading them. You're trying to make ChatGPT sound more human. That's what I have hired you to do.

• 4:31Ben Brock Johnson?

  Oh, God. I've just talked to somebody who like left journalism and this is their job now, is they're like reading prompt responses and grading them and talk about.

• 4:45Speaker 05

  That's a job?

• 4:46Ben Brock Johnson?

  Yeah, dude, it's grim. It's frickin' grim.

• 4:50Speaker 05

  But it pays pretty well. Do you want to guess how much it pays?

• 4:55Amory Sivertson?

  Yes.

• 4:55Ben Brock Johnson?

  Forty, forty, forty-eight dollars an hour.

• 4:57Amory Sivertson?

  I'm gonna say sixty dollars an hour.

• 4:59Speaker 05

  Yeah, okay, you guys are right. Yeah, it's like fifty to a hundred dollars an hour, which, look, for people coming out of the journalism profession is not bad. Not bad at all.

• 5:07Amory Sivertson?

  Yeah.

• 5:08Ben Brock Johnson?

  I think my starting job in journalism paid around sixteen dollars an hour, so.

• 5:13Amory Sivertson?

  Yeah, mine did too.

• 5:14Ben Brock Johnson?

  But just for inflation, of course.

• 5:20Speaker 05

  So several people have been posting an email recruiting for this type of job. So probably you're the person you talked to, Ben, got a similar email. I know people who have gotten these emails. I want you to take a look at this Reddit post.

• 5:35Amory Sivertson?

  Posted in r/jobs, the title of the post is, "Is Crossing Hurdles a Scam?"

• 5:44Amory Sivertson?

  The email says, "Hi, I'm from Crossing Hurdles, a global recruitment firm. We would like to refer you for an interesting opportunity that involves leveraging your expertise to train AI models. You'll be matched with relevant opportunities aligned to your expertise in strategic analysis, operational modeling, and stakeholder communication." That was written by AI, no doubt. It's just like, what? What will I be doing? No idea.

• 6:08Ben Brock Johnson?

  This posting reads like BS to me. I will say that.

• 6:12Amory Sivertson?

  What about it?

• 6:14Ben Brock Johnson?

  This just reads like so many of the scam job, like work from home, flexible hours. Like, I just feel like we've seen this in the past, this kind of language. It's, it's almost like receiving a message from a Nigerian prince. You know what I mean?

• 6:30Amory Sivertson?

  I don't like that it's "join expert network, management consultant." It just feels like random employment words smashed together.

• 6:39Speaker 05

  Yeah, you can see why the original poster here and a lot of the commenters thought it was a scam. But also, you just like can't really tell, like, what is this, what is this job? But this company, Crossing Hurdles, is real and so is the company that they're recruiting for. In the job description, it says they're recruiting for Merckor. And what Merckor is doing is hiring people to do what you all just did, to read real ChatGPT conversations. And Merckor is contracted by OpenAI. And this is called Project Lily, this project where they're trying to make ChatGPT basically sound more human.

• 7:21Speaker 05

  So the reason we know about Project Lily is because of a reporter named Joseph Cox.

• 7:29Speaker 04

  I'm a co-founder of 404 Media and I'm the host of the 404 Media podcast where we investigate and talk about surveillance, privacy, the digital underground, AI, and increasingly the intersection of all of those because they actually do overlap a fair bit nowadays.

• 7:49Speaker 05

  What is Project Lily?

• 7:52Speaker 04

  So Project Lily is this internal program at OpenAI that uses contractors to read, review, analyze real ChatGPT users' prompts. And I'll just say straight away that this is different to stuff around safety, right? We've seen a lot of concerns about safety when it comes to OpenAI and ChatGPT, where the very sycophantic 4O model, which was really sucking up to people, that has allegedly been part of people unfortunately taking their

• 8:32Speaker 04

  own lives and doing all sorts of other dangerous things. OpenAI has introduced humans to try to mitigate that, to review potentially dangerous messages and that sort of thing. This, Project Lily, is very different. This is not really about saving anybody's life or really protecting anybody, per se. It is much more about making the product better. It is about improving the models.

• 9:05Speaker 05

  So at the beginning of this conversation, I gave you the pretend assignment where you read the Endless Thread, fake Endless Thread LinkedIn post that ChatGPT came up with. But Joseph has seen the dashboard you'd really see as a contractor working on Project Lily.

• 9:23Speaker 04

  They log in, they select what is called a task, they are then presented with a real ChatGPT user's prompt, they are told to summarize what they think the user is actually asking. You know, is this person asking you to make a slide deck? Are they asking you to reword something or whatever? The contractor is then asked to sort of rate and review a series of responses generated by.

• 9:50Ben Brock Johnson?

  Chat GPT sort of select the best one and explain why. It is kind of quite boring, rote, mechanical work. But they're trying to inject a human element and human taste and decision making into the training of these models.

• 10:10Speaker 05

  Which is interesting, right? Because as popular as Chat GPT already is, it seems like OpenAI still thinks that they need a human touch to give their product an edge.

• 10:24Amory Sivertson?

  I don't like this. I really don't. I will say, I come from a, I'm really resisting the robots in my life. And when I find out that AI has been used for something, or if I can tell that AI has been used for something, it's like a real turn off to me. And so the idea that we are now paying people to help the robots sound more human, I just feel unsettled by all of it. And I think that we are contributing to our own demise.

• 11:03Amory Sivertson?

  One, one job post at a time or one, one prompt at a time. And I know that that is not an opinion shared by everyone. But, but yeah, I'm, I'm really uncomfortable with it.

• 11:15Ben Brock Johnson?

  I have a slightly different view. The difference maker for me is I would feel a lot better about this if I felt like the governor, you know, you know how cars have governors on them? Does that make any sense to you, Grace?

• 11:33Speaker 05

  No, I thought governor was a person I voted for.

• 11:38Ben Brock Johnson?

  When a governor is in a car, it makes sure that the car's maximum speed is not unsafe.

• 11:46Speaker 05

  This is a device.

• 11:48Ben Brock Johnson?

  It's a device that's in a car.

• 11:51Ben Brock Johnson?

  And I think what is missing from the world of AI right now is governance more broadly. And what I mean by that is safety checks on how AI gets developed and what we do about it and how we define danger to humanity, et cetera, et cetera. And I think like, to me, the, the idea that a robot is going to take over some aspects of our jobs across industries and across different kinds of work, we should accept that reality, but not without care. And I think right now, the world we're in, there's like no care being applied.

• 12:34Ben Brock Johnson?

  There's no governor in the engine. And that's where I get worried. And that's when I get into a similar headspace of Amory of just like, ah, no, never mind. I'm, I'm out. I'm out the game.

• 12:47Speaker 05

  Yeah. So how these reviewers are told to like, do this job to help maintain the car that is Chat GPT and make it even better. They're told to highlight at least three parts of Chat GPT answers and explain what's wrong with them. And they've been told some specific things to watch out for.

• 13:08Ben Brock Johnson?

  A Chat GPT response should be given a lower score if it is just full of emojis. Because I think as we've probably seen, if you've been on LinkedIn for more than five minutes, everybody on there is using AI to write their posts. And it is full of emojis and tick lists and great, amazing takeaways from the forfluencers who all over LinkedIn. Chat GPT is trying to get rid of that.

• 13:36Speaker 05

  So earlier, we referenced like in the post that we were, that you all were grading, there was like one emoji. It didn't seem, that didn't seem to be a big factor in your score. But if you are working for Project Lily, maybe you would highlight that. Or I don't know, there's other language, like, do you remember anything that you would have, if you had to highlight three parts that you would be like, ah, this is what I'm going to flag. Do better, Chat GPT.

• 14:02Amory Sivertson?

  I would highlight the use of the word fascinating. That makes me go, ugh. If you love great storytelling, that jumps out to me as feeling like, eh. Of course, everybody loves great storytelling.

• 14:19Ben Brock Johnson?

  Also, this, this phrase jumped out to me, unexpected human stories. For some reason, it's like, who's, what's unexpected here? Is it the stories are unexpected? We don't expect the stories when they show up? Is the, are the humans, do we not expect humans? Like what, I don't know. For some reason, that phrase jumped out to me and bothered me.

• 14:42Speaker 05

  So human reviewers are looking like at stuff like emoji use and just like language that's too robotic, roboty. But they're also looking to see if Chat GPT is too sycophantic, which as Joseph mentioned above, as we heard to mention above, has been a problem for OpenAI, particularly regarding suicide and suicide prevention. And these human reviewers are checking to see if Chat GPT is anthropomorphizing itself. So like, if it refers to having experiences it can't have because it is a robot, like cooking, if it's like, oh, when I cook dinner, I do this. You're, as a reviewer, you're going to flag that. You're going to be like, no, you didn't, because you don't exist. You can't cook.

• 15:28Amory Sivertson?

  Yet.

• 15:29Speaker 05

  Yes.

• 15:32Speaker 05

  So Joseph spoke to a contractor who actually does this work. And they told him that the guidelines change a lot, like the stuff like looking for emoji use, like what the, the tells are.

• 15:44Speaker 05

  There's at least one more thing that these human reviewers are going to be on the lookout for. And that's personal information. We're going to talk more about Project Lily and its implications for privacy after a break.

• 16:11Speaker 00

  Support for this podcast comes from Is Business Broken? A podcast from the Marotra Institute at BU Questram School of Business. What gets lost when work stops looking like a nine to five job? If I am not committing to be working with you exclusively at a particular point in time for a certain amount of time to do certain types of work, then you can't necessarily expect the employer on the other side to provide guarantees either. Follow Is Business Broken wherever you get your podcasts and stick around until the end of this podcast for a preview of a recent episode about the changing dynamics of the gig economy.

• 16:49Speaker 01

  Flowing ad budget on metrics that look great till the CFO sees them? That's bull spend and marketers are calling it out in dashboard confessions.

• 16:59Speaker 02

  I remember telling my boss, it'll be good for the brand when leads were slow. Yeah, it, it wasn't.

• 17:07Speaker 01

  Cut the bull spend. LinkedIn lets you target by company, job title and more. Advertise on LinkedIn. Spend $250 on your first campaign and get a $250 credit. Go to LinkedIn.com/campaign. Terms and conditions apply.

• 17:20Speaker 03

  Whenever fall comes back around, it always feels like a fresh start. This year, I've been layering in some styles from the new Lululemon fall drop because the weather is shifting and I find I have to be ready for the entire day and not just one part of it. Lululemon's fall line has some really versatile transitional pieces. So I can just wake up, throw on a matching set, add a jacket or a fleece and head out, knowing that I'll feel comfortable enough to go wherever the day takes me. Check out the new fall arrivals in store and online now at lululemon.com.

• 17:51Speaker 05

  Okay, Ben and Amory, we are talking about OpenAI's Project Lily in which human reviewers help train Chat GPT by reading users' real conversations with it. About, yeah, Amory's not a fan.

• 18:06Amory Sivertson?

  No.

• 18:07Speaker 05

  About a year ago, we did an episode about AI and relationships. And we asked listeners how they were using LLMs. Do you remember some of those use cases?

• 18:19Ben Brock Johnson?

  Yes.

• 18:20Speaker 05

  Like what were they?

• 18:20Ben Brock Johnson?

  I got into a disagreement with my wife and I tried to figure out how to navigate that disagreement by talking to Chat GPT and formulating my arguments better before I go back to argue with my wife.

• 18:32Speaker 05

  Yeah, like it was pretty personal stuff. People were telling, were asking GPT about like the intimate details of their lives. Sometimes using Chat, I think we got people who are like, I'm basically using this like a therapist. And I, I don't know, because we didn't ask this at the time. But I bet a lot of them were probably assuming that only a machine would read whatever they were writing, right? Like they weren't assuming that a person, a real person might be reading this.

• 19:00Ben Brock Johnson?

  Okay, okay.

• 19:02Speaker 05

  And Joseph says that that's actually been kind of a selling point for a lot of these big LLMs.

• 19:07Ben Brock Johnson?

  A lot of the recent marketing from Chat GPT and OpenAI is that, hey, if you're too embarrassed to ask a human this question, you can ask it to us and we will answer it in confidence and no one will know about it.

• 19:24Speaker 05

  Do you want to take a guess at how many users Chat GPT has per week?

• 19:29Ben Brock Johnson?

  680 million.

• 19:31Speaker 05

  900 million.

• 19:34Ben Brock Johnson?

  Ah, get out of here.

• 19:36Speaker 05

  And some estimates say in August, it reached a billion users per week, which is just really like blows my mind.

• 19:45Amory Sivertson?

  But these are people who use things for like, you know, like Chat GPT, what's an egg replacer in my, in my muffin recipe? To people who are like in a quote unquote relationship with a chatbot, right? So it's, these are just users generally. It really runs the gamut in terms of, okay.

• 20:05Speaker 05

  And who knows how often they're using, but like a lot of people are turning to this every week.

• 20:09Ben Brock Johnson?

  I seriously doubt even 1% of those users or something have any idea that a human may look at the prompts they're entering into that machine.

• 20:23Speaker 05

  So OpenAI has said that they remove personal information before these chats are reviewed by humans. And I asked Joseph how they said they did that.

• 20:33Ben Brock Johnson?

  First of all, in the dashboard, when I've seen it, it has removed sort of the username of the user. So that's not there. So it's sort of de-identified somewhat in that way. OpenAI also says it uses this privacy model to

• 20:48Speaker 05

  Process the prompts before they reach the contractors. So this will do stuff like look for names, look for email addresses, look for phone numbers inside the prompt and then redact or remove them.

• 21:04Speaker 05

  Ben and Amory, do you all feel like that's sufficient? Even if it's just a tiny, tiny percentage of conversations that these human reviewers are looking at?

• 21:13Amory Sivertson?

  I mean, it's more than I was expecting, but I'm sure that there's something that I'm not even thinking of that's going to make me feel horrible again.

• 21:23Ben Brock Johnson?

  Yeah, I don't know. I mean, it's really hard because people can be identified by a very small amount of data these days, I feel like. So hard to answer that question.

• 21:33Speaker 05

  Yeah, well, first of all, the algorithms or whatever model they have in place aren't even catching all of that personal information before humans see it. So part of the job of these Project Billy reviewers is just to flag like, oop, some personal information slipped through. But Ben, like you said, almost any piece of information is personal when you're able to put it into context. These prompts are sometimes so detailed and people don't think their communication is going to be read by a human. That includes a lot of information that, yes, it may not have their name, but I'm pretty sure there was the right prompt. You would be able to figure out who was typing that.

• 22:16Speaker 05

  Even in 1997, there was something called the Bill Weld experiment. And this researcher found that all you needed to figure out someone's identity was their zip code, their gender and their birthday. And she was able to de-anonymize them, governor of Massachusetts, health data from a data set. So like, it takes very little information to figure out who someone is. Perhaps less than people realize.

• 22:45Speaker 05

  So Joseph knows this because he's a journalist who covers privacy and surveillance. And we know this, but a lot of people probably don't realize like what type of information actually is personal, actually can lead right back to their names. OpenAI does disclose this human review, by the way, at least kind of. But it was previously pretty hard to find. Since Joseph first came out with this story, OpenAI has made it a little easier to find out how to opt out of this human review. Now, do I think it still could be more explicit? I mean, yes, because I still don't think the vast majority of users are going to have any idea, especially when you compare it to, I don't know, something like Google's Gemini,

• 23:29Speaker 05

  which unfortunately is forced on me at work because we run some of our email infrastructure on Gmail. You open a Gemini chat and it says, humans may review this content. And it is right there in front of you. I don't think ChatGPT is doing that yet. Joseph's reporting was on OpenAI and Project Billy specifically, by the way. But he did also reach out to Anthropic, whose LLM is Claude. And they said that they too use human review. And Google Gemini is disclosing this a little bit more clearly. So a lot of these big commercial LLMs are doing some version of this. So I think that people just need to treat this less like a group chat, like a signal where you know it's encrypted,

• 24:13Speaker 05

  and more like a post on social media, which, yes, it may not be fully public, but it's almost semi-public in a way in that a human could actually review this later on.

• 24:26Speaker 05

  So I think that would be a pretty big shift for a lot of people to think about ChatGPT less like a private conversation and more like posting on social media. But I thought that that was an interesting idea. How do you think more people knowing about Project Billy or maybe having this shift in thinking might affect how they use LLMs?

• 24:49Ben Brock Johnson?

  I will say I don't love the, and I get it because we all do this. Like I do this all the time. I'm like, you know, what we, what we really should do is as users, be a little more thoughtful about how we behave in this world. And I just feel like it's like one of those things where it's like, nobody's doing that.

• 25:09Amory Sivertson?

  And they shouldn't have to. I mean, this is, yes, it kind of, at this point, takes a combination of users changing their behavior with this technology. But, you know, it also takes users demanding that the data be treated differently by the overlords that are making this technology available and sort of encouraging people to use it in more and more personal ways. Yeah, I think that is, that's good savvy advice for the individual. And also it's going to be really hard to follow. And instead, maybe we say, no, sorry, you can't read our conversations.

• 25:52Amory Sivertson?

  In which case they'll say, okay, well, then we can't make this technology any better. In which case I'll say, great, get rid of it. I'm done.

• 26:00Speaker 05

  And historically, people often have taken the path of least resistance, even when confronted with some serious privacy violations when it comes to tech. Like a lot of times we find out about privacy violations and we're just like, hmm, but it's so convenient. So I'm going to pretend I didn't hear that. But to both of your points, like this can't be done on the individual level because privacy is a team sport. Like just because you never post something personal to ChatGPT, like, I don't know, maybe your partner is uploading all of your arguments to it. Or maybe your boss put some information about you at work into it. Like, we kind of have to, as a society, like come up with norms of how we're going to think about privacy

• 26:46Speaker 05

  because we're all in this together, like individuals. It's just a drop in the bucket ultimately.

• 26:54Amory Sivertson?

  Thank you, Grace.

• 26:55Ben Brock Johnson?

  Thank you for looking into this. And it's, it is fascinating how this stuff is continuing to evolve and.

• 27:03Amory Sivertson?

  Fascinating, you say, Ben, is it?

• 27:06Ben Brock Johnson?

  Busted.

• 27:09Speaker 05

  Okay.

• 27:09Ben Brock Johnson?

  I'm going to go down another rabbit hole, rabbit hole, rabbit hole.

• 27:12Amory Sivertson?

  Is Bot Ben filling in for Ben on this taping?

• 27:24Speaker 05

  Endless Thread is a production of WBUR in Boston. This episode was produced by Frannie Monahan and me, Grace Tatter.

• 27:33Ben Brock Johnson?

  It was co-hosted by me, Ben Brock Johnson.

• 27:36Amory Sivertson?

  And me, Amory Sivertson. Mix and sound design by our production manager, Paul Vikes. It was edited by Dave Shaw. The rest of our team is Emily Jankowski, Dean Russell, and our senior managing producer, Sumitajoshi.

• 27:48Speaker 05

  Endless Thread is a show about the blurred lines between online communities using AI for work and AI using you for work.

• 27:57Ben Brock Johnson?

  If you have an unsolved mystery, an untold history or another wild story from the internet, or you want our help in murdering robots before they murder us, you can send us an email, [email protected]. Bye.

• 28:30Speaker 00

  Support for this podcast comes from Is Business Broken, a podcast from the Marotra Institute at BU Questram School of Business. Follow Is Business Broken wherever you get your podcasts and listen on for a preview of a recent episode featuring Brandeis University Professor David Weil on a question at the heart of the gig economy. When an app controls how you work, is it really just a platform or is it your employer?

• 28:56Amory Sivertson?

  It is very true that the platforms allow, let's say, drivers to find riders much more easily in a way that would be impossible absent the Uber or Lyft technology. Yes, the technology is connecting people, but the technology is also doing all the kinds of things that we usually associate with management. Setting the rate of pay, setting the pace of work, deciding on who gets the job and who doesn't, monitoring the performance of that party. There's a lot that these technologies do that are all about what we would call management.

• 29:37Speaker 00

  Find the full episode by searching for Is Business Broken wherever you get your podcasts and learn more about the Marotra Institute for Business, Markets and Society at ibms.bu.edu.

• 29:52Speaker 03

  When fall comes back around, there's a sense of a reset. And new pieces from Lululemon's fall drop are an inspiration to push your goals and elevate your routines. As your calendar fills, easy-to-layer styles meet the demands of your day. And as you challenge yourself to try even more, versatile sets transition with you. The Lululemon fall lineup brings new levels of comfort and confidence, whether you're starting something fresh or starting something over. Check out the latest arrivals in store and on lululemon.com now.

• 30:22Speaker 00

  Hey, Bose here. On your next track, listen for the decisions. Distortion on the bass, the vocal right up front. Every choice deliberate. With Bose, you'll hear it all. Your music deserves Bose.