How Amazon Automated My Job: A Conversation With Elaine Kwon

Elaine Kwon was a vendor manager in Amazon's retail organization when the company started to turn her tasks over to machine learning in a program called "Hands Off The Wheel." She joins Big Technology Podcast to discuss how and why Amazon rolled the program out, how she and her fellow vendor managers reacted, and what it portends for the rest of us. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:03Alex Kantrowitz?

  Hello, Elaine.

• 1:04Speaker 01

  Hey there. How are you doing?

• 1:06Alex Kantrowitz?

  I'm good. Great to see you again. It's been, I think, probably two years, so we have a lot to catch up on.

• 1:11Speaker 01

  We do. We do. Thanks for having me on.

• 1:14Alex Kantrowitz?

  Oh, yeah, it's great to have you. I'll just pause for the ad break and then we'll get started.

• 1:29Alex Kantrowitz?

  Hello and welcome to the Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. We talk a lot about automation on this show, and today we're going to speak with someone who's seen it firsthand. Elaine Kwon is co-founder and partner at Quantify, and before that she was a vendor manager at Amazon. And so it's white-collar automation program called Hands Off the Wheel, Get Off the Ground, and Go Operational. She joins us today to talk about her experiences and the implications automation might have on work and our global economy. Elaine, welcome to the show.

• 2:04Speaker 01

  Thank you for having me, Alex.

• 2:06Alex Kantrowitz?

  Thank you for coming on. We spoke in Seattle back when things were in person. As I was trying to research what Hands Off The Wheel was all about for my book Always Day One. And the conversation just stuck with me and I've thought about it, you know, over and over as time goes on. So I thought, well, we talk theoretically about this stuff often on the show. Let's talk about how it actually works from a practical standpoint. So I'm really stoked to have this conversation.

• 2:36Speaker 01

  Sounds good. And I would say the implications that that has had even since 2016, which is when a lot of this got started, has been tremendous, even to this day. You know, even conversations I'm having with founders of companies even this week.

• 2:52Alex Kantrowitz?

  Oh, wow. Yeah, that's, that's wild. Okay. Yeah. And I'm very curious about how that's gone. So let's go back to the beginning quickly. You joined Amazon in 2014 as a vendor manager. So a vendor manager is in Amazon's retail organization. What does a vendor manager do inside Amazon? And in particular, what was the job description in 2014 before all the automation came in?

• 3:19Speaker 01

  Oh, boy. Uh, I don't.

• 3:21Alex Kantrowitz?

  Let's go down memory lane.

• 3:22Speaker 01

  Yeah, I know. I don't remember the exact J D, but what I can recall is that the role was a transition between what a traditional buying role entailed, as well as what Amazon's demands entailed. And so, you know, in theory, the vendor manager is supposed to be the buyer. And there was a little bit of buying involved, especially within the fashion realm, which is where I was working. But the conflict of the role itself came into being in the fact that

• 3:56Speaker 01

  at the heart of it, we weren't actually doing much buying. Now, if you're familiar with retail.

• 4:00Alex Kantrowitz?

  And by buying, by buying, sorry, you're about to explain buying means you are on the phone with brands like, in your case, Gucci and Versace. And I'm going to spitball here, potentially saying, you know, we need this many handbags and these fulfillment centers at this price. Is that, and the fulfillment center is obviously the warehouse that all this stuff gets stored and then shipped out to people once we hit buy. So is that broadly to the description of the job?

• 4:27Speaker 01

  Yes, and I would say it even goes further, which is, you know, if you're talking to a Macy's buyer, a Nordstrom buyer, you're going down SKU by SKU, colorway by colorway, line by line, as to exactly what should be in your stores. And that's really where this position harkens from. And I think where, you know, doing so at Amazon is really different, or was really different back then, is that we still had that spirit. A lot of, a lot of my peers came from that exact background, and that's what they were used to. But in reality, you know, a lot of Amazon's buying spirit is very much about the everything store, which is buy one and see if it sticks. Buy one of everything, see if it sticks.

• 5:11Speaker 01

  And when you apply that to a role that had originated from this idea of curation of taste, of being able to predict what future trends can and should be, you're at a, you're at an impasse. And so I, I think that the vendor manager role, it was a great learning experience. I really, really loved it. But it was really different than I think what perhaps others from the outside looking in might think it is.

• 5:35Alex Kantrowitz?

  Right, right, because if you're at Macy's, you're, you're, and you're a buyer, right? Or you're managing vendors, right? The type of brands that come in, the Versace's and the Gucci's. You're like, okay, I think this bag is going to be in style. I have limited shelf space. I'm going to buy these, and that's your job. At Amazon, you're dealing with these fulfillment centers that can fit an entire NFL's worth of football games on the land that's in there.

• 6:02Speaker 01

  The limitless shelves.

• 6:03Alex Kantrowitz?

  The Sundays, yeah. So you're just

• 6:05Speaker 01

  Exactly.

• 6:05Alex Kantrowitz?

  out there buying like crazy and, and, or and then, and then seeing what the internet demands when they are on that site. And that's the first party retail, right?

• 6:15Speaker 01

  Exactly.

• 6:16Alex Kantrowitz?

  Amazon just stocks products in these warehouses, and then people go ahead and hit buy on their computer, and then they get shipped out.

• 6:24Speaker 01

  Yes, I think the the going idea, the going thought process back then was that if we, you know, we're Amazon and at the time, we don't really care about being profitable. We just want to be the best e-commerce site in the world. So the customer will know what they want. So how about we just have everything and they'll tell us what should sell. That was the going thought process behind it. Now, where things changed and where we are today, I would say it really started in 2016. Couple of years later, where Amazon realized that they had to start valuing profitability a lot more than they had before. And, you know, we can't hold it against them.

• 7:07Speaker 01

  Every company must, you know, no, no, let me rephrase that. No company can distribute our profitability forever. Right. And I think Amazon had the privilege of doing that for as long as it did. But a lot of things started changing and coming down the pipe in in 2016, where you start realizing, okay, you know, there's a lot of inventory sitting at these warehouses because we decided to buy one of everything. A lot of it isn't going anywhere. What do we start doing about that?

• 7:33Alex Kantrowitz?

  Yeah. And so this sort of stems with Jeff Bezos telling investors, if you're going to buy Amazon, you're in it for the long haul. We're not going to, you know, generate profits right away. We're investing. And that's why Amazon's multiple, right? When you look at profit and and at stock prices and and still in the stratosphere. It's wild. So let's go back to the practical side of these things. So we're going to get to the automation. But, you know, while you're doing the job, how are you placing the orders? Are you like in a software tool where you're like, you know, basically, are you in Excel? Are you in an Amazon software tool where you're like locking the orders and making forecasts? And are you on the phone with these brands and saying, okay, this is how we want to do it?

• 8:16Alex Kantrowitz?

  It's like, what does this job look like to you from a practical standpoint when you're doing buying?

• 8:21Speaker 01

  Yeah, so a little bit of everything you just said. You know, every company is different. You know, I learned that back then, working with all these brands and still very much is true today. Even with the proliferation of so many more D to C brands all over the internet. But every single brand's different and so, you know, you spend some time working with your portfolio of brands, understanding what they're trying to, what their goals are and then ultimately how Amazon can fit into that. And it's a bit of push and pull. Now, for a lot of brands, it'll often be a push from my side, meaning, hey, I really want you to grow, you know, this much in order to do that. These are all the things that we have to do or want to do. And again-

• 9:04Alex Kantrowitz?

  So what would those things be?

• 9:06Speaker 01

  I mean-

• 9:07Alex Kantrowitz?

  Price negotiation or?

• 9:08Speaker 01

  Sometimes price negotiations, sometimes preparing for peak seasons in advance and trying to be strategic about it. Sometimes it's a matter of even just, you know, inventory logistics excellence, right? Understanding that, hey, we're opening up a new fulfillment center in this region of the country. We want to make sure that your products are there. Let's make certain moves supply chain wise. So it involved every aspect of what it means to thrive within the e-commerce space today. But I think where it got difficult is that, again, as Amazon realized, we got, we had to keep more of the money that we're actually making here. You know, a lot of things started to change, meaning even that role, like the vendor manager's role

• 9:52Speaker 01

  or what that team's directives were. They started, you know, one of their new goals where we have to become more profitable. Our category of products has to become more profitable. Our transactions, every single one is now being measured, you know, with a new profitability standpoint. And so that's also where, you know, when I'm sitting in a, you know, in a, in a meeting with a brand, it's a lot easier to talk when all of us are on the same, you know, page trying to grow top line revenue. Everybody generally wants that. It's a very different conversation when I'm sitting there and saying, I need you to- Give me a better price. I need you to give me X dollars in funding, or I need you to,

• 10:33Speaker 01

  you know, make a big change in your processes in order to accommodate what I'm trying to do here. So it it it starts changing a lot of discussion, and I think that's where, you know, that's where we are today, which is that there is very little trust in this environment, especially when, you know, I mean, you're talking about even multinational publicly traded brands that, you know, everyone knows about. Even they struggle when it comes to negotiating with Amazon because frankly, they they don't like to compromise.

• 11:08Alex Kantrowitz?

  Yeah, and and as you're doing this, they're collecting the data, right? They're they're collecting data on what what the company bought, what it was priced at, what it sold at, and what demand was for what products and all that stuff. And then all of a sudden, I think, you know, probably around the time that you started, there are some folks on the machine learning side inside Amazon that say, hey, what if we, you know, actually turn this over to automation to make sure that we can do this, you know, as efficiently as possible with the best margins? They start calling it Project Yoda, from what I heard, like, some people are like, instead of us doing this with as humans, why don't we use the force? And the force was machine learning.

• 11:53Alex Kantrowitz?

  That's what happens when you put a bunch of computer nerds in charge of this stuff. It always comes back to Star Wars.

• 11:59Alex Kantrowitz?

  And eventually it starts, so they they start, the machines, I imagine, start to get good enough where the forecasting and the prices that they want, the negotiation range, becomes good enough where they think they can start to turn some of it over to that machine learning stuff. So now now let's go back like to your, from your perspective on the ground.

• 12:21Speaker 01

  Can I can I can I make a quick comment about what you just said, though?

• 12:24Alex Kantrowitz?

  Yeah.

• 12:25Speaker 01

  So that's the funny thing. Even when I was an employee there, I had very similar thoughts to what you just said.

• 12:32Alex Kantrowitz?

  Really?

• 12:32Speaker 01

  Mm hmm, mm hmm. But the reality is, and this isn't just applicable to Amazon, but many other companies, many other industries as well, is that, you know, a lot of automation and even to to, I would say, to a degree, machine learning in many of these pieces of technology is still incredibly.

• 12:53Speaker 01

  It's not ready to operate on its own or in a mass scale. And that that definitely goes for this situation that we're talking about when it comes to Amazon.

• 13:05Alex Kantrowitz?

  Yeah, so let's hear the story about how this stuff starts to become operational and sort of the pros and cons of it. Um, so what I I I'd love to hear like from your perspective at the beginning. Uh, when did you start to hear that some automation might come into that process? That you had in terms of buying and ordering from these brands?

• 13:23Speaker 01

  Hmm, for those interested, I remember first hearing the phrase hands off the wheel. You know, I think it was late 2015, early 2016. And it was interesting because it was it was discussed very laissez faire, very nonchalant, as if, you know.

• 13:40Alex Kantrowitz?

  Don't worry about these machines.

• 13:41Speaker 01

  Exactly. It's like, you know, like this is just just something that's happening. It's gonna change, but it's no big deal. But I feel like if if anyone is taking a critical look at the situation, you can see the writing on the wall, which is stuff's changing, the priorities that we had are changing. What does this mean for the work that we used to do in the in a particular way? So, you know, uh, the very first thing that they automated were promotions. Uh, the creation, management and execution of things like discounts, you know, pricing changes, uh, coupons, that sort of thing. And um, it felt pretty innocuous, you know, I think most people were like, okay, you know, like this is just a new tech change, no big deal.

• 14:24Speaker 01

  But at least for me, I kind of looked at it and I'm like, all right, this may not seem like a big deal to some, but for me, I kind of saw that as, okay, they're they're trying to make some of what people are doing every day. Uh, no longer part of their job. So what's what's next? Because this is not just the, this is just a tip of the iceberg, right? And so I remember seeing that. I remember having conversations with brands, introducing them to the tool. And one caveat I think we should make is that when we talk about automation at Amazon, we're not talking about automating something for everyone, for everybody. We're talking about automation so that Amazon no longer has the responsibility of having to do a certain thing.

• 15:06Alex Kantrowitz?

  Yes, the brands on the other end might still have to do the inputs, but Amazon would basically take the human activity out of it.

• 15:12Speaker 01

  Exactly. They no longer have to staff. The person, the people, the team, the overhead in order to accommodate what was happening manually before. Um, and in fact, what ended up happening and is still the case to this day is that they, they pushed a lot of that work onto brands, manufacturers and sellers uh with very little, actually no forewarning, um very little training, very little communication. They just kind of woke up one day if they logged into their Amazon portal and they realized, oh, what's this new thing? You're telling me that all and and and it was especially a bit of a rude awakening for,

• 15:52Speaker 01

  you know, brands that let's say you had already committed a significant sum of money to Amazon. And you let's say you've already paid a certain amount of money to Amazon for the year saying, hey, I want to work with you closely and fund our partnership so that we can have a really big, beautiful campaign during Mother's Day or a huge promotion during holiday this year. Now, all of a sudden, you have to do that work yourself. You're not being given much help at all. And in many cases, you may have to pay for that promotion yourself. Uh, and so, you know, it was a bit of a rude awakening, I think, to say the least.

• 16:32Alex Kantrowitz?

  Right. And so just like taking it back to a practical level. So these brands would be working with you as a vendor manager. And that was your job, right, was basically like we'll promote you on the site this day and we want X amount of inventory. And then, you know, somebody who's on the other, I'll just use a a brand. Let's not, I know it's not necessarily the one, but I'm just gonna use it as an example. Somebody who's on the other line at Gucci, you know, who's been speaking with you now all of a sudden, you know, the whole idea of when they're going to get promoted on the site, how much uh inventory Amazon needs and what fulfillment centers and at what price. And, you know, even the negotiation in terms of price in some areas. Ends up turning from human to a portal that they would log into because the machine

• 17:17Alex Kantrowitz?

  learning is doing this on the backend. Is that right?

• 17:21Speaker 01

  Yeah, I would say so, and certain things came more gradually than others, but the long story, the long story short is essentially that that's exactly right. Even to this day, you know, we have a great partnership with Amazon, you know, my company that is, and we talk to them all the time, but you know, they will, they're very, they're very open and honest about their limitations as people now in a way that I think we felt a bit uncomfortable to do so back then. And so now they're just very transparent, they're like, I have no control over that. I have no influence over this. I have no way of, of helping you with this. And it, sometimes it baffles, you know, companies on the other side because they're like, wait a second, you're, you're actually in charge of this thing, of whatever this thing is that

• 18:06Speaker 01

  we're discussing. What do you mean you can't change it? What do you mean you can't make an exception for a mistake that's, that's actually wrong? And that's, that's, that's the conversation I'm having a lot, very often, unfortunately.

• 18:20Alex Kantrowitz?

  Yeah, because overall, it seems to me that they said, okay, our algorithms are going to do the best possible job for the least possible overhead, figuring out how to maximize profits, as you mentioned, and get the right stuff into the fulfillment centers at the right time at the right price.

• 18:38Speaker 01

  Yeah, yeah.

• 18:39Alex Kantrowitz?

  Oh, well, sorry, go ahead.

• 18:41Speaker 01

  Oh, no, you, you hit on the, I, I love that you brought up the ordering side because that is, that's probably the biggest bane of everyone's existence when it comes to some of the automations that's happened, that's happened.

• 18:52Alex Kantrowitz?

  Right. So you would send like a purchase order back in the day for a certain amount of inventory? And now what does it just show up in people's inboxes?

• 18:59Speaker 01

  So they always showed up in people's inboxes.

• 19:02Alex Kantrowitz?

  Right, but there was a person who would send it.

• 19:04Speaker 01

  But there was, there was usually many people who touched it before, you know, usually it would involve me, you know, as someone from the vendor manager team, it would involve the in-stock management team whose entire job was to create, manage, watch, and, and, you know, organize all the POs that are happening at any second. And so there used to actually be a set of people that you could talk to if you said, hey, you know, This new thing is happening, a new trend that we're watching in the market. We want to make sure that we buy enough of X, Y, and Z product for this upcoming season. You could actually do that, but not anymore. Not once hands off the wheel really got implemented and not, not once it hit inventory

• 19:49Speaker 01

  and ordering algorithms. So everything suddenly became, no, we, we, we can't touch it anymore. Our algorithm is either going to order it or it's not. We can only hope to try to influence it and hope that what we kind of like the poking and prodding will influence the animal to behave in a certain way. But we actually have no way of guaranteeing whether certain changes will happen.

• 20:14Alex Kantrowitz?

  Okay, after the break, I want to speak with you about really the human side of this, how you and your colleagues reacted and you know, what Amazon did to all the people whose work it automated. So folks, hang on for just a moment, we'll go to break and we'll be back right after this. This episode is brought to you by Avpoint. Everyone's racing to roll out AI right now. Copilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what Avpoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate.

• 20:59Alex Kantrowitz?

  Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. Avpoint, the unifying trust layer for AI. Head to avpoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast.

• 21:31Alex Kantrowitz?

  Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a white glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form and request your free risk assessment. Their team will show you just how exposed your executives are

• 22:16Alex Kantrowitz?

  and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real world attacks. The link's in the show notes.

• 22:32Alex Kantrowitz?

  And we're back here for the second half of the Big Technology Podcast with Alain Kwan, former vendor manager at Amazon, who watched the automation program Hands Off the Wheel, which is, I talked about Project Yoda. It eventually turns into Hands Off the Wheel, where the vendor managers literally took their hands off the wheel and let the machine learning do the work. And she is now co-founder and partner at Quantified, which is a company up in Seattle, which I'm sure we're going to hear a little bit more about in the second half here. So let's go back to your reaction when you start to see some of this stuff go down. But you know, I know that Amazon employees are all pretty close to each other. What happened like at happy hour or dinner or, you know, just kind of hanging out in the office

• 23:15Alex Kantrowitz?

  when you and your colleagues start to see some of the stuff that you were doing end up being turned over to machines? Were you all like, was there like a moment of realization where it was like, oh, the machine learning isn't charged anymore? And it's kind of, oh, it's our hands off of the wheel?

• 23:33Speaker 01

  There was mixed reactions. I think some folks were righteously frustrated, annoyed, angry, that carefully laid out, thoughtfully laid out plans were suddenly out of their hands. Some of them, I think, including myself, started realizing, okay, this is the end of the line, essentially. You know, it's not that this role is going to disappear forever, and it hasn't. There are still vendor management teams in existence. But this role is going to change a lot. You know, instead of having full autonomy over how

• 24:12Speaker 01

  these brands will be projected on our site in terms of, you know, what we're going to sell, how we're going to sell it, what types of everything down to, you know, just how they're going to be merchandised on a detail page. All of that is going away. So what does that mean for me? And I know that for myself, one of the reasons why I took the job was because of that autonomy, was because of the freedom and the ability to really build these partnerships and say, how are we going to help each other achieve amazing growth together? When you start taking away, bit by bit, all of the tools that, you know, the team has in order to achieve any of those goals,

• 24:52Speaker 01

  now it starts feeling more and more, what's the best way to describe it? You're still being asked to hit astronomical goals, but you have very, very little ability to change anything to achieve them, if that makes sense. And that just felt like, you know, for me, you know, it's not something that I think spending my time on would be worth. So that's when I started realizing, all right, we need to find a new way to help brands grow, frankly. And that's actually where I started thinking about the future of e-commerce. What does this mean? Because this is not just Amazon. You know, every single e-commerce marketplace is trying to

• 25:31Speaker 01

  do both things. They're trying to grow and trying to scale how they can show the diversity of all their products online. That's not a small task. So this is not just me. Amazon problem we're talking about, Amazon may have carried out in a certain way. But, uh, you know, Zalando, Tmall, Alibaba, MercadoLibre, you're, you're talking about the fact that all these companies are fighting to figure out, how do we show this product off in the best way possible to, to convince you, customer, to buy it from us while still scaling how we do all this work on the backend to cost us as little as possible.

• 26:10Alex Kantrowitz?

  Right. And Zalando, the German e-commerce company that you mentioned, actually hired Ralph Hebrick, who headed up this whole program inside Amazon. So now he's off and I'm sure his minions are off doing the same thing in other companies.

• 26:24Speaker 01

  Oh, yeah. We're seeing it happen. It's very much happening. Yeah.

• 26:27Alex Kantrowitz?

  Yeah. And, and Elaine, one of the remarkable things that I find when I spoke with you about this and some of your colleagues or former colleagues, uh, is that it, everybody who's experienced this says, oh, it actually made a lot of sense and there's not a lot of ill will toward Amazon, even though like, you know, people are, when, when people speak about automation and how, you know, work has been automated, you know, they, they are fearful of what happens to the worker and you were the worker, but yet you maintain like a pretty sunny attitude about that. Can you explain a little bit more about your, your feelings?

• 27:00Speaker 01

  You know, I'm a, I, I, I think I'm a, I'm a fairly practical person. So, you know, even back then, when I saw what was happening, I didn't get mad about it. I wasn't angry about it myself. It was just more like, all right, this makes sense. I understand. I understand why this is happening. Now, is this going to hurt some people, some businesses? Absolutely. But for myself, I kind of looked at it as an opportunity because, you know, these are what e-commerce marketplaces and companies, retailers are doing in order to survive themselves. That doesn't mean that the brands and manufacturers and sellers are. All alone by themselves without any aid. That's where my thoughts went, which is okay.

• 27:43Speaker 01

  This is going to be a growing chasm in this industry. How do we, how do we bridge that gap? What do we need to do in order to solve this? Because this problem is maybe small today, but it is going to grow. And sure enough, you know, five, six years later, here we are. And like I said, even this week, I've had three conversations where, you know, with different CEOs about issues related to stemming, or I would say stemming from this exact thing.

• 28:11Alex Kantrowitz?

  Wow. Okay.

• 28:12Speaker 01

  Yeah.

• 28:13Alex Kantrowitz?

  And, and okay, now that as we're talking about it, I just grow, grow kind of curious. So if I'm a vendor manager inside of Amazon, how does this stuff show up in my, does it just, is it like an automate button in my tool that I use to buy? Uh, what did it look like?

• 28:27Speaker 01

  Like, how does the buying happen?

• 28:29Alex Kantrowitz?

  Well, like, yeah, when, when you're turning stuff over to the machines, um, yeah.

• 28:33Speaker 01

  I don't, um, you know.

• 28:35Alex Kantrowitz?

  Was it an automate button or was it just like, okay, that task that you were doing, you're not doing anymore?

• 28:41Speaker 01

  Yeah, there were, there were steps that we no longer did anymore, you know, before we would actually go, like I said, we would, we would go through line by line and actually talk about products, what they were, what they're going to offer to the customer and decide, do we want to buy it? Yes or no? Now, uh, and again, not super familiar with how the tools may have changed, but

• 29:01Alex Kantrowitz?

  Right.

• 29:02Speaker 01

  Uh, my understanding, my understanding and what we saw happen back then was that, you know, the vendor, the brand now creates their new product listings, whatever they want to sell on Amazon, they create them on their own. They do all that work by themselves. They submit it. If, uh, once it's processed and created by Amazon, which is a process that, you know, the vendor manager doesn't touch anymore, um, then the algorithm will decide based on the product information submitted, whether or not they want to buy.

• 29:32Alex Kantrowitz?

  Unbelievable. Yeah. And one of the things I heard, I wonder if you experienced this, was that the company like set hands off the wheel goals were like a certain percent. Oh, you're smiling. So, so, well, go ahead.

• 29:45Speaker 01

  I mean, you're bringing back, you're bringing back memories.

• 29:47Alex Kantrowitz?

  I'm sorry. I'm sorry. Some of this is fun. I'm sure some of it is less.

• 29:52Speaker 01

  You know, some of these things I haven't thought about in quite a while, but you're absolutely right. They set goals, you know, which again makes sense.

• 29:59Alex Kantrowitz?

  Like the percentage of stuff that should have been automated?

• 30:02Speaker 01

  Yes. Which again, makes sense. If you're, if you're leading a team and you're trying to hit big changes and provide benchmarks for where the team should be, but in this case, the benchmarks are how much less of your job we want you to do.

• 30:15Alex Kantrowitz?

  Yeah. Yeah.

• 30:17Speaker 01

  How much more we, we want you to teach or tell these vendors and brands, do your job instead?

• 30:23Alex Kantrowitz?

  Yeah. I remember speaking, uh, one of your former colleagues who told me basically as soon as they had this meeting, there was one year where apparently leadership handed down these major hands off the wheel goals. And then this person sitting in this meeting was basically like, all right, it's time for us to find new jobs, you know, as a joke, but it turned out to be kind of serious.

• 30:43Speaker 01

  Yeah. Yeah.

• 30:44Alex Kantrowitz?

  And the interesting thing that I found was a lot of, you know, and you know, you can fact check me on this, but like from my understanding, Amazon didn't like go out and, and, you know, do mass firings of vendor managers. A lot of people ended up moving into different roles inside the company, product manager and program manager. So what happened there?

• 31:04Speaker 01

  Correct. Correct. I would say, you know, everyone's path, it's very much sort of choose your own adventure. You know, they weren't, they weren't out to, at least from my experience, they were not to fire anybody. It was, I mean, the people that worked in these roles, I'd say for the most part, very talented, very smart, very driven, great. And specifically at Amazon, there's this idea that if you're hired into Amazon, you know, you're capable enough to serve in other roles, other teams, other functions, even if you may not have a ton of experience in that particular, with that particular skill set yet. And so there was a cool, there was a cool opportunity for many that was, you know what?

• 31:42Speaker 01

  I'm gonna go, you know, interview some other teams internally and see if there's a a different role that might, you know, appeal to me. So that's what I would say many did. A a small handful stayed behind within the team itself. Uh, you know, folks like me, I um, I ended up getting poached by a startup at the time and and kind of took that route. Uh, all the while still thinking and obsessed with this problem, which is why we're here today and uh, what I'm, you know, what I've been working in ever since. But um, it, I would say those, i- it wasn't, it wasn't a wide public announcement, but I think everyone again. Kind of realized, okay, this may be the end of an era. And so.

• 32:26Speaker 01

  Take it, you know, keep your eyes open and start thinking about what your next path really looks, looks like.

• 32:32Alex Kantrowitz?

  Yeah, a- and this is sort of something that's pretty amazing to me about what happened. And Amazon, and I think other companies should really take note when they start to do this because. There is, there has been this feeling of like, okay, well, automation's gonna come in and then people are gone. But in reality, it's not really the case. And what Amazon did was ended up putting them, I mean, project manager and program manager jobs. Are what I think of as essentially professional inventors inside the company, people who are there to to shepherd along new products. Uh, and uh, a- and build the next new thing for Amazon. And it's interesting, okay, they were working on this one thing, machine learning comes in. Now all of a sudden, you know, help the company grow in another way. And that's, I think, one of the, you know, magical parts of the Amazon work culture that gets overlooked.

• 33:16Alex Kantrowitz?

  And there's a lot of criticism of Amazon, a lot of it, you know, definitely well deserved, but there's definitely some business brilliance there. And this is a big part of it. And. I think like one of my favorite stories that I came across when I was writing Always Day One was. Uh, Dilip Kumar, who was running pricing and promotions. Uh, inside Amazon, right? Probably something close to what what you were doing. He goes to work to spend a, after, you know, in maybe twenty fifteen, he goes to spend a year, year and a half. Shadowing Jeff Bezos as his technical advisor. He comes out, the thing's on its way to being automated. He's not going back there. So he ends up getting together with a bunch of people from the retail organization and building Amazon Go, which is the checkout free retail store.

• 33:56Alex Kantrowitz?

  Uh, that's gonna be a major part of Amazon's, you know, brick and mortar in real life shopping.

• 34:02Speaker 01

  Yeah, the first one is actually one, one block from me right now.

• 34:05Alex Kantrowitz?

  So you're right in South Lake Union, huh?

• 34:06Speaker 01

  Yeah.

• 34:07Alex Kantrowitz?

  In Seattle.

• 34:08Speaker 01

  Yeah.

• 34:08Alex Kantrowitz?

  Yeah. And it, it is like, people ask like, oh, how does Amazon reinvent itself so often? And this is, I mean, this is the key example in my mind when it comes to the way that the company works.

• 34:19Speaker 01

  I agree. I think that there's a lot of, a lot of really interesting and very applicable principles that, you know, even I've taken into my own business practices. But there's also a lot of warnings too. I mean, I, I would be the first person to tell you that, guess what? There's still a vendor management team in existence. The problem, and this is again, my two cents, take it or leave it. But the problem, in my opinion, is that there are so few of them that are now, you know, still expected to manage huge goals, huge portfolios of brands. But because they've been given these hands off the wheel automated tools, I think the expectation was, well, you should have no problem, you know,

• 35:01Speaker 01

  managing a hundred times what people used to before, because you won't be doing all of these things that people used to have to. But the problem is, is that automation or any tool that is automating a piece of work is only as good as the complexity of the rules designed within it. And the, the fact of the matter is, is Amazon has so many products. The diversity, the range of products really cannot even be fathomed. And so every rule we've seen that goes into part of, you know, whatever tool's being automated, it makes mistakes. It makes a lot of mistakes. I mean, even right now, we're finding that thousands and thousands of products are being mistakenly flagged as pesticides.

• 35:47Speaker 01

  Because someone implemented a rule to look for anything that could be a pesticide and suppress it, keep it off the site. And now you have pearl accessories, headbands, hand creams that are being, or even t-shirts with silver dur to help sweat proof, you know, sort of experiences. They're all being flagged as pesticides. And so, you know, I, I, I give this example to illustrate this idea that, you know. There is a lot that can be automated, but when you're dealing with the level of scale that Amazon's currently operating at, even the smallest rule can have devastating effects, if not managed incredibly well.

• 36:27Alex Kantrowitz?

  Yeah, that's what from my conversations with the company, it's the people, the vendor managers really became auditors. So they went from making the orders to like auditing the system, making sure that it's ordering appropriately. And I remember speaking with Ralph Hubrick, who I mentioned before, had a machine learning who's since left. And he talked about how like, you know, they couldn't get the order of white socks right. And they were trying to figure out what was going on. And there was like, okay, the vendor manager went in and audited the system and found that they had 56 different ways of writing white socks. And so the whole, you know, machinery just ended up blowing up because of that. So it does totally change the job. And the scale is also really interesting. I think he mentioned that people were going from, uh, vendor managers were going from,

• 37:10Alex Kantrowitz?

  you know, managing like a thousand products to 100,000. And then was it then followed up and was like, well, that's not the exact number. I was just using that to illustrate the scale, but really is 100 times what they were doing for. And there's no way to keep that, the precision that you would have had otherwise.

• 37:26Speaker 01

  I would agree.

• 37:28Alex Kantrowitz?

  So what does this portend for the rest of the economy? I mean, the one interesting thing I find about the tech giants, when it comes to machine learning, is that they've had in-house research organizations for much longer than anybody else. And a lot of them have the, uh, you know, the granddaddies of the field, you know, Facebook has Jan Le Kuhn, who basically is the guy who helped deep learning, you know, take off in a way people used to laugh at him. They said that technique will never work. Then it worked and Facebook hired him right away. Uh, Google has, uh, I think his name is Jeff Hinton, who's another one of the, you know, uh, originators. And then Ralph, of course, you know, came to Amazon. So these companies all had a head start. Uh, and I wonder, is this something

• 38:13Alex Kantrowitz?

  that ends up coming to the rest of the economy? Can people, um, who aren't working for the tech giants anticipate this, you know, in the workplace at some point?

• 38:21Speaker 01

  Yeah, that's a great question. And I think it's one that I have very frequently because as you can imagine, you know, our company is all about a set, we are here to help all the brands, manufacturers and sellers of the world thrive within these spaces.

• 38:38Speaker 01

  It can be done, but it is incredibly, it's incredibly difficult for a single brand or manufacturer to do alone. Exactly because of what you just described, which is, you know, very few companies have the software, the data, the expertise, as well as the know-how to navigate what is actually a maze for each of their accounts. Imagine a brand that sells on Amazon, Walmart, Zalando, Tmall, and Rakuten, which is just a, it's just a fraction of the incredible e-commerce marketplace that they could be selling. Let's just say just five. Each of those five is its own maze in that each of these companies were built independently from one another.

• 39:22Speaker 01

  All of their algorithms on the backend are very unique and, and, and specific to that marketplace and that particular pieces of software. And so, um, from the consumer standpoint, it feels like it's pretty similar. You can go from Zalando to Amazon and you're like, all right, I search for something, I add to cart, I'm done. But that's where the similarities pretty much stop. And so for, you know, we talk to and work with, you know, again, multinational publicly traded companies. I would say it's kind of interesting because they tend to be the first to realize, we don't have what it takes to do this. So they're usually the first to reach out and work with companies like ours. Um, I would say medium-sized businesses, uh, depending on the leadership,

• 40:07Speaker 01

  will reach out immediately or they'll usually try it on their own. And when things start getting difficult, stagnating or failing for whatever reason, that's when they realize, we need help. And then last but not least, we're seeing a proliferation of awesome small businesses, um, startups, VC-backed businesses that realize that they know enough to realize they don't know enough to do this internally. And so that's usually when they reach out. But it does take, um, frankly, the best defense to kill our offense, right? And I feel like, you know, it's a combination of uh, data analysis and expertise that comes from having an actual team of people that have done this, not just once, not just twice,

• 40:46Speaker 01

  but hundreds of times over many, many years, uh, combined with software. Um, and that's what it takes today.

• 40:54Alex Kantrowitz?

  So you've built a company to help other retailers navigate this stuff.

• 40:57Speaker 01

  Exactly.

• 40:58Alex Kantrowitz?

  Which is important. I mean, I imagine it's a very big opportunity. And now, when it comes to just the idea that automation can make its way into the workforce or into workplaces beyond these tech giants, do you anticipate that that's going to happen? Do you, are you anticipating using it in your own workflows? Where do you think automation as a whole goes, you know, from where it is today? Because right now, it seems to be in some small number of companies, but a lot of people are predicting it's going to go much broader.

• 41:29Speaker 01

  I think the question that we actually ask ourselves is, what isn't going to get automated? Or what's going to get automated last? That's actually the conversations we have internally. Yeah, because, you know, when you start, especially when you're in the position we're in, which is we work with incredible companies, but many of them, and not to be able to see a lot of the trends that are happening. And when you start seeing that, you realize, okay, there is, everyone's different, but also a lot of what's happening is not unique. And I think that's really where the question is. It's like, what's, what's the last thing that's going to be automated? What's the, and in my opinion, it'll likely be the most difficult thing, which is critical thinking, adapting to new pieces of information and data.

• 42:13Speaker 01

  I would say COVID has very much thrown all of the forecasting that was run by machine learning into the drain. And that's been one of the really fun, difficult, but very interesting challenges that we've been helping brands with, which is, how do you figure out what's going to happen when no one knows what's going to happen? And, and that's been, I would say, it just kind of goes to show you again that,

• 42:38Speaker 01

  I feel like it's a long while yet before we see the level of automation that I think is depicted in movies and in television at times.

• 42:48Alex Kantrowitz?

  But it's coming.

• 42:49Speaker 01

  It's coming and it's happening step by step.

• 42:52Alex Kantrowitz?

  How should people think about, I mean, you've obviously adjusted pretty well to having gone through this experience. How do you think people should look out for, you know, the potential that stuff that they do might be automated? And what type of skills do you think they should be investing in to make sure that they're able to thrive, you know, in a situation like this?

• 43:11Speaker 01

  Yeah, I was actually speaking to a graduating class at a university recently that asked me exactly that, which is.

• 43:18Alex Kantrowitz?

  Let's do the commencement address.

• 43:19Speaker 01

  Exactly, right, but like, you know, young people are wondering exactly that. They see all the changes that are happening and they're wondering, shoot, you know, do I have what it takes to truly make it today? What do I need to know that perhaps I wasn't taught in school? And frankly, there are two things. One is critical thinking, which I briefly mentioned before, but that is actually the number one trait that I know. For example, we hire and we work in the business of automating a lot of things for others. But that is one factor that frankly, most machines still have not yet even come close to touching.

• 44:03Speaker 01

  Being able to say, actually look at a situation with all of the nuances and complexities at hand and be able to communicate real time what's happening and what should happen. That's another level that again, you know, I think is going to be in demand for quite a while yet. The second of which is also may sound counterintuitive, but it's actually the ability to build trust. And the reason why I say that is because as long as companies are still being run by people, the number one detracting factor that I see coming between two companies or, you know, different partnerships is really whether or not the companies understand one another and are able to trust one another. And so I'd say if you have the ability to connect and truly build trust

• 44:45Speaker 01

  in a way that goes deeper than just superficial connection, you can find amazing opportunity to survive beyond the age of automation.

• 44:55Alex Kantrowitz?

  Yeah, I love how you said as long as companies are still being run by people. Who knows, maybe, you know, one day that'll be different. That'll be interesting. Maybe you and I will just give it over to the robots.

• 45:06Speaker 01

  I'll sit on the side and take notes for the machine.

• 45:09Alex Kantrowitz?

  Right. I think there's some book that talks about like, you know. Automation communism paradise, or whatever it is, where like just let the machines do the work and we get to hang back and and drink martinis, which okay, I'll be into that.

• 45:23Speaker 01

  Oh, man.

• 45:24Alex Kantrowitz?

  Uh, long term, are you are you um bullish on the the future of work and on, you know, people's ability to continue to find, you know, meaningful employment despite the machines? Or are you betting on the machines basically kicking our butts?

• 45:42Speaker 01

  That is a great question. Um, I certainly hope they don't kick our butts, but I do think that, you know, I think the future of work is going to require that everyone, every job, every function that we all have is going to incorporate a certain level of technology that we never have had before. Um, and I think that, you know, even the way that we ha- are currently have adapted to working today is just one step in that direction. But even simp- what's what's, let's say, what we would call traditionally non-technological jobs, like, you know, uh, being a UPS delivery driver, even that today incorporates a level of using technology, you know, scanners, taking digital photos and automated thumbprints.

• 46:27Speaker 01

  I mean, they are using technology in a way that they didn't before. And I think that it's, we're only going to see that increase to the point where, you know, again, the things that humans are left doing are the things that only humans can do.

• 46:39Alex Kantrowitz?

  Right. Is Amazon gonna do this drone delivery thing, or is that bull?

• 46:43Speaker 01

  I don't know. I will let you know if I find out, but I I I'm not sure.

• 46:48Alex Kantrowitz?

  They haven't made a lot of progress since Jeff Bezos made that flashy announcement. He's gonna leave the company before it happens, so.

• 46:54Speaker 01

  Oh, man.

• 46:55Alex Kantrowitz?

  All right, great. How can people get in touch with you or learn more about what you're doing at Quantified?

• 46:59Speaker 01

  Yeah, um, there, feel free to, you know, I would say, look me up on LinkedIn. My name's Elaine Kwon. I'm the managing partner at Quantified. And you can reach us on our website as well, quantified.com. That's with K W O N T I F I E D.

• 47:16Alex Kantrowitz?

  Awesome. Elaine, so great to catch up. It's been too long and I'm really glad we had opportunity to chat again.

• 47:22Speaker 01

  I agree. This is a lot of fun. Thanks, Alex.

• 47:24Alex Kantrowitz?

  Super fun. Thank you. All right, everybody. Elaine Kwon, joining us on the Big Technology Podcast. It was really great to have her on and it's great to have you listening. Major thanks to Nick Guatini, our editor, who does a great job with making sure this thing sounds good every week. Red Circle, thank you for hosting and selling ads. Thank you, Main Street, our sponsor. And thanks to you all for listening. If this is your first time, we do this every single Wednesday. We have tech insiders, outside agitators, and hopefully you'll learn something new each time you tune in. So please subscribe if this is your first time here. If you're a longtime listener, ratings go a long way to help us continue to grow, help those, train those machines to pick us up and distribute us places that we wouldn't

• 48:07Alex Kantrowitz?

  have been otherwise. So it takes about 10 seconds, if you're willing to give us a rating on the Apple Podcast app, that would be terrific. I won't even ask you for five stars. Any rating would help, although, you know, those five would be appreciated. The machines love them. That is gonna do it for us this week. We will be back next week. As I said, thanks again for listening to the end. Appreciate it very much and we will see you next Wednesday here on the Big Technology Podcast.

• 48:44Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.