Wharton AI Professor: How to Work With AI Agents Without Losing Your Edge | Ethan Mollick

Ethan Mollick is a Wharton professor who co-directs the Generative AI Labs and was named by TIME as one of the 100 most influential people in AI. His new book Co-Intelligence: The Next Phase of AI just came out. AI can draft most of your work in seconds — but if you only fix 10% of what it writes, what are you actually losing? In this episode, I ask Ethan how to work with AI without handing over your own thinking. He explains why he still types every first draft himself, why he brainstorms alone before opening Claude, and what happened when AI fact-checked his new book and found errors in the papers he was citing. We talk about the four skills you need to use AI well, why entry-level jobs no longer train people the way they used to, and how he actually manages AI agents step by step. At the end, I ask the question I keep thinking about: what happens when AI learns taste? Links: 📌 Subscribe to my free newsletter where I go deeper on AI tools, career strategies, and building with AI: https://siliconvalleygirl.beehiiv.com/subscribe?utm_source=spotify&utm_medium=description&utm_campaign=futureproof-sub&utm_term=ethan-mollick https://siliconvalleygirl.beehiiv.com/subscribe?utm_source=spotify&utm_medium=description&utm_campaign=futureproof-sub&utm_term=ethan-mollick ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠𝕏 : ⁠⁠⁠⁠⁠⁠⁠https://x.com/siliconvalleymm⁠⁠⁠⁠⁠⁠⁠ https://x.com/siliconvalleymm%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0 🔗 Instagram: ⁠⁠⁠⁠⁠⁠⁠⁠https://www.instagram.com/siliconvalleygirl/⁠⁠⁠⁠⁠⁠⁠ https://www.instagram.com/siliconvalleygirl/%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0 💼 LinkedIn: ⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/marinamogilko⁠⁠⁠⁠⁠⁠⁠ https://www.linkedin.com/in/marinamogilko%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0%E2%81%A0 📌 My Companies & Products: ⁠⁠⁠⁠⁠⁠⁠https://Marinamogilko.co

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  I hope I don't have to cancel again tonight.

• 0:02Speaker 01

  Chronic migraine means 15 or more headache days a month, each lasting four hours or more. Botox, onabotulinumtoxin A, prevents headaches in adults with chronic migraine. It is not approved for those with 14 or fewer headache days a month. Botox, the number one prescribed branded chronic migraine preventive treatment, prevents, on average, eight to nine headache days a month versus six to seven for placebo. Prescription Botox is injected by your doctor. Effects of Botox may spread hours to weeks after injection, causing serious symptoms. Alert your doctor right away as difficulty swallowing, speaking, breathing, eye problems, or muscle weakness can be signs of a life threatening condition. Patients with these conditions before injection are at highest risk. Side effects may include allergic reactions, neck and injection site pain, fatigue and headache. Allergic reactions can include rash, welts, asthma symptoms and dizziness. Don't receive Botox if there's a skin infection. Tell your doctor your medical history,

• 0:45Speaker 01

  muscle or nerve conditions, including ALS Lou Gehrig's disease, myasthenia gravis or Lambert Eaton syndrome, and medications, including botulinum toxins, as these may increase the risk of serious side effects. Ask your doctor. Visit botoxchronicmigraine.com or call 1-800-44-BOTOX to learn more.

• 1:00Speaker 02

  Your sausage McMuffin with egg didn't change. Your receipt did. The sausage McMuffin with egg extra value meal includes a hash brown and a small coffee for just $5. Only at McDonald's for a limited time.

• 1:12Speaker 03

  Price and participation may vary. Promotion pricing may be lower than meal pricing.

• 1:15Speaker 04

  We can easily give up skills to the AI and we may never get those skills back.

• 1:19Marina Mogilko?

  This is Ethan Mollick, one of Time's 100 most influential people in AI. He leads the generative AI labs at Wharton. When everybody has Claude advising on the strategy, how do you avoid sounding just like everyone else?

• 1:33Speaker 04

  You don't want the AI making those choices, even if they're good ones. You want to be inserted into that process. I'm deciding when to insert my own agency into the system rather than just sending off agents to do work.

• 1:42Marina Mogilko?

  The adoption piece has to happen from scratch. Every two weeks, that's what I'm feeling.

• 1:47Speaker 04

  So we have to purposely add friction to our lives, inserting key areas so we maintain an advantage.

• 1:51Marina Mogilko?

  So you will never delegate generating ideas to AI?

• 1:54Speaker 04

  I wouldn't say never delegate, but you.

• 1:58Marina Mogilko?

  Ethan, welcome. You teach entrepreneurship and innovation at Wharton. Time named you one of 100 most influential people in AI. About half a million people read your Substack, One Useful Thing, and your new book, Coexistence, just came out. Welcome to the show. When you teach your students, you also ask them to start companies while they're in class and you tell them to work on the most ambitious ideas. How do you teach ambition?

• 2:28Speaker 04

  That is always a hard question, right? Ambition looks like, we call this entrepreneurial efficacy in the entrepreneurship world, right? Same kind of thing, agency. And it's actually quite tough to sort of do. We don't have like a playbook for it. But I actually find that people create barriers that you can kind of help them overcome by getting them to, by just asking them to be more ambitious and being in a room where peers are more ambitious. So a lot of what I just asked, it's really easy now is like, what could you do more than that? And that was a hard question before AI because you stopped because you had the ability to only get so much done. And now, as the frontier moves out, pushing for ambition is a much bigger deal.

• 3:05Marina Mogilko?

  What are the coolest ideas that you've seen in your class?

• 3:08Speaker 04

  So it's funny because like the last version I taught was six months ago. And so already that was like the beginning of a Gentic, Engendic works. My next semester starts in January. So I actually, when, after ChatGPT came out, I had been following, you know, obviously models for a while. But after Da Vinci, which came out right a couple of days before ChatGPT, and ChatGPT came out and then a couple of days later, I had a class and I basically just dropped the entire class and said, I'm going to show you how to use ChatGPT. And by the end of the class, one of my students had actually stopped paying attention and had coded up his entire project for the semester during the class period. So, and showed me the results at the end. And so every year since then, the ambition level has gone up further and further. What was most interesting this time was all of these people who are subject matter experts.

• 3:51Speaker 04

  I've got surgeons and I have biomedical device people and I've got finance people and I have, you know, NGO and humanitarian people, soldiers all in this class. And suddenly they were all building real stuff, right? Like, you know, and so there were really interesting ideas. How do we track our entire supply chain? Because a lot of companies don't actually know their entire supply chain. And how we figure out where is the financial weakness in that chain that we might have to watch out for? Really interesting set of ideas. There was a bunch of really interesting stuff around, like specific problems in medicine, about how do you actually give information to patients in ways they understand as opposed to just forms? So, I mean, just really interesting, but very narrow, right? Focused on something people know. And part of what is interesting is like, software can be many things. And if you're trying to think about solving a problem that you have deep experience in,

• 4:36Speaker 04

  suddenly there's a lot of capabilities there that are really exciting.

• 4:39Marina Mogilko?

  Are there any problems that you're telling them to avoid solving?

• 4:42Speaker 04

  There, among those of us who teach entrepreneurship at some of the leading business schools, we, you know, we used to exchange notes about the topics we see people address over and over again that we know will never work out. But like, we're always conflicted. Should we tell them it's not going to work out or let them try and figure it out for themselves? So there's a few that are sort of standard problems that are actually kind of a good example of where technology hits its limit. One of the most common is we are going to replace brokers for houses. Right? It seems like a really big, like you're paying a couple of percent to these people who like are doing increasingly less work for you. This is obviously a case of disintermediation. And every time the idea fails, because it turns out there's a deep social integration of political integration and legal integration of brokers into the process that you just can't dislodge. So sometimes the reality of the world

• 5:27Speaker 04

  is quite different than what you people think. And then the other is students thinking too narrowly about their student problems. So everyone always wants, you know, better ways to like get homemade food to them or better ways to, you know, hold stuff between semesters that probably aren't as universal once you're out of, out of college.

• 5:43Marina Mogilko?

  This is so interesting. You're mentioning these ideas and then I can draw parallels to businesses that actually worked. Like DoorDash was solving the problem of delivering food on campus. And then when you talked about real estate, like Airbnb, right? We would have thought like, oh, who would have rented an apartment from strangers? And like, yes, there's a social element of trust. But then Airbnb took over and now we trust Airbnb. How do you know your idea is not going to survive? The real world?

• 6:08Speaker 04

  Well, I don't know.

• 6:09Marina Mogilko?

  You just try, right?

• 6:09Speaker 04

  That's why you don't tell people, right? Like, these are the highest failure rate ideas. But obviously because they want to do it, right? Because they keep coming up, they're not bad ideas. The whole idea is you can't filter them out initially. Like, there should be a better way to buy and sell a house, right? And, you know, people do want home cooked meals more easily than a meal delivery service, which is sort of what they're asking for now. It's like, how do I have my mother cook a meal for me as a son or father is essentially what they're asking. And so there should be a way to crack this. There's demand for it. But there are harder problems than people think. And in some ways, AI makes it more ambitious, right? Because it's no longer the technical piece. But the numbers look really good. And then they just hit a point where it's like, oh, there is a challenge here. So whoever cracks it makes a lot of money. But it is also an idea that everyone is, we've seen people throw themselves against over and over again. So maybe some people who are watching or listening will be the ones to break the real estate broker experience.

• 6:54Speaker 04

  But it's not as easy as it appears, I guess.

• 6:56Marina Mogilko?

  Exactly. It's like, when do you give up? Because there are so many shiny things around you and you can build things in a day, right? It's so hard to focus on a problem when you hit the wall. Do you agree with this phrase? It is as easy to work on a big problem as on small problems. Basically, the amount of effort that you put in to building a small business versus large business is the same because you're going to have the same headache, hiring, firing people. Do you agree with that?

• 7:18Speaker 04

  In general, like, I mean, I think if you're trying to build a business from scratch, right? Aim big because you may not get there anyway, right? And also ambition helps motivate people. It gets you the best people to work for you. It creates a real self fulfilling prophecy, right? But I also think that it is, there are phases to a business, right? So that initial phase might be difficult no matter what. Scaling presents its own challenges versus running a small business, right? So they're not quite equivalent, right? Like trying to get to the next level. If you're going to raise a Series B and a Series C, that's very different than you're running, you know, a, like, you know, a local business of some sort, right? So I think.

• 7:54Marina Mogilko?

  Different skill set.

• 7:55Speaker 04

  Yeah, yeah. I mean, if you start from a blank slate, right? And you want to take over the world, go fully ambitious. But also, by the way, I think that, you know, we keep talking about ideas. I think people aren't being ambitious enough about the kinds of ideas they pursue, partially because of, they're sort of in a Silicon Valley mindset about software, right? And I think a lot about, I studied the history of technology, among other things, and have done work on it. And if you look at who got like really rich during the industrial revolution, right? It wasn't people selling steam engines, even though this keeps being the analogy in Silicon Valley. It's like, I want to sell picks and shovels. You don't want to sell picks and shovels. You want to dig a gold mine with new gold mine technology. And I feel like we're kind of missing that point here. If you can make a business more efficient, just do it yourself right now, right? Overcome those barriers directly.

• 8:35Marina Mogilko?

  What is going to happen to software companies when you think about entrepreneurship? Is it really worth starting a software company these days?

• 8:42Speaker 04

  So I think, you know, it's good news for entrepreneurship overall. Like the early data looks really good and you can understand why it's a really good idea. I think everyone's a little too like SaaS brained at this point, right? Which is like, what do you do to make money? Software as a service. And I think that that is under threat, right? Like, it's a different sort of environment than it was before. I spend a lot of time with big companies now, as well as entrepreneurs, and there are people getting it. There, there's a lot of conflict over what the future of software looks like. On the other hand, I think software is increasingly becoming something that's table stakes for doing a lot of other interesting work. So I kind of would love to see more entrepreneurs branch out from software being the thing they sell to actually making or doing something because there's never been a better time to do that.

• 9:21Marina Mogilko?

  What do you think is the future of SaaS? Because we're seeing all of these companies that copy AI, right? That has started a few years ago, but now I think their revenue is going down significantly because of AI, because it's basically the new models are replacing what they were doing.

• 9:35Speaker 04

  So it's kind of interesting, right? You have a bunch of stuff happening at once. On one hand, everybody's a coder now. I mean, and that's not just a theoretical thing, right? So there's this great set of studies out of Anthropic suggesting that actually managers now write better code than coders, at least in first passes, right? Because it's about giving clear instructions more than being great at code. And things are transforming quite quickly, right? You've got 100 times more productive people. So companies are also completely in a mess as a result. They all have their targets on SaaS vendors. There's not a large company out there that doesn't want to think about how do we become more independent, how we build our own tools. And so it'll be a while before they get that all right. And a lot of them are still very happy. I mean, as far as I know, both OpenAI and Anthropic still use Slack from Salesforce. They still probably use Workday.

• 10:20Speaker 04

  So there'll be more choices being made about what kind of software vendor to go with. But I think the more interesting thing is that companies are realizing that a lot of these startups that were selling them stuff are just selling them the things they could build already. They don't have their own LLMs, so they're selling LLM capabilities that they already have. And that's creating a really interesting dynamic.

• 10:39Marina Mogilko?

  So what do you think is going to happen in a couple of years? Because I already see myself, like, for example, we're using the software because we're getting to the limit of the number of subscribers and the price can basically, like, can we just try and vibe code this and like connect Amazon services for like the newsletter stuff, right? I don't know. Like, what do you think is going to happen in a couple of years?

• 10:58Speaker 04

  I think that software will be democratized. So the whole thing, I teach at Wharton, I teach MBA students, I teach undergrads, and a lot of the MBA students are technical. So like, the key to success for the last 10 years has been find a technical person who can code for you. I think it's incredibly freeing to view this like other kinds of fields where you can get, approach it in different kinds of ways. There'll be software-heavy companies where you really need top engineers working with AI to succeed. And then there's going to be like software as a toolkit that we get things done. And I think democratizing, that's incredibly helpful. I mean, you don't want to be a software company. You want to make your newsletter, you want to make your videos, and there were things standing in your way. And software, by the way, is one of them. It was also, you needed to be able to do your own finance.

• 11:40Speaker 04

  I mean, when I did my first startup company, I did not realize, it was the late 90s, first internet bubble, but I did not realize that you could pay somebody a couple cents a pay period to handle taxes for your employees. So I would calculate taxes in Excel every week for the first few weeks, not realizing you could just do that. So even just advice, all of the kind of pieces you get, like, make this much easier now than it ever was.

• 12:01Marina Mogilko?

  If you're listening on Spotify and this conversation was useful, please, please, please rate our show. I see all your views, but I'm not getting the actual feedback. This can really help our team to decide what to fill next, and it can help someone who's never heard of us decide whether to give us a listen. Thank you so much. Using AI and the process of it, I really like the study that you mentioned, the Kenya entrepreneurs, where people had experience in business, they used AI and it helped improve their strategy versus people who did not have experience. Can you talk about that, when everyone is using AI, to strategize a lot?

• 12:35Speaker 04

  Yeah, so it was an interesting study. It was GPT-4, right? Long obsolete system. There's now 180 models better than GPT-4, 19 of which can run on your phone. So, you know, big difference. And it was only giving advice. So the fact that I think the number was 18% profit increase for the best entrepreneurs, that's a crazy number. Like, we do studies all the time, we try and move the needle on entrepreneurship. That is a crazy number. But it only helped entrepreneurs who are already being successful, because it helped them expand in the sort of way that you're talking about. Entrepreneurs who are already failing couldn't implement the advice. They were already going down the hole, and there was no way to kind of pull back. So it was, so we know this stuff can be effective. And that was before coding, right? That was literally, what should I do now? And entrepreneurship, like, it turns out, by the way, about 50% of Americans have had

• 13:18Speaker 04

  a good idea, or they think they have had a good idea, for business in the last five years, and almost nobody pursues it because of all these barriers. So even just the advice to overcome those barriers is a real golden age for entrepreneurship compared to what we've had before.

• 13:30Marina Mogilko?

  How do you know when to trust it? You mentioned newer models. Do you think the study will no longer apply? Because newer models are so much better, or you still need to have this judgment?

• 13:38Speaker 04

  I talk about this in the new book, but I think there are sort of four things you need to use AI successfully, and that's deep knowledge, right? Some expertise in something. And that's, by the way, like, entrepreneurship is all about getting disciplined expertise, right? You go to an industry area, you interview people, you learn about the industry, you test ideas out. I mean, that is building expertise, and I think that's every bit as important as it was, right? Otherwise, you're just sharing Claude ideas. And Claude ideas are, look all very similar to each other, right? Second is wide knowledge. Increasingly important for entrepreneurship is knowing a little bit about the fields you're not in, right? So if you're in software, you should know a little bit of design and a little bit about accounting, because that helps you apply the AI more widely. And then agency, which we've been discussing a little bit here. And then finally, some sense of taste, like selecting good ideas out of the slot pile

• 14:24Speaker 04

  is also really helpful. So I think that what we're seeing with the more agentic systems is they can run longer periods of time, right? When GPT-4 could do four minutes of work independently with a 50% success rate, now we're up well over 17 hours of work. But you still need to be able to have some guidance in these systems. And even as they get really good, it turns out the human touch actually does matter because variation is what makes entrepreneurs successful.

• 14:46Marina Mogilko?

  Yeah, and that's the whole challenge, right? When everybody has Claude and Claude is advising, or ChatGPT, whatever they're using, advising on the strategy, how do you avoid sounding just like everyone else?

• 14:56Speaker 04

  Yeah, I mean, it's kind of funny, because you realize that people are more important than we thought, or that some people thought, right? So everybody who was early on to using Claude as a good writer back, you know, when Opus IV, V, or whatever, they wrote really well. And for a while, nobody knew what Claude writing sounded like. And now, if you open LinkedIn or X, it's just nauseating, right? It's like, what's carrying the weight? What is it? It's just all the same stuff. And even not just the little phrases, like, the way it approaches things, the kind of language it uses. And you're like, well, they should, if you didn't hire a marketer, you're probably making a mistake. Now, the marketer could do a lot more work than they could do before. But their human variation actually matters quite a bit. And I think we're seeing that in a lot of other cases, which is the AI pushes people towards a sort of a consensus strategy or view.

• 15:40Speaker 04

  We have some experiments where we found that we can make the ideas almost as diverse as a group of people. But if you map the ideas that the AIs come up with and map the ideas that a group of humans come up with, the AI still doesn't cover the full range of human ideas, even if you're pushing it really hard. So this variation piece is going to become increasingly important. Like, what are you adding to the system to guide or steer it so you're not just clouting about everything? And it would be okay, like, it's okay if you're using Claude for finance and that's not your competitive advantage. But in the area where you are narrowly aiming, you need to stand out, you need to differentiate, or you need a moat that has to be human to some extent, even as the models get incredibly capable.

• 16:15Marina Mogilko?

  Yeah, you mentioned taste, and this is a question I ask a lot of my guests. How do you work on taste? What do you tell your students?

• 16:21Speaker 04

  We know a little about teaching people taste, right? Like, you need to see good and bad examples of things. So like, you want people to explain to you how to pull something apart. So I, one of the, I bring into, an art professor, actually, in my classes now, who talks about sort of thinking about, like, how does a designer see the world, right? So you get this kind of perspective, like, what makes a font good or bad? What makes an image interesting or not? And so I think we need language to discuss things for taste, right? If you talk to somebody who really understands movies, they have a taste for movies that's individual, but they can probably talk widely about that. In the same way, like, venture capitalism, like, venture capitalists are all about taste, right? They develop a sense of taste for a particular kind of problem space or not. So I think that seeing a lot of variation is really important.

• 17:05Speaker 04

  Learning to be disciplined about this is another example of why I think education has some value here. Being exposed to lots of options, I mean, those help give you a sense of taste that you need.

• 17:15Speaker 00

  Reports due to, then meeting prep, then...

• 17:18Speaker 01

  Chronic migraine doesn't care about your to-do list. For people with chronic migraine, 15 or more headache days a month, each lasting four hours or more, Botox, anabotulinum toxin A, prevents headaches in adults with chronic migraine. It is not approved for those with 14 or fewer headache days a month. Botox prevents, on average, eight to nine headache days a month versus six to seven for placebo. Prescription Botox is injected by your doctor. Effects of Botox may spread hours to weeks after injection, causing serious symptoms. Alert your doctor right away, as difficulty swallowing, speaking, breathing, eye problems, or muscle weakness can be signs of a life-threatening condition. Patients with these conditions before injection are at highest risk. Side effects may include allergic reactions, neck and injection site pain, fatigue and headache. Allergic reactions can include rash, welts, asthma symptoms, and dizziness. Don't receive Botox if there's a skin infection. Tell your doctor your medical history, muscle or nerve conditions, including ALS, Lou Gehrig's

• 18:02Speaker 01

  disease, myasthenia gravis, or Lambert-Eaton syndrome, and medications, including botulinum toxins, as these may increase the risk of serious side effects. Ask your doctor. Visit botoxchronicmigraine.com or call 1-800-44-BOTOX to learn more.

• 18:14Speaker 02

  Your sausage McMuffin with egg didn't change, your receipt did. The sausage McMuffin with egg extra value meal includes a hash brown and a small coffee for just $5, only at McDonald's for a limited time.

• 18:27Speaker 03

  Pricing participation may vary. Promotion pricing may be lower than meal pricing.

• 18:30Marina Mogilko?

  How is it different from judgment? Because AI is doing a lot of judgment calls. Like, you present the data and analyzes it, tells you, like, this is a better decision. How is it different when you approach it from the taste perspective?

• 18:42Speaker 04

  I don't like the idea of judgment, per se, because AI is obviously doing judgment. You can't do a 17-hour task without the AI making judgment calls. Just like, we don't have creativity in the list, because the AI is obviously making creative decisions. The question is whether they match the kind of judgment and creative decisions you would make, right? So we can call that under taste or knowledge, but it's a kind of a different question, right? So judgment is this hard thing, we don't really know how to teach, and we don't really have a good example of it, but some people, you know, what context is that judgment happening? It becomes a question. So it's a hard, it's a hard thing to ask, right? Is like, what is judgment? And so the AI is exercising some kind. There's obviously a human judgment element that's different, but we haven't had to build the language around differentiating these things. I kind of think about what's happening in math right now, right?

• 19:25Speaker 04

  Which is this sort of explosion of proofs generated by AI. And one of the things mathematicians will tell you when you talk to them is a bunch of revelations about their field, or maybe clarifications about their field that outsiders didn't know, right? There's an aesthetic element to math that actually matters, right? To a mathematician. That if the proof doesn't help teach you something beautiful or interesting, there's no value in it for the field of math, even if we've solved this problem, who cares? So we're starting to differentiate things that we didn't have to differentiate before, because the only kind of judgment and intelligence we have is human, so we didn't have to come with a different category.

• 19:58Marina Mogilko?

  And it's so interesting, when I think about... And taste and judgment, it feels like people who have experience in the field are better positioned to learn that versus someone who's just starting out. Instead of getting time to learn basic things, they need to go straight into judgment, straight into taste. Is that how the market is progressing with AI?

• 20:15Speaker 04

  There's a couple of things happening at one time, sort of when you think about markets and education, right? So first of all, we are temporarily in a crisis in universities. I mean, we're always in a crisis in universities, so this is not new. Anyone who's been in education knows this, but we are in a temporary AI crisis, right? And we already kind of know the outlines of that solution, which is AI is incredibly good tutor when used as a tutor. And it's very bad if you're just cheating and getting answers from it, even if you think you're learning, you're not, right? So eventually we'll have AI tools outside of class. Inside of class, we'll do more action learning. I mean, if you've been to a business school, law school, you've already done these kind of case studies and discussions and activities. And, you know, we'll do in-class tests, we'll figure it out. The real problem is where people actually have developed taste and judgment is in the world, right?

• 20:59Speaker 04

  And I, we had this great deal for the last few thousand years, which I would send you, you know, a newly trained recruit who didn't know anything. And we had apprenticeship, right? That's how the last four thousand years of white collar training have operated. So you'd get a really junior person who is desperate to show you they were good. They work for cheap because they wanted to move up the realm, the rungs. And they would do the work that a middle manager didn't want to do. And in return, they would get a little bit of pay, they'd get to show off their stuff and they'd learn the ropes. And the middle manager would get their work passed off to somebody who could at least do some of the work for them and make their lives easier, right? And that's how they moved up the management chain. And that all broke, right? Because every, you know, your incentives as an intern or as a first year associate or whatever you are, your incentives are a hundred percent deliver the best product you can. Who's better than you, Claude or ChatGPT or Gemini?

• 21:42Speaker 04

  So you will deliver Claude or ChatGPT or Gemini answers. And this is not like being digitally native because you don't know what you're delivering, you're just handing stuff over. And every middle manager has realized, why would I give feedback to a human who takes a lot of time to do the work? I could just have Claude do it myself. So we've just broken the entire system that forms professional taste and judgment, and we haven't figured out a full replacement yet.

• 22:02Marina Mogilko?

  What are your thoughts? What are you telling your students?

• 22:05Speaker 04

  So I think that there are a couple of things. One is at the individual level, we now have to start being really disciplined about when we're going through cognitive surrender, when we're giving stuff up to AI. And that's really hard to do. We're built like, first of all, entrepreneurship, which is what I teach, right? And what we've been talking about here, like, that's about trying to make your life as easy as possible so you can focus on the hard stuff, right? And, you know, but then you learn nothing, right? As a result of that. But it's also true in every other aspect of life. Like, we can easily give up skills to the AI, and I think we all feel tempted every day to do this. And we may never get those skills back. So we have to purposely add friction to our lives in certain key areas that we maintain an advantage, but we're not incentivized to do so. So the individual level, it's going to be making choices. At the organizational level, companies are going to do one of two things.

• 22:50Speaker 04

  They're going to either follow the easy path or hard path. And I worry about the easy path, which is only hire senior people. And we're seeing some signs of that. I know you've had some economists on talk about this, there's some early signals that junior jobs might be threatened. We're not a hundred percent sure yet. As you, you know, or the other option, the harder option, is that we actually make training a serious part of jobs, right? So you actually have to give people time to be inefficient and learn on the job. And we, rather than learning, be sort of an accental thing, we'll have to teach them. Maybe you have to actually add an extra year to universities that we teach you to be a, you know, a Google person or a McKinsey person or, you know, whatever it is. And there's a year of customized training for you to teach you, because we know how to teach and assess people in classrooms that we don't know how to do in the real world anymore.

• 23:32Marina Mogilko?

  This is such a tough problem. And then I'm thinking about a professional who even completed that step and they graduate being good enough, but not exceptional. And the models keep improving. What is happening to these good enough people?

• 23:46Speaker 04

  So everything is too early to tell anything yet, right?

• 23:49Marina Mogilko?

  I know, but do you ever, like, when I think about, like, where are we headed?

• 23:52Speaker 04

  I mean, I, but also, how long is good enough, good enough, right? Like, the issue is, right? Like, I mean, that's progressing too. So, you know, this concept that we developed in our first paper where we examined this, we called the jagged frontier, that AI is good at some stuff you wouldn't expect and bad at some stuff you wouldn't expect. And it's hard to know in advance what that is. As long as the frontier remains pretty jagged, there are always bottlenecks that humans kind of need to fill in. And those can be really weird ones. Like, if you think about, like, AI replaces a lawyer, well, it turns out a lawyer does lots of things with friction in the world that the AI can't necessarily do, even if you had a robot doing the work, right? Because there's human systems they interact with, there's legal requirements they interact with, some people want to talk to a human. So there creates these bottlenecks that, that, that actually reward different talents than before. You see this happening in coding too, right?

• 24:35Speaker 04

  Like, what makes a good coder now is completely different than three years ago because you're not writing code anymore. So I think one thing I would say is good enough is a bundle of skills. And it may very well be the people who are like mediocre coders, but really good at the vision about building software.

• 24:50Marina Mogilko?

  Or communication.

• 24:51Speaker 04

  Yeah, or communication. And those will change too over time, right? I'm not going to say the AI will never do that task, it probably will. So I think we're going to see ourselves ping ponging across what skills are valuable in a really rapid way without an easy answer.

• 25:03Marina Mogilko?

  Yeah, and I really like the jagged frontier study. And I was getting thoughts, maybe it's us who don't understand the models, good enough? Because sometimes you re-prompt and you get the result that you wanted.

• 25:14Speaker 04

  Well, my, one of my favorite phenomena is watching people who know nothing about a field, getting the AI to do like impossible things. And you're like, wait, how'd you do that? You're like, I just asked. You're like, you can't ask the AI for that. And they did, and it like works, and you're like, oh, great, you know, what are we doing here? So, I mean, there is this sort of weird phenomenon. There, I mean, there's two things that we know make you better at AI. It used to be AI was a level, mostly a leveler, right? If you're a bad performer, you got a boost. That still is that way. But what we're increasingly seeing, and there's some nice results of Anthropic as well as some other studies, that suggests that the people who are, have the most expertise get the most gains from AI at this stage. And that wasn't true until agents kind of came around in the last year or so. But the other thing we know is that there's some AI whispering ability that is like knowing how the AI thinks, that might be sort of theory of mind.

• 25:58Speaker 04

  Like, if I know people really well and I get to know the mind of Claude or ChatGPT or Gemini, whatever I'm using, I become more effective in working with it. And so those are sort of two different skill sets. And then there's like a mysterious third category, which is like, some people seem to be good at this in ways we don't know. There's this great study by Alex Emes and a few others, he's now at, he was at Chicago now, he's at DeepMind. But they measured people creating agents for negotiation. And a couple of interesting things. One thing they found is your skill multiplier becomes much higher because the agent engages in many negotiations. So a small difference in skills in creating good agents has a huge impact on bottom line. The second thing they found was that the people who are good at creating agents are not only people who would be the best negotiators in real life. I mean, even, there was even a gender divide.

• 26:41Speaker 04

  So one of the things that we find repeatedly in studies for a wide variety of reasons is that women tend to outperform in negotiate, underperform in negotiation. And there's been a lot of work on this, it's been a big topic of concern, but women made much better negotiation agents than men did. And, you know, and again, we don't actually know, like, I don't even know how to teach you how to do that necessarily.

• 27:02Marina Mogilko?

  Is that a management skill?

• 27:03Speaker 04

  I mean, there is a management skill, there's a theory of mind skill, there's an expertise piece. But like, things are happening so quickly that, you know, like, again, we can't pull them apart that easily.

• 27:13Marina Mogilko?

  This is, and I just had Felix from Anthropic in the studio, and he was talking about how it's very important to try and understand your workflow. Like, what's going on in your head when you create a perfect text? Or when you go through a perfect negotiation, like, what was your thinking like? So you can translate that onto AI and this is how you build a workflow.

• 27:34Speaker 04

  I feel like that's kind of made up. I mean, it might be true. To me, the analogy that makes the most sense is somebody who's running organizations and managing people is like thinking of AI like a team I'm delegating work to. And if I can do a good job delegating to a team and realizing where a team goes wrong, which is very different than knowing my own process, right? The AI will not replicate my process. And part of what I try and do is like preserve stuff for me that way also, right? Like, I write, you know, it might seem crazy since I do Substack and X, everything, I write all my own stuff, right? I write first drafts of everything on my own. I mean, yeah, I type it all out, right? Because if I don't do that, the AI is very seductive and will absolutely like dominate what I'm thinking in any direction. And then you're just going to read like Claude results filtered through me.

• 28:14Marina Mogilko?

  But like, do you use voice at all?

• 28:16Speaker 04

  I use voice sometimes for this. I find like just typing on a computer is easier for a lot of this stuff. But yeah, voice and like, you know, and like, you get good at figuring out how you generate ideas. Like, for me, weirdly, I come up with my best ideas is during Soul Cycling, which I think I'm the only guy and the oldest person in the room. And somewhere, you know, but somehow being in a loud room where I'm biking and people are yelling at me about achieving my goals, I tune it all out and I come up with good ideas. I don't know why. But everyone has, learns their own way of thinking. I've talked to another scientist who like, I need to be extremely overtired one day a week and then I come up with my best ideas. So like, you figure stuff out in that kind of front, but you also have to preserve parts of what you do that are very human. I don't think it's necessarily about replicating my human process.

• 28:57Speaker 04

  It's about preserving a human process and then delegating work the way I would to a team. Like, I have a really good idea, here's how I want you to execute on that idea.

• 29:05Marina Mogilko?

  So you will never delegate generating ideas to AI, is that what you're saying?

• 29:08Speaker 04

  I wouldn't say never delegate, but you have to be careful. I mean, we actually have old evidence on this. Brainstorming was invented, I think, nineteen fifty-five, and by nineteen fifty-seven, we knew it was a bad idea. Because basically, anytime you do a group ideation process, your, your ideas become fixated on whoever says the first idea out the gate. And then there's a judgment process. And so anchor on other people's ideas. So you lose natural variation. So the way you should do brainstorming before any group process is everyone writes down ideas individually and then only start generating ideas as a group. Once you've done individual idea generation, you end up with much better ideas with higher diversity. Right? Same thing with AI. You should absolutely use AI for idea generation, but only after you've generated ideas for yourself.

• 29:47Marina Mogilko?

  So here's what happens after a single podcast. When I learn something new about AI, I go ahead and change some processes in my company, see if it works, see if the models are really powerful enough to handle that thing that I just delegated, and then I write about it. I don't type my emails like Ethan does. I actually talk to my AI and then edit them. And I also have a person helping me with that. But it's basically a summary of what we tried with AI, what worked and what didn't. And of course, a lot of strategic stuff, how I navigate this media career and building businesses as a mom, as a woman, as an immigrant. So if you want to get a closer look at my process, subscribe to my free newsletter, Future Proof, and let's grow together.

• 30:30Marina Mogilko?

  And you talked about this effortful thinking and having AI as companion. Can you describe that process?

• 30:35Speaker 04

  Yeah, I mean, I think that what you want to do is you're kind of trying to mentally sweat while doing this. Like if you're not kind of, if it's not uncomfortable, you're probably not doing enough learning or thinking. Like it works just like exercise or anything else. Right? And so in learning, we call this desirable difficulties. Like if you're not encountering difficulties, you're not learning. And so there's certain mental processes like ideation that can't be streamlined that much. Right? And it's very tempting to do that. And we don't recognize, we've learned to recognize the cost of like, we never walk around or, you know, we never, you know, we never exercise. We haven't learned the same kind of thing with our brains because there was no alternative. Right? You know, you could Google something, but that was not the same kind of thing as AI do my thinking for me. So I think that we have to be recognizing that we're doing that kind of effortful work.

• 31:19Marina Mogilko?

  So interesting because I got to thinking that maybe like the way I get ideas could be a workflow. Like maybe there is something that's happening in my life that leads me towards certain idea and maybe AI can help me pinpoint.

• 31:32Speaker 04

  It can. I mean, I just think you have to be careful. And by the way, there is a, in ideation, there's something called the equal odds rule that turns out to be true, which is that some people generate both more ideas and those people who generate more ideas generate better ideas than most other people. And it's not necessarily deeply correlated with intelligence. There's creativity is a trait. Some people are just more creative. So if you're less creative, I think we don't need to hold creativity on a pedestal. Right? Then maybe AI is helping you out more, right? Generating ideas. If it's an area that you have competitive advantage, kind of going back to where we were before, or writing, or whatever it is, or podcasting, right? We can generate an AI podcast. It will not be as good as an expert having done this a lot and thinking through this. So at least for now, that human edge, right, is a real one. And that's where you want to make sure you're not really giving stuff up. Right?

• 32:17Speaker 04

  I worry less. I mean, ideation is just one of many traits. Like, most people don't have good ideas. That is okay. We lionize ideas too much. Right? Execution, you know, emotional connection, there's lots of other great things people could do. And so you want to be strategic about when you're using these systems.

• 32:30Marina Mogilko?

  I really like what you said about people who generate more ideas tend to generate more good ideas. And I think we just need to be learning how to constantly generate things and like, maybe write them down or test them out because it's so easy to test out ideas.

• 32:42Speaker 04

  Yeah, I think that that, ideas that don't do anything aren't that useful. The whole discipline of entrepreneurship before AI, right, was built around this idea, especially Silicon Valley entrepreneurship and the lean startup and all the other dominant methods, strategies that we teach people to become good entrepreneurs. All of those strategies come from this idea of learning at the cheapest cost you can. Right? So it was all about like, don't just come with a beautiful idea in the building, right? As the cost of software development dropped, the goal was get out there and talk to people, show them your product, right? Do customer interviews, create minimum viable products and learn quickly. And then all the VC sort of reorient around this too. Right? It was originally built around giving you a giant check to scale your idea.

• 33:25Speaker 04

  And it became, how do we give you seed rounds and pre-seed rounds and why combine it or all of the, basically the entire evolution of entrepreneurship was based around the idea that it got cheaper and cheaper to test ideas over time. And so now we're really hit rock bottom cheapness. Right? Like you can, don't trade, AI is not a customer when you simulate customers, but it's a pretty good first pass at being a simulated customer. There's now no longer an excuse to not at least push your ideas a little bit.

• 33:50Marina Mogilko?

  Yeah.

• 33:51Speaker 00

  I hope I don't have to cancel again tonight.

• 33:53Speaker 01

  Chronic migraine means 15 or more headache days a month, each lasting four hours or more. Botox, onabotulinumtoxin A, prevents headaches in adults with chronic migraine. It is not approved for those with 14 or fewer headache days a month. Botox, the number one prescribed branded chronic migraine preventive treatment, prevents, on average, eight to nine headache days a month versus six to seven for placebo. Prescription Botox is injected by your doctor. Effects of Botox may spread hours to weeks after injection, causing serious symptoms. Alert your doctor right away as difficulty swallowing, speaking, breathing, eye problems or muscle weakness can be signs of a life threatening condition. Patients with these conditions before injection are at highest risk. Side effects may include allergic reactions, neck and injection site pain, fatigue and headache. Allergic reactions can include rash, welts, asthma symptoms and dizziness. Don't receive Botox if there's a skin infection.

• 34:35Speaker 01

  Tell your doctor your medical history, muscle or nerve conditions, including ALS Lou Gehrig's disease, myasthenia gravis or Lambert Eaton syndrome, and medications including botulinum toxins as these may increase the risk of serious side effects. Ask your doctor. Visit botoxchronicmigraine.com or call 1-800-44-BOTOX to learn more.

• 34:51Speaker 02

  Your sausage McMuffin with egg didn't change. Your receipt did. The sausage McMuffin with egg extra value meal includes a hash brown and a small coffee for just $5. Only at McDonald's for a limited time.

• 35:03Speaker 03

  Price and participation may vary. Promotion pricing may be lower than meal pricing.

• 35:06Marina Mogilko?

  Think a good idea is worth more now versus five years ago?

• 35:09Speaker 04

  I think a good idea is worth more when combined with execution. Right? I think that there is, that you, that an idea alone has never been worth that much. I mean, it's great, but you know, one of the conflicts that typically happens in entrepreneurship classes all everywhere is the person who came up with an idea wants all the credit for the idea. And one of the things you see is, you know, as sort of a semester goes on and then a lot of startups out of work and turn into real businesses and raise lots of money and end up being very successful. But you kind of see the process of like, oh, everyone trades, the person who came up with the idea is like, that was the imaging thing. And then by the time you're, you know, six months in, the idea has changed. It's rarely the idea that has made everything successful. Right? It's a set of explorations and everything else.

• 35:49Marina Mogilko?

  The execution.

• 35:50Speaker 04

  So I think ideas have always been sort of a dime a dozen and the AI can generate those ideas now better than anyone else. But if you actually execute on them, there's real value there.

• 35:58Marina Mogilko?

  I agree 100%. And when executing ideas, you become a manager, either of people or agents. And there are so many people who are now not forced, but they have to manage agents because of their work. What are the tips that you will give them, the management tips that they should be deploying?

• 36:13Speaker 04

  It's the revenge of the MBAs, right? Like I teach at, you know, I, you know, I teach at a business school. So it's like very, it's gratifying to be like, haha, everyone has to be a manager now. All my business people were looking for technical people and now all the technical people need management skills. But actually, those management skills do apply. Right? So what you need to, it turns out, I mean, we've been doing a lot of experiments with that at the generative AI lab at Wharton, which is what I lead. And a lot of the experiments basically show, you know, look, techniques that work on people work on AI. And so I think what you need to be thinking about is how do you manage a process? Because one of the dangers of, of the cost of an AI system is have assigning a lot of work, not even the token cost, just the effort cost of monitoring work is quite high.

• 36:57Speaker 04

  So we built all these systems for doing that. Right? So if I'm specifying things for an agent, I will specify things like, here are the points I want you to check in with me on about the work. Here's where I want you to give me diverse examples that I can select from. So I make a choice, right? I'm deciding when to insert my own agency into the system rather than just sending off agents to do work. I am making sure to specify what good and bad looks like in really kind of clear ways. And I'm watching really early on the process to see whether or not there's a failure in my ability to communicate to the agent. Right? Oftentimes they'll head off in a direction that sort of like, you know, Claude has its own obsessions, right? They'll head off in those directions unless you intervene quite early on and often. So, you know, those management skills about thinking about, am I clearly specifying what I need?

• 37:40Speaker 04

  Am I, you know, am I establishing a feedback loop? Right? Am I dealing with, you know, reducing uncertainty over the course of a project? Those things end up really applying here.

• 37:49Marina Mogilko?

  Interesting. What is the biggest mistake that you think people are making when they're managing those agents?

• 37:54Speaker 04

  I think they're trusting the agents too much. You'll get an amazing result. Like I, everyone loves sort of, I do a lot of single prompt, like this is awesome kind of stuff. But the guidance process is incredibly important. And I think that it can be very disempowering, both because the agent knows things you don't, but also because the interfaces we have to work with AI now are increasingly hostile to augmentation and embracing automation. Right? They hide the chain of thought, right? And even the chain of thought, thought summary is no longer pure. It's supposed to feel more like magic and that's disastrous for being a human in the loop. I don't think we can be in the loop anymore, but managers are not really in the loop. They assign work and they have people check in on them. So we have to now create forced positions for the AI to report back to us. Otherwise they will try and do magic for you and you don't want that.

• 38:38Marina Mogilko?

  Can you describe a process in your own workflow where you can show us how you create this result and how we create the feedback loop so that people can learn?

• 38:48Speaker 04

  Sure. Like I was exploring a sort of random topic that interests me in the humanities, which is, and I've done this for entrepreneurial pitching as well, which is there is a lot of research that stories have shapes. Right? And so there is, and so like there's a rise and fall stories and, you know, and like the Cinderella stories. And this is like a longstanding question. And I actually did some research before with primitive LLMs about whether or not we could take entrepreneurial pitches and find the shape of a good pitch. So it's a topic that's really interested me for a long time. And as agents have gotten better, one of the things I've had them do is assign them to explore this topic. Right? And so there's a few things there. One of them is, you know, I was assigned Then to kind of generate a bunch of ideas on how to do this. And then I, but I didn't have it then execute its favorite idea.

• 39:33Speaker 04

  Cause that would have been bad. So I had to give it feedback part way through. So give me a bunch of ways that you might approach this, figure out what the shape of story research is real. And then, you know, it was like, speculate for me, just like I would for a student. Give me some thoughts about how you might approach this and give it feedback again. Great. Why don't you execute one, two, three, and four, first draft of all of them. Show me the results and then let's pick which example works better from this. Great. I really like what you're doing with three here, but number two has this really nice feature. Let's talk about how we combine those together, right? Why, you know, can you show me where the, you know, this looks wrong. Can you show me why you think this went wrong? So it's almost like a Socratic method kind of thing of me guiding the AI and giving myself lots of choice points where the really tedious, annoying work, get, you know, figure out where 5,000 story of 5,000 stories are.

• 40:18Speaker 04

  Build the semantic coding tools I let the AI do. So I get the interesting work for myself. And like, that's, if there's one message of sort of the latest book and where I'm most desperate to try and communicate with corporate America. It's like, we have an automation world that will be bad for everybody in a lot of ways. But an augmentation world is like the dream of work, right? You do the interesting stuff and the AI does the crap stuff and you get to just direct an organization and it's really fun.

• 40:41Speaker 06

  Before we keep going, I'd love to get to know you a little better. I share a lot about my work here and I'm curious about yours. What do you do and what you'd like more help with? We've put together a little survey. Your answers will help me decide what to cover in future conversations and what resources to put together for you. I'll leave the link in the description. Please, please, please take a couple of minutes and tell me about you, about your business, about your ambition. So I can make better content for you. I really, really appreciate you. Thank you so much for doing this.

• 41:12Marina Mogilko?

  The problem with this that I see is that the models progress so fast. So you build a workflow, you describe things and then, I don't know, the new model comes out and they tell you it's better to not give it a workflow because it's so good at understanding what data to use for a particular workflow. And sometimes it thinks about things that you didn't think about and it can make a better workflow. How often do you have to rewrite the whole thing?

• 41:34Speaker 04

  So that's not a workflow I was giving you, right? That wasn't step one, I do this and step two, I do this. This was the way I would manage a researcher team or, you know, or a startup team.

• 41:42Marina Mogilko?

  It's a one-off thing.

• 41:43Speaker 04

  Everything, everything's an interaction where I'm learning from what the system does. I don't know in advance what's gonna be good or bad. If you're trying to launch a startup company, have a deeper conversation, reach some sort of conclusion, come with a marketing plan, you know, do financial analysis. All of these involve lots of choices. You don't want the AI making those choices, even if they're good ones. You want to be in, you know, inserted into that process. So it's sort of a meta process, right? The idea is like, assume you have a really smart person or a really incredible team doing work, but you still want it to be your work. When should you talk to the AI about this, right? You still want that visibility into the system. Otherwise you're just, it's just Claude. Even if it's brighter than you, you're inserting nothing into the process anymore. And so anyone could type the same sentence to get the same result.

• 42:22Marina Mogilko?

  What are the skills, I know you wrote about it. What are the skills that you're just delegating to AI and you, you said you can't wait to delegate a few skills?

• 42:31Speaker 04

  Basically everything having to do with, so fact checking is great.

• 42:36Marina Mogilko?

  And it's really good at it now.

• 42:37Speaker 04

  Really good. Actually, it's kind of, it's creating a kind of crisis on how good it is. I saw with my latest book, when I delegated to the, to the AI to fact check it, right? And I had human fact checkers. It's a big release, right? But not only did the AI spot errors that I made, but even more often it spotted errors in the stuff I was citing. So it's like table two in this really complicated academic paper has an error. And I'm like, what do I do with this? Like, do I email, like, what, you know, we have a whole method of correct, like correction in science. It doesn't make any sense anymore because it's, but it found an error in an AP story I was citing that it was, cited the wrong city and event took place in like, it's like meta fact correction, right? So like superhuman fact checking and I've been fact checked by the New Yorker and other things like, this was pretty amazing.

• 43:21Speaker 04

  And then there's just like a lot of like busy work, right? That the AI does really well, like never fill out another student form for your child if you can avoid it, right? Like immunization forms, it does really well. So, you know, and, you know, then there's the tempting line of like, oh, should I have it do all my mandatory training for me? And the answer is like, not yet, I think. But, um, so I think that there, when I talk about deskilling myself, like I cannot wait to descale myself on a whole bunch of useless things I had to be good at doing, filling out forms, you know, particular kinds of legal documents, signing NDAs. Like I wanted to be able to handle those things because I think it's well within the range of what the AI can do.

• 43:53Marina Mogilko?

  That worries me a little because I had this thing a couple of days ago, I was on an airplane. So the wifi was really shitty and I had to fill in the form. So I asked my dot to fill it in for me, but I just said, fill it in. But what it did, it sent it and I only realized in the morning and I'm like, oh my God. And my team was like texting me, this doesn't sound like you. And I'm like, oops. But then I realized how convenient that was, like, because it completely took it off my to do list. And it's kind of scares me because dot drafts all the replies and I just correct maybe 10% of it, but it's so good. And I'm like, what's, what's gonna happen?

• 44:27Speaker 04

  So I had to set a line where I have a dot and I also have a codex scheduled tasks going a couple of times a day, summarizing stuff and, and giving me what's happening in my world. And, and I won't send drafts to anyone who I like, I have, I write stuff to myself, to people who I have any kind of connection with or I feel bad doing it, right? So that's another line I'm trying to do. It drafts a lot of nice rejections for people aren't students yet, questions about my class. But I, I mean, I had the same thing where the AI emailed my dean about something, you know, when I was like, don't do this. And so, but this is part of why also the sort of YOLO, you know, trusting everything. Like you do have to build a set of trust with this, just like with people. And like, once you build that, you're like, okay, I know why it fired off that message, right? Dot is a little too proactive. I mean, again, no hiding stuff, right?

• 45:12Speaker 04

  Like when I have a scheduled task running codex, I have a system prompt, right? And I have, I understand the context it's in and I've been tweaking that, that prompt for it over time to make it good. And, you know, and using for low stakes before higher stakes. And dot just encourages you like, oh yeah, just ask the questions, but we don't know it's system prompt. We don't know, there's no place to change the instructions about how proactive it should be. And that's a little more ominous.

• 45:34Marina Mogilko?

  I think you could, it's just when things are moving so fast, it's so hard for me to find time and reprogram my whole system because there's this new thing that just came out.

• 45:43Speaker 04

  But I actually also think that part of what makes dot and muse do what they do is they're much more magical. Like there are less opportunities to get yourself deep into what they do. And in some cases, that's great, right? Who wants to worry about it? And the other, I don't know the behavior I'm expecting. So my trust has to go after experience, right? And there is a little bit of danger of like, we're taking too much stuff off our plate too quickly. Not because writing, you know, an NDA and signing it back is useful. I, I trust that the AI will do a good job spotting errors in an NDA, right? In fact, if I, my gut feeling would be, it would be too nervous about signing NDAs, right? Which are like not that important or like, you know, but like it would be too nervous about it. But so like, the question becomes, how do you start to work with that? And look, I've had to work with, build human teams many times over my life and there's a trust building process.

• 46:26Speaker 04

  And what I worry a little bit is being taken out of the loop of not the AI doing the work, but the trust building process of understanding what the AI wants to do. If you're used to models, the model families feel fairly similar to each other. So like, I, I had a good feeling for the, you know, for the four series, you know, and the GPT five series, right? And then, but then you have a new kind of pre-trainer, a new model comes along and suddenly like, you know, Opus five, five doesn't feel quite like, you know, Opus four, six. You don't trust the same way. So there is this issue. And I, I mean, by the way, if you're having this problem personally, can you imagine what's happening at companies right now? Like organizations, they're, they're being forcibly updated to everything. And like, you show up and like your Claude code just updated overnight for, and you did training for 500 people.

• 47:10Speaker 04

  And now this is like, they've decided everything has different names now.

• 47:12Marina Mogilko?

  What are you telling managers of those organizations?

• 47:15Speaker 04

  So what most of them are doing is it turns out like, the thing is LLMs are so unreasonably effective at everything that almost all their sins are forgiven, right? So like, they don't have a choice. Like it's either like, and most of them adapt, right? I mean, I think muddling through is underestimated as the default for humanity, right? Like, and I often point out to organizations like, look,

• 47:38Speaker 04

  if you had told me in advance before COVID, that every organization on the planet would have to leave their office with no notice and not come back for a year and a half with no contingency, no planning. You would have thought, like, if you had predicted in advance, you would say like, hey, about 20% of companies will go under as a result of this, right? As far as I can tell, not one of the 5,000 biggest companies in the United States went under as a result of COVID. We adapted and I think that people are better adapting than we think. We have a whole bunch of systems that are built around normal times and normal technology that are all absolutely breaking right now. But I think we're, people are proving to be more durable than we think. Not everyone, not every organization, but more than you think.

• 48:13Marina Mogilko?

  Yeah. It's just making some of us upset.

• 48:15Speaker 04

  Yeah. I mean, but well, it is upsetting and coordination is really hard, right? Like, and I get used to something and it gets updated and like, it's not even the model is as much as the interface is changing. Like, when do I trust that? That's changing all the time. It's your intimate personal assistant and they're giving a brain surgery every day with no notice.

• 48:31Marina Mogilko?

  You know what was upsetting for me last week? So I had this whole process of coming up with questions for the podcast, selecting titles and thumbnails. And I have all the data from my previous podcasts. And then I just asked our grown, the new Gemini's model, did I'm interviewing this person? Come up with questions, just that, without giving any context. And it went like, it gathered all the amazing data. And then what was striking for me, it went into my granola, found out that two months ago, I had a conversation with a YouTube strategist. And they pinpointed something about that type of guests that I don't remember. And it was never in my strategy, just because, you know, we just talked and that's it. And it mentioned that and it created the whole workflow based on that. And I'm like, oh.

• 49:13Marina Mogilko?

  So now I realize there's so much data that I haven't included in my process, which makes it probably not the best process in the world because Argon is so powerful now at identifying those things. Have you had any?

• 49:25Speaker 04

  Well, I mean, this is all academia right now. Like, we are, we, you know, you talk to lots of academics and we've been doing science roughly the same way, you know, for 100 years or so. And there's all of this stuff that is like improvised and jury rigged and limited by what humans we are. And we don't know what to do. Like, forget like the obvious stuff, floods of junk papers coming after every academic. Like, you know, that, like, and the first wave of trivial but true papers, right, that we had to deal with that. Now we're like starting to get AI doing really good research. And then also, just like to get really nitty gritty on the science side, like, there's also this kind of crazy thing, which is like, you kind of realize that all of science is P-hacking to some degree, because we're humans, we have to select topics that interest us.

• 50:08Speaker 04

  So we're pre-selecting topics that we think are interesting, right? And that we think will result in a paper. And now the AI doesn't care about any of that. So it can generate an infinite number of topics and papers. And like, how do we fit that into a science that was human driven? So that's a long way of saying like, yes, all the time, right? All the time, I'm like, I put my book in, it's like, oh, have you thought about this? There's starting to be like deep contradictions, like chapters five, seven, and nine have raised a question you don't really address. You're like, oh, wow, actually it did. Like, and I've had lots of human readers, and it's not going to bother any of them. I know I've written lots of, you know, I've written books, but I'm like, oh, that's a really profound point that I was like, you know, and so what do I do with that, right? And like, then the level goes up. It's a little nitpicky points of like, oh, chapter four, you're vague about this. It's now like, oh yeah, that is a really deep question about human autonomy.

• 50:53Speaker 04

  And like, compared to the AI's ability to get things done, that's really difficult. Like, I, you know, it cannot write my book yet, right? For a variety of reasons, but wow, it has some profound things to say. And like, am I obligated to incorporate that?

• 51:07Marina Mogilko?

  What would be your advice to people who are, again, we met, we talked about people who are upset with the process changing all the time? What is the mindset hack that they should be adopting to just go through it and thrive with it?

• 51:18Speaker 04

  I think you have to have a little bit of excitement and curiosity. I think there's a mindset of like, I think we all are nervous and a little resentful. And like, no, and I'm a very enthusiastic person, use AI for all sorts of stuff. And like, there is no doubt that that is real, right? And I think it's okay to acknowledge that. It's also okay to acknowledge that you have imposter syndrome, because we all do. Like, I'm as up to speed as anyone, you are too. Everyone's making this up. You've had Anthropic and OpenAI people in the chair, they're making it up, right? We're all figuring this out as we go along. So I think it's okay to feel nervous, like, leaving aside existential anxiety, I think it's okay to feel nervous, it's okay to feel a little resentful. I think that there's also a positive side. And I think that's kind of how I view the, like, but suddenly you're able to do so much more. So part of what I do is focus on the more you can do, and with a sense of play and

• 52:02Speaker 04

  experimentation, and that can help you get into better mindset. And just acknowledge the resentment's real, like, it is annoying and like, the labs are not doing us any favors. Like, they have an unreasonably good technology, and then they just keep doing new stuff with it. And sometimes it works, sometimes it doesn't. Sometimes they'll just cancel a whole thing that you depended on for a while. Like, when you interview professors, like, half of them want to quit, and the other half thinks it's the most exciting time they've ever worked. And I think you see the same with executives, even entrepreneurs, right? There's like half, it's like, I didn't sign up for this, right? Like, I didn't want to do it this way.

• 52:31Marina Mogilko?

  It is what it is. I had a question about local models. When do you think they're applicable for individual users? Because I tried a few of them, and I don't think I'm ready to delegate stuff to local models, because they're a few steps behind the frontier. But at the same time, with this all progressing so fast, we're going to have, you know, Argon or Astra on our phones in a few years. What happens to an average user? Do you think we're going to use local models more? And what happens to those large companies? What do you think is going to happen in like, say, five years? Because I think there's a research that says by 2030, 50% of enterprise tasks are going to be run locally. Do you think?

• 53:06Speaker 04

  I think that's a made up number that has no basis in reality. I mean, I mean, first of all, when I talk to enterprises, right? A lot of them don't want to run local inference. And they certainly don't want you having a local, like, they don't want every machine running a model, right? Like, they've been using SaaS vendors for years, it's not that weird, you know? The token costs are an issue, and they're thinking about that. But like, I think a lot of people, so this is a meta point about AI, which is, people keep fixating on the problems of today and assuming they'll last forever.

• 53:31Marina Mogilko?

  Exactly.

• 53:32Speaker 04

  You know, all the legal stuff that we had to do to be like, is, you know, can this do NDAs? We need specialized legal AIs? Nope, just better models. So I think it's, it's a little easy to get fixated and not ride the exponential.

• 53:43Marina Mogilko?

  Is there an overarching thing that we should be thinking about when thinking about the progress with AI?

• 53:48Speaker 04

  I think continuing to be more ambitious and trying to aim to have at least a couple of things that don't work yet, unsaturated personal benchmarks that you're kind of aiming for, or business ideas. So I can still not get a high taste academic paper from scratch out of these systems. So I can get academic papers that are true and correctly written, but they're not interesting. And they're not telling us anything fundamentally new. So that's a big one for me as an academic who studies this stuff. I mean, you can't vibe code a true startup from scratch at this stage. Like, you can't take an idea and say, execute. I mean, that's a big gap, right, between that and reality. You know, I do a lot of work in game development, that's been a topic I wrote about, I've done stuff in. And we're getting closer, but like, the game development from scratch, there's still lots of interactions. Writing a good story that I actually want to read, right, that's coherent.

• 54:32Speaker 04

  So there's a lot of them out there when you kind of, all of these would have seemed ridiculous as even possibilities, you know, two years ago, but we're close.

• 54:40Marina Mogilko?

  That's exactly what can be, I mean, fascinating and frightening, because the overarching thing of, you know, a good game, good research is taste again, right? What if in two years, AI learns taste? I know we talked about not projecting the problems too far into the future, but what do you think the future is going to look like when AI learns the taste?

• 55:01Speaker 04

  I still, so, I mean, we can speculate at any point that AI just does all human stuff better, right? And at that point, I don't have anything useful to say, right? In some ways, because it's like, it just does stuff. I think that the jaggedness is underestimated continuously. And I think the nature of what taste is gets underestimated as well, which is that when we talk about taste, right, we mean your tastes. And that universal, like, like, there's still going to be some similarity to the AI systems in the, for the foreseeable future, that will become tiring for most people under most circumstances. Claude ideas, you know, Claude websites, and you can get to do incredible stuff with your taste added in. So there still remains jaggedness at the outside edges. When that goes away, I mean, what do we do?

• 55:44Speaker 04

  We shrug and say, then, you know, I don't have an answer at that stage.

• 55:47Marina Mogilko?

  I don't know. Buy stocks of these corporations, they're about to IPO. I guess that's a pillar, a safety pillar. Thank you so much, Ethan. That was very interesting, very entertaining, and thank you for sharing your workflows. And please read Ethan's new book, and I really, really like your Substack.

• 56:04Speaker 04

  Thank you so much, and this is a real.

• 56:06Speaker 02

  Your sausage McMuffin with egg didn't change, your receipt did. The sausage McMuffin with egg extra value meal includes a hash brown and a small coffee for just $5, only at McDonald's for a limited time.

• 56:18Speaker 03

  Price and participation may vary. Promotion pricing may be lower than meal pricing.

• 56:22Speaker 07

  What's the secret to being a great host this season? Keeping things deliciously simple, like mixing Simply Cranberry into a festive drink as guests arrive, or serving up a warm mug of Simply Apple on a chilly night. That means less time behind the scenes and more time in the moment with everyone. Crafted with real fruit, Simply juices and fruit drinks make any holiday drink effortless, inviting, and completely delicious. Simply, say yes to simple.