The Professor Who Required His Students Use ChatGPT In Class — With Ethan Mollick

Ethan Mollick is a professor at the University of Pennsylvania’s Wharton business school and writes One Useful Thing on Substack. He is one of the world’s foremost researchers on practical applications of generative AI and is an immensely engaging speaker. Professor Mollick joins Big Technology Podcast for a vibrant discussion of how AI changes schoolwork and office work, covering his decision to make his students use ChatGPT in class. Tune in for a insight-packed interview that will illuminate where the field is heading. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  An author and a Wharton professor speaks with us about AI's application in schools and the workplace and talks through the latest in artificial intelligence research. My conversation with Professor Ethan Mollick, coming up right after this. This episode is brought to you by Genesis. What does it actually take to put a Gentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large scale agentic CX transformation, including the business decisions, trade offs and lessons along the way.

• 0:45Alex Kantrowitz?

  You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility and peace of mind. That's how companies like Razor were able to cut their inference cost by 15 times. Visit akash.network slash Alex for a free consultation. That's akash.network slash Alex for a free consultation.

• 1:17Speaker 01

  LinkedIn presents.

• 1:23Alex Kantrowitz?

  Welcome to Big Technology Podcast, a show for cool headed, nuanced conversation of the tech world and beyond. Ethan Mollick is here with us today. He's a professor at the Wharton School at the University of Pennsylvania. He writes One Useful Thing on Substack. It's a great newsletter. You should subscribe. I've been looking forward to this conversation for a long time. Ethan, welcome to the show.

• 1:43Speaker 01

  I'm thrilled to be here. Thanks for having me.

• 1:46Alex Kantrowitz?

  Great to have you. I think in terms of the people who have actually gone and used AI and written about AI and its applications in school and at work, I don't think there's anyone as prolific as you, anyone who's, you know, who's built an audience and an expertise like you have. So I'm thrilled to be able to speak with you about this. One of the things that I've noticed is that interest in chat GPT seems to be dropping off. You're a professor. Is that because kids are out of school?

• 2:13Speaker 01

  Yeah, I mean, there's this sort of stat that everyone's sort of talking about that the numbers are dropped. And it's any one of a number of things, right? It's entirely possible that subset of people are losing interest, right? It's not an easy system to use. And if you're using it for fun, you bounce off it pretty quick and move on. It's very possible that it's a big engine for cheating. Schools is out of session. That might be the cause of the drop. The third is there's now actual competition with chat GPT. So maybe people are using Bing or Claude or Bard. So it's hard to kind of make a big jump at this, but I don't think it matters very much because I think you're going to see some churn. Not everyone's going to find this useful right away. That's not really the point.

• 2:49Alex Kantrowitz?

  But I have been thinking about it and it's like, okay, not everybody's going to find it useful, but some people are talking about this as a revolutionary thing that's going to change everything. There's certainly notes of that in your writing in terms of the way that it's going to change education. So how do you square that with the fact that something that can be so revolutionary can lose some adoption or have the adoption tail off early on?

• 3:09Speaker 01

  Well, I think, I mean, you have something that is many things at once. AI is a general purpose technology, right? It can do many different things. It can obviously, you know, create content and, you know, cheat on tests. But general purpose technology is something that comes around once a generation. Think the internet or steam power and has huge impacts on all aspects of productivity and work. So chat GPT is just the tight, you know, the sharpest edge of GPT and general AI use across the many different domains. So if a lot of people are playing with it as a consumer, finding it boring and moving on, I don't think that's a big deal because I think the consumer use is the least interesting. I think the question is, okay, what are people adopting this stuff for real work? Are people building wrappers around this and using the API instead?

• 3:53Speaker 01

  Like, I don't think it like the actual use is going to change over time. And I think that how many people are downloading the app or playing with the website is probably less important than we have a sort of technology shock overall.

• 4:07Alex Kantrowitz?

  Okay, so let's talk about one of those actual uses, which is schoolwork. So, I mean, I visited a university. I went back to Cornell, where I went to school sometime, you know, over the past semester. And just kind of was sitting at the lunchroom, asking kids, hey, how are you using this stuff? And it really was unbelievable that every single one of them had heard of it, was using it. They were using it in ways that I didn't really anticipate. They didn't understand concepts that were taught to them in class. So they were going to the LLMs and getting them to teach them in the gaps where the professors weren't able to cover. How have you seen it actually changed the way that education is working so far? And what is the vibe among you and your counterparts about how this is going to, you know, change your jobs in terms of educating students?

• 4:51Speaker 01

  So I think you're right. I mean, everything goes to cheating first, and that is important and interesting, but in some ways, the least important and interesting, we'll deal with the cheating problem, right? There are ways of solving that. They aren't by detecting AI writing, by the way, which is impossible, but there are ways of dealing with the cheating problem. What they're, but the really profound thing is, like you said, using this as a universal tutor, using this as a way of creating, you know, of creating new ways to teach new exercises. I've made AI required in all my classes, my entrepreneurship classes, and like, worked on entrepreneurship classes, done pretty well. I think out of the intro classes that I know that people teach, people have raised like two billion in capital over the last decade or so, you know, plus, like, it's, they do a lot really well. Now they're doing even more. I actually ask people do impossible things, but you're, there's tons of different use cases.

• 5:34Speaker 01

  So explaining things, explain why I got things wrong, people building demos with it, getting feedback, you know, it's, so it's beyond just cheating, but I also think it fundamentally changes work. People are going to be cheating on essays, people will be cheating on problem assignments. All of homework is threatened, right? We're facing what I call the homework apocalypse, but I think we could build a better world afterwards.

• 5:54Alex Kantrowitz?

  Yeah, there are, there were other interesting applications that I hear that I, now that I'm thinking about it, there are law students, and I guess this might be in the cheating bucket, but it is pretty resourceful, law students who are on call in their law school class that are, you know, basically chatting with Bing to make sure that they have the answers on specific cases. So how do you deal with that as an educator?

• 6:14Speaker 01

  I saw the same thing. You put a Harvard Business School case into it. And if I type into, and I just say, tell me what I should say to sound really smart. Crack the case, it will definitely do that. So it invalidates whole ranges of how we used to do work. Right? I don't think we do assignments the same way anymore. Now, the plus side is we've known for quite a while that the way we were teaching classes was not the ideal way to do it. The ideal way to teach is actually using what's called active learning and flipped classrooms. So that's where you do your learning outside of class and you do activities and applications inside of class. So lectures are generally a waste of time for the way most people do them compared to actually doing things. So what I hope to see happen is less emphasis on homework as a way of practice.

• 6:59Speaker 01

  And more homework, more of the homework outside of class is the initial learning. And then the practice happens in class. But that is a radical change from how we used to work, how we used to teach. And people aren't ready for it, right? As you said, like, no one's ready for what's about to happen. Essays don't work the same way anymore. Problem sets don't work the same way. In-class conversations don't work the same way. We have to rebuild around these, these new technologies.

• 7:21Alex Kantrowitz?

  Right. And long-term, maybe we will, but short-term, as you mentioned, this is really going to be an issue. I'm just going to quote from your piece, "The Homework Apocalypse." So first of all, it really is astonishing to me, A, how much the internet changed the way that people do homework. So you write, "One study of 11 years of college courses found that when students did their homework in 2008, it improved test grades for 86% of them, but only helped 45% of students in 2017." And that's because half of the students were looking up homework assignments and answers on the internet in 2017. So if we expand that, doesn't that, in the short-term, at least, before we go to flip classrooms, just totally destroy everything we're trying to do in education?

• 8:04Speaker 01

  I mean, people are already cheating. There's 20,000 people, at least as of last year, before ChatGPT came out, who's in Kenya's full-time job is running essays, right? Like, this is, this has been a longstanding thing. We've just kind of ignored it. So you can't ignore it anymore. And also, a different kind of cheating is enabled. And also, what cheating is changed? Is it cheating if I ask ChatGPT? To help me come with an outline, but I don't use that outline. Is it cheating if I ask for feedback on an assignment? Is it cheating if I ask it for 10 different ways to write a sentence when I'm stuck? We have to redefine how cheating works. We have to redefine how plagiarism works. Like, this is not an easy sort of first step. So it's a shock. It happened all at once. And, you know, as you said, school districts aren't ready. I think in the long term, we emerge much stronger. In the short term, we're gonna see people do all sorts of crazy stuff.

• 8:48Speaker 01

  I mean, they already are trying to use anti-cheating tools, which don't work. But I expect people to be going back to filling out Blue Book exams and, you know, as an interim solution. I expect people to be given as homework assignments where people are scrutinized and quizzed about whether or not they answered things. People will be asked to use Google Docs and teachers will go over line by line to make sure that they are, you know, entering them at the right time because you can look at track changes. Like, there will be all kinds of crazy stuff that isn't helpful at first as a stopgap measure until we figure out how to do this right.

• 9:19Alex Kantrowitz?

  So I think we'd both agree that, like, looking things up on the internet isn't cheating when you have homework, right? And you're, I mean, assuming the professor hasn't told them, don't look stuff up on the internet for this, right? I think that's normal and that's already decreased, like the

• 9:32Speaker 01

  Well, that is cheating.

• 9:33Alex Kantrowitz?

  And you put it in the recall. That's cheating?

• 9:35Speaker 01

  Yeah, if you look up a problem set, if I assign you a problem set, I'm grading you.

• 9:38Alex Kantrowitz?

  Okay, that is a cheat. That's cheating. Yeah, for sure.

• 9:42Speaker 01

  Doing internet research isn't cheating for writing an essay, obviously, but also, you know, but that homework assignment was about looking stuff up on the internet that are exact answers, right? It was about searching, you know, Chegg or whatever to find out the right answer to a problem. And that's really become the issue. And, but you couldn't do that with essays, right? Unless you paid someone, which was kind of really cheating. But now you can, right? And the AI will generate essays for you. And by the way, it has vision, so show it your geometry problem and it'll solve it. Like, that's a pretty interesting world.

• 10:11Alex Kantrowitz?

  Are your colleagues coming to you and be like, I mean, obviously, they're looking at you as a guy that understands this. Are they coming to you and they say, I busted a student using ChatGPT in an essay because here's the word as an AI assistant, I cannot or I can. What do you think about that? Has that happened?

• 10:26Speaker 01

  I mean,

• 10:26Speaker 01

  Look, Edward and I'm not seeing much bad cheating. Like, I mean, look, I did have one person at one point during my, when I created where their assignment switched fonts partway through to the Wikipedia font, and that was a dead giveaway. So there's bad cheating out there. But I actually assigned my students to cheat in class. So as part of the way of learning what large language models operate. And by the time, what I tell them to do is fake an essay about a personal experience, reflecting back on a personal experience and applying a class concept to it. Very standard reflections, actually a very powerful tool. Applying class knowledge to a real thing is a very powerful tool. So it's a perfectly good assignment. But I asked them to fake it, and they have to use at least five different prompts to tell me what happened. And I will tell you, by the time you prompt the AI four or five times with, you know, and there's multiple techniques, we can talk about that with prompting.

• 11:12Speaker 01

  By the time you've prompted the AI four or five times, the results are really good. And the AI detector detects them, and they don't feel like, they don't say as a large language model, and they don't end with in conclusion, and they don't have the chat GPT style lists of things that go on in them, because you can make it much more human writing.

• 11:29Alex Kantrowitz?

  So go a little bit deeper into this idea behind assigning students to use AI in class. Does that, how does that get across some of the learning objectives that you have? And do you think that's going to become more prominent?

• 11:41Speaker 01

  Well, there's, there's about three different ways of using AI in class, right? So the surface stuff I gave you is that, right? The surface level stuff is you do an AI assignment, you know, focused on AI and learning AI. And that's great. I mean, but, you know, I'm a fairly advanced user of AI. I get to teach my students that, you know, there's a, and so that assignment has a reason. Then the second level of assignment is letting people use AI and taking advantage of its strengths and weaknesses. So ask them to do a report, and they use AI, and they have to critique the AI's answer, right? And that actually can be very helpful. The third level of AI is integrating AI deeply into everything you do. And that lets you do the impossible. My syllabus, I used to ask people at the beginning of my class, they had to do a class outline, where they were, you know, they would outline the business idea they had. Now, the outline assignment insists that on top of doing an outline, they have to do at least one impossible thing.

• 12:27Speaker 01

  If they can't code, they have to code. If they can't do HTML, they have to do HTML. If they can't draw, they have to do drawing. And every assignment they turn in has to be critiqued by at least three famous entrepreneurs through history, which they then use GPT to give them perspective, because getting outside perspectives matters a lot. And I expect them to use AI for feedback. And now they do tons of more work than they did before and get far, much further than they did before. So there are different levels of embracing AI for school.

• 12:53Alex Kantrowitz?

  So much of the problem with our education system seems to be, and this is speaking from educators, that it is memorize and spit back. I guess that's why the lecture homework system doesn't work very well, because people sit and lecture, they memorize what the professor said, they come to the final and they write that down, or they type it down and then spit it back. And their ability to retain what they heard is sort of evaluated. Do you think that these things, these LLMs and the being integrated into class work can actually help create more of the critical thinking and more of the inventiveness that we need to see in students and in the education system moving forward?

• 13:31Speaker 01

  So I'm gonna actually push back a little bit. I actually think, so I think we need to learn more facts. And that sounds horrible, but it turns out-

• 13:40Alex Kantrowitz?

  I'm shocked. Okay, go ahead.

• 13:42Speaker 01

  All right. So here's why, okay? Because in order to be able to beat large language models, or at least to be able to work with them, and who knows how good they're gonna get, really, we could talk more about that, you need to be an expert. Like, if you're in the 50th percentile, that's not good enough. You need to be an expert in something to be able to monitor the LM, use it well, be a cyborg that it helps you out with. And we actually know how to build expertise. And unfortunately, there is no shortcut. Like, people want to teach critical thinking as if that's some sort of magical shortcut over what we're doing. It's not. The way you build expertise is unfortunately horribly grinding. It starts with having a deep basis of facts. You need facts. And then you need to start to see the connections between those facts, which let you move the facts from your short-term memory to your, and your working memory to your long-term memory. Then

• 14:25Speaker 01

  you need deliberate practice, right? You probably heard the 10,000-hour thing. That's not right. It's not 10,000 hours. But you do need vast amounts of grinding deliberate practice that pushes you really hard to apply those facts in different ways. That's how you become a chess master, right? You study a whole bunch of moves and games, and then you play enough with enough against increasingly hard opponents and learn from your mistakes until you get better. So you get good at tennis. That's how you get good at being a carpenter. That's how you get good at being a professor. So we can't skip the facts phase, because that's how humans build expertise. Regurgitating and spinning back facts, so you start to see the connections between, okay, you know, and you're making tons of them here in the room, right? Between all of these different topics, you're an expert in technology. That requires you to know a lot of facts about technology that you can pull together and pull, you know, and throw back out in useful ways.

• 15:08Speaker 01

  Being an interviewer, right, requires knowing a lot of facts. So we can't skip that phase. We can't outsource that. And so in some ways, the AI requires us to actually double down on the basic knowledge, because we can only build the advanced knowledge from the basic knowledge.

• 15:23Alex Kantrowitz?

  Interesting. So then what do you think the future of of education is going to look like?

• 15:27Speaker 01

  I think the what makes building expertise hard is that process of deliberate practice is hard. It requires coaching. It requires instruction is mixed in like you have to get lessons. It has to increase the difficulty level on a constant basis. It has to be as engaging as possible. You have to be utterly focused on it. These are things that's really hard to do in a classroom, really hard to do. So we sort of outsource it to a bunch of homework and a lot of grades of doing the same stuff over and over again. And then you start to work as a junior producer and, you know, in in podcasting and you work your way out. You do prep work and then you work your way up to mid level. Then eventually you get good at like we we depended all of these systems to do it. AI might be able to let us skip these things and make expertise easier. A tutor, and by the way, if you look at the Khan Academy's Camigo, you can start to see the direction here.

• 16:13Speaker 01

  A tutor that actually knows where you need to go and makes sure the work is always just hard enough without getting frustrating. That meets you at your level that gives you feedback and helps you through the process of deliberate practice. That's an exciting potential future, right? So I think we can get better at building human knowledge expertise. We know how to do it. We just the the most effective way to teach is what's called direct instruction and one on one tutoring. Right? Like literally the idea of a one on one tutor, people who get one on one tutoring acted 98th percentile of a class. Like, it's amazing, but one on one tutoring is incredibly expensive. AI can do that. So I think we're looking at a world that unlocks that.

• 16:47Alex Kantrowitz?

  So do you think that this is going to turn to AIs?

• 16:49Speaker 01

  Well, I think part of the teacher will turn to AI. I think the tutoring instruction will be AI based. I think you're still going to use classrooms because we need to work with each other. We need to apply our knowledge. We need to get feedback from what mistakes other people are making. So that's that flipped classroom idea in class. You'll be doing exercises, activities that will be AI driven, but the teacher will still play a vital role. And they'll be able to provide guidance, support, all the other things you need in a classroom. And outside of class, the instruction that used to be delivered through textbooks and through lectures is going to be delivered through AI.

• 17:18Alex Kantrowitz?

  Last question about education. Look, the education system, you know this well, moves extremely slowly. It's not a fast moving system and everybody inside it, not everyone, but almost everyone seems to be extremely resistant to change. So I'm hearing that, you know, these ideas from you and they they seem to make sense. But then if you ask me like realistically, on what time horizon this stuff is going to happen? 20 years? I mean, what do you think the actual speed of change is realistic for us to expect?

• 17:46Speaker 01

  Well, that goes back to that dropout GPT downloads that you saw before, right? Systems are slow, right? And people overestimate how quickly change happens, but systems resist change because it's not just one thing. You can't just make one thing different because the rest of the system needs to operate too. The same thing will happen with jobs, by the way. So the education system is slow. It's slow for a reason because there's a lot of interlocking pieces. Like, you can't just advance things as one teacher because you need to fit into a class system and you need to be able to fit into getting creative. The kids need to get able to get to college and you need to know what they're doing and hold them accountable. Like, there's reasons for this grinding slowness. It's not just bureaucracy, right? It's that too, but it's all these interlocking systems in place.

• 18:29Speaker 01

  And it has to fit into teacher tenure and, you know, and what parents expect. There's lots of pieces, curriculum. So those are going to take a very long time to change. I think that you'd be surprised at how quickly people can move when it's radical. I mean, schools moved imperfectly online with two days' notice. It's insane. Like, I think COVID showed that we can adapt quickly than we need to. But I also think it overestimates the need for systemic change and how much systemic change needs to be just in US school systems, right? The exciting thing about something like Bing is it's available in 169 countries around the world. If you are in Botswana right now, you, you know, you have access to the same, same GPT-4 that you could get access to as, as, you know,

• 19:11Speaker 01

  BlackRock. Like, there's no difference. And if I give you the right two paragraphs or prompts, you can learn something new. So I think that we're underestimating the global change on this. I think you're right. Things aren't going to change overnight, but I think there is a capacity to do it.

• 19:25Alex Kantrowitz?

  Does that mean all these predictions? And so let's shift to the workplace now. I mean, it does take time to do this and workplaces are also slow to change, even though they might be a little bit more nimble than, you know, educational institutions or government. Do you think that because it takes this amount of time to adapt and change, that all these rumors, all these, you know, this panic, oh, AI is going to take my job, might be a little bit overblown right now?

• 19:48Speaker 01

  So I don't, I think that's part of the reason. So to talk about AI taking my job, we need to get a little bit academic, which is we don't like to think about jobs in academia. I mean, we do, obviously, we have jobs.

• 19:59Alex Kantrowitz?

  Tasks.

• 20:00Speaker 01

  Right. We think about the task, the bundle of work that you do, and we think about the systems that your work is in, right? And so we could talk more about tasks a second, and it seems like you want to have, you know, we'll talk about that. But it does matter, right, that you're part of a system, and that does kind of, but on the other hand, task adoption is very quickly, you know, you can easily change a task. And because individuals are incentivized to do action, and because the productivity performance impacts are so potentially huge, like 30 to 80% performance improvements in some tasks, that creates a huge incentive for companies to shift quickly. So I think we will see faster movement. I'm already seeing companies do faster things. Part of it is that the APIs that you use to access GPT-4 are kind of so easy to do, that you can kind of roll

• 20:45Speaker 01

  your own solution remarkably quickly.

• 20:47Alex Kantrowitz?

  Yeah, one of the things that really surprised me in reading your writing is that it seems like this change is actually being driven more by the individuals than the companies themselves. And there are people, you did a poll asking, if you use AI at work, do you tell anybody? And 50% of people were like, no. So, so what do you make of this idea that this is being driven by, you know, the bottom up versus the top down? In fact, many companies are banning AI, which is quite interesting.

• 21:14Speaker 01

  And if you talk to people, the companies that banned AI, they're all bringing their phones to work and doing all the work on their phones, and then emailing themselves, right? I mean, you know, let's go back to cheating, right? Like, in school, cheating is bad. Shortcuts are bad. At work, if you can figure out a way to do all your work in 10% of the time, you are going to do that. Humans are extraordinarily built to respond to economic incentives. So if I set it up so that you could do more work and less time and less effort, and you can outsource boring tasks, you will do that very quickly. And so we're seeing people experiment all the time. Experimenting with companies is expensive. Experimenting with your own jobs is super cheap, right? You try chat, it doesn't work, try again, doesn't work, maybe you give up. You try, it starts to be interesting, you keep going. Trial and error is cheap for your own tasks, expensive for other people's.

• 21:58Speaker 01

  So it's very easy to adopt at the individual level. The benefits are huge individually, as long as no one knows you're using it. And I talk to people all the time who outsource 90% of the work, and just don't tell anyone about it.

• 22:11Alex Kantrowitz?

  What happens when people find out?

• 22:13Speaker 01

  Well, so I think that's the real question. If you're a smart company, you're going to be working really hard to make sure the answer to that is they get rich and rewarded, right? Because what I want to do is find out what everybody's been experimenting on. All these companies are used to doing experiments, to doing, to doing change from the top down. As you said, they're used to having a boss do things. They have an innovation group, they hire McKinsey. That's not the way that this works. There's no reason McKinsey knows more about how to automate some mid-level manager's job than they do. So I need to get that middle of manager willing to talk about what they're doing, to share that information with me. And to do that, I need to make sure that middle manager doesn't feel like they're going to get fired. I need to make sure that middle level manager knows they'll get rewarded, that I won't fire other people because they've, they've ratted out this capability.

• 22:57Speaker 01

  I want to make sure that people feel incentivized to be part of the team. So this is where having a hostile work environment where you hate the boss but are grinding away at work, I'm never going to share anything. If I'm for a company where I'm all in on that company, maybe that's a little bit different.

• 23:09Alex Kantrowitz?

  Right. And so it sort of goes to this question of like, all right, so what's going to happen to my job? It's like, actually, if you can take this, these tools and make yourself more productive, you become more valuable to the company. And I think that like one of the companies really never talk about like, ah, like, you know, we wish we did the same thing with less people. They always talk about, we wish we did more, we just have a labor constraint or a cost constraint. And this could potentially flip that. Is that how you see it?

• 23:34Speaker 01

  Well, I mean, there's lots of different constraints, right? Like what you might be, lots of companies also fire people, right? Because I mean, let's be realistic, like if you get a productivity gain, that's what people tend to do, right? Is like, or they view that there's overhead, they slash it. But if you could figure out ways to turn that extra overhang of people productivity into positive growth, your company will be much more successful, right? And, you know, if you're firing people left and right, when they, they become more efficient, they're going to stop becoming more efficient because they know what happens. So it's, you have to embrace the idea that there might be some short-term inefficiency to get this long-term gain.

• 24:12Alex Kantrowitz?

  But it does tweak the employment contract just a bit, don't you think? Because you get paid to do a job. The, the contract is basically you're doing this job. This is what I think, you know, someone on your level is able to do. Then you find that way to be, you know, much more efficient. And then all of a sudden it's like, you know, it, it really does, you know, say, basically as the technology has shifted everything there. And it's no longer like, it's, it's almost a, I don't want to say dishonest, but it's a completely different way of approaching the job. If someone can do it like, shouldn't, shouldn't, I mean, yeah, okay. You're shaking your head.

• 24:44Speaker 01

  No, I'm agreeing with you actually. I, I think, I think, I, I, I think that you're right, that this is completely shattering. But every, I mean, this is GPT, a general purpose technology breaks things, right? Like this is like, this is like, old systems are going to get broken everywhere and we have to reconsider them. What does it mean to work with an AI? Like what, how much am I a free agent working with an AI to help a company? How much does my employment contract matter? If I get all my work done in an hour and a half, do I get the extra seven and a half hours on, you know, seven hours on my own, like, or six and a half hours on my own? Or if you're a banker, 16 hours on my own? Like what, what happens, right? We don't know. What, what happens when like, a lot of our systems at work are built around human limitations.

• 25:27Speaker 01

  They're built around enforcing limits on human, like keeping everyone operating at the same pace, making sure your gear in the machine. What happens when that breaks? We don't know. Like this is, you know, it's very funny. People talk about the singularity, right? Is this thing that is like, the AI gets smarter and that's a murders us all. And I'm definitely think, you know, that seems worth worrying about, but it's also the original meaning is it's like a mathematical point that we can't predict what happens afterwards. That's already going to happen at work. I don't know a hundred percent what the future of work is going to look like. I don't know a hundred percent what the future education looks like, because we're just assuming that everything stays the way it is, right? What happens is work gets more, more easy to automate with AI. What kinds of categories of work change? Like, we're barely scratching the surface.

• 26:07Alex Kantrowitz?

  Let's go through a scenario that sort of hems to what you've been discussing. So I am a manager. Somebody comes to me and says, listen, I read Ethan Substack and I'm confessing, I'm using AI. You shouldn't fire me. Here's the article. This is why, by the way, I'm doing a hundred percent of my work in 10% of the time. So does the, what is the man, what is the next step for the manager there? Do they say, we actually need you to do 10 times the work in a hundred percent of the time? Or do they say, yep, that's good. Teach us how to do it and we're going to have basically give everybody, put everybody on the two day work week.

• 26:41Speaker 01

  So there's a whole bunch of options, right? I mean, I'm serious. Like, so one option is something we already know works really well, which is let's job craft this. People are more motivated and do better jobs when they go through job crafting, which means working together with management to figure out what their job is. So you could say, hey, what outsourced stuff have you done? The 10% that's left, do you find that engaging? Is that what you really want to do? Because a lot of the early state work of AI is about freeing us from judgery, which is kind of the good, it's ominous in the longer term. In the short term, it looks pretty good, right? Like, my job is a bundle of tasks that includes many things like flit expense reports. If AI does that, yes, like, that's the best thing in the universe. I hate expense reports, right? And so you could ask this person, what have you outsourced? How do we get you to do more of that 10%? Maybe we're giving you a raise because you're doing 10 times the work now.

• 27:23Speaker 01

  Maybe we're giving you a million dollar bonus because we can spread that idea across the entire company. We're saving that much. Extravagant rewards, rebalancing work towards what people want to do, right? Thinking about what systems we can use to support that person. It becomes part of a conversation.

• 27:39Alex Kantrowitz?

  Exactly. It's interesting. So I know, I know we kind of danced around this question earlier about, about jobs. But I mean, like, so we don't know exactly what's going to happen there, but there, there has to be some sort of impact, don't you think?

• 27:53Speaker 01

  I mean, there's going to be, right? I mean, and again, general purpose technology, it's going to be a huge,

• 28:01Speaker 01

  like a huge generalized impact, right? It's going to affect everything in all kinds of different ways. So some jobs are going to change and disappear, right? And like, when the telephone system was moved to a digital telephone system, it resulted in a giant wave of change. A lot of people lost their jobs. A lot of things ended up happening very quickly. But most times when technology change happens, people get better jobs and higher paying jobs and the nature of work changes. But we tend to look at the back on those historically as not in the moment, which is what we're going through now. So there's going to be disruption everywhere in the nature of jobs.

• 28:36Alex Kantrowitz?

  Yeah, and it's also, there's something that you hinted at that I think we should expand upon, which is that taking away the drudgery is really nice in the short term, but ominous in the long term. I mean, after reading your work, I didn't really fully think about how this could potentially change like the nature of meaning that people get out of their work. And you had this one line that's just amazing. You know, you're talking about, I think, letters of recommendations. And you start to say that, you know, we can create documents mostly with AI that gets sent to AI powered inboxes where the recipients most respond mostly with AI. Even worse, we still create the reports by hand, but realize that no human is actually reading them. We can view the destruction of busy work as freeing. And we do not yet have

• 29:20Alex Kantrowitz?

  to start our, oh, this is about setting our time on fire as a signal. But like, go back, let's, let's go back to this idea of the AI's community. I mean, this is one of these things where like, you know, it seems like maybe one day, instead of us getting on the podcast, my AI will interview your AI and it will be listened to by AI users who will summarize it for people. What is that? How does that change our society? How does that change work?

• 29:44Speaker 01

  I mean, we have, we have to reconstruct meaning, right? Like, this is, this is, this is the biggest crisis. I think that everything else is secondary. I am sending a report. As a mill manager, the report that I would write about my employees' work, even if no one reads it, was valuable because it means that I checked the work, right? And that my boss looked at the, looked at this, you know, and saw that I was doing this. And it's like, okay, the systems of monitoring are in place. Now I hit a button in Google Docs, it creates a automated report that looks plausible. I send it over Gmail with a fake message to my boss, who then hits auto reply and gets a message back. We've just taken the intellectual meaning, but also the reason for doing this work has disappeared entirely. Like, we have to reconstruct this.

• 30:28Speaker 01

  This means our work systems were built, the reason we have org charts is because in the 1850s, somebody wanted to make sure we had org charts, or early robber bar, you know, railroad barons wanted to build a structure that would let them control vast geographic train networks from the top. So they built systems that look like train networks that do that. Henry Ford realized that people that he could hire, if he hired low paid employees, he couldn't monitor and control them and what they were doing. But if they did one task over and over again, he could keep an eye on it. So thus, assembly lines. Agile development is all about the idea of that we could track bugs and changes at the end, but we still need stand up meetings to coordinate work. All of those things are going to change as a result of AI being able to coordinate to interact, to change, you know, and that's going to change work.

• 31:13Speaker 01

  The companies that figure this out first are going to win.

• 31:16Alex Kantrowitz?

  It's almost a chicken and egg thing, right? Where it's like, okay, the AI is changing, changing work, but it's also commentary on how much of our work is meaningless, don't you think? I mean, you, in your, in your essay, write about how, and this is what I was getting at with setting the time on fire, which is a concept that you've talked about, but basically people are asked to like, write letters of recommendation, not because they're actually going to read it, but just to kind of show that the professor would put the time in to recommend the student, and it's a signal. And it just seems that if you can automate this and have AIs talking to AI, like so much of the system of work today is kind of garbage, don't you think?

• 31:53Speaker 01

  Yeah, I mean, it made, I mean, look, work is broken in lots of ways. It literally, as a management scholar, one of the dominant methods of understanding how organizations work is called the garbage pail method, which is just like stuff happens, right? Like people meet each other, it all gets tossed together and stuff happens. AI can help a lot, right? It can help free us from drudgery from bureaucracy, but there's a lot of people who work in bureaucracy. There's a lot of people who, you know, the reasons part of those systems exist and part of them just grew that way. And, you know, and we have to reconstruct all of how that works. As you were saying, the problem with the letter of recommendation is not just that I could push a button and write it, but that the letter of recommendation that I push the button and write is going to be better than the letter of recommendation I spent an hour writing, because the AI will read all the documentation, all the material and be able to produce something that's much better.

• 32:35Speaker 01

  Like, so the problem is I turn in the old fashioned school, you know, like properly done, morally correct letter of recommendation, and I'm actually hurting my student's chance of getting the job compared to pushing the button. And we don't have job, things built around that. I mean, do I just send the prompt I would have sent? Like where I say, I would have told the AI, I like this person, they're a good job, give them a good recommendation. Or what do I do? We don't know, we don't, that's, but that's one minor system out of so many that's going to be broken.

• 33:05Alex Kantrowitz?

  It does seem, so there's been this question of like whether AI will automate all of our work. And I can keep coming back to it, but it does, it does seem at a certain point, like the further we get into this, we could effectively just like set it and forget it with our economy. And maybe I'm kind of delusional, but, but it does seem like we'll still make work for ourselves. Like, why don't we just like have the robots kind of take care of our basic needs and just kind of live?

• 33:29Speaker 01

  Well, I mean, but that, of course, is the bet, right? We do, like, we're coming back to the billion dollar question, the trillion dollar question, actually, I guess, or what are the companies, like seven trillion, 12 trillion? I don't remember off the top of my head. That's the question, right? The question is, first of all, how good does this get? Right? Like, there's too much obsession, not that it's wrong, about like whether we build artificial general intelligence and a machine god, totally get it, I'm glad people are worried about it. They absolutely should be, government should be worried about it. We should all have concerns about, you know, whether the machine god will save or kill us. But like, there's a lot of steps between there and now. And the real salient question for technology is, is this as good as it's going to get? I think the answer is probably no. Will change be linear or exponential in the future? That we don't know.

• 34:11Speaker 01

  If change is exponential, then maybe the robots take caring of everything is something that happens in the next five or 10 years. If not, if it just keeps getting better, we have a gradual squeeze. If you're not in the top 10% of workers, you know, in your task, you're probably, AI, you know, AI will replace you, then top 8%, top 7%. So I think the question about what happens with our leisure time, what happens with work, depends like right now, if AI were to pause where it is today, I think it would be 10 years of us absorbing the impact, but I think we'd be fine. Right? Like, there, it clearly needs oversight, it does not, there are very few categories of workers it completely replaces. It tends to do drudge work, right? I think we'd be fine. Now, what happens if it gets 10 times better? I don't know.

• 34:54Speaker 01

  Then, then we start to think about how work gets automated and changed in much more profound ways.

• 34:59Alex Kantrowitz?

  We have abundant resources on this planet and in the modern world that we've developed, but we still fight over them. Like, it just seems so ridiculous in some ways that we're getting this technology and we're still gonna fight over them.

• 35:11Speaker 01

  I mean, it's gonna help. I mean, the question is whether it makes us fight worse, you know, is the, is the open question.

• 35:17Alex Kantrowitz?

  Right.

• 35:17Speaker 01

  Right. I mean, look, it is hard. It's hard. When new technologies come around, everyone gets utopian and cataclysmic at the same time, right? And so this is the like, but this is something quantifiably different. The question is just whether it keeps going, right? Where we have it right now, it is a complete change in technology and it's very exciting and it will be a huge revolution on par with the internet or maybe steam power.

• 35:43Speaker 01

  But if it gets 20 times better, then it starts to get really weird, right? Because then we actually have, you know, what, and if it starts to be able to outperform humans at every task, that starts to get very strange. And we don't know what happens then. We don't know what the limits are. We don't know the outcomes. And that's where like the entire sort of system modifies.

• 36:01Alex Kantrowitz?

  Right. Because if you talk about like, all right, the technology gets good enough that you only need the top 10% of workers and 90% you don't need. Well, you still need workers and you're not at like maximum production. And that drives inequality in a society and destabilizes it. That's what you mean by the system shifting.

• 36:16Speaker 01

  Potentially, right? I mean, at the very least, you need government intervention to make things happen. But, you know, again, if we look at the history of work, change will be slower than we think generally. And people generally tend to get better jobs. We just don't know if this is the end of those rules, right? The historical rule has been technological change, every other time it's happened, has resulted in generally an increase in, you know, in GDP, an increase in quality of work. There's been some exceptions. Will that keep happening is the big question.

• 36:49Alex Kantrowitz?

  Ethan Malik is here with us. He is a Wharton professor and he writes the One Useful Thing newsletter on Substack. On the back end of this break, we're going to talk about a few new models and a bit of a lightning round and take some questions that popped up on Twitter and threads over the past couple of days. All right, back right after this. This episode is brought to you by Avpoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data. And most teams have no single way to see it, secure it, and prove it's under control. That's exactly what Avpoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate.

• 37:34Alex Kantrowitz?

  Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. Avpoint, the unifying trust layer for AI. Head to avpoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door.

• 38:19Alex Kantrowitz?

  The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a wide-glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwell.com/bigtechnology, fill in the quick form, and request your free risk assessment. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwell.com/bigtechnology. Learn how to stop online threats before they become real-world attacks. The link's in the show notes.

• 39:10Alex Kantrowitz?

  And we're back here with Ethan Malik. He's a professor at the Wharton School of University of Pennsylvania. He writes the great One Useful Thing on Substack. It's a great newsletter. We've been talking a lot about his most recent pieces here. Let's do a quick lightning round. So Ethan, you've been experimenting with some of the bots that have been coming out from the non-open AI, non-Microsoft groups. So what is your, so I'm going to ask you about your experience with two of them, first of all. So what is your experience with Claude from Anthropic? And how do you find it different from OpenAI?

• 39:41Speaker 01

  So Claude was developed by people at OpenAI, and they said they left because they were worried about OpenAI's risks it was taking with its AI. So it's supposed to be a friendlier and less harmful AI. They just released Claude version two, which is somewhere between GPT 3.5 and GPT 4. So it's a really good model, probably the second best model out there right now. And one of its really great capabilities that it has in the short term over anyone else is it has 100,000 token, you know, context window, which basically means it can hold a book and a short book in memory. And that means you can upload PDFs to it, and it's very good at working with documents. So it's very good at summarizing, at annotating, at combining documents. I find it very useful for those purposes. Because

• 40:24Speaker 01

  it's safer, it also is a little bit preachy, so it can be a little bit more annoying to work with. But it is, it's a very powerful model. And if you're working with documents, I'd strongly recommend trying it.

• 40:34Alex Kantrowitz?

  Yeah, you have this chart of breaking down all the different models. And I think for Claude, it's like, good, but a bit too preachy.

• 40:39Speaker 01

  Exactly.

• 40:40Alex Kantrowitz?

  It can be that way.

• 40:41Speaker 01

  Well, I don't really mind some of the others, because without guardrails, AI gets really bad really quickly. But Claude can get a little over the top.

• 40:50Alex Kantrowitz?

  You mentioned that Anthropic researchers are dreading what they're building, which is, which is really strange, right? It's like they're going ahead. This is covered in the New York Times article, but they're going ahead and they're really freaking out about the nature of AI. And this is a question I ask sometimes, but it's just so unbelievable to me that some of the people with the biggest concerns about AI are like, man, this stuff can really destroy society. Let's build the next new model. What's happening there?

• 41:14Speaker 01

  I've got, there are, there are three theories, right? Theory number one is it's all cynicism. I don't think that's the case. I mean, some people are being cynical and marketing it, right? Or paying, you know, paying lips, you know, like, but, but that's one option. The second is that they are genuinely worried, but not that worried. Right? So they're not like, they have to say that they're, you know, have, believe in possible doom, but they really think it's going to be okay, which, you know, a lot of researchers do. And a lot of researchers are also very worried. The third option is to take what they're saying at face value, which is that they actually think that they are building, that if they don't build a safe version, people will build a dangerous version. That's always a little bit weird to start an arm's race that like, if we don't burn down the village first, someone else will burn it down. It's sort of a, it's sort of a disturbing viewpoint, but that, that may be a serious kind of approach, right?

• 41:59Speaker 01

  But they may think that they could shape the future that way. So it's some combination of those things, right? I, I do believe, talking to people at AI Labs, many of them are true believers. They really believe they're building, uh, you know, super intelligence. Whether or not they're right or wrong, I don't think anyone really knows.

• 42:15Alex Kantrowitz?

  What do you think about this Pi bot? It's Reid Hoffman's bot, right? It's, uh, supposed to be a very friendly bot. I started talking to it a little bit. It can even like play like, it gives you an option to be like a therapy role, which was interesting. I was like, I'm about to convince, confess my deep, dark secrets. And it's like, your secrets are safe with me. What's up with Pi?

• 42:35Speaker 01

  Pi has gone from driving me absolutely insane to intriguing me, right? So like, it is, it vomits emojis. It tries to be a friend really hard. It will not do work to save your life, to save its life, which is something I push really hard on, you know, and, but on the other hand, it is pretty good at chat. Like it's pretty good at adapting to you in the way AIs are pretty good to keep a chat going. If you have not tried the app, I strongly recommend it because there's an ability to have, get on a call with it, which is near real time conversation back and forth, which is pretty nuts. So I admire the attempt to create a AI that doesn't do the thing the other chat bots do. Is there a market for it? Is it an open question? Is it like, is this good or bad for the world? Like high engagement bots are kind of a high risk also in their own weird way.

• 43:18Speaker 01

  So like, it can be friendly, but it takes away time. But I've been increasingly impressed by its ability to, to be interesting to talk to at the very minimum.

• 43:26Alex Kantrowitz?

  Yeah, it's definitely fun to speak with. Do you think we need all these? I mean, there's so many. There's ChatGPT, there's Claude, Bing Bot, Pi, Character AI. Like, this stuff is going to consolidate eventually. It has to, right?

• 43:39Speaker 01

  I don't know. I don't know. I mean, I think the models could get specialized too, right? I mean, I don't, I mean, if we're talking about, you know, electricity providers or, you know, railway systems, like, having more than, you know, I don't know how much consolidation there is. And I actually don't think for large language models, there aren't that many contenders, right? Like, and the question, of course, is, is there a flywheel? Once you've built a large language model, right, does that LLM, like, give you an advantage, right? Does that help you code the next model? And so, there's only a few companies that have significantly big, large language models so far, right? And so, OpenAI, which has Microsoft stuff built on top of it, right? There is Claude and Anthropic. There is, there is

• 44:20Speaker 01

  the company behind Pi, and there's Google with its Bard, right? Those are the models. Like, you know, Elon Musk's announced a new model he's building, but it's going to be years of training to get that thing up and running. So, the question is, are there really going to be that many? And will they specialize the way Pi is the conversation bot, AI is the document, you know, Claude is the document bot, and GPT is your general purpose tool?

• 44:46Alex Kantrowitz?

  Right. Yeah, that's interesting. That's an interesting possibility. Do you think that prompt engineering is going to be a new job? I think that's kind of a ridiculous idea. I think everybody's going to learn how to do this. What's your perspective?

• 44:58Speaker 01

  I agree with you. And not only is everyone going to learn to do it, it's just going to be, do it for you. If you already use something like, I mean, go back to Pi, you never prompt it once. It keeps a conversation going. If you ask GPT what I should ask, it will tell you what to ask. If you use AI, you know, art system like Mid Journey, it went from like, when I did Mid Journey prompts, you know, a year ago, those prompts would be these elaborate invocation spells, Kodak Chrome, Art Station 7, like, we just didn't know what we were doing. Throw everything into it to get a good picture. And now you just say, show me the thing, and it shows you the thing. Like, I think that prompting is going to get easier. I don't know anyone who works in an AI lab who doesn't think that prompting is just going to get easier to the point where prompt engineering is kind of silly.

• 45:41Alex Kantrowitz?

  Yeah, I agree. What do the US AI efforts look like in comparison to China as some as an academic who studies this? I'm curious what you think.

• 45:49Speaker 01

  I'm not an expert on Chinese AI policy. The US is typical US stuff, which is laissez faire while a thousand different court cases resolve themselves and there are Senate hearings like it is going to be a slow process to figure out what all this means. Japan, for example, has said pre-training. The training data is can it be protected by copyright, which is interesting. The US has to take a position on this. China, the early evidence seems to be a desire to regulate these systems. But on the other hand, you know, national militaries are probably spinning up these. So I think regulatory systems are going to be really interesting. I think Europe is really being very careful and we'll see what happens with that. I mean, I think there's a lot of change happening all at once.

• 46:28Alex Kantrowitz?

  Code interpreter is just something that you've picked up on and plug different plugins that can help you make sense of data. What's your perspective on those?

• 46:37Speaker 01

  I think that that's some of the most exciting stuff out there, right?

• 46:39Alex Kantrowitz?

  Yeah, don't worry about it. Yeah.

• 46:41Speaker 01

  So code interpreter lets an AI, let's GPT, both take in data, give you data to download and also run its own Python. It gives it a Jupiter notebook essentially to do its own work in. And it turns out increasing evidence suggests that when you give AI tools, it becomes much more capable. So given the ability to use tools, it first of all solves all the problems that have made AI really annoying to work with, right? So if you've ever tried to use AI to work with language, like that is a hard thing to do, right? It doesn't work. It doesn't understand sentences and paragraphs the way we do, right? If you ask it to count the number of words in a sentence, it can't do that. But code interpreter will write a little program to do that. It'll solve math problems that it couldn't solve

• 47:23Speaker 01

  by writing a little program. And it starts to really complex stuff like it will you throw data at it and it will apply theory and it will I give it a paper. I give it, you know, I did a little experiment where I gave it the NBA playoff data and had it find interesting stuff. And it found all these cool hypotheses and then graphed it. And then I threw in some work by by Tufti, who's a famous graphic design infographic design person and said, apply these skills to make the graph better. And it did. Like there's a really powerful democratization of coding and analytics that is already starting to happen.

• 47:56Alex Kantrowitz?

  Do you have to know how to code to use them?

• 47:58Speaker 01

  Yeah, I don't. I can't code in Python to save my life, but I do know stats. So I can look at the statistical outcomes. But no, I don't need to know code to do it.

• 48:07Alex Kantrowitz?

  We got some questions from social media. So one person asked, how is accuracy necessary? In terms of like, let me just read the actual question. Taka, ask him about the necessity of accuracy, whether it's a core need for enterprise use cases for LLMs. So should tools like Code Interpreter, here we go again, give companies greater confidence that LLMs won't hallucinate as often and provide reliably accurate outputs?

• 48:33Speaker 01

  So this idea of hallucination is a real problem, right?

• 48:37Alex Kantrowitz?

  Yeah.

• 48:38Speaker 01

  I would argue that it's probably not as big a problem as people think. So first of all, hallucination rates are dropping. So we don't have a lot of comparative data, but there was a really cool study where they gave the variety of AIs, the neuroscience board qualifying exam, neurosurgery board qualifying exam. And unsurprisingly, GPT pass the flying colors, but they tracked the hallucination rates. Bard hallucinated 44% of the answers, GPT 3.5, ChatGPT 22%, and GPT-4 only 2%. So hallucination rates do seem to be dropping. And then when you attach to code interpreter or data or PDF, it drops further. But hallucination is a genuine problem, which is why right now there's huge advantage in that expertise thing we talked about. If I'm an expert, I could check over the results and fix problems.

• 49:21Speaker 01

  And I do need to do that with code interpreter. So I think that, you know, that it's less of an issue. It favors experts. And also, there's a lot of jobs where like accuracy is the number one thing. If you're doing marketing writing, it's not that big a deal. Even if you're doing customer service, a lot of the interaction, if there's small errors, they don't make as much of a difference as getting the big things right. So the other question is how accurate you need to be. For some applications, AI is totally out because it hallucinates. But for a lot more than you think, it works.

• 49:46Alex Kantrowitz?

  There's some concern that people are, we're going to develop brain chips with AI inside them and attach them to our brains. And we could like have our data sucked out and stuff like that. What do you think about that?

• 49:57Speaker 01

  That feels that in a world where we're living in science fiction, that still feels too science fictional right now. So I don't think involuntary brain chips are in the cards and physical worlds and biology are much harder than software. So I think let's not get ahead of ourselves too much, I think.

• 50:14Alex Kantrowitz?

  Would you get a brain chip if you could?

• 50:16Speaker 01

  No, I mean, no one's getting generation one brain chip. I mean, you know, and also like, you know, I think that, and I think when you look at what the actual brain development, you know, the chip stuff is, it's not telepathically communicating with AI. I think we need to be really careful about, you know, even in a world where a lot of hype is coming true, there's still hucksterism going on. So there is no telepathic brain chip in the near future. We don't know how the, you know, we don't know how LLMs work. We don't know how humans' brains work either. Like it's, we know how they work, LLMs work technically, but we don't know all the details of, you know, why a particular decision's made. I think we're going to be a long ways off from connecting AI directly to our brains.

• 50:49Alex Kantrowitz?

  Yep. Are there models outside of LLMs that you're looking at or interested in?

• 50:53Speaker 01

  I mean, so LLMs underlie a bunch of other things, right? So large language model and the transformer technology that powers it is what is powering art-based AI dialogue and, you know, podcast creating AIs, you know, all of these things. So like the large language model technology is the transformer technology and attention mechanism that's at the heart of, of, of this technology are what's driving all of these sets of tech changes all at once.

• 51:20Alex Kantrowitz?

  Okay. Last question for you. What are you looking forward to in the fall when students come back and how are things going to be different in terms of the way that you teach? Cause you basically had to deal with this mid, midstream this year. What does next year look like?

• 51:32Speaker 01

  I'm excited. I mean, I think like it's great to take the, like it has allowed me to reimagine how teaching works in really exciting ways that I think will benefit my students and make things interesting. But that's for me. I think a lot of other people are terrified and or putting their heads in the sand or they try AI better. They've heard of AI. I mean, I hear talk to each other all the time. We've heard of it, but I haven't really tried it. Like, I think that we're in for sort of a mess.

• 51:55Alex Kantrowitz?

  Yeah. Okay. I can't wait to watch. I hope we can keep in touch. Ethan Mollick, thanks so much for joining.

• 51:59Speaker 01

  Thanks for having me.

• 52:00Alex Kantrowitz?

  Thanks for being here. Thank you, everybody for listening. Thank you, Nate Watney for handling the audio, LinkedIn for having me as part of your podcast network and all you listeners. If you enjoyed, please hit five stars on Spotify or podcasts. First time here, if you hit subscribe, that would be awesome. Thanks again to Ethan. I've been looking forward to this conversation for a long time and it delivered all the way through. So really appreciate you being here. All right. We'll see you next time on Big Technology Podcast.

• 52:30Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.

• 52:59Speaker 03

  In this life, you can't win.

• 53:02Speaker 04

  When the world gets you down.

• 53:03Speaker 03

  No!

• 53:03Speaker 04

  Escape to Pluto TV.

• 53:05Speaker 03

  This is objectively fantastic.

• 53:07Speaker 04

  Relax into the movies and shows you already know and love, all for free.

• 53:11Speaker 03

  Schwing!

• 53:11Speaker 04

  Even our news is uplifting.

• 53:13Alex Kantrowitz?

  You stay classy, San Diego.

• 53:15Speaker 04

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 53:20Speaker 03

  We're not worthy!

• 53:22Speaker 04

  The best things on TV really are free.

• 53:24Speaker 03

  You just have to know where to look.

• 53:26Speaker 04

  Escape to Pluto TV. Stream now, pay never.