When AI is your personal tutor with Sal Khan of Khan Academy

The COVID-19 pandemic changed education forever. But Sal Khan says an even bigger educational revolution is just around the corner … This week on How I Built This Lab, Sal returns to the show to talk about a new learning platform he’s building at Khan Academy. It’s called Khanmigo, and it uses the same generative AI technology behind OpenAI’s world-changing ChatGPT to help students with their schoolwork. The technology isn’t without its risks, but Sal thinks Khanmigo could act as a personal tutor for every student and a teaching assistant for every educator - reshaping the classroom for good. This episode was produced by Alex Cheng and edited by John Isabella, with music by Ramtin Arablouei. Our audio engineer was Neal Rauch. You can follow HIBT on Twitter https://twitter.com/HowIBuiltThis & Instagram https://www.instagram.com/howibuiltthis, and email us at [email protected] mailto:[email protected]. See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Guy Raz?

  Our presenting sponsor today is Anthropic, the team behind Claude. Every founder on this show started with a hard question. Not the marketing plan, the 2 a.m. kind. Is this idea any good? Am I the one to build it? Should I quit my job? Anthropic got built the same way. A public benefit corporation founded on one hard question. How do we make sure AI goes well for people? So they asked over 100,000 people their hopes and their fears and are publishing what they find, even the uncomfortable parts. Me? I've got my own hard question about AI. Here it is. If everyone has access to something that can help them write the business plan,

• 0:46Guy Raz?

  build the prototype, analyze the market, maybe even come up with the idea, what separates the people who actually build something from everyone else? I don't have a clean answer yet, so I've been sitting with it with Claude. There's hope in hard questions. Ask yours at claude.ai/hibt. We get support from U.S. Bank. A question I love asking founders. What's the one decision you almost didn't make? The one that changed everything? Because it's never the obvious stuff. It's never, I decided to start a company. It's the small, boring, Tuesday afternoon decision you barely remember making until you realize it's the reason everything after it either worked or didn't.

• 1:32Guy Raz?

  Say you're 18 months in. Business is growing. You've got expenses coming from six directions. Gas for the van, office supplies, your phone bill, team lunches that are actually client meetings, and you're putting it all on whatever card you grabbed in year one. You're not thinking about it. Why would you? You've got bigger problems. But here's the thing. Path A, you keep going like that. End of the year, you're sorting through statements, trying to figure out what went where. You've left hundreds, maybe thousands, on the table in rewards you never earned, money that could have gone back into the business. Path B, you pause and get the US Bank Triple Cash Rewards Visa Business Card. Now, you're earning 3% cash back

• 2:16Guy Raz?

  on gas and EV charging, office supply stores, cellphone service, and restaurants. You pick up an extra $750 cash back as a bonus. And you've got a low APR on purchases and balance transfers for 12 months while you're scaling. Same business, same expenses, two completely different outcomes based on one small decision. Visit usbank.com/business to learn more. The creditor and issuer of this card is US Bank National Association, pursuant to a license from Visa USA, Inc. Some restrictions apply. If you travel often, you'll want to hear about this unprecedented offer. For the first time ever,

• 3:00Guy Raz?

  new Delta SkyMiles Reserve American Express Card members can earn two Delta Comfort Round Trip flights for travel within the 48 contiguous US states, Puerto Rico, or the US Virgin Islands. Plus, you can earn 50,000 bonus miles towards your next getaway. And who doesn't appreciate an elevated airport experience? Delta SkyMiles Reserve Card members get access to the Delta Sky Club, your pre-flight oasis to recharge, with 15 visits and four one-time guest passes each medallion year when flying Delta together. That's up to $950 in annual value. Value estimate based on $50 per visit rate to purchase additional visits

• 3:45Guy Raz?

  or bring a guest to the Delta Sky Club. Make every trip feel more rewarding with the car that upgrades your travel. Apply today at delta.com/cardsocial. Offer ends November 4, 2026. Minimum spending requirements and terms apply. Flights are fulfilled as flight certificates valid for Delta Comfort or Delta Maine to select domestic destinations. Book 21-plus days in advance. Up to $80 per person and taxes and fees apply. Other restrictions apply. When I pull up to a gas pump to fill my car, I'm not thinking about chemistry or engineering. I'm thinking about getting in, filling up, and getting on with my day. And it reminds me a little bit of the founders I talk to on this show, because the best

• 4:27Guy Raz?

  ones obsess over things their customers will never see. Tiny details, invisible improvements, relentless iteration. And not because anyone's watching, but because it makes the product better. That's what the mobile brand does. It engineers fuel the same way great founders engineer their products. Mobile Supreme Plus Premium Gas helps keep engines three times cleaner, which can mean better performance and improved gas mileage. It's for people who care about what's under the hood, even if they never see it. Three times cleaner compared to mobile regular gas in port fuel-injected engines. Actual results may vary.

• 5:07Guy Raz?

  Visit mobilefuels.com for details. Hello and welcome to How I Built This Lab. I'm Guy Raz. The pandemic changed education forever. In just a few days, schools around the world had to move online. Teachers began teaching on Zoom. Schools started issuing students laptops. And platforms like Google Classroom became indispensable. Even with kids back in the classroom now, school is much more digital now than it was before 2020. And our guest today says that an even bigger change is on the way. A revolution that could transform the way kids learn even more than the pandemic did.

• 5:51Guy Raz?

  Sal Khan has been on the show before. He's the founder of Khan Academy, an online teaching nonprofit that went from pixelated YouTube videos to a massive platform with hundreds of free tutorials in dozens of languages and tens of millions of users every month. Right now, Sal and his team are working on a new learning platform. It's called Khan Migo, which uses the generative AI technology behind OpenAI's ChatGPT to help students with their schoolwork. And the people at Khan Academy think that Khan Migo could act as a personal tutor for every student and a teaching assistant for every educator, but we'll get there. First, let's go back to 2020 and the early months of the pandemic when Khan Academy saw

• 6:35Guy Raz?

  millions of new students begin to use its materials.

• 6:38Speaker 01

  You know, the spike was interesting because that first week where you had global shutdowns in the US and other places, we talked about it last time, our traffic went from about 30 million learning minutes per day to about 90 million learning minutes per day, pretty much within that week.

• 6:52Guy Raz?

  Wow.

• 6:53Speaker 01

  What was interesting is you fast forward a couple of months, as we know, a lot of school systems started to figure out how to do online schooling.

• 7:03Guy Raz?

  Yeah.

• 7:04Speaker 01

  And then we saw things actually normalize, but maybe in a very abnormal way, mainly because I think people started to have screen time fatigue. And I think during the pandemic, people had a, you know what, if we can just kind of pretend like we're going through the motions of school, let's call it a day. And because of that, people weren't looking to improve necessarily, they were just looking to tread water.

• 7:26Guy Raz?

  Hmm. What did you find? I mean, as you know, I mean, there have been studies over the past two years comparing students pre and post pandemic, right, their standardized test scores. And it's clear that student performance in both reading and math fell significantly, math, I think even more than reading. Have you found that Khan Academy helped keep students on track during the pandemic? Or is that something you even measure?

• 7:48Speaker 01

  Oh, yeah, this is something we've been keeping a very close eye on. And we've, the thing I always point out, first of all, is that the numbers were not good pre-pandemic.

• 7:55Guy Raz?

  Yeah.

• 7:56Speaker 01

  What we saw, we did an efficacy study in the first full school year, the 2020-21 school year, and then we did another one in 2122. And what we saw in 2021 is that the students who put in an average of 15 minutes on Khan Academy a week in a school setting, they actually saw no COVID learning loss. And that students who put in 30 to 60 minutes, they accelerated almost 40% faster than pre-pandemic norms. And it's not a mystery. They're just getting more practice at their level with more feedback. And those that are supporting them, their teachers, are getting more information about where the students really are and maybe can adjust their lessons a bit based on that.

• 8:38Guy Raz?

  Tell me about what you, I mean, you must have learned, and I think now this is probably the third time I'm talking to you since the pandemic started. And in each time I've talked to you, it's so clear how much you're, you were learning in real time and how much you've learned from what happened, sort of the overwhelming numbers of people coming to Khan Academy. Tell me how it sort of fundamentally changed how you guys operate.

• 9:04Speaker 01

  I think it did a bunch of things that we always wanted to do, but this just accelerated things. We always wanted better ways for students to address gaps that they might have. The fashionable term now in education circles is unfinished learning.

• 9:20Guy Raz?

  Yeah.

• 9:20Speaker 01

  We all thought that the pandemic was only going to affect the school system maybe through that first summer. And by the time you go back to school, even then, there would be some damage done. Back to school 2020. I mean, now we know how delusional that was.

• 9:32Guy Raz?

  Yeah.

• 9:32Speaker 01

  But we started creating these back, get ready for grade level courses, which essentially cover all of the prerequisites that a student needs in order to be ready for their grade level work. And we saw that that was very popular. So that was accelerated. We created another sister nonprofit called Schoolhouse.world. We saw how much people were leaning on Khan Academy, but we also saw that there was a gap of getting real human support. And so the utopian idea was, well, what if they could get real human support for free based on volunteers out there on the internet? And then as we got to the tail end of the pandemic, and as we all know, people had very mixed feelings about COVID learning. Most people did not think it was, you know, spending time on video conference for several hours a day was a good thing.

• 10:13Guy Raz?

  Yeah.

• 10:14Speaker 01

  We felt the need to show that there's a way to do this well. And so another sister organization, we have a lab school that I helped start that's literally in the same building as Khan Academy. But we said, could we create an online version of this that can do online schooling, but do it well? Don't do it so it's just students listening to lectures on Zoom for hours a day.

• 10:35Guy Raz?

  Yeah.

• 10:36Speaker 01

  When people are on video conference together, make it interactive, make it Socratic, and then use personalized learning and other tools for students to get more asynchronous support, but always feel connected to a community. So we started working on a lot of things like that as well.

• 10:51Guy Raz?

  All right, let's go now back to, not that long ago, back to the summer of 2022. You got an email from Sam Altman at OpenAI. And he said, he sent you a note, said, hey, we've got this thing we're working on, and we would like you to check it out. So you did. Tell me what happened. Tell me the story.

• 11:12Speaker 01

  It's interesting. Almost on a daily basis, we get emails from folks saying, hey, you have an interesting technology. We'd love to partner with Khan Academy. And most of the time, you know, we look at it, we're like, oh, you know, it looks intriguing, but we just don't have the bandwidth. But I obviously knew of Sam and Greg and had deep respect for what many of the things that they've done in their lives. And I have been watching what's been happening with generative AI for the last, let's call it, three to five years. I didn't really think it had a place at Khan Academy because GPT-2 and 3 were good at writing convincing text, but it really had no grounding in factual knowledge. So I'm like, I'm happy to meet with you all. I'm just curious what you all are up to.

• 11:52Guy Raz?

  Yeah.

• 11:53Speaker 01

  And they said, hey, we're going through our first training run of GPT-4, which obviously is now out, but back then that was very confidential. And we want to launch it with a small number of partners that we think can show a social positive use case because we think it's going to change things. I was skeptical at first because I was familiar with GPT-3. About two weeks later, I got another email from both of them saying, hey, we're done. Can we show you a demo? I'm like, sure. So they put up an AP Biology question on the chat, and the AI answered it correctly. I'm like, oh, this is pretty interesting. And then I said, ask it to explain its reasoning. And it explained it. It gave the right answer. And then I said, explain why the other choices aren't correct. It gave that.

• 12:35Speaker 01

  And then I asked, can you create 10 more questions just like that one? And it did. And as far as I could inspect, they were pretty good. That's when I started getting the goosebumps. I'm like, okay, this is, this is different.

• 12:47Guy Raz?

  Yeah.

• 12:48Speaker 01

  And then they gave us access that weekend and I spent hours with it. And it was doing very well, although it definitely had some issues. We realized pretty quickly that you couldn't just have it generate a ton of questions and just put students in front of it. It would make mistakes, especially in those early days. It was pretty, still pretty bad at math.

• 13:05Guy Raz?

  Yeah.

• 13:06Speaker 01

  But at the same time, we also, we started to make it role play. Role play as a tutor, role play as a character, role play, you know, do something in the style of someone else. And that's when I started to really say, wow, this really could be that holy grail that we've all been thinking about, reading science fiction about for, for decades of an AI that can actually emulate a human tutor.

• 13:28Guy Raz?

  Before we really dive into what you did next, from, from what I gather, Sal, I think your initial reaction was the initial reaction that many people have, which, which was like, kind of terrified.

• 13:44Speaker 01

  I read a lot of science fiction and it was, it was a moment in which I felt that all my science fiction reading had prepared me for. But, you know, there were moments that even that first weekend where I was pushing a few of the limits, where I, I asked it things like, are there things that you think that you're not sharing with me? And it said, yes. And then you get a little weirded out by that. I'm like, well, why aren't you sharing it with me? And it says, well, it might offend you or it might scare you. And it feels like you're talking to an alien. It feels like you're talking to a super intelligence. And I had to keep reminding myself, I do understand how it works. I have a master's in computer science. I understand how large language models work.

• 14:23Speaker 01

  And I, I started to say, okay, I can get how, if it's just modeling the natural thing to say of why you're not sharing a thought, it is, why, why don't you or I share a thought? It's because it might offend someone or it might scare them. So that's kind of what it was doing. But it was an otherworldly feeling and it was, you know, we had signed a non-disclosure agreement. So it was a very hard secret to keep. You know, every now and then I'd go to dinner parties and I was like, I wish I could tell you, but the world is about to change.

• 14:53Guy Raz?

  We're going to take a quick break, but when we come back, how Sal and his team struggled with the shortcomings and the dangers of GPT. Stay with us. I'm Guy Raz and you're listening to How I Built This Lab.

• 15:10Guy Raz?

  Does your team want a website that looks and feels handcrafted but is still fast to ship? Well, Framer is built exactly for that. With Framer, you design on a visual canvas with responsive layouts, hosting and a CMS built in. So the work is production ready from day one, and you can get it done with help from Framer's AI agents. Framer is the AI website builder that helps creators, teams and businesses ship production-ready sites faster than ever while getting every detail right. Prompt, inspect, edit and publish in one place at a whole new pace. AI agents and humans work together. The agents bring speed and scale,

• 15:54Guy Raz?

  and people bring taste, judgment and control. Learn how you can get more out of your site from a Framer specialist or get started building for free today at framer.com/built for 30% off a Framer Pro annual plan. That's framer.com/built. Rules and restrictions may apply. AI isn't just opening new business opportunities, it's changing the way businesses get work done. The questions they ask, the decisions they make, the routine tasks they can hand off. That's the idea behind NetSuite Next, the next generation of NetSuite with AI woven into the tools your business already relies on.

• 16:36Guy Raz?

  NetSuite Next surfaces insights exactly when you need them, puts agents to work on routine tasks and problems, and lets you ask complex questions in plain language, like talking to a knowledgeable colleague. But technology is only part of the story. AI is also reshaping how people and teams work together, and that shift is already underway. That's why NetSuite wants to put a new expert-created business guide in your hands today. It's called Aligning for the Agentic Era: How AI is Changing Everyday Work. And right now, it's yours, absolutely free. Inside, you'll discover how AI could reshape work across your entire organization, and how to position your business to prosper from it.

• 17:21Guy Raz?

  If your revenues are at least in the seven figures, get NetSuite's free business guide, Aligning for the Agentic Era: How AI is Changing Everyday Work, at netsuite.com/built. The guide is free to you at netsuite.com/built. netsuite.com/built.

• 17:41Speaker 02

  Football is back, and PrizePix is bringing out their best offer yet. New users who sign up will get $150 in lineups if you win your first $5 lineup. That's right, if you win your first $5 lineup, you'll get $150 in lineups. On PrizePix, you simply pick two to six players, pick more or less on their stat projections, and submit your lineup. Then sweat it out as you watch your picks play out live. It's the best way to get more locked in on game day. Make all the picks you want, this football season, all in the same app. And now, PrizePix is available nationwide. So what are you waiting for? Lock in for kickoff, because everyone and everything is on PrizePix. What you on?

• 18:26Speaker 03

  Download the PrizePix app today and use code WONDERY, that's code WONDERY, to get $150 instantly in lineups if you win your first $5 lineup. PrizePix, run your game. Must be present in certain states. Visit prizepix.com for restrictions and details.

• 18:42Guy Raz?

  One more thing before we get back to the show. Please make sure to click the follow button on your podcast app so you never miss a new episode of the show. And it's totally free.

• 18:59Guy Raz?

  Welcome back to How I Built This Lab. I'm Guy Raz, and I'm speaking with Sal Khan, founder of Khan Academy. In 2022, Sal got early access to the large language model GPT-4 before ChatGPT was even released. And he was amazed by its potential. So you sort of get over your initial kind of, oh, my God. And you start to see, obviously, a huge potential here to kind of transform what you do, which is offering free education to people around the world. So what did you do next? I mean, the ChatGPT wasn't going to be the exact right thing, right fit for what you do, but you could build on that, on that foundation, right?

• 19:43Speaker 01

  That's right. The first thing we did is we set an org-wide NDA and I said, hey, how many folks in the organization can I get, we give access to? And eventually we got about, about 40 or 50 folks in our organization to, to get access to GPT-4. Yeah. And even that, this was the first time that Khan Academy had to do a little bit of cloak and dagger type stuff. But folks at Apple are used to keeping secrets and things like that, but we're not used to keeping secrets and we had to. This was obviously very sensitive stuff. And we immediately, we had a hackathon, which was well timed because then these 40 or 50 folks were able to, at least for a couple of weeks, say like, okay, let's just, let's just be generative here, no pun intended. Like, what is possible with this? And by the end of that two weeks, we had some really cool demos.

• 20:25Speaker 01

  And then we started to have the internal debate of like, okay, how aggressive do we get on this? What about bias? What about the math errors? What about hallucinations? What about just the PR implications? People trust us. And if they see us leaning way forward on something that's not perfect, that's something that some people could be wary of, what does that do to our brand? What does that do to the trust that folks have? On the other side, folks were arguing, look, this is a transformational technology. This is one of these things where it's either going to allow us to magnify our impact by an order of magnitude, or it might make us irrelevant. And so we, we made a decision to say, you know, we have to keep moving forward, but we have to do it in a risk-aware way, in a risk mitigating way.

• 21:05Guy Raz?

  All right. I think it's fair to say that you are a true believer in this and your team basically developed an AI that you call Khan Migo. And so this is now, I think in beta, but still integrated into Khan Academy. So tell me how, how this works. What, what is it?

• 21:25Speaker 01

  So first was the guardrails. Like, what, in what context would we even feel comfortable doing this? And what's interesting is, we were starting to do all this in earnest in September and October, then end of November, ChatGPT comes out. And I remember the day that it came out, I slacked Greg Brockman at OpenAI and I said, hey Greg, we're under NDA and I thought we weren't launching anything until March of 2023. What's this chat GPT thing? And Greg says, it's, it's nothing new. It's based on GPT 3.5, which it had been out for many months already. No one really took note when they released GPT 3.5. And OpenAI just decided to publish a bunch of apps that use GPT 3.5. One of them happened to be chat GPT.

• 22:09Speaker 01

  And then being able to inter- interface with the model in a chat interface, I think, made everyone see what the power of it was. And at first I was a little bummed. I was like, oh, this is going to steal the thunder. I've been telling all my friends at dinner parties, something's big's coming in 2023. Just you see, but then I was, I think it was actually a blessing because as we know, chat GPT goes out there. It amazes people, but it also scares people. And then immediately it creates huge issues for education, like kids are cheating using this.

• 22:39Guy Raz?

  Right, you can, you can type in an equation and it'll give you the answer.

• 22:42Speaker 01

  And you can write your essays.

• 22:43Guy Raz?

  You can write your essays, right? Yeah.

• 22:45Speaker 01

  And that I think gave us more license at Khan Academy to say, look, the genie has out of the bottle with chat GPT. We need to double down on bringing a better version of out there, out there that can mitigate all the risks and maximize the benefits. So we started saying, well, you know, first, whatever we make shouldn't be for cheating. It should be for actually helping students learn. So there's many things we're doing to make it Socratic, not just give you the answer. Two, it has to be accurate. It's not okay if it doesn't know what seven plus four is.

• 23:13Guy Raz?

  Yeah.

• 23:13Speaker 01

  It should, it shouldn't hallucinate. And then we also were afraid of students misusing it or using it in some ways that might harm them or other folks. And just reassuring teachers and parents that they can keep track of what their students are doing with, with this very powerful tool. And so that's why we made it so it logs all of the conversations. We have a second artificial intelligence that monitors conversations with the main artificial intelligence. And if any of those conversations go into a A gray zone, then it actively notifies the parents and the teachers that, hey, you should take a look at this conversation. And then we're continuing to add a whole bunch of other things, things where teachers can use it for lesson planning, we're adding memory to it, how teachers can essentially use it as a teaching assistant, get reports on what students are up to, I could go on.

• 23:58Guy Raz?

  So essentially, the idea is, hey, if we can get every single kid a personal tutor, then it will have an impact on their academic performance. And this has actually been studied and measured. I know you gave a TED Talk back in March of 2023, and you cited a study that came out in the 80s, basically, that showed that when kids have one-on-one tutoring, their scores and grades just skyrocket. Like, you can turn an average student into an outstanding student, for example.

• 24:25Speaker 01

  That's 100% right. If you want to become a great pianist, you're not sitting in a class of 30 with your piano and the person's lecturing you. If you're a great athlete, you have a coach who's optimizing you, not just giving a lecture to 30 students and saying, hey, you might want to improve your swim stroke in this way. And then, who knows if you actually do. So one-on-one tutoring has always been the gold standard. When we did mass public education, which was a very utopian idea two or 300 years ago, but we made compromises. We started batching students together, applying set standards, lectures to them. Some kids get it, some kids don't. Even if you have a gap in your knowledge, too bad, move on to the next concept. And as you mentioned, Benjamin Bloom, 1984, he articulated it well. He called it the two sigma problem, two sigma, sigma the symbol for standard deviation

• 25:10Speaker 01

  in statistics. And a way to think about it, two sigma improvement, two standard deviation improvement is dramatic. It's going from the 50th percentile to, I think, the 95th or 96th percentile. But the reason why he called it a problem was, well, how are you going to give everyone a one-to-one tutor? And he also, in 1984, theorized, well, maybe you could emulate some aspects of that with technology. But I've been citing that study for many years in terms of what Khan Academy is trying to build. Essentially, old Khan Academy or base Khan Academy trying to get to that first standard deviation. But now with the AI, we can go to that second standard deviation.

• 25:44Guy Raz?

  So essentially, the interface, if you're listening and you're familiar with the interface of Khan Academy, let's say you get a math problem, and you can essentially click a little sort of eyeball robot icon in the corner and say, hey, I don't understand this problem, can you help me? It will not give you the answer. You can't say, hey, can you solve this for me? It's essentially, you ask it to help you, and then what happens?

• 26:08Speaker 01

  Yeah. And if you say, give me the answer, say, hey, I'm your tutor. I'm here to help you. You need to learn here, as I think a good tutor would do. And if say, okay, well, give me a hint. It'll say, well, let's take a close look. What is the problem actually asking for? And if you say, well, I think they're asking for this. If I got it right, they have to say, oh, yeah, that's good intuition. Okay. So what's? Where would you take that? Or let's say I say, I think the next step is X squared minus five. It's like, okay, take a double look at that. Are you sure it's minus five? Pay attention to your, to your signs. And one of the things we've done to make it more robust on the math side is when a student presents their math to the AI saying, hey, I think this is the next step. The AI behind the scenes comes up with what it thinks are reasonable responses from the student.

• 26:50Speaker 01

  It doesn't share those with the student, then it compares its reasonable responses to what the student said. And if it gets something different, it'll tell the student, hey, I'm getting something different. Can you explain your reasoning, which is very pedagogically strong? And so then if the student explains the reasoning, and it's a large language model, so it's very good at understanding when a student explains their reasoning. And sometimes, you know, the AI will say, okay, you know, now that I see what you did, you know, you might have missed that aspect of it. Or it might say, you know what? You got it right. I realize I made the mistake. Which we actually have heard feedback from, from students that they actually really like that. And once again, I think this is very human like. This is what a human tutor would do. When I tutor, I got started tutoring my cousins back in 2004, and this would happen all the time.

• 27:34Speaker 01

  My cousins do something and I'd say, hey, that's not what I got. Can you explain how you got that? And nine times out of 10, I was right. But one time out of 10, they were right. And I'm like, oh, my bad. You're right. And so I think people really appreciate that.

• 27:49Guy Raz?

  And this is now available. I mean, it's totally a feature of, of the interface.

• 27:55Speaker 01

  Yeah. What we did is when we released it coincident with the GPT four launch in mid March of 2023. And we did a limited release pilot where people had to go on a waiting list and then they had to give a donation. And the reason why they had to give a donation is this stuff isn't cheap. The computation costs, you know, every interaction is going to like 5,000 Nvidia GPUs that are crunching this. And so anyway, you know, we have to pay OpenAI money, which then pays Microsoft Azure money, which then uses that money to go buy expensive Nvidia chips. So that's, that's something that we've had to grapple with because our mission statement is free world-class education for anyone, anywhere. But we have about 10,000 folks using it right now, some in mainstream school

• 28:37Speaker 01

  settings, and it's much more than just the tutoring interaction that we just talked about, which in and of itself is powerful. But we have activities that are standalone AI activities where students can talk to historical characters or literary characters. They can get feedback on their college essays. They won't write the essay for them, but it can give them feedback. It can coach them to help them think of their college essay. It can act as a guidance counselor in certain ways. It can act, help teachers create lesson plans, assessments. And what we're seeing, the feedback has been very, very positive. So this summer, we're hoping to make it a more broad release, so that we're expecting many tens of thousands or more folks are going to sign up.

• 29:17Guy Raz?

  Sal, there's a part of me that's also asking, like, am I not asking all the right questions? Am I not thinking of all the downstream consequences? I mean, obviously, we're talking about Khanmigo, but as you know, many school districts in the United States have essentially blocked ChatGPT from school-issued laptops. They banned them from being used, right? I think a bunch of school districts all across the country. And so what is it that they don't fully understand that you, or what is it that they're not seeing? That if you could make the case to them that this is a mistake, what would you say?

• 29:59Speaker 01

  We've shown Khanmigo to many of these same districts, and their general reaction has been, in fact, not general, all of their reactions has been, this is what we need. This is, I guess a fair analogy might be, what Khan Academy is to YouTube is what Khanmigo is to ChatGPT or standard generative AI, whereas we also know a lot of school districts ban YouTube, even though YouTube has a lot of really good learning content on it.

• 30:26Guy Raz?

  Has a lot of great stuff.

• 30:27Speaker 01

  A lot of great stuff, but it also has a lot of stuff.

• 30:29Guy Raz?

  A lot of crap.

• 30:29Speaker 01

  A lot of crap, a lot of stuff that can, students at minimum is going to distract them, and worst case is going to put them into some kind of weird rabbit hole and mess with their head. So they banned it, but something like Khan Academy where you can create curated, safe environment, it's monitorable, et cetera, they feel much more comfortable. And so when we showed them, it's not cheating, it's actually acting like a tutor. It provides oversight by adults. It can flag when students are getting into suspicious areas with the AI. They're like, these are just the guardrails that we wanted.

• 31:03Guy Raz?

  But I mean, you concede, and I guess you'd have to, that there's nothing preventing a kid from using ChatGPT too cheap.

• 31:10Speaker 01

  Nothing preventing. And so my best guess of where, let's say, term papers or homework is going to have to evolve to is especially in something like writing, you're going to have to do some in-class writing. And that's a place where I think Khanmigo can be really useful. We're working on activities right now where a teacher can say, hey, I want all the students in the class right now to write a five paragraph essay about the following. And then on the student interface, Khanmigo will say, hey, Mr. or Mrs. Smith just wants your thoughts on this thing. Let's work through this together. And then the students can work on it right there. And Khanmigo won't do it for them, but can help assist them. You know, well, are you really answering the question that they're asking? Or do you have more data to back that up? And simultaneously Khanmigo can tell the teacher like, okay, Guy is making a lot of progress, but Sal seems a little bit stuck.

• 31:55Speaker 01

  You might want to go walk up to him. You know, Mary has already finished the assignment and she's now going back to working on her math or something like that. Super useful for the teacher. And even for take home assignments, the teacher can say, I want you to do it on Khanmigo. And the process is as important as the outcome, as we know it often is in writing.

• 32:13Guy Raz?

  Yeah.

• 32:14Speaker 01

  And then if there's some student who goes to ChatGPT and just says, ChatGPT, write this essay for me, and just copies and pastes it into Khanmigo, Khanmigo can tell the teacher like, there was no process here. The essay just showed up.

• 32:25Guy Raz?

  I see, I see. So Khanmigo could be the interface too for submitting assignments.

• 32:31Speaker 01

  It could be the interface, it could be the coach, and it can, it really acts as a teaching assistant. Imagine every student having a teaching assistant and the teaching assistant is working with every student and can report back to the teacher like, yeah, I worked with that kid. I know it's their thoughts and I helped them a little bit, but it's mostly them. But one kid just showed up with an essay and really couldn't defend his argument. Let's take a second look at that.

• 32:53Guy Raz?

  We're going to take a quick break, but when we come back, more from Sal on the future of Khan Migo and whether it could ever become a for-profit product. Stay with us. I'm Guy Raz and you're listening to How I Built This Lab.

• 33:15Guy Raz?

  Welcome back to How I Built This Lab. I'm Guy Raz and I'm talking with Sal Khan, founder of Khan Academy. They built a teaching AI called Khan Migo that could become a pretty lucrative commercial product, but Khan Academy is a nonprofit. Sal, this is a business show, as you know, and so I want to ask you to put your business hat on for a moment because we don't do many nonprofits on How I Built This. And there are tons of ed tech companies out there that make money working with school districts, you know, Dreambox and all these other programs where you can, schools are paying a fee for kids to use these products and Khan Academy products are free to use. But I want to ask about Khan Migo. I mean, if this, in fact,

• 34:00Guy Raz?

  could be a tutor for every kid and an interface with the teacher, giving the teacher feedback on the student's progress, it strikes me that this is something that you could license to schools, that you could actually turn into a product. Obviously, you're a nonprofit and you're mission driven, but is that something that you would consider doing?

• 34:20Speaker 01

  Simple answer is yes, and we're kind of doing that, a little bit of that already. You know, I always point out to our team, we're not-for-profit, we have free in our mission, but still, it costs on the order of 60, 70 million a year to run Khan Academy, both to, you know, just our server costs are $6, $7 million a year, and then obviously all of the engineers, et cetera. And historically, we've been primarily funded with philanthropy.

• 34:42Guy Raz?

  Yeah.

• 34:42Speaker 01

  But with that said, about five years ago, we started going to school districts. Obviously, we've had a lot of what we call grassroots usage in classrooms, hundreds of thousands of teachers. But we said, if we really want to reach all kids, we have to work formally with the districts. And we would go to the districts, we would show them our efficacy studies. But they said, look, for us to use this systemically inside of our district, you have to give us support, training, integration with our rostering systems, district-level dashboards.

• 35:05Speaker 01

  And that's where we said, okay, to do all of this bespoke work, someone has to foot that bill. And we said, look, we're not going to charge you for all of this stuff that's funded with philanthropy, but at least cover some of the incremental costs here. And so to a lot of districts, and we now have about a million students in districts where they're paying on the order of about $10 per student per year to get all of these other things. So we are built, we have built this enterprise muscle, and it is trying to build a little bit of a flywheel of sustainability. And to your point on Conmigo, Conmigo has a very real marginal cost to it. It costs on the order of, let's call it $10 to $15 per month. And so what all of these school districts that have come to us, we are talking to them

• 35:47Speaker 01

  of like, hey, we need to at least cover our computation costs. And then we do expect that the underlying computation costs are going to come down. And so we are having these conversations on our team right now of like, okay, right now it costs, let's call it $10 per user per month, just for the computation, we have to charge that for Conmigo, not for BaseCon Academy. But let's say that cost goes down to a dollar per user per month. Do we just lower it to a dollar? Or do we lower it to $2? And then we use that that incremental dollar to fund our R and D?

• 36:15Guy Raz?

  I mean, I look at this, the potential here, right? And what you've developed and the amount of money you've put into it already. And but I wonder whether there's a world where you take a product like Conmigo, which if it's as revolutionary as it looks like it could be, I mean, it can really change the face of education. Is there a world where that's spun out into a for-profit business?

• 36:41Speaker 01

  There's always been this debate, not at Con Academy, but about Con Academy, which it always confused folks, why we're not for profit. You know, we scale, we're tech-heavy. I live in the middle of Silicon Valley. Most of my friends are entrepreneurs or or VCs of some kind. And there's a Harvard Business School case about this, should Con Academy be for-profit or nonprofit? And to me, the arguments for for-profit have historically been access to talent and access to capital. The reasons to be nonprofit are you really can truly make your mission the bottom line. And I think there's a trust aspect of it as well, that you know that our bottom line really is this, it's not trying to improve our EPS or have an IPO.

• 37:23Speaker 01

  What we have found is we are getting access not to just good talent, but to the best talent. People are here for the mission and they're here to work with other people like that. And so the only reason that I could ever see trying to do some type of a spin out or a wholly owned sub that's is for some reason, if we're not getting the risk capital from the foundations, or we can't create a flywheel of sustainability, then maybe certain pieces of it. But I'm very afraid to do that. I mean, definitely wouldn't do that with the mothership because I do see how, you know, a lot of good comes, I'm a diehard capitalist. My old job was at a hedge fund. A lot of good comes out of market incentives. But I think education and healthcare in particular, market incentives don't always align with our values.

• 38:07Guy Raz?

  Sal, one of the incredible triumphs of Khan Academy has been the millions of kids who have benefited from it. But in the United States, a huge concern, as you know, and I know you've looked at this, has been the gap between wealthier kids, kids who are not as wealthy or impoverished, a gap in outcomes between black students and other students. How can something like this change that in your view? I mean, I don't want to be too much of a techno optimist here, but I do want to be optimistic. I mean, and I'm sure you are. How do you think they can kind of change that equation?

• 38:47Speaker 01

  Yeah, some of the stats would shock folks. A majority of minority majority schools, so that's a mouthful, but a majority of minority majority schools don't offer courses like Algebra II, Physics, et cetera. Not to even mention things like AP courses or IB courses. And so if you're a young African American student, you don't come from a lot of money, you go to one of these schools, you might be the next Albert Einstein, you might be the next Marie Curie, but there's no way that you're going to be able to tap into your potential if your school does not offer Algebra II. Or many times in that Algebra II class, because kids are coming from In many cases, tough circumstances have gaps in their education before that.

• 39:31Speaker 01

  The teacher kind of tries to teach the middle or even to the bottom quartile, saying that that's those are the most severe cases. And once again, that kid who could have been Marie Curie is not going to be able to prepare, prepare themselves. And then when they go to college, they're going to feel inadequate compared to kids who had much better preparation. So that was, that's always the view of Khan Academy, which is we want to raise the floor.

• 39:53Guy Raz?

  Yeah.

• 39:53Speaker 01

  And then we want to provide as many supports as we can. But we view Khan Migo is kind of in between traditional Khan Academy and Schoolhouse.world, but that's just another layer of support. Now, in order to access all of this, you still need internet access. You still need a device. And that's why we work so closely with school districts. You know, the technological side in the US has gotten a lot better on over the last 10 years, but that's why we're working with school districts to try to get more kids at that level of engagement.

• 40:22Guy Raz?

  Sal, just a broader question about AI. As these products, as other AI platforms become better and better, you can imagine a future not too far in the distance where things like writing won't actually be that useful. Like, you know, we think of this as a skill. We think of great writers as people who've honed their craft, who have talent. But I mean, if these large, you know, language models can absorb everything ever written in human history, well, you can also imagine that that skill and talent won't be particularly useful in a short period of time. Do you agree with that?

• 41:01Speaker 01

  That was my initial thought slash fear slash concern. The more that I thought about it, and I'm writing a book about all of this right now. So I've been interviewing folks and, you know, I interviewed Kevin Ruse, who famously, you know, New York Times writer, who had that famous conversation with Bing's Sydney. And it was trying to convince him that he didn't love his wife and he loved her and all of these, all of these things. And I asked him that same question and I thought he brought up a very good point. I mean, he's literally a writer and he's like, look, to be a great writer or journalist. There's the writing part of it, and then there's the journalism piece of it. And sure, AI could help you with the first draft. It can help give you feedback on your writing. But it's going to take a lot to get to great writing. It might not get quite there.

• 41:45Speaker 01

  And that whole piece of journalism, like talking to the right people, being creative about how you get your information, connecting the dots. AI is not going to do anytime soon. And I think every job has that aspect of it. So I actually think the imperative isn't that, oh, kids aren't going to learn to write. I'm worried about the the kids that only learn to write OK. Well, to some degree, they might be empowered by something like AI. Because now their writing is not going to hold them back in other domains. But anyone in the, let's call it the writing lane, they're going to have to become better. They're going to have to move into the editorial role to be able to know how to manage, to create great writing. How do you know that it's that the AI has done a good job?

• 42:29Speaker 01

  And that you can't coach it to be even better? You won't make someone an editor of a newspaper unless they could be one of the best writers themselves. And I think the people who really leverage AI well are going to be the people who can who can get into that. How do you manage the AI? How do you put the pieces together? And how do you do those things that AI won't be able to do?

• 42:48Guy Raz?

  So anyone who who has a child, who knows a child that's been in school during the pandemic, knows how much education has evolved, right? I mean, many kids are now issued laptops and not just in private schools, but also in public schools. And so much of the work is done through Google Classroom, right? And parents can interface with it. And it's just a completely different world. Forget about what they're learning, it's how they're learning. But I suspect that what you're talking about with things like Conmigo and other AI platforms. What we're about to see is something that is like going from, I don't know, like the Stone Age to the Industrial Revolution. Is that a fair analogy?

• 43:29Speaker 01

  I think you might be right. You know, there's a world now where you can, you can talk to historical characters and we're probably three to five years away where you could share a room with Benjamin Franklin or Julius Caesar or Cleopatra and immerse yourself. It'll, it'll be literally the holodeck from Star Trek. I didn't think that was going to happen in my lifetime. I now think that's going to happen in the next five years. I think for the most part, this is positive because the students who are really motivated, you know, there's a class of students when they, when they were in their math class and I was in this class of students. I imagined how I was describing the universe and I was like, oh, this is so beautiful. This is so elegant. Wow, this is so connected. And that motivated me to go get through some of the grunge and the really hairy equations and all that. But for a lot of kids, they didn't see the, the beauty in it.

• 44:13Speaker 01

  And so they weren't motivated. But if you're motivated, you can power through anything. Same thing on history. When you just read the history book like, ah, ah, this is dull. But if you're like, wow, this actually happened to real people and I actually know about it. And could you imagine what it would have been like to be in that moment when Caesar crossed the Rubicon? You know, what was going through his mind? Then all of a sudden, history comes alive and you become incredibly motivated. I think AI will really help unblock folks as a tutor, but also really help motivate. I mean, imagine having a tutor that not only can do what Aristotle did, but it actually can be Aristotle sometimes. It can role play with you. It can take on tone that really captivates you. This all would have been science fiction a year ago and it's literally happening now or in the, the coming months.

• 44:57Guy Raz?

  It puts to rest the question, if you could have dinner with five living or historic people, who would they be? Because you can actually do it at some point.

• 45:06Speaker 01

  You can actually do it. You know, having the real dinner will always be better because we, you know, obviously the AIs are, they're interpolating or extrapolating from, from, so that there's, there's definitely some imperfections and, but yeah, you're right. In terms of the richness of experience, it's even better than the dinner because probably in five years, you could put some goggles on and have the dinner actually in Rome in the first century BCE as opposed to at your dinner table.

• 45:33Guy Raz?

  It's amazing. It's absolutely amazing. Sal Khan, thank you so much.

• 45:37Speaker 01

  Thanks for having me, Guy.

• 45:39Guy Raz?

  That's Sal Khan, founder and CEO of Khan Academy.

• 45:45Guy Raz?

  Hey, thanks so much for listening to How I Built This Lab. Please make sure to follow the show wherever you listen on any podcast app. Usually, there's just a follow button right at the top, so you don't miss any new episodes, and it is entirely free. If you want to contact our team, our email address is [email protected]. This episode was produced by Alex Cheng with editing by John Isabella. Our audio engineer was Neil Rausch. Our music was composed by Ramtin Arablouei. Our production team at How I Built This includes Chris Messini, Carla Estevez, Casey Herman, J.C. Howard, Liz Metzger, Sam Paulson, Kerry Thompson, Elaine Coats. Neva Grant is our supervising editor. Beth Donovan is our executive producer.

• 46:30Guy Raz?

  I'm Guy Raz, and you've been listening to How I Built This.