The AI Tutor For Every Child and the Next Frontier of Education, From Khan Academy’s Creator Sal Khan

The future of education is right at your children’s fingertips. Sal Khan, CEO and Founder of Khan Academy, joins Sarah Guo and Elad Gil this week on No Priors. For over a decade, Sal Khan has been trying to reform education, beginning with tutoring his cousins in math. He's the father of the YouTube "chalk talk" format, and has now served tens of millions of students through Khan Academy. He guides us through how Khan Academy is using AI to personalize a student's educational experience, transporting students into immersive learning experiences that allow them to debate historical figures, to assisting teachers with lesson plans that address the learning gaps keeping students from reaching their full potential, to a Khanmigo, a tutor for every child. Prior to founding Khan Academy, Sal worked as a hedge fund analyst. He holds an MS in business from Harvard University, as well as an MS in Engineering and a BS in Computer Science from MIT. Show Links: Khan Academy - CEO & Founder - Khan Academy | LinkedIn Khan Academy Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @salkhanacademy Show Notes: [0:00:06] - Sal Khan's Journey [0:08:41] - Mastery Learning and AI in Education [0:19:53] - Future of AI Tutors in Education [0:23:10] - Education's Future With Generative AI [0:29:35] - Connecting Learning Through Tutoring and Collaboration [0:33:22] - Implications of GPT 4 on Education [0:40:42] - Future of Education and Job Skills [0:46:47] - Importance of Traditional Skills in Education

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Sarah Guo?

  Sal Khan democratized learning with his educational YouTube videos that turned into Khan Academy, which has 150 million learners from all over the world today. 15 years later, he's as energized as ever about making learning personalized with artificial intelligence. This week on the podcast, Elad and I talk with Saul about the impact of AI on education. He says, surprisingly, we're on the cusp of the biggest positive transformation that education has ever seen. Khan Academy has recently created Khanmigo, a chatbot tutor that can nudge learners in the right direction if they get stuck. Saul, welcome to No Priors.

• 0:37Speaker 01

  Thanks for having me.

• 0:38Sarah Guo?

  Saul, can you start by giving us some background on yourself and how you ended up starting Khan Academy?

• 0:44Speaker 01

  Sure. You know, you go way back. My original background was in tech. I go to business school. I end up being an analyst at a small hedge fund. And it was shortly into that, it was 2004, I was a year out of business school. I had just gotten married. I was based in Boston at the time. I was born and raised in New Orleans. My family was visiting me up in Boston after the wedding, and it just came out of conversation. My 12-year-old cousin Nadia was having trouble with school, math in particular. She was being placed into a slower math track. Her parents, my aunt and uncle, hadn't gone to school in this country, so I don't think they understood the implications that on that track, she wouldn't end up taking calculus in high school, et cetera, et cetera. So I took it pretty seriously. I said, hey, I'm up for tutoring you, Nadia, if you're up for receiving it. And she agreed, so she goes back to New Orleans. I start tutoring her remotely.

• 1:28Speaker 01

  She actually gets caught up with her class. She was actually having trouble with unit conversion. She gets a little ahead of her class. I call up her school. I said, I really think Nadia Rehman should be able to retake that placement exam. They say, who are you? I said, I'm her cousin, and they let her. And that same Nadia that started off as, for lack of a better word, a remedial student, was then put into an advanced math track. So I was hooked. It was fun to stay connected with family. I really enjoyed geeking out on the math, and it seemed to be really helping my family. And so I started tutoring her younger brothers. Word spreads in my family, free tutoring is going on. Before I know it, I'm tutoring 10, 15 cousins. And I still had the day job. I was working as an analyst at a hedge fund. And I saw a common pattern with my cousins.

• 2:13Speaker 01

  They just, the main reason they were struggling was they had gaps in their knowledge. Oftentimes, the reason they were having trouble with that algebra equation was because they weren't fluent in dividing decimals, or negative numbers, or exponents. And so I started making software for them, which I did for fun, because I, that, that part of my brain wasn't being used fully in my hedge fund job. I started making software that would generate problems for them, give them immediate feedback, provide hints if they needed them, allowed me to keep track. I put a little database behind it and allowed me to keep track of what they were doing and when they were doing it and what they were getting wrong and right. And I, I called that Khan Academy. That was the domain name. It was available. It was kind of a fun family project. And about a year later now, we're about 2006, and I was showing this off at a dinner

• 2:57Speaker 01

  party, and by this point, my family had moved out here to Silicon Valley.

• 3:00Sarah Guo?

  Do you still have the hedge fund job?

• 3:01Speaker 01

  The same hedge fund job, but actually, it's interesting. I have a whole theory that we can keep referring to, that benevolent aliens are using Khan Academy to prepare humanity for first contact. And

• 3:12Sarah Guo?

  Okay.

• 3:12Speaker 01

  Because there's a series of things in, in, in my journey that seem like it was almost like, you know, people were moving me around in a way that, that, that benefited this. But one of them was my boss, I mean, it was a two-person hedge fund at the time. His wife decided to become a law professor. And it was interesting, because at first she thought she was going to go to UVA. And in case, you know, we would have ended up living on a farm in Charlottesville or something. But at the last minute, she decided that she wanted to take the job at Stanford. And so my wife and I came out here and my wife ended, who, who's a physician, she ended up doing her fellowship at Stanford because of my boss's wife, not wife's choice. But that, that landed us here in Silicon Valley, which I think ended up becoming very important

• 3:56Speaker 01

  for the future of Khan Academy. And then I was showing this off at a dinner party in, in Silicon Valley to a friend. And he said, well, you know, this is cool, but how are you scaling your lessons? I'm like, well, I'm not, like, it's hard to do with 15 cousins what I was doing with one. He said, you know, there's this YouTube thing. Why don't you record some, your lessons as that? And I immediately thought that was a horrible idea. I thought YouTube was somewhat frivolous for dogs on skateboards, cats playing piano, et cetera. And, you know, I was like, yeah, what, how do I even record that? This was before phones had cameras, et cetera. But I decided, oh, there's gotta be a way to record your screen while you're drawing something. And it's called screen capture. And I just started using it and I started just covering things that I found myself repeating a lot for my cousins.

• 4:40Speaker 01

  And my cousins, in famously or infamously, told me that they liked me better on YouTube than in person. I think what they were saying is they liked having an on-demand version. There's no judgment, no shame. If they had to review things, it was available all the time. But they really still appreciated having me in their life, I think. But then YouTube, just a lot of people started discovering it. A lot of people were also discovering the web app. I actually had to shut down registrations because it was crashing my $30 a month Java serverlet web hosting that I was using at the time.

• 5:14Sarah Guo?

  At what point did you open it up to anybody that wasn't a cousin?

• 5:17Speaker 01

  Well, YouTube videos by def- you know, I saw no reason to make them private.

• 5:22Sarah Guo?

  Right. I meant the like Java serverlet app.

• 5:25Speaker 01

  The Java serverlet app, initially I had it opened up to anyone who wanted to register. And I was actually, you know, my day job, I was an analyst at a hedge fund, but I was emailing local schools and seeing if they wanted to use it. And some of them were interested in using it. Famously, the first school that used Khan Academy, this was in 2007, was Sidwell Friends, which is, I didn't even know, they're very elite school in DC. It's where all of the, you know, senators and presidents' kids go. But they were actually a fifth grade classroom, this teacher was John Marmino, was one of the first, and he was just using this homebrew software that I was, I was creating. But I actually stopped registrations on that at 10,000 because, once again, it was crashing my $30 a month web hosting and my cousins weren't able to use it. I mean, talk about, you know, backwards silicon, you know, so many things in Silicon

• 6:10Speaker 01

  Valley, people would love to have the problem. Of an app that more people want to use. I was like, nope, no more for anyone else. But by 2008, it, there were about 50,000 folks who were using the YouTube videos on, on a monthly basis, which, you know, compared to where we are now seems miniscule, but at the time seemed like a lot compared to my 15 cousins. And I, at the time, I didn't think I was going to quit my day job. I actually liked it. I was getting paid well. But I did set up as a not-for-profit, thinking that maybe I could get some funding, maybe do this on the side, hire some folks, et cetera, et cetera. So I set it up as a not-for-profit mission, free world-class education. But then a year fast forward, it kind of took over my life. At that point, it's like, you know, a hundred thousand or something, folks were using it.

• 6:54Speaker 01

  This is all I was thinking about. And, uh, you know, I, I kind of came to terms with, we were trying to save up money for a down payment on a house in Silicon Valley, which we all know is not a joke, even back in 2009. But at the end of the day, you know, real, real wealth is being able to do what you, you, you feel like is your, is your purpose in life, I think. And as long as you have enough money to, you know, put food on the table, go to a restaurant every now and then, hopefully send your kids to college, et cetera, et cetera. If, if you can do what you care about and feel a sense of purpose, that's probably a bigger deal. And so that's, that's what I told myself and my wife, and we gave, and I took the plunge to do it full-time.

• 7:30Sarah Guo?

  It's, it's amazing story. At some point, did you like learn more about education? Cause this feels very much like, oh, I know algebra and I have some instincts about what these kids don't yet know about algebra. Like, was there like reading about, you know, the two sigma problems somewhere in there, like pedagogy, any of it?

• 7:47Speaker 01

  Simulators in the early stages, no. In the early stages, it was very intuitive driven. And it was really, uh, you know, I think a lot of, I'm, I'm guessing that both of y'all, we were all in the category of kids that did pretty well academically when we were in school. And I think when you do well academically in school, you, there's one of two narratives you can tell yourself. One narrative is, hey, I'm just special. I'm just good at this. I'm just gifted. The other narrative is, you know, what I'm doing. It doesn't really feel all that special and some of my peers that are flunking the same classes that I'm getting A's in, they can beat me at chess. They can, you know, learn a video game or a puzzle or solve a puzzle faster than me. So I don't think it's about like pure cognitive capability. And then the more, I used to tutor peers in high

• 8:31Speaker 01

  school as well. And the more I did that, I just realized it was the same thing I saw with my cousins. And the benefit I had and a lot of us that did well in topics like math and science, we, we didn't allow those gaps to fill. Whenever we felt a little bit uneasy about how fluent we were with something, we, we gave ourselves a little bit of extra practice and we weren't just memorizing things. We always wanted to understand why, you know, why does the long division algorithm work? You know, let me think about that. Does it? Okay, that makes intuitive sense. Why am the place values work, et cetera, et cetera. So I think when you do that, then later and later, math frankly gets easier and easier, not harder and harder. And if you do the opposite, where you, where you don't understand the conceptual ideas behind the algorithms and the equations, and then you also have

• 9:13Speaker 01

  fluency gaps. I mean, the number of classrooms I now visit where seventh graders, they understand exponents, but they literally, you know, three times seven, they're grabbing, grabbing their calculator. I'm not exaggerating. I literally saw a seventh grader in the Bronx for three times seven, grabbing their calculator and I literally slapped their hand. I don't know if that can get in trouble for that anymore, but like, I literally slapped their hand. I was like, no, you figure out three times seven. And the kid, he started to use his fingers. And I'm like, this is on some level malpractice that no one sat down with this kid for like, an evening and said, you have got, like, it has to be automatic. It's got to be in milliseconds that you know what three times seven is. Cause he, he understood all the concepts of exponents and algebra and all of that, but the cognitive load of

• 9:55Speaker 01

  having to do that. So anyway, I was experiencing that from a very intuitive point of view. Obviously, once Khan Academy got more traction, started working with more educators, people would tell me, oh, you're just talking about mastery learning. I was like, yes, that's exactly what I'm talking about, which is just that you always have the opportunity and incentive to fill in your gaps. Oh, you're talking about differentiation, that everyone is learning at different paces and you have, I was like, that's exactly what I'm talking about. And then people are talking about, well, yeah, mastery learning, Benjamin Bloom, even though it's arguably the oldest way of learning, that you keep working on something if you haven't mastered it yet. He coined it as mastery learning. He also wrote this whole, you know, two sigma problem, that if someone gets one-on-one tutoring, personalization, mastery learning, can they get, can get accelerated dramatically. So

• 10:38Speaker 01

  yes, it was intuition for me that seemed to work, but now I'm very familiar with a large body of literature that actually backs it up from a, from pretty, pretty rigorously.

• 10:48Speaker 02

  Could you talk a little bit about the range of things that Khan Academy does now? Because you have everything from Khan Labs to Amigo to a variety of other efforts. It'd be great to just kind of get a view of, you know, broadly what you focus on currently and what are the areas you're most excited about?

• 11:02Speaker 01

  Yeah. And the answer I'm going to give you will probably make any like textbook strategy person cringe, because, you know, oftentimes people say, just focus on one thing. And maybe it's coming from me that I feel like the, the opportunity to transform education is not going to happen if you just do it narrowly. So I view our charter, you know, the mission, free world-class education for anyone, anywhere, which by itself is a very big statement and almost delusional when I was one guy operating out of a closet, you know, a little over a decade ago. But we are, we are covering all of the core academic material from pre-K through the core of college. That's the goal. We want to do it so it's personalized. We actually even want to do it so it can eventually result in credentials.

• 11:43Speaker 01

  And so what we are doing already on Khan Academy, so we have Khan Academy Kids, which is used by nearly 2 million kids, primarily in America. It's, it's actually kind of turning into like the Sesame Street of this, of this generation. Really great Africa studies. Kids are spending an average of 90 minutes a month on it, which is pretty high for an average. That's a large chunk of all of the kids in America between the ages of three and seven who are using that. That's all subjects. That's math, reading, writing, character development, social, emotional, whatever you want to call it. Then as you get into older grade levels, it's called Big Khan. You know, that's where we just keep going from just the basics of math all the way. Way through calculus and statistics and multi-variable and on and on. Science, we have science, a pretty strong progression from, I would say, late elementary

• 12:26Speaker 01

  school through early college as well, going all the way to biology, chemistry, physics. And this is not just videos. This is, most of our resources are actually behind the exercise platform, where you can get as much practice as you need, deep item banks, immediate feedback. Teachers can keep track of what they're, what's going on and assign through the platform. We've added humanities, so you know, American history, civics and government, world history, we have art history. So we're filling out that grid of all of the core academic material, financial literacy, computer science. So, so all of that economics is happening. Early on, in 2012, 2011, I wrote a book, The One World Schoolhouse. And the first third of the book was kind of like the history of education. The middle third of the book

• 13:08Speaker 01

  was my journey getting here. The last third of the book was, in a world where things are changing and tools like Khan Academy exist, what should education look like? Just, it's almost like a first principles exercise. Like, could we have full year schooling, mixed age classrooms, personalization mastery throughout, more time for hands-on if you can do the other stuff more efficiently. One thing to write about it, a whole other thing to implement it. On top of that, my oldest child was about to enter kindergarten and I felt like a hypocrite if, if I was telling everyone to do mastery learning and if my kid just went to the local school that was not doing mastery learning. So we started Khan Lab School, literally underneath, on the first floor of the offices of Khan Academy. That was back in 2014. That is now

• 13:48Speaker 01

  a K through 12 program. We've had three graduating classes already. So I'm happy to talk more about that. We started another online school with Arizona State called Khan World School. We just started last year with that, but that's now sixth through 12th grade and there's some very exciting things happening there. And then most, well, I'm glazing over a lot. We are working on credentials. So we have a pilot with Howard University where kids in Title I high schools are getting mastery on Khan Academy and that's resulting in college algebra credit, which is, solves a major need. Happy to talk more about that. And then, most recently, we're doing a lot with generative AI. We started partnering with OpenAI about a year ago. So, well before it was really a big thing and we had to keep it quiet for a while. But Khan Migo now.

• 14:35Speaker 01

  You know, it's a, in our minds, it's a tutor for every student. It's a teaching assistant for every teacher. It can support students while they do traditional work on Khan Academy, answering questions, making it relevant, motivating them, even advising them. We have functionality where it can be a guidance counselor. It can be an academic coach, but they can also debate the AI and practice their fine, you know, fine tune their argumentative skills. They can talk to simulations of historical or fictional characters. And on teachers, they can use it to create lesson plans, the grade, create rubrics, refresh, refresh their own knowledge.

• 15:06Speaker 02

  Mm hmm. You've said that we're on the cusp of the biggest positive transformation that education has ever seen. How important do you think AI is relative to the broader set of access that you have through things through YouTube and online coursework and all the rest of it? Is this a complete game changer? Is it an add-on? Like, what's the relative degree of importance of this shift right now?

• 15:25Speaker 01

  I think in the very short term, it is going to be a meaningful add-on. I think if you go three to five years in the future, it will be a game changer. And the reason I say that is going back to Benjamin Bloom, but I think this even predates Ben. The goal standard was always to have a personal tutor. You go back to, if you were a prince in most of, if you were Alexander the Great 2,300 years ago, you had Aristotle as your personal tutor. And Aristotle would speed up, slow down, motivate you when you're feeling down, like do all of these things with you. And that was always the goal standard. Two, 300 years ago, utopian idea of mass public education, but in order to do that economically, we had to make compromises. One of which is you don't get a personal tutor. You don't get a one-on-one teacher. We're going to batch you into groups of 30. We're going to move you at a set time or pace.

• 16:09Speaker 01

  We're going to apply some lectures and standards and homework, et cetera. On the test, some of you are going to get a hundred percent, some of you are going to get 80, some of you are going to flunk it. Too bad. The, the batch needs to move forward. And somehow we expect those of you who didn't know 20 or 30 or 40% of the material on the simpler stuff to understand the more advanced stuff. And what, what, what happens, those gaps accumulate and kids start falling off. And then you eventually put kids on different assembly lines when you start tracking them. And that kind of worked during the industrial revolution where you didn't need a lot of people in the knowledge economy. You needed kind of basically. Educated people to work in factories, et cetera, et cetera. You know, not, not, not so acceptable anymore, but that was a compromise that we had to make.

• 16:51Speaker 01

  Benjamin Bloom, nineteen eighty-four, comes up with the, writes this two sigma problem where he showed one-on-one tutoring, two standard deviation improvement, two standard deviations takes someone from the fiftieth percentile to the ninety-sixth percentile. So an average student then becomes a very strong, becomes an exceptional student. He calls it a two sigma problem because there's no way you're going to be able to do that in a real classroom. Like, we don't have the resources to give every, everyone a tutor. And then he even, back in nineteen eighty-four, tries to see, well, what, what could you potentially do if you approximate a tutor using technology? And he, he kind of throws out the conjecture that you could get one standard deviation of improvement. And to a large degree, everything that we've been doing at Khan Academy was how can we approximate, start to scale pieces of what a tutor could do.

• 17:35Speaker 01

  So a micro explanation, that's what a video does, and it's on demand whenever you want it. Exercises with immediate feedback, that also starts to approximate. And then give teachers and give tutors and give parents information so that they can also act more like tutors. As opposed to just giving a lecture to everybody, they can look at the data where kids are and then do more focus interventions. Have groups of three or four, the kids that need help with the negative numbers while the other kids keep working. Now work with the five kids who have, need help with decimals while the others. So it was all about, okay, in a class of thirty, can we help approximate tutoring? Well, generative AI, and I didn't think this was going to happen in my lifetime. So when we started, we're playing with GPT four about a year ago. It blew my mind that you could actually get it, not only to pretend to be a tutor, it actually had good quote tutor moves.

• 18:20Speaker 01

  It was, it was being truly Socratic. It had some big issues with it around the math and the hallucinations. We've been working pretty feverishly to mitigate those pretty significantly. It's pretty darn good now. And so it's already, and we, we've already started putting it out in real schools out there. We're already seeing it's increasing engagement. More kids are not only engaged, but they're getting, they're, they're not getting as blocked. Teachers are definitely, you know, we haven't done rigorous studies on it yet, it's very early. But anecdotal, teachers like that's answering questions that the students either were afraid to ask or that I as a teacher would not have been able to get to. Teachers are already really enjoying the teaching assistant functionality. It's saving them a ton of time writing lesson plans, rubrics. They kind of feel less alone as a teacher.

• 19:03Speaker 01

  And, and now we're looking at ways that it can even help address some of the problems that generative AI introduced where. We actually just made an announcement today with Instructure, the folks who make Canvas, the learning management system that goes out to its most of higher ed and a large chunk of K-12. We're like, well, if you use the AI to create rubrics, lesson plans, and then the AI essentially administers the assignment with the student and doesn't do it for them, but does it alongside them, then the AI can report back not just on the outcome of the assignment, but actually the process. Like, hey, I worked with the student, we brainstormed thesis statements, and then they were having a little bit of trouble backing it up, but that we eventually got there, and it took us about four hours. And so you can be pretty confident in that situation that the kid didn't just copy and paste from ChatGPT.

• 19:45Speaker 01

  So I think in the next year or two, it's going to be a really useful tool that's going to increase teacher productivity and support a lot of students. I think when you start going about three years out, we're already working on the notion of memory for the tutor. So the tutor, if you tell it, if you ask the AI, why should I learn this? And it says, well, what do you care about? And you say, well, I love women's soccer. The World Cup's going on, so that's fun of mine. So I love women's soccer. It'll say, okay, it'll make a connection to that. But the next time, it should remember that you like women's soccer, so that it doesn't have to ask you again. And so it's creating, but it could, you know, you like language in a certain, you like more casual language, or more formal language, or your reading level, whatever it might be. So we're actually already prototyping

• 20:28Speaker 01

  that it remembers these things about you. We're making it very transparent to the user, so that they can say, no, I'm not into women's soccer anymore. I'm into gymnastics, or whatever it might be. But we think that type of memory, and the type of memory where it can refer to previous conversations, will allow it to be more, allow it to graduate from being just a on-demand help for certain tasks, to being something that can have a long journey with you, have a narrative. I know it sounds a little bit wild, but I could imagine in five or 10 years, you're going to get your best college recommendation from Conmigo. Because Conmigo's like, I've been working with this kid for hours for the last 12 years. So let me tell you about this. And by the way, they love gymnastics. And you have a

• 21:09Speaker 01

  great gymnastics program at whatever. So I think that's, and you're going to have, you know, we're already playing with the text-to-speech. I know people have heard text-to-speech before. What's about to come will blow your mind. It's hard to differentiate from a real human being, which is scary on other dimensions of life, but it's good for tutors. And then I think in three to five years, yeah, you're going to be able to video conference with your tutor, which then makes it like a real thing that you can have a long journey with.

• 21:36Sarah Guo?

  Have you ever read the Neil Stephenson book, The Diamond Age?

• 21:39Speaker 01

  Not only have I read it, I used to give it away to people. Every employee at Khan Academy used to get The Diamond Age. And you're referring to it because it takes place in this neo-Victorian, not-too-far-off future in China. And this member of this neo-nobility gets this AI tablet app for his granddaughter to educate her, The Young Ladies Illustrated Primer. And it gets bootlegged, and it gets in the hands of 200,000 orphan girls who live in barges, and then they essentially just take over. And so I've always used Young Ladies Illustrated Primer to our team at Khan Academy. It's like, this is what we're hoping to build in the long run. And I never thought we were going

• 22:21Speaker 01

  to fully build it. I thought we were going to be able to approximate it. But already, Khanmigo can do some things that are maybe even beyond what The Young Ladies Illustrated Primer did, where you can talk to Don Quixote in any language that you want. You can get into debates with it, et cetera, et cetera. And I think in the next three to five years, almost everything that Neil Stephenson imagined, I think he wrote the book in 1994, I think is actually going to be a reality, which I didn't think was going to happen in my lifetime.

• 22:52Speaker 02

  How does that impact the structure of the actual school? So say that you think ahead five or 10 years, and you're in the context of an advanced K through 12. Maybe it's Khan Labs, maybe it's something else. What happens, or what changes, or how do you interact with the tutor versus the classroom environment? And I know it's hard to predict the future, but I'm sure you've thought a lot about this stuff.

• 23:10Speaker 01

  I think, big picture, at a place like Khan Lab School, where even before generative AI, we've been, and at Khan World School, actually, we just documented, we were almost embarrassed to say the results. Steve Levitt of Freakonomics fame, he actually looked at it himself, because he was like, is this real? And he's like, oh, it's real. I mean, these kids are learning about three to four times faster. Not 3% faster, or 30% faster, three to four times faster.

• 23:34Sarah Guo?

  Wow.

• 23:35Speaker 01

  And at Khan Lab School, we've consistently seen it at least 1.5 to 2 grade levels in a year. And this is before generative AI. So I think in some of these settings, we haven't had the same issues that in the traditional school system. Kids are getting about point seven grade levels per year. Now, I think if you go to more affluent areas, areas where parents are college educated and they can afford more supports for their kids, you're probably seeing at least a grade level or maybe closer to what we're seeing at Khan Lab School or like 1.5, two grade levels a year. But I think in environments where basic fluency has been less of an issue, you're now going to free up more time for teachers, so that they're going to spend less time doing things

• 24:20Speaker 01

  like grading and lesson. Most teachers spend about half their time doing that kind of stuff. So that that gives them more energy for themselves, but also more energy just be with the students, which I think is really powerful. Students are going to feel a lot more supported. And I think if you if you fast forward three to five years, kids are going to feel very supported in that kind of that basic fluency and that core academic. But I think it's going to be like the magic school bus. Like, you're going to be able to put on your like Apple visors or whatever, that you're going to be able to like go on with the with the teacher and like, you know, jump into the the circulatory system or time machine yourself to ancient Greece and have a debate with Socrates. And you're going to be like, I think we're going to have, and you're going to have, you know, one of the things we do at Khan Lab School is just try to create more time and

• 25:05Speaker 01

  space for students' passions. And so you're going to have a world where a middle school student is going to be able to create like studio quality techno music or, you know, a Lord of the Rings quality, epic, epic science fiction movie or fantasy movie. So so I actually think it's going to be very exciting. And I think if you go into, let's call it more mainstream classrooms, I think we're going to go a lot further towards solving the just the core fluency issue and not just in math, but in writing as well, because that's the other thing. The main problem with writing, it's very resource intensive. It's hard to get feedback. It's very expensive for the teacher to give feedback for 40 kids. It's mind numbing, frankly. Kids have to wait a week to get that feedback. Then they may or may not get a chance to iterate on it. And so most people aren't getting much practice.

• 25:48Speaker 01

  And that's why you're seeing a big problem with writing on top of the problem with math. So I feel confident. I mean, already this year in Newark, New Jersey, which, you know, is not a special case school district. The North Ward had just started using Khan Academy pretty intensively in November. This is pre-generative AI. As of June, seventy percent of those six thousand kids use Khan Academy at a level that's associated with pretty profound efficacy gains. We're still waiting on those test scores, but I think we're going to show for the first time in like US history or recent history, a pretty large acceleration of a large urban school district in math. And that's pre-generative AI. So I'm feeling actually very optimistic about that.

• 26:29Speaker 01

  And then, yes, especially for those students whose families might not get an opportunity to, you know, put them in AI summer camps like we do with our kids or, you know, take trips or, you know, I think AI is going to bring the world to them in really, in really powerful ways. They're going to be able to practice their debating skills. They're going to be able to connect things to other ideas. They're going to be able to create pretty profound things. They're going to be able to also go on that magic school bus and it won't, it's not going to be something that requires a ton of resources. So I'm, I'm overall hopeful, you know, there's always forces of friction and cynicism and et cetera, et cetera. But because teachers have a lot to gain here, it's going to save them a lot of time and energy. I think you're going to see faster uptake of this than you see of other, other solutions.

• 27:13Speaker 02

  There's the old saying that the future is here is just not equally distributed. And with things where you have sort of strong structure in place, like the education system, is this a five-year transformation, a 20-year transformation, afford it? Like, how long will it take for schools to adopt this despite all the incentives to do so?

• 27:30Speaker 01

  Yeah, I want not to be too aggressive in my predictions, but I think it will be safe to say that in five years, almost all teachers will be using generative AI for lesson planning, grading, progress reports, rubric creation. I think that is already, I, that might happen by, by like late next year, honestly. So that's, that's going to happen fast. So I think in a world where almost all teachers are already using the tool. I think having a generative AI tutor supporting the student, I think that's for sure going to be mainstream in three to five years. I think, you know, because ChatGPT introduced the emergency around cheating, like it's not like it's a nice to have. It's like a problem. It's like broke the education system.

• 28:15Speaker 01

  People need a solution. And so I think that has to be fixed in the, and I think it's actually going to be fixed with generative AI as well. That's, that has to come into play in the next year or two. So, you know, five years in the future, you might, when you just visit a classroom, you might not superficially see a lot of differences. But I think when you start double clicking on it and you realize that, wow, teachers have a lot more time now for student facing things. A lot fewer students are stuck. There's a lot more space for differentiation now. I think it's, that that will be transformative. And I think you're going to see it in student engagement, motivation and, and in things like test scores.

• 28:49Sarah Guo?

  So in this novel, The Diamond Age, like one of the things that happens is because these tutors are so powerful and they have memory, right? Some of the things you're working on and they're engaging because they have narratives and the dragons that's like teaching you calculus with, and physics with like your environment. There's a, there's a lot of engagement with that and much less interest in some cases in sort of like group schooling settings. Like, can you talk a little bit about how you think about that? Cause you also started Schoolhouse, schoolhouse.world with our mutual friend, Shashir, founder of Coda. And, and, you know, this is like small groups online and believing that there's still the importance of this. Can you, can you talk about sort of how those pieces fit together and what the motivation was there?

• 29:32Speaker 01

  Yeah, absolutely. And, and just a little bit more detail on Schoolhouse. We started during the pandemic, you know, that was a moment where like Khan Academy's usage went up three X. And it was clear that it was great if people are using Khan Academy, but they, they needed, everyone was isolated and they needed more human support as well. And I'm always thinking about, okay, how can you give something at scale that used to feel really expensive? And tutoring, live human tutoring, for the most part is expensive. But I was, well, what if you could leverage volunteerism? And so it was a bit of a, you know, optimistic idea that there would probably be a lot of people who want to do, you know, what I did with Nadia, that a lot of people would like to, love to do that. And so we set up schoolhouse.world as a way, you know, it's, it's another nonprofit, a sister nonprofit to Khan Academy. Its mission is to connect the world through learning.

• 30:17Speaker 01

  And the whole idea is, is that you get free tutoring and who gives the tutoring? It's, it's volunteers. And a lot of those volunteers could be near peers. They could be high school students, college students, but some of them also are teachers who teach at a fancy private school and want to give back. They might be a retired professor, et cetera, et cetera. It's, it's gone well. I mean, it's definitely at a smaller scale than Khan Academy. It's on the order, Schoolhouse is on the order of about 10,000 folks per month, but it's doing some pretty powerful things. Already a large fraction of the kids who take any given SAT are already using Schoolhouse to get live human tutoring. Khan Academy had been the official, has been the official practice for kind of the asynchronous tutoring aspect of the SAT or the, the individual practice, but now you can get, you can join

• 31:00Speaker 01

  cohorts and be, be, become part of that. And, you know, I think the power of Schoolhouse is as powerful as it is to get help academically. I think that's social connection, especially social connection across borders, et cetera, et cetera, hugely powerful. Same thing with Khan Lab School, Khan World School. We think it's all about the human to human connection. Khan World School, which is an online high school, but its anchor is a Socratic seminar where kids are getting together almost every day and they're debating things. Now, it's not five hours on Zoom. We think that's wrong. That's mind numbing. That was what was wrong with pandemic schooling. Like that's, that's literally going to ruin kids' eyes and like ruin them. But, you know, an hour, two hours, but you also don't want completely asynchronous. You want some interaction.

• 31:43Speaker 01

  And at Khan Lab School, even though people think it's Khan Academy, it's downstairs from that, of course, yes, we do use Khan Academy, but if anyone visits, you're going to see far more collaboration and interaction amongst these students and far more talking amongst the students than you will see pretty much at any other school. And I've become a believer more and more and more, yes, it is important to improve test scores, et cetera, et cetera, and we are doing that. But that's social, emotional, that character development, that ability. Ability to collaborate with other folks. That's arguably the more, the more important thing. And if I, I almost didn't do it by design out the gate, but if I'm, one of the things I'm proudest about it at all of these projects we're doing is that element is that, that the collaborative element, the connected element.

• 32:25Speaker 01

  You know, whenever we bring in new faculty to Khan Lab School, they're, they're always like, it's weird how collaborative these kids are. Even when you get into high school, even in Silicon Valley in high school, where a lot of other schools get quite competitive, secretly quite competitive, but everyone knows it. That's not happening at KLS. The kids are genuinely collaborative. I think part of that is more time and space to pursue your passions. I think a lot of part of it is mastery learning. It isn't about, you know, elbowing out your peer. It's about, hey, everyone here should learn it. And I think a lot of it's about the peer support, which we were trying to scale at Schoolhouse, but it's always been part of Khan Lab School.

• 32:59Speaker 02

  How did the relationship with OpenAI and GPT-4 start and the, the origins of Konmiga? Was that something you came up with and you went to OpenAI? Did they come to you? Like, I'm a little bit curious about the origins of both the product and the relationship.

• 33:11Speaker 01

  Yeah, you know, this story is going to be just another one of the, the data points of behind the benevolent alien theory.

• 33:17Speaker 01

  It was about a year ago, end of summer of 2022. I got an email from Greg Brockman and Sam Altman, who I was familiar with. I didn't, I didn't know super well, saying, hey, we're working on our newest model. We'd love to, to talk to you about it. And I was skeptical whether it would have any real strategic bearing on what we do at Khan Academy. But just as someone who's followed the industry and has always been fascinated by this, you know, I, I knew I had been following GPT-2, GPT-3, DALL-E. So I was like, oh, are they talking about GPT-4? That's kind of cool if I get to be on the early, get to see what's happening there. So we had that meeting and, and they hadn't even finished the first training run of GPT-4.

• 34:02Speaker 01

  But they said, you know, we think it's going to be done in about two weeks. And we think this is going to be the model that really wakes up people to the power of generative AI. We want, two reasons why they were talking to us. The first was. We want to to launch with some social positive use cases. To us, the most obvious ones are education and healthcare. And y'all are the first call we're making. Like, and we want to do it with organizations that people trust and are going to like, do it in a thoughtful way. And so, as far as I know, we were literally the first people they called. And, and, and I said, well, you know, I'm, I'm up for looking at it. Oh, then the second reason why they want to chat with us is it turns out that Bill Gates, when he saw GPT-3, he's like, oh, this is cool.

• 34:44Speaker 01

  But we all know GPT-3 really didn't have a good handle on knowledge. And he, he told the OpenAI team, he's like, I'll be impressed if this could pass the AP Biology exam. He literally said that. And so I think part of the reason they also reached out to us is like, y'all have a lot of AP Biology items. Could we test this?

• 35:02Speaker 01

  And we're like, yeah, sure. I was a little bit of like, what's in it for us? But like, yeah, sure, that's kind of, I, I mean, I knew what was in it for me. I just wanted to see what this was. So about two or three weeks later, they said, oh, you know, can we give you a demo? I was like, yeah, sure. So it was me, my chief learning officer. We get in and they had a AP Bio question. And they said, Sal, what's the answer to this? And I was like, okay, it was about osmosis, I'm like, it's C. And they said, all right, let's see what it said. And you know, it said it's C. I'm like, oh, that's, that's, maybe it's like, I was like, ask it to explain why it's C. And I know everyone's used to chat GPT and even GPT-4 now, but this was before anyone had seen any of this stuff. And I asked it to explain and it explained it, explained it quite well. And then I said, explain why the other choices aren't right. Explain that quite well. I said, and that's when I started to get the goosebumps.

• 35:46Speaker 01

  And I, I said, create five more questions like this one. It did it. And that's when I'm like, okay, this, this is, this is a game changer. And then they're like, would you like access? I'm like, yep, please give us access. So that, that weekend, I literally couldn't sleep. And, you know, my, myself, our chief technology, our chief learning officer, we're literally slacking each other like four in the morning with like, things that we had gotten it to do that like, seemed like science fiction. But most notable was act taking on personas, like acting as a tutor, being able to engage in a Socratic dialogue. And we also were discovering some math errors and stuff like that. We were surfacing that to OpenAI. It turns out that they did have some mistakes in their training data, which we, you know, so that was our service to humanity above and beyond the CoreCon Academy work. They realized like, oh yeah, there's some.

• 36:30Speaker 01

  But then we started to say like, look, if we can, if we can figure out ways to put right guardrails around this, make sure there's safety, security, we can mitigate the math errors, we can mitigate the hallucinations, this is a game changer. So that's when we started working. ChatGPT comes out a few months later. I slack Greg Brockman. I'm like, hey, I thought we were under, under, under, like, what are you launching? And like, no, this is just on GPT three point five, but everyone's, you know, seems to be taken by this, even though what we're working on for was so much better. I was bummed at first, but I was like, oh, in hindsight, it was a good thing because it threw out this very imperfect thing that was not designed for education. The education world took it the hardest because kids are going to cheat now, et cetera, et cetera. And then when we launched

• 37:10Speaker 01

  with GPT four on March 15th of 2023, it almost allowed us to directly address all of the concerns that people were already having around ChatGPT. And so that's, yeah, that's how we got to where we got to.

• 37:22Speaker 02

  That's really cool. How did you end up thinking about safety? You mentioned, for example, you wanted to ensure there was safety around the tool and probably certain answers or certain questions could necessarily be asked. Like, how did you end up approaching that?

• 37:33Speaker 01

  So step one, we, as soon as we saw it, we're like, oh, this could be used to cheat. And so like, okay, how do you help without cheating? And so that's when we said, we started putting guardrails in the, the system prompts, so to speak, where it says, don't give the answer, make sure the student does the bulk of the work. It's okay to help them, but primarily ask leading questions. You know, we have, we have a lot. We've tested a lot of prompts and to just get to that sweet spot and we're constantly tweaking it. So that's the first guardrail. Second guardrail was, you know, this is a new environment and we don't know where kids are going to take this. So let's make it transparent to other stakeholders, what's going on. So we said, okay, we're going to record every

• 38:14Speaker 01

  interaction with the AI and make it accessible to parents and teachers if the student is under 18. And, you know, we can debate that for, you know, there's good arguments why older students, there might be some constructive stuff that they could use the AI for that's probably not good to share with parents. But anyway, that's it. We took the most conservative stance out the gate, especially for under 18 there. And then the, the next one is, you know, we have a second AI that's moderating the conversations. And if it looks like it's going into a unhealthy place, not only will it kind of shut down that, that, that thread of the conversation, but it will actively notify parents and teachers. And so that's, those are, and then we, we are trying this best to do, to do digital literacy as well to students to recognize like, look, Here's what you could use generative AI for.

• 38:58Speaker 01

  Here's where you should be skeptical of generative AI, et cetera. And we we are feeling increasingly confident that that combination and and also none of the interactions, and this is to credit to OpenAI. None of the interactions are being used to train the AI. And and arguably, there could be benefit in the future for that to happen. But because this is all new frontier and it's all very sensitive. We and OpenAI just said, yeah, let's just keep this as kosher as possible. Let's just make this like, you know, as safe as possible.

• 39:28Sarah Guo?

  If if you think about, you know, going beyond Khan Academy, you've also spent 15 years generally thinking about education. Do you have a prediction for what happens to like university with all of these technologies?

• 39:39Speaker 01

  Yeah, I I think I I don't think generative AI dramatically changes what was already going to happen to universities. I I think it's not news to anyone that the return on investment for for university, especially if you have to take on debt, is is is mixed. Depending on on who you are and what you do with it. Obviously, we have a student debt crisis. And even if we were able to forgive some, it's not going to solve it for the for the next generation of students. And so universities are bloated with cost. They're not particularly good at, you know, either informing or preparing students for, you know, in certain cases they are. And in other cases, they're not of of preparing students for what they need. I think you're going to have a world where the very elite universities. People people join those.

• 40:23Speaker 01

  Yes, it's it's it's almost like you're joining a caste, you're joining a a club. So I think they're going to be fine. I think that, you know, whether or not they can justify the ROI, they're going to be fine. I think now there's some interesting things that are happening around, you know, legacy admissions, et cetera, et cetera. So that'll that'll be interesting. I think the community colleges at the other end of the spectrum, they're very flexible and they are very good at thinking about what are the needs? Like, OK, are there are there are there industry certifications that could be useful? OK, we can support students for that. Are there other types of, you know, mini badges that could help someone get a job? We can do that. So I think the community colleges that are low cost and nimble and are able to adapt to the changing environment. I actually think they're going to do fine, too.

• 41:08Speaker 01

  I think that vast middle of universities, especially the ones that are charging, you know, sixty, seventy thousand dollars when you include room and board per year. And a lot of the kids are graduating with debt that is very hard to even cancel with bankruptcy. And they're not, they're not getting the pay to be able to, you know, I, I, I tell everyone, you know, all this student debt stuff, they should be holding the universities accountable. Like, where did that money go? Like, who got that money, right? Someone lent it to the student and then that went someplace. It went to the university. They took the money. Thank you very much. And now they're just kind of hiding when you have trillion dollars or two trillion dollars of debt and it's ruining these kids' lives. I think if you, if you, if you just said, you know what, the, the universities are accountable for 10% of that, their, their tune will change overnight.

• 41:54Speaker 01

  Overnight, they will start informing students much more about their career outlook. If you were to like, yeah, not everyone's going to become a museum curator. Let me just tell you that right now. And by the way, the kid who you heard about, who, who just became a museum curator, majoring in this major, it turns out that their grandmother like built the museum, right? Like, it's, it's a hard job to get. So don't think that this is going to lead there. Just be informed. And I think the universities, if they were on the hook for even a small fraction of that debt, would almost like not let you major in something unless they felt like they were, you were going to pay back the debt. So I think you're gonna, you know, there's a reckoning there. The university cost can't get bloated forever. I think some of what's going on around admissions, the scrutiny, I think it's healthy.

• 42:37Speaker 01

  And, you know, I'm a big advocate of a competency based world, where it's not how long you sat in a chair or, you know, how long you threw frisbees and went to frat parties. It's, it's, it's what do you know and what do you not know? And so we are working and I know others are working to create competency signals, credentials, that if you do it, employers will recognize it. And it doesn't matter if you did it when you were 16 or when you were 36, you're now qualified for that job. I think that's where employers want to be. And I think it's not going to, quote, disrupt college, but it's going to give college an alternative, which I think is very healthy.

• 43:12Speaker 02

  I think one of the big shifts that's coming as well is just the type of skills that are going to be relevant in a more and more AI-related world. So if you think ahead 10 years, 20 years, 30 years, et cetera, are there specific things that, for example, you're encouraging your kids to learn or that you think that people should really be focused on as they think ahead for their own children, you know, in terms of the skill sets that will be relevant in a generative AI world?

• 43:35Sarah Guo?

  A lot, and I both have like preschool kindergarten aged kids, so these are active decisions.

• 43:40Speaker 01

  No, it's a good, good, good question. It is something I think a lot about. I, you know, I think there's two, there's at least two metaphors that can help us think about that problem. One metaphor could be, you know, we can go back to the 19th century. And I'm not just going to talk, you know, a lot of people made the metaphor of like steam engines and horses and all of that, but I actually think the camera is the best metaphor. Because being an artist was a real thing. It was really a technical field. You were a portrait painter. And like the best artists, you know, you would study for years to be as accurate to reality as you could, and they could like, can look at how the light moves and all of that. And all of a sudden, a camera comes out and all of a sudden, the artist starts saying, okay, that's the end of art. Like this thing can capture reality better than, you know, anyone can.

• 44:23Speaker 01

  But then very quickly, people realize that, no, in some ways, this liberates the artist. That this, this allows us to think about, I mean, it's not a coincidence that the impressionist movement kind of coincided with the advent of the camera. People start saying, no, it's not about capturing reality. That's art should be about the impression. Art should be about the feeling it conjures. And it actually, all of a sudden, led to an explosion of like what art is. It allowed art to come out of that trap of just painting nobility and like these grand scenes and start getting into things that evoke and really challenge us. I think the same analogy, you know, people are saying, oh crap, this thing can write pretty well. This thing can code pretty well. This thing can create movies, create images pretty well. So what that tells me is it kind of liberates the creator to move beyond that.

• 45:08Speaker 01

  Someone who can elevate and integrate and manage these tools. The other metaphor I would say is imagine if, just as we're saying with Conmigo, every teacher now has a teaching assistant. So now every coder is not going to have an army of coding assistants. Every writer has a writing assistant. So people are going to have to move into the managerial ranks quite quickly and figure out, okay, instead of being a coder, I'm now an architect. Instead of being a writer, I'm now an editor. And in order to do that, you have to know those skills really, really, really well, arguably better than previous generations. So I think it's even more important that people get not just adequate, but excellent at writing, communicating, coding.

• 45:51Speaker 01

  I think creativity, because there's so many outlets, someone who's creative, who leverages these tools is going to be unstoppable. They're going to have like godlike power. And that's good. I know there's, you know, there's these debates with the Screen Actors Guild, and it's a very sensitive issue. By the way, I think just as YouTube, there's a lot of talent that would not have been discovered without YouTube. You would not know about Justin Bieber. You would not know about me. Yes, I just put myself in the same category, very self-aggrandizing. But, you know, think about how many movies we've all seen that had like a hundred million dollar budget and were horrible, like horrible waste of time movies for a hundred million dollars. You're now going to have a ton of people be able to create movies with similar special effects and acting and screenplays and everything. And a lot of them are going to be horrible. And even more of them are going to be horrible.

• 46:33Speaker 01

  But every now and then, we're going to discover the Justin Bieber of movie making. And, and like, that's an amazing movie. And that's some kid, he or she did it on a budget of a thousand dollars, not a hundred million dollars. So how do we foster that? How do we give more time and space for that? But I think if you have kids, it's kind of very traditional advice. They should just get really good at reading, writing, math. I've never been a subscriber of like, oh, there's Google. You can search stuff. You don't have to learn anything anymore. Nope. The more that you have rapid access to knowledge in your head, the more that you are fluent with your mathematics, that you have information and concepts at the tip of your fingers, these tools will accelerate you more than anyone else. And the kid that has to get the calculator for three times seven or has to go to Google to figure out when World War II was,

• 47:17Speaker 01

  the world is going to pass them by.

• 47:20Speaker 02

  Amazing. I think that's a really great note in terms of an optimistic view of, you know, it's the same basic skills. It'll be important 20 years from now and kids really just need to learn the basics. And you've provided such a great platform to achieve that by. So thank you for all the work you've been doing.

• 47:33Speaker 01

  Oh, thanks for having me.