Founder Eric Steinberger on Magic’s Counterintuitive Approach to Pursuing AGI

There’s a new archetype in Silicon Valley, the AI researcher turned founder. Instead of tinkering in a garage they write papers that earn them the right to collaborate with cutting-edge labs until they break out and start their own. This is the story of wunderkind Eric Steinberger, the founder and CEO of Magic.dev. Eric came to programming through his obsession with AI and caught the attention of DeepMind researchers as a high school student. In 2022 he realized that AGI was closer than he had previously thought and started Magic to automate the software engineering necessary to get there. Among his counterintuitive ideas are the need to train proprietary large models, that value will not accrue in the application layer and that the best agents will manage themselves. Eric also talks about Magic’s recent 100M token context window model and the HashHop eval they’re open sourcing. Hosted by: Sonya Huang, Sequoia Capital Mentioned in this episode: David Silver: DeepMind researcher that led the AlphaGo team Johannes Heinrich: a PhD student of Silver’s and DeepMind researcher who mentored Eric as a highschooler Reinforcement Learning from Self-Play in Imperfect-Information Games: Johannes’s dissertation that inspired Eric Noam Brown: DeepMind, Meta and now OpenAI reinforcement learning researcher who eventually collaborated with Eric and brought him to FAIR ClimateScience: NGO that Eric co-founded in 2019 while a university student Noam Shazeer: One of the original Transformers researchers at Google and founder of Charater.ai DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker: the first AI paper Eric ever tried to deeply understand LTM-2-mini: Magic’s first 100M token context model, build using the HashHop eval (now available open source) 00:00 - Introduction 01:39 - Vienna-born wunderkind 04:56 - Working with Noam Brown 8:00 - “I can do two things. I cannot do three.” 10:37 - AGI to-do list 13:27 - Advice for young researchers 20:35 - Reading every paper voraciously 23:06 - The army of Noams 26:46 - The leaps still needed in research 29:59 - What is Magic? 36:12 - Competing against the 800-pound gorillas 38:21 - Ideal team size for researchers 40:10 - AI that feels like a colleague 44:30 - Lightning round 47:50 - Bonus round: 200M token context announcement

• 0:00Speaker 00

  Thing that remains to be solved is general domain long horizon reliability. And I think you need inference time compute, test and compute for that. When you try to prove a new theorem in math, or when you're writing a large software program, or when you're writing an essay of reasonable complexity, you usually wouldn't write a token by token. You'd want to think quite hard about some of those tokens. And finding ways to spend not 1X or 2X or 10X, but a million X the resources on that token in a productive way, I think is really important. That is probably the last big problem.

• 1:00Speaker 01

  Hi and welcome to Training Data. I'm delighted to share today's episode with Eric Steinberger, founder and CEO of Magic. Eric has an epic backstory as a researcher, having caught the attention of Noam Brown and becoming one of his research collaborators while still a student in high school. Eric is known for his exquisite research taste, as well as his big ambition to build an AI software engineer. We're excited to ask Eric about what it takes to build a full stack company in AI, his ambitions for Magic, and what separates a good AI researcher from a legendary one.

• 1:34Speaker 01

  Eric, welcome to the show. Thank you so much for joining us.

• 1:37Speaker 00

  Thank you for having me, Sunny.

• 1:38Speaker 01

  Okay, so let's start with who's Eric. You're a Vienna-born Wunderkind, whose early passion for math turned into a, I think what you described as a full-fledged obsession with AI by age 14. Take us back to age 14, Eric. Like, what were you up to? How did you become so obsessed with AI?

• 1:55Speaker 00

  Uh, thank you, Sonia. I, I, I think I just had my midlife crisis when I was fourteen. And um, I, I was just looking for something meaningful to do. Spent about a year looking at physics, math, bio, medicine, just anything really that seemed valuable to the world. And um, at some point bumped into just simply the idea of AI. It hadn't sort of occurred to me until then. Uh, and if you could just build a system, a computer system that could do all this other stuff for me, like, great. Like, I don't have to decide. So uh, it felt like my decision paralysis was uh, sort of resolved. And I, it, it was this weird moment where I could just see the next thirty years of my life unfold in front of me. And I was like, okay, this is clearly what's gonna happen. Like, I have to, I have to do this.

• 2:37Speaker 00

  And uh, that was, it was quite, it was quite nice. Um, I like predictability. So uh, it, it was great to like know what the world will look like.

• 2:45Speaker 01

  And you started loving math. Like, why, why AI then?

• 2:49Speaker 00

  I think I'm naturally attracted to math. It's just what my brain sort of gravitates to. Uh, AI just seems useful. Uh, uh, the thing that's most important to me is just what is useful for humanity and the world. And uh, math is nice, but not useful at, at some point. Like, you know, seventeen dimensional spheres are, are probably not going to be the best career choice, uh, if you want to be useful. So uh, it seemed like something that I could get good at, uh, but also just the most important thing ever. Uh, and so the, it was a very clear choice. AI just, just like, as, uh, it was clear ten years ago. It's just, it wasn't close. And now it's close and clear.

• 3:28Speaker 01

  Can you tell the story of how you got to Ferrer? I think it is such an epic story.

• 3:32Speaker 00

  Sure. I, I, I mean, so when I started with, at fourteen, I had, I didn't really know how to program. I, I didn't get into programming out of curiosity about computers. I just wanted to uh, solve AI, basically. So after a couple of years of just warming up on my own, I reached out to one of David Silver's PhD students, who's the AlphaGo DeepMind co-founder. And this PhD student, I guess at that point he was a graduate and sort of worked at DeepMind. I asked him if he could like spend a year with me, just every two weeks bashing my work. Sort of trying to do so sort of like super speed up, mini kind of PhD-like experience where I could just learn how to do research.

• 4:17Speaker 00

  And I sent him this like giant email, you could print it out, I don't know how many pages it would be, would it be a lot of pages? Where I was basically just saying like, I want to build this algorithm you made in your PhD. So I want to beat this algorithm you made in your PhD. Here like, here's a list of 10 ideas. I don't know if they're gonna work and I think I need your help to figure that out. And then over a year, we eventually got there. And he was like, his name is Johannes. Johannes was kind enough to just bash me every two weeks, roughly. And yeah, it was brutal dude. Like, just like, because I was like, he held me to the standard, you know? And I was like, like, don't be nice just because I'm in high school.

• 4:54Speaker 01

  And you were in high school.

• 4:54Speaker 00

  Yeah, I was in high school. And then when we were done, I just graduated high school when I finished. So this is when I finished the project that I was trying to get to with this. And then Noam Brown, who is obviously one of the best RL researchers in the world, reached out because he had worked on something similar, it turns out. And we sort of had like some ideas that were very similar and some ideas that were a little different. And so we just both published this and he reached out and then I got to work with Noam Brown for two years, which was great. And then that continued and so I got bashed for another year.

• 5:26Speaker 01

  You were a high schooler, he was Noam Brown.

• 5:29Speaker 00

  Well, I mean, he published a paper called Deep Counterfactual Regret Minimization. And I published a paper called Single Deep Counterfactual Regret Minimization. And mine beat his by a little bit and.

• 5:38Speaker 01

  So you won up Noam Brown as a high schooler?

• 5:41Speaker 00

  I think I just graduated. And I also, it took him like three months to write this paper and it took me a couple of years.

• 5:48Speaker 00

  It, yeah, I mean, slightly. I'm sure he like would have come up with this the next day. But like the sort of the gap between the two things. But it was, it was just, yeah, it was like obsession is the right word. I just, I do things like 100% and.

• 6:05Speaker 00

  Yeah, so that was a lot of fun.

• 6:09Speaker 00

  Kept working in our all with Noam Brown for a while. Yeah, so that's how I got to Fair. Noam Brown worked at Fair at the time and he reached out. I was actually, I was at university then. And basically just worked part time as a researcher at Fair while studying. And anyway, so that was it.

• 6:28Speaker 01

  That's awesome.

• 6:30Speaker 00

  It was a lot of fun. Noam is great. Like the brainstorm ping pong sessions with Noam Brown, dude. There's like nothing like this, where you would just think, there's like this problem. And it's sort of like, you know, maybe you would like start a six month research. Like, no, Noam and I would get on a call and it was like, we just discuss it and it's done. That was so great.

• 6:46Speaker 01

  I love that. I love that. What makes him so great as a researcher?

• 6:50Speaker 00

  I think it's a number of things. As a researcher generally, from more from a meta level, he is fantastic at picking the right problems. And then spending a long time just grinding to make it better, better, better, better. He's very good at the whole like compounding thing in research. Also making bets that aren't obviously the right bets when he makes them. He makes them earlier, I suppose, and making those like differently. So he's generally very good at picking problems and then attacking them consistently. He's also just very smart. I guess that helps.

• 7:24Speaker 00

  He works really hard. He used to do 100 hour weeks during his PhD. I don't know if he still does them, but he used to, he used to work really, really hard during his PhD.

• 7:32Speaker 01

  I imagine he still is. Okay, so Noam arranged for you to become a researcher at Fair while you were still a university student. If I'm correct. And you were juggling both.

• 7:41Speaker 00

  That was my first semester, I think, or something.

• 7:44Speaker 01

  So you were juggling that, you were juggling being a collaborator to Noam at Fair. And then you became obsessed with yet another problem, climate change. And I actually started an NGO that is incredibly popular, climate science. So you just didn't have enough on your plate.

• 7:59Speaker 00

  That was actually too crazy. That's when I dropped out. I was like, this is crazy. This is too much. I can do two things. I cannot do three. It was like my conclusion after, after doing like, I did three months of that. That was terrible. That was fucking awful, doing like all three things. Cause it's just like, you're just can't do well at three things. I mean, Elon can, but maybe I'll learn it in 10 years. But the, I couldn't at the time. So I dropped out at the time. But yeah, so I started NGO. I generally, I just think like charity stuff is awesome and hugely under. Appreciate it like it's it's sort of like super high status to start a startup. But I I think it should be equally cool to start a charity. You're like helping the world in other ways.

• 8:41Speaker 00

  And so, yeah, I mean, we we started it as a it's a nonprofit, but we started it like a startup. People were working insanely hard. We had clear objectives. It was, you know, a software product effectively. It's it. It was much more similar to a startup with the exception that there was no money in, no money out.

• 9:00Speaker 00

  It was just very weird, but the yeah, it's mostly volunteer driven, or I guess is I just no longer run it. But yeah, so that that was that was an interesting experience. You'd think I would like learn transfer a lot from running a quote unquote company to running a quote unquote company now. But they are so different that I could like there was like no transfer at all between climate science and magic. Like a thousand volunteers, twenty hardcore engineers, no money at all. I can't tell you how much we raised because it's not announced yet, but the, you know, giant. It it's it's like completely different in every imaginable way. So, but yeah, it was a lot of fun.

• 9:35Speaker 01

  So Eric, climate science became an incredibly successful nonprofit. And it wasn't just any nonprofit. What made you decide to kind of hand over the reins and hand over the torch on that and and go start a company in AI?

• 9:49Speaker 00

  I just thought AGI was further away when we started it at all. I I would never have started anything else if I thought AGI was so close. And once I realized it is, there was just like no other. I I was, I mean, I I my initial thing was always AI. That's what I did as a kid.

• 10:06Speaker 01

  Yeah.

• 10:07Speaker 00

  I care about various issues in the world, but it's none. None of them are my like unique calling in any way. I just, you know, I hopefully be in a position to donate a bunch of money and whatever. But the thing I care about fundamentally is AGI and.

• 10:24Speaker 00

  It was like, oh, damn it. This is not 20 years away. So I have been running around with this AGI to-do list, which is somewhat of a meme internally. Because if I just get going through it, then we're trying to fix all these problems. You have seen it. Yes, we showed it to you. And I've been running around with a version of this. It was actually like in 2017 or so, I was still in high school. I don't know why, but some conference invited me to present my AGI to-do list. It was wrong at the time. I was also sure it was wrong, but at some point, so there was one thing I just couldn't at all figure out. And I don't like blue sky research in the sense of like just staring at a wall and trying to like figure out what the right question is. I really like to have the question and then look for the right answer when starting an intense project.

• 11:09Speaker 00

  Because you need to know which direction you run in to really plan for it. And many things seemed clear, but it seemed completely unclear how to make these models reason in the general domain. And that became more clear with language models, especially code models. And so, yeah, when I saw just some of these early, some of the early results in this space, I was like, okay, I know all this stuff from the RL world. I have a bunch of other thoughts. This seems great. Like, we should just take LMs and make them do the RL stuff. It's a very simple kind of proposal, but I think that's sort of where, I mean, it makes a lot of sense.

• 11:52Speaker 00

  RL has been doing this for 10 years. It works in restricted domains. If you can make something work in 20 restricted domains and you have something else that works in a general domain. If you can combine them, maybe you get both the X and the Y axis. And then, you know, you have your beautiful top right corner of the matrix. And yeah, so it seemed like it seemed pursuable. And if something, when something becomes, when something as important as AGI becomes an actually executable to do. Obviously, there are still like things to figure out, like, you know, details of the algorithms. How do you make it efficient, et cetera, et cetera. Like, so like we do everything at all. Many, many things to figure out, but the direction was clear. And yeah, so it seemed like the right moment.

• 12:32Speaker 01

  Okay, we're gonna circle back to your AGI to-do list later, because I'm curious about it.

• 12:36Speaker 00

  Sure.

• 12:38Speaker 01

  I want to brag about you for minutes, because I

• 12:39Speaker 00

  It might be wrong, still. I don't know. Until we have AGI, it is a hypothetical AGI to-do list, but we're trying.

• 12:46Speaker 01

  I think the research field is tracking pretty closely to your to-do list. I want to brag about you for a minute. I think you've been incredibly humble about your background. But, you know, as a high school student, you did catch Noam Brown's eye. And as, you know, as one of his colleagues at the fair, you became one of his top collaborators. Not even just one of many, because there's such talented people that work there, but you're one of his top collaborators. And, you know, when I speak to folks that know you, they just say extraordinary things about your capabilities as a researcher, your creativity, your work ethic. As far as I can tell, you work non-stop. I think you texted me at 2:00 a.m. in preparation for this podcast. So I think it's safe to say that No, no, thank you, silent mouth.

• 13:28Speaker 01

  Anyways, I think it's safe to say that you are one of the brightest minds of the current research generation already, and will certainly be one of the legends that people talk about for the next decade. And so with that in mind, I'd love to ask you some questions of advice for aspiring researchers. And so maybe first off, you did it all from a very untraditional background. How did you do it? And like, do you think that, like, what advice would you give to others in your shoes?

• 13:53Speaker 00

  I can only really speak for the sort of profile of goals and person I am. I think I was lucky in the sense that I knew very, very early with Portina, as we said, exactly what I wanted to do with my life. I had no doubt at all.

• 14:08Speaker 00

  And uncertainty can be paralyzing to a lot of people. I also had a very clear sense that I did not at all have a plan B. Like, there was no other path in life that I would have been even, like, remotely above the neutral line on. Like, it had to be build AGI. Everything else is completely irrelevant. So, you know, I understand. So for many people, like, you know, a well-paying job at Google is a great Great achievement. I mean, I would, if it's on AGI, it's fine, but it, it, you get what I mean. Like, I, I just knew that there was nothing else I could do and like be fulfilled in life. I look back when I'm ninety, I'd be happy. So in a way, like burning the boats very, very early

• 14:47Speaker 00

  gives you the opportunity to just be, be like, do, do things that you'd otherwise do ten years later, which again, even like, I, I sucked at the beginning. It took me two months to understand the first paper I tried to understand. Um, like, I was terrible at programming for a long time. But when you're a teenager, you, you, you're like a decent researcher. You don't have to be great. Like, you know, you, that gets you, that gets you, uh, things like a great mentor who then bashes you for a year, uh, which, you know, was, was, um, very, very helpful. Uh, and, and you know, then you get better and you're still young. So your, sort of your brain shapes more easily, maybe. I don't know. So I feel like I benefited a lot from being early. But within that, um,

• 15:28Speaker 00

  I'd say just like, go for the end goal immediately. Doing anything sort of like, oh, I'm gonna do a PhD because I need a PhD to get into, that's all bullshit. Uh, like, you don't. Uh, it's just completely bullshit. Uh, the other thing is like writing five-page, uh, emails to people actually works. Um, writing like, I get a lot of these like two paragraph meh things now and grateful I get emails, but don't, I understand now why people think this stuff doesn't work. It certainly does when you're like, here is how I'm gonna beat your algorithm. Please help me. Uh, five pages, at least in my experience, every single time anyone I, I, I wanted help from in this way, uh, was very helpful. So I suppose be proactive in seeking like the best people in the world to, in a time-efficient

• 16:13Speaker 00

  manner, just distill their brain into yours and be, show them that you can make use of that. If you, if you, if you tell someone who's very good effectively, hey, I'm gonna make good use of this. If you want to coach someone, I would love to be that person. Uh they'll they'll usually do it. They they won't do it for ten people, but if they do it for one or two, that's enough. You just have to to to win that seat, I guess. Um so that that's been really helpful in my experience. Also, just not shying away from learning new things. Um like, I again, I didn't get into programming because I'm curious about computers. I'm I'm not very curious about computers. Uh I I just like AI and the computers are the thing that are necessary to. So it's fun. I enjoy programming now. It's it's great, but I wouldn't have gotten into it, I think, if if it wasn't for AI.

• 16:53Speaker 00

  But still, like, you get into it. So so like, don't be shy. Like, uh we we interview a lot of people who who don't know, um, you know, how how you'd implement an L L M. And it's it's kind of crazy to me. Uh if if you're a researcher and and you you couldn't like implement Sharding or whatever, like, it's it's just insane. Um so so really understanding the whole stack, going down to uh but sort of not bottom up, really top down, like, here's the thing I care about. This is the problem I want to solve. Okay, like, what do I need? What do I need? What do I need? Uh and and then like all the way down. Um and um and they're like much more competent people at kernel programming and hardware design or whatever than I could ever ever dream up to be. But I understand enough of it to do better work at the top of the stack than I could if I didn't. Um so um.

• 17:36Speaker 00

  Uh I think fundamentally, you need to understand to the domain you work in. Um it's also really good to just read everything. Um like, I used to read. I don't know. I don't have a precise number, but just every paper I could, I every paper I would see, basically. And eventually, you get so fast at it that you can, like, that's feasible. And you like build a database in your head of like, oh, this is similar to this thing. It's sort of like, like, but this was sort of my eye-opening moment, or Bill Gates has this interview, like, oh, yeah, if you learn enough things, they're all like similar to each other. So it's not linear. It gets easier. And at that point, I was like, I should read every paper. Um and uh so thanks for the advice, Bill. Uh obviously, it's like through a video. I I never met him, but um. The uh so I just started reading every paper, and that's really, really helpful, because a lot of the best ideas that we had that work really well now at at Magic.

• 18:19Speaker 00

  Were enabled by random things that are like, oh, this it would never work without this random thing that I would have to have come up with in tandem. But because I have this database in my head, I can go like, oh, yeah, like this. And then so often like one good idea is enabled by three other ideas that others have come up with. And and so so it's always just like this composition of stuff. So so having a large database is really helpful. Yeah, and then just never stop, like never, never stop. It can it takes like ages to do good stuff, to do good work. And at any point, there was actually one moment with Johannes, the the the deep brain research scientist who mentored me for a year in high school.

• 18:56Speaker 00

  Where we had a version of the algorithm that wasn't very good. It just it was all right. And and we're thinking like, ah, should we publish this? Like, we were both not really happy about it. And he was like, close to giving up on me. It was like, oh, you know, like, maybe this is just not gonna work like I wouldn't wanna publish this. And so I was like, dude, fuck you. I'm just gonna get this done. And then we got it done like a month or two later. And so I think like I remember going on a walk after this. And just being like, can I do this? Yeah, I don't know if I can do this, but there is no other option. So I just better get it done. And then I went back home and started programming again. It was still like sad that day, but the next day was fine again. They just keep going.

• 19:38Speaker 00

  So I think you have to, I think that's a pretty formative experience because I actually wasn't sure if I could do it. And then we just did it like super soon after. And so I really like haven't felt that insane level of like doubt and pressure since then, which has sort of enabled. I think it's actually beneficial. You have to be realistic, but you don't wanna, if you if you stop, you, yeah, I mean, so anyway. So I think those would be the main things. Also, be like really fucking honest about what you suck at to yourself, because otherwise you're never gonna get good at it. Like, you need to search for the bad things. And instead of like trying, actually, yeah, I think like as a researcher betting on your strengths is good only to the extent that you don't have

• 20:22Speaker 00

  necessary conditions that are completely missing. Like you can't bet on your strengths if they're not enabled. This is again like back to like the engineering thing, for example. So yeah, I don't know. I'm rambling, but stuff like that.

• 20:34Speaker 01

  That's great. That is such a fascinating glimpse into like the inner mind of what it takes to be a great researcher and, you know, behind all the glamour of of training large, large models. And so like, thank you for providing that, that peak. And I'm really glad that you mentioned kind of reading every paper voraciously and having this database in your head. Because one thing I've heard from your collaborators is that your superpower is understanding and absorbing new research. And so I'm curious, do you agree? Like, do you think that is your superpower as a researcher? Or what, what kind of traits do you think have made you such an exceptional researcher?

• 21:06Speaker 00

  So I think initially in the R L work I did, it was synthesis where I would read every paper and I would go like this thing plus this thing plus that thing with this modification. I think that's what they would mean. That, yes, was definitely very helpful. I think is a good way to do research. Generally, there's enough work to for synthesis to be a successful strategy. I guess to an extent still that I tried very hard after it. This is actually in reading, just to bring this up. I realized this and I tried very hard to get better at leaps, like coming up with totally alien crap that just there's no reference for it at all. And because, because ultimately like, so if you take like the transformer, for example, right? Like attention existed.

• 21:45Speaker 00

  The idea of stacking a bunch of L S T M blocks existed and you just had to remove the idea of recurrence, really like that. And like a bunch of a couple of other things that were necessary, right? Residual streams, like the, the residual update and transformers existed from, existed from ResNet. So it's like, it's synthesis. But there is an amount of leap in there to, to, to make it all work. Like it's, it's, it's a little more complex than just taking components and putting them together. You need to come up with new things too. Like, you know, the normalization and the, the, the, the head square, square with the head, which is actually incorrect. But anyway, everyone now knows this.

• 22:21Speaker 00

  But roughly like you should do some of this. So there are some new ideas in there that like really help make it work. But, but it's still a large amount of synthesis. So, so I suppose like most good ideas are synthesis, but there are always some, like in the best ideas, there are some leaps. And, and I'm trying to get better at those, but still it's mostly, I guess, like take. Buy things and throw it away, the stuff that doesn't work in them, make the things work and configure. But yeah, I think, I think some, some stuff needs leaps. But, but yeah, I guess like, no, that's a recipe. Like take L L M's, make them super efficient, block context, giant, throw R L on it, make it all work together. It's still mostly synthesis, I guess. You're right.

• 23:00Speaker 01

  Uh, who do you admire most in the research world? And like, what do you think those folks' superpowers are?

• 23:05Speaker 00

  Uh, Shazir, uh, no, Shazir.

• 23:06Speaker 01

  Yeah.

• 23:07Speaker 00

  Uh, uh, uh, yes. Um, he, uh, what is his superpower?

• 23:14Speaker 00

  Uh, I guess to an extent synthesis. Um, he is, um.

• 23:20Speaker 00

  I mean, he's, he's just the best at synthesis. Uh, he is also great at everything in the stack. Uh, he can, uh, like he, he has no weakness really. Like he could implement the whole thing, uh, on his own if he had to, uh, run it. Uh, he sees the future, I think, in, in a way like it's very unconstrained. And, and that, you know, I, I think I remember sort of crediting, uh, you know, a number of the labs for scaling laws. Uh, this guy made a presentation where he was zipping through, uh, uh, essays or completion or whatever, written by like models of various scale. I was like, this is a hundred billion parameter model. This is a three hundred million parameter model. This is a billion parameter model. This is a five billion parameter model. This is on YouTube somewhere. It's hilarious. And you feel like, what, what if we make this bigger? It's, it's, he's sort of presenting it this hilarious way.

• 24:05Speaker 00

  Um, and then I agree, everyone else is like super scientific about it. Uh, I think Noam is generally just, uh, if I had to put it, he's very, very intuitive. Um, I, I think they're like, a, a lot of labs and, and researchers are sort of, uh, uh, and, and I think it's also, this is not a bad thing. Uh, it's very good, uh, are very evals driven, very mechanical, right? Like it's sort of very empirical in a way. Like Noam, Noam sort of just notes, he was like, God, this, this would work and then it works. Um, so, um.

• 24:33Speaker 00

  I think that's a superpower. That is just extremely great synthesis. He has the larger, he has a larger database. Because he's been around for so long, he just, he literally knows everything. I mean, he invented half of the stuff that everyone's doing now. The, the, uh, he, there's, there's no one who compares. Um, I, I'd say, um, there, there are a number of other people, I guess. Just that you, you, you shouldn't fail. Out of all the people who are sort of the OGs of deep learning, I think you could have hinted there's, deserves by far the most credit just because he like, went through all the bashing when, uh, when, when it was like, oh, this will never work. They're like training like tiny, tiny, tiny, tiny, tiny things. And they're like, this will never work. And he's somewhat stuck with it. I think that's, that level of grit and, and belief in something that is now obviously working,

• 25:19Speaker 00

  deserves a huge amount of credit, uh, whether Capsulelets work or not, whatever.

• 25:25Speaker 01

  Yeah.

• 25:25Speaker 00

  He, he, he like, you know, i- it's, it's incredible to come to something like the, the conclusions that, that the world is at now. And if you look at some of the older papers, a lot of the ideas that are important now were in there already. So, so that's important. Um, and I think he just deserves a ton of credit. Um, I mean, Noam Brown had, um, uh, the, the army of noobs. Noam Brown, uh,

• 25:48Speaker 01

  I should name my kid, I should name my kid Noam is, is what I'm.

• 25:50Speaker 00

  It's a very good strategy. Yeah. Uh, the, um, it's a great strategy, actually. I think a hundred percent of Noam's that are like somewhat popular and are well-known in the research community are great. Yeah, no, he's, he's, he's also amazing. I mean, a number of labs were working on what he was working on during his PhD and he basically soloed the thing. Uh, and, and was like way better and way faster than labs that put 10 people, including some really famous names, uh, on it. And if you just look at the paper track, track record, like,

• 26:20Speaker 00

  I was like, like, here's like the rest of the field and then Noam 100x efficiency. And then here is the rest of the field that Noam does it again. And it's like, it's just consistent. Uh, I think the consistency with which he has just bashed, um, out, uh, these 100x multipliers in, in RL, uh, data efficiency and, and computer efficiency is, is crazy.

• 26:40Speaker 01

  Yeah.

• 26:41Speaker 00

  So, so yeah, Noam, Noam, uh, the Noam army is pretty good.

• 26:45Speaker 01

  I want to go back to this concept of leaps are still needed in research and that you still have this AGI to do list.

• 26:50Speaker 00

  Yes.

• 26:51Speaker 01

  What do you think are the most interesting unsolved problems in AI right now?

• 26:55Speaker 00

  Well, so a lot of it is solved now, I think. And the thing that remains to be solved is general domain, long horizon, reliability, and I think you need inference time compute, test time compute for that. So you'd want,

• 27:11Speaker 00

  when you try to prove a new theorem in math, or when you're writing a large software program, or when you're writing an essay of reasonable complexity, you usually wouldn't write a token by token. You'd want to think quite hard about some of those tokens. And finding ways to spend not 1X or 2X or 10X, but a million X the resources on that token in a productive way, I think is really important. That is probably the last big problem.

• 27:46Speaker 01

  Fascinating, the last one, okay.

• 27:47Speaker 00

  I hope so. I think it's reasonable to think that is the last big answer. I mean, look, over the last few years, all of this other stuff got solved. Like, oh, can we do multimodal things? Can we do long context? Can we do it? All this is gone. Reasonably smart models, you know, they're quite efficient now in terms of cost. I mean, you'd have to be a reality denier to like not see what's coming. I mean, this is just a, this is like a realization to a lot of people in the LLM space, but like RL has been doing this for like ages. So it's just like so clear that you need to do that. Or I mean, maybe you don't need to, maybe you can get away without doing it, which would be insane, but if you don't need to, it will still help you a lot.

• 28:28Speaker 00

  Like, it's just, it's just like, do I want to spend a billion dollars on my pre-training run and then like a little bit more money on inference? Or do I need to spend 10 billion dollars on my pre-training run? I'd rather, you know, like 10 billion would be great, but I'm going to be, I'm going to be, I'm going to prefer spending one.

• 28:41Speaker 01

  And is bringing, is bringing like the LLM and RL worlds together, is that like a research problem? Like, there's still like fundamental, like unsolved science problems? Or is that like a, you know, we have the recipe, we just need to do it and have the compute and the data?

• 28:55Speaker 00

  I think there is no public successful recipe right now. There are good ideas, like, okay, even if you take best of N. Make N large enough, it's sort of, you know, it's not terrible.

• 29:06Speaker 01

  Yeah.

• 29:07Speaker 00

  The, so there are ideas. I don't know that the final idea exists. I think there's just a lot of room up from what is currently known, but, but there are ideas. See, I, I, I think it's very unlikely that even if you stop progress in research, we would not at some point hit something that everyone would agree is AGI. It's just, it's just that I think we can do better and maybe it couldn't solve Remont, right? Maybe it couldn't do all these like super hard things, but it'd be pretty good. And then now like, I'm just curious, like, okay, like, what's the actual, like, if we did all the things, how good will it get? So, so I think there is research left to be done. And there are a lot of ideas floating in the world now. Everyone's sort of working on this.

• 29:50Speaker 00

  But I don't even, I don't know that the current set of ideas is even final. Like it'll be, it'll keep moving, I think.

• 29:58Speaker 01

  Well, let's transition to talking about magic. Maybe just, what, what is magic? You've, you've been very mysterious today. So maybe just.

• 30:05Speaker 00

  Yes.

• 30:06Speaker 01

  Share a little bit about what you're building.

• 30:07Speaker 00

  Yeah, I mean, we're trying to automate software engineering. The, it took us a while to figure out how to train super giant models. That's a pretty interesting engineering challenge. I mean, fundamentally, we're trying to automate software engineering from a, the product side. And, and, and like a subset of that is a model that can build AGI. Cause if it's, if it's like, if it's a great software engineer, then it should be able to do everyone's job at magic, like if we can do everyone else's job. Like that, that would be a subset. So the idea is that you could use this to recursively improve alignment as well as the models themselves in a, in a way that isn't bottlenecked by, by human resources. And you know, there aren't that many Noam Chisiers in the world.

• 30:51Speaker 00

  If, if I had a Noam Chisier in my computer, I could spin up a million of them and maybe alignment would just be solved. I, I, I'm just like, it's like simplifying a ton and very idealistic in the statement. I'm happy to turn this whole thing into a scalable oversight podcast if you'd like. But the.

• 31:08Speaker 00

  The core idea is like, okay, like if I could just clone what we are doing into a computer and then press yes on the money button to run a cluster to do the work we would be doing next week, that would be phenomenal. So I think we sort of were pursuing these two things in tandem where we want to ship something that's a good AI software engineer for people to use. It's like, I think one of the going to be one of the first domains to see higher levels of automation. And I don't like talking around. I don't think the whole assistant pitch is going to last very long once these models are good enough to automate. Like, there's just no way the economy is not going to do that. And I think everyone knows this and they're just like, they just don't like talking about it. It's totally fine. The world, we used to all be farmers. We're not farmers. We're fine.

• 31:50Speaker 00

  Everything prefer, everyone prefers this. I think we'll figure our, figure our way out in the economy if it produces the same or more stuff with less inputs. Like, we should be able to figure that out. That's not a hard problem. And like, from like economic principles, you just have to like figure out distribution. Anyway, but that's what we're trying to do. We're trying to automate software engineering and as a part of that, automate ourselves in doing the work we want to do.

• 32:12Speaker 01

  And so the reason they go after software engineering then is that is the kind of lever that allows you to to automate everything else.

• 32:18Speaker 00

  It's like the MVP of AGI, right? Like the the minimum viable AGI.

• 32:23Speaker 01

  Yeah.

• 32:23Speaker 00

  Because then it creates everything else. Like, yeah, we wouldn't train something like Sora. Sora is great. You know, fantastic. Generic video is awesome. It's just not interesting from an AGI perspective if you believe that models can code themselves.

• 32:38Speaker 01

  Totally. And so out of all the companies that are trying to build an AI software engineer, you were probably the only one that is really taking a vertically integrated approach and training your own models. And that is either insanely brave or insanely crazy and probably a combination of both. I'm curious, like, why, I know you love training models. And so I know that's part of it. But like, why do you think you need to own the model to get this right? And like, how do you motivate yourself in the kind of the David versus Goliath of like, knowing that OpenAI exists and has great people and cares about coding and is great at building models, obviously? Like, how do you think about that entire dynamic?

• 33:18Speaker 00

  I think you need, well, to build the best model, you need to build the model. And we want to solve these fundamental problems. You can't rely on any, like if the API guy sold it, then what the hell are you, you know? We might as well start the company three years later. It goes to the point where we started, right? We, we started working on this stuff two years ago. So we, we have, you know, it took us some time to learn how to train these large models. Like it was really, I think it took OpenAI two years to get from G P T three to G P T four. As, as well, and I thought we could be like much faster and this is gonna be great. It is, it's a pain. So it's definitely an engineering challenge, but it's necessary. Like, it's not like, it's not like we're doing it just because it's fun or because I like training models.

• 34:00Speaker 00

  It's, it's a massive financial investment that people trust us with. That it, it, and it, it's, it's not like it's one of those like one to one ROI investment. It's like if it's work, if it works, it's fantastic. And if it doesn't work, the GPUs ran and the money is gone. So like you're, you're getting a lot of people's trust of doing that. It's certainly not something you should do just because it's fun and you enjoy it. Fundamentally, I think the value will accrue at both at the, at the A G I and at the hardware level. And never at the application level. There's no incentive at all to offer an API. If the API creates a hundred billion dollar company, you will just build that company internally. It is, and if, if OpenAI doesn't, someone else will.

• 34:44Speaker 00

  Like it's, it's, it's, it's just incredibly unimaginable to me. That that would be how you would build these companies in the first place. And from a business perspective, I don't think that's necessarily the right way. Maybe there's some partnership potentials. You could like, oh, we'll get like special access or whatever. And then

• 34:59Speaker 01

  But why is that different from like cloud computing, right? Like there's, there's been many ten billion, a hundred billion.

• 35:04Speaker 00

  I mean, it's much, much, much harder to build Netflix and Airbnb and Uber than it is to build a chat interface. Like. Fundamentally, Magic is an application you press download on that we have a couple of guys working on and it's just there. Like, it's not, you know, you can build this with like YC pre-seed money. The moats in, I guess I could just make the API twice as expensive for the next model and then launch my own product and then undercut every, it's, it's really fucked to not own a model. In, in, in, in this domain, and in any domain that's going to generate a ton of revenue for a single company. In the case where it's distributed, maybe it's fine, but I don't think this will be. So, so it's necessary both for a market, which is good for us, because

• 35:45Speaker 00

  the market is incentivized to fund folks like us, which, which it isn't in other domains, like have, have fun writing like an email assistant. You're not going to get that funded anymore. So, so, so that's, that's helpful. But fundamentally, the reason we train our own models is because it's necessary for our mission. And I just wouldn't be interested in building like a nice little SaaS wrapper. It's just not like, that's not, that's going to happen anyway.

• 36:11Speaker 01

  And, and I think though about competing against the 800 pound gorillas, like you've raised a lot of money, but some people have raised boatloads of money.

• 36:19Speaker 00

  They've raised a lot of money too. And so as well, some people have, have 100 million plus in revenue a year that they could spend. So it goes beyond even the ones who could raise. Yeah, absolutely.

• 36:29Speaker 01

  And so how do you like, how do you motivate yourself to compete in the, in the, you know, reality?

• 36:34Speaker 00

  The question is how much does it cost to build AGI and not how much money can you raise? Because if you can build AGI for however much you can raise, and you're having more might help you, but it won't get you there substantially sooner, right? Like if you have all the right ideas and you get, you can build it with a certain amount of hardware, like by definition, like, okay, if someone had like 100 times more hardware, would it be like computing that much faster or whatever? But it, it doesn't seem like a material advantage if your estimate for how much compute you need to build AGI is not as high as the revenue these companies could generate or the, the funding they raise is in fact much lower. So, and I think that is the case. So it's not by any means accessible. It's very damn

• 37:15Speaker 00

  hard to get that much money. But it's not 100 billion. It's, it's, and if I'm wrong, I'm wrong and it'll be 100 billion and we will not have 100 billion and that's it. But if we can get to that point where we have AGI and a couple others have AGI and then like the, the, the sort of the benefit of additional computers there and you show an ROI, it's, it's like a reasonably even playing field in, in terms of.

• 37:38Speaker 00

  Additional revenue, you're just, you're gonna bring AGI to the market, you're gonna raise more on it. It's so that the starting conditions of like have this hardware is like you need sufficient hardware, but you don't need more than sufficient. And so that's a bet, that's not a, you don't know, but I think it's a bet with the high enough probability of being right, that it is reasonable to compete in this space. And I think it is actually, it is reasonable to think that

• 38:06Speaker 00

  like the ROI of having quote unquote sufficient funding might be better than the ROI of having like infinite funding early on.

• 38:14Speaker 01

  And is there like an ideal?

• 38:16Speaker 00

  For investors, that is not, not for me, for investors.

• 38:19Speaker 01

  Is there an ideal like team size for researchers? Is there a certain point at which you reach kind of like diminishing marginal returns of adding on the extra researcher?

• 38:27Speaker 00

  So one of my biggest weaknesses, especially early on at Magic, was just scaling the team effectively. Like we were very single threaded on a very small number of people doing basically all the work. And I think we're getting better at that now. It's also, you just need a certain level of maturity of your code base and of your research ideas and everything to properly segment them.

• 38:52Speaker 00

  So early on, I would have said five for that time. Now I would say closer to 20, and I'm not including like folks working on other stuff. I'm including folks working on like the models and everything. I'd say closer to 20. I could imagine that in a few months, I'll say at a slightly larger number, especially when you get into large scale deployment, you really want to have very, very good processes around just having high reliability, availability of services that are detached from each other, et cetera, et cetera. So then you can segment even more, which obviously stuff we're working on now.

• 39:26Speaker 00

  But.

• 39:28Speaker 00

  It sort of grows over time. I I don't see it ever exceeding like the tens of people. And right now it's in the low tens, very low tens. But I don't know, maybe it's it's a skill to be able to utilize. If you're able to utilize 200 people, you're just a better CEO than I am. The the, no, seriously, if if you can, i- i- i- it's it's a good skill. And and I think a part of why I say a smaller number for us is that there is a ton of stuff we just don't do. Like, if we if we built a video model, that would just be a separate team. They built a video model and like, you know, that's that's more scaling. So so to an extent, we're more focused and that's why we're smaller. But also, if we could double the team and be twice as fast, that's I mean, I would do it any day.

• 40:09Speaker 01

  Um, back in, or was it late 2022 when I first met you? Um, at the time, like, it was marketing assistants and email assistants were all the rage. And you were the first pitch that I heard that was AI that feels like a colleague. And I just remember that really sticking in my brain. So in some sense, you've been thinking about kind of like agents, uh, to use a buzzword longer than anyone else.

• 40:33Speaker 01

  I'm maybe share your vision for that and like what you what you think it takes to build a great agent.

• 40:37Speaker 00

  Fundamentally, there are two tiers here, I guess, three. One is useless. The next is assistant that you have to micromanage. And then the the next is the thing that manages you, basically. Where you're it's sort of more like a colleague. You could, I think the layer where it's exactly even doesn't really exist because it's sort of this like little thin point. Once the model is more competent than you are, um, you are there to give it guidance on what you want to be accomplished. And um, answer clarification questions, but you'll never have to tell it like, here's a bug. I'm not saying that this is V1 of everything. I'm not saying this is V1 of our product. But fundamentally, that has to be the goal.

• 41:20Speaker 00

  The way I feel when I talk to my best engineer, that's how I want to feel when I talk to magic. Where we have a discussion, he's almost always right. And then he just writes the code and then he, someone else reviews it and then it works. Like that experience where my job is exclusively saying like, here's kind of what I want. Uh, and then they help clarify even right? Like, ah, this one here specifically that, that, that you, you, it, it, it should feel like that. And everything else doesn't matter to the user. Uh, like what tools the agent uses, how it works, does it run locally in the cloud? Does it need a VM? Does it have a browser? I don't care. Doesn't fucking matter. Our problem, not your problem. You care about your problems getting solved.

• 42:04Speaker 00

  So fundamentally, that's what I think matters to customers. And everything else is dependent on the exact product shape, exact domain, except, except everything. And like, I'm stubborn as fuck. I just don't want to launch anything that isn't that. Uh, we will probably have to, uh, but the, I, I, I, I just really want to get that thing. Like, I want to talk to my computer, go and have lunch and come back and it built AGI. Like, that's the, that's, that's the end goal, right? And, uh, there'll be, there'll be checkpoints, but, but that, yeah. I don't think every, anything else matters. Uh, the, the, how, how you accomplish that is, uh, is, is up to each individual company.

• 42:44Speaker 01

  Yeah. How far away do you think we are from, from that? Or I guess maybe, maybe to break it down into a little bit more.

• 42:49Speaker 00

  I mean, we met in twenty twenty-two. You learned how to extrapolate Eric's timelines. So, uh, maybe, yeah, one and a half or double everything I say, but, uh, I think very soon, like, very small number of years. I don't want to give a number now, but very small number.

• 43:07Speaker 01

  Less than ten.

• 43:08Speaker 00

  Oh, definitely less than ten. I mean, way less.

• 43:10Speaker 01

  Wow, okay. Cause I'm seeing some of the, uh, like the Sui agent, uh, stuff that just came out. They're at like fourteen percent on Sui Bench, which feels like.

• 43:18Speaker 00

  I mean, fourteen percent. I just don't care about fourteen percent. Like, I, I, I, I mean, we, we, like, I don't know if eighty or ninety is good enough. Like. I like, I, I, I think you need ninety nine, like even ninety six, I don't trust my computer. Like, I don't want to review the code. If I have to, like, the tier of product where I have to review the code is fundamentally different from the tier of product where I don't have to review and understand the code. And like, you're not talking about ninety five when you, when you don't want to review. You're talking about ninety nine point something. You're talking about whatever my developers accomplish, uh, plus some, same as with self-driving cars, right? So the, the difference with self-driving cars is like, you die if the thing crashes. And here you just have to review code. So it's launchable before, but, but fundamentally you need way, way, way more. And like usually the last few, right? Like the, the nines are hard to get.

• 44:03Speaker 00

  So, yeah, but, but no, I, I think in, I, I don't know, I, I don't. People have, um, I mean, models have surpassed all these benchmarks. I mean, just recently the math benchmark, right? Like way faster than even like for prediction markets assumed than like, I, I don't see that stopping. There's just too much, like if, if everyone was stuck and I got realistic, some perception in the public that, oh, G P T four is like only like not getting much better. No.

• 44:28Speaker 01

  Um, okay, we're gonna close out with a lightning round. Um, one more answers. Uh, one, what's your favorite A I app? Not magic.

• 44:36Speaker 00

  Probably all the invisible ones still, like my spam filter and all this stuff. Uh, just the, the things that keep life, you're working, uh, I think are still at the moment more useful than the sort of A G I like apps. Cause if you took them away, like life would just be awful. Um, like recommendation algorithms, uh, for whatever, uh, I, I think that's really useful.

• 44:58Speaker 00

  Um, other than that, uh, yeah, I think whichever, uh, uh, you saying other than, uh, the, the, let's say the other than the programming world, um, other than magic. I, I actually don't, uh, yeah, but I'd say whichever model is currently like best would, it's a very boring answer, but I'd actually pick the spam filters, et cetera. The recommendation services first.

• 45:20Speaker 01

  What paper has been most influential to you?

• 45:22Speaker 00

  I don't think this paper is relevant at all in the world anymore, but it was the first paper I ever tried to deeply understand, like I spent months on it and re-implemented it and everything.

• 45:33Speaker 00

  And so it was most influential to me as a person, not so much to my current work. And the paper is called DeepStack. It's one of those neural networks plus imperfect information game solving papers. It's reasonably complex for time.

• 45:50Speaker 00

  Yeah, so if folks are interested, it's like nowhere near Sota now, but it's just an irrelevant type of algorithm. But at the current time, that's right, right now, back then it was useful. So that was very influential for me because it was just my first touch point with research, really. I had no idea how to do research at all. And then I sort of just was like, I'm gonna dig into this. The way people like hyperlink spam on Wikipedia where you like rabbit hole, I did that with this paper.

• 46:15Speaker 01

  I love that. Okay, that's my week in reading. Last question, what are you most excited about in AI in the next one, five and 10 years?

• 46:25Speaker 00

  Just what it's gonna, how society is gonna integrate with it.

• 46:31Speaker 00

  I think that's, we're getting to the point now where it's really gonna impact over the next one to five years. It's really gonna impact how society does stuff and beyond just, you know, another tab in your browser that speeds you up by some percentage on some tasks. I think it'll get much more significant in that timeframe. And ultimately, you should, like the only, I am not one of the intrinsic curiosity type of people. I know most researchers are, I really am not. I just care about the outcome and that is the outcome. So I'm most excited for the outcome.

• 47:04Speaker 01

  Eric, thank you for joining us again. Last time I recorded the podcast, we weren't actually able to talk about the thing that got us so excited about magic, which was you had shared with us your long context eval and our own kind of AI researchers had gotten really excited by what you'd accomplished on that. And that was actually what led to us investing in magic in the first place. So you just made some exciting new announcements around the eval. I was hoping you could share it with our audience.

• 47:30Speaker 00

  Yeah, for sure. Thank you so much. Yeah, I mean, we've been running around with this hashes eval for a while. Basically just being frustrated by the needle and haystack eval. And you know, everyone keeps complaining about it. And now that we've decided to announce our where we're currently at in terms of our context work. Instead of just like, you know, blah, blah, talking about like, oh, we have so many tokens of context. It felt reasonable to to share the eval as well. I mean, we've we've used it in our fundraising, obviously. And thanks for for backing us. And generally just used it to guide our our architecture development and our research.

• 48:09Speaker 00

  So yeah, it felt right to open source it and let others compare their architectures and their results with ours. And then, you know, we, yeah, it's exciting to share. And thank you for having back on to talk about it. Yep.

• 48:24Speaker 01

  Thank you. Can you say a word on what's broken about Needle in the Haystack and what your eval does differently?

• 48:29Speaker 00

  Yeah, for sure. You know, with Needle in the Haystack, basically what you're testing is like, find this weird thing, the needle, in this giant pool of not weird stuff, the haystack. And so really, all you need to be able to do to do this is to sort of take like a little backpack and walk from the start of the context to the end of the context and like find the weird thing, put it in your backpack and return it. You, if you have to, like, sort of, you have to sort of implicit prior that there is, this thing is weird. So you're more likely to remember it, which sort of means that you actually don't need to remember the whole, the whole context window. You don't, you don't need to know all of it. And so so that allows, you know, some models, I would say, to to sort of look like they're doing this, doing long context really well.

• 49:12Speaker 00

  And really, it's not working as well. So we decided to just go the complete opposite, like super hardcore mode and just replace everything with random noise. There's no semantic information at all. I guess it's just randomly generated letters, basically just hashes.

• 49:30Speaker 00

  And if you did something like Needle and Haystack in a pool of hashes, you really have to know the whole thing. But then what we do is we also do a hop. So it's not just you find this one thing, but you find this one thing and then you find another thing. As obviously, you know, you can keep that going. But that, those two dimensions, I think, really are the important quantitative components of context. There are other things you can measure much better in more domain-specific evals. So of course, we care a lot about code and so we look a lot at that too internally. But I think, you know, from a general purpose context evaluation perspective, and the reason we chose to open source this eval and only this eval is just that, you know, I think this quantifies exactly what you want to measure when you think about long context,

• 50:15Speaker 00

  that everything else is sort of domain-specific. But yeah, you want to be, you don't, you want to be forced to remember the whole context window when you're talking about the context window. Otherwise, is it really that big? Like, yeah.

• 50:27Speaker 01

  Totally. I remember our own researchers were just blown away by the purity of the eval and how well-done it was. And so, thank you for what you're doing and thank you for open sourcing it, especially in an age where long context is becoming more and more important. Congratulations.

• 50:42Speaker 00

  Thank you so much for having me back. Yep, cheers.

• 50:43Speaker 01

  Of course. Thanks, Eric.