Own or Be Owned: Why Every Company Needs Its Own AI Model (Yash Patil, Co-Founder & CEO of Applied Compute)

Yash Patil https://www.linkedin.com/in/yash-s-patil is the 23-year-old founder and CEO of Applied Compute, a $1.3 billion company helping businesses train custom AI models on their own data: smaller, cheaper, and purpose-built for the work they actually do. Before founding the company, Yash dropped out of Stanford and spent two years at OpenAI working on post-training infrastructure and Codex. He left with one core conviction: every company that runs its critical workflows on someone else’s model is building on shifting sand. Applied Compute is his answer to that problem, already serving customers including DoorDash, Cognition, and Mercor. In our conversation, we explore: Why “own or be owned” is becoming existential for any company that relies on frontier AI modelsWhat it was like inside OpenAI the weekend the board fired, and then reinstated, its CEOWhy post-training is where competitive advantage is now being built, and what reinforcement learning with verifiable rewards actually isWhy evals have become the new production environment, and why companies will never share them with frontier providersHow a specialized model built for DoorDash outperformed frontier models on a narrow, high-value taskWhy cost, not capability, is now the primary driver pushing companies toward custom modelsWhy Yash believes AI’s transformation of the economy will unfold over decades, and why near-term fears about mass job displacement are misplaced— Thank you to the partners who make this possible Brex https://www.brex.com/mario: The intelligent finance platform. Guru https://www.getguru.com/: The AI source of truth for work. Persona https://withpersona.com/generalist: Trusted identity verification for any use case. — Transcript: https://www.generalist.com/p/own-or-be-owned-why-every-company — Timestamps (00:00) Introduction (03:50) Fable 5 and the case for owning your own models (09:22) Why Applied Compute is betting on custom AI models (12:30) Yash's early influences and first projects (17:42) His brief time building at Stanford (19:29) Leaving Stanford for OpenAI (25:58) Inside OpenAI during Sam Altman's firing (28:18) What Yash admires about Sam Altman (29:43) Teaching models to reason (35:39) The core insight behind Applied Compute (39:40) How Applied Compute works with its customers (45:55) Why model training never ends (48:56) Why not every task needs a frontier model (51:25) The culture and people of Applied Compute (54:50) Applied Compute's training infrastructure (58:43) The coming compute crunch and other predictions (1:03:48) Final meditations — Follow Yash Patil X: https://x.com/ypatil125 Website: https://yashpatil.me https://yashpatil.me/ LinkedIn: https://www.linkedin.com/in/yash-s-patil — Resources and episode mentions: https://www.generalist.com/p/own-or-be-owned-why-every-company⁠ https://www.generalist.com/p/own-or-be-owned-why-every-company — Production and marketing by penname.co https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  There is truly some atrophying happening by relying too much on AI tools. The one thing I'm worried about people losing is this ability to actually deeply think about how things should be architected and designed. The best engineers right now are the ones who are like, learn how to code before AI. Because at the end of the day, I think humans will be the overlords over all of these AI tools. And you will still need to do critical thinking to know what you want to build. Palantir is one of the few companies that signs massive contracts without knowing if they can solve the problem. They'll sign like huge, you know, hundred million dollar deals before they've started working on the problem, right? Yeah. I think we've been into that category where there's open research questions. And people who like to work on things that are uncertain are naturally attracted to companies like AC.

• 0:42Speaker 00

  I know there's a lot of folks who are like, oh, 50% of jobs are going to go away in the next two years. Personally, I think that is not going to happen. I think just the diffusion of AI into the real world take time.

• 1:02Mario Gabriele?

  Demand for intelligence is outrunning the chips that supply it. Meanwhile, tokens are being sold at subsidized prices that won't last. Demand keeps climbing and companies are discovering that using a frontier model for every task is like cooking with a blowtorch. Spectacular and the wrong tool for most jobs. Yash Patel lives at the center of that squeeze. The 23-year-old ex-OpenAI researcher runs Applied Compute, a $1.3 billion company that trains custom models on a business's own data. Smaller, cheaper, and built for the work that that company actually does. One year in, and Yash's startup is already serving a mix of tech startups and established corporations,

• 1:46Mario Gabriele?

  helping them enter the AI era, all while owning their own intelligence. In this episode, we discuss why open weight models are quietly taking over real workloads, why Yash believes AI's transformation of the economy will take decades rather than years, and what it was like inside OpenAI: The Weekend the Board Fired CEO Sam Altman. I'm Mario and this is The Generalist. The best founders aren't spending their time on expense reports. They're busy building. Brex is the agentic finance platform that makes that possible. High limit corporate cards, banking and AI that handles the back office automatically. So your team never has to.

• 2:26Mario Gabriele?

  Expenses get captured, books get closed and spend stays in policy without anyone chasing it down. Your team gets their time back to focus on what actually moves the company forward. Vercel, OpenAI, Anthropic, Granola and Deepgram already run on Brex. One in three startups in the US does too. It's time to get Brex. Go to brex.com/solutions/startups. Every revolution in AI creates one question that never changes. Can you trust the output? AI for work is incredible, but without trust, it's just leading to faster mistakes. The challenge isn't building an AI that can answer questions.

• 3:09Mario Gabriele?

  It's making sure those answers are right. That's where Guru comes in. It's the AI source of truth that connects everything your company knows. So every insight, every answer, every recommendation is grounded in verified knowledge, not outdated information or hallucinations. When your teams and your AIs share one trusted foundation, everything moves faster with fewer redos, fewer blind spots and more confidence in every decision. Because in the age of AI, truth isn't just power, it's protection. See what Guru is doing for thousands of companies like Spotify, DHL and Stripe at getguru.com. That's getguru.com. Yash, lovely to have you here.

• 3:54Mario Gabriele?

  And we've been chatting already sort of before we've sort of kicked this off. And I think it makes sense to maybe start with some of the things we were chatting about. If that sounds good to you.

• 4:03Speaker 00

  Of course, yeah.

• 4:04Mario Gabriele?

  Yesterday, Fable 5 came out. A big day in the world of AI and has implications for all of us, but including on the sort of the business you're running. I know you've got a few thoughts on it. What came to mind for you when you sort of saw it?

• 4:19Speaker 00

  I think first of all, the model is amazing. Like a ton of our team members were using it. It's like clearly feels like a step function. You know, I think. There have been moments in time where we've seen better and better models, and this is definitely like a point in time that should be noted. I think, though, the thing that you're referring to is kind of when we were looking at the system card, and this was kind of blowing up on Twitter and whatnot.

• 4:45Mario Gabriele?

  Yes.

• 4:45Speaker 00

  You know, there were some stipulations about how you could use the model, particularly some of the guardrails where the model might not answer in a super intelligent manner, or it might sort of like answer around particular types of questions. And I think in the past, it's been focused on things that are very reasonable, like, you know, like safety concerns and things like that, things that kind of are consensus view. You know, models helping with this kind of stuff is not great. But yeah, in this case, it was, you know, queries related to like AI model development, which really sparked this question of like, hey, how much control do the frontier model providers have

• 5:29Speaker 00

  in how you use these models? And, you know, if they do end up sort of holding out capabilities for particular types of work, what does that mean for the downstream users of these models? So the sort of natural conclusion and what the core thesis of our company is, is in a sort of own or be owned. Like, having ownership over your own model and your own intelligence is actually really existential for a lot of people. And, you know, if you don't start investing in owning your own models, you might kind of be in a tough position when, you know, these things do change underneath you. It's kind of like a modelless company is sitting on shifting sand.

• 6:11Mario Gabriele?

  Yeah, and you know, the, I think the consensus things that people understand, you know, that Anthropic might not want to help you with is like biochemical weapons and such. But, you know, folks that are relying on these models to do more AI research, discovering that that is sort of maybe in some places subtly being degraded is

• 6:30Speaker 00

  Totally.

• 6:31Mario Gabriele?

  You know, difficult to To adapt to.

• 6:34Speaker 00

  Totally. And like, you know, there are stated reasons for why they're not doing this, particularly around like, you know, safety in terms of model development where it's unclear whether having everyone have access to these tools is beneficial for humanity. But the the big thing is who's going to make that call?

• 6:54Mario Gabriele?

  Yeah.

• 6:54Speaker 00

  And the people who make the models are going to make that call. So and then the this not so charitable view of this, right, is Anthropic is doing very well and making some extremely powerful models. And this kind of prevents other people from being in that race as well if they want to use Anthropic models to build up their model development stack. So it's a little anti-competitive. Yeah.

• 7:17Mario Gabriele?

  Yeah. My view would be that I think they really believe it. Like, I think they sincerely worry about this stuff. Like, everything they they put out into the world is fairly consistent around that. Do you I don't know. Do you find it to be more of an anti-competitive behavior?

• 7:33Speaker 00

  No, I I don't. I actually don't think it's anti-competitive. I think it does come from a really good place, which is like, you know, I have friends at Anthropic and I really do respect the leadership. They handled the whole project class wing and deployment of Methos really well. Like, you know, they do get some flack for it. But I think if it if you do have a powerful new technology, it makes a lot of sense, you know, for you to take things slowly and roll it out to preferred partners first. Also, you know, with this new Fable model, there's like some stuff around data retention, which makes sense. Like, you cannot cover the full surface area of what these models can do just within the walls of Anthropic. So you actually need to see how people use it. I think the important

• 8:17Speaker 00

  point is, and these things can be both true at the same time. I think, you know, Anthropic is acting in really good faith. But also, at the same time, it does raise the concerns of like, hey, if you don't have control of your model, if Anthropic ever does want to hold out a capability for competitive reasons, you know, what happens to the to the users of those models? Because, you know, I think I think Anthropic is an amazing business. They're like, I mean, they're obviously doing Doing incredibly well and they're moving into the application layer and things like that. So there are products that rely on anthropic models that are going to be competitive with whatever anthropic puts out into the application layer.

• 8:57Speaker 00

  So I guess the question is, what happens when you're relying on those and they want to push their products higher up the stack?

• 9:04Mario Gabriele?

  Yeah, there was always the phrase, I think it was around Apple, when they would release something that sort of immediately threatened something within their developer ecosystem, Sherlock, right? If you would say that it's like anthropic, every time one of these models comes out, it's like immediately someone is depositioned potentially and in these different ways. To come back to the idea of owning one's intelligence, maybe we should talk about it in the context of your company, Applied Compute. Like, what does that mean? Why is that so important? And why is that the challenge that you're spending your time on?

• 9:37Speaker 00

  Yeah, so what we do at Applied Compute, maybe I'll give like a little bit of overview is we help companies own their intelligence by training custom models on their data, being able to serve these in production at scale, and then continuing to improve them the more they're used. In practice, what that looks like is we benefit from the open source community, basically taking open weight models that are extremely powerful, actually. I think if you sort of track the development of open source models compared to closed source models, they've kept up. And a lot could argue that they've sort of been closing the gap from when the first open weight models started to come out.

• 10:18Speaker 00

  But yeah, we basically are able to harness these open weight models, combine it with customer data to create task-specific or specialized models. So the idea really comes from the fact that we don't think all AI workloads are homogeneous, right? There are places where you want to use the right model for the right task and you want to optimize for cost, latency, performance, modality, deployment, flexibility, that kind of stuff. So our vision is that AI is actually going to be very decentralized in the sense that many companies will Own their own models rather than there being a sort of single model that does everything.

• 10:57Mario Gabriele?

  Yes, and so to, you know, put it just in a probably a too dumb analogy for for folks, it would be sort of, you know, a Goldman Sachs of the world, uh, you know, has all of this useful data. I think you've even used maybe Goldman as an example before. That's top of mind.

• 11:12Speaker 00

  Yeah.

• 11:13Mario Gabriele?

  You know, has all this useful data context and, you know, just unleashing a general model into that loses a lot of what, you know, the value of this place is. And you also don't want to necessarily give that data away.

• 11:26Speaker 00

  Totally. Yeah. I mean, the, the way these companies operate on the inside is like a lot of what makes them special and differentiated. So, uh, you know, I had Brendan, a good friend of mine, um, and, and CEO of Merkle. He was the one talking about the Goldman stuff and he was over at our office last week, kind of talking about what they're seeing in enterprises. And, you know, I think the, the sort of manifestation of what differentiates a, a company when it comes to AI is is kind of coming up in this talk around evals, right? So, you know, companies have their own definition of what good looks like, how they, you know, do certain tasks and procedures inside of their company. And they're sort of codifying it within evals.

• 12:08Speaker 00

  And, you know, there's no way they would ever sort of give those evals away to, to, to the, the, the model providers. Because that's a lot of their, their secret sauce. Uh, you know, there's some things that they might, you know, want to share to sort of improve the capabilities of the models, but anything that's sort of proprietary and unique to them, that's what they're going to want to hold close.

• 12:30Mario Gabriele?

  Mm hmm. So many of these topics are, I feel like, so live right now, like the question of the value of routing and cost.

• 12:38Speaker 00

  Openpocalypse.

• 12:39Mario Gabriele?

  Yes.

• 12:40Speaker 00

  Yeah.

• 12:41Mario Gabriele?

  So I want to come back to it, but maybe to, to sort of set the stage a little bit, I'd love to hear about your story a bit. I was seeing that when you announced Applied Compute for the first time.

• 12:52Speaker 00

  Yeah.

• 12:53Mario Gabriele?

  There was a, a tweet from like someone you went to middle school with, which was very sweet. And it described you in such an interesting way that it made me very curious, describes you as. Uh, the middle schooler hanging out with high schoolers and someone who had a very unusual sort of unconventional mind.

• 13:09Speaker 00

  Yeah.

• 13:09Mario Gabriele?

  Yeah. What, what were you like as a kid? Were you just building things from the very beginning?

• 13:13Speaker 00

  Yeah. So, um, that's, that's actually really funny. I, I know exactly what you're talking about. It's a friend of mine. Her name's Sarah. We, we went to, we went to high school together. Uh, but she was a few years above me. So, you know, on a little tangent, there is this club called Science Olympiad.

• 13:28Mario Gabriele?

  Yes.

• 13:28Speaker 00

  We had like, there was like this middle school where a lot of people went to high school, like this specific high school afterwards. She was there in high school. I was there in middle school and those, those Science Olympiad team is often like hung out together. So that's how I got to know her before, before I was there. But, um, yeah, in terms of, in terms of childhood, like, uh, I can tell you a little about my family. So, uh, you know, a small family of, of four. Uh, mom is, is like a pediatrician, runs her own sort of medical practice. Uh, my dad's like an electrical engineer, works on chips and, and things like that. Very relevant now. Um, and I have an older brother, uh, who, who, uh, is in tech and, uh, is like a software engineer.

• 14:11Speaker 00

  Now works at a really amazing company called Chi, Chi Discovery, plug for Chi.

• 14:15Mario Gabriele?

  Yeah, I love it.

• 14:16Speaker 00

  Uh, Chi Discovery. Um, but growing up, yeah, like my brother was actually the person who got me into, into coding. So I, I feel like older brothers always like get way less credit than they deserve. Like they really paved the, or, or, you know, older siblings in general paved the way for the younger kids. And yeah, he was, he was the first to sort of start fiddling around with like building apps and games and things like that. And one of the things he did sort of when he joined high school is he like built this grade checking app. So the grade checking system in our, uh, school district was terrible. Everyone hated it. So he just like built this app and then got a bunch of his friends to use it.

• 14:58Speaker 00

  Eventually the whole school is using it. Eventually a bunch of schools in the whole school district were using it. And I just thought it was so cool that he is a single person was able to like build this cool thing. And now like everybody I know is is using it. So that really got me into this this mode of like, oh, wow, I want to do that too. Basically, classic like copy the copy of the older sibling. So my dad actually encouraged me a lot to to build these kind of like little fun, unique apps or platforms and stuff. But he doesn't know how to code. So he would give me the ideas and then he'd say, go, go build it. I'd go and build it and show him and he'd be like, oh, you should do this or that or something. So like, you know, for example, so

• 15:40Speaker 00

  my dad grew up in India, like a village, and it was very communal type type living. And the way it would work is each family would like cook a bunch of food of like the thing that they were really good at cooking.

• 15:52Mario Gabriele?

  Yes.

• 15:52Speaker 00

  And then they would like go and like sort of trade food with like other families, things like that.

• 15:57Mario Gabriele?

  Oh, sweet.

• 15:58Speaker 00

  Yeah. So so he was like, oh, you should just, your your mom makes really amazing burritos. And then we know this neighbor makes really good barbecue. So he was like, you should go and like build something that helps us trade food or we're not. Built it. It was really fun. No one ever used it.

• 16:13Mario Gabriele?

  But it was harder in America.

• 16:15Speaker 00

  Exactly. But it was a lot of fun to just kind of like iterate on this thing and see something come to life. And that that project actually got me my first internship.

• 16:25Mario Gabriele?

  Oh, wow.

• 16:25Speaker 00

  Yeah, my my neighbor, he's in tech and he was working at this small fintech startup that was like four people. And my dad was like in classic dad fashion was like in the yard talking to him about, oh, you know, yeah, she's working on this thing, yada, yada, yada. And he was like, oh, do you want to do you want to come and work at our startup for the summer? And I was I'd never done anything like that. So I was like, oh, that's super cool. This was like kind of freshman year of high school. So yeah, he took me to their office, which was just an apartment building, like one unit there. And I thought it was super cool to see like, all of these just normal people like.

• 17:09Speaker 00

  Working like it's such a small group of people working on this, this sort of like hackathon project.

• 17:15Mario Gabriele?

  Yes.

• 17:16Speaker 00

  You know, it was, it's very different than when I visit my dad at work and he's, he's in the cubicles and like, you know, like hundreds of people and things like that. Yeah. So it was like, I was like, cool. This is like a bunch of people who are doing exactly what I wanted to do in my free time with my dad.

• 17:30Mario Gabriele?

  Yes.

• 17:31Speaker 00

  But

• 17:31Mario Gabriele?

  As a job.

• 17:32Speaker 00

  Doing it as a job. And so I was like, this is super cool. So I entered in there and a couple of other places. That's how I kind of got into startups and stuff, but um, yeah, it was, it was kind of cool.

• 17:42Mario Gabriele?

  Amazing. And then you, you go to Stanford and I think I've seen you say somewhere that, um, you know, you're a very good student in middle school and high school.

• 17:50Speaker 00

  Yeah.

• 17:51Mario Gabriele?

  And then when you came to college, it was sort of like, uh, you know, I've arrived and I'm going to basically watch the lectures and just build stuff.

• 17:57Speaker 00

  No, no, exactly. So I mean, in, in middle school and high school, grades were super important. I think it's like a common thing and a lot of immigrant families as well. But yeah, when I went to, to college, it was kind of like a, give, give me an inch and I'll take it. So I was like, oh, like, you know, classes are online. That's cool. I'll just like watch the lectures online. I spend most of my time just like hacking on like basically manifestations of what I was working on like childhood, like middle, middle school and high school. So, but for like the university setting. So nothing, I want to be very clear. None of this was impressive or like interesting or anything like that. I was like working on like every single generic college campus coding project you can think of, like

• 18:39Mario Gabriele?

  Yep, yep.

• 18:40Speaker 00

  social calendar for like me and all my friends.

• 18:42Mario Gabriele?

  You have to do these things. You get these reps.

• 18:44Speaker 00

  Exactly. Or like a dining hall voting app or something like that. Cause you know, some food is better than others and things like that. But yeah, just, just through that, like found other people who were kind of doing the same thing. And I think Stanford's a really cool place because there's these sort of natural communities that form and, um, you know, there were like little informal clubs of like, oh, let's get together and hack on a project or something like that. And then sort of the main organization I was affiliated with at Stanford was Treehacks, which was like the school's hackathon, um, that we throw for, for like everyone across the, the nation and even some folks internationally. So yeah, spend most of my time just like kind of. Working on projects and stuff rather than a attended class.

• 19:30Mario Gabriele?

  You'll have to correct me if I'm wrong, but you're a Thiel fellow, but you also seem to have gotten a degree. How can this happen?

• 19:36Speaker 00

  So I did not get a degree, actually. Yeah, yeah. So I dropped out of Stanford during my sophomore year.

• 19:44Mario Gabriele?

  Okay.

• 19:44Speaker 00

  About halfway through my sophomore year. So yeah, was basically there for a year and a half. Dropped out to join OpenAI though, and then after OpenAI had started, started applied.

• 19:55Mario Gabriele?

  And OpenAI, I think, you know, the way you've talked about it before is like you were just so mesmerized by

• 20:03Speaker 00

  Oh, yeah.

• 20:03Mario Gabriele?

  this technology. Like, can you, yeah, tell me about that.

• 20:05Speaker 00

  I mean, I looked super fondly upon OpenAI because, yeah, obviously the technology was amazing, but the people were like even more amazing. I, so I actually didn't really take any AI classes when I was at Stanford. So joining OpenAI is how I learned a lot of this stuff. And there's, there are like, were incredible people on the post-training side that kind of took them under their wing and like taught me a lot of stuff. So, you know, Luke Metz, Barrett Zoff, a bunch of folks on the post-training side were like, they were my managers. And I was just like, kind of look, like, looked up to them. They're legends in like the language modeling space. They like made, kind of made the first versions of ChatGPT with John Schulman.

• 20:50Speaker 00

  It was actually really funny. So on the, on the sort of, so Barrett, who used to be my skip, told me when he was, when, on his last day, when he was leaving OpenAI, he was like, Yash, by the way, I don't know if you know, but when you were joining, you were originally supposed to be on like ChatGPT engineering team. And we just, we just really needed more people on post-training infrastructure, which is on the research side. So it was like, last minute, we pulled you in. And I didn't know, because you don't know your team before you join. And it kind of, you know, that's basically the reason why I got into all this AI and research stuff is completely by luck.

• 21:27Mario Gabriele?

  Yeah, set a few things in motion.

• 21:28Speaker 00

  Yeah, set a ton of things in motion. I feel super grateful for like the people that were there and then the fact that like, They were willing to like, teach me a bunch of stuff and sort of take me under their wing. And yeah, I mean, like this, this guy Ian Osborne, who's now at DeepMind threw my 21st birthday party. Uh, it was, it was a lot of fun. I, I had, I was lucky to be surrounded by like some really, really awesome people.

• 21:50Mario Gabriele?

  Am I right in recalling that you got the OpenAI job by like cold emailing Sam?

• 21:55Speaker 00

  Yeah. So the backstory there is rewind to like freshman year. We had been hacking on some projects and stuff and my friend and I both had like internships lined up for the summer, but we were like, oh, let's just like keep working on cool things and, and like, you know, keep, keep hacking on stuff. Um, so we, Sanford is a, it's a very unique place because I think there are a lot of people who encourage folks to, to do this, like to go and just like build stuff. So we were, you know, there were a few venture firms that were like, hey, like here, take a angel check or something like that and go and spend the summer building something. And then if, you know, if you, if you end up not wanting to continue, it's totally

• 22:36Speaker 00

  fine, whatever. Uh, but growing up, like I knew that nothing comes, comes free. So, um, we, prior to this, I had been invited to like Sam's place for like a dinner. He, he like, um, puts on some dinners sometimes for, you know, people who are building stuff on campus, whatnot. There were like, I think a like couple dozen folks who, who went. Um, and Sam is like someone I deeply, deeply admire. I think he's amazing. And so I had watched all his YC videos and read all his blogs and stuff like that. Couldn't work up courage to actually talk to him at that dinner. But, um, a couple of weeks later, when we were like, oh, should we do our internships or not? Shot him an email and was like, hey, like there's some people who are

• 23:19Speaker 00

  offering to sponsor us for the summer. We don't really know who they are. We're not even sure if we feel comfortable. Like what do you think we should do? Do you know these people? And he was like, you should totally do it. If you're, this is back, back when Sam was kind of talking about like, oh, people should spend more time building stuff and take time off of school to like try and explore things. He was like, if you want, I can basically give you a small stipend, um, to just like pay. For rent and food to go and work on it and I'll sponsor you for the summer, which is like deeply generous of him. And, and we were like, wow, this is cool. Like, and so, um, so yeah, so we, we, we just did that for, for the summer.

• 23:58Speaker 00

  Worked pretty closely with someone who was like running, uh, sort of his, his family office at the time, who became a friend. And then, uh, ended up shutting that down though. It was, it was like an e-commerce company. Uh, but ended up shutting it down and coming back to school and like returning the, the money and things like that. But yeah, that's, that's how I got to know him. Um, and then when ChatGPT came out, I was like, wow, this is the coolest thing I've ever seen. I have to work on it. So then I was like, oh, hey, Sam, I don't know if you remember me, but like, you know, I think ChatGPT is super cool. I'd love to work on it. Like, are there any ways I can apply for a job? And then he was like, oh yeah, like we have this residency program. You should apply for it. So he introduced me to the residency folks.

• 24:44Speaker 00

  Talked to them. They were like, oh, you have to drop out. And I was like, I'm fine with that. I can, I can take time off. And so, and they were like, no, you really have to drop out. Like you have to like commit to never coming back.

• 24:54Mario Gabriele?

  Burn the boats.

• 24:54Speaker 00

  Yeah. And I was like, um, I don't know if I get fully convinced by parents of that. And then, so I was like, Sam, like, you know, should I, should I do? And he's like, let me chat with them. The residency is like a six month thing. So you'll have the option if you want. Then I interviewed, uh, was very lucky to get the, the residency position and then joined like basically like a week after.

• 25:17Mario Gabriele?

  They needed you to drop out like almost as a proof of conviction somehow?

• 25:21Speaker 00

  I think at the time it was just, it was actually just a much smaller company, right? So like, I think they're, they're, it's not the, you know, massive company that it is today. This was actually something that I, I don't think they had done before. Like they hadn't had people do the residency that had like dropped out of college just yet. But after, after I did, I think there were a couple of other folks who became more mainstream. But yeah, I guess I, I think they just wanted like commitment and stuff. But I, I already kind of knew at the time. I was like, I would be so lucky to be able to work here. You know, I totally would. If it was up to me, it was not up to me. My parents had to, had to have a say, right? So.

• 25:56Mario Gabriele?

  Clearly the right choice.

• 25:58Speaker 00

  Yeah.

• 25:59Mario Gabriele?

  You ended up living through like. One of the most interesting periods of technological history ever and also one of the most dramatic like weeks of technological history ever.

• 26:10Speaker 00

  Yeah.

• 26:10Mario Gabriele?

  I'm sure there's lots of that you don't want to talk about.

• 26:12Speaker 00

  Yeah.

• 26:12Mario Gabriele?

  But you were there when Sam was ousted, came back. What was that experience like for you as someone having what was, I think, their first proper job? Like, that's a very dramatic thing to go through.

• 26:25Speaker 00

  It really sucked. So I remember that week pretty, pretty well and I don't, I don't want to be over dramatic or anything like, you know, things, things happen and, and stuff, but like, it was, it was kind of this feeling of like, oh my gosh, like, things are going so well, we're making so much progress, everything is, you know, everything's happening, you know, why are we halting the, the progress? Like, let's just go, like, things were going so well. And so I just remember there were like a bunch of, basically that whole week was actually like, you know, we couldn't go into the office. All our laptops were shut down for security reasons and whatnot. So it was like every day there'd be someone who'd be, who was like, oh, I'm like hosting at my place, just come hang out. Um, cause we, no one had anything to do, there's no work.

• 27:08Speaker 00

  So, um, so yeah, I just remember like everyone being down on the fact that it wasn't even about like the situation, like, you know, everyone wanted Sam to come back and, you know, no one more than me, cause I think he's really awesome. But everyone was kind of sad about the mission just being on pause. They were like, this is, this is weird, like what we were doing so well. So, and then it was a roller coaster too, because, you know, Sam was, uh, talking with the board and things like that. And, you know, we, we all thought he was coming back and stuff. And then there is an announcement that, you know, he wasn't, and there was someone else stepping in. But eventually, right, I, I think, um, we were super excited that Sam came back as CEO.

• 27:51Speaker 00

  And, um, it's actually funny, I remember, so after that was announced, everyone was going to meet back up at the office. And I lived, so I lived across the street from the office in this, like, the building, the Madelon. And so I like came into the office and Sam was actually coming down the stairs. So, um, gave him, he, he gave me a big hug and I definitely remember. For a while, but that was like probably one of the coolest coolest days was just like it's back on track.

• 28:17Mario Gabriele?

  Yeah.

• 28:17Speaker 00

  Let's go. Right? So.

• 28:18Mario Gabriele?

  It sounds like, you know, there's obviously been, I'm thinking of all the stories that have been out over this year at the books. Sounds like you still are basically, yeah, a big supporter of Sam and think he's a good executive in that respect.

• 28:31Speaker 00

  Oh, yeah. I think, I think, yeah, a little side tangent on Sam. I think he is an amazing CEO and his superpower is he's able to make impossible things sound extremely possible. Like, you know, if I were to tell you, hey, we're gonna spend, you know, hundreds of billions of dollars to build the biggest supercomputer in the world and train a model that can like, you know, do like, you know, 80% of all knowledge work, you'd think that was crazy.

• 29:01Mario Gabriele?

  Yeah.

• 29:01Speaker 00

  But somehow he's able to go up in front of hundreds of the smartest people in the field and be like, guys, this is possible. We can do it. And I think that extends like the whole, like the whole leadership team as well. Like Ilio is really, really good at this too. He had his, uh, you know, the feel the AGI stuff like, yeah, I really admire Sam's ability to make impossible sounding things sound possible. And that's really how you get the best people is you work on really hard problems because all the best people want to work on hard problems. No one wants to, no one who's really, really smart wants to work on things that are, that are easy.

• 29:37Mario Gabriele?

  Mm hmm. I think that's a, yeah, I mean, that is a skill. I will make you answer for, for Sam's journey. In terms of your time at Opening Eye, you mentioned that you, you ended up spending it on post-training and this sort of had, you know, such an impact on, on what you're building today. For folks that, I'm sure most of the audience or a lot of the audience will, will be really intimately acquainted with this, but like, you know, for folks that maybe aren't, can you explain a little bit about, you know, why post-training matters and how it's sort of led to this work you're doing with Applied at the moment?

• 30:08Speaker 00

  Totally. Yeah. So, so kind of a high level overview of, of training models in general is there's a couple of phases of training. You know, a lot of people talk about the, the big expensive, high capex. Phase, which is pre-training, which is let's go and take internet scale data, use this architecture called the transformer to sort of learn language patterns. And out of that falls intelligence, right? So you get this model that can do next token prediction. And that's kind of like how we got the first sort of GPT models where they just got really good at predicting the next token. And it felt like these models could actually reason and answer questions and things like that.

• 30:50Speaker 00

  After you do pre-training, you basically have a unaligned model that will just predict the next token until you tell it to stop. To make it useful, that's when you move to things in the post-training realm. So there's a couple of different types of post-training. You know, when we were first training these like chat models, right? Where it's like a user sends a message and assistant sends a message. These are a variety of techniques of like supervised fine tuning, as well as like reinforcement learning with human feedback to sort of steer the models to give responses that are coherent. You know, in the, in the format and tone that a human would like and whatnot. Yeah.

• 31:31Speaker 00

  But you're sort of just basically doing behavior cloning on the supervised fine tuning side. So you're saying, here's a bunch of examples for good answer. If, if someone inputs this prompt, these are examples of what good completions look like. Yeah. And then you just do, you know, you just train the model to reduce the, the loss on those like target tokens. Now, what people have been talking about when they say post-training today is something quite different than supervised fine tuning or reinforcement learning with human feedback. It's this, this, this idea of RLVR. So reinforcement learning with verifiable rewards. It turns out if you take a really small, like a relatively small amount of high quality data as compared to like when you were doing pre-training or SFT or something like that.

• 32:17Speaker 00

  And you actually have a model try to attempt the same problem a bunch of different times, and you have a verifiable way of checking if the model gets the answer right. You can use this new reinforcement learning algorithm to sort of show the model, hey, these are examples of what good looks like, these are examples of what bad looks like, and sort of nudge the weights to do more of the good reasoning versus bad reasoning. This thing that people talk about when they talk about chain of thoughts, so all these tokens before the model outputs an answer, is actually entirely an emergent property. Turns out like the models learn to steer themselves, learn to think, and that's how

• 32:59Speaker 00

  they get more accurate and get, you know, basically like hill climb any sort of task that you give it. So what the first domain that this started with was math. Part of it is because, first of all, it's like got a really lot of really good properties, which is like easily verifiable. You don't need that many complex tools or things like that to be able to do math. And the second thing is just like everyone on the research team at OpenAI was like competitive math people or things like that. So it was like a domain that they deeply understood and also were very excited about. And there's also a lot of other philosophical reasons around like why math is like kind of the root of like a lot of other things.

• 33:41Speaker 00

  So basically, the first experiments were like actually hill climbing on a variety of different math problems, ranging from like basic arithmetic to sort of like harder, you know, proofs, like, you know, algebra, calculus proofs, whatnot. And turns out that the model is able to learn on sort of easier problems and sort of internalize that and be able to unlock harder and harder problems. So that's this like hill climbing that you've seen. And there was kind of like a couple of scaling laws that came out. So obviously you have the pre-training scaling laws, now you have this like post-training scaling law, and now you have this test time compute scaling law, which is like if you actually allow the models to reason for longer when you're using them in production, they give you better answers.

• 34:25Speaker 00

  So that's this like reason, when you pick a reasoning level in ChatGPT or something like that, that's it just reasoning for longer, spending more test time compute before giving a final answer.

• 34:36Mario Gabriele?

  This episode is brought to you by Persona, the B2B identity platform helping businesses verify users, fight fraud and build trust. Fraudsters are already using AI to spoof faces, voices and documents. So your defenses need to adapt just as fast. Persona helps secure some of the internet's largest and most trusted platforms with identity verification. If you're building a product where trust matters, identity should be a priority. You've probably already experienced Persona without realizing it. Verifying your LinkedIn profile, signing up for Etsy or renting a scooter with Lime. Trusted by leading companies like Square, Brex and Twilio, Persona gives you the building blocks to create identity flows that adapt to your customers, risk tolerance and locales you operate in.

• 35:21Mario Gabriele?

  Whether you're verifying age, onboarding businesses or automating KYC. It's fully configurable, so you can launch in days, not quarters. Want to see for yourself? Generalist listeners get a free year of the starter plan. Head to withpersona.com/generalist and check it out. You're doing this work at OpenAI. What was the moment that you realized there's a need for the sort of service that Applied Compute offers, where you are sort of going into these companies and doing this more hands-on work?

• 35:51Speaker 00

  So I think Microsoft AI, when they put out their thinking models recently, you know, the title of their paper is like the hill climbing machine or something like that. I think the way of thinking about RL is it really is that. It's like if you define the hill to climb well, the algorithm works. So the way we were training these models was actually just like on the post-training side was, hey, let's go and make a bunch of high-quality data sets that target the capabilities that we want the models to get really good at. Like basically, we'd start with an eval and be like, okay, if this is what the eval looks like, let's just like go make training data that looks like the eval.

• 36:29Mario Gabriele?

  Yes.

• 36:30Speaker 00

  That was the way that we were proving these models. So the natural conclusion was kind of like, hey, you know, everyone is going to have the, their own evals and the things that they want these models to be really good at. And evals extend beyond just, uh, you know, the intelligence of the model. It's like, oh, it needs to, you know, be this intelligent in this cost and latency regime, or this modality, or something like that. So if everyone's going to have their own evals, everyone should also be able to train their own models that do well on those evals, right? So, so our sort of natural conclusion was like, hey, maybe since we have this hill climbing machine, the best way to sort of manifest it in the world, obviously the frontier models are amazing.

• 37:14Speaker 00

  Um, you know, these will be like workhorse models for a lot of things.

• 37:16Mario Gabriele?

  Yes.

• 37:17Speaker 00

  But if people want to be very specific about the task that they're trying to optimize for, they should have the ability to go and push train their own models on their own data.

• 37:26Mario Gabriele?

  Yeah, evals are, you know, the, the scorecard for all of these models and that's so general.

• 37:31Speaker 00

  It's the new prod, it's the new PRD.

• 37:33Mario Gabriele?

  Yeah.

• 37:33Speaker 00

  Yeah, exactly.

• 37:34Mario Gabriele?

  And, and the reality is that every organization has a different scorecard for what they consider valuable. And that's, you know, sort of what you're saying is like, you know, if you have a different set of things that you're optimizing for, it starts to make sense that you have a machine that is made to hit those optimizations.

• 37:49Speaker 00

  Exactly. And if we go back to the, the Goldman example, right? Like if a lot of those evals are going to stay private, people are going to want to own their intelligence, own the post-training capability to go and create those models that are good at those evals, right? Yeah, so I, I think that was the primary motivator. And then I think the main, uh, sort of driver for why people are thinking about this right now is cost. Uh, the token apocalypse has really shown, hey, you need to use the right model for the right task. Um, even beyond just like optimizing for evals, right?

• 38:20Mario Gabriele?

  Yeah, that's, I, I'm curious like how you think about it today. Clearly that's top of mind. I think in the past, you've talked about how general models sort of set the floor and, you know, specialized models can sort of raise the ceiling in some sense.

• 38:33Speaker 00

  Yeah.

• 38:34Mario Gabriele?

  In some ways, it feels like the distance between those two might be getting smaller, but the cost at which you are accessing these different offerings is sort of like, has it become more of a cost story in your head?

• 38:46Speaker 00

  Cost has become like a, the main driver, I think, right now. But I think in principle, the argument still stands, which like, take Kirkland and Alice, right? They caused waves in the Twitter community, I'm sure broadly as well, about how they're going to spend like half a billion dollars to, I forgot what the exact quote was, but essentially the idea was to create AI tools that their competitors can't also use, right? So I think the idea is, and this extends beyond just the model, to be clear, right? The model, the model is part of the moat, but also training that model inside of your harness with your context, that's actually how you're going to get differentiation.

• 39:29Speaker 00

  So I think a lot of people are thinking about, hey, if I'm using the same stuff as my competitors, how am I different? And that's also another driver for why they should train their own models on their own data.

• 39:41Mario Gabriele?

  And I think it's been just almost a little over a year, right?

• 39:45Speaker 00

  Yeah, yeah.

• 39:46Mario Gabriele?

  And you've gone from starting the company to, you know, a last valuation 1.3 billion.

• 39:50Speaker 00

  Yeah.

• 39:50Mario Gabriele?

  Been a good year.

• 39:51Speaker 00

  Been a good year, yeah.

• 39:53Mario Gabriele?

  Can you tell me a little bit of how you've started to work with folks and what that process looks like with these companies?

• 39:58Speaker 00

  Totally, yeah. So we work with like a variety of, of companies across AI natives, digital natives, and large enterprises, and hyperscalers. So basically, if you look at how money is spent on these models, right? There's some amount spent on like training, and it's like a non-trivial amount, but like most of the money and most of the usage is on inference, right? So when we were thinking about what the future of open models looks like, or models that people are going to inference a lot for different things. Our conclusion was kind of if you are going to train, or if you are going to use open weight models and specialize that model, you're going to do some sort of training on top of it.

• 40:41Speaker 00

  So the actual wedge into inference is getting really good at training. So what we do with our customers is we sort of, we have like a post-training platform where they can integrate all of their data. And we have a team of applied researchers that actually deeply embed with the customer, work with their ML and research teams

• 41:02Mario Gabriele?

  Yes.

• 41:03Speaker 00

  to train these custom models. And we really want to make sure that the economics are in their favor. So we try to have most of the stuff that they pay for come on the inference side. Right? So it's like, once we have a really good model and you guys want to use it, then we can co-host it for you.

• 41:20Mario Gabriele?

  Yes. The sort of forward deployed engineer model is something that I think people obviously associate with Palantir as a successful example of it. With Palantir, I think the way they've made it work is that clearly they found a way to get good leverage on it, such that

• 41:37Speaker 00

  Yeah.

• 41:37Mario Gabriele?

  the 50th version of it, of them taking on a project is very different than the fifth. What's the version of that for you that stops it from being sort of just pure consulting leverage?

• 41:50Speaker 00

  There is always some component of this that is going to be unique per company. That's kind of the whole thesis, right? It's like each company is going to have different data and want a different model. But really, it's the infrastructure that is repeatable across all of these companies. Right? It's like the scaled infrastructure for training these models, the scaled infrastructure for serving them, and then the sort of techniques for capturing production usage, turning that into more data that we can go and continually train. So our equivalent of Foundry, which is Palantir's platform that they use to give to all their FDEs to go and move faster.

• 42:27Mario Gabriele?

  Yes.

• 42:27Speaker 00

  We have our own post-training platform where all of our applied researchers and researchers from the companies we work with can both collaborate to train different models. And really, I think the way of thinking about it is it's the infrastructure to make training performant and easy.

• 42:45Mario Gabriele?

  Mm hmm.

• 42:46Speaker 00

  Yeah.

• 42:46Mario Gabriele?

  Does it end up becoming more of an ongoing engagement than people might realize from the outside? I could imagine that, you know, One version of it would people would think, oh, you go in, you're sort of doing an AI transformation and then you know, you're out the door. But I would imagine like the demand for, you know, improving this, deepening it is like, leads to quite long engagements.

• 43:08Speaker 00

  Totally. Yeah. I mean, so I think there's always a desire to continually improve these things, which is why, you know, when we deploy a model in production, a lot of our, our product is, is built around, hey, how do we take the usage of this model and turn it into more data that we can continue to, continue to train on or build context around the model so that, you know, it's, it's learning actually the more, the more you use it. So yeah, I think the, the idea is that the engagements don't end. Obviously like the way we generate revenues by serving the models and we need to make sure that those models are always performant.

• 43:47Mario Gabriele?

  And, and maybe even to get even more sort of tangible, like, can you maybe talk us through an example or two of like what taking this more specialized contextual approach unlocks for a customer, for instance?

• 44:01Speaker 00

  Yeah. So, so I think one of the case studies we have on our website is this engagement we did with DoorDash, which is, you know, they have more than a hundred thousand merchants that come to their platform every year and they want to create a new DoorDash storefront, right? So when they, when they do this, they come with a bunch of unstructured information about their business, um, you know, what, what, who they serve, things like that. And one of the things is like a picture of their, their menu. So it turns out this was actually a example of where we were, where we were able to exceed the frontier in terms of quality is we trained a special, small specialized model that was able to take pictures of menus and turn them into high fidelity DoorDash storefronts.

• 44:45Speaker 00

  Basically the process was create an eval for what good looks like.

• 44:49Mario Gabriele?

  Yes.

• 44:50Speaker 00

  Create the training data for, for us to go and like, uh, do RL training on top of, train the model and then deploy that and host that.

• 44:58Mario Gabriele?

  The trend feels like. You know, if applied compute continues to grow, and this is part of sort of a broader movement, is like sort of trending towards more of these small models where you do sort of say, here's a set of tasks that like are maybe a bit more narrow, a bit more deterministic, that like, we can actually very inexpensively do this very effectively. Is that roughly how you see things going?

• 45:24Speaker 00

  Yeah. And so I think the motto is better, cheaper, faster. What people are starting to realize is that these open weight models provide a ton of flexibility around ownership, and they're actually good. Like, you can actually go and use them to perform optimally on your task. So I think you're going to see just a lot of people for, you know, reasons related to cost. I think cost is the big driver right now. But there's just so many other downstream benefits you get by owning your own model.

• 45:56Mario Gabriele?

  When you sort of move towards the world of, you know, a company having multiple of its own models for different tasks, does that create like some sort of maintenance debt for them that they have to sort of stay on top of? And yeah, I mean, there's probably still cases where that's very much worth it for them. But is that something you've noticed come up?

• 46:16Speaker 00

  Yeah, I mean, so this is part of like the continuous improvement part. As there's obviously like model drift and you know, the distribution of tasks might be changing slightly, but over a long period of time that really, really matters. Our view is just like, training and inference loops, they're like this right now, but they're starting to move closer and closer together. Right now, the form is like, people are just getting a lot faster at making new data and training the model again. But eventually, these things are going to be quite intertwined. I think there are some cool examples of this, like that are kind of like the, maybe the first sort of beginnings of it. Like, if you saw cursors, composers, real-time RL

• 46:59Mario Gabriele?

  Yes.

• 47:00Speaker 00

  work. I mean, that was them doing like online training by like, taking a bunch of user trajectories, you know, getting, extracting the implicit reward, taking a training step. And then deploying a new, new model in production. I think like that is just kind of an indication of where things are going to go is like, you're actually going to be doing more and more continuous training.

• 47:23Mario Gabriele?

  Yes.

• 47:23Speaker 00

  Because as your models being, it's kind of like an exploration technique, right?

• 47:26Mario Gabriele?

  Yes.

• 47:26Speaker 00

  So.

• 47:27Mario Gabriele?

  And, and, and so, you know, what you're describing there is almost like a continuous learning model where it's like, every time I happen to do this task, it's actually training this model to be better the next time around.

• 47:38Speaker 00

  Totally. And, and the name of the game is learning from sparse rewards, right? So if you think about the different phases of training, pre-training is like super data inefficient, right? You need to like learn the whole, or you need to train on like the whole internet in order to like learn the language. Then you, we had like this like SFT stuff, which was kind of behavior cloning. So you say,

• 47:58Mario Gabriele?

  I'm not familiar with that.

• 47:59Speaker 00

  Yeah, yeah. So, so SFT is described as behavior cloning because you're giving it examples.

• 48:03Mario Gabriele?

  I see.

• 48:04Speaker 00

  Of what the correct outputs should look like. And so you give it a bunch of examples and then for a new task or like a new prompt, the model is like sort of interpolating like, oh, like I should, I should say these things, right?

• 48:16Mario Gabriele?

  Okay.

• 48:16Speaker 00

  So, and then we kind of move to RL, which is like a lot more data efficient, which is, I'm going to take one data point, one math problem, generate a thousand trajectories, all with different thinking tokens or whatnot, all with different rewards. And then I'm going to do backprop on those and there, there the model will kind of learn what good reasoning looks like and bad reasoning looks like. The ultimate thing, right, is being able to do super, super sparse, uh, learning, which is like, you go and roll out a trajectory, you have some indication of whether it was good or bad or something. And you're able to directly learn from, from that. That is difficult and that is the problem to solve and continual learning.

• 48:57Mario Gabriele?

  On the, you know, open weights models, you, you were mentioning earlier that you think like maybe the gap between open weight and closed models is, has sort of narrowed over the past year. In some ways, if I look at like the hardest benchmarks, I almost feel like it's widened. But actually what we're learning is that it does, just beyond a certain threshold, it almost doesn't matter.

• 49:18Speaker 00

  Yeah.

• 49:18Mario Gabriele?

  Like, I, I don't know. Do you think? That's wrong.

• 49:20Speaker 00

  Yeah, yeah. So I think what we were talking about before is frontier models are amazing and you should still use them for your hardest tasks. But if you have like a hundred dollars of spend, maybe 20 of those dollars go to the frontier models and the 80 dollars can be actually 20 dollars that go to cheaper, better, cheaper, faster models that you train on this task, right? So I think like, like the frontier models are obviously very, very, very good. But you know, you don't need to use them for everything. There's all these memes on Twitter, right? Like where like people are using some big sword to cut like us.

• 49:56Mario Gabriele?

  Yeah, or a blowtorch for, you know.

• 49:58Speaker 00

  Exactly. So I think like, you know, there's a lot of truth to that. And that's what, you know, you were talking about model routing and things like that. I think there's going to be some pretty interesting innovations there. We're working on some of that as well.

• 50:09Mario Gabriele?

  The other thing I think about with open weight models is like China has really dominated that so far.

• 50:15Speaker 00

  Yeah.

• 50:16Mario Gabriele?

  You know, it feels like America is clearly leading with some of the frontier models. But yeah, it's like, how do you think about the fact that an implication of this could be that a lot of American companies become like actually quite dependent on Chinese models, that the rules could get changed up on them, those could go closed at some point? Like, is there a robust enough open model, open weight system right now?

• 50:40Speaker 00

  So, so I think that, you know, you're right. It all boils down to incentives. So I'm extremely excited that companies like Nvidia are investing a ton in open source models. And Nvidia has a real reason to do it, right? Like the dream for Nvidia is that every company is, has their own model and is running it on Nvidia chips. So I think that there are real, there are companies like Nvidia, Nvidia being the big one, that have true incentives to keep models open. And I think there is enough of investment and a lot of smart people working on open models

• 51:21Speaker 00

  that I think it's going to be a pretty, pretty safe and big bet over a period of time.

• 51:26Mario Gabriele?

  One of the things that I think is really interesting about applied compute, sort of outside of the technology piece, is that you seem to have managed to attract a huge amount of ex-founders, which is always a very bullish sign, I think, for an investor, you know, to see that happen. How have you managed to do that? It's something like two thirds, right?

• 51:46Speaker 00

  Yeah, so it's almost. Part of like the business model is ever because we're doing this like sort of forward deployed approach where we're embedding deeply with with customers. It requires people who are extremely high agency and like ambitious and move fast. So I think it just naturally like the kind of work just naturally attracts these these types of folks. And then, yeah, I think I think like what we we in our culture, we sort of decided, hey, we want to hire ex-founders who are who fit this archetype and they just themselves attract other like-minded folks.

• 52:26Speaker 00

  So yeah, I think it hasn't been anything really special like we're not trying like recruiting from some very specific pipeline or something like that, but it's just this kind of natural snowball effect where you have the type of work that we do plus the initial team that we got together just attracts more people of the same mentality.

• 52:44Mario Gabriele?

  At a higher level, like what are the things that you think are maybe particularly distinctive about applied computes culture? Like what do you care about that? You know, other other startups you know, or other places you've worked maybe wouldn't care about as much?

• 52:59Speaker 00

  So there's a lot of this like nine nine six stuff.

• 53:02Mario Gabriele?

  Yes.

• 53:04Speaker 00

  We're not a shop like that, like we, and I deeply respect people who who choose to operate like that. I think the type of culture that we want to embody is people are just very excited about their work. And we have no requirements on how long people work or things like that, but it's a very bullish sign when people are so excited about what they're doing that they want to come in on on on weekends and and and sort of spend a bunch of time in the office. So yeah, I think the things that we prioritize or what we look for when people are joining the team is like being deeply mission aligned. So, you know, we want people who who want to come and work on hard problems, but also really believe that,

• 53:46Speaker 00

  you know, something like this should exist in the world and that this is actually very important for basically every company out there. And then take a lot of pride in working on hard problems that don't have like the thing that's very unique about our company, right? And it's actually somewhat related to a Palantir type. I remember having this unique moment where it's like Palantir is one of the few companies that signs massive contracts without knowing if they can solve the problem, right?

• 54:16Mario Gabriele?

  Wow, how interesting. That's so true.

• 54:17Speaker 00

  Yeah, like, you know, they'll sign like huge, you know, hundred million dollar deals where this is like before they've started working on the problem, right? So, so I think there are, there, there's like a unique set of companies that, that do that. And, and obviously like, I think we, we, we fit into that category where there's open research questions about like what we're trying to go and get really good at and solve. And people who like to work on things that are uncertain and there's a lot of greenfield research to be done are just naturally attracted to companies like AC.

• 54:48Mario Gabriele?

  When you talk about, you know, wanting people who want to do the hard thing, like what's been the hardest, thorniest, technical piece of this so far for you?

• 54:55Speaker 00

  Yeah. So I think like, one of the really cool places of work in the company right now is basically building the best ML infrastructure for training open weights models. So, you know, we're obviously in this compute crunch, the more performantly you can train models and the more, you know, sort of like accurate they can be, that's like true differentiation, right? There's, there is actual work. It's not like RL is completely figured out and it's kind of like a commodity thing. Like there's real research happening at the ML infra layer. So we've been hiring a bunch of these like really, really smart researchers that are actually able to go deep into the stack, work very close to the hardware to actually make training more performance.

• 55:39Speaker 00

  And that's one of the ways we're differentiating amongst like other things that are out there.

• 55:44Mario Gabriele?

  Can you, you know, maybe part of this is too secret sauce, but like, can you talk a little bit about what that training stack is? Like what that looks like?

• 55:51Speaker 00

  Yeah. So, so basically the, you know, the, we, we procure our own GPUs and then we, when you're actually going and doing training, right? The way training works is you're alternating between like doing inference. So, so you're, you have some checkpoint, some weight checkpoint that's like a policy step zero or something. You go and try a bunch of attempts at a problem, right? And that problem is actually in some sort of replayable environment, right? So say we're training a coding model, you're like spinning up a sandbox that has a code base on it. And potentially even more complex than that, like you have, you know, services that are running that the model need to interact with, yada, yada, yada.

• 56:29Mario Gabriele?

  Yeah.

• 56:30Speaker 00

  All needs to be replayable though, right? Cause you're repeating this like a thousand times.

• 56:33Mario Gabriele?

  Yes.

• 56:34Speaker 00

  You go, you go and do the inference on that, get a bunch of model attempts, assign reward to them based off of like some verifier. And then use some sort of objective to be able to go and take the next step on the model and get your policy step one. And so one of the really interesting problems is like, okay, you're doing this sort of alternating of sample, train, sample, train, sample, train, sample, train.

• 56:59Mario Gabriele?

  Yes.

• 56:59Speaker 00

  It's like always slower to do things sequentially, right? So what do you actually want to do is you want to do things asynchronously where you can actually be sampling on your next set of problems. While training is happening.

• 57:12Mario Gabriele?

  Oh, wow.

• 57:13Speaker 00

  And you can actually, there's like really cool science, science questions around this of like, how far can you push that? How asynchronous can you be? How off policy can you be? Can I be sampling problems using policy, you know, step one or something, or sorry, policy, some, some earlier policy.

• 57:29Mario Gabriele?

  Yeah.

• 57:30Speaker 00

  While I'm training like the last policy. So this is where a bunch of the ML dynamics come in or how do you make these off policy corrections where you're trading off? Performance and speed and throughput versus like ML dynamics and, and then stuff like that. So these are the kinds of like infra problems that a lot of our team is working on. It's like, how do we make this stuff really performant? And it really matters for longer and longer fries and tasks, right? Like, you know, if you're running your model for like an hour, your total training runs can be super, super long if you're doing things synchronously, right?

• 58:04Mario Gabriele?

  Yes.

• 58:05Speaker 00

  Train inference, train inference. You really do want to do things super asynchronous in order to like, cut down on your total GPU time.

• 58:13Mario Gabriele?

  You're sort of optimizing around.

• 58:15Speaker 00

  You're layering it.

• 58:15Mario Gabriele?

  GPU time, because that's like the most constrained.

• 58:18Speaker 00

  And so this is the other cool thing, right? It's like, so there's that that side of the house, but then there's also environment creation, right? Why do people care so much about performance sandboxes and things like that? The reason is because if you're waiting on CPU or clothes, if you're idling your GPUs for CPU or clothes, you're probably wasting a lot of money. So there's like a ton of performance engineering on the environment side of the house to make training faster as well.

• 58:44Mario Gabriele?

  You know, as you were talking about this sort of imperative, for every company to own their own intelligence, it sort of occurred to me, maybe this is a very naive thought, like, isn't the sort of logical extension of that, that at some point it'll also trickle down to the human level? And all of us will need to have our version of that. Like, does, you know, does AC in five years, you know, start to become, you don't need a forward deploy engineer, but you have a version of it that's like more self-serve?

• 59:11Speaker 00

  Yeah.

• 59:12Mario Gabriele?

  You know, each of us has our, our own model?

• 59:15Speaker 00

  Yeah.

• 59:15Mario Gabriele?

  Own set of, own set of models?

• 59:17Speaker 00

  Yeah, I mean, we've so far, we've been pretty enterprise facing, but maybe this is, this may be the market number two.

• 59:21Mario Gabriele?

  Excellent.

• 59:23Speaker 00

  No, I think you're right. I think a lot of people have been very excited about this idea of like personal AI. And I think maybe a bit of a jump to like a broader point, but if you think about personal AI, that's like, you know, something that deeply understands you, how you work, and maybe running all the time, right? So I think maybe a broader point is just like, as these things start to trickle down, the demand for compute and inference is going to like balloon like crazy. You know, that, that, that's something that's like, kind of like a broader, maybe, I actually don't even know if the market is fully pricing in how much compute that we'll need.

• 59:59Mario Gabriele?

  Yeah.

• 1:00:00Speaker 00

  All of this stuff. Like if everyone has their own personal models and that are constantly being trained and inference and things like that, like, we just need a ton of compute.

• 1:00:09Mario Gabriele?

  Which I don't know, it seems likely that will happen to me. Maybe that's, maybe that's too, maybe I'm.

• 1:00:16Speaker 00

  No, no, I think, I think people totally want it to happen. The problem is, I mean, I think they're just like, right? There are like physical limits, right? Like to how, you know, how much the supply chain can accommodate? And like, not even like on the chip, like, you know, there, there are ASML machines, like lithography machines that like take a really long time to build. So that's why I think like, you know, we're in for, this kind of ties back to like token apocalypse. We're going to be in a compute crunch and inference demand is going to crank like crazy. The ability to train Pareto optimal cost performance models, it's actually going to matter a lot.

• 1:00:54Speaker 00

  And I think we're seeing the early beginnings of that and it's just going to get worse until, until people really start to invest in this.

• 1:01:02Mario Gabriele?

  What's your sort of, uh, you know, not AI 2027, but your view on the next couple of years? Like, what do you think the world is looking like from your perch?

• 1:01:11Speaker 00

  You know, I know there's a lot of folks who are like, oh, 50% of jobs are going to go away in the next two years and things like that. I think like, personally, I, I think that is, is not going to happen. Um, and it, it probably is inspiring a lot more fear that maybe it doesn't.

• 1:01:27Mario Gabriele?

  Is maybe misplaced?

• 1:01:28Speaker 00

  Yeah. I, and I, and I think it's coming from a good place, which is just like, we need to be prepared for this sort of stuff.

• 1:01:34Mario Gabriele?

  Yes.

• 1:01:34Speaker 00

  Kind of being on the front, front lines, um, like doing this forward deployed work and whatnot. I think just the diffusion of AI into the real world take time. There's a lot of change management that needs to happen. You know, companies are like large institutions that take time to adopt new things. And we were talking about this before, right? Like OpenAI is this, you know, it's OpenAI, the research and deployment company. And, you know, deployment is so hard that they created a company called Deployco, the OpenAI Deployco, right? And same, same with Anthropic and stuff like that. So I think like, people know that it is going to take some time to go and deploy this stuff. So I don't think overnight everyone's going to lose their jobs. There will.

• 1:02:19Speaker 00

  Be like areas of the economy that I think will experience more change than others just due to the nature of the job. But to me, this is like a multi decade long rollout and diffusion of AI into the real world.

• 1:02:31Mario Gabriele?

  Wow. What has surprised you most in terms of, you know, that change management? Maybe the slowness of some of these enterprises, like what has maybe could you? You couldn't have counted on?

• 1:02:42Speaker 00

  I mean, in some ways, we knew this would happen, but just seeing it up close. Makes a lot of sense is there are just established procedures inside inside of companies. And one of the difficult things is like, you know, to change those procedures, you often have to touch multiple layers of of management and and the actual people who are doing the work. And it is just a difficult coordination problem, right? And and I think the other thing that is also obvious, but we've just, you know, Michael Chen on our on our go to market team put out a really good like field notes guide to what for deployed engineering has been like is like, no one is data ready. Absolutely. No one is data ready. Things are like, obviously always very messy and whatnot.

• 1:03:26Speaker 00

  But if data is kind of everything, even just the state of where it is and even the best companies is it's not where it needs to be to actually go and make any sort of deployment extremely smooth right now. So I think there there are just these natural sort of roadblocks that will slow things down when you're trying to roll out AI and like big organizations.

• 1:03:48Mario Gabriele?

  Well, I always like to end with a couple sort of thought experiment questions. If you were given the chance to conduct any experiment with no operational constraints and no budget constraints, what's an experiment you'd like to run?

• 1:04:02Speaker 00

  Oh, this is a fun one. We talked about in the in the office, which was if we all reverted to to to cavemen and like lost all modern technology, how long would it take us to reproduce the iPhone? So knowing what we know now, how long would it take us to get all the infrastructure in play? Like, No fabs, none of this stuff.

• 1:04:27Mario Gabriele?

  Man, that's really.

• 1:04:28Speaker 00

  Just sand.

• 1:04:29Mario Gabriele?

  I need to think about that for like a day at least.

• 1:04:32Speaker 00

  Yeah.

• 1:04:33Mario Gabriele?

  I don't know. What did you come up with?

• 1:04:35Speaker 00

  Well, so, and I guess the darker side of this is you have to factor in everything. So like people, you're going to war against each other, things like that.

• 1:04:45Mario Gabriele?

  Yeah, yeah, true.

• 1:04:46Speaker 00

  If we were just all lost, all modern technology.

• 1:04:49Mario Gabriele?

  Hundred percent.

• 1:04:49Speaker 00

  I concluded that we just never would.

• 1:04:52Mario Gabriele?

  You think we would never happen again?

• 1:04:53Speaker 00

  I think we just, uh, yeah, like first of all, most of the population, this is getting a little weird, but I think most of the population would not just like know how to survive the first like.

• 1:05:01Mario Gabriele?

  Yes, I think that's right.

• 1:05:02Speaker 00

  Sixty to ninety days and then, um, you know, we'd have to, there's the ability to go and build all this infrastructure takes a ton of time, you know, if you sort of were thrown into chaos and dystopia, like people are obviously going to get mad at each other and create factions and fight. And so I think we'd spend most of our time fighting than actually building a thing.

• 1:05:23Mario Gabriele?

  Wow, I'm gonna be thinking about that for a while. That's a good one. Okay, sort of last question then, I suppose. If you could assign a book to everyone on earth and you knew they would read it and they'd understand it, what would you want to give to people?

• 1:05:36Speaker 00

  I really like this, uh, book called The Design of Everyday Things. It's actually one of the first tech books that my brother, brother gave me. And I think it really set like a good framework for how to think about solving problems and in non-complex ways. And here's like a crazy segue for you, right?

• 1:05:56Mario Gabriele?

  Oh, great. I love it.

• 1:05:56Speaker 00

  I think in today's age, when you have AI tools, particularly coding.

• 1:06:02Mario Gabriele?

  Yeah.

• 1:06:02Speaker 00

  You can actually, by going faster, you can, you can actually be going a lot slower. So like taking the time to, like, the one thing I'm worried about people losing is this ability to actually deeply think about how things should be architected and designed.

• 1:06:17Mario Gabriele?

  Yes.

• 1:06:17Speaker 00

  Because if we rely too much on AI tools and, you know, I think every company has seen this in some capacity with like this AI vibe coding or whatnot. I'm, we're all for AI. I mean, my main thing that I worked on at OpenAI was Codex. But I, I think there is truly some atrophying happening by relying too much on AI tools. So, yeah, back to the basics, back to the roots.

• 1:06:39Mario Gabriele?

  Yeah, I mean, things like judgment, taste, all of that stuff is, is very easy to stop.

• 1:06:46Mario Gabriele?

  Actually building when you have these tools in front of you, right?

• 1:06:49Speaker 00

  Totally. I think the best engineers right now are the ones who are like people who like learned how to code before AI because they actually deeply understand systems, thinking, and things like that. And that would be my worry as like a new grad engineer or something like that is getting too reliant on AI tools. Because at the end of the day, I think humans will be the overlords over all of these AI tools and you will still need to do, you know, critical thinking to know what you want to build. They might help you build it, but you need to know what you want to build.

• 1:07:23Mario Gabriele?

  Well, I think that's a that's a nice optimistic place to end. Thank you so much. I really enjoyed that.

• 1:07:27Speaker 00

  No, thank you. This was awesome. Yeah. Thanks for having me.

• 1:07:31Mario Gabriele?

  That's it. Thank you for listening to this episode of the Generalist Podcast. Please subscribe on Apple Podcasts, Spotify, or your preferred podcast app. Ratings and reviews help others discover these discussions. So if you enjoyed the conversation, I'd be grateful if you could take a moment to leave one. For all past episodes and more, visit us at degeneralist.substack.com. See you next time as we continue to explore the future.