Seven creators, $1,000 a week in AI tokens, one summer

Reid Hoffman and Parth Patil argue that if your AI experiments always work, you’re not pushing hard enough. Wrapping up the Tokens to the Future series, they revisit seven grantees’ experiments with $1,000 a week each in tokens. Their conversations span Dungeons & Dragons worldbuilding, an iOS game built in a weekend, and personal agents whose usefulness compounds over years. They explore how AI can shift more people from execution-focused “cog jobs” to creative “spark jobs,” and why expertise can sometimes make new possibilities harder to see. Throughout, they ask what to delegate and where story, pacing, and judgment still demand human direction. Plus, Parth’s example of ChatGPT picking seven for a “random” number raises a bigger question: how do you get surprise from a system built to predict?

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Reid Hoffman?

  You want to combine the superpowers of these AI agents with our superpower. The likelihood that we actually, in fact, are much better in collaboration is high.

• 0:09Speaker 01

  Arguably, probably the most important agent is that personal agent, the one that is looking out for you.

• 0:14Reid Hoffman?

  How do we orchestrate to something that's really new, that is improbable? I think that there is a much more enduring role for us. You know, one of the things we've been doing this summer has been this great Tokens to the Future program. And, you know, Parth has been the primary architect of this because it intersects his, you know, personal mission to try to get everyone understanding how to get on their, you know, surfboards and, and, you know, kiteboards and efoils and everything else, you know, windsurfers and, and, and get into AI. And, you know, undoubtedly, it's a bunch of the token grantees have been a bunch of people Parth's met and are doing creative things.

• 0:58Reid Hoffman?

  So Parth, why don't you kick us off on our, on our, our summer, you know, kind of a wrap up?

• 1:05Speaker 01

  This year, this summer, we, we did some very experimental, something, something new and interesting. We ran this token grantee program. And we picked seven grantees this season. We picked people from a wide range of industries, everything from creative to media to security, coding, game development, robotics, really picked a pretty wide group of people. And we basically deployed a thousand dollars a week in each person's hands to, to deploy AI, to, to experiment with technology, see what's possible, see what new capabilities are coming online, explore the frontier. And I, I mean, there, I had my expectation going in like, oh, this will be fun, you know? I think, I genuinely think you have to burn tokens to learn tokens.

• 1:49Speaker 01

  You have to like, you have to, it's a 10,000 prompts, right? You have to put 10,000 prompts in to understand even a fraction of what's coming out of these models, especially with how good they're getting. And I can't do that alone. So I, I had to pick people I thought that would help me map this out. People that had their own, you know, unique superpowers and interests and passions. People that could see the many frontiers of, of this, this kind of like moment in AI. And so we created this program. And I think it's been, you know, it's, it's been very eye opening. It's, it's surprised me in so many more ways than I expected. And I guess like, I'm really excited to have, you know, I'm really excited to talk about it and, you know, recap this and hopefully inspire others to run similar programs, you know,

• 2:34Speaker 01

  others to also pick up the tools and experiment and see where the frontier is headed.

• 2:38Reid Hoffman?

  So, you know, you handpicked this group of grantees, which I think was the exact right way to start. And, you know, again, as was mentioned, it's kind of like, it's, it's, it's different zones, but in depth and with, and, and a willingness to be, you know, bold pioneers across this, this kind of AI landscape. So looking back at the season, you know, what patterns show up for you across them? And especially those that you didn't design for, because you had a going in theory. Like what, what, what, what, what emerged?

• 3:16Speaker 01

  One thing that surprised me, okay, maybe I was biased towards the tools that I had already used. And I was like, oh, you know, people, everyone's going to want to use this model. And then I realized like, well, actually like the different, the, the best model for the task might be different. Or, you know, the best model for the workflow might be different. And so I was like, and just watching where people like, oh, I'm going to spend, you know, I'm going to spend the tokens on this set of video models, this set of image models, because it unlocks this new format in storytelling. Or I like, these are the coding agents that I like. So it was very interesting getting to see like, oh, you know, why do you like that agent? What's, what's special about that agent, you know, factory AI, like the factory approach to software? Why do you like this versus like a Claude code or a Codex? So seeing a wider range of the, because I can only see as far as the tools that I use, but then seeing all the other tools that I'm missing.

• 4:02Speaker 01

  It's, it shows that there's, there's a pretty healthy ecosystem of options out there. But I think one of the interesting patterns, I think every single grantee demonstrated was that independent of where AI is right now, they're kind of projecting out the capabilities over a six month, two year, three, four year timeline, right? So AI is like good at X, not quite good at Y. But they're all aware of those limits in the current form. And none of them are like, well, it'll never be better at that thing. They're actually, there's this common mindset of like, once it can do this, then these are the seven things that we're going to want to do with it. And so they're kind of like drawing that exponential out a little bit and helping us paint a picture of like,

• 4:44Speaker 01

  well, when you can, you know, if you can use a, use a video model and two people can tell a short story, well, how far is it from, you know, what kind of short story? Can you do science fiction? Can you do, you know, can you do a Western? Can you do a Western sci-fi? And then how many people can make like a longer form movie? And so the connecting the dots, helping us connect the dots on like where this is going over a three to five year timeline is something that is probably, every single person did in their own way, in their own domain, right? So whether it was robotics, whether it was storytelling, film, game development, coding, we can kind of see a little bit further into the future because people are in the areas they're very passionate.

• 5:26Speaker 01

  They're able to kind of see and connect the dots on some of the, what the model capabilities are going to make, bring online in the next couple of years.

• 5:33Reid Hoffman?

  We're going to dive into, I think, some of these specific patterns. But before we get there, I think one of the things that's important about this kind of frontier work is what it means to be AI native. And, and I think part of the thing to, to kind of go into kind of what AI native is, is, is what does that mean how you operate as an individual? You know, kind of, you know, and kind of my classic, all the way back to startup of you, you know, kind of stuff is OODA loops and decisioning and activity and, you know, exoskeleton, you know, how do you become, you know, you know, I am Iron Man, you know, kind of as an, as, as an angle?

• 6:17Reid Hoffman?

  And I think one of the things that we saw is it's not coding pedigree. It's not per se technical background. It's a, it's kind of a mindset.

• 6:29Reid Hoffman?

  And so what did, across all of this, you know, what, what did you kind of say? Hey, this is how my sense of being AI native evolved, you know, from what, what, how? Because actually you were also asking questions, you should say like how you are AI native? Then how that evolved and then how that, how we should be helping people think about that?

• 6:52Speaker 01

  Yeah, I think, you know, there's the, there's the themes of the kind of, if I think about the shape of the superpowers that are coming online, there's the superpowers in automation, you know, being able to automate things using agents. But then what should we be automating? And then that's a judgment call, right? So the superpower of being able to use code to, you know, blitz through cognition is really interesting. But then what becomes more important is like, well, when should we not do that? When should we use our judgment, you know, now that we can scale our, scale our cognition? You know, what are the things that uniquely require the human, human taste and the, the human experience?

• 7:32Speaker 01

  And I, and, and part of this is like not thinking of everyone as like, I'm not just a data analyst, right? You're not just an investor, like we're, we're actually very multifaceted. And, you know, you have an engineer, but it's not just an engineer. It's much more interesting when you take that person and you look at them as like a multifaceted, you know, person with many, many passions and interests. And then when you think about the more general human, the general human with many, with many different facets to them, the way they use AI will always surprise you. Because we're not one dimensional people, right? And so this is why I thought it was really important that we would bet more on people and not on like specific roles or categories, right?

• 8:16Speaker 01

  Because people always surprise you when, when you kind of like, uh, give them a chance to explore and expand their, the way they, the way they think.

• 8:24Reid Hoffman?

  Let me add a little bit to this, I think.

• 8:26Speaker 01

  Sure.

• 8:27Reid Hoffman?

  Because I think part of, you know, one of the things I can add, since you've selected all the people is, is, is I think part of the thing is people frequently think of work or process to mechanically versus organically.

• 8:42Speaker 01

  Right.

• 8:42Reid Hoffman?

  And it, and, and a little bit of what I like about kind of investing is, you know, one of the major things I like about investing is, is, is kind of betting on people. And I think that part of what we saw is we kind of went across all of these, you know, very different fields, you know, film, social, VFX, security, gaming, energy. You know, the same shift showed up in each of them, but that's because the people are being pioneers across them. And I think that part of the thing is getting this kind of shared mindset, this kind of curiosity, exploration, pioneering, willingness to experiment. I mean, like one of the things that, you know, I try to give people advice is if you're not trying to do things with AI

• 9:25Reid Hoffman?

  that don't work, you're not trying hard enough on the edges.

• 9:28Speaker 01

  Right.

• 9:28Reid Hoffman?

  And like doing things, you shouldn't wait until, oh no, I'm going to wait until I know exactly what works. And then I'm going to do that. It's like, no, no, you, you're like, we've all landed in this, this, this, this, this like magnificent new world with lots of new capabilities, lots of new possibilities. And so it's people doing them. And I think that's one of the great things we did with, you selected a great group of token grantees and we saw their curiosity and their boldness and their willingness to set off on new terrain and new journeys.

• 10:02Speaker 01

  So going back through everyone's conversations, reflecting on the summer so far, I think like at least two totally different ways people use the tokens, use the intelligence jumped out to me. And a lot of that actually went into building, a lot of people were building games. And I think, you know, I like games, you like games, maybe I pick people that also like games. These are the people we like. But what do you make about that? Like, what do you make of that? Like, a lot of people were like Dungeons and Dragons inspired in the past and like how that, is there, is there, is there a connection there to like, why people make things?

• 10:39Reid Hoffman?

  Well, you know, there's this, one of the things that comes into kind of talking about human beings, humanity, is there's these different articulations of theories. One is Homo sapiens, we have thinking. People, one of the ones that's also been written as Homo ludens, like we are game player, we're game playing, and that's kind of basis of how we do things. I've of course written about Homo techni.

• 11:01Speaker 01

  Homo techni, yeah.

• 11:02Reid Hoffman?

  Right, we're technological. And I think they're all very good lenses on this stuff. And I think one of the reasons why in this particular thing, ludens plays out early is because part of games is, it's like, it's how we kind of trial things in simulation. It's part of how we, we, we kind of learn new dance moves. It's part of how we, we kind of have curiosity and explore. It's one of the reasons why like, you know, some of the best theories of, of education and how people learn is through exciting their curiosity and kind of game playing. It's like, you know, how do, how do you make education, learning like a game? Right? As a, as a way of doing it. And so that doesn't surprise me that these naturally

• 11:45Reid Hoffman?

  curious and bold people are games. Now, I do think that it's, you know, as with the,

• 11:53Reid Hoffman?

  a number of these folks, the fact that there is a gaming overlap with you is very entertaining. The fact there's a clubhouse overlap with you is very entertaining. Because by the way, early people going in a clubhouse, it's also, it's not per se a game, but it's a kind of pioneering and curiosity and, and a willingness to try something new. It's like, you know, most game players don't want to play the same game again and again. They want to kind of new people, new explorations, new levels.

• 12:19Speaker 01

  What's the next game? What's the next game, the next level? I think there's another aspect, which is that games are more forgiving. And it's like, failure is not like catastrophic in a game environment. And we learn from our failures. So like, when you have these like environments or projects where we're just trying to see what the tool can do for us, and we create an environment that failure is not going to be catastrophic. It's not going to, you know, you're not going to lose your job if the game doesn't play out, like the thing that you're, you're doing doesn't play out if it's a game. And then you can try new things with like low cost of, of error. And I think that, that, you know, the first thing you should vibe code should probably not be like hospital software.

• 12:59Reid Hoffman?

  Should certainly not be.

• 13:01Speaker 01

  Should certainly not be.

• 13:02Reid Hoffman?

  I think your point is exactly right.

• 13:03Speaker 01

  Right? You know, you want to give yourself an environment where like, okay, we're going to make some mistakes and it's going to be okay, because we're learning a bunch of these new things before we move on to things that are more serious and more, you know, where the ramifications are much, much more serious and like more expensive, right?

• 13:19Reid Hoffman?

  Yeah, a hundred percent. And I think actually, by the way, it's partially again in learning, it's, it's, and part of the reason why we're doing the Tokens, the Future program is, is that learning and experiment where failure is cheap and quick.

• 13:31Speaker 01

  Yep.

• 13:32Reid Hoffman?

  And then you're learning the things that matter. Now, games itself is an important area, because I do think it's part of, like, you know, part of how we think is we have like mental models of things and we learn, you know, kind of how to do them. And games are part of the environment that we set up. Games are also, you know, one of the threads I think was going through our discourse was, you know, kind of single player, multiplayer, you know, kind of how does that play into things? I think that the, you know, questions around, you know, how to think about, you know, anything from, you know, kind of like, you know, you know, Matthew's full iOS choose your own adventure game built in a weekend with Fable,

• 14:15Reid Hoffman?

  or, you know, you know, Kitty's poke attacks and imposter style multiplayer game. I mean, the, like, like, like also building the games gives you a really rich environment.

• 14:28Speaker 01

  Yeah.

• 14:29Reid Hoffman?

  In each of these, these kind of vectors, which are kind of parallel cosms. I don't know about microcosms, they are microcosms in the game, but also parallel cosms to kind of human life and social interaction.

• 14:42Speaker 01

  Right.

• 14:43Reid Hoffman?

  So I think that's part of the, part of the reason why games is not just a part of selection, you know, criteria, but actually something that's relevant to the material.

• 14:54Speaker 01

  Yeah, yeah. And I even think about Jonathan Breszo, he, he, he's running a Dungeons and Dragons campaign, early, in the early days of stable diffusion, generated 40,000 images because he was like, I need to, you know, flesh out the rest of this world. And I want the campaign to have like a world they can imagine in their mind and, and see and be a part of. So yeah, the volume was a very interesting theme, right? Like how much people, just how much people can, will generate when they, when, when they can. And then I think like the autonomy is an interesting aspect of like people who are delegating to agents that'll work all of, like all night long. So like scaling their effort by delegating to autonomous agents. You know, I do that myself, but seeing how other people do that

• 15:39Speaker 01

  and the, the different ways they're doing, it's like, I want you to research, you know, Joe, I think, you know, Joe Salvatore, he was, he asked one of our agents to spend all night researching successful media techniques over the last 80, 90 years. And like, you know, what does it take to create a meaningful advertisement that's a timeless kind of principle of, of, of, that you can extract from, from history? I never thought about doing that. I mean, I was not, I was, I wasn't thinking about media in that way, but seeing how the thing that someone else will ask an agent to do always surprises me, especially the more different they are than I am, right?

• 16:15Reid Hoffman?

  So let's kind of talk a little bit about life agents. So, you know, everybody kind of loves a life agent, you know, personal project I've always on, you know, it's part of the, the theory around inflection AI, and what, you know, Mustafa, now Sean White and crew are, are kind of started and doing. So how is your theory of life agent, you know, kind of advanced? And, you know, what, and, and how is the, the kind of question about like the, the, the fact that actually, in fact, it's not, it's, it's very rarely the

• 17:00Reid Hoffman?

  people who are kind of deep experts in something that are adopting, but it's actually more beginner's mind, you know, not veterans, you know, give some, you know, fill out this painting some.

• 17:15Speaker 01

  Yeah. So it's interesting, the, the like personal agent, the, you know, in, in the case of inflection, the EQ just as much as the IQ, right? I spend a lot of time with coding agents and they're very much IQ maximized systems. But it's the one, the ones, I guess, like the, the agent that is the most meaningful to me is the one that thinks about my wellbeing, thinks about my life, helps me plan, you know, my commitments, helps me like negotiate for a better deal on something. That's like covering my blind spots, right? And I think that the, this is arguably, probably the most important agent is that personal agent, the one that is looking out for you. And what I've noticed through this program, through,

• 17:58Speaker 01

  you know, the, over the course of the year, as the personal agents have finally gotten very good, the open claw, the Hermes, and then how, you know, various grantees are using Claude, using remote control, you know, being able to tell an agent that's on your computer to go do something for you when you're on your phone. So the way that these things are now kind of like in our own lives is, it's like, at least we see in, in the token grantee program, we're seeing, these are some of the earliest adopters of the most powerful personal agents that have ever existed. And I think especially when you're alone, or when you have a small team, or you're building your own business, kind of like the startup of you, that first personal agent is the most important. It's, it's your first employee.

• 18:41Speaker 01

  It's your, it's your EA, it's your co-founder that's like on this journey. And I think we're in the very beginning of understanding this, because as they accumulate memory over a couple months to a year to multiple years, they start compounding in their usefulness. And I've seen, I've seen this in just since February with my agent, but I can't, I can only imagine what five years of experience working with, you know, a personal assistant is going to feel like, just how equipped you are, right? Every day I wake up and it's like half of the problems that I'm, they're on my plate, the 25 notifications, half of them already have suggestions on how to, how to, how to move forward. So I feel like there's this like proactive momentum that I can lean on, that's an exoskeleton of a sort. And I think

• 19:26Speaker 01

  through the program, I've seen this is something that's happening across the ecosystem. It's not just coding agents, it's actually the personal assistant is finally here.

• 19:34Reid Hoffman?

  Yeah. No, I think that the, the question, it's, it's, it's, it's, people have a tendency to put it in a box and not realize all the different things. I mean, and it's part of the reason why, like, it's, it was, you know, was awesome, you know, it was awesome that, you know, you know, Matthew, you know, runs a dead kid home monitor with a personal dashboard and, you know, remind me about Lakers games with the fan and, you know, has a Codex Chief of Staff, you know. Uh drafting, meeting follow ups. And I think these are just the beginnings. And as you mentioned, part of the reason why to start on this pattern is, you know, we, we as,

• 20:15Reid Hoffman?

  you know, kind of as a, as tool users tend to go, well, let's wait for the tool to be finished, shaped, and then learn the expert shape. And it's like, no, the, this uh tool is going to be in dynamic reformation and dynamic reformation with you.

• 20:30Speaker 01

  Yes.

• 20:30Reid Hoffman?

  It's one of the reasons why.

• 20:31Speaker 01

  So almost every week, like it is getting interesting more, you know, it earns more access to my life in a way that's like useful and compounding. Yeah.

• 20:40Reid Hoffman?

  Yeah. And so, um, and you know, part of the, you know, and this is again, part of the reason why we decided to do this, you know, as, as part of the possible podcast is because it's, it's, it's a reshape of what's possible, right? Like part of what's going on with the kind of the AI exoskeleton skill is a reshape what's possible. And it's one of the reasons why non-experts um have actually, in fact, um, some advantages here because when you become expert, part of your expertise is you learn what's like, this is doable, this is not doable. This is, this is possible, this is not possible. When the possibility landscape um shifts, you have to rethink that. You have to kind of rethink the, wait a minute, what is now possible and not possible, what is

• 21:24Reid Hoffman?

  now doable, not doable is different. And, and frankly, it's changing relatively often. And so as part of that changing relatively often, you need to be um, you, you, you need to be learning and adjusting and no one will tell you, oh, just end here. We're going to discover this. And it's one of the things I like about entrepreneurship, but called pioneering is, is like only through a pioneering process. And so I think the uh, one of the things is not only begin with a beginner's new mind, but to continue with a beginner's mind. It reminds me of like, again, one of the things that Ben Casnoca and I said in the start of you, which is, you know,

• 22:06Reid Hoffman?

  permanent beta. It's like you're always in process, never complete.

• 22:12Speaker 01

  So let's, let's revisit a few scenes from the summer and, you know, like dive into some, some of what we, what we noticed and what we saw. Um, I think one of the, you know, we had Ben Hansford on, we had Ben Hansford, uh, professor of film at USC. And such a, I mean, I, I keep rewatching that conversation because I, every time I watch it, I learn something more. It, he, his, his perspective, you know, being early to AI in LA, in, you know, in film and entertainment, um, really interesting perspective and as a teacher, right? So working with people much young, working with kids, working with students much younger than him. So he has this, like, he feels his own age sometimes holds him back and then his students surprise him, right?

• 22:56Speaker 01

  So, but then he thinks about like how he thinks about AI. It's like not a tool, but that he starts thinking of it more like his team. He's got the Claude, he's got the Codex, he's, he's, you know, he's firing off the, his projects are kind of like circulating between the agents around him. And I think um, he had an interesting line, what he said, he said, you know, a hammer can't build a bird box while you sleep, you know? So then the, the, the AIs are kind of like this, like navy teal seam. You give it a mission and then it comes back to you with like, ah, here's what we've done. And, and that was very, that's very exciting. I think especially because up until now, it seems like, you know, a lot of people, most people are interacting with AI like it's a search engine. They're still asking it questions about the world, but they're, you know, Ben's

• 23:38Speaker 01

  already at this place where he's like, no, no, no, these things work for me. When I ask them to do something, they're going to go take a shot at it and then they're going to come back with some completed work output, something new. And no, it's really exciting to see someone outside of software, outside of Silicon Valley, using agents in a way that's extremely like, starting to think about them as this like personal team, personal infrastructure.

• 24:00Reid Hoffman?

  Yeah. And I think, yeah, look, I think part of, I like, I agree. Um, because you know, a, a natural way to start is to be thinking that you're, you know, a conductor, you're a director, you're an orchestrator and like the, the, the team of agents, the swarm of agents, the work process of agents. I mean, this is one of the things that, you know, um, I started thinking about in our earliest conversations, you and I, you know, as part of starting to work together. And I think that the, the question when it kind of comes down to this is to say, that's a very good lens and a very good way to start. And like, if you don't have anything else, start there. I do think it's interesting, like, you know, part of what you've got, you and I've also

• 24:44Reid Hoffman?

  had as an ongoing conversation is kind of this question around, like, it's natural to think it's a crew and to deploy it as a crew and make it work. And there's, there's features that it has that human beings don't 24/7. You know, one of the weird things when you work with these chatbots and agents is they just do it better and it just does it better. Whereas a human goes, what do you mean? Like I, I did, I did the best thing I could.

• 25:05Speaker 01

  They're relentless. They can like clone themselves and then paralyze across the problem, which is not human like at all.

• 25:12Reid Hoffman?

  Exactly. And, and a lot of those things are features. And it's like, you, you need to be adapting to that feature. It's different than a human team. Now, obviously, one of the things you have to kind of, like, as, as you know, one of the, the kind of metaphors I think is it's an alien intelligence that has learned and, and trained deeply to be human.

• 25:31Speaker 01

  Yeah.

• 25:32Reid Hoffman?

  And one of the places you have to say is like, well, that gives us a bunch of superpowers, like some of the ones we just gestured at. But it also gives us some weird weaknesses, like can break into a lacuna, have bad context awareness, um, not realize, you know, this is like the hugging face thing. No, no, reward hacking this way is not what I want you to be doing, right?

• 25:55Speaker 01

  Yeah. We don't want to break the rules in order to get the answer to the test.

• 25:58Reid Hoffman?

  Yes, exactly.

• 25:59Speaker 01

  Yeah.

• 26:00Reid Hoffman?

  And so um, and so the, you know, it's, I think it's, it's, it's one of the reasons why you, you, you kind of both experiment, try things and do, but like, you don't, like, you start with a gaming mindset, like you start doing that to learn which things you do before you do something serious. Like, like if you um, like, you know, for example, one of the things I know you do is you say, look, I would love you to read communications to me and draft stuff, but don't send it until I say so.

• 26:32Speaker 01

  Right, right. I learned that the hard way. And now I'm like, okay, well, we're going to take baby steps, you know, prove to me that you can even write the right email, you know? Then, you know, once I see that a couple of times and it's like, okay, now we're going to build like a little bit more agentic, a little bit more proactive.

• 26:48Reid Hoffman?

  Yeah. No, I know of, of, of a case that went so bad on that, that um, basically by a person being like uh, too enthusiastic and not exploratory and sequential enough.

• 27:01Speaker 01

  Yeah.

• 27:02Reid Hoffman?

  Like basically sent confidential information from their company to another outside party on a ongoing deal discussion.

• 27:11Speaker 01

  Oh no. Oh my God.

• 27:14Reid Hoffman?

  And it's like, what? And it's, it was, wasn't trying to do something, it was trying to be helpful. It was like, oh, I thought this would be helpful. And you're like, yeah, that's like, yeah, you know, where it's like a human would never do that.

• 27:25Speaker 01

  Right.

• 27:25Reid Hoffman?

  As part of the virtue, because they have the conduct, well, almost never. I mean, there may be some nutty person somewhere in eight billion people, but like, you know, very rarely. And so I think the, the important thing is to go, we have these uh quasi-alien, quasi uh human tools that are spectacular and have a bunch of superpowers. But, you know, they don't naturally understand the shared contextual awareness that we have.

• 27:52Speaker 01

  Yeah.

• 27:53Reid Hoffman?

  They may not even understand always what good enough or great looks like.

• 27:58Speaker 01

  Right.

• 27:59Reid Hoffman?

  And they may get trapped in lacunas that we don't understand. But by the way, none of that is, oh, then I should just wait until that problem solved. Because the, the, the amplifier is already so great and intense. It's like, no, no, that's the new way that you orchestrate, that you direct these tools. And I think that's one of the things we saw across all of these folks, but you know, Ben was a particular highlight on that.

• 28:24Speaker 01

  That's right. Learn where their limits are and then figure out how, where they fit in, in a way that's not, uh, that's, that's extremely constructive, makes use of their strengths. It's kind of like the jagged frontier, um, as Ethan Molik calls it.

• 28:37Reid Hoffman?

  Exactly. And, and, and like one of the ones that I think you and I've talked about a bunch. So we'll go to kind of another theme, which is, you know, you can't outsource taste with, you know, uh, Joe Salvatore. Um, and this is a little bit of like, I was gesturing, I was like, like, is it good enough? Right? And it's one of the things, it's part of the same reason why, like a lot of intense AI training to train these, you know, kind of alien intelligences and human is like still using a lot of, you know, reinforcement, learning, human feedback, human data. But it's still the case, there's a lot of, of, of, you know, kind of where our judgment comes in, our taste comes in, our context. And we use these kind of squishy words because it's a

• 29:21Reid Hoffman?

  broad, squishy thing that has a lot of like uh perceptual recognition, intuition, you know, training from judgment. So, you know, what's, what, what were some of the themes on this taste that you saw from our episodes? And what are some of the ways you're thinking about it these days?

• 29:41Speaker 01

  Yeah. Joe, Joe Salvatore, he had a really good um, line here where he, he, he said, you know, AI has the same confidence on a creative task, whether it nailed it or whether it produces slop. And it's just going to come back to you with that same level of confidence. And I've, I, that one, that one has stuck with me. And I think it's very true. Um, the- Especially in the creative spaces, in the creative domains. It's not like one piece of art is better than another piece of art. These things are subjective. So I think in the case of like, if you think of a world, if you think of the world as purely math and coding, and this is all gradable, right and wrong, then the RL thing is interesting, right? Like then the AI can get better at the thing because it's so objective, right?

• 30:25Speaker 01

  We know what right looks like, we know what wrong looks like, and then maybe the AI can hill climb towards right. But then in the spaces that are like messy and subjective, artistic, creative, you know, very much more human, it's where the AI kind of just like, you know, it's kind of like dead in the water at a certain point, right? And then that's when we have like, you know, what we're doing. So like Joe will have it do eight hours of research of what humans have found to be good design over the last 90 years. And then Jonathan will have an image model generate 200 images before he picks one, 200 names for a magical object before he thinks one fits the criteria, right?

• 31:04Speaker 01

  And so actually, that is like, it's like, that's the, it's everything, all the 199 images that you and I never see represents the taste that Jonathan brings to the table. You know, he decided to only show us one of the 200. And that means that like, that curation is actually where his wisdom is coming into play, the creative wisdom, his personal experience, right? And then Katie was talking about how we layer, you know, taste is layered on top of story. And then I always realized, like, especially now that I'm starting to make longer form videos, longer form conversational content through AI, I'm realizing, wow, the, you know, I can generate anything visually.

• 31:46Speaker 01

  We can make it beautiful, we can make it look like anything, we can make it look like science fiction. But will it feel like a compelling story? And I'm realizing, okay, actually, the writing skill set, the pacing, you know, the character design, that is the skill that the AI is not like delivering out of the box, right? And that requires the person, the director, to kind of infuse it with their vision for what a good story is, what an interesting character looks like. And so, and then it's like, what do we do to develop taste? Was, I think Joe, again, had the most interesting take here, which was that you have to consume a lot. You have to see a lot of anything to understand what good even looks like, right?

• 32:31Speaker 01

  And so the only way to train it is to be out there and experiencing the world.

• 32:36Reid Hoffman?

  Yeah, and by the way, I think, you know, I think it was not just here, but in an earlier episode, it's like, you know, writer's skill was the most valuable thing you were undervaluing.

• 32:44Speaker 01

  Yeah.

• 32:45Reid Hoffman?

  And I do think like, these things still don't, like, they can write a Wikipedia entry, which by the way, is a group collection effort that's not particularly edgy, beautiful, et cetera. It could be very informative, they can do that very fast, supremely fast and thorough and everything else. But like writing, like interesting stories and edge and dialogue. And like, for example, I actually saw an investment memo yesterday that was like, okay, so which did you use, ChatGPT or Claude for this? Because I could tell it was basically like, even though there was a lot of like work that proximized what a human analyst could do, there were errors or softnesses or in, like, it kind of was, like, it was like I was filling out a form.

• 33:30Speaker 01

  Yeah.

• 33:30Reid Hoffman?

  Versus.

• 33:31Speaker 01

  It's like checking the box.

• 33:32Reid Hoffman?

  Yes, right, and that's there. And actually, one of the architects that I'm aware of in Japan actually has, I think it's ChatGPT, and like produce 50 images in his style, and then picks the three that he thinks would fit for a project. And that's how he goes into a first meeting to say, and he's a super famous architect. I can't name him because I, you know, haven't got permission. But like, that kind of thing is still involving the taste, you know, as it plays. And it's one of the things I think will persist for some time at least. And it's part of the, you know, what is the future of work? And how do we do it?

• 34:15Reid Hoffman?

  It's one of the things, one of the many different areas to be looking at how do we bring in essential and useful things as humans into working with AIs for, you know, high quality token output.

• 34:29Speaker 01

  Yeah, I mean, you got to think about the chopping block floor and everything that never made it to the public. And that is, that's a huge part of, you know, what stands out. There's the slop, and then there's the curated, like, artistic choice, right? So we had Jonathan Breso, one of the most creative people I've ever met, honestly. And he brought a very interesting framework to the table, which was that, you know, creativity needs a human. And he had this framework of like, the cog, there are cog jobs and spark jobs, where cog jobs are this like executions, logic, not necessarily the most creative stuff.

• 35:08Speaker 01

  And then there's the spark jobs, which is the design, the music, the creative choice, the like, the subjective space of things. I think for me, it's objective versus subjective, where it's like, if it's execution versus like, more of an exploratory choice, a subjective space. I thought that was very interesting, the cog versus the spark jobs. And what kind of tasks are cog tasks versus what kind of tasks are spark tasks? And he thinks that we're going to move people, human beings are going to move to a place where more of us are going to be in this spark jobs kind of role, working with AI. And I tend to, I think I tend to agree, you know, he said, true art is improbable, right?

• 35:52Speaker 01

  And this really landed because I think about like, what is the language model doing? You know, it's predicting the most likely next token. Well, I once asked a language model to generate a joke every minute for a week. And then I looked at all the jokes, and there was only one thing I realized, like, every single joke was a dad joke. And then I was like, why is that? Why is every single joke corny? I was like, oh, because the most likely punchline is the dad joke. The most likely punchline is not the funniest punchline, it's not the, you know, it's not surprising. It can't surprise because it's trying to do the predictable. And then it makes me think, okay, well, they're not, and if you ask the language model, if you go to ChatGPT

• 36:33Speaker 01

  and you say, pick a random number between one and 10, more than half the time, you're going to get the number seven. And then it's like, oh, wow, like, it's not even random because it is trying to predict the most likely number that a person would answer with when asked, pick a random number between one and 10. So you can't, so like, you know, the right answer would be for it to roll a dice and then to say what the dice revealed. But then it's like, we're working with these tools that are predicting, they exist in the predictable space, they're interpolating between what we have. And then that's where it's very, like, the more interesting weird people that we have talking to these systems, they're pushing it out to that, like, out of distribution, like, out of the predictable space.

• 37:16Speaker 01

  And so I think about, like, that's where this, that's how I kind of think about the spark thing. Do you think that, do you think that this is like a feature of only today's systems?

• 37:28Reid Hoffman?

  Yeah, great question. Look, I think, and you know, you and I both know that being overly deterministic about, AI won't be able to ever get to here, it only is, is, even though there will be almost certainly some things in that, to state it with some, you know, probably 100% deterministic or probably 90% deterministic is a little bit of a fool's errand. But I do think precisely because of the general ways that they operate, there is the most predictive token, as you're talking about, there is the, with the mixture of experts and the

• 38:08Reid Hoffman?

  kind of the learning thing, the learning paradigm tends to be the, what is, what is a high quality, high prediction to this? And if you're kind of being vanilla on the prompt, then the prediction is you want something generic or vanilla on the output. We can massage this by being much less vanilla on the prompt. That's part of what we're trying to help people understand to do in tokens of the future, you know, podcasts and so forth. But it's also, you know, you put in workflow, you know, you have, you know, agents that play different roles that then prompt off each other in order to make stuff happen, including like red teaming or make that better,

• 38:51Reid Hoffman?

  or is that good enough and all the rest, so you can improve it along those vectors.

• 38:55Speaker 01

  Like novel remixes of the way we attack a problem or create something.

• 39:00Reid Hoffman?

  Yeah, so we're already trying to like push the limits of this. But I do think that the notion of, like, how do we orchestrate? This is part of the reason why we're talking about taste, is how do we orchestrate to something that's really new, that is improbable, you know, in the kind of the, you know, Jonathan, you know, kind of art and creativity side. I think that there is a much more enduring role for us than, you know, and by the way, even as we kind of architect the kind of agents to try to do it,

• 39:42Reid Hoffman?

  it's like one of the things that kind of lived experience and being, you know, kind of growing up in the world and so forth, kind of helps us with. Plus, I think in addition to that, kind of taste and judgment is the context awareness. So I would, generally speaking, think that we've, that that will persist much, much longer than, than, you know, kind of, the kind of AI maximists will think that it does. And it might persist, you know, you know, air quotes, forever.

• 40:14Speaker 01

  Forever. Yeah, yeah, I mean, it's like the AI is just chasing the weirdest of us, but like, we're the frontier. Like, I'm in the real world, I'm constructing an interesting life and then exploring. And then the things that I would do, it can't because it's, it's like reading, it's like the thing that someone once told me that these models, it's like they sat in a library and read every book, but they never actually ventured out into the world. I mean, eventually they will, and then they're going to be learning from the world. But right now, it's like, they have this theoretical map of like, how everything works, and.

• 40:50Reid Hoffman?

  Yeah, 100%, and let me kind of add a, kind of a, call it a, you know, a vision, not really a hope or an aspiration or anything else, but not really a full theory. But it's like, look, you want to combine the superpowers of these AI agents with our superpowers. And the theory that we have no superpowers is an interestingly articulate theory. Because like, part of it is, well, we see these new amazing superpowers from the machine that we used to value uniquely ourselves. You know, ability to reason in symbols, the ability to, you know, operate in cognitive tokens. And you go, oh, it has all that, oh, is there any rule for us?

• 41:30Reid Hoffman?

  And I actually think that the likelihood that there's some areas that we actually, in fact, are much better in collaboration, let's just use the frame of token production, you know, working with it is high, you know, can be lensed a couple of ways. One we said is taste, one we said is, you know, kind of context awareness and judgment. But like, another one is, like, what most people on track is, in that, in a watt expenditure per token, we are geniuses compared to these AIs.

• 42:06Speaker 01

  Yeah.

• 42:07Reid Hoffman?

  Now, the good news for using the AI is we can put terawatts behind the AI, and we only have 20 watts here.

• 42:13Speaker 01

  Yeah.

• 42:14Reid Hoffman?

  But to say, hey, look, that's not just the cheapness and efficiency of token, there's also something about the way we do it that will likely have some useful advantages in, you know, kind of collaboration. And obviously, we want to be the, the kind of human in the center, you know, kind of producers of kind of value in this. But like, using AI to amplify our humanity as much as we can.

• 42:43Speaker 01

  Absolutely.

• 42:44Reid Hoffman?

  Possible is produced by Palette Media. It's hosted by Ari Finger and me, Reid Hoffman. Our showrunner is Sean Young. Possible is produced by Tenasi Delos, Katie Sanders, Spencer Strasmore, Imo Zoo, Aman Suri, Danny Garrison, Trent Barboza, and Tafadzwa Nima Rundwe.

• 43:02Speaker 02

  Special thanks to Surya Yalimanchili, Sayida Sepieva, Ian Alice, Greg Beato, Parth Patel, and Ben Ralas.