AI is Supercharging Writers with Sudowrite's Founder, James Yu

In this episode, Nathan is joined by James Yu, Founder of Sudowrite, an AI writing tool. They discuss how James started Sudowrite after using GPT-3 for his own fiction writing, how Sudowrite is able to be a supportive writing partner, and Yu's experience developing its tech stack and working through the backlash he's received. f you're looking for an ERP platform, check out our sponsor, NetSuite: http://netsuite.com/cognitive Subscribe to the show Substack to join future conversations and submit questions to upcoming guests! https://cognitiverevolution.substack.com/ We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. SPONSORS: Shopify | NetSuite | Omneky Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. TIMESTAMPS: (00:00:00) - Episode Preview (00:04:28)– How James started using GPT-3 for fiction writing in early beta (00:08:47) - Using few-shot learning to get good results from early GPT-3 (00:10:25) – Focusing messaging on being a supportive AI writing partner (00:12:08) – Helping fiction writers feel less lonely through pseudo-conversations with AI (00:16:01) - Sponsors: Netsuite | Omneky (00:17:22) - Evolving from paragraph-level assistance to full story understanding (00:20:19) - Building the product for centaurs: half Human, half AI (00:32:15) - Using multiple models together to "kitbash" the best outputs (00:39:21) - Iterating on prompts over time to get a customized "garden" (00:45:18) - Keeping temperature high for creative writing, but avoiding cliches (00:54:21) - Separate paths for professional writers vs casual creators (00:57:30) – The continued role of human-centric art and culture (01:00:10) - The potential for automation to replace repetitive jobs (01:07:48) - Yu’s experience launching an AI writing tool and the backlash he received (01:16:24). - Advice for future AI product launches to avoid or minimize backlash (01:18:25) - Focusing on delivering value to customers during controversy (01:28:44) - The idea of an "AI bundle" to reduce churn and customer acquisition costs (01:42:06) - Recommendations for underrated AI apps - Claude, Pika, Midjourney The Cognitive Revolution is brought to you by the Turpentine Media network. Producer: Vivian Meng Executive Producers: Amelia Salyers, and Erik Torenberg Editor: Graham Bessellieu For inquiries about guests or sponsoring the podcast, please email [email protected]

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Sudowrite is serving customers who are capital W writers. I think in this medium term, these LLMs do open up the possibility that, hey, if you've never even considered writing a story or a novel, tools like Sudowrite can help you achieve that. We can let you give us both the 50,000-foot view all the way down to the leaves of the scenes themselves and who's in the scenes. We're not out here to replace writers. We're out here to create tools in cost of writers who are giving us feedback all the time about how this helps their lives and less than be more creative. We work hand in hand with novelists who were literally something like cried, tears of joy when they actually got this into their hands.

• 0:42Nathan Labenz?

  Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas, and together, we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello and welcome back to The Cognitive Revolution. James Yu is a co-founder of Sudowrite, an AI writing assistant built for fiction writers. I first encountered James and Sudowrite on Twitter back in May, when he introduced a number of new product features with a video. And for some reason, presumably stemming from growing fears of AI replacement

• 1:25Nathan Labenz?

  in general, and the ongoing Writers Guild strike in particular, ended up getting an absolute wave of hate and vitriol in response. The video, which has now reached more than 8 million people, has one of the crazier ratios you'll see, especially if you take the time to watch the video and try the product, as I did, and see just how thoughtfully James and team have built Sudowrite. Fascinatingly to me, their homepage headline speaks not to AI capabilities, but to human needs. They describe the product as the nonjudgmental, always available, willing to read 30 drafts, AI writing partner you always wanted.

• 2:04Nathan Labenz?

  It's clear from this conversation that James is someone who loves both AI and writing, and he's building something that helps him write better. So we focused mostly on how they've built the product over the last three years, starting with the original GPT-3, which they do still use in some cases. And then we also cover various language model tips and tricks that may generalize to other applications, my idea for an AI commercial bundle, his experience of anti-AI online dogpiling, and more. As it happened, just as we were recording, news broke that a deal had been reached between the Writers Guild and the Hollywood Studios. And James has since publicly endorsed the AI portion of the agreement, which seems to protect writers' traditional contractual rights and prevents studios from forcing AI tools on them,

• 2:49Nathan Labenz?

  while also preserving the writers' freedom to use AI as part of their creative process going forward. I'm sure we'll talk a lot more about this precedent-setting agreement in the future, but for now, if you're enjoying the show, please do send it to a friend. We do a super wide range of episodes, and my hope is that each one can serve as a niche entry point into the broader AI discussion. But for that to happen, I do need some distribution help from the core listeners. So please, send this one to a fiction writer in your life. And with that, I hope you enjoy this conversation with James Yu of Pseudo Write. James Yu, welcome to the Cognitive Revolution. Thanks for having me. I'm excited to have you.

• 3:29Nathan Labenz?

  You have built a cool product and had a bit of a dramatic summer since you launched the most recent version of it, and excited to get into all of that. I guess I should say, right out of the gate, you are a founder of the company Pseudo Write, and this is a company that helps people create long-form content. And you've been doing it for a couple of years already. I, just in checking you out, saw that some of the quotes on the website were from a launch back in 2021, when you got some really nice coverage, including mentions in The New York Times and The New Yorker and so on. So I guess, for starters, I'd love to kind of hear what you saw early on in the GPT-3

• 4:12Nathan Labenz?

  era and how that kind of translated so quickly into a product. So I think you're very much ahead of the curve relative to most who are getting into it a little bit later now.

• 4:22Speaker 00

  Yeah, I think we've actually been around about three years and it was mostly kind of in private beta at that point. But yeah, as soon as GPT-3 came out, yeah, I was at that time heavily writing fiction, focused on science fiction. And when it came out, I was, you know, I was seeing all these people just like create memes and all these things with GPT-3. And you know, this is like the first model, right? Da Vinci, like version 1.0. But even back then, I sort of asked myself and my co-founder, Amit, like, can this actually be helpful for creatives? And that's really the harder, you know, what we've been working on. And I started using it in my own writing and I thought it was helpful. So then we started

• 5:03Speaker 00

  building Sudowrite really as a toy back in like kind of late 2020. And even in that nascent state of GPT-3, it was the hallucinations and all those things actually were a great feature, not necessarily a bug for fiction writing. So I think that's actually part of why it was working for us. And yeah, we also were very involved with fiction communities and just really started seeding Sudowrite and this kind of way of writing with writing groups. We were just very organically from there. Yeah.

• 5:36Nathan Labenz?

  Yeah, boy, it takes us back into the history of language models, which folks who've listened to this show will, you know, kind of know that progression. And if anybody wants a recap, our Riley Goodside episode is probably the best kind of history of models and how they were fine tuned and how that kind of plays out in terms of how users, you know, can best interact with them. But I'm struck, you know, from the, just going back again to that Da Vinci 1.0, as you said, if I recall correctly, the context window was just 2,000 tokens in that original version. And also, you didn't really have any of the instruct training. So you really had to kind of set something up in this sort of highly suggestive, you know, autocomplete sort of structure.

• 6:20Nathan Labenz?

  And you know, that doesn't give you a ton to work with, right? If you're, if you're limited to that narrow context and trying to write long form fiction. So I'd love to hear kind of how you bootstrap your way into getting that to work for you. And you know, then how you started to turn that into a product where obviously, you know, it does a lot more now than it did then.

• 6:41Speaker 00

  I think a lot of it just came down to user expectations. So actually, our first feature that we had was called Wormhole. It wasn't called, you know, Right or like all these other things. We called it Wormhole because hey, this is what, you know, five other versions of you and other multiverses could possibly write for the next paragraph. And for us, that that really set the tone for like, okay, it might be a little wild. It may not follow your tone exactly, but it might inspire you, right? So you set the expectation that, hey, this is not always going to be precisely what you will write next. So then our target really was when you're just absolutely stuck, right?

• 7:25Speaker 00

  When you're writing something to just get you unstuck. And maybe it's just the wrong thing. And we actually put that on a pedestal to say, hey, that may actually be good. Because if it writes the wrong thing, it maybe gives you an understanding of how you want to write the right thing, right? How, how you want to actually approach this. So yeah, actually, without some that with that expectation in mind, the constraint of the smaller context window wasn't really a problem. It was more that you're at this cursor, it reads whatever about, you know, thousand words or so, and just gives you the next like 500 words. Well, we don't insert it straight into the document. And that's the other thing too, like, and, and, and we still do that today. It's like, we give you cards, so you consider them.

• 8:08Speaker 00

  You should read them over, shouldn't just immediately just insert it. So yeah, I would, I would say it comes down to those constraints. And obviously that's very localized, right? And that's kind of the working with DaVinci was like, okay, it really works very locally. And as the models got better, the instruct models and obviously today, which you can do for, we've been able to widen that lens so that we can help you with a larger part of your story, not just the next paragraph.

• 8:34Nathan Labenz?

  Yeah, very interesting. Do you remember, just out of pure curiosity, any kind of clever prompt tricks that you used in those early days to get the right kind of output back?

• 8:46Speaker 00

  Yeah, it was a lot of few-shot things, right? Just giving examples. And funny enough, I would give examples using, for example, we had a describe tool that was like giving, that would give good descriptions based on something that you highlighted in your text. And that's something like author, fiction authors need to do to get very rich descriptions. And what I would do is I'd give a few shots from my own stories and how I described things. So I would actually say like the first iterations of that tool was probably more like a how James would describe this kind of thing in his stories, right?

• 9:19Nathan Labenz?

  You casked yourself in your own product.

• 9:21Speaker 00

  Right, exactly. But because we also took the context of, you know, what the user is giving them on your script, it didn't matter too much. It was like giving the AI a bit of flavor, like, okay, you want me to go into this literary latent space. That's really just kicking into there. So there's a lot of tricks like that of just like giving few shots, doing a lot of, I would say a lot of prompt, I think prompt engineering is one thing, but I think this prompt evaluation was the big thing. Like, we did so many panels between me and my co-founder and our writing groups that were very invested in helping shaping this product. Getting their feedback on the wide diverse set of fiction and fiction writing was very key in those early days.

• 10:04Speaker 00

  And today as well, we still do that today.

• 10:07Nathan Labenz?

  Yeah, interesting. Okay, well, I want to get into a little bit more of the details of the product and how all this is continuing to evolve. But for a little bit more context, I just noticed the headline right at the top of the homepage, which is very colorful. The non-judgmental, always available, willing to read 30 drafts, AI writing partner you always wanted.

• 10:29Nathan Labenz?

  And I guess, you know, what jumped out to me about that was how sort of non-utilitarian it is, you know, you mean, you could imagine another version of it that's like, this thing writes the best copy, you know, you buy it now, or, you know, some maybe more artful, but still kind of clearly purpose-oriented version of that. This is much more about the need of the author, really. It kind of speaks to, you know, the emotional needs of somebody who's maybe, as you said, stuck in the writing process. I'd love to hear a little bit more about how, you know, was that something that you started with originally? Is that something users have led you to? Who are, I mean, you mentioned a little bit

• 11:13Nathan Labenz?

  about kind of your early users being committed writers, but I wonder how that's evolved over time. But it just really caught my eye as a super fascinating way to position the product.

• 11:24Speaker 00

  Yeah, for sure. Yeah. And this, this has remained consistent. For example, compared to write to like an enterprise writing tool that is trying to help you write emails, right? In that world, you're not really thinking about sort of the, less about maybe the emotional connection, but more about productivity, right? And so that wasn't really the reason why I started using GPT-3. It wasn't necessarily so that I get like more words per minute. It was really about getting me into the right headspace. So based on a lot of our early interviews, it's funny, we're talking a lot of fiction writers and our early beta customers, some of them would say, hey, I love your tool, it helps me get

• 12:04Speaker 00

  unstuck. But actually, the other part that I found is it makes me less lonely. Because as a fiction writer, most fiction writers are loneloves, right? They're writing, I know, writing on spec, they don't have a writer's room. They're not like the top 90 percentile like Hollywood writers, right? So they don't have that luxury. And maybe they have a writing group, like even then, it's very asynchronous. And most of the time that you're spending is yourself, your keyboard, your brain.

• 12:34Speaker 00

  And that's basically it, right? So they began to really like anthropomorphize the AIs than like, hey, this is actually reading my stuff and giving me relevant kind of suggestions. It feels like they are reading. I feel less lonely and a lot of them, some of them are saying like, hey, even my partner, my life partner won't even read my manuscripts anymore. Or, or I'm too embarrassed to show my first draft to my ready group, but I'm not embarrassed to show it to an AI. Right? And so when we started hearing this over and over, I think that's where we really just see it of like, this is not merely like, give me the next few words of my manuscript kind of thing. I think it goes beyond that.

• 13:15Speaker 00

  It's really a different way of you working with your ideas and a way for you to connect, you know, connect with your own work, right? Through the lens, I would almost think about like these large language models as a cultural lens into maybe adjacent texts in the latent space that you may or may not read. Or maybe if you, for example, you go into a library, you read an author that really speaks to you. These language models can fill that space as well. So yeah, we've always, we've always spoke to that part of messaging.

• 13:51Nathan Labenz?

  Yeah, that's really, really interesting. It reminds me a lot of an episode we did with Eugenia, the CEO of Replica. And one of the things that she said, cause they also started, you know, not even just early in the GPT era, but actually pre GPT models with their virtual friend product. And I'll always remember when she said to me, you know, we, we knew, we kind of recognized early on that we might not be able to create an AI that could talk, but we could create one that would listen. And I was like, wow, that's just so profoundly speaks to, you know, the, the sort of. Perhaps often neglected by most technology products needs that people in fact have.

• 14:32Nathan Labenz?

  And it sounds like you have kind of a similar relationship to your users. I think that's really super fascinating. So, okay, so most of these folks are kind of long tail independent, which makes a lot of sense, right? I mean, most fiction is probably written by folks that have never actually sold a book, but are doing it for the love of it. And I can certainly imagine how that would be a bit lonely at times. How has the product evolved to today? I mean, when I checked it out, I think one of the elements that was most striking about it is that you've built up both a pretty powerful hierarchical system for kind of cascade, you know, starting with kind of a kernel of a story idea and then really developing

• 15:17Nathan Labenz?

  that across a number of different dimensions and even kind of cascading down into chapters and then paragraphs and ultimately, obviously, it cascades out into word by word. I'd love to hear how you describe that. And then I also want to get into a little bit more of kind of the different angles that the product kind of presents because there are sort of character based lenses on the work and like you also described this kind of rich description based angle. And I think it's all super interesting. I'm not really a fiction writer myself, so I'm kind of learning about the craft also through studying this product. But yeah, take us through how the product has evolved and how people get the most value from it today.

• 16:00Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 16:04Speaker 00

  Yeah, sort of going back to that Da Vinci 1.0 world where it was very localized. The big unlock that's happened with GPT-4, you know, all the newest models is that we can get now, we're at a point where the AI can really understand a story from the 50,000 foot view. So I see it as like we are evolving from that local like paragraphs to maybe multiple paragraphs to scene by scene, understanding and now to the whole story arc, understanding. And So what we're trying to build up and we're still, I think in the beginning phases of that, of this, like, as you mentioned, this hierarchical thing, the hierarchical, like, the way we structure

• 16:46Speaker 00

  is that if you use StoryEngine, which is our new hierarchical sort of like way of writing with pseudorite, how do we get an author to mind meld with the AI in the right structure, in the right format, so that the AI has the right context at all times? Because even with the kind of the newest context windows that are out there, I know Claude also has like 100K context window, but the pricing doesn't really work out. And in many cases, like, do you want to always be using like your whole novel is part of the inference? Probably not, right? So even in that extent case, it's better to give tools, at least to today's day of the art,

• 17:28Speaker 00

  for the author to give the right context to the AI at the right time. And so, yeah, when we're working in this very localized space, when it's just like, give me the next paragraph, we had little fine-grained tools to be like, oh, what, give us this essence of like, what is the tone of this scene? Is it ominous? Is it happy? Maybe give us some idea of like who's in it. But very, very small amounts. But now with StoryEngine and kind of like this new hierarchical way, we can let you give us both the 50,000-foot view all the way down to the leaves of the scenes themselves and who's in the scenes. And so this allows the AI to pull out the relevant information so that it can do whatever the task is that you want it to do. For example, is it ideating the next plot point?

• 18:13Speaker 00

  Is it just, you know, give me an idea of like the next paragraph or so? But I can do that now where it does feel less like a slot machine. Because in early days, DaVinci 1.0 is like slot, full-on a slot machine, you know, maybe one out of like 50 times, it's like, wow, this is a really big gem that I'd love to add to my manuscript. So we're trying to battle this slot machine problem, which is like, can we get us every one or one out of two pulls, you get something that I'm like, oh, wow, this solves a problem that I have in my storytelling, or unlocks something. So I see it as like humans and the AI sort of meeting in the middle as kind of way, and not trying to get the AI to do everything, right? Because I think that's, especially a lot of Twitter demos that are kind of out there,

• 18:58Speaker 00

  they're like, oh, yeah, the AI can just do everything. Well, that's not the point, really. I mean, we are Really building this product for centaurs. So half human, half AI, or whatever percentage you want, but you know, but with the human at the head of the centaur, not the butt. Because I think that's when we lose the plot on like, why are you even writing fiction? Right? That's kind of what you know, you're still the humans at the head and can we give the human as much control at whatever fidelity they want, right? So this is the other big challenge that you mentioned, like, yeah, how do fiction writers like work? Well, they work in as many ways as there are fiction writers because every fiction writer has their own way of writing.

• 19:39Speaker 00

  Now, this is in a converse to like, for example, going back to the writing corporate emails. There's probably less ways to write a corporate email than there are to write a novel. That's the challenge of training this type of tool because like we want to make it flexible enough. Uh, so that even if an author doesn't adopt StoryEngine, doesn't adopt parts of our system, it can still be useful. And that I would say is our primary UX challenge delivering this is how do we work in the many, what the myriad ways that fiction writers do and at least be good enough for like a good baseline of like how people are writing, whether they're free writing or they're like outlining first and they like to get everything in place first. Uh, we want our system to work in both of those ways.

• 20:23Speaker 00

  And if from a programming sort of part, uh, you know, I'm a programmer and also a writer. I kind of think about like Sudare, what we're aiming for is almost like an IDE for storytelling. Couldn't really make that happen before, you know, writers have used lots of different kinds of templates and Excel spreadsheets, but it needed a language model to make this actually happen because to actually pierce the veil. What is actually happening in these scenes? And now it's possible and that's kind of our North Star internally. Like, can we make it as programmers have had IDs and all these things that helps them structure their programs for ages? Fiction writers and convers have basically nothing, like they have to do it all a lot of that schlep themselves.

• 21:06Speaker 00

  So how can we reduce that schlep so they can focus on part they love most is storytelling part?

• 21:14Nathan Labenz?

  Kind of, you know, your IDE thing made me think about like the Copilot thing from a few months ago where somebody sort of deconstructed the prompt that Copilot uses and kind of figured out like, how does it go around your file structure and what is it grab? And what is it stuffed into place? And so I'm going to kind of try to riff a little bit. You tell me, you know, where I'm going right or wrong in terms of how yours might be working now that, you know, we're kind of getting to where this is becoming possible. I guess in the two modes of writing, right? There are some, you know, some people, as you said, kind of start with an idea and then they maybe flesh out an outline and whatever. And you can help them with that.

• 21:57Nathan Labenz?

  And then I imagine as you get down to kind of the leaves, as you put it, you know, of the actual individual pages and the paragraph by paragraph, what's happening to kind of help generate ideas in that context? I imagine that you would have some sort of summarized view of the book as a whole, right? Like, we're writing a book that, and interestingly, some of this stuff might not even be generated by the author. I'd be really interested to know if like, there's any sort of summarization or kind of characterization that may be going on in the background that is then, you know, kind of fed into a downstream prompt. But again, if I was trying to build something like this, I'd be like, you know, we're writing a certain kind of book. If that's not clear, I'd maybe, you know, use a summarization or something to try to get some sort of headline statement. You know, the book consists of this many chapters.

• 22:42Nathan Labenz?

  Maybe here's like an outline of the chapter titles or something. We're currently working on, you know, a chapter 22 scene, whatever. You know, here's kind of a little bit of the context immediately before. And maybe if it exists, like, here's a little bit of context after. Like, your job is to fill in the blank. Is that like roughly kind of what you're doing? Or I mean, how would you kind of complicate my base analysis there?

• 23:07Speaker 00

  The main challenge is context stuffing these models to give the right information at the right time for the right function that you're doing. For example, if you're Want a good description of this character in a scene. You might need to know like the fears of this character, right? Or maybe like what this character has done in the past, maybe not, right? But you need to be able to give those pieces to the AI. So the AI can make a decision on like, okay, is this kind of what I need? I just throw this stuff away, but it's very challenging, right? Because for example, maybe maybe very nuanced kinds of things that happen. Uh, well, here's an easier problem. It's like, this character died in chapter six.

• 23:51Speaker 00

  Probably should not reintroduce this character in chapter eight, right? Uh, so those are the kinds of problem, problems that we are working on right now. Which in the, we have parts of it, but in the ultimate case, I think we, what we, what we wanted to be able to do is have, give the AI an understanding of the entire timeline. And this is like more detailed than like the outline of the novel, like a timeline where the characters are, you know, the state of the world. And then given the states of the world, where are sort of the global contexts that you bring in that are static? For example, like this character is afraid of fire, you know, or something like that. Um, or what is the premise of the novel? What's the genre? What's the style?

• 24:32Speaker 00

  And really it's being, that's all this kind of like stay is being fed into some decider that figures out like for the AI, which of these things are most relevant? And some of that's heuristic. Um, uh, something that we're also starting to do now is more like doing a semantic search or vector search across these things to give the right context to the right LLM. Because the agent, you know, this is also trying to understand like the different characteristics of the LLM. It's like, what are some things good at? What are some things not good at? For example, she fifty-four is great at being consistent and following rules. But man, it sounds like C three PO. Sometimes it's like, uh, it's not really, uh,

• 25:15Speaker 00

  um, the best creative prose writer, right? Um, but then Claude for an anthropic, right? Is very good at creative writing, but maybe less good at following kind of rules. So. I would say there's a big context piece, but then there's also a piece of like, understanding the pen, if you, if you will. I consider these different L L M's like as pen, different styles of pens and mixing and matching them. Some authors, man, they really love using like a Claude functions. So some authors, they, they have very distinct opinions about this too, which is very interesting. Um, but for us to be able to provide like good baseline is we need to really understand the characteristics of these models at the

• 25:58Speaker 00

  pros level. So I would say it's a combination of those two things. But that first piece, that's a long road. I think that, um, ultimately like we were still building to that system that can really, how do we, almost like a time and break down like the novel writing process into these constituent atoms, like these components of kind of like how I'm approaching a scene as a human. We sort of used that as a guide for, okay, how would we get the AI to also approach this scene like a human would? How do we give it the right context, right? Uh, because a human can also, we can't fit the whole novel in our head as well. So we also have a context window limit, right? So we're doing the same

• 26:39Speaker 00

  sorts of processes. So we're taking an almost appetit mortic kind of view, uh, behind the scenes and algorithms.

• 26:46Nathan Labenz?

  Let's talk about these kind of leading models, because you touched on that. I think that's, that's a super fascinating thing, right? Like we've seen this progression from GPT three, world's biggest autocomplete, you know, this sort of, uh, savantish thing that could kind of pick up anywhere. And all of a sudden you're in the middle of an internet forum or, you know, whatever. Uh, and now we've got way more dialed in, RLHF'd, RLAI-F'd, much more consistent. I was kind of expecting that you might say that both GPT four and Claude two kind of had some problems in this respect because I do find them to be, I don't get a lot of variety. Like if I run Claude two over and over again,

• 27:28Nathan Labenz?

  I feel like I get almost the same thing back every time, kind of no matter what, uh, which doesn't seem like maybe what you would want. Maybe it is a little bit, I do think it's in my experience, a little bit better at imitating a user's style compared to a GPT four. But I kind of had expected that you were maybe gonna say, actually, we use 3.5, you know, we kind of maybe do some background processing or whatever with the frontier models, but then, you know, we actually get maybe a little better results from the kind of one step down, like the 3.5s or the instance, because they're a little less RLHF to death, so to speak, or maybe even like open source models, you know, because access to the base model can really open up, obviously, the window of possibility.

• 28:12Nathan Labenz?

  But it sounds like you, I mean, you know, give me more on kind of what models you're using and the mix. And in particular, if you think that there's something I can be doing to get more diverse results out of the frontier models, I would love to understand that for my own purposes, too.

• 28:26Speaker 00

  So, to be clear, we're still using a lot of DaVinci models. I know they will be sunsetted soon, but I actually still like them. Because for certain use cases, the RLHF has this characteristic, as you're saying. It smooths out all the rough edges and just makes things a little bit more predictable in some ways. Or like, I like to say the C3PO is like, first, we must consider this and blah, blah, blah. You can sort of like smell that from like a mile away, right? So we still mix it up. We still use like a variety of DaVinci models behind the scenes in tandem with like the big one,

• 29:10Speaker 00

  the GPT-4s of the world. But there are also ways to, as I say, it's not jailbreaking, but it's like trying to get it into a particular latent space is, I think, less effective than DaVinci, but it is possible. And there are ways to describe certain types of styles of which you want in a literary sense, which can get you to have more varied styles. And so one of the things that we have on our, I'd say, is something called MatchMyStyle. So where we can, essentially, like you can upload a whole scene of yours, and then we will try to get it to a prompt that will kick GPT-4 into a latent space that's like at least closer to your style without having to fine tune, right?

• 29:55Speaker 00

  Yeah, I'll check that out because that's that's kind of yeah, if you if you're describing it because like I think GPT and all these things have read a lot of literary reviews. So so he understands the pairing between like how a literary review will review this particular piece of prose and the prose itself. That gives you closer. It doesn't give you a hundred percent, and that's actually a concern. So that's. And that's why we are now exploring more like open source models and other things because I think what I'm seeing in the market is that. Especially for enterprise use cases, which makes a ton of sense for OpenAI, that they do want to smooth out these rough edges and solve enterprise problems. Not necessarily solve creative prose problems.

• 30:36Speaker 00

  We might go down the path of like, yeah, either fine tuning or getting our own open source models where we can have more control. That's going to be in our books this year to do more of that. That being said, you know, I think a lot of these people who are using a more whole hog to like generate like like full scenes, for example. They're using that as a starting point, right? They're not using it as a final draft. This is the first draft. A lot of authors have been able to come up with tricks to get more varied style. One example is called kit bashing. So it's where you generate using two different models. And then you have a center column where you kit bash and just copy paste this paragraph from here to strike it from here. So almost like a pastiche.

• 31:20Speaker 00

  So you can combine sort of the best of like, for example, Claude versus the Da Vinci versus a GPT four. It came to how artists do this too, you know, like kit bashing where they basically have a photograph and they just paint over it. So there's some new techniques now being discovered by authors. Using multiple models at the same time, and a lot of them are also using their kind of like open source models as well. In tandem with pseudorite, more sophisticated ones. Where really they're just like taking multitudes of outputs and just mixing it, mixing it all together. Which is really fascinating. I don't personally do that, but a lot of authors do.

• 32:01Nathan Labenz?

  Yeah, really interesting. Okay, so. Just for super specific techniques, because I think people are always looking for little nuggets that they can apply in their own projects. When I try to get something to write in my style, and the main thing that I've done that for actually is just the intro essays that I write for these podcasts, and then sometimes like the Twitter thread that I'll post to promote it when we release it. I've gone back and just grabbed like five intros from earlier podcasts that I feel pretty good about, just put those in like writing samples one through five. Then I give the transcript of the current podcast.

• 32:43Nathan Labenz?

  And, you know, obviously this kind of surrounding instructions are like, here are five examples of the writing style. Here's the current discussion. Please write. I would say, I use Claude too for that, because the context window basically requires that the length of the podcast is typically too long for anything else. I find that even above like an hour and a half is about as much as it can take. And if it goes much beyond that, I have to start to break the transcript into chunks, or it somehow kind of loses the thread, even though it does technically fit into the 100K context window. And then basically what I get out of that is like, okay.

• 33:25Nathan Labenz?

  You know, I'm interested too in your, in kind of where you think the number of successes needs to be before people find it useful. You kind of said one in 50, you know, back in the earliest days. And now, you know, maybe you kind of alluded to like one in two or one in like a small number anyway. For me, more often than not, I end up doing a full rewrite. It still sometimes helps me to get some kind of boilerplate down and something to react to. You know, it's almost maybe, I don't know, I don't think we're quite to drinking game status on this yet, but I often refer to this Simpsons scene where Mr. Burns says, He said, art unveiling and the art is unveiled and he says, I'm no art critic, but I know what I hate.

• 34:05Nathan Labenz?

  And I do feel like I kind of behave that way sometimes where Claude two gives me something back and like, that's not it, but it somehow gets me writing faster, even if it's not it. So that's kind of how I approach it with Claude two. And sometimes I mess around with things like having it assess my writing style. It sounds like you're doing some of that also kind of give kind of a like a critique or an assessment or, you know, just a description of the style before then trying to like go into that mode. I guess going over on the GPT side, the OpenAI side, this I I'm finding these days like dumping all the kind of instructions into the system message seems to be the thing to do.

• 34:47Nathan Labenz?

  You know, any, any coaching, how can I get my, um, how can I get my little micro app that writes these intros to do a better job for me?

• 34:52Speaker 00

  So yeah, within GPT line, definitely, you don't use chat GPT. Use the playground. The system message does matter. So if you basically set the tone of like, what kind of writer you would like it to be. And so you describe your style. But I would say like, the other thing to think about is, um, if you're giving a very long prompt, and I think the, the, the, there've been studies recently, which is LLMs are kind of, with a really long prompt, they just like ignore the middle. Kind of like a human would, right? Which kind of makes sense with RLHF. So the most important bits of that prompt will be at the beginning and the end, and especially the end.

• 35:31Speaker 00

  Um, so for example, yeah, how we prompt for style is that the style goes last. Because you really need that to be almost the tip of the pen. And so if you're putting any style information, I'll reemphasize it again, right at the end, be like, and write this in the style of blah, blah, blah, blah, blah. You describe your style in detail and that will yield better results in getting it to follow this sort of tone and writing of what you want. Now, if you care less about the tone and more about the content than, you know, you sort of, uh, ignore that part and be like, okay, you just get to rewrite it anyway. Uh, that depends on kind of like your, your workflow. But I'll say a lot of authors, yeah, they are, they're definitely rewriting a lot of the things that are coming out.

• 36:16Speaker 00

  So here's, here's the, I'm not sure if you're doing this, but if you're asking to not do something, sometimes it's like not great because then it will do the thing. I call this the pink elephant syndrome. Don't think about pink elephant. You're gonna think about elephant. So L L M's have this syndrome as well. If you say like, don't be overly cliche, it actually might be more cliche. So it might be better to just give statements or even random kind of like quotes that are not cliche at all. So that it gets into latent space of like, okay, you want me to be in this like, really creative space where I don't talk about cliches. Yeah, but it's a, it's a tough wrangling thing. I mean, in the, in the evaluation part is tough,

• 36:58Speaker 00

  right? For text versus images. I mean, images, you can just, yeah, you see things that I just thought and you're like, I can firmly evaluate that. But if you're like generating paragraphs and paragraphs of this thing, like you don't want to be pushing that lever too many times. A baseline that we have, uh, is to write is about one and two, that an author would use something that comes out 50% of the time. That's like the gold standard that we've had, um, so far. But I think for individual basis, that number is just going to be much lower because you might be writing this prompt bespoke each time. You might not be dialed into a particular set of like, instruction that you're like, and I think that's okay, because you're sort of like, interacting with this thing in the chat, like kind of interface. But I always say, if you want

• 37:43Speaker 00

  to get more repeatable is, uh, to keep a spreadsheet of the prompts that work well for you and start iterating and making notes of like, okay, this wording makes it worse. It's me have to do this, you know, four times, like two times and just start iterating that prompt over time until you have like this prompt that works really well for you, Nathan, right? Nathan prompt, right? And so, yeah, we're seeing a lot of authors who are, a lot of them use chat sheet, B T and tendons. So they all, they also have like, a whole set of prompts that they keep for themselves that works for the way that they work. And that may actually be a new. Practice for writing in general in the future that you are not only sort of gardening

• 38:26Speaker 00

  your craft, but you have a whole garden of prompts that you're like constantly evolving, that are working really well to LLMs that you love. That just may become a common practice.

• 38:37Nathan Labenz?

  Yeah, that's a great transition into, I think, the second half of the conversation. And I want to kind of widen the view from the app and all the techniques. But before I do that, I just want to ask a couple of more things on the techniques. How, can you tell me a little bit more about kind of the background information processing that you are performing on the user's behalf? Like if I show up, let's say I have, you know, maybe a draft already in a Google Doc or whatever. And I'm like, all right, I'm going to come try this tool. And then the first thing I do is just paste in, you know, 20 pages of like something I've already got. It sounds like you're doing a number of things

• 39:19Nathan Labenz?

  to kind of figure out, okay, who is this person? How do they write? What is this? Like, what are the, who are the characters that are in here? And I haven't actually done that exact workflow. So I haven't experienced, you know, how all that plays out. But it sounds like you're kind of coming in pretty hot and like processing that information to kind of synthesize a bunch of things. Love to hear more about that because that too, I think is something that may spark some, you know, creativity and others, you know, well beyond the creative fiction realm.

• 39:49Speaker 00

  Yeah. So we have, if you start with StoryEngine, we have a brain dump field, which we always made like free form text that you can put anything in that, right? So yeah, whether it's like pages from your manuscript or, you know, Wikipedia article. And so we felt very strongly that it needs to be unstructured because I think, you know, as a writer, you're going to have all these random notes coming in, right? But then that's like the first step because then from there is essentially if you were to try to replicate this with your, you know, plot too, it's like, how do you then get, because the AI is just really good at being able to start structuring these into a synopsis or like an outline. So just asking it for that and then just start putting that in back into like the brain dump of

• 40:31Speaker 00

  what you have, it will get better and better like understanding of the 50,000 foot view in a way that LLMs understand. So I think that's the major challenge. I mean, I think our minds work very associatively, but when you're writing, you have to linearize it. And when you're writing with an L N, you not only have to linearize it, but you have to like hierarchically linearize it. So I would say if you for your own purposes, you may want to keep your own silos of like, okay, this is the 50,000 foot view of what you're writing. This is the now like a closer up view. So each time getting closer all the way down to the leaves, because if you, once you have all of that sort of filled out, you can then pick which fidelity to give to the L N at the right time.

• 41:16Speaker 00

  And that's kind of what we're doing in our, in our backend. And I would say it's still very, I would say it's still V one and kind of like, we were always improving that process. It's really tough to pick the right fidelity to get the A I to understand for this particular task. But I think that's useful writing technique in general, right? So you should be able to know, kind of like, keep your thesis in mind for this essay that you're writing. But also know, like, in this particular section, how is it building to that thesis? But how is it also like introducing new questions or new information that is going to get the reader to want to read more, right? Continue reading, right? There's always a tension between those two things, right? No, no silver bullet there.

• 42:00Speaker 00

  I, I think that in the interesting part is that you can get L N to help you create these kind of like sub dock, these different fidelity documents, which will actually help you. Cause I think you reading that and parsing that yourself will give you a better understanding of what you're trying to achieve with the piece, right? Because I think thinking hierarchically will unlock like, oh, wow, maybe this is not even the right thesis. It better that you know that earlier than later, right? So I actually saw a lot, a lot of our users, when they go on strange and they're like, oh, wow, I now understand like maybe my novel is not what I think it's really about. I need to question myself.

• 42:39Nathan Labenz?

  Yeah, that's interesting. I, I imagine too, you know, one thing that the language models are so good at is adopting all these different perspectives. I almost wonder if you could create like a focus group style feedback where you could provide like a whole bunch of different perspectives. I haven't done this on creative stuff so much, but done a little bit of experimenting with like, if I tell the language model, you know, that it's a person and it has a certain belief, you know, is it able to kind of? Come up with correlated beliefs that, you know, a person likely would have depending on, you know, what we kind of generally know goes together in people. And it is pretty good at that. So I kind of suspect that you could, you know, kind of call forth like

• 43:21Nathan Labenz?

  a bunch of different perspectives on a piece of work and give people like a sort of diversified, you know, feedback that they otherwise might really struggle to get. And again, that's probably just an interesting technique in general that, you know, that extends well beyond creative writing. Are there any settings, you know, this is again, very technical, but, you know, people who listen to this are, many of them are like building stuff. Obviously, different language models are a big deal. It sounds like you're not yet into the open source and fine tuning those, but likely coming. Are there any settings or other kind of techniques or tricks that you find to be particularly relevant? Like, I assume you have temperature up. I wonder if there's like a different temperature setting,

• 44:04Nathan Labenz?

  you know, does like OpenAI versus Anthropic demand different temperature settings? Top P is also one I used to use mostly to eliminate crazy long tail stuff. I'm guessing you don't want to drop your top P at all. But anything in that vein, you think is kind of interesting or might show the way to others?

• 44:23Speaker 00

  Yes, we do keep it relatively high, but we also have a low knob to give the user to dial that in. Yeah, Cheechee top P temperature does help kind of shape these things, but they also like have confounding effects with each other. So that's the other thing. It's like so multi variant at that point. Like, for example, you just put temperature like really low, then you probably need to start fussing with like your frequency penalties and your presence penalties because it can create sorts of problems like repetition or it can introduce other problems. So I would say, if there's any advice, I would

• 45:03Speaker 00

  say like, if you're fussing with one thing, you be aware that it may degrade performance for another thing that you're not even aware of, right? So, yeah, so even for us, like, we're very, we're very cautious about changing temperatures and changing these settings once we have it sort of dialed in. But we do a lot of testing of these things to get statistically all significance on whether things are good or not, for some definition of good, whether it's like user actions and various other things. But it's, it's tough because yeah, for creative writing. You really do want off the wall ideas like hallucinations are great like in in our world, right? So keeping temperature relatively high and really the big challenge that I see for

• 45:48Speaker 00

  creative writing is cliches. It's like getting the L and not be cliche is a harder task, especially with RLHF. Temperature is not, you know, top, even top P like the not like one hammer that will like fix that. And you know, fine tuning is another option too, but that has downsides as well, where it can get too dialed into a particular like world, right? And kind of the richness of having the full, the full gamut is, is lost there. So yeah, I would say I would have like a very like dialed in kind of like one, one silver bullet there as well. It's really like, you need to experiment a lot and take notes.

• 46:33Speaker 00

  It's not an easy, it's not a trivial thing. If you want to make it repeatable, like good for you.

• 46:39Nathan Labenz?

  So are you still paying six cents per thousand on the original DaVinci? They haven't changed the price on that still, have they?

• 46:46Speaker 00

  Yeah, I don't think they have. But most of our workload now has moved to either GPT-4 or Turbo. So, but there are definitely some things, yeah, there are, are still, yes, I think it's, I think it's six cents. Yeah, they haven't changed that pricing. Although I think with the new, I think you're introducing a new model, right? Turbo instruct, which maybe a little cheaper as well. But yeah, that's the other challenge, the economic challenge. How do we balance context windows and model choices and while keeping our prices sort of affordable for the average writer? That is a whole nother, right? Whole nother podcast of like, how do you even tune these things for pricing?

• 47:28Nathan Labenz?

  Yeah, there's certainly a lot that goes into it. I mean, most folks, I think, still kind of have the luxury of Being like, just use the best model and then figure it out later. And we've seen enough of the price drops and can probably assume that more, yet more price drops are on the way. Although if you're attached to an earlier model, and I've been there too, where it's like, yeah, I don't really want to, I have like a fine tuned thing that's working really well. And the fine tuned one even goes, oh, it's not six, it's now down to two. They had dropped it to two, but the fine tuned is still 12. So yeah, excuse my mistake.

• 48:09Nathan Labenz?

  Still 10 times, even the base DaVinci, I believe is still 10 times more than the latest like turbo. So it's non trivial difference, but yeah, they did a two thirds drop before the 90% drop. Yeah, the way, the cost thing I think is definitely super interesting. The context does get long, you know, the Claude one at 100,000 tokens. It's like, yeah, that sounds great. You can fit the whole Great Gatsby into it. But every time you click that button, you're looking at, you know, potentially can be, you know, can be a dollar, right? Per generation, if you really stuff the whole thing in there. Slight hammer to write one paragraph. So yeah, that's, you know, that can be non economical pretty quickly. Your comments about the kind of frequency penalty and whatnot too.

• 48:53Nathan Labenz?

  I think, is that mostly an old model thing? Like, I guess what I would, how I would summarize my experience today is, I guess, for starters, what is temperature? What is top P? If folks don't know, temperature is the parameter that governs how the final token is chosen based on the distribution that is generated by the model. Because the model at every step is not just generating a token, right? And most people probably will know this, but instead it's generating a distribution over all possible tokens, which could be like one on one token and zero on everything else, but generally speaking is much more, you know, kind of long tail than that. And so then you have the question, well, do I just take

• 49:37Nathan Labenz?

  the most likely token every time? That's temperature zero. Or do I kind of sample from this distribution proportional to the relative numbers? I believe that's temperature one, although this stuff is not super well. Yeah, I think you can go above temperature one as well. The way I understand that is that you're then sampling even more from the tail than it's like weights would naively suggest. And then top P is basically saying, okay. I may want to eliminate super long tail, you know, super unlikely things from being selected. Maybe I do want to have some diversity in my choices. I don't just always want to take the most likely token. But if there's something that's like under 1% likely, then I just don't want to consider that at all.

• 50:22Nathan Labenz?

  If you want that kind of behavior, then you can bring the top P down to say 0.99. Then by definition, anything that is, you know, smaller than that, that delta is just not considered at all. Typically today, that takes me far enough with the frontier models. And I would say only in kind of the earlier models have I needed to really mess with the like frequency penalty and those other kind of things. Does that jive with your experience too?

• 50:50Speaker 00

  Yeah, mostly. I think presence penalty is powerful because if you want to write things that are not sort of present in the prompt, right, it is helpful. But yeah, I would say overall, though, it matters much more within the prompt than these parameters, tuning these parameters to like micro, micro settings, right? And that is like the bigger, a higher order like influence to the quality of the output.

• 51:20Nathan Labenz?

  Use cases are everything, you know? Most of the stuff I do is about trying to get the job done, you know, right? And in your case, it's the job is writing and it's definitely opens up a lot of different considerations. So, okay, thank you for indulging me in some of this detailed stuff. I think it is, you know, can be a little bit tedious to wade through, but hopefully people take away from it, you know, some tips and tricks that they can use as they're building their own projects, even if those are just things that they're, you know, often building for themselves or a few teammates. Turning then to the big picture. So I guess maybe for starters, I'd love to hear your vision of kind of the future of creativity. You alluded a little bit to it with the centaur, maybe also the future of entertainment.

• 52:05Nathan Labenz?

  One thing that really struck me from the website is like, hey, with pseudo right, you can get a full novel written in a week or maybe even just hours. It's striking that. You know, in that case, certainly the AI is doing a significant amount of the actual, you know, banging out line by line that may, people may have all sorts of feelings about that and the quality may, you know, vary widely. But notably, that is kind of opening up a possibility of like, the content can get generated faster than we can consume it. And so there's kind of this potential inversion of like, instead of, you know, creative content always kind of being one to many, I'm really curious about this possibility where it can become kind of one to one, or even like choose your own adventure. But

• 52:49Nathan Labenz?

  I'm putting too many words in your mouth. So forget what I said and just tell me what your vision for kind of the future of creativity and entertainment is.

• 52:57Speaker 00

  You know, today, you know, we used to write is serving customers who are capital W writers, right? I mean, these are professional writers or they're self-pubbed or, but they consider themselves as writers. I think in this sort of medium term, these L L M's do open up the possibility that, hey, if you've never even considered writing a story or a novel, tools like Sudowrite can help you achieve that writing goal that, you know, maybe was ever about a reach, or maybe you didn't have all this craft knowledge that you had before. What we're hoping is that through using tools like ours, that you also, you will upgrade your craft along the way because you'll see a lot of storytelling tricks that are happening and sort of foundations there. Now, if you go even further in the future,

• 53:39Speaker 00

  yeah, I think that there will probably be a possibility of this one to one kind of thing happening. I just, you know, I have a seven year old and, you know, he, you know, we use G P T sometimes to do storytelling and just like play around with it and just like tell bedtime stories. One thing to consider is that, you know, when you're a seven year old, the things in the world are default. So he's coming into a world where these intelligent agents are default. You know, they're not new. They are things that are like, yeah, of course dad, A I's can storytell. Of course, they can paint too, but I can do it too. You know, I could, he's a, he loves drawing. He also loves telling stories. But he doesn't

• 54:21Speaker 00

  have this like sort of adversarial relationship with it. He's like, oh, but maybe the computer can help me, or maybe I can flesh out some ideas, right? So I think that there will, and it can do that at sort of scale and personalize. So I think there's this sort of, I think there's a few things I'll play out. One thing is this sort of like a personal kind of craft, not scaled kind of media production. For example, I recently had it through a birthday party for my, for my son for it's like Zelda theme. And I created like a whole AI video where she incorporated his face on Zelda. And like it was a scavenger hunt, AI voice, like Zelda was talking. You know, the kids were like, oh my God, they know, and how is Zelda know my name?

• 55:06Speaker 00

  You know, like, I'm not going to share this, right? It's really only just for me and, you know, the kids that came to the party, right? But this is something I could not be able to do before, right? Um, I wouldn't have hired like voice actors to play Zelda through this, you know? But with AI, there's all this is like possible. So I think that for this kind of like micro kinds of media where it's not, it's not about like sharing it to the world. It's possible now and that's, that's pretty cool. And I think that they will, maybe, maybe that won't necessarily be like fully like monetized in like a scalable way. But I think it would just be like akin to like making your own website. It's just like very easy to do.

• 55:44Speaker 00

  Now I think at the other end of the spectrum, yeah, will there possibly be the rise of like a parallel track of novels or films which are fully AI generated? Or at least like it's like AI generated, but with a producer, a human producer in the, in the middle of that. I think that, you know, you're to, to your point about content being, uh, writing content more fascinating to read. I mean, we're sort of at that space, right? Right. You can't read everything. You can't, uh, you can't like watch every film, right? But I think that will probably even emergence of like a film created just for you, Nathan, right? It's just, just like, oh, this is a, uh, it has your interest in mind, you know? That is a possibility, right?

• 56:27Speaker 00

  I'm not fully bothered that will happen at scale still. I think it will be still a little niche to do that, just because I think there is a water cooler effect of like, in that world, we'll still seek out, uh, things created and directed and sort of like produced by humans, right? Not just fully automated. And I think that there is something to that where we will still look at correcting, right? Um, so I think it will be parallel track. Beyond that, I think it's a little harder to make kind of predictions there. I wouldn't posit the state like it's all gonna be like AI general all it was just like all that. I was surprised something I think the truth will stuck be somewhere in the middle.

• 57:07Speaker 00

  And maybe there will be a rise of a different kind of media that only be possible via AI. Whether it's like yeah, G or eventual or like a fully immersive game that is generated on the fly. That has a particular reality and it's just like impossible to be like have humans creating that on the fly.

• 57:26Speaker 00

  But I think there will still be books. There will still be films, there will still be people writing books, the quote unquote old fashioned way. The old fashioned way using it using pseudo right.

• 57:37Speaker 00

  But in that world, yeah, it's still stringing together. I think that it's very lengthy, you know, these things that, you know, things have been able to like kill books all time. But actually, reading has gone up. If you look at the past like few years, book sales have been on up, right? Even in the in this world where Tik Tok and all these second kind of things exist. I think that will still be a cornerstone in culture.

• 57:56Nathan Labenz?

  Yeah, I always say everything everywhere all at once for kind of all of this stuff. So I totally agree that there's not like one, you know, outcome in all like that everything. You know, coalesces into, but rather even more diversity and choice in media than we already have, which is already pretty extreme. Your comment about kind of the blurring of the line. And I mean, I guess in in creative writing, this is like this line is already pretty blurry for a lot of people, right? But I think this this idea of the blurring of the line between like what is sort of quote unquote productive? Or like economically, you know, valued work versus just kind of the similar, you know, maybe even same activity that is leisure.

• 58:41Nathan Labenz?

  I expect that to be a pretty interesting development. And I feel like you're kind of right at the center of it. In as much as, you know, if a lot of jobs that people don't want to do can become largely automated. And there I think about, you know, you know, not that many people like want to get up and go work at a call center all day and take, you know, thirty calls an hour that are largely repetitive or whatever. Not to say there's nobody, but you know, I sort of see a lot of that kind of stuff going away. And then there's this kind of fear that like we just sort of consume, consume, consume. And we become like extremely passive and, you know, kind of low agency.

• 59:24Nathan Labenz?

  And this feels like, you know, it feels like there is kind of a positive vision in here somewhere of like, you know, being creative, kind of to the degree that you want, like being exploratory, having stuff that's kind of interactive, generating stuff for you that's engaging and interesting, but also giving you opportunity and, you know, to shape it, to craft it directly, to, you know, choose options and kind of explore in a way that's still like meaningful. I'm really interested in that space, you know, how are, how are people going to, I mean, it's a big assumption, obviously, that like a lot of jobs are going to go away, but I do think a lot of jobs are going to go away. That's not to say that necessarily employment, you know, drops because we may sort of classify

• 1:00:08Nathan Labenz?

  a lot of new things as, as jobs in some sense, like creating these, you know, bespoke birthday parties for your, you know, neighborhood kids. But yeah, I do think we need to kind of develop this vision for like, what is a positive, you know, interaction mode that we can have? How are we going to spend our time in kind of relationship to all these new AI tools that isn't about like purely eliminating or purely being more productive, but is kind of a new like fun way to be and explore. So I really love that you're, you know, kind of creating that and also doing it in the context of writing. It's kind of meta in that respect.

• 1:00:49Nathan Labenz?

  Are there things that you would recommend that we check out? I always lament that positive visions of an AI future are like pretty few and far between, but you strike me as one of the people that I should be asking, you know, what should I read or engage with, you know, whether it's, you know, a novel or otherwise, to inspire myself, you know, about what the future might look like?

• 1:01:14Speaker 00

  That's a good question. I, I love sci-fi. I feel like sci-fi is this meta thing because a lot of the people who are creating technology are reading it. And so that it's sort of like comes out in the products, like they are like designed from these worlds, which were actually a lot written, you know, 40, 50 years ago. So yeah, I would, I would say like, it's not, if you want to like discover, I mean, sure, you can go on Twitter and see all these like demos of new AI things. I think it's harder. There's so much noise. Right. So I actually like reading like, you know, older books and see like, oh, what, what were sort of the visions of the future there?

• 1:01:56Speaker 00

  Because I think it's easier than to extrapolate like what would be, you know, thirty years from now. If you look at like what people thought it was going to be, you know, whether it's reading like Gibson, you know, or reading different sci fi, you know, like a, a, a three body problem, you know, just exposing yourself to different, some very different points of view, uh, both culturally as well. I think can widen kind of what can see is like the bubble of AI. This is your sort of point about elimination of like skills, right, and jobs that are necessary. It's kind of funny because I do think novel writing will still be there. So it's funny enough that maybe in one version of this future, like we can just indulge our creative pursuits, right?

• 1:02:40Speaker 00

  And think that, or at least get more points, right? So I think that we run positive outcome here. But I think the other point about a lot of these AI systems is like controllability, especially like the earlier versions of all these things, whether you're talking about mid journey, where you're talking about, what are you talking about, Sudowrite or ChatGPT? It's like, we're still in this pre iPhone moment. And maybe ChatGPT is sort of iPhone moment, but I still think that there will be another iPhone moment where an AI totally understands your brain state and understands your intent and can do it. And you can control it and not feel like the slot machine. Because I think once we get to that point where we have this sort of melding, I think it solves a lot of problems and it's kind of like agency.

• 1:03:25Speaker 00

  So I think that you can now fully express kind of like what you want to create, right? And you can also fully express as, you know, a person on the job, like kind of the jobs that need to be done, right? But you have put humans in the control of that. Now, I'm not sure how that squares with, well, it, the AI just. All the programming for us and we just like take the human totally out of that loop. I do have some worries about that, probably not, not to the same spectrums of duma-isms. Uh, that are out there, but I think at least some humans should be in that loop directing something. Uh, but there'll probably be less, uh, folks there. So that is a problem that we will need to contend with.

• 1:04:10Speaker 00

  Uh, there's always this joke that like, oh, a lot of artists are up in arms, but you know, mid journey, but then all the programmers like, ah, finally. I don't have to write a test anymore. Yeah, I can just do that, right? But there's a little bit of danger in that too. I think that we need, we need to be able to, I think, understand these systems. And maybe it's not understanding at the leaf level, they can have a core understanding as some fidelity in that chain. I think that will be important going forward.

• 1:04:40Nathan Labenz?

  There's a notion, I guess it's kind of presented in some cases as an AI safety agenda called cyborgism, which is, uh, kind of along the lines of what you're discussing here. Uh, that might be of interest to folks to go check out. And it's kind of what Elon Musk said was the motivation for starting Neuralink as well. You know, it's the idea that we need the, you know, we need to be first of all, able to communicate with machine systems at a higher bandwidth than we currently do. And, you know, potentially with higher fidelity to our, you know, internal mental states than we can capture with language. Um, so yeah, the future could get very weird in, in those regards.

• 1:05:26Nathan Labenz?

  I, I saved this kind of to the end because I, you know, I don't think it like defines your story by any means. But I do want to touch on the experience that you had when I first reached out to you. I think it was maybe two to three months ago now. You launched a big update to pseudo right and put out a video, you know, nine minutes or so on Twitter, showing some of the new features and how it works. And it was an interesting thing because.

• 1:05:54Nathan Labenz?

  You know, if I had seen that in total isolation, I would have been like, cool product. You know, one of lots of things that are coming out that are using AI and, you know, a thoughtful implementation certainly of like, how to apply these tools to the storytelling task. And yet, that would pretty, you know, at least for a minute, went pretty sideways for you when, I guess, I mean, you tell me why you think it happened, but timing, you know, was maybe an issue and all of a sudden people were like, you know, this guy is, you know, public enemy number one of writers. So I'd love to just hear that story kind of from your perspective and see if there's anything that we can learn from that, because it certainly seems like, you know, the kind

• 1:06:37Nathan Labenz?

  of thing that, I don't know if people are getting canceled anymore by the, you know, the woke mobs that some folks used to be concerned about, but now there, there may be an, you know, anti-AI contingent out there looking to, to cancel people. So yeah, tell me how that all went down.

• 1:06:53Speaker 00

  Well, it's my first time really getting canceled. I was definitely got dog piled there for a while. Yeah, I still stand by that demo. I stand by what I said. I think if I were to update with at least a little bit of nuance, it's, I would say, this is great at the first draft. It's not about writing the entire novel. And if you want to, I mean, so this is the other thing. There's, this division is maybe more complex than, than even just anti-AI. I think that there are also shades of like, we have a lot of indie writers who are using us. And actually, their audiences, a lot of them, like, they actually know that they're using AI assistance for help. And they're like, this is great. You can write like 20 books instead of 10 books a year? Give me more of this, you know?

• 1:07:37Speaker 00

  So there is really seeing that feedback and being like, hey, it's really capturing your tone well. And, you know, and I think there were, there were a lot of folks out there who were saying like, oh, how could you even find any novelist to even work with? You worked, literally worked with hundreds of novelists who were begging for this product, right? So I think that's the, that was the other thing, the one I was lost in this, in this tweet, it's just like, it wasn't, this product was not born out of just my mind and isolation, just like everything else. We worked hand in hand with novelists who were, literally some of them like cried tears of joy when they actually got this into their hands, right? So it's stark contrast and it was some of the reactions out there.

• 1:08:22Speaker 00

  Now that being said, I totally understand this third rail that's happening right now. You know, I don't, you know, writers are not getting paid compensated. Well, now, for example, in definitely in what's happening in Hollywood right now, it looks like there's some deal that's going through. I think that's part of that timing and maybe the nuance in that video, which was lost. And I think that writers should be compensated. They should be compensated fairly for the work they are making. But I think that that, the question of that, it should be separated from the tools that they use, right? So Zuray is always, it's a tool. It's not, we're not out here to replace writers.

• 1:09:02Speaker 00

  We're out here to create tools in cost of writers who are giving us feedback all the time about, you know, how, how this helps their lives and, you know, less than being more creative. You know, that was the largest, we got, we had so many influx of new users as well from that. It's the awareness as well of like, wow, this kind of tool actually even exists. And even if they don't, you know, a lot of authors are not using us in that kind of way, right? So some of them are using us in a very AI maximalist way where they are playing whole scenes. We're bringing them to lots of things like, some of us also are just using it in a very like brainstorming way. Just like, okay, what do you think of this scene? Not even using any words that are coming out of it.

• 1:09:42Speaker 00

  So I would like to say like, yeah, it's very diverse, as diverse as people using Photoshop back in the day. We also saw that sort of backlash too. But digital art, really art, no one questions that today, right? You can find a few of them. And I don't see that's happening with majorities and sort of art generation. But I love to shine a light on the creatives who are pairing this with their craft, right? And like, how can we build a tool that is mindful to that as well? And also pointing them in directions and, and, and increasing their craft. For example, we also run weekly classes with writers to talk about how do you still write for your rank? But also just talking about how do you become a better creative writer?

• 1:10:25Speaker 00

  So we're already invested in this space. I've been doing, you know, we haven't pivoted. You know, we've, Ame and I are both fiction writers and we, you know, built this. We keep, keep on building this thing from the core thesis for three years, you know? And we're going to keep building this, you know, because we are really seeing that this is, I think that creative writers will see these kinds of tools as. Uh, maybe not default, like in this stressful thing, but it's just like using any other writing tool like grammar check. But it was definitely inexperienced and again, getting dog piled, but it definitely sort of also renewed my seeing the folks on the other side, I mean, being like, whoa, can I not talked about like me using AI for writing in the public?

• 1:11:08Speaker 00

  And that kind of stuff really, that's, that's really concerning. Uh, there is like, it's like wish hunts happening in the author world where they are just trying to out these authors is using AI. And I think that is just like, I think that will be a transitionary state as well that is happening. I mean, that's where, that's where things happening out there in these groups and enclaves.

• 1:11:28Nathan Labenz?

  Do you think, I mean, the timing, obviously, because this was kind of, I don't know, it was like the earliest days of the writer's strike, but at least there was, you know, it was underway at the time that you put this tweet out. Seems like that's probably the biggest variable. And then I guess, do you know, like, trying to understand what is really happening here? Is this like, who's responding to you? Are these, um, cause I did an episode, you know, which I sent you, uh, to try to win a little, you know, trust from you to get, uh, to, to get you, you know, comfortable to do this in the first place. We did an episode with a couple, uh, three members of the WGA and kind of talked to them about, you know, what their AI related concerns are and also how they're using it in their own practice.

• 1:12:12Nathan Labenz?

  And, you know, the three possibly biased, you know, sample of, of those that were up for coming on an AI podcast, but they were not at all like hostile to technology. It seemed like their thing was really just, you know, we don't want AI to start getting assigned like certain contractually defined roles that have payment rights attached to them. And so it seemed like they were pretty, you know, if they could get that kind of agreement, they were going to be pretty content to continue to experiment with all sorts of AI tools.

• 1:12:50Nathan Labenz?

  Do you have a sense for like, to what degree the reaction that you got was driven by

• 1:12:57Nathan Labenz?

  people who are like actually writers versus just kind of general internet vitriol? And you know, aside from timing, you kind of mentioned a little bit of like more nuance. Maybe the thing would be to emphasize more like, you know, the kind of it starts with why type thing? Like, why are you doing this? Who you are, your own passion for writing and storytelling at the top? Is the reaction that you got something that you think really comes from writers? Or does it just kind of come from like general internet culture? And second, what would you say to future AI product builders? You know, when it comes time to put things out in the public?

• 1:13:37Nathan Labenz?

  What can they learn from your experience to try to minimize the risk of any of that kind of blowback coming their way?

• 1:13:45Speaker 00

  Yeah, I just, I think the top, the higher order bit is Twitter or X. It's just like, there's no, there's most, I was, you know, looking at the analytics of people watching video like, like a zero percent, basically like many 90% watched the first like five seconds, which had like basically no content. Right? So it's mostly, I think the mechanism of retweeting and tweet docking that got this place. And it's interesting because I think there's probably, it was probably on the tip of like, a few people retweeting the right verbiage that got the dunkers to come in. And if it hadn't tweeted it, maybe it would have just been kind of like semi, not even a viral tweet.

• 1:14:29Speaker 00

  Right? So I think there's mechanisms in, in the structure of the social media that caused this issue. I think that a lot of them were our writers, but I would say mixed, right? There were probably writers and also just general internet dumping happening. But I mean, I became sort of just like post a trial of like, oh, this is the AI, AI bro coming. Right? To take all our writing jobs away. Nothing I could have, at that point, it was just, there's nothing you can do with that kind of deluge, right? So you cannot reply to every tweet. I was thinking, it's was advice, uh, maybe Twitter is not always the right, uh, not always

• 1:15:11Speaker 00

  the right conduit, um, for these kinds of announcements. I would say also like putting customers first. I, I think that if I were to redo it, maybe having one of our customers talk about like how they use this in their workflow to maybe like human, humanize it a bit more to be like, okay, this is an actual use case. It's not like I'm just contriving it out of thin air. So I do wonder if also just like I patterned it after like, oh, Twitter demos for the tech people. It's like, okay, we just like take this demo, but we don't take it like at sort of face value. It's just like, okay, this is like a demo of what could, how could it be done? I would say like, if you actually have a real showcase an actual real problem, and if you can with a customer, I think that would be, uh, that's even more, uh, wise there.

• 1:15:56Nathan Labenz?

  Have you raised capital? Are you hiring a team? Or are you guys just bootstrapping this, uh, as your own passion project? What's the story there?

• 1:16:04Speaker 00

  Yeah, uh, we raised a small round in 2021. Um, and we have small team, seven folks, including me, uh, so four developers and we're profitable. So ever since, ever since March. So we're sort of in this place where I want to build a team mindfully and, you know, find folks who are, you know, like, actually, I think three are, four of our developers are like have English degrees. They come from that world, right? Who are, who are also writers. Uh, so we're looking to really just like grow mindfully and just really focus on this particular problem. And now we have the line to sort of do that, uh, with kind of our structure in place.

• 1:16:43Nathan Labenz?

  When this storm hit, it varies a lot. You know, your experience and kind of what you have to do varies a lot depending on whether it's just you or you've got a team that you're, you know, representing and who's looking to you for answers. What did you do? Did you just kind of keep your head down generally and let the storm pass? Or was there anything else that was useful?

• 1:17:01Speaker 00

  It's tough. I mean, when you're, when you're in a dog pile like that, you, the, the one thing you want to do is to reply to everyone. Uh, but we, we basically kept our heads down, uh, and just really go and focus on our customers. Because, you know, they're the ones who they're coming in that we've got so many requests, feature requests and things like that. So really just building and focusing on the writers who are using us. And getting our team to focus on that. Obviously, it's a little bit hard when I'm just like, wow, this little rectangle is now like screaming at me all the time.

• 1:17:33Speaker 00

  But I feel like, yeah, I definitely became a little more resilient after that experience. So I would say that if you, if anyone else is in that same position, it's like, it's the internet, it's structural, it's how the incentives mechanisms and these kinds of social media apps are made. And it also made me mindful of also being a dunker, right? I mean, just like you see all these opportunities to dunk, just don't, don't do that. Like, there, and actually, in that, in that world, there were folks who had more nuanced discussion on it, which I appreciated. Where I actually had conversations that were not like a streaming test. So I look for those folks as well.

• 1:18:14Nathan Labenz?

  First of all, it's awesome. And congratulations that you're profitable with the company. I imagine that like your actual paying customers little noticed this or like barely cared, right? I imagine it didn't have a big impact on the business?

• 1:18:29Speaker 00

  No, I mean, yeah, I think, yeah, I think someone posted in our, in our community Slack about it. And a lot of people were like, wait, what? What's happening? The impact on the business is that it's part of the reason why we are profitable. We got, we got a lot of exposure from that tweet. And it actually exposed a lot of writers. I think they're in all, in any of these vitriol, I think, on the internet, there's always a big middle, like, population who are not saying anything. And they're like, let me check this service out. And let me make up my own mind about, you know, whether I want to use it or not, right? So that, that's the hit. But they're not tweeting, they're not retweeting, right? So I think that's the other thing to keep in mind as a founder of a company, that you're not hearing from a lot of people.

• 1:19:15Speaker 00

  You're hearing mostly from people who are out there from at this particular time and have a particular agenda.

• 1:19:23Nathan Labenz?

  Another question I have for you. Because of your success monetizing the app and also just this this moment where you got this like surge of visibility, traffic, you know, probably new trial users, you know, some of them paying customers, I'm sure, but probably many were just kind of like, let me just hit the free trial button and, you know, generate something and kind of just see what it's all about. I think that's a challenge for a lot of AI app developers right now. It's like the most of the ones I talk to are kind of like, yeah, you know, there's a lot of interest. We, you know, have maybe had a viral moment or two or, you know, whatever, or some minor ones. So we're like getting people that are interested in us. That's, you know, my company, Waymark, we've had kind of, I'd say two viral moments this year, not as big as yours, but definitely big.

• 1:20:09Nathan Labenz?

  You know, by certainly any, you know, by any standard, any previous standard of just like tweeting about the product, you know, we went a lot further than we had in the past. Yet then what, you know, we've kind of seen and what has happened to a lot of others too, is like, you have a lot of people that just come through, try the free trial. If you are using a kind of expensive model, you know, and you're trying to, which if you're trying to deliver like a premium experience today, you probably are using a model that has, you know, somewhat non-trivial cost, right, for for each user that comes in. Then you have this kind of, okay, well, how do I, where do I draw the line? How do I, when do I insist that people pay so that this is like economically sustainable for me or for the business?

• 1:20:48Nathan Labenz?

  And then you also have a lot of churn problems, or at least most of the apps that I talk to do, because yours may be an exception to this, because I can imagine writing might be more of like a lifestyle, you know, where people are just really into it on a on a long-term basis. With Waymark, we have, you know, small businesses that create videos and a lot of times they're like, I need one now. I don't know when I'm going to need one again. So I'll subscribe so I can do the thing and then I'll immediately cancel because I got what I needed for now and I can always kind of reactivate later. So, I mean, to some degree, this is timeless, like SaaS business problems, but I do think it's a little bit different in the world of AI, particularly because. Two things. One, we're having these like viral spikes everywhere right now.

• 1:21:30Nathan Labenz?

  And two, you have this kind of not huge, you can kind of stomach it, but like non trivial token cost and it creates like an even more kind of high stakes question of like how and when and how aggressively do I monetize? So I'd love to get your thoughts on that too. Because it sounds like you've dealt with all of it.

• 1:21:48Speaker 00

  What you touched on is like, yeah, it is a SaaS problem. I think that AI companies are still startups. I mean, a lot of it still just depends on distribution, just your the age old thing of like reducing your churn. It's not necessarily about like model choice and all these things that we talk about on Twitter all the time, which are important, but there are also just like foundational things like, are you pricing your your service properly? Are you capturing enough value there? Are you providing enough value to those users? Are those users actually like telling other users about your product? Right? So I think that this whole panoply of like startup advice and SaaS advice that still runs through

• 1:22:30Speaker 00

  the AI is not like a panacea. You just don't sprinkle that emoji sparkle thingy and suddenly your company just takes off. So I think that is a big divide between, I think a lot of people see these Twitter demos and they're like, oh, so cool. It's so awesome. Man, users are gonna like eat that up. But the gulf between taking that demo to making a profitable business is immense, right? And I think it really still comes down to, do you know your customer well? I think that's the biggest thing. And like, do you also know that domain well, or or have or get advice or just like, or become an expert in that to become an indispensable

• 1:23:13Speaker 00

  tool, right? That is used often. I think that's the kind of like what you're talking on the the churn problem, because I think a lot of these AI ideas are not viable because

• 1:23:26Speaker 00

  Not viable businesses. Maybe they are like gray as a demo. But if it's not repeatable customers who have a hair on fire problem, you still run into the from A, A, as he says, it can be even worse because the consumption model of these AI, the margins are like worse, right? Because imagine your database costed like thirty cents or every row that you like save, right? And it's kind of like, and you're doing it every single time, every update. That's kind of what we're talking about, right? So that's the other, the unique challenge of AI is how do you tune? Uh, like your context windows, like how many free like, um, uh, inferences you give to trial users? So you had to pay a lot more attention to that than a regular SaaS would pay attention to their database costs.

• 1:24:11Speaker 00

  Uh, because it's just like it scales more easily per customer. And so we do a lot of that. We do a lot of tuning, but one, one thing that we have stuck to is that we've taken more of an Apple-like approach, which is, hey, we're going to give you the best models. You're using the best top of the line models, latest state of the art. But you're gonna have to pay a little bit more for that. We're stuck to that. So I think that's that, uh, and I think that you should, in your best one philosophy that you need to like figure out whether it is that you're going to use state of the art models? Or are you going to be like, okay, I want to give you a lot of volume? Or maybe bring your own key or doing things like that. I think you need to like be very crisp without that.

• 1:24:52Speaker 00

  Uh, because otherwise you just do sort of random things here and there. I think you should top down, figure out that philosophy. Because that will determine your audience. Uh, and you want to be mindful about choosing the audience of like, okay, are you going after prosumers or more like professional? Folks who are willing to pay? Or are you going for like more consumer types of experiences where man, you're gonna have to have a lot, a lot of consumers, right? Uh, to be make that kind of work. Um, I think for us, we're a little bit in the middle, but we skew more towards like the prosumer professionals who are uh, working writers and they're, you know, they're working day in, day out and using our, using our trial databases.

• 1:25:32Speaker 00

  Personally, I find that to be an easier space to be in because then you can actually charge for value, right? That you're actually providing. It's still not easy, even with AI. And a lot of, a lot of times it's funny because I'm like working on this for some cool like new AI futuristic feature. But then other times I'm just like, man, I just need to make our like flow for the pricing better. It's like, this is not like rocket science, but you need to work at it. Those kinds of things that are just like normal startup things.

• 1:26:05Nathan Labenz?

  Here's a kind of speculative question. I've been starting to circulate this amongst some AI app developers. Because of this kind of high token cost, like for us at Waymark, somebody comes through. First thing they do is they, they set up like a small business profile. And then once they have that set up, they can kind of generate videos that use that profile. Plus they're like runtime instructions. Setting up the profile kind of takes a certain amount of fixed cost, if you will, of the AI. Actually, a lot of it, that is on the image understanding side. And we actually should ask you too about possible multimodal extensions of your product in the future. But, okay, we have that one fixed cost.

• 1:26:49Nathan Labenz?

  And then as you generate each video, there's, you know, some AI model cost with that too. We kind of figure it's something like 15 cents per, you know, random user that comes in, does the free trial. And then we're like, yeah, what, you know, our retail price, whatever, if it's 30 bucks and we need kind of one and 200 to, you know, subscribe to like pay, you know, for all the usage that we have for the other, you know, 199. And obviously those, you know, things can kind of, everything's connected by springs to each other, it seems like somehow. But. I do think that kind of sucks. So I've been dreaming of this idea of the AI bundle, which is kind of, you mentioned like bring your own token, which is another kind of angle on it.

• 1:27:34Nathan Labenz?

  One big challenge, of course, is like, most people don't have a token. You know, they've used ChatGPT, you don't get a token from ChatGPT, you gotta be on the platform to do that. So, you know, right off the bat, if you're asking people to like bring their own token, most of the time, you know, most writers, you know, most small businesses are just gonna be like, don't have one, what's that? So there's a couple of different ways you could kind of orchestrate something like this. But I've been thinking about this notion of the AI bundle, where maybe, you know, taking inspiration from a cable bundle, you could set something up where the average consumer gets to say, okay, I'm gonna subscribe to, you know, tier A, tier B, or tier C, whatever, you know, maybe for a hundred bucks a month, I get some baseline access to a thousand apps.

• 1:28:16Nathan Labenz?

  And then you, as an app developer, if you participate in the, in the bundle, and again, this could obviously get complicated, but let's just say for a simple starter, you know, you get 10 cents as one in 1,000 apps that constitute this, you know, hundred dollar bundle. And then what you have to do is maybe give all those users that come in, you know, for that free trial, you kind of have to give them like your base plan, which you have like a 30,000 word initial starter plan. And obviously, again, all this could kind of fuss around. But I wonder, would you be interested in participating in, in something like that? If somebody could say, hey, I'm gonna go sell a million people on a bundle, you know, that, that would translate into basically, you know, 10, 10 cents a month.

• 1:29:00Nathan Labenz?

  That basically translates into like a million dollars annual revenue for the participating companies. You could still ups, you know, upsell them to your premium plan or whatever, you know, once they kind of get beyond the bundle limits. But you would kind of serve whoever comes in that's a bundle subscriber, and you would just get kind of more consistent revenue. That way, you can welcome those people, hopefully with more, you know, open arms and more access. They can get more value without hitting like super early limits. You, you have to worry less about kind of churn from these like pop in, pop out users. Probably also have to worry less about kind of managing these viral moments, you know, which can be.

• 1:29:41Nathan Labenz?

  Good and bad and a kind of weird mix where it should be, it should be all good, but in some ways it's not necessarily always all good. Do you think you'd be interested in participating in that kind of bundle as a app developer?

• 1:29:52Speaker 00

  That sounds super interesting. Obviously devils in details and like, what is the share of sort of payouts and how does that work? I mean, maybe this is actually just like a cable network, right? It's like cable inference cable.

• 1:30:06Nathan Labenz?

  Yeah, because people love their cable bundle so much, you know, I'm trying to.

• 1:30:11Speaker 00

  No more streaming, right? Go back to cable box. Cause it's kind of like the streaming model today, right? In AI. Yeah, I think that's interesting. I think I would have questions of like, well, it might be more diffuse, right? The people who are, they may not be extension writers, right? Obviously. But I guess it depends on like, I guess you need to get to a certain size to make that work out. Yeah, it's one meeting because I was like, how would you actually implement that? Like, do you then? Are you like man in the middle link like tokens that you like this bundle like manage? And there's like some, maybe you negotiate deals with OpenAI. I mean, that could also be interesting.

• 1:30:55Speaker 00

  Maybe that's some how you could like transfer, get those discounts over to customers. Yeah, I mean, it's something I've, yeah, I'd be interested in chat about that. The other thing I've been thinking about is like, what if? I mean, OpenAI and Claude like made their own login system, right? To be like login with OpenAI. Oh, no, you don't have to log in. Oh, you like Claude, login to Claude, to the AI application. Suddenly your intelligence, your AI, your favorite AI goes with you anywhere, right? I think there's a coordination cost. I'm not sure why OpenAI would be incentivized to do that, right? I don't think they really care about making it cross compatible. But I think that's better for the consumer. I would love to have that to be like, okay, any app, I just log in with my inference provider.

• 1:31:38Speaker 00

  Have it that be like the entry, the entry point and the intelligence and maybe even has all my prompts. It has all my settings in there. I think that's also an interesting idea, but I'm not sure how you overcome coordination costs there.

• 1:31:54Nathan Labenz?

  It is tough.

• 1:31:56Nathan Labenz?

  And unclear who would be able to pull, I mean, the most likely. Candidate to pull that off would probably be OpenAI. And I love the login with OpenAI or login with Anthropic or Claude or whatever as well. That's kind of, I think part of how this could work would be like you log in to validate that. I mean, that's how the cable bundles work, right? Like you, or when you're online and you're going to go use some ESPN app or whatever, like you log in with the cable bundle login. So something like that would probably make a lot of sense here. I think the idea too of bringing personalization with you, I'm with that also. I think I might do like do a whole show on this and kind of flesh out this vision a little bit more broadly.

• 1:32:36Nathan Labenz?

  But they have the ChatGPT Enterprise, which connects to your stuff. And they're beginning to power vector database search where you just like connect your Google Drive or connect your Dropbox or whatever. And they ingest all that stuff, they chunk it, they index it, they embed it. And now they also do all the querying. So it's all this kind of vertically integrated experience is coming to retrieval augmentation. Even that could be kind of extended to the individual level and could bring a lot of value to this login experience, right? If I could, you could log in to bring your identity and bring your sort of access. You could bring your like system prompts, you could bring your prompt library.

• 1:33:19Nathan Labenz?

  You could even bring like your whole vector database profile that could kind of come with you anywhere. I think that is definitely super interesting. It's a little, the vision for that is a little bit foggy. But that's why, you know, I'm probably not the one to do it. But maybe I'm the one to talk about it until somebody at one of the bigger players actually decides to implement it.

• 1:33:44Speaker 00

  I mean, it feels like the hope of OpenAI plugins was sort of that, but it feels like they pumped the brakes on that. And I don't hear much about plugins being talked about, but they said they have affordances to be able to do vector search on your company's private or your app's private data and then bringing that into ChatGPT.

• 1:34:03Nathan Labenz?

  The kind of big quote from Sam Altman was, a lot of people thought they wanted their app in ChatGPT, but what they really wanted was ChatGPT in their app. And it seems like, in my experience, developing stuff, the

• 1:34:16Nathan Labenz?

  Plugin paradigm and the function calling capability that the newest models have is basically two sides of the same coin where, you know, when you create a plugin, you're basically declaring, here are the calls that you can make to me. And when you're using the function calling, you're basically saying, here are the functions you have available. So that they present like slightly differently and you, as of now, you kind of have to, you know, coerce them from one into the other format. But broadly, it seems like that's kind of converging toward, regardless of the interface, there is a way where the model can be made aware that it has these tools or these functions

• 1:35:00Nathan Labenz?

  or, you know, affordances is a great word too, available. And, you know, it's getting good at knowing how to take advantage of those. And so then that distinction probably starts to kind of gradually fade away, right? Whether I want to, I don't know, we'll see. ChatGPT could get really good. But at least for now, you know, there's enough value in all these other apps. Like, I don't really want a paragraph summary of like flight options that are available. I want to not have to deal with the advanced, you know, search and sort thing on kayak. I just want to be able to say what I want, but then still kind of see that in the like richer format and have all those controls if I do want to dive into them.

• 1:35:41Nathan Labenz?

  So it does seem like it's bringing the AI to the app that's kind of more the thing. But again, that kind of, you know, speaks to some need to like grease those interactions because right now it is, it is tough. If, you know, folks like you or me are in this spot where we're like, well, we can give you, you know, how many tokens can we afford to give you before, you know, this just becomes unsustainable? And if they could take that problem off the table and be like, yeah, now you don't really have to worry about that. You can kind of, everybody can use your app, not necessarily for free, but freely. And then they can fall in love with it and then they can pay for your like, you know, higher tier plans. I think they could do just a tremendous amount of good for. The ecosystem that surrounds the core technology.

• 1:36:27Speaker 00

  Yeah, as a consumer, you really want to, the first few inferences are not going to be representative of your experience with this. You do need more time, I think, with an AI app.

• 1:36:36Nathan Labenz?

  Yeah, totally. It sucks to hit that barrier after like two clicks, you know, because the one out of two that you mentioned earlier is probably tough to achieve in your first two. And, you know, we, Waymark too, we have this problem. We actually have a pretty, our thing is very well defined. And I think we do have, you know, one of the highest success rates of any app that I've seen out there. But still, it does kind of suck because we put people in a position very quickly where it's like, you either have to buy if you want to, you know, really do this, or you're kind of, you know, hit the end of the the experience. And I just wish we could, you know, kind of open that up a little bit more without having to, you know, just eat all the cost and make it kind of a tough thing to sustain.

• 1:37:20Nathan Labenz?

  Anyway, again, thank you for coming down this brief detour rabbit hole with me. Anything else that you want to talk about today that we didn't know? Multimodal was my one other thing. That's obviously a super hot topic. We got Dolly three out. We got image understanding coming. Do you envision extending your product to, you know, obviously you have interest in creating visual assets given what we talked about at the birthday party, but is this something that you think Sudowrite will do as well?

• 1:37:46Speaker 00

  We probably won't do it ourselves, or if we, we do have like a small visualize button, which is these Dolly on the back end, but we really remain focused on the text and the story outline, editing and story understanding. However, one, one place where we could possibly expand is that maybe StoryEngine becomes sort of backend to other apps to want to be able to create legible stories. Like if we have a lot of folks who already have their stories, like in our particular format, maybe with the mid journey front end, it's like you can create storyboards. But, you know, that's really like far down the future. I think there's a lot of story problems as it is, even just with text.

• 1:38:29Speaker 00

  That are perhaps even more authority in some ways than than images. But yeah, we remain focused on on that. But it's interesting to see OpenAI like it feels like they're moving into a world where it's like everything to everything. I think it's really smart. And I wonder how that will be implications of that. I mean, I imagine like that thing asks for accessibility. That's really cool. Imagine like in the future, if you're a 360 camera in the middle of a meeting room, AI just recognizes sticky notes that you're putting on the board, the brainstorm session. Like, there are some interesting applications there with storytelling too, like having a writer in a writer's room.

• 1:39:09Speaker 00

  Having like an Alexa's type device that can see everything, hear everything and then make suggestions that are highly relevant, right? To the context at hand. I think there are some opportunities that will unlock if, you know, if I'm excited to hear what OpenAI will announce at their developer conference, like, hey, maybe that will unlock some things there. Another sort of tangential thing is like some ideas I've had around. What if, because a lot of people dictate their stories now. So what if you could just walk around San Francisco with your air buds in and just like? Talk for two hours and the end of it, you come home and you have whole outline. It's already like pre like assembled and all that from your rambles, right?

• 1:39:51Speaker 00

  Because so those are the more types of like multi-modal things with at the service of like, how do we get great stories out of people that we're thinking about?

• 1:40:00Nathan Labenz?

  Any apps you would recommend from your many travels that you think people are maybe sleeping on or, you know, obviously. I used to ask this question to everybody and then everybody would just say chat GPT and so I kind of.

• 1:40:13Speaker 00

  What's wrong with that? He's like 80% is good for everything, right?

• 1:40:18Speaker 00

  I mean, this is probably not unknown, but Cursor, I mean, I, as a programmer, I mean, I love, I've been using that nonstop. Just because they can rewrite and just highlighting it's just like so powerful in terms of being able to rewrite functions, things like that. That goes beyond what Copilot can do. Interesting that Copilot is sort of like lame, have this little headway, but it feels like Copilot will catch up as well. I think Copilot X is the thing that they're building, but I've been loving that. Yeah, of course, that's the name, other AI product that I've been using.

• 1:40:49Nathan Labenz?

  On the image side, do you have any favorites? Like what, um, did you do anything, you know, special or non-obvious for the Zelda birthday party?

• 1:40:56Speaker 00

  Oh, yeah. I mean, I used a combination of Mid Journey and Pika, uh, Pika Labs, uh, for just adding some motion. Uh, I found that to be pretty effective. Um, but also using it, also just using Dolly as well to just get flesh out some prompts, um, a- along the way. Oh, ChatGPT is really great at making scavenger hunts, themed scavenger hunts that are then kind of like is relevant to a normal house. Man, it can generate like a hundred different clues that are like Zelda themed and it's just like, now go to the cherry fountain in, you know, and there's like refers to something in the bathroom. Like, wow, this is so good. And there are these kind of like bespoke things, right? Uh, but really the coming of the combination of these, uh,

• 1:41:41Speaker 00

  these tools. For the AI voice stuff, um, I just found it just like a random off-the-shop website that was doing something that had a good Zelda voice. So I may remember the name of it, but really just like exploring, like it was fun to explore those tools.

• 1:41:54Nathan Labenz?

  Thank you very much, James Yu, founder of Suda Right, for being part of the Cognitive Revolution.

• 1:42:00Speaker 00

  Thanks, Nathan.

• 1:42:01Nathan Labenz?

  It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please, don't hesitate to reach out via email at [email protected], or you can DM me on the social media platform of your choice.