Applied NLP solutions & AI education

We’re super excited to welcome Jay Alammar to the show. Jay is a well-known AI educator, applied NLP practitioner at co:here, and author of the popular blog, “The Illustrated Transformer.” In this episode, he shares his ideas on creating applied NLP solutions, working with large language models, and creating educational resources for state-of-the-art AI. Sponsors: Fastly https://fastly.com/ – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipFly.io https://fly.io/changelog – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog https://fly.io/changelog and check out the speedrun in their docs https://fly.io/docs/speedrun/. Featuring: Jay Alammar – Website https://jalammar.github.io, GitHub https://github.com/jalammar, X https://x.com/JayAlammarChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Jay’s popular blog (with posts including “The Illustrated Transformer”) http://jalammar.github.io/co:here https://cohere.ai/Topically sandbox - topic modeling https://github.com/cohere-ai/sandbox-topicallyco:here’s prompt engineering guide https://docs.cohere.ai/docs/prompt-engineeringUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:07Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners at Fastly for shipping all of our pods super fast to wherever you listen. Check them out at fastly.com. And to our friends at Fly, deploy your app servers and database close to your users. No ops required. Learn more at fly.io.

• 0:42Daniel Whitenack?

  Well, welcome to another episode of Practical AI. This is Daniel Whitenack. I'm a data scientist at S I L International, and I'm not joined by Chris today, but we have a very exciting guest. Today, we have with us Jay Alammar, who you might know from a bunch of great content out there, including the Illustrated Transformer and other articles also with Cohere. Welcome, Jay.

• 1:09Speaker 02

  Thank you. Great to be here. Thank you for having me.

• 1:11Daniel Whitenack?

  Yeah, yeah, it was great to see you at E M N L P back in December. We tried to sit down and have a conversation then, but it didn't quite work out, but still really great to see you there. What were your impressions just generally about like the experience at E M N L P and like the N L P crowd gathering there and what they're talking about? It was like just after Chat G P T came out. So yeah, what was your impression and some takeaways?

• 1:42Speaker 02

  Yeah, these are incredible events, like the amount of condensed download that you get in a conference like this of so much research, a lot of it you've caught up to, a lot of it is like new and for you to explore. And also like a lot of people to meet that, you know, in my case, most of them I've never met before. A lot of them like I've connected with online, but then having the chance to sort of, yeah, go deep into some of these topics and think through with people what they're thinking about. And also like observing what themes are sort of coming in, let's say, multiple poster presentations or workshops. So yeah, definitely awesome running into you. I'm glad we're catching

• 2:26Speaker 02

  up after that. I'm glad we got to shoot also one of your works in a video we've rolled out. And it's part of my excitement. When I was there, I was like, you know, people who don't go to conferences, they never really get the sense of what happens inside. And so I'm glad that you were doing, yeah, a bunch of these interviews during the conference to cover it for people who were not there. And I think there's room for a lot more sort of content like that. So yeah, it was definitely interesting and eye opening and quite intense in terms of just how much information and social interaction is done in five days.

• 3:04Daniel Whitenack?

  Yeah, I love it, but at the end of each day, I'm so tired. I like, I love being around people. I actually do enjoy that. But being an introvert, I'm absolutely exhausted afterwards.

• 3:16Speaker 02

  Hundred percent. And it's like on a weekend. So it's like also you're, you know, you're working through and then, you know, you don't have a weekend and so you're, you're working through it. So yeah, definitely need a day or two after to sort of cool down.

• 3:28Daniel Whitenack?

  Yeah. And you mentioned some of the content that you've been creating. I think many probably know your name or would have like recognize your blog and some of the blog posts, like I mentioned from like Illustrated Transformer, but then like you have so much after that as well. You were just mentioning some recent things you put out about stable diffusion. Where did you start developing this passion for like more of the educational side of this kind of state of the art AI stuff? And what's your perspective on that? And what, from your perspective, are you trying to kind of achieve with some of the things that you're putting out?

• 4:08Speaker 02

  Writing publicly and like learning publicly is maybe one of the greatest life hacks I've ever stumbled into. It started about eight years ago when I was just getting into machine learning. There was an event that happened, which was TensorFlow becoming open source. And I thought, okay, now it's a good time to jump into machine learning. I've been eyeing the field with interest. I've been seeing a lot of the like deep learning developments, but I had no exposure to it previously. So I was, you know, not necessarily working in the industry, but I was extremely interested in it and I wanted to learn. It's very easy for you to spend some time learning about a thing, but then, you know, three months in, six months in,

• 4:48Speaker 02

  you need some sort of an artifact to point to that really gives you a sense of, you know, how much you're progressing in that field. And for me, some of these initial artifacts was, yeah, writing a tutorial that sort of captures what I've learned in those two or three months. It also serves at me just writing down my notes for me to learn a concept a little bit better. So I did that a few times and it developed in so many different ways. So it opened so many doors that pulled me closer to machine learning and helped me learn more and more and more. Like if I'm to understand a new paper that came out, I can read the paper and, you know, maybe understand or grasp, let's say, you know, 20% of it.

• 5:31Speaker 02

  But if I'm to explain it, I really have to go in, you know, much deeper into it, or if you're going to implement it as well. Because if you're writing about it or explaining the work, you don't want to write something out there that is, let's say, incorrect. So in a way, it forces the social circuits of my brain to exert that pressure for me to learn. And so that's been extremely useful and that's opened so many doors. So through the blog, Udacity reached out to me and I worked with them for about two years, creating lessons on their deep learning and NLP nano degrees, which include creating a bunch of videos and code examples, which I continue to do after.

• 6:09Speaker 02

  So that included, let's say, the YouTube channel that I've been, you know, creating some of these videos on that explain, you know, a bunch of these models. But yeah, it's a very fast-moving field and there's so many exciting things happening every, you know, couple of months. And when I get some time and I sort of have a... My eyes on something that is especially interesting. Yeah, I would sit down and try to sort of do a write up. And so some of these milestones of, let's say, models have been, so yeah, the transformer was a major one. G P T two was a big one, G P T three, BERT. Retrieval augmented transformers, I've written about that, really interested in multimodality.

• 6:51Speaker 02

  So when a language model has the ability to read or like look at images or generate images. So I have a write up about one of these models. And then, yes, the latest has been image generation models. So stable diffusion, how these models work and how text language models factor into their composition is also sort of another sort of fascinating. Component for me in that. So, yeah, definitely one of the most useful things that I do and just, I'm really excited to see people, you know, finding that content helpful in their journey. People love that visual aspect of, you know, don't assume that the reader knows everything about the topic.

• 7:34Speaker 02

  So a lot of people jump into the field, either, you know, they jump from computer vision to N L P. Or they jump from being a Python backend developer to becoming, and so just thinking about some of the content pieces as being gentle on ramps for them. Is something on my mind as I write these and I encourage people to, you know, write whatever they learn. And we need so much educational content out there that it's the most useful life hack I can think of.

• 8:02Daniel Whitenack?

  Yeah, I totally agree with you. I think generally the podcast, as we've developed it, you know, Chris and I have developed it over time. Our focus is mostly on sort of like community and like bringing guests in and all that. But the one sort of like selfish, amazing thing that I've found also is like having conversations about all of these topics. And occasionally Chris and I do like just episodes, him and I, where we dive into a topic like we did one on chat G P T. We did one on stable diffusion and others where like having to talk about these things like in a time window and try.

• 8:39Daniel Whitenack?

  To also not say that many inaccuracies. It is a challenge, but it's been really useful for my own learning, that's for sure. Do you have any tips or sort of cheat codes for those that are like, "Oh, I would really like to kind of get into this arena of creating educational content around technical topics around AI or NLP or whatever it is." Because like you say, there's such a need for good content around these things out there, more so than, like, the research paper is great, but, you know, follow-ups to that and educational content around things is really useful. So, any tips or, or things you'd like to share for those out there that are thinking about

• 9:24Daniel Whitenack?

  either blogs or videos or whatever it might be?

• 9:27Speaker 02

  Yeah, so like, the first one is start. Just pick a medium and start with it, whether it's audio, whether it's video, whether it's like, say, short tweets or tweet storms, or a blog, or, you know, tech talk videos. So just pick something or experiment with a bunch of them until sort of you find the channel that you're comfortable with. But definitely what I see a lot of people do is waiting and waiting and waiting for them to have, let's say, their magnum opus and have that sort of masterwork be the first thing that they release, which is the wrong way to go about it. So you'll definitely improve a lot quicker.

• 10:06Speaker 02

  People will not, you know, judge you harshly for sharing something that you've learned out there. And a lot of people are held back by some sense of imposter syndrome because, you know, they're learning something and, you know, there are so many experts out there about this thing. So they're like, you know, "What can I add to this conversation?" And in fact, you can add a lot. You're seeing it with fresh eyes, you're seeing it at a different time where there are different resources. Even if the only thing that you're doing is putting together a curated list of, you know, resources that you've found helpful is a useful thing in its own sort of regard. So definitely starting and with time, finding your own voice and finding your own comfort

• 10:49Speaker 02

  and improving your craft is something that will come with practice. You'll definitely not be happy with your first output, but as long as you're putting it out, you're learning and you're sort of nudging your way closer into that place. To me, it's always useful to sort of, yeah, emulate your heroes. You know, what kind of content do you consume? What kind of content did you find helpful? What exactly about it was helpful to you? So in my case, you know, coming into machine learning, like I can identify a few writers and bloggers who, you know, whose code or articles really were helpful to me. So Andrej Karpathy, for example, you know, had this article about RNNs, the unreasonable effectiveness of RNNs.

• 11:33Speaker 02

  That was one of the first times where text generation really clicked for me and that it's finally really possible for software to generate text and that is, you know, somewhat coherent. I learned a lot from, yeah, just the styles of Chris Hola, from Andrew Trask and sort of writing a neural network tutorial with just 13 lines of Python. And that was really one of the first times where machine learning sort of really clicked for me. So, yeah, that would be the, I think, the second one. Like, see the sense of what connects with you. Try maybe to emulate it. Don't be shy of stealing. You know, the Beatles spent years and years just doing covers until they were comfortable with their own sort of sound. And so that would be the second one.

• 12:18Speaker 02

  But, you know, really comes down to just create, put out there, get some feedback, continue creating and just move the cycle.

• 12:26Daniel Whitenack?

  I remember at EM NLP when we were chatting a bit, one of the things you mentioned was how you were really enjoying, I forget the exact words you used, but it was something like making NLP boring, or something about applied NLP, or something like that. What did you mean by that? And what are your kind of passions around like your day to day work at this point?

• 12:49Speaker 02

  Yeah, so I mean, we're blessed to be working in a field that is very hot, very rapidly moving. And every now and then you see something that is super impressive beyond the capabilities of what you think software should be able to do. Now, you can see a demo out there, or you know, you can come across social media posts that show that Model X does Y, which is, you know, mind blowing. But then how reliably really is the model able to do this specific task? So, a massive GPT model is able, let's say, to answer a question about programming. And you can see a demo, or you know, a screenshot online of it answering a question correctly.

• 13:32Speaker 02

  But then is this really reliable for you to build a product around for this specific use case? And you know, the example here is that for some use cases it is, for others it's not. But if you're just seeing these demos, you have to really ask yourself, is this a cherry picked example, or is this reliably something that the model is able to do? And for this example, which is answering questions about programming, that is a, you know, large class of complex problems that models are currently not able to really be reliably good at, as evidenced by Stack Overflow banning the posting of answers from GPT models. So that's one example of, you know, use cases where, yes, you will see flashy demos, but you know, the reliability isn't there.

• 14:17Speaker 02

  But there are other use cases where the models can reliably generate sort of use cases. And so I work with and collaborate with a lot of developers and companies who are trying to roll out these models and they, you know, would come in with a specific understanding, but then when they try to use it for a use case where the model is expected to reliably be correct, they, you know, find that the demos in the real world are a little different. And so, yeah, there's a little bit of education and let's say a learning curve of how to best roll out these models and how to think about the various use case and how they differ, for example. And so, yeah, I'm really excited in these, let's say, playbooks of how to roll these models out reliably,

• 14:59Speaker 02

  which ones are, you know, ready to be used and now for various use cases. Like one example here is, let's say, neural search and semantic search and it's using embeddings to create search systems that go beyond just keyword search systems. So that's a very, let's say, reliable and mature use case of AI. It's not part of, let's say, the generative AI hype out there, but it really should be. And so that's a little bit of the example that I feel we as an industry owe the people who are just catching up to these developments to, you know, have a discerning eye of, yes, there are a lot of exciting things here. Let's not get sort of tricked by the hype across everything and have

• 15:43Speaker 02

  some non-realistic expectations for some use cases that are still futuristic. The models are still able to do incredible things right now, but the some of them will continue to be developed. Yeah, that's a little bit of how I think about it in the, I mean, that is definitely prefaced by saying that there is something really special happening here and software being able to understand, quote unquote, understand and generate text coherently, you know, has a potential that is beyond what we can really fathom. So it's right to be excited about it, then yeah, go deeper, be a little cautious, be discerning of cherry picked examples versus reliable use cases.

• 16:24Daniel Whitenack?

  When I started first consuming like your blog content and other things and thinking about transformers and then like going over to a notebook, trying to do a few things. At a certain point, it was quite difficult to like overcome that barrier and start to integrate some of these things into your applications. And we've moved to a time when it's so easy to pull these models in. Now, the problem is not so much like good tooling around this because it is fairly easy to do this at this point in terms of like an integration, but more so around like the workflows and best practices and how to judge like is this model a good fit for this

• 17:06Daniel Whitenack?

  use case or how could I use this model, that sort of thing? Would you say that's also how you're seeing it? And I'm also wondering, you know, you mentioned talking to clients and customers and and people doing, you know, building up new applications around these technologies. Have you seen a shift in terms of like most of those people being now like software engineers instead of, let's say, like data scientists or that sort of thing? And if so, how does that influence kind of how how we think about building AI tooling and who we're building it for maybe?

• 17:41Speaker 02

  For the first point, definitely, there's a lot of, you know, playbooks being created. So first, it started out with prompt engineering. So how do you get the model to do some behavior that is useful for your use case? And they're definitely capable of doing it. But then that's really not enough of a competitive advantage for you to build a product around. And so your playbook would need to include a bunch of other components. So can you have access to some proprietary data that others cannot have? And then one of the, let's say, also differentiating factors here is can you fine tune your own models so that you just improve upon just the baseline model that is publicly accessible or even open source in some cases?

• 18:26Speaker 02

  How can you continue to improve the quality of your own fine tunes and continue to collect the data that improves that model? And let's say observing the generations of your own model. If you're a company or a product that is, you know, building around a specific model, there are dynamics there that really affect the economics. So In image generation, for example, it's extremely useful to have a public gallery of your model's generations. So Midjourney has one of these, where, you know, if you want to create a specific kind of image, you can, you know, give the model a few prompts and explore that generative space. But you're really helped by the model having a massive

• 19:07Speaker 02

  gallery of hundreds of thousands of examples that really nudge you towards the best way, direction, visual space, uh, kind of prompt type or description of style. And so that's, let's say, another example of one element of a, let's say, generative AI playbook. Keeping on top of the research is also another source of, uh, really good ideas for how to roll out a lot of these models. So in, you know, two, three years ago, there was a lot of these demos when, let's say, GPT-3 first came out of the model answering something factual and bringing, let's say, a specific fact. So asking it a question and the model, and it's surprising when that happens, but then you know with time that that's not a

• 19:49Speaker 02

  necessarily reliable use case for these models up until now. But there is a way towards one where you're not just asking the generative model for the information stored inside of its parameters, but you aid it with a search component. There's a part of your system that goes and searches a database or the web and then retrieves relevant articles and presents those in the prompt to a generative model and in this process sort of improve what kind of a model. So that's another sort of element of if you want to tackle this use case, you know, don't just rely on the pre-trained model's parameters and information stored there, but let's say augment it with a retrieval component.

• 20:31Speaker 02

  And, you know, a bunch of companies are figuring out, yeah, a bunch of these. And I think as a community, we're also sort of working on that together. I'm, I'm working on sort of some write-ups to. Try to codify and, you know, make public some of this sort of gray knowledge that's coming together across generative AI use cases on both the text and image space. Definitely the last few months, to address the second part of your question, there's been an absolute explosion in the public's excitement of text generation models. So yes, ChatGPT is one of these models and then the generative chat battles happening between Google and Bing and these product rollouts and sort of the waves that they're

• 21:15Speaker 02

  making throughout the industry are definitely putting text generation and language models at the tops of the minds of a lot of people and a lot of developers are sort of trying to figure out, you know, how they can start using these models, how they can think about them. Yeah, it's been an absolutely tremendous couple of months to see that growth, which is not just developers, but it's like people in the street, just, you know, your parents coming up and saying, okay, we finally sort of get what you do, which has been absolutely surprising.

• 21:46Daniel Whitenack?

  Yeah, I really like how you highlighted this sort of like gray area, gray matter, whatever you described around this, like, knowledge of how to put these various pieces together into a solution. I guess it's sort of like solutioning with a certain set of like potential pathways forward with a state of the art model. So like used to, like, when there was sort of data science hype before all the AI hype, it was like, okay, you need training data, you're going to train a model. That was sort of like the playbook. Now you're in this scenario where you're like, okay, well, I have a pre-trained model. Am I going to do some type of fine tune? So that's like something there.

• 22:29Daniel Whitenack?

  Am I going to focus on prompt engineering? Am I going to chain multiple things together? Am I going to do some retrieval and pull in like external knowledge into that? So there's like so many of these things where like the chaining and the assembly of the solution is actually where the value comes out, which I think is a really good thing for people to think about. I'm wondering, because you've, you're in this all the time, you're like, you're seeing new problems and I know you've even showed me, you know, cool solutions you put together around like topic modeling and like labeling topic names with generative models and that sort of thing. Like when you come to something like that.

• 23:08Daniel Whitenack?

  How do you parse through like this intuition around, well, is this a situation where I'm like chaining multiple models together? Is this a situation where I really need to focus on prompt engineering? Is this a scenario where maybe I should be focused on fine tuning? Any sort of guidance or thoughts around like how to develop the intuition around that sort of solutioning? Maybe part of it is just experience, but any suggestions around that?

• 23:37Speaker 02

  Yeah, I'm in complete agreement to what you said. There is this frontier forming between where you train models and where you're just a user of models. Because of just the very high quality of pre-trained models and their ability to solve general problems. So their general problem solving. Vast majority of my work is above this frontier, so not on the modeling layer. I would want to explore, you know, what is possible with pre-trained models. Generally, we have to fine tune, but then, and that is aided by just the surprising thing of these large generative language models. They do few shots generation really well.

• 24:21Speaker 02

  So if you give them three or five examples of a type of generation or a style of generation that you want, they tend to catch on to that and give you something that is, you know, good enough for a lot of use cases. And then, you know, I know that fine tuning would be the next step to that. So if that is not enough, and if there's a use case where, okay, I have, you know, 500 labeled examples or 1,000, that's when I would sort of try to reach to to fine tuning. But in context of just providing a few examples to the model, really helps solve a lot of use cases. And so, yes, the solutioning sort of aspect is where the headspace sort of, you know, tried to think about.

• 25:02Speaker 02

  Just using the APIs of these models and yeah, how to chain them together, how to think about, yeah, fine tuning. And not fine tuning, but let's say embeddings and then using those embeddings for specific tasks for retrieval and then chaining that with generation. So this is a vastly underexplored and let's say new frontier. Because and you can completely Spend forty years just learning the training layer and the various model architectures and the various ways to improve the data and fix the data. So we need people in all heights of the stack. But then the engineers right now have this widely available sort of an underexplored

• 25:46Speaker 02

  area of what can you do with pre-trained models, a lot of them sort of via APIs. And you can definitely do a lot.

• 25:55Daniel Whitenack?

  Could you just give people maybe who aren't familiar with Cohere some general intro to what Cohere is trying to do and what they offer?

• 26:04Speaker 02

  Absolutely, yes, glad to hear that. Cohere offers an API for large language models. And the goal there is to make using language models easier for every developer or company out there without thinking about, you know, hiring an army of people to train large transformers. Our founders came out of Google Brain and one of them was the co-author of the Transformers paper. And we have teams that are focused on training, let's say, two kinds of models. So the generation large GPT models, we train, let's say, both of these families in-house and continue to develop and improve them. And the other family is text embedding models. And these are the ones that can power use cases like neural search, semantic search, text classification.

• 26:47Speaker 02

  So if you want to classify messages by topic or by sentiment, they're very sort of capable in that. And the latest release has been the multilingual embedding model that supports over a hundred languages. So if you want to do semantic search or neural search, you don't have to build a hundred different pipelines for each language that does, I don't know, stemming and, you know, very language-specific pipelines. You can just throw it all at the embedding model and just retrieve, you know, the best results. And so that's the core tech and the company offers all of that via API. And yeah, we invest a lot in the content and educational side. It's still an area that is quite new.

• 27:29Speaker 02

  Large Language Models as a Service is a new brand of company. It's only been around for, you know, two years. And so, yeah, we focus a lot on the educational side of the various concepts that are needed there to help both developers but also a general audience capture and build those intuitions. And that's, you know, something that companies had to do throughout the development of technology. So. The majority of executives now would know what an API is, for example. But you know, fifteen years ago, maybe twenty years ago, API or big data or cloud hosting were all sort of, you know, deeply technical words or words that, you know, didn't yet develop. And that is the same now with things like embedding and fine tuning and base model, yeah, language model.

• 28:14Speaker 02

  And so, yeah, definitely the education is a part of that. And a lot of it is just us, you know, learning with our developer community and sharing the common lessons. Uh, so developer number ten thousand doesn't have to repeat the mistakes of the previous developers. We're really passionate about that as well.

• 28:31Daniel Whitenack?

  Yeah, part of the advantage of having some of this, and I'd be curious to hear your thoughts on this as well. So, you know, going back to like something I said earlier when I was first learning about transformers and trying to get hands on with these things, like there was no like easy cloud API for me to just like access and use. I think now like even for open models, that's that's shifting a lot where you can, you know, host things on hugging faces inference API, or you could, you know, use any number of these services. So you could use Cohere's large language models, you could use like replicate and what they have in their cloud APIs and that sort of thing.

• 29:14Daniel Whitenack?

  So a lot of these are available now where it's a small number of lines of code and you're able to access the API. What do you think are the implications of that for, you know, how people are thinking about using models maybe differently than they thought in the past because of these access patterns? Um, maybe it's less people are are thinking about the training side and just like chaining. I I don't think it's an accident. We've seen a lot of like chaining things recently. But yeah, I don't know. I'm curious to hear about your thoughts on the implications of how this landscape is

• 29:53Daniel Whitenack?

  changing to where like we've kind of gone from, oh, let me download model weights to like I could chain this API together with this API and that sort of thing.

• 30:04Speaker 02

  I mean, definitely it makes it a lot easier for much more larger group of people to start experimenting with these models. Because it just lowers the barrier of entry so much. And it enables people to not think about moving tensors across GPUs and watching out for GPUs running out of memory and updating model weights. And it's just another abstraction. And you can think about it just like every other cloud service out there. So if you wanna build a new website, you no longer need to buy a physical machine, ship it to a data center, maybe go physically to that data center and sort of, you know, put the code on. It's been abstracted away as a service you can reliably access.

• 30:48Speaker 02

  Somebody else is, has the world's, say, foremost experts making that service reliable for you. And you can focus on your core business problem, the core sort of product that you want to do. And knowing that these other pieces are there and are being handled by people whose sole job is to, you know, maintain the quality and increase the quality of these models and the uptime. And this is especially a factor when these models are massive. They need to be on so many different machines and GPUs. And it's such a hassle to deploy your own model. Like, you need a PhD maybe to really wrap your head around everything that is involved in something like.

• 31:33Speaker 02

  And so, yeah, it just frees up people to think about, okay, this is an API. Think about the frontier of the next level of services that are now finally possible, that weren't possible before. And let's say we saw new industries come out of these developments in AI. So AI writing assistance, for example, is a type of industry where there are so many companies now that didn't exist before. And these companies just rely, in general, rely on APIs. And they can focus on really creating the best, you know, domain knowledge for them to help out their customers. So it really helps in the specialization and sort of that abstraction of not having to worry about this lower layer in the stack where others are sort of handling it for you.

• 32:15Speaker 02

  And fine tuning becomes as easy as uploading a text file rather than, you know, a process of babysitting a model for a week to see, you know, what happens. And so that definitely increases. The cycle of experimentation, but also the ease of deployment in accelerates, let's say, the coming of the next generation of products that are just now possible.

• 32:40Daniel Whitenack?

  Yeah, and one of the things that we talked about before we started recording was some of your excitement around like multimodal models and where those are going. I know that's also increasingly easy to kind of like tie different modalities together. I think the light bulb is going off for a lot of people. I even just had a conversation yesterday where someone was talking to me about like, oh, a large language model, could it do this or that like for their use case? And I said, yeah, but like, what if you just add the image component on as well? Like, you can, you know, generate the copy for your ad and the image for your ad,

• 33:18Daniel Whitenack?

  you know, with generated text and that sort of thing, which I know a lot of people are trying and a bunch of other things too. So what's on your mind in terms of kind of multimodality? I know you've written a lot about stable diffusion and other things recently. Where's your mind with respect to that? And what are some of the use cases that that you're thinking of or the sort of applied things that are interesting to you in that space?

• 33:44Speaker 02

  Yeah, so on the research front, I've written about Gato, DeepMind's Gato model that does images and text and a lot of other modalities, which is an interesting research development. On the more applied side, we've released a notebook that does a little bit of prompt chaining. So a few researchers from DeepMind had this paper called Dramatron where they, you know, shared a system and a bunch of prompts that uses language models to write a screenplay. And it doesn't write it from one prompt. It's, you know, seven different prompts that do different things of the story, and then you end up with the screenplay. So there's a prompt to generate the characters and a prompt to generate the setting and then

• 34:24Speaker 02

  the beats of the story. And so you build this knowledge hierarchy and then, and so we have a notebook that showcases how to do that with coherent models, but also plug in some calls to stable diffusion models to generate, okay, so these are the descriptions of the characters. What might these characters look like? So that is an AI image generation sort of flow. This is a description of a setting where the Part of the story takes place, how can we sort of visualize it enough? And these are sort of the flows. And we see, you know, libraries kind of like a langchain that are empowering a lot of this chaining of the various text models, but also potentially image generation models.

• 35:07Speaker 02

  So yeah, that's the most, the use case that I went down. And it was only really possible because, you know, like for me to have a quick time to experiment with them, because there are APIs that do them. So on the stable diffusion front, I can share this code because it's an API call to like, say, stability AIs, stable diffusion models. But I've been wanting to do that with Midjourney, but Midjourney does not have an API that you can call. I have to do, you have to do it through like Discord, for example. And so that's also, that's a differentiating factor for these various products, you know, which ones support API access, which don't. So it will factor into, let's say, developer adoption for them on the infrastructure layer.

• 35:50Speaker 02

  But yeah, that would be specifically the one that I've experimented with the most.

• 35:55Daniel Whitenack?

  Awesome. As we kind of wrap up here, usually we ask our guests sort of like, what's on their mind moving into the next, you know, six months or so? What are you, what are you most excited about? What have you not explored yet? But what's like on the top of your list to, to dive into? Any thoughts?

• 36:16Speaker 02

  Yeah, I'm really excited about use cases that use both generation and embedding in the same sort of flow. That's one area where I think about a lot. And let's say the topic modeling and let's say cluster naming use case is one of them. And we've open sourced a library called Topically that does exactly that. So that's one area where I'm working closely. And I think these models can help us really understand large collections of data using that and, you know, create interesting visualizations of them as well. So yeah, for me, it's mostly, yeah, the interaction of these sort of two systems, hopefully in, let's say, supporting multiple languages as well.

• 36:57Speaker 02

  So multi-modality is interesting, also multilinguality is interesting for, you know, systems that can support even more and more data that's out there. I think these are most likely the ones that are just top of mind for me at this time.

• 37:11Daniel Whitenack?

  Awesome. Well, thanks so much for taking time to chat, Jay. I'm glad we got to do this and I hope to have you back on the show in a year so we can talk about all the fun things you've explored in the interim. So, thanks a lot.

• 37:26Speaker 02

  Amazing. Thank you so much. So good to catch up with you and chat about all of this. I look forward to speaking again.

• 37:41Speaker 00

  Thank you for listening to Practical AI. Your next step is to subscribe now if you haven't already. And if you're a longtime listener of the show, help us reach more people by sharing Practical AI with your friends and colleagues. Thanks once again to Fastly and Fly for partnering with us to bring you all Changetalk podcasts. Check out what they're up to at fastly.com and fly.io. And to our beat freak in residence, Breakmaster Cylinder, for continuously cranking out the best beats in the biz. That's all for now. We'll talk to you again next time.