The perplexities of information retrieval

Daniel & Chris sit down with Denis Yarats, Co-founder & CTO at Perplexity, to discuss Perplexity’s sophisticated AI-driven answer engine. Denis outlines some of the deficiencies in search engines, and how Perplexity’s approach to information retrieval improves on traditional search engine systems, with a focus on accuracy and validation of the information provided. Sponsors: Neo4j https://graphstuff.fm/episodes/2023-finale-llms-and-knowledge-graphs-throughout-the-year?&utm_campaign=UCGenAI&utm_content=AMS-SrDev-ToFuDev-UCGenAI-Audio-None-GenAI1-GenAI-NonABM&utm_medium=Audio&utm_source=PracticalAI&utm_justglobal= – Is your code getting dragged down by JOINs and long query times? The problem might be your database…Try simplifying the complex with graphs. Stop asking relational databases to do more than they were made for. Graphs work well for use cases with lots of data connections like supply chain, fraud detection, real-time analytics, and genAI. With Neo4j, you can code in your favorite programming language and against any driver. Plus, it’s easy to integrate into your tech stack. Backblaze https://www.backblaze.com/cloud-backup/personal/landing/podcast/practicalai – Unlimited cloud backup for Macs, PCs, and businesses for just $99/year. Easily protect business data through a centrally managed admin. Protect all the data on your machines automatically. Easy to deploy across multiple workstations with various deployment options. NordVPN https://nordvpn.com/practicalai – Get NordVPN 2Y plan + 4 months extra at nordvpn.com/practicalai https://nordvpn.com/practicalai It’s risk-free with Nord’s 30-day money-back guarantee. Featuring: Denis Yarats – LinkedIn https://www.linkedin.com/in/denisyarats, X https://x.com/denisyaratsChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Perplexity https://www.perplexity.aiPerplexity | Blog https://www.perplexity.ai/hubPerplexity | Getting Started https://www.perplexity.ai/hub/getting-startedUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:06Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related tech is changing the world, this is the show for you. Thank you to our partners at fly.io, the home of changelog.com. Fly transforms containers into micro VMs that run on their hardware in 30-plus regions on six continents. So you can launch your app near your users. Learn more at fly.io.

• 0:45Speaker 01

  What's up, friends? Do you remember when ChatGPT launched? I do. It felt like the LLM was this magical tool out of the box. However, the more you use it, the more you realize that's just not the case. The technology is brilliant, don't get me wrong, but it's prone to issues like hallucination on its own. But there's hope. There is still hope. Feed the LLM reliable current data, ground it in the right data and context, then and only then can it make the right connections and give the right answers. The team at Neo4j has been exploring how to get results by pairing LLMs with knowledge graphs and vector search. Check out their podcast episode about LLMs and knowledge graphs throughout 2023 at graphstuff.fm.

• 1:30Speaker 01

  They share tips on retrieval methods, prompt engineering, and so much more. Don't miss it. Find a link in our show notes. Yes, check it out. graphstuff.fm, episode 23.

• 1:50Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I am founder and CEO at Prediction Guard, where we're safeguarding private AI models. And I'm joined, as always, by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How you doing? Chris. Doing great today, Daniel. How's it going? It's going great. I am sometimes perplexed throughout my workday, but generally the week has gone well. And I'm really excited because hopefully our guest today will provide a lot of ways for us to navigate through the perplexity. I hope he finds that funny.

• 2:31Daniel Whitenack?

  Yes, yes, of course. We have with us today Dennis Yaritz, who is the CTO and co-founder at Perplexity. Welcome.

• 2:41Speaker 03

  Thanks for inviting me and great to be on the show.

• 2:44Daniel Whitenack?

  Yeah, yeah. Well, we've been wanting to make this one happen for a good, long while. Of course, been following the things that Perplexity has been doing. Really impressive and inspiring work. And yeah, just really excited to hear a little bit about that story. And also this kind of space of answering, giving people knowledge with generative AI, with language models, search more generally, maybe. Could you set us up by maybe talking through, of course, people have followed, everyone's kind of followed this surge of generative AI

• 3:21Daniel Whitenack?

  over the last couple of years, but there's been kind of a segment of that that has focused a lot on kind of answering questions, discovering knowledge, curiosity, exploring topics, and intersection with search more generally, I guess. So could you kind of help us understand maybe that journey, how that's kind of developed? And how also you and the co-founders of Perplexity kind of came upon your approach to that?

• 3:51Speaker 03

  Yeah, that's definitely been like very fascinating, almost two years, I guess. I think like a general like web search, right? So like, what do you want to get ultimately get is an answer to your question, right? So it's the current iteration that we had like when it was Google and all the other like classical search engines is is approximation of to get to this point, right? So you have your question, you ask it as a form of query, and then you get like a bunch of documents that are very relevant, but you have to still do an ultimate sort of like, you have to do like additional work to get to the bottom of it, right? So you have to scan through the documents, you have to for yourself understand. What is, is this a true answer?

• 4:32Speaker 03

  If it's a not true answer, kind of like have to trust the information. And as you can imagine, it's a lot of work, especially if you're trying to search for something very complicated, maybe things that are not so obvious. And would it be nice to kind of like avoid that step, right? Where you just like answer a question, ask the question and you get an answer right away. So that's kind of like ultimate destination where we're trying to get. Obviously, there has been, it's been tough to get there over last like decade or so. It's, you know, there's been like a lot of work, but it never quite worked.

• 5:12Speaker 03

  There is this like a much higher level of hallucinations, much higher level of maybe not perfect synthesis of the information. You kind of like basically get like a Frankenstein. So like instead of like a hearing and a nice, easily readable, readable answer, you get like some just like basically extracted pieces of the information and you just like concatenate together, so like not very pleasant. And, you know, it's funny that when we started, so one of our angel investor was Jeff Dean, requires like no introduction. And he was saying, you know, Google actually wanted to always build something like this, but it's just because they have like such high expectations for accuracy.

• 5:56Speaker 03

  Because like, you know, millions of billions of users using Google, right? And like if you like hallucinate like 1% of the time, you're gonna get a lot of unhappy people. And so they would never able to, because of the models were like not as strong as they are right now. They were never able to get to just like 99.9% of accuracy, right? And that's why kind of like this work never pan out.

• 6:21Speaker 03

  But something great happened, you know, two thousand twenty two, right? So we, when we started our company, we kind of, you know, both myself and Aravind, my my co-founder, we've come from like academics, right? We've been doing like a lot of research and like language modeling, reinforcement learning and stuff like that. And he was actually at OpenAI at that time, like we've been very like clearly following improvements of GPT models like GPT two and then GPT three. That's where it's actually got very interesting. And it was kind of like became obvious and obvious there is going to be something there, right? And this was the primarily motivation for us to start a company. We wanted to do, build an answer engine from the get go, but it was kind of like very ambitious.

• 7:06Speaker 03

  You like, I remember we would go to the investors and there was a say like, oh, we're going to build a search engine. And they're like, start like, you know, looking at you like you're crazy, which is makes a lot of sense. They were so like, oh, there's Google already. And they had like a fair point, but we we still kind of like weren't like very discouraged by that. We knew there is like something there and we start like prototyping. So we, the first version of Perplexity, we actually created as a slug bot or like Discord bot, where kind of like was a very primitive combination of a search engine plus. At that point, it was like Da Vinci two model, so it's still, you know, pre chat GPT. And it's kind of like worked much better than I expected. It was like very quick, like we put it up this demo like in a couple of days.

• 7:49Speaker 03

  And it was like, you can already see that it in certain cases, it is very helpful. Like we, because this was like very early company, we were like trying to, we were like trying to hire like one of our first engineers. So we, and we didn't know like how to organize the insurance for him or like what it. And so we actually use this bot to like ask those questions about insurance. Because if you go to Google and start asking the questions about insurance, you're going to get a lot of ads. And you're just going to get very quickly, you know, disappointed.

• 8:20Daniel Whitenack?

  I've appreciated that same thing in founding a company and using these models in that, in that way. It's been useful.

• 8:27Speaker 03

  Yeah, exactly. And then it was kind of like useful, so we were kind of like been trying it out and like playing with it. But then what really happened, it started happening is like, I think it was like a couple of weeks before chat GPT. They, OpenAI released this Da Vinci three model. And I literally remember very clearly, like changed the, like literally the model name and started asking the same questions that I would asking before. And I could see right away, it's just like so much better. It's just like so much better at understanding what you of your intent, so much better at like understanding where like, what should it say, what it should not say, so much better like synthesizing the answer. And I was just like really like blown away.

• 9:09Speaker 03

  And you know, um, I was like, okay, so there is like definitely something there. And then it was like, chat G P T happens like a few weeks after and we were like, okay, so we have to, like our initial product was actually, because as I mentioned, we came from academic background, like our citations were like the core component. Because it was like very clear to us from the beginning, like if you want to get an answer, you want to make sure it's accurate. You want to make sure you can verify it. And and so the citation was like this sort of like first class citizen in in our product. And then when chat G P T came out, we was like, okay, so one of the biggest point of uh feedback for them was, okay, so I don't know, like if this is accurate information, if it's not, if it's hallucination, if it's not, how would I verify it? And that's why we like recently decided, okay, so this seems like a good opportunity to release our product. We like

• 9:54Speaker 03

  literally in a matter of like two days, put up a website uh connected to our like backend that we had. And just like, obviously did not expect that it's gonna uh be um people gonna use it and like the usage is gonna grow as much as possible. But coming back to your uh original question, I think like what happened is just like literally this in a matter of like days or month. I mean, obviously it follows like a lot of uh years, many years of research, but like it was very clear step function in the quality of the generated answers. And you can like literally, if you if you sort of like spend some time playing with it, you can clearly see that it's now becomes very uh very good. We also realized at that time, because like models only gonna get better, things only gonna get like faster and cheaper. So there is something, there is a lot of stuff to build here.

• 10:39Chris Benson?

  For those who are still kind of learning about your organization and what you're offering, could you step back for a second? Like if you were talking to Jeff Dean or another investor, kind of giving them the elevator pitch about what you're doing specifically and how it's differentiated from the GPTs and stuff out there, how do you define that? How do you describe yourself in terms of the specific opportunity that you're pursuing?

• 11:05Speaker 03

  The fastest way to get the most accurate answers to your questions. I think that's essentially like Answer Engine. We're kind of like one of the first people who coined this term.

• 11:15Chris Benson?

  And can you differentiate that from a search engine a little bit there?

• 11:18Speaker 03

  Yeah, yeah. So basically in a search engine, right? So you get like all the way. It's like, as I mentioned before, in the search engine, you have to search first, you get the documents. Let's say you have like top 10 results and you have to scan through all of them and identify the information to get your answer. Here, we kind of like, we do this step for you. So we take the first step of the retrieval of the relevant documents and then we synthesize them into a human readable, nicely formatted answer that you can then like, if needed, if you want to get more information, then you can click on the citations that are kind of like nicely attributed for each sentence and then you can like learn more information. And so we kind of like wanted

• 11:59Speaker 03

  to do things like very simple. We were like, early on identified, so there is like two things we care about. This is accuracy and then speed, because this is, you know, you want to get information fast. Google trained us all to get instant results, search results. I think that was like very important. And early on, it was like kind of like challenging because those models were like very slow. Our like infrastructure was not very advanced to do that. So like you, I remember like very first version, you would have to wait for like three seconds or like five seconds to get an answer. It was like very slow, but because they were like, it was like such a better experience than just like looking the search results at Google. So people still would

• 12:39Speaker 03

  But our ultimate goal was like, okay, so can we be as fast as possible? So yeah, so the the main differentiator is just we care a lot about quality. So we minimize the chance of things being inaccurate or like hallucinate. And we want to do it as fast as possible. And so that's that's kind of like distinguish us from Google because, you know, Google doesn't, for example, generate the answers, even though like more recently they started doing this, which is kind of like validated our idea. And chat GPT probably primarily focuses on like different things, you know, but I guess also more recently they started doing a web search as well.

• 13:29Speaker 01

  What's up, friends? I love Backblaze. I'm happy to have them as a sponsor. Backblaze makes backing up and accessing your data astonishingly easy. This is a service I personally use. Go to backblaze.com slash practical AI. You get unlimited cloud backups for Macs, PCs, businesses for just ninety nine dollars a year. You can easily protect business data through a centrally managed admin, protect all the data on your machines automatically, easily deploy across multiple workstations with various deployment options. You can add on enterprise control, including granular access permissions, advanced single sign-on, group management controls and compliance support. They even offer multiple restore options, including rapid recovery in the event of

• 14:14Speaker 01

  data loss or ransomware. That sucks. You can access your backed up data from anywhere in the world using their web app or their iOS or Android app. You can even restore by mail. They'll give you a hard drive with all your data shipped to your door. You buy a hard drive restore, send the hard drive back within thirty days and get a full refund. Get one year file retention and version history, over fifty five billion with a B files restored for customers so far. Visit backblaze.com slash practically eyes so they know where you came from and continue to support the show. This is a service. Obviously recommended by me, but also by New York Times, Inc. Magazine, Macworld, PCworld,

• 14:55Speaker 01

  LifeWire, Wired, Tom's Guide, Nine to Five Mac, and just so many more. You receive a fully featured or no risk trial at backblaze.com/practicalai. Again, their support in the show, go there, play with it, start protecting yourself from potential bad times. Start today.

• 15:38Daniel Whitenack?

  So you mentioned a few things there. You mentioned web search, you mentioned retrieval, you mentioned the large language model. So at least in kind of how I think about it, and maybe others categorize it differently, there's one element of information that you can get from an LLM, which is I'm going to put in a prompt and it's going to generate text and that may contain some facts or made up facts or some text, but it may be informational, right? So there's some sort of knowledge that can be gained there. And then in a second case, there's a way to retrieve on the fly, external data. So that could be like from

• 16:19Daniel Whitenack?

  your company's documents, it could be from the web, whatever, and then inject that into prompts into the model, which kind of grounds it and like you say, would give you a citation. There's also, you know, more agentic approaches to this where maybe there's multiple ways that you could get knowledge, maybe doing a web search or searching a certain database that you have access to, or any handful of sources that you've kind of curated as tools and you call them in a more automated way. So I'm wondering from your perspective, obviously you are part of a team that have been exploring this very deeply from very early on, like you say, when these models made that jump.

• 17:03Daniel Whitenack?

  From your perspective now, both in terms of what you're building with Perplexity and also kind of how you generally see the ability to get accurate information from these models, how do you view those kind of categories in terms of their utility and what you're relying on for the accuracy element specifically?

• 17:23Speaker 03

  The way I see it's going to unfold, I think the tools and sort of like agentic behaviors, I think that's where it's going. I think it's going to be the main bottleneck for this right now is just like models are not smart enough yet to take into account and sort of like reason all of the information that is out there. But I think it's going to be like a main component. So there's going to be like models, they're like very powerful already right now. They're like trained on a lot of data, like basically internet, they have a lot of internal information, internal knowledge, and they can do like already like very good job of synthesizing information. There's like certain things that they don't

• 18:04Speaker 03

  do well and perhaps they never going to do well, those things like, for example, like, you know, like computation, like when you need to do like some, maybe around like some code or like do like some sophisticated like math computations that like the LLM architecture or I guess transformers is like going to struggle at that. Also, you know, because those models are like so big, sometimes it's going to be very expensive to them update very frequently. So you need a way to ingest something, some new information that just like happened and it's still not part of the LLM weights. You have a way to, and this is what we sort of like specialized. Also, some private documents, as you mentioned,

• 18:47Speaker 03

  right? So sometimes if it's like enterprise, you know, you have a, some of the documents that obviously the model is, was not trained on and you kind of like maybe want to reason about those documents, right? And there is like all kinds of other tools that you can, yeah. Eventually, there is going to be like agents that is going to do like actions, maybe you're going to like book a ticket, like buy a ticket or something like that and and stuff like that. So I think it definitely where it's going, it's it's going to be like a synergy. Uh, everything's going to come together. We just need a top level powerful model that's going to kind of reason behind multiple things and we'll have to, need to have like long context windows, maybe like some memory as well. And then, you know, just like utilize those tools as much as possible.

• 19:30Speaker 03

  How we are perplexed to thinking about it, we are like, okay, so and there's like multiple ways and multiple sort of like applications, multiple use cases of this general approach. We are primarily focusing on the information retrieval part of it. So kind of like initially web, web is the main component because there's like lots of things to instruct from web, but also we're thinking about how to integrate a new different data sources. Maybe like some of the different databases, maybe there's like some more complicated or like more specialized documents like, you know, maybe there's like PDFs or something like that, or like some financial data, things like that.

• 20:11Speaker 03

  The other aspect that I'm I'm very excited about and and we're working on is kind of a do, you know, right now we can answer, we can do like a great job of answering like complicated questions that you can get answers on Google, but like still not something where you can ask expert and then can get an answer, right? Like, what if I have like a question that is just like requires like multi-step of reasoning, so it requires like, okay, like search in web multiple times, analyzing information that during the issue retrieval and then like maybe refining the search. So kind of like those things that may maybe going to take some time, but like if you do it yourself, you would spend like, let's say like an hour. So here, maybe the system would spend like a thirty seconds and it's going to save you a bunch of time.

• 20:54Speaker 03

  So kind of like those use cases that's um that can answer like very complicated questions. And we believe that there is a world for for those type of questions. Technology like this is going to be useful.

• 21:04Chris Benson?

  As people are using, you know, an answer engine like yours more and more often going forward, and you kind of alluded a moment ago to to the fact that, you know, L L M's are not the the be all, you know, there are things they don't do well, like like mathematics and such and and a variety of other things I'm sure that we could all throw out there. But they're really powerful at what they do, but clearly there is a place and a need for both. The L L M's, you know, these largest models that get all the, they kind of suck up all the air in the news cycles, as well as many smaller models that are specialized, you know, a mathematics model that, you know, that you plug in. As we're looking at trying to, to use answer engines to retrieve information and that information is increasingly multimodal in nature,

• 21:48Chris Benson?

  in terms of what you're asking, how does the architectures of those come together? This is a space, it's not the first time we've asked it here, but it's evolving so rapidly. We're no longer hosting a model, you're now hosting a, a whole collection and they may be mixed with models that you're APIing out to and such. How does that look to you as you're building this company at this point?

• 22:10Speaker 03

  It's going to be always like trade off between, um, like if you have like one powerful model. I mean, yes, it can do like lots of, uh, general things like super well, but like it's going to be slower. It's going to be more expensive. One of our like key principles is just like, we want to do things very fast. So we can get like that answers as fast as possible. That means you have to design your system, like orchestration system in a way where certain things will have to rely on like customized models, like and something that is much smaller, much faster, but it knows it's like a specialist model. So it's not a model that like knows how to do everything, but it knows how to do like one task.

• 22:51Speaker 03

  And basically the, the challenge here is like, how do you balance between this like general models? And this like a specialist models. And I think we've, we've been doing, um, this like from the very beginning. So like when you send the request to perplexity, it's just like not one model. There is like, I don't know, at least like 10 different models trying to, um, do lots of things with, with your request. It's like all kinds of like ranking models, bunch of like embeddings. All different like classifiers and and stuff like that. And and like the the other trade off here is just like with general model, let's see. Um, one of the uh big, and I think it was like actually very critical component of uh why like company like perplexity in the first place became uh possible is.

• 23:35Speaker 03

  It's like the speed of iteration, like you literally can change the prompt and you can just get a new product like in a in a matter of like hours. Like imagine couple of years ago, if you wanted to build something like, you know, perplexed or like whatever other like GNI product. You have to collect data first, have to train the model, launch the product, see if this product makes sense, does it have market feed or not? If it has market feed, then you like start collecting data and and then and then you just like keep improving. So what been possible was GPT models like and and API is just like this kind of like flipped over. So you uh very quickly can build a product, see if there is a uh any signs of life for this product. And then you start collecting data, which is, I think, uh honestly, the most important thing.

• 24:19Speaker 03

  And once you collect data, you can like distill it, you can uh build like many other like smaller models and kind of like optimize the experience. So you can make the models faster, you can specialize them. I think this is the key. I think this is the honestly was like one of the most fundamental um changes in in the development. Uh and and we kind of like took advantage of of this uh theme like early on and then still using. But i- it's still tricky, kind of like imagine like every time you have this like specialized models and you have like, if you have tons of them, you have to then like. Treat each model like care about each model separately. So if you want like to train this model, so you have to spend some time on it. So uh you have to like evaluate it. So it becomes uh uh more difficult to manage.

• 25:03Speaker 03

  But on the other hand, uh you know, uh you have some great benefits. So the key is just like you don't want to go like too overboard with those models, like if everything is on uh like customized models. But also you also clearly don't want to have just like one model that's going to do everything. So.

• 25:19Daniel Whitenack?

  Yeah, I I have a question maybe related to that, which I think is a pain point a lot of people are feeling. And I'm I'm guessing your teams have felt, which you even mentioned this, like you you can have. Make a small change in your prompt or create a new prompt and all of a sudden it's almost like you have a new product. Which is sort of like amazing in one sense and really frustrating in another sense. Cause as you were just alluding to, it's like, oh, maybe I have these like seventeen different things chained together. And they all have prompts that I've worked really hard on. And then like tomorrow, you know, Lama seventeen comes out or something.

• 25:58Daniel Whitenack?

  And now like it behaves, it has a different character of behavior than the previous model that I was using. I'd love to use it, but now I have all of this. It's almost like AI model debt that I've that I've got in my system. Do you have any perspective on that or any anything that sort of has happened in your experience in this regard?

• 26:21Speaker 03

  Yeah, yeah, this is clearly been a been a thing and it's been happening quite often. And if I would guess that's going to continue happening. One thing to be like realized early on is, okay, so this is going to be the case. There is going to be like, there is not going to be like one model that rules them all. I mean, even though like for some time it was like GPT four, but like now we can see there's like particular like anthropic, you know, Gemini, like Lama. There is like going to be a future where there is several frontier models, right? Because of that, we decided, okay, so like let's design our infrastructure and our system in such a way that it's going to be model agnostic.

• 27:00Speaker 03

  Right? So and then that means, okay, so there's like a ways where you can evaluate each component independently. There is a way where you can quickly change things up to adapt for like a new model and stuff like that. And that's, it took some time to get there, but it's, I feel like it was like a very correct decision for us. And then, so for example, one of the advantage we have over, let's say, like things like chat GPT or like Lord or like basically one model providers. Companies is just like, we can seamlessly integrate many different models and like our users can like, okay, decide this is the, they want to use this model or they they wouldn't do that more. Like later on, as we progress, I think we can even like. Decide based on the complexity of the query,

• 27:45Speaker 03

  or like the type of the query, we can like route to like a particular model that does the better job for those type of queries. And like, you know, minimize like maybe like some of the queries are super simple, you don't need to run like a very large model for those like to answer. So then you can like, I can optimize speed and things like that. So I feel like you have to just make a system in a way where it's like agnostic to the model.

• 28:19Speaker 01

  What's up, friends? Have you ever had trouble accessing that favorite sporting event? Or that awesome show? Or that film even? Because it's not in your region. Well, our friends at NordVPN can help you switch to a virtual location, to a country where it is available, unlocking a world of entertainment. Plus, it's not just about streaming. It's your go-to for online security. Protect your bank details, your passwords, and your entire online identity. If you're traveling, they can shield your data on public wifi, keeping you safe no matter where you're at. They also have this cool feature called threat protection. That means you can say goodbye to viruses, malware, and phishing sites, because NordVPN will protect you no matter where you're at.

• 29:03Speaker 01

  And one of the fastest VPNs globally, they ensure no buffering while streaming and no stops for your ISP from bandwidth throttling. It might sound costly, but think again. NordVPN costs less than a cup of coffee a month, and you can use one account on up to six devices. To get the best discount off your NordVPN plan, go to nordvpn.com-practicalai. Our link will also give you four extra months on the two-year plan. There's no risk with our 30-day money-back guarantee. Once again, go to nordvpn.com-practicalai.

• 29:59Chris Benson?

  So to follow up on what we were talking about before the break there, I know that you were talking about really building around model agnosticism to be able to handle that. I couldn't help but wonder occasionally as we get a new model out, it breaks new ground on modality, you know, being added in a whole new approach, that kind of thing. And so how do you, as a business builder, who is having to try to accommodate all these different models? When you have one that jumps out and has a completely new thing added in that was unexpected prior to the announcement, how do you, in the organization, how do you guys kind of pivot to accommodate that?

• 30:40Chris Benson?

  And keep that, the agnosticism, and yet provide that extra, you know, functionality that's now available? How do y'all tackle that problem?

• 30:47Speaker 03

  The most important thing to be is to not be kind of guard and try to anticipate what's going to happen. I think that's, that's very important. And it's kind of like, for the most part, I wouldn't say it was like too hard to predict what's going to happen. But after that, I think it's primarily like product decision, right? So like, do you, does this new feature benefit your product or not? Do we want to like build something in product or not? So for example, one great example was image uploading and kind of like multimodality, right? So we, we knew, it was like last year, we like knew for sure that this is going to happen at some point. Because, I mean, there was like already like some smaller models that kind of like supporting that. We knew that, you know, much better model is going to come out.

• 31:31Speaker 03

  And then because of that, we kind of like in advance start, first of all, like understood like, okay, so this is going to be important for our project. So we can support like this and this and that, those use cases. And we decided to build infrastructure in advance and kind of like anticipated. Obviously, we didn't fully like predict like how exactly it's going to operate, but it was like very close. So it was like required like little, like a couple of days to adjust. And then we were like one of the, you know, very quickly can release it as a product. So it's having the system that it kind of general enough that can like support those like new modalities. It's very important, get some great deal of anticipation.

• 32:13Speaker 03

  But also, you know, like sometimes, you know, like certain features that maybe come out, maybe they're like not useful for your product. You don't, you also don't wanna like put everything into, okay, so like this is a cool feature, let me just add it to your product. So like that, that's always not, not that great idea in general, so only things that make sense. And if, if those things makes sense for your product, you likely already like thought about like, how do you, how would you implement them in advance? So it's just like makes it a bit easier.

• 32:38Daniel Whitenack?

  While you were talking, I was thinking there's, there's sort of one axes that you have to navigate here around model releases and functionality and modalities, all of that stuff. There's sort of another maybe around UI and user experience. I, it's sort of multiple people making the comment, oh, well, like the chat interface came out with chat GPT. So everyone's sort of like focused around the chat interface. Is that the best way to utilize this sort of technology in the long run? There's probably a lot of exploration that's still open around UI and user experience with this type of technology. And certainly chat is, is relevant and, you know, we're using it a lot already.

• 33:22Daniel Whitenack?

  I'm wondering from your perspective, especially as we see this functionality maybe more embedded in the physical world around that, whether that be like in our glasses with, with meta glasses or in like kiosks in airports or whatever, whatever those things are. What is your perspective on how important it is to explore new types of, of, of UI or user experience with this technology?

• 33:51Speaker 03

  I believe like, and I mean, we, we kind of like, we're like very confident. Early on, it's just like chat interfaces is a temporarily thing. It's just too limiting. It has like a lot of constraints and that's why I like, you know. We didn't follow the uh usual route like it was all of the like chatbots they were like literally copied chat chat G P T and kind of like okay so put like chat interface we kind of like thought a little bit more about this and we designed it I feel like ultimately right now we're still in this like early stage where like people care about the model itself you know so like the model is the thing but uh as this thing they get more advanced like as more people start using G N A I products I feel like the the main thing is going to be product itself like what kind of things can product do

• 34:34Speaker 03

  if do you do it like better than this do you have like the best U I do you have the best U X and and that's why we we kind of like early on was like been thinking about those things and we kind of uh design our product in a way that is the mostly suitable for the things that we wanted to do right so if it's like searches like it doesn't make we we we knew that like chat doesn't make sense for search it's just like that's not how people search for information that was like a very big factor I think in our success um the other thing I guess we also even like last year we start like prototyping and experimenting with this like uh concept of uh generative U I so something where L L M can uh guide like what kind of like U I elements you can generate and then like

• 35:19Speaker 03

  sometimes you you know like one of the things in chat interface like if you want to ask like a follow up question sometimes it doesn't make sense to you know ask it and as a sentence right so maybe you want to like show like a checkbox or like a button or whatever like if there's like a it's just like especially on the mobile like everybody uses phones right it's just like not very uh convenient to type especially you if you're on the run so you'd rather like press a button that's why I like uh maybe uh speech uh and what I guess like voice technology is going to be one of the uh interesting modalities for sure I feel like an interesting interface because it's kind of a it has a lot of uh advantages obviously has like lots of disadvantages too but uh uh definitely going to be interesting and I think like going forward as we

• 36:02Speaker 03

  go towards like agentic behaviors and like more things but going to become possible I think It's definitely not going to be like chat interfaces. It has to be something else.

• 36:12Chris Benson?

  I'm really fascinated by this topic and it's, I think, something that both Daniel and I have some passion for and just in, you know, daily use and stuff. Yeah, I'm wondering with you thinking about that kind of productization, do you think that's something that perplexity engages in in a direct way or supports other companies through that? And then there are so many times in the course of a typical day where I'm wishing I had other ways of interfacing with these capabilities that we're talking about. And I'll give you just a, just a tried example. I, I will take my dog for a walk at a nearby park every day and that's my thinking time. That's where I'm really trying to be creative and I'm walking and I have to keep walking.

• 36:55Chris Benson?

  You know, I don't want to stand right now. I have to, I stop and I pull out my phone and it's frustrating and people are going by me and I'm trying to hold my dog and but I want this experience. It might be while driving, might be while walking the dog where this seamless way of utilizing these capabilities that we've grown accustomed to come about. Are you, A, how do you see yourselves being part of that next journey on the interface side? And B, do you have any ideas on how to get there? It's just, that's the next thing I want.

• 37:27Speaker 03

  Definitely looking into this, I think, and we consider like multiple options either like certain things I think we can do ourselves for certain other like things we probably have to work with some, some other like partners. Cause I mean, yeah. But I, I truly share your kind of like experience and I think it's, yeah, it's just like, if you sit in front of a computer, I think they are the, by far the best interface is a keyboard. I don't think you can do better than that. But yeah, if you, you know, occupied with something else, if you're driving a car, maybe you're walking you, there has to be something else. Like even like phone is, you know, it's, it's okay. Maybe like even like taking notes, maybe you like say a command and something like that, but and you get the voice back. That's already like something.

• 38:09Speaker 03

  But it misses like visual information. So you kind of like want to add that. So that, that means you probably have to have some sort of like glasses on. I think it's, it will definitely happen and we, we will try, we for sure, we like, we spend a lot of time this year improving our mobile app to do uh voice to voice. And we can like invest a lot into like voice generation. So for example, uh you can, yeah, ask like various questions and like, you know, like if you, if you need something quick, like you walk in, you, you like, I wanna have like a quick uh look up of information. So we support that. There is something, for example, for if you drive a car, we have this uh, you can read up like the stories or like discover

• 38:53Speaker 03

  uh from, from perplexity. That's also like uh AI generated voice. So it's kind of like you listen to a podcast or like a, uh so that's super important. Yeah, I think that the next step is vision and sort of like, how do you get there?

• 39:06Daniel Whitenack?

  Maybe one challenge that I've been thinking about this entire time while we've been talking relates to definitely a danger that I think people have identified as related to this new technology, which is you've already mentioned that you're kind of doing web retrieval or retrieval from certain sources as kind of a primary way of grounding answers of ensuring um accuracy of citations. But I know a lot of people are concerned about and thinking about this sort of idea of data poisoning where we're putting out actually a lot of generated content on the web, right? And that proportion of human and generated content is going to change over time, which means even for retrieval systems, especially if you're doing

• 39:50Daniel Whitenack?

  web related searches, there's a potential that you could retrieve generated content itself and get in this kind of weird loop. Of course, there's a separate problem for like the models and how they're training and I mean, this affects a lot of different areas, but I know probably you, you've been doing a lot of thinking about this because it's kind of key to how you operate as a system. Any perspective on that, that you feel like people should keep in mind, kind of moving forward or things that you're thinking about in terms of whether that's data curation or validation or I know a lot of people are talking about detecting generated content and that's maybe hit or miss. So

• 40:32Daniel Whitenack?

  there's all of this kind of connected stuff, but generally around this idea of data poisoning or generated content on the web. Any thoughts?

• 40:39Speaker 03

  To me, it's uh, it's kind of like a technological problem. I feel like it's Very reminiscent of like spam classifiers, right? Like it's just like whole another level or like, but let's say like 20 years ago when you like receive emails, yeah, you you would receive like a lot of spam and then eventually people develop technology that can like detect it. I feel like something like this will happen. So it's always going to be like a constant battle like the between like generators and discriminators. So at some point like generators maybe it's going to be better.

• 41:11Chris Benson?

  It's like fighting malware.

• 41:12Speaker 03

  Yeah, yeah, exactly. It's it's the the same concept. It's definitely going to be an issue for sure, but my hope is that and my belief that is the good guys, the good generators is going to be just like it's from machine learning fundamentals, discrimination is much easier problem than generation. It's like much easier to tell what is good, what is not than than generated. So and usually seems like we've been more successful in like detecting this stuff and I I don't see any reasons why it's not going to continue.

• 41:44Chris Benson?

  So this has been really fascinating as as we wind up here and have you here for you know, one more question as we we've talked about the future and have been kind of, you know, talking about what our expectations might be and how those might be fulfilled. Are there any other areas that we haven't addressed that you're that you're interested in? And possibly as as part of that, any way of kind of summarizing your own vision without it being just answering questions that Daniel and I have thrown at you, but your own vision for what the future looks like to you and what you want it to be and what perplexity is trying to realize it as to kind of paint a picture of what we might see over whatever timeframe you want to address.

• 42:28Speaker 03

  Yeah, yeah, so I'm I'm very excited basically to get to a point where, you know, I have any question, any any problem I want I want to have and then just like go to perplex and it's like get an answer or like suggestion or even like perform an action for this. I'll already mention this thing where kind of like in in increasing the quality or like the complexity of the type of questions you can ask and then making sure that the system will be able to handle those. I think this is. Definitely going to be the future and think we work hard on that. It kind of like opens up another dimension, you know, you can just, you can ask like lots of simpler questions

• 43:08Speaker 03

  or you can just like one have like one hard question. So it's kind of a, you know, complexity versus quality. And I think ultimately, when you get information, you usually use it for some sort of like decision making, right? And so if we can then take this information that we retrieved for you and synthesize in a form of answer, can we also then do like some decision making for you? And can you perform actions on your behalf? So like imagine, I don't know, like you imagine like you researching something, maybe you want to like research to buy like best running shoes, right? So it's a pretty painful procedure right now. Cause there's like so much stuff in the internet, like you don't know what you trust. So that that's why like, you know, if you identify usually like running shoes, you just like stick with them forever.

• 43:53Speaker 03

  Because it's just you, you, you trust, but like things like that. So imagine if you, somebody will do like this, like research for you, like really nails it down, like weighs all the pros and cons. And then suggests like it's like, do you wanna, this is the possible like variance, you wanna buy this stuff? And then you say yes, and then it goes like automatically buys it for you. And then like two days later it's delivered and you, you only need to type like questions once. You don't have to push your credit card in and stuff like that. And there's like tons of examples. So kind of, I would say the, the, the future is like first, you would have to make sure that you can nail sort of like the information retrieval part and kind of like generating the most useful information.

• 44:35Speaker 03

  The next step would be like decision making. So like make decision based on this information. And then third step to actions. So that's, that's where I think I would be excited to get to that point.

• 44:47Daniel Whitenack?

  That's great. Yeah. Uh, thank you for being willing to take time to dig into a number of topics that I know. I know our listeners are exploring themselves and also interested in. And certainly perplexity has been leading in a lot of these areas. So keep up the good work and, and thank you for, for the work and the perspective and taking time to talk. Appreciate it.

• 45:09Speaker 03

  Yeah. Thanks for the great questions and thanks for having me.

• 45:19Speaker 00

  All right, that is Practical AI for this week. Subscribe now if you haven't already. Head to PracticalAI.fm for all the ways. And join our free Slack team where you can hang out with Daniel, Chris, and the entire changelog community. Sign up today at PracticalAI.fm/community. Thanks again to our partners at Fly.io, to our beat freaking residents, Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us. That's all for now. We'll talk to you again next time.