AI-Powered Business Communications with Dan O'Connell, Chief AI & Strategy Officer at Dialpad

In this episode, Nathan sits down with Dan O’Connell, Chief Strategy Officer at Dialpad. They discuss building their own language models using 5 billion minutes of business calls, custom speech recognition models for every customer, and the challenges of bringing AI into business. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. We're sharing a few of Nathan's favorite AI scouting episodes from other shows. Today, Shane Legg, Cofounder at Deepmind and its current Chief AGI Scientist, shares his insights with Dwarkesh Patel on AGI's timeline, the new architectures needed for AGI, and why multimodality will be the next big landmark. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. --- SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. X/SOCIAL: @labenz (Nathan) @dialdoc (Dan) @dialpad @CogRev_Podcast (Cognitive Revolution) TIMESTAMPS: (00:00) - Introduction and Welcome (06:50) - Interview with Dan O'Connell, Chief AI and Strategy Officer at Dialpad (07:13) - The Functionality and Utility of Dialpad (17:20) - The Development of Dialpad's Large Language Model Trained on 5Billion Minutes of Calls 19:56 The Future of AI in Business (22:21) - Sponsor Break: Shopify (23:56) - The Challenges and Opportunities of AI Development (31:17 ) - Prioritizing latency, capacity, and cost when evaluating AI (39:41) - Most Loved AI Features in Dialpad (42:01) - The Role of AI in Quality Assurance (43:10) - The Future of Transcription Accuracy (44:06) - The Importance of Speech Recognition in Business (46:59) - Personalizing AI for Better Business Interactions (47:01) - The Role of AI in Content Generation (52:47) - The Challenges and Opportunities of AI in Sales and Support This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Nathan Labenz?

  It's not that I'm like racing to replace people or, you know, cut costs or whatever, but I always kind of come back to sort of in, you know, a Bezos style, like, what does the customer really want, right? And the customer wants like immediate response 24/7, like where I can pause the conversation where I want to at my convenience and be able to come back and pick it up, you know, right where I left off and maybe even switch modalities. And, you know, ChatGPT offers me all these things today. Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas and together we'll build a picture of how

• 0:45Nathan Labenz?

  AI technology will transform work, life and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello and welcome back to The Cognitive Revolution. As we head into 2024, I've been thinking a lot about where we are in terms of AI's impact on knowledge work. While 2023 certainly brought explosive growth in AI adoption, to be honest, things have moved a little less quickly than I had expected. At retail prices, $1 billion only buys one to two GPT-4 API calls for each of the world's 8 billion citizens, which really just goes to show what a tiny toehold language models have established in knowledge work globally.

• 1:29Nathan Labenz?

  Even if this were to go 100x over the next year, it's still just one GPT-4 API call per person per day, still a tiny fraction of the knowledge work that humans are doing. So why this delay relative to my admittedly very high expectations? And what are the likely solutions? First, while I've definitely argued that OpenAI has modes, I have been a bit surprised by how long it has taken for other companies to match the quality of GPT-4. It's fair to say, I think, at this point, that GPT-3.5 level models are effectively commoditized. But GPT-4 is different. Only Anthropic and now Google with Gemini are really even in the ballpark in the West. Though it's worth noting, and I do hope to

• 2:12Nathan Labenz?

  have another episode on this soon, that Baidu's Ernie 4.0 also appears to be a worthy contender.

• 2:19Nathan Labenz?

  A second major issue is that language models are weird, and the know-how to successfully implement them into automated workflows remains relatively scarce. Most companies are naturally excited about opportunities to save 90% of their costs by automating tasks that nobody enjoys anyway. But if they don't have or know someone who can execute on such a project, they really have no choice but to wait.

• 2:42Nathan Labenz?

  A third issue is that AI agents, which promise to allow people to delegate work to AI on a real-time, more ad hoc basis, have not matured as fast as I thought they might. In part, that's due to ongoing GPU shortages. I've repeatedly said that I think GPT-4 Vision, originally introduced in March, but only now hitting general availability, will boost agent performance. And indeed, we already are starting to see new reports of much better performance at significantly lower cost, simply because computer interfaces are designed to be interpreted visually, and GPT-4 Vision now makes that possible. What else is standing in the way? Another challenge is that it turns out that we can boost AI performance significantly by decomposing tasks into subparts.

• 3:27Nathan Labenz?

  But then, when we try to string long chains of these subparts back together into meaningful work, it becomes very tricky to determine what information to give the AI at each step along the way. Too much information can be unwieldy, and in any case, makes the agents slower and much more expensive. While too little information leads to bad decisions and overall failure. GPT-4 fine-tuning, which is also still in early-access-only mode as of now, will probably help here. Fine-tuning in general is very useful for shaping model behavior. And regular listeners will know from the recent emergency episode on the Mamba architecture that I expect state space models to deliver more coherent, agentic behavior as well.

• 4:10Nathan Labenz?

  But even if so, this would pose another challenge. Where are we going to get all the long-episode, context-rich, how-we-work-in-practice sort of data that we're going to need to train these long-context AIs? Where does that data exist today, if at all? One likely source of such data, I think, are the software platforms in which people currently do their day-to-day work, which naturally capture records of the work and also implicitly define the space of possible actions that a human or an hypothetical AI agent might take at any given time along the way. So with all that in mind, I was very interested to discover DialPad, a communications platform

• 4:52Nathan Labenz?

  that uses AI to deliver assistance, automation, and insights to human users, who today are mostly sales and customer success teams. Having been in business for years and having grown a customer base to more than 30,000 businesses, DialPad is now in a unique position with a proprietary data set of more than 5 billion minutes of sales and service calls, which they've now used to train their own transcription and large language models. This looks to me like a near ideal opportunity to begin experimenting with AI employees. Though, as you'll hear in this conversation with Dan O'Connell, DialPad's Chief AI and Strategy Officer, he's not expecting these changes quite as rapidly as I am. And based on my predictions last year, I have to say, there's a pretty good chance

• 5:37Nathan Labenz?

  that he'll end up being right. Yet, at the same time, regardless of how long it takes, it does seem clear to me that huge amounts of routine knowledge work will ultimately be done by AI. So while I'll continue to hold myself accountable for the best possible predictions that I can give, it does also seem to make sense for me to bias my analysis toward shorter timeline scenarios, simply because those are the ones in which AI scouting will ultimately become most valuable. As always, if you're enjoying the show, we'd love it if you'd take a minute to share it with your friends, or leave us a review on Apple Podcasts or Spotify. You would be amazed at how few reviews we get.

• 6:20Nathan Labenz?

  I literally get 10 times as many personal notes as we get online reviews. And yet, I understand that it does make a huge difference to podcast distribution. So I would really appreciate it if you would take just a minute to write a short online review. Now, without further ado, I hope you enjoy this conversation about AI and the present and future of knowledge work with Dan O'Connell, Chief AI and Strategy Officer at DialPad.

• 6:46Nathan Labenz?

  Dan O'Connell, Chief AI and Strategy Officer at DialPad. Welcome to the Cognitive Revolution.

• 6:53Speaker 01

  Nice. Thanks for having me, Nathan.

• 6:54Nathan Labenz?

  I'm excited to talk to you about the platform that you guys have built and are continuing to build and are making more and more AI-centric. One of my calling cards in doing this show is I do my homework. So I've gone in and created a free trial account and bounced around and called myself and transcribed a little bit. So I guess I would briefly just describe DialPad as a sort of almost everything hub for the work that sales and customer service teams in particular are actually doing. Like you're actually in DialPad a lot doing your work. Tell me about the product and your customers, I guess, for starters.

• 7:33Speaker 01

  Yeah, so you see me on video. I'm smiling and laughing as you're describing. You did exceptionally well. And I would add, you know, communications platform is the easiest way to think about it. We do voice video messaging on any device anywhere in the world. That can be used for internal communications between teams, but it can also be used, as you mentioned, for external communications, supporting sales organizations or customers. And we build AI throughout that platform. So how do we go capture a conversation, transcribe it, and then focus on building features that help with assistant automation and insights from those conversations?

• 8:08Nathan Labenz?

  Assistance, automation, and insights. Okay, we'll come back to that. I looked you up on G2 as well. Good reviews, upper right quadrant. 56% of the businesses SMB, 37% mid-market. What would you say is kind of the state of, you could even maybe broaden a little bit beyond AI, but I'm very curious about AI adoption, just knowing this business segment as I do and knowing that they're not always the quickest to embrace new technology.

• 8:37Speaker 01

  Yeah, and those are obviously self-reported from G2. We would classify ourselves on our revenues. And then just for context, for size of business, we're north of $200 million ARR, business, 30,000 customers. Pretty much a third, a third, a third across SMB mid marketing and enterprise. We have some really big brands that use this Stripe, Twitter, Uber, HubSpot. Then we also can go and power, you know, really small businesses that might be a law firm of two or three people. I would say like the general thing that we see from our customers are very much people that believe in the cloud to no surprise. But when you thought, when you talk about communications, there's still a lot of people that run their whole PBX system or phone system in a closet with wires behind it.

• 9:20Speaker 01

  And for us, it's really much, we see a lot of people that believe in look, managed services should live in the cloud. And those people tend to be on the forefront of wanting to try out AI powered features. So they believe in, you know, turning, having a transcription and being able to map sentiment as opposed to getting into kind of the, the arguments around wiretapping laws and call recording laws and things like that. Those are not the types of conversations that I would say like many of our customers are, are, are pushing.

• 9:49Nathan Labenz?

  Yeah, interesting. Okay. Well, I think that's a pretty, the third, a third, a third is a interesting accomplishment, really. I mean, to build software that can serve the enterprise and small business at the same time is a real challenge. I've, I've dabbled in that a little bit. And again, this goes beyond, you know, the, the AI focus, but the just raw number of features and configurations and, you know, next thing you know, you're kind of, how do we not become Salesforce? While we try to meet all these different customer requests and at the same time, you need something that's really intuitive for small businesses who don't need, you know, or want any of that stuff.

• 10:28Nathan Labenz?

  You know, I guess one thesis I've had and you, I think you've largely addressed it so far with just disciplined product development. But I've had the idea that a lot of that complexity maybe gets smoothed over, over the next year or two with natural language interface, like maybe everybody who has a crazy, complicated platform can start to go down market because they can, you know, hide 17 menus and say, you know, what do you want to do? And just translate the user need into.

• 11:00Nathan Labenz?

  Configuration on their own platform as appropriate. Does that seem realistic to you, even if it's not maybe your most pressing problem relative to other software platforms?

• 11:09Speaker 01

  Yeah, I actually agree with you on that. And I think, you know, what the interesting thing of, you know, I think one of the unique opportunities for large language models actually is putting them on top of analytics. And you think of business intelligence platforms today, because as much as we're a communications platform, as I said, like the stuff that got me really excited at this to tie this back is, hey, do you want to go power communication on any channel? We want to go understand it in real time. So let's go capture it, transcribe it. And once we have it transcribed, well, now it's in text format. We can do all sorts of things with it. And if you put a large language model on top of that, you suddenly have this robust analytics platform. If you think about building an analytics platform today, or even a database, you need a bunch of filters to go and search things. Right? And so that creates a lot of complexity. Everyone wants different filters.

• 11:54Speaker 01

  They want their data in different ways. And to me, one of the biggest opportunities now is you can put a conversational search interface on top of analytics. And that, to your point, streamlines the complexity of the software you need to build, simplifies the experience that users go and have. And I think really starts to unlock the potential of like this last, you know, I call it like the last offline data set, which is these conversations. So I get really excited thinking about that. Try to, you know, kind of tying it back to what you're saying. But yes, I think there's actually a simplification that happens. And we naturally think of simplification coming from the SMB market as opposed to the enterprise market. But I do think that's something that's going to start to play out.

• 12:34Nathan Labenz?

  Okay, cool. So could you give me a little bit better sense of how, when I signed up, that, you know, there was the list of 10 possible, you know, departments and roles. And so I wasn't quite clear on like just how broadly the software gets deployed within an organization or kind of, you know, maybe it gets deployed very widely, but there's, you know, certain people that are the real in there hour by hour users. Like, am I right to believe, and I have kind of some big questions in mind that I think depend on this answer? How much of it is like? The sales and customer support versus like the whole organization?

• 13:17Speaker 01

  Yeah, I would say probably the fastest growing segment of our business is providing that uniqueness communications and AI for sales and support organizations. That said, the vast majority of those customers, if you take those 30,000 customers are deployed across the internal communications and as I said, our friendly competitors are, you know, to actually put it out there and clarify for folks are going to be the teams, Microsoft Teams, Zoom, Ring Centrals of the world. And then when you get into the sales and service side, the five nine and eight by eight and talk desks of the world. But as I said, the nice part about our business, you know, as I said, as we saw into these three

• 14:00Speaker 01

  segments, we also sell this one consolidated piece of software that can support these different users. So we might be deployed just on a sales organization or we might be deployed just within a service organization. But that gives us again, these kind of upsell opportunities to go and demonstrate the value of a unified communications platform, the power of AI, and then go talk to them about the internal communications that are there. But the vast majority of our customers, my long-winded answer is vast majority of us are deploying this across the organization. And then they're leveraging different features depending on, you know, the persona or the use case that they have.

• 14:35Nathan Labenz?

  Gotcha. Okay. So for the power users, how much of their time on a given day would you say is like in the dial pad UI versus tasks where they sort of have no choice but to leave the dial pad experience to go do something else?

• 14:56Speaker 01

  Service and support, they can live in that application. As I said, you know, if you think of a contact center agents probably sitting there waiting for, you know, taking inbound calls. We've got integrations with, you know, all sorts of the CRM's that they would go and have Zendesk, HubSpot, Salesforce to go pull information and provide them with the record information they would go that they would need or help them drive assistance from those conversations. But the intention is, you know, the way that we use dial pad internally is that's our internal communication platform and that's also what our sales and support teams are using. So I look here, I'm like, that app is loaded for me up in my browser window, front and center, you know, my entire workday.

• 15:38Speaker 01

  But as I said, there's always going to be these other interfaces that I need to go and engage with. But we want to make this the single destination to go and power the communications when you need to have a video meeting, personal phone call, you know, a business phone call, and then to go and pull the contextual information that you need from different integrations or workflows, whether that's again from from systems of records or from ticketing systems, whatever it might be.

• 16:01Nathan Labenz?

  Yeah, so it sounds like you're developing your own version of kind of the rag paradigm as well, like access into even broader knowledge bases than sales and support and AI question answering, all that kind of stuff I imagine is on your mind?

• 16:21Speaker 01

  It is. Yeah. I mean, yeah, you think about recruiting, right? So you think about opportu- kind of these tangential markets up and up where, you know, recruiting organizations or just recruiters. You and I need to have a con- maybe, you know, you're interviewing for a role and I need to make sure I'm asking the right topics and doing that a consistent manner. We can help guide that conversation. So there's everybody that's doing the interview is asking the same questions to the same person. But we can go capture those responses and then write them into Greenhouse, you know, for example, which would be a CRM or Assistant of Records for recruiting organizations or recruiters to go and leverage. But again, it's really about, hey, we want to go power those communications wherever they're happening and then starting to provide either context to the user

• 17:05Speaker 01

  or start to automate tasks and workflows that stem from those conversations.

• 17:10Nathan Labenz?

  Okay, cool. Let's change topic for a second, then switch back to more practical, you know, closer to the user, more product experiential things in a minute. So I saw that you just announced not too long ago the creation of this DialPad GPT large language model, which has been trained on, and I'm just reading from the blog post, five billion minutes of business calls and online interactions. The world's first business-focused LLM.

• 17:43Nathan Labenz?

  Five billion minutes is a lot of calls. I guess for starters, like, you know, I'm translating that in my head to 500 billion tokens. If I'm thinking just 100 words a minute of conversational speech, that becomes, you know, 5% of GPT-4 training scale. If I, you know, triangulating effectively here. So that's, that's pretty big. I guess, you know, a bunch of questions around that, like, where does all that data come from? Is that just stuff that's been recorded in the platform over time? And then, you know, how did you actually undertake the, the project of creating your own language model? I imagine, you know, potentially some partnership involved or?

• 18:25Speaker 01

  Yeah, so, so one thing is, so we had started a startup called Talk IQ. And that was one of the first real-time speech recognition engines back in 2016. And at that time, you know, give a little bit of context to, to, to the listeners is, at that time, if you were going to go do transcription, you would typically go capture, capture a conversation, you know, WAV file or an MP3 file, you could put it through a third party. Typically take you, you know, if your, if your conversation is 30 minutes, it can take you 30 minutes before you get an output back. Um, so there's this big delay. And we thought there was this really interesting opportunity, especially for sales and support conversations. Say like, look, if, if you can go capture and, and, and build a streaming engine and understand a conversation in real time, well, then I can start to route a conversation, you know, based on what Nathan might be asking about. I can start to map its sentiment in real time.

• 19:09Speaker 01

  I can start to guide a, a person. I can do live agenda tracking, like all these opportunities open up. So we got really, really excited about that opportunity. Um, so we built a real-time speech recognition. We started to leverage some open source software. I think, you know, a lot of people, when they say, hey, we're, we're building something from the ground up, you tend to start with, there's fantastic open source software out there. So we started with CallD, which was a model for speech recognition and build this fantastic, this really accurate, fine-tuned model to go and do long-form conversations. And I say long-form, meaning. When you do a phone conversation, that's very different than when you talk to Google in Siri and say, what's the weather or set a timer. And these were really complex,

• 19:51Speaker 01

  difficult challenges, just roughly a decade ago, which is kind of funny to think about. So we built this engine. And then when you're building a startup, what you need is distribution. And so DialPad at that time was, as I said, a really broad communications platform. Their founding team came out of Google. They had built Google Voice. And so it was a really kind of peanut butter and chili moment to use an analogy of, hey, we can go understand the conversations. You got the distribution of power in the conversations. And over those years, as I said, I've been here now for five and a half years. When people leverage our platform, they can opt in to show that data. And these are all long form commerce, business conversations or enterprise conversations. And the vast majority of them are sales and

• 20:32Speaker 01

  support related. And so that becomes this fantastic training set for us, again, that people opt into. When we leverage the data, we obviously anonymize it, strip it of any person identifiable information, et cetera, et cetera, et cetera. And when we started to then, you know, I think the world got enamored a year ago with kind of large language models, fortunately by ChatGPT. And so we were really quick at saying, look, this is suddenly this great opportunity. We got this training set. We go power these communications and we put this large language model on top of it, like all these opportunities open up. But these foundational models have some challenges with them. Capacity, you talked about token limits. So if you think about a transcript, initially, you would have to

• 21:15Speaker 01

  go break up a transcript six different times. And you know, if you even, that creates challenges even to start to do things like to do summaries. And so it became, you know, the challenges for these foundational models for us were capacity, the token limits, latency, you know, how quickly were we going to get an output back, cost. And those challenges were things that we didn't think we could wait for. They're all engineering problems that get resolved over time. And so what we did was actually focus on building our own large language model. And so there's two ways that we've approached that. One is, so you take fantastic open source software, you fine tune it. Fortunately for us, as I said, we've been at this for some time.

• 21:56Speaker 01

  We've got a bunch of experts and everybody from conversational neuroscience to linguistics. We do our own labeling and know how to build these models and they have the data set and they know how to do it. And then we're also working on building our own large language model from the ground up. And when I say large language model, it's ultimately a smaller language model. It's going to be built specifically for specific industries and use cases.

• 22:21Nathan Labenz?

  Hey, we'll continue our interview in a moment after a word from our sponsors. So you're doing this training in-house, like you're managing your own cloud? I would have guessed that you were partnered with like a mosaic or something like that to get over the, you know, the gnarly parts of the large scale pre-training.

• 22:39Speaker 01

  Yeah, we do everything in-house. We have our own GPU hardware, so we have got our own capacity for A100s. And the nice part for us is we do all the bare metal. So even our telephony platform, what's unique for us is because when then to tie this back to like the world of communications, a lot of people, when they think about building a communications platform today, think about Ultilia as a CPaaS provider. And there's APIs to go power, you know, voice and messaging. So why go build that yourself? I mean, for us, we think that the best businesses verticalize their stack. You can see that in in different industries, whether it's without, you know, now building your own silicon. You can see it in automotive industry with businesses like Tesla.

• 23:22Speaker 01

  But I think the biggest uniqueness that comes from owning your stack is obviously the pace of innovation and then cost advantages at scale. And those, those two things allow you to actually bring, I think, uniqueness and better markets or better products to the market faster than anybody else. And so again, I think. I don't know if we would make always the same decisions that we have in the past. If we were to start a startup today, but based on the proprietary data that we have, the people that we then know how that you have and the team of experts. We've done a lot of this in-house. That doesn't mean that we don't, you know, we do partner with with OpenAI to leverage the foundational models in certain aspects of our business.

• 24:03Speaker 01

  We also partner with Google, leveraging Google for text, which is one of our Iceland, which is one of their large language models today to power some features as well. So we do plug in some partners in different places, but the vast, vast, vast majority of things we do is all done in-house.

• 24:17Nathan Labenz?

  Yeah, I guess I'd love to hear, this is, I think, such a big challenge for so many, right, to know kind of what to build, what to buy, what to partner for in a world where things are moving extremely fast. And I think you, you know, I'm a little bit surprised by your answer. And I think it, you know, it's it's a flex because most, maybe not most, but certainly I think a lot of companies out there might say, oh, you know, let's, uh, llama two, you know, we're, let's save money or let's, you know, own our stack or combination of the two, whatever, let's go, we'll fine tune that, you know, we'll, we'll be, you know, we'll control our own destiny and whatever.

• 25:02Nathan Labenz?

  And then it's like, but if you don't have a real skill set and kind of a knack for it, right, and there's, there's multiple different critical components, I would say, to the skill set from, especially if you're doing large scale stuff from scratch, managing a cluster to kind of, you know, or OpenAI recently described it as the artisanal process of shaping language model behavior. And it seems like I would expect so many companies that may be great at what they do to try to add on this type of discipline and sort of get bogged down in it. And next thing you know, like OpenAI has kind of released their next version before you've made, you know, llama two really work.

• 25:46Nathan Labenz?

  So do you have, is it, is it about just the team that you guys had already assembled that you think is just differentiated? Or do you have strategies that you could recommend for how to avoid falling into that trap? Because I do think a lot of people are headed that direction if they try to follow your specific path.

• 26:04Speaker 01

  Yeah, and I, and I think that's why I say, you know, even here, I try to be really keen on this stuff is I think, you know, fortunately, because of the decisions we made in the past, and it's been kind of fun to see how this has played out. I don't always know if we would make the same decision, you know, starting from scratch. So some of that is, you know, we have a team of experts that's been in these fields for, working with these technologies for 10 plus years, uh, just at the forefront of building product. But a lot of the people on our team, you know, we've got, you know, over a team of 50 focused on AI, uh, 18 PhDs, we've got 16 patents, we show up at the best NLP conferences in the world and win awards for the models that we're building and developing.

• 26:46Speaker 01

  So some of it, I think, much like anything, it always comes back to the team of experts that you have and the access to the data set that you can go and leverage. So as I said, because of those two things of that decision, you know, six years ago for DialPad to be focused on AI then has played out nicely to position us today. That gives us, I think, some advantages over relying on a third-party API to go and do things. And, you know, when that, that, when ChatGPT, you know, was giving us some excitement, you know, two weeks ago, I can tell you, we were probably one of the few businesses that were, I say unconcerned, we were all, you know, following this, you know, because I thought it was just a really interesting business, that business thing that was going on. But we were probably one of the few businesses that was like, hey, this doesn't impact us.

• 27:31Speaker 01

  Like, we're not worried about whether our summarization is going to go down because the company may, may cease to exist overnight. Um, we don't rely on them to do that. Um, and so again, I think that always why, you know, as a, as a technologist, why it's always to me, I do believe in like, control your own destiny, innovation, like that's where you're going to really win or lose markets. Um, and don't, I say outsource, but you know, for lack of a better word, that, that's alluding me, but don't outs, don't outsource what you think are the, the, the most important parts of your business. And I think right now, I do believe, and I know it sounds a little bit generic, given the, you know, the, everything in the news and so far, but I do believe that AI is the biggest opportunity.

• 28:13Speaker 01

  And so, um, I think there is a better experience, more cost-effective experience is because we are a business that you can do by doing that yourselves, but that comes with a bunch of landmines and a bunch of challenges and it's not easy for sure.

• 28:27Nathan Labenz?

  Yeah. What do you think businesses should prioritize? Most. I mean, you kind of talked about a few different values there where there's almost always trade offs, right? You've got, on the one hand, what's the absolute best performance that I could get, you know, if I don't worry about latency or, you know, financial cost, then there's like, you know, if I put some constraints on those, then what's the best I can get? And then there's like, you know, now how much would I trade off that performance for, you know, further improvement in latency and cost? I think even that decision making structure is really tough for a lot of folks. It's, you know, it's pretty unfamiliar

• 29:08Nathan Labenz?

  territory, just given the new, you know, just traits of this technology. I mean, it's like, it's weird, right? It's, it's definitely slower than most software we're used to. It's more expensive, you know, on the margin than most software we're used to, probably in both of those by like orders or can be orders of magnitude. Obviously can do, you know, tremendously more stuff than most software we're used to. But do you have, you have kind of any coaching for how people should think about those relative dimensions of like, of the good in their AI product development?

• 29:42Speaker 01

  Yeah. And I can give you, you know, this is our matrix. I think obviously it depends on the business that you're in. So well, I'll start with, we have a matrix around that. Number one is latency. And what I mean by that is, we think the biggest opportunity for our business is how can we do things in real time for a conversation? What I mean by that is, you know, if we were going to go, we do things like agent assist, so we can like pull information from a knowledge base, present it to your user in real time in terms of like milliseconds.

• 30:10Speaker 01

  If that assist card didn't show up for three seconds, it loses its utility even a second later. And if it's 10 seconds, there's just no point in even building that feature. So everything when we talk about always starts with the most innovative features, the most interesting things we can do have to be as close to real time as possible. So that means latency really matters. Two is then, how do we get these features? If you can go do that in real time, how can you get them in the hands of as many people as possible? So that again, comes back to the question of like capacity and what's available from these large language models. And those are still challenges today for a lot of these models, which is like, at what scale can they operate, right? Or can you get them? And then the third is cost.

• 30:54Speaker 01

  And I think cost is one of those things that you can solve over time. If you're really providing value to a user, then you can always get the additional, the additional cost or price that you need to cover, kind of maintain your margins or improve your margins, whatever it might be. And margins tend to improve over time, as I say, you can always optimize that. So the cost piece tends to be, you know, third on the list, you know, there's other things, but to give you the way we think about it, like, cost is not the blocker for us. Like, we're, we've raised, you know, a half a billion dollars in venture from the best investors in the world, Andreessen Horowitz, Google Ventures, Iconic Capital, just to name a few. And we think there's just this unique moment in time to drive innovation. And so let's really focus on like, the latency piece, because that's going to provide the best experience and get it in the hands of as many people as possible and worry about

• 31:39Speaker 01

  the cost later. And if you're an earlier stage startup that is more capital constrained, then, you know, cost might be a number one of that list.

• 31:47Nathan Labenz?

  Yeah, interesting. So then how do you think about performance in terms of just like the quality of the insights or, you know, whatever the task is? Because I have mostly thought about, now, I haven't really done anything in this sort of highly real time way that, you know, and that I totally appreciate the rapid depreciation of a tip, you know, that once the moment is passed, it's passed. So totally get that. I haven't operated in that environment. But my intuition in most of the things that I've built has been try to maximize the quality of the AI output first. Subject to, you know, it does, even if it takes three minutes, you know, in terms of

• 32:31Nathan Labenz?

  latency, then that can, can start to be a deal breaker. You know, again, it depends on exactly how much better, you know, the best configuration of best model might be relative to my alternative. But do you get, do you think of it in terms of like? We want to get above some threshold of utility and then then we focus on all these other things or how does that kind of fit in?

• 32:50Speaker 01

  Yeah, so let me, yeah, so I'll reframe my answer of assuming that you are pleased with the output of accuracy. And so when we build models, there's two ways as ours, if you do come up with thresholds of what we think are going to provide quality output and utility. Two is we all dogfoed these these these features ourselves, meaning our support team goes and leverage and sales team leverages the models that we build along with every one of our employees. If we don't think it's it's providing utility is a good feature, then there's no way that we would expect anybody ever to go and pay us for it. So it starts there and then the other part is when, you know, and I think this gets skipped by businesses or kind of not thought about is the feedback loop for users, which is one labeling anything that's AI generated.

• 33:35Speaker 01

  And then two is providing that feedback loop for users to say, hey, this recommendation was good or bad, or this summary was good and bad, or the identification of this this action item was good or bad, whatever it might be. And those are things that I think are intrinsically really important for this and having that feedback loop to get back to the teams. We're fortunately always also on a biweekly release schedule, meeting every two weeks. So when we talk about tuning our models and making adjustments, we are constantly looking at that data, going back, redeploying models. And again, there's that constant feedback loop to go and make sure that there's both utility and function in it. And I think that has to be, obviously, you know, as I said, if I was to reframe my answer,

• 34:18Speaker 01

  like, hey, assume that you're doing the right things and like building an accurate model that provides utility, then we're focused on like these parts around latency, you know, capacity and then cost. Hey, we'll continue our interview in a moment after a word from our sponsors.

• 34:32Nathan Labenz?

  Yeah, it's definitely more art than science still at this point. And, you know, I think we're kind of in this this moment where, again, the dynamics are just changing so quickly. I've noticed in my own fine tuning work, which I've mostly done on the OpenAI platform. And the reasons for that are interestingly also starting to change. Like, it used to be because I couldn't match the performance anywhere else. And it was kind of simple as that. Now, I would say it's pretty reasonable to expect that I could match a 3.5 fine tuned for my use case, which this is at Waymark, you know, we do. Essentially video script writing and related tasks.

• 35:12Nathan Labenz?

  And I think I could probably match the 3.5 fine tuning with not like a huge amount of effort on top of a, you know, mix stroll or whatever the case may be. But now the better reason for me to stay with the open AI is that they're like pricing actually works out more favorably for us and the hosting is just much simpler, like the managed hosting relative to having to, you know, work with other platforms to provision dedicated instances and so on. You know, and some of those like really don't have much in the way of auto scaling yet at all. Now it's like on that other dimension that that they're kind of ahead.

• 35:50Speaker 01

  And you bring up like auto scale, like these are all things that we have to go through, right? And again, communications platform, there's different demands, sorry to interject, and there are different demands, right? You can imagine like a meetings. Well, guess when meetings pop up? Top of the outer, top of the hour, bottom of the hour. Right? Those are the two spikes. So even just having auto scaling on a communications platform for all of these things that you know, most people are not starting their meeting, you know, 1117 on those pieces. But these are all, you know, as I said, it's important to have this stack and to have these controls and capabilities, you know, where, as I said, many of us are former Googlers. And so we're obviously at Google Clouds and these are all of the things that come into play of auto scaling and doing that scale for sure.

• 36:32Nathan Labenz?

  Cool. Okay. So let's go back to kind of product and, you know, practical user experiences a little bit. You want to just run down the kind of most common, maybe most loved AI features and experiences that you are powering today and, you know, maybe we can kind of start to get into some that are emerging as well?

• 36:52Speaker 01

  Yeah, most loved, obviously real time transcription. We do a summary on top of those conversations, which I think the world is suddenly becoming enamored with, you know, instantly summarized blocks of text. But we do some cool, really cool things with summarized text. And so you imagine a supporter in sales conversation, we can go label the purpose and the outcome of those conversations. So suddenly you can take unstructured data, made it structured. Previously, people would go and have to label it, literally people in their contact center go review calls, you know, tag this stuff. The agent assist features, as I said, which is like, hey, my first job was in a contact center and I was terrified of the first conversation I was going to get because I was like, what is this person going to ask me? And so again, being able to guide people and pull information and present it to them in the moment that they need it.

• 37:36Speaker 01

  And then the last one I would highlight are things that are, I think, the perfect application of AI, which is we can infer customer satisfaction from conversations. And you think of what happens for, you know, in the support world is you and I have this conversation. You know, at the end of it, you send me a survey and you're like, hey, Dan, what was your experience? And I'll tell you, I'm like, Nathan, either it was really good or really bad. Like those are the only, you know, the two people you kind of hear from. And so for us, it was like, hey, look, we can go infer customer satisfaction with really high accuracy. We can do it across every conversation. There's no change in behavior and there's no new software that needs to happen. And you get, you know, 100x volume of the data that you're getting. So it's much more representative.

• 38:19Speaker 01

  And so these are just a few of the things, as I said, that we're working on. But we're really focused on how do we go capture these conversations? And then, as I said at the beginning of our chat, really focused on delivering features that are focused on assistance, automation, and then telling you things that are happening to help you make better business decisions.

• 38:39Nathan Labenz?

  Of the stuff that the AI does today, how much of it would you say in the absence of AI, gets done by a human versus simply doesn't get done at all? Like presumably, if you, if there was no AI transcription, very few of these calls would be transcribed, right? And but, but maybe many would be summarized, you know? How, how do you kind of think about how much is this doing work that otherwise wouldn't have happened versus, you know, taking work off of humans' plates?

• 39:10Speaker 01

  Yeah, I think about it in two ways. One is what happens is you get into like the crazy world of quality assurance. And they say crazy world and that you see me smiling and laughing as they say, in like the nicest way possible. But it is what happened before even transcriptions was people are listening to, sitting down. You know, if you're in a contact center, sitting next to Nathan, listening to the call and then making sure that you're following the process and you're handling things in a friendly manner. So one is like, the stuff's even happening. It's either happening today and there's not even a record of it. Or two, it's happening and it's really, really, really time intensive. Meaning somebody's probably not even providing the transcript or taking the notes.

• 39:52Speaker 01

  But they're saying, hey, here's the scorecard and the rubric and I'm giving Nathan a grade to say whether he's doing a good or bad job. Or you get into like the legal world, right? Or somebody may want a record of a conversation, whatever it might be, or a doctor's office. Right? Somebody's probably sitting there, maybe taking notes or the doctor's taking notes. So I think honestly, our application of the things that we're focused on are these mundane tasks that I think are really ripe for AI to actually do it. Transcription accuracy is near human levels. As I said, it's kind of everyone's being a lot of businesses are in the same realm of kind of 90, 90% plus accuracy. And then you get into this land of, hey, we've got, we can go summarize that information and structure it in really fantastic ways.

• 40:37Speaker 01

  And I think there are immense opportunities to either go and completely automate tasks or augment the tasks in really fantastic ways to free up people.

• 40:47Nathan Labenz?

  Just as an aside on the transcription, are you still using the original stack that you created some years ago there? That's all been whisperified at this point?

• 40:57Speaker 01

  That is, but yeah, so we do, so all of that still we do in-house for, and you know, I'll share a little context on that. We're on Nvidia Nemo Toolkit is our latest models. And again, when you talk about telepathy, there's different codex. And so everything is fine tuned for the codex that we go and leverage and use. And so really high create transcription accuracy. And when you're talking about, you know, driving automation assistance and insights, you keep hearing me say those three things and grain it and people. Transcription accuracy becomes the foundation of all of that. And you know, you've probably, you know, I've had, I listened to some of the other podcasts and people will talk about kind of garbage in, garbage out on data. But there's always that argument that comes up from investors or analysts around look like,

• 41:41Speaker 01

  isn't speech recognition in A R S R, isn't that just a commoditized, you know, commoditized technology? And I'm like, I don't think so. I think we're a long way from solving that problem when you get into accents and words that don't show up in the dictionary and proximity to microphone, all sorts of, all sorts of complex challenges around it. But that to me is the foundation for all of this, that if you can't accurately transcribe information, then everything else goes out the window. And again, that's why we recognize that and think it's important to go and have an in-house speech recognition team and build those models and fine tune those models again for our network of telephony.

• 42:19Nathan Labenz?

  Yeah, interesting. The codec aspect to that is it's always a lot of little nuances and once you really get into the weeds on these things.

• 42:27Speaker 01

  Yeah, people will come to you and say, hey, that you know, they'll see a bench, you know, much like anything and benchmarkings always have caveats and nuance too. And I always encourage people to, to, to understand that, which is you may see a benchmarking report of a business that does speech recognition and says, you know, they have the highest accuracy. And it all comes down to, well, what training set was that? And is what network was that on? There's all these things that matter around the codecs. And so for us, as I said, is we have the best accuracy on our codecs and telephony network. So there's nuance and that's not us saying, you know, we're not a provider of speech recognition for somebody to do as an API. So somebody can't go take our models and plug them in on the T-Mobile or Verizon network

• 43:10Speaker 01

  or ATV, you know, whatever it might be. And that matters. And I think it's important for people to understand the context of these decisions and the context of these benchmarks.

• 43:21Nathan Labenz?

  Yeah, interesting. How much do you personalize? I guess you could, you could customize to the level of the customer or you could personalize down to the level of like the customer's customer. And I wonder how much of those two levels of, and I guess you could even, you know, do it in multiple ways, right? This could be. You know, context management at the prompt level of, you know, here's a profile of this business or a profile of the customer. You could go into a rag setup. You can go into fine tuning. How do you think about, you know, I guess I'm just thinking these tips, right? Like tip pops up. Obviously, that's got to be specific. Certainly got to be specific to the business, right? It can't just be like general, you know, try to smile more. Maybe it even needs to be often

• 44:05Nathan Labenz?

  specific to the individual and customer too. So yeah, personalization tells everything.

• 44:13Speaker 01

  We start with custom models for every, so custom models for every customer. And so that is the ability for them to go and fine tune the speech recognition model that they would have. And I always go back to, look, there's going to be us in startup in SaaS land. We tend to come up with acronyms that are new. We come up with funky spellings of product names to be unique. So you, those tend to be the most important things to a business, right? Are the uniqueness of those products or those acronyms. And so again, it comes back to you need to accurately transcribe them. Where we want to get to, we would like to get down to, hey, Nathan, if we're providing, if you're, if your business is a customer of DialPad,

• 44:54Speaker 01

  we would like to provide Nathan his own model for himself. And I always go back to, you think of somebody that has a name like Sarah, and there's multiple ways to go and, and, and spell that. You want to piss somebody off really well is show them a transcription or a summarization and summarize their name wrong every single time you see it. And so those are things that we're working on. We don't have that problem solved today. But again, these are all the challenges that start to pop up at scale. And you bring up two interesting points is, well, if you're a customer, can we then go and build another model for the customers that we're leveraging your platform for? Or can I start to have a unique model for the individual employees that might be, you know, leveraging, leveraging the piece of software? So those are all still

• 45:37Speaker 01

  opportunities for us that we haven't solved. I don't think anyone has, you know, solved it at those levels. But when we think about like, where do we want to get to? And what are the annoyances that show up in this stuff? Those are the very real annoyances that show up. Just thinking about transcribing somebody's name.

• 45:52Nathan Labenz?

  Yeah. If you're doing this at home, one very practical tip that I have, which I don't think would work at your scale, but works for me when I'm just. Processing podcast transcripts, for example, is to take a raw transcript and then run it through a language model to kind of clean it. And at the top, I'll just be like, here are all the proper nouns that you need to know to clean this up so that you get the company names right and all that sort of thing. Again, that works at low scale. I don't mind dropping, it could be like a dollar on Claude or whatever to do that. Not going to work for you at the scale of all your customers, you know, many, many calls. But how much data do they need to do this sort of customization? Like, is

• 46:37Nathan Labenz?

  there, is there a rule of thumb of you need X hours of speech? And I assume that they need ground truth as well of the, you know, the correct transcript of those inputs?

• 46:49Speaker 01

  Yeah, the hours, I don't have the number of hours. I'll ping my co-founder and I'll get you the hours. But there is a limit to say, look, and this always comes up for customers, you know, to put it in relevant aspects of people. You know, we have to go, we have customers all around the world. And so somebody will say, hey, we want to go have transcription in a language we don't have today. And so then we need to go and find the audio from that and make sure that there's enough to go and do it. And to your point on the ground truth, this all stemmed from our first startup was when we were building Talk IQ, we had to go build our own telephony platform. We had to go build our own speech recognition. We had to go build our own NLP and we had to go build the tools at that time to even do it, which was, how do we go capture the audio,

• 47:35Speaker 01

  literally start to create kind of 15 second clips of the audio and then build these tools that our labelers could go through to say, okay, you need to start by accurately listening to this 15 second clip. I need the ground truth, which is go and transcribe it. Somebody has to make sure then that is accurate because it's a tedious task at scale. And then we have to go, as I said, is take those and then go and build the models and improve the models from it. So these are all of the very real challenges that, as I said, you know, because we have this past experience. Fortunately, kind of went through or faced with those problems back then. And as I said, much of that has been solved today.

• 48:18Speaker 01

  And you know, different startups have showed up to go even build tools to do it.

• 48:22Nathan Labenz?

  Yeah, I'm struck as I'm listening to that description that it seems like it's probably gotten an order of magnitude more efficient even just to set that up, right? Because now, you know, there's the Facebook model, for example, that supports like a thousand languages or something. So it may not be up to your standard on language 900, but it probably saves 90% of the time of, you know, having to go hire somebody on Upwork to get started.

• 48:46Speaker 01

  It does, and we do, and you do kind of hacks around it, right? Which is, so you know, you showed up today and you may say, hey, maybe you want to have Dutch. And we would say, oh, we don't have Dutch today. So what we could go and do is we could go plug in, you know, Google File Speech, we could go plug in Lex, SpeechMatics, you know, Deepgram, like Whisper, you knew it. And you assume if like, again, goes back to like where we started the conversation, he made sure like, who's going to get you the best accuracy on the codec that you have. So we would go and do that. And then that becomes training data for us, right? So suddenly as people just use the platform and they opt in the data, then that helps us over time go and say, we can then go and do this ourselves and reduce costs and improve accuracy faster than a third party.

• 49:31Speaker 01

  So those are all like the very real things that we go through. And there are times we would take that approach and there were times that we would potentially plug in a third party and just let it be. And that usually stems from, you know, thinking about the opportunities from the business, which is like, hey, this language might not be in high demand. So we never, you know, we're okay with the pass through costs of just using a third party for it first sight. Hey, this is, you know, there are very real opportunities for us at scale to go and do this.

• 49:58Nathan Labenz?

  How about on the content generation side? In my limited free trial, you know, I didn't have enough content in the platform to kind of really see how that can work. But obviously this is another thing that has to be customized, you know, to really be useful. And I've done some experiments trying to get AI to write as me. Haven't quite got out of the uncanny valley yet, I'm sorry to say. How are you guys doing in terms of generating content? How do you think about measuring how well you're doing? And you know, that I'll that I'll project into the future, I guess, from there.

• 50:34Speaker 01

  Yeah, we're we're early on that. So as I said, like, kind of when we think about like content generation as a communications platform, we think the easiest thing is like reply, you know, generating a reply. You and I have a conversation and one thing that we've kind of noodled on is, hey, can I go send me in the summary? And if I send you the summary, then I want to have an email that accompanies it. And can we go auto draft the email based on, again, the context of the conversation we just had? I tend to have like similar opinions to you of this this valley that you're in where I'm like, I think there's some utility there, but is it more trouble than it's worth? I use chat GPT for go write me first drafts of blog posts, you know, as a starting point for things.

• 51:18Speaker 01

  So I do think there's utility and some some some opportunities there when we get into content generation. We're still early in exploring that, but I also kind of have this uncanny thing of like, I don't know, for things like email responses, you know, for example, is that, is it really worth the time? I mean, is it really going to provide a ton of value for people? Are they going to end up just customizing it themselves and writing it? As you start to engage with a large language model, you suddenly realize that you've spent as much time as you would just drafting just drafting a two sentence, you know, and sending it.

• 51:50Nathan Labenz?

  That ultimately comes down to just how good the generations are, though, right? Like, it's, um, this is a, this seems like a very dynamic situation. Like, I have not got it to work on, I guess, I haven't even really tried it with GPT 3.5 as much as I probably could have. I've been kind of holding off to try fine tuning GPT 4 on my own writing to see how close I can get it. You know, presumably, if you could fine tune GPT 4 on a decent body of business correspondence for a business, you would probably, you know, you may not be writing, you know, love letters to partners or, you know, taking over corporate strategy, but presumably you could like,

• 52:34Nathan Labenz?

  get pretty high acceptance rates on like routine communication, I would think, right?

• 52:39Speaker 01

  I think you're probably right. I don't, you see my hesitancy. I don't know if I'm like bought into that, like meeting like the the general utility, like. I think there's a ways to, I think there we're still a ways off from this stuff. And I, and I generally think, and I, and I say this, meaning I think there are gen, there are very real opportunities for large language models. And I think what will still play out this, this next year is I still think we have a ways to go. Um, and I think they're going to do some really, some things really broadly, really well. And we're still going to need the next level of development to start doing things that are really going to seamlessly work and be like the next wow factor shows up.

• 53:19Speaker 01

  And I just think like, this is one of those examples where I'm like, I do a lot of, as I said, like content generation today. And I'm always like, it's a great first draft. It's a great starting point. And I think about putting those features in our, in our app and potentially charging for them. And I'm like, I don't know if that's, that's the, like the wow factor feature that we should go and. I think that it's like a quality of life feature to say like, hey, end to call. We can go generate this amazing recap and structure it. And I'm like, do I need to have the auto-generated email to go along with it? Like it's a couple of sentences and is it really going to do it that well? And is that where we should be spending, you know, our product and engineering time? And there are different competitors in the space, like Teams is very much focused on that with Copilot.

• 54:04Speaker 01

  You know, Zoom does that a little bit with like AI companion. And I'm just like, I don't know if I'm, I don't know if I'm personally convinced on that use case yet. And especially one that you might potentially charge. I just think there's some extra development that needs to happen. And to your point, that might be like, hey, there's a bunch of fine tuning that you individually need to then go and do. And then you think about, well, how do you do that at scale for a user? And ones that that user doesn't know, these are just the things that pop up. Sounds great, but you're like, okay, so then you want every user to hop into your app and suddenly do fine tuning to customize the emails.

• 54:35Nathan Labenz?

  Well, actually that's one of the big reasons I thought this conversation was so interesting is because in looking at Dialpad, I've been kind of thinking, this feels kind of like the gym that is both capturing the data of what people do.

• 54:52Nathan Labenz?

  Incompletely, and maybe there's a little caveat we can discuss there, but definitely capturing a lot of what people do, especially in these sales and marketing roles that are the most intensive users. And then also, the dialpad environment seems like a very natural place in which to deploy and evaluate and ultimately refine an AI agent, because you've got the possibility of what are the actions are much more clearly defined than if you're just like, oh, go use a computer or go use the web at large. I guess, there's a bunch of different questions here,

• 55:32Nathan Labenz?

  but if it's not there yet, or if the next generation of model quality is not going to take us to, even, I mean, you're saying like, maybe not even on email drafting, but I'm thinking even a little farther than that, like, virtual AI employees that may start limited, but can hopefully get in there and actually advance your work for you. What do you think is missing that would stand in the way of that happening in 2024?

• 56:00Speaker 01

  Well, I think, and I'll get to take a small little tangent and tie things back. The content generation piece to me, the part that excites me, and I think is the higher utility on that, is like, understanding the recommended next best step or automated workflow that somebody should take. And what I mean by that is because you tied into, hey, we can go understand these conversations, and we're tied into a CRM and a database, so we know the outcome, which is if I'm using specific language in a conversation, and that language leads to a more positive outcome for the business, then the next time we see that conversation happening for a different person, we should highlight to them, hey, here's the right course of action that obviously takes it.

• 56:45Speaker 01

  So when I think of the content generation, it's more around generating content around, here's the next best action, as opposed to me thinking about content generation of like, here's the email, right, to go and write.

• 56:59Nathan Labenz?

  Okay, well, let me keep pushing you to think a little farther into the future. I feel like... If I imagine AI realizing its potential, a lot of sales and customer service ultimately ends up getting mediated by AI. And it's not that I'm like racing to replace people or, you know, cut costs or whatever, but I always kind of come back to sort of a Bezos style, like, what does the customer really want, right? And the customer wants like immediate response 24/7, like where I can pause the conversation where I want to at my convenience and be able to come back and pick it up, you know, right where I left

• 57:41Nathan Labenz?

  off and maybe even switch modalities. And, you know, ChatGPT offers me all these things today. And it's like, you know, I had a conversation while driving this weekend about the new states-based model moment that's happening. And I would say ChatGPT was, you know, maybe not the best conversation partner in my life, you know, possibly for that conversation, but certainly among them. And, you know, for many people who, you know, aren't as connected as maybe I am, like, probably is the best conversation partner that they could have. So do you think that I'm like, wrong about that? Or, you know, going back to the, you know, the stats at the beginning, you have like, third, third, third, these small and medium sized businesses, enterprises maybe, maybe don't have to go this way, but

• 58:26Nathan Labenz?

  like a small business just can't staff the phones 24/7, right? And can't like, pick up all the calls that are, you try calling the local restaurant around here at a lunch rush and it's like, they don't want to even answer, right? Just you just have to walk in and wait in line. Seems like there is like a qualitative change that can happen here. And, you know, obviously it comes with a lot of potential disruption. I don't know. It seems like you're not, you're not buying that vision. I'm not sure if it's because you think that technology's not gonna pan out, or you just don't want to go there.

• 58:56Speaker 01

  Automating support, like doing digital deflection. I think like, like first generation chatbots have been like really good password reset bots. Obviously, you can go put a large language model on your chatbot and it's going to be able to handle much more complex requests and sound much more natural. Fully believe in that for support. And I would say, you know, you can go look, I think, you know, today, probably 80% of people probably start engaging with a brand for support on a digital channel. So the very first thing they want to do is they want to go up this next generation experience with a large language model. Go get better answers, more natural sounding answers as quickly as possible. For sure, we're working on that. We believe, I believe wholeheartedly in the opportunity.

• 59:36Speaker 01

  This automated sales one, I don't, I don't, and when I say that, meaning like outbound sales and creating an outback, because there's some startups working on this one. I don't know about that. And we had, a couple of us had dinner with Mark and Drayson the other week, and Mark had mentioned that he has this idea at times that might be two bots to chat with each other. It's like the sales bot and the, you know, each of us have, Nathan has his own automated bot of him and understands, and I have my own that's selling. I don't know if I agree with that. Like, I personally think people like to buy from people. I also think when you think of like a support use case, that yeah, I want to engage with a digital bot, but the second I need to escalate that to a phone, I don't know if I want to talk

• 1:00:20Speaker 01

  to an automated voice bot. As, as, as good as that bot might be, I think there is, when you're really frustrated about something, talking to a human and then you get into all sorts of things. Like, do you let somebody know whether that's a voice bot? If it's so good and you've got speech synthesis and everything? And I don't know. And I, and I do think that it's going to take time. As I said, when I go back to the time thing, like, I don't think that stuff shows up in 2024. And I know, I've seen some really impressive demos. And at the end of the day, you're like, you can still tell. And so that's why I said, I don't think in the next year, you suddenly have an experience where we're like, you can't tell. Maybe on a digital channel for sure, but not on a voice channel. And I'm happy, as I said, I'm, if somebody's got a really cool demo out there, like, I would love to

• 1:01:04Speaker 01

  see it and play around and like, for sure. But those are some things that, that we do, like, we spend some time thinking about as a communication platform of like, hey, does this automated sales agent show up? And does somebody want that? But I very much view it as like, hey, the next generation chatbot on a digital channel, for sure. Instant answers, better answers, faster answers. All of that stuff is like an immediate big opportunity for us.

• 1:01:29Nathan Labenz?

  So what is your kind of 2025 and beyond expectation? Like, mine is pretty much that all this stuff does happen. I'm not as sure if we get like AI automated science, although even that is looking like increasingly likely, but definitely sort of AI mediated sales and support seems quite likely to me. Yes, I would agree like they are still kind of uncanny valley, even the chat GPT voices. You know, they don't handle interruptions very well. They don't handle my long pauses very well. And so they start talking when I'm still trying to talk. I was looking back at the transcript with chat GPT. And I'm like, stop interrupting me. I've known for my long pauses for God's sake.

• 1:02:15Nathan Labenz?

  So, you know, they're not quite there. I would definitely agree with that. I'm also a big supporter of the notion that I don't think we should have AIs deceiving people. So I don't even think we, even if it does sound, you know, fully human and could pass. I think we should probably, you know, either as responsible corporate, you know, actors, or as a government sort of say, you know, we're going to just be clear when you're talking to AI. That seems healthy, but I don't know, you know, if it's good enough, I don't really think people will mind. And it does feel like the ability to respond immediately. And again, you know, drop, you know, put the phone down for 10 hours and then come back and pick it up later after my kids go to bed when it's convenient for me like, and seamlessly do that.

• 1:02:58Nathan Labenz?

  You know, that seems so helpful, not to mention, you know, the ability to pass the cost on to the customer. Like, I guess you could think that, you know, even, you know, 20, pick your year like, and I'm not saying like I care exactly what your specific year is, but do you think it's so far off that it's just like not really something that we need to be concerned with right now? Because if it's, you know, if it's not 2024, but it is 2025, like it's still very close.

• 1:03:25Speaker 01

  The reason I have to express that hesitancy, and I, and I think honestly, like we don't deal with, there's going to be, you know, much more smarter and greater people that that will tell me it's faster and I'm, and I'm wrong. My experience with technologies has been similar to that trough of disillusionment. And I think, you know, I look at self-driving cars as a great example of this. We're now a decade into this journey. If you asked me a decade ago, based on whatever people have said, we would have figured this was a solved problem. And I think these things turn out, when you really get into like the complexity of human language and the workflow, especially work and some of the complex workflows that we have every day, those turn out to be much more complex, harder problems in the real world.

• 1:04:06Speaker 01

  And that doesn't mean that you can't be really enamored and amazed by a demo, or that there's not all of these little things that we do every day, mundane tasks that are ripe for audit, complete automation or experiences. But as an aggregate, I think it just is going to take more time. And I don't have, I don't have a guess on time. If somebody said that with the, the, the only thing I, I, I feel is when I think about a year. And I'm like, look, I've been blown away by large language models and chat GPT. And even just, you know, even just technology in Joma over the past decade. I just have this inkling and this hesitancy that, that it's going to take more time.

• 1:04:47Speaker 01

  And I hope I'm wrong, uh, because as I said, as somebody that grew up in Silicon Valley and a technologist, as somebody that just loves innovation, I hope I'm wrong. But I have this sinking feeling, unfortunately, that it's going to be harder and more complex than we think.

• 1:05:04Nathan Labenz?

  That's really interesting because I also kind of hope I'm wrong. But in the other direction, I, I feel like I see this coming at us so fast.

• 1:05:12Speaker 01

  What have you seen that you're like, this is it?

• 1:05:16Nathan Labenz?

  Well, just, you know, broadly extrapolating the last two years, you know, just going back to literally two years ago today. And we're recording on December 19th, 2023. So December 2021, there still wasn't an instruction following model in the public. The state of the art for me at Waymark was fine tuning GPT three. And I think we had just gotten, we had moved from query scale to Da Vinci scale. And that was like enough to create our first script writer that was, you know, like, okay, we, this will definitely be useful to our customers. Our use case is quite narrow compared to, you know, certainly the dial paid use case.

• 1:06:00Nathan Labenz?

  And, you know, certainly the broader agent challenge globally. But we have gone from couldn't get actual utility from it in late 2021 to it is a totally qualitatively different experience two years later. And that includes the script writing having gotten very good. The image understanding having also progressed by leaps and bounds such that we're now able to choose the appropriate image out of a user's image library with, you know, pretty high consistency. The, the text to speech has also gotten dramatically better.

• 1:06:40Nathan Labenz?

  Again, it was not even two years ago, but probably one year ago that we went from, you know, hey, this just doesn't sound good. We, we partnered with media companies. So, and we do also sell direct to small business, but our kind of standard is would a media company that like owns TV stations or owns a cable network, would they put this on their air? And, you know, feel like that is the kind of thing that isn't going to, you know, make people turn their TVs off, right? So, and it wasn't, you know, a year ago it wasn't. And now most of the stuff that we put on the air is AI voice. And it's not perfect, but you know, all of these things have kind of gone from 20, 30%

• 1:07:23Nathan Labenz?

  of human to like 80, 90% of human in just this less than two year timeframe. And I don't think we're done, you know, we're, I look at our model and I'm like, I see actually a lot of low hanging fruit still where some of the techniques that we've developed for past generations were, are now like hurting us, you know, we've, we, we've, you know, tried to fit so much into so few tokens and whatever in the past. Now it's like, you know what we should really do is go back and look at that again, cause like the tokens are a lot cheaper. The models can handle a lot more of them.

• 1:07:58Speaker 01

  To me, it's like, it's that, it's always like that last mile and the complexity of that last mile. So you said like, I totally, and I go back to like the bad analogy of like the, the self driving cars, cause I live in San Francisco, so we, uh, you know, Waymo cruising around all the time, you know, and doing, doing a good job. Uh, but that's not without issue. And so I, I think it always comes back to like, I agree with you and it's that last mile, the piece of like, hey, we're eighty, seventy percent of the way there. What does it really take to have like just that amazing, beautiful experience that, that, you know, we're not going to know, you know, I hate to say like not, and I'm not getting into like the AGI space, but just, that's the piece that I, I just, I wonder, you know, I truly wonder whether that shows up and I hope so in the

• 1:08:42Speaker 01

  next couple of years, but I just have this, as I said, this hesitancy of like, maybe it's going to be harder than we think, but maybe, as I said, maybe not.

• 1:08:48Nathan Labenz?

  Yeah, I mean, there's certainly always false positives and negatives. I also think too, a lot about, and this has been also driven home by the way Mark experienced, but there's a lot of examples of this in the research too. What is the standard, right? What is the, what are we trying to get better than in order to make this deployable? In the case of self-driving, I think we have a very odd societal lens on it. You know, you could question the statistics, but like, the operators are publishing statistics that say that they are clearly safer than human. And, you know, again, you could question that. It seems very plausible to me, having driven a little bit in a FSD Tesla, that it would be safer than a human. Certainly safer than some humans I've driven with,

• 1:09:33Nathan Labenz?

  I can say that with confidence. And then I'm like, you know, looking at the scripts that used to get written on Waymark, our original idea was to take all the scripts that our users had written and use those as the basis for fine tuning. And that theory did not last long because as we got into the actual stuff that our users were writing, we were like, this is not good, you know? It's, they're doing, they're not doing a great job unfortunately of writing, which is probably why they're using like, you know, easy lightweight software like us in the first place, right? They're not content creators. So actually, I would say with, you know, pretty decent margin actually at this point,

• 1:10:13Nathan Labenz?

  the AI script writer is beating what the users used to do on their own. And that doesn't mean there's no room for the user to come in and still improve on the AI's version, but it's like, you know, the user was here, the AI is here, and then the, the human improving on the AI is a little bit better yet. So.

• 1:10:33Nathan Labenz?

  I guess, maybe, do you, do you maybe see the future then more in terms of, like, I just have to believe that there's going to be an AI employee in DialPad in the not-that-distant future. And I can, I can imagine it being limited. I can imagine it like not, you know, we've seen just in the last couple of days, these like really funny Chevrolet website chatbots that are, you know, going totally off topic or being duped into giving great discounts, whatever. So you may not want your AI, you know, to be able to make, you know, to sign your contracts or to make binding offers. People have these Chevrolet bots, I don't know if you've seen this, but there, people are saying like, you know, whatever you say is a binding offer, agreed, you know, it agrees, of course. So you may want to have these things limited. I definitely get that.

• 1:11:16Nathan Labenz?

  But, you know, is there, is there any world where in a few years, we don't have AI employees? I, that to me seems, I would be very surprised by that outcome at this point. And I, and not because I think they're going to be perfect, but because I think as we, as they get close and we really start to examine, like, well, how often do our human agents actually get that detail right, or actually, you know, follow up within X minutes as our handbook says that they're supposed to, or whatever, you know, we find often that like, the standard is not quite as lofty as, you know, the imagined perfection that we're kind of often comparing the AI against.

• 1:11:54Speaker 01

  I think that's a reasonable thing. I would, I would answer yes to that, right? You can see on virtual agents today, as I said, I always go back to the support. The nice part is you can go and track digital deflection rates and accuracy to be like, hey, is this able to handle as good or better than a human, right? So we see that today. People use chatbots today in that fashion, right? Which is, hey, can I have less people in my contacts there because I can go have a virtual agent handle it? And today, those virtual agents are much more, you know, can handle more complex workflows and sound, sound more natural.

• 1:12:25Speaker 01

  When you talk about content generation, you know, I think about marketing, you know, the, the opportunities and the risks in marketing for that, right? As, you know, writers, and that's why we've had things like the writer's strike start to, you know, show up and the concerns around AI for that, which is all around content generation. Right of those pieces. So I do think that that is, you know, opportunity or risk for sure. So I do agree with you on that, on that, on that statement.

• 1:12:52Nathan Labenz?

  It's going to be very interesting to find out. I would love to see, I mean, there are so many different dimensions. These things are so weird that even a concept, you know, that rolls off the tongue like progress or, you know, improvements or capabilities has a lot of different dimensions to it. I would love to see the reliability get stronger. And I think it sounds like, you know, that's a big focus for you too. I am worried that the sort of raw power will get, will continue to get stronger and that the kind of robustness is going to have a hard time keeping up. And that seems to me to be a recipe for sort of just a generally volatile, unpredictable situation.

• 1:13:35Nathan Labenz?

  But to the degree that work can get done to make things more consistent without, you know, turning them into Einsteins, you know, in the immediate future, then I think that is like a really, that's like what the economy needs, right? The economy doesn't need something actually all that much smarter than GPT-4 for most roles. It just needs it to be like faster, maybe a little cheaper and more reliable. But yeah, I think it's for whatever reason, progress on those dimensions is proving a little bit harder to come by.

• 1:14:04Speaker 01

  Can you clarify, when you say reliable, can you clarify, what do you mean?

• 1:14:07Nathan Labenz?

  Yeah, I mean, there's, again, even that has kind of different dimensions to it. But I've been thinking a lot about this state space model moment recently. And, you know, I think that one of the key limitations of the current transformer-based language models is they're purely episodic. The, the episodes are getting longer as the context windows get longer, but they have no mechanism to carry anything forward from one episode to the next, right? So it's like, they always have total amnesia. That kind of gap between, you know, they obviously have these, they have like massive knowledge encoded in the weights and then they have kind of a little context and kind of nothing in between.

• 1:14:50Nathan Labenz?

  But now we have like rag and stuff in between, but you know, kind of hacky stuff in between. Something, an integrated memory to me feels like maybe the way that we get to this higher level of reliability. And again, I think there's a lot of ways to measure or kind of conceive of reliability, but to some degree, it's like predictability, legibility, consistency, being told once that you did the wrong thing and then like actually listening to that and doing it right again in the future. And those are the things that I think the the transformers kind of fall down on today where they aren't, you know, and that prevents them from being like reliable coworkers.

• 1:15:32Speaker 01

  That makes sense.

• 1:15:33Nathan Labenz?

  Any thoughts on memory? This is like a real kick of mine at the moment. And I'm gonna have an insanely long monologue episode coming out about it. But is that an area that you are thinking about as well?

• 1:15:48Speaker 01

  Yeah, I mean, to your said, like the context window and being, you think of like applications for customers, right? Which is, it all stems from being able to personalize the content, right? Or that experience. And it all stems from, yes, being able to pass information from, as I said, you classify it as like multiple episodes, but multiple conversations in the past. And today, the way that you handle that is you are pulling information from a system of record and trying to provide that context to it. It doesn't have the full context because again, old records might be partially intact. They might not even exist. Nobody has them. And I think to me, that's again, comes back to like our biggest opportunity of what I think really excites me about the platform of really any

• 1:16:30Speaker 01

  communications platform is you can suddenly have a record for every conversation, assuming you want to record, you know, I'll, I'll put the the preface out there of like, assuming you want to have a record of it. Great. If that, if that is true, then suddenly you have this really amazing opportunities that open up of saying, suddenly I can go personalize the content in the moment. Suddenly I have this understanding of the past seven conversations that we had. And that really then opens up the opportunity for these virtual agents or virtual sales assistants, you know, however you want to kind of frame it. So I think the lot figuring out this next step of like the long-term memory. That to me is like something that needs to happen again. Now, I have this hesitancy of like, these are all the things we have to have happen.

• 1:17:15Speaker 01

  Right. And you know, how quickly, how quickly does that change? But you know, GPT four and the, and the token limits, like those created big challenges just is like, go back to like our, you know, to take a step back. It's you take a long form transcript, like we would have to go break the transcript up. And you're like, hey, go summarize this piece of the transcript. And then suddenly you've taken, you know, a 30 minute conversation. You've got to split it up 16 times and you're taking 16 summaries and asking for an aggregate summary, like these are all the challenges that show up. And you're like, well, the better way to get a summary from that is if the token limit and is increased, like I make one request for a summary. Guess what? That summary that I get from that is much more, it's got better context.

• 1:17:58Speaker 01

  So these are all of kind of, I think that the big implications for these next experiences that we want to see. And I think for me, as I said, like, why get excited about this is you see these experiences and they're kind of still, you know, to use your words, like 70, 80% of the way there and you can see what it's going to look like when it's, when it's 100%.

• 1:18:19Nathan Labenz?

  Fascinating conversation. I especially love getting into just kind of the somewhat divergent views of the, or you know, expectations, not necessarily views, but expectations for the short-term future. Anything else you want to cover today that we haven't got to so far?

• 1:18:33Speaker 01

  No, I think we covered like, we covered a lot of ground. As I said, like, you know, reasons we, we kind of chose our path of like trying to do a lot in-house and it was fun talking about, as I said, like the, the views. I was like, I gotta bookmark this and we'll come, you know, hopefully come back a year or two later and see how this plays out. But you know, for me, as I said, I think there's just tremendous opportunities, especially in sales and support and marketing. I think those are like three just right areas for opportunity for AI to be leveraged. Right. Whether it's around content generation, just understanding conversations and powering the systems and automation within them.

• 1:19:09Nathan Labenz?

  Dan O'Connell, AI and Strategy at DialPad. Thank you for being part of the cognitive revolution. All right. Thanks, Saifan. It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please, don't hesitate to reach out via email at [email protected], or you can DM me on the social media platform of your choice.