Ep 45: Founder & CEO of HeyGen Josh Xu on TikTok’s GenAI Dilemma, Trust and Safety and the Path to Interactive Avatars

Joshua Xu is co-founder and CEO of HeyGen -- the fast-growing AI video creation and translation platform. You can upload a video, or create a new one from a script using an AI avatar as your star, and HeyGen will translate it into 175 languages. HeyGen now serves over 40,000 customers and is generating $35+ million in revenue.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Jacob Effron?

  Today on Unsupervised Learning, we had HeyGen CEO Joshua Schu on the podcast. HeyGen recently raised $60 million at a $500 million valuation from investors like Benchmark, Thrive, and others. And Joshua had some really interesting perspectives on how enterprises are using AI video tools today. He also hit on how HeyGen's thought about security guidelines when it comes to voice cloning, as well as the dilemma faced by platforms like TikTok as they balance their content creators and AI-generated content. He also hit on future areas for HeyGen's product expansion. I think folks are really going to enjoy this episode. Without further ado, here's Joshua.

• 0:33Jacob Effron?

  Well, Joshua, thanks so much for coming on the podcast. Really appreciate it.

• 0:36Speaker 01

  Yeah, awesome. Yeah, thank you for having me.

• 0:38Jacob Effron?

  Amazing. Well, I was thinking different places to start. And I think one thing I'd love to know is what it's like to be on the inside of a company like HeyGen when you really go viral? And I feel like there's been all these moments in HeyGen's history, probably, where that's happened. The one that I feel like I was first exposed to, I believe, was when I think Elon and others started dubbing the speech from the president of Argentina at the World Economic Forum and putting it in different languages. What is it like to be in the inside of HeyGen when something like that happens?

• 1:07Speaker 01

  Yeah, sure. It was definitely very exciting so that people start to appreciate the value of the product and also seeing the magic happen, right? I always had talked to my team about that. We're building the AI product. We're inventing the AI experience like disrupting the old way of creating, video-creating content. And I think our role is more like a magician. And I think with all these different areas of AI technology evolving, I think there's a many opportunity for us to build a magic moment. And I think the one you mentioned about video translation dumping was a magic moment that when you actually experience that,

• 1:52Speaker 01

  people can actually, in front of a camera, they can speak into different languages with a natural voice and then also the facial expression on that.

• 2:02Jacob Effron?

  Did you have any warning that was coming? Or you just wake up and open Twitter one day and you're exploding?

• 2:07Speaker 01

  No, we didn't know. I guess we were pretty much like in the heads-down shipping mall. And we just keep shipping the things that we think is cool. But I think one thing we know that internally is that as soon as you keep shipping. Keep improving the product experience, keep listen to the customer. And ultimately, you would have a, you know, a moment that actually hitting the market, yeah.

• 2:30Jacob Effron?

  What was the first magic moment for you with the, with Hagen product?

• 2:33Speaker 01

  Um, it was definitely the moment where, when I created my first avatar.

• 2:41Jacob Effron?

  Yeah.

• 2:41Speaker 01

  And then when I was watching myself that speaking in front of a screen, I think that feels very magical. Uh, you know, and I think some other magic moment, I would say. When you actually start, you know, putting your avatar into day to day use, right? Give you one example is that there was a time I want to share a product update email internally to the, to the team. And instead of filming myself, I have a script ready. And, and I have a chat G P T polished that a little bit and put it into my avatar. And then I can generate a video that way. It felt much, much easier and I don't have to like jump on the camera to record it anymore. And that was, uh, you know, amazing, yeah.

• 3:22Jacob Effron?

  Yeah, I was kind of wondering whether you were going to show up to the podcast or whether, you know, there was going to be an avatar with a script. Uh, but I guess, uh, I'm glad, glad to have gotten you in person.

• 3:30Speaker 01

  Yeah, yeah.

• 3:32Jacob Effron?

  You know, I'm, I'm curious, like, you know, how you describe like where we are in A I for creative tools right now. Like what, what do you, what is, you know, if you were to kind of set context for our listeners, like. What is possible right now? What's still kind of science fiction and not yet possible? Uh, how, how do you think about kind of where we are?

• 3:48Speaker 01

  Yeah, so like first of all, um, you know, Hagen is an A I video platform. That helps people to create, you know, localize and personalize the, the video, right? The way how, how I look at it is like this, you know, um, in the past or even today, majority of the video production. Is actually, you know, first of all, we need to use camera to film it, like what we do over here. And then that is sort of like the first step. The second step, we take the footage out of it and then do some editing, right? Do some post-production as well. So that was the two main steps to in order to produce a great video. And you know, always start with the camera, started with the footage.

• 4:30Speaker 01

  But I think with generative AI, one thing would be changed. I think it's now it is actually possible to generate a footage from using AI instead of having a camera to film it.

• 4:41Jacob Effron?

  Right.

• 4:42Speaker 01

  And you know, when we started the company initially, the mission was to like actually would place the camera.

• 4:48Jacob Effron?

  Right.

• 4:49Speaker 01

  And because without that, there's still many people, they many businesses, they don't have access to that expensive camera. Or for individuals, someone maybe, you know, just camera shy or being introvert. And they they're not comfortable in speaking in front of camera. Now we can have AI that really help that process like making 10x faster, 10x cheaper using AI to generate the footage.

• 5:12Jacob Effron?

  And when you started things, was it like, you know, I mean, did you feel like yeah, these avatar models are ready? Or like these capabilities are ready to go, or was it kind of a bet on, you know, where things were headed?

• 5:22Speaker 01

  No, I think, you know, it it was definitely, I mean, when we started the company, you know, there was no not such term called generative AI. It's more coming from a vision.

• 5:32Jacob Effron?

  Yeah.

• 5:33Speaker 01

  And I believe that technology will actually ultimately solve the problem in terms of generating the video. And you know, I I have always have believed in that, you know, a source of video can be encoded into a computer. That means it's just a zero or one encoding, it's a binary format, right? Then, you know, you you always have a way to figure that out, have a machine to learn about it and generate that. And maybe coming back to the point that I I think the new way we imagine of how a video can be made is that you generate the footage. And even the whole editing experience could be very different, right? So when we look at today's editor, they're mostly timeline editor.

• 6:08Jacob Effron?

  Yeah.

• 6:08Speaker 01

  And then you have a multiple layer in the timeline. And then, you know, the the whole point of the the reason why a timeline editor exists? It was actually because of the camera was very expensive, right? Because camera was expensive, we kind of when you film film the footage, you need to take it 10 times and 20 clips and stuff like that. And then you pull import all of them into a timeline editor, you pick up the best one, do some editing. But the thing is that, once you can actually generate the footage now, you no longer need to actually, you know, have rely on the timeline editor again. So I would imagine that in the future, near future, actually, the whole editing experience will be vastly different. And, you know, we may not need to operate on the timeline.

• 6:50Jacob Effron?

  What do you think it looks like in the future?

• 6:53Speaker 01

  I mean, obviously, we are still exploring that. Some of the very exciting industry progress on that would be, you know, now you can generate a video using text, right? You know, from a text to, you know, to a video output. And, you know, I think it would be a combination of different UX, like writing the script, editing it as a light documentation, you know, having different sorts of UX to navigate through on the 2D camera.

• 7:23Jacob Effron?

  How do you kind of prioritize across, like, everything one can do with a video? How do you prioritize where to focus?

• 7:29Speaker 01

  I would say, you know, it comes back to the original part, right? You know, we are building a magic moment. We want to invent a way how we can, you know, create a video for customers. And I think what's very, very important is that that's your magic walk. And, you know, I think it comes down to really on the on the topic around AI quality, right? So when we say about this new workflow, this new technology can replace the old workflow in terms of a few minute footage. Can this generated footage actually replace that camera process? So it always comes down to the AI quality. And, you know, when we say about quality, about the model, the AI model, it's not only about solving, I would say, a mathematical problem,

• 8:13Speaker 01

  because, you know, when you have a better loss function, it doesn't actually have to necessarily make your video look great, right? I think what does make the video look great has many different aspects, you know? Like, for example, the footage of the lightning, you know, the naturalism on the person, how they speak, whether they actually, you know, have their body motion and gesture matching the script that the content they want to represent. And all those comes down to the quality of the video. I think that's the number, always the number one focus for us.

• 8:45Jacob Effron?

  How do you evaluate those models? And figure out if you're moving in that right direction, given it seems somewhat subjective, or at least, you know, requiring some pretty substantial aesthetic taste?

• 8:54Speaker 01

  Yeah, I think it it I I would say it it is a very tough question. And you know, I think usually it comes down to like we being a team. Uh internally, we need to be very, very good at that. Right. And you know, um the question I will always push everybody in the company, including myself, is that, all right, can you actually build a great avatar? And that you you you are happy with that you will be using that in your day to day work. And I think that as ethics specs are uh somewhat subjective, but also, you know, once you actually uh get engaged on those type of problem every day, you kind of have developed some sense.

• 9:30Jacob Effron?

  So how do customers use Hagen today? Like, obviously, it's such a a broadly applicable tool. Like, what what are kind of the main use cases you see?

• 9:36Speaker 01

  Sure, um like we have over forty thousand customers on the platform today.

• 9:41Jacob Effron?

  Yeah.

• 9:42Speaker 01

  And they mainly use for uh three main use cases. Create, localize, and personalize, right? When when we say about create, they can create the avatar or pick select, you know, one of the stock after we we have on the product and type the text and create a video. And without having without on the camera anymore. And the second biggest use case is localized, you know? And people can take existing video, the non-Hagen video, and we can help them to localize into more than one hundred seventy five different languages and dialects. And still keeping the voice tone, the facial expression, and the person, and apply the lip sync to that. And you know, the last piece is about, you know, personalization, right? And you know, now you can create one video,

• 10:24Speaker 01

  but you can actually use Hagen to personalize that particular video into more than over a hundred thousand different variations, depending on who's the customer, which which is initially working on what other problem they are facing. And you can personalize the video messaging the same way as you personalize the email.

• 10:41Jacob Effron?

  Yeah. One thing I think is interesting for creative tool companies is it feels like sometimes there's this tension between, you know, building for the most cutting edge, you know, filmmaker like artist types versus, you know, your average person in marketing that, you know, never really plays around with video tools. How do you guys think about like that, you know, building for your very sophisticated users versus the fact that anyone can pick it up and and and figure it out?

• 11:03Speaker 01

  I think, you know, one thing to call our Hagen's not for. Uh it's not built for the professional uh video editor who already have access to camera. Well, they have the skill set to actually making their video editing on their own using some sophisticated softwares. He just actually for the rest of the ninety nine percent of a user who are not professional player, we call that. And you know, people, uh I think one example you you gave is a great like the content person, right?

• 11:29Jacob Effron?

  Yeah.

• 11:29Speaker 01

  And you know, like marketer, they usually produce the content, write the script. But they may not have a necessary have the skill set to actually produce the video. And Hagen is the tool to actually help those people to enable that. And our mission is to actually want to uh you know, enable the visual storytelling to everybody. And you know, especially for people who doesn't have access to expensive camera, who doesn't have know how to use the sophisticated video software.

• 11:54Jacob Effron?

  I think, you know, an interesting challenge folks face in your situation is just teaching, you know, users how to do an entirely net new thing. Like, you know, I think a lot of AI products have this uh challenge.

• 12:04Speaker 01

  Yeah.

• 12:05Jacob Effron?

  How have you thought about like when that marketing person opens Hagen for the first time? You know, what what have you learned, I guess, about making it as easy as possible for them to like to figure out? You know, it's a new way of doing things. Maybe they haven't used an avatar before. They don't know the capabilities. What have you kind of learned doing that now for for a few years?

• 12:23Speaker 01

  Yeah, I think, you know, uh it was a journey that in in the very beginning, we were very, very focused on in terms of like presenting the magic, right? So this is possible. I think, you know, there's a lot of walk around telling people what it is possible today. And what does it look like today? And where is the technologies at? Because, you know, Hagen is a very horizontal platform. And we have uh many, many different use cases like, you know, ranging from marketing, sales, support, customer success. And then also with a lot of, you know, uh you know, training, internal and as well as, you know, common creators. And, you know, it it is a um, you know, important effort that actually for us to invest on, you know, showing case what are the different use cases in different verticals.

• 13:07Jacob Effron?

  Yeah, right. And so ideally, I guess, if you're a new user, you try to get them to their use case as fast as possible. And then show them kind of one of these, one of these magic moments.

• 13:15Speaker 01

  Yeah.

• 13:16Jacob Effron?

  I guess like there's a lot of folks, you know, I feel like the avatar space has really taken off over the last year or so. And you know, there's a bunch of companies in the space. Like, how have you, you know, what actually differentiates things on like the model layer? I mean, you talked about kind of realism and this feeling, you know, being able to convey emotion and whatnot, but would be helpful as you think about like what makes a good avatar and what's required from a technical perspective to make that happen.

• 13:40Speaker 01

  I think it's really centralized around one question. Is this footage engaging, right? And you know, there's because ultimately we will build this video and share with someone else. And I think especially in the business context, the purpose of a video in the business world is to actually deliver a message effectively, right? You know, if people drop off right after three seconds, that means that message is not effective. But engaging is one aspect that we keep thinking about. You know, expression is important to get engaging, you know, and I think the mental model we had is that we don't always talk about speak with our mouth movement. We actually speak together with our head movement, our eyebrow and our body motion and gesture together, right?

• 14:26Speaker 01

  That is actually the most challenging part in the puzzle. How do you actually coordinate all these different pieces together? And you know, kind of integrate the system together to build an engaging experience together.

• 14:38Jacob Effron?

  How do you actually build that model? Like, is, you know, both, I mean, is there a lot of just like watching video footage of people speaking or like, how do you go about training a really good avatar model? Hey folks, Jacob here, just taking a quick break from our episode with Joshua to let you know that we run a ton of really interesting episodes on unsupervised learning with founders and cutting edge thinkers in AI. If you're enjoying this episode, please consider sharing with a friend. We're always trying to get the word out about the podcast. And now back to Joshua. How do you go about training a really good avatar model?

• 15:07Speaker 01

  First of all, we build the entire like video layer in-house and we have a dedicated research team building all the avatar models, not only lip sync, but also driving a lot of the body motion as well, so rendering the full body as well. You know, we actually recently rolled out our avatar 2.0 that was actually not only built on rendering the face, actually it was a model rendering the entire body, right? And I think what is very important over there is that

• 15:38Speaker 01

  You know, when you actually build up the whole solution out there, um, you know, it it is quite important to notice that. Essentially, I think I think the process could be look like this. And, um, you know, it's just like, assuming there's a person sitting here. Um, he's watching a lot of a talking video every single day, right? And then I think the point was that, all right, cool, then can you actually figure out a way uh to reproduce those type of content? And I I think that learning process would be same for the AI model as well. Um, you know, and um, I think data is actually one one of the biggest uh piece that um that the puzzle we need solve. And also, I think continuously improve the mobile model architecture

• 16:21Speaker 01

  to being able to capture those like variants, um, you know, capture those um your dimensions and integrate them them together.

• 16:28Jacob Effron?

  How do you think about kind of, you know, this one really engaging avatar model versus, you know, there's many different ways to be engaging?

• 16:35Speaker 01

  Yeah, sure. So I think um um i- i- if we roll it back, right? So I think the simplest way to create an avatar, you know, actually, you could actually use a photo.

• 16:45Jacob Effron?

  Yeah.

• 16:46Speaker 01

  But the problem with the photo is that, let's say, I put my photo and create avatar, and the AI model would not be able to figure out how I talk, right? I usually talk about, you know, things that when I move the gesture like this way, or, you know, my facial expression is a little bit different than than other people. And, you know, that's why we introduced something called, you know, the video avatar, where you actually need to submit the footage. And I think the idea was that the AI model will mimic the whole behavior of you speaking inside that footage. And I think that personalization came from that particular.

• 17:19Jacob Effron?

  It's amazing how fast you get it. I feel like there it you there's not much you need to upload to.

• 17:23Speaker 01

  Yeah, and we usually require like from, you know, thirty seconds to two minutes, and that will be good enough for us to really learn about your talking style. And when I say talking style, right, not only about how you move your your mouth, but also everything together. We are also in the process of having, you know, a larger model to really capture more footage there so that we can capture. The different mode that you might have, like for example, the presentation mode.

• 17:47Jacob Effron?

  Right.

• 17:47Speaker 01

  And you know, like interview mode, right? And then uh maybe something else when you have a uh and then we didn't build up that thing inside the, you know, um um uh the AI model itself. And it it it and then the question comes from, you know, okay, how does customer can use that particular AI model? Uh I mean, obviously, we will provide control and where you can use customer can set that which mode you want. And I think another thing, it is very possible with the AI today is that, you know, you can kind of figure out and and this building as a recommendation for customer based on the script, based on the content they want to talk about. And then you can kind of like adaptively changing the avatar mode and behavior uh in in inside of you.

• 18:29Jacob Effron?

  Yeah, no, it's amazing. I feel like the the fact that you made these models more engaging, you know, when avatars first came out, I feel like it was just for learning and development where it was like the the alternative was so boring that like it didn't really matter how engaging it was. It was gonna be better than, you know, reading a a bunch of learning and development things. But now, obviously, you've been able to expand use cases to marketing and and a whole host of other things. You know, one other area of potential expansion that I think is really interesting is today, a lot of this is asynchronous, right? Um and I think the the potential of kind of like synchronous generative streaming is is really interesting.

• 19:00Speaker 01

  Yeah.

• 19:01Jacob Effron?

  Um obviously, I imagine that is technically quite challenging.

• 19:03Speaker 01

  Yeah.

• 19:04Jacob Effron?

  Um and so I'm wondering if you talk a little bit about some of the technical challenges there and then also, let's say we solve those, kind of the world you see if we're able to get to to asynchronous.

• 19:12Speaker 01

  Yeah, so uh but first of all, we we actually have the, you know, interactive avatar uh version, uh it's a beta version on the product today, where you can actually um um upgrade your avatar into an interactive version of avatar where you can actually send your avatar into attending, you know, some of Zoom meetings.

• 19:30Jacob Effron?

  Yeah.

• 19:31Speaker 01

  And being able to interact uh in the in the real time, right? I think some of the challenging, uh yeah, you're right, you know, once you hit the quality uh bar and then the the next question is about the performance. And I think, you know, obviously, you know, the model will become big, big, big. Become bigger and bigger, and I think the model architecture will be more complicated. And the challenge is more around how to navigate through that. Keep optimizing the inference speed over that side. But, you know, I do think that will bring a whole new use cases. If you look at a lot of advertising, you know, platform today, like Facebook, Google. When I look at the same video ads, you know, even though we get recommended into different ads. But when we look at the same one from same brand, we always are looking at the same video.

• 20:15Speaker 01

  A better way to actually, you know, utilize that experience is that. What if we can look at a different video based on our, you know, preference, based on our, you know, you know, watch history, right?

• 20:28Jacob Effron?

  But it seems like basically to to render this on the fly, you're saying essentially today, these are like bigger models that it's hard to run inference on that fast to to get the models, you know, to to generate that.

• 20:37Speaker 01

  Yeah, but I think, you know, that is, it's definitely a bit technical challenging. It's a very challenging problem. But I think, you know, I'm I'm very optimistic about that, you know, within 12 months from now. I think a lot of this technology will actually be able to run in real time. And, you know, and and even some of them will be able to run in real time on device.

• 20:57Jacob Effron?

  What about like full body movement? I know that's obviously been another area you've been focused on. It seems like you've done a lot there, like talk a bit about some of the challenges associated with with doing that and kind of where you guys are today.

• 21:08Speaker 01

  Yeah, sure. So I think, um, um, let me just circle back why the full body rendering, uh, full body of that is very important. Uh, yeah, it just comes back to how how do we define, uh, you know, what, uh, uh, a human presenter, uh, you know, is engaged, right? I think the the body motion, as body gesture is actually pretty important out there. And, you know, um, I I I mentioned a bit earlier that the three point old version of the Higien Alpha that is actually a full body rendering.

• 21:38Jacob Effron?

  Yeah.

• 21:39Speaker 01

  And, you know, uh, we have not included gesture yet, but that will be the pretty much the next step from from that. Um, you know, I I think the challenging would be more around. I think that's just like in general a new area. I don't think we have a lot of data. We have really understand the model, what's the right model architecture to capture that. But ultimately comes back to a thing that, you know, it is zero or one or encoding. We will always have a way to figure that out.

• 22:06Jacob Effron?

  So obviously, there's a lot of folks working on kind of pure text to video models like the, obviously the Sora folks and Pika and others. How does that kind of intersect with what you guys are doing long term? Can you imagine a world in which they complement each other? Or is it like, you know, do you think you both go and expand into what each other is doing? Or how do those two kind of live alongside each other long term?

• 22:25Speaker 01

  Yeah, sure. So the way how I look at it is that, first of all, at HeJin, we focus primarily in the business videos, right? So when I try to look at it, you know, in order to solve the video generation for the business context. And they essentially, there were two paths together, in my opinion. One part is that, you know, we sort of like generate the entire video from end to end. And then the generation unit would be pixel by pixel, right? And then, and then frame by frame. And that's probably some of the text to video technology to talk about. And I think there's another part is that, you know, you build an orchestration engine.

• 23:05Speaker 01

  And sort of like, you know, being able to capture text, script, voice, sound, music. Avatar footage as A roll and some of the B roll generation altogether. And we believe in the second approach because that provides more, you know, you know, control consistency as well as the quality. And when we talk about the business video, right? What does the brand or enterprise or business want in terms of video? They exactly want control, consistency and quality. And I think this, this second part in terms of a technical approach will allow us to be able to like deliver, you know, much better experience on that. I mean, obviously, especially when we talk about control. Yeah, we can talk about that. You can control the video or even image, right?

• 23:50Speaker 01

  Using a prompt, but you know, actually a better user interface probably using a mouse to handle that on the 2D dimension of a camera. So I would see that, you know. Uh in in this case, we would work with this uh all all the, you know, text to video partners closely. Because that is actually important piece of the system.

• 24:09Jacob Effron?

  Yeah.

• 24:09Speaker 01

  Where we take that as a B role, but we kind of build a whole Audi, build Audi, you know, orchestration engine on the top. And that service there that uh interrupt the customer, you know, directly.

• 24:19Jacob Effron?

  That's so interesting. So it's yet another input into, you know, you might want an avatar or something, some generative scene for something else. And um you know, you might wanna tailor a voice for something and basically you guys are, you'll build some of these features yourself. You'll kind of input in the the text to video that's the best and kind of orchestrate across all of it to provide this really holistic video experience.

• 24:39Speaker 01

  Yeah.

• 24:39Jacob Effron?

  We've talked about kind of like full body synchronous, you know, text to video. Any other areas that you're paying attention to that you're like, hmm, in the in the next six, twelve months, that might change this kind of video space a lot?

• 24:50Speaker 01

  I think another important piece, I would say, uh is technically possible, but um I think the experience has not really, you know, uh came up to market yet. It's really the um the brand personalization layer, right? When when I say that is, you know, we we obviously, let's say, we are using ChatGPT today, right? You can actually give it a prompt that, hey, give all the context about the company. And then the the ChatGPT will help you to write a script in your brand tongue. And then, you know, already realize that what's the context, what's the background, what's the history of our company and what's the product? And that is still not possible with the video today, right?

• 25:29Speaker 01

  So I think, uh I I mean, one important piece of the puzzle was actually to dissemble the the video into different component. And then when you try to uh assemble them together in the final uh video format, uh I think that is the brand personalization you can keep in. And you know, one one one great experience would look like would be, hey, you know, we just take a URL or what? A taste, look at some of the, you know, past video that the company made and sort of like the AI model would learn. What's the color tone, you know, what's the style, what are the uh you know, opening and, you know, um video clip? And it can sort of like bake all this element into the final assemble uh process. I think that would be another very, you know, exciting feature.

• 26:14Jacob Effron?

  How would you do that?

• 26:16Speaker 01

  I guess, you know, very similar to, you know, the chat GPT context window.

• 26:22Jacob Effron?

  Yeah.

• 26:23Speaker 01

  You sort of like, you either spin up all those fine tone. Yeah. Or either you have a context window. But really, I think, I think the underlying, you know, idea was that you want to have some user input as a, as a memory. And then you can bake that into embedding so that it can feed that into the AI model.

• 26:44Jacob Effron?

  You know, one thing I'm interested for your thoughts on, I feel like a lot of time in the AI startup world, you know, the people are like, well, the incumbents are so well positioned to do this, you know, they'll get around to it. And, you know, obviously in your world, you compete with some pretty impressive incumbents. Obviously, your former employer and Snap, you know, there's TikTok and other folks that are kind of like prosumer video focused. Obviously, when you were leaving Snap, I'm sure you thought about you could build these features within Snap or in a startup. You know, how do you think about like the way those platforms evolve where they have the distribution already versus kind of, you know, what you guys are able to do at Agen?

• 27:16Speaker 01

  The way how we look at the whole video space or generally the creative content space, right? I think we are at a stage on the creative tools stage. But I think when we look at the distribution, when we look at the platform piece, they're still owned by lots of, you know, passing company developer, you know, especially evolved in the, you know, mobile camera phrase. And the way how I look at it is that, you know, it is, first of all, it's about who's the customer. And initially, when we talk about building the video creation tools, it was for people that has the camera.

• 27:57Speaker 01

  They are relatively professional and advanced in terms of using sophisticated editing tools. But now I think he just really build for people, the content people, you know, the, the people who don't have access to those things. And I think the product experience will be majority very different, right? And there's definitely many opportunity out there for us to build a whole new experience for, for those type of people. I think the way how we look at it is that there would be, it is not really competing on the old market, but it's actually opened up a, there's a new market opportunity that, you know, yeah, in this case, startup and incumbent would both want to work on that. And I guess it would be really depends on how, how, how the different product can capture the customer needs

• 28:42Speaker 01

  and build a product feature that, you know, satisfy the customer, what they want.

• 28:46Jacob Effron?

  And it seems like both Snapchat and TikTok have been pretty focused also on some similar, you know, end markets and avatars and whatnot. Like, what do you think of what they've done so far? Do you think the focuses are the same or do you think they kind of diverge at some point?

• 29:00Speaker 01

  The way how we look at it is that, I think, when you talk about, we will talk about some of the incumbents, you know, especially vertical video and short form video platform, right? I think the key idea was to unlock creators that can, you know, easily create a video using a mobile camera. I think that was the key, you know, value that drives that. And I think the key value for, for a new platform like us is that it's not about that, right? Because we want to make the camera obsolete and we actually want to be enabling the, you know, the, the video creation experience without the camera. And I think there would be always a dilemma for some of the platform.

• 29:43Speaker 01

  I give one example is that, assuming, like obviously a lot of the platform are built for, are centralized around creator, right? And, and but, you know, now this technology is actually, you know, not for the creator who actually have access to a camera. And assuming there's a 10% of the content that are being generated on AI today. And how would people rank those content? How would people recommend those content compared to the rest of the, you know, 90% of the camera based? I mean, the question would be, all right, we can mix them together, but what if that ratio become 50%, 50%? That means that if you kind of ran these two content together, that means that the existing conector would be ma-

• 30:24Speaker 01

  Vastly, you know, getting less views, getting less attention for customer, right? And and that in that case, you know, if if that stage arrived, then we would have to like build up a new platform really for the AI generated content. I think that's where the new opportunity for a new platform will emerge.

• 30:41Jacob Effron?

  It's so interesting because basically, yeah, to your point, the TikToks of the world, you know, they are in service of their content creators and, you know, are promising a platform that if, you know, they're competitive places to get like the Charlie Demilios of the world to post and they say, hey, you post here, you'll get the biggest reach, the most exposure. And they'll face some dilemma as the AI content gets more popular of do we promote this or do we suppress it? Because it directly competes. So do you think ultimately you guys have like a consumption platform as well for all this AI generated content?

• 31:08Speaker 01

  That is not the the mission. And we just want to build a great creative tools to enable everybody have access to visual storytelling. But I would say we could either see that that's a possible as a new opportunity for new platform emerged. I guess we will see.

• 31:26Jacob Effron?

  What have you like learned and obviously you guys have made, I think, like this big enterprise push in the last, you know, three, six months, it seems like it's going very well. What have you like learned in servicing those customers? Like any things you were surprised by that you needed to do in the product or things you didn't expect?

• 31:39Speaker 01

  I think one piece about, you know, enterprise customers, I think, obviously, I think the quality requirement there is much higher, right? And, you know, in terms of a brand consistency, in terms of like, you know, the the quality of the output of the video, that's one. And I'm obviously that aligned with our product strategy, we have been always really focused on that. I think another very important piece for we did enterprise world is that, you know, it's about integrating the technology and the product into the day to day workflow. There will be many, you know, mixing piece about the integration and workflow we are actively working on.

• 32:18Jacob Effron?

  What are some of like the the integrations that are end up being most important?

• 32:22Speaker 01

  I think it really depends on which which use case we're talking about. If we're talking about marketing, I think, you know, integrating with existing CRM, existing go to market tools are very important, right? I mean, in fact, we actually was on, you know, at HubSpot last week. We were, you know, announcing the partnership and we build some of the integration and app. Integrating it into the app ecosystem of HubSpot so that people can actually, you know, using HeyGen as a tool together with their marketing product, with their CRM product together. I think that's one thing that would dramatically improve the customer experience.

• 33:01Jacob Effron?

  That's kind of pulling data both for the context to provide HeyGen, but then also making it really easy to ship a HeyGen.

• 33:07Speaker 01

  Yeah, exactly.

• 33:08Jacob Effron?

  You know, one thing I feel like you've been super thoughtful on is the kind of security side of this. And, you know, I think you guys have been, you know, really forward in some of your guidelines around voice and, you know, what you're able to do and not do. I'd love to hear just how you've thought about that. You know, it seems like from the outside sometimes that in, especially in the like prosumer world, the people that like push the boundaries the most and let you do, you know, the Donald Trump voice or the Taylor Swift voice or whatever it is, like end up getting the most eyeballs or something. And, you know, you guys have obviously thought about this a lot. You know, talk to me about like how you've thought about that over time. And, you know, the decision to, I guess, you know, if it's impacted you basically not to, to go up to the line or maybe even past it.

• 33:44Speaker 01

  First of all, trust and safety is critical for our business. We are serving some of the largest enterprise customers. And, you know, it is really important for us to like having, you know, the trust and safety implemented correctly in the platform. So the way how we look, we look at it is that there's a two big piece about the trust and safety work at HeyGen. One is about creating your avatar, the other one is about, you know, the video content you're creating, right? So for every single avatar that people are creating on HeyGen, we would always require a consent, a video consent format. And we have, you know, advanced AI to actually match the consent. Actually, it's the same person as the footage. And meanwhile, we also have a dynamic generated passcode where, you know,

• 34:29Speaker 01

  it kind of expired every 10 seconds or 15 seconds so that, you know, really giving a secure layer that to protect that. So that, which means that it is almost impossible for people to actually creating someone's avatar without their consent. And that's one. And, you know, meanwhile, we also have a, you know, build up the moderation team, not only. Be reviewed by AI, but also have a, you know, moderation team actually doing human reviews um on on their footage to making sure everything is uh you know expected. And that's one piece. I think another piece is about, you know, we have our platform moderation policy that we do not allow hate speech, you know, misinformation, fraudulent, uh you know, uh any political uh campaign

• 35:14Speaker 01

  and and information like that. And you know, obviously, it's a hybrid uh solution between AI model uh review as well as the moderation team uh human review. And just making sure everything, um all the content being produced on HeyGen compile with our uh you know uh policy.

• 35:31Jacob Effron?

  What about like um IP partnerships? Like, how does that play in here? Like, I don't know, you could imagine over time, like, you know, actors giving their permission to use their voices or uh you know, I don't know, fictional characters, people want Iron Man or whatnot, like.

• 35:44Speaker 01

  Yeah.

• 35:44Jacob Effron?

  How you, have you thought about that today or where do you kind of think that space goes?

• 35:48Speaker 01

  Um obviously, we have a a partner partnership with some of the uh you know, existing actors and they uh they build it out on HeyGen so that um that's part of the stock up that we have. And you know, we haven't explored much about, you know, around the IP, but I think one thing would be very interesting to explore is that especially now that you can generate a new voice, right? You can generate a new person that AI generated version. And that could become the future IP, actually. And you know, um especially, I'm not sure, look at that, the the latest uh model from uh at image generation. It could actually pers- persist um the consistency across different generation.

• 36:33Speaker 01

  And which means that that opened up a whole new possibility for uh for people to actually generate a new IP. And you know, what if we turn those image or or into video, right? And then, you know, there's a um um there's a new role out there and that I guess we are very excited to explore.

• 36:49Jacob Effron?

  Yeah, it's like AI influencers. I guess there was that one Michaela, I think, on Instagram for a while that was that was pretty popular.

• 36:54Speaker 01

  Yeah.

• 36:55Jacob Effron?

  Yeah, there's so much you could do with these with these uh with these AI influencers. It'll be interesting to see those those emerge. Do you have any on the platform yet or not yet?

• 37:02Speaker 01

  No, no, not yet, yeah.

• 37:04Jacob Effron?

  Yeah. Yeah, one thing I'm also curious that a lot of, you know, are a lot of founders think about is, you know, you're a hot AI company. I'm sure people are always trying to throw money at you. And you probably have access to endless amounts of fundraising. How have you thought about like the appropriate amount to raise and how, you know, capital intensive in the business? Agen ends up being and I feel like a lot of folks are trying to figure this out now where there's, there's a lot of capital available.

• 37:28Speaker 01

  I think especially in the whole AI category, some of the biggest, you know, cost factor would be actually around GPU and talent. And, you know, I think that's one, but I think one thing I have learned quite different. I think, you know, when we look at the finance model about a software company, right? And, you know, it's actually very different from the AI category, in my opinion, right? So when we, when we sell software, your marginal cost is close to zero. And on, on serving additional customers, but it's not true for AI. Because, you know, you, you are actually, you know, consuming a lot more GPU compute in that case.

• 38:13Speaker 01

  And I mean, obviously GPU is not zero today, right? And I think that's one thing that's different. But I think another thing I want to call out is that with all this AI technology evolving, the individual employee also gets much more efficient than before, right? I mean, I personally get much more things done with GPT helping me around and, you know.

• 38:35Jacob Effron?

  And it sounds like Agen for your product updates and whatnot internally.

• 38:38Speaker 01

  Yeah, exactly. Yeah. What was funny is that I was requested on the customer test premiere yesterday. And I did it with my avatar, right? So, and so that I don't have to like film it myself anymore. But I'm trying to say that, you know, I think there's a, some new generation company that is really AI native and which also means that the teams are also AI native. It gets much more efficient, right? And I think especially if you look at some of the, you know, earlier player in the AI space.

• 39:12Speaker 01

  The suggestion of growth is like insane, right? And you know, I think one example, we also look at a chat GPT get to a hundred million user put extremely fast than any of the past company uh in the past, right? And I think the go to market would be also accelerated because the whole industry, wherever is excited about AI. So I would say uh surprisingly, you know, we actually require less capital to build a great AI company.

• 39:41Jacob Effron?

  So imagine one thing you could do is, you know, get have a very generous free tier, which you have a free tier and but like, you know, I feel like companies like yourselves are, there's always this test, this tension of like, you know, you face inference cost every time you people run these models, but like people discover the magic moments from it for being free. And so I could imagine you could just like make the free tier even more, like, you know, just bombard the market with as much hey Jen as possible like.

• 40:04Speaker 01

  Yeah.

• 40:04Jacob Effron?

  You know, obviously you do some of that, but how have you thought about that?

• 40:07Speaker 01

  Yeah, I think that's a that's an interesting, interesting idea. I think if look at some of the uh LM space, I think LM definitely have a free offering.

• 40:17Jacob Effron?

  Totally.

• 40:18Speaker 01

  Obviously, we have our free tier uh as well. And I guess, you know, you at some point you will start to see a marginal gain as you open up more free uh usage. And I guess we haven't really, you know, uh focus on exploring exploring that yet, but I guess there will be, yeah, very interesting idea.

• 40:36Jacob Effron?

  Is there anything that like you don't do today because the inference costs are too expensive? Like you have the technical capabilities, but you're like, you know, God, because inference seems to come down, you know, a dramatic amount every six months. Like, is there something you're waiting for it to get a bit cheaper?

• 40:50Speaker 01

  We we don't wait.

• 40:52Jacob Effron?

  Yeah.

• 40:52Speaker 01

  We always build a product ahead of twelve month.

• 40:55Jacob Effron?

  Yeah.

• 40:56Speaker 01

  Like I give one example is that um and when we design the product today and the decision is made based on the um um the future of twelve month. What are the model capability uh this one? Uh and what is the um the cost? And you know, what are the different piece of the puzzle would evolve together? So um yeah, so I guess we actually always build ahead.

• 41:22Jacob Effron?

  Yeah.

• 41:22Speaker 01

  Yeah.

• 41:22Jacob Effron?

  That's awesome. Well, we always like to end our interviews with a quick fire round where we get your your thoughts on some uh some broad questions.

• 41:28Speaker 01

  Sure.

• 41:29Jacob Effron?

  Uh you know, maybe to start would be, what's one thing that's overhyped and one thing that's underhyped in AI today?

• 41:35Speaker 01

  I think overhype it is the.

• 41:39Speaker 01

  Speed in terms of like um really getting AI into playing a huge value in some of the business and enterprise. Right. I think um underhype is actually the the the final um impact of that. And I think um we always tend to when when an innovative technology like emerge, you always think that it's like it should be. Go um you know much faster than we than we uh expected, but I think um I think that's like something. Um yeah, yeah, it happened in the issue in my opinion.

• 42:13Jacob Effron?

  Yeah. What's something you've like changed your mind on since starting Agen?

• 42:17Speaker 01

  I I mean, there's many things I changed my mind. I would say I may just like get a lot of learning along the way uh it it alongside the journey. But um you know, since we are very technical, talking about a lot of technical piece today, I think I can give one example on the technical piece. And um I think in the early probably twenty twenty one. Um you know, um the whole metaverse was a bit hype at that time, right? And I mean, obviously, there's a a huge advancement in terms of the three D, uh you know, um graphics technology. And because essentially, when we try to solve the problem about video generation, there's really, there's also two ways of solving that, right? One is that generating pixel by pixel.

• 43:02Speaker 01

  And the other one is actually building the three D model layer as the underlying, um you know, uh guy guiding model first. And then you put pixel on top of that. And I think since the um you know, uh stable diffusion, I think that completely changed my mind. We invest on three D technology very early on and was trying to explore that path. But um when when I saw stable diffusion, I I I just realized that, all right, this technology. This technical part is gonna go much, much faster because you know, it it would allow you to be able to train on the large scale of data. And I think that was one thing, you know, uh we have been changed, you know, in terms of technical part.

• 43:42Jacob Effron?

  So interesting. On the customer side, like anything that surprised you, like, you know. Something that you didn't think would be a big deal, but actually was like really impactful, or maybe something you thought would customers would love, but it ended up liking as much.

• 43:54Speaker 01

  Yeah, sure. So, um, you know, this is actually many example like that. I have on average one to to customer call a day. Uh, so I always ask about that question, actually. And I think, uh, uh, I think I would be always surprised that how detail-driven some of customer will be paying attention on the avatar, uh, uh, quality, like the engagement. Because I think it is tricky to actually look at someone else avatar versus you are looking at your avatar on yourself.

• 44:28Jacob Effron?

  Yeah.

• 44:29Speaker 01

  You would have a much higher bar in that case. And then, uh, I think I also learn new things from talking to customer. And I would say the second thing, uh, is that, you know, we always wanted to make, uh, you know, the, uh, I I think the streaming avatar is one example that we we we try to, uh, have the avatar team talk with customer in real time. But, uh, I think we still need to improve the performance out there. And we did, I think, getting a lot of pe- feedback around that, uh, we we we I guess we need to continue to improve on this part.

• 45:01Jacob Effron?

  What's the most like detailed piece of avatar feedback you've gotten? Like a really like in the weeds like?

• 45:07Speaker 01

  Sure. So I was talking to, um, uh, a customer, uh, actually yesterday. Um, uh, I said last night, he was, he he's a, he's in Eu- Europe. And, uh, uh, I think he give a lot of feedback around the gesture. Right. So, um, you know, uh, he produced avatar video longer than sixty minutes, uh, actually five or six minutes. And he was complaining about, uh, I mean, one of the ping point that, uh, the customer had is that, you know, sometimes the gesture of the person does not match the, uh, script or the content. And, you know, but it was fine in the first few minutes and it was start to like, uh, you know, uh, move around the gesture randomly, uh, at the end of the video.

• 45:50Speaker 01

  Uh, and then, I mean, that was not easy to identify unless you actually make a much longer video. And, um, yeah, and then he actually give a lot of feedback around how he would like that avatar to behave in terms of, you know, in this session and that session, uh, not only the body motion, but also the gesture. And and I found that very, uh, very amazing, yeah.

• 46:10Jacob Effron?

  That's so cool. Um, which AI startup like outside of the video world are you most excited about right now?

• 46:16Speaker 01

  This is probably not a startup, but you know, definitely a new generative AI product. It's a Google Notebook LM. I'm not sure, you you look at that, but it was a very, very cool technology, in my opinion, that you can put in a blog post and some text or URL and then it would generate a podcast format in a video format, in an audio format. And what was funny is that-

• 46:37Jacob Effron?

  I better watch out, that sounds, that sounds like it's coming from me.

• 46:40Speaker 01

  Yeah, yeah, and what's funny was that I send out a weekly product updates myself to the entire company every week. And I did one yesterday and then, you know, and I I I've kind of feed in my entire product updates into the Google Notebook LM and then it generated a soundbite for two minutes voice. And then, and I shared with him and then it opened. And because now you can actually listen your product-

• 47:09Jacob Effron?

  Yeah.

• 47:09Speaker 01

  -in the podcast format with two people actually talking in a conversation way. Yeah, I I really, really like that.

• 47:14Jacob Effron?

  If only there were avatars to put on top of that.

• 47:16Speaker 01

  Yeah, I mean, that that the next step is actually to turn it into an audio podcast and a video interview, yeah.

• 47:21Jacob Effron?

  Yeah, yeah. Well, I guess be- before I leave the last word to you, my last question that's shamelessly broad is just, what do you think like, you know, video creation workflows look like in 2030? If we fast forward like way, you know, way in the future, what's your vision for kind of where this is all going? Like when, you know, when when people are learning this and, you know, this next generation, when they, when they learn this?

• 47:43Speaker 01

  I always have a very simple mental model to think about how Hagen as a product will evolve and what are the problem, what weighs the bar in terms of building product experience for the customer? Is that I always imagine when I need to create a video, I would imagine I have a video agency sit right next to me, right? And, you know, if you try to think about it, what is the process look like for for you to create a video with a video agency? You talk about your ideas first, step one. Step two, the video agency will help you to film the footage and then once they have the footage step throughs that they they go to editing and then they present it to you and then you give some feedback and then they go back to edit it. And then we have the final video.

• 48:26Speaker 01

  I think, you know, five years from now, I would say-

• 48:32Speaker 01

  Everybody will have their video agency on the pocket, right? So where hopefully Hagen provide that experience that you know you can actually interact with Hagen with the product. And like like you're talking to a specific, you know, personal video agency for you. I think that is the ultimate, you know, uh, uh, vision that or experience we want to enable.

• 48:55Jacob Effron?

  Yeah. What do you think people will make with that? Obviously, you've seen this first wave of use cases on the first generation of Hagen. Now, the twenty thirty version, what do you, what do you think you and I, some of the things we might make?

• 49:05Speaker 01

  I mean, sure. So I think the way how we look, really look at it is that. Um, I mean, obviously, um, any type of, you know, um, you know, text content, audio content or video content we are making today can be generated with A R in that way in the much faster, lower cost, right? I think, but really, when we look at the power of a creative tool is that you open up new use cases. There's many you can say you actually don't know, right? I think one example I always want to go back is like, right, cool. Imagine in two thousand twelve, when we have a mobile camera. Do we actually know we will have Instagram, Snapchat, TikTok?

• 49:46Speaker 01

  And this type of content, uh, you know, uh, today is very, very hard to do, to imagine like that. But I think what's really powerful with it, with the creative tools that when you improve the tool, when you lower down the equation barrier, you will open up a whole new world that I'm very, very excited to explore what people will be doing with that.

• 50:06Jacob Effron?

  Yeah. You're obviously so passionate about the space, like, did you grow up really into this or was it when you joined Snap or when, you know, how, how did this kind of come about?

• 50:15Speaker 01

  Sure. I think, you know, um, I think, uh, definitely highly related to my experience at Snap. You know, working with, uh, um, you know, a lot of product and growing together with the company. And, you know, um, watching a lot, I, I, I always can just consider myself, I grew up my career in a, uh, you know, mobile platform era, right? So that when actually mobile camera be evolved and then when we actually saw all this different type of content platform emerge like Instagram, Snap and TikTok we talked about. And yeah, definitely put a lot of thought around it. And I think presently, I felt, you know, the most joy for me come from when I create something,

• 50:57Speaker 01

  people use my technology, use the tool that I build to create something on their own. I think that brings me the most joy.

• 51:04Jacob Effron?

  Yeah, well, super cool. I mean, I'm sure there's lots of threads folks will want to pull on. I'll leave the last word to you. Where can folks go to learn more about HeyGen, about you? Anywhere you want to point it.

• 51:14Speaker 01

  Sure, yeah, I think definitely go visit heygen.com to, you know, try out the product. It's free to sign up and it's also free to create your own avatar.

• 51:23Jacob Effron?

  Amazing. Well, Joshua, thanks so much. This has been an amazing conversation.

• 51:26Speaker 01

  All right, thank you, thank you, Jiehao.