Josh Woodward: Google Labs is Rapidly Building AI Products from 0-to-1

As VP of Google Labs, Josh Woodward leads teams exploring the frontiers of AI applications. He shares insights on their rapid development process, why today’s written prompts will become outdated and how AI is transforming everything from video generation to computer control. He reveals that 25% of Google’s code is now written by AI and explains why coding could see major leaps forward this year. He emphasizes the importance of taste, design and human values in building AI tools that will shape how future generations work and create. Mentioned in this episode: Notebook LM: Personal research product based on Gemini 2 (previously discussed on Training Data.) Veo 2: Google DeepMind’s new video generation model. Paul Graham on X replying to Aaron Levie’s post that “One approach to take in building in AI is to do something that's too expensive to be reasonably practical right now, and just bet that the costs will drop by 10X or 100X over time. The cost curve is on your side.” Where Good Ideas Come From: Book on the history of innovation by Steven Johnson. Project Mariner: Google DeepMind’s research prototype exploring human-agent interaction starting with browser use. Replit Agent: Josh’s favorite new AI app The Lego Story: Book on the history of Lego. Hosted by: Ravi Gupta and Sonya Huang, Sequoia Capital

• 0:00Speaker 00

  What I found too, building products over the years is it's very common. Everyone talks about product market fit. You'll know it when you see it and all that, which is true. But at least for me, I've always felt in the first part of building products, you iterate a lot on the product and sometimes you forget to iterate on the market. And finding the right market side is also just as important as the right product. And you have to connect those two. And so I think that in these early stage things with Mariner, that's where we are. It's like, is it possible for a computer to, like an AI model to drive your computer? Yes, that's a huge new capability. Is it accurate? Sometimes. Is it fast? Not at all yet. Like, that's kind of where we are in terms of the actual kind of use case or the capabilities. And then it's about finding the right market.

• 1:02Speaker 01

  Today, we're excited to welcome Josh Woodward from Google Labs, the team behind exciting Google AI launches like Notebook LM and the computer use agent Mariner. Google Labs is Google's experimental arm that's in charge of pioneering what's next in how we interact with technology by thinking about how the world might look like decades from now. Josh is helping to reimagine human AI interaction from the provocative claim that writing prompts is already becoming archaic to the emergence of multimodal AI as a default user experience. He shares insights on the rapid innovation culture in Google Labs, offers a glimpse of what's next in generative video and much more.

• 1:41Speaker 01

  Josh, thank you so much for joining me and Ravi today. We are excited to hear everything that you're doing over at Google Labs. Maybe first to start, you mentioned a provocative topic to me on your way in here. Writing prompts is old fashioned. What do you mean by that?

• 1:55Speaker 00

  Okay, so thanks for having me. I do think it's old fashioned. We'll look back at this time from an end user experience and say, I can't believe we tried to write paragraph level prompts into these little boxes. So I kind of see it splitting a little bit right now. On the one hand, as a developer and AI engineer, you should see some of the prompts that we're writing in labs right now are these beautiful, like multi-page prompts. But I think for end users, they don't have time for that. And you have to be almost like some sort of whisperer to be able to unlock the model's ability. So we're seeing way more pull and traction. I kind of seen this in other products in the industry too right now. How can you bring your own assets, maybe as a prompt?

• 2:39Speaker 00

  Drag in a PDF or an image, sort of recombine things like that to sort of shortcut this giant paragraph writing. So I think it's going to kind of divide. I think as engineers, AI engineers, you'll keep writing long stuff. But I think most people in the world, we're probably in a phase that'll sort of fade out here pretty soon.

• 2:59Speaker 02

  So the form of the context will change, right?

• 3:01Speaker 00

  That's right.

• 3:02Speaker 02

  So you still have to get, give the model something.

• 3:04Speaker 00

  Yeah.

• 3:05Speaker 02

  Right, but it might be that you can communicate it via a picture or communicate it via like, just look at this set of documents.

• 3:11Speaker 00

  Yeah, your voice, a video, any of that. These models love context. So the context is not going to go away. But we're making a lot of bets right now that the type of context and the way you deliver the context, that's changing really fast right now.

• 3:26Speaker 01

  I love it. Okay, we're going to go deeper into the future of prompts and multi multiple models in this episode. Maybe before we do all that, say a word on what is Google Labs? You know, what's the mission? And tell us a little bit more about how you sit, where you sit with inside Google.

• 3:43Speaker 00

  Yeah, so Google Labs, if anyone's heard about it, and we had one a long time ago that went dormant for a while. And this is kind of back about three years ago it got started. It's really a collection of builders. We're trying to build new AI products that people love. So they can be consumer products, B two B products, developer products. It's all zero to one. It tends to attract an interesting mix of people. Maybe people who have been at Google a while, but also a bunch of startup founders and ex-founders. And so we kind of mix these people together. And we basically say, what's the future of a certain area going to look like? So the future of creativity or software development or entertainment. And they go off in small little teams and they just start building and shipping.

• 4:25Speaker 00

  And so that's how it operates and it sort of sits outside the big traditional Google product areas, but we work a lot together. But there's kind of an interesting interplay there. And I think that's been part of what's been fun about it is you can kind of dip in and maybe work with search or Chrome or other parts of Google. But you also kind of have the space to explore and experiment and try to disrupt too. And that's, that's kind of what we're up to.

• 4:48Speaker 02

  How do you create the culture inside of labs that you want? Right? If you think about, there's got to be a lot more failure presumably than there are in other parts. There's got to be a different metric for success than there is at just the sheer scale of Google. So what is the culture you're trying to create and how do you create it?

• 5:03Speaker 00

  So we really pride ourselves in trying to be really fast moving as a culture. So we'll go from an idea to end user's hands 50 to 100 days. And that's something that we do all kinds of things to try to make that happen. So speed matters a lot, especially in kind of an AI platform shift moment. The other thing is we think a lot about sort of big things start small. And one of the things, if you're in a place like Google, you're surrounded by some products that have billions of people using them. And people forget that all these things started with solving usually for one user and one pain point. And so for us, we get really excited if we get like 10,000 weekly active users. It's like, you know, we'll celebrate that. That's a big moment when we're starting a new project.

• 5:47Speaker 00

  And for a lot of our other kind of groups inside Google, their dashboards don't count that low, right? So there's kind of this moment where, you know, the size of what we're trying to do is very small. It probably looks a lot like companies you all work with, honestly, from that perspective. And then I think the other thing we're trying to do is because we sit outside the big groups at Google, we kind of have one foot in the outside world. We do a lot of building and kind of co-creating with startups and others, but also one foot inside Google DeepMind. And so we've got kind of a view of where the research frontier is and more importantly, where it's going. And so we're often trying to take some of those capabilities in. So we take a lot of pride in sort of finding people who are very creative.

• 6:32Speaker 00

  People who are almost like see themselves as underdogs. They have kind of a hustle to them. And so we have this whole doc called Labs in a Nutshell. And my favorite section in the doc is called Who Thrives in Labs? And there's like 16 or 17 bullets that just list them out. And that's kind of how we try to build the culture. But you do have to normalize things like failure. You have to think about things differently around promotion, compensation, all these things that you kind of would do in a company, too.

• 7:00Speaker 01

  You mentioned the deep mind links. I think that is super cool. What have you found is the kind of ideal kind of product builder persona inside labs? Is it somebody with a research background? Is it somebody with a who comes from a successful consumer product background? Is it, you know, is there the magical unicorn that's great at both research and product?

• 7:19Speaker 00

  Yeah, yeah. We take as many unicorns as we can find. And we actually have found some, which is great. You do look for that kind of deep model expertise as well as kind of like a consumer sensibility in terms of

• 7:32Speaker 01

  And those people exist.

• 7:33Speaker 00

  They exist. They're great, too, if you can find them. And we also kind of have found ways to kind of train or develop people. So that's another thing we think a lot about is like, how do you bring in often? People that might not be the normal talent that you look for. So like, we're always in the interesting kind of zone of like, who's undervalued? Who's kind of like really interesting, but maybe not on paper? But when you interact with them, you look at their GitHub history. I mean, there's all these different signals you can look at. But yeah, that's kind of how we would think about it.

• 8:05Speaker 01

  Really cool. How do you decide what projects to take on next? Is it is it bottom up, top down? How does that work?

• 8:11Speaker 00

  Yeah, great question. We kind of do a little bit of a blend, actually. So at the top downside, we're looking at what are the areas that are kind of on mission for Google? That are strategic to Google? Because we sit inside it. So we're thinking about ourselves in that broader context. So that may be, for example, like, what would the future of software development look like? There's tens of thousands of software developers at Google. And obviously, this is an area that AI is clearly going to make a big change in. So we'll be thinking about, could we build things for other Googlers? But also externally, how do we build things like that? So we take that kind of top down view. Think of it as almost, I'm from Oklahoma, we like to fish a lot in the summer. But like, you're trying to figure out what's the right pond to fish in. So we put a lot of thought into those like ponds to fish in.

• 8:54Speaker 00

  But then we let a lot of these teams, often they're four or five person teams, come up with the right user problems to go try to solve. And that's where we kind of meet in the middle. And I think for a lot of other teams, they might look at what we do as a little chaotic. You know, we don't have like multi-quarter roadmaps, like we're trying to survive to the next, whatever, 10,000 user milestone and then try to grow it. But I would say it's kind of that sort of blend.

• 9:19Speaker 02

  What's one of the products that you guys have built that you're excited about now?

• 9:23Speaker 00

  Oh yeah. So I guess if you've ever used the Gemini API or AI Studio or Notebook LM or any of VO, any of these things. These are products that we've kind of worked on from labs. I mean, maybe I'll talk about one that's maybe well better known and one that's coming up. So the very excited about where Notebook LM is going. I think we've hit on something where you can bring your own sources into it. And really AI just like grips into that stuff. And then you're able to kind of create things. So a lot of people maybe have heard the podcasts that it came out last year. There's so much coming that follows this pattern. So watch this space. There's just a lot you can do with that pattern.

• 10:05Speaker 00

  And I think what's really interesting is it gives people a lot of control. They feel like they're steering the AI. We have this term on the team and actually one of the marketing people came out was like an AI joystick. That you're kind of controlling it. So that's interesting. I would say there's a lot of stuff coming right now. We're very excited about VO. Google's imagery model and sort of video model and where those kind of come together. So we've got really interesting products coming along in this space. I think maybe we can talk about that some at some point. But I think generative video is kind of moved from this moment of almost possible to possible.

• 10:40Speaker 02

  Let's talk about it now. Tell us.

• 10:41Speaker 00

  Yeah, yeah. Well, I think it's, it's interesting. These models are still huge. To run like VO2 takes hundreds of computers, right? So the cost is very high. But just like we've seen with the text based models like Gemini and even the ones from OpenAI and Anthropic. You know, the cost has reduced like 97 times in the last year. So if you kind of assume cost curves like that, what you're going to see with these VO models, what's kind of brand new, say with VO2, is it's really cracked really high quality and physics. So the motion, the scenes, the, if you talk to a lot of these AI filmmakers, they talk about, what's your cherry pick rate? Which is a term for like, how many times do you have to run it to pick out the things that's really good?

• 11:25Speaker 00

  And what we're seeing with something like VO is the cherry pick rate is going down to like, one time, got what I want. And so the instruction following, the ability for the model to kind of adhere to what you want, is really cool. So I think when you put that in tools, you're now able to convey ideas in a whole different way.

• 11:43Speaker 01

  What do you think are the solved problems and the unsolved problems in AI video generation? Because I remember, you know, last year, it was like, you know, even last year, there were all these, you know, there was so much talk about, you know, generative video is, you know, a physics simulator, for example.

• 11:58Speaker 00

  Right, right.

• 11:59Speaker 01

  It can kind of emulate physics and it's like, that's amazing. Is the physics stuff solved, do you think?

• 12:03Speaker 00

  Yeah.

• 12:04Speaker 01

  Like what else is, you know, what's done and then what's to be solved?

• 12:07Speaker 00

  Yeah, I would say physics is a hard thing to solve forever. It's close. I would say it's close enough, yeah, but a year, six months ago, a year ago, a few years ago, you had Will Smith eating, you know, pasta. It was a disaster. And then even last year, you had kind of these videos of like knives cutting off fingers and there were six fingers, you know, it was like, that's where we were. So I think physics, tons of progress. The ability to do photorealistic quality, very huge progress. The ability to kind of do jump scenes and jump cuts and different sort of camera controls, that's really coming into almost solved. There's paths to solve all this stuff. Still going to solve the efficiency and serving costs, I would say.

• 12:49Speaker 00

  And probably still have to figure out a little bit more of like, the application layer of this. Because I think this is another big opportunity, is we've seen, like a lot of other modalities with AI, you get kind of the model layer, you get kind of the tool layer, and then the real value, we think, is in this application layer. And so I think that's really interesting to rethink workflows around video. And I think that's pretty wide open right now.

• 13:13Speaker 01

  Do you think that models are capable of, you know, even having video that is malleable at the application layer? So for example, if I want to have character consistency between scenes, are the models even capable of that? Or I imagine you want model sterability in order to be able to kind of work with it at the application level? Like, what is model readiness? And what's required in order to be able to do magic at the application?

• 13:35Speaker 00

  Yeah, so I was talking to a couple of AI filmmakers this week, and what they're really interested in is exactly what you're saying. Character consistency, scene consistency, camera control. It's almost like we need to build an AI camera. You think of some of the cameras that are kind of filming us right now. This is sort of like decades of technology that's kind of been perfected for a certain sort of input-output. And I think we're on the verge of kind of needing to create a new AI camera. And when you do that, you can generate infinite number of scenes. You can generate like, oh, you're wearing a red sweater, now make it blue. And not just in that scene, but in like a whole two-hour film. So there's all kinds of ways that we're starting to see these prototypes that we're

• 14:17Speaker 00

  working on too internally, where this is here, like it's coming. Where kind of entire, I think, things that used to either be too expensive, or too timely, or required a certain skill level. We kind of talk internally in the team about how do you kind of lower the bar and raise the ceiling? And what we think about that when we're building products is how do you make something more accessible? Or how do you make like the pros take it and just blow, you know, the quality out of the water and make it incredible stuff? So that's what we're seeing with video. It's kind of right at that point where both are happening.

• 14:51Speaker 02

  There's an interesting tweet from, or post from Paul Graham recently on this idea, I think of, based on this pace of progress, he's like, you sort of want to be building things that kind of don't quite work.

• 15:02Speaker 00

  Yes.

• 15:02Speaker 02

  And are way too expensive.

• 15:04Speaker 00

  Yes.

• 15:04Speaker 02

  Right?

• 15:05Speaker 00

  Yes.

• 15:05Speaker 02

  Because they're gonna work.

• 15:06Speaker 00

  Yeah.

• 15:07Speaker 02

  And their cost is gonna come way down.

• 15:08Speaker 00

  Yep.

• 15:08Speaker 02

  Right? And so I would imagine that has applicability for you guys too, particularly in video.

• 15:12Speaker 00

  That's exactly how we do it. Yeah. I mean, right now, I don't know off the top of my head, but each video eight-second clip generated is obscenely expensive. But we're basically building for a world where this is gonna be like, you're gonna generate five at a time, not even think about it. One of the actual principles I've kind of learned just over the last few years, working on this AI stuff, is Make sure your product is aligned to the models getting smarter, cheaper, faster. And if your core product value prop can benefit from those tailwinds, you're in a good spot. If any of those are not right, question your existence. That would be my summary takeaway on that. I like that.

• 15:50Speaker 01

  How far do you think we are from having economics of video generation that are right side up? Where it costs less to generate the thing than the economic value of generating it?

• 16:02Speaker 00

  Yeah. Oh, wow. This is tough. This is a prediction you're never really sure about. I don't know, but I would say one thing we're seeing, just as we're modeling out a lot of costs, because we're starting to put VO into some of our own tools that are coming out, is we're probably going to need innovation on the business model side, in addition to just the product and the application layer. And what I mean by that is you could, our first thought was, oh, let's just make a subscription and then just charge per usage on top. That might be a way to do it. Another way to do it is when you talk to some of these creatives, whether they're in Hollywood, or even these AI filmmakers that are popping up, they're kind of like, okay, I want this output and I'm willing to pay this much.

• 16:43Speaker 00

  And it's kind of a pay per output, which you've seen in other cases that AI companies are starting to do some of this too. But for sort of film and video, that's a little bit how you'd think of doing a project if you were a producer. But now you're kind of imagining it at like the individual creative level, which is kind of interesting. So that's more of like, almost like an auction type model, potentially. So I think there's a lot to explore. I think we're probably though, you know, the pace things are moving, it's on the scale of like quarters, I think, where it starts to get interesting, as opposed to like many, many years. So that's, yeah, I think there's a path.

• 17:18Speaker 02

  You talked about the pace of progress a couple of times?

• 17:20Speaker 00

  Yeah.

• 17:21Speaker 02

  Do you think it's accelerating? You have a unique view in the DeepMind, and let's use that as a harbinger for some of the others.

• 17:27Speaker 00

  Yeah, yeah, as a proxy. Yeah.

• 17:29Speaker 02

  What, where are we at? Are we accelerating? Are we on a crazy trajectory and maintaining the same one?

• 17:37Speaker 00

  Yeah.

• 17:37Speaker 02

  I'm interested.

• 17:38Speaker 00

  Yeah. I keep thinking it will slow down and it's never slowed down in the last three years. So, you know, you think, oh, pre-training might be plateauing. Inference time compute, a whole nother horizon opens up. And I think there is so much, there's an author on the team we actually hired, his name's Stephen Johnson. He co-founded Notebook LM when we first brought him on. He talks about this notion of like, there's adjacent possibles. He has this really interesting book on the history of innovation. And I feel like right now, it's like you walk into this room and there's all these doors that are opening up into these adjacent possibles. And there's not just like one room and one door. It's like one room, like it feels like 30 doors that you can go explore.

• 18:23Speaker 00

  So I think that's what it feels like on the inside.

• 18:26Speaker 02

  I love that visual of the rooms and then the adjacent possibles. I'm going to steal that and maybe take it and call it my own.

• 18:34Speaker 01

  Plastic VC over here.

• 18:38Speaker 01

  What do you think the future of video consumption looks like for us as consumers? Like, am I still looking at Hollywood style feature films that are created by Hollywood Studios, just done a lot more cost efficiently? Am I looking at a piece of content that's dynamically generated to what you know about me? And it's only for me to watch? Am I like, what, what, what do you think the future of consumption is as a consumer?

• 19:00Speaker 00

  So this is one of those that could go in spite in many different ways, I would say. I'd say some of the things we're excited about and what we see. So I think the future of entertainment is way more steerable. So right now, you think about, you sit on your couch like this and you maybe scroll through something or whatever you cast it on, you bring it up on the TV. So it's going to be way more steerable where you can kind of interject if you want and maybe take it certain ways. We think that's one area. We think another is personalization, like you said. If you think today about YouTube, TikTok, any of these algorithms that can kind of figure out, this is what you're interested in. Imagine that, I think, way more extreme, that could be kind of fine tuned to sort of what you want to share with the model. I think the other bit is a lot of this, I think, is going to be generated on the fly.

• 19:43Speaker 00

  So another theory we have is that just like there was a rise of kind of a creator class couple, whatever, 10, 15 years ago that powered YouTube and the rest. There's gonna be a shift, or maybe it's a different set of people that we think of as like curators, where you curate stuff and you work with the model to maybe create things. And I think another loop in that is how you can remix all this. And so that's another big part of what we see in the future of entertainment is that there'll be like, "Oh, I kinda like that, but then make it more like this." And if you think, you know, at some level, the cost, the time, the skills required of this is literally maybe just like tapping a button or just describing it, and you get kind of different versions. That's kind of where we see some of this going.

• 20:26Speaker 00

  It will be really interesting to see if like some of these same percentages hold. Like, we know today that a lot of times, a certain percentage, like 90, 95%, just consume from platforms, and you have a very small creator class. So like, will that balance change? But I see a totally different ways you could think about content platforms that have some of these native controls. Like, for example, will we expect UIs that have a join button? Where, you know, today our UIs maybe have a play, pause, whatever. Save, bookmark something, star, heart it. Like, will there be like new things where you join and they're like, "Oh, hey, Sonya, Ravi, what do you wanna talk about?" Do you know what I mean? And I think like, that's totally possible.

• 21:09Speaker 00

  We're building that in the Notebook LM today. So that, you can imagine, play it forward. You've got avatars or human-like characters or not, with lip reanimation, voice cloning, all that can come together in sort of new ways, I think.

• 21:24Speaker 01

  Do you think movies and games start to blur?

• 21:27Speaker 00

  Yeah, I think that's a real possibility. Yeah. There's a whole interesting intersection that's happening right now between movies or video content, games and sort of world-building, and 3D. And it's really unclear to us right now where that's gonna go, but there's so many areas right now where we're seeing learnings from each, and even down to some of the training techniques, we're finding things like that.

• 21:51Speaker 01

  So actually, that's something with my questions. Like, if you look at all the companies building generative video models right now, some people are kinda going straight from the, you know, the pixel stream, so to speak? And some people are going from the 3D angle with the idea that, you know, to really do video right, you need to get 3D. Do you have an opinion on that?

• 22:07Speaker 00

  Yeah, we've actually got bets on both sides right now. I don't know. I don't know. Get this. Yeah, we're hedged on this one. So on the 3D side, we have this project we got started where we basically said, like, take six pictures of a sneaker and create a 3D spin of it. And we put that on search. It's been really great and it's amazing how it fills in the details. But I think what's interesting is we've been going down that path, something like VO2 shows up. Now you don't need six photos anymore. You need like two or three. And you can basically do like an entire product catalog, like every product that's ever been indexed at Google, just overnight, sort of can create it. So now you've got a 3D object, basically

• 22:48Speaker 00

  of any object, bookshelf, chair, whatever, from any angle that you can pan tilt, zoom, relight, and now that's like an object that you can drop in anywhere. So that's kind of the 3D angle. From the video angle, it's interesting, like kind of the world building. We had this little prototype we built. We were like, wouldn't it be cool if you could recreate landing on the moon for like every classroom and like give teachers a tool where they could put the kids in the like, you know, lunar module as it's coming down. So we built this thing. It's kind of terrifying, actually, because we also built a little side panel where you can inject problems where it's like, oh no, something's on fire in the back. They like simulate things. We had a little fun with it. But that was interesting because the models you could say like, look right. And it would actually

• 23:34Speaker 00

  fill in the details. And so you start to get this, that's where it feels like it's kind of blurring. And I guess why we're hedging on both sides right now. Yeah, we're not sure.

• 23:42Speaker 02

  2025, everyone's talking about agents.

• 23:44Speaker 00

  Yes, yeah.

• 23:44Speaker 02

  Computer agents.

• 23:45Speaker 00

  Yeah. You just said it three times.

• 23:49Speaker 02

  Exactly.

• 23:49Speaker 00

  Probably being a VC again.

• 23:51Speaker 02

  I've been called a VC twice today. This is a very big insult. Can you talk to us about Google Mariner?

• 23:58Speaker 00

  Yeah, yeah. So Mariner is one we put out in December last year. This is a fun one, actually, because we started seeing this capability developing in the model. We're trying to understand if you could let these models control your computer or your browser. What would happen, good and bad? And so that was a good example of a project where we went from, hey, this capability is kind of showing up. Let's put it into, right now it's a Chrome extension, just because it was quick to build. Idea in people's hands, 84 days. Very fast, very fun. A lot of memories made on that. But I think what's interesting is you're seeing both across Anthropic, OpenAI, obviously Google, and a bunch of other startups in the space are all hitting on kind of the same idea that models are not just about maybe knowledge and information

• 24:43Speaker 00

  and synthesis and writing. They can do things and they can scroll. They can type. They can click. They can not only do this in one browser in one session, but like an infinite number in the background. So I think with Mariner, what we're really trying to pursue is like, of course, there's the near term thing of like, can it complete tasks in your browser? But the bigger thing is, what's the future of human computer interaction look like? When you have something like this, kind of not just one of these things, but basically like an infinite number kind of at your disposal. And so that's what we're chasing with that project.

• 25:19Speaker 01

  What do you think the ideal use cases are? Maybe even in the near term for Mariner? Because I think all the demo videos I see, not necessarily from Mariner specifically, but with computer use more broadly are, you know, here, have this agent go book a flight for me or go order a pizza on DoorDash for me.

• 25:34Speaker 00

  Right.

• 25:34Speaker 01

  Like, that's nice, but like, I like doing those things.

• 25:37Speaker 00

  Yeah, yeah, yeah. You're pretty good on those on your phone.

• 25:39Speaker 01

  Booking a flight is one of my, one of my delights in life. And so what do you think are the killer kind of consumer, consumer use cases?

• 25:48Speaker 00

  Yeah, well, that's what's interesting. It may not be consumer. It may be enterprise. And one of the things we're seeing when we do all the user research right now on Mariner, because we have it in Trusted Tester and people are playing with it and giving a lot of feedback, is it's really these high toil activities. Toil is kind of an old fashioned word. It doesn't get used a lot. But this is when people talk about it. It's like, this is what makes me grumpy. And this thing is helping me solve it. But what's interesting is a lot more of those are showing up on the enterprise side. Just to give you a couple of examples from yesterday, we were hearing from one of the teams and they're basically, they have this co-browser use case. So imagine you're in like, Call center somewhere, some customer calls in. They right now have this very complicated way.

• 26:32Speaker 00

  The agent in the call center can like remotely take over your machine that's not working, browse through things and do something for you. They were like, we would love to have Mariner do this. Um, and that's like a way, another one we heard, which was kind of interesting was people, they're like part of a sales team or something. They have, take a customer call, then they've got all these next steps they need to do, and they just want to fan that out. And it's often updating different systems that are all probably, I don't know, some SaaS subscriptions they're paying everywhere. And they're just like, the UI is clunky. It takes a long time. I just want to send Mariner to do all this. So these are the kinds of things that are kind of interesting that are just naturally coming up. On the consumer side, I don't know. Have you found one yet in your mind that you like?

• 27:15Speaker 00

  Cause I, we're, we've got a few, but I, it's, I'm curious.

• 27:19Speaker 01

  I'm thinking, I'm trying to think what the toil I have in my everyday life.

• 27:22Speaker 00

  Yeah.

• 27:23Speaker 01

  Talking to Robbie. I'm kidding, I'm kidding. Talking to Robbie is the best part of my day.

• 27:29Speaker 00

  Yeah, I appreciate that.

• 27:32Speaker 02

  I think that, but I like the framework, even if we don't have the exact use, the framework of like, what are the things that are the heavy lifting that you don't enjoy?

• 27:41Speaker 00

  Yeah.

• 27:41Speaker 02

  Right throughout the day?

• 27:42Speaker 00

  Yeah.

• 27:43Speaker 02

  That take up time away. And I do think that that was actually the same logic that yielded things like DoorDash or Instacart.

• 27:49Speaker 00

  Right, right.

• 27:50Speaker 02

  Um, you see how I had to get Instacart in there?

• 27:52Speaker 01

  I do.

• 27:53Speaker 02

  Okay, good. I'm just making sure that that was there. Um, on the enterprise side, when you think about it.

• 27:58Speaker 00

  Yeah.

• 27:59Speaker 02

  Um, how are you testing that? Are you testing that with existing, you know, uh, customers? Are you testing that with Google Cloud customers? Like, who are the enterprises that you guys will actually like test things with?

• 28:09Speaker 00

  Yeah, so in that case, we kind of go across big and small. So there will be some cloud customers. We have a lot of cloud customers who always want the latest and greatest. Just give us that. They have like labs equivalents inside their companies, right? So those are awesome test beds. We also work with a lot of startups. Um, and I mean, if there's others listening to this that are interested, let it like, DM me, let me know, like, cause we're always trying to learn kind of from different sides of the market. What I found too, building products over the years is it's very common. Everyone talks about product, market and fit. You'll know it when you see it and all that, which is true. But at least for me, I've always felt in the first part of building products, you iterate a lot on the product and sometimes you forget to iterate on the market. And finding the right market side is also just as important as the right product.

• 28:53Speaker 00

  And you have to connect those two. And so I think that in these early stage things with Mariner, that's where we are. It's like, is it possible for a computer to, like, an AI model to drive your computer? Yes, that's a huge new capability. Is it accurate? Sometimes. Is it fast? Not at all yet. Like, that's kind of where we are in terms of the actual kind of use case or the capabilities. And then it's about finding the right market. But yeah, to answer short, it's kind of, in these early days, we do lots of stuff really quickly. And what I kind of coach our product managers on and other people on the team, because we have engineers and UXers, they all go to these sessions, is like, don't look at the dashboards. It's too small numbers right now. Look at their eyes. Like, look at the customer's eye.

• 29:38Speaker 00

  And when you show them stuff, do they light up or not? You know what I mean? And like, that's kind of the signal you're following. It's way more art than science at this stage.

• 29:46Speaker 02

  Can we go back for a second, just to the context point? Because I was thinking about this vis a vis, like, you working at Google, right? And you talked about bringing your own, you know? Is there a world where someone can just opt in of like, Google knows a lot about me, right? Already, you know, my searches, my Gmail, my calendar. Is there a world where you can just sort of opt in and be like, I don't want to bring it all now. I just kind of want you to use what you got and make magic, right? Is that something that could happen? Because Google's uniquely suited to be able to do something like that, probably more so than anybody. Yeah. Is that something that you guys can play with in labs or have a possibility for? Or is that not possible?

• 30:24Speaker 00

  We do some, more kind of internally with some of our own, like, data on the team, right? Where I've opted into a lot of things, where it's just like, take it all. Like, let's make good stuff. But I think you'll see some of that come through in the Gemini app too, where you can link different things. But I think it's actually an area that's like actively kind of being explored too. Of like, what types of data is like the most interesting and the most useful? And of course, also the right controls where people feel like, okay, I'm not just giving it away. Yeah, so I think that is an area though, that we do experiment on some. But I'd say right now, a lot of the experiments are more on our own stuff, as we're trying to figure out.

• 31:03Speaker 02

  You're gonna have to tell us separately some of the things that you could have done now, now that they know everything about you, you know? Like, what is the magic that can be created for you?

• 31:11Speaker 00

  Yeah, I think certain things that immediately come to mind that are pretty powerful is you can... You can see things like in my own data, I feel like I have a second brain. There is a true, like, there's always been this vision of a second brain and tools for thought and all this stuff. And I feel like you can get pretty close to that. And I think the Gemini model specifically is really good at long context, the ability to have this like impressive short-term memory. And so Gemini 2, that's an area we're really trying to exploit right now, like how to use that.

• 31:40Speaker 01

  On Mariner.

• 31:41Speaker 00

  Yeah.

• 31:41Speaker 01

  Similar questions to what I asked on, on Vio.

• 31:44Speaker 00

  Yeah.

• 31:45Speaker 01

  When do you think we'll have computer use that is accurate enough and is fast enough to do some of these use cases you talked about?

• 31:52Speaker 00

  Yeah, that's another one. It's kind of hard to tell at the pace though, right now. I mean, not just inside Google, but what you're seeing from some of the other labs too. They're on like about an every month or two rev. So you can imagine just this year, we're going to see four, five, six revs of each of these things, right? Again, that's just what we know is happening. Um, I think the areas that are a little bit trickier, harder right now is how the computer like finely or precisely navigates like the X, Y coordinates almost. You almost want like a lat long of your screen. And that's still kind of really interesting jagged edges on that, I would say. The other big area I would say is like this, it's more of a human thing.

• 32:35Speaker 00

  Like, when do you want the human involved or not? When do they want to be involved or not? And kind of creating the right construct almost is like, hey, I'm about to buy something. Oh no, I want to know about that. Or I'm okay for $5, but nothing more than that. Do you know what I mean? And so there's a whole bunch of almost like hardcore like HCI like research and like really going deep on the empathy of like how you set those controls. That I don't think any of them, including the Google Mariner one right now, we don't have. I mean, we do certain very blunt things like, don't buy anything. Don't consent to any toss. You know, like, there's sort of like crude things right now that you can do. But I think people are going to want a more fine-grained way. So these are some of the things that are, I consider more unsolved.

• 33:21Speaker 00

  Again, that principle, just banking on the model's going to get smarter, faster, cheaper. Uh, and you're going to get like four, five, six or seven revs this year. Uh, yeah.

• 33:31Speaker 01

  Okay, I have a meta question.

• 33:32Speaker 00

  Yeah.

• 33:33Speaker 01

  How come all of the research labs converged on computer use at like, as far as I can tell, the same exact point in time? Was that an accident? Was that just all the technology happened to converge at the same time? Like, what happened there?

• 33:44Speaker 00

  It's a good question. I mean, this is, I don't know the specifics there of each of the other labs, but I would say, you know, when you read about the history of innovation, and there's like, all kinds of things on this, there, it's not uncommon that discoveries kind of happen around the same time. And I think there's kind of a new paradigm now with these models. And I think lots of people are seeing the potential in certain ways. And I'm sure there's also, I don't know, people changing labs and other things that are cross pollinating all these ideas too. But it does feel like it's one of those, is kind of how I'm interpreting it, is like, I think similar with coding, right? You saw, there's already, even the agent stuff right now, there's lots of this stuff kind of bubbling, um, which makes it really fun, but also keeps you on your toes, right?

• 34:29Speaker 00

  Because this is kind of the underdog mindset here.

• 34:31Speaker 02

  Are you gonna hire any other authors? The reason I ask is I was thinking about, I think Matt Ridley is the one who's written about some of these things, about like adjacent innovations.

• 34:40Speaker 00

  Yeah.

• 34:40Speaker 02

  And you have Stephen Johnson. Maybe, why did you hire Stephen Johnson?

• 34:43Speaker 00

  Yeah.

• 34:44Speaker 02

  How did that happen?

• 34:44Speaker 00

  Yeah.

• 34:45Speaker 02

  And are you gonna think about other people that don't have obvious backgrounds that you would bring into labs?

• 34:50Speaker 00

  Yeah, yeah. So, um, the quick story on Stephen was, um, the guy who kind of restarted Google Labs, this guy named Clay Bevore, who's awesome.

• 34:57Speaker 02

  Our mutual friend.

• 34:58Speaker 00

  Mutual friend, exactly. And, um, he and I are big fans. We've basically read everything Stephen had written. And Stephen was a very interesting guy because for like decades, he's been in search of the perfect tool for thought. And so Clay, Clay cold emailed him. Uh, we were both subscribers to his Substack. We kind of messaged them and we're like, we love you. Will you come work with us? We can build the tool you've been wanting to build. Like, that's where it started, actually. And this was like, I mean, it was like, summer 2022. So like, before any of the, you know, ChatGPT moment or anything. And Stephen picked up the phone and he was like, yeah, let's do it. So he came in, he was a visiting scholar. The job ladder didn't exist. Uh, I had to go figure out

• 35:41Speaker 00

  With our HR person, how to create a role that he could take on. This is very kind of unconventional in that way. And then the rest is kind of history, obviously. I've read a bunch of Matt's books. I don't know Matt. He'd be awesome. So if he's listening. Yeah, exactly. He's listening. He can come talk to both of us. That's right. That's right. I would say we've done this quite a bit. So we've actually brought in musicians. I'm actually really, we're trying to figure out right now, like a, like a visiting filmmaker. That's cool. So it's kind of a model. Steven kind of pioneered it. He was the first one. That it's like how to bring in, it's a big value in labs of how do we co-create? We don't want to just make stuff and throw it out there. We actually want to co-create it with the people that are in the industry.

• 36:23Speaker 00

  And what we find when we do that is you actually get way beyond the like, oh, that's a cool toy AI feature. You get into the workflow. And if you're working with someone like Steven Johnson, who's written, you know, dozen plus books. There's a certain way he thinks about and almost like a respect for like the sources and the citations. All that stuff comes through a notebook LM. And we're doing similar stuff with music and video and imagery and other stuff. Yeah.

• 36:47Speaker 01

  Is the goal to create net new products that you can take from one to a hundred to a billion standalone? Or is the goal to, you know, find product market fit with things like notebook LM? And then really fold them into the Google mothership, so to speak.

• 37:01Speaker 00

  Yeah, it's interesting. So when we first started, I would say it was all about build something, graduate it. So kind of a traditional incubator sort of model. It's been interesting as it's gone along. We've done that some cases like AI studio and the Gemini API. We graduated and it's now in DeepMind and they're kind of running with it. Something like notebook LM, we're just going to keep in labs right now for the foreseeable future. Cause it's kind of a different creature, like it's only possible with AI. And a lot of the stuff we're working on now, I mean, we'll have to see how many of these we can put together that actually can kind of get escape velocity. But we're really interested in turning them into businesses and making them sustainable. And kind of, you know, that's been a lot of the, the focus actually is like take big swings. And that gets back to your point, a lot of these won't work.

• 37:46Speaker 00

  Um, because if you're just, if they're all working, you're not swinging big enough.

• 37:50Speaker 02

  Of course, yeah.

• 37:51Speaker 00

  So it's like trying to find that balance, but that's definitely, we start with kind of, could we make this a business, work backwards from that? And if we end up graduating it, that's still a good, good outcome for us. Another good outcome is we stop it and was like, cut the losses. We did our hundred day sprint or whatever, move on to the next thing. Yeah.

• 38:08Speaker 01

  You mentioned at the top of the episode that you try to do some top down thinking of, you know, what are the most interesting pools for us to be building in?

• 38:15Speaker 00

  Yeah, yeah.

• 38:16Speaker 01

  What are your predictions on the most interesting pools to be building in for 2025? Like, where are you hiring talents? Like, where are you sniffing around? Where are you co-creating with the DeepMind folks?

• 38:25Speaker 00

  Yeah, yeah. There's a lot happening with agents. There's a lot happening with video, some of the things we've talked about, computer use. But I think about those pawns a little bit different. I think about them, we have this doc called Labs is a Collection of Futures, and it's 82 predictions about the future, um, which is always dangerous to make one prediction about the future, let alone 82. But the thought experiment on the team where we got to this was, imagine you're in a room like this, the ceiling just opens up and this little capsule comes down. We all jump in it and it slings us into the future. It's 2028, you can get out, you get five minutes, look around, write down everything, and you're brought back to the present. And then write what you saw. And that's what this doc is, is so what's the future of knowledge look like? What's the future?

• 39:09Speaker 02

  Even though prompts are old fashioned, that's a pretty good prompt that you gave to the team. I was just telling you right now.

• 39:14Speaker 00

  Yeah, yeah. So that's, you know, we think about, we think about it at that level, at kind of a high level. So say something like, what's the future of knowledge going to look like? We think it's going to be one piece of that prediction, one of the 82, is that it's infinitely remixable. And anything that comes in can be transformed and become anything on the way out. If you believe that, then you take certain bets and you build products kind of with that future in mind. So that might be one of them. But I think like, going back to maybe some of the ones that a lot of people might be listening or building, I do think we're kind of at the moment for video. We're at the moment for very interesting agent stuff with the thinking and reasoning models. And I think there's also maybe something kind of under the radar right now

• 39:56Speaker 00

  a little bit. I still think coding has major leaps we're going to see this year. And so those would be some of the ones that are top of mind for us.

• 40:04Speaker 02

  Are you guys doing work on coding out of labs too?

• 40:07Speaker 00

  Yeah, we are. We are. So right now at Google, 25% of all the codes written by AI.

• 40:12Speaker 02

  Yeah, I saw that. Jeff Tean said that.

• 40:13Speaker 00

  Yeah, that's right. That's right. And that's up a lot in the sense of just how fast the progress is. This is an area that I think there's kind of two approaches you could think about, like how, again, think of lower the bar, raise the ceiling, right? How do you make coding available for people who could never write code before? Massive opportunity.

• 40:32Speaker 02

  Like Sonya, you know, like I've been coding my whole life. I mean, Sonya is some of that.

• 40:37Speaker 00

  Well, it's kind of interesting because some of the most interesting stuff happening here, I don't know if any of you have played with like Replit's agent stuff. Really interesting, right? Couple of weekends ago, I'm with my fourth grade son. We are struggling right now in our household to implement chores. We created a chore tracking app. 28 minutes, 45 cents. Done. We're daily active users. And so it's a way to kind of get into software in a world of kind of software abundance. That's really interesting. So we've got some stuff in that area. We're also interested in how do you take a professional trained SUI programmer and make them like 10x to 100x? And there's kind of, I think, interesting bets on both sides of that. Yeah.

• 41:22Speaker 01

  What do you think is overhyped in AI right now?

• 41:25Speaker 00

  Oh, that's an interesting question. I wish we'd move beyond the chatbot interface a bit. That's one area that feels like we're kind of reusing that in a lot of places, Google included. I'm also not sure.

• 41:40Speaker 00

  There's still a lot, I think, of like people jamming AI into stuff. Like AI itself is a bit overhyped. I wish we were a little more precise about how disruptive or like where to apply it. And so I think, again, we're trying to think a lot about like workflows, not just taking existing product and bolt on AI. So I think that's maybe a little, there's a race, like you're seeing the first generation of AI, put it in. And it reminds me a lot, actually, when I first started at Google, it was like right as the iPhone moment was kind of just happening and taking, taking hold. You know, when Steve walked on stage in 2007 and said, this is the iPhone. If you look at the App Store three years later, which is roughly where we are in this

• 42:20Speaker 00

  AI revolution, the App Store in 2009-ish, I went back and checked, websites that have been shrunken down to fit on your phone, flashlight apps and fart apps. These were like the highest top downloaded things that were happening. So I think we're kind of in this stage where the real stuff is going to start to come out kind of this year, next year, the next year. That's when you start to see the Ubers, the Airbnbs, the Instacart, the things that really change kind of how you do stuff. And so that's, that's kind of my thought on it.

• 42:51Speaker 02

  All right, then Sonia asked you the overhyped question. I'll ask you the under the radar, underhyped question. What are some areas that deserve more attention within AI?

• 43:01Speaker 00

  We talked about coding a little bit. Maybe just one other thought on that is I think if you can get code models that can kind of write code and self-correct and self-heal and migrate and do all this stuff, it just makes, you think the pace is fast now, that totally changes the curve. So I think that's a huge, I still think it's underhyped. Like it's hyped a lot, by the way. But I think as hyped as it is, it could be hyped more. That's one. I don't think we've fully internalized the notion of like, what does long context or like infinite context mean? It gets to some of your personalization questions potentially, but it also gets at some of the stuff we were talking about around, how can you make things like a mariner literally just keep going?

• 43:46Speaker 00

  And so that whole notion of long context, I mean, you see a lot from Google, but we're investing a lot in that because we think that's a strategic lever. That's important, especially as you get more agentic, chained together kind of workflows. Maybe another one, I think there's, there's not enough talk about taste. And like, I think if you believe the value is going to be in the application layer, if you believe there's going to be some percentage of AI slop, you can just see a few of these trends. And I think there's going to be a value in good taste and good design. And it doesn't mean it has to be human created necessarily, although I think there's going to be a high value on that too.

• 44:28Speaker 00

  As like human crafted content becomes more artisan. But I think that's another one, I would say. I think maybe related to that, it's like veracity and truth. And sort of what is real? Like, these are things that I think are going to become way more important than they already are today.

• 44:44Speaker 02

  I think that context point within there, I like really firmly agree with on like, what can happen with you, your infinite context point? Because if you think about the relationship in your life where you have like the most context, shared context, it's probably with your spouse.

• 44:58Speaker 00

  Yeah.

• 44:58Speaker 02

  Right? And if you think about that, what ends up happening is you can communicate with your spouse, literally with just like, like the flick of an eye.

• 45:04Speaker 00

  Yeah.

• 45:05Speaker 02

  And all of a sudden, they know exactly what you mean. They know it's time to leave the party, whatever it might be.

• 45:09Speaker 00

  That's right.

• 45:09Speaker 02

  Right? And you think about, that's the aspiration for what can happen with infinite shared context.

• 45:15Speaker 00

  We know that's the ceiling.

• 45:16Speaker 02

  Exactly. Right? And so you think about, you're like, think about how far away that is from now, where you're like typing things in.

• 45:22Speaker 00

  Paragraphs.

• 45:22Speaker 02

  About what it is. And your point of like, well, hold on, there's all these different ways you can communicate it. And that can get to know you better if it has memory. And so I, I think there's so much gold in there of it just being able to keep going.

• 45:33Speaker 00

  Yeah.

• 45:34Speaker 02

  Right?

• 45:34Speaker 00

  Yeah.

• 45:34Speaker 02

  But giving it the right context and whenever it needs.

• 45:37Speaker 00

  We think of any company that you all back, or even Google, like what's one of the most painful things? Is when a long-term employee leaves. Because all that context walks out the door. So I think it's exactly right, whether it's a personal relationship or a work relationship. Yeah.

• 45:50Speaker 01

  Okay, we're going to wrap with a rapid fire round.

• 45:53Speaker 00

  All right. Yeah, it sounds good.

• 45:54Speaker 01

  Okay. Favorite new AI app?

• 45:56Speaker 00

  Oh, I mentioned it earlier. I'm having a lot of fun with Replit.

• 45:59Speaker 01

  Love it.

• 46:00Speaker 00

  The new agent thing and on the phone. I think they're doing some really interesting stuff there.

• 46:04Speaker 01

  You know, one of our partners, Andrew Reid, is known for slinging, like, creating these amazing memes and sending around. It's now so easy to create an app. He just creates these all the time and sends them to me. They're really good.

• 46:16Speaker 00

  Yeah, we have this concept of disposable software. You use it once and you kind of throw it out after you're done with it. So, yeah.

• 46:24Speaker 01

  Okay, what application or application category do you think is gonna really break out this year?

• 46:28Speaker 00

  Video.

• 46:30Speaker 01

  Okay, recommended piece of content or reading for AI people?

• 46:34Speaker 00

  Ooh, that's an interesting one. Um, you know, this one's not a traditional AI pick. I think probably a lot of the listeners here, I was gonna say, over the break, I read a lot. And one of the books I picked up was, actually, is The Lego Story. And it's the history of Lego. And it's on its third generation of family ownership. I'd recommend that one. It's a really interesting, uh, yeah. Here's why, though. There's a pivotal moment in the company's history where they had 260 products. And maybe for a lot of founders that are listening, you can imagine your company could go in like all these different ways. You're trying to figure it out. And the grandfather, the CEO at the time,

• 47:16Speaker 00

  basically identified like the little building blocks, this is it. And he bet the company on it. And he bought these incredibly expensive machines. And so I think it's like, I like to read biographies a lot. And this was one that really stood out.

• 47:29Speaker 02

  Josh has incredible taste in books. And he has this wonderful reading list that he's been kind enough to share with me.

• 47:36Speaker 01

  Oh, no way.

• 47:36Speaker 02

  That's really wonderfully curated. It has this very good formatting as to when it's something you really gotta read versus not. And so you should, to all the listeners, you take Josh's suggestions seriously.

• 47:47Speaker 01

  I actually really want a great AI reading app. That's like my wishlist app. In part because I have terrible memory, but out of, out of everything I've ever read or listened to, which I think is a different set of things than all the books on the planet. Like, there's all these things that are kind of on the tip of my tongue and ideas that connect. But, you know, they're all kind of in an abyss and they're all pretty inaccessible to me. And, and so something that surfaces, some of those thoughts and ideas that I've had, things that I've read, you know, that next layer of thought I have from reflecting on two different things that I've read.

• 48:19Speaker 00

  And the connections probably across them. It's a good idea.

• 48:23Speaker 02

  I think even within that, like, just the hard copy version, the Kindle version and the audiobook version being like seamlessly intertwined, you know, like even just at the most basic level, you know, so that you can continuously pay attention to something that you like. And then we can get to the version that you said. Yeah.

• 48:42Speaker 01

  Request for startup. Okay. Pre-training hitting a wall. Agree or disagree?

• 48:46Speaker 00

  Ooh.

• 48:49Speaker 00

  Maybe lean agree. I think there's still stuff to squeeze out there, but I think a lot of the focus has shifted. Yeah.

• 48:57Speaker 01

  Nvidia, long or short?

• 48:59Speaker 00

  I don't give stock advice.

• 49:02Speaker 00

  Index fund.

• 49:04Speaker 02

  Do you ever sit with Demis and be like, look, as someone between us, we won a Nobel Prize. Do you ever start with that? Because, you know, that feels like something that's true. You know, between the two of you, there's one Nobel Prize.

• 49:18Speaker 00

  It's all one directional. That's Demis and John Jumper. Those are the people that won the Nobel Prize, not Josh Woodward.

• 49:27Speaker 01

  Okay. Any other contrarian takes, Nai?

• 49:30Speaker 00

  Any other contrarian takes? I guess maybe I'll leave it with this. I think we are kind of, one thing is like, what a time to be alive and building. Because I feel like there's this window where there's like so many adjacent possibles opening up. I think the second would just be like, I'd encourage people listening to like, really think about, of course, there's the models and who's winning and the back and forth. But like, what are the values you're building into your company? Because I think this is one of those moments where there's going to be like tools created that shape like follow-on generations. I think it's really important people think about that. And like, are you trying to replace and eliminate people? Or are you trying to amplify human creativity? I mean, there's like one that's like, you know, it immediately comes to mind when I'm

• 50:13Speaker 00

  thinking of video, for example. I'm on the side of wanting to amplify human creativity. But I think there's like, there are these moments that happen in our valley here where like things change and they change often for generations and they can change for good or bad. And so I would just encourage people that are in spots where you're building and you have this incredible technology that's only getting smarter and faster and cheaper to put it to good use and think about the consequences downstream.

• 50:40Speaker 01

  Thank you so much Josh for joining us today. We love this conversation.

• 50:44Speaker 00

  Yeah, thanks again.