GPU dev environments that just work

Creating and sharing reproducible development environments for AI experiments and production systems is a huge pain. You have all sorts of weird dependencies, and then you have to deal with GPUs and NVIDIA drivers on top of all that! brev.dev https://brev.dev/ is attempting to mitigate this pain and create delightful GPU dev environments. Now that sounds practical! Sponsors: Fastly https://fastly.com/ – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipFly.io https://fly.io/changelog – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog https://fly.io/changelog and check out the speedrun in their docs https://fly.io/docs/speedrun/. Featuring: Nader Khalil – X https://x.com/NaderLikeLadderChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: brev.dev https://brev.dev/ DISCOUNT for our listeners 🔥: Use coupon code “practical-ai-2023” for 5 hours of free GPU compute!brev.dev doesn’t offer credits often, so the credit redemption button is hidden by default. Go to this link https://console.brev.dev/org/ejmrvoj8m/settings?credits=true to expose the button. Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  There's a lot of optimizations around the GPU spend. So the way that it's being backed up for the volume, we're doing like intelligent backups, I guess, where we can back up just the amount of volume that's actually being used. So you're not paying for unused volumes, even when your instance is off. There's auto stop, making sure that your instances aren't costing you a lot when you're not using them. You can use brev scale, which lets you de-allocate the GPU or get a more powerful instance if you need it. So flexible compute needs without having to reset up or install anything. And there's the obvious benefit of not running a container locally if you're on a Mac that kind of like casually eats up like 20 gigs of RAM.

• 0:45Speaker 01

  Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive and accessible to everyone. Subscribe now if you haven't already. Head to practicalai.fm for all the ways. Special thanks to our partners at Fastly for delivering our shows super fast to wherever you listen. Check them out at fastly.com. And to our friends at fly.io. We deploy our app servers close to our users and you can too. Learn more at fly.io.

• 1:20Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm a data scientist at SIL International and I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

• 1:35Chris Benson?

  Doing good, having a good 2023. And this is going to be the best year for artificial intelligence ever.

• 1:43Daniel Whitenack?

  Yeah, well, I mean, it must be. Yeah, we finally did our chat GPT episode and that was really cool because I don't know if you saw, Chris, this is the first episode where we had, I think, like over 10,000 downloads in the first week. So thank you to our listeners. It's awesome to see that. We're glad that was useful. And we're going to keep the good content rolling right along because this week we've got something super practical, which I think everyone deals with what we'll talk about today. But we were privileged today to have with us Nader Khalil, who's the co-founder and CEO at brev.dev. Welcome.

• 2:22Speaker 00

  Hey, thank you. Thanks for having me.

• 2:24Daniel Whitenack?

  Yeah, so I alluded to like a problem that we all face, which is environment management. And like, I'm developing on this environment, I need to have these dependencies, or I use this environment, now I need a GPU, or Chris is on my team and he needs to replicate my environment. All of these sorts of things, whatever, you know, category you put those in. So how, how I guess in terms of, you know, you're digging into this problem now, but how did you get there? What started you along this path of really thinking deeply about dev environments?

• 3:01Speaker 00

  Man, we've had quite a twist and turn of a journey to get here. And yeah, I mean, the ultimate goal is just monotonous machine problems getting in the way of creative development. And that's, it's funny, when I, I went to UCC at Barbara, I studied electrical engineering and computer science. And when I moved to SF to work, I was actually building cloud dev environments at Workday. And I did that for two years, and in December 2018, actually just before that, I was getting a beer with a bar owner, and he was telling me how he had a thousand clicks on his Google Ads, but his bar was empty, other than me. And he shows me that his metrics on his Google Ads, he goes, "Make it make sense." And I realized, he had a really good point. Digital ads work really well for digital businesses, because if someone clicks on an ad that's an Amazon ad, you've entered Amazon storefront. There's nothing like that for the physical businesses like his.

• 3:45Speaker 00

  And so, he's just using a really bad medium. So, my co-founder and I, kind of like same co-founder with Brev, we pretty much realized, there was like a way for us to backdoor the Uber app. And so, we put tablets and Ubers and Lifts, and we let local businesses advertise on them. And if you tapped our screen, we would reroute your Uber to that location.

• 4:03Daniel Whitenack?

  Yeah, that's legit. Yeah.

• 4:05Speaker 00

  Yeah. You go out with friends for drinks, you see buy one, get one free margaritas, you tap the screen and we take you there. You get a free drink, our owner knows his ad work, the driver got a tip, everyone won.

• 4:15Daniel Whitenack?

  Perfect.

• 4:15Speaker 00

  And so, that was really exciting. I end up, that's what I quit my job to go do. We did that for like two years, completely bootstrapped. We like ran out of money, I poured my 401k into it. We got into YC for that, we got to like a quarter mill ARR. And essentially Demo Day was March 2020, which was right when the shelter in place happened in SF. And so, we got to see our 400 cars go to seven overnight, right actually the week of Demo Day. So, we didn't raise a dime, obviously, but...

• 4:41Daniel Whitenack?

  Oh, I feel bad for laughing, but I can't help it. Yeah.

• 4:46Speaker 00

  Have you seen that GIF on the internet of the raccoon with like cotton candy? And it's just like, "Where did it go?" It was like very much like eight March 2020 for us. But uh, it was funny because with a physical business, you have a physical fleet, right? We have physical operations. You imagine like physical hurdles being the hardest part of that. And in January 2020, we're starting YC. We're like, we got to like 15 K MRR. Things are working and we need to just three, four X the fleet. And uh, that was like really hard for us. We found out from one of our drivers that uh, Uber and Lyft have these parking lots half a mile from SFO Airport, where drivers go wait for these really valuable airport rides. So I go to the parking lot and Uber security kicks me out right away. They're like, I'm not a driver. So I'm like, okay, well, I'm a leasern.

• 5:30Speaker 00

  So I went to a gas station, I bought cigarettes. I light one up and just walk back on the lot because now I look like a driver taking a smoke break. And I got right past Uber security. I'm on this lot till like 4am talking to every driver. We four Xed our fleet that night. So like, there was never a physical hurdle that got in our way. But once we got those drivers live, everything else went to shit. We had like our advertiser dashboards really slow, like all these random problems. One of them was like the ads when they flipped on our tablets would just disappear and flash white. And if that happened at night, it's jarring. And so riders would turn off the screen and you lose revenue for the night. And so it was really funny having like really weird physical problems, but like we can sneak past Uber security and solve those.

• 6:11Speaker 00

  But like, no, when we have to like sit at our computers and fix something, it's like our dev environment slowing us down. And so it was almost like instantly when my co-founder and I, like when essentially the pandemic killed that business, my co-founder and I look at each other and those 20 days of January where we were trying to deal with our dev environment issues, we couldn't replicate these issues locally, just so many weird bizarre issues. We're just like shooting in the dark. Um, that was the only time with that business I had like a pit feeling in my stomach, like we've forgotten a Simon or something. And so we were just immediately, how do we solve our previous problems? And so we spent like a year and a half in Pivotland with a good North Star. We built a very heavy abstraction, I guess. It was kind of like what Replit is now. At the time, Replit didn't have databases, so you couldn't really build applications in it. So we had this.

• 6:56Speaker 00

  We essentially said, hey, if we force our dev environment opinions on you, you can't have problems we didn't already know about because we forced your decisions. And so you wouldn't have problems, it'd be a really smooth experience as long as you did everything that we supported. And so you get cron jobs out of the box and Tolia was already hooked up and a database was already there, but you have to use our version of Python for your APIs, things like that. And so it was an interesting experience in like the broader, like everything outside of a dev environment, when you need to start using, when you want to run tests and you need more tooling and those things aren't supported. So it was a great way to like plunge into the space, but ultimately we learned that a good abstraction is only good if it pairs well with the problem it's solving. And if you're good at solving problems, you're going to have new ones to solve, which means you'll need new abstractions or a flexible abstraction.

• 7:40Speaker 00

  And so that's when we kind of pivoted away from that and built like the current version of Brev.

• 7:44Daniel Whitenack?

  And a lot of what you described, I mean, I've never tried to sneak tablets on an Uber or something like that, which sounds like a really fun thing to try to do. And I love, I love that story. Probably a less fun thing for me in my life is like the general like arena of the very kind of specialized and weird dependency issues specifically related to like machine learning and AI sorts of environments. And the differences that people have between like trying to prototype something locally and then trying to scale it out in a reasonable way. Did that factor into your thinking when you were building this in terms of like these

• 8:25Daniel Whitenack?

  like data science people out here, this like explosion of AI tooling and all of that? Or was that something that came along the way as you were kind of going in this journey and thinking about like, what kind of problems these abstractions were thinking about, what kind of problems these solve?

• 8:41Speaker 00

  Yeah, so it's definitely something that we learned along the way. We initially started by trying to solve our own problem. We at Brev exclusively use Brev for all of our own development. It's just a much, kind of to your point, right? You're not dealing with environment issues. We upgraded, we have a blog post about when we upgraded from Golang version 1.17 to 18, it caused a memory leak, but our co-founder fixed it, his environment. And so when I wanted to update my environment, I just reset and I'm on the latest. And so being able to just move your environment that way, it is really, it makes everything a lot easier. What we've learned is that some of our power users were AI developers because AI dev environments are really complicated. And they specifically asked us to support GPUs. And when we started to support GPU instance types, it just kind of opened our eyes to how many, I guess, this kind of

• 9:24Speaker 00

  raw DevOps problems there are within the ML ops space. You know, GPUs are really expensive. A lot of times the GPU is sitting idle. If you need to do some sort of development, you might spin up a GPU just because there's the off chance you do some GPU development right now, but a CPU would have sufficed. So the way Brev works is it's, the idea is you can move your dev environment between different instances. So you can, if you're not using the GPU, de-allocate it and just go to a really cheap pennies per hour CPU instance. And only when you need the GPU do you turn it on. We also have AutoStop, so I learned from Workday. They were burning a lot of money every month because developers forgot to shut these instances off. This also happens from individual developers. So if you don't use your Brev instance, we'll automatically power it down. You can start it again from the CLI and it's just, and it's back up and running.

• 10:08Speaker 00

  So Brev is a CLI that makes it really easy to spin up these dev environments and we connect your local tools to that remote instance. So we kind of, the CLI wraps SSH. So all you have to do is run brev start and start coding and not really have to worry about the actual like environment issue.

• 10:22Chris Benson?

  That sounds really cool. Let me ask you a kind of a baseline question that as I'm learning about how you've done this, but I'm starting kind of from where I'm coming from and probably where more than a few of our listeners have. Like I'm used to, you know, using Docker and, you know, getting in a container and it has, you know, access to an Nvidia GPU, kind of the way a lot of folks are doing it. Can you kind of tell us a little bit about what the difference is between that, kind of that classical approach that a lot of people use? And in what ways are you differentiating and stepping up from, from that into brev.dev?

• 10:57Speaker 00

  Yeah. And can you explain to me maybe like how, where are you running this container? How, are you running this on your machine? It has the Nvidia GPUs. Yeah.

• 11:05Chris Benson?

  You have to have a set of images that you have, you know, set up. There's a bunch of configuration ahead of time, which I know I don't have to do on yours. But essentially, I'm having to say, okay, I have a GPU available in someplace on the network or maybe in the cloud, and I'm going to do those configurations. And then maybe I'm on my laptop, maybe I'm on a server. But a lot of people are, you know, logging into containers to do the work and then trying to move the container around and be able to access those resources from different locations. I know that I'm starting from that because it has some good things, but it also has some real pain in the butt aspects to it in terms of having to make it all work. And so I'm kind of wanting, it sounds like what you're describing upfront is a really good user experience.

• 11:49Chris Benson?

  And so I'm trying to get a sense of like what the differences are in the two.

• 11:54Speaker 00

  Yeah, so I think at a minimum, if you want to just run a Brev environment with or without a container, whether or not you have that set up, the way we kind of handle this is with a simple bash script, knowing that every Brev environment is running the same version of Ubuntu. We have the specific version listed in our docs. Running a bash script is, bash is ubiquitous. It's available, you can make, and you can run anything on it. So, or you can install anything with it, rather. So you can start with just a bash script if you don't want to run a container, if you just want to like try something and have that run. So we leverage this a lot for some of our templates. If you have a bash script committed to your repo that has setup instructions, Brev can automatically run it when you spin up an instance. So you create a new environment, you give it the Git repo and the path to the script that you want it to run, and that script will get run immediately for you when the

• 12:39Speaker 00

  instance is created. The user experience is creating the new environment, whether in the CLI or through the UI, setting the path to that setup script, or you can also just start with one of our templates. And then from your terminal, you run brev open and we'll open up VS code connected to the remote instance or brev shell if you, we support Vim, Emacs, JetBrains, whatever IDE it is that you want to use or code editor. And then if you do have a containerized workflow, anything that you were going to run in your terminal, if you're going to run docker compose commands, if you're going to run COG, if you're using replicate, anything that it is you're trying to run, you can just put in the bash script and know that that's going to reliably run for you or someone else that you're sharing this with. But I think the big thing here is There's a lot of optimizations around the GPU spend. So the way that it's being backed up for the volume,

• 13:24Speaker 00

  we're doing like intelligent backups, I guess, where we can back up just the amount of volume that's actually being used. So you're not paying for unused volumes, even when your instance is off. There's auto stop, making sure that your instances aren't costing you a lot when you're not using them. You can use brev scale, which lets you de-allocate the GPU or get a more powerful instance if you need it. So flexible compute needs without having to reset up or install anything. And there's the obvious benefit of not running a container locally. If you're on a Mac, that kind of like casually eats up like 20 gigs of RAM.

• 13:52Daniel Whitenack?

  I actually, so I haven't used it a lot. I have to be honest, but I did spin up a couple of environments in brev.dev leading up to this conversation because I wanted to understand a little bit more about it. And it was really fun. Like Chris was saying, like, I think it's true that the sort of dev and onboarding experience is really nice. And I was using like the UI configuration. And the experience I had was that, and I don't know, I'm kind of curious, like what you've heard from other users, I guess, is my question. Because my experience was similar to like, okay, I created the dev environment like with the UI. It's a little bit different UI than I'm used to, but there's like familiarity with certain of the things, right?

• 14:36Daniel Whitenack?

  I'm pointing it to a Git repo. I'm maybe defining, like you're saying, like a startup script or something like that. I'm naming it. Okay, it's creating this thing. I added GPU, whatever. There's similarities between that and like what I would create in an instance in the cloud. But then I have this dev environment. And I think the point where like something switched in my brain was I was, you know, local in my terminal. And I, I actually even forget the command now, but like brev, brev open, the environment name, that's what it was, brev open. And it, it just popped up VS Code. And then I realized like I had my VS Code open and I could open like a terminal in VS Code, but that was running in the environment

• 15:21Daniel Whitenack?

  that I created remotely. So that's where like things switched in my brain. Like, oh, I'm now using that environment that I set up and I could share that environment with someone else and then they could pop open their code editor and see this. I'm curious, other people that you've talked to, people that have started using it, where are those light bulbs going off for them? And what are the things that they're really like getting excited about, I guess?

• 15:45Speaker 00

  I mean, I just did a slew of user interviews and the first question I always ask is what does Brev do? It's always really exciting to hear that from someone before I have an opportunity to like kind of accidentally influence that conversation. And the biggest things we hear is that Brev is the most delightful or easiest experience to run anything on a GPU in the cloud. So that's been a lot of our focus is, you know, dev environments are kind of the thing that gets in the way of what you're trying to do. And so that's been our focus from the beginning, but there's a lot more complicated workflows, especially with AI. And just the the dramatic cost, like we have one user whose Google Cloud bill was like about $280 with just running on their GPU instance, but with something like Brev scale, they brought it down to about 25 bucks.

• 16:29Speaker 00

  I think their exact thing was like 27 or something dollars. And so that's like a, you know, 10x reduced cost, just because that GPU was sitting idle while they were like actively coding and building things. So I think our goal is just to have something that is a much more delightful and really simple experience, but also saving a lot of money. A lot of what we're focused on right now is integrating with other clouds, you know, we to get this far have been just been built on AWS, but we're partnering with like Lambda Labs right now to support their GPU instances because they're a third of the cost. And we're leaning deeper into actually a container strategy, which will let us provide kind of like start and stop across clouds, which I think will be really exciting. So this is something that's we're getting ready to release over the next two weeks. And I'm going to start kind of talking a bit more about, but

• 17:11Chris Benson?

  Actually, I was just gonna say you can go ahead if you want to dive a little bit into that right now. Cause you really piqued my interest with that. So if you don't tell me now, I'm gonna pester you later.

• 17:22Speaker 00

  Yeah, well, um, the way that we're approaching it, I think is we're going to, and it's a bit experimental still right now, but we'll have something out within two weeks. Our team has pretty quick velocity. We're a small but potent and passionate team. And so we really want to be able to support start and stop across any, uh, anywhere that there's a GPU available for us in the cloud. And it might not be at a large data center. It might be at a small one and that's okay. If it's a cheap GPU that's in a region that's not going to introduce a lot of latency for you, you should be able to leverage it when while we have access to it. And if it's rugged from us, if we have, if you stop your instance, you should be able to start it again. And it might not be on the same instance in the same data center, but that's okay. We're really just trying to optimize on, you know, GPU itself, a GPU is a commodity. It's like you just want the cheapest one and you want to be able to run your code on it easily.

• 18:06Speaker 00

  And so that's, yeah, in like two weeks, I think we'll have a pretty exciting launch on that.

• 18:09Chris Benson?

  That sounds pretty cool. So there's another aspect of that that's got me thinking with you looking at multi-cloud and you kind of said it could be a small data center, it could be, you know, I'm getting the impression there can be a lot of diversity potentially in what you're targeting for getting your GPU. What are some of the kind of considerations somebody might have for if they're using brev.dev? Like, how might they decide? And is there any strategy yet other than just kind of whimsical on saying, hey, I want to go with this one or that one? Is it just a cost thing? Or could there be other considerations that you guys have thought about in terms of being able to provide, you know, like going to a small data center here at this company rather than the big AWS one in Northern Virginia

• 18:52Chris Benson?

  over here? Any thinking around that?

• 18:55Speaker 00

  Yeah, so just to clarify on the kind of whimsical approach, are you talking for us as we try to find GPUs that we can?

• 19:02Chris Benson?

  No, for the user perspective, because if I'm, am I correct in thinking they can kind of choose where to target on that? Or is it something you're doing behind the scenes?

• 19:10Speaker 00

  Our goal is to make it really easy, but expose as many options to a user as they want. So for example, we'll default right now to a region that makes sense, but you can always open up the region and pick one that you would like. Again, right now, we're only working with AWS, but that'll change really quickly, like in these next two weeks. So we always want to make it an option for a user to see transparently where their instance is coming from. There's, I don't think, reason for us to hide that. However, we do have an option for you right now to connect your AWS account. And what I've noticed is only like two users have bought, like two individual users, not teams, have used that. And I think what that means to me is the specific location of the GPU doesn't really matter. It's just like, hey, I want to run this on an A100. Go run this on an A100.

• 19:50Chris Benson?

  Gotcha.

• 19:51Daniel Whitenack?

  So one of the things that is sort of a question running through my mind is, I thought it was really powerful. Like, when I opened up the environment, I had the environment, I could run stable diffusion or whatever, because I had a GPU in the background, I had enough memory, like all those things, that's really nice. And I could see how that would allow me to sort of understand the environment that I'm like eventually building towards in terms of what I want to release in production. And I could share that environment with other people. What would be like? Like, from your perspective, as both the founder and creator, but also a user of Brev, what is like the workflow that you've seen work in terms of going from that local dev

• 20:33Daniel Whitenack?

  and sharing local dev environments with other team members towards like something you would run in the same type of environment in production? Like, okay, I've now used a Brev environment to like, figure out how to run this, you know, fast API code that serves my model or something like that. And now, I want to run the same type of environment, but I want to deploy that in my AWS or something. Like, how, how did that work? And how does like Brev, you know, factor into that, I guess?

• 21:04Speaker 00

  Yeah. So it's really funny, right? Think about how many times you have to kind of do like the same redundant work. And all of this being like, not the thing you're trying to actually build. So you go and install everything so you can work it on the dev environment. Then you go and install everything so you can go run, run your tests if you have a pipeline. Then you go and install everything so that you can deploy everything in production. And like theoretically, we've already done this. And so I'm good friends with the team at banana.dev. We love working together. I think our products are both very synergistic. And something that we're working on is if somebody has a Brev environment, they should be able to click a button and it deploys on banana. It's a serverless GPU for production, right? That's, that's the, a helpful way to look at this is, there's two types of compute. There's interactive compute and non-interactive compute. If you're deployed on production, that's a non-interactive compute, right? Your API is up and running.

• 21:49Speaker 00

  You don't need an active shell into it. And in fact, that might even be an anti-pattern. If you have interactive compute, you're actively developing, you're open to the terminal, you're, you're running things and seeing it live and you're making iterations to it. And so, if you look at Brev and banana, for example, as like, interactive and non-interactive computes that are very, that work really well together, you can take your interactive dev environment on Brev, get things running, and once you're done, press a button, move it to banana, so that it's non- interactive, it's not costing you as much, it's just, it's on the serverless model. And then, if you have a server error, right? If you have some, you get some sort of sentry log. On your Banana server, then you should be able to click a button and then open it up in Brev and interactive compute, so you can figure out what's wrong, fix it and send

• 22:33Speaker 00

  it back. And if you're able to have that kind of workflow, you're taking away a lot of this DevOps overhead, because at the end of the day, we're just trying to build. That's, I think that's where I see the future heading, is how smooth can we move between the states that the user essentially wants.

• 22:49Daniel Whitenack?

  Yeah, I think that's a really insightful sort of direction, because I see this efficiency gain with Brev and sharing environments for that interactive compute that's really important. But then if you can make that connection to the production deploy, that's huge, because now, there's still so much of the time, there's this friction that you talked about, where even if I'm developing against a cloud instance, there's some sort of non-negligible labor costs of me going through the headache of going and deploying something to production, and it's still not the same, or there's some issue like you

• 23:31Daniel Whitenack?

  were talking about when things go wrong and there's debugging. So if you can replicate that environment, both in an interactive and non-interactive way, I personally think that's really, really powerful and interesting. I think, actually, just a note, I think we've got scheduled to have an interview with Banana upcoming, so listeners, watch out for that one. I'm excited about that.

• 23:54Speaker 00

  Really exciting product, and I think a really, really exciting space of just the ML AI ops dev tooling coming out right now. Yeah, definitely really excited. And to kind of take what you said even a step further, you might be reading a research paper and you see a Google Colab notebook that has a model, and you want to go take it, fine tune it for your own, whatever you want to do with it, and then go ahead and deploy it. I mean, Brev is kind of in the center of like interactive compute where we could take a if we have an import tool for Colab notebooks, where you can kind of import it on Brev, change the compute that you want, get something more powerful, fine tune it the way you'd like, maybe even use a template for API frameworks. So you get like Flask APIs set up already for you. You can kind of continue to modify from there

• 24:38Speaker 00

  and then hit the production button and go to banana. That's kind of like the dream workflow I see, where we're behind the scenes always finding the cheapest GPU for you to do that. You're able to get as powerful of a compute needs as you need. It's really simple to go from like Colab to something scaffolded with like APIs that are ready for you to deploy to production. And again, we just get to focus on the fun part.

• 24:57Daniel Whitenack?

  All right, so Nader, I'm looking through the templates that you have at brev.dev. And you know, just to give people a sense of like some of the things that you can kind of spin up an environment quickly and do right away. I see a couple of different stable diffusion, stable diffusion, stable diffusion version two, Dreambooth, TensorFlow, Whisper, Clip, Image Captioning, all sorts of different things. But then there's, you know, environments that you have templated out for things like Go and Rust and you know, other environments that people might be interested in. You already alluded to the fact that you're a quickly moving, you know, small team. And I'm wondering like,

• 25:40Daniel Whitenack?

  out of all the sort of like areas that you know, you could focus on, it's probably, one of the things I would guess is it's maybe difficult to position this for a certain group of people that really need it because it's kind of a common need across, you know, all dev environments. So I'm wondering how you, it seems like you've kind of brought some focus to the area of GPUs and data science, AI type of workflows specifically. Do you think that's mostly been driven by this sort of GPU element and the complexity of those environments? Or how do you think about like, where to head from here in terms of like the verticals and the industries and the specific dev workflows

• 26:24Daniel Whitenack?

  that you're thinking about and you're focusing on? And how is that working? And what are you hearing from users in that respect?

• 26:30Speaker 00

  Yeah, so it's kind of funny. Before we leaned into the AI ML workflows pretty heavily, you're right, right? It's a dev environments is like, who is your target audience? People who code, right? And that's kind of a very naive answer for a very early stage of the product. I think what we learned is you really want to be able to solve someone's problem as quickly and acutely as possible and get out of the way. And I think that's been a big change in direction for us. Even if you look at the like, the way that the product onboards, you need to have the CLI so you can run Brev open. So we used to say, oh, well, when you make an account, we'll tell you to install the CLI right there. But the user doesn't know yet why they want to install the CLI. They haven't, they haven't had expressed desire to open their dev environment yet. So the way that we changed it is it's just focused on getting your environment created. When your environment's created, then you see an open tab.

• 27:14Speaker 00

  When you click the open tab, there it tells you install the CLI because you haven't yet. And so that's, you know, the user says, I want a thing. And then we can kind of show and not really impose. And so when we were thinking about like broadly dev environments, when we initially started this tool, or when we initially started building Brev, it felt like we, you know, someone says, hey, my local environment is not working. And so we'd say, great, we can make one for you in the cloud. But now we're not just introducing Brev as a tool to solve their environment issues. We're also introducing the cloud. It's a separate thing. And so in terms of like acutely solving the problem, we're not doing that. We're introducing the element of the cloud, which they have not yet expressed a desire for. And so what's great about the GPU use cases is we're meeting people where they are, which is in the cloud, right? They're saying, I am trying to access an A100 that does not

• 27:58Speaker 00

  exist on my MacBook Pro, and I want to get this running, right? So the cloud intention is coming from them, not us, right? And we're not kind of like sneakily trying to introduce something else that way we can get them to use Brev. It's just meeting the user where they are, making it in the issues with using a GPU in the cloud is that they're really expensive and they're really painful to get set up. And then, of course, all the dev environment issues. And so that's been a really great focus for us. And we're leaning in as hard as possible to the ML Ops tooling. The dev environment issues are much more severe here. It makes a lot more sense for, there's, there's a lot more room for us to delight users by making a much better experience. And going back to that container strategy, if we can move between different clouds, We can also move between one local cloud, which is your actual computer.

• 28:41Speaker 00

  So I think the way that we kind of want to approach broader dev environments is you should be able to run something on your computer and then say, I now have a need for the cloud. I want double the RAM. I want a GPU. I want something. So you can start local and then move it to a cloud. And that's the way that I think we can ultimately broaden from ML dev environments. But this is a huge focus for us right now. And what I really want to do is rather than think about so many of those other use cases, how do we get really tight integration with Banana? How do we get a really easy way to go from a Colab notebook to something that you're now fine tuning on a much more powerful GPU? How do we find an interface with other clouds? And like, that's where we're focused right now. And there's a lot of work to do here.

• 29:22Chris Benson?

  Yeah, clearly. You have such a focus on kind of accessibility in terms of the user experience. And, and you know, you have a bunch of different ways of connecting in, you know, like I use VS Code, so I went and looked at that. And you have the guides that address different common models that we would be interested in that are really popular right now, like stable diffusion. And you talk about the different clouds. Could you pick one, whatever one you want, and just kind of walk us verbally through? And I know it's audio only, but if you can walk us verbally through kind of what the workflow looks like and what people might expect, just to give a sense. It looks really good, but I'm trying in my head. I'm trying to put it all together from an end to end. And I bet you've done this before. So I'm hoping you can kind of just give us a little narrative that's easy to follow on that.

• 30:06Speaker 00

  Yeah, so let's say you want to run Dreambooth and you want to make a bunch of cool photos of you and your friends. So you can go to our Dreambooth template. It says, click a link. Any environment, you can actually make a URL to easily share it. So we've made one that's a URL template for running Dreambooth. So you click the link from our blog post or from the guide in our docs, and it will take you to the dev environment page with everything filled out. It has the GPU that you'll need selected. It has the volume, the amount of hard drive that you need, the repos that you need, the setup scripts. You don't, you won't have to worry about anything. Just pretty much hit the create button. When you do that, the environment, essentially what we're doing behind the scenes is spinning up the GPU that you need. We are installing everything that's needed, all the dependencies that are needed for that.

• 30:49Speaker 00

  When you're done with that, with the brev CLI, run brev open in the name of your environment, and it'll open up the S code to that environment. And in the read me, it'll say, upload 10 photos of yourself in this folder. And we kind of show you how to run, how to train, and that's it. So the idea is that, you know, in like four minutes, you have a GPU running everything. And all you have to do really is focus on the fine tuning that you kind of want to focus on.

• 31:14Chris Benson?

  That sounds great.

• 31:14Daniel Whitenack?

  Yeah. Could you share a little bit also about like, because part of this, I think, is like, I'm doing a specific thing in my environment that I've created, which is special to me. But now somehow like I need to share that with Chris, right? How would that work out in this type of scenario?

• 31:34Speaker 00

  Yeah, so there's a few things that Rev does behind the scenes. There's like things that are, that we intend for you to share. But every environment that I have, I have my own get aliases. Like when I type C, that's, that's a function for git commit, right? S is get status. There's a bunch of things that I just expect and I have set up in my Zish RC. So you can set up your own developer preferences. And every time you create a dev environment, we take whatever was shared in the template, and then we add all of your settings on top of it. There's also Hashacorp Vault is hooked up by default into every instance. So you have like an encrypted secrets manager. So I have my AWS credentials encrypted and it stays in my AWS account. And every time I create a dev environment, if I, if my co-founder shares one with me or someone on the team gives me their environment, I reliably know that my terminal settings

• 32:18Speaker 00

  are all going to be loaded in, my AWS credentials will be loaded in. But also there's scopes to the encrypted secrets manager. So you can say that if someone shares this environment, make sure that these secrets are added into the environments, like an environment scoped setting. So it's up to you to decide what you want to be shared. We're not going to share secrets that you don't want. You're not going to share your AWS credentials if you don't want it. You're never sharing the machine with somebody. You're, we're just setting up one for them and setting it up kind of identically, so.

• 32:47Daniel Whitenack?

  Yeah, yeah. Which I guess gets to that sort of idea of templates, right? You're creating a template which you intend another person to use, but maybe in a slightly different way than you used it, right?

• 32:58Speaker 00

  Yeah, exactly.

• 32:59Chris Benson?

  So I'm gonna, I'm gonna throw out kind of a random question, and it's okay if you haven't gone here. I just want to ask, have you ever thought about having one that is essentially, you know, we see these services that companies will run, and then they'll end up deploying it kind of on a private server or something so that it can go into a secure environment, that kind of thing, as a standalone instead of being web accessible? Any thought toward doing something like that, where you could use it in a non-public environment?

• 33:26Speaker 00

  Yeah, so that's how larger teams will use Rev. So at a minimum, you can just deploy all of the instances, so that the instances themselves stay in your AWS account, but we can also deploy the entire control plane behind your VPC, so nothing's really exposed out. But that's kind of more on the enterprise route. Individual developers, I don't think have those.

• 33:44Chris Benson?

  Totally get it. And it's the enterprise route that I was kind of asking about is like, you know, you'll have large organizations that have their own GPUs and stuff like that, but they're still just GPUs. And so I was wondering whether like, you know, moving into that, if the control plane can say, "Okay, I'm gonna hook up what you have in your data center, here's your workforce, and you kind of have your own environment." So that's something clearly y'all have been thinking about doing.

• 34:06Speaker 00

  Yeah, and something that we actively support, and we have teams that we're talking with that are going this route. It's a, you get all the same benefits where, you know, you can still scale down your instances, scale up your instances. Obviously, you might not benefit from some of the cheaper GPUs in the other clouds because they're not behind the VPC. But if you're on like AWS or GCP, we can absolutely do that. And we know from like an individual user's perspective, if you're gonna pay an extra eight hours by accident, because, you know, we always forget to shut our instances off. If you're a team of 180 engineers, that cost just is amplified. And I saw that at Workday when I worked there as well. So definitely, we've kind of had some of those learnings brought into the product as well. So, yeah.

• 34:44Daniel Whitenack?

  Yeah, that's awesome. I'm just thinking like, looking back at my own sort of progression and like trying to run some of these things myself, and like just thinking back, not, I mean... The tooling has improved, right? But the environments were still difficult, right? So like either I had like, you know, the consumer GPU card that's in my like workstation here, or I'm trying to use one in the cloud and like the GitHub repos there. And like the tooling, like I can understand what's happening in the code, right? Like that has gotten much easier. I can deploy stable diffusion in like very small number of lines, right? But the environment is still quite difficult. So I think this is really exciting and encouraging.

• 35:29Daniel Whitenack?

  I'm wondering what encourages you? And what are you thinking about kind of like looking towards the future? What excites you about this space?

• 35:39Speaker 00

  Oh, man, what excites me about the space? I, uh, I think, I don't even have to say it. There's, there's just so much to focus on in every realm, right? Within interactive and non-interactive compute, like I've talked to Eric at Banana about just how we both, both of our teams are about the same size and we're both 100% focused in our space. And it just feels like there's an infinite amount just looking down. So looking up, there's even more. Uh, you guys mentioned your last episode was on ChatGPT. And I, I think AI is really exciting, not so much in that it's going to replace us all, but it kind of lets us be more creative directors of our own lives. If you think about any creative process as having like some generative aspect and then some like malleable aspect. So if someone's making clay, they like throw a bunch of clay down. That's the generative.

• 36:23Speaker 00

  And then you kind of like form it nicely into the bowl or cup that you want, right? And that's the kind of like morphing it in. So there's always those two aspects. And when AI is able to help us just kind of really push on that generative side and we're still in control of the output, we're still the ones that are kind of morphing the final product. Uh, I view it as like, uh, just an extremely empowering thing. And so it's been really exciting seeing what all the developments in the space. Really bought into the idea that you make things a little bit easier and you can just dramatically increase the affordance for things to happen. And so as much as possible, how do we get rid of machine problems and that people who want to build really exciting things and build the next new affordances and the new models essentially be able to do that with as little friction as possible. And that's not just within their fine tuning

• 37:05Speaker 00

  and resource constraints that they might have, but also in terms of like moving it and shipping it and delivering it. And so on one hand, the things that are being built is very exciting, but on the other, the energy in this space is huge, right? I think everyone has been so inspired by what's been done recently with ChatGPT and the recent AI models that are out, that it's just, it's galvanizing a lot of people to build a lot of really cool things. Everyone I know, especially here in San Francisco, founder or not, is, you know, it's funny seeing founders who have nothing to do with AI thinking about AI side projects, right? That's galvanizing people. And so everyone is really excited about building this stuff right now. And I just hope we don't lose that energy and just make things as frictionless as possible as we do that. Yeah, even I, I'm guilty. I have a little

• 37:48Speaker 00

  Saturday project I'm throwing together with some generative AI stuff, right? There's a lot of really cool stuff that's happening. So.

• 37:55Daniel Whitenack?

  That's great. Yeah. Well, thank you, Nader, and your team for helping us, you know, reduce some of that friction and get people's ideas out there. Like, this is, this is super exciting. And I think, you know, speaking of friction, I think one of the things that you mentioned prior to our conversation is that you'll spin up a coupon code for for our listeners for some compute on brev dev and getting some of that, you know, removing even some of those barriers for our listeners as they're getting started. So we'll make sure and include that in our show notes. So please take a look at that. Get on brev dev. I did it. It only takes a couple of minutes. It's awesome. So yeah, thanks, Nader, for coming on the show and telling us about what you're doing.

• 38:38Speaker 00

  Absolutely. Thank you guys so much for having me. Really love the conversation. And by the way, Chris, you mentioned Lockheed Martin earlier. My mom was a nuclear engineer and worked at Lockheed Martin as well. So that's awesome.

• 38:47Daniel Whitenack?

  Oh, thanks for telling me that. I'm definitely in good company and awesome. Cool. Yeah. All right. Thanks. See you, guys.

• 39:04Speaker 01

  All right, that is our show for this week. If you dig it, don't forget to subscribe. Head to practicalai.fm for all the ways. And if Practical AI has benefited your life, pay it forward by sharing the show with a friend or a colleague. Word of mouth is the number one way people find shows like ours. Thanks again to Fastly for fronting our static assets, to fly.io for backing our dynamic requests, to Breakmaster Cylinder for the beats, and to you for listening. We appreciate you. That's all for now. We'll talk to you again on the next one.