Why Physical AI Needed a Completely New Data Stack

The future of AI is physical. In this episode, Lukas Biewald talks to Nikolaus West, CEO of Rerun, about why the breakthrough required to get AI out of the lab and into the messy real world is blocked by poor data tooling. Nikolaus explains how Rerun solved this by adopting an Entity Component System (ECS), a data model built for games, to handle complex, multimodal, time-aware sensor data. This is the technology that makes solving previously impossible tasks, like flexible manipulation, suddenly feel "boring." Connect with us here: Nikolaus West: https://www.linkedin.com/in/nikolauswest/ Rerun: https://www.linkedin.com/company/rerun-io/ Lukas Biewald: https://www.linkedin.com/in/lbiewald/ Weights & Biases: https://www.linkedin.com/company/wandb/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  We had this idea that visualization would be hard to monetize, particularly for this like physical AI kind of applications. You need to visualize every single step of everywhere you might interact with data. So we've actually redesigned the data model probably four times at this point.

• 0:16Lukas Biewald?

  Oh, wow. You know, AI and robotics, it seems like such a hot topic. What are you seeing that people might not realize that's working now that's in work a year ago?

• 0:23Speaker 00

  So we're just really seeing an incredible progress in sort of ability to do quite advanced manipulation. This has been a really difficult problem in robotics for a long time. Another one is reinforcement learning. And classically, that's been for walking and motion. And what's starting to work a lot now is kind of combining some of that reinforcement learning with imitation learning.

• 0:43Lukas Biewald?

  Are there benchmarks that should be created?

• 0:44Speaker 00

  There aren't any great benchmarks. The way often to solve real problems is to co-train and co-design for your hardware. It's like you build, design these things together. And then it becomes quite hard to benchmark in a good way.

• 0:59Lukas Biewald?

  You're listening to Gradient Dissent, a show about making machine learning work in the real world. And I'm your host, Lukas Biewald.

• 1:08Lukas Biewald?

  All right, this is a conversation with Nico West, who is the founder of rerun.ai, which is a company that I've admired for a while. They're in a similar domain to weights and biases, robotics and embodied AI and augmented reality. And they have a very cool logging product that operates at super high scale. And then a database or a kind of system of record for bigger customers to track everything they do. They've made a lot of interesting decisions, some same, some different than than weights and biases. So it's fun to get into the weeds about how their product works and how they think about it. It's also just interesting to talk about robotics and the state of the art, because it might be the coolest application of AI right now. Hope you enjoy it.

• 1:50Lukas Biewald?

  All right, so maybe I'll start with a little bit of why I wanted to to talk to you on this this podcast. So, you know, so you're the founder of rerun, which is a company that does logging for robotics and also kind of functions as a system of record, it seems like for robotics companies. And it's very similar in some way. To what we do at Weights and Biases for, you know, AI teams. And, you know, looking at your website, it seemed like you emphasized a lot of the same things as us, like a simple API, you know, like high-speed logging. And it also seems like you have a similar customer love to Weights and Biases that I really respect and admire. And maybe not a lot of people know this,

• 2:32Lukas Biewald?

  but the original use cases of Weights and Biases were pretty heavily in robotics. Actually, OpenAI, when we first started working with them, their main focus was robotics. This was before GPT. And you can actually find an interview with Wojciech, very early interview with Wojciech, one of the founders of OpenAI, talking about robotics projects there. It must have been like five years ago. So, anyway, I mean, you started your company a few years ahead of us in a different domain and, you know, made a lot of different decisions. So I was really excited to talk to you about kind of your kind of creative and product process. And also, I think everybody loves robotics and the applications and maybe call it embody I know, so maybe get into that too.

• 3:15Speaker 00

  We get compared to Weights and Biases every now and then. And that was one of the early signs that we were really happy about our API. When people are saying, yeah, it's like Weights and Biases for robotics or spatial computing and that kind of, that kind of field. So, yeah, let's, where do we want to dig in first?

• 3:36Lukas Biewald?

  Yes, maybe we should start with like, what are the range of use cases that you're typically used for? And how does rerun work there?

• 3:41Speaker 00

  Got it. Yeah. So maybe just set the scene. So rerun, we have an open source project and that is very popular. And that project is an SDK for like logging, modeling, querying and visualizing, really multimodal data and particularly multimodal data that changes over time.

• 4:02Lukas Biewald?

  So I'm picturing like sensors, like a LiDAR.

• 4:06Speaker 00

  Yeah, like 3D sensors, setup for multiple cameras moving around. Maybe you have regular RGB and you have motion sensors and you have whatever other normal signals and tensors. And you just, you think it could be, so this, actually our first early beachhead was in spatial computing. Or I'm going to reality, so like companies building headsets and that kind of stuff.

• 4:30Lukas Biewald?

  And so what are they logging there?

• 4:32Speaker 00

  You're logging like sensor, like images, you're logging camera calibrations, where a camera is in space, as well as just normal text logs. And you're logging time series of, that could mean whatever, just, you know, CPU time or some confidence of a neural network or anything like that. So just both the input sensors that you have and any internal computation that you might do. So yeah, the output of a neural net, so you know, little bounding box that you detect or anything like that. So, yeah.

• 5:08Lukas Biewald?

  And where has it gone from there?

• 5:10Speaker 00

  So, yeah, it started out as that really focus on, we just wanted to make it like 100 times easier to like debug these sort of system, multimodal systems, like computer vision or robotic systems that do things over time. And that's, and that really was the focus quite heavily in the beginning and got a lot of, yeah, first usage in spatial computing and then quite broadly, I mean, it's used in like hedge funds and like weird different things. But then now very heavily in robotics and particularly learning first robotics. And so on top of that commercial sort of open source project, we built a, or are currently building a data platform.

• 5:54Speaker 00

  It's a bit more like a cloud database or a data lake house, even if you wanna kind of get into that. So for, really focused on. On kind of making it a lot easier to get data from like from my robot to like in the right shape and the right subset of that into to training on it, basically. So that like running that like record data, curate and train loop as fast as possible, but not taking part in the training, which is what, you know, Weights and Biases does, but the kind of data pipelines leading up to it.

• 6:28Lukas Biewald?

  Interesting. So I mean, I can see how, you know, you start in kind of VR, AR, and then, you know, you move into robotics, but how, what is the hedge fund logging on your, on your platform?

• 6:41Speaker 00

  Yeah, I think we just have all this breadth of random use cases as well. I guess they just, it ended up, we, we put a really lot of effort into performance and the ease of use. And so it's actually, it turns out, it's a pretty difficult to get really fast, simple to install, streaming visualization of even simple things like time series and so on. And so we put a huge amount of effort into, we built a in-memory database from, from scratch, basically, that runs inside this like visualization app. And the other, that can speed things up and, and it's like highly performant and easy to use, basically. It's not perfectly meant for that, so it's not like that's a, I don't imagine that will

• 7:23Speaker 00

  take off in a big way, but we find a lot of those things, random use cases like that as well.

• 7:28Lukas Biewald?

  Interesting. And, and I know it's just like Weights and Biases, it seems like your kind of core API call is a dot log.

• 7:35Speaker 00

  Yeah.

• 7:36Lukas Biewald?

  And I think like one of the things we often thought about was, you know, how kind of opinionated to be about, you know, people's data types and like how opinionated to be about displaying them. Like, how did, how did you like think, think through that trade off?

• 7:49Speaker 00

  Yeah, so that came from, I mean, first, it's a really important trade off. And I think one that we got, I don't want to say like right, but got, got pretty good.

• 7:58Lukas Biewald?

  You can, you can be proud of your product.

• 8:00Speaker 00

  Okay, yeah. It's, I'm, I'm quite proud of it. It, it did, of course, there are always like things that we want to improve, but I've built similar systems many times. So the three of us co-founders, we worked on a, a 3D like computer vision company along, like a long time ago, and we built something similar there. And you, where the idea was, yeah, you want to be able to log all your data, including this like images and 3D and all those, like the weird, sort of weird data.

• 8:32Speaker 00

  And had kind of built several versions of this at different companies and came in with this idea that it was really important to have it be incredibly low friction when that's what you wanted. So if you're debugging one of these systems, there's so much state in your head, right? Very similar to debugging, like machine learning training. So it just needs to be like dump and forget, just no thinking as like high level, very easy. But then with experience building these things in maybe more rigid ways and you can always get stuck somehow. So it's very important for us to have it be flexible so that you can kind of, okay, I added in this like maybe output from a neural net or a 3D point cloud or something that my neural net is estimating.

• 9:18Speaker 00

  But I just, I want to understand like what was the confidence level of like that, this one point? And I want to just hover it and look at it. And then you need to make that work where I can't upfront know what all the users' data models are going to be. And I have a very flexible data model. So, uh, so we designed a sort of new data model from scratch, inspired by entity component systems. This is like a way of modeling data in games. So we couple of the early employees, just had a gaming backgrounds. And yeah, basically we designed the entity component system.

• 9:53Lukas Biewald?

  And what does it mean? How does that work?

• 9:54Speaker 00

  It's basically, um.

• 9:59Speaker 00

  It, yeah, let's see if I can explain this like very, very simply, but uh, instead of objects. Okay. If you're a normal object oriented program, you might have a, you know, object, you say, okay, you have an image. It has its size, it has the memory buffer with the things in it.

• 10:16Speaker 00

  You just, you have sort of a memory buffer is one thing. Image size is another, and you can kind of just compose all these components. And then there are systems that know how to interpret. So there's some visualization system that says, oh, if I see a like image or like image buffer and a size and all these things that I need, then I will just draw it. And so it's, it's a bit more flexible than saying, okay, we just have this fixed object that we know about how to deal with. We can like easily compose, um.

• 10:48Speaker 00

  Sort of things that you might want to visualize or log or understand more freely, basically.

• 10:54Lukas Biewald?

  And so what are the most common things that people are logging here?

• 10:58Speaker 00

  So if you go to sort of right now, where it's like really popular right now, where Rearness is particularly popular in like heavy learning-based sort of AI-first robotics. And there are people are logging like motion sensors, image or video. So it's really important to have flexibility and high performance and how you represent video as long together with just other debugging system things that you might have in a robot. So those will be the most important things there. And then point clouds and LiDAR kind of sensor.

• 11:36Lukas Biewald?

  So maybe we could talk about this, the kind of state of the art of AI and robotics. It seems like such a like a hot, a hot topic. Like what are you, what are you seeing that people might not realize that's, that's working now that didn't work a year ago?

• 11:50Speaker 00

  I think this, even this last couple of weeks has been like incredibly exciting. I don't know if you've been following all the releases.

• 11:56Lukas Biewald?

  Maybe not. What are they?

• 11:59Speaker 00

  So we're just really seeing an incredible progress in, in sort of the ability to do quite advanced manipulation. This has been a really difficult problem in robotics for a long time. It's, classic robotics is just like built around being incredibly precise. You pre-program every movement upfront and just everything has to be down to the sort of millimeter precise. And, but this kind of flexible, messy manipulation, like folding laundry is the classic task, has been really elusive for robotics for a long time. And it's just sort of folding, folding laundry has kind of gone from being impossible to sort of boring over the last year, basically.

• 12:43Speaker 00

  And it's really kind of come through these like end-to-end learning methods.

• 12:49Speaker 00

  First, on those manipulation tasks, basically imitation learning. It's a kind of robotics version of supervised learning.

• 12:58Lukas Biewald?

  Can you describe imitation learning works for?

• 13:00Speaker 00

  Sure.

• 13:01Lukas Biewald?

  Folks, man, I know.

• 13:01Speaker 00

  Yeah, so the simplest case is you have it, you tell to operate your robots. That means just a person is remote controlling the robot. And you record everything that happens. So let's say you want to fold clothes, like a one episode, you record data as episodes. So that might be start, take, take a piece of clothing from the hamper or whatever, and put it on the table and fold it and, you know, put it in a pile or something like that. That could be a one task. And so that, um, yeah, so you collect data like that, just basically demonstration data. And you, you collect all the state of the robot at every timestamp.

• 13:41Speaker 00

  So what are all the joint angles and some perception or proprioception input, basically, that you can use and, and in simple terms, just train your, your, some neural net to, uh, to repeat that, that task. But this is the real messy world, so, um, it's never going to be an exact replica of the task. But that's the kind of robustness that you try to train in. So I think it was about two or three years ago, that started to kind of really work in a sort of more mercurious ways. And that really started this really, really heavy excitement in, in the sort of AI-first robotics that's, that's happening right now.

• 14:25Speaker 00

  And now, and there's been, that's sort of one of the learning paradigms. Another one is reinforcement learning. And classically, that's been for, uh, like walking and motion and, and things like that. Um, and what's starting to work a lot now is kind of combining some of that reinforcement learning with imitation learning. So we're seeing some like really, really awesome, um, results and very robust robotics sort of manipulation handling coming out over the last couple of months. That's, that's, uh, yeah.

• 14:54Lukas Biewald?

  And, and do you think it's been like algorithmic insights that have made these strategies work? Or is it, you know, a bigger focus on training data or a kind of better hardware platforms that actually, you know, work better or more computer? Or what's, um, what's driving this?

• 15:10Speaker 00

  I think it's all of the above.

• 15:14Speaker 00

  My view tends to be that generally the things that matter more, maybe more this like ecosystem or like economic engine or something like that. I think what what I maybe think see as the big arc is that the sort of LLM hype, Chappie GPT moment kind of thing really showed to the world the power of like really scalable machine learning. And then that really primed everybody to be looking out for signs of could we have a scalable learning work for robotics. Because previously it's been this thing, yeah, you add more training data, it gets worse. Is it just the models don't generalize? And then, yeah, a couple of years ago, there was a bit of a breakthrough that just you could sort of start seeing the signs of

• 15:59Speaker 00

  scalable methods kind of working for robotics. So that includes using transformers, but there were a bunch of innovations that had to be made in terms of just how do you model a robotics problem, which is different than a text problem. So it makes, but then that gets going and then more people get money pulse in, more people get excited. So there's more hardware being built, so that gets cheaper and cheaper and then people want to, everybody knows it's scalable, so that means you have to have a lot more training data, so that, but there's enough excitement to invest in in collecting data.

• 16:34Lukas Biewald?

  My impression of robotics right now at this moment in late 2025 is, you know, I I'm super excited about it, obviously. And I see these like amazing demos, you know, all the time. But, you know, then when I visit our robotics customers, the demos don't seem quite as impressive as like the YouTube videos, you know, that I watch. And I'm not talking about the Boston Dynamics ones, where it's obviously like, you know, like kind of more traditional robotics, but even the ones where, you know, you see this really kind of fluid motion, you know, clearly like trained.

• 17:07Lukas Biewald?

  Well, I don't know. I mean, I still, I mean, you know, I would love to buy a robot that, you know, folds laundry, but I don't have one. Like, what's like the the gap? Right now, how far is the gap? And how would you characterize like the state of the art of of robotics?

• 17:21Speaker 00

  I think there are different like kinds of robotics companies. So I think there's a lot of that. And the ones that are more maybe Swinging for the fences directly, I want to call it that. And you see these incredible demos and then it's actually quite far from a product. But I do see another batch of robotics companies that have a very different approach. They're much more practical. They build robots, scrappy robots early and ship to customers and like have a working product. And they're just aggressively deploying that. So those don't get as much hype because it's not as exciting to look at a kind of not aesthetically pleasing warehouse robot that is just like.

• 18:01Lukas Biewald?

  So about the working robots that you see like in warehouses, like what's, what's happening now?

• 18:06Speaker 00

  So they're really using a lot of the same models, but maybe not as heavy focus on research. So in parallel to all this that's been happening, there's also been a lot more open source models coming out also. The same way in LLM land, there's, there are some great open source models heavily driven by Hugging Face.

• 18:31Speaker 00

  So I see them being just kind of scrappy, using these like VLA models.

• 18:40Speaker 00

  That's vision language action models.

• 18:43Speaker 00

  But maybe just being more practical about building other systems around them and kind of using teleop where needed. And then kind of just have a more product and like scrappiness sort of approach to making things work.

• 18:58Lukas Biewald?

  To be like practical, like what are you actually seeing like, are there? I mean, I've talked to guests on the podcast, I think of every, you know, realm of this, every like spectrum. And certainly like, you know, we talked to folks that have like, you know, carts and factories where you can like put something on it and the cart will like go. You know, to the next location, that seems like clearly factories are doing that. A lot of people talk about, you know, like kind of picking stuff up in factories and moving it. Although that one doesn't seem as, I mean, that's some people have been dreaming about that for like years and years and years. Like, is there actually like picking up objects like happening in real world factories? And is it mattering? Is that, is the city there past that in terms of like what's actually deployed in production?

• 19:39Speaker 00

  I would call it that there are pockets where it's working, but it hasn't been scaled up.

• 19:43Lukas Biewald?

  And what are those pockets? That's I guess what I want to know.

• 19:45Speaker 00

  Okay, so pockets are more like, it is actually possible to do this now. So we see companies that have deployed the tens to a hundred robots in like manufacturing and they use

• 20:00Speaker 00

  these kind of flexible learning based sort of manipulation for like moving stuff in manufacturing, for instance. So just be simple stuff like pick and place. And yeah, that's basically, they have that working, but it's it's not like at massive scale yet, basically. But there are there are smaller companies that are are growing their amount of robots and like and kind of making it actually work in production right now.

• 20:24Lukas Biewald?

  These are vendors, robot vendors that where factories buy from these companies.

• 20:28Speaker 00

  Yeah, yeah, exactly. I haven't heard from, I mean, it may happen, but I haven't heard like the factories themselves doing it and doing that yet.

• 20:35Lukas Biewald?

  I see.

• 20:36Speaker 00

  Yeah.

• 20:37Lukas Biewald?

  And then like, what about on the consumer side? Like I've, you know, had a couple of rich friends that have bought like humanoid robots and they sort of seem to kind of collect dust in the corner.

• 20:44Speaker 00

  Yeah.

• 20:44Lukas Biewald?

  Right now, and I've I've been to, you know, kind of a robot fighting league.

• 20:49Speaker 00

  We were there too.

• 20:51Lukas Biewald?

  Which we're now sponsoring, but you know, I'd say the fighting seemed a little maybe unimpressive, you know, compared to the hype around the fighting.

• 21:00Speaker 00

  Yeah.

• 21:01Lukas Biewald?

  Like, again, what's what's the state of the art there? Like, if a robot can actually fold laundry and then the state of the art is actually like past that, why isn't there a consumer product that folds laundry?

• 21:12Speaker 00

  So I think you can think of, if you just think of what it would take to fold laundry in your own home. There are more things to it than like setting it up, putting the, if if if you had a very expensive robot, but it required you to set everything up perfectly. Maybe some people would buy that, but most people wouldn't because of just the total setup and the kind of management of it. So just there's like a, the physical world has such a huge sort of

• 21:43Speaker 00

  long tail or fat tail, or whatever you want to call it, of this weird, small variations that make things a lot harder. So at the end of the day, you're not, it's not just a robot that can do a task, you need to build a product. Right? So I think that distance and the whole systems aspect of like deploying robots is like, is very difficult, right? You you need to service robots, you need to have, you know, onboarding tutorials and all these kind of things. So I think we're still. That's not where we're at yet, that like all these like most advanced frontier robotics, like intelligent robots are at the sort of full product level yet. But we're seeing some people ship, starting to ship robots for the home soon, I guess.

• 22:27Speaker 00

  Or it's exciting, but I don't, I don't think we'll see full autonomy in the home for a bit.

• 22:33Lukas Biewald?

  What do you think will be the first applications that people use in their house?

• 22:37Speaker 00

  Probably picking up, I mean, the first one, the ones that are used right now is like vacuum cleaning, right?

• 22:42Lukas Biewald?

  Well, I feel like vacuum cleaning has been around for a long time.

• 22:44Speaker 00

  Yeah, there's some newer vendors that significantly make that like a lot, a lot better.

• 22:51Lukas Biewald?

  Interesting. What are you thinking of?

• 22:53Speaker 00

  Well, I like Matic a lot.

• 22:54Lukas Biewald?

  I have a Matic in my house, actually.

• 22:56Speaker 00

  So that's like big step up from Arumba, for instance, right?

• 22:59Lukas Biewald?

  Actually, it's funny. My daughter was asking me what's better about a Matic than a Roomba? And I think we like the sort of like mapping that it does. But honestly, that's not really like an autonomy feature. What is better about a Matic than a Roomba?

• 23:13Speaker 00

  The autonomy, the mapping gives you autonomy, right? So that allows you to handle more robust things, like something is in the way, it doesn't get stuck. It can not drive over your cat.

• 23:24Lukas Biewald?

  Right.

• 23:25Speaker 00

  The collection of these things make it more robust and make it more useful. It's just you get into this world and the, or sort of this realm in the physical world, where there are just so many small things that can go wrong. And they add up and you know, if the probability of one of them happening is high enough, you end up not using the product, basically.

• 23:42Lukas Biewald?

  Sure, sure.

• 23:42Speaker 00

  So I think there's a lot of that, the same lines of robot liability, kind of.

• 23:47Lukas Biewald?

  It seems kind of funny. I mean, I feel like I remember when Roomba launched, it must have been like 2004 or something like that. And you know, here we are 20 years later and it seems like robot vacuums are still by far the main consumer use case.

• 23:59Speaker 00

  Yeah, and I think that's, I mean, that is, this is the simplest, right? So it makes sense for that. I would guess robot vacuums that also pick up toys. That's probably the next, next step. I know some folks are working on that.

• 24:16Speaker 00

  So in terms like really practical things that can work autonomously, I guess it's just like small steps like that will happen first.

• 24:22Lukas Biewald?

  What about fun things? I feel like suddenly we see robot dogs everywhere.

• 24:25Speaker 00

  Yeah.

• 24:26Lukas Biewald?

  Like, what else are you seeing in that domain?

• 24:29Speaker 00

  I haven't interacted that closely with the companies that are building more toys, but I mean, I'm super excited about that. I think there's so much fun things like that that can can exist. There are like AI-powered robot toys and so on, but that's not a space I know really well.

• 24:46Lukas Biewald?

  I guess, what else are you seeing in terms of applications?

• 24:50Speaker 00

  I think very broadly, we're seeing things in like agricultural picking. It's just something that I didn't really realize was the breadth of how many places there's this like last mile kind of logistical pick and place thing. So there really is a lot in different parts of the logistics chain, as well as like the many times of like small autonomy, you know,

• 25:16Speaker 00

  last mile delivery, these kinds of things that put them in the same, and it's sort of close to self-driving, but sort of the same realm as robotics as well, drones. There's just a lot of different applications there.

• 25:30Lukas Biewald?

  Yeah, are you saying, I mean, I may be overly focusing on robotics, like, do you see other like embody AI or other applications like outside of robotics that are meaningful to you? Like, how does your customer base break down?

• 25:43Speaker 00

  So, yeah, I think our sort of customer and user base breaks down according to just if you're trying to do something intelligent for the physical world mostly, and that will split into the big categories being AR, spatial computing kind of things, autonomy more broadly and robotics. But then there are There's just like a long tail of smaller things and

• 26:10Lukas Biewald?

  Totally.

• 26:11Speaker 00

  I mean, there can be really small things like a media thing, a little AI thing in like film production that makes like focusing easier. Just like there are huge amount of these like small little problems. It's quite a lot in like security, surveillance, like analytics for construction, that kind of, that kind of stuff. They're just basically any physical world thing that's happening out in the world. Someone's probably building a product to either observe, analyze it, or actually improve it.

• 26:40Lukas Biewald?

  Are you seeing like a lot of satellite data applications?

• 26:45Speaker 00

  We don't see a lot of that internally because they're, they're just product features. We haven't built a lot of like specialized satellite data handling. We have basic mapping, but that's a whole very deep field. So we don't see them necessarily as huge rerun users yet. But I'm sort of, aside from that, see some cool, cool stuff happening.

• 27:05Lukas Biewald?

  Cool.

• 27:08Lukas Biewald?

  All right. I kind of wanted to go like a little bit deeper into your product if you don't mind. You know, I think like one of the things that we really, you know, designed our, our products for is making sure that doesn't like break the, the user's process. Was that like a big thing for you? I'm curious how you like to actually stream the information back to like a central server.

• 27:28Speaker 00

  Yeah. Yeah, that was important for us too. So I mean, just things like you should, can never crash the user's process and, and things like that, of course.

• 27:40Speaker 00

  So we originally designed it heavily in cases where,

• 27:48Speaker 00

  let's say like, we focused a really lot on high-performance streaming. So there's the all the robustness aspect. I don't know that I have anything particularly like interesting to say, other than that we spent a lot of time making sure it could never crash. But then robotics, we, so we didn't design it for the, the case of, hey, there's always a server listening. So quite often in robotics, you can't guarantee that. And the kind of normal thing to do is write to disk first. So the two kind of core original use cases, like either writing to disk or streaming to a remote viewer or something like that, a remote server. So we have been designed for, for both of those and our sort of on-disk format is.

• 28:34Speaker 00

  Basically the same as the streaming format. And so to make these things fast, we built the sort of we you log data. And depending on the size, you can control these things, but we basically run a micro batcher. And such that you buffer up a little bit of signals in memory. And if it's you're having really, really fast time series, maybe you buffer up more. And then sort of send it over the wire and sort of chunks. I like that instead. And that balancing that and kind of doing that efficiently is like one of the important parts of getting reasonably good, good performance out of these things.

• 29:12Lukas Biewald?

  I mean, one of the things, decisions we made that I'm never sure if it was the right decision or not is we had the logging be open source because it's like running in the, the customers or the users environment.

• 29:22Speaker 00

  Yeah.

• 29:23Lukas Biewald?

  But then we made our visualization, the centralized visualization web server. Close source.

• 29:29Speaker 00

  Yeah.

• 29:30Lukas Biewald?

  I think you made that whole thing end to end open source.

• 29:32Speaker 00

  Yeah.

• 29:32Lukas Biewald?

  Is that right?

• 29:33Speaker 00

  Yeah, exactly.

• 29:33Lukas Biewald?

  And then how did you, so is that why you had to make like a different product to monetize? Or did you have a, like a more enterprise version of the visualization server? How did you think about that?

• 29:44Speaker 00

  So we, we had this idea that visualization would be hard to monetize. Actually because of how important it is. So we had the idea that particularly for this like physical AI kind of applications, you need to visualize every single step of the, like everywhere you have, might interact with data. You need to, you need to visualize when you're like prototyping in a notebook. You need to visualize when you're like, you have like maybe one robot or one device like next to your desk. And you're like kicking it to figure out like what's happening live. You want to visualize your like data pipelines as you're like preparing for training. Eva, just like everywhere. And you want to use it for your operation. It's just too widely deployed.

• 30:28Speaker 00

  There's no good business model that lets you have visualization and all those different points that is actually that works. If you you could, you can make the business model around the per-seat license or something like that that works for your expensive engineer that's looking at the data all the time. But you also want to have the same visualization for your support person who looks at every now and then, or like that balance becomes very difficult. And having it closed source, if it's so, so central, it actually becomes very difficult to adopt for a lot of companies. So we basically decided to have the base visualization be open source and it kind of always has to be, just so it's like easy to embed. And you need to be able to build your own tools around the visualization that are custom to your company.

• 31:12Speaker 00

  So you just, you should be able to take the visualizer and embed it in your application and customize it. So the open source visualizer is sort of client side. So that's kind of how we made that split, that like there's a logging SDK that runs in your process and it directly communicates, maybe via the file system, but like directly communicates with a visualizer app. And there's no real serious server in the middle.

• 31:38Speaker 00

  So that's kind of how we started that out. And I wouldn't say the commercial product is a different product, but it's a cloud sort of backend for it instead that supports like very large amounts of data.

• 31:51Lukas Biewald?

  I see. So we might have actually ended up in the same place, although you might have more emphasis on the client side visualization, which we've actually just implemented recently.

• 32:00Speaker 00

  Yeah, it's kind of, I think, similar in that manner, but just took a different route, perhaps.

• 32:04Lukas Biewald?

  Interesting.

• 32:07Lukas Biewald?

  I guess, have there been interesting API design trade-offs that you made? Is there any like API decisions that you sort of like regret? Like, I remember we made a ton of decisions in the early days, very quickly with like no information. And it's amazing how that some of the weirder ones that we made have like, you know, stuck and haunted us for years.

• 32:27Speaker 00

  I think there are more detailed things that I don't know the worth, like getting into whatever specifically how we designed the tensor data model, things like that.

• 32:39Speaker 00

  I would say rather we designed things wrong and then redesigned them. So we've actually redesigned the data model probably four times at this point.

• 32:47Lukas Biewald?

  Oh, wow.

• 32:48Speaker 00

  So we that and that has been like very effortful, right? And it's been hard, you know, we had to keep some backward compatibility and kind of do all these things. Been been quite difficult, but but yeah, we we kind of really cared about getting those core pieces right and been ready to redo it.

• 33:03Lukas Biewald?

  I mean, I mean, this may be a tune in the weeds and people might just skip this episode. But like, what are decisions that you made in the data model that you then changed in like the next version of the data model?

• 33:14Speaker 00

  Well, the first one was this. We started out with a less flexible API. So we started out like in the beginning, it was like, because this is getting very detailed, right? But it was like r dot log underscore image. And then you'd like give it all the fields. And there was still a flexible way you could give your own custom data into that. And so that was the first huge rewrite where we kind of formalized the sort of system to have a higher level of like archetypes, which are sort of easier to use wrapper objects that you can log instead. So now it's like r log then parentheses image, say, right?

• 33:56Speaker 00

  And yeah, so that was a huge one. And then a lot of the under data model things that we've been changing are maybe more not as clear on the surface. But it's it tends to have to do with how you represent columns of data. It's like more like being able to log more data for multiple time step at the same time, for instance, like here's a whole column of data or here are like five columns of data. And sort of those APIs and exposing that. And then so that's been one set. And but the most maybe difficult to do and impactful things have been maybe more how it's represented in memory and how the data model. It works sort of in the query engine of the database.

• 34:39Speaker 00

  So those rewrites have been the most, very impactful for performance and things like that and storage and pretty painful like to redo. I think we've redone that three times or something.

• 34:50Lukas Biewald?

  Yeah, totally. That's funny. A lot of similarities between our companies. What database are you using now to store all the data?

• 34:58Speaker 00

  Yeah, so it's custom, basically, everything. So the whole, everything open source is built in Rust from the ground up and we use Arrow.

• 35:10Lukas Biewald?

  Yeah, we do too, yeah.

• 35:11Speaker 00

  So that's basically built around Arrow and everything is custom. So it kind of comes down to this, the special properties of what we call like physical data. And so physical data will be multimodal, multi-rate and often episodic. And that just changes how you need to store that data. It doesn't, like, you can't fit that kind of data in a table.

• 35:37Speaker 00

  Because it's multi-rate, that means maybe you have motion happening very fast and you have like images coming very slowly. So they just take that on a robot that's happening at different rates. It just doesn't work if you can't fit it in like a normal tabular model. So just have to store data differently. So, you know, it's a new file format for doing that efficiently. You could think of it as like a sparse version of Parquet or something like that that we use. And the indexing systems, we've like kind of built all those things from scratch.

• 36:13Lukas Biewald?

  I was curious about your customers, like, you know, across all of your customers, are there kind of like consistent blind spots or things that they consistently discover when they start to use rerun to be able to visualize all of their? All the logs?

• 36:29Speaker 00

  So, let's say, I guess you can split on like customers and users to start with as the users of the open source.

• 36:37Speaker 00

  I think the really big qualitative thing there, so that what the open source really is mostly giving them is like visualization. And do, and but it being radically simpler for them to do, so they do a lot more of it than like there's higher performance and so on. So, everybody is already doing some kind, some amount of visualization. So it tends to, it tends to be maybe about volume, like you look at like a lot more things. You're debugging the intermediate steps of your data pipeline and so on. And so you, it just is a higher volume and therefore higher hit rate. So we'll have people tell us like that are working on some big self-driving car project and they found bugs in their training data pipeline that have been around for three years.

• 37:19Speaker 00

  But so those kinds of things, so that would be on the, on the open source side. I think on the, on the commercials side, the biggest like aha things tend to be that they could, they can build like just significantly simpler data pipelines because we have a different storage layer than is used in kind of normal data. And so just the fact that you can build, yeah, this, how simple the, the kind of data pipeline structure can get is, is kind of tends to be one of the really, really big sort of aha or like unlocks that they can, can do that very differently.

• 37:57Lukas Biewald?

  So one place that

• 38:00Lukas Biewald?

  guests on the show have really differed in perspective in kind of physical applications is how much, how they felt about simulations, like it almost felt like, almost like a religious feeling in some of the interviews I've done. I'm curious where you land on sort of like training in simulation and then like what fraction of your customers are, are training in simulation?

• 38:21Speaker 00

  I think I'm more on the agnostic side personally. Just coming from that I'm not personally building these things, so I try to just kind of keep an open mind.

• 38:28Lukas Biewald?

  Totally.

• 38:31Speaker 00

  So we see both, I would say, the users of our commercial platform do not only rely on simulation. Like, if you're like really just relying on simulation, then you have a little bit lower need to care as much about how you're like real saving your data, basically. So if you don't care about saving your data at all, then you don't need a sort of database to store it. So because of that, we get a sort of bias towards people who care about real data. And I see that's also what I see mostly in the companies that are building products that interact with the real world. I don't think anyone knows yet that it's possible to do that well with just simulation. You most likely need a real lot of, like, a lot of real world data to do that.

• 39:15Lukas Biewald?

  Totally.

• 39:18Lukas Biewald?

  So you're a practical guy, like me, building for practical people, practical product. But I was curious, have you ever built a feature in your product because it kind of give you a spark of joy, even if there's no real, like, obvious case for it?

• 39:33Speaker 00

  Like, it would give me personal joy, you mean, that it's there?

• 39:39Speaker 00

  I don't know that there's no case for it. Probably we have, but I've like, you know, forgotten about it. But I think there are some aspects where we've really, really prioritized like small, mini interaction performance. It's like it's, we really care about like how fast it is to like scroll back and forth in time. So the rerun viewer is like, feels a little bit like a video editor or something like that. You can scroll back and forth and just having incredibly low latency on that. Like, that probably doesn't matter as much as we care about it, but

• 40:12Lukas Biewald?

  Maybe it does.

• 40:12Speaker 00

  It just feels really nice. It just has this like, oh, it feels amazing.

• 40:16Lukas Biewald?

  Totally.

• 40:18Speaker 00

  And I think that people react with that. We get a lot of people who are just, I don't know, that gives you something visceral, but you can't put that on a spreadsheet, for sure.

• 40:25Lukas Biewald?

  Totally.

• 40:27Lukas Biewald?

  I love that answer.

• 40:30Lukas Biewald?

  Do you?

• 40:34Lukas Biewald?

  I guess, do you think, okay, I have a couple of rapid fire questions here. Just let's see where they go, okay? What do you, do you think there's like an underrated robotic startup that you'd wanna call out?

• 40:44Speaker 00

  Yeah, I don't, I don't wanna like pick favorites here.

• 40:47Lukas Biewald?

  And this is being positive, though.

• 40:48Speaker 00

  Okay, yeah.

• 40:51Speaker 00

  Underrated.

• 40:54Speaker 00

  Let's see, I would say one on the sort of modeling side that I think is talked about too little, at least. There's not, it's like generalist AI and I'm very impressed by, by what they've done. On the sort of ship.

• 41:05Lukas Biewald?

  And why, tell me why you think they're.

• 41:08Speaker 00

  I think that they had a really lot of foresight on the method of how they're gonna collect data. And like, really went for that very seriously.

• 41:18Lukas Biewald?

  And what was the method, can you say?

• 41:20Speaker 00

  Yeah, it's public, so they, it's called UMI, but basically, yeah, I don't know if you've seen these videos where people put like, they're holding like, it's a person, but they're holding a little robot gripper, kind of. So that, with a camera on it and so on, so they're like more puppeteering a fake, fake robot. And so they bet on that and really scaled that up and, and they're showing a really, really impressive sort of dexterous manipulation. So I was impressed by the, the sort of strong bet on the, on that direction. Like really making that work, it's been really cool.

• 41:56Speaker 00

  Yeah, on the kind of, there are two companies, the other example, I think are great, that are more on like, yeah, ship and deploy. I think Ultra is a really great, great company and the more like pick and place for logistics.

• 42:07Lukas Biewald?

  And why, why do you think of them?

• 42:10Speaker 00

  They're just very, I think, good at shipping and learning from that and like building real systems, but still being AI first. I'm just sort of seeing them doing that very well. There's a European company called Syriac, that's like really good at that same thing, they're in a different space, more manufacturing, but, but I've seen them do like very practical, like shipping oriented and I think that's, that's really cool to see.

• 42:34Lukas Biewald?

  Hmm, cool. Okay, do you think that there's new data formats that you're gonna need? Say, if we roll forward three years? Like, what do you think are like, maybe new sensors coming along? Or new types of objects that you're gonna need to visualize in your product?

• 42:48Speaker 00

  So I think force and tactile touch are clearly becoming a lot more popular. So, and it's natural, like you'd expect. It's this difficult if you have the, if you imagine like, losing your sense of touch or in your hands, picking things up is a lot more difficult. And so unexpectedly, that helps for, for robots too. So that, I think, audio, so more, more things, audio as well, those kinds of things.

• 43:18Lukas Biewald?

  Okay, so in the LLM space that I'm much more familiar with, the language space, actually, my, you know, where I started and where we see more of our customers these days, there's tons of benchmarks. And there's always fighting about like, okay, like, is this a good benchmark? A bad benchmark? People are like too oriented around benchmarks. But I feel in robotics, you don't really see benchmarks as far as I can tell. Are there benchmarks that you think are meaningful? Are there benchmarks that should be created?

• 43:41Speaker 00

  This is like a hot topic in the field. I don't have a super strong opinion that's better than anyone else in the, like,

• 43:49Lukas Biewald?

  Well, you're a guest, so give us your opinion, yeah.

• 43:51Speaker 00

  But I'll say, yeah, it's a huge problem for the, for the field that there aren't any great benchmarks.

• 43:59Lukas Biewald?

  Maybe who can fold laundry the fastest?

• 44:01Speaker 00

  Yeah, but you end up with this difficult situation where the way of than to solve real problems is to co-train and co-design for your hardware. It's like, you build, design these things together. And then it becomes quite hard to benchmark in a good way. How are you gonna logistically do that, right? So you, for something to be practical, almost has to be simulation based. And then you have all the problems with simulation. And not being, like, being particularly difficult for like manipulation and interaction in the physical world. And, you know, perfect simulator, and that doesn't exist. So, yeah, I actually don't know what the solution is, but it's, it's super, super important. But I imagine we'll see.

• 44:43Speaker 00

  Much more of companies driving that for themselves, the sort of internal benchmarks. And then companies will have these forms of robots that they evaluate like, hey, we have a new model, deploy it and they'd run tests live on real things. That's the way of testing things. And I would imagine that will continue to be that way for a bit.

• 45:05Lukas Biewald?

  Okay, so a few years ago, when I was messing around with robots in my garage, you know, it seemed like everyone is using this thing called Ross. And it seems like everyone kind of hated Ross, but like still used it. And I was kind of surprised you on your website. It seems like Ross just has incredible staying power. Like you still integrate with it. What's your like, do you have like a take on it, whether it's like good or bad? And what is like, why hasn't it been replaced if so many people are griping?

• 45:31Speaker 00

  Yeah, I think it's one of these things where it's almost a part of every roboticist's journey to have a stint of like, I'm going to rebuild Ross, but right this time. Like very many roboticists have done that at some point and then given up.

• 45:48Speaker 00

  I think the really core thing is that like, yeah, they're approved, like Ross has approved that network effects work. That there's just a build an ecosystem around it and kind of evolving standard. And it's just very hard to kind of come out of that. And then the other part is that it's solving a problem that is difficult to make general. So like the, at the heart of Ross is it's kind of a message passing system. So there's a way of a common set of message definitions. And her message could be how to represent an image that you can pass around. Or, you know, a 3D pose, basically where something is. So there's any data that you might have. And it's very, it turns out very useful to have a standard set of those.

• 46:33Speaker 00

  And most companies will have their own versions. But if you have a set of standards, then you can build pluggable systems. Like someone builds a pluggable, open source, like navigation module. So, and those things all add up. And that's just really, really hard to get out of.

• 46:51Speaker 00

  Yeah, we actually see a lot of, or a bunch of, a little bit more serious efforts now to replace Ross.

• 46:59Speaker 00

  Let's see how it goes, I guess. It's definitely a difficult problem, but I think in particular it's hard to fund.

• 47:04Lukas Biewald?

  Right.

• 47:06Speaker 00

  So that Ross, I think Ross would be better now if it had been better funded, but it's just hard to find a good funding model.

• 47:14Lukas Biewald?

  Well, it's astonishing because so many well-funded companies are using it.

• 47:16Speaker 00

  Yeah, yeah.

• 47:17Lukas Biewald?

  Do you think there would be more of like a maybe an open source collaboration like you see with databases or something else?

• 47:24Speaker 00

  Yeah, I don't know that we see the databases really have that at super scale until they're... Yeah, maybe there are a couple databases like that, but usually there is a team first that like really takes it really, really far. And then that kind of continues. I don't know why that hasn't happened with Ross, but it certainly hasn't.

• 47:43Lukas Biewald?

  Well, I mean, it could segue into one of my final questions, which is, you know, when you look at some of our joint, you know, customers that are doing these general purpose, you know, robotics applications that, you know, are raising it like astonishing valuations pre-revenue, like we've never seen these kind of valuations pre-revenue before. Does it feel like a bubble to you or not?

• 48:03Speaker 00

  Yeah, it's hard to know. There's some of that, but I think, yeah, it's, they are going after astonishingly large markets. And I think there is something particular about general robotics, which is like, there is, I see the point. Is the very, one of the hardest parts of robotics is making a decent price point for the hardware. And you kind of need a really lot of scale for that. And the best way to generate scale is to be able to serve a lot of use cases with the same, same hardware.

• 48:41Speaker 00

  And so that also requires a lot of scale because then you need a very large amount of data, basically, and compute to make that happen. So I kind of see why a lot of capital gets amassed to go after it.

• 48:53Lukas Biewald?

  Are you a fan of the humanoid form factor or not?

• 48:58Speaker 00

  I mean, I'm a fan because I'm just sort of an optimistic person, I think. So I would love to have a humanoid that was like doing my dishes and my and so on.

• 49:09Lukas Biewald?

  Do you think it will be a humanoid?

• 49:12Speaker 00

  I think long run for home tasks, I think there are a set of tasks that are likely to be humanoid or humanoid-like. I think in the shorter term, in many cases, I think sort of a semi-humanoid is a much more practical form factor, like a wheeled fixed base with arms and that are humanoid-like.

• 49:34Lukas Biewald?

  It's like Jetsons.

• 49:35Speaker 00

  Yeah, a little bit that kind of thing. I think the thing that human-ish form factors have going for them is data collection. So just like you could imagine, maybe it's more efficient to have four arms, but it's really hard to have a human control four arms at the same time. So as long as the real world like data collection is important, I think that's just the like, that's a really important factor that will be hard to get away from.

• 50:01Lukas Biewald?

  Is there a tool out there in your space that you wish existed but doesn't?

• 50:06Speaker 00

  That's a good question.

• 50:09Speaker 00

  I don't know. I don't know that I've.

• 50:12Lukas Biewald?

  You'd probably make it if there was.

• 50:13Speaker 00

  Yeah, I'd probably make it in that case, yeah.

• 50:16Lukas Biewald?

  All right. And last question, I guess, what do you think is the biggest breakthrough likely to see in the next year or two in your space?

• 50:25Speaker 00

  Honestly, I don't know. I think significantly improved robustness. And that really is one of the main thing that that matters. I think over the next year, seeing more and more like larger, longer and more complicated tasks be able to be performed like in a robust way, like self-correcting and so on, and a little bit more like learning on the fly. I think that's a likely trajectory if things go great this year. And my guess is it's going to take a little longer for this more high-level, combining that with like really great high-level reasoning as well. So maybe that's the year after. But no, I'm not just guessing.

• 51:06Lukas Biewald?

  I guess in your experience, how is training embodied AI or robotic systems different than training LLMs?

• 51:13Speaker 00

  Yeah, so maybe it can start with some things that are common. I think the things that we see the absolute best sort of AI team, LLM teams and physical AI teams do is is going to have the researcher, the modeler work on both the data and the modeling together, like the same person is kind of making complicated decisions about exactly how to pre-press data, what data to include and the modeling.

• 51:41Speaker 00

  And so that that's like super, super core. And it just turns out that that's a lot easier to do for LLM, LLM teams.

• 51:51Lukas Biewald?

  Why is that?

• 51:53Speaker 00

  Also, yeah, there are maybe like three core things that you need to be doing. So you need to be looking at your data just like all the time. And for text, you can, I mean, just read text, like it just kind of comes in. And if you have other signals, you just like the raise you to visualize. But doing that for physical data is very difficult. That's a little bit more closer to solve, like for tools like rerun.

• 52:16Speaker 00

  The other part becomes this, like you want to have very flexible, like make it possible for the researcher to kind of be editing the data pipelines and so on. And on the, and you may need to have like high performance when you do that. So you want to have both of those things, high performance and flexibility. And for LLM data, that's like, yeah, text and normal things that you can use like sort of robust, well-proven storage formats like Parquet, Iceberg and things like that. And like high-scale data processing engines like Spark and sort of Databricks. Those, those kinds of tools have been around and matured quite a lot.

• 52:57Speaker 00

  That allows you to be like easy declarative database style APIs to edit, edit how the data comes in. But for robotics, like none of that exists, basically. So there, all the data formats are built for. A world before machine learning. So they're not flexible. They're really optimized for like fast logging rather than like flexible querying.

• 53:20Lukas Biewald?

  And when you say flexible querying, can you give me an example?

• 53:22Speaker 00

  Sure.

• 53:22Lukas Biewald?

  I'm imagining like I want all the, you know, places where it was recorded at night or something like that.

• 53:27Speaker 00

  Yeah, exactly. Let's say I want, I want all the times that it was like at night and the grip, the left gripper didn't open. And there was a failure of type, whatever, type B. And so that might require you to query the metadata and the signals sort of inside a specific recording.

• 53:47Lukas Biewald?

  In fact, are you doing some kind of like semantic embedding to make that possible?

• 53:50Speaker 00

  Some, some users do that, but you could just be normal querying, like this signal value is above five. Or this, and this other thing is this, and the text is whatever, like the text description. And so the kind of normal robotic stack doesn't have a query, a sort of a format where you can represent this sort of multi-rate physical data with, with sort of 3D or whatever, all these different things that you can store it in a file format and have some kind of just database query engine that can operate on top of it. So you end up having to, you put in these file formats that are optimized for saving logs. And then there is no query format. So you, the way you answer all these questions is you write a huge parallel job

• 54:35Speaker 00

  that reads the file into memory, like reads field by field, everything. And you're like writing imperative custom code for every such query and running it on a huge parallel job and aggregating it manually. So it's just like a huge amount of work where it should just be like a SQL query or something like that.

• 54:54Lukas Biewald?

  I feel like I'm seeing, you know, kind of a converging some, in some cases of, you know, L M models and embodied AI models. Like for example, you know, one of our customers, Wave, actually uses an L M to describe what the car is doing as it's driving around. Which is really kind of interesting. Do you see other things like that happening?

• 55:14Speaker 00

  Yeah, I mean, particularly for data annotation. That's a really, really big one. So you'll see a lot of teams doing like they'll train one model or use an open model to just annotate their robotics recordings. To just say, oh, here's where it started or stops. This is the task that it's doing right now.

• 55:34Speaker 00

  Things like that. And obviously, using computing embeddings for images and motion and stuff like that for search and curation. So both of those are are super common as well.

• 55:43Lukas Biewald?

  One thing that I think about, you know, for weights and biases is like, you know, we see more and more like agents like interacting with our products. And we've been pretty passionate about like doing really good visualizations.

• 55:53Speaker 00

  Yeah.

• 55:53Lukas Biewald?

  But I kind of wonder, like, you know, I think a lot of people now, they just pull the data out of weights and biases. And then, you know, use Streamlit or Marimo or something to just like, you know,

• 56:02Speaker 00

  Yeah.

• 56:02Lukas Biewald?

  conjure like their own custom visualizations. And I've been thinking like, oh, like, do visualizations even really matter in like 2027? Do you have a perspective on that?

• 56:12Speaker 00

  I think it will continue to matter. But potentially, it will not be good. Like, people will do more and more custom visualization based on this. So we actually see that happening quite a bit. Like you use Codex or Claude or whatever to just, oh, I need this special case to visualization and just kick off an agent and it'll build your specialized tool. So if like SaaS style applications, where like building a very dedicated tool like that, I would imagine that gets harder and harder. And the underlying systems, assuming they're, they need to be very like high scale and robust. I think that I would imagine that to have some more time on it at least.

• 56:56Lukas Biewald?

  Okay, is there, this is another just, you know, I feel like I'm like just fighting my own demons here. But is there any trade offs that you have between, you know, kind of flexibility for people in the first case where they're just kind of setting things up and just like kind of messing around?

• 57:11Speaker 00

  Yeah.

• 57:11Lukas Biewald?

  Versus like real scalability at like production volumes?

• 57:17Speaker 00

  So I don't know that it would be the exact about scalability for production volumes and flexibility. To be honest, I think we kind of solved that. That took like two years to like on iterating on this data model thing. And I think we got a really good design that can kind of fit both. Of course, we don't get all the way like, for instance, or the way our system is designed, we'll never get to our logging being useful for like really low latency tail operation. It's just not built for that.

• 57:45Lukas Biewald?

  Yep.

• 57:46Speaker 00

  So we won't get all the way there. But I think maybe more importantly on like usability. So we want to have a thing that's flexible enough for researchers to use. And one of the things that go into that is we don't want to require you to like compile a schema before being having to log something.

• 58:09Speaker 00

  But in production use cases, having systems that are a bit more built around like a pre-compiled known schema. This is exactly all the things that can come. I guess that gives a little performance, although you can work around most of those performance problems. But that can be actually lead to a bit more, that can fit a bit more naturally into those systems. So that like ease of low friction for a researcher usability versus hey, I'm plugging this into my already existing schemas and sort of production system. That then has taken us a lot more work to kind of work for there too.

• 58:49Speaker 00

  And we still have a little left to go on that, on that journey to like really cleanly integrate with those sort of production systems, so that like robot production systems like that. Yeah, that's been a big trade off for us, for sure.

• 59:02Lukas Biewald?

  What about, do you get a lot of open source contributions?

• 59:07Speaker 00

  We do, but it's not, we get like a medium amount, I would, I would call it like.

• 59:13Lukas Biewald?

  Do you think it's like worth the overhead of managing it?

• 59:18Speaker 00

  I think it is, for us, it's not, if it was just for the sake of the contributions, it would not be worth it. There's a, we need to review pull requests and so on. And so for that purpose, no, not really. So for us, open source is, it gives a lot of trust, right? So it's like very easy to adopt. And I think there's those just, it makes it very easy. So rerun gets built into a lot of other open source tools. So one of the most impactful open source projects in robotics learning has been Hugging Face's LaRobot project. And they built rerun into that for the visualization. And that wouldn't have happened.

• 1:00:00Speaker 00

  And then we built into like Nvidia's new simulator engine and gets built into that. And so those kinds of things matter a lot in terms of contributions. I think it's actually more in terms of some company wants, or some person or someone really loves rerun. And, and they have some little thing that they are really annoyed about and having a way for them to do that. And maybe it would have practically been easier if they just asked for it and we did it. But that kind of participation builds a kind of relationship with people. And I think that adds up more, more than the particular thing.

• 1:00:37Lukas Biewald?

  Awesome. All right. Well, thanks again.

• 1:00:40Speaker 00

  Yeah, thanks.

• 1:00:42Lukas Biewald?

  Thanks so much for listening to this episode of Grading Descent. Please stay tuned for future episodes.