Deep learning in Rust with Burn 🔥

It seems like everyone is interested in Rust these days. Even the most popular Python linter, Ruff, isn’t written in Python! It’s written in Rust. But what is the state of training or inferencing deep learning models in Rust? In this episode, we are joined by Nathaniel Simard, the creator burn. We discuss Rust in general, the need to have support for AI in multiple languages, and the current state of doing “AI things” in Rust. Sponsors: Neo4j https://www.neo4j.com/nodes-2023?utm_source=Changelogpodcast&utm_medium=nl&utm_campaign=Nodes&utm_content=Ad-1 – NODES 2023 is coming in October! Fastly https://fastly.com/?utm_source=changelog – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipFly.io https://fly.io/changelog – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog https://fly.io/changelog and check out the speedrun in their docs https://fly.io/docs/speedrun/. Featuring: Nathaniel Simard – GitHub https://github.com/nathanielsimard, LinkedIn https://www.linkedin.com/in/nathaniel-simard, X https://x.com/nath_simardChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: burn-rs https://github.com/burn-rs/burn: This library strives to serve as a comprehensive deep learning framework, offering exceptional flexibility and written in Rust. Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:08Speaker 00

  Welcome to Practical AI. If you work with artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners for helping us bring you Practical AI each and every week: Fasi.com, Fly.io, and TypeSense.org.

• 0:42Speaker 01

  What's up, friends? There's so much going on in the data and machine learning space. It's just hard to keep up. Did you know the graph technology lets you connect the dots across your data and ground your LLM in actual knowledge? To learn about this new approach, don't miss Nodes on October 26th. At this free online conference, developers and data scientists from around the world will share how they use graph technology for everything from building intelligent apps and APIs to enhancing machine learning and improving data visualizations. There are 90 inspiring talks over 24 hours, so no matter where you're at in the world, you can attend live sessions. To register for this free conference, visit neo4j.com/nodes. That's N-E-O, the number four, j.com/nodes.

• 1:43Daniel Whitenack?

  Well, welcome to another episode of Practical AI. This is Daniel Weitnack. I am the founder at Prediction Guard, and I'm joined, as always, by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

• 1:58Chris Benson?

  I am doing very well today, Daniel. It is fall weather out, and I'm enjoying getting outside.

• 2:04Daniel Whitenack?

  It's fall. It's raining here today.

• 2:06Chris Benson?

  Yeah, it's a little cloudy out, but I'm enjoying it's nice weather. And so, you know, it's like, part of me wants to stay inside and do the fun things, like especially about what we're going to be talking about today.

• 2:16Daniel Whitenack?

  Right.

• 2:16Chris Benson?

  And part of me wants to get outside and enjoy the weather.

• 2:19Daniel Whitenack?

  Well, it's that time of year where you just want to curl up next to a fireplace and burn some firewood.

• 2:26Chris Benson?

  Oh my gosh, you took us right there. I'll tell you what, before you say that, I'll just say this is an exciting episode coming up because I think this is a little moment where we're going to talk about our industry maturing a little bit through one effort. And with that said, I'll let you go ahead and do the intro.

• 2:44Daniel Whitenack?

  Well, the connection to burn is because burn is a deep learning framework that's built in Rust. And today we have with us the creator of burn Nathaniel Simard. Welcome Nathaniel.

• 3:00Speaker 04

  Hi, thanks for having me.

• 3:01Daniel Whitenack?

  Yeah, well, I admitted to you before the episode that I am basically uninitiated in terms of Rust goes. I've, I've looked at various articles. I think that I've run Rust programs just in a sort of hello world sort of way. Probably my biggest use of Rust has been using Rust in the Python lenter called Ruff, which is really great. So that's kind of a circular thing, but for those others out there in our audience that might not be as familiar with Rust

• 3:41Daniel Whitenack?

  as a programming language, could you just tell us a little bit about the sort of like, what is Rust and why Rust?

• 3:50Speaker 04

  Yeah, Rust is, I think it's falsely being categorized as a low level programming language, probably because of historical reason. But it's very general programming language that can be used for high level stuff as well as low level stuff. So the main reason to use Rust is maybe when you need to go through multiple abstraction boundaries without having to pay for performance. So yeah, this is how I defined it.

• 4:16Daniel Whitenack?

  And I could be wrong about this, but I think one of the great features, along with go having a really great mascot. We've got, isn't it a crab? If you see crabs or something for Rust, isn't that a thing?

• 4:31Speaker 04

  Yeah, I think it's a, it's a cute crab.

• 4:34Daniel Whitenack?

  It's a cute crab.

• 4:35Speaker 04

  That's the mascot. Yeah.

• 4:38Speaker 04

  I think it's important for programming language to have that.

• 4:41Daniel Whitenack?

  Yes.

• 4:41Speaker 04

  You have Python or the snake with the with this programming language we've got. I don't know what is for Go.

• 4:48Daniel Whitenack?

  It's the the gopher, the Go gopher. Yeah, it's it's quite nice.

• 4:52Speaker 04

  Yeah, the gopher.

• 4:53Chris Benson?

  It's funny, and you mentioned that Go is actually how Daniel and I got to know each other. We met in the Go programming language community, and we were kind of the two data oriented people at the time. This is going way back. There are many, many data oriented people these days, but got to know each other. Subsequent to that, I had been hearing about Rust for a while, and I got very interested in it. Not only for because, as you pointed out, it's a fantastic general purpose programming language all around. But it also does have a lot of really amazing low level features and performance capability that attracted me to it. So I'm not nearly as accomplished in the language as you are, Nathaniel. I still love Go, but Rust is now another programming language that I have fallen in love with.

• 5:38Speaker 04

  Yeah, I think Go is really well suited for web services. So we've got a lot of tooling around that. It's very pragmatic to use it for that stuff. So yeah, Rust is getting there, but we've got the whole async stories behind that.

• 5:51Chris Benson?

  Yep.

• 5:52Daniel Whitenack?

  And for Rust itself, you mentioned kind of people have this stereotype of Rust as a low level programming language. But could you give maybe some examples of the the types of things either you've built in Rust over time or that are possibilities, just to kind of give people a sense of what people are doing with the language? Obviously, we're going to be talking about deep learning, which is, thanks to you, something that can be done with the language. But what are some of the other things that are out there that people are doing right now with Rust?

• 6:25Speaker 04

  Well, I think it was first created as a replacement for C++ to write browser engines. So this is maybe why it was known as a low level programming language. But now I think it's used in game engines. It's also used to do web frontend. So you've got like Lettuce and Deoxys, which are frontend libraries like React and Vue. So this is pretty high level. We've got also command line libraries that you can use like meta programming so that it's very easy to do your command line arguments, all of that kind of stuff. So yeah, there is tons of things that are built with Rust, high level and low level. So you can mix and match with your own applications.

• 7:05Speaker 04

  Of course, there is like the web services with Tokyo, the SING front time, Action, a lot of, if you want to do web services, there is also libraries for that. Yeah, this is a project on top of my mind.

• 7:18Chris Benson?

  It was one of the first languages that really embraced WebAssembly and got it out there. It's interesting, coming, speaking as kind of a novice in the language and coming from most recently Go, there's always this debate on Go versus Rust that you tend to see in articles out there. And I really found room for both of them and I go back and forth at this point. I will point out, whereas Go is one of those languages that has runtimes that kind of manages memory, Rust has a really cool feature to it. It's not specific to what we're talking about today, but the compiler ensures that you don't have memory faults, seg faults, which is something like 70% of all the bugs in software, according to Microsoft.

• 8:01Chris Benson?

  And so it has a really interesting way of approaching ensuring that you can produce bug-free software, or at least much fewer bugs in it, far fewer bugs in it. So it's a pretty cool language. I'm just curious, as we're talking about the language in general, what's your favorite feature or what are some of the things that made you turn to Rust versus some of the other languages you may have worked in?

• 8:21Speaker 04

  Ah, this is hard to just choose one feature. I think this is the whole package.

• 8:26Chris Benson?

  Said like a real Rust aficionado there.

• 8:28Speaker 04

  Yeah, but like my favorite feature is not the reason why I started writing in Rust, but now I think my favorite feature is just associative types, because it can abstract data types, like something that is really hard to do with other languages. So, yeah.

• 8:45Daniel Whitenack?

  And could you explain a little bit of like, when might that be useful or how is that useful in terms of like when that might come up in your programming?

• 8:54Speaker 04

  Well, it's when you need to abstract the type you're going to use, but you let the implementation decide the types. Normally, like you have the generics. The generics you have to, maybe you have a list and you have to say, okay, I want a list of strings. But it's when you use the list that you decide the type. Where associative type is, okay, I've got maybe a list, but I don't know of what. It's the implementation that decides of what's going to be the list. So sometimes it makes sense. For instance, in burn, we've got the backend, a backend traits, which we can have multiple implementations like CPU, GPU. And we have associative types for the memory, for the tensor handles, for all of those things that you can manipulate at a high level,

• 9:38Speaker 04

  but you don't have to know which type it is. It's to the implementation to decide.

• 9:43Daniel Whitenack?

  I'm just gonna ask maybe like an ignorant question, but I think maybe some people out there might be wondering it. If I'm working in Python, this is a language where I don't have to compile my Python code. Some of the things that we're talking about here with the compiler and other things, a lot of people don't think about, although there's some intersection with that. So could you describe like when you're writing a Rust program, what does that look like in terms of is it a statically typed language? You're talking a little bit about type there. It sounds like you talked about a compiler. So am I right in that it's a compiled program? And then you can run the binary on some architecture?

• 10:28Daniel Whitenack?

  What is it like to work in Rust as compared to something that people might be very familiar with like Python, where a lot of people that are probably listening to our episode or have their Google Colab notebook pulled up right next to them, right? And they're doing all sorts of things with a Python interpreter. What is the workflow and programming like in Rust as far as how the language is set up and how you work with it?

• 10:56Speaker 04

  Obviously, it's a bit different than working in a notebook. Like you said, it's the strongly typed static programming language, similar to like C++, Java, all of those older language. So for people that comes from Python, maybe you're aware of the Python type int that you can use. It's a bit like that, but you have to use that everywhere in all of your functions and definitions. And the workflow, something that I like is that in Rust, it's, I think it's the one of the only programming language that does that, is that when you write the function, you can just write the test below it. So that's kind of the way where you can get some feedback on what you're actually writing.

• 11:37Speaker 04

  And it encourages good practice because you're writing a test that can be reused all the time. It's not a script that you're trying to just run on the side. You can actually commit that and it describes how the code should run. And that's how you get interactivity with this. And since you have a packet manager, which is Cargo, it's pretty easy to just execute the code you're writing.

• 11:59Chris Benson?

  To follow up on that, you know, Cargo, the package manager, is based on a lot of the best practices we see in some of the other programming languages. For instance, in JavaScript and the Node community, you have npm. And there are several others and the Rust community really drew from kind of best practices on that. Another thing to kind of follow up on the compiler notes that Nathaniel was mentioning was, was a lot of Rust developers kind of see the compiler almost as a pair programming partner in a sense, to where instead of just hitting compile from time to time like you would in Java or something like that, the compiler is so comprehensive that it kind of helps you and you kind of use it to write the right code.

• 12:42Chris Benson?

  And you get to the end of the process and know that your code will actually work without runtime errors. So it's a different way of thinking about being a developer. It takes a little bit of a mind shift to adjust over to it.

• 12:54Speaker 04

  This is very different. Like in Python, an important skill is just to be able to read the stack trace. Because you're going to have a lot of exceptions when you run your programs and you you have to learn how to debug your program. This is kind of a hard skill you have to do when you learn Python. In Rust, it's you have to learn how to write the compiler errors. But they made, at least they tried to make it as easy as possible. Even sometimes you've got links to the documentation. It opens a browser, it can read why you have that error. It explains the reasons why. So this is a different set of skills. And yeah, this is quite different from the workflow you use with Python.

• 13:33Daniel Whitenack?

  Maybe just one more question about Rust in general before we dive into some other things. What is the Rust community like in terms of whether it be? Is there active channels where the Rust community communicates with one another? Conferences, meetups? What is the Rust community like? And is it growing? How is it changing over time as you've been with the language for some time? How has it developed in the time that you've been part of the community?

• 14:07Speaker 04

  I'm not sure about all of the community, obviously, but I think it's pretty friendly. Like there are some Discord channels where you can just go and ask your questions if you want to. There is an active GitHub issue, so the language is open source. If you have a problem, just open an issue and people maybe are going to help. So this is a pretty inviting community, I think. This is part of the reason why it succeeds, I think. Because if you don't answer questions, you don't help people use your technology, it doesn't really work out. I never went to a conference for Rust yet, but I know there are many. So maybe I'm going to go to some later.

• 14:45Chris Benson?

  You know, one of the topics that has been kind of a recurring topic between Daniel and me over a number of episodes, we've been tracking kind of the maturity process of the AI community and kind of what it takes to kind of level up and to take it to the next level. And on a number of different occasions, we've talked about the fact that if you look at other communities that have arisen before this one, often it takes kind of broad support. Whereas in the kind of the early days, you know, that we're really still in, in my view of modern AI, it has been largely dominated by a single programming language, which most of our listeners are very aware of, which is Python, which has really been kind of

• 15:30Chris Benson?

  the focus of where all the work is. It's where all the APIs have been focused on and everything. And we've discussed quite a bit about how for AI to mature, it needs to become more broadly available to other languages. And so that as you have different types of use cases, addressing, you know, different business needs, and that requires languages other than just Python all the time, how do you get to AI and what kind of bridging do you need to do? It leads me Nathaniel, as I wanted to ask you is it's clearly a need that the community has had to be able to start getting Rust and other languages in there. I'm curious, how did you approach this? What was it about trying to get Rust

• 16:13Chris Benson?

  working as a framework that could work with AI tools of the day? How did you get into that? What was your motivation? What did you see as the need at a personal level?

• 16:23Speaker 04

  Well, I started working on BIRD because I was experimenting with asynchronous neural network and I wanted to make something a bit, not standard, let's say that. And I needed like multi-trading, concurrency and stuff like that. And it was really hard to do with Python. And I have a software engineering background. So I said to myself, well, if it's hard for me to do that, then maybe it's too hard for any researcher to do that. So that's why maybe we don't have yet an architecture for that kind of stuff. So I say, well, Let me try and make a framework in a language that has support for high-level programming and concurrency and all those things. And yeah, it's pretty much the description of Rust.

• 17:06Speaker 04

  So that's why I started writing a framework in this language. And then it's just was a personal project for a long time. I just was experimenting with it and yeah, it grew with time.

• 17:17Daniel Whitenack?

  When you first started thinking about burn and these problems that you were looking at, what was the current support for doing, whether it be kind of quote unquote traditional machine learning, like, you know, random forest, SVM, whatever that is, in all the way up to kind of deep learning in Rust? What was kind of the state of things? I'm looking at your burn repo and I see you've at least been submitting pull requests since July of 2022. Maybe I'm sure some of it goes back further than that. So back to those days, what did the ecosystem look like in terms of its support for these things?

• 17:59Speaker 04

  Well, I don't think there was a lot of deep learning framework in Rust. So there were some experiments, but nothing really pragmatic that you can use. So I think there was a library for normal like SVM, random forest in Rust. I never used it, but yeah, I don't think it's comfortable yet to side chat learn in PyTorch, which is very complete.

• 18:23Daniel Whitenack?

  It's interesting because some of the sort of early stuff that we were doing in Go was similar there. There were certain packages like for whether it be kinds of regression or hypothesis testing, statistical things, but not really a robust deep learning framework. One of my questions would be in Go, I know one of the struggles with trying to support really robust deep learning is not necessarily the fact that you can't create a nice package with a good API, but a lot of these sort of specialized

• 19:05Daniel Whitenack?

  Libraries and toolkits like CUDA and GPU support make things a little bit more difficult. So it might not be that, but what, what did you see at the time you started working on Burn as the big challenges on the Rust side? And has that been the case as you've developed the package? Or have other things become the kind of dominant challenges over time?

• 19:33Speaker 04

  Yeah, all of those things are hard to work with, like CUDA, having your own GPU kernels, all the drivers, not necessarily easy to install on all platforms. There is a GPU libraries in Rust that has write kernels. This is like WGPU, so it's targeting the web. But when I started working on Burn, I acknowledged it was pretty important to be generic over the backend so that we can write the best backend for the specific hardware you're actually targeting, because it's probably always going to be faster to write CUDA for Nvidia, to write like low-level C or Rust maybe with S AMD support for CPU, or to write

• 20:17Speaker 04

  with the Metal Graphics Driver for Mac. So I was aware that one backend cannot be written for all of them. And I, I just defined the API and I just used Lit Torch as a backend because there was already bindings to Lit Torch and Rust. So this allowed myself to iterate over the abstraction over the user space API and not necessarily worry about speed and writing all of the kernels, just getting the abstractions in place and the software architecture in place. And it's more pragmatics. It's probably like as fast as Lit Torch by default. And then I can just go and write more kernels afterwards, which is what we're doing right now.

• 21:03Chris Benson?

  I'm curious, do you feel, given the low-level capabilities that Rust brings to bear that so many other languages don't have, and that when you're looking at, whether it be GPUs over time, and I know you're talking about using libtorch in this case, but do you think that as you move forward, that that low-level capability that you have in this language that other languages don't bring to bear, will be a helpful part of kind of developing it and maturing burn over the years ahead? Does that low-level give you an advantage that you might not have with other languages that we're trying to integrate in?

• 21:40Speaker 04

  I think so, mostly in the part where we need to handle memory. So that's an important part of deep learning frameworks, don't have to waste memory. We can leverage all of the type system of Rust to actually do graph optimizations and all of that kind of stuff that we're going to work on soon. And I think it's going to be easier to work with Rust to do that with good performance than it will be with maybe another programming language with the garbage collection, because it has fine control over the memory. So not necessarily to write GPU kernels. When you do that, you're actually writing compute shaders, so it's not relevant to Python or C++ or even Rust.

• 22:20Speaker 04

  But if you want to handle memory and write the optimization pipeline, then I think Rust can be really useful.

• 22:27Daniel Whitenack?

  And just to get a sense of kind of the current state of burn, what is possible in terms of support and what you can do right now? And what are some of the highest requested things that you would like to work on but kind of aren't there yet?

• 22:48Speaker 04

  I don't know. There are so many things that I want to work on, but time is just, is limited, so quite hard. What I'm really excited to work on is kernel fusion and really optimize the compute pipeline with lazy evaluation. So that's something I'm really excited to work on.

• 23:07Daniel Whitenack?

  Could you dive into that a little bit and kind of what that might mean for a user specifically?

• 23:14Speaker 04

  Yeah, in terms of user, it's just going to be faster. So this is really like optimization techniques that the framework, the deep learning framework can use. So yeah, there isn't a lot of impacts in terms of user API and usability, but it's just going to be faster.

• 23:31Daniel Whitenack?

  Gotcha. And would you say that right now, in terms of what people are doing with the package? Now, you mentioned that part of what got you into it was building kind of experimental models or architectures that maybe you were experimenting with on the research side? So I'm wondering with this package, what are you seeing as the people that are using it? What are they most doing with the package? Is it that sort of experimental research implementation side? Is it taking models that aren't maybe experimental and embedding them in, you know, Rust applications where they wouldn't have been able to before?

• 24:16Daniel Whitenack?

  Is it something else? What are you seeing in terms of what people are doing over and over again?

• 24:20Speaker 04

  I think a lot of people are using it because it's easy to deploy on any platform. Because we have different backends, so you can deploy on WebAssembly. You can deploy on even a device with auto operating systems. So this is pretty great in terms of deployment flexibility. But even though I started the framework because I had like a research idea I wanted to do, the goal of Bernie isn't necessarily to be only for research. I wanted to go with a kind of a blank sheet and thinking about all the constraints and who is going to use the framework. So I'm always thinking about like the machine learning engineer perspective, the researcher's perspective, and even the backend engineer's perspective.

• 25:01Speaker 04

  So the one that is going to write the actual low-level kernel code and CUDA kernels and stuff.

• 25:08Chris Benson?

  So there's kind of different user profiles or use cases that you can assign to the framework.

• 25:13Speaker 04

  Yeah.

• 25:13Chris Benson?

  Kind of as a follow up to that is you were looking and I noticed that you had quite a few people that were making contributions. You're for being a relatively young project overall, you have a lot of people involved in it. So, I mean, it looks like it's really getting a lot of traction. How do you kind of organize the work around it and kind of satisfy the interests of each of those personas along the way? You know, is there one that tends to lead or do you tend to try to have certain people that do different ones? How do you approach that?

• 25:45Speaker 04

  To be honest, I'm not sure. I think the key is just to be reactive. So if there is an issue, just go and comment it. If there is a bug, try and go fix it. And I think the most important work I can do is in terms of architecture, like setting the stones in place. But then if we, I want to extend, maybe add more tensor operations. Or if I want to add more neural network modules, then I can open issues and people that are interested can just assign themselves and actually do a pull request. And I just have to be really, really conscious about that, do code review correctly, be kind. And I think that's, that's pretty much it. I don't have any other secret.

• 26:26Daniel Whitenack?

  So Nathaniel, I've deployed a lot of models as part of my day job. Let's say that I am interested in Rust and I am interested to maybe take some model that I might have experimented a little bit with in a Colab notebook or something like that. And I want to make it, like you said, have the support for multiple backends implemented in a maybe more efficient application. What would be the process that someone would have to do to, let's say, get one of the kind of popular quote unquote models these days up and running in Rust using Burn?

• 27:10Daniel Whitenack?

  Is that something that's possible right now? How are people kind of pushing the edges with respect to that?

• 27:16Speaker 04

  Well, I think there are two different strategies. So we're actually working on being able to import a next model. So if you have maybe an image classification models, then maybe our import is going to work. It's still in WIP, but if there is no crash, it's going to work, not all operations are supported. But maybe for other models, you maybe need to write the model from scratch using our framework and then translate the weights and you would be fine to deploy it. So it's a bit of work, but working it with Burn is quite intuitive. So the API is similar to PyTorch, the modeling API at least.

• 27:56Speaker 04

  So it's not that hard depending on, obviously, the size of the model and the complexity of the model.

• 28:03Daniel Whitenack?

  Yeah, and I think I saw a few on the repo that you've, people have already sort of done this. What are some examples of some of these that people have brought over into Burn?

• 28:14Speaker 04

  Yeah, I think there are community models for Llama, for Stable Diffusion, for Whisper. This is thanks to the community. I didn't actually port those models. But yeah, since it's open source, I think if you actually do the work to port maybe a model, I think it's great to share it with the community. People can start using it. So yeah, we have a few, but we would like more.

• 28:37Daniel Whitenack?

  Yeah, so call out to the listeners out there that are Rust people in the audience. Check it out and submit some of your own model implementations. That, that's a great way to contribute, I'm sure. You mentioned it having a similar API to PyTorch. And I'm kind of looking through some of the documentation here. I'm wondering if you could just comment on a few of the things that you call out as far as features of Burn and kind of explain what you mean by some of those things. I think we already talked a little bit about the customizable, intuitive, user-friendly neural network module. So this kind of familiarity with maybe a PyTorch API, maybe there's more to that.

• 29:19Daniel Whitenack?

  But you also mentioned this comprehensive training tools, including metrics, logging, checkpointing. Could you describe that a little bit in terms of what the thought process is in the framework around these things, which are definitely important practically, as you said, for the machine learning engineer, for the actual practical person who's trying to build models?

• 29:42Speaker 04

  Yeah, and even the researchers, sometimes they don't want to actually write all of the training loop if that's not the core of their research. Yeah, there is a library, which is called Burn Train, which tried to bring a training loop to the user, so they don't have to write it. So you've got like a basic CLI dashboard where you can follow all your metrics. You have your logging, so if you want to maybe synchronize the drive to maybe a Google account, you can probably do that. So it's similar really to PyTorch Lightning, so for the PyTorch user that are familiar with the projects, but we also have that for Burn and we just have that. It's just easier to get started with the framework. I think it's essential for now, if you're starting a new framework, to provide that.

• 30:27Daniel Whitenack?

  We already talked a little bit about the versatile backends. I don't know if you want to say any more about the other options for that. You mentioned Torch and WebGPU, but I see a couple others here mentioned. Are there any callouts that you'd like to make there, both in terms of other options, but when also those other options might be useful? It might, people might not realize in the audience, when you would want to use a Torch backend versus something else?

• 30:56Speaker 04

  Yeah, I think the Torch backend is probably the fastest if you have an NVIDIA GPU. For the CPU, I'm not sure, it depends on the model, but we also have an NDA backend, so NDA is similar to NumPy but for Rust. This is maybe the fastest backend, but this is extremely portable, so you can deploy the backend everywhere. So if you've got a small model, it can be very handy to have that or to write unit tests and stuff like that. We also have a Gendal backend, so Gendal is also a new framework built by Hugging Face in Rust, so they're trying to make it easier to deploy model with that. So we actually have their framework as a backend for Burn, so you can benefit from their work.

• 31:41Speaker 04

  And yeah, we have the WebGPU backend as well, so we can target any GPUs. So if you don't have NVIDIA, don't be sorry, we have you covered.

• 31:50Daniel Whitenack?

  Awesome. So I also noticed on your GitHub repo, in addition to kind of the familiarizing us with kind of the capabilities and features, you also have the Burn book, which I assume was a little, maybe inspired by the Rust book. That seems to be a common thing. What is the Burn book and how can we best use it? What's it for in your mind?

• 32:13Speaker 04

  Yeah, the Burn book is to help people getting started with the framework. So it's like a big tutorial slash reference that you can use to actually start using Burn. At the beginning, it tells how to install Rust, how to get started with the language, how to make a basic models, the training loop, the data, the data pipeline, all of that. So it's just with all the explanations and stuff like that. So it's really to help people getting started with the framework in an easy way.

• 32:42Daniel Whitenack?

  Of the people that are coming through and learning from the Burn book, interacting with you on the repo, do you see a lot of people coming from the non-Rust community in because they have either, you know, performance-related things or maybe their company is exploring deploying things in Rust or other people, that sort of thing? So people coming from maybe the Python community? Or do you see more people kind of Rust engineers who are already building things in Rust? And so now that everybody wants to integrate AI into their applications, you sort of have the influx from that way. Are you seeing both?

• 33:24Daniel Whitenack?

  Which side is kind of coming your direction more?

• 33:28Speaker 04

  I'm not sure necessarily about the backend of users of Burn, but I think the main pain point is that they want to deploy their model reliably and they're coming to Burn to do that. And some of them, once they get familiar with the framework, they actually port also the training part. So they can have all of their machine learning workflow working with Burn. So it can be people with Python background or Rust engineer, I'm not sure. But I think this is the main traction point.

• 34:00Chris Benson?

  I will offer kind of a burn newbie perspective on that myself. When I ran across burn and reached out to you, I was really excited about it. In part because as this industry is maturing and affecting as a, you know, many other vertical industries out there. We are seeing AI capability being pushed out from only being, you know, in data centers and stuff out into onto the edge. And you can define the edge in many, many ways, obviously. But the place where processing is happening and even training is happening is evolving over time.

• 34:36Chris Benson?

  And if you look at businesses and their other use cases, the, the fact that they are, they need AI in all these other industry things that they're doing. All these other businesses, they may be, you know, platforms that are mobile, like such as we have, you know, autonomous cars out these days. And you name it, all sorts of stuff that are increasingly relying on AI. And they're turning because those are autonomous things, they need the performance. In many cases, the safety and low level performance capability that Rust offers. I know that I got super excited when I came across burn because I'm in this AI world, but I'm also in this high performance, things moving around time and space world as well.

• 35:18Chris Benson?

  And being able to combine those into one, have one language that is able to do both at the same time. And deploy out to the edge in a very safe way and highly performant way was hugely exciting. And it's been a point of conversation that I've had with colleagues for quite some time. So I think you've hit a sweet spot with burn that is going to get probably as people become aware of it. You'll get a lot more uptake because it solves what would otherwise be a big problem that they're going to be faced with, you know, in the years ahead.

• 35:49Speaker 04

  Yeah, and I think it's not just about like, there is a good, a good amount of solutions to just deploy inference model like with PONIX and stuff like that, but it's not going to cover the training part. And I think it's valuable to be able to do training everywhere. Like maybe the next generation of model, you're going to call backward during inference. We don't know that. It's cool to have like one tool that you can do both on any platform.

• 36:15Daniel Whitenack?

  As you kind of look to the future of the project itself, I maybe have kind of two elements to this question. What are some of your hopes for what Burn becomes into the future as a framework? In terms of like the sweet spot and what it does really well, what people turn to it for. So what, what is your kind of hope and vision for the project, I guess? And then for yourself in terms of your own work and how you're using the project or other things. What is your hope for the future? You have your own interests, obviously, in terms of developing AI-related applications. So I'd love to hear both of those things if you have a comment on them.

• 37:01Speaker 04

  I think I would like Burn to be widely used for maybe complex model. I think Ross really shines when you've got complexity. So if you've got this convolutional neural network with just a few layer, maybe the benefits of using Ross isn't as massive, maybe for deployment. But if you've got like a big models and a lot of complexity in the building blocks, then I think Burn will shine in that place. So I would like to see like innovative new deep learning application being built with it, as well as maybe just normal deep learning models like that we're familiar with, like ResNet, Transformers, all of those ones, but deploy on any hardware.

• 37:44Speaker 04

  So that everybody can run maybe locally some models, maybe not the big ones, but at least the smaller ones. And what I would like to do with it is maybe more research. Like I said previously, on maybe bigger models, maybe asynchronous neural network, like try to leverage the culture and nature of the framework.

• 38:04Daniel Whitenack?

  Yeah. And as we kind of get close to an end here, just for those, because it is a podcast, people are listening in their car and maybe taking mental notes of some things or on their run, where do people go to find out more about Burn? And what would you suggest, let's say it's a newbie to Burn, what should they do to get familiar with it and try things out? So where do they go? And what would you suggest they start with?

• 38:35Speaker 04

  I think the best place to start is to go to the website. So it's just burn.dev, pretty simple. And from there, you can just go on the book that we spoke about and just follow along. If you are not familiar with Rust, we're going to provide links so that you can get familiar with the language. And then you can come back afterwards, follow the rest of the book. And if you're interested, you can also go to the GitHub, try the examples. You can run them with one command line. So you can try to do N friends or to event launch a training on your own laptop. So that can be great. So yeah, that would be the place I would go to start.

• 39:16Daniel Whitenack?

  Awesome. Well, thank you so much for taking time to join us and not Burn us. You are very kind. So thank you. Thank you for your time. We're really excited about what you're doing and hope to have you on the show, maybe next year or sometime to see all the exciting things that are happening in deep learning and Rust and Burn. So thanks so much, Nathaniel. Thanks a lot, man.

• 39:40Speaker 04

  Thanks to you for having me.

• 39:45Speaker 00

  If you enjoy the music you hear on Practical AI, you'll be happy to know we've released two full-length albums for purchase or streaming. Just search for Changelog Beats in your music app of choice and check them out. Volume Zero is called Theme Songs, and it includes special remixes in addition to the classics. And our first volume is called Next Level, featuring many of the video game-inspired tracks you've heard on Changelog podcasts over the years. Check us out, Changelog Beats. Thanks once again to our partners, fastly.com, Fly.io, and TypeSense.org. That's all for now, but we'll be back with more Practical AI goodness next week.