Training AI Models Without a Billion-Dollar Data Center | Steffen Cruz of Macrocosmos

Training a frontier AI model today requires hundreds of thousands of GPUs, months of compute time, and a budget that only a handful of companies on earth can afford. Steffen Cruz, co-founder and CTO of Macrocosmos, thinks that model is about to break, and he's spending his time building what comes next. His project IOTA, operating within the BitTensor blockchain ecosystem, uses distributed training to split large language models across thousands of devices located around the world, coordinated by blockchain, and powered by surplus cheap energy wherever it exists. After nine months of research, the system can reproduce baseline benchmark performance using what Cruz calls "wonky vegetables" - unreliable, churning, globally distributed compute - and turn it into something indistinguishable from centralized training if you use the right approach. The conversation with Craig Smith covers the mechanics of how this actually works, why the blockchain's role is far narrower and more practical than most people assume, and why the Mac mini stockpiling trend creates an unexpected supply of distributed compute that can earn passive income when idle. Cruz's target: a 70 billion parameter model by mid-2025, trained at 10-20% of what it would cost through a hyperscaler, and aimed squarely at the legal firms, hospitals, and cash-strapped startups that have been waiting to train their own sovereign models but couldn't afford the price tag. Subscribe to Eye on A. I. for weekly conversations with the people building and deploying the future of AI.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Craig S. Smith?

  The blockchain, in effect, is a registry of addresses, and it points toward the actual assets that are being registered. There's no training data on the chain, there's no compute on the chain.

• 0:15Speaker 01

  Today, we are training models, we're training multiple models all at once, using pockets of cheap energy, which translates into cheap compute. By the middle of this year, I would like us to reach 5,000 compute nodes. I think this is respectable-sized cluster that you can train a model that will get people's attention and make them understand that there's a lot of utility in this technology. I think that's an important milestone for us.

• 0:40Speaker 01

  So my name's Stefan Kruse. I am the co-founder and the CTO of Macrocosmos. I hold a PhD in subatomic physics in the University of British Columbia. So I was a physics researcher for the beginning of my career, and then I pivoted to AI when it became apparent to me that there's a lot of opportunity for scientists that are just about to graduate from their postgraduate studies. And there's just, there's an entirely new science sort of being born and developed in real time for us. And I think my decision is one that a lot of other people have made as well, where they want to go from being in what can feel like the sort of the old-school machinery of traditional science in academia,

• 1:24Speaker 01

  where things move slow and your contributions can be very narrow and sparse, to becoming a pioneer of this entirely new, exciting thing, which is not only abstract and theoretical, but it's being applied in new and interesting ways constantly. So when I saw that, I concluded my PhD, and I decided to leap, leapt out of academia and into sort of the application of AI and also physics in various private sector domains, such as manufacturing and systems optimization. Following that, I discovered BitTensor, which is a very interesting AI blockchain project. And ostensibly, the purpose of BitTensor is to allow AI to be developed in a way that

• 2:06Speaker 01

  is fully democratized and is sort of done in a global way with incentives. And I thought that was such a fascinating proposal that I decided to join the network. I've been within the BitTensor ecosystem for around three years now, and I have been relentlessly experimenting and building within BitTensor the kind of things that I think would make a lot of sense and would bring a lot of value to the world, using the blockchain as a prerequisite tool to enable AI research to be done in various ways and scales that would be very difficult otherwise.

• 2:40Craig S. Smith?

  Yeah, and specifically, well, first, let's talk about BitTensor. But before you began, we were talking about how there are a few of these blockchain ecosystems for either training or marketing AI models or AI applications. And I'm familiar with SingularityNet. BitTensor is much larger, is that right? Is it the largest of these or one of the largest? And how many are there?

• 3:17Speaker 01

  I believe so. So BitTensor is actually over a hundred projects under a trench coat. And I think that's also why it feels so encompassing and so large. And this is what is very interesting about BitTensor is it doesn't seek to solve a specific narrow problem in the AI industry. It's not like trying to solve something like, how do we provide people with access to models? Or how do we train models? Or how do we do one of a plethora of different things? It actually has done a lot of foundational work on how do we make it possible to build a base layer for people to come and solve just about anything they can imagine in a way that it uses the blockchain as a reward mechanism, as a coordination layer, but then kind of BitTensor does a very good job of stepping out of the way and letting people's creativity drive them through.

• 4:03Speaker 01

  To to build what they want on top of it. So so BitTensor feels very big and very uh sort of like it's everywhere because there are today, there's a hundred and twenty eight different teams that build on top of BitTensor. They're all building something different. A lot of them choose to work within AI, but there's also some that work outside of AI as well. And so for that reason, I think that BitTensor punches well above its weight.

• 4:25Craig S. Smith?

  Yeah, and and for those who are not familiar with the blockchain and even me, uh the blockchain i- in effect is a registry uh of addresses, right? And it points toward the actual assets that are being registered. I mean, this is the thing about blockchain art that was such a uh uh craze for a while that you're not actually buying a piece of. I remember Trump was selling uh these

• 5:00Craig S. Smith?

  I think playing cards or something like that on the blockchain and uh you're not actually buying anything. You're you're registering your ownership of something.

• 5:12Speaker 01

  Exactly.

• 5:14Craig S. Smith?

  But the actual asset is a digital asset that can be copied millions of times. It's up to you then if you want to enforce your ownership to track people down and and uh and that sort of thing. So uh so in in uh in the the projects that I've seen related to AI, they it's a registry that points to an AI uh project, or is there something more happening on the blockchain?

• 5:46Speaker 01

  So in a project like BitTense, the blockchain is used in a number of different ways, but fu- fundamentally, it's it's an immutable database that is globally shared. And that in itself enables a lot of things that Bitcoin pioneered some fifteen years ago. Um what what becomes possible when you have this sort of shared globally distributed state is you can create a store of value and then you can start building higher and higher. Order commodities on top of that. So Bitcoin sort of addressed the most principal version of this. But what was demonstrated as a result of that is you can actually aggregate a huge amount of compute,

• 6:26Speaker 01

  or let's just call it effort. Once you have this interesting distributed sort of system architecture, people come to it and because there's economic incentives involved, people are constantly innovating and optimizing and they're trying to find a way to sort of percolate to the top and to become the most competitive people that are doing whatever this mining task involves. In the case of Bitcoin, the mining task is just guessing random numbers faster than anyone else can. That's why you just throw a more powerful computer at it and you become a more competitive miner on the Bitcoin network. And other crypto projects have tried to orient that effort, that sort of collective coordinated effort of all the participants. They've tried to orient that towards different tasks, whether it's distributed file storage

• 7:10Speaker 01

  or creating a global residential proxy network or solving data scraping from the web. And all of these things are really interesting, whereas what BitTencer uses the blockchain for fundamentally, as you described, the registry component is very important. It's also sort of a shared synchronization clock. The notion of blocks and a blockchain gives everyone sort of a drumbeat that you can actually anchor work to, which is an integral part of a lot of our research on BitTencer. We have this sort of shared clock, which the blockchain regulates. But ultimately, you can do as much or as little on-chain, as they call it, or off-chain as you want. And I think some of the benefits of doing things on-chain is you have complete auditability.

• 7:53Speaker 01

  You can see precisely what the details were that went into a specific piece of work or transaction. So it gives you transparency. The problem with that, of course, is if you've got billions and billions of transactions being written to a database, your database gets very big very fast. And then it becomes kind of unyieldly for people to actually have a copy of, which can interfere with the true decentralization ethos of it, and you end up with just a few copies. So I think that there's a pragmatic middle ground where you store exactly what you need. More to ensure the core parts of the work that you care about are captured and and transparent for everyone to order, review and verify. Whereas the rest of it, you can you can store elsewhere, you can store offline, you do a million

• 8:36Speaker 01

  other things with it. So I I hope that adds a little bit of light to the question, Greg.

• 8:39Craig S. Smith?

  Yeah, in your case with um macrocosmos, you're providing compute for training models. Is that right? Explain precisely what macrocosmos does.

• 8:52Speaker 01

  Happily. So today we operate three subnets in BitTensor. Each subnet is, you can think of it as effectively a different project, a different service. And our three different subnets, I would consider them to be three different three different axes or three different use cases for the blockchain that I think are very potent and powerful. So one of them is just the aggregation of raw compute at scale. And if again, harking back to what I described about the Bitcoin network, I think one of the most important things of our time is set to be AI itself. And those who train the models and own the models will have an enormous advantage.

• 9:32Speaker 01

  And so a big part of the purpose of BitTensor is to provide an alternative way for us to create these models, to train these models, to deploy these models such that they're not held in walled gardens and we're having to pay for the privilege of access to them. But we want to reinvent that in a way that's more democratic and also borderless, something that censorship resistant, something that cannot be controlled. And so our primary, our flagship project within BitTensor involves what's called distributed training of large language models. And that's quite a quite a mouthful. Distributed training is quite different to how a lot of the frontier AI labs train models today.

• 10:15Speaker 01

  Typically, you have huge warehouses stacked from floor to ceiling with GPUs, which are specialized computers for training AI models and serving AI models. And you've got hundreds of thousands, hundreds of thousands of them in the largest data centers in the world. And they're all plugged up together with extremely high data transfer speeds. And as a result, you can basically use the whole thing as a single computer. And this is fantastic, and it's been demonstrated that you can just scale the amount of compute and the amount of data you throw at a model, and they get predictably better and better and better. This is called a scaling law. Now, there's an alternative to this, which is called distributed training. Distributed training doesn't rely on having a single warehouse stuffed with computers.

• 10:55Speaker 01

  But actually, i- it's based on the premise that you can train an equivalent model using computers that are distributed around the world. And now, this has a lot of very interesting outcomes. When you don't do everything in a single data center, one of the first thing is, one of the first things is that the, the capex of actually building this massive warehouse in the first place is massively alleviated. There's also the local energy that is required to run one of these massive data centers can also now be distributed around the world in a way that creates less impact on local communities. Generally, much, much less detrimental to the environment. And lastly, it allows you to do something that is actually fundamentally not available. Once you've built your data center and stuffed it with GPUs,

• 11:40Speaker 01

  sort of the cost of training models is already baked in to that initial build process. Whereas in the case of distributed training, which is what we care about, we can actually perform what is effectively cost arbitrage. So if there's, if there's a surplus of energy in Iceland, very, very cheap energy, we can actually use that pocket of energy, even if it's only for 12 hours of the day. And we can target a lot of that compute and use it in a very elastic way. And in fact, that's what we do today. We are training models, we're training multiple models all at once, using pockets of cheap energy, which translates into cheap compute. So it has all of these different economies of scale, which are certainly unusual, but are becoming more and more accepted as a viable alternative.

• 12:23Speaker 01

  And as our appetite for bigger and bigger models is only going to increase as the years go on, we're going to find that we need to start looking at alternatives before we hit a hard ceiling on a lot of this. Because things like the Stargate project and the Colossus project, these are multi-billion dollar GPU build outs.

• 12:41Speaker 01

  And so we think that just like the the fundamental physics experiments like the Large Hadron Collider, at some point, you require the budget of a nation's state. It's $20 billion to build a bigger ring to smash protons or electrons. You you need to start thinking about different experiments because it it just becomes unpalatable eventually. So so we're we're trying to get ahead of this problem. I think that by 2028, we're actually the world, the mainstream, the Overton window about training models is going to shift and we are going to have to start thinking about how do we do this in a way that arbitrage is more efficient in both cost and energy. So we're we're trying to do a lot of the early work right now on that. And BitTensor is a wonderful place for us to do this research because it we are able

• 13:22Speaker 01

  to use the blockchain to reward people that contribute their compute to our training experiment.

• 13:28Craig S. Smith?

  Let me ask a couple of questions. I had a guy on the program who was doing something interesting. He was, and I don't remember whether it was blockchain based, but but he would aggregate spare compute from data centers or, you know, whether on premise or or, you know, independent data centers around the world and then offer that compute to AI projects at a at a cheaper price than than the hyperscalers or the big clouds could offer.

• 14:13Craig S. Smith?

  And and so is it that's one question, is it like that and you're just tying it to the blockchain to to organize it and distribute it? The other is, you know, for a long time, people were talking about federated computing so that you wouldn't have to lose control of your data, but you could make it available for training models. And again, that that was using the blockchain to register the the data, I guess.

• 14:56Craig S. Smith?

  So first of all, is it related to either of those concepts? And and then the the deeper question and the one even talking to the people I've spoken to, I've never. Really gotten a hold of the chain again is just a registry. There's no training data on the chain. There is no compute on the chain. The compute resides, as you said, uh, maybe in Iceland. What is the link between the chain and the compute in Iceland?

• 15:32Speaker 01

  So starting with your first questions, the relationship between those two examples you gave and what we're doing. I would say that they're cousins, but they're not, uh, they're not siblings. So they're, they're a little bit different. For example, federated learning uh is built on the idea of anonymizing client data, but being able to continuously train models. We were working with a slightly different version of this, which is not so much the edge devices providing training data today, but the edge devices are providing a train compute. I suppose would be a way to think about that. And secondly, I think regarding what BitTensor actually does or what it's, what the role of the chain is,

• 16:15Speaker 01

  is at its core, it's really not that revelationary at all. At its core, what the blockchain is effectively doing is it's providing a trustworthy, transparent record for everyone to audit, especially when there are, when there's code that's being run on the blockchain as well. People understand how this code is going to interact. It can't be tampered with, which means that it's predictable. And that's a proxy for safety and trustworthiness. So if, if someone had already deployed a smart contract, for instance, and we, we're not actually building a smart contracts at Microsoft, but I guess just to illustrate the point, if someone has written something in the smart contract, then the creator of the smart contract can't later change the terms and conditions on you.

• 17:01Speaker 01

  It, it, the point is it's a, it's a modular block of code that will run under these conditional expressions. And that makes it, um, that, that makes it something that you can effectively count on. So I believe I've given you a bit of an illustration of one of the projects that we're working on a bit tense, so which is distributed training. This project is called IOTA. And as I mentioned, I think that it's a precursor to some very powerful downstream technologies that we may found are an instrumental part of the way that we create AI in the next decade. Yeah. We really do believe that there's also a real change in the way that people are thinking

• 17:40Speaker 01

  about personal agents and personal compute. So I've seen personally in the last few months, a lot of people are starting to stockpile Mac minis. I'm not sure if you've seen this trend, but basically now that agents are starting to become kind of economically useful a little bit, we're seeing the beginnings of it, I think is a fair appraisal. What we're starting to see now is people want one of these things running for them privately around the clock, day in and day out. So they're buying a dedicated computer and on that computer, your personal agent is running and it's checking your emails. It's maybe doing some online shopping. Maybe it's doing some work for you on the side or some hobbies. And this is wonderful use case. And this is also a really interesting place where our technology of IOTA developed by Macrocosmos

• 18:24Speaker 01

  comes in because all of these computers that people have now built at home. Well, there's two ways to think about it. That's your personal agent that is creating value for you personally. But wouldn't it be great if that compute could earn passive income? Almost like you own a property and you can Airbnb it out when you don't need it. And this is actually what we're offering as we develop IOTA is we've created something called Train at Home, which is people that have unused devices at home. Let's say MacBooks, Mac minis, or even consumer GPUs. They can plug those into our network and that means that they become part of this global supercomputer. So we can use them for training, training models, which we can commercialize. And they can also basically create some passive income just from having that device sitting around that was otherwise underutilized.

• 19:11Speaker 01

  So I think there's something very parsimonious about that arrangement of things. And the more people that are choosing to buy personal devices to run these agents, you realistically don't need the agent running 24 hours a day. You need pockets of productivity. I want to make sure it, you know, plans my weekly agenda. I want to make sure it checks this, that and the other. Maybe there's only four hours of the day that you actually need this thing. And you can actually get returning your initial investment within, you know, 12 days, 30 days, by plugging it into a network like this and also being part of something that I believe has a great purpose, which is the democratization of AI itself. So it's a really interesting intersection of timelines right now with agents becoming demonstrably more useful and engagement increasing. And we basically can use all of that consumer compute.

• 19:55Speaker 01

  We can glue it together and we can train models at greater and greater scales and greater and greater utility. So, yeah.

• 20:03Craig S. Smith?

  Yeah, yeah. As a matter of fact, I'm running Mac Mini stockpiling. I think started with Open Claw. And

• 20:11Speaker 01

  Exactly.

• 20:12Craig S. Smith?

  I'm running Open Claw on this computer, which I know is a bad idea. And I have an unused Mac Mini upstairs. So I'll probably go fix that after this call. But the, so, so how does somebody use this? Well, first of all, yeah, back to the point of, so you've got compute

• 20:40Craig S. Smith?

  around the world that is plugged into the network.

• 20:47Craig S. Smith?

  Do people who are using a macro cosmos, do they need to know what's how the, what the link to the blockchain is?

• 20:59Speaker 01

  Absolutely not.

• 21:00Craig S. Smith?

  Yeah. And, and the blockchain is just keeping track of where the compute is and what's available at any point in time. Is that right?

• 21:13Speaker 01

  Yep, and it's just responsible for rewarding participation. So it's making sure that any contributions made by anyone's devices are fairly rewarded. That's really the way I think about this. The blockchain is just a really convenient way of taking care of compensating people for their contributions. The actual onboarding experience for people is we keep this as low cognitive load as possible. Right now, it's a one-click app store download. This thing will just run passive in your machine, adding some more progressively more useful user experiences. Things like telling it, "Oh, you can only use my computer when I go to sleep." So by default, it's disabled until 10:00 PM and it's only enabled from 10:00 PM till 6:00 AM.

• 21:59Speaker 01

  So that will be a simple user control that will, you know, get it out of the way of when you're actually trying to be productive with your machine. That's assuming it's your primary machine. If it's a secondary machine that just sits at home, we're also creating a way for agents to decide, "Oh, I finished my work. I'm not going to need to do anything for four hours." So the agent just decides, "Hey, I'm just gonna go and make 20 bucks." And that would be nice to come home from work too, right? It'd be nice to come home from work and you're like, "Hey, what have you done today, Claw?" And it's like, "Oh, hey, I finished all my work by 9:15 AM and you were gonna be out till 5:00 PM." So I thought, well, I thought I'd just shop around and see how I could make a little bit of side cash. And here you go. So I do think that as our computers become more autonomous, you know, thinking about our relationship with computation,

• 22:41Speaker 01

  it's gonna change fundamentally and our computers are not gonna sit there waiting. They're gonna be very proactive and they're gonna be very resourceful, especially when you've got these clever, clever little agents that are running the show inside of the machine. And as a result of that, I think we need to really rethink what your computer can do for you. You know, our relationship to that world. And I think bringing this back to the top about BitTens. You can think of BitTens as it's just a bunch of places where you can use that machine and you can go and make some income from it. Whether it's providing something like human intelligence or whether it's training models or whether it's detecting if an image is real or fake. You can almost think of it as a mechanical Turk for agents and humans alike.

• 23:26Speaker 01

  Where you can go there, you can monetize your skills and your experience or just your raw compute.

• 23:31Craig S. Smith?

  Yeah, and for the training.

• 23:35Craig S. Smith?

  How does, let's say I have a large model that I want to fine tune. How do I do it with Microsoft Cosmos?

• 23:49Speaker 01

  So today, our work is focused on the first stage of training models specifically, which is called pre-training. So pre-training has historically been by and large the most computationally expensive part of creating a model. It's the part where they systematically inhale the entire internet, which usually takes months and tens of thousands of GPUs. Beyond that, fine tuning is comparatively a much shorter computational task. So I think it's the essence of pre-training requiring a long time horizon workloads. That is actually why we're so interested in moving this from a centralized to a decentralized context.

• 24:31Speaker 01

  And there's a lot of pioneering research that needs to be done that we hope will be very useful. Even outside of the Web three world, we hope that this is really useful research to anyone out there that is interested in economically or capital efficient ways of training models. We hope that this is a contribution to the field. That's really how I see this because what we're doing at the root of our work is we're taking a network of very unreliable compute. People are joining, people are leaving. It's just constantly churning over and churning over. How do we create something that is persistent and stable out of something that is fundamentally so noisy and unstable? And this, and it's, and it's core, that's the problem we're trying to solve.

• 25:14Speaker 01

  And it just so happens that the, the, the, the use case for this persistent compute fabric we're creating is trading models. But there's nothing stopping that from doing something else. Perhaps there's a world where universities could use a similar decentralized network for their traditional HPC cluster jobs where they're doing academic research. That's not necessarily AI related at all. It could be bioinformatics, it could be physics. The same thing, the same thing is still, uh, still remains to be true, which is. If you need 10,000 nodes of compute for 12 hours, what's the cheapest way you can get them? And we're trying to build software that basically orchestrates compute around the world to act like it was all plugged into itself and it's a supercomputer.

• 25:58Speaker 01

  And I think it's a, it's a fundamental thing that we're working on that we're applying towards training models, but I hope has a much broader value to the community.

• 26:07Craig S. Smith?

  Right. So on pre-training, how does somebody use it?

• 26:11Speaker 01

  Absolutely. So we're sort of emerging from, from research mode right now. We spent about nine months on IOTA so far. And we've demonstrated that we're able to reproduce a lot of important baseline benchmark performance metrics using this. What we call it the wonky vegetables, sort of the ugly vegetables that you get in the vegetable aisle that no one necessarily wants to buy. We're going to turn that into a premium soup. And we're basically graduated from research mode right now, where we're showing that the two are indistinguishable. If you treat them in the right way, if you use the right spices, it tastes just as good. And what, what we're moving to next is thinking very deeply about what is the, what is the commercialization opportunity for a technology like this?

• 26:54Speaker 01

  Who are our target audiences? We think that, that the world is going to train more models in the next 10 years, not less. We think that researchers, small startups, especially cash-strapped startups, academia, all of these people are interested in training more models. But perhaps for budget reasons, it's very hard for them to do so. So we're, we're sort of targeting the, the commercialization of IOTA as, as two things primarily. One is what we call supply side. So supply side would basically mean people that have surplus GPUs. So the NeoClouds, the hyperscalers, perhaps they have 10,000 GPUs in their inventory, but they can only rent out 9,000 of them at a given time. Or 9,500 of them.

• 27:35Speaker 01

  Well, for us, them plugging in their compute into our network allows them to increase their overall utilization, which goes straight to their bottom line. So they'll be happy with that. And even better, that if you actually to look at the usage of all of those GPUs that a hyperscaler has, you might find that, well, it's, it's rented out for four hours. And then it's not rented for two hours and then it's rented out for four hours. And that little two hour gap is exactly what we're trying to utilize. These short, interruptible bursts of compute. And we turn that into, again, a continuous stream of compute is exactly what excites us. So for the supply side, it's basically anyone that has GPUs. What they have to do right now is if they can't rent it out, there's sort of a buyer of last resort on the market.

• 28:20Speaker 01

  They'll rent out those GPUs that cents on the dollar for inference tokens. And we can basically say, well, hey, instead of doing that, how about you plug them into our network? And we can basically give you a better margin. Because trading is a higher order commodity than inferencing. And as a result of that, we believe we could pass on the benefits to them as suppliers. So that's the supply side. On the demand side, as I mentioned before, we have researchers, we have startups. We have all kinds of different profiles of users that we know right now they're training models all the time. What we want to do to that, for those guys, is basically provide them with an interface to train models in a way that's recognizable to them. So there's already very popular libraries like PyTorch, like TensorFlow.

• 29:04Speaker 01

  These are very popular libraries that are basically used by everyone in industry that's training models. Well, we want to make it as simple for them in terms of abstractions as if they were just building models in the normal routine way. So with no additional cognitive overhead, you basically say, I want to train a model. You're not going to painfully specify, oh, I want to use a Nigerian GPU for 12 minutes. Of course, you're not, right? What you're going to do is you're going to say, this is my, this is my sort of executive level objective. This is what I want to get done. These are the parameters that I want to have precise control over for reproducibility purposes, for deterministic purposes. All of this section down here, feel free to go and arbitrage and get me the cheapest version. And then we have this, now we have a two-sided market, right?

• 29:48Speaker 01

  Now we have a sort of supply side and a demand side that are sort of dancing together where we're arbitraging, offering cheaper rates. And Iota is a technology that basically is this infrastructure layer that is, is the one that brings this compute to market, pairs it up with users that want it. And we believe that that's something that is phenomenally valuable.

• 30:08Craig S. Smith?

  Yeah, well, and then how does the, the, under the hood, how do you get the, the, the compute to the, to the model? Cause they're not sending the model around all over. Uh, yeah.

• 30:26Speaker 01

  The way that you get the compute to the model is, it's, it's effectively a, an orchestration deployment layer. Something reminiscent of Kubernetes, but Kubernetes that deploys to heterogeneous compute nodes around the world. So it's, it's a dynamic deployment layer that basically takes some containerized code that the user writes, sends it out to those machines, and then very importantly ensures that those machines around the world have a tunnel to communicate with each other. And that tunnel is the part that makes them act like one big blob of compute instead of 50 different random pieces of compute. So it's effectively that it's, it deploys anyone's code as an interactive system or a network of compute that can, that is sort of geospatially aware.

• 31:13Craig S. Smith?

  Right. And, uh, and that orchestration layer, um, is, where is that based?

• 31:22Speaker 01

  The orchestration layer is something that we maintain and operate. So you would effectively dispatch your code from your laptop or anything like that. And your code would then be sent securely to these backend nodes, wherever they might be distributed around the world. And then you would effectively, as a user, you would have the ability to monitor logs, monitor trace data, the usual sort of dashboarding, tooling, things like that as the user. But effectively what's happened is your code has now been sent out, cloned, replicated to all these different copies. And all of those copies are hosting a section of the model, which is what makes IOTA very, um, in my opinion, very novel.

• 32:04Speaker 01

  We, we don't actually have each of these compute nodes running a full copy of the model. They're actually running a small sliver of the model. It's called model parallelism. And in effect, what this means is you can train really large frontier sized models using very small building blocks. Like you can build a huge Lego tower out of small pieces. It's the same idea. And the way it works is in order to train the model in its entirety. You have to root information from all of the sections of the model and knit that together. And as a result of that, you can basically train this this sort of distributed model architecture as if it was all stuck together in one place. And that's that's why it's taken us nine months to get out of research mode, to be honest, Greg. It's it's quite a formidable task.

• 32:48Craig S. Smith?

  Yeah, and and is the blockchain important for the synchronization? Because you need a very tight synchronization between nodes of or between different compute, right?

• 33:06Speaker 01

  So the way that we utilize the blockchain today for IOTA's purposes is we need a registry of everyone that's in the network right now. So it exposes them as, hey, I'm an available worker. Here's how you can find me and here's my unique address that I know to identify you unambiguously. So we we use that as sort of

• 33:29Speaker 01

  an authorization layer, an identity layer.

• 33:33Speaker 01

  Also with that, we we have an off-chain layer to IOTA, which is constantly tracking the contributions of that node. And then, once we have computed the total work done by each node, that is sent back down to the blockchain layer where it triggers a payout loop. So everyone that contributed that compute is then rewarded in the IOTA token for their contributions of work. And and this actually can be articulated in a very tokenized context, or even a US dollars context. That all you effectively need is building blocks here is is you need to know exactly who it is that's assigned to this model training experiment. You know where you can find them, you know, how to glue them all together, and you know how much work they've done.

• 34:15Speaker 01

  And if you've got all of those things, you have an operational distributed trading network.

• 34:20Craig S. Smith?

  How large is the network and how do you? How do you grow the network of compute?

• 34:28Speaker 01

  Yeah, so when we began, the source of all of the compute for our distributed trading experiments was people that came to our project in BitTensor, the IOTA project, with, they have their compute, they know what we're here to do. Usually, there's a philosophical alignment, or sometimes they just see it as an opportunity to make some cash. But basically, people show up armed with usually pretty powerful hardware. So the actual, the supply side of computing BitTensor is very much not a concern. People know that BitTensor's there. People know the rules of the game. So we were almost always oversubscribed in terms of people that wanted to contribute to our experiments. However, in December,

• 35:13Speaker 01

  November, December last year, we wanted to go beyond the scale of the network that is supported natively by BitTensor. So the maximum number of people that could be participating in the IOTA experiments was 256, which, if you consider that to be a data center, it's tiny. It's a pebble. So we actually worked around that, and we've now made it effectively a limitless-sized system by, instead of having every unique entity as a unique address on the blockchain, we now keep track of it off-chain effectively. And what this allows us to do now is we can have an arbitrary-sized registry of participants. So we had 2,500 people download our macOS app in the first two weeks since it launched.

• 35:58Speaker 01

  And now all of those people are available at a moment's notice to join our network. And they're paid out identically to the ones that are strictly on-chain with a BitTensor. We have a transparent payout system that people understand. It's a predictable system. It rewards, it directly rewards contribution in proportion to basically hours of work done per day. But the nice thing about this is it's not limited by the amount of storage capacity for the blockchain, or the number of slots that a blockchain can have as miners. We actually decided to just rewrite a lot of that using different rails, so that it's not, it's not limited in that way. And now, as a result, we have 500 or so today running. The amount of work that we have to give the miners to do is actually quite...

• 36:45Speaker 01

  It changes over time. There's times when we run lots and lots of experiments, we train lots of bottles. And there's also time when we only need a small amount of compute and we only pay for what we need. We're not just, the forster isn't just always running. We actually have experiments that are live or we have production runs where we're going to say, this is the model we're going to train. We're going to go find 500 devices around the world. Fortunately, again, the bit tensor piece has made it very easy to find 500 random people in the world with powerful computers that I can just plug into my network, which is the miracle of technology, really. And so we can effectively not worry at all about where that compute comes from. And then downstream, we basically pay everybody out for that work that they've done.

• 37:27Craig S. Smith?

  Yeah. And is there, do you have a sense of scale of comparative scale that that would give me and listeners an idea of of how this would compare to

• 37:44Craig S. Smith?

  to a major data center?

• 37:47Speaker 01

  Yeah.

• 37:48Craig S. Smith?

  Yeah.

• 37:48Speaker 01

  The the largest data centers in the world are measured that the ones that have started to come online are now hundreds of thousands of GPUs. So these are multi-billion dollar projects. They're absolutely enormous. We're not, we're not at that scale yet. But the nice thing is the technology that we're building scales in a very beautiful way. In fact, you know, to go from a day, to go from our network with 10,000 nodes to 20,000 nodes, there's nothing that we fundamentally need to change. We just need to expand the the distribution surface of what we're doing and our infrastructure layer scales to much, much higher capacity. So again, this is the nice thing about not being committed to a bricks and mortar

• 38:30Speaker 01

  build out in the way that the centralized compute paradigm is. In our case, if we want to double, triple or 10x the capacity of our compute network, we simply need to go out, increase the surface of the network. So what I think is in the books for us is by the end of this year, I want, well, by the middle of this year, I would like us to reach 5,000 compute nodes. I think this is a respectable sized cluster that you can train a model that will get people's attention and make them understand that there's a lot of utility in this technology. I think that's an important milestone for us. Something around the 70 billion parameter model size, I think, is when you go from, you know, you graduate from the sort of small models that are proof

• 39:13Speaker 01

  of concepts to things that are actually taken much more seriously and enterprise customers that have been waiting to train an in-house model, whether it's a legal specialist model or whether it's a medical model or something like that. Perhaps they've been waiting for a while, but they couldn't really afford it because it's a very high price tag that comes with training a model of that scale. Because of the discounts that we can offer using this energy and cost arbitrage approach, I think that we're going to start to see a lot more people train their own sovereign models, their own enterprise models. And so by mid this year, it's important to us that we demonstrate that that is possible with the technology that we're working on. And in a year, a year and a half from now, I'd like to go beyond 100 billion parameter models. So the parameter count of the model

• 39:57Speaker 01

  is very much the number that we think about when we consider what it takes to make this technology something people take seriously. I think we need to have a portfolio of models that we can point at and say, these are just as good as the ones that you would have gotten in a centralized context, but at 10% the cost, at 20% the cost. I think then people really start to take notice and it becomes an interesting proposition. And that's where we'd like to get to.

• 40:24Craig S. Smith?

  Yeah. And we were talking about people stockpiling Mac minis. The people that are joining or contributing compute to macro cosmos are GPU people with GPUs, spare available GPUs, right? Not CPUs.

• 40:48Speaker 01

  It's yeah, we're not going to get anything done if we wait for CPUs to do the work, unfortunately. Perhaps there are workloads in the future. As I mentioned before, we can imagine this technology is something that allows you to do more than just model training. We're working on a more fundamental infrastructure problem, not just a model training problem in many respects. And there are a lot of scientific problems, computational expensive, scientific problems that are CPU limited. And in those cases, absolutely, it would be very interesting to apply this technology in those cases. But today, we support effectively CUDA devices and silicon devices, which are Mac Minis, the new MacBooks.

• 41:28Craig S. Smith?

  Is this online yet? I mean, can, are people training on this network yet?

• 41:35Craig S. Smith?

  And if not, when will it be available? Do you have people lining up?

• 41:40Speaker 01

  Yeah, you can find us at iota.macrocosmos.ai. That's I O T A, which I should probably have introduced as an acronym for the incentivized orchestrated training architecture.

• 41:53Craig S. Smith?

  Okay.

• 41:53Speaker 01

  So iota.macrocosmos.ai is where you can learn more about this project. Today, we are, we don't have any customers in yet. As I mentioned, we're coming out of research mode right now. We're trying to really calibrate the system and tighten this thing up. There's a lot of moving parts that we want to make sure we have really nailed down. But we have startups that are already ready and committed to working with us, training models towards the second half of this year, which is incredibly exciting. So we'll have some collaboration results and some early partnership results by the end of summer.

• 42:24Craig S. Smith?

  Okay, great. And in one other question, this is for compute, but we spoke about federated computing. Could you register data on the network or make data available? And then for pre-training and then have Macro Cosmos send the model around to different

• 42:55Craig S. Smith?

  blocks of data to train the model?

• 43:00Speaker 01

  It's a very interesting idea. One of our other projects in BitTensor is actually a web-scale data scraping service, where we decentralize the efforts of hundreds and hundreds of miners that scrape social media data for us. And we can use that for many reasons, anything from journalism to marketing, brand analysis, all the way to AI model training. So we actually have an entirely standalone project which is dedicated to data scraping, which we think is an important part of our... I call it sort of virtuous cycle. We have data, we have compute. We also have a lot of what we call innovation networks, which I didn't get much time to talk about today. But to your question directly about whether we could actually outsource client-side data

• 43:44Speaker 01

  and use that for model training, my answer is we already are. We actually just have it in a dedicated, highly scalable system called Data Universe.

• 43:54Craig S. Smith?

  Okay, okay. Well, this is fascinating. I'm gonna be paying attention to see how this scales. Do you think this model will be replicated by others? If you...

• 44:11Speaker 01

  Yeah. I think it's very important that it is. Do you mean the technology or the actual downstream model artifact?

• 44:20Craig S. Smith?

  The technology, the technology. Because, as you said, the appetite for compute, for pre-training compute is gonna grow. The, you know, the number of models out there is only gonna grow. And compute is expensive and a lot of it's underutilized.

• 44:47Craig S. Smith?

  So it just feels like, like there would be other people doing this. And another question is, are you working with any of the big cloud providers to maximize their GPU utilization?

• 45:09Speaker 01

  To your first question, I certainly, I see other teams right now that are working on very similar projects. Some of them are focused specifically on model training, or others are more oriented towards this orchestration technology. I think we're one of the very few that's combining those two in the way that we're doing it right now. So I do expect the world of devices, you can call it the Internet of Things, you can call it whatever you want, but I think that there is a second law of thermodynamics at play here, which is things get more connected over time. And I think that especially when there is a lot of value added and a lot of potential to having the orchestration of multiple devices to do higher order workloads.

• 45:53Speaker 01

  So in our case, there's simply not enough memory on one Mac Mini to train the models that we want, which is why we have to do all this work to glue them together. And there's many, many other problems where there's just not enough resources available on one device to do what it is that you need. So I think that there's a lot of other industries that are already thinking carefully about how can we get access to the kind of compute in the specific topology we need it. And I think that what we're building is going to be very interesting to them. Devices, as I said before, devices are only going to get more intelligent and more autonomous, and they're going to become much more useful if you can stick them together like Lego bricks and make these bigger pieces out of them, these different constellations. So I see a bright future for that effort right there. Whether we're the ones that really take this all the way to the top of the mountain,

• 46:37Speaker 01

  or we get halfway there, I'm proud of the work that we're doing. And I think that it's, again, I think it's very valuable for everybody that we work on this.

• 46:46Craig S. Smith?

  Okay, great. So iota.macrocosmos.ai

• 46:56Speaker 01

  Yeah, you can find everything you need about us just at macrocosmos.ai. And you can find all of our projects and our research in decentralized AI.