#243 Greg Osuri: Why the Future of AI Depends on Decentralized Cloud Platforms

This episode is sponsored by Indeed. Stop struggling to get your job post seen on other job sites. Indeed's Sponsored Jobs help you stand out and hire fast. With Sponsored Jobs your post jumps to the top of the page for your relevant candidates, so you can reach the people you want faster. Get a $75 Sponsored Job Credit to boost your job's visibility! Claim your offer now: https://www.indeed.com/EYEONAI Greg Osuri's Vision for Decentralized Cloud Computing | The Future of AI & Web3 Infrastructure The cloud is broken—can decentralization fix it? In this episode, Greg Osuri, founder of Akash Network, shares his groundbreaking approach to decentralized cloud computing and how it's disrupting hyperscalers like AWS, Google Cloud, and Microsoft Azure. Discover how Akash Network's peer-to-peer marketplace is slashing cloud costs, unlocking unused compute power, and paving the way for AI-driven infrastructure without Big Tech's control. What You'll Learn in This Episode: - Why AI training is hitting an energy bottleneck and how decentralization solves it - How Akash Network creates a global marketplace for underutilized compute power - The role of blockchain in securing cloud resources and enforcing smart contracts - The privacy risks of hyperscalers—and why sovereign AI in the home is the future - How Akash Network is evolving from a resource marketplace to a full-fledged services economy - The future of AI, energy-efficient cloud solutions, and decentralized infrastructure The battle for the future of cloud computing is on—and decentralization is winning. If you're interested in AI, blockchain, Web3, or the economics of cloud infrastructure, this episode is a must-watch! Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A. I. Twitter: https://twitter.com/EyeOn_AI (00:00) Introduction & The Biggest Challenges in AI Training (02:36) Greg Osuri's Background (04:50) The Problem with AWS, Google Cloud & Traditional Cloud Providers (06:40) How To Use Blockchain for a Decentralized Cloud (10:17) Akash Network's Marketplace Matches Compute Buyers & Sellers (14:42) Security & Privacy: Protecting Users from Data Risks (18:25) The Energy Crisis: Why Hyperscalers Are Unsustainable (21:51) The Future of AI: Decentralized Cloud & Home AI Computing (26:42) How AI Workloads Are Routed & Optimized (30:24) Big Companies Using Akash Network: NVIDIA, Prime Intellect & More (45:49) Building a Decentralized AI Services Marketplace (55:09) Why the Future of AI Needs a Decentralized Cloud

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  There are two big challenges for AI training, right? The one was data. We didn't have data. There's a data limit as to how much you can get data to train. And second was energy. What we saw with DeepSeek, it can use synthetic data. So it's very, very amazing using synthetic data. I can actually solve the data problem. But what we cannot solve is the energy problem. I think that's why it's very, very important. If you're doing training to focus on distributing your training runs versus trying to go with the traditional mechanism of centralizing your training runs. Because we're going to hit a cap in two years and we have no solutions.

• 0:31Craig S. Smith?

  I'm past the point of looking for jobs, but I'm not past the point of looking for people to hire. And when I need to do that, I turn to Indeed. Imagine you just realized your business needed to hire someone yesterday. How can you find amazing candidates fast? Easy, just use Indeed. When it comes to hiring, Indeed is all you need. You can stop struggling to get your job post seen on other job sites because Indeed's sponsored jobs help you stand out and hire fast. With sponsored jobs, your post jumps to the top of the page for your relevant candidates.

• 1:14Craig S. Smith?

  So you can reach the people you want faster and it makes a huge difference. According to Indeed data, sponsored jobs posted directly on Indeed have forty-five percent more applications than non-sponsored jobs. Plus with Indeed sponsored jobs, there's no monthly subscriptions, no long-term contracts, and you pay only for results. How fast is Indeed? In the minute I've been talking to you, twenty-three hires were made on Indeed, according to Indeed data worldwide. There's no need to wait any longer. Speed up your hiring right now with Indeed.

• 1:54Craig S. Smith?

  And listeners of this show will get a seventy-five-dollar sponsored job credit. To get your jobs more visibility at Indeed, to get your jobs more visibility, go to indeed.com/eyeonaai. Eyeonaai, as always, all run together, E Y E O N A I. That's indeed.com/eyeonaai right now.

• 2:22Craig S. Smith?

  And support our show by saying you heard about Indeed on this podcast. Indeed.com slash I on AI for a $75 sponsored job credit.

• 2:36Speaker 00

  Good to see you again, Craig. Great to be here. Thank you so much for having me. My name is Greg Osiri. I'm the CEO and founder for Overclock Labs, a core contributor to Akash Network. Background, been a programmer all my life, been an open source developer for little over 15 years. And early days of my career helped contribute to this container native ecosystem. A lot of my software I've written is still by use by projects like Kubernetes, Docker and whatnot. And we, you know, big users of the cloud and we noticed that cloud, as we know,

• 3:17Speaker 00

  is increasingly becoming more important in our daily lives, considering most of the workloads now are hosted on the cloud. And we felt as, as it gets more prominent, it also is getting more closed. So we felt the need to have a transparent and open cloud. Beginning with how the resources are priced, to how the resources are secured, distributed and whatnot. And in line with what we saw with Kubernetes and what we saw with Docker and Linux really, we wanted a similar cloud. And that's really our work on Akash began.

• 3:58Speaker 00

  And it began as an open source project. It still is a very, very mature open source project at this point. But, you know, if you go back, the name Akash means in Sanskrit, the sky. The sky is where the clouds are formed. So that's, that's where the name came from. And, you know, we, you know, we were inspired by this super cloud concept by Cornell in 2015, where they prophesize that you can actually, if you decouple the resource layer, that means the layer where the resources are acquired from the control layer, the control plane. You can effectively create a mechanism where Anyone can be a resource provider.

• 4:43Speaker 00

  And that resource provider doesn't necessarily needs to be the controller. So that's the concept of super cloud. So really, and we created Akash as this a mechanism to bring the control plane in a decentralized mechanism where no one controls the control plane. It is a community that essentially operates on consensus. And a resource plane where anyone with compute can plug in and offer that to a market at incredible prices today. Akash is perhaps the fastest growing cloud and the most cost efficient cloud. Apples to apples for compute resources, you have 10 times the cost savings compared to your Amazon's the world.

• 5:26Speaker 00

  Whereas for accelerator resources, be GPUs, you have nearly two to three times cost advantage compared to your traditional cloud. So that obviously is contributing to the current incredible growth we're experiencing.

• 5:41Craig S. Smith?

  Yeah. So as I understand it,

• 5:47Craig S. Smith?

  part of the concept is that there's a lot of unutilized or underutilized compute in the world.

• 5:55Craig S. Smith?

  Data centers scattered around the world that are are not being used at full capacity. And those people can make their resources available to anyone in the world, just as Google server farms are available to anyone in the world. And the blockchain layer, the the control layer consists of

• 6:25Craig S. Smith?

  it, are they smart contracts then that that you enter into with the provider? How does the blockchain layer control the resources that are added to the network by people with excess compute?

• 6:40Speaker 00

  Of course. So to understand lay of the land, you essentially have about eight, seven point two million data centers in the world, right?

• 6:51Speaker 00

  And if you break them down, there are about eleven thousand professional data centers, meaning over one megawatt capacity. The rest of them are smaller data centers, could be closet, you know, in a office or whatnot, that is considered data center. So these eleven million data center, eleven thousand data centers are owned by enterprises, you know, out of them, about a thousand of them are hyperscalers.

• 7:16Speaker 00

  And their utilization rate, I mean hyperscalers, if you remove them, they tend to have higher utilization because the businesses, but at ten thousand or so enterprise data centers, their utilization rate is somewhere around fifteen percent. Right? So there's enormous underutilization in these ten thousand professional one megawatt capacity data centers. And in the

• 7:44Speaker 00

  the not, the professional grade, not, I mean, sort of the, the, some of professional grade data centers, we don't exactly know the data that very well. They, you know, some cases they tend to have high utilization, but some cases they don't, but there, the, the, the fact remains that is heavy underutilization. Right? Obviously for AI data center, you have higher utilization, non-AI, you have lower. And if you're AI particularly, you have, you know, if you're a professional data center in AI, you tend to have a lot of use when you're training. And not necessarily when you're not training.

• 8:21Speaker 00

  And a lot of times you, you know, if you're training, you need the latest and the greatest chip because the cost advantage is significantly higher if you have better chips. And now the question is what happens to the chips that you made investments to? H100s are now being replaced by H200s. Now, H100s are perfectly great chips and Nvidia made about 2 million of them. They're really good chips. But what happens to those chips, right? So, once you're done with training. So, there's a lot of underutilization and inefficient way of resource distribution in the landscape, right? Now, what Akash does is it takes this compute essentially, or providers that have this compute,

• 9:03Speaker 00

  can come and offer Akash and tenants, that means people that use compute, approach Akash with an ask. Essentially, they create an order saying that, hey, look, I am willing to pay $2 an hour for H200s. And this is my configuration, this is what I need. They place their order in an order book and providers that can fulfill the order bid on the order. So, essentially creating a reverse auction marketplace, right? So, the tenant here sets the price. If provider can fulfill, great, you know, if they cannot, tenant obviously doesn't win the bid.

• 9:46Speaker 00

  So, and once the provider bids and the tenant accepts the provider's bid, a lease gets created. This is what we call a kind of a contract, right? Between the provider and the tenant. And there's a requirement to make an escrow payment from a tenant. So, tenant pre-pays, that funds are held in an escrow and those escrow payments are distributed to the provider as provider fulfills the service. So, that's how it works. Now, the layer where the tenant creates the order is decentralized. It's open. So, all the information besides the private information, there's no privacy.

• 10:29Speaker 00

  Private information is collected, but private information when it curtails to the application is all private. It's not exposed, whereas...

• 10:40Speaker 00

  Whereas the resource information, that means the resources requested, the price, that is public. So you have this rich public data, it's phenomenal, you get how much people are paying for what, right? And that rich data is used to price the resources by providers, whatnot. So, once the provider and tenant gets into an agreement, the blockchain goes away and the relation becomes peer-to-peer. So I am talking to provider directly. There is no intervention, there's nobody in the middle, right? Blockchain comes into place to enforce a contract and enforce payments, right? In a decentralized manner.

• 11:19Speaker 00

  So enforce payments in the sense if the escrow account is out of money, the workload gets undeployed, essentially. That enforcement happens on the control layer. If the provider, you know, doesn't fulfill the order, the lease gets canceled, the money goes back to tenant. That layer is enforced. Every time somebody needs to pay for the resources, that layer is enforced on blockchain. Blockchain is just a coordination mechanism that enforces control, but not an execution mechanism, because blockchain is actually not a good.

• 12:00Craig S. Smith?

  That's right, but the enforcement isn't, doesn't happen on the chain either, does it?

• 12:09Speaker 00

  Well, enforcement is really pulling the resources back, I mean, pulling money back, right? If the provider falls, provider doesn't get paid. If tenant falls, the tenant, you know, the money goes to provider. So that's sort of like mediation happens on the blockchain. Well, ultimately, it's the providers, you know, that will comply as long as they agree with the protocol.

• 12:34Craig S. Smith?

  Yeah, but when you said once, you know, a match is made, then it's peer-to-peer, that's off outside of the blockchain, right?

• 12:46Speaker 00

  So what happens between the provider and the tenant? It's up to the provider and the tenant, right? Now, provider can choose to give the tenant more resources if they want to, but the question is, why would they?

• 12:58Craig S. Smith?

  No, that's right.

• 13:01Craig S. Smith?

  Just on the enforcement, I don't quite understand if I'm...

• 13:06Craig S. Smith?

  If you have compute resources, I have a workload. I find you through the Akash network or through the blockchain.

• 13:22Craig S. Smith?

  Or then, and then you and I, I'm sending in my workload off.

• 13:31Craig S. Smith?

  Chain and privately peer to peer. And something. How does the the chain know whether or not something's gone wrong? I I go back on the chain and. And report it or or. Or what, how does that happen?

• 13:51Speaker 00

  Yeah, the first obvious thing is you didn't get the resources that you need. So there's a verification mechanism that chain enforces on the provider. So there's checking, there's all kinds of things that, hey, did you get the resources you said you're going to need? The provider did not, well, the contract yanks, you get your money back. You know, so it's obviously, once you get the resources, you don't usually have a problem. And then you have problems later, yada, yada, yada. If your workload gets dropped by the provider, it gets yanked, you get the money back, right? So there are certain aspects of like, and quote unquote enforcement, because I use this word loosely, because no, no one can control you. That's all point here.

• 14:34Speaker 00

  But if you do not comply to the contractual obligations, you contract gets yanked. Now, is there indemnity in the sense like, well, what happens if some nefarious activity happens? Yes, there's absolutely indemnity in in certain category providers that are audited, right? So not all providers are, it can be anonymous by default as a provider. But, you know, people are reluctant to deploy on someone without knowing who they're deploying to. I mean, it could be anyone. So there is a degree of identity that's exposed from a provider standpoint to the tenant. In case you need indemnity. Right?

• 15:18Speaker 00

  There's also something called TEE or Trusted Execution Environment. Where if you need to run your entire workload in a fully private mechanism, fully.

• 15:29Speaker 00

  Encrypted mechanism all the way from the chip level, even chip to chip, you know, communications to be encrypted, you can use TEE, which is Trusted Execution Environments. Now with TEE, of course, the challenge is there's a little overhead in terms of how, you know, in terms of performance because of encryption, but it gives you security guarantees. If your application requires that level of security guarantees, you can take advantage of the TEE so that nefarious activity wouldn't occur.

• 15:58Craig S. Smith?

  You were talking earlier about the data centers around the world that are independent, right, that are not a part of the hyperscalers. How does that, can you give, is, do the hyperscalers account for 80% of compute and these independent data centers account for 20% compute in the world, or is it the other way around? Or, yeah, can you give us a sense of scale of what, not what's on the Akash network today, but what your sort of addressable market is?

• 16:42Craig S. Smith?

  I mean, how that compares to the Googles and AWSs and Azure's of the world?

• 16:49Speaker 00

  Yeah, so we measure, so compute is a, there's no direct way to measure compute because it's very, there are too many dimensions to be measured. But if you take down like the amount of energy being used, you can get a fairly good idea. So, you know, there are about 1,030 megawatt data centers. Okay, I mean, half of which are located in the US, but, so, so 30 times 1,000 is 30,000. That's a 30,000 megawatts, right? And there are about

• 17:31Speaker 00

  Ten thousand one megawatt plus data centers, right? So one megawatt, between one mega and thirty thousand, thirty thousand megawatts. So about ten thousand there. So if you, from a, from a high, like, one megawatt about, um, you know, situation, I think about sixty percent, if I remember correctly, sixty percent are hyperscalers, forty percent are non-hyperscalers. And I think that number is growing, hyperscalers are growing. Hyperscalers are limited by two main things. One is energy and second is water, fresh water. So you need to have good energy and clean water in order to build a hyperscale data center,

• 18:16Speaker 00

  and that takes a very long time to acquire. In the United States, we have our entire grid capacities around one point two terawatts, right? So twelve hundred gigawatt capacity. Um, and, uh, we are not good at, you know, building more energy because of regulation and it's quite a lot. If you look at the interconnect requests, that means a connect request that want to connect to the grid, the new supply, there are about one, one point nine gigawatt capacity that'll come onboard in the next fourteen years. But if you look at the timing, they're very, very, very slow to connect to the grid.

• 19:01Speaker 00

  There are all kinds of problems, why that is, I mean, all the way from aging grid infrastructure to regulation to, if you want the most efficient, uh, you know, power, which is a nuclear reactor that can produce up to a gigawatt, I mean, eight hundred megawatts, one point two gigawatt capacity, like the old school reactors. The last one we built took about fourteen years in the U.S. And about thirty two billion dollars, right? So it's very, very expensive to build. And also with the new regulations, the the gen three reactors, uh, the the cost per kilowatt hour is about fifteen cents, I believe, ten to fifteen cents a kilowatt hour, which is prohibitively more expensive than natural gas, right?

• 19:43Speaker 00

  Uh, so there there are a lot of, it's very, very complex and very challenging environment to bring more more data center. So there there's a big question like, how can hyperscale survive? You know, sustain building more data centers? In fact, look back, Nvidia tried to build a hyperscale data center. Um, and Nvidia owns all the chips. Like, the question you may ask is like, why wouldn't Nvidia go to the data center? They couldn't get more than twenty megawatt capacity. I know this for sure because I talk, I know the guy who tried to build it in Nvidia. So it's very, very hard to get capacity. And the last one, nasty nuclear reactor we had was the three mile long, three mile, I, my, in Pennsylvania, that was worked up by Microsoft, right?

• 20:26Speaker 00

  Um, so supposedly, you know, the the Stargate project is supposed to bring more nuclear, but let's see where that happens. So, so your better bet, I think, is can we actually build non-hyperscale, like, sub ten megawatt, sub ten megawatt, you can get away with

• 20:43Speaker 00

  over highly dense, uh, solar powers, wind, you can, you can do renewables for ten megawatts. And that tends to be a bigger trend now. So, so I I I'm extremely skeptical if we are able to move as as fast as we want in the hyperscaler, uh, market. But I think we have a bigger chance to move in the, in the second tier market, which is the one megawatt to ten megawatt. Uh, and I think we have an enormous, uh, opportunity now to move in the hundred kilowatt to one megawatt capacity data centers too. These are small modular data centers. These can be fit in a container, right? Microsoft is experimenting with them.

• 21:28Speaker 00

  All kinds of innovative things you can do in terms of cooling, in terms of like placement and whatnot. And the advantage you get with a smaller hundred, hundred, uh, kilowatt or, you know, to to one megawatt data center says, tapping into distributed grid. The solar energy everywhere, right? So we actually wrote a paper, I mean, Akash, I appreciate that with you. I mean, we did this analysis on like, okay.

• 21:53Speaker 00

  If AI is going to be so important in our lives, if I want everything in my house to be an agent driven, so this conversation right here should be, you know, recorded by an agent, should be processed, insights, I have a year old at home, I want to know everything she's doing at home while I'm working, hopefully have an agent that warned me if she's getting into things she shouldn't be getting into, I want the whole house to be automated, every conversation to be recorded, but I would hate the conversation to be stored in a cloud, because I do not trust anything that leaves my home network as known shit. Okay, now can I have a sovereign AI in the home? I think most people would want an AI in the home as long as it guarantees privacy,

• 22:39Speaker 00

  right? And I'm building that, but okay, what do I need to get the latest and the greatest AI in the home? Well, DeepSeek, you know, 365B can run very well on H200 clusters, it needs about eight H200 clusters, and that costs about half a million dollars, that cluster, right? Now, we did some study, we said, we, you know, feasibility study, like, okay, is there any way you can have sovereign AI in a semi-professional data center that takes about 30 kilowatts of energy in the home, that is cost efficient? And the answer was yes, absolutely. If we can acquire

• 23:19Speaker 00

  about five of these H200, eight by eight chip clusters, this is called HGX clusters essentially, about 40 chips, over earning about, you know, first state about 2.3, you know, dollars per hour, 80% utilization with 20% overhead for cooling, your CapEx slash OpEx plus OpEx can be recovered within five years by placing them on a cash. What you're doing is you're literally allocating one HGX cluster, HGX has eight chips, to your home use. So that's completely dedicated, hundred percent utilized to your home use.

• 24:02Speaker 00

  The rest four of the HGX clusters you're offering on the market for eighty percent utilization, since you already have a cooling and energy infrastructure. Now, we can further reduce the cost by having solar panels. Now, the big challenge of solar is obviously storage, right? So solar only comes for eight hours a day, or max six hours, if you're lucky, actually. But storing like, say you have a hundred kilowatt capacity, or thirty kilowatt capacity, storing that much capacity for over ten hours requires a three hundred kilowatt battery, which is prohibitively expensive. It'll cost you about two hundred thousand dollars. So, but if we can sell it back to the grid, like, okay, overproduce the energy,

• 24:45Speaker 00

  use that energy to sell it back to the grid. In Austin, you get about four cents a kilowatt hour, three to four cents, depending on your region. That way, you can pay for the compute, you know, by tapping back into the grid, right? So grid will sell you back at ten cents. I know it's a scam, but grid will sell you cents as you sell it at four cents. Still, but still prohibitively, you know, not prohibitively, very, very feasible. So we did all this study. That's why tapping in to a home network, a semi-professional home network, and this will take one forty-two U rack, which is, can't put in closet. When AI becomes very important, if AI has to really enter the home, which I think it will,

• 25:30Speaker 00

  I think that's when we're going to see an explosion of these data centers. It could be an office, it could be anyone that want to get a rig. You know, it could be someone in university, for example, they want to get a rig. As long as you have solar, you can completely, actually, I mean, you know, very efficiently run. So there are about seven point two million of these data centers. We had about eight point six million, peaked in twenty seventeen. Obviously, cloud came and like killed a lot of these data centers, but I think there's going to be resurgence of the data centers for privacy, for ownership, for cost reasons. And that's one of our goals, is to have a decentralized AI. And we're going to achieve that by decentralizing the energy production and energy. Consumption as well.

• 26:13Craig S. Smith?

  Right. But for the time being, they're not in the home, primarily they're small data centers scattered scattered around the world. As they sign up, if I have a workload, do you have an orchestration layer that that routes my workload to the right data center? Or or do I have to do that on my end?

• 26:42Speaker 00

  Yeah, so it wouldn't do the data center selection for you by design. Because the application, the blockchain is not application aware for privacy reasons. Remember, a blockchain is a public chain. And in order for it to route, it has to be very application specific, because every application has a different way of scaling. And but so it has a bidding engine, doesn't have a matching engine by default. And, you know, application specific information is one reason. Also, a lot of times you get back a bit with better resources that you may need, better price performance that you may need, because a lot of times, people don't actually know what they need.

• 27:28Speaker 00

  You know, that's why you go to the store and be like, oh, I thought I need this, but I need something else. So a lot of times, you know, people can, sometimes you can compromise too. I mean, you don't exactly get what you want, but there is something that's equally good. You know, you should be able to purchase that, right? Like, so that's why it's very, very hard to develop a matching engine for humans. And so Akash doesn't have automatic routing. Now, you can, you can lease out multiple data centers on Akash and have your own routing mechanism, bring your own router, we call it. But that, that again is very application specific, because you understand the application way better than.

• 28:07Speaker 00

  You know, the infrastructure provider, Akash gives you the sovereignty and control for you to be able to have flexibility to write your application everywhere. So, yeah.

• 28:16Craig S. Smith?

  So you you put a bid on the on the chain, is that right? The bid goes on the chain, it's seen by all the providers attached to the network. They bid on it, or I'm sorry, you put a RFP or whatever on the chain, they bid on it. And then you select whichever one you want, or you can split your your workload between multiple providers, I would guess. The what happens if?

• 28:50Craig S. Smith?

  Uh, I mean, well, two questions first, in the beginning, you're going to get a pretty reasonable number of bids. But as this network grows, the day could come when when you you get more bids and you can reasonably go through.

• 29:13Craig S. Smith?

  Is there going to be an application outside of the chain that can sort through these? Or is is, I mean, is that something that you guys are going to offer so that people can find the optimal partner or or a compute provider?

• 29:31Speaker 00

  Yes, big selectors, we call them. It is on the roadmap. So you can select a big selection strategy. You know, that's optimal for you, and these are please create strategies optionally, right? So it can be like, hey, cost first, optimal for cost, latency first, optimal for latency. Or combination of cost and latency, a cost first and latency later. You can have all kinds of rules, a rules engine based sort of like, in fact, we are looking a little beyond rules engine, we're looking at agent based mechanism where agent can adapt to different situations, considering agent knows your application, you can read your code, understand your code, how it works, and make the appropriate choice for you instead of a rules based system where it could that could be fragile.

• 30:17Speaker 00

  Yeah, so yeah, we definitely have, and we, in fact, we have applications built on top of Akash, different providers like Prime Intellect. Nvidia actually use Akash, they have their own, you know, product called Brev. There's a product of Venice, they're all different products that actually have their own selection engines because they auto scale. The idea of Akash was it gives you primitives, it gives you, it's like Lego box, right? It gives you all these cool things. And as a programmer, you can decide what to do with those Lego box, right? So more often than not, people don't.

• 30:50Speaker 00

  There are a lot of people use Akash directly, but most of our usage comes through the distributors, meaning Nvidia's, Prime Intel X, and Venice of the world, where they offer in Bitmine, a whole lot of things out there. They offer their own flavor of Akash on what they think, you know, is the best winning strategy for the bids, right? Absolutely, I mean, there should be a bid selection engine. Will Akash provide it? Probably will give you options, options that'll make your job easier.

• 31:20Speaker 00

  Assuming that you're a power user and you understand how these bids work, but we want to be as explicit as possible in terms of granularity, that you have full control over who you get.

• 31:34Speaker 00

  And I think that is a very important capability.

• 31:37Craig S. Smith?

  And you mentioned price and latency. When you, when, when the bids come in, do they come with some metric that measures latency for, for your, where you are, where your workload is?

• 31:52Speaker 00

  They give you the region where, where, where the, where the, you know, workload will be placed, everything you would need to make the selection, right? And so you can write a latency checker, essentially, that'll check the latency depending on where your users are. So typically, what you would need is you would analyze your user traffic and you would select a mechanism where it'll sort the data center that's in 95th percentile of your user activity can be achieved within 50 milliseconds of network latency, right? So that's really what you wanna, wanna do. But those controls need to be written and that need to be specified based on your application. Some applications, latency is a very complex thing because.

• 32:38Speaker 00

  Like if you have hyperscale applications like Facebook, now you most definitely have different shards on different regions, right? So California users come to California and New York users come to New York. And more importantly, the data should be available in these local regions, right? And what happens when you have, when you have a New York user trying to access California or when they travel and New York user goes to California? The this little, it's latency based selection is not as easy, one would think. It's it's easy when your application is simple, but as it gets more complex, it gets very, very, very challenging, right? Like, so, uh, yeah. So you can't just have a latency engine because, you know, you may not actually give the right information based on the application.

• 33:22Craig S. Smith?

  Yeah. And and then the other thing you mentioned, uh, that you you may not want, uh, you know, to put certain information with a hyperscaler because you're concerned about privacy. Uh, how how do you know how secure uh these compute resources are on the Akash chain? I mean, how the, uh, you know, you may be accessing, uh, you know, a small data center in in Hong Kong. You know, you're not there. You don't have people

• 34:03Craig S. Smith?

  looking at their security protocols. How do you? How do you ensure that, uh, you you're secure?

• 34:12Speaker 00

  So the two main mechanisms Akash, uh, provides, one is a decentralized auditory mechanism, where if the provider says, hey, they're compliant, they're HIPAA compliant, they are, you know, whatever standard, uh, compliance they have, they have their their tier four data center, they are, you know, the tier four in terms of redundancy and whatnot. These auditors go and verify that off chain. They literally run, run applications, they run audit checks, they physical proof, they're paper proof, they actually go verify and they post their results on chain saying that hey, I have verified that this provider says who they are, their identity, more importantly, because if something goes wrong, you want to go after them, kind of thing, right?

• 34:55Speaker 00

  All the information is posted on chain. I mean, in fact, not the, the verified information by multiple auditors, right? So as a tenant, you can choose the auditors you want to select and these auditors are public figures, like Overclock Labs, Mr. Akash is one of the auditors, for example. Right? Second mechanism is we, we provide something called T E E or Trusted Execution Environment. What T E E does is it gives you a non-custodial way to encrypt runtime, not just transport. So your transport is obviously encrypted, right? That means communication between you and the provider is encrypted. But what's not encrypted is what's in the memory because it's hard to encrypt. Well, you know, so anyone with physical access to the machine can theoretically like with, if they're talented enough, can theoretically look inside what's, what's in the machine,

• 35:40Speaker 00

  what's, what's running. To avoid that, you want to encrypt the memory itself. Um, and that's achieved through Trusted Execution Environment. Now, the trade-off, obviously, there is, you need more resources because it's encryption, right? It needs more compute, it needs more GPUs. Um, so if you're doing AI, Nvidia has this amazing T E E mechanism in that newer chips, H200s and H400s, where they encrypt memory within the chip itself and they also encrypt the, the transport between, between the chips using NVIDIA Link. NVIDIA Link is 3.2, uh, uh, TVPS is very, very fast, about a second. Um, so it may slow down a little bit. We, we see right now, we're seeing about 10% overhead, which is not bad trade-off if

• 36:25Speaker 00

  you want privacy, right? Especially if you're unknown data center. But if you're deploying on something called like Equinix, which is in US, which is a professional data center, you don't have that problem. So you're more, you're more relaxed when it comes to like, hey, okay, we can trust this data center or whatnot. But ultimately, yeah, you have to. You have to trust somehow, right? So either through encryption trusting or through through uh through auditability trusting. But uh this way you actually have a lot more uh you know, tools at your disposal that you normally don't get when you go to a data center directly and with them, right? So you have all these like infrastructure that Akash gives you that you can't, you don't normally get.

• 37:08Speaker 00

  So going to hyperscale is just trust the brand and say, okay, Amazon will not do something. You're trusting that Amazon would not do something. I mean, this is a good amount of truth to it, but we know how people, how this trust could be you know corrupted. We just don't know what's happening. If you're a big company trying to use Amazon, they'll let you open the boxes. They let you, they literally show you like if you're having one million, if you're Department of Defense, they literally let you audit everything, right? But if you're a, you know, Josh Moore, no one cares. They they just, you know, no, you you wouldn't even know if someone has access to your data. It's very famous with ChatGPT because one of the engineers under interview, they said, yeah, we look at users' prompts.

• 37:51Speaker 00

  So I felt violated because like, okay, my prompts are good Lord, right? You know, all kinds of things. And I'll be judged by the person that is looking at my prompts and that feels, you know, violation of privacy, right? And if you have anything on the cloud, there's no guarantee that these people are going to look at your thing. But if they adhere to certain standards and they don't comply and you have some degree of like, you know, indemnification, right? So that's why I feel like if you want ultimate privacy, you have to run the home. No, there's really no way. Second best thing is T E E. If you can't run the home, well, the overhead. And third best thing is probably like ultimate trust in the in the provider.

• 38:34Speaker 00

  But Akash gives you that T E E, which is much better than a cloud provider.

• 38:38Craig S. Smith?

  Yeah. And then the other question is, um, if I'm running a workload and and um and my data center goes down, uh, or or maybe the, you know, there's an an outage.

• 38:56Craig S. Smith?

  And and the backup doesn't happen, you know, the generators don't kick in or whatever. What happens to my workload?

• 39:05Speaker 00

  It dies. And so you have to know how to be redundant pre-application. And that's one of the big areas. Uh, you had the same problem with the cloud today too, and cloud work. Amazon has about 200 outages last year, right? And we all saw what happened with CrowdStrike. Somebody could push bad code and the entire US airline infrastructure came to a standstill. Because people couldn't fly. Uh, so you know, centralized systems generally have a lot of faults that they don't expose. But so one of the key design patterns to use Akash effectively is to be redundant from the get go. Akash makes it cheaper and makes it more optimal to be redundant, but there's really no silver bullet. Uh, and this happens all the time, right?

• 39:49Speaker 00

  So we see uh, I S like, that's why you want to see when someone says there are tier three or tier four data center. That means they have a dual uh, ISP ideally with with a Starlink backup. And we uh, happy to share you the A P we wrote recently, like, okay, how do you design a home data center? With dual ISPs with a Starlink backup. Uh, where dual generators, you need a gas, I mean diesel generator with automatic switch off as well as a gas generator. Uh, and ideally solar panels. So we have all kind in USBs, I mean, the uninterrupted power supply, UPSs and whatnot. So you know, so the so all these aspects are verified. So you reduces the chance of false, but false happen.

• 40:31Speaker 00

  But you cannot prevent false, but what you can do is come back up faster, right? So no matter like Mike Tyson says, no matter what, how planned you are, you know, your plans are only as good as you get punched in the face, right? So so you you will get punched in the face when you're running infrastructure. And that's the reality of infrastructure. And you talk to anybody that tells you infrastructure runs infrastructure will tell you there will be faults. No matter how hard you think you built your data center is. Uh, but it ultimately comes down to redundancy. You can run in a quorum, for example, like if you have a database, well, don't run a single database, run. You know, a master slave, sorry, you're not supposed to use the words like a leader follower style database, uh, where you, you know, if your, if your follower goes down, eh, the leader

• 41:14Speaker 00

  will self-elect a different leader or, you know, spin up a new follower database that'll always be alive, right? So, uh, you need to be a little bit more professional to, to use Akash. Now, natively, now there are applications on top of Akash that makes all these things easy, uh, you know, you don't have to think about redundancy, like you just click a button, taken care for you, right? Uh, but that's very application specific. Akash doesn't, uh, provide by design those redundancy mechanisms because again, depends on the application, right? How you, like a lot of times when you're scaling databases, you need to think about consistency, what level of consistency you want. If you want immediate consistency, um,

• 41:55Speaker 00

  you know, so when you have multiple nodes in a database, right? Um, uh, uh, and you want to make sure all the nodes are presented this, giving the same data. Well, if you have multiple nodes, if you want to be redundant, you have to have them in different locations around the world. Well, they're not going to present the same information because there's latency, there's an east to west, there are milliseconds latency, right? So if you want an immediate consistent database, you gotta make sure the, all the, uh, data sets are replicated, it's about 300 milliseconds. So when you write, you can't read immediately, you gotta wait 300 milliseconds. That's called immediate consistency. Well, if you don't care about immediate consistency, I mean, eventual consistency is fine with you.

• 42:37Speaker 00

  That means you write a data, uh, uh, entry in California, that entry takes about, you know, you're assuming the reader is reading back from California and not reading from New York, um, and you're fine with that. So if you're building social media, you're fine with that. But if you're building something like a banking application where someone withdraws cash from ATM in, in, uh, in California, should not be able to withdraw within 300 milliseconds in New York. You need immediate consistency. That's why. Scaling and redundant redundancy planning is very application specific.

• 43:15Speaker 00

  And so that's why Akash gives you, doesn't make the choice for you. And you have to design your redundancy on how you want to see it.

• 43:23Craig S. Smith?

  So this is, this is new, right? I mean, how long has the network been operating?

• 43:31Speaker 00

  About four years now. Four years.

• 43:34Craig S. Smith?

  Four years. Yeah. And as you said, the, the, the demand for compute is presumably going to rise sharply as, as AI spreads through the economy. Do you have a, a target for how many, how much compute you'll have on the network by X date or, or any sort of roadmap that way?

• 44:05Speaker 00

  Yeah, so we're growing at 10X now every year, which has been phenomenal. We hope to continue to grow 10X, you know? You know? So right now we have about, you know, we want to get to about 10,000 GPUs by the end of the year, which we think can, now about 100,000 GPUs by the end of next year. About a million GPUs within the next three years. And we're doing fairly well. We're doing really well. We're growing really, really well, growing at 20% month over month now. And, and more importantly, it's not how much compute you want to get. Whereas it's about how much compute we, we have on the network gets used. So utilization rate is extremely important.

• 44:50Speaker 00

  So we don't, we scale responsibly because every time there's a new compute and a new provider that comes to Akash. They should be able to sell out their inventory, right? If they're not selling inventory, there's no point coming to Akash. So, very, very important for us to make sure there is a good equilibrium between the provider's capability to sell compute and the tenant's capability to scale their workloads, right? Again, if provider sells everything, I mean, there's no inventory, tenants will be upset. So, uh, so that's why you gotta be very, very careful in how you, how you scale the systems. Uh, our utilization rate right now is seventy percent, very healthy, very, very good.

• 45:34Speaker 00

  Now, that utilization rate can go up as we scale because there'll be more resources generally in, in, in the, in the, in the pool. Uh, but so our constraint is utilization rate and we don't scale without that. So we, we did very, very well now.

• 45:50Speaker 00

  Um, and you know, I think

• 45:54Speaker 00

  our next big goal is, uh, transitioning from a resource market to a services market. A resource economy to a service economy. Right now, think of Akash as just a marketplace for resources, like coming to a commodities exchange where you can trade gold and whatnot. But it is not a marketplace for, for services. So a lot of times when you're building AI systems or any system, as a matter of fact, you have, you use several resources like databases, vector data, vector, uh, databases to inference services, to agent, you know, hosting platforms, to whole lot of resource services that one would orchestrate together to build. And a lot of times when you look at the cloud providers, the resources are open source systems.

• 46:38Speaker 00

  Look at Redis, for example, Elastic Cash on Amazon is Redis. Or, or, you know, uh, the, the SQL, I mean, the database services that Amazon offers is MySQL or Postgres. These are open source protocols and Amazon really takes them and, and white labels them and sells them. We feel like if we empower open source developers to be able to provide their services on Akash, we can effectively have a sustainable open source ecosystem. Right now, a big challenge for open source software is sustainability. Docker, for example, Docker is one of the most widely used, you know, open source container infrastructure system. But they couldn't figure out a business model after raising it a billion dollar valuation. They sold to Pinnas for to Nutanix.

• 47:24Speaker 00

  They're no longer an open source company. Kubernetes, one of them, eighty percent of the globe uses Kubernetes. They can't keep their contributors together anymore. Right? Look at every, I mean, besides Linux, which has a very weird founder and is very special founder. Besides Linux, we don't have any example in the world that remain purely open source. Right? Successfully.

• 47:54Speaker 00

  And so we have to change that. We have to create an economy for open source contributors to be able to sustain themselves. To build open source software. And I think that is what Akash is going to transition to the service economy. And that's coming next year. I think with that and also like our, we also measure revenue per GPU. Right? So revenue per GPU is right now at twenty dollars per GPU per day. Which is fairly good. We were ten dollars a year ago and you know, we did a good job in increasing the revenue per GPU. But with additional services, we can further improve our revenue per GPU to about fifty to a hundred dollars. Right? That comes with premium services at the top of it. And that revenue goes to open source developers. Right?

• 48:36Speaker 00

  So, so we not only look for like just capacity per se, pure capacity. We also look at how do we increase our value per resource that we provide? As well as how do we scale while maintaining good utilization? That's very, very important for the health of the network. The more healthier the network is, the higher chances of success we have in the future. So, yeah.

• 48:58Craig S. Smith?

  Well, so how does someone, I mean, once you have the services up presumably,

• 49:07Craig S. Smith?

  you'll have people that can onboard

• 49:15Craig S. Smith?

  tenant, as you call it, or a user and walk them through the system and get them connected and do all of that stuff. But for now, how does somebody use the network?

• 49:29Speaker 00

  So you could go to Akasha Network and there's a button that says deploy. There's an application called Consol. Consol or Akash Network is a great way to get started. You know, there's $10 like free trial if you want to go try it. It's actually very good to use, highly encourage folks to try it. No signups is very, very straightforward. People love it. If you want a more easier system in terms of

• 50:00Speaker 00

  in terms of like getting like SSH style like access, you can go to Prime and Select. They can use, they use Akash underneath. You deploy a resource, you'll see the power.

• 50:12Speaker 00

  So there are several ways, but I think the best way is to go directly because you get the most cost advantage. Right now, last, I don't know if you have any H100, there's 200s left on the network because they, they go like hotcakes. But each 200s are $1.99 an hour, which is the lowest compared to Amazon. I believe it's Amazon's $4 or $6 per hour, but like three times cheaper. And the reason why we're cheaper is because I'll send you the link where we did a cost analysis, right? You actually amortize your, your investment while with, with, you know, with, with the decent utilization rate fairly easily. So there's no reason why you should be paying Amazon margins. I mean, Amazon makes record level margins from you, right?

• 50:56Speaker 00

  So you remove all the margins, the resources are actually fairly cheap. That's really what Akash provides. So that's why resources are very, very cheap, Akash.

• 51:06Speaker 00

  So, but they go out like each 200s are gone. I would have been like, our utilization rate was like 98% last we saw H200s. That's a problem with like hot chips. You don't really have availability and we're trying to improve that as much as possible. But A100s are available for, I mean, sorry, H100s are available for. Dollar twenty cents, which is also very, very competitive compared to Amazon. So highly encourage folks to check it out. If you want to see the pricing, you can go to akashadnetworks slash GPUs and you'll see the GPU pricing and you'll find them extremely competitive. You also give, you also get to know how much availability there is on the network so you can plan.

• 51:44Speaker 00

  And I highly warn you, it's very addictive, especially when you have, especially when you, when you automate. So a lot of our users just automate the hell out of it because the moment they see a GPU, I guess we sort of swept away, especially with H two hundreds. Deep secret runs, deep secret runs really well on H two hundreds. So it's very addictive because people are like constantly bidding, constantly bidding, getting into this like really cool game. I think there's also somebody that's building features where they actually purchase the compute and they resell the computer at a higher rate. So it's a very, it's a very fascinating market, actually.

• 52:18Craig S. Smith?

  Lastly, you're not the only decentralized cloud computing platform out there.

• 52:26Craig S. Smith?

  How do you regard yourself in the, in, among the competition?

• 52:33Speaker 00

  So we're definitely the leaders, we're the first one. And we're the leader, so there are a lot of copycats, I mean, success gets copycats and that's, that's a reality, right? We are not the only one, but we are the first and the biggest open source cloud. So new copycats are actually closed source. They take our code and fork our code, we've seen that several times, right?

• 52:58Speaker 00

  Now, not to disregard the competition, but I think it's, it's a good thing that people are looking at our model and seeing the success behind our model and trying to have their own flavor and they obviously want to stay closed source because, well, you know, it's advantageous to be closed source because you can ship code faster. So Akash is very decentralized. I think from a pricing standpoint, we're also, we're much better. From a pricing, we have much better SDKs, we have much better user experience, we have much better, you know, in, in several aspects and also you have a lot of.

• 53:33Speaker 00

  We have about thirty thousand developers in our in our in our discords. Anytime you have a problem, you just go to the discord and this helps solve your problems really quickly. So enormous community and we have enormous participation, right? We have about five hundred contributors that come and build Akash. So if you see a feature that you want to build, you can write a proposal yourself. So if you're an open source developer, if you like building open source, Akash is a platform for you. Because it not only lets you use an open source system, but also lets you build and pays you. Like if you have an idea that you think you can run on Akash, you have, I believe, twenty five million dollars in the community pool. Then you can apply for a grant and get and use Akash. A lot of university students are using Akash that way too.

• 54:18Speaker 00

  So it's a it's not a company you're you're dealing with. It's a community you're going to be part of. So that's a big difference between using Akash and a company with its products. It's not a corporation, it's a community, right? So if you are that type of person that believes in a in a that communities can actually like solve societal problems, Akash is for you. But if you are one of those people that want to pay a company, that can leverage decentralized stack as well, you know, there are there are options and, you know, of course, like companies, you know, out there that are offering something similar to Akash out there that you can take advantage of.

• 55:01Craig S. Smith?

  Is there anything that I didn't cover that you'd you'd want listeners to know?

• 55:06Speaker 00

  No, I think just, I think I want to emphasize why this model is is going to be extremely important. And I think why people doing AI, especially training, should be considering some of these models. Like, our biggest challenge for AI, there are two big challenges for AI training, right? The one was data. We didn't have data, there's the data limit as to how much you can get data to train. And second was energy. We solved the data problem, or you saw what we saw with DeepSeek, it can use synthetic data. So it's very, very amazing using synthetic data and using a mixture of experts mechanism. You can actually solve the data problem.

• 55:50Speaker 00

  But what we cannot solve is the energy problem. I think that's why it's very, very important if you're doing training to focus on distributing your, your, your training runs versus trying to go with the traditional mechanism of centralizing your training runs. And I, you know, because we're going to hit a cap in two years and we have no solutions, right? People are, that's why we have $500 billion supposed investments and a lot of the investment is going towards power obstruction. But by the time you see, realize the benefits of the investment is going to be late. So highly encourage folks to, to take decentralized distributed training more seriously and look into mechanisms employed by news research. News is a top team. They came up with something called distro recently.

• 56:34Speaker 00

  Google deep mind came up with a paper called dialico. There's several companies that are having their own approaches and some of them reduce the, uh, one approach is like reducing the amount of communication between, between the nodes to, to train better. Some of them have better verification mechanisms employed locally to train better. Uh, and I, I love to see more, more work, more different approaches and more experimentation in the space. And that's really, I think, going to benefit all of us. Uh, uh, you know, and really take the power away from the open access of the world. If you want to really disrupt them, you have to think of a decent situation.

• 57:13Craig S. Smith?

  I'm past the point of looking for jobs, but I'm not past the point of looking for people to hire. And when I need to do that, I turn to Indeed. Imagine you just realized your business needed to hire someone yesterday. How can you find amazing candidates fast? Easy. Just use Indeed. When it comes to hiring, Indeed is all you need. You can stop struggling to get your job post seen on other job sites because Indeed's sponsored jobs help you stand out and hire fast. With sponsored jobs, your post jumps to the top of the page for your relevant candidates.

• 57:56Craig S. Smith?

  So you can reach the people you want faster and it makes a huge difference. According to Indeed Sponsored jobs posted directly on Indeed have 45% more applications than non-sponsored jobs. Plus, with Indeed sponsored jobs, there's no monthly subscriptions, no long-term contracts, and you pay only for results. How fast is Indeed? In the minute I've been talking to you, 23 hires were made on Indeed, according to Indeed data, worldwide. There's no need to wait any longer. Speed up your hiring right now with Indeed. And listeners of this

• 58:38Craig S. Smith?

  show will get a $75 sponsored job credit to get your jobs more visibility at Indeed. To get your jobs more visibility, go to indeed.com/eyeonai. Eyeonai, as always, all run together, E-Y-E-O-N-A-I. That's indeed.com/eyeonai right now. And support our show by saying you heard about Indeed on this podcast. Indeed.com/eyeonai for a $75 sponsored job credit.