The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin

More than 25 million users are using NEAR-powered applications. Co-founder of NEAR protocol and Transformers author Illia Polosukhin joins hosts Sarah Guo and Elad Gil to discuss the intersections of crypto and AI technology, what we should expect from AI agents, decentralized data labeling, why AI’s alignment problem is really a human problem, and more. Show Links: Illia Polosukhin - Co-founder of NEAR | LinkedIn NEAR Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @ilblackdragon Show Notes: (0:00:00) - Blockchain, AI, and Web3 Intersection (0:06:39) - How We Might Combine Blockchain and AI for Cancer Research (0:23:35) - Inference and Decentralized Data Labeling (0:30:13) - AI SaaS Strategic Challenges (0:38:18) - The Future of Hardware Accelerators

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:06Sarah Guo?

  A blockchain operating system might just be the key to a democratized Web3. In fact, more than 25 million users are already getting a taste of this, thanks to Near. This week, Elad and I are joined by Ilya Polosukhin, the co-founder of Near and co-author of the Landmark Transformers paper, to discuss the interaction of blockchain and AI technologies, what we should expect from AI agents, how to handle the content authenticity problem, and why the alignment problem in AI is really a human problem. Ilya, welcome to No Priors. Thanks for doing this.

• 0:35Speaker 01

  Thanks for inviting.

• 0:37Sarah Guo?

  You are one of the authors of the original Transformers paper. We've also had Gnome and Jacob on. How did you get involved with that seminal work in AI?

• 0:46Speaker 01

  I worked on a team on natural language understanding focused on question answering. And the state of the art at this time was LSTMs, recurrent neural networks, which you could not launch in production at all because they're too slow and take a fair bit of time to process as document scale. So Jacob at the time was using attention for query similarity. And he had this idea like using attention for encoder-decoder type. I kind of jumped into it and with the shush we're playing around with, can we actually get it to train and understand the order of words and do translation just based on, you know, attention?

• 1:31Speaker 01

  So yeah, it was pretty cool to explore that and obviously grew into something very interesting and awesome.

• 1:38Sarah Guo?

  You originally co-founded Near in, I think, 2018, meaning for it to be an AI-focused company. What was that initial mission and how did it become a blockchain company?

• 1:49Speaker 01

  Yeah, so we started with the idea that we wanted to teach machines to code. You know, we had transformers coming out. There was a lot of kind of really interesting push in 17, 16, 17 around AI. And so our expectation was we kind of would ride the exponential growth of AI, which has happened in This year, we thought it will happen in seventeen, eighteen. And so with that, we got a really interesting dataset around language to code. But more interestingly, we had a whole community of developers, mostly students, who were doing crowdsourcing for us. So we would give them code, they would write descriptions, we would give them descriptions, they would write code for them, write tests, like all kinds of tasks.

• 2:33Speaker 01

  And we actually faced the challenge of paying them because a lot of them were in China, in Eastern Europe, kind of other countries where there's monetary control problems, people don't have bank accounts. And so we started looking into blockchain just like to solve our own problem. The AI kind of exponential explosion didn't happen at the time. And so we saw an opportunity of we can actually build a blockchain that we would use to solve this first and focus on that while kind of waiting out the AI thing to really happen. And as you go into the blockchain rabbit hole, you realize there's a lot more that meets the eye.

• 3:11Sarah Guo?

  Yeah, yeah, ended up being a pretty big mission.

• 3:14Speaker 01

  Exactly.

• 3:15Sarah Guo?

  So you call NER a blockchain operating system. For any of our listeners who haven't used it, like, what does that mean?

• 3:23Speaker 01

  So the idea is that we wanna kind of go up stack, right? We want kind of an environment where you can discover and use Web three experiences, you know, benefits from them and not need to think about the low level, you know, implementations and quote unquote hardware that runs under it, right? So similarly, how operating systems on your phone, you know, kind of abstracts out all the complexity of, you know, networking and payments and everything, you just use it and you have apps that developers can build. And so that's really what we're trying to achieve and kind of build this framework and platform for everybody to build their applications in Web three and really deliver it to the user, to consumer.

• 4:04Speaker 02

  Where do you see a lot of that overlap coming in terms of Web three and AI? You've thought very deeply about both. I remember when I first met you, you were just switching from sort of NER's original mission and at the blockchain based mission. And, you know, you were known as a team that could literally build anything, right? Like you had yourself and Alex and Pyguy and all these like amazing people. And you went down the direction of building blockchain, in part, I think, originally around this data labeling kind of mission and the ability to do payments and things like that. And now I know you've been thinking a lot again about how these two worlds interact or intersect. Where do you think are going to be the biggest places of overlap between AI and blockchain or Web3?

• 4:41Speaker 01

  There's few levels of interesting intersections. I think the most obvious one that everybody talks about is various marketplaces for resources, right? Be that compute, model, or data, right? So data crowdsourcing. So those are pretty obvious, right? Web3 is really good at creating marketplaces, creating traceability, and providing like an equitable place for everyone to participate. Now, the more interesting ones is where AI kind of agents, right? Which, you know, we've seen like initial versions of, but obviously they're going to continue evolving. If you equip them with a blockchain account, right? They are now becoming an economic agent that is able to pay other people and pay other AIs to do work, right?

• 5:26Speaker 01

  And they can communicate, right? And I think one of the things that a lot of people who are, oh, like these language models are just the same advancements like as everything before, missing the point that this is the first time that a machine is able to communicate with people in the same way, right? There's no more need in an intermediate human that interprets data and then tells it to other people. Now a machine can communicate directly to people. And so it can task them with work. It can provide them context. And so I really think one of the most interesting cases is organizations that are run completely by AI, right? Where quote unquote CEO role is taken by AI agent, who is tasked by, you know, by community or board of directors or whatever is oversight governance is,

• 6:12Speaker 01

  to, you know, hit specific APIs and follow specific mission. They can even give specific feedback with training data when they don't think it's doing the right job. But what it does is like creates this kind of a new layer of management that potentially removes a lot of middle management right now, which is like transforming information and context for each individual person and giving them specific area of work. And then gather like kind of harnessing their creativity and putting it back together, right? And I think that's a very interesting use case that kind of really melds blockchain and AI together.

• 6:44Sarah Guo?

  Why, like you have a traditional biotech, cancer, research, commercial entity? Like why blockchain and why AI for that?

• 6:53Speaker 01

  I use this example, right? We want to, you know, continue making progress on solving cancer, right? And it's a very complex problem, right? There's a lot of like specific sub cancers that, you know, need research. And so all of this and like coordinating people, doing experiments, propagating information, recruiting, you know, people recruiting the candidates, right? All of this requires like somebody to do this work and kind of organize the process and really set up a lot of pipeline and, you know, funding and all those things. And right now there's so much overhead around everything from, you know, how grant funding is allocated from the nonprofits that collect money for research, how like experiments are set up, the information sharing, like all of those pieces are really kind of broken. And so you can actually have, you know, like coordinated effort

• 7:38Speaker 01

  that is designed just to do that. And it can consume all this information and kind of specifically task, you know, who is the best person at doing the experiment? Which lab is the best at doing this specific sets of experiments? You know, fund them for this, you know, amount of money, you know, oversee their delivery and then kind of iterate. And, you know, if, if it thinks this lab is not doing a good job, fire them without having like extra, you know, personal affiliations that, you know, people do have. I'm actually excited about some folks are already building some examples of this in like a simpler forms, but I think we'll see, you know, first organizations like this, probably even this year, where potentially with a simpler mission, some kind of more straightforward like KPI metrics,

• 8:22Speaker 01

  but where kind of this information propagation and onboarding of people happens already through a kind of language model AI agent.

• 8:33Speaker 02

  A similar version of this that I've heard people talk about, and it may be the first step towards it, is actually providing on the job feedback via an AI versus like a human manager with the idea that it depersonalizes the feedback, right? So if you have a agent or an AI providing feedback, some surveys at least have suggested that the average employee may be more comfortable with that because it feels more objective, it feels depersonalized, it feels like it can be provided in a directive way. And it seems like that's one aspect of sort of this AI as CEO concept that you're describing. Do you think the first place that it'll show up is DAOs? Or do you think it'll show up in a different part of the community?

• 9:08Speaker 01

  Yeah, I think DAOs is, and especially what happened with DAOs, there was a lot of people who are really excited about DAOs kind of as a concept. And so they put a lot of time running them. But it's actually a very like not interesting job, right? It's like you onboard new members, you explain to them all the same thing, you know, you answer to their questions. And so that's the part which like you can already automate, right? You can like have a Discord bot that is like have all the context about the DAOs. You know, interactions and kind of onboard new people and gives them like new, you know, tasks to start with and kind of coordinate them. So I think that will be the first place where this kind of start showing up. And as well, because you have like payments kind of very like there. And you don't have any social constraints that usually you have in like regular organizations.

• 9:53Speaker 01

  Like, you know, I, I, a lot of people will revolt if you like tomorrow say, hey, by the way, your new boss is this AI model.

• 10:00Speaker 02

  Yeah, yeah. How, how do you think about AI in the context, or I should say blockchain and AI in the context of things like alignment?

• 10:07Speaker 01

  Yeah, so I think this is a very interesting topic. So I have this view that we need human alignment instead of AI alignment. So right now, kind of when we talk about, you know, hey, we need to align AIs with like human values. But the reality is that, you know, all the problems that exist, they all exist because of humans doing things. And, and they've existed before. I actually like to use the Byzantine full tolerance problem, right? Which is basis for blockchain. But they, their, its roots are in, you know, history where there was people propagating misinformation. And you were trying to like figure out how to prevent misinformation in the army, right?

• 10:47Speaker 01

  So this is like a really old problem of misinformation and kind of like how to work around that. And so I think what we need to start doing is figuring out how do we build a society that is actually able to deal with uh, kind of effective misinformation at scale, right? So like we've kind of built it like a lot of our society has started building up tolerance to misinformation around, you know, TV and mass media. But we don't have like a system and framework around dealing with it at scale. And that's what AI brings. Things just scale to the same problem. And so this is where reputation, identity and kind of systems around our social, I call it operating system that powers our commu- kind of communities is really important, right?

• 11:32Speaker 01

  How do all these these pieces work together? And how does they actually operate when there is malicious actors who potentially are able to, you know, at in mass create like very personalized misinformation or create, you know, fake political actor that is, you know, convincing every individual exactly in what uh they think, you know, that government should do to get elected. And the this is where Web three comes in as like a set of primitives, right? We have cryptography to authenticate content and create a path. Everything from, you know, you take a picture with camera. Some of them already have a secure enclave that can sign the image that's taken.

• 12:12Speaker 01

  And so as that image gets processed, we can actually propagate that information and have a proof that it came from, you know, specific time and place and then being processed by a specific set of filters. Right? So that can give you like an anchor. Then you still need to know kind of who is publishing what, right? Like we're recording this podcast, you know, people listening to it. It could have been completely generated at this point, right? But if, for example, we all sign the, you know, the final podcast and say, hey, yes, we've recorded it. And this is valid content. Now, when somebody is listening to it, they can check that indeed, hey, this content is signed by us. Now the question of us comes in, right? So this is where kind of identity and reputation is important.

• 12:54Speaker 01

  And so this is where kind of unchain identity becomes your kind of coalescence of all of the content and all the interactions that you do. And then that links to kind of, you know, reputation in different communities uh and uh provides context for people who are watching for this content to be able to understand, you know, who is this person who's talking or where they're coming from? And what are the information values they have? So I think like it's it needs to be a kind of systematic approach and it'll start with pieces, right? I think one of the Important pieces will be kind of a green lock similar to SSL transitioned on the content, right? Like as you go to YouTube, as you go to, you know, New York Times, you actually will see that like,

• 13:39Speaker 01

  hey, this content been signed by this party and this party is in some trust root or trust community, like graph of communities that you are following, right? So that's probably like one important piece. And again, blockchain and cryptography is just like tools to enable that product experience. And then from there, you know, we need similar things on the government level, right? When you file paperwork, when you file, you know, your identities, the fact that your SSN is a, you know, number that you give to everyone, which is like supposed to be secret is like, for example, ridiculous, right? So things like that is like, all of this needs to improve and kind of upgrade to this new level where like a massive amounts of kind of at scale of things that have been happening now are possible.

• 14:23Speaker 02

  What do you think is the most likely form of blockchain based identity? Because, you know, the blockchain really has been the earliest place where you've had programmatic actors interacting around economic and other utility functions, right? It really is money as code. And effectively, smart contracts are ways to programmatically interact with that, right? So you, you had almost like the execution layer without the intelligence and now we're adding the intelligence. You have the cryptography, but you're missing a real sense of identity, which is needed if you have an agent or bot representing you interacting with another agent, which is probably where a lot of things will work in the future online. What do you think is the most likely form of identity on the blockchain and why hasn't it happened yet?

• 14:59Speaker 01

  It has happened to some extent, right? We have, you know, like millions of people actually using blockchain right now and they're using it more for financial use cases and kind of that's their financial identity.

• 15:11Speaker 02

  The wallet is identity kind of thing.

• 15:12Speaker 01

  Yeah, well, it has became an identity, right? And the reality is like your quote unquote private keys are your identity, but that's just too hard of a concept for people to actually work with, right? And so on near, we actually changed that. We, you know, you have a properly named account. So like mine is root near, which can have lots of different private keys accessing it with different permissions, right? I can give a key and in a way permissions to an agent. For example, interact on behalf of this or I can withdraw it, right? I can give it to a specific application, etc. So like a more extensive model is needed, that's one. We need to have more social interactions kind of being spawned from this. And so this is again blockchain system is powering actually social interactions and kind of communication.

• 15:57Speaker 01

  We actually have a project working on chat and other ways of using now this identity in more places. It's mostly because we didn't have a critical mass of these applications that are using this identity. So to for it to really become kind of the core. And if it's not the core, it's not as useful because nobody, you know, like, hey, you don't have it. So like, we're not going to use it as a default thing everywhere. So like, we really need to kind of go over like, again, I think S S L is really good example of something that's like, it it delivers value, it's clearly valuable, but it was such a like uphill battle to get it there, right? And so I think like, until you have this critical mass of like,

• 16:39Speaker 01

  kind of websites switched and browser support, it it didn't become a default, right? So we kind of need like the same here to happen. Like, we'll need to have a critical mass of applications using, you know, identity. And then uh then we kind of see the like in browsers or wallets or whatever like applications to hold it. And then we'll see a transition function happen where like, hey, oh, you don't have it. Like, you should get it because it's actually easier and better to use it. And and it gives you like more financial freedom as well and more upside.

• 17:09Sarah Guo?

  Where do you think the most likely failures like system wide are are to be? Like with, um, you know, growing capabilities in AI? Like where where do these mitigants in terms of uh reputation systems with blockchain? Or like content providence are are likely to, how how's that going to manifest in ways that affect us?

• 17:29Speaker 01

  Yeah, I think there will be, probably next year will be very interesting in US. Because I think this this will be a place where everybody will just take whatever their toys they have in toolbox and do it even just for kicks, right? Even if it's not malicious, although some players will be malicious. And I think that what we'll see, everything from like completely fake narrative candidates to like, I would be very interested to see like a webpage where you land and you know, you log in and it literally generates specifically for this user, based on their interest, a agenda for this candidate. Right? So like hyper focused, you know, marketing for candidates based on like who this voter is, right?

• 18:14Speaker 01

  So things like that, like we'll have all those possible things where the media will kind of be flooded with like, you know, you can spin up new media right now and just generate content about your candidate like that you want. Uh, and then market that. So like you can have like all kinds of things now just exploding without any way of like framing it on the user's side. If like, does this have history? Is this coming from the right sources? Has it been validated? Right? And so I think that's gonna be a really, um, uh, important. I think the other side actually is law enforcement, and this is sadly already happening. The people are using these tools now in very malicious ways right now. And law enforcement don't have a like really good ways to deal with this.

• 18:58Speaker 01

  And so I think everything from this like on camera, like signing, we need this now. Like they really have no way to like kind of identify, uh, if the image was generated or not. And similarly like for, you know, audio recordings and things like that. Like there needs to be a kind of additional kind of levels of verification. Uh, and this goes into actually like video calls and voice calls. Because, uh, right now, somebody can call you on the phone and play a recorded, like generated audio of somebody they recorded 30 seconds of, right? And this can be this very nefarious means, right?

• 19:36Sarah Guo?

  It's a huge consumer fraud problem already.

• 19:38Speaker 01

  Well, it's huge consumer, but it's also like be- beyond that is becoming like a real criminal problem. Like criminals are be able to use these tools now and it's like the barrier of entry there is like very low. And so uh this is where like you really need like, you know, the phone calls, the kind of all of this, like you need more information identification and like kind of cryptography in- embedded into the system, otherwise it's completely going sideways really quickly.

• 20:03Speaker 02

  Yeah, this is where people would be using APIs like Element or LFG or Eleven Labs to create a voice snippet, right, where they'll upload to your point thirty seconds of voice, train a model. And then the output sounds like close enough to the person that you could fool a financial advisor or a bank or somebody else to, you know, do transactions on your behalf or things like that.

• 20:24Speaker 01

  Yeah, or and you like swipe their phone and and now you're able to like impersonate them completely, right? So yeah, so this is like a real problem and like having kind of authenticated pass is required there. Uh to really stop it and like we have actually like the phones are actually have so much already. Like we have Face ID and fingerprints, we have, you know, there's secure enclaves that sign things that are like haven't been hacked as far as I know, like so there's like a lot of the pieces are there. Now we just need like a product stack that actually pushes it uh to the user and and like to the products.

• 20:58Speaker 02

  Yeah, that makes sense. I guess one other area where some people have talked about overlap between the blockchain world and the AI world is around training. And there's almost like two or three different forms of that. One form of that is there's a lot of GPU capacity that was purchased for mining on the crypto side. And given how valuable GPU is now on the training side, there's all sorts of sort of models to aggregate GPU specifically for training in different ways, you know, aggregating access capacity. And then separate from that, there's ideas around, well, can you train a model in a distributed way across the blockchain more generally? Do you think either of those things are concepts that will work or how do you think about them relative to the future?

• 21:42Speaker 01

  Yeah, I mean, it's interesting because it it sounds like such a no brainer that hey, let's grab those GPUs that, for example, Ethereum just moved from proof of work to proof of stake. Let's grab those and start using them. The challenge is the GPUs there are like not the ones that AI folks wanna use, right? Like, kind of all the AI is really zeroed in on like, how do we get A100s or H100s? And the GPUs that like folks used for Ethereum mining and like similar is like older ones like that are not also focused on like floating point arithmetic, for example, as much. And so the challenge was more around like people who did did it, like Corviva is probably a good example, right?

• 22:27Speaker 01

  They were mining company. Like, it's more that they had a know-how how to build data centers and they can like get access to massive, like, talk to Nvidia and like get massive access to that versus like repurposing the same GPUs. Although, I mean, obviously, like for smaller models, for some specific, uh, maybe inference things, there's there's maybe transition. There's a question of decentralized training, right? Uh, in general, right? Like, hey, we have like lots of GPUs everywhere, can we train it? And the reality right now, the the requirements on bandwidth, right? Like, people who are training these models right now, they have like a, you know, 800 gigabit connect, right, between these GPUs, right?

• 23:07Speaker 01

  So, maybe you have 100 megabits on between this, usually not, and you need to like replay and like, uh, work around problems for decentralized. So I think decentralized training right now is like still not as realistic, although there's some research, people are trying. I think an inference is really interesting because we do need so much more compute for inference than we need for training, right? Like, it's it's a very interesting, like, economy of scale. You train once, like Lama trained once, and then everybody runs it everywhere. And so the inference is where I think there's a lot of interesting cases. One is, you want it to be private, right? Right now, if you're doing inference, uh, you need to send it to some service, and that

• 23:50Speaker 01

  service may or may not record it, and, uh, both input and output. Second one is you want large capacity at like that can scale with more usage, right? Tomorrow I have, you know, ten X more users, I want to be able to scale with that. And so this is where I think using some of this hardware that exists as well as kind of leveraging maybe new methods of privacy and coordination. That can again crypto has like NPC, like multi-party computation, there's zero knowledge proofs, et cetera. Like they can be leveraged to achieve that and have kind of secure, like secure decentralized inference.

• 24:32Speaker 01

  So I think that's way more realistic than training and also way more needed.

• 24:38Speaker 02

  And then I guess one of the really early applications that Nir was thinking about was data labeling. And to your point, the ability to pay people who are doing data labeling for AI purposes, right? And since that time, I think a number of companies have really grown out in terms of the data labeling world in a centralized way. There's scale.ai, there's Serves, there's a few others. Do you think the best solution in the long run is still a decentralized model where you're using tokens to pay effectively for labeling? Do you think things will stay in the centralized world? Like, how do you view all that evolving over time?

• 25:10Speaker 01

  Yeah, I think decentralized kind of a web three marketplace is a more effective way to do this. And it kind of provides few interesting benefits. One of them is that it opens up kind of the market, right? Where you don't need to set up like a local office and kind of hire people and train them, et cetera. Like you can just open up global market, anybody can join. And you have a very specific rules, right? That if they follow, they get paid, right? So I've used Mechanical Turk before, for example, and you can actually, as a client, you can just decline paying them, right? So people in Mechanical Turk, like the workers have very low kind of way to push back.

• 25:52Speaker 01

  If I say, at the same time, they don't have any like quality and knowledge assessment on the platform, right? So, so I think having quality, knowledge and this kind of escrow model all embedded into one marketplace that opens up for everyone. And you know, anybody everywhere can get paid at any time. Like offering that both the people who doing this work want because they kind of are more protected actually. And it's like fair game. And then the people who want to give tasks, they can actually get access to like way larger workforce. They can like specify specific parameters. They can, you know, price it at whatever level they want. That's going to be the kind of future of it.

• 26:30Sarah Guo?

  Can you talk a little bit about what makes the quality control problem for annotation hard here, right? Because one thing that I've seen with significant research labs is like still continued in sourcing of annotators for both pre-training sets and RHF because some of the external services and marketplaces can't get to the level of quality that they're looking for in particular domains. So can you just describe the dynamics there?

• 26:58Speaker 01

  Yeah, so I think there's two parts. One is like domain knowledge, right? That generally like hard, like it's hard to tap in into like a specific centralized service, right? Because they need to kind of like, for them to do payments, do all those things, they need to set up a subsidiary in whatever country they have their workers, they need to train them, they need to hire them. Maybe it's contracts, but like they need a lot of overhead that they do that. For example, developers, let's imagine, you know, you're building a new really cool developer platform which uses, you know, language models and you want to fine tune on code, right? Well, the existing platforms, like them hiring a bunch of developers to actually do this, right?

• 27:41Speaker 01

  And, you know, if they're doing this full time, that's like super complicated. Then kind of building out the validation tooling for how to like cross validate that the work has been done. Now, on the lefty marketplace, you know, any student can join and like do this, right? They don't need like, you know, join it, like get a contract with a specific company. They don't need to have the company in the local region to work with them. And like students, you know, for coding, for example, are really interested in doing this because they are usually don't have much money. And this is a way for them to practice their work anyway. And then as a task giver, you can actually specify the specific way you want the cross validation to happen. And one of the things we've done, it's like honeypots, right?

• 28:26Speaker 01

  Where you actually specify specific types of incorrect answers that people need to mark as incorrect. And otherwise they actually lose the buy-in. And so there's like actual like very. Clear like economic game theory where people have buy-ins, they they lose them if they like do poor quality of work. And so they have

• 28:49Speaker 01

  like way more incentive to do this versus like, let's say, if you're working on a contract, there's like way more leeway usually if you're not doing your work, right? So there's like just way higher kind of self evaluation as well that happens. And so, I mean, there's a lot of pieces that needs to come together for this to be like high quality. But again, it just opens up this marketplace and makes it effective. And it in a way removes a lot of the human part as well.

• 29:13Speaker 02

  One thing that I think is really neat about how near approaches innovation is you do both internal sort of near roadmapping and product development. And then you also have a series of things that you either spin out or spin up, or you're sort of involved with this sort of these ancillary companies or projects or efforts. What areas are you most excited about over the next coming year in terms of either near or some of these other efforts that you're involved with?

• 29:34Speaker 01

  So we do actually have a project in this Web three AI data marketplace that we are spinning out to focus on. Now, like they build a product, they have all the pieces. Now it's like ready to actually go to market and bring customers. I think the the really interesting area is kind of partnering with existing kind of either already Web three enabled or interested in Web three teams who want to give access to more functionality to their users, right? We have, for example, Sweatcoin, which is really good example of like, it was a Web two project that had one hundred twenty million installs that had a ton of people using it every day

• 30:15Speaker 01

  kind of for a very specific use case, right? Kind of tracking their steps and, you know, maybe getting a discount on their next shoes. But now as it's transforming to Web three, they're kind of opening up, right? And you can now participate in economic activity, you can, you know, learn about new kind of innovations that happen in the ecosystem. You can now, you know, but like as they integrate more into blockchain every system, I can potentially interact with like on the social side, do the tasks and gigs. And so like you kind of really open up the what before was like a very limited kind of economy to really this like, you know, composable open web. I think that's really exciting and like we will see probably more and more examples of that. And finally, I'm really interested in kind of, as I mentioned, like, because we have now

• 31:00Speaker 01

  open web and socialware, the kind of what I call future of SaaS. So I think a lot of, between Web three and AI, a lot of SaaS will actually start being replaced. Because right now, what SaaS is, is like one database with a specific UI for a specific problem. The database is the same between CRM, the hiring tool, marketing tool, even some of the project management tools, right? The database underlying is like not that different. And it's been just like the front end. And like interconnecting all of these databases is like a ton of work. It always breaks, right?

• 31:39Speaker 01

  But now you can have like the database you own, right? So using kind of Web three tech tech. And then you can build all of this front ends on top, either through kind of blockchain every system, shared components, or even through describing with natural language, some of the interfaces and business processes you want to have, right? So the way people will interact is like kind of their business operations and all the tooling they need will start to change. And so I'm really excited about this space. And like, we have one company that is kind of, you know, starting to build out some of the things in this space. And over next year, we'll see kind of that evolving.

• 32:16Speaker 02

  Do you think that moves to an agent driven world? In other words, when you imagine the interfaces on top of this that are sort of driving these business processes for future SaaS applications? Do you view them as sort of traditional UIs? Or do you view them as agents that are interacting programmatically or some hybrid?

• 32:31Speaker 01

  It will be a hybrid. So I, like in my imagination right now, at least, I expect like, you can describe a business process, which is like, hey, you know, when we have a new creative from like marketing department, spin up a Twitter campaign and create me a dashboard that tracks the conversions on our product, right? And so what it does, it like creates, you know, the pipeline of those things. And then it also creates a page where I can see like normal user interface of like analytics.

• 32:58Sarah Guo?

  So it might be more generated dynamic UI.

• 33:01Speaker 01

  Exactly, yeah. And it's like adjusted for specific use case you need. And probably there's like a bunch of templates that is like, you know, fine tunes for your specific problem, like, and this is possible right now.

• 33:12Speaker 02

  Yeah, I guess it kind of moves you down the path of what you were talking about in terms of like AI as CEO or AI as project manager, where you're kind of morphing into a world where you're delegating to an AI to drive a bunch of activities and then come back to you with the results like you would an employee or a coworker, which is very different from the world of UI today, where you just go to the same spot to see analytics, you go to the same spot for communication, which is your email, you go to the same spot for, you know, interacting with the workflow. And you're saying this should be more of a dynamic world where things get brought back to you based on a series of tasks that you provide out.

• 33:45Speaker 01

  Yeah, and it's like probably a shared environment as well, where, you know, we probably will co-work on a business process and, you know, we'll share one display, but then we'll maybe fork it because I'm more interested in conversion and you're more interested in retention, for example. And so, so that's kind of the dynamism right now that also doesn't exist, where like we all look at the same, you know, Jira task management. And I'm like, I don't really care about half of this stuff, right? But it's not a filter problem. It's like, I want different information showed in a different way.

• 34:14Sarah Guo?

  Author of the paper that changed the world, here we are in 2023, is it bigger transformers all the way? Are there other architectural directions that are worth thinking about that you're paying attention to?

• 34:25Speaker 01

  I think there's definitely something around like, how do we get these models to have the capacity to like let themselves think before outputting or like kind of process more? And I, I think it's like still within the transformer's structure and it can be like advanced, but I haven't seen anything that's like really matches my intuition around this. But I think the, like, the simplicity of this architecture and like, indeed, like the, the amount of optimization that's going into this right now is just, it will be really hard to match. And kind of, you know, with enough expressivity, you can express any function. So like, it's not, this is not the problem at this point of like, hey, we don't have an expressivity, right? It's more around how do we, I like,

• 35:11Speaker 01

  Compose a data set that's cleaner, better, or add some self-critique and understanding of like, is this content correct? Or I need more time to think versus, hey, I'm forcing you to output next token, even if you don't have an answer yet. So I think that parts we really need and I think they kind of fit into the architecture, but just require more engineering and more different types of tasks as well for training. I think like, you know, the fact that we're just using a big language model is kind of interesting because this is not a task you would expect.

• 35:47Speaker 01

  Everything to be able to just predict next token. So like, you know, starting to, obviously, R L F H been already helpful, but like, starting to like, hey, can you critique this answer? What would be the better answer, et cetera?

• 35:59Speaker 02

  Do you view that as a training or fine-tuning thing? Or do you view that as an inference thing?

• 36:03Speaker 01

  I mean, it's going to be like a combination, right? So I think we just need an architecture that at training time, you're able to... So like, I mean, the simplest thing is like, instead of outputting a token in the next, right? You can actually give it like, you know, empty token, for example, for some period of time. And then when it says like, okay, I'm ready, give it to output next token, right? And so this way you can train it to like, think more before outputting. And then at inference time, you can vary it, right? Like, hey, I'll give you more time to think, you know? Or like, no, you have no time to think. But then you can train it to like, actually be able to like dynamically to output this. So again, this is like a very simple thing, but like, you can keep expanding on this.

• 36:46Speaker 01

  You know, output it and then feed it back and like, is this the right answer, like, et cetera? So there's a few different models. But I think the, to Jacob's point, like, the fact that this model is like doing a really effective search in kind of this knowledge space means that probably like pushing more into that concept is more useful than doing more searches at inference time. Because like, it means you already lost all the semantics if you're doing search at inference time.

• 37:13Speaker 02

  I think you made a really interesting point where it's possible the transformer architecture increasingly is getting locked in. And there's two components to that. One is it just seems to run really well on the main silicon that we're using right now for AI, which are GPUs. And then secondly, there's so much optimization work going into it. And so much being built around it that it effectively creates optimization that just won't happen for any models anytime soon. And so you effectively end up with this interesting feedback loop or lock-in effect for this set of models. Do you think that we're in a spot now where this is just kind of the future for the next five years or 10 years or something? Or what do you think is the likelihood that other approaches or architectures will emerge anytime soon?

• 37:54Speaker 01

  I mean, there might be another architecture that reasonably fits with the same silicon. I think that there's an interesting example of there's a company that built an alternative, silicon that is kind of allows to process things in pipelines. And so like the chips are actually like kind of smaller compute chips, but they're kind of all like in a grid and the data flows from one side to another, right? So the example there is on one side, it's like a really interesting architecture can build really cool things with this, but it doesn't fit transformers very well, right? Like you can do transformers with it, but it doesn't fit very well. And like your cost that like you get like, you

• 38:36Speaker 01

  know, cost to output ratio is not that interesting. And so in comparison to, you know, you're just optimizing on GPUs or using some of the new hardware accelerators. And so this is where example, like, I mean, and I'm not to speculate here on specific company, but, you know, I wouldn't expect they will have like a ton of people lining up because like there is a ton of alternatives for transformers that are coming in. And like somebody would need to like go in and develop a lot of new architectures that fit better as this model. And so it'll be really hard for them to like be a viable business and kind of have the economies of scale that we're having right now to just kind of continue optimizing

• 39:19Speaker 01

  and building best state of the art chips, right? So unless somebody's like really investing in this, I think it will be more around like what, what else we can do with current silicon, right? And kind of combinations of this. And then, I mean, maybe there's something new will come out.

• 39:34Speaker 02

  Yeah, but when things lock in technologically, they actually tend to lock in pretty strongly until there's a really big sea change or sort of the optimization of those things hit a asymptote. And it's interesting because I think a prior example of this kind of chip plus software reinforcement loop was really the Windows and Intel.

• 39:52Speaker 02

  Monopolies of the 90s, you know, they used to call it Windel for Windows and Intel because it was such a strong mutual lock-in effect where you had chips that were optimized for Windows and Windows optimized for the chipset and it just kind of kept going from there. And so this is, I feel like, a stronger version of that in some sense, where you have the underlying compute architecture and the most important model reinforcing each other in a way that kind of locks both of them in.

• 40:15Speaker 01

  Yeah, and what changed that is pretty much kind of mobile, right? And creation of ARM devices, ARM chips that are kind of optimized for mobile and then came back into PCs, right? So yeah, so unless there's like a completely new form factor, which hard to predict, right? But also it's like, that's a lot of investment to go from not just software, not just hardware, but like full stack, right, innovation.

• 40:39Sarah Guo?

  Yeah, I think it's unclear if this is a strong enough market force, but the short-term, you know, demand-supply imbalance around GPUs with all of the growth of applications, especially as like you think any of these applications work, like inference needs grow, right? Your ability to build enough, for Nvidia really, to build enough GPUs to service the demand is like, it's blocking a lot of companies, right? And I think the question is like, there is more incentive to make heterogeneous hardware work than there ever has been. And like, can that catch up with the full-stack optimization that you described? The CUDA-like investment that Nvidia has made?

• 41:21Sarah Guo?

  It's super unclear, but I think like, there's been no reason to chase that until, you know, this past 18 months, and I think now there is.

• 41:29Speaker 01

  Yeah, but at the same time, we have like every single, you know, large companies doing their own hardware accelerator, as well as, you know, a bunch of folks who are kind of spun out of those. And so like, we're gonna have a, you know, a market full of hardware accelerators, which are still optimized for transformers, or at least like similar structured architectures hitting the market like this year and next year.

• 41:50Sarah Guo?

  Yeah. Ilya, this is great. I hope you will, after Alad and I work through all of the transformers authors, like Pokémon-style, gotta catch them all, I hope you'll come back for a reunion episode, but thank you for doing this.

• 42:02Speaker 02

  Yeah, thanks for jumping on.

• 42:03Speaker 01

  For sure, thank you.

• 42:06Sarah Guo?

  Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.