Ep 3: NEAR CEO Illia Polosukhin on the Origins of the Transformer Paper and The Overlap Between AI and Crypto

Jacob and Jason sit down with NEAR CEO and Transformer paper author Illia Polosukhin. They discuss Illia’s fascinating journey from Ukraine to Google and AI to crypto, the origin story behind the “Attention Is All You Need” paper and the overlap between AI and crypto. Illia also shared his thoughts on AGI and the problems that excite him most in AI right now. You can find Illia on Twitter (@ilblackdrago) and learn more about NEAR (@nearprotocol)

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Jacob Effron?

  This is Unsupervised Learning, a podcast about the latest in the rapidly developing AI landscape and what it means for businesses in the world. And really excited for today's episode. We are joined by Ilya here, who's had a really fascinating career, worked at Google, where he focused on deep learning, TensorFlow, and probably most notably was one of the eight authors on the famed "detention is all you need" paper that introduced the Transformers architecture, which has really catalyzed this entire wave of large models. As if that wasn't enough, he then went on to co-found the NEAR protocol, which is a scalable, robust, and usable blockchain for builders. It's achieved quite a lot of developer activity and traction in the ecosystem. Ilya, thanks so much for coming on and chatting with us.

• 0:41Speaker 01

  Yeah, thanks for inviting. Great to be here.

• 0:43Jacob Effron?

  Well, you know, before we dive into our specific questions, would just be helpful to get a bit of context on yourself, you know, how you came to work in AI in the first place, then how you made the transition over to the crypto side, and then kind of just giving listeners a bit of context on what NEAR is.

• 0:57Speaker 01

  For sure, yeah. Looking back, my fascination with AI started somewhere when I was 10, I think like watching the artificial intelligence movie. And so I actually was always imagining some form of like an assistant that you can talk to, that can do stuff on your behalf, including write code and, you know, hack into systems and trade on a market when I was a teenager. And so I was actually like, I was really interested in how like the systems would work. I remember building my first neural network somewhere when I was like 15, you know, just in Bosco. Not that I knew how to train them, but anyway, that was really interesting and so got me excited. And so I went to study applied math with a focus

• 1:42Speaker 01

  on machine learning. And at the same time, like my family is from very humble means, so I was starting to looking for a job from the first year, and I wanted to get a job in machine learning right away. So from the first year of university, I was working for this machine learning company, which had like a small office in Harke, where I'm from, and was the headquarter in San Diego. And the interesting thing about that company was that they were doing machine learning before the machine learning world existed. They started in like early 70s, and they were doing econometrics like then. And so it kind of evolved into machine learning. They were doing decision trees, which like... For a while, we're powering and still probably powering a lot of the kind of actually production scale machine learning right now in the world.

• 2:24Speaker 02

  So from a really young age, you're super interested in AI. So you you get involved and in Ukraine, you find a company that's working on this, who has headquartered in San Diego. I think you're going to have to help our listeners get a straight line here. So so so you're in Ukraine, working on machine learning. Somehow I we got to Google, working on find yourself on the attention paper. Now, new crypto, like walk us all the way through this so quickly.

• 2:52Speaker 01

  Yeah, there there's, you know, there's some order to that madness. But so, I mean, first, like, so I worked for this company and they invited me to pretty much move to San Diego to work out of the headquarters, which although I was actually interested in doing PhD in machine learning, they kind of gave me an offer, which being a poor kid from Ukraine, you really have to have to be insane to refuse. And so I moved to moved to San Diego, worked there. It was a lot of interesting work, like we did actually projects, not just doing like tooling, but also did a lot of projects for other companies kind of semi consulting way. But so that was all kind of decision trees, machine learning, and I always was interested in neural networks. Like, I read books, I read actually neuroscience.

• 3:36Speaker 01

  There's actually a really good book about how like in detail analyzing neuro cortex of cats. And so when I saw the cat neuron that came out of Google, so that was Andrew Yang and Jeff Dean build this kind of large scale at the time GPU cluster that they trained unsupervised way, that when inspected, they found that there is a neuron that reacts to cats, right? And so like, when I saw that, I'm like, that that is the stuff, that is what I need to do. And so, okay, I need to go to deep learning. This is happening. This was like twenty thirteen ish. And so I started interviewing pretty much for kind of companies that had that level of things, which, you know, Google being the top one. And although

• 4:19Speaker 01

  they were really trying to put me into ads where they were using the decision trees, which I was expert on, I was really fighting against that and what and and so got into research team, which was natural language team that at the time was just formed a new department called machine intelligence, which was Google Brain and this kind of natural language team kind of under one umbrella. And for me, actually, Natural language, like, a lot of the work on, like, Google Brain at the time especially was around images. And for me, images was always like, there's, you know, like, thousands and thousands of species in the world that can understand images and like, react and do stuff. And there's only one species that can understand language and like, answer questions and reason about things.

• 5:04Speaker 01

  And so like, definitely you wanna work on something that's like, requires way higher.

• 5:08Speaker 02

  Wait, wait, to be clear though, that was cats, right? I mean, it's cats that understand the, the language, so.

• 5:16Speaker 01

  So natural language was always exciting. And so the way I, I thought, the way to understand if the machine can reason and, and understand things is through question answering. And so, kind of, our primary focus for the team was question answering, like, how do we teach machines to answer que- read text and answer questions? Like, this is literally how we test kids, if they understand things, right? Reading comprehension, SATs, all this stuff. That's like, this is the only way we know that the other, like, box is able to, to understand the reason. And so that's kind of what was the focus of the team. We did, you know, we launched some stuff, like, we did a lot of deep learning, we have some papers around that. And it was a really amazing team as well. The main problem was, we were using recurrent neural networks,

• 6:01Speaker 01

  at the time that was kind of the stuff for language. And they were really, uh, well, they were really hard to train, that, that was one. And so attention, like, there was a tension pretty much came up, and so that, that mechanism actually allowed to kind of propagate gradients way more efficiently, kind of bypassing the recurrent steps. So that was one. But then the other thing is like, they were really expensive to run. And so we would not be able to launch anything in production, right? Like, and this is, I think, a really important thing for everyone right now in AI. It's like, these things are not cheap. Like, when, when you see words being output by the model, that's not because they want an, a cool, cool animation. That's because that's how fast the machine is actually producing stuff.

• 6:43Speaker 01

  And so, so to launch something in production, which we had some models in production, we dumbed down the models. So we would take this, like, really sophisticated, you know, recurrent model that reads words by word, you know, attends back, do all the stuff. We had like, even more, like, we have a paper, it was like hierarchical recurrent models. And then we dumbed down it to beggar words. So it literally just looked at all the words. And then like did a few levels of fully connected. And then output it like scored the answers. That that was like going from this like. You know, sophisticated generative model to a really dumb beggar words, don't have any context. Model. And so that's kind of was, you know, a lot of, a lot of the motivation for, for something like transformers came from like.

• 7:28Speaker 01

  We cannot launch recurrent models in production. They're really slow. And same time, beggar words actually worked okay. Like when trained properly, we were getting actually really, really interesting results.

• 7:38Speaker 02

  So, so that's interesting. We, we, we still haven't gotten to, to near crypto yet. Cause like I, I, we, I wanna hear that story still. I wanna make every share, everyone hears this.

• 7:46Speaker 01

  Yeah.

• 7:46Speaker 02

  But this is important because transformers is obviously one of the most transformative papers. We've had in, in this, in any space, but in obviously in AI in general. And it came because of cost savings. That what we're basically hearing from you is like, hey, we can't really do this stuff because of. It, it's just too expensive to do. So we gotta find a different way to do this. So. A constraint, which is really what happens in the real world too, is a massive constraint comes into play, even inside Google. It says, we gotta find a better way to do this effectively. Is that what, what you're saying there too?

• 8:13Speaker 01

  Uh, that was part of it. Yeah. I mean, so kind of Jacob had a very interesting insight around this like. Self attention, like what, what that would mean, right? For kind of extracting information. Like for me, kind of this idea always sounded, if you watch the movie Arrival. There was this like the, the aliens would talk in like the whole sentence as a whole, as one thing, right? As one concept.

• 8:35Speaker 02

  Yeah.

• 8:35Speaker 01

  And then you kind of need to press out what it means, right? And this is really. And that's kind of was the insight around the, the transformer self attention module, right? That like kind of. Ingest a whole sentence at once, right? And that kind of attends to itself, uh, to understand the meaning. And but like at the end, yeah, it was like, it was way more efficient to train. It was. For for like translation type task, you can ingest the whole, you know, input as one, like it's one process, like it's highly parallelized and then you can output maybe less efficiently, but still was like way faster than the SDMs.

• 9:10Speaker 02

  So question for you and this one too. So, okay, so so you and I go back a ways at this point. And I remember you texted me something when you got off a plane once. You're like, hey, cool. Look, I wrote this thing on a plane. It was like, I'm taking, and now I can, I can do near contracts in JavaScript as I think what you did. And that that plane ride is like, hey, it's supposed to be in Rust, but now I can do it in JavaScript, all that sort of stuff. So you clearly like to program too. And you're one of the great developers of the transformer. So which side of this fence was the most exciting and fun for you? Is it working on the transformer architecture itself? Or is it kind of like figuring out the self attention stuff?

• 9:43Speaker 01

  I mean, it's kind of the same thing, right?

• 9:46Speaker 02

  Well, yeah, it's one of those is like you're implementing it. And the other one's like, hey, we're sitting down and kind of figuring it out. And, you know, the math side of the fence, like, I know what side I would sit on personally.

• 9:56Speaker 01

  Definitely on implementing. So so me and Ashish, we really kind of started implementing. And then the whole like making it really work, like and scale and and and beat the results. That's when like a lot of really good researchers on the team join and kind of, you know, kick it out of the park, right? And so generally, like, and by the way, I left also halfway through. So like, a lot of that work was kind of done already without me. So generally, yeah, like, I'm definitely more in, you know, let's hack something together. Let's see how it works and and get it going. And then there is really, you know, smart people with a lot of intuition who have kind of, yeah, really good understanding how this models work. And obviously, like, from there, a ton of people contributed to making this thing actually scale

• 10:40Speaker 01

  to the level it is right now, right? Like, that's not a small key by any means.

• 10:45Speaker 02

  No, I can't imagine it is. I mean, this is, when you're talking about like, you know, at the forefront of everything, you're always going to be, there's always going to be two or three or four people, in this case, eight on the paper or whatever, but that get named. But then there's a team of people and teams of people behind it. It's just the way that the nature of of software in general. But even so, you know, it had to be an interesting time to be there. And also, particularly now, because like, when was that paper released? Was it 2017? Was that?

• 11:11Speaker 01

  Yeah, 17, yeah.

• 11:12Speaker 02

  17? Yeah. So here we are about five years later, right? You know, six years later at this point. And it's still groundbreaking foundational papers, though.

• 11:20Jacob Effron?

  I'm curious, like, when you guys were working on it and released the paper, like, did you realize what a monumental change it was? Or like, you know, to what extent? Obviously, it's like catalyzed the entire space now. You know, was that clear at the time? Is that something that's become clear as folks have done more with it? Like, you know, what extent did you realize this was this was kind of game changing when you guys did it?

• 11:37Speaker 01

  Yeah. So, I mean, it was definitely an interesting way, like completely different way to look at things. So, but I don't think it was clear that it's like that different right away. I think for, again, I left midway. And so I know the team kind of, when they started applying it to different problems, right? Like, not just translation, but like, I think images and other stuff, that's where they realized that this is like something really interesting. Like, if you can use the same thing and it works for other problems, that's like, that's when you started like, but I mean, for me, I think when people started using it in like pre-training, like the was the BERT is probably like a really good, like, again, BERT is like very close to me in question answering space. So I think that's where I was like, okay, well, this is something that like can scale and be used in, in, it's pretty much was like the most effective way to do pre-training, right?

• 12:31Speaker 01

  Which like you couldn't do. And then at the same time, it's like, then you can reuse that for, for question answering, for this different other tasks. Like, that's a fundamental block that we can build on top, right? And so I think that that was when I kind of realized it. But at the same time, to choose a story, how did we get to blockchain? So I left Google and we started a near AI company. And it was actually because at the time, we believed that we're on a exponential slope of AI, which, you know, maybe we're right now, maybe not.

• 13:05Speaker 02

  We'll get to that one in a minute.

• 13:06Speaker 01

  Yeah, we'll see that, but but we believe we're in an expensive slow because we saw a lot of progress. And so we wanted to kind of catalyze that and build, well, Copilot, that's what we wanted to build. We wanted to build a machine that actually helps people to code. And even better, you can just tell it what you want and it writes code for you. And so we had really interesting data. We had some subspaces where we wanted to apply this. And so we started building a startup near United to do that. And well, we kind of were missing, I think, three hundred million dollars worth of compute. At the time. But what we did instead, we had a lot of crowdsourcing. So we built a crowdsourcing network of students.

• 13:46Speaker 01

  Were willing to work for very small payment to write code for us, to write descriptions for code for us. And so kind of do a lot of like this coding related tasks. It's like for them were kind of interesting practice and for us was invaluable data. The challenge we faced was that they were in China, in Russia, in Ukraine. In kind of all the countries that have some form of capital control and trouble of sending money from US. And that's where blockchain comes in, like, hey, we need to solve this problem. We've heard of this like global payment system, right, which, you know, whatever. So we started looking at like Bitcoin, Ethereum, none of them were robust enough for doing this, right? They were like way too expensive for sending like, you know, cents per cask.

• 14:31Speaker 01

  They're way too complicated for everyone, they're like somewhat brittle in different ways. And that's where like, okay, well, that's a problem. We have a problem. We're systems engineers who know how to solve this kind of problems, like we know how to scale systems and build. System like that. And so that's where we're like, okay, well, AI kind of slowed down, like two thousand and eighteen, there was kind of not. Like the progress wasn't as as we've seen it in fifteen, sixteen, where things were really like churning. And so and we gave ourselves in the first a year to to try to to get to something. And so like, okay, well, we need to to pivot anyway. There is something we can do, it's clear. We were very arrogant, we thought we're gonna do it in half a year, we're gonna launch like layer one in half a year.

• 15:17Speaker 01

  We got it pretty quickly off it when we like started talking in the ecosystem and like learned the reasons why nothing scales and the complexities of all this. But yeah, so that's kind of how we went through this and like realized that we actually have a really interesting opportunity building a scalable infrastructure for this. And the interesting twist on that is somebody in the community built a crowdsourcing system called NearCrowd on top of Near that has actually been doing a lot of data labeling in the community for past like two years and employing like, I don't know, like one, two thousand people daily to do that. So kind of closing the circle there.

• 15:57Speaker 02

  That's funny. So basically, we could still that down too. You actually had a different problem and a constraint, and so you had to go solve that one too by building.

• 16:08Speaker 02

  I see a pattern here. So every time I have a problem, I'm going to call you at this point.

• 16:12Speaker 01

  I'll build a platform and a framework.

• 16:15Speaker 02

  That's right. Thank you. Thank you.

• 16:18Speaker 01

  At Google, TensorFlow was same story. Like the TensorFlow was built, I mean, amazing researchers, amazing team of Jeff Dean kind of built the foundation. And I was like, I had a team that needed to build stuff on it. And it was like super raw, the framework, the APIs, everything was super raw. And so in a, like not in airplane, but in airport, before the flight, I built this like scikit-learn API. So scikit-learn is like this Python framework that like a lot of people know in Python. And so I built the API on top of TensorFlow that was like, it was like, I don't know, 120 lines of code, pretty much. It just wraps all the complexity of like of TensorFlow at the time under it.

• 16:59Speaker 01

  And I got my team to start using it and then I actually open sourced it as a separate thing. And like, but a bunch of people started using it and that's like contributing to TensorFlow result because like we pulled it in into the, became like estimator framework at the time. So yes, that happens a lot where I'm like, oh, I'm gonna fix this small problem. It should be quick.

• 17:22Speaker 02

  The funny part of it is like right at the same time that you were rolling out and doing near AI and then going into near crypto is right when I joined GitHub. And even at GitHub, so you know GitHub was well funded at the time. I think it had a total of $250 million and more than that on the balance sheet. Making 100 and you know 2017 making about $150 million a year or something like that, you know. After I joined it was actions and packages and eventually co-pilot as you, you know, we launched there. We couldn't build a bunch of those as an independent company, even with that much cash on the balance sheet because of how expensive these things were. Actions started out as a workflow tool because it was the cheapest way for us to get the entire thing out there as a CI element.

• 18:02Speaker 02

  It would have cost us 60, 70 million dollars a year to build this, even with all the data that we had and the systems. And co-pilot, we had been talking about that for quite some time too, in a slightly different vein. But when we estimate how much it would cost to build this and we just couldn't do it, we just, there's no way. So post-acquisition by Microsoft is when the coffers were unleashed and you could go and do some of these things. But it's amazing how expensive this stuff is.

• 18:25Jacob Effron?

  Yeah, it's, it's, it's super interesting and I think, you know, the overlap between AI and crypto you alluded to is really interesting, right? I mean, you know, obviously with data labeling, but even with the increase of this like reinforcement, learning from human feedback. It's, it's not hard to see how kind of crypto could be an interesting way to, to enable that at scale. I think just generally I'd be curious, there seems to be a lot of overlap of people that are interested in crypto and AI. You know, there were for a while, there was, you know, crypto people funding a lot of these AI companies. You know, it seems like a lot of overlap between the two, like I'm curious how you think about like the, the overlap both today between AI and crypto. And then as you zoom out, you know, five, 10 years, 20 years, uh, that, that, what that overlap looks like.

• 19:00Speaker 01

  Yeah, for sure. Well, definitely there's a big overlap. I think there was a lot of like AI people going to blockchain and now blockchain people going to AI. But I think generally it's like, it's the people who are like on the front of kind of of a technological shift, right? Like both AI and Web three broadly speaking is like a major technological shift. Where everything else is probably like just an additive thing. And so I think a lot of it spans from that. So here's a, like, here's how I, how I think about kind of AI blockchain intersection. So AI right now is built on top of three pillars. It's data, it's models, architectures and compute.

• 19:41Speaker 01

  So for data, we already talked, it's crowdsourcing, it's, you know, it's also a very important piece, which is, you probably want a more neutral kind of global community that is actually contributing to alignment data sets. Like, you don't want this to be controlled by name a company, because that name a company will always have a bias.

• 20:02Speaker 01

  We'll not be mentioning any, any previous employers of anybody on this show.

• 20:09Speaker 01

  But, but anyway, but like, I mean, any individual or any like small team, they, they will, they will introduce bias by, by all means, right? And so, like, I think there's a really interesting question of like governance of data and how kind of this alignment. And by alignment, I don't mean just like construct alignment, right? Like the how to follow structures, but also like actually value alignment, like what's moral, what's not? How to not scream at people, how to not threaten them to be get killed, right? Stuff like this, we need the data set like that. There's actually no like large open data sets to train on as far as I know right now for this. And so, so that's, I think, a huge space where, again, between crowdsourcing, DAO governance, kind of global

• 20:51Speaker 01

  community of the, of the blockchain, I think is, is really important. Models is really interesting because they're really very open, like it's, papers are published, you know, source code is mostly open. And researchers are like-

• 21:05Jacob Effron?

  That may, that may be starting to change, right? Like, I think, you know, Anthropic OpenAI seem less inclined to publish these days.

• 21:11Speaker 01

  So, so here, here is like, my belief is open source always wins, right?

• 21:17Jacob Effron?

  Yeah.

• 21:19Speaker 01

  It's, it's just an inevitability, like, anything that can be open source will be open source eventually, right? Like, and so it may be lagging, like, it may be that, like, open source is like one generation earlier. Because there's a really interesting, like, kind of this balance that, like, OpenAI doesn't want to. That means that the second, the, the one after OpenAI in whatever leaderboard it is, they will open source because that's a way for them to like get all of their attention, right? And so, like, you may not have like the last, but like the, the reality, the reality of this right now is people throwing, like- Stuff at the wall for the most part. It's not, it's not like there's like in systems engineering, like everybody's thought through, there's an architecture, there's a trade offs, everybody clearly understands what the trade offs are and people go build.

• 22:03Speaker 01

  But it's a lot of experimentation and people like, what if we try this? Like, I think this will work. But like a lot of the research is based on kind of intuition and, and really trying things out. And that's why having a bunch of researchers working off kind of similar setup is really important. And so I think this is where there's a really interesting kind of blockchain coordination component comes in where can we have kind of an open coordination of model? And Hugging Face is kind of a good example of like a in that direction, but like something where, you know, you have leaderboard, you have ways for people to experiment, to fork off each other, to kind of

• 22:40Speaker 01

  like work in the same environment. And like, how do you incentivize that? Right? Because a lot of these researchers sit right down in universities and like don't have access to resources. Like, how do you incentivize? How do you structure that? I think there's, there's an interesting kind of space there where blockchain is a tool.

• 22:56Speaker 02

  That is interesting because the three pillars that you mentioned, two of them are naturally incented to be closed. Like one of them is like proprietary data or whatever, but compute is a closely held resource. And so that like it's, it's the models themselves that are open source. And I, you, you and I have talked about this in the past personally, like I believe in open source. My, my entire career is based around it. Data is the one element where there's like only elements of it that are open partially, but you know, compute, no one is out there. You gotta schedule it, you gotta buy it. It's a tightly controlled resource.

• 23:25Speaker 01

  Yeah. And so compute, I think is really interesting because on one side, right now, the requirements for compute for training, for training are so extreme. Like, I mean, we're talking about supercomputers, right? This is like a, a highly connected thirty thousand dollar GPUs with interlink that's eight hundred gigabit per second, which is faster than interlink between a GPU and CPU. Like it's literally faster to send data to another GPU than to save it to RAM. Right, that's just ridiculous, like, you know, but that's that's a setup that these things are running at. And so, like, that is, like, that's a lot of expertise, a lot of money, investment.

• 24:04Speaker 01

  And, yeah, that's why it's like, very limited number of parties have this access and very, like, and it just control, like, access to it is is highly controlled. Now, there are two ways this is gonna be evolving from a perspective. One is commoditizing that, so there is already research around like how to use a more commoditized setup by like pipelining differently. And so you lose some, you gain some in in different ways. And so like, obviously, this is where having access to more of this, like, and by commoditized, I still mean like, you still need a A one hundreds, but you don't need like this like highly connected setup. So it's still, it's like a normal cloud, not not, you know, pay pay ten million dollars to to play a little bit, uh, what it is right now.

• 24:49Speaker 01

  And so, and then the other setup is kind of people building out just like pods of compute and renting them out, right? And so, like, the expertise will continue to spread, like, that's just inevitable. And so there will be more of this pod and then you kind of need to like manage allocation of resources to this pod, right? So you can imagine a marketplace where you're like, hey, I want a hu- a thousand GPU pod or whatever, a thousand GPU pod. And like you bid on it and then, you know, a bunch of people bid on it and so you like you get access to one of the spots from.

• 25:21Speaker 02

  You you could totally see this model in like an ads model sort of way, or even just like, you know, in crypto where people pay to jump the line in certain cases and stuff like that, too. Which has its own own issues and stuff like that, but the point being like, right now, it's a very closed, um, tightly controlled.

• 25:36Speaker 01

  Yeah, like you you either, like, this name and then you have access or you nod this name and you don't have access. Like, like, there's no, like, even if you have money, you are not able to, like, you cannot buy actually with your one hundred if you nod Google, Microsoft or Amazon.

• 25:49Speaker 02

  Exactly.

• 25:49Speaker 01

  Or like a few and few others, like you, they just don't gonna sell you.

• 25:52Speaker 02

  And that to me is one of the things that's like, you know, gonna hold back, obviously, the space a little bit, but it's it's it goes back to, I mean, in this world, money matters so much and money in the name, money attached to the name even matters. It's kind of a weird space. So I think it'll evolve, obviously, but well, it'll be tough. So I don't think a lot of people put together the crypto element of that, too, which is fascinating.

• 26:12Speaker 01

  Yeah, but I think in crypto, there was a whole way around like AI and blockchain and people thought, okay, we're gonna train all the all the AI on this like leftover GPUs from interior mining. And so just wanna say, no, that's not gonna happen.

• 26:27Speaker 01

  First, those GPUs are actually not the same GPUs that

• 26:30Speaker 02

  They're different GPUs. They use the same word, but it's like saying a Hyundai and a Porsche. They're they're both cars. So they're very different. Awesome. So Ili, I like to ask this question. I'm curious about this one, obviously, and Jacob's already gonna laugh because he knows exactly what question I'm asking here is. Where do you sit in this this race towards AGI? Like, what do you think is gonna happen? What do you and what's your your general view on this?

• 26:55Speaker 01

  Well, our company was literally called Near AI. And the idea was to rename it to Here AI when we get there.

• 27:03Speaker 02

  And to be clear, too, you your your near co-founder is off doing something at the moment in this space again, isn't he?

• 27:10Speaker 01

  Exactly, yes. So I I believe we don't have like a good definition of what what intelligence is, what sentience is like. So we we all kind of like always discussing different things. And so here's the things that like I believe and that are highly like likely in actually very short time now.

• 27:31Speaker 02

  And short being like two years or short being twenty years?

• 27:34Speaker 01

  Five to ten, five to ten.

• 27:35Speaker 02

  Five to ten.

• 27:36Speaker 01

  We want some really intelligent assistants, right, that kind of replace most of the mundane work, right? I think everybody at this point, okay, like we we see how this is happening. We want a knowledge kind of assistant that is able to summarize, understand, augment pretty much our own capability. And honestly, Google have been this for past like ten years and down like. I like to say this is like, everybody's like, hey, we want, you know, intelligent assistants. We want, you know, superhuman intelligence. I'm like, type in google.com. You literally, you literally already have it. It's this thing that knows more than anyone else in the world about everything. It just have no logical reasoning, right? It has no, it has no like common sense, but it knows everything, right?

• 28:21Speaker 01

  It's like a ba- baby that like uploaded the whole like encyclopedia, right? But now we have something that's like, has a lot of common sense and at the same time bullshit itself. So this is where, there's definitely an interesting, you know, proper combination of those two things where like it actually understands where it doesn't know things and it's like relying on common sense to summarize things. So, so we'll get to that, which is like a, a knowledge assistant. So there'll be a kind of skills assistant, right? Thing that you can like teach to do stuff and it does that. There will be a knowledge assistant. And so the question is, will there be a, something that has its own will thing? And I, I actually don't know that humanity wants that, like, like realistically.

• 29:06Speaker 02

  I have kids. I don't want that. No, not, I'm like, I want to rethink that whole thing too.

• 29:12Speaker 01

  Al- although, you know, there's like character AI, for example, which is trying to build that, right? Which is like, you know, creating arti- kind of artificial beings and connect to it. So, so maybe we want that as well. So there will be those. And so if you unite all those three things into one thing, right? That, that's where we get to some, like, for all intents and purposes, it would be really hard to distinguish it from, like, from another human, right? And so that's, that's what like I call, you know, if you, if you connect these three pieces, that will be a kind of general intelligence. Now. Where things get really complicated is, okay, now if you have that and somebody decides to leverage this highly scalable

• 29:53Speaker 01

  system that is able to like reason, summarize, you know, comprehend, talk at a level beyond like human or above. What are the ways this things can go really wrong, right? And so this can, this can go into like, obviously, you know, finding zero data vulnerabilities, you know, of the kernels and network cards and hacking all of the networks and, you know, making sure nobody else builds an AI. It can be going into like, you know, flooding the system, our social system with, you know, frivolous lawsuits, you know, fake political candidates, like things like that, right? Like there's so many dimensions now that, and like that's, again, this is not like AGI level even of capabilities you need to start doing this, right?

• 30:35Speaker 01

  This is like a little, maybe a little bit more advanced than we have right now, but like we already have, we already see this kind of happening, right? This like

• 30:42Speaker 02

  Yeah.

• 30:42Speaker 01

  Fa and some other startups.

• 30:44Speaker 02

  So then that's my question here for you on this one too, is that you, you've got a certain size of these camps, you got the, let's just call them like two extreme ends of a spectrum. You got the total optimists of AI, then you got the doomers on one side and the doomer, you know, and like, let's just, you know, not talk about whether or not one is right or wrong, but I'm just curious too, what your view is, do we have to get all the way to the will for the doomers to be correct? Like, hey, that's a dangerous time for us or I don't believe that to be the case. I believe that we can start to see people use these tools in ways that just would, as a society, we're fully not ready for at this point. Like, like you said, it could span the legal system overnight or somebody could use

• 31:22Speaker 01

  So this, this, these things are already used, like even before G P D like thing, the machine learning was used to for various reasons, right? It's not data mining on, you know, stolen

• 31:32Speaker 02

  Would people use tools for evil if we release them? I don't, wait, hang on a second. Isn't that not part of the?

• 31:38Speaker 01

  There's some kind of code, right?

• 31:40Speaker 02

  Yeah.

• 31:42Speaker 02

  Well, we need an open source license that says, no, you're not allowed to do that because people will definitely adhere to that.

• 31:47Speaker 01

  Yeah, don't be evil license, right? There's some companies won't be able to use it.

• 31:54Speaker 01

  Anyway, yeah, so first of all, I'm an optimist, but I'm an optimist that understands that like, hey, this will be used in bad ways. Like same as anything else, like every single tool will be used in good and bad. And good will overrate bad, but the bad is like the repercussions of bad is usually like way more visible than even sometimes a good that. Like crypto right now is really good example, right? For example, we use crypto to raise money to help people when war started in Ukraine. FTX completely annihilated that, right? Like that's kind of the like the bad, the bad of things like and FTX is not even a crypto problem. Like, you know, it's the regular, like, somebody was doing this.

• 32:31Speaker 02

  It's the nature of a tool, right? So the, the, the more powerful the tool, the more powerful the positive and negative repercussions of it are. So it's the same thing as having like a bow and arrow versus a machine gun versus a tank versus a whatever. But even so, like it's just in general, the more powerful the tool, the more, more powerful good it could have, but the more destructive power it could have, which is. This is why I don't understand the doomer side of the fence because like that, that just means that that's the exact evolution of all of us to a degree. We have to actually figure out how to work with these tools. It's just the nature of it.

• 33:02Speaker 01

  Yeah, exactly. And so it's, I think it's really important to, to have a plan, which is like, hey, you know, if you have some, something that has capable of finding zero, no, zero day vulnerabilities. In kernel, for example, like what, what, what is our plan here? Like, how are we doing things? Are we like isolating things that are like not connected to the internet? Are we like, what, like, we should be thinking about this kind of stuff. And we probably should be starting to think about it now. So I, yeah, I think again, machine learning has been used for nefarious reasons for, you know, years. Hands down, social engineering, all of the stuff have been done before by humans and like troll farms in some countries already. So like now this kind of things come together, you know,

• 33:46Speaker 01

  and will be used at the highest scale. And again, they exploiting the existing problems with the system like, you know, the fact that you can create frivolous lawsuits in legal system in US, it's a problem with legal system, not with AI, right? Like, and it's been exploited, you know, by patent trolls and everyone all the time without AI. It's just like now you can do it at scale. And so, so I think the question is like, yeah, we have all the systems. They're kind of designed like pre-digital. A lot of them are pre-digital. And so we need to like digitize them. We need, you know, we need to embed cryptographic identities. We need some form of like incentive structure that is like actually like mechanism design, not just like thrown together, you know, in seventeen hundreds.

• 34:29Speaker 01

  And like we need to kind of start designing things in a way that's like for the modern world. And that like society needs to adjust to that. And again, it's not, it's not because of the AI. It's because like we're living in a world where technology is ruling, ruling the, the system now way more than ever been. And like this systems are not designed for that.

• 34:48Speaker 02

  Curious here too. And then I gotta get back on track. We actually have a script that we are way off.

• 34:53Jacob Effron?

  It's always more fun when you're off script anyway.

• 34:55Speaker 02

  Yes, this is, this is fun. So I'm curious. This is actually, I'm a VC now. You've, yeah.

• 35:04Speaker 02

  So one of the things that I think about a lot in this space is how the economic incentives are not aligned to have this conversation. It's way more profitable to build an OpenAI or an Anthropic or Cohere or something like that, like that sort of thing, or something that will span the legal system than it is to build defensive mechanisms or things that will protect and things that, and it's just the nature, again, going back to like historicals, it's the nature of tools, like defensive mechanisms or things that protect or things that are able to, you know, put harnesses in place and don't really get the same attention as the other, other elements like. But in this case here, it's, I feel is probably more important because it, it spans the entire, like our entire society and all of our technical underpinnings. Like you mentioned the zero day stuff, like, you know.

• 35:49Speaker 02

  I keep on saying, I don't think society is ready for this. I know for certain there is no company in the world that is ready for a machine learned neural net that is very smart to attack its systems, like nobody. And we don't understand the ramification that could happen from that. Is there a way that we could change, one, how people care about that sort of thing? Or can go to work in it, or the incentive structure? Or, you know, you're very passionate about this one thing. We've talked about this in the past too. Like, how can we approach this differently?

• 36:20Speaker 01

  Well, so I mean, I think this is like a combination of things. One is, we need to raise awareness of this, right? I think last few weeks with like Bing conversations, right? Have highlighted some of the problem. But I think it still, it still lives in this like consumer

• 36:37Speaker 02

  Yeah.

• 36:37Speaker 01

  space. I think it's, it did not hit like, hey, your company is vulnerable type thing. Or your government is vulnerable, or your, you know, economic system is vulnerable type thing. And so I think showcasing that is important, but also starting to think what are the solutions, right? So, so having some form of a gathering of minds who are brainstorming on a, what are the ways this can be? Kind of the current systems can be exploited. And then, you know, proposing concrete solutions, right? And concrete solutions span, again, like, I do think the blockchain actually is a big part of the solutioning here because it provides a lot of the kind of authentication, cryptography, you know, traceability, like simple things like that.

• 37:20Speaker 01

  Like, for example, news articles should contain cryptographic signatures of the article itself, but also any quote should be actually signed by the person who gave the quote. Like, that should be a standard we use because like, otherwise you can do whatever you want. Like, you can write whatever you want, the article can be completely fake. Nobody knows what's going on, right? Like, things like that. And then your viewer, whatever, your Apple News, for example. Validate that this is like indeed a legit article that has the provenance and has like, you know, it's linked to a press release, it's like being authenticated, the the company that this article mentions have cosigned on it, et cetera. Like you can have all this kind of mechanisms. And we have all the technology for that. So like that's, you know, si- simple to to orchestrate.

• 38:04Speaker 01

  It's just now, we need to showcase that this is a problem. Like you can generate a million articles, each one of them, you know, like for each individual person, like they go to a website and you show them an individually generated article that convinces them exactly on specific point in specific way that convinces them, right? Like we can do that. And that's obviously not a great world to where everybody's kind of manipulated through through that. But like that's currently happening like that. Like main point here is this is currently happening already. I will not talk about specific news outlets, but like this is just done ma- manually right now, right?

• 38:39Speaker 01

  So anyways, kind of generally speaking, right, raising awareness, showcasing solutions, kind of, there's like few fundamental pieces that are needed, right, to defend. And it's a combination of AI and blockchain kind of coming together, right, as solutions. And then almost like bringing it into what happened with cyber security, right? Like cyber security was like, nobody cared about it until everybody started really caring about it. And now this is, you know, multi hundred billion dollar industry with like tons of really powerful, like really important companies that are creating like layers and layers of protection. Like I think we need to kind of get to that level. And so hopefully it will not be because everything got broken.

• 39:18Speaker 02

  And that's the thing, like thi- in this case here, it's like everything could get broken fast. Whereas in cyber security, you can hear the story about how like one company got compromised and um, it could be different here.

• 39:28Speaker 01

  In cyber security, people weren't preparing, like, like it's not, like, you know, people were hacking systems for past fifty years. It's not a new thing, right? It like just here, things will go like to extreme, right? But again, like I think like cyber security firms should be starting to think about it now and kind of creating, now creating next layer of of protection against this kind of system.

• 39:50Jacob Effron?

  Yeah, I mean, I feel like a lot of it goes back to, you know, this is like a uniquely exponential pace of improvement, right? And I think actually, I I'd love to hearken back to something you said earlier on this conversation that was that was intriguing was, you know, we've obviously had this exponential pace of improvement, you know, in in AI. And I think you expressed some skepticism that that might continue in the in the short term. And I'd love to just unpack that a little bit.

• 40:09Speaker 01

  I I so it wasn't skepticism, it was more like, it's more that, at least from my experience, every time there is a kind of hype, and it looks like everything is moving really quickly, that usually follows with like a a some period of kind of things are moving slow. And so, because it's like, it's a step function, right, there's like a step function happens and then like, on the step function, there is like still some improvement, but it stops being exponential. I mean, even right now, like, we still love linear improvements, right, but it's scale. Like linear improvement is not, is not like exponential.

• 40:44Jacob Effron?

  Yeah.

• 40:46Speaker 01

  So, and like, and and you know, yes, control more and more computer it and will become like somewhat smarter, but it's not solving the fundamental problems that we know these models have. Now, there's a lot of people trying to solve them, and so, I'm sure there will be like kind of really smart people trying every possible combination of every possible idea, until one of them works really well, and that's when we get a get another step function, right? So that's kind of, at least, again, based on experience, what's been happening, right? Like, there there's like a fundamental thing, then a lot of people need to experiment a lot to get to kind of next step. And then, again, a lot of experiments is just like, we see the step function and we get excited because we expect the new next step function.

• 41:31Jacob Effron?

  Yeah, I mean, if you weren't working on near, like, what what are, you know, some problems in AI that you might be trying to solve right now?

• 41:37Speaker 01

  Well, so there's two core problems, right? One is the kind of reasoning, like, these models are still not like reason very well. And so there's a trick, right? How to get like ChatGPT to be smarter is to tell it to like explain itself as it thinks. And so that's just a a showcase that there's a missing piece there, like, like, if you need to tell it to do this, and it cannot do it itself. That means like we need to like build that somehow. And then the other part is, yeah, this combination of, you know, superhuman intelligence of like search engines, but has no common sense and way to like kind of summarize it. And this like common sense embedded in like this extremely large model,

• 42:19Speaker 01

  like how to combine this, like how to get the model to, instead of trying to remember every single thing in the world, it's able to like, let me look it up, right? Like, because like really smart people right now don't like, hey, I know everything in the world. No, it's like, I'll look it up and I'll be faster at looking at than you.

• 42:36Speaker 02

  Let me just say that I think we actually found our definition of AGI here, which is common sense, because that definitely is superhuman and definitely what most humans don't have. So if we can get to common sense in these systems, we're there. But no, okay, that's fascinating. I like the reasoning one, too. I mean, that's, that's obviously going to be an interesting step if we can ever get to that point.

• 42:57Speaker 01

  Oh, for sure.

• 42:59Speaker 02

  Yeah, and for sure. Like, one of the things that I think about when you say that, too, is I think we naturally go, end up going through these phases where we have, like, what I colloquially call these expand-contract phases, where it's like, everyone kind of comes together, works on something really quickly, and then they get some breakthrough, and then they can go away and go do and experiment all these different phases. And you can call them different words if you want, but basically, all industries, all movements kind of end up going through this. And it feels like we're kind of on another one of those where like, everyone had that breakthrough, and we're kind of going off in all these different directions to experiment. But at some point, we're gonna have to come back and say, hey, the next hotness, the next thing, the next platform, the next blah, blah, blah is gonna be this, therefore that's gonna be the breakthrough.

• 43:39Speaker 01

  Hopefully, it's gonna be open source and powered by blockchain.

• 43:44Speaker 02

  Well, I mean, yeah, I mean, I definitely think that's gonna be the case for what it's worth. Even if it's closed in the beginning, somebody will figure it out and reverse engineer it or blah, blah, blah, the thing. Next thing you know, it ends up on GitHub or Hugging Face, and there you go, you know?

• 43:56Jacob Effron?

  And then what was the second problem? You mentioned, I guess, the first big one being this like common sense and reasoning. What's like the second big problem in the space you see today?

• 44:03Speaker 01

  Oh, that's what I mentioned, like, the retrieval of the, like, how to not require this model to remember everything in the world in weights, but instead being able to use external kind of knowledge base, right? Like, I need to look it up pretty much, kind of model, right? Because, and again, like, right now, like, the fact that we need to keep scaling the models, to me at least, you know, it's, it's not, like, obviously, it remembers more things. Like, it's not, it's not, it's not just that it gets, like, I mean, there's some level of, like, new capabilities that get, appear from that.

• 44:38Speaker 01

  But like a lot of it is still like a lot of this this model is just storing knowledge, right? And like, we don't need to do that. And so like, how do we, you know, compress that, use the external knowledge, like either look up at the moment, or, you know, have it pre embedded or whatever. And so kind of decomposes to pieces because that that also can like make the inference cheaper and potentially kind of, you know, simplify a lot of things.

• 45:02Jacob Effron?

  Yeah, that's super interesting. Well, we always like to end these podcasts with a quick fire round. And so we'll ask you a few questions and get your thoughts. We like to do the same set of questions. And so, you know, maybe to start, obviously, you know, AI Twitter is very active these days. There's lots of things that get discussed. What's one thing that you think is like overhyped that's talked about a lot and one thing you think is underhyped?

• 45:21Speaker 01

  I mean, I'm mostly on trips with Twitter, so I don't even know what's AI Twitter. AI Twitter for me is like, I saw a new, for example, optimization function that says they're better than Adam. That's pretty cool. Like, would be cool to try, right? I'm more interested in that kind of stuff. I think the probably like underhyped is just like, yeah, like, how to really make this inference is cheaper, right? Right now, just the cost, the sheer cost of running any of this is, I mean, it's like cents per query, right? Like, it's just insane.

• 45:52Jacob Effron?

  Yeah. Yeah. And then I guess next would just be, you know, how do you see governments getting involved in AI going forward?

• 45:58Speaker 01

  Well, I mean, usually I'm on the side that's like, you know, just give the money and let the let the industry figure it out. Again, I think what needs to happen here is really around not about policing AI. I don't think that's possible. And like, at the end, it's a bunch of source code that, you know, if it's not going to happen in US, it's going to happen somewhere else, kind of thing. I mean, same as blockchain. And so what they should be doing is is actually adopting these technologies and using them for like, for their own kind of use cases, but also for our own defense systems, right? And so I think like, what government should be doing is really be like, the consumer of this and, you know, through that actually fund and to Jason's point around like, lack

• 46:43Speaker 01

  of incentives, they should be funding the defense company, like AI defense companies through pretty much buying that as because, you know, imagine some other country has capabilities comparable to OpenAI, and they do have nefarious agents.

• 46:59Speaker 02

  This is where I spend most of my time thinking about the defense side of things when it comes to regulation, which is, one, we always lament the fact that most people in government don't actually understand tech in general. And this compounds that problem by orders of magnitude in these types of systems here. And the other is, the other actors in the world won't have the same sort of thing. So you put regulations on, one, it only favors incumbents, the Microsofts and Googles of the world. Two, you're not putting any restrictions on China, they're not going to abide by that. So what's going to happen on that front? So this one is always fascinating to me.

• 47:31Jacob Effron?

  No, I mean, this has been, this has been like a fascinating, wide-ranging conversation. I know folks are really going to enjoy it, and I'm sure they're going to want to learn more about you and some of the work you're doing at Near. So what's the best way for them to do that?

• 47:42Speaker 01

  Well, Twitter is the best, so @nearprotocol for Near, and then @billblackdragon for myself.

• 47:48Jacob Effron?

  That's an awesome handle.

• 47:49Speaker 02

  We'll have to make sure that's actually in there, too. We're going to get you tens of followers, it's going to be amazing.

• 47:57Jacob Effron?

  Game-changing. No, but seriously, thanks so much. This has been an awesome conversation.

• 48:02Speaker 01

  For sure. Thanks for inviting me.