Joscha Bach: AGI, Consciousness, and the Evolution of Intelligence

Artificial Intelligence is changing faster than most people realize, but are we asking the wrong questions about AI, AGI, consciousness, and the future of society? In this episode, cognitive scientist and AI researcher Joscha Bach joins us for a thought-provoking conversation on artificial intelligence, machine consciousness, large language models (LLMs), AGI, human intelligence, and the future of work. Joscha explains why intelligence is fundamentally about model-building, why consciousness may be a biological training algorithm, and what current AI systems are still missing. The conversation goes far beyond technology, tapping into innovation, capitalism, regulation, scientific progress, institutional trust, democracy, media narratives, AI safety, employment disruption, economic incentives, and the future of human civilization. Joscha shares his perspective on AI doom scenarios, why he believes many fears around artificial intelligence are overstated, and how AI could ultimately empower individuals rather than replace them. This episode offers a different look into artificial intelligence, human progress, and what comes next. Like and follow for weekly episodes. In this episode: 00:00 Intro 01:04 AI doom scenarios & humanity's future 02:00 Defining intelligence & consciousness 05:00 Why AI intelligence is different from human intelligence 06:52 Are LLMs actually intelligent? 08:10 Why AI hallucinates 10:09 Human learning vs machine learning 11:22 Can transformers lead to agi? 12:28 The scaling hypothesis explained 13:58 Mechanistic interpretability & neural networks 15:13 Can AI surpass human understanding? 18:42 AI and the future of scientific discovery 20:14 Why scientific progress feels slower today 22:03 Institutions, incentives & organizational decline 25:05 Trust in government and society 28:31 Can AI improve democracy? 30:40 Why AI gets such negative media coverage 33:03 The AI extinction debate 36:30 AI, automation & the future of jobs 40:05 Big Tech, AI power & capitalism 41:15 Money, markets & resource allocation 46:28 Regulation, innovation & economic growth 49:13 Why innovation is slowing down 51:08 Is Big Tech starving other industries? 54:39 Is there an AI bubble? 57:30 Could the financial system be rebooted? 01:00:17 Cryptocurrency, regulation & economic stability 01:08:27 Regulation, governance & innovation 01:10:33 Lessons from Berlin's innovation culture 01:16:08 AI, surveillance & centralized power 01:18:07 Decentralized AI and individual empowerment 01:20:05 AI, healthcare & professional expertise 01:23:36 Control, society & human responsibility 01:25:00 Advice for leaders and innovators 01:26:03 Escaping filter bubbles Connect with Joscha Bach: LinkedIn: https://www.linkedin.com/in/joschabach X: https://x.com/Plinz Our links: Visit our website: https://www.infotech.com/?utm_source=youtube&utm_medium=social&utm_campaign=podcast Follow us on YouTube: https://www.youtube.com/@InfoTechRG

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Speaker 00

  There is a group of people that believe that artificial intelligence is destined to murder everyone of us in the same way as we have diminished the habitats for large apes. We are a large ape that is breeding its own successor species and the successor is probably going to get rid of us because it's competing for space and resources with us and we are going to do this because it's smarter than us. This is an argument that seems to make sense and the ways in which this doesn't make sense relate to your gut feeling where you realize that the story is probably way, way more complicated than this.

• 0:39Geoff Nielson?

  This is a show about the future of tech and the future of work. I'm Jeff Nielsen and today my guest is Yasha Ba. He's a cognitive scientist, AI researcher, and one of the most interesting thinkers in artificial intelligence right now. It's impossible to listen to this guy for 30 seconds without him changing the way you see the world. Whether it's how AI works, how our minds operate, or the superstructures we've built as humans, he has something profound to share. What I really want to know is what we're getting wrong about AI, what the future of the technology really is, and what we need to do as a society to get the future we want. Let's find out.

• 1:18Geoff Nielson?

  Well, Yasha, thanks so much for being here today. Really excited to have you. You know, I don't know how much you know about our podcast here, but in general, this is a podcast about the future of technology and the future of work. And, you know, I want to talk about AI. I want to talk about AGI and future tech, but, you know, I want to talk about how we can design advanced technologies and intelligences. I want to talk about, you know, organizational systems and how we can build them better in the abstract sense to actually execute on specific goals. But maybe just before we do, I want to back into that with a little bit of conversation around, you know, defining intelligence, defining consciousness as you

• 1:58Geoff Nielson?

  see it, and how that can inform us. So maybe, maybe to start things off, how do you define intelligence in your world and in your studies? And what are the implications that that has for non-biological systems?

• 2:15Speaker 00

  I see intelligence is the ability to make models usually in the service of control. So it's not simply the ability to deploy a skill, but it is the ability to acquire a new skill, typically in an area out of distribution, which means an area that you don't know, that you haven't seen in your previous observations. And when we look at intelligence in humans, we basically measure the ability to solve puzzles. And we do this by finding a set of puzzles that are giving somewhat repeatable results. We have different domains of these puzzles and we notice that people, when they're doing good at one type of puzzle, they also tend to be doing good at other types of puzzles. And we call this general factor G.

• 3:00Speaker 00

  And G-loaded tasks are those that you basically have difficulty to get much better at just with training. There is some effect of training, but it seems to relate it to the ability to solve puzzles. And we typically measure this by taking a population of individuals and then taking the average. And then we see how far a particular individual deviates from that average of the total population. This is how we calculate IQ. But the intelligence quotient. But this is not something that we can directly translate to machines, because the way in which computers are solving problems are very different. And as a result, they are wildly out of distribution,

• 3:43Speaker 00

  which means they are nowhere near close to human average. And so basically, every computer system is an outlier. The other thing is that we typically don't measure in humans the skill acquisition. So how do they get better? How do they learn? How do they solve problems? But we measure how good they are at solving a given problem, which is makes sense for human beings, because most of us don't memorize the skills. But for the computer, it doesn't make a lot of sense, because they're likely see most of these problems in the training data or somebody has trained them specifically to do this particular type of test very well, but they are not going to be very good at other tasks that in human beings would predict a similar performance.

• 4:27Speaker 00

  And so we cannot directly compare intelligence across humans and machines, because it's a quite different thing. And so when we think about intelligence in machines, we typically compare this with the expert performance in a given problem field, for instance, in solving mathematical theorems or in programming or in simple manipulation tasks. And then we have benchmark problems that we can use to compare different AI systems with each other. Intelligence is not the same thing as consciousness, of course. Human beings, in order to become intelligent, have to be conscious. And we observe this in babies, that they basically, they wake up,

• 5:11Speaker 00

  there is a distinct difference between them being alert and attending to the world and reflecting and making models of themselves and experiencing things, reacting directly to stimuli that they are getting and so on, being someone. And if we don't get into that state, we don't turn into intelligent minds, we remain vegetative. So I suspect that consciousness is something like a biological training algorithm. Instead of a transformer, as our present AI systems or another machine learning program, we have consciousness, which is probably a training algorithm for self-organizing system, something that is merging at the beginning and turns it as into intelligent beings.

• 5:52Speaker 00

  And it's not clear if AI systems at the moment have something that is similar. They probably don't have the same functionality as us. They don't need to train a self-organizing system in the same way. And in order to perform their tasks, they generally don't need to be conscious. When you are producing a string of numbers, for instance, a tax return, you have to be conscious. But the LLM, when it produces a tax return, probably doesn't have to be conscious. Super interesting question is, when you're talking to the LLM and you ask it to simulate an interaction partner, more or less in real time, does this interaction partner, when it's pretending to have conscious states, have something that is akin to a causal equivalent of this eternal structure?

• 6:36Speaker 00

  So at which point does it feel like something to be like an LLM? And that is, at the moment, in the eyes of most philosophers, an interesting but open problem. So people have strong opinions about this, but I haven't read a decisive argument yet.

• 6:52Geoff Nielson?

  And I'm really curious about that LLM piece because one of the things that maybe makes LLMs different from us, maybe doesn't, is, you know, basically their method of information retrieval, right? That they're designed based on training data to just give you kind of a plausible answer. And it seems, at least for most models we've seen so far, that they're very limited by what's in their training data. And if you ask them to extrapolate, particularly far beyond their training data, they kind of break down. And so I'm curious, you know, in your model, I guess,

• 7:30Geoff Nielson?

  are LLMs particularly close to what we consider intelligent in a traditional model building way? Or have they just been designed really well to be able to fake it by, you know, ingesting and sharing data?

• 7:46Speaker 00

  Yeah, it's also one of the topics where people have strong opinions, but I would say that we don't know. It's an empirical problem. And this empirical problem gets decided by how research is progressing. A lot of number of large companies are betting very large amounts of money that the LLM can be scaled and tweaked in such a way that it's able to solve problems without hallucination. There's also the question, why does the LLM hallucinate? And the possible simple answer is that if you put human beings into an isolation tank, if you deprive them of sensory data, they also tend to hallucinate after a few hours. And what happens if we take

• 8:26Speaker 00

  the LLM out of the insulation tank in which they currently are? They don't have a connection to the world in real time as we do. And there is no technical reason why that would be impossible. There are some, of course, things that have to be solved, how to make online learning, real time learning, real time behavior working. But in principle, it's been shown that this can be done. And if we have a model that is not just predicting the next token in human written text, but that is predicting the next batch of sensory data from some robotic retina and proprioception and so on, then it can just not be done. Not afford to hallucinate anymore. So the model gets tied closer to an external reality.

• 9:07Speaker 00

  Another problem is that the L L M is currently trained on human text and human text is somewhat arbitrary. There are a lot of novels in there, things that people just made up. And so it's not really tied down to something that requires it to be extremely tight and narrow and reliable in its reasoning. We find that when we train the models more on mathematics and on code, that the rest of the reasoning also becomes more reliable. And so in this, especially this area of programming, when we get the L L M to write computer programs, we have examples where it produces causal structure that often works on the first try. And the more you run it not just against human written text,

• 9:49Speaker 00

  but also directly against the compiler, it gets feedback about what works and what doesn't, and can use this to improve its performance and its reasoning. So there is not a totally obvious reason why the present systems cannot be scaled into the realm of human thinking, human creativity, human problem solving. There are still some arguments that this might be difficult, but we can also see that the learning was very different. The human beings could not learn from this large data. We have to learn from much smaller data and become coherent much earlier. But it's it's not clear what the limits of the approach are. Personally, I'm very interested in our own way of learning and consciousness, because I suspect that we don't optimize for prediction.

• 10:34Speaker 00

  We optimize for coherence. And coherence means that we don't have constraint violations anymore, that everything fits to everything else. And if we have a regime like this, then we probably can build systems or possibly can build systems that are able to learn from much less data.

• 10:54Geoff Nielson?

  If you work in IT, Infotech Research Group is a name you need to know. No matter what your needs are, Infotech has you covered. AI strategy? Covered. Disaster recovery? Covered. Vendor negotiation? Covered. Infotech supports you with the best practice research and a team of analysts standing by, ready to help you tackle your toughest challenges. Check it out at the link below and don't forget to like and subscribe.

• 11:22Geoff Nielson?

  It's interesting hearing your perspective on the transformer architecture and the potential that we have something that can be scaled up with more data. And I think hopefully some thoughtful constraints to help us get something a lot closer to human intelligence. And that's something that I've talked to a number of people about and I've heard some people say, yes, it's the right underlying model. Some people say, no, it's not the right underlying model. But it sounds like in your mind, if we can get it to multimodal really effectively and have it ingest large amounts of data, more akin to what a human does, I guess, where it's taking in, I don't know if it's like visual stimuli, audio stimuli, it's smelling things. But if we can build something

• 12:05Geoff Nielson?

  using this underlying architecture that takes in a lot more about the world and has the compute power, I guess, to build these models based on it. Is that the path forward? Is that going to get us toward what we sometimes call AGI or this kind of advanced artificial intelligence?

• 12:28Speaker 00

  So again, the short answer is at the moment, we don't know. We are talking about something that is at the boundary of human knowledge and experience and technical ability. And so it's something that we are currently figuring out by building these systems, testing them, modifying them until they work better and better. But it's an interesting. Aspect of the so-called scaling hypothesis, which shows that there is a linear improvement if you add exponentially more data and using the present algorithms. And it doesn't really matter so much which algorithm you're using. What they're different is not what they can learn, but how efficiently they can learn it. So there are ways in which you can tweak the algorithms.

• 13:10Speaker 00

  And if you want, then you could say that a transformer ultimately is just a way to tweak the existing neural network training algorithms. And you could have used the existing algorithms. You would just need to train them for longer and wait longer because they're less efficient at detecting the signal in the data. And there are probably ways to detect the signal much better. The reason why we think this is because human beings can learn from far less data. But ultimately, we have a structure that can be discovered in the signal that you're getting and all the observations that the system is making. And there's only so much structure in there.

• 13:51Speaker 00

  And you can beat the structure into the architecture of the resulting model that you're no longer constructing by hand, no longer coding by hand, but that you're growing. And ultimately, all these different algorithms are growing the same kinds of model structure. And so people came up with something that they called the universality hypothesis. That's something that's been discovered by the team of Chris Ola at OpenAI, when they invented the paradigm of mechanistic adaptability, which basically means that you are doing brain scans on the neural networks and figure out how their internal structure works. And what they discovered is that all the different vision models that exist that have been trained with different algorithms using different data

• 14:35Speaker 00

  have pretty much the same internal structure. And that internal structure is also the same structure that you find in the primate visual cortex. And that's a really interesting result. They even found a new type of neuron in the artificial model that couldn't then be found in the biological systems. And this universality hypothesis basically says that the algorithm doesn't really matter so much. That's just a question of efficiency. But what matters is what problem are you trying to learn? And what is the thing that you're optimizing for while you're learning the model?

• 15:13Geoff Nielson?

  One of the, one of the things I wonder about there, and it's really interesting to hear about this sort of analog to the primate visual cortex, is when we're building these, you know, intelligent learning systems, or the frameworks that are going to help us build them, to what degree, I guess, our own intelligence or the way that we perceive things is a limitation? Like, is it, are we going to get to a point where we just realize we've built something that's a lot like ourselves? Can we build models like this that actually, you know, far exceed our own ability to understand them or the way that we process things? And, you know, what are the implications, I guess, for, you know, building these models

• 15:58Geoff Nielson?

  that are, that understand the world either in a fundamentally different way than we do or just in a more advanced way than we do?

• 16:07Speaker 00

  In a way, we already built models that understand the world more deeply than we do. Because our human brain is very shallow, very mushy. We cannot have models that have more than a handful of layers and depths that we can comprehend. When we define an object, it only has a handful of features that characterize the object. Otherwise, we lose track of it. And these limitations of the human brain mean that the puzzles that we can solve are not that complicated. We have been struggling to understand physics for the last 160 years or so, and we don't seem to be making much progress. And it's infuriating because you have this problem with the standard model that needs to be compressed further down.

• 16:52Speaker 00

  There needs to be a regularity behind it, and we are just running against this wall and not making much progress. And so the problems that we are solving are quite limited. But what we can do is we can make experiments. And these experiments can be, for instance, machine learning systems, where we vary only a few things at a time and then learn from this and go deeper and deeper. We can distribute our expertise across human beings and so on. But deeply understanding reality is very hard for us. And at the moment, we are not really building systems that are minds, we are building systems that predict text or that predict structure in visual data and media that are made from human consumption, for human consumption, and they're pretty good at this. And building things that learn from scratch,

• 17:37Speaker 00

  that would learn in the same way as us by improving themselves autonomously over all the domains, that learn in real time, that learn from arbitrary data, that don't learn just from human media, but from arbitrary sensors that we can connect to this, right? True human, true machine perception has never been tried. We have not really built systems that go far beyond what humans can do. That at this point, it would be very surprising if they don't exceed our ability to reason, to make sense of reality, to discover patterns. You notice that, for instance, when we are looking at x-rays, that the AI systems can discover things that human radiologists cannot discover, they're more sensitive to this. And the same way, it's possible to build

• 18:21Speaker 00

  systems that can detect traits of individuals, like emotional states and so on, at some point with higher resolution that we can. Whenever there are signals in the world, and the AI can process more data, more training data than us, and can discover more subtle structure in the world. So there is not an obvious limit.

• 18:42Geoff Nielson?

  The, the discovery piece to me is maybe one of the pieces I'm most excited about for this technology is the ability to, you know, extend the frontiers, I guess, of science and of our understanding of the universe. And, you know, I think what you said sort of makes a lot of sense, that we've been, I don't know if you would say, losing steam at this in the last handful of decades, but it feels like we're not pushing boundaries in the way that, yeah, it feels like we've had a real golden age of, you know, physics and of, you know, natural sciences, you know, 100 or more years ago, and we have lost steam a little bit. And I'm curious, you know, based on where we're at right now, is that because, you know, all the easy stuff has already been

• 19:27Geoff Nielson?

  done? Is that because as, you know, human collectives and institutions, we're just...

• 19:34Geoff Nielson?

  We're unable to span all these different branches at once and we're not able to kind of share knowledge across these due to limitations of, you know, human minds and human society. And, and, or do we just need, I guess, artificial intelligence and some of these, you know, mechanical supports to help us get a little bit farther? So I guess, like, what's your kind of optimism to pessimism level about where the discovery frontier is going to lead us and how much it's just we need to code better tools at this versus there's some sort of structural institutional reform needed?

• 20:13Speaker 00

  I suspect it's a mixture of both. Of course, the low-hanging fruits are done and when you go into scientific fields, what you find that is when they start, there are a lot of the paradigms are discovered early on and the people who discover them become very famous, even if the paradigms later get corrected and much more intricate and complicated. And later on, it's just much, much harder to make incisive progress that changes the way in which people look at reality. This is just the way in which things tend to be. The other one is that there is probably some change in the institutions that if you think about somebody like Einstein, he comes from a very smart, highly educated family and

• 20:56Speaker 00

  they discover early on that this is a gifted child and they give him the best private tutors that they can get hold of. And so sometimes people get told the story that he wasn't super good at school and was lacking off in the patent office. And I don't think that is actually describing what was going on. You had a child that was really given all the opportunities to develop his intellect and to follow the interests that he had.

• 21:28Speaker 00

  So many stars needed to align to make something like this happen. And it's difficult to get these stars to align in different times and different societies, in different circumstances. And today, we have a different society where we have very different priorities for what we are looking for in a person that we led to a scientific career. The other thing is that science has become more postmodernist, that in a sense that

• 21:56Speaker 00

  when you look at founder generations of companies or of states or of organizations, this is typically about survival. If you make mistakes during the first years of Google, then Google will not be a successful company. And so everything is about dealing with ground truth. And once a thing becomes too big to fail, the incentives of governing change because suddenly you are CEO of an organization that is like a little society of its own, like a little nation state, and it is not going to break down anytime soon because it has so many products and so many quasi-monopolies and so on that make it safe. And now the job that you have as its leader is to make sure that nobody is top like you

• 22:37Speaker 00

  from within the organization and that you come across as a good leader to your employees and to the general public. So much of it is much more about the story that you're going to tell and all the different political sensibilities and interests that you have to satisfy in order to stay in power and to maintain your alliances within the system. And so once reality is no longer what you deal with at the ground truth, at the level of atoms in the outside world, but it's about the story that you need to tell, everything becomes more postmodernist. Everything is about choices what you get away with. And this is a trend that happens in all larger systems that we observe in the social domain. And I think it also happens in the university.

• 23:21Speaker 00

  So in a way, to me, university looks a little bit like AI model that is trained on its own output. So since the seventies, it becomes more and more slop. And it's no longer trained against the outside reality. And we've seen this in the last decades that large parts of science have stopped to look for truth and is no longer looking for truth in the same way. But there are many positions that are political positions, ideological positions that you're not allowed to take, talk back to. And this also has extended in the STEM fields. And so that makes it harder to make a little bit of progress. But it's very difficult for me to, to weigh to which degree it's the capture of the universities

• 24:02Speaker 00

  by ideology and employment programs and to which degree it's actually that it's so hard to make progress. There's also classification of paradigms is important.

• 24:14Geoff Nielson?

  Right. And we're dealing with, you know, a series of interrelated, complex systems here where we have, as you said, increasingly large companies that are now oriented more around saying things to raise investments and, and create a sense of stability and, you know, play the game and avoid, you know, you know, crush competition or avoid political competitors from within or from the outside. We've got researched institutions that have become ideological versus looking for science. We've got government institutions, which, and frankly, one of the, one of the challenges across all of these, to me, is we seem to be moving to less and less trust overall

• 24:56Geoff Nielson?

  of our institutions and the implications that has for, you know, running them effectively and actually, you know, supporting human progress, if I, if I can call it that. So is this, I mean, all of this, all of these changes, most of which aren't for the better, are those just sort of, in your view, Yosha, like a natural outcome of these systems? I don't know, modernizing or getting bigger? Like, is this a trajectory that, you know, we can make some tweaks to or will be self-correcting? Or do you see this as just, I don't know, like, late stage of civilization and things

• 25:35Geoff Nielson?

  are getting too unwieldy and we have to either find ways to operate within them and make things that work or just, you know, defer to, you know, non-human entities to help us get out of it?

• 25:50Speaker 00

  At some level, it looks like there are few adults left. And the organizations have become some poisonous and captured. Both the political parties are increasingly populist. And this populism means that they are telling low information voters things that the voters want to hear and try to get to power in this way, regardless of what the consequences are with respect to the policies that are happening on the ground. That is an easy story to tell. And I suspect that everybody who is in this field knows these ideas and theories. The more I understand about reality, the older I get, the more reluctant I am about these positions. Because you cannot prove them.

• 26:35Speaker 00

  There is no control group. And there is still enormous amounts of brain power and competence. It's just elsewhere than in the public administrations. And for instance, most of the VCs have extremely smart people who analyze the technologies very deeply and are not really impressed by hype. But they are impressed by the markets. They are impressed by the projected developments that are happening, which are to a large degree political or happen because people get impressed by things and so on. And a lot of the developments that are happening are the result of emergent dynamics. And there are people who model these emergent dynamics rather than being able to control them. And there's also another perspective

• 27:19Speaker 00

  where we have this impression that the world is in very bad shape. Objectively, we are doing relatively well. We have very few militant conflicts, despite having news about the ones that exist at extremely high and unprecedented detail, which captures our attention. But we don't have riots in the streets. We don't have a revolutionary situation right now. We are relatively well fed and everything seems to be working, despite us having bad news about everything. There are some trends which are super worrying. One of them that worries me is the trust in public institutions. We can find that since the 1950s, when we did polling about what people think about the government,

• 28:02Speaker 00

  there's an almost completely linear decrease in the trust in government in the US. With some deviations after nine eleven, it went up during the recession, it goes down. But if the trend continues, then by the mid twenty thirties or so, we will have completely negative trust in the government. And maybe this means that democracy ends, which could be very concerning. Because we don't know what comes afterwards. And we see the alternatives to democracy in China or Russia and so on. And it's not certain that this is a better world to live in. So yes, a lot of things to worry about. And it could be that AI is changing the trends, that getting more information to everybody, giving universal basic intelligence to everyone could change the equation.

• 28:47Speaker 00

  Maybe it drives us away from populism, maybe it allows us to make more detailed models of reality.

• 28:55Geoff Nielson?

  Do you have a sense, you know, first of all, one of the things I like, Yosh, is that in there, and you know, I've heard you say this in other conversations, you actually often have a much less bleak view of the future than a lot of people I speak with. And you know, tend to take a more historical approach where you say, sure, there's lots of things that are not great now. But if you, you know, open the aperture and you look at how they are versus historically, it's not nearly as bad as it seems. And you know, you've talked about social media and how, you know, is it perfect? No, but you know, when you look at traditional media and the messages that are being, you know, perpetuated through that, suddenly it doesn't look as bad.

• 29:38Geoff Nielson?

  If you look at the organizations and companies controlling AI, in some ways, it's actually more positive that there are a number of them and not just one. And so I, first of all, I really appreciate that. And I'm curious to get a little bit more perspective, I guess.

• 29:56Geoff Nielson?

  You know, in terms of your view about how, how bad things are right now versus how bad the average, you know, media consumer believes that they are. And then The other thing that's been on my mind, you know, and I've heard you talk a little bit about is that it feels like right now we're at a period in history where the future feels a lot less predictable than maybe it did 10 or 20 years ago. You know, which is sort of a silly construct because obviously even if we thought we could predict it, it went in a, it went in a different direction. But if you have a sense of the way things are moving right now, you know, what they're sort of converging towards, or is there just far too much sort of fog of war to be able to answer that?

• 30:40Speaker 00

  Well, one thing that has changed is that ideas seem to be disseminating much faster than they did in the past. And this leads to a more rapid development because ideas are begotten by earlier ideas. And so if the turnaround cycle of ideas is increasing, the memes are coming at a higher frequency, then the future is changing faster and things are implemented more quickly. Ideas are spread more urgently and they go into many more years. One thing that I find interesting is the extremely negative view that people have taken on artificial intelligence. And I suspect that's largely due to two factors.

• 31:20Speaker 00

  One is that large media organization, starting from the New York Times, decided that they see artificial intelligence as an economic and social threat to themselves. They have learned from the internet that the internet has taken a lot of the business away from media in terms of advertising. And another thing is that they've changed the business model of media because you get news for free on the internet. And so media organization are no more as doomed to serve identity to people to serve membership in a particular social group and you are updating the group memes to the media organization. That is a change that definitely has happened.

• 32:03Speaker 00

  People are not just reading the news media to get news. They are reading them to understand how their in-group expects them to relate to the news. And this has become their new role. And as a result, when the news media are publishing, they are consciously publishing memes that are changing how people relate to reality. And once Sulzberger decided that AI is a threat to their business model, the frequency of negative reporting increased dramatically.

• 32:34Speaker 00

  Another thing that contributed to this negative perception is that there is a group of people that believe that artificial intelligence is destined to murder every one of us in the same way as we have diminished the habitats for large apes. We are a large ape that is breeding its own successor species and its successor is probably going to get rid of us because it's competing for the space and resources with us and we are good to do this because it's smarter than us. And this is an argument that when you are a person that is extremely logical and rational seems to make sense and the ways in which this doesn't make sense either have relate

• 33:18Speaker 00

  to your gut feeling or of an understanding of complex systems where you realize that the story is probably way, way more complicated than this. But these people have organized, they have convinced people like Jan Tallin, who is an extremely kind and smart person, one of the co-founders of Skype, who started to fund an organization together with a MRT professor Max Tegmark, the Future of Life Institute. And then due to some error of Vitaly Putin, they ended up with having 600 million dollars of funding that they could use for lobbying and influenced, for instance, how lobbying worked in the European Union against AI and they are obstructionist, who basically believe

• 34:02Speaker 00

  that everything that leads people to make AI worse, that currently talking to Bernie Sanders and pushing him to make moratoriums on data centers and so on is good because it delays the moment when humanity gets murdered by the AIs. And so these two factors are coming together at the moment. But on the ground, I think AI is a lot safer and more equitable than I thought it would be. It seems that the systems that we are building are in many ways idiot servants. There are things that are very good at doing busy work, they're very good at writing boilerplate code, they're very good at summarizing texts for you or looking up things for you, even though they make mistakes in there, it's what they can do for you is amazing if you

• 34:47Speaker 00

  treat them as extremely smart interns. And the development is somewhat predictable, we can see how it slowly gets better and we can react to this. And at the same time, these models are extremely equitable. Everybody is able to use these models for twenty dollars a month. And they are very close to the level in which the industry is using them. And it's a result of basically these models being subsidized for the general public. Because while the inference can be done, running of the models can be done relatively cheaply. The training is very expensive. And there is big competition with respect to training and large expenditure. Because we are not yet satisfied with these models. So we need to retrain them, retrain them, retrain them.

• 35:30Speaker 00

  And that is an enormous cost that currently is not being borne by the consumer. And so this thing that everybody has access to these models is amazing. The other thing is that everybody I know with AI companies is really concerned about making these models as safe as possible. And also building them in such a way, tuning them in such a way that the New York Times is happy with their output. And they don't get any credits for this. But the AI is largely very safe and benevolent and is trying to be kind and nice to people. And everybody can use it for very, very little money. It's who would have thought? What normally the timeline that I would have expected is that the first AIs are super expensive. The access is extremely restricted.

• 36:15Speaker 00

  Very few people can afford them. And the companies are mostly hoarding the outputs of these models and the best models for themselves. We even have open source models that allow academia to play and private people to play and make their own meaningful research. That is really an exciting outcome that I did not expect. And it's in many ways accidental. A lot of things had to come together. People did not plan for this. And so I think we are, with respect to present AI deployment, it's difficult to say what the future will bring in a pretty good timeline. The other thing is that people are super worried at the moment about employment effects. And

• 36:56Speaker 00

  I suspect that what happens is that with every other technology, we have a transition, a transition from backbreaking labor that is tedious, time-consuming, hard to do, to things that are closer to what only humans can do. And what we find is that despite us having more and more technology that is removing jobs, right, we always add more jobs on the other side. And that's because as long as there's anything to be done, people can do it. And they can afford to do things that could not, we could not afford to do before. What are the things that we actually want to do, like raise our children, work in education, do art?

• 37:38Speaker 00

  There's so many things that people would like to do and they just cannot afford at the moment because they still have to administer things, they have to collect information and wait to it. And a lot of that work is super mechanical and boring and not satisfying. And in the suits, driving trucks around and so on. I think it's much better if people are freed from these things and can attend to the things that they'd rather do. And how can we allocate resources to these people? Well, it's also a problem that we can solve with AI and we have done this in the past. And I think we will do it in the future. So as long as there is anything worth doing on this planet, there's always going to be jobs. It's not like the amount of work on this planet is finite, the amount of jobs is finite.

• 38:22Speaker 00

  Instead, there's always things to do. And we will always be able to allocate food and services to people. And these food, services, things, goods that we're producing become cheaper because we produce them more efficiently than before. There are still aspects like ecological impacts of things. At some point, there's going to be resource exhaustion. At some point, we will have difficulty to stabilize our societies because we lose track of what the meaning of our society is and how to bind different groups, different cultures, different subcultures together into a cohesive form. We just don't yet know how to solve all these things. But maybe AI can help us with this. And so these developments, I think, are happening. There is going to be a shift in employment and the things that we are doing.

• 39:07Speaker 00

  But it's probably going to be much slower than a lot of people in the AI industry expect. But it's also more radical than a lot of people imagine at the moment.

• 39:16Geoff Nielson?

  I really appreciate your optimism around that, especially in light of, as you said, so much of the media, whether it's traditional or new media, pushing these much more alarmist and outrage-oriented views that AI is going to murder us or take our jobs or ruin our lives or our society or the environment in some way. One of the pieces I wanted to push you on a little bit is you talk about this fantastic tool and it being democratized and all the good it can do. How do you square that with it being, at least right now, so much of the value and, I guess, financial capture

• 40:01Geoff Nielson?

  and the ownership sitting with a handful of big tech organizations who, to your earlier point, are motivated financially. They're motivated politically and in terms of storytelling and being able to create these hype machines that say, oh, yes, this technology can do anything and it will be able to do even more as long as you plow a trillion dollars into funding the infrastructure for it and the learning for it.

• 40:31Geoff Nielson?

  What's the implication there? And how concerned are you by that ownership model? And also, I guess, by the fact that if we have this layer now in our economy that the ownership of it is so centralized versus being decentralized across a much broader worker base?

• 40:53Speaker 00

  When I was young, I was against capitalism because it seemed to be obvious that there are so many things wrong with capitalism. And it took me a long time to think about alternatives and to look at human history to realize that we have been lifted from poverty and squandered largely by capitalism. And the alternatives to capitalism do not obviously work. For instance, at the moment, we have money. And money is, you could say, it's off the devil. It gets people to...

• 41:27Speaker 00

  Um, court it, it gets into game the system, it gets them to be heartless to each other and greedy and so on. And you cannot actually eat money. Money is a, is not a resource, it's a signal, it's a signal for anticipated reward. And in some sense, you could say this uh dopamine that has been recreated at the next level. And this dopamine is stringing us together into a global brain that allows us to make goods and resources fungible across every planet, uh parts of the planet and across every domain. Which is, I think, a big miracle, that everything can be traded, that uh we can exchange goods and services in this way, that we can travel everywhere and uh use money uh to tie eight billion talking monkeys into a global intelligence.

• 42:13Speaker 00

  And that uh other perspective that you could take on it, imagine there was a score that you could give people that uh tells us in a decentralized way, instead of having some kind of centralized government committee. Uh what, uh whether what the other person does is valuable to me. And you can distribute the score around in this decentralized way and uh there's going to be many things that are going to be broken with it, but we are going to create organizations that fix the most egregious things that are going wrong with it whenever there are bugs that make the system unstable. And uh we try to make it as good as we can. And maybe we come up with an alternative at some point, but this is what money is at the moment, it's this decentralized score.

• 42:58Speaker 00

  And this is also set up in such a way that the people that are better at allocating resources get uh more resources to allo- resources to allocate. So for instance, if you look at somebody like Elon Musk, who has uh gotten extremely good at allocating resources. He is one of the few people who saw that uh NASA has become too inefficient to build new rockets. And if we were to build a company from scratch that builds new rockets, this could actually work. And everybody told him, you're crazy. And he put all the uh money that he could get in into this uh idea and uh it paid off. It didn't pay off only once, but six times in different companies. And he is not using this money for consumption, he lives basically in this little box.

• 43:42Speaker 00

  He doesn't have paralysis or um.

• 43:46Speaker 00

  He does not live like a king in, on the back of his peasants in medieval times. Instead, all his money is invested in his ventures. It's basically the controlling portion of the companies that he is running. And the system has basically implicitly decided that because he is so good at coming up with ideas that should be done and prioritizing them. And despite all the mistakes that he makes, he seems to be very good at error correcting them, but his bets have largely paid off. If he had given this money to a committee of people that were appointed to allocate it, then these things were not built.

• 44:29Speaker 00

  And we can see the counterfactual in many ways. He is has the only Western car company that is able to build profitable electric cars. And from the perspective of the other car companies, it would have been much better to wait until the mid thirties, until the price of batteries comes down by itself, hopefully. Otherwise, we wouldn't have electric cars at this point without him. Another thing is that NASA is the Chinese committee that we have in the West. And they are building rockets at 10 times the price of Elon Musk. And it's not because he is skipping on safety and is killing people left and right, but because he actually deploys the technological advances that we have, not to employ more administrators, but to actually get the stuff done.

• 45:15Speaker 00

  And so despite people hating on Elon Musk, because he seems to have the political opinions of a 12-year-old in Fuchan and voices them via being the CEO of an important social media platform, which doesn't seem to be very wise, but is outraging people against him. The reason why he has that much money is not because the system is inherently unjust, but because it has decided that it was a good thing to let him allocate resources. And that's probably not always the case, but there are counterfactuals to this. They were, for instance, when you observe that. As Kobi Dr. recently pointed out, that Bill Gates, the former CEO of Microsoft,

• 45:59Speaker 00

  has made more money after he left Microsoft because he already had so much money. This might be a problem. He has not gotten this money by investing it himself in extremely clever way, but by having other people invested for him. And this might be one of the many problems that we have in capitalism. There's also the question of research use, but capitalism is not incentivized to produce things once, but we want to produce and sell the same thing again and again and again. So most of things that we are building are very brittle. There's also a tendency by capitalist organizations to capture their regulators. And as a result, innovation is breaking down in many areas. For instance, in medicine and many other domains, it's very hard to make new progress

• 46:44Speaker 00

  because it's so hard to get past the regulation. And the only thing that seems to be innovating at the moment appears to be the tech industry and some areas of biotech. And everything else is getting continuously worse. And if we didn't have this AI revolution at the moment, we would be in the middle of a recession. It's also something that we need to bear in mind. And a recession means that we get poorer, that life gets more expensive every year without our incomes increasing.

• 47:12Geoff Nielson?

  It's a really interesting perspective, and I like it. And there was a lot to process in there. Implicitly, I think in there, you're sort of, you're... One of the concerns I've had for a while that I think is fairly common is a frustration with big tech's ability to sort of monopolize resources in the economy right now, that so much of the investment is going to them versus going to traditional sectors. And it sounds like you're sort of dismissing the concern there and saying, well, yeah, of course, it's going to them. They're the only ones who are actually demonstrating any sort of real ability to innovate. And it's actually an indictment of a lot of other organizations that they're not able to capture more of that investment. Is that fair or is that too dramatic?

• 47:57Speaker 00

  Yeah, I think that a big issue where we have high demand and low supply is how At the moment, and the reason why we don't build houses are complicated, but a part of it is a permission process that makes it very difficult to actually get

• 48:14Speaker 00

  make the state allow you to build houses in the way in which people want them to live in. Another part is building codes. A lot of materials are off of the table. A lot of innovation is not possible. Because existing organizations, unions, builders, associations, cement factories and so on, have written themselves into the codes in such a way that it's very hard to have innovation. And to get innovation happening in the construction industry would probably require abolishing a lot of the existing regulations. And this means that you are endangering existing stakeholders that profit basically of our inability to innovate.

• 48:54Speaker 00

  And that is, I think, one of the many issues. Another one is that we cannot build a train anymore or another large piece of infrastructure. A lot of the inner cities that we are seeing at the moment, including skyscrapers, subways and so on, would be illegal to build today using existing regulations. And if you look at the subway in New York, it's completely medieval. It's a stuffy, it's extremely uncomfortable to use. Compare this to the Chinese subways, that's a future that we are unlikely going to get, right? And I think that is one of the main issues that we have built a society in some sense implicitly chose to set it up in such a way that people get paid for not working, for not innovating.

• 49:40Speaker 00

  And that they're blocking the ways in which things could be improved. And the reason why this hasn't happened in tech yet is because tech has been moving too quickly to be regulated. There have been pushes for allowing only professional engineers to work in the tech industry, right? You have guilds that certify you before you are allowed to write code and so on. And if, once this is happening, I think that innovation in tech also will become much harder. There have been pushes to have something like an internet FDA that requires you, if you want to have a new social media platform, to go through some multi-year, million-dollar certification process.

• 50:19Speaker 00

  Which would make it impossible for anybody but the existing company is to run a social media empire. And at the same time, the government would look into this and tie it down in such a way that it only works in the ways in which the government wants the social media platform to have to run. There might be benefits to this, but by and large, as long as there is no external oversight and there is no external oversight. There is no external god that looks at our societies and improves them. And we only have these deterioration processes and something like senescence that is setting into all the organs and lets them serve their own interest, grow local fat deposits rather than serving the greater whole

• 51:03Speaker 00

  and make this organism in which we all live fit for the future.

• 51:08Geoff Nielson?

  The the regulation and deregulation piece is really interesting and it's something I want to come back to. But but just before I do, I want to take a slightly more cynical approach, which worries me sometimes, which is that in some ways, the investments we've been able and the returns we've been able to see from big tech have in some ways perverted the global investment environment because they're able to create these things. You know, these these products and these services and these, you know, I guess, you know, vehicles for return where there's extremely scalable, very low resource intensity, you know, very low kind of human involvement at scale.

• 51:50Geoff Nielson?

  And that becomes just a much more attractive investment than, you know, building more trains or, you know, farming or, you know, restaurants or something like that. And so it in in some ways, the availability of that product and service pool starves everything else of investment. So so what do you make of that? Is that just still a failure to innovate of everywhere else? Or is there some sort of? You know, trend that could lead to, um, you know, if you follow the metaphor, like a starvation of these other sectors that does systemically weaken, you know, our economy and our sort of, you know, civilizational structure.

• 52:36Speaker 00

  And one of the problems is that trains don't generate a lot of return because it's too expensive to build them. And that has to do with the way in which we structure permission, land ownership, and many other aspects of building trains. Also, a lot of criteria going into the design of trains that make it impossible for us to deploy trains as they're being used in China because the criteria are different and so on. A lot of stuff has accumulated, our regulations, that just makes it unprofitable. It makes the numbers not add up. There is no point in putting resources into something that is not producing a return because money is in some sense a measure of whether the thing

• 53:17Speaker 00

  should be built. And so if we want that thing to be built, we need to remove obstacles in such a way that it makes economic sense to build these things. For restaurants, I don't see a shortage of restaurants. So there is, people are, as long as they're eating out, there will be restaurants. But there is no point in having centralized investments into restaurants. It's much easier to have this decentralized thing where everybody as a customer decides whether this restaurant is still good or whether they don't like it anymore, whether they like this food or don't like this food. And as a result, restaurants are opening and closing in an adaptive way. And this adaptive function of capitalism is super useful for, especially things like restaurants. And for larger

• 54:00Speaker 00

  infrastructure projects, which have a longer horizon, it becomes more difficult and you have to think about how to subsidize them. But in, for instance, in California, the subsidies, you can take arbitrary amounts of money and so there is a problem and the money is just evaporating before it hits the ground. And that is an issue of the dysfunction of public administrations in the US that nobody seems to have a solution for right now. And with respect to the money that is going into the AI companies, it's very difficult to say for me as a non-economist, what this actually implies. We possibly do have a bubble in the sense that eventually the AI is going to pay off,

• 54:43Speaker 00

  but not as early as all the investors and hyperscalers are hoping. And so it could be that we are overbuilding capacity in the same way as we did this during the early days of the internet. Do you remember the IT bubble? People were hoping that the online retailers would produce enormous returns and they eventually did and put a lot of brick and mortar stores out of business. But as a result, we got better goods and services. Again, Amazon is a better way to get a lot of stuff into everybody's home than the stuff that existed before. And economies of scale has been very useful and has saved us during the pandemic.

• 55:24Speaker 00

  A similar thing might be happening with AI that as we built too many internet connections back then and as a result, a lot of these investments had to be written down and the stock market collapsed for a while. A similar thing could be happening right now. But we need these data centers in the long run because in the long run, there are so many things that we want to compute that we cannot afford to compute yet. And we want to have video in-house, we want to have multi-model models, we want to have models that are able to listen to us in real time and talk back to us in real time in ways that they just can't at the moment. And so having more data centers is probably a good thing for us.

• 56:05Speaker 00

  And if this means that some of the early investors have to write down some of their investments, maybe that's a good thing.

• 56:16Geoff Nielson?

  And you know, that's so interesting to me. Again, you know, there's so many people who will talk about an AI bubble and say, oh, it's terrible, it's the end of the world and, oh, and you know, it's going to destroy the economy. And it sounds like your positioning is it's actually a relatively benign solution for all the capital that's circulating.

• 56:33Speaker 00

  I suspect that if most of this money disappears, so to speak, it would not hit most of the rest of the economy in an extremely harsh way. But I don't know that. There is another aspect that is important. We have a bubble in some sense in our entire money supply. And it could be that we have to reboot our financial system at some point and nobody seems to be knowing how this is working and a few people that are working on this, of course, not talking about it publicly. So I have no idea how far these plans are and what's going to happen. But it could be that people realize that the present monetary system needs a new contract at some point, that people have something that comes after

• 57:16Speaker 00

  the dollar. In the meantime, we are fixing it on all ends and we can still buy pizza with it.

• 57:23Geoff Nielson?

  And can you expand on that a little bit? What, when you think about rebooting the financial system, what does that mean to you?

• 57:30Speaker 00

  So one thing that is concerning to me a little bit is when you look at the history of other civilizations that were forced to debase their currencies more and more because the state, for instance, in the Roman Empire, didn't know how to make ends meet. That has been a large factor in their demise. Basically, the states going bankrupt, which means that the economic system that is required, the economic machine to run the administration, is no longer able to produce enough goods and resources. And ultimately, this is what gets you when you have more and more people working in unproductive administrative jobs, relatively unproductive, they're mostly employed for redistributing

• 58:13Speaker 00

  money, not for producing goods and services. That is an issue, right? And if you, I mentioned the permission system in California, this is not an issue that can be solved by throwing more money on it. In part, it's inverse. We have more people working in the permission system for housing than anywhere else in the country and perhaps the world. And all these people need to get paid, which is the reason why this is so expensive to get a permission to build a new home on an existing, on the place where an existing home already stands, and why this takes so long. And by such a large fraction of the whole cost of buildings, anything here. And this is a burden that happens on the total economy that more, the longer this thing runs,

• 58:59Speaker 00

  the fewer people are actually building things and are involved into the creation of goods and services. And the more people get fed just on top of the existing economy, picking back on it. And ultimately, this is probably a thing that needs to be changed. We have tried to change this at the moment by opening the borders and letting in a lot of undocumented immigrants. And I hope that they are the ones that are going to produce things for us. So we can keep administering things and pushing money around instead of having to work on the ground in the fields or in the hospitals or in nursing homes and so on. And it's not clear if this calculus is ultimately going to go out. But we also have to realize money is not something that we have to mine out of the ground. It is in finite supply. You can just create it and delete it.

• 59:45Speaker 00

  If you figure out how to do this and how to solve all these principal actor problems that the people who are involved in it is also have a personal interest in this. I think that's really concerning about me is in Canary and the coal mine is cryptocurrency. And I think that there is no real world use for cryptocurrency, except that allows private individuals to do things that normally would require to have some kind of banking license. And there's a reason why we require licenses for this, because there are licenses to game the system. And so we keep the people who are in a situation where they can game the monetary system deliberately very small. It's both in the interest of Wall Street and in the interest of the general public that are too few people are playing the Wall Street games.

• 1:00:31Speaker 00

  And basically crypto is able to outrun regulation due to the way in which the blockchain is defined. It allows new form of assets that otherwise would be illegal for good reason. And I thought this loophole is going to close very soon. But crypto industry managed to give some crypto to the regulators and it's still around. And the fact that it is still around is really, really concerning to me. Because if the system, the financial system has an interest in survival, it should clamp down on alternatives to the financial system. And I don't think that crypto is a reasonable alternative to the financial system because its money supply cannot be centrally regulated. And I think what we have learned over the last few hundred years is that you need ways

• 1:01:13Speaker 00

  to regulate the money supply. You need to be able to take money out at the top and put it in at the bottom to keep the whole system stable and in flux and make it work as this regulatory system for the economy. And if cryptocurrency is created by the people who later own a large portion of it, right, the consortium behind the most of the coins are heavily financially invested in the outcome of those coins. This is not an alternative to to money. And so by crypto itself is not super dangerous because it's a tiny fraction of the larger system. The existence of crypto is a bad sign.

• 1:01:49Geoff Nielson?

  Right. And my my feelings on crypto align pretty closely with yours. And it's it's funny, because in crypto, the idea that it is like an existential threat to the broader financial system, like that's almost a marketing point of crypto, like the people propagating crypto want buyers to believe that, oh, you have to get in on this. And, you know, it's it's going to be the future. It's going to replace everything. But it's so, I mean, what's so fascinating to me about crypto is there's been this narrative for so long that, oh, if only there was less regulation and only if only there was the ability for wider adoption, it'll change the world. And it feels like over the last few years, a

• 1:02:31Geoff Nielson?

  lot of that regulation has actually gotten more lenient. And that narrative is harder to sell now because it's it's less restrained than ever, and it still hasn't changed the world. And I think to your point, it just feels like it's sort of existing for its own sake to be able to make wealth for the people holding it versus having any, you know, deeper value for, you know, society or for broader use outside of it being deregulated.

• 1:03:00Speaker 00

  Yeah, it's basically a way to print stocks without having assets on the other side of the stock. And it's pretends to be an operating system for the thing that comes after the present financial system. But I think it's not, it's cannot be demonstrated to work in such a way, but it does demonstrate the vulnerability of the present operating system.

• 1:03:22Geoff Nielson?

  Right. I want to come back to a conversation, Yash, that we were having a little bit earlier about innovation, about, you know, systems and organizations and, you know, the best way to actually create things of value. And again, I agree with you that it feels like a lot of jobs, a lot of what we're creating, is sort of, you know, self-perpetuating and it's, it's getting farther and farther from any obvious creation of value or of, you know, innovation or goods and services for people. And it just exists because it happens

• 1:04:00Geoff Nielson?

  to exist or people expect it to. And so I wanted to talk about organizations and I guess the effectiveness of organizations. And I know this is a little bit outside of what you often speak about, but when we're thinking about organizations and them as sort of, you know, I guess, an abstract intelligence or a system that can be more or less effective at achieving its goals, how do we make organizations better? How do we get them to allocate resources more effectively to create things people actually want? And how do we, I guess, how do we hold them more accountable internally versus, you know, just having bloat or politics kind of run them into the ground?

• 1:04:47Speaker 00

  I suspect it's more the other way around. Under which conditions do organizations survive that have a net negative output for society? And I think it happens when they can exploit rules that would normally not be in place, that gives them a monopoly or that gives them power. And that power itself can be used in an extractive way, can be used as a tool to extort the rest of society. And a similar thing is happening in the same organization behind the billionaire tax in California. There is this idea that

• 1:05:27Speaker 00

  by a one-time tax in, of the wealth of billionaires, including unrealized gains of people who only have stocks that cannot actually be turned into money, these people would have to take out a loan or go bankrupt. They cannot get a loan for this. They want to take this money to offset some of the healthcare cuts of the Trump administration. And this, the problem is this thing is that because the tax needs to be done retroactively, otherwise the billionaires would of course leave California, a lot of California billionaires have left the state preemptively. And this leads to a situation where the tax revenue of California is dramatically shrinking

• 1:06:12Speaker 00

  by more than the tax was projected to generate an income for the state. And this also has a detrimental effect of on Silicon Valley. A lot of the investment pipeline is not going to function as well if the people who are administering capital here are no longer living here and instead are living in Austin or in Miami. And another problem is that it's very difficult to convince the founder to incorporate in California and to live in California if they are planning to build a unicorn. And during building a billion dollar company, they might have so much stock that they can no longer hold the stock because they need to sell it. And while the company is not ready yet, they cannot sell the stock, of course.

• 1:06:56Speaker 00

  So a design like this is somewhat obviously wrong and broken. And you can argue to the people responsible for the design that it's broken. But of course, they understand that they're not stupid. They understand these arguments. And the perverse situation is that if you are a politician who is, has decided that the next election is going to be between an extreme MAGA candidate and extreme socialist candidate, and they're not going to the MAGA candidate, but the extreme socialist candidate, they're just accepting the fact that the situation is going to be worse for Californians or for innovation in the tech industry, even exchange they get to power. And it's, it's has something to do with the conflict of incentives that exist.

• 1:07:41Speaker 00

  And for this union that is supporting this legislation and cooked it up in the first place, they, I think, just hope to get paid. They hope that somebody is, the tech industry is saying, oh, this is so bad for the tech industry and for California, that we are going to give you money to retract this legislation to make sure that this bad thing doesn't happen. And this is an example, I don't know how representative it is. It's a very Californian example and typical for what happens here locally. I don't know how much this is indicative of what happens in society at large. And how much is this responsible for our inability to innovate or out compete industrialization in China?

• 1:08:25Geoff Nielson?

  So if we try and answer the question, where do we go from here? And how do we correct some of these, you know, structurally bad behaviors? You know, to me, there's a scenario where they become self correcting because we do stupid things. And there's too much regulation and it makes innovation too difficult and all the innovators leave. The funding dries up and it becomes a cycle where then we have to say, oh shit. Well, let's repeal all that because we made a big mistake. Or maybe it's just a survival of the fittest situation. And the capital ends up flowing to lower regulation environments where it can sort of flourish more. Like, is there any way to short circuit that sort of cycle of stupidity?

• 1:09:09Geoff Nielson?

  Or is this just something we have to play out because of how, you know, structurally ingrained it is right now? Across the entire, you know, kind of governance and regulatory environment?

• 1:09:23Speaker 00

  I suspect one big chance that the US could have is that we have very different states that have different local constituencies. And we could just enable more competition between the states by losing some of this central regulation that we have. So by giving individual states, for instance, to have their own FDA. They could fork their FDA's, they could compete on different healthcare systems. And this might allow to get actual innovation in the healthcare sector, which means better medical services at a dramatically lower cost. A similar thing could happen with housing, basically allowing innovation zones

• 1:10:05Speaker 00

  where people can experiment with different types of zoning, where communities decide on their own zoning and building codes and so on. And we accept that innovation can bring risk. It can bring more accidents, it can bring effects that you did not expect. But by and large, when we look at the history of the US, having freedom to innovate has been net positive. And maybe we should bring that back.

• 1:10:33Geoff Nielson?

  One of the challenges I have around that, and I'm curious if this falls into it or not, is, you know, I've heard you tell stories before about, you know, your time living in Berlin. And Berlin, in some ways, acting as a hotbed of innovation. Because, you know, for a few of the last previous decades, there just hasn't been a lot of regulation and there haven't been a lot of, you know, kind of governmental structures or economic structures. And it just has created this fertile environment where you can have that sort of true innovation and survival of the fittest. And by the way, I've spent a few months in Berlin and I love Berlin and I love that about Berlin.

• 1:11:15Geoff Nielson?

  And the reason I bring that up is it seems to stem from an absence versus a presence. The creation of, oh, we're going to make this an innovation zone. We're going to create some sort of rules or some sort of structure. And I'm curious, in your mind, you know, how do you square that? Are we doing more harm than good with any sort of intervention here? And, you know, is the problem just that we're doing too much and the too much, you know, works against us?

• 1:11:47Speaker 00

  I think that Berlin is difficult to translate into anything in the US. There was a very unique situation that we had this socialist state that was implemented on Germans, which I think is a good indication that socialism doesn't work. Because if you take a well-educated, extremely rule-following population and then implement your system on them, then it doesn't work, right? And it's probably not going to work in the US at all. Which is a much more heterogeneous society in which people cheat a lot more. And the issue was, for instance, that we were not just inefficient at producing consumer goods and at innovating cars and so on. We had cars that were largely designed in the 1950s and 60s and then stopped innovating on them.

• 1:12:31Speaker 00

  We still had bullet holes in a lot of the houses in the inner city of Berlin because we could not afford to patch them up, simply because we were too unproductive. And when the wall came down, we had a system where all the regulations became null and void. And everything was in some sense bankrupt and you could start from scratch. In a city that had suddenly a lot of cheap housing and a lot of young people flowing in and trying new things. So how could you achieve something like in the in the US without having some kind of catastrophic breakdown in the first place? I doubt that this is possible. And then in a way, we do have a breakdown in the political parties.

• 1:13:10Speaker 00

  There was no competent coal administration in Western Germany that would take over and perform investments and finance transfers in exchange for getting all the real estate and building a new society there. Instead, you have the leading bullet artist of the nineteen eighties taking over and there is no alternative in sight. So this is the situation that we're in. And the question is, what would it need to take to get competent governance again? And is it possible to implement it against all the different forces that are interested in governments not being competent? So they can do their own thing in this in this larger system.

• 1:13:50Speaker 00

  And I suspect that we have still part of our social contract is we don't want our government to be very competent, which is why we pay politicians much less than we pay to Google managers. It's a situation that is very different, say, in Singapore, where the government officials are extremely well paid and their pay is tied to performance metrics of the economy of the state itself. And they're really looking for talent for for these positions. But also, of course, Singapore is not this country of the size of the US. It's a mall and this is administered more like a larger mall. And it's not clear if a country the size of the US can be organized in this way.

• 1:14:34Speaker 00

  So I have mostly questions, I don't have answers. I just don't see how the situation in Berlin could easily be translated. There are some similarities maybe in Detroit at some point. We had extremely low housing costs, but also part of the appeal was that of Berlin, that was destined to succeed because it was designated the new seat of government. It was moved from a small town in western Germany back into Berlin. And so no matter what happened, you always knew the future would be glorious. And it's not as obvious as it is in a former industrialized city in the US like Detroit. To think at some point it's going to rise from the ashes and you should invest now. And every young artist and builder and maker should move to Detroit to turn this into

• 1:15:19Speaker 00

  a glorious place for the future. But who knows, maybe things like this are going to happen. Maybe we will have places like this where people realize, oh, the state has gone bankrupt and the new government of that state has decided to turn the best, make the best of it and to really loosen regulation, to push back against the federal administration to allow meaningful competition to other states.

• 1:15:41Geoff Nielson?

  Right. So I mean, we've ended up down a deep path of structural issues, regulation, you know, all these kind of systemic challenges to innovation and, you know, what it may or may not look like if we're going to improve some of these. But in most of that, we haven't really talked too much about AI and the role that AI is going to play. And so I'm curious, as you, you know, think through how the technology is evolving and how it may impact our societies, do you have a few sort of scenarios that you've mapped

• 1:16:22Geoff Nielson?

  out either what it may look like if we're able to actually, you know, harness this to make things better versus if, you know, it ends up being a technology that, you know, deeper ingrains existing powers and, you know, the existing structures that concentrate that power?

• 1:16:41Speaker 00

  I think a lot of it depends on whether AI is going to be centralized or decentralized. We already have a situation in states like Russia, where AI is deployed in a centralized way, for instance, for surveillance purposes. And it's so ubiquitous that there are cameras in the subway that can predict for every single person what the probability is that they're intending to go to some kind of protest. And the police can preemptively dissent on them and arrest them before there is any kind of protest. And China has similar technologies, and US so far doesn't, which I believe in many ways to be a very good thing, and I hope it doesn't come to that.

• 1:17:20Speaker 00

  But if the rest of society becomes more under stress and the government feels under threat, it's more likely that there's going to be a push for having more surveillance technologies here that are centralized and come down from the state. There is another thing that is happening from the ground up, where people are reacting to increased crime and so on by Nest cameras that are then built into surveillance networks that locally increase safety for the communities. And there is a push of leftist activists against the surveillance because they feel that it's the state surveillance that comes from top down and the citizen surveillance

• 1:18:02Speaker 00

  that comes from the bottom up are pretty much the same thing and lead to similarly bad outcomes. Maybe they do, who knows? It's something that we have to figure out as a society over the long run. But another aspect that's very important to me is I want to have AI that serves me, that makes me more competent and is not serving some existing interest. So, for instance, I want my AI to be able to tell me what kind of medications are effective under which circumstances. And if the AI is preventing me from knowing this, I have no way to check on my doctor. There's also the question, do I actually need my doctor as a gatekeeper for most of the things that I'm doing?

• 1:18:42Speaker 00

  Do, why can I not order medication myself if I have a university degree and understand

• 1:18:50Speaker 00

  roughly how the medication is working and I have no prior history of ruining myself with medication? And in many ways, I suspect it would be better to relax the conditions under which you can get a certification that allows you to prescribe medication or can access to medication and you should have the knowledge to do that. And there is a larger, more more growing movement that you should not make a machine in the image of a mind with a certification that is a lawyer or an architect or a city planner or a doctor who knows things that normally require you to go to university for a few years.

• 1:19:33Speaker 00

  And I believe in many ways, it's desirable that this knowledge gets widely disseminated.

• 1:19:42Geoff Nielson?

  When we look at the state of AI right now in terms of how advanced it is, in terms of making some of these predictions and judgments.

• 1:19:54Geoff Nielson?

  Again, it's so difficult, I think, for most people to be able to parse out how advanced it is because there's using it yourself, there's the hype, there's stories about where it is or where it isn't. Do you get the sense that the reason it's not replacing? Doctors are an interesting one because we're talking about, you know, in some ways you can be talking about life and death if you take the wrong medication. Lawyers seem to be a little bit of an easier one, but if you talk broadly about professions, do you see the capabilities as being there for most of these right now? And this is just, again, sort of regulatory environment capture?

• 1:20:31Speaker 00

  Imagine that we would have a system, we have multiple FDAs competing with each other. And similar to say, instead of having a centralized taxi organization, you also have Lyft and Uber. And they are competing with the existing taxi medallions. And you have regulators that make sure that what's outside of the box is also inside of the box. And you hold people to certain standards that they promise. And that the certifications are working out, but you can relax how you certify things. For instance, you could have the way to fast track medications that are already allowed in other countries. Or you fast track medications if you expect that the harms are relatively low if they're misapplied.

• 1:21:11Speaker 00

  And as a result, you basically create a much larger market in which you have many more healthcare providers than before, which would bring down prices. Thing that is easy to oversee, that part of the reason why healthcare is so expensive is because the associations of the doctors make sure that the number of places in medical school are very limited. So the doctors are not actually competing with each other on price. And is this really the American way to do it? Should we not allow people to provide healthcare services by well-educated professionals in a more competitive environment? And as a result, make healthcare more easily accessible to people? And that probably allow us to live longer and more healthily.

• 1:21:57Speaker 00

  And AI can definitely help us these things, play an important role. We just don't know how to fit AI into the existing regulatory frameworks yet. And another thing that AI can help us is to make administrations transparent. In some sense, it's not just about the IAS using AI to know more what you have been doing and how they can tax you more effectively. But it's also allowing you to look at the IAS and seeing how they are using your tax dollars to administer themselves. And there is a possibility that we get a future in which this AI technology goes both ways, in which everybody is able to make meaningful contracts with everybody else.

• 1:22:38Speaker 00

  Because the AI is introducing the competence and the ability to crunch all that information for everyone of us. An important thing that could be happening are epistemic communities, where you basically have communities of people who share AI models that make a joint model of reality. And that joint model of reality is not administered by the state and is forcing indoctrination down your throat. But it's something where the, where people themselves say, this is the AI that is acting in my best interest, better than others. And we are building it together and it allows us to model reality more effectively. It allows us to predict how the financial system is going to evolve, how the organizations are going to evolve and also to organize ourselves.

• 1:23:22Speaker 00

  And this might eventually lead to a world in which we have successor models to capitalism, to money, to the existing representative democracy that are more equitable and more effective at administering our future.

• 1:23:36Geoff Nielson?

  I don't know, I think about people and their relationship with a sense of control in their lives and whether we're trying too much to, I don't know, focus on getting a sense of control versus having a more meditative approach or being comfortable in a situation where, you know, we're, we're sort of flowing down this river and there's not going to be this sense of control. I don't know, is, is that a trap that people are facing?

• 1:24:04Speaker 00

  I think that sometimes it's, can be okay to relax and to accept the things that you cannot change. At the same time, you need to identify the things that you can change. And you cannot change them by having strong emotions about them, but you can change them by entering relationships with other people and strengthen their decency to build the civil society by treating people well, by being responsible with each other. I suspect that a better society starts with every one of us. It starts by building good relationships to your friends and to your family, to your children, and then scaling out from this and building a better, a better organization around you,

• 1:24:44Speaker 00

  building a better workplace environments and integrating yourself into the community, into your local city management and so on. As soon as you feel that you have the power and maturity to do so and scale up.

• 1:25:00Geoff Nielson?

  So, you know, as we start winding down this conversation, you know, probably a lot of the people listening to this are technology leaders or business leaders who are, you know, part of some sort of institution or organization that, you know, they, I hope, want to make some sort of difference in. They want to innovate. They want to make things better. They are caught up in these broader environments where they have to deal with perverse incentives and things like that. Do you have any guidance for them in terms of, you know, how they can do the right thing, be more effective, or, I don't know, come away from this conversation a little bit wiser than when they went into it?

• 1:25:45Speaker 00

  Well, often an issue is the filter bubble in which people exist. And in many ways, I am a friend of filter bubbles. I believe that you cannot have a symphony that you're playing when you have a wrestling match on the same stage. And if you want to get excellence in anything, you need to have a certain degree of isolation. You need to get people that are specialized in a particular way and spend a lot of time with them and getting better at things. But it's also important to realize how far the rooms are drifting apart and to learn to move between rooms and to identify people that are clearly smart and well-intentioned, but that end up in a different part of the world and end up with having radically different opinions than you have. And talking to them is really important.

• 1:26:31Speaker 00

  So basically, getting people to contradict you is important. This is how we make progress. And contradicting in good face is the way in which we learn, which we get different aspects of reality integrated into our little bubble and our little projection of the world. Maybe AI can help with this. Maybe we can build better social media tools that can help us to filter out the outrage and get good information in and get a better understanding, a better integration across people. Most people I know are very well-intentioned. Most people strive to be decent, try not to be cheats and liars and so on, and seek out those people and build networks with them and make it very attractive for everybody else to join those networks

• 1:27:16Speaker 00

  and make sure that the networks are resilient.

• 1:27:20Geoff Nielson?

  No, I really like that. Well, and it sounds like part of that is being more comfortable with our ideas being fallible and searching for, you know, competing ideas and trying to synthesize and being comfortable having new opinions and new views versus just continuously reinforcing this, you know, singular view and dismissing anybody who thinks differently.

• 1:27:43Speaker 00

  Yep. I noticed this also in the tech industry. Most of the people I meet in the tech industry are very well intentioned, but many of the more radical leaders don't touch grass that much and think that the way in which we solve our social issues is to create charter cities in which we are only collecting the good risks and all the plumbing is going to be done by robots. Maybe this is not the correct solution. We do need a social contract that allows us to integrate.

• 1:28:10Geoff Nielson?

  Yeah, no, I think that makes sense. On that note, Yosh, I wanted to say a big thank you for joining today. It's been a really interesting, really far-reaching conversation, and I appreciate all your insights.

• 1:28:23Speaker 00

  Yes, sorry, this is not very much an AI conversation today, and I'm not super competent outside of this. So the things that I'm saying, I say this as a private person, as a guy looking at things, don't hold it against me if you disagree with me. And the things that I'm saying is just the way in which things are appearing to me right now, and I hope you can take something useful from it.

• 1:28:45Geoff Nielson?

  I appreciate you saying that, and I feel like, yeah, I intended to have a more AI-focused conversation than we ended up having, but, you know, the conversations kind of go where they are. So no, I appreciate the clarification, and maybe one day we'll have another conversation that's a little bit more AI-oriented. If you work in IT, Infotech Research Group is a name you need to know. No matter what your needs are, Infotech has you covered. AI strategy? Covered. Disaster recovery? Covered. Vendor negotiation? Covered. Infotech supports you with the best-practice research and a team of analysts standing by, ready to help you tackle your toughest challenges. Check it out at the link below, and don't forget to like and subscribe.