Intelligence is collective, not artificial — Prof. Michael I. Jordan (UC Berkeley / Inria)

Michael I. Jordan, described by Science magazine as the most influential computer scientist alive, has never thought of himself as an AI researcher. In this conversation he explains why that distinction matters. SPONSOR: --- Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open. Apply now: https://cyber.fund --- Jordan trained as a statistician and cognitive scientist, and his career has been spent building machine learning systems that work in the real world: supply chains, commerce, healthcare, and large economic systems. When the field rebranded itself as AI and then AGI, he did not follow. Instead he argues that the framing is wrong. AI is better understood as a collective economic system than as a race to build a disembodied superintelligence. We talk about why AGI is mostly a PR term, what machine learning achieved before the LLM hype cycle, and why the assistant-on-your-shoulder vision may be less compelling than it sounds. Jordan explains why explanations need to be actionable, not merely mechanistic; why AlphaFold's missing error bars matter; how prediction-powered inference changes the picture; and why drug discovery is an incentive-design problem rather than a pure pattern-matching problem. ERRATA: Science magazine ranked him the most influential computer scientist, not Nature --- TIMESTAMPS: 00:00:00 Cold open: A demoralizing message to young builders 00:02:04 CyberFund sponsor read 00:02:50 From symbolic AI to machine learning systems 00:05:42 Why AGI is mostly a PR term 00:08:48 A collectivist, economic perspective on AI 00:11:33 Why LLMs need system design, not hype 00:14:50 Predictability beats faux understanding 00:17:55 AlphaFold, bias, and prediction-powered inference 00:21:48 Stop anthropomorphizing intelligence 00:27:44 Drug discovery as an incentive problem 00:32:29 The three-layer data market 00:38:07 Social knowledge, markets, and culture 00:45:39 Creator economics beyond Spotify 00:48:30 How science-fiction AI narratives mislead young builders 00:51:45 AI should improve humans, not replace them 00:56:42 Safety is a property of the whole system 00:58:12 Silicon Valley gurus and the cream off the top 01:00:47 Game theory, mechanism design, and contracts 01:04:39 Conformal prediction, e-values, and anytime inference 01:08:11 A new liberal arts triangle for the AI era 01:11:30 The Bayesian duck and markets as uncertainty reduction ReScript (transcript, PDF, refs etc) - https://app.rescript.info/public/share/fb68f94af29d3745c6cf6125e01328b5 --- REFERENCES: person: [00:02:50] Michael I. Jordan (homepage) https://people.eecs.berkeley.edu/~jordan/ paper: [00:06:01] A Collectivist, Economic Perspective on AI https://arxiv.org/abs/2507.06268 [00:18:09] AlphaFold https://www.nature.com/articles/s41586-021-03819-2 [00:20:36] Prediction-Powered Inference https://arxiv.org/abs/2301.09633 [00:33:47] On Three-Layer Data Markets https://arxiv.org/abs/2402.09697 [01:04:39] Conformal Prediction with Conditional Guarantees https://arxiv.org/abs/2107.07511 [01:04:51] A Tutorial on Conformal Prediction https://www.jmlr.org/papers/v9/shafer08a.html [01:06:00] E-Values Expand the Scope of Conformal Prediction https://arxiv.org/abs/2503.13050 [01:08:23] Computational Thinking https://www.cs.cmu.edu/~CompThink/papers/Wing06.pdf other: [00:28:20] How Should the FDA Test? https://rdi.berkeley.edu/events/sbc-assets/pdfs/Summit%20session%20speaker%20slides%20submission%20form-s1-5%20%28File%20responses%29/Slides%20in%20PDF%20%28Please%20name%20the%20submitted%20file%20as%20_firstname_-_lastname_-slides.pdf%29.%20%28File%20responses%29/27-Michael%20Jordan-Session%20V.pdf#page=15 [00:28:40] Michael I. Jordan Session V Slides

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 0:31Speaker 01

  Low energy, unfocused, foggy? You might be dehydrated. Whether it's hot yoga, over 100 degree weather, or too much sun. Gator Light, with a specialized blend of five electrolytes, was scientifically designed to help move fluids into the body faster for rapid rehydration. Shop now at retailers nationwide. Gator Light: Hydrates Faster Than Water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 0:59Tim Scarfe?

  Major said that you were the most influential computer scientist.

• 1:03Speaker 03

  It exists in the real world. This is an abstraction, but it's a real thing. It's like F equals MA. It's a set of, it'll make predictions. So if I write down a game, just like I wrote down F equals MA in some coordinate system, I can now predict what'll happen. I don't think we need to. See, I think this anthropomorphizing of intelligence and understanding and all that is not necessary, not appropriate, and is a distraction for many, many problems. Why say it understands? I think it's science fiction, and I think science fiction's important for society, but it's also, at the level it's being promoted and those kind of voices, it's really hurting 25 and 20-year-olds. You know, these young folks of whom there are huge numbers are excited about technology,

• 1:46Speaker 03

  and they want to build things that help their family and help their country, actually more their family than their country, honestly. And they see real opportunities in doing that. And they're kind of being told by the leaders, "Well, we had our fun. We developed a bunch of algorithms. We did it, and we were just..." Interested in the pure, you know, understanding intelligence, even though they didn't understand intelligence, they built, you know, gradient descent algorithms. And now you guys, you can't do this because it's dangerous. It's gonna, it's gonna wipe out humanity with a, with a high probability. Or it's super intelligent will arrive soon, so there's nothing left to do. That's in your lifetime. That is so demoralizing, so demoralizing. And that, that thing, I think that bothers me the most. I mean, the second part that bothers me is there's no economic thinking going on there.

• 2:29Speaker 03

  So the current generation is just way too, you know, there's not much thought going on, not much intellectual stuff. Uh, it's just, yeah, it's possible to build it. It's possible to steal the data from wherever you want to, because that's what the internet allowed to happen and not return any value to the person who originated the data. It's possible to run gradient descent on that, but you need huge amounts of money, but it's now possible to get it from people who aren't thinking very deeply. I, I don't think it's bad to build systems you don't understand, but I think this level of detachment from reality is unusual for human history.

• 3:03Tim Scarfe?

  This episode is supported by Cyber Fund. If you're building at the frontier of AI, they want to hear from you. Cyber Fund believes the future belongs to AI natives who want to achieve the impossible. And that is why they're introducing the monastery for AI native founders. It's an environment of pure focus and rapid execution for founders operating at AI native speed. And they're offering teams $2 million each to participate. Apply now at cyber.fund. And what do you think about the term AGI, by the way?

• 3:44Speaker 03

  Uh, AGI to me is just a bit, it's, it's a PR term. Uh, and it's, uh, some people think it's, it's fun because you have to have these great aspirations. I think it's just distortionary. I think it confuses young people. And as I will talk about today a little bit, I think that one of the things I find most alarming about the so-called thought leaders that one will see often on podcasts and other venues is the alarmist tone or the exuberant tone. And I think 20 and 25 year olds are watching that and saying, hmm, am I going to be exuberant or I'm going to be alarmist? Those are the two choices. And I hope that this conversation we're about to have is one that makes it clear to young people that there is other ways to approach life and technology.

• 4:28Speaker 03

  I've never actually thought of myself as an AI researcher. I didn't read an AI book. The term was coined in the 50s and John McCarthy and others had particular goals in mind for coining it. And they had particular methods in mind like logical inference and so on that didn't really quite pan out. In the meantime, in the 60s and 70s, you know, 80s, something arose called machine learning. The actual methods like decision trees and nearest neighbor and logistic regression and hidden Markov models were developed in other literatures, mostly statistics, operations of search and so on. And that led to industrial success stories. So, supply chains and commerce and transportation systems all used and still to this day use vast amounts of machine learning.

• 5:11Speaker 03

  They used gradient based methods and you know, the cloud was developed to handle machine learning workloads at Amazon, in fact. And so that's the tradition I came up in. I was trying to think about systems building at scale that would also serve multiple people. The AI buzzword returned, I think, you know, maybe five or so years ago, because the data that got to be, started to be used was language data. And so the box now is not just making predictions about supply chains or commerce or prices or whatever, it spits out a human fluent language. And people said, oh my God, we've solved the old AI problem. In fact, in some ways, by if you define the AI problem narrowly, like the Turing test, yeah. But there was this ongoing tradition of machine learning and by that time had incorporated people from all different kinds of fields. And it was really having an impact, the industry

• 5:56Speaker 03

  still is. But the AI buzzword returned because of L O Ms. And now, to my view, it's been a distortionary effect on the path of research, on how we think about where research should go. But also on the path of how do we think about business models? And how do we think about where technology is going? And AI wasn't enough. They had to create this big hyped up buzzword A G I, which we will talk a lot about economics, you know, as a source of intelligence, a social intelligence. And when it's put together with machine learning style intelligence, you can now talk about at scale, not just numbers of computers and amount of data, but numbers of humans. And that's critically important to me that the role of humans as producers and consumers in these emerging systems should be respected, amplified and thought about.

• 6:42Tim Scarfe?

  Professor Michael Jordan, such an honor to have you on M L S T, especially given that nature said that you were the most influential computer scientist a little while back.

• 6:51Speaker 03

  It's funny because I was trained as a statistician and a cognitive scientist, but I'll take it.

• 6:57Tim Scarfe?

  Amazing stuff. Well, Michael, you've just published a paper called A Collectivist Economic Perspective on AI. Give us the elevator pitch.

• 7:05Speaker 03

  I was never an AI person. So in some ways, it's easy for me to come in and look at people who are self professed AI researchers and sort of say, what are you doing? What is your, what's your, what's your point? What's your goal? I think sadly, they often don't have a very clear goal. It's, it's that humans are intelligent. Humans are a computer. The brain is a computer. And if we mimic that and take aspects of it and parallelize it and make it more powerful, it'll just do great things. And it kind of stops there. It's not that there's a goal in, you know, society that we're going to try to do this or that. It'll just solve problems for us and then we'll be happy.

• 7:49Speaker 03

  And it, you know, I got away from Silicon Valley partly because that's just the way that people talk and I got tired of it. And there's not a lot of intellectual, you know, let's call it deeper, the long term thought going on. I now became a rat race and a money race and all that. So. So yeah, my my perspective, maybe it comes from a long tradition of other people having sort of social science perspectives on intelligence. We are social animals and a lot of our intelligence comes by the fact that we aggregate. We aggregate opinions and thoughts and, you know, we have cultures and so on that retain them. And moreover, the society provides a context for our intelligence.

• 8:34Speaker 03

  A smart action in one context is not in another context, and it's all very fleeting and contextual in the moment. And so, social science ideas are needed to appreciate what that means. When I say social science, I include economics. So game theoretic. The context is somebody else out there is trying to take advantage of me, or maybe to collaborate with me, and I don't really know. And so I've got to put out feelers and do signals and create mechanisms where we can interact effectively. And economics studies that in a mathematical way. That attracts me because I am a mathematically inclined person. I'm not a critiquer of AI. I want to make it right, and I want to make it better and understand what it means to be intelligent in this world, and safe and interesting, and think about long-term issues. And so to me, you have to

• 9:17Speaker 03

  do that, you know, formally or mathematically at some level. It's not enough just to build things and put them out there. So when I say a collectivist, I just mean that most of this technology is based on inputs from billions of people. So there's already a collective putting input in, and it's meant to serve billions. So there's a collective it's serving. So there's really a big network that's kind of latent there. And then economics criticals, I don't want to just sort of, you know, say words. I want to say I want to write down actionable mathematical ideas.

• 9:48Tim Scarfe?

  This is interesting, isn't it? Because I think in the 1970s, Dreyfus came up with this idea of the first step fallacy. And, you know, so we we create something, and it's related to, you know, the McCorduck effect as well. We create something so amazing, and we just think we're only one step away from being able to do anything. So these systems, they're incredible, right? They produce beautiful text. They can solve problems. They can do programming. And isn't it weird that they don't actually help us that much? We thought it was going to revolutionize-

• 10:18Speaker 03

  It's not weird at all, because the the the model there is the old AI model. Let's just build something intelligent. And and it's only got upgraded a little bit. It's going to be a better search engine. That's fine. I do think the search engine was major progress for humanity. But now it became more than a search engine. It was like a secretary sitting on your shoulder, helping you, whispering things to you. And it's just a dumb business model. I don't think many people really will want that. They'll turn the damn thing off. They want to think for themselves. They want, you know, maybe at the end of the day, a summary or something, but they don't want this all the time, you know, they're interacting with this entity thing. It's not a very good business model. And in the meantime, we have huge healthcare systems and transportation systems and finance

• 10:59Speaker 03

  systems that are all based on data flows among many billions of agents and, you know, are ripe. They already have a lot of machine learning in them and they're ripe for thinking in a more economic way. What are the agents and what are they trying to get out of it? And what kind of cooperation and competition is latent there that you could, you know, make, improve? You know, markets arose thousands of years ago and, you know, we learned about some of the principles, but we can improve them. And thinking arose, you know, billions of years ago or whatever. But we're not perfect. And not just in terms of thinking, but we're also not perfect in terms of narrowly following our own agenda and hurting other people, even though we don't want to. Humans are wonderful. We want to prize human life and creativity and, you know,

• 11:44Speaker 03

  emotion and love and so on and so forth. It's fundamental. But humans are also bad or do bad things. And that's where technology should be able to aid you. And so you need to think about the system, the old reset of these systems. They're not really systems. They're, you know, big statistical boxes that do inputs and outputs. That's not a system's way of thinking. There's a lower level system, of course, the computer system. But I want to be above that. I want to say, what ecosystem does this belong to? Who's it interacting with? What rate and what kind of quality and what kind of values are being created? And when I say value, I mean often mean money. I want jobs out of this thing. I don't want just it to answer and do things for us. I want it to create opportunities for,

• 12:29Speaker 03

  for work and creativity and so on.

• 12:32Tim Scarfe?

  Rich Sutton is quoted quite a lot in respect of design versus evolve. And I watched a wonderful talk by David Deutsch the other day and he was kind of talking about explanations. And he said that physicists obviously go for these principled, low-level explanations. But sometimes you get these high-level coarse grannies that are just really good and maybe economics is one of those. But what do you say to folks from Silicon Valley like Ilya Sutskevich? And they're just talking about human value functions and they're saying, okay, well, you've got these LLMs and we just turn them into multi-agent systems and we get all of the economic stuff that you're talking about for free. What would you say to those people?

• 13:09Speaker 03

  I mean, it's just not a good way to think about engineering. I mean, if you were a chemical engineer back in the 40s and 50s, saying, we're just going to throw a lot of stuff together and make it work. Well, you could do it, but you'd get a lot of explosions and a lot of economically non-viable things. You'd hurt a lot of people. And I think a lot of these people are not thinking about all the people that are being hurt already. You know, Facebook and so on, you know, it's damaged a lot of young people. A lot of teenagers are having mental health problems. And this is just not something that was talked about by computer scientists at all. And now we're talking about yet another level of displacement of, you know, jobs may go away, but you know, that's tough. It'll create new ones, of course, like always. You know, I just don't like to talk that way.

• 13:52Speaker 03

  And you know, so you've got to say, well, step back a moment, what is your point? Are you trying to create a new kind of market where people could come in and have their talents valued and appreciated? And where bids could be put out for things that people might need and, you know, and collaborations can emerge. And there could be producer-consumer relationships being explored and understood and developed. And this could all be a mix of computation and humans. I think eventually we'll all kind of merge. But, you know, along the way, just doing something so disruptive with all of these metaphors. It's not good social science or not good mathematics. It's just metaphors.

• 14:33Speaker 03

  And yes, you can build it because the previous generation of people created these amazing things that collect data. And we can do gradient descent on it and ad hoc architectures. And yes, that works, it's amazing, but let's not give so much credit to the people that did that. It's the people 20, 30 years ago who did that. So the current generation is just way too, you know, there's not much thought going on, not much intellectual stuff. It's just, yeah, it's possible to build it, it's possible to steal the data from wherever you want to. Because that's what the internet allowed to happen. And not return any value to the person who originated the data. It's possible to run gradient descent on that, but you need huge amounts of money. But it's now possible to get it from people who aren't thinking very deeply. And so, you know, I'm, I'm, it made me seem more dark than I want to.

• 15:17Speaker 03

  I mean, there's some lot of good in, you know, builders. But we also, every previous era of engineering development, electrical engineering, chemical, mechanical and all, had some builders, but they had a lot of concepts and they had a lot of thinkers. In fact, all of those engineering disciplines had something like Maxwell's equations or Newton's equations to help them kind of hear. No, it's just people that are very smart and who can code and then have lots of intuitions. And, and it seems to, and I don't ever see anything that feels deeply intellectual to me. It feels like science fiction.

• 15:50Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 16:20Speaker 04

  Low energy, unfocused, foggy? You might be dehydrated. Gatorade Zero powders have been scientifically designed to help improve hydration balance in the body with a clinically proven electrolyte blend and no artificial colors or flavors. Great tasting hydration, no matter where you're going. Shop now online or at retailers nationwide. Gatorade hydrates better than water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 16:49Tim Scarfe?

  Well, I suppose another thing that doesn't help is that these, these systems are like soup. And there's even a field called mechanistic interpretability that tries to kind of dig into the soup and, and it's almost like they're, they're searching for UFOs. They're trying to find these principled circuits that do reasoning or, or do whatever the thing is. And I guess you could say cynically that it's not like when engineers build a bridge.

• 17:11Speaker 03

  Well, I'm a little less negative than that. I don't think it's bad to build systems you don't understand. But then you've got to kind of put things around it. And the things that are beeping around are like buzzwords, like AI safety. It's a buzzword. Okay, what you really need, I mean, a human, you can't explain to me why you picked this Airbnb over another one or whatever, all the choices you've made today are inexplicable to me, they come out of your brain. And I don't need to know all the whys and wherefore's of your choices. And what I need to know is that you're somewhat predictable and that if I make certain options available to you, you know, you're likely to take this one versus this one, and therefore I can make my own plans and we can start to interact and so on. So that's

• 17:54Speaker 03

  part of economics, is the economic style of thinking says, I don't understand all these other entities out there, but there's certain, you know, rules of thumb that I can use, or, you know, quantitative predictions I can put in place that allow me to interact and not get hurt and even get value out of it. You know, so no, I don't think it's necessary to understand all the details. Now, the input-output behavior, you often have to understand better than we can now. For example, if I'm denied a loan at a bank, and the bank used this big AI program based on past data, I want to know why. And why doesn't mean that you look in the internals and show me some circuit. No one's going to want that. They're going to want, well, there was like 50, here's like 50 people that are pretty much like you, according to the, you know, embedding we're using in this big network. And of those

• 18:39Speaker 03

  50 people that are like you, some of them got the loans, some of them didn't, and here, here, let me just show you what those people are like. And you start to say, oh, I see, they differ from me in this way, that's actionable to me. I could now change things. So you have to build systems around this predictive system. That's a nearest neighbor system, for example. And that system will supply what people might consider more like an explanation. And so it's not just trying to go in the internals of something. Again, chemical engineering is, you know, there's, there's, there's certainly thermodynamics and lots of things are understood, but lots of phenomena were not understood for a long, long time. You mix up a bunch of stuff and, you know, certain waves are created and certain things happen and you exploit that and go and move on. But you understand something about input, about behavior

• 19:20Speaker 03

  and constraints and so on. I think the current generation of neural nets will continue to, you know, they have very nice scaling behavior, they'll continue to be there.

• 19:33Speaker 03

  But they really have to be thought of as a product, part of a bigger ecosystem. And then you kind of ask, well, what can the neural net do in this context? And what's it missing? And what if I have multiple of them? And how do they engage with each other and with us? And what is needed? What transparency is needed for the overall interaction to be an effective one? Whether or not I understand all the details or not.

• 19:55Tim Scarfe?

  For some reason, and correct me if I'm wrong, I have an intuition that behaviorism is bad, that just by not having any mechanistic understanding and only looking at the outputs, there's the famous example, isn't there, of the hen didn't know his neck was going to be broken? And one example of this actually is AlphaFold. So I interviewed John Jumper last week at Google. And you did some analysis on those 200 million predicted proteins, and you found they were very good, but there was something missing, but you could robustify them.

• 20:21Speaker 03

  You could robustify them, that's correct. And I think that's a good example. So I'm a big admirer of AlphaFold. I don't think it's like an L O M. I think it's targeted. It was for a particular set of problems and it does it very well. The issue that we found empirically was that when you ask certain kinds of questions,

• 20:40Speaker 03

  you know, in particular, we did one where we were looking whether quantum fluctuations in a protein were associated with phosphorylation, meaning the protein was active or not in the cell. And you might think that these fluctuations, which lead to strands hanging off, are kind of like, you know, bad proteins, evolution wouldn't use them. But it turned out that a lot of them seem to be phosphorylated, meaning they're reactive in the cell. Hmm, that suggests a hypothesis test. Is there an association between yes, no phosphorylated and yes, no quantum fluctuation? So that's a little two by two table. And you do a statistical test on that. And the problem is that if you just use known protein data, that's been crystal, you know, there's a crystal structure known. You don't have enough data to test that hypothesis with high power.

• 21:24Speaker 03

  And so you can't reject the null hypothesis, that there's no association, even though it looks like there is. On the other hand, you use 200 million, you know, proteins out of AlphaFold, you can test hypothesis with high power and you reject the null hypothesis. But what we found is that the confidence interval on that statistic of that two by two table was extremely narrow and way far from the truth, the true value of the gold standard value. And we found this in domain after domain. You know, so why is that? Well, what's happening there is that there's probably not many. Examples in the training set of proteins with quantum fluctuation. Because it's not been that studied in the past, and it's hard to crystallize.

• 22:06Speaker 03

  And so, not many examples means that it's quite possible AlphaFold won't give out a great answer. But it won't tell you that. It doesn't give you out error bars, and it doesn't know specifically on the question you're asking. That's where I want the error bars. And it didn't know about that question when it was, you know, built and designed. Okay. Alright, so now I have a good statistical question. What if I add a little bit of ground truth data to the 200 million? Can I shift the error bar so it stays somewhat narrow? So I have high power, but it covers the truth. And the answer is, yeah, there's a methodology. We developed something called prediction powered inference that does exactly that. And so it'll cover the truth, just like in a classical statistical setting. But it's using this rather highly biased architecture. And it's now, it's not biased overall. In fact, its accuracy is high overall.

• 22:51Speaker 03

  But for the question I'm asking, it might be very biased. And that's gonna happen a lot in science, because scientists are rarely interested in just studying the past over again. They're interested in brand new things on the edge of knowledge. And that's where specifically these foundation models will be most poor and most highly biased. So there needs to be, around any foundation model, the ability to maybe collect a bit of ground truth data to merge it in with some procedure like this, and then to give out a more trustable answer. That's all not science fiction. That's what can be done and what really needs to be done. And I'm sure the AlphaFold people are on board with that, that they would not find that weird or surprising. But a lot of other people out there talk about bias and all that, and they either don't worry about it. I say it'll go away, we have enough data.

• 23:36Speaker 03

  Or they just critique the architectures and critique the outputs, but they have no scientific, you know, method in mind that'll help us go forward. So that's kind of the state we're in.

• 23:46Tim Scarfe?

  I challenged Sean a little bit about the extent to which AlphaFold understands, and he was basically allergic to the word "understands."

• 23:53Speaker 05

  We are not trying to tell you everything. We are not a model of the entire cell. These machines let us predict. They let us control. We have to derive our own understanding at this moment, right? We can experiment now on the artifact. We can look at the 200 million predicted structures, not just the 200,000 experimental structures in order to help us understand, but it doesn't do the act of understanding for us. It does the act of predict and maybe control.

• 24:25Speaker 03

  Why should AlphaFold understand?

• 24:28Tim Scarfe?

  Well, what would it mean to? I mean, he was sketching it out to me. He kind of said that this is a weird alien artifact and it's not like it's kind of created. It's refined. There's this recycled pathway. You can put the thing through multiple times. You can kind of corrupt it halfway through and the network is just iteratively kind of, you know, it solves the complex bit first and then it's refining, refining, refining. And like, could we interpret that as an understanding process?

• 24:50Speaker 03

  I don't think we need to see that. I think this anthropomorphizing of intelligence and understanding and all that is not necessary, not appropriate, and is a distraction for many, many problems. Why say it understands? You know, some of my heritage comes from seeing in real life in industrial settings, machine learning algorithms being rolled out 20, 30 years ago. So I, when I first went to the West Coast, I visited Amazon and around 2000, they were using huge amounts of data to do supply chain modeling using the neural networks of the day was random forests. And it was really working and they could make really fantastic predictions of, you know, whether certain ships would be delayed in the Indian Ocean or whatever. And so certain parts wouldn't arrive in time. And the overall supply chain takes billions of products and sends it to 100 million people per day.

• 25:34Speaker 03

  And so you cannot, there's no way that any human can understand what's happening in that big, big box.

• 25:41Speaker 03

  But it's not necessary. And in fact, you can ask, does that overall system understand, you know, transport and logistics? And the answer is, who cares? It's, it does a very important optimization and prediction process that allows an engineering system to be built around it. It brings down uncertainty. It makes you possible to do kind of stockpiling and, you know, planning. And that's what you ask for. You don't care whether it's, has to have a word like understand it or intelligence applied to it. That's for the media. The media, that's, that's kind of my problem with a lot of these people rolling out AGI and AI terminology. The media laps it up and they know that. Even though we don't have a clue what understanding or intelligence, we in our research

• 26:25Speaker 03

  realize we don't care or need it. We want to build good systems.

• 26:28Tim Scarfe?

  Yes, it's interesting because I agree that we live in this complex adaptive irreducible system. We can't essentialize it. And folks like Francois Schollet or even David Krakauer, they talk about intelligence as the adaptation synthesis, of course, grand representations. But what if there is a bit of a step? So let's not anthropomorphize it. Let's say that understanding is about like not the endpoint, it's about the path which led us there. And we know that in the real world, we're a collective intelligence and there's the blind man and the elephant, and we all take our own paths in lives and we have different perspectives on the same whole. So what if like a better form of understanding is just being able to reconstruct the thing from your perspective using building blocks rather than trying to essentialize it?

• 27:11Speaker 03

  You know, that's all sounds great. It's just not the language that those of us who research would use. I mean, we would think in those terms a little bit, of course, but we would try to turn it into some kind of an equilibrium or optimization problem. And here's the information that's available, and here's the data, and here's the power and the, you know, the error rates. And we try to put a little bit of structure around it of that form. And, you know, there's always this creative moment. Like, I remember when in high jumping, I used to be high jumping, interested in high jumping when I was a kid. And, you know, you would go up to the bar and you'd jump over it in various ways. And there were the barrel roll, rolling across, you know, was the technique the Olympians were using. And then there's this guy, Dick Fosbury, came along and he says, no, if I

• 27:53Speaker 03

  go backwards, I can do better. And no one had thought about doing that. As soon as he did it, everybody did it. And that's, and, you know, it went up like by a half a meter or something. I don't know. And so what process led to that? Was it an understanding process? You know, it was just a little bit, let's try something different, mixed in with the ability to try it out and to do tests. So a huge amount of industrial planning is try it out and see what works. Was it called A B tests? And those are done all of the time and I've got nothing against that. It's not based on understanding. But it's led to optimized systems that can do things that, you know, people hadn't thought about before. So a blend of that with understanding.

• 28:37Speaker 03

  But just understanding, you know, I was a cognitive scientist and I, you know, and interested in neuroscience. One should be interested in those things. They're fascinating. But they aren't, they aren't the leading edge of thinking of how to build systems that, you know, work in the world. And they're not the leading edge of trying to be leaving the next generation systems. If you put, a lot of people keep saying, well, we got to put logic back in our symbols. Because that came from the, that came from our previous kind of view of what humans are doing. Probably, humans are capable of doing some logical reasoning and probably have some symbols. Whether they're kind of built in some complicated network or they're reified somehow. I don't know. My intuition is as good as yours.

• 29:16Speaker 03

  But really, the goal is to, you know, in a, I tend to be an engineer at heart. You know, a mathematically inclined engineer. I want to say, what are you trying to achieve? Are you trying to displace teachers? Are you trying to make doctors better? What are you trying to do? And what, what would be the abstractions and the, and the points of entry into that problem? And then how can you kind of pull back from that and do it in some general way? That's, that's elegant and will inspire others.

• 29:42Tim Scarfe?

  It's so interesting seeing different scientists, you know, from a multidisciplinary perspective, attack this problem. Physicists, for example, they, they work very, very low level and they talk about, you know, the, the dynamics of particle systems and whatnot. And what I'm really fascinated in, I mean, you come at it from an economics perspective, which is traditionally dominated by this agential lens. And you talk about equilibrium and incentives and so on. How, how does that come into it? How, how would you take a very complex system and almost kind of decompose it into this new frame of thinking?

• 30:12Speaker 03

  It's not been done really enough for me to have, you know, tons and tons of, of great examples. But, um, you know, we've been looking at kind of modestly scaled examples where there's a, um. So for example, we looked at a little bit of drug discovery and kind of the, the regulation, you know, the, so. I'm a pharmaceutical company. I test out all kinds of proteins and I throw them in animals and maybe a few humans to sort of see what's working. And I have some understanding, quote unquote, and in other words, I know somebody have the evolutionary biology behind it and so on. And that guides me. But at some point, someone's got to really test this out in the real world and decide, regulatory agents has got to come in and say, yeah, that goes to market or it doesn't. Okay, so now you've got a kind of tangled web of scientists and pharmaceutical companies, and not just one, but many, many of them, and proteins.

• 30:55Speaker 03

  And now you've got to think about how that system is behaving. So hopefully, the regulatory agency is trying to, overall, over the entire system, have the number of false positives be low and false negatives be low. That's what the goal is of the problem. So it's a statistical problem. Oh, but wait, a classical statistical problem, you would just go gather IID data, independent identically distributed data from some source. Here, no, the data is coming from the self-interested pharmaceutical companies. What's their motivation? Money, and whatever. Maybe they want to help people and money. And all that's kind of hidden from you as a regulatory agency. Okay? So now, the economic mindset kind of comes into play. He says, "Well, it's hidden from me, but it's not arbitrary.

• 31:39Speaker 03

  You know, I can kind of probe in various ways." And so that becomes very economic. So economists think about how you set prices. You know, so if I've got a lot of people coming on my airline, there's a thousand people that just arrived who want to go from here to London. Every one of them has a different price point, and that price point will shift in the moment. How eager are they to get? It's not just because they have a lot of money. It's because they have needs. And I don't know what those are. So what I do is I set up various services and various prices that kind of bracket the possibilities, so that overall, it's likely I'll make enough money and everybody will be kind of happy and the service will go forward in life. So it's a blend of knowing a few things and admitting that you don't know other things,

• 32:21Speaker 03

  but putting it in a system that actually can work with that kind of mix of asymmetries and incentives. So the incentives there are that, you know, there's a certain service and price. If you pick that one, you're likely to be able to get on the airplane. You're likely to have the goodies you need or whatever. And that then doesn't make you do something. It incentivizes you. And in the pharmaceutical world, if I could get them to be incentivized to mostly send in drugs they've done some testing on, or they have some belief it's a pretty good one, and not just throw arbitrary ones at me, then maybe the overall system will actually have the error rate you want it to. Okay? Because if you don't do that, then if it's a drug that'll make a ton, you know, a billion

• 33:06Speaker 03

  people will use, then you're going to make money whether it really works or not. Okay? So you need it to get to market. How do you get it to market? Well, you just throw it at the regulatory agency, and there may be a false positive. They just got a false positive and they put it on the market. You make a ton of money. And if there's enough of that, the incentives are all wrong and the overall system will not control type one and type two errors. Those are examples we've actually worked on, but I just hope you can appreciate when all this stuff starts to really roll out in society, it's not going to be there's a few big L L Ms and everyone consults them like a search engine. That's just not the model. It's going to be there's local data, like I told you about with prediction power. Everyone's got to vet what's coming at them. There's got also be local data because I collected it with some expense and I want to just give

• 33:48Speaker 03

  it away. Okay. Thank God. Finally. Anthropic is paying people for money. Yeah, that has got to be the future. So I'm going to have some competitive value in my data and I'm not going to just give it out. All right. And so now if you start interacting with lots of lots of people that want to get some value out of the interactions, you have to talk about the incentives. What's the incentive for them to send the data, but not only send the data, send correct data, send truthful data. Don't just add or not be adversarial. And so I cannot imagine, you know, a fully full fledged version of all this rolling out in society and all of our decision making throughout our lives without a deeply microeconomic perspective accompanying the gradient descent on data.

• 34:30Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 35:00Speaker 04

  Low energy, unfocused, foggy, you might be dehydrated. Gatorade Zero powders have been scientifically designed to help improve hydration balance in the body. With a clinically proven electrolyte blend and no artificial colors or flavors. Great tasting hydration, no matter where you're going. Shop now online or at retailers nationwide. Gatorade hydrates better than water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 35:29Tim Scarfe?

  You spoke about this three-layer model. So there was an example where you know, you might have, you know, consumers and they might have their data and you've got Google and then, you know, Google is using the data, the consumers are getting a service and then Google might sell the data over here. That's kind of like a traditional model. Let's start with that.

• 35:44Speaker 03

  Okay, so those are really kind of like bore-atom kind of things. We're being scientists there. We're trying to say, what's a minimal model that exhibits some of the behavior that we want to study here? So let's think about a data market because data is not just now something you analyze to build a big L M. It's also something you'd sell and buy and has value. And also there's privacy concerns about data. Okay, so let's put a little minimal model together where we could study that. All right. And so one we've done is called, we call it a three-layer data market. And it has, it exists in the real world. This is an abstraction, but it's a, it's a real thing. You've got a user or multiple users coming into some platforms. The platforms provide a service like, you know, imagine, you know, payment service. And as I use that, they get data from me.

• 36:29Speaker 03

  They learn about what kind of purchase I've made and so on. And they use that data to make their service better. That's a good little, nice little loop there. All right. The problem is that rarely do they make enough money off of that service. They, you know, take a small cut that the merchants don't like to give them. So they have to do other things to, to, to stay in business. So typically now, for a long time now, probably 20 years, they've been selling their data to third-party data buyers. And these are not evil people, just trying to, you know, ruin people's privacy. They're, they're trying to do market research, learning what, what would work and what are people really doing? This is behavioral studies. And so the data is valued to them. They pay for it. All right. So, you know, Google doesn't need this because they create this artificial advertising market,

• 37:14Speaker 03

  which we could talk more about, that kind of superpowered all this nonsense. But other companies like Mastercard would do, have to, have to sell their data. So, so now it's a three-layer thing. And as soon as that third layer was introduced, the, the equilibrium has to shift because the, the user who's sending their data in just lost something. They lost a little bit of privacy. Some third-party that I don't know anything about is getting data about me. And I can't just accept that, you know, but I can't walk away, you know, so there's a stress on the system now. So in an effective economic system, what would happen is that the you wouldn't just wait for the regulator to come in, the government say, you know, no, this can't be done. What you would do is that the platforms would say, well, we'll

• 37:59Speaker 03

  offer you a tunable level of differential privacy for some cost. Or we'll just say that this our company, I'm Google, I'll offer you a level, you know, point three. And some other company says, well, I'll offer you level point seven. Okay. So the user looks at that and says, ah, point seven, that's, that's better. I really care about my privacy. So I'll go there. That company, they will get start, sort of get more data and their service will get even better. And whoop, you got a little nice little feedback loop there. But now the data buyers will look at the data from that person. At point seven means more noise has been added to the data. It's less valuable to the data buyer. Data buyer will say, I'll, I'll spend less, I'll give you less money for that. I'll give more

• 38:40Speaker 03

  money to Google. And so now you can see there's conflicting tendencies here. The incentives are aligned, but they're not, you're not optimal for everybody. And so now the mathematics is not just an optimization problem. Mathematics is an equilibrium problem. But it's an equilibrium problem that involves statistical assertions, data, and how much you can predict with this data and so on that. So you quantify that with error bars and statistical predictions. So you put that all together in a big mathematical system, and you can find the equilibrium as a function of various system parameters. So for example, is there a minimal level of privacy that regulators could require or not? Or, you know, is there some heterogeneous

• 39:22Speaker 03

  privacy budget, you know, et cetera, et cetera. You could put in various, those sorts of things. And now you do a little plot of how the equilibrium moved. And the equilibrium have overall utilities for all the three players summed up. That's the social welfare. You can ask, how high is the social welfare or that equilibrium versus this one versus this one? And another regulator could look at that and say, well, I prefer this one because it's overall higher social welfare. And laws could be made at that level. Okay. So even though this is a toy little, you know, little toy model. It has the ingredients that I'm very interested in predictive models, data markets, but money, incentives, and a real system that really is already kind of working, but people aren't thinking about it very well, just like in the drug discovery domain. But if you take

• 40:06Speaker 03

  an economics point of view, you can make a lot, you can make the system better.

• 40:11Tim Scarfe?

  Of course, so modeling it as a dynamical system which we can simulate, and then we get these modes of

• 40:17Speaker 03

  No, we don't have to simulate it. In that case, it's you can actually write equations and calculate equilibria. It's a Stackelberg game, and you can actually find the equilibria. You know, in other cases, you would simulate. But the point is, a lot of my machine learning colleagues don't know much about fixed point algorithms and finding equilibrium and how they shift as you shift various parameters and all that. That's economics stuff. Machine learning people are really good at optimization, but this is not an optimization problem. It's a, and there's all these algorithms and other branches of mathematics that find Pareto frontiers and do it statistically and do it as a function of size of various markets and size of populations and all that. And it's kind of amazing in this era that the

• 41:00Speaker 03

  two have almost never met. The economists never had a lot of data to inform their design of their market. So they just wrote down a bunch of equations and made rational assumptions and all that, and then found equilibrium mathematically or otherwise. And the machine learning people never thought about the equilibrium. They just had a lot of data and they used it to do the obvious thing, predict the next word in a string of words. But the future has got to be that those came, those branches come together. The economics equilibrium perspective is critical, but the, oh, it's got to be adaptive perspective is critical. And also, you alluded to earlier, some of the machine learning Silicon Valley types just saying, well, we've got all this data, therefore, all the behavioral stuff's already built in.

• 41:42Speaker 03

  And that's too naive, obviously, but it is a useful point of view in a certain sense. The economists do make rational assumptions they shouldn't have to make. And if you put in data instead of that assumption, you'll probably do better. You'll have some of the behavioral economics already built in. But if you do it outside of any economic thinking whatsoever, you'll just make a mess of things. And of course, that's what Silicon Valley seems to be pretty good at.

• 42:05Tim Scarfe?

  What is the difference between like data and the kind of knowledge that you're talking about?

• 42:10Speaker 03

  Social knowledge is very ephemeral and it's very in the moment. You know, I could walk down the streets of Copenhagen here and there's all kinds of little markets out there and what's available and at what price and you know, what I might like and all that, it's all super ephemeral. And that's kind of, I think, a better way to think about all this. You can't just gather enough data to know that that person walking down the street there, they're going to come buy this product. It just, and all of our decisions that are our choices, you know, cannot be that you have enough data to cover all of that. You know everything about what's going to happen, even in the next, you know, 10 seconds. So you have to be a little more humble about, you know, that I have a lot of ignorance, but that doesn't mean I can't build a safe system like a market of some kind that could, people

• 42:54Speaker 03

  could come in and they cannot get cheated, they can get value and it could evolve over time. It could shift in ways and I don't have to be, I'm not the god figure at the top, you know, designing the human value function or whatever to put it in so that it responds particularly well to what humans really want in my god vision view of the world. No, it's got to be a system that permits bottom up preferences to be expressed in the way the human wants to in the moment. And that's not built in and the system respects those things and wants to maybe learn more about them and then use them in the moment and then maybe keep some of it, maybe it's all ephemeral and goes away. But there seems to be a huge naivete about what data, even if you have, you know, whatever

• 43:38Speaker 03

  exabytes or whatever of data, you're going to miss all the details that are probably the main thing that matters for a particular kinds of class of decisions. Even AlphaFold, based on huge amounts of data, doesn't do well on certain queries. And yes, they'll patch those and it'll do better and better, but always the new questions that people will ask will always be on the edge of knowledge or often be on the edge of knowledge. And people aren't thinking about that, they're thinking about, well, I just got to replace the teacher. Because the teacher is working not on the edge of knowledge, they're working back in there, you know, all the stuff that's already known. That's fine. You could aid teachers, but good teachers also kind of know how to migrate to the edge of knowledge.

• 44:13Tim Scarfe?

  Yeah, and when we do abstraction and idealization, it's always a little bit lossy. And I'm fascinated by this observation that market, you said markets were around before capitalism. So it's this bottom up thing, it kind of, it's a natural.

• 44:25Speaker 03

  This is not capitalism. That's one methodology for making markets work, but it's not the only one.

• 44:29Tim Scarfe?

  Exactly. So it's a, it's a, it's a natural phenomenon, you know, the emergence of something we might call markets, and it's constructive, and it's divergent and diverse. And then you're saying somewhere the rubber can meet the road, so we can create abstractions, we can do some kind of modeling, and we try and do it in such a way that we don't, it's not too lossy.

• 44:50Speaker 03

  Absolutely. Human culture creates abstractions. Individual humans create abstractions too that work for them. And, and when those abstractions are kind of useful enough and they can be, they kind of get promoted into the culture, and that flows up and down all the time. And indeed, that's something that systems could perhaps help with. I'm not going to just trust systems to take on that burden, but that could, it could be helpful. And so indeed, it's not just the individual cognitive entity that creates abstractions. And we should just re-identify that. Yes, it comes, but cultures create abstractions. And you can study the micro economy of that or whatever or not, you can just sort of say those abstractions came, are part of the culture, and they're useful, they've stayed around and, and they're useful.

• 45:32Speaker 03

  And, and going forward, it's not that we're going to just keep the old ones, but we're going to build systems that allow new ones to emerge. And that is not the god figure figuring it all out and putting them in there. I, you know, so Silicon Valley says, well, you just got it, we got so much data, we'll have so much that we can do all of it top down. And they're forgetting somehow that, first of all, the data came bottom up, the data was all contextual, and the data was supplied by people and so on. But they're also forgetting that it all has got to continue to. Be a micro level that is kind of going to be beyond their ability to sense and we're not going to want them in so much in our lives. You know, the after the search engine, which I thought was a fantastic piece of technology, allowed access and all that sort of then a lot of it was very prime.

• 46:16Speaker 03

  We're going to put glasses on you, we're going to put, you know, things around cameras in your home and all that and we're gonna, we're gonna know all the details of your life and we're gonna, you know, make your life better somehow. And that just that equation did not calculate for me.

• 46:27Tim Scarfe?

  Yeah, so this idea that that culture is the abstraction hard drive in the sky. And both cultures very adaptable, so we can delete strategy. I mean, knowledge decays very quickly and organizations maintain knowledge and really good bits of knowledge stay around for a long time. And, you know, down at the bottom up, we're creating new bits of knowledge. So how does that whole ecosystem work? How do we kind of designate good things that stick around and how do we find new bits?

• 46:50Speaker 03

  We don't, you and me, the answer is you and me don't. But that's something that good intellectuals do. That's what, you know, economics, like there's a whole field of behavioral organization, you know, how does, how do organizations effectively emerge? And those are, that's really interesting. It's not everything, but you know, there's a lot known there and some of it's mathematical, some of it's not, some of it's best practices. But those are the kind of ways that these AI ecosystems should be talked about and not just in terms of neuroscience and, you know, metaphors of the neurons and physics metaphors and all the stuff that it was part of my heritage too. But it just felt so lacking when we actually see these things, the rubber hitting the road as you, as you say.

• 47:36Speaker 03

  And so, yes, behavioral organization, how people are organized into things that promote, you know, not only, you know, good revenue for companies, but also promote democracy and so on. So there are people that talk about all these things. And I just don't think they seem to have much presence in Silicon Valley. And maybe that's for good, you know, Silicon Valley let them go, you know, they'll just burn a lot of money and, you know, cause some headaches and then they'll also creates things like search engines and, and I think a lot of the companies are also really focused on creating value. I do think Amazon is different from Meta. Amazon has got a business model, bring packages to doors. And behind that, they create some technology to support that and it's mostly to the good in my view. You have to worry about labor markets and so on and so on, but those are all good things to worry about.

• 48:21Speaker 03

  But just creating computational artifacts that make predictions and that if you were to wear goggles, you would be able to, you know, live in their world. It's not a business model. It's a it's a science fiction dream that may not be may not be helpful for humanity.

• 48:38Tim Scarfe?

  Well, can we explore that because you gave the example of Spotify. So you know, we were talking about this three layer thing before and and now we've got this weird incentive structure where Spotify are actually incentivized to generate the songs with with AI, right?

• 48:50Speaker 03

  Yeah, they are. And I'm a, you know, I have a project. I'm a scientific advisor to something called United Masters, which is an alternative, which has musicians keep their their their work and and United Masters connects them to brands and to other kind of opportunities so that they're kind of more like a real artist, not just like that they their their song got streamed and they got a little bit of money. Because Spotify indeed is not, it's close to perhaps a monopoly, but it's not incentivized to pay people. There's not a pricing. There's monopoly prices, if you will. And so one would hope that somehow the market will fix that, that enough young artists will say, I'm getting screwed here. I'm not making any money.

• 49:34Speaker 03

  And another service will emerge. But we are in an era where some of these services do become monopolies pretty quick. And that's I leave to my economist friends to think that through and to look back at historical examples and think about, you know, is regulation needed or is or is there other market making mechanisms that'll make this more healthy for human beings? I'm not against Spotify, but you know, it should be part of an ecosystem that actually rewards the artist more. Right now, an artist is getting paid very, very little. I think I don't believe the prices are being set under competitive mechanisms. But I think there's this broader, you know, macroeconomic view of what are these systems doing and what are they what's their role in society going to be? And with the search engine, I was many of us were puzzled about how they would make

• 50:19Speaker 03

  money. It just didn't seem like, you know, there was a money maker. And then the whole advertising thing was a bit of a surprise, at least to me, that it would become so huge. Of course, the underlying thing is that people expect things for free. All right. And so Google couldn't kind of make payments. But I think they made a mistake at some point. I think like with YouTube, when they acquired YouTube. YouTube is more than just pointing people to a website, you know, YouTube is incentivizing creators to create things that people will watch. At that point, I think a socially responsible Google to critique them a little bit would have. Said, oh, we've created a market here. We've created a producer consumer relationship. We've got to make that market a little bit more valid and we could actually have that when someone's watching things, they can.

• 51:02Speaker 03

  Have a some sort of an economic connection to the person that made it and then it can be incentives flowing that this person now incentivized to make more because here's my audience. Connect it directly. Instead, it was all going through Google and then Google was putting advertisements next to make a ton of money for themselves. And then there's a modest incentive, give back a little bit of money. That to me was a huge mistake and then Facebook made it even worse.

• 51:25Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 51:55Speaker 04

  Low energy, unfocused, foggy, you might be dehydrated. Gatorade zero powders have been scientifically designed to help improve hydration balance in the body. With a clinically proven electrolyte blend and no artificial colors or flavors. Great tasting hydration, no matter where you're going. Shop now online or at retailers nationwide. Gatorade hydrates better than water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 52:24Tim Scarfe?

  So you've buttered up against folks like Geoffrey Hinton and Stuart Russell at Berkeley. And these guys are painting a picture that this technology is recursively self-improving, that it is agential, that it's not a cultural technology, it's a thing in of itself. And this seems a little bit science fiction on the first read.

• 52:45Speaker 03

  Very science fiction.

• 52:46Tim Scarfe?

  What do you think?

• 52:47Speaker 03

  So I think it's science fiction, and I think science fiction is important for society. But it's also, at the level it's being promoted and those kind of voices, it's really hurting 25 and 20-year-olds. You know, these young folks of whom there are huge numbers are excited about technology, and they want to build things that help their family and help their country, actually more their family than their country, honestly. And they see real opportunities in doing that. And they're kind of being told by the leaders, well, we had our fun. We developed a bunch of algorithms. We did it, and we were just interested in the pure, you know, understanding intelligence, even though they didn't understand intelligence. They built, you know, gradient descent algorithms. And now you guys, you can't do this because it's dangerous.

• 53:31Speaker 03

  It's gonna wipe out humanity with a high probability, or it's super-intelligent, it arrives soon, so there's nothing left to do. That's in your lifetime. That is so demoralizing, so demoralizing. And that thing, I think that bothers me the most. I mean, the second part that bothers me is there's no economic thinking going on there. It's zero. It's really about cognitive science mentality or neuroscience. We figured out how the brain works. It's gradient descent with a lot of distributed neurons. And the fact that these LLMs are working so well shows that we figured it out. It wouldn't work so well otherwise. Now, I think that's dubious. We don't, the brain is way beyond. I mean, you ask the neuroscience if this has anything to do with the brain, basically they'll say no. It's a nice metaphor, it's cartoon.

• 54:15Speaker 03

  Does gradient descent work at massive scale? Yeah, more than we would have ever imagined. But is it showing its weaknesses? Yeah. Can it be fixed? Certain areas, yeah. You know, you build certain verticals, they'll do good things. And it'll make mathematicians go faster, but it won't put them out of business and so on.

• 54:36Speaker 03

  It's having a big effect on society. I worry more about labor and capital relationships than I worry about it deciding to take over. So the rest of it, that to me is more on the ground sort of, you know, how does the next generation take technology and work with it? And I don't think that voices like that are actually helping that generation to actually perceive what they should work on and why.

• 55:04Speaker 03

  Superintelligence versus extinction. Those are your two options. And god damn it, those aren't the only two options. There's a huge number of very positive things that can be done at human scale.

• 55:17Speaker 03

  And let's hope that enough of the young mentalities kind of get behind that.

• 55:23Speaker 03

  But they don't have enough examples of people out there who made money by making, did Sam Altman make a life better, you know, not clear? And, and, and, you know, I think in previous generations, there was a little bit more, you know, here's people that are out there making things that, you know, vaccines or whatever. Oh, I want to be like that. And right now, not so good.

• 55:44Tim Scarfe?

  I don't know if I could get you to be a psychologist for a minute and try and understand why these, I don't know whether it's the search for purpose, but have you noticed as well that some folks think that there's going to be a utopian future? And when you, when you speak with them, there's quite similar DNA. So they, they also think that it's recursively self-improving, it's going to be superintelligence and so on. But if, if I was to press you to say, why do you, I can understand, right? These things are so clever. But why is it? Why do they believe that?

• 56:12Speaker 03

  I mean, they're clever in a way, in a recognizable way at some level. They're, they're taking all this human cleverness and packaging it in a new way. And again, I think it'll kind of always be missing a little bit of the point because it's not in the moment. It's not the ephemeral stuff.

• 56:30Speaker 03

  But that doesn't mean it can't be even more clever. And I kind of think that that's okay. I think that for me, the goal here is not to build a superintelligence and have a dictator tell or anything like that. It never was. And I'm kind of shocked that some people seem to think that was always the goal. To me, it just never was. Rather, again, I think I said this in the very beginning, humans are wonderful. You know, I'd hate to have robots taking over from us, and I don't think that's gonna happen. There's just too much good about human nature and about what humans are, you know, able to produce that are shockingly beautiful and creative and inspiring. We need to support all that.

• 57:15Speaker 03

  Now, the issue, though, is that, the flip side, is that we are not, we're far from perfect. People really hurt a lot of people, and they are being empowered to do yet more of it. And we are very narrow-minded. We also don't understand, like, you know, people hurt other people often because they don't understand their motivations. They got a misunderstanding. Well, how many wars were created because someone didn't understand the intentions of the other side? And they said, "Well, let's just proactively, let's just bomb them." That's just all the time. That's how humans act and think. What's missing there is an appreciation of uncertainty and information signaling and sort of eventually game theory arose to help people think it through a little bit, but it's still extremely rough.

• 57:56Speaker 03

  And if you look at our political system, you know, and aged, you know, charlatan leading a country, you know, this is our optimized human system for making decisions that, you know, of the highest kind. There's so much room for improvement of the human being. And democracy's gotta be the way, but democracies right now are a few aged people sitting in various rotundas. And and various capitals, you know, not knowing what they're talking about mostly. We have a very broken human system in many, many domains. We have a few that are map. I think the universities are pretty good, and I think a lot of companies are pretty good. And a lot of human associations of various kinds, various skills are pretty good. Uh, but we have so many broken ones.

• 58:41Speaker 03

  And so to me, that's what AI is about. AI is about uh helping the things that were too hard for humans. And aiding the information flow so that humans could actually make the good decision in the moment that most of them really wanted to make. And and and and not making the bad decision that they were afraid they had to make because they didn't know enough. So there's so much to me, opportunity, if you think about it at that level. That's what AI is about to me. AI is not about this replace the human with the computer, the recursive self improvement stuff. I mean, it just feels like a metaphor. You know, we work with recursive algorithms. We work with improving algorithms. I don't see that getting out of control like, you know, a virus that somehow, you know,

• 59:22Speaker 03

  we're we're going to work with these systems and we're going to, uh, like I say, hopefully mostly focus on getting right some of the things that evolution didn't quite get right for the human being, especially at scale of seven billion. Evolution perhaps didn't prepare for that. Um, and focusing on that to me is what AI can be about. So I'm positive, I'm bullish about AI in that sense. I I and I'm appalled by the dialogue has become between the people that have all the money and want to just build something for build it sake and the people that are just anti intellectually saying it's terrible, it's gonna, it's gonna destroy all of humanity. That's the dialogue in in the public eye right now. And that's just said, so I find that so harmful. And um, and it does bother me that people that worked on it for all these years think

• 1:00:05Speaker 03

  that we've reached the end, that somehow the gradient descent is like the brain and therefore you could take multiple brains and you could fuse them together. And oh, my God, it's gonna just do uncalculable things. Such as such science fiction. It's it's whether it, you know, even true or not, it's we're not even thinking about it. What's the path? What? How do we engage younger people to do things that are actually positive? And what mechanisms are you going to talk about? What kind of education are you going to talk about? What what goals are you going to set? And the thought leaders are not talking any of that kind of language. And, you know, I think I think it's unusual for human history that thought leaders are are are are heading off in these two directions.

• 1:00:41Tim Scarfe?

  It's also complex because there are very real security and safety risks of having any autonomous software, you know, just doing things without direct human supervision. So we should say-

• 1:00:50Speaker 03

  Well, yes and no. Think about airplanes. You know, that's the classic example, but there's very, very few airplane crashes at massive scale these days. There used to be a lot when I was a kid. And it's because of the autopilots.

• 1:01:02Tim Scarfe?

  Yeah.

• 1:01:03Speaker 03

  And mostly now planes are flown by autopilots and the human can come in as need be, but it's because of that. So there's this blend of automation with human is actually the most effective way to go. Again, it's improving. Humans didn't evolve to be flying this big thing up in the air. And so you can improve upon human ability there. You put the two together, you can do something that's helpful for everybody.

• 1:01:27Tim Scarfe?

  I suppose in that case, it's quite a well-specified problem. So we want to go from A to B and here are the parameters.

• 1:01:33Speaker 03

  Yes and no. I mean, you have multiple planes in the air, you have clouds, you have, you know, change in weather patterns, you've got some person who did something stupid. You know, it's easier because yeah, up in the air, there's a lot of room. In 3D, it's a lot more room than in 2D. But in 2D, you've got all these cars floating, flying around, and you've got tens of thousands of people dying each year in each country. It's a mess at some level, even though it's very important and effective for many of us, so we do it. But a hybrid system that had a lot of autonomy with some human and so on, but you've got to think about it at the system level. Just putting a superintelligence behind the wheel of a car, dumb way to think about technology.

• 1:02:11Tim Scarfe?

  Is there any hope? I mean, I don't know what you think would be the thing that would make these folks update.

• 1:02:17Speaker 03

  I think that Ilya and others have done some great things. I mean, they built some systems that all of us are not only using, but kind of it's changing our thinking and all. And I think that's kind of what I get out of what they're saying is that I'm a builder. I'm not a, you know, you think I'm a guru and a thinker and maybe, I mean, maybe I think I am too, but maybe I'm really better as a builder and I can build things. And with the resources that are now available to you, and again, it's not just the money, it's the whole internet and the whole, you know, all the things that previous generations of people did. That, one thing that bothers me a lot about these people, not the Elon Musks or the Sam Aldmans, they're just coming in, taking the cream off the top, you know, from all this effort that people put in. And a lot of these people are, you know, rightly, not just that they wanted the credit, they're just annoyed that this is the direction that

• 1:03:02Speaker 03

  these people are now taking it. Without the appreciation of why were these people building these things? Not for you, but I had other goals in mind. So, you know, I think that, yes, these are some builders and there's some very impressive builders, but it is, I don't, and I think there's this system, we call it Silicon Valley, whatever, that these people live in and where, the more outrageous, the more far-flung, the more physics, biology-inflected, neuroscience-inflected that your language is, the more you sound like a guru. And, and people enjoy that, that posture and that activity. And it creates great amount of money.

• 1:03:45Speaker 03

  They don't care about the wealth perhaps, but it creates them, allows them to yet be more prominent. Because they can now have another company that tries some other crazy thing. And if it doesn't work, that's a sign of you had a great idea. So it's a, it's a, I wouldn't want to be in that world. And I am trying to becoming a bit of a historian. I mentioned chemical engineering, electrical engineering, but you know, you look back at the history, there were some glimmers of some of these kinds of things. But I think this level of detachment from reality is unusual for human history. This level of my crazy science fiction, 25-year-old dreams are all, that's what I'm going to pursue for the rest of my life, whatever, you know, come hell or high water.

• 1:04:25Speaker 03

  And then at some point I'll flip because I realize, oops, I didn't really have a great goal in mind at all. And what have I got here? Oh, I've just spent a lot of money and I got this thing and I don't really know what to do with it. And I'm worried about it. You know, that to me is a sign of a certain level of immaturity, frankly.

• 1:04:42Tim Scarfe?

  Circling back to, you know, you were talking about this statistical contract theory, which is when, you know, we, we have things with an information asymmetry and we model incentives. A lot of folks in the audience would have heard of game theory, right? You know, what's the difference?

• 1:04:56Speaker 03

  Oh, well, game theory is a mathematical discipline, you know, it started with von Neumann in the 20s. And it's got many, many branches to it. And it's a mathematical way of thinking, really.

• 1:05:09Speaker 03

  And one way I like to think about it is that it's like F equals ma. It's a set of, it'll make predictions. Okay, so if I write down a game, just like I wrote down F equals ma in some coordinate system, I can now predict what will happen. And in the case of F equals ma, I integrate a differential equation. In the case of game theory, I write down the game and I calculate the Nash equilibrium, or the correlated equilibrium, or some other equilibrium concept. And I say, here's what will happen in nature. Because my little mathematical model has the kind of appropriate, captures the appropriate ingredients. And for F equals ma, yeah, the thing follows a parabolic, you know, curve. It means the theory is right. And then Einstein says it's not quite right, and he makes a better one. And in game theory, same thing.

• 1:05:54Speaker 03

  You look at, okay, do those equilibrium actually characterize how systems and organizations and people behave? Sometimes yes, sometimes no. But those aren't, that's not the end all. So there's all kinds of other equilibrium, Stackelberg equilibrium, and sequential equilibrium, and various kinds of figures of merit, you know, various social welfare constructs, various regret constructs, and all sorts of things. It's a whole huge field of its own. And let's think about it eventually kind of being as big as physics, because it's all about strategic interactions and so on. And, you know, not molecular interactions. But, but now you can also ask the inverse question. In physics, the inverse question would be, I want to build a bridge.

• 1:06:36Speaker 03

  So my goal is not just to see if something follows a parabolic path or something. I want that bridge to stand up. So I invert F equals ma. Okay, I go from the goal back to the design that would ensure that that thing stood up. All right, and so most engineering fields are inverse problems. They go from the goal back to the design. Whereas the forward direction is science. You say, here's the, here's the setup, here's the prediction. And is the prediction realized or not? So, okay, yes, it is, that means the model must be good. So what's the inverse of game theory? Okay, well, it's outside of economics, not talked about perhaps that much. Game theory sounds like it's sort of everything. Well, the inverse of game theory is what's called mechanism design.

• 1:07:19Speaker 03

  And mechanism design says, oh, I want a certain outcome in the world. That this person gets paid, that the wealth is divided equally, that, you know, there's some fairness or some market that's created. What game do I design so that that outcome is realized? So I'm the designer of the game. I'm not taking the game as given and then looking at what it predicts. Mechanism design has got many pieces too. I work in contract theory. That's a part of mechanism design. It says, what if I have two entities interacting? They're not symmetric. One knows more than the other and they have to interact each other. That's, that's contract theory. Auction theory is another part of mechanism design. Where I've got a bunch of people coming in and I think of them as symmetric. I don't know who's got more money than who wants to bid more than others.

• 1:08:01Speaker 03

  But I have this mechanism called an auction that reveals their value. And the outcome is that the person who wanted the painting the most got it. That's that desire, that's one desired outcome. Anyway, long story short, game theory is a super rich, not so old discipline. You know, a hundred years now. That's, that's continually evolved and to continue to supply all kinds of algorithmic ideas for those of us who are in the business. So I'm, I've been mostly a statistician in my career, kind of worried about uncertainty and probabilities and decision making and uncertainty. But when I go to equilibria and games and, or economic ideas, then I gate theory is kind of part and parcel of the thinking.

• 1:08:38Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now, then this is a job for Indeed sponsored jobs.

• 1:09:08Speaker 04

  Low energy, unfocused, foggy, you might be dehydrated. Gatorade Zero powders have been scientifically designed to help improve hydration balance in the body with a clinically proven electrolyte blend and no artificial colors or flavors. Great tasting hydration, no matter where you're going. Shop now online or at retailers nationwide. Gatorade hydrates better than water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 1:09:38Tim Scarfe?

  You've said that we need to be thinking about, I mean, we've spoken about incentives, we've spoken about collectives. The other big one is uncertainty quantification. Now, there's this wonderful field in machine learning called conformal prediction. It was invented by my professor at university, Volodya Vork. And, you know, so we learned about the transductive confidence machine.

• 1:09:58Speaker 03

  Oh, nice, yeah.

• 1:09:59Tim Scarfe?

  These measures of strangeness. So that would be like the distance from a hyperplane on an SVM. And you can basically kind of, you know, I suppose calculate something like a p-value, right, and have a confidence region.

• 1:10:10Speaker 03

  An e-value, actually.

• 1:10:11Tim Scarfe?

  An e-value. Go on, tell me more.

• 1:10:13Speaker 03

  Oh, well, I don't want to get into technical talk about e-values, but just now we're kind of, you know, Vladimir is fantastic. And I don't know what he thinks of himself as, but I think of as a statistician, you know, with game theory background too. And, you know, he's in the school of like the Phil Davids in the world and the David Blackwells who spilled out of statistics to do all these other things. And so, yeah, classically, p-values were just kind of a one-shot quantity that statisticians would talk about. That was like Fisher, that said, I've got a model of what's going to happen in the world. It gives a probability distribution on the outcomes. Some outcome arrives, it looks very improbable under that model, the model must be wrong. That's kind of the p-value. And so the p-value is the tail probability. The problem is if you do that repeatedly, and you look at maybe the

• 1:10:58Speaker 03

  smallest p-value along the way, that's called p-hacking, and that gives you wrong answers, mathematically and then in practice. So e-values are different. It's an expectation of some non-negative random variable, or a non-negative supermartin-gale in more generality. So you're watching this evidence kind of accruing, and you make sure the expectation of that evidence is less than or equal to one at each step. And then you can think about a multiplicative kind of evidence gathering. That if it's always an expectation less or equal to one, then it'll kind of stay below one. And if it's non-negative, it'll just kind of decay away. So under the null hypothesis, I've got this stochastic process which is kind of decaying away. Well, I can look at that at any time and sort of assert that it's decaying away. And I can look at it repeatedly and keep asserting that. And

• 1:11:43Speaker 03

  I can have control. There's something called Vils' inequality that Vladimir And others have exploited that says that can be controlled over the entire path of this thing. So now we can do statistics in a new way. It's called anytime inference. We can peek, we can change, we can gather new data, we can do this and up the way they very liberating. And Vladimir is, uh, yeah, one of the leaders of that and E value is, uh, is a, one of those Martin Gales stopped at a particular time. By the optional stopping theorem, you could stop it whenever you want. So that has opened up a lot of connections. In fact, our statistical contract theory.

• 1:12:19Speaker 03

  What is a contract memory was like services and prices. Well, the services are like evidence gathering.

• 1:12:27Speaker 03

  And, um, and then the price also is, is part of the, it's, it's a random variable. And, um, it turns out that we can have an incentive and compatibility in contract land. If and only if E value in statistics land. So there's a nice tight connection between game theoretic probability and the theory of incentives. So I, to me, uncertainty quantification is rarely just here's an error bar. That's kind of classical statistics and it's more what the context is here. The context might be a contract or it might be some other evidence gathering mechanism. And this, this way of thinking opens you up to a broader class of, of, of, of, of evidence gathering.

• 1:13:11Tim Scarfe?

  Very cool. And I should say in your paper, you have this figure of a triangle, which we'll put on the screen now, but you're kind of saying there's you know, there's, there's economics and there's computer science and there's statistics.

• 1:13:20Speaker 03

  Well, these are, these are thinking styles. I don't even, even call them by those disciplines. So there, there was, there was a paper by Jeanette Wing and, you know, a couple few decades ago, talking about computational thinking. So it says, oh, computer science is, um, developed these thinking styles that are more abstract than just computers. It's, it's, uh, like modularity and abstractions and APIs and all that. And why don't we teach everybody in all the sciences and all the disciplines to do computational thinking? I think that's totally right on. That's, that's great. But lots of algorithms don't come about from those kind of computer science principles. They come about from thinking about inferential uncertainty and how do I gather data to make predictions about things that don't yet exist and think about incentives. How do I make

• 1:14:01Speaker 03

  sure that incentives are in place? And I called those two kinds of thinking, one of them inferential thinking, so not just statistics, a lot of fields have inference in them. And then economic thinking, it's not just economics, it's social scientists of all kinds and legal scholars and so on. When you put those three together, you get a pretty good platform for training of the next generation and a pretty good platform for problem solving of the kinds that we've been talking about this entire time. Just one of the fields, just computational algorithms and optimization, that kind of gives us LOMs, fine, great, but doesn't give us any of the context around the LOM. The incentives kind of gives you the whole thing we've been talking about. And then statistics to me is critical. It thinks about what kind

• 1:14:44Speaker 03

  of errors I'm going to make, how to make sure the data is controlled so I don't make the errors. And we put the three together, yeah, they also bring some partners. The economists talk to the behavioral psychologists, the computer scientists talk to the physics people or whatever, the statisticians talk to the legal people, whatever. There's a whole sub-communities that come together. So to me, if you put on this triangle there and you think of it as a rounded, it starts to become a new way to think about academia. This is the liberal arts of the era. This is the core. And now, my colleagues in the humanities might disagree, the core is still the humanities, but I just don't think it's

• 1:15:25Speaker 03

  touching the core intellectual issues of the era, which is about data and about compute and all. But I want to put the ingredients in place that those things are thought about in a societally responsible way.

• 1:15:38Tim Scarfe?

  But could you bring this to life? So you famously spoke about here's a language model, and I'm going to ask it, how confident are you about the answer? And it tends to be quite modal, so it'll either be like 1, 0, or nought. And what's the difference? Why does the language model not really have any idea about its confidence?

• 1:15:56Speaker 03

  You should ask the language model builders, because all they're doing is predicting the next word, and there's not any thinking about uncertainty quantification in doing that. And you can graft in ideas, but they're dubious. They're often putting up dubious prior in or. And, and so you can, you go to the statistician and, and that's what, you know, people have done. And they've said, okay, I could just treat it as a black box and I can put conformal prediction around it. It's a nice method, doesn't require a lot of assumptions. So yes, that's true, but it makes a lot of, you know, it's not, there's an exchangeability assumption. The data, you know, if you scramble it, you get, it's the same.

• 1:16:36Speaker 03

  And, and so while I think all of that's really crucial and important, I, I tend to think more about the broader context. So, um, you know, I gave an example in that article that you mentioned, um, of, um, you know, a, a duck who goes to a lake and the, the, this is a statistician duck. So it's kind of calculated that over the last year, there tends to be, you know, twice as much grain on that side of the lake than on this side, two to one ratio. All right. So now the next day I need to decide on the duck, uh, which side of the lake I go to. And the, the Bayesian duck, um, who has those probabilities would then do the maximal expected value. And they go to the left side of the lake with probability one because they're all right. But the actual ducks don't do that.

• 1:17:19Speaker 03

  They go to probably two thirds to that side of the lake and one third of the other side. They're hedging. But, but it's not just a hedging thing. Hedging would just do occasionally going to the, the other side of the lake. They're actually getting the right ratio. And, and so the explanation is that you weren't thinking about the context, right? Of this uncertainty. Okay. It's not just you, the individual duck, probably you evolved in a world where there are many ducks. And if all the ducks went to the same side of the lake, obviously you've missed out on a resource. And so is there an algorithm that allows many ducks to cooperate here? Um, well, if they all have that same uncertainty, then they can sample with probably two thirds and go to this side versus one third. And that's actually a Nash equilibrium of the bigger system. All right. So the, the right way to think about uncertainty there is that in the context of the population,

• 1:18:02Speaker 03

  what should be, how should I use my uncertainty? Uh, another kind of uncertainty, that's kind of the economic side. Another uncertainty in economics is the one I've alluded to, information asymmetry. You know, things I don't know and you have expertise I don't know about, but we're going to work together and I'll maybe give you a contract. A menu of options. But even if I interrupt you for a while, I still might not know. You'll, there's things you're gonna know that you're not gonna give away to me. And maybe you'll hedge, you know, you'll lie a little bit, so I don't know about that. That'll never, that's not just sampling. That's a different kind of uncertainty. Okay. And then finally, there's what I like to call providence. You know, that's more like a database kind of uncertainty. If you, if I wanna do a medical operation and you're a doctor and you look at the data for people like me,

• 1:18:46Speaker 03

  here's the, you know, if you do the operation this way, the probability of survival versus this. And I look at that, I say, great, but now you tell me, oh, that data was gathered 10 years ago. And I'm gonna say, okay, my confidence in it should go up. All right, well, classical statistics, you know, could talk about that. In fact, I'd be more of a Bayesian to think about that, but it doesn't. It's just sort of the data is the data. And it should be in a bigger system that its data is flowing around. It should always be tagged with metadata about how old it is. And that should be quantitatively brought into the uncertainty quantification. We're not doing anything like that right now. And so the poor L L Ms, you know, which are basically doing none of the above, have to strike out a little bit in all these directions if they're gonna start to do like what humans do.

• 1:19:28Speaker 03

  We are pretty good at getting these little bit of providence. Oh, it's old data, I discount that. We get a little bit of context. Oh, there's a social environment here, you know, I should just do the same thing, should randomize. Oh, there's some sampling uncertainty, you know, and so on. We put all that together almost seamlessly. And then we do this in a social context where if I don't know how to get from here to the other side of town, I will ask someone who looks Danish. I know something about how to gather more data and so on. So the poor L L M has none of the above. And so what should it say when you ask, how sure are you? And all it's doing, to the best of my knowledge, is that it's just, well, in the past, someone asked a human on the internet, how sure are you of that equation you just wrote down? And someone said something, oh, I'm very sure because of this or that.

• 1:20:13Speaker 03

  And I think it just mimics that, those kind of assertions, but that's not reasoning under uncertainty.

• 1:20:18Tim Scarfe?

  And if we did have epistemic, you know, quantification. What would be the the main uplift from that? Is it about I know I don't know something, so I'm going to kind of lean in and try and do more epistemic foraging in that area?

• 1:20:32Speaker 03

  Well, again, I think we're now in statistics land. You know, the statisticians are all about what species are present on the island. Have I sampled enough to know that there's not a new species? That's these are classical areas of statistics. Optimal experiment design, you know, for that subpopulation, I don't have enough data. I'm making a bad inference and data collecting in the context of inference and the collecting in the context of, you know, making assertions and doing that repeatedly, that's what statistics has long focused on. So I think give them credit for handling a kind of active form of uncertainty reduction. But again, for me, uncertainty reduction in the in the large comes about from much broader sets of of components like a market.

• 1:21:15Speaker 03

  Like if I if I I use example in the paper where I'm, you know, I want to have a restaurant like this or pizza and I need tomatoes. And so if I had to forage for tomatoes every day, that would be pretty uncertain whether I would have pizza that evening. But because there exists a market where someone else did the foraging, there's a stable amount of tomatoes every day. I can count and build my restaurant, assuming that that's true, that my certainty for finding tomatoes went down. Therefore, I can build on top of that and do other things. Markets mitigate uncertainty and they they don't do it because someone designed an optimal experiment design or, you know, ran or did some multi armed bandit, you know, not directly. But because the market did try various things out, there's incentives for people to explore and exploit.

• 1:21:58Tim Scarfe?

  Professor Jordan, it's been an honor having you on the show. Thank you so much.

• 1:22:01Speaker 03

  All right. It's been my pleasure. I've enjoyed talking to you.

• 1:22:04Speaker 06

  Now at Kroger, your points add up to more ways to save every trip. Use them your way to take dollars off groceries at checkout or save at the pump. You can even save up to $35 on your next fill up. Same points, more choices. And with low prices on fresh, quality food, plus digital and personalized offers tailored to how you shop, your dollar goes further. Redeem your points online or in the app. Kroger, fresh for everyone. Restrictions apply. See site or store for details. When you spend enough time around great companies, you notice that the best ones treat trust as a growth strategy, not a box to check. Vanta helps companies build and prove trust from day one.

• 1:22:48Speaker 06

  Its Agentic Trust Platform connects compliance, risk, and trust at enterprise scale. Vanta delivers a 526% return on investment over three years and pays for itself in just three months. Learn why over 16,000 of the fastest-growing companies use Vanta at vanta.com.