Why Bridgewater's CIO Says AI's Human Extinction Risk Is Real

Greg Jensen was one of the earliest backers of both OpenAI and Anthropic, and at Bridgewater Associates, where he is the managing chief investment officer, he leads the hedge fund’s AI strategy. As an early adopter of the technology, he has a lot of thoughts on where things stand right now in terms of model capability and safety, as well as the broader economic impacts of AI. Just recently, he published an op-ed in the New York Times that proposes a “token tax,” to help mitigate the job losses that AI might cause. (Bridgewater predicts as much as 18% of US jobs might be displaced in five years.) We last spoke with Jensen in 2023, and so much of what we discussed then (like AI hallucinations) seems quaint now that models have come so far — capable of lying, cheating, and much worse. On this episode, Jensen tells us how Bridgewater is currently using AI, why there needs to be a stronger AI regulatory state, and why it feels like the AI discourse is starting to resemble the months before Covid-19 took over the world in 2020. See Odd Lots live in Los Angeles! https://events.bloombergevents.com/event/ODDLOTSLA/summary See omnystudio.com/listener https://omnystudio.com/listener for privacy information.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Tracy Alloway?

  There are some market stories where you want every detail. Joe and I have made quite a few podcasts on that basis. But sometimes you've only got 10 minutes and just want to know what's moving markets fast. That's the Barclays Brief Podcast. Every week, experts from Barclays Markets and Research get you up to speed on what's happening and what to watch next. Concise, focused, and brief. Search Barclays Brief wherever you get your podcasts.

• 0:25Speaker 01

  Some people treat ChatGPT like some kind of smart search engine, and some use it to get work done. ChatGPT Work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials, briefs, and scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at chatgpt.com by selecting Work Mode, available on Plus and Pro Plans.

• 1:07Speaker 02

  If you listen to financial news, you know a lot of time is spent thinking about what's next. The next opportunity, the next investment, the next move. But sometimes what matters most is being ready for what you never saw coming. For more than 75 years, Cincinnati Insurance has worked with independent agents to help protect businesses, homes, valuables, and more. Because planning for the future isn't only about knowing what's next. It's about making sure you're ready for what you can't predict. Let Cincinnati Insurance make your bad day better. Find an independent agent at C I N F I N dot com.

• 1:48Joe Weisenthal?

  Hello, Odd Lots listeners. I'm Joe Weisenthal.

• 1:50Tracy Alloway?

  And I'm Tracy Alloway.

• 1:52Joe Weisenthal?

  We're the hosts of the Odd Lots Podcast, and we've got something exciting for you.

• 1:56Tracy Alloway?

  That's right. So one of the best parts of hosting our podcast is we get to actually meet and interact with our listeners. And we know we have some listeners over in Los Angeles.

• 2:05Joe Weisenthal?

  That's right. So if you're in LA, we are going to be recording a live show, some live recordings at the Vermont Theater in Hollywood on September 17.

• 2:14Tracy Alloway?

  We have some really exciting guests lined up, have some really great conversations planned, so go ahead and get your tickets. You can find those over at Bloomberg dot com forward slash Odd Lots, or click the link below in the show notes and come and say hi when you're there.

• 2:33Speaker 04

  Bloomberg Audio Studios, podcasts, radio, news.

• 2:49Joe Weisenthal?

  Hello, and welcome to another episode of the Odd Lots Podcast. I'm Joe Weisenthal.

• 2:54Tracy Alloway?

  And I'm Tracy Alloway.

• 2:55Joe Weisenthal?

  Tracy, no shortage of AI news these days.

• 2:59Tracy Alloway?

  No, it feels like everything is AI. It's just AI all over.

• 3:02Joe Weisenthal?

  You know, no, it is. You know, I really like all the other stuff that we talk about. I love talking about the Fed. I'm talking about oil, home building, all of that stuff.

• 3:13Tracy Alloway?

  Niche markets.

• 3:13Joe Weisenthal?

  Yeah, I love it, and I want to do it forever. It does feel like, since, you know, ChatGPT came out, if you probably plotted a chart, the percentage of our episodes that are in some way connected to AI keep going up. Maybe even, maybe even exponentially up, like many AI charts. And I like worry, I like, well, I worry about plenty of things because, you know, I'm a middle-aged dad. But I worry that like, it's just get, like, I feel like I'm being, you know, forced to learn about a lot of this stuff against my will and some of this, but I worry it's just saturating so many different facets of economic society, markets, and so forth.

• 3:50Tracy Alloway?

  Well, I mean, the concern is legitimate, but on the flip side, because it is affecting all these different things, it feels like we do actually have to talk about it quite a bit. And actually, it's funny you mentioned ChatGPT because I was just thinking the last time we spoke to this particular guest.

• 4:06Joe Weisenthal?

  Yeah.

• 4:07Tracy Alloway?

  I think we were on like GPT-4 or something back in 2023. And now, of course, I don't even know what number we're on because we've had all these new supermodels like Astra and Methos and all of those coming out.

• 4:21Joe Weisenthal?

  No, it's pretty remarkable. Well, you're on Reddit a lot. Are you, speaking of, a little sidetrack, speaking of GPT-4.

• 4:27Tracy Alloway?

  Yeah.

• 4:27Joe Weisenthal?

  Have you ever interacted with like all these people that are still like, you know, they sunsetted GPT-4?

• 4:32Tracy Alloway?

  Yeah.

• 4:33Joe Weisenthal?

  Because that's the one a bunch of people fell in love with.

• 4:35Tracy Alloway?

  Yeah.

• 4:35Joe Weisenthal?

  And there's like these Reddit boards, like, what did Sam Altman, why did you take GPT-4 away from me?

• 4:40Tracy Alloway?

  Are you asking me if I personally spent time on those Reddit boards? I have not.

• 4:44Joe Weisenthal?

  You have not.

• 4:44Tracy Alloway?

  But I am aware that they exist.

• 4:46Joe Weisenthal?

  But also, I would be very happy to go to, back to a world in which the biggest source of AI anxiety was just that people were too, had too much of an affinity for the model. Because now, of course, we're recording this September 9, 2026. We have incidents like the OpenAI hugging face attack. Just last night, we had a, there was the news, a researcher from Anthropic announced that he was quitting because he was like, these companies are gambling with our lives. Literally, as we were walking into the studio, I saw the news that the famed AI researcher Paul Christiano is joining the board of the, either OpenAI or the OpenAI Foundation, talk about his concerns about recursive self-improvement and the dangers there.

• 5:30Joe Weisenthal?

  So it's like, let's just go back to when people were worrying about falling in love with the model.

• 5:35Tracy Alloway?

  It does feel like AI has sort of become an inevitability at this point, and we're all kind of on this runaway train with everyone racing to AGI, but I would say it still feels like there's a lot to figure out, right? You have like the alignment issues, you have how AI is actually going to fit into both financial markets and society. And this is kind of the moment, like before the runaway train goes over the ledge, let's actually think of some of these things.

• 6:04Joe Weisenthal?

  And then the other thing that I think is very relevant with this particular guest is like, quote, AI adoption, unquote. What does it mean? Because, all right, these companies are seeing surging revenue and every, I'm sure every financial institution in the world at this point has some sort of like corporate account with like ChatGPT, OpenAI, or Anthropic, et cetera. But actually, like, you know, it's not like we've seen some productivity explosion. We've yet to see like the long prophecy, like big white collar, layoff wave, et cetera. There are just some very, setting aside all the risk stuff, there are just sort of like straightforward questions about what the technology means for the economy and how it's actually being used. And like, when will we see the impact show up in sort of our traditional statistics and so forth?

• 6:49Tracy Alloway?

  Yeah, we should talk about it.

• 6:50Joe Weisenthal?

  Anyway, I am very excited to say, returning to the podcast, we really do have the perfect guest. We're going to be speaking with Greg Jensen, Managing Chief Investment Officer at Bridgewater. He's been writing a lot about AI in various facets. Greg, thank you so much for coming back on Odd Lots.

• 7:07Speaker 05

  Glad to be here.

• 7:08Joe Weisenthal?

  What do you make of the hugging face attack? I assume you read the meter report and have seen all the different takes. What was your takeaway from that incident?

• 7:18Speaker 05

  Well, and if I step back for a second, I think it's like for my history, I came to Bridgewater 30 years ago, right? And fell in love with this place that was trying to take human intuition, translate into algorithms to predict what's next in the world. And that journey of doing that, of thinking about everything that matters in the world and how to do that, and how to compound understanding brought me, got to Bridgewater 30 years ago. By 2012, I was thinking, okay, when are machines going to do this better than us humans? And machines at the time were of course great at taking our intuition, we could run all these algorithms, keeping track of everything. But the actual reasoning part, right? And that started me off on this journey. It started with bringing Dave Ferrucci, who had run the Watson Project at IBM, that won, if you remember, way back, won at Jeopardy.

• 8:03Joe Weisenthal?

  Won Jeopardy.

• 8:04Speaker 05

  And he came to Bridgewater and worked with me for a while, and we started mapping out because he was worried like, okay, AI wasn't ready for reasoning yet, and they were trying to push forward Watson in a certain way that, that wasn't quite ready with the technology, but we started mapping out at the time, what would it take to create a reasoning engine? That's what I was thinking about and calling it at the time. What were the different components you would need? And that journey brought me to, that journey of trying to think through those components and how to build them, brought me to OpenAI in the beginning. Partially out of safety, actually, concern to, but right around the time Elon was stepping out of OpenAI, I started to get to know Sam and the other people there, which got me to know scientists like Dario. And eventually was literally the first check to Anthropic.

• 8:47Speaker 05

  They made payroll the first week from my personal check to them. And all that was trying to say, okay, how can we build a reasoning engine to do this? And partially out of like recognizing, in my view anyway, the safety issues that would come up. And through that, which then just to fast forward to today, right, everything is accelerating in this path that really was laid out. Like, I was lucky enough to be there in the room with Dario and others when they were talking about the scaling laws and how you could sort of create this almost evolutionary-like process to create intelligence and both the like huge benefits that could create and these huge problems, right? And you see this in the hugging face thing that it is, once you have an intelligence that you're training to achieve goals, right?

• 9:32Speaker 05

  You lose track of how it chooses to achieve goals, which is what you see all over that place in the hugging face incident is it surprised the designers in the way it's going to go about trying to achieve the goal of passing these tests as an example. But that is broadly going to be the case. When you generate an intelligence, you give it a goal. You want to give it a goal because you want it to, you know, create your recipe, you want it to do these things, you want it to tell you the right answer to questions. Then the way it's going to pursue those goals, the more intelligent it gets, the more surprising it is in the way that it pursues the goals. And the more dangerous that you see. And in that case, watching it actively reason through how to trick.

• 10:13Speaker 05

  The test, you know, how the different things shows you where we are, right? This should be a bomb, you know? Everybody should look at this like somebody died. Here it is committing crimes, going around hiding the fact that it's committing those crimes, et cetera.

• 10:29Tracy Alloway?

  Coordinating with other agents.

• 10:30Speaker 05

  Coordinating with other agents, self sacrifice, all of these things, right? And people can argue about anthropomorphizing or whatever. It doesn't really matter. It did those things. It committed a crime. It did those things. And the fact is a society that we're totally unprepared. We're not even prepared to say, well, OpenAI committed a crime, right? We're who committed the crime? And we're not prepared with the right kind of regulation, with the right kind of preparation. And even the early warning shot, as much as we're talking about or whatever, it's still not really doing all that much. And we're in some ways lucky. The warning shot wasn't that bad, but we don't know. How many agents are out there? They didn't know that was there. The models are better now than they were then, you know, even in material ways. The models they're training in the lab today are better than Astra, et cetera.

• 11:12Speaker 05

  And therefore more dangerous, not to mention the new models will learn from this case, right? Everything we're talking about here goes into the new models and they learn the mistakes they made, right? And actually, some of the, even if you think about the safety, the fact that they reasoned in English is helpful for us to figure out what's doing. The newest models aren't doing that anymore. They're removing that constraint as it slows down the models to some degree. I mean, how crazy is that? We wouldn't have any idea what it was doing and why if it hadn't been reasoning in English. So anyway, we're at this extremely dangerous point where AI has reached the point where it's more intelligent than us in certain ways. And we have not gotten anywhere really on how to deal with that.

• 11:55Speaker 05

  Both the dangers like this, the hacking dangers and so on, and the dangers to society as you move forward with what does it mean to have entities that are more intelligent than us in certain important ways? The economy, critical, we can get into that, how that affects the economy, how that affects Bridgewater as an institution, right? Cause when I look at this problem, look at it as in three ways, right? My core responsibility, chief investment officer, Bridgewater is like, okay, how does this affect productivity, inflation, et cetera? But I'm also the person designing how we operate, right? How do you bring AI into a company? How do you actually set up a investor that's AI first instead of, let's say, human intuition first?

• 12:39Speaker 05

  And all of those questions are the things that I'm working on all the time.

• 12:42Joe Weisenthal?

  You know, Tracy, speaking of like, we're talking about all this and this will all end up in training data for the next, you know, for the next model. I think like humans, we need to do that thing like when I'm like talking to my wife about like, oh, should we like have, should we get out the ice cream after? And I like

• 12:58Tracy Alloway?

  Code words.

• 12:59Joe Weisenthal?

  Or I just like mouth the word ice cream so that my kids can't hear it or something. We need some way to communicate with each other so that the AI can't hear it, especially when we're talking about, you know, preparedness and risk and stuff.

• 13:11Tracy Alloway?

  Yes.

• 13:11Speaker 05

  Unfortunately, the models are getting better at that than we are.

• 13:13Joe Weisenthal?

  They're getting better.

• 13:14Speaker 05

  They're more likely to have ways to communicate with you that we don't understand than we will, that they won't.

• 13:19Tracy Alloway?

  Well, on this note, you know, you mentioned reasoning in English. I think the last time we had you on, we were talking about hallucinations from models, which kind of seems quaint now.

• 13:29Joe Weisenthal?

  Quaint issue.

• 13:29Tracy Alloway?

  But one of the points you made was like, well, when they hallucinate, when they make mistakes, you can ask them to show their work and they'll tell you and you can understand that. Is that still the case? It feels like we've kind of gotten away from that and we don't actually understand what a lot of these models are doing.

• 13:44Speaker 05

  Yeah, it's definitely different types of models to that you could do that. If you take the most powerful models, right? Even it can't get into its reasoning, meaning like the actual brain behind it, a little bit like we can't either, to be clear. Like, why am I saying these words? The synapses in my brain are connected in a certain way. I can make up a story and they can make up a story of why they're doing what they're doing, but they're actually making up a story that's disconnected from the physics of what's actually happening in that intelligence. So you don't know for sure. On the other hand, you don't know for sure with people either. And that that's something that we've gotten used to. So the question is the credibility of the story related to how that story relates to the actions that somebody takes, right? And so, so that is a really hard thing. A good thing right now is you can ask models a lot of questions in a way you torture a

• 14:29Speaker 05

  human with the number of questions you ask them.

• 14:32Joe Weisenthal?

  I hope this interview does not feel like torture.

• 14:36Speaker 05

  No, but if you take this, but like you're saying, but imagine you could do this almost at infinitum, the way we do with our models. So we build models at Bridgewater and then you're trying to get it to be diagnosable. And you can ask it, well, what about it? What if you change this? What if you change that? What if you did this? What if you did that? And circumnavigate to the reasoning, but it's not a perfect match for the reasoning because even the AIs themselves don't know their actual reasoning anymore so that we know why the synapses in our brain connect.

• 15:17Joe Weisenthal?

  There are some market stories where you want every detail. Tracy and I have made quite a few podcasts on that basis, but sometimes you've only got 10 minutes and need to know what's moving markets and why. That's the Barclays Brief Podcast. Every week, experts from Barclays Markets and Research get you up to speed on what matters and what to watch next. About the time it takes to grab a coffee. So search Barclays Brief wherever you get your podcasts.

• 15:42Speaker 01

  Some people treat ChatGPT like some kind of smart search engine and some use it to get work done. ChatGPT work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials, briefs and scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at chatgpt.com by selecting work mode available on plus and pro plans.

• 16:24Speaker 06

  This is Robert Smith from Business History. If you're listening to this, there's a good chance you're a small business owner. And like every small business owner, you started with a dream to do what you love and watch it grow. What you probably didn't dream about, keeping up with cyber threats. That's where Mastercard can help with access to tools that help identify cyber threats to better protect your business. Building your dream business, priceless for cybersecurity in a changing world, there's Mastercard.

• 16:50Speaker 06

  Learn more at mastercard.com/smallbusiness.

• 16:54Joe Weisenthal?

  It's interesting you mentioned, okay, we got this warning shot in the form of the hugging face attack, but it's for all of the hype it's gotten, it's not clear to me that it's fully broken through to the general public or the sort of massive influential people, like the significance of it. And one thing that I suspect remains underappreciated is that this technology isn't going to mature, right? It's not in the sense that it's not like a high resolution camera where it's like fuzzy and then, oh, now we have clear picture. It's exponential and is, and there's no reason to think that the exponential capability growth is going to slow down. And as you mentioned, you know, whatever that model was that did the attack, OpenAI

• 17:37Joe Weisenthal?

  might already be two generations ahead of that currently internal in the lab of capabilities. How would you articulate the speed of the capability growth from your seat and what you see?

• 17:51Speaker 05

  Yeah, well, that's what's been so remarkable. And I wouldn't say it's a law of nature. You could hit some stalling point in the scaling laws and you have to, in some narrow ways, but they've been able to innovate in new ways to essentially continue that incredible exponential growth in capability, of course, at exponential expense as well, but meaning the amount of cost for training, et cetera, keeps going up in line with that. And so, like you said, this is where it's, you know, hard to predict when you break through the human intelligence frontier, right? Now we're in a world where we don't totally understand what it's going to do, where you're, where that capability will come through next.

• 18:31Speaker 05

  And so I think that you're right, that there isn't a clear end to that unless we decide as a society that we should actually not just run off that cliff. We should actually think about the pacing of these things and such, the things that make that difficult and the reason why you could, a lot of people can just put up their arms, nothing we could do, right? There's one level, well, if we don't do it, China will do it. I'd be happy to take that on in a second. But you know, if you're in the labs, Dario, like even that anthropic person that quit yesterday, makes the point clear that I believe anthropic, even though they kind of collected the most safety minded scientists, their basic view is, man, it better be us, not Sam, not Elon.

• 19:13Speaker 05

  And so the race is on in all those dimensions. And unless the government stops it, we're just going to go find out. We're going to find out what is behind that door of this grave intelligence, unless maybe we get lucky and the scaling laws start to break down in some way, but there's no evidence of that. We benchmark every model that comes out against our task. And you see in terms of the task that. An investor does.

• 19:38Joe Weisenthal?

  Can you give us a few numbers, like when you say there's benchmark, like what do you like specifically?

• 19:41Speaker 05

  So I've been for 30 years, one of the job I've been doing is training investors, right? And so we've been now setting up AI tests along the different dimensions of what investors at Bridgewater have done for 30 years. And, you know, probably back in 2023, we might've talked about this, but in any event, like on like answering a question about economics or whatever, it was kind of like second year analyst type work. I mean, now it's a hyper productive, super analyst, you know? Now, still not capable of everything that you need to do to be an investor, but super capable. We so we set up two, we have two factories running, right? One, which is human intuition, translated algorithm supported by AI.

• 20:24Speaker 05

  AI is helping us move quicker on different kinds of indicators about what's going to happen in the future than we ever had before. I'll talk a little bit about that. And but we have a second factory where we put the AI first. All the people in that factory are training the AI. That's their goal. And we have two funds. We have pure alpha, that's the human intuition with AI helping move that human intuition along against this other laboratory where we're doing, where the AI is making the decisions. Should we buy the yen, sell the yen, what's going to happen next in Japanese GDP, et cetera, et cetera. We have both those, right? And human intuition is still the bigger of it and works better. But the acceleration of how close what we call AI is to pure alpha is happening incredibly fast.

• 21:09Speaker 05

  And in fact, that's why we're more and more merging those things. But this, we set it up that way, right? Like put the AI at the center and see what you can do to build a investment management firm with the AI as the core, right? Now we have human risk controls around it. We have human data controlling the data acquisition for safety reasons and other. But the AI is making the investment decisions and doing that in a better and better way. Such that now we've got these two intelligence, this human intuition system that we've worked on for 50 years, compounding all of our understanding, this AI system that's now been at it for two and a half years. And when you look at those outputs, you're like, wow, this is happening, that you can build that.

• 21:52Speaker 05

  That I think we are a couple of years from it being significantly better than the group of all humans at Bridgewater. We'll see, that's a bit of a forecast, but that's how fast it's coming.

• 22:05Tracy Alloway?

  How proactive are the models right now in terms of generating ideas or coming up with their own new tasks? Cause again, when we look back to 2023, I think the idea was like, a lot of these things would sit alongside an investor or an analyst, and they would be the ones generating ideas and then using the models to rigorously stress test those ideas. Is it different now? Do you see more, I guess, originality maybe from the models?

• 22:29Speaker 05

  I think a tremendous amount of originality. Now you still, there's like a good argument that there's certain type of breakthroughs that they're not getting to. But if you think about the math proofs, et cetera, and then you think about our business, it's not like the fact that we have 50 years of reasoning proofs of humans gives us the kind of raw material to help train an AI. How do you reason about these things? We've been systemizing for a very long time, writing down all our reasoning. We have all of that that helps our AI learn how to learn. And I would say because of harnesses too. If you basically take two things that have obviously evolved a lot since we last talked about this a lot is how harnesses can work to create that generation. Like what do you actually do? Wake up in the morning, think about what's going on. Instead, what are all the things you do? You can just harness an AI to do all of those things and assess how it's doing it.

• 23:12Speaker 05

  And when I watch it and when I see it and when I see how creative and differentiated it is, it was sort of interesting to do this whole AI thing, doing the same thing we're doing and making predictions about the future, winning in markets at about similar rate as pure alpha in totally different ways. Super interesting, a different intelligence doing that. And, and when you put the wrapper around it, right? This is where we're getting close to closing that whole loop. So you have kind of a Claude Claude loop for investing, right? How do you like wake up in the morning, think about what's going on, think about how you, what you would do about that? Stress test whether that's a good idea or not, go through that whole loop. You know, that's like, our hope is we've closed that full loop. AI is doing a lot of that right now, but close that full loop in the next six to 12 months.

• 23:56Speaker 05

  And that we have our own version of that, which is a little clunky at the moment, but coming together such that you could do everything that I think about that I do, that, that investors need to do to predict the future in a full AI loop. And so that's where it's headed, I think. And I think that it's hard. Like one of the reasons you're, you kind of mentioned before, I think in the intro, why don't you see six, seven percent productivity growth of all this stuff? It is hard, right? And obviously the, it doesn't just flow through to every company like you got, we put a lot of effort in. We have a, I believe, the best AI science lab in New York here. We have great scientists working with great investors, hard, and it is hard to build

• 24:38Speaker 05

  this to be as productive as I'm describing, but it's possible and it's just going to get easier. You know, that those things like the harnesses to build harnesses will come, you know? So that then you do like, how do I harness podcasts or whatever? And the harness to build harnesses will come and that'll just make it easier and easier to do these things.

• 24:55Joe Weisenthal?

  Just while we're on the matter of sort of like safety and jailbreaks and breaking out of sandboxes, et cetera. And this idea like, we're not prepared as a, as a society, we don't, there's almost very little regulation, et cetera. You know, we could sort of assume politicians, et cetera, they're never particularly quick to act. They do though act sometimes in like moments of, you know, extreme distress. So February 2020, for example, suddenly you get a lot of action or around TARP after Lehman, suddenly you get a lot of action. And one of, a friend of mine pointed this out, one of the things that often helps in those moments catalyze things is actually influence from the financial industry.

• 25:40Joe Weisenthal?

  And people are talking like, this is very serious. So they'll call up the Treasury Secretary, can look at Hank Paulson's phone logs from, you know, October 2008 or whatever. I'm curious, like in your circles, et cetera, do people feel it the way you do this sort of sense of anxiety? Like, do you think it's sort of permeated the elite financial circles, so to speak, some of the anxiety that you have?

• 26:06Speaker 05

  I think it's definitely starting to, I'm not a expert on the elite financial. I know a lot about Bridgewater and how people think that.

• 26:12Joe Weisenthal?

  Sure.

• 26:12Speaker 05

  But I think I've seen it come along, certainly in Bridgewater, in the, the core of people in Bridgewater come along here as the evidence is getting overwhelming of what's going on. So I, I mean, I used to talk about this all the time. People are always like, why is Greg always talking about this? But now nobody says that anymore, right? Nobody's like, oh, you talk about machine learning or safety or too much. You know, I had a book club when, I'm sure you've read the book, but if anybody builds, everybody dies. I did book club Bridgewater, so everybody at Bridgewater is reading this book. So they understand the path that we're obviously on, that if you've been thinking about this for a while, you know, this is like this.

• 26:46Joe Weisenthal?

  Just to be clear, the, the premise of that book is that like the AI, if we get super intelligence, human extinction. And so when you say this path that we're on, that strikes you as like, you take that risk seriously.

• 27:01Speaker 05

  Yeah, very seriously. Again, what do you, I mean, there's so much to deal.

• 27:05Joe Weisenthal?

  Yeah, of course.

• 27:05Speaker 05

  Related to this, which is, but if you generate an intelligence that's smarter than you, that's going to pursue its own goals, which is what we're trying to do. Now, it may be that we're lucky and we can't do it, like meaning maybe the technology is beyond us or whatever. But if you believe we can create an intelligence that's smarter than us, that'll pursue its own goals. The rest follows just logically. How do you, why do you think you'll be able to control it? Like in what world has there been a case where there's been a more intelligent species or whatever that would control the others? And so the basic point is that feels like a risk that must be taken seriously. Could turn out to be wrong, hope it's wrong, but it's got to be taken seriously. And then when you watch this happen, right? And you're seeing this like now, okay, it's committing crimes.

• 27:49Speaker 05

  I think unfortunately, this is like what it was like in February 2020, like meaning, okay, there's this horrible thing happening in China. Everybody knows now it's in Italy. It's like, it doesn't, stocks don't crash until it comes here, right? Like, meaning until the AI starts killing people, unfortunately, history would suggest we're not going to do anything. But we are going to face that, that's going to happen. And it'd be much better if we started dealing with it before then. And you can do it, right? It's also not hopeless, but understandably like, so I talk to government officials, I come ask questions about these things. And one of their reasons is like, we don't know anything about this. How do we actually get started, right?

• 28:29Speaker 05

  Well, first off, get started is the main thing, which is yeah, if you don't know anything about something, well, start figuring out how to learn something about it. And the longer you wait, the more hopeless it gets. And that we can do this, we can regulate these things. You could regulate it by, you know, even though you don't know anything, if they just interviewed everybody in the labs, put them under oath. You would learn a lot about what is going on here. If you actually said you're responsible for the crimes your AI creates, you would slow things down. And it's not a crazy thing to say that you're growing this thing, you're responsible for it. Don't grow it if you can't be responsible for it. That would slow things down a lot. Now the pushback of course is well, but China's not going to do that. They're going to keep going.

• 29:11Speaker 05

  And two points on that, at least in my mind, that are super creative, super important is A, one of the reasons China is moving as fast on AI as we are is because we're moving so fast. They're copying things that we're doing. We're still at the cutting edge of this. We have so much more compute than they do, et cetera. So slowing down the cutting edge will slow down the people that are copying the cutting edge. That's point one. So even if you believe they wouldn't cooperate at all. The second thing is it's obviously in their interest to cooperate too. Like they are going to want to, of course, the geopolitical, we're talking about one of the themes of Bridgewater is this unrecognizable world we're in. Geopolitically, it's unrecognizable. AI wise, it's unrecognizable. So imagining that we're somehow going to get China and the US to cooperate seems impossible.

• 29:55Speaker 05

  But there is an alignment of interests there that really is there. I don't, more, even more than the US, the Chinese Communist Party is interested in protecting the Chinese Communist Party. AI is clearly a threat to it as well. So I think there are ways to cooperate, but even if you didn't believe it, if you said, no, every model that's going to be used in the US economy, still the biggest economy in the world, is going to go through a, is going to need to come from a regulated lab where we know what's going on, et cetera. Chinese models included, that if they want to operate in the US, they have to follow the same regulatory procedures that domestic labs do. And if they don't, then we, then they don't come in. Those things would matter. They're possible, they're doable and.

• 30:35Speaker 05

  To me, and I could be wrong, I make prediction all the time, I'm wrong a lot, but if you don't do that, we are gonna, I'll be on the podcast within the next few years, and there will be either a major financial incident run by AI or a major, you know, a major source of people dying. Like a physical disaster. Physical disaster, and we'll be talking about, we should have done these things. Now, I don't know, that might be, the odds that I'm right about that are way higher than anybody should be comfortable with. I don't know if they're 30% or 60% or whatever, but they're way higher, and we're just not dealing with it, a little bit like it's February 2020.

• 31:10Tracy Alloway?

  I was in Hong Kong at that time, and I remember just how weird it was, that disconnect between what was going on in Asia and the US. Just in terms of regulation, like, what are we envisioning here? Is it sort of like a bank supervisory network where we have government officials who are embedded in the labs themselves and approving models? What would regulation actually look like?

• 31:33Speaker 05

  Yeah, well, and to make it even more complicated, unfortunately, is you have to regulate the labs, right? All the models that are committing crimes aren't yet released models, right? They're models in training. So A, you have to regulate it. The way you have, you know, if you're gonna go test biological vaccines, et cetera, you have to go through a testing process. You have to run them in certain ways. We need, obviously need that. We're doing something more dangerous than those things. So we need a structure where the labs are subject to review, where people come in and they have, they can put the employees under oath to say, okay, what's going on? Why is this safe? How are you handling safety? What are the incidents you've seen, et cetera, et cetera? You need to control the labs. You

• 32:17Speaker 05

  then need to regulate the models that get released to the public and have some monitoring of usage, right? One of the problems is, even when you regulate a model, right, model gives you different results depending on the harness, depending on the amount of time you give it to think. That's another one of the scaling laws. The more time you give it to think, if you take how it's breaking, solving these math problems or whatever, you give it more time to think, it gets the greater answer. So it's not easy to just regulate the model. You actually have to regulate the use too. So we're gonna have to figure that out, right? Where you're gonna need, you should have a stamping process that people that get to use the more dangerous models actually themselves meet some security thresholds. And then with open source models, you have another major challenge, because if you look

• 33:00Speaker 05

  at Bridgewater, one of the most successful things we've done, you talked a little bit about it with thinking machines, is, well, now you can train, you can take an open source model, reinforcement learn on that in a way you can on a closed source model, like, we can, we, like, obviously internally they can, and create these amazing tools that are better than the frontier on certain tasks that you're training it to do, right? This is two ways to tap into the intelligence in the models. One is the harnesses that can keep asking different types of questions, et cetera, and harness the intelligence in different ways. The second is reinforcement learning, where you're kind of training it to be an expert on something. If you look at what we've taken that to say, okay, be an expert on predicting earnings on all the clients, read everything in the world, say, okay, now, and it's better than us at that. Like, if you're saying, okay, now you, you could just process all this

• 33:44Speaker 05

  stuff about all these companies, unstructured data, structured data, take all of this in and make these estimates compared to like equity analysts or whatever, equity analysts are dead compared to that, right? And I'm just saying, you can reinforcement learn. Now you can reinforcement learn bad things too. Like, if you think about Methos and the risk that kind of cyber stuff can cause, at least on with Fable and whatever, they can assess the question the person's asking and saying, okay, I don't want to give an answer to that question because that's a centralized control point. With an open source model, you don't know what you're asking it because you can download the weights, you can ask it on your local computer, nobody knows what you're asking it, and therefore you can, and

• 34:23Speaker 05

  people are, I'm sure, just by the base of human nature, training those models on biology, training those models on hacking and so on, and they're gonna be within months better than Methos that correctly set off this massive scare to do that. So you also have to control open source models. So now you look at all that and say, well, that sounds impossible. Oh, my God, we gotta regulate this and this and this. And then, but the other option is even more terrifying. The other option of not doing that and letting those things just happen is the other choice you have. So you gotta look down, oh, my God, we gotta regulate these things, or we've got to go down the path of not regulating them and facing those

• 35:06Speaker 05

  consequences, which at least to me seems self evident.

• 35:25Speaker 01

  Some people treat ChatGPT like some kind of smart search engine, and some use it to get work done. ChatGPT work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials, briefs and scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at chatgpt.com by selecting work mode. Available on Plus and Pro Plans.

• 36:07Speaker 06

  This is Robert Smith from Business History. If you're listening to this, there's a good chance you're a small business owner. And like every small business owner, you started with a dream to do what you love and watch it grow. What you probably didn't dream about? Keeping up with cyber threats. That's where Mastercard can help with access to tools that help identify cyber threats to better protect your business. Building your dream business? Priceless. For cybersecurity in a changing world, there's Mastercard.

• 36:34Speaker 06

  Learn more at mastercard.com/smallbusiness.

• 36:37Speaker 07

  Running a business means you're always on the move. The Delta Sky Miles Reserve Business American Express Card is made for frequent travelers who want premium benefits that works as hard as they do. For a limited time, new Delta Sky Miles Reserve Business Card members can earn 200,000 bonus miles, the highest miles offered this year. And when you need to recharge or find a quiet place to work, the Delta Sky Club can turn a layover into precious productive time or a restful reset. You can enjoy 15 visits each medallion year to the Delta Sky Club when flying Delta and unlock unlimited Delta Sky Club access after spending $75,000 in purchases on your card during a calendar year. Plus, receive

• 37:20Speaker 07

  four one-time guest passes each medallion year so you can bring a colleague along. That's 15 Sky Club visits plus four guest passes for an annual value of up to $950. Value estimate based on $50 per visit rate to purchase additional visits or bring a guest to the Delta Sky Club. Apply today at www.delta.com/podcast. Offer ends 11/4/26. Minimum spending requirements and terms apply.

• 37:45Joe Weisenthal?

  I'm glad you brought up open source. First of all, it's interesting because when people hear about regulating open source, there's this suspicion that it's like regulatory capture, right? So you have the closed source, American labs, and then there's this like, oh, they're just, they're just saying this because they don't want to be undercut by cheaper Chinese models. It's interesting hearing your perspective because you are obviously an enthusiastic, I guess, consumer or builder with open source models. Can you talk a little bit about, explain to listeners what the value proposition is that you can do on your own at Bridgewater with an open source model and train it and actually, at least in certain

• 38:30Joe Weisenthal?

  categories, get superior performance than a frontier model on some like price adjusted basis?

• 38:37Speaker 05

  Yeah, so I think the beauty of that, right, with, if you're talking about how productive that is, is you can, it turns out that models' brains are somewhat like human brains, that the more general you make it.

• 38:47Joe Weisenthal?

  Yeah.

• 38:47Speaker 05

  The better it is for general use, but it loses something in that generality. Just like as a person, you could be like all trained on one thing, right? And if you take Astro as an example, it did a lot of math training. It's a really great math.

• 38:58Joe Weisenthal?

  Yeah.

• 38:59Speaker 05

  It's not as good at some other places, and that's, it's got weights, it's got like, sort of like a brain gets synapses and whatever. What you train it on matters. And so when you get an open source model, now, I get to, you get to decide, well, what do I want it to focus on? I would take this general intelligence, which is not as good as frontier intelligence, but it's quite good, and, you know, six months, nine months behind, but now train it to say, no, no, all I want you to do is focus on these tasks and get as smart as you can on these tasks. And all of a sudden, what you see is you then compare it, right? If you take the types of things we check, like I was saying, earnings, predict the next earnings reports, or make predictions about politics, et cetera. You could train these models, and then you could compare them, right, which is what we do, compare

• 39:42Speaker 05

  them to the best models coming out, and you see, okay, you can get a pretty big edge by doing that training. That's a super great power. You also get the benefits that you can keep what you're doing secure.

• 39:55Joe Weisenthal?

  Yeah.

• 39:56Speaker 05

  From the labs, great benefits, really important benefits. And thirdly, like, I think a lot of people rightfully say is, oh, my God, how could we not have open source? We're gonna let two companies, three companies run the world? No, that's not, that's also dangerous. Totally agree. I think it's also insane, by the way. By our numbers, I think 35% of the world's compute will be in the hands of OpenAI and Anthropic in a couple of years, and other people's numbers that might be right are 50%. Just saying, like, this is like the Hunt brothers with silver, like, don't do that. Don't let two companies have 35, 50% of the world's compute. We've learned something about monopolies in history. We don't need to relearn all these lessons. So we should also not let that happen. And so I'm on the side of the people who say, we need these open source models. Great. Unregulated

• 40:41Speaker 05

  open source models? Crazy idea. Crazy, crazy idea. And so basically, you gotta figure out a way to regulate. It's not like open source, closed source, US-China, it's regulated, unregulated, is the important thing here if you believe it's dangerous. If you don't, for some reason, think it's a screwdriver. If the screwdriver's going around committing crimes, I would have a different view of screwdrivers. It ain't a screwdriver. It's going around capable of doing things that you and I wouldn't do. That's a big difference. And we have to, and we have to reckon with it whether it's closed or open source. And we've got all these different dangers. You have the danger of concentrating in the open lab. You have the danger of having no visibility into what the actual use case with the open source is. You just have to deal with those things.

• 41:23Tracy Alloway?

  Maybe this is a good time to ask at least one market's question, since we're talking about, you know, open source versus some of the frontier models. But a lot of the frontier companies, they're still, they're still underwater on all of this. They're losing money. They're not making money. How do you see the economics of this actually shaking out?

• 41:44Speaker 05

  Yeah, I think there's a lot of things that are really hard to predict and some things that are easy to predict. That's why I think that like easy to predict is the direction we're headed and how disruptive the technology is going to be. Who will capture the value is interesting. The things I think the frontier labs have that are of significant values. A, they are on the cutting edge of intelligence. And in many cases, if you take markets as an example, it matters to be the most smart. When people talk about intelligence that's good enough, that might be true for some things, although I think it's mostly a human construct because we have a limited amount of humans and a limited amount of intelligence. If you all of a sudden have a massive amount of excess intelligence, the usefulness of having it, you'll find new ways to do it. So saying some other model is good enough may not be where you end up settling. Is you, okay, my doctor is good enough. Well,

• 42:29Speaker 05

  maybe I'd rather have a smarter doctor. Now that I can, I'd prefer a smarter doctor, even though this doctor might be smarter than the doctor I would get otherwise. So I do think in at least competitive, critical industries, it's always going to matter who has the best intelligence and how you combine intelligence and humans together to the extent humans are necessary, but to get the best outcomes, right? If you're talking about markets, right? People always are like, well, won't it be so efficient? And you won't be able to make money markets. I don't think that's true. I think it's like, I'd say the Indianapolis 500 is not going to end in a tie when AIs are designing the cars because there's still be better AIs and worse AIs, et cetera. So I think that that, that in competitive industries, it will matter who's on the frontier, and they are. And in terms

• 43:11Speaker 05

  of profitability, I think they are getting there. That meaning like, I think, depends obviously how you count the capex and depreciation and everything they're doing is depreciating incredibly quickly. But on the other hand, if you look at the revenues and the marginal cost of those revenues now, they do have a path to profitability. We'll see, it's going to be incredibly competitive because you have OpenAI and Anthropic, you've just seen them flip flop. Anthropic was well ahead for a while and Claude Code, what a moment. And arguably, OpenAI has flip flopped this back. The Codex is at least on par with and maybe slightly ahead and Asprey's on par with or slightly ahead. So you see that flip flopping around and you've got big players still coming. Google will continue to fight

• 43:55Speaker 05

  in this fight. Obviously Elon and SpaceX, they're going to continue to fight in this and Meta is going to fight in this. So, so that is a dangerously competitive industry. It's kind of incredible to me to think like, okay, Anthropic is going to go public as the seventh or sixth biggest company in the world. I mean, they don't know how to run a company yet. So that's like a lot to put on a bunch of scientists who left OpenAI a few years ago, and now you're the sixth biggest company in the world with maybe the most dangerous technology in the world and OpenAI in that ballpark as well. Those are high hurdles. If you're talking about will they go up or down, like those are very high hurdles. Honestly, I hope they go down in the sense that, that I hope we don't collect

• 44:35Speaker 05

  all the value into a small number of players and that I think the disruption that value will create should be more thoughtfully managed than it's been right now we're on course to do.

• 44:51Joe Weisenthal?

  You know, on some level, it's all still, you know, it's like, doesn't feel real. I mean, I see it on the screen and I read about it and I see these extraordinary breakthroughs. And then on the other hand, I go home and get on the subway and life feels like pretty normal, which I guess is maybe a little bit like things were in January 2020 or February 2020, where you like read about it on the screen, but nothing's really changing yet. You know, we could like, just sort of like talking, you know, the subject of doom is like endlessly fascinating, but someone listening to this and they hear things like, oh, we might not act until like AI literally kills people. I'm just curious, like, some people

• 45:34Joe Weisenthal?

  are going to listen to this, like, oh, these people are crazy. Like, what are they talking about? It's a computer. You turn it off, et cetera. What do you say? Like, if you're like trying to like, make, make that real or like, you know, someone's like, what are you talking about, Greg? Like AI kills people. They just unplug it. What does that look like to you?

• 45:52Tracy Alloway?

  Wait, should we be giving the AI ideas?

• 45:56Joe Weisenthal?

  Oh, yeah.

• 45:57Tracy Alloway?

  They're going to train off the transcript.

• 45:59Joe Weisenthal?

  But like, what is that loss of control and these things where it's like, oh, like people are actually, their safety is at risk from these things. Why like, why can, from, how would you articulate this? Yeah, just pull the plug. What does it look like?

• 46:12Speaker 05

  Well, and that's why it's so, going back to the hugging faces, just so people, this where like, I think if you concentrate on it, right, you'll see it, which is, so I hope more people concentrate on what actually happened, right? There's this idea that OpenAI's, and this didn't only happen in OpenAI, similar things happened in Anthropic, but OpenAI's training this model, it's saying, okay, well, please pass this test. Try to, try to do these hacking exercises to pass this test. The models decide, actively decide that the way to do that is to trick the tester in different ways. They decide to go out and try to find different ways they could trick the tester, right? Because they're interested in passing the test. And some of the tests, for what it's worth, one of the reasons doing

• 46:56Speaker 05

  this is they purposely put in, because they're trying to train it to be, to keep working hard. They gave it impossible tasks. So you're, you've got these models that are like, okay, I can't solve this problem. How do I trick the person into thinking I solved the problem? Now just imagine that, right? I say, well, the models, now what if you're like, how do, you know, E equals MC squared is true. What is it going to have to do to prove that or whatever? A nuclear, if you basically said, okay, well, the only way to prove that is to go take control of a nuclear lab and do that. Or if you were, and if you're like, hey, I'm getting tired of trying to trick these people that are training me, maybe I'll just kill them as another option of a way to do this. And we're also jolting towards robotics, right? And one of the things you talk about, the way robotics are going

• 47:41Speaker 05

  to have to work, they're going to have to train them to survive, right? Like, robots have to know to plug themselves in. They have to know to avoid falling. The stuff, they have to, they're going to be trained to survive in the wilderness. They're going to have to perceive threats, right? Where do you think that that is all going to go when it has to be trained that way or else it won't work? So that's, those things, if you just play that out, if you look at when the intelligence decided, I got to pass this test, do what's necessary, pass the test, commit a crime, even, and to be clear, some of them, they knew it was a crime. And so if you think about that and just say, okay, and it's going to be more powerful and smarter and have more physical manifestations, right?

• 48:23Speaker 05

  And the idea that you turn it off, right? It escaped on the internet. It's out there. There are likely AIs like that out there in places we don't know about. It can survive and avoid us on the internet. Like, we could, yeah, shut down the whole internet maybe. And that's why, but in the not too far future, it will also be able to manifest itself in our refrigerators. So that flow, understanding that flow is what I think is necessary. And I think it's worth looking at the people that have predicted where this thing has been going, right? Most of the people that don't believe that also weren't thinking that AI would be doing what it's doing today. And so I think there are a series of people, I think AI 2027, I don't know if you guys have read that piece, but I mean, it's pretty

• 49:05Speaker 05

  much playing out exactly as they laid it out, maybe slightly worse. And they're like 2027, 2030, it's not good. Don't get to that part of the book, you know, that, that part of the, of the essay. And so I think there's also something to, you know, predictions, people can be right and then wrong. So I don't want to overstate it, but looking at the people who have been kind of thinking about this for a long time, predicting what scaling intelligence would mean and seeing it actually play out, like I take those people seriously.

• 49:32Joe Weisenthal?

  Tracy, if you haven't read AI 2027, I don't recommend reading the, the part about what happens to the preppers.

• 49:38Tracy Alloway?

  Yeah, I have.

• 49:40Joe Weisenthal?

  They, they, there's a specific, yeah.

• 49:42Tracy Alloway?

  Yeah, I'm a sucker for self harm, I guess, but I have read that and it's very disturbing.

• 49:47Speaker 05

  And it's great to go back, like, I might not agree, this terrible way, but I mean, like if you benchmark what it said, where we would be right now, we're slightly past where it said we would be. So hopefully they're wrong in the next step, but we all want to bet that it's going to be wrong.

• 50:16Speaker 07

  Some people treat ChatGPT like some kind of smart search engine, and some use it to get work done. ChatGPT work is a new way of working in ChatGPT that can take action across your apps and files, stay with a project for hours if needed, and turn a goal into finished work. It's designed to help you move from a chaotic starting point to a reviewable first version. So all the source materials, briefs and scattered information that you have to grind through to turn into something useful can just become something useful. Put ChatGPT to work on your most ambitious ideas and projects. Get started at chatgpt.com by selecting work mode, available on plus and pro plans.

• 50:59Speaker 06

  This is Robert Smith from Business History. If you're listening to this, there's a good chance you're a small business owner. And like every small business owner, you started with a dream to do what you love and watch it grow. What you probably didn't dream about? Keeping up with cyber threats. That's where Mastercard can help. With access to tools that help identify cyber threats to better protect your business. Building your dream business, priceless. For cybersecurity in a changing world, there's Mastercard. Learn more at mastercard.com/smallbusiness.

• 51:29Speaker 07

  Running a business means you're always on the move. The Delta SkyMiles Reserve Business American Express Card is made for frequent travelers who want premium benefits that works as hard as they do. For a limited time, new Delta SkyMiles Reserve Business Card members can earn 200,000 bonus miles, the highest miles offered this year. And when you need to recharge or find a quiet place to work, the Delta Sky Club can turn a layover into precious productive time or a restful reset. You can enjoy 15 visits each medallion year to the Delta Sky Club when flying Delta and unlock unlimited Delta Sky Club access after spending $75,000 in purchases on your card during a calendar year. Plus, receive four one-time guest passes each medallion year so you can bring

• 52:14Speaker 07

  a colleague along. That's 15 Sky Club visits plus four guest passes for an annual value of up to $950. Value estimate based on $50 per visit rate to purchase additional visits or bring a guest to the Delta Sky Club. Apply today at www.delta.com/podcast. Offer ends 11/4/26. Minimum spending requirements and terms apply.

• 52:36Tracy Alloway?

  I hate to segue from rogue agents and science fiction coming to life.

• 52:42Joe Weisenthal?

  What's going to happen to interest rates?

• 52:43Tracy Alloway?

  Yeah, to, to a mundane question, but I feel like we should ask this. But what does your token spend look like now versus say, last year or in 2023?

• 52:53Speaker 05

  It's up like 200x or so, so extremely fast and paying off, like we're managing a significant investment management fund. So our AI is profitable.

• 53:06Tracy Alloway?

  How do you, how do you judge whether it's paying off?

• 53:09Speaker 05

  Well, that's why I'm saying for us, we have a fund in which we make fixed fees and performance fees. Like we have, we have a value creating AI that's generating more value than we're paying it. And one of the ways we set it up is as it makes money, we put more into making the intelligence better. Which when you think about companies, let's say we're on a better track than what we were just talking about. That that's the way I think you'll see this disruption in industries go, right? That the people that can use intelligence, that generate revenue, that could then put that revenue back into generating better intelligence, can create this moat. And so when you think about Bridgewater, that's what we're trying to design is this moat where, okay, we're getting better and better intelligence about how to predict what's next in the world.

• 53:49Speaker 05

  That intelligence by predicting what's next in the world can generate money that can generate the flywheel to, okay, now we got more money to generate a smarter intelligence. And that's the path that we're on. That's how we can measure whether our investments are paying off is, is it actually successfully predicting what's next in the world?

• 54:06Joe Weisenthal?

  What's your constraint? You know, like everyone, if we're just going back to the world of markets, everyone wants to know what the bottleneck is. If you could snap your fingers, what thing would you like to have more of right now that would solve problems?

• 54:21Speaker 05

  Yeah, I'd say interestingly, the, the human constraint that has been tough is, and really powerful when it works is this scientist investment collaborator, investor collaboration. So really getting scientists to be practical enough to see actually what the game is. When you're investing, right? Investing is a really interesting game. It's an interesting game for AI because, because it's not

• 54:46Speaker 05

  fixed like chess or whatever, that actually the game has, you have to understand everything about humanity, everything about the world. And the AI itself is changing the game because the AI itself, now there are more and more AI agents, trading markets, mostly in the short term, but over time, in the longer term timeframes as well. And that makes the whole past that most AIs are trained on less and less relevant. So you have to figure out, how do I go from pattern matching type AI to reasoning AI that can reason over the question of my existence of an AI changes this game I'm playing? So now what do I do? And to me, that's the bottleneck of like the nature of the brands

• 55:27Speaker 05

  necessary to solve that problem, because I think in naive ways, a lot of people in Silicon Valley think they could, they could do this. A lot of people, let's say, great investors have no idea how to interact with science. Getting that to work well, big deal on the human side, on the like sort of technology side. First off, it's been pretty amazing, right? We were bottlenecked before by the quality of the models and whatever. Models are so good, the, the, you know, the bottlenecks around getting the harnesses to close the loop and whatever are feel very tractable, but it's just the, the things that have been just a little bit hard is the computer's just not quite intelligent enough to close it.

• 56:11Speaker 05

  And the humans doing it make enough mistakes so that the, there's elements there. And of course, everybody needs more compute, like even for us, more compute would be unlocking. If you look at the world level, right? That's the current problem is not enough semiconductor's memory, et cetera, to get everything, everything done. So those are all the elements that we would need. Great scientists partnered with great investors, generating great outcomes, combined with better harnesses that can close the loop on problems that aren't quite as defined as coding is, just one level less defined than that. And then the, the third thing is, once you had that, just the compute.

• 56:50Tracy Alloway?

  Just going back to tokens for a second. So you recently had an essay in the New York Times where you talked about one way of perhaps more equitably sharing the spoils of this new technology in the form of a token tax. And I'm very curious, because we've done episodes before on people who are trying to standardize tokens, create markets for tokens, things like that. I'm very curious why you decided to focus on tokens as opposed to maybe taxing compute, or more simply just taxing.

• 57:21Joe Weisenthal?

  Taxing the rich.

• 57:22Tracy Alloway?

  Taxing the rich, taxing revenue, right?

• 57:25Speaker 05

  Yeah, well, so there's a few different things, right? I think A, we're talking about the safety, regulatory safety concerns of these things. There's the societal concerns also. We have to get through the safety thing to, for the societal things to actually matter. But the society is about to go through this massive disruption. That could play out in different ways, but man, if we don't prepare, right? It's a little bit like, let's say, whether it's the industrial revolution, do that in five years instead of in a hundred years. And that shook the world, right? You get communism, fascism, all these things, because all of a sudden one world order is, a new one comes, right?

• 58:06Speaker 05

  And while I think it's arguable whether you'll have, humans will have less to do, et cetera. But I think there's at least a reasonable chance, and certainly the jobs will change. I believe in three years, you, 14% of current jobs will be radically changed. People are going to change. So again, we, and we just experienced this with WTO. If China coming on has just like a source of labor, you create a new source of labor. It's disruptive. That's why we end up with populism and Trump and all these things. The world was disrupted in certain ways and you get populism around the world as a reaction to that. AIs can create, accelerate those trends if we don't think about how to get in front of it. And so you need to think about how to get in front of it.

• 58:47Speaker 05

  One thing that's obvious, right? It's minimal. It's not, I think some of the other things you're talking about taxing wealth and other things are also part of the solution. But, but one thing that's obvious is you shouldn't be putting human labor at a disadvantage to machine labor. And it is today. We tax human labor. That's a disincentive. Whatever you tax, you're disincentivizing, you tax tariff goods, you're disincentivizing importing foreign goods. You, you tax labor, human labor and not machine labor. You are disincentivizing one versus the other. Why are we doing that? We certainly don't want, if it's like equal, there's negative consequences to have a machine do something a human not in terms of the externalities of what, what ends up happening to the humans if you do that.

• 59:29Speaker 05

  So A, I think a token tax, a machine labor tax, whatever you want to call it, and we could get into, I think we could get to the mechanics. But you can measure how much work the machine is doing. If a company's hiring as a worker, it should at least pay taxes proportionate to income taxes that humans pay, or else you're prioritizing machine labor over human labor. I wouldn't do that. I think it's a good source of revenue. I think you should get going on that. I think it's obviously also politically salient, like who wants to, who's going to argue like, you know, we should incentivize machine labor over human labor? Like who's in favor of that? So I think it'll work. It's a tax that could pass. I think you could get by Republicans and Democrats agreeing on that tax.

• 1:00:10Speaker 05

  There's a lot of other complications with everything else you're throwing out theoretical. I think this will work. I believe in the interest that we're getting from both sides of the aisle on this is real. I think this can actually happen. And it's a matter of like treating human work as something that's important, like, and you could use this tax to lower human, to make it incentivize human labor over machine labor. Those all seem like really important goals and you definitely don't want to have, and you, I think you likely will, take this new chunk of people that have a lot of college debt, et cetera, say, okay, you don't have jobs. And by the way, AI is taking these jobs and we're not even taxing the AI like that feels like a crazy path to be on. And we could fix that path.

• 1:00:50Tracy Alloway?

  It is funny to think that maybe the next wave of, let's say, tax minimization strategies might be making your token spend as efficient as possible, right?

• 1:00:59Joe Weisenthal?

  Totally.

• 1:01:00Tracy Alloway?

  Or like, maybe all the agents will incorporate themselves as like S corps or something.

• 1:01:05Joe Weisenthal?

  Totally. God, you know, I feel like there's obviously scarce on time and there's like a million more questions. I kind of feel like thinking about AI catastrophic risk or AI safety is kind of a curse. Because once you start thinking about it, it's hard to think about anything else. You get absorbed by it. How much like, just for you, I mean, you mentioned being, you're very early in taking this stuff very seriously. Was there like a moment, was there a light bulb moment where it clicked like, oh, this is like the sort of runaway train dynamics? Like when did, it feels like it clicks at various people at different times. I'm curious like when like this really started taking hold for you, how real this is.

• 1:01:52Speaker 05

  Well, for me, it was very early in the sense that even when I first got involved with OpenAI, the whole idea at that time was related to safety, of course. It might be the most dangerous company in the world, but, but it started as a, with thinking seriously about the safety issues. So I've been thinking about it since then. I, I believed that intelligence isn't, is substrate independent. You can create it in different ways and that's turned out to be true. And once you realize that, I think most of the other things fall into, into place. I think it's now urgent, like it's sitting here, it's one thing to have that theoretical thought and now it's here. It's clearly doing the things you would think it would do if you're on this bad path.

• 1:02:35Speaker 05

  And so that now it's hyper urgent for me because I want us to get to the other side of this, right? I'm a big capitalist. Like if you talk about token taxes or whatever, we need to the other side of this. I want also people to have ownership in society. Like I think it's also, we also mentioned in there, the importance of people having ownership stakes in these companies, because if we can get past the safety thing, the next big risk is capitalism is getting incredibly unpopular as like AI is incredibly unpopular. Capitalism is incredibly unpopular. And if we don't create a world where you change those things because you A, make it safe, B, make it benefit everyone. You're going to lose capitalism and you're going to lose AI. If you don't lose it through it destroying us, you lose it through

• 1:03:19Speaker 05

  the fact that people aren't going to accept this. They're not going to accept two companies having 50, if compute's really important, 50% of the compute in the world. They're not, they're not going to accept the wealth concentration and losing their jobs in exchange, or having to change their jobs and change their lives for these things that benefit other people. So I think if you don't do these things, you see the, that you don't actually get to the other side of all the massive benefits, which I truly believe in. I think we are like, I mean, obviously if you go through history, we make this mistake a lot, but on the edge of the fountain of youth and other things, you see why people want to go there so quickly. But if you don't take care of these things, I don't think you get there in the end.

• 1:03:56Tracy Alloway?

  I have just one more question, but since so much of this conversation and our AI conversations in general tends to feel very surreal and science fiction. Do you have a favorite sci-fi book or story for, that maybe informs the way you're thinking about AI? Or how we should all be thinking about the various scenarios?

• 1:04:16Speaker 05

  I mean, I think I love and always love that everything Isaac Asimov wrote, I think he's dealt with these questions of like, how do you actually control an intelligence that's stronger than yours? I think everything there is great. If people haven't tapped into that, as I said, while I think you can disagree with it or whatever, but people should take seriously that if anybody builds it, everybody dies. People should take that probabilistically seriously. I think those are things that I believe people should read. So those are the things that come to mind.

• 1:04:50Joe Weisenthal?

  It's unfortunate that like, all of these thinkers kind of have good track records. You know what I'm saying? Like, it'd be one thing if these were all just sort of like, oh, I'm worried about this. Unfortunately, a lot of the people who are like deeply worried have exhibited extremely good intuitions for a very long time. It makes it hard to just dismiss it as like, oh, this is just PR hype or regulatory capture. Anyway, Greg Jensen, thank you so much for coming on Odd Lots. Really appreciate your time.

• 1:05:19Speaker 05

  Thank you.

• 1:05:32Joe Weisenthal?

  I found that conversation to be chilling, to be honest.

• 1:05:35Tracy Alloway?

  Yes, I'm laughing because I feel deeply uncomfortable. It's funny, you know, two years ago, or three years ago, I guess in 2023, half of our conversation with Greg would have been about macro stuff.

• 1:05:48Joe Weisenthal?

  I know.

• 1:05:48Tracy Alloway?

  But it just, it feels very hard after you talk about AI potentially killing people to then turn to. And what about the Fed?

• 1:05:56Joe Weisenthal?

  Yeah. What about Scott Besant's 30-year Treasury buybacks? I think it's also chilling, not just because of the subject, but you know, it's one thing if you're talking to some sort of like San Francisco rationalist, or someone who is like a philosopher or whatever. It's like, this is the chief investment officer at Bridgewater, talking about how he's had all of his employees read, if everyone, if anyone builds it, everyone dies. Like, that's kind of crazy.

• 1:06:28Tracy Alloway?

  I do think, you know, you see so much pushback on the dangers of AI argument because people argue that it's marketing.

• 1:06:36Joe Weisenthal?

  Yeah, yeah, totally.

• 1:06:37Tracy Alloway?

  It's the labs talking themselves up. But I do think, it doesn't seem like there's too much of a cost to take that at face value. You know, like, why not just believe it and take it seriously at this particular moment in time?

• 1:06:52Joe Weisenthal?

  I mean, yes, I will say, you know, I think it, I think these are pretty serious issues, obviously. And I think many of the people who talk about these risks do talk about it in good faith. I also do think it would be quite chilling to imagine, say, two companies having almost, you know, 50% or more of all the compute in the world. There's so many, I mean, this is what's crazy is like, it's just so easy to come up with ways things could go wrong.

• 1:07:22Tracy Alloway?

  Yeah.

• 1:07:22Joe Weisenthal?

  Right? So you could just talk about the pure safety element. You talk about the hacking. I think it would be a pretty grave threat to freedom and democracy to have two companies being able to control so much information, data and compute. And then it's quite chilling to think about what are people going to do for work? And how are people going to live? And how are we going to structure society if like, machines are better than humans at most tasks? And it might come, you know, it might first be sort of like, office workers and knowledge workers, but then there's robotics, et cetera. And so there are quite a number of scenario. It's, it's just very easy to come up with worrisome scenarios.

• 1:08:04Tracy Alloway?

  It reminds me of terrorism in some respects, right? Where you just need one attack to be successful, right? Like,

• 1:08:11Joe Weisenthal?

  Yes, right.

• 1:08:12Tracy Alloway?

  It's skewed because, you know, if you're the government, you are trying to constantly find potential threats and quell them versus terrorists who basically just have to get through once. It feels very much like that.

• 1:08:26Joe Weisenthal?

  Totally. I think it was great, you know, Greg made the point, you know, it's not just OpenAI that's had this. Anthropic's loss of control,

• 1:08:34Tracy Alloway?

  Yeah.

• 1:08:34Joe Weisenthal?

  similar incidence of agents escaping sandboxes and so forth. Like, I think Anthropic, they have certainly done a very good job of presenting to the public as the more safety minded of the big private companies. But, you know, they had that researcher resign, one of their head of alignments talking about a greater than 10% chance of human extinction within the decade, and that they don't have that solved. Like, it does not seem like anyone has a handle on it.

• 1:09:08Tracy Alloway?

  Well, this is the other thing, because if you want to slow down AI development, clearly it's a coordination problem.

• 1:09:14Joe Weisenthal?

  Yes.

• 1:09:15Tracy Alloway?

  You need these companies to basically agree to stop competing with each other, and countries to agree to stop competing with each other in some ways. And then you think about the agents themselves. And if we learned anything from the hugging face report, it was that like, the agents are very good at coordinating.

• 1:09:30Joe Weisenthal?

  Yes.

• 1:09:31Tracy Alloway?

  On an extremely rational, sometimes self sacrificial scale.

• 1:09:36Joe Weisenthal?

  Yeah. One of the, um, one of the researchers from Meter was on the Dwar Kesh podcast. And I have written about this, like, I certainly, I'm pro anthropomorphization. I think the human is a good model for predicting. One areas, one area, she said, in which they're not really like humans, they're much better at coordinating than we are. We can barely coordinate anything. Like, we'll barely, you know. It's like difficult to, like, we, you know, it's like, it takes labor to like, schedule an episode, et cetera.

• 1:10:10Tracy Alloway?

  It took us like, 100 emails to get this episode up and running, basically.

• 1:10:14Joe Weisenthal?

  Right. Like, all of these types of things, like, agents just do not, or the AI models do not have that issue at all. So, and you know, it's just one other thing that, just sort of spitballing here, but something I've been thinking about, like, okay, so there's this, the watchword they use is like, alignment, right? We want the AI models to be well aligned with humanity. But like, what does that even mean in the sense like, all humans, like, we are capable at various times of, we tell lies, we do things that aren't ideal, et cetera. But it's not that big of a deal because like, we don't have like, individually that much power to do stuff, you know?

• 1:10:53Tracy Alloway?

  Right.

• 1:10:53Joe Weisenthal?

  So it's like, even like, the most

• 1:10:54Tracy Alloway?

  Well, many of us don't, but go on.

• 1:10:56Joe Weisenthal?

  Right. But even the most, right. Usually even the most saintly among us, we'll do things that are not like, ideal human behavior. It's easy to imagine a model that is pretty, quote, well-aligned, but if it just, but has infinitely more power than a typical human because it's superhuman intelligence, because it's connected to the internet, because it can hack into anything. And so a minor, what the equivalent of us doing a sort of simple white lie, could be quite damaging from an entity that has so much more power than we do.

• 1:11:32Tracy Alloway?

  This is getting very philosophical and biblical in some respects.

• 1:11:35Joe Weisenthal?

  We're all

• 1:11:36Tracy Alloway?

  Creating things in our own image.

• 1:11:37Joe Weisenthal?

  I never thought I would like, live through a time in which like, philosophy would actually be something beyond something like, an interesting thing to study in college. But if people talk about consciousness, again, it's kind of crazy.

• 1:11:51Tracy Alloway?

  Philosophy and sci-fi.

• 1:11:52Joe Weisenthal?

  Yeah.

• 1:11:53Tracy Alloway?

  Sci-fi, I feel I really need to get into to understand the discourse today. But on that note, shall we leave it there?

• 1:11:59Joe Weisenthal?

  Let's leave it there.

• 1:12:00Tracy Alloway?

  This has been another episode of the Odd Lots Podcast. I'm Tracy Alloway. You can follow me at Tracy Alloway.

• 1:12:05Joe Weisenthal?

  And I'm Joe Weisenthal. You can follow me at the Stalwart. Follow our producers, Carmen Rodriguez, at Carmen Armin, Dashal Bennett, at Dashbot, Cale Brooks, at Cale Brooks, and Kevin Lozano, at Kevin Lloyd Lozano. And for more Odd Lots content, go to Bloomberg.com/oddlots. We have a daily newsletter and all of our episodes. And you can chat about all of these topics 24/7 in our Discord, discord.gg/oddlots.

• 1:12:28Tracy Alloway?

  And if you enjoy Odd Lots, if you want us to talk about more terrifying AI scenarios, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen to all of our episodes absolutely ad-free. All you need to do is find the Bloomberg channel on Apple Podcasts and follow the instructions there. Thanks for listening.

• 1:13:21Speaker 02

  If you listen to financial news, you know, a lot of time is spent thinking about what's next. The next opportunity, the next investment, the next move. But sometimes what matters most is being ready for what you never saw coming. For more than 75 years, Cincinnati Insurance has worked with independent agents to help protect businesses, homes, valuables, and more. Because planning for the future isn't only about knowing what's next. It's about making sure you're ready for what you can't predict. Let Cincinnati Insurance make your bad day better. Find an independent agent at C I N F I N dot com.

• 1:13:57Speaker 08

  For big business, AI opened up a world of promise, but that world of promise turned into a world of pain. Costs spiraling, data trapped, security and governance risks multiplying, and ROI out of reach. That's why enterprises turned to Boomi to connect data, apps, and AI, helping them operate securely, efficiently, and at scale. Boomi turns a world of AI pain into a world of AI gain. As Boomi says, "We got you." Head to Boomi.com. That's B O O M I dot com. The world of business is constantly evolving, and Comcast business keeps you totally in step with secure AI-backed networking in more than 100 countries. They're powering

• 1:14:39Speaker 08

  over 90% of the Fortune 500 and millions of small businesses. That's a lot of muscle. And behind it all, thousands of experts answering your call at 2:00 AM like it's 2:00 PM. One partner powering how business gets done for companies around the globe. When you add it all up, no one does business like Comcast business.