Pod Insights Real Vision: Finance & Investing We Need Both Symbolic and Statistical AI About this episode 🔥 Get 10% OFF in NGRAVE's shop with the code "Realvision" http://realvision.com/ngrave David Mattin, founder of New World Same Humans, joins Pradeep Ravikumar, professor at the Carnegie Mellon University School of Computer Science, to explore the state of AI today, the distinction between symbolic and statistical AI, and why we need both. This episode is sponsored by NGRAVE, maker of ZERO, the world’s only crypto wallet with the highest security certification. of the coldest hardware wallet ZERO and stainless steel backup GRAPHENE. NGRAVE brings you the highest security in a touchscreen experience to safely manage all your crypto offline. Enjoy a 10% Real Vision discount at realvision.com/ngrave. with the code REALVISION Elevate your brand with Real Vision. Connect with us at [email protected] to explore advertising Learn more about your ad choices. Visit podcastchoices.com/adchoices
Transcript 0:00 Speaker 00 Hey everyone. Today's Real Vision Daily Briefing is sponsored by Engrave, maker of the coldest hardware wallet Zero and stainless steel backup graphene. Engrave brings you the highest security in a touchscreen experience to safely manage all your crypto offline. Enjoy a 10% Real Vision discount in engrave.io shop with the code REALVISION. Now to the top analysis of today's markets.
0:36 Speaker 01 Hi there, everyone. A belated Happy New Year from me. It's my first time back on the platform in 2024. I hope you are all extremely well. I'm joining you as usual from London, a very chilly London. And we are in for a huge, huge treat today because today we have a special guest. His name is Pradeep Ravi Kumar. He is a professor of machine learning at Carnegie Mellon University. And he's here, of course, to talk to us about AI. We've all watched this AI moment that we've been living through for the last two, three years in amazement, with wide-eyed wonder, essentially. I want to dive deep
1:19 Speaker 01 into that with Pradeep in this session. We'll look at where things are heading in 2024, how things are going to develop, the moves we think some of the big players, including OpenAI, will make. But we'll go deeper than that. I want to look with Pradeep at the issues that form the center of his work. I think it's fair to say that he believes that we're at a crucial threshold moment for machine intelligence. And I want to dive into that and what it really means, what it means for where all this is heading, and the impact it can have on the real economy and our lives. So without further ado, Pradeep,
2:03 Speaker 01 thank you so much for joining us. How are you?
2:07 Speaker 02 Yeah, I'm doing great. Also Chile and Pittsburgh, but still doing great. Thanks for having me.
2:15 Speaker 01 Thank you so much for joining us and I hope we can both stay warm during this conversation, warm enough. So look, let's start by just learning a tiny little bit about your background and your research work right now, because that will set us up for everything that will follow.
2:36 Speaker 02 Yeah, so my research actually dialing back 20 years was in statistical machine learning. So this is actually all of AI these days involves statistical machine learning. And this is something that really picked up steam in the 90s. And during my PhD in the 2000s, that's where we laid a lot of the foundations for the statistical machine learning in terms of figuring out statistical associations between a lot of data and making predictions and so on. So all those foundations are right now being used in all the state of the art AI models.
3:17 Speaker 02 And so fast forward to the present. So now I'm really looking forward to the next evolution of AI beyond statistical machine learning. And I'm sure we will talk a lot more about that today.
3:36 Speaker 01 Yes, indeed, we will. And that taps into this big thematic distinction that we'll get into between statistical and symbolic approaches to machine intelligence. Because I think it's fascinating and it's so crucial for everyone out there to understand if they want to think about where this is heading across the next few years. But let's start with where we're at right now. I mean, it goes without saying.
4:05 Speaker 01 For most of us out there, the AI moment we're living in now, ChatGPT, Midjourney, Generative AI, all of that, it's just, it came as a huge surprise. Where do you think this is heading across 2024? Because so far, we've had huge excitement and a degree of wonder with the capabilities and the outputs of these models. It feels like, you know, we've done that now, and the questions are more around, or the questions I'm asking myself are more around, how does this have an impact on the real economy? Where does it head across 2024 in terms of jobs, economic impact and so on?
4:50 Speaker 01 How are you looking at that? And how are you looking at the role of the big players, you know, OpenAI and DeepMind in all of that? Can ChatGPT have an impact on the real economy? Or is it just not ready to?
5:05 Speaker 02 Right, I think yes and no. If you think about ChatGPT and large language models, they are very good, great even, on figuring out statistical associations between word sequences and understanding essentially just a collection of words. But with the more modern multimodal foundation models, they can also understand images, even audio and all of these things together. And so basically, an upshot of this is that if we have data that consists of images and text and audio,
5:46 Speaker 02 then we're going to have these AI models that are going to understand what we want from them. And I think this is going to be a big unlock because if you think in terms of how, let's just say, we participate in the economy, let's say there is a manager and he wants, he or she wants their direct reports to do something. They typically convey this using audio, using text, potentially images, right? Maybe we communicate using memes, but. The the interesting thing is that if you can have AI models that can understand this, then they that that's a potentially huge unlock.
6:29 Speaker 02 And so I think what we might see over the next year are what I would say are things both above and below these large foundation models. Where you can have things above where you could basically have these these large language models suggest certain tools to use. And then you could have other wrappers that can use those tools. And then you could also have things below where it can clean up how you interact with the with these foundation models. And so I imagine that there would be a lot of companies in that space that can help us do that.
7:15 Speaker 02 But at the same time, I would say that it's not not all of the things that we want to do take the form of analyzing text and images and audio.
7:28 Speaker 02 Right? So I think that's going to be a realization that we are going to also see. So there's going to be these positives that I mentioned, but there also could be potential negatives, or I don't want to say disillusionment, but because I think there are going to be other solutions along the lines of what I'm going to talk about later on. But people are going to realize that just the large language models are not going to work if you have data that is not just images and text.
7:53 Speaker 01 Right. So, yeah, I mean, essentially what ChatGPT and other large other similar tools fueled by large language models can do is understand your text inputs, your language inputs, and spew out some new some new text. And there are instances in the workplace where that is useful, right?
8:13 Speaker 02 Right.
8:15 Speaker 01 Do you think that that can eat jobs yet?
8:21 Speaker 02 Um. Technology has always, so one perspective of technology is that it takes energy and uh transforms it into something instrumental. And by and large, if you think about, I guess, the industrial revolution and hundred years back, a lot of it was in the physical world. But if you think about even our jobs, right, so for instance, we are communicating actually uh you know, on video or the internet right now. And so a lot of our jobs don't actually involve the physical world, though again, there is some physical infrastructure for the internet. A lot of it just involves the world of ideas.
9:05 Speaker 02 A lot of our jobs involve ideas and moving ideas around. And there is an interesting question on whether you can take energy and transform it in an instrumental way to move ideas, not just atoms. And if you think about a lot of the jobs we do, including personal computers and, you know, we have Microsoft Word, PowerPoint, a lot of it are tools, technology that help us do that. But it's basically a lot of steps, very laborious. Um, and we can think of a lot of uh the new wave AI models as essentially simplifying those steps.
9:46 Speaker 02 Um, and the once we do that, that's such a huge unlock for humanity that I think it'll create more jobs rather than just destroyed jobs because there are many companies that just won't be feasible using basically our current set of technologies. And so for instance, if I have a very, very enterprising um entrepreneur in India, say, and they do not have enough funding to have a team of ten um people. So this is a company that's never going to exist. But with these tools, maybe it exists. So I wouldn't think of that as.
10:30 Speaker 02 So let's just say they start this company with two people. Instead of 10 people, I wouldn't say that you lost eight jobs. I would say you gained two jobs because that's now a company that wouldn't have existed. In fact, if you look at the downstream effects, um, then that would actually create even more jobs. So I think I'm pretty excited about the possibility of essentially using energy to to change things in highly interesting instrumental ways.
11:00 Speaker 00 Hey, everyone, we're going to take a quick break right now to hear a word from our partners. We'll be right back with more of the day's top analysis on the Real Vision Daily Briefing.
11:12 Speaker 01 Yeah, and that's such an interesting perspective because, you know, one of the big questions I'm asked and one of the refrains we constantly hear about ChatGPT right now is, oh, what about my job? Any knowledge worker, you know, many knowledge workers, what about my job? I'm concerned about my job. Is it going to eat my, you know, my task in the workplace?
11:34 Speaker 01 It doesn't feel as though ChatGPT is is sort of competent enough or multimodal enough, right? To just eat an entire person's job, even if their job is pretty is is essentially knowledge based. Would you agree?
11:49 Speaker 02 Absolutely. Uh, and this would also tie into some of the discussions we're going to have later on. But I think a lot of what ChatGPT and these large language models are going to do, I think are best understood as productivity enhancers. They are not going to be end-to-end solutions, um, because you cannot trust those as end-to-end solution. So we, for instance, Microsoft has this GitHub Copilot where it suggests, um, code snippets based on various, um, specifications you may have. And it even takes that existing code and tries to complete it forward. But you cannot trust it completely.
12:31 Speaker 02 See, you, what I find is that it enhances the productivity of a programmer by two X, three X. But it cannot replace the programmer, so you still need the programmer. And um do we ever get to a place where we can use this end to end? Uh, not in the near future.
12:52 Speaker 01 Yeah, I mean, and that's a, that's a really interesting perspective and potentially a reassuring one for many people out there. Um, and it has to align with, certainly aligns with mine, and it has to align with most people's, you know, anecdotal experience of using, for example, ChatGPT, which is exactly that, you know, it feels like a sort of jetpack for the mind, it feels as though it can enhance your productivity. But you, you can't just set it, you can't just set it off and, and have end to end replacement of entire, entire tasks.
13:25 Speaker 02 So actually, if you look at the word co-pilot, Microsoft actually gave a lot of thought to what do we call this device? And they are calling it a co-pilot, so it's very clear to the people using the system that this is something that is a companion, that's something that's going to help you. But it cannot be something that is end to end and you cannot trust it fully. But at the same time, um, you can kind of interact with it in a very different way and so on. So you can kind of learn how to interact with it, but it's still a co-pilot. And I think uh that was very smart nomenclature that exactly captures what it is.
14:07 Speaker 01 Yeah, yeah, I think that was a very sort of wise choice of words. Um, and an accurate one. And so you have that sense of it being, um, these tools being a productivity enhancer, sort of letting us think further faster, I suppose. Um, and I also love your perspective on the way it can help create jobs, because as you say, you know, you might have a company that needs five employees. Um, but it can't possibly afford that. Well, now it could have two employees plus a whole lot of A I. Um, and you haven't lost three jobs you've created to, because that company otherwise would not have existed. So do you see already where we're at now with these large language models and
14:53 Speaker 01 tools like ChatGPT? Do you see that sort of democratizing?
15:00 Speaker 01 Start-up culture and the ability to to do your own thing and start your own thing. You know, that's part of the that's part of the path we're on that I find interesting.
15:11 Speaker 02 Absolutely. I think this is something that we can think of as there are certain tools. Microsoft had wanted to do this with just computing itself. So this was actually Bill Gates' vision, which is that he wanted to bring the power of computing to individuals. He wanted to democratize access to computing to individuals. And of course, we have gone far from that. We now have smart devices rather than just desktops sitting on our on our tables. But now, if we move that forward, we not only will have computing devices,
15:54 Speaker 02 but we will also have programs, ways to interact rather with these computing devices that is more democratized. So we can think of programs as one way in which we communicate with these machines, these personal computers. And this was a language like the priests of yore where they have speaking in, you know, strange languages to these strange machines. And we needed to disinterme- we needed to intermediate our interactions with these machines through these priests, which are these programmers. But if you can allow normal non-programmers to also interact with these machines,
16:39 Speaker 02 that wouldn't that actually accelerate or be actually a more natural next step of Bill Gates' vision of having computing devices at the hands of everybody, but also in a way where they can interact with it fully. I can ask my smartphone to do certain things that I have in my mind, but I'm presently not able to do so because I won't be able to create an app. But now, if I can do that, isn't that amazing? So I think that's actually a very, very exciting frontier, I think.
17:14 Speaker 01 It is hugely exciting, isn't it? And I mean, if you're Roughly our age, you obviously remember the kind of the whole, you know, the birth of the app store and back in the sort of early twenty teens, I guess it was when every third friend you had was was was creating their own app. And the constraint on that for many people is that they couldn't code. You know, everyone had an idea for an app they wanted to make. Most people can't code, so they simply couldn't make that app.
17:44 Speaker 02 Right.
17:45 Speaker 01 My feeling is we're removing that constraint.
17:49 Speaker 02 Yes, absolutely, or at least reducing that constraint a little bit. So it won't be as simple as our grandma being able to code an app.
18:01 Speaker 02 Just from scratch. So we are still not there yet, but we are definitely reducing the barriers, I would say.
18:08 Speaker 01 Yeah, yeah. So the the potential for a sort of flourishing of creativity around computing is huge because to a much greater extent, it feels to me, you know, if you can imagine it, you can have a go at creating it, you know. Whereas that simply wasn't true before.
18:28 Speaker 02 That's right.
18:29 Speaker 01 Everyone on on Twitter or X, as we're now supposed to call it, is excited today about the launch of this new like R one rabbit device. You know, this little device that it is like an AI companion. There's also the humane AI pin. So we're seeing this incredible, the burgeoning of this of this wave of innovation around these AI companion devices. Do you think that's going to become a bigger part of people's of the story in twenty twenty four and a bigger part of people's relationship with AI across the coming year and beyond?
19:08 Speaker 02 I love the vision at the very least, but I'm a little bit skeptical, I have to say. So I love the vision of reducing friction. So Apple is very good at this, where
19:22 Speaker 02 many people completely underestimate and underrate friction that prevents us from various tasks. And if you think about Apple, they try to make these devices so that the friction of using those devices is reduced. And if you think in terms of natural language interfaces, that reduces friction. So if I want the computer to open a particular app, instead of me going, opening the computer and then trying to find the app, if you can just say, please open, you know, DoorDash and it opens it. That's great. And simple things like this, I think, can be enabled.
20:06 Speaker 02 But I think if we wanted to be a full-fledged companion in terms of saying, hey, book me a flight to London at the end of this month,
20:20 Speaker 02 I'm not sure we are there yet. So I think if we dial down our expectations for these companions, I think we are definitely going to get something. But I think to have a completely
20:36 Speaker 02 have a computing device that is going to be completely disintermediated
20:44 Speaker 02 in the in the sense of all interactions that we have with these. If it's completely going to be intermediated by natural language and audio, I don't think we are there yet. I mean, Amazon tried to do this with Alexa, where they tried to, and Alexa is pretty good. Right? So voice recognition is pretty good. They tried to get people to do various things with Alexa and they failed. Because there's still The just amount of friction, it's not perfect. Um. And if it's not perfect, I don't want it to book a flight, right? So I think that's essentially the problem.
21:25 Speaker 01 Yeah, yeah, exactly. And I think they're going to have to play with kind of voice and audio and screens quite a lot to find the perfect combination of the two, because that, you know, when I, when I book a flight, I want to be able to see the options in front of me. It's no good just telling me what they are.
21:43 Speaker 02 Right.
21:43 Speaker 01 You know, I think that those, those details are crucial. It's gonna be, I mean, my other concern for those startups like Humane and the, the, the little rabbit device that's going on now is, aren't they gonna get crushed by Apple? Like when Apple puts a large language model on its devices and can just roll that out across a billion people, don't you think?
22:08 Speaker 02 Absolutely. I think if Apple or Google, uh, for that matter, they do have this Gemini Nano that they already planted, push onto their pixels or Pixel Pros. Um, that's really the next frontier in the sense of having, um, these powerful models on our computing devices, uh, natively, uh, so that they don't need to communicate through the internet through some server and pay some cost and so on. Um, once we have that, then it is, I think, a worthwhile question to say, do I need a PIN? Or do I need a completely different device? Uh, probably not. Uh, I, I would definitely agree.
22:52 Speaker 02 Uh, but, um, on the other hand, uh, companies like Google and Apple are not going to roll out half-baked products, right? So they won't roll out something unless it's close to perfect. Especially Apple, they love to reduce the friction, but they're not going to re- uh, uh, release something unless the friction is completely gone. And so I think there's still some time for a company like Apple to actually release something. So I think in the next two, three years, these companies have, I think, free rein. Maybe they even get acquired by Apple for all we know. So there's still, I think, a good space to explore, uh, potentially.
23:35 Speaker 01 Because the dream with Apple is some sort of vastly upgraded Siri. And maybe they, you know, they don't call it Siri. That is a true on-device virtual companion that just knows everything about you. And you can talk to it. It's like a counselor, it's an assistant, it's a friend. Um, but you think, you, do you think that Apple have not done that yet? Because it's so hard to perfect and we know they're a perfectionist company.
24:02 Speaker 02 Exactly. Uh, that's really what is stopping them right now. Um, it is always possible to roll out a half-baked product right now, but Apple won't be the one to do it. Uh, and so they are going to wait, uh, and just even after coming with a product, they're going to, uh, experiment with it. And from a technological standpoint, we are nowhere near, uh, the, the place where, uh, a company like Apple would be happy with the product. So we are just not there.
24:33 Speaker 01 Now, speaking of the next frontier for machine intelligence, let's really dive into this. Because I know that fundamental to your work is this idea that to really move beyond where we are now and to really see an impact on the real economy, sort of a deep economic impact.
24:55 Speaker 01 We need to move beyond the focus on the statistical approach to machine learning that sort of governs this moment. So this can start to feel somewhat technical. It doesn't actually have to be that technical really, and you explain it so brilliantly. So let's get into all of that. Just let's start by talking about what we even mean by, you know, statistical and symbolic approaches and what it is about the current moment that makes it the statistical moment. Um, and that means diving in a little bit to the history of, of AI, doesn't it?
25:32 Speaker 02 Right, yeah, absolutely. Yeah, I, I think it's, uh.
25:37 Speaker 02 I think a great point to talk about the history of AI. So if you dial back all the way to nineteen fifty's, when we had these powerful computing devices, people thought, oh, that our next thing is to just build AI. If we have powerful computers, how hard is it going to be to build AI, right, is what they thought. So the first wave of AI was using what are called symbolic approaches. They are so called because if you think about any beliefs we may have or any statements we may have. So for instance, let's say David is a great journalist at Real Vision. So this is a statement.
26:19 Speaker 02 And I can basically have various symbols to represent this statement. There's an entity named David,
26:29 Speaker 02 who's a person and who's a journalist who works at Real Vision. And so all of these are, I can have symbols to represent this knowledge. And then I can also have reasoning to go from certain propositions to certain other propositions. So for instance, given all of these statements, I would then perhaps have another proposition that says David does great interviews. And so that's a reasoning step where I go to a completely different proposition and go from A to B. And these were the kinds of programs that people were building early on. And we can think of these as largely rule-based systems,
27:14 Speaker 02 where these rules operate on symbols. And the heyday of these were actually in the eighties and including in the nineties, where one prototypical instance is called Dendral.
27:27 Speaker 02 Where what it was was it was um given compounds, so let's say hydrogen peroxide, H two O two. So given various measurements, I think spectrographic measurements. Um. Maybe I want to know uh the composition of this compound. Uh. It's two hydrogen atoms, two oxygen atoms. And they built a system that was better than organic chemists. Um. And the way they did this was using a bunch of rules. So they had various symbols related to the measurements, they had various symbols related to the atoms and they um. Uh. Related the measurements to the atoms and so on. And so given the measurements, these looking at these rules, they'll say, oh, I think it's hydrogen peroxide.
28:10 Speaker 02 And there were these things called expert systems that were these gigantic rule-based systems sitting on top of symbols. That people were building in the eighties. That, again, um, the similarity to the present is uh quite interesting. Where people, the companies and industry were spending uh millions upon millions of dollars to build these large-scale systems. And people thought that this was the future of AI. And it turned out not quite. Um. Interestingly, Deep Blue that beat Kasparov uh in chess was another instance of such symbolic. Uh. Approaches where there the symbols related both positions.
28:53 Speaker 02 And particular actions that you can take. So I think there are sixty four um um. Um. I I forget how many exact moves you can make from any given position. But what you can do is that you can roll out. So I will play, you know, a particular position, then the opponent could play any of other sixty four different possible positions. And I can just completely roll this out till the game finishes. And then see if I move my, you know, knight to this position, then do I win? And I can just roll this out, but this obviously will take too long. And so they had various tricks to reduce basically the search. And this is how basically Deep Blue beat Kasparov.
29:37 Speaker 02 It is all very kind of rule-based approaches. Um. But like I said, um, this kind of fizzled out. Because this was too laborious, because where are you going to get the symbols? Where are you going to get the rules, right?
29:56 Speaker 02 But then there was a little bit of despair in the late eighties came this new field called machine learning. There, let's take the dendral example. So I have certain measurements, spectrographic measurements, and then I have what the compounds are. And instead of having organic chemists provide rules and what the symbols are, what are atoms and whatnot. What if I just give a training data that says that, okay, here are the measurements, here is the compound, here's the measurements, here's the compound. I just gave thousands and thousands of such mappings and let the machine learn how to associate bits of the input, the statistical
30:36 Speaker 02 associations between parts of the input and the parts of the output, and it will automatically learn that. And so this was the statistical learning phase, the second wave of AI that really picked up steam in the nineties and 2000s. And if you look at 2010s, there was even further boost in terms of these architectures called deep neural networks that, together with the foundation that people built, really turbocharged the statistical revolution. So that's where we are presently in terms of where AI is.
31:18 Speaker 00 We're going to take another quick break to hear a word from our partners. We'll be right back with more of the day's top analysis on the Real Vision Daily Briefing.
31:29 Speaker 01 Yeah, thank you. And I mean, it's just such a fascinating and expertly delivered, very brief history of machine intelligence. And so you have, you know, the initial first wave, which, as you say, almost grew with the birth of computing itself. There was an almost naive optimism that we've created computing, it can't be long until we create a human-like intelligence. And the root to that is via symbolic representation. Am I right in understanding that part of the reason that fizzled out is because?
32:07 Speaker 01 That symbolic approach is can be effective for highly structured sort of rule-bound domains. Like, for example, chess, you know, it's a it's a very structured, rule-bound game. I mean, there's literally a set of rules. And and and you can sort of hard code those rules in. It's pretty laborious and then and then apply just brute force computational power to get answers. But if you think about, for example, how do we get a car to drive around, you know, the open road? There's no set, you know, I mean, there's loot, there's no strict set of rules for dealing with all that elements. It's it's an extremely fluid environment, an unstructured environment.
32:53 Speaker 01 And that's where the symbolic approach tends to break down. That's my like amateur understanding amateur compared to yours, understanding of of this. Is that right?
33:03 Speaker 02 That's absolutely right. Um. So where the symbolic approaches were basically incomplete um for complicated domains. Incomplete meaning that they were not able to capture all of the things that are going on. Like, for instance, you talked about cars driving. So there are a lot of things that are going on in terms of the environment. And can you have symbols and rules that capture everything? If you can't, then it's incomplete. That's number one. Number two, the rules may not also be what we call sound in the sense that. I may come up with a rule that says that um, oh, um, if you see a car in the other lane, then don't switch lanes or something.
33:48 Speaker 02 But in certain cases, you may need to violate that. If let's just say another car is just coming speeding behind you, you're just going to switch lanes no matter what. Right? So there are many instances like that where rules can be wrong. So they are mostly right, but they are not going to be perfect. And so that's another big part of why these symbolic approaches failed. There is also, you talked about chess, right? Where with the deep blue system. It is very, very computationally intensive to roll things out in terms of, okay, so I will play this and the opponent will play that and, you know, what's gonna happen.
34:28 Speaker 02 And so they needed a supercomputer for this. But these days, using statistical approaches, we have very simple, very small programs on our computers that are even better than Magnus Carlsen, for instance, or any human chess player. So it turns out that even in highly closed domain settings, it is computationally better to use statistical approaches. And so for all these reasons, but primarily what you, I think, so nicely talked about is where the statistical revolution really won out. And that's where every
35:11 Speaker 02 pretty much every AI system that we are presently using uses the statistical machine learning.
35:17 Speaker 01 Yeah. And I remember the huge excitement when Deep Blue beat Kasparov. And it's kind of mind blowing now when you load a chess program on your, on your phone and realize that it's, it's more powerful than any human chess player has ever been.
35:33 Speaker 02 Right.
35:34 Speaker 01 That was kind of one of the high points for the symbolic approach, certainly in the popular consciousness. Chat GPT, I, I think it's fair to say is one of the high points for, for all of us and in the, in the public conversation for the statistical approach, right? Because large language models are, represent the statistical approach.
35:55 Speaker 02 That's right. So, yeah, if you think about these large language models, the way they work is that I have a sequence of words and I need some way to compactly represent the sequence of words. So let's just say the sequence of words is an entire novel, right? But we need some way to just summarize that in some fixed length storage. But then, given this, I also need associations between this and what will come next.
36:32 Speaker 02 And so this is basically figuring out statistical associations on steroids. Where I want to compactly represent a sequence itself, I need associations between parts of the sequence. And then I'm going to figure out statistical associations between word sequences and what's going to come next. And this turns out to be very good with things like grammar, because that's basically what grammar is, right? Where you see all of these things and what should come next? There are certain rules that we have, but these rules are again quite complicated. Sometimes you follow them, sometimes you don't.
37:15 Speaker 02 It's complicated in many ways. And we presently are using these large language models to automatically figure it out just from statistical associations with a large amount of data.
37:29 Speaker 01 So you have these models essentially learning about the deep statistical relationships between words and even parts of words as we, you know, use them in the English language, trained on this vast amount of text. And that just gives them, I mean, people have been surprised at how competent that makes these models at producing language that pretty much sounds as though it could have been produced by a human being. I mean, that took even experts like you by surprise, right?
38:01 Speaker 02 Absolutely. I think nobody expected the statistical association based models to be so good. We always knew that with bigger models and with more data, these models would get better. So that, there was no doubt about. But the capability that we are seeing, that has really caught even researchers by surprise, definitely. What it suggests is that a lot of.
38:37 Speaker 02 I guess statistical associations capture a lot of the things that we care about. Um, but not all, um, as I'm sure we will come to.
38:47 Speaker 01 Yeah, well, exactly. And let's, let's come to that because. You know, I won't give the game away for you, but essentially you believe that. To move to the next big, to, to really advance from where we are now. And in particular, to see an impact on, on the economy and to make this AI moment truly useful. And impactful, we need to move beyond this statistical approach moment. Um, talk to us about that. Why are we hitting the limits? What limits are we hitting when it comes to the statistical approach? And why do we need to move on to get to a more useful place? And what does that look like?
39:28 Speaker 02 Right. Um, so if you think in terms of a lot of a formal knowledge, let's think about Latin number theory or algebra. Uh, linear algebra and so on. So some of our advanced math courses. Then a lot of it is relationships among different things, right? Um, relationships between numbers, relationship between symbols. And it is not always the best thing to. Talk about this knowledge in terms of statistical associations. So for instance, number theory, right? So I may have various theorems involving primes, for instance.
40:09 Speaker 02 And, um, it is not always possible to represent this knowledge. By saying that, okay, so if I see one and then a plus and then another one. And it equals, then the next thing should be two. Now, this is an important, uh, number theoretic fact. But it is not always possible to just represent all of number theory, just using these statistical associations. It is also not efficient to do so. Um, so it's both the capacity as well as, uh, efficiency reasons. Um, let me actually give a, uh, uh, an example, an application that we in our present day may want. So let's say I'm a hospital system.
40:54 Speaker 02 And I have a little bit of unstructured, a little bit of structured data in terms of what my doctors are doing, what are treatments they are giving, how the patients are fed, how much it costs, how much the hospital is making. And if you think in terms of what the hospital administrator might want, they may want certain insights in terms of what are certain procedures that are costing the most or are the least cost effective, or things that are most helpful for patient outcomes, who are the doctors that are really good. These kinds of insights have numeric
41:35 Speaker 02 reasoning that is critical to make, to come up with those conclusions. Or let's just say I'm
41:47 Speaker 02 a company and I have a whole bunch of branches, a bunch of shoe stores, let's say I'm Nike and I have a bunch of shoe stores. And I want to look at the results of all of the shoe stores to figure out what are things that are working, what are things that are not. And so here I need to do numeric reasoning that is much, much more complicated than just simple statistical associations between symbols like numbers. And right now, if you're going to use chat G P T or large language models to surface these, it's going to hallucinate, it's going to think of statistical associations that could be interesting, but by and large not going to be useful. So what
42:29 Speaker 02 we really need is essentially the kind of, I think it's interesting to think about the kind of human culture itself, our scientific enterprise itself. We have developed things like algebra and other number theory and so on quite late, right? We had developed language seventy thousand years ago. But and if you think in terms of associations between bits of images and whatnot, that actually goes back to originally the Cambrian explosion millions of years ago.
43:10 Speaker 02 All species have vision, for instance, and what is vision but statistical associations between all of pieces of the image? That's the only way they a lion is able to say, hey, there is a gazelle, I'm gonna go, I should go and eat it, right? It has these capabilities, but it wasn't able to develop algebra, right? So we need this additional capability to do the kinds of things that companies want. And there is a reason, you know, lions can develop algebra. Similarly, I think we need to have this additional component that does this kind
43:51 Speaker 02 of numeric symbolic reasoning and calculations that we can rely on.
43:58 Speaker 01 Right, right. So to make this truly, you know, for many corporations, many startups, to make this truly useful and therefore to have that kind of big impact on the economy that people are talking about right now when it comes to AI, you know, in layman's terms, we need to make it good at maths. It needs to be able to handle numbers. You know, I'm kind of unilever, I have like endless data about sales and kind of supply chains and all this. And right now, a large language model can't can't handle that. I mean, we can all step to chat GPT and see for ourselves that it can't really do maths at all.
44:36 Speaker 01 How do we, and this is where some of your work, you can talk a little bit about some of your work. What it sounds like what you're driving at is essentially a new fusion of the symbolic approach and the statistical approach in terms that we ordinary mortals can understand. How are we going to do that? How can that work? And then I'm interested on your thoughts on who is going to do it? Who the winners could be here? Because we're talking about a big next evolution for AI with potentially massive rewards for the players that get it right.
45:12 Speaker 02 Absolutely. And so here, I think, so if you think about the first wave that is symbolic, the second wave of AI that is statistical, we need a third wave of AI that is both statistical and symbolic. Some people call it neuro-symbolic by associating neuro to the statistical revolution. And so what we really need are the statistical symbolic or neuro-symbolic third wave of AI that combines both of these. So one way to think about this is maybe, again, going back hundreds of thousands of years, right, to to to human evolution. So we have a prefrontal cortex that
45:55 Speaker 02 evolved at the very end. And if you think about other parts of our of our brain, our neural architecture, that's basically what our statistical models are. It's all about statistical associations within bits of the input. And we are older parts of the brain are already doing that right now. But if you think about the evolution of our own brains, mammalian brains, then around, I would say, a few hundred thousand years ago, bits and pieces of the prefrontal cortex started getting
46:35 Speaker 02 involved. And I think if you look at actually Homo sapiens, I think it is maybe, again, a few hundred thousand years ago, where it fully developed. So what is the difference? Why do we need the prefrontal cortex? It's basically symbolic reasoning, sitting on top of the kind of statistical associations. So what we really need to build is the prefrontal cortex that's
47:05 Speaker 02 We can evolve it at the end, but now I think it's the right time. We now have evolved the other parts of our brain, done a reasonable job with that. But we need to have this more symbolic element. And I think a lot of the architectures that people are exploring are along similar lines, where you actually have something that sits on top and that synthesizes. That we can think of it as a hierarchical architecture, that you have a more symbolic module that sits on top of this statistical module and that synthesizes things into symbolic elements and does reasoning with it.
47:48 Speaker 02 So a lot of this, people are still developing. So I can tell you that DARPA is investing heavily in this, in third wave AI. So I'm part of basically, so DARPA has funded my group as well. And they have funded a few groups all over the country to push this neuro-symbolic AI forward. DARPA is always forward-thinking. So whatever DARPA is doing, it's going to be the state of the art in a few years, in two, three years. So you're already seeing, so you talked about the rabbit companion.
48:31 Speaker 02 So if you look at the presentation that the CEO made, you may have heard the phrase neuro-symbolic. Yeah. And what I would say is that you are going to see this phrase neuro-symbolic more and more in successful companies, because you do need this third wave to actually build something useful. And really, that's.
48:59 Speaker 02 In, I would say, in a year or so, you're gonna get, again, I don't know how much Rabbit is doing in a neuro-symbolic direction. But I think in a couple of years, by twenty twenty five, you're definitely gonna see companies use this neuro-symbolic technology in some way at least.
49:17 Speaker 01 Do you think this is the direction OpenAI and DeepMind are pushing in? Because we all noticed at the end of last year, you know, DeepMind's Gemini multimodal model is apparently very mathematically competent. Then there was some ruckus about how much of that demo was a little bit massaged or faked. But it does seem to have maths competence. Have they achieved that by, as you say, kind of bolting on some form of symbolic reasoning to the underlying statistical model?
49:50 Speaker 02 Not yet.
49:51 Speaker 01 Right.
49:52 Speaker 02 The, the, the way, the way that some of these companies, again, there, there could be different approaches that they are taking is. So one example, which actually Google came out with a year or so ago, it's called Chain of Thought. Where it is, if I think in terms of, oh, one plus one equals two. It is good at figuring out just from the statistical associations what should come next. But if I have a long word puzzle and I'm asking it for the answer. It won't be able to do that just from statistical associations. But if I ask it to go step by step, then each step it may have seen from the statistical associations in the data in terms of what should be the next step.
50:38 Speaker 02 And by forcing the model to go step by step, you can actually do a better job than what it was doing previously. But it's still not a, this kind of a neuro-symbolic thing where it's not, still not something you can trust. And so right now, these companies are still in the statistical wave and not the neuro-symbolic wave, primarily because.
51:09 Speaker 02 There are many, many ways to do that and it's still not fully clear. Uh, we are building the foundations of that. So I would say, but things are moving so fast. So I think it's still a year out, I would say, in terms of how to achieve this.
51:22 Speaker 01 Yeah, but I mean, this is so profoundly important for people to understand that we're in this symbolic, sorry, this statistical wave right now. And the next big wave is coming and it's going to be about this fusion of the old symbolic approach. Updated for a new age with the current statistical approach. We're going to hear a lot more about, you know, neuro-symbolic AI. Um, and just as you say, you know, and I love this comparison to the human brain.
51:53 Speaker 02 Right.
51:54 Speaker 01 And we'll talk a bit about that in a second, you know, our brains do both. So if the dream is a thinking machine, a machine that is, i- is in some way comparable to, to the capabilities of the human brain, you would expect you have to have both approaches.
52:11 Speaker 02 That's right.
52:12 Speaker 01 Who else is positioned to win? I know you have some interesting thoughts on that, you know, I mean, OpenAI must be thinking about this, DeepMind must be thinking about it. Who, who else?
52:23 Speaker 02 I think, um, OpenAI and Google and so on, primarily because of the business models are focused on modalities like text and images. Because that's where the large amounts of data lie. You need that to build the statistical models. But if you think in terms of when neuro-symbolic methods are required, it's probably going to be in domains where you do not have such large scale data. And I think we, probably the dark horses are going to be companies that are working with data. They are working with databases.
53:01 Speaker 02 So I would say Oracle is an example of a company that is working with databases. So I mean, if you look at most enterprise customers, a lot of the data is, they do have documents and images, of course, but a lot of the data are in relational databases like Oracle. Um, and there are of course other companies as well, but. If companies like Oracle can take some of the bones of the statistical revolution, combine that with the the symbolic components and have a neuro symbolic solution,
53:43 Speaker 02 that can mimic how, let's say, human data analysts work with these with these databases. Databricks would be another example, but these would be the kind of dark horses, not just Google and OpenAI, where a lot of what they are developing are probably going to be commoditized to a large extent. But again, you know, we can never tell how much things can be commoditized. Because if you think about Google search, why wasn't search commoditized? Why is Google still so predominant in search? Because that's just retrieving relevant web pages from a corpus of web pages.
54:25 Speaker 02 We know the technology, why aren't there a million competitors? So I don't want to say that I'm sure things will be commoditized, because there are network effects, or there are not network effects, but I guess there are kind of, if people are using something, they will continue to do so. But I think that companies that are going to push in this neuro symbolic direction for enterprise customers and other customers. Because again, if I think in terms of our own everyday lives, so we have our health monitoring devices and all of those are numbers.
55:08 Speaker 02 So I want my AI to understand me in all my numeric glory. And for that, we need neuro symbolic models.
55:18 Speaker 01 Right, and I mean, Oracle are fascinating, you know, obviously tech giants, but not not one that people would immediately step to when you talk about the future of AI. So many of us will have heard that here first, and we will be watching Oracle and this and this neuro symbolic wave that's coming with great interest. And as you say, it's going to be super interesting, because of course, OpenAI is, you know, is trained.
55:47 Speaker 01 On a massive amount of text that was just sort of free floating out there that they could just grab. And that's now somewhat controversial, but essentially they just hoovered up a whole ton of like human culture and used it to train this this model. Um, there isn't necessarily the same giant numeric data set, right? Just floating around out there. So it's going to be the people that have access to those kinds of data sets.
56:17 Speaker 02 Exactly.
56:17 Speaker 01 That opposition to win here, right?
56:20 Speaker 02 Exactly. That's right.
56:21 Speaker 01 Yeah. Yeah. And so that that's where someone like Oracle comes in. And, you know, if they have a ton of structured numerical data that they can train on, even if they train something that only people using, you know, Oracle databases can use, that's still massive. Um, yeah, it's very, very interesting. Okay, look, it's just been so, thank you so much for joining us. I mean, we could go on for another hour. It's such a fascinating conversation, and we must have you back because the unfolding of this big third wave, which I don't think, which I don't think many people yet are aware of, but is coming and is so important. I think it's
57:01 Speaker 01 a fascinating story to track. And I'd love us to track it, you know, through the year, through through the next few years. I know many people out there will be watching and wondering on something you said earlier, or one of the themes you touched on earlier, which is, look, excuse me, this, this kind of mirrors the way the human brain evolved. You know, we have this underlying statistical approach.
57:29 Speaker 02 Right.
57:30 Speaker 01 But then we evolved this sort of symbolic representational intelligence too, and human intelligence is about both.
57:38 Speaker 02 That's right.
57:39 Speaker 01 Does that mean that this fusion of statistical and symbolic is the route to the fabled AGI? You know, a truly generalized human-like intelligence and then even superintelligence?
57:55 Speaker 02 Right. Um, I, um, uh, you know, so there's a joke, right? They say that if you don't have any idea, then something's going to occur. Say it's going to be 20 years out, going to be 30 years out. And so you have all of these surveys that say that researchers say that, oh, it's twenty thirty years out. You should always take that with a grain of salt, because then if somebody says something is twenty thirty years out, they're essentially saying, I have no idea, right? So maybe it'll occur, maybe it won't. I think understanding um the kind of human intelligence, or even it doesn't need to be human, having a completely general intelligence
58:42 Speaker 02 is a very, very ill-defined thing. I mean, uh uh uh being able to understand language and images, ChatGPT or large language models can already do that. And that is a general intelligence to some extent, I would say. Um but it's still, I would say, very far from the kind of general intelligence that we would associate with humans. So I think AGI in the very ill-defined sense that people have in mind is very, very far away. I wouldn't find me the bold claim that it's not twenty thirty years away, it's hundred years away, because
59:24 Speaker 02 if you, again, think about the waves of AI. We had the first wave that mirrored the computing revolution. Um and then you had the second wave, the statistical revolution, which um again, you know, a a a lot of it was uh developed over decades even prior to the development of computers, because the field of statistics is much, much older. And regression was developed by Galton in the late nineteenth century, for instance. But I would find it interesting that regression came after calculus. It's a more subtle, statistical associations are more subtle
1:00:07 Speaker 02 uh to develop algorithms to to infer those are more subtle than even calculus. But that took so many years, I wanted to say, because, you know.
1:00:19 Speaker 02 Now we are coming up to the third wave, which is also to flesh this out fully is going to take many decades. We will come up with solutions, don't get me wrong, but to fully flesh that out is going to take decades. And that's just three waves. And it seems hubristic to say that that's all we need, right? To, to, this is what we can see presently, but there are things like, for instance, our working memory. So here is a mystery. Our working memory in the prefrontal cortex can only handle 10 symbols at a time. Why is that? What is so critical to having 10 symbols? That cannot just be a capacity thing,
1:00:57 Speaker 02 because our brains can handle a lot of things. Why? So we have no idea, right? And we are, there's just so much we don't understand. And I imagine that it's going to be, you know, five, six waves of AI, you know, it's going to take 100 years. So it's probably a problem for our grandchildren, probably.
1:01:15 Speaker 01 Right. Fascinating perspective. And certainly, it feels to me that the, that the reflected light this journey is throwing on the nature of human intelligence and sort of essentially how much of that we don't understand is, is also a thrilling part of the, of the picture. But look, Pradi, thank you so much for joining us. Absolutely fascinating conversation. Where online, if anywhere, can people learn more about you and follow your work?
1:01:46 Speaker 02 My website is where all of my research is available. I just came onto Twitter a couple of days ago.
1:01:57 Speaker 01 Well, you have, you have some fun in store. Welcome.
1:02:02 Speaker 02 Right. So.
1:02:04 Speaker 01 It's not, it's not always, you know, it's not only highly civilized and sort of intelligent conversation about machine learning, I hate to tell you, but it can be a lot of fun.
1:02:14 Speaker 02 Right, right. So it's all, so I guess it's in the second wave, it's all in the statistical associations.
1:02:22 Speaker 01 Well, he's training his own model, isn't he, Grok?
1:02:25 Speaker 02 Yeah.
1:02:25 Speaker 01 So yeah, we'll see how that plays out as well, especially with an election coming up. But, okay, so your website, people can Google you, find your website, check out the fascinating work you're doing. Thank you so much for joining us. And like I say, please come back in future because we will be tracking, of course, continuing to track this incredible AI moment we're living through. Where it's heading, what it means for all of us as professionals, as investors, as human beings, what it means for the for the economy at large out there. We'll be continuing to track all that. And the exponential age theme on Real Vision all through this year, and I cannot wait. It's going to be a fascinating ride.
1:03:08 Speaker 01 But for now, Pradeep Ravi Kumar, thanks so much for joining us, joining us and everyone out there. See you soon.
1:03:15 Speaker 02 Thank you.
1:03:21 Speaker 00 Hey, everybody. Today's Real Vision Daily Briefing is sponsored by Engrave, maker of the coldest hardware wallet, Zero, and stainless steel backup graphene. Engrave brings you the highest security in a touchscreen experience to safely manage all your crypto offline. Enjoy a 10% Real Vision discount in engrave.io shop with the code REALVISION.
Previous How Blockchain and AI Can Lead an AR Revolution Next #962 - Will Tech Earnings Deliver? with Dale Pinkert Page updated October 20, 2026 · transcript and summary are machine-generated