The Next Gen AI Models: Reliable, Consistent, Trustworthy — With Aidan Gomez

Aidan Gomez is the co-author of the "Attention Is All You Need" paper that launched the AI revolution and CEO of Cohere, an enterprise AI company. Gomez joins Big Technology to discuss the myths, facts, and realities of today's AI landscape. Tune in to hear why the real value of AI isn't in flashy consumer apps but in automating crucial back-office processes that could save businesses billions. We also cover the truth about AI capabilities, the likelihood of AGI, synthetic data training, and whether an intelligence explosion is possible. Hit play for a refreshingly grounded discussion about where AI is actually making an impact, from one of the field's pioneering voices. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Want a discount for Big Technology on Substack? Here’s 40% off for the first year: https://tinyurl.com/bigtechnology Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  An author of the original paper that launched the generative AI revolution joins us to sort out the technology's myths and facts. And when we'll see a return on all that investment? All that and more is coming up right after this. This episode is brought to you by Genesis. What does it actually take to put a Gentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience? And what companies need to get right in the AI era? Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial to go inside a large scale agentic CX transformation, including the business decisions, trade offs and lessons along the way.

• 0:44Alex Kantrowitz?

  You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility and peace of mind. That's how companies like Razor were able to cut their inference cost by 15 times. Visit akash.network slash Alex for a free consultation. That's akash.network slash Alex for a free consultation. Welcome to Big Technology Podcast, a show for cool headed, nuanced conversation of the tech world and beyond. We have a great show for you today because we're joined by Aidan Gomez, the CEO of Cohere, an AI platform for enterprise.

• 1:26Alex Kantrowitz?

  And he's also the co author of the famous Attention is All You Need paper, which invented the transformer and started this whole AI thing. Aidan, great to see you. Welcome to the show. Thanks, Alex. Thanks for having me. Great to have you here. I want to begin with some myths and facts about AI. We have debates all the time on the show. Where's the technology going? Is it worth the investment? And no better person to ask than someone who was there at the beginning and now currently an entrepreneur in the space. So earlier this month, OpenAI raised 6.6 billion, but the reporting says they might be losing 5 billion per year. And, you know, in some ways, okay, you need the investment to build the models.

• 2:08Alex Kantrowitz?

  But in another way, it's like, okay, you know, where does this end? Because the compute, the data, the energy to train these models keeps getting bigger. The requirements for that keep getting bigger and so does the money. And, you know, it does this ever become sustainable? So what do you think?

• 2:24Speaker 01

  I certainly understand the urge for people to see the numbers being spent on training and be concerned that it's not going to recoup in value. But I think that those numbers are actually small relative to the long-term value that the technology will deliver. I think it is now time to prove that. So last year was very much the year of the proof of concept. People were getting familiar with the technology. It was their first time working with it. And so there were a lot of small tests and experiments. But this year is very much one of going to production and getting these models into the hands of people at scale.

• 3:05Speaker 01

  Of course, we're already seeing a high degree of ROI in the sense that there's now hundreds of millions of people who are using the technology. It's actually in their hands. It's part of their day-to-day. And so that certainly is ROI. And with what Cohere focuses on, the enterprise side, we're starting to see this technology get into the hands of employees and get into enterprises. It's a much slower process. It's a bigger lift. You have to integrate with existing systems within enterprises. You need to train employees on how to use this technology. But that's well underway now, and we're seeing quite dramatic growth in adoption. So I think we will find ROI, and it's coming soon.

• 3:48Alex Kantrowitz?

  And we'll talk more about the specifics of Cohere and ROI in the second half. But let's keep on this line because it goes to another one of these myths and facts, which is that the next set of models are going to be this godlike set of models. And you've talked about how there's going to be a lot of cost at the beginning, right? And that's necessary cost to train these massive models. And the sense that I've gotten and from my reporting, one of the things I've heard is companies have been willing to make those investments because they think that this next 12 or 18 months in model development is crucial, and the capabilities will advance significantly. As they put more compute data and energy into the process.

• 4:33Alex Kantrowitz?

  So let's go to this myth, in fact, number two, which is, does the next set of models give us that, you know, godlike AI model? And so many people are

• 4:42Speaker 01

  I don't know about godlike. I don't think I'd ever use that term to describe what's coming. I think we're going to have some really powerful and useful tools emerge. I think that's what's coming. The idea that we're building AGI or something that's just going to like solve all our problems for us. I think we need to set that aside. I don't actually think that's

• 5:06Alex Kantrowitz?

  We'll get to that. But let me ask you more pointedly on this, on this next generation of models. Okay. So you say we're going to see some new tools. What does the next generation? Because they are being trained on much more resources than, than have they have been previously. So what tangible step forwards are you expecting to see from the next generation of models?

• 5:25Speaker 01

  I think the reliability and trust factor is huge. And also just the competency, right? The accuracy with which it gives you answers. And so all of those are going to increase. I don't see a step change coming, but I see a steady, continuous course towards very high accuracy, very high reliability AI.

• 5:50Alex Kantrowitz?

  There's been so much hype in the industry. This is one of the things that sort of comes along in this discussion, which is like everybody I speak to who's on the ground says, yeah, we're not going to see a step change with like, let's say, GPT-5, but exactly as you described, more reliability, more consistency. Is there, I mean, is there worry that some of the air is going to come out of the, you know, this AI moment? If, you know, because again, like I, when I say godlike, I'm not, I don't believe that's going to happen, but I'm reflecting what a lot of the hype is starting to expect. And so if it's just steady, you know, steady improvements in reliability, which is actually like, you know, we both agree pretty big. But do you think that that sort of takes some of the steam out of this moment for AI?

• 6:34Alex Kantrowitz?

  Because people will look at the step change as, as a failure, given where the hype is.

• 6:41Speaker 01

  Well, listen, I'm not one of the people who's saying we're going to be building

• 6:44Alex Kantrowitz?

  Oh, yeah.

• 6:45Speaker 01

  Godlike AI.

• 6:46Alex Kantrowitz?

  You're doing it.

• 6:47Speaker 01

  Yeah, I don't, I don't have much to say towards those claims. What I would say is that even if, you know, like, just as a hypothetical, even if the technology froze and what we have today is all we get, there's so much good to be done. There is so much work to go do to implement this technology across the economy, really boost productivity, drive better outcomes, build tools. So the technology does not have to move in order for incredible value to be realized. We just need to go do it. And it takes a lot of effort and time and work to go realize that value.

• 7:31Alex Kantrowitz?

  Okay. And again, more of that is coming in the second half where we go a little bit more tangible. But let's stay with the theoretical, or at least like the industry stuff. What do you make of the fact that the GPUs, so we talk about like the ingredients? Again, this is coming from your paper, right? The ingredients that are required for these models to get better. They need data, they need compute, need energy. And the compute right now is starting to go like through the roof in terms of the amount of compute that's being used to train models. So just for some context, so Meta's Llama 3 model, which was like state of the art like 10 minutes ago, it used 16,000 GPUs to train that one. Now we're hearing that Elon Musk is building this super cluster.

• 8:16Alex Kantrowitz?

  I think it's called Project Memphis that has 100,000 GPUs. So multiples of what the cutting edge is being used to train on. So I'm curious if what, what you think that increase in GPUs are going to get us first and foremost. And then we'll talk about whether the right way to scale these models is with just throwing more compute, because I know you have a nuanced take on that. But like first and foremost, like if you go from 16,000 GPUs at the state of the art to 100,000, what do you think that delivers?

• 8:50Speaker 01

  Definitely delivers a bigger and better model. You have more compute. We know that scaling up improves things. There's questions around saturation and whether continuing to scale up is justified, whether there's going to be enough gains from that strategy to justify the increasing cost. My personal perspective is that, you know, building a massive model, it's not actually useful for the world if it's too big to be consumed, if it's too expensive to actually deploy. And so for Coher, we've been very focused on building the right size of model. But if your question is, what is more compute unlock? It will be a better model. Objectively, we know that scaling leads to more capability, a smarter model that's

• 9:34Speaker 01

  more reliable. And so that's the output.

• 9:38Alex Kantrowitz?

  And does that ever end? I mean, that's one of the big debates here is that, you know, basically you could add compute and data basically to infinity and it will continue to improve. Or is there a tipping, you know, sort of a saturation point?

• 9:52Speaker 01

  I don't think within any achievable scaling up for humanity that we'll reach that tipping point. It just saturates. The gains become much, much smaller. And so you're much less willing to want to pay double the price for a minute difference. But it is pretty consistent that bigger is better and that just continues, but it tapers off over time.

• 10:19Alex Kantrowitz?

  I mean, OpenAI has talked about how like their goal is to build AGI. A lot of people in the industry talk about AGI as North Star. I know Coher is more like, let's make this practical for businesses. But I want to get your sense because because that's not your North Star, I think you can speak a little bit more about like, more honestly about what it means and whether it's achievable. So do you think that, let's just use this definition of AGI as intelligence that's as capable as humans in the tasks that humans do? Do you think that that is something that we should even be thinking about? Or is it a marketing tool like, and is it achievable?

• 10:57Speaker 01

  I mean, with that definition of AGI, I think it's both achievable and a fairly reasonable target. So we can, we can measure how good humans are in any particular task. And then, yeah, I mean, it's a reasonable goal to want to create technology that can perform that well in that task.

• 11:16Speaker 01

  So I think, based on that definition, I think when we start to think about, you know, you described it as like godlike models. These are being described as well beyond.

• 11:28Alex Kantrowitz?

  So yeah, so I think my definition is probably artificial general intelligence. And I think that this godlike is the superintelligence thing that a lot of, and I think a lot of people will use AGI as a synonym for superintelligence, which seems wrong to me, but there is this belief that once we hit AGI, we've already reached superintelligence because if it can do everything that humans can do and doesn't get tired, doesn't need to sleep, doesn't need to get paid necessarily, you're already at superintelligence. But sorry, go ahead.

• 11:56Speaker 01

  Yeah, but no, I think that that definition of AGI is a reasonable one. And I think it is exciting. I think that that's, that's definitely the target. What we want to do is we want to create machines that have this unique property that humans have of intelligence. And we want to be able to deploy them in the places to take work off of the shoulders of people and then put it on to these machines to make work better and easier. And in order for you to do that, in order for you to trust the machine enough to shift that work over, it better be as good as the human. Otherwise, you're, you're paying some price, you're reducing inaccuracy. Things get worse, not better.

• 12:38Speaker 01

  And so that, that's a very reasonable objective.

• 12:43Alex Kantrowitz?

  And when do you think we might reach it?

• 12:47Speaker 01

  In many respects, we're already there in many fields. The models are as good.

• 12:52Alex Kantrowitz?

  Right, we're still like.

• 12:54Speaker 01

  Yeah, but I don't think those two things are in conflict. I don't think that.

• 12:57Alex Kantrowitz?

  Really, why not?

• 12:59Speaker 01

  Well, because I think that we will never see mass unemployment of humans. I think that this technology is going to unlock more opportunities. It will let us do more as opposed to scaling back what we do. Humanity is very. Supply side constrained, not demand side. We always, we want more, we want better, we wanna be healthier, we wanna do more, we wanna have things be cheaper. And so we have all this demand and we're trying to keep up with our own society's demand. And this technology, its true promise is in bringing productivity and letting us do more.

• 13:43Speaker 01

  Now you can zoom in and you can like pick a specific field and you can say, this field might be automated by AI. And I think that's true and you know, we should be thinking about retraining and shifting certain skillsets over to other new domains, like retraining people. But in general, at the macro scale, I think this technology will create much more opportunity than it will take away.

• 14:09Alex Kantrowitz?

  I mean, if we have AI technology that can basically do work for us, whether it's knowledge work or whatever, right? And we already have a lot of technology that can automate, you know, factory work. Why are we continuing to work?

• 14:23Speaker 01

  It brings purpose and meaning to a lot of lives and we enjoy it. I think that the right form of work is something that fulfills you and that is enjoyable, intellectually interesting, compelling. And that's really what I wanna spend my time doing is that, as opposed to number crunching or, and maybe someone else enjoys number crunching, but for me, I'd rather outsource that to my Excel spreadsheet. So yeah, I think that work in its best form is incredibly fulfilling and that's never, that's something that humans will never give up, will always wanna do that. But if we can hand off and if we can have an assistant that is, you know, on 24/7 and

• 15:08Speaker 01

  has access to all the information and tools that I have access to, and I can ask it to do things for me, that's a very compelling value proposition. It changes work in a way that is extremely positive, I think, for, for almost everyone.

• 15:21Alex Kantrowitz?

  How far away do you think we are from having reliable assistance? Like, a lot of people looked at OpenAI's O1 reasoning model and they're like, oh, this is just kind of like a step toward assistive AI. What do you think?

• 15:34Speaker 01

  I think the notion of using reasoning or letting the model have an inner monologue to work through problems, think through them, make mistakes, but then realize that, catch mistakes and correct them. I think that's a crucial piece in improving not only the accuracy or robustness or usefulness of the model, but also the trust in the model. Because you're able to inspect how it arrived at its conclusions, how it decided to do what it did, you actually trust it much more. It's explicitly written out. And so I think we've all known these sorts of tools would need to emerge.

• 16:19Speaker 01

  And yeah, I think it is a big step towards dramatically more reliable assistance, ones that you can trust and work with and give feedback to. I think it's really exciting.

• 16:28Alex Kantrowitz?

  Okay. And then where does that put you on the fear around AI? I mean, if AI can sort of go step by step, figure out these processes, realize where it went wrong, go back, take action, right? I think that's sort of where people get weirded out, is when these things start to take action on their own. What can they do that we're not prepared for? So what do you think about that? Are you worried AI might cause harm to people?

• 16:52Speaker 01

  I think it's really important to remember that we get to choose where we deploy models. It's not like they get to choose where they work or what they have access to. We have to plug them in and we have the opportunity to implement safeguards. So to make sure that before these models are put in any very high-stakes situations, that there's oversight, that a human has to approve high-stake actions. It's not carte blanche and the model is now smart and we just plug it into everything and say, go at it. It's very much intentional and we're going to need to be thoughtful and careful about that. So I'm not scared of like a doomsday like Terminator.

• 17:35Speaker 01

  Scenario, I think that media has certainly instilled that with lots of sci fi stories. And it's a very compelling story, which is why well before AI was remotely competent, we were coming up with stories about how this might happen.

• 17:49Alex Kantrowitz?

  Yeah, but it's not just media, right? It's like also AI leaders are saying, you know, how many people signed that statement that said we should be treating AI risk the same way we treat climate change and nuclear? Why do you think there are so many people in the industry that are stirring up the fear around this stuff?

• 18:09Speaker 01

  I think that's a great, that's a great question to ask them. I did not sign that letter and so.

• 18:15Alex Kantrowitz?

  So it puzzles you as well?

• 18:17Speaker 01

  Yeah, yeah. I mean, I'm empathetic to the fears because yeah, like this is a very salient story. That's why it's been so popular in sci fi and all this sort of stuff. So I'm actually understanding of why people are so attached to those stories. But as more and more evidence emerges that these models are much more controllable than we may have thought, that they're a little bit less capable than we may have thought. It's harder and harder to make that narrative. And I think you see the discourse shifting now. I think the discourse has begun to shift away from doom and existential risk. And now it's much more about practical concerns, which I'm really happy to see.

• 18:58Speaker 01

  Stuff like, okay, this technology could be really useful for healthcare, but it could also cause harm if we don't do it the right way. And so specifically, how do we set up the safeguards to make sure that harm doesn't happen? Same thing with finance, right? And like distributing loans or something like that. Or with people using them maliciously to pretend to be human and trick people. How do we prevent those things? That discourse is super productive, but that's very effective. And so things are shifting in that direction now and I'm excited to see that change.

• 19:29Alex Kantrowitz?

  Now, some of this fear comes from this line of people saying, oh, there are emergent behaviors in the models, right? That basically that they've found them able to sort of come up with things that are outside of their training set. And there've been some papers that say, okay, actually, they don't really have any emergent properties or emergent behaviors. And as someone who wrote the paper that kicked this all off, what do you think about that? Can LLMs have any emergent behavior or discoveries that they weren't trained on?

• 20:00Speaker 01

  I think that they can, what's the right word? I think they can interpolate between skills. And so if they've seen how to do A and they've seen how to do B, they can get kind of the average of A and B. But they don't just go completely beyond anything that they've seen. I've never seen a model behave in a totally unexplainable way. They're really good interpolators. If you show them different domains, they can blend domains quite well. But yeah, I've heard the same thing about emergent behaviors. And I think the research is really inconclusive there. There's not a lot of compelling evidence that says we're going to have some total step change or capability takeoff.

• 20:41Speaker 01

  Even in the latest state of the art research, a lot of it's about synthetic data and models teaching themselves. And so self-improvement is this notion of can a model actually teach itself without human intervention? This is now a huge part of model building. It's a big part of how we create data at Coher. And before this started to become mainstream and actually part of the production process of creating these models, people were saying self-improvement. These things are just going to take off. They're going to become superhuman overnight and we won't be able to control it. Well, it turns out that doesn't actually happen.

• 21:17Alex Kantrowitz?

  Right. So this intelligence explosion or intelligence takeoff is not something that happens.

• 21:23Speaker 01

  No, it's not happening. It's not happening. It improves, it can self-improve for a while and then it tapers off. And so, yeah, you get some good improvement out of it, which is why we use it. But then it plateaus. It doesn't just keep going forever. And so I think the evidence points firmly in the direction of a lot of those fears may have been misled.

• 21:42Alex Kantrowitz?

  Now, talk a little bit about that. It's interesting you bring up the use of synthetic data and having the machines self-improve, because one of the big questions about whether this plateaus is, you know, does the world run out of data to train the AI? And I was watching one of your recent interviews where you talked about how, you know, back in the day, you could run up to anybody and they can add knowledge to a model, but as the model got smarter, the models got smarter and smarter, it became less easy for people to add supplemental knowledge to them, which points to like sort of running out of available data to make these AI models smarter. So how does like AI-generated data actually solve that problem? And where is synthetic data being used to make these models better?

• 22:26Speaker 01

  Yeah, so I think the example you gave is a good one, like where it's getting harder and harder to get the data that incrementally improves the model. And it's important to note that that's because the model is getting so much better. And so before, we could just grab anyone off the street and they could teach the model something. And then that signal started to go away. And so we had to go to undergrad students in bio to teach the model about bio. And then we had to go to master's students, then PhDs. And we're kind of at that level where we're currently hiring PhDs to teach the model in their specific domain. But then after PhDs, where do you go, right? Like, I guess professors? What

• 23:06Speaker 01

  about after that? So I think the models are catching up with the state of knowledge across a bunch of different fields. I would say that synthetic data probably doesn't get us out of that, that issue. Actually, I don't know if synthetic data, outside of easily verifiable domains like math, it's hard to use synthetic data to drive outcomes. So we'll be able to do it in

• 23:32Alex Kantrowitz?

  So how is it being useful for you?

• 23:34Speaker 01

  Not for making our models fantastic philosophers, or making them fantastic social scientists, or something like that. For that, we rely on humans. What we do use synthetic data for is for crafting how the model responds to stuff, and in domains that are verifiable, like math, like coding. In those places, it's actually quite effective. But that's still a huge domain of interest for people building and deploying these models. We want them to be good at math and computer science. And so more and more, synthetic data is becoming a huge chunk of the data that we train on.

• 24:16Alex Kantrowitz?

  Okay, fascinating. One last part of this discussion is sort of what methods help get this AI to improve? And there's been a question of whether LLMs can take it like all the way, or whether you need to combine LLMs with different forms of training, whether that's reinforcement learning, I guess that's part of it already. But the other side of it is, do you have to like build world models with like robots going out in the real world and learning things like things like gravity? And what happens when you bump into things which you just can't convey in text? So I'm curious if you think the current methods are able to get this field to the promised land, or whether they need to be combined with others?

• 24:57Speaker 01

  There's definitely proof points out there, which suggest large language models, or like the transformer architecture is capable of handling a bunch of different modalities. And so you can merge not just text, but video and audio as well into the model. So you can give them a much more balanced experience of the world. You can show them the world. You can show them videos that demonstrate physics. You can let them see, hear, speak. And so as a platform, it does seem like this is a pretty good platform, as far as they go. There's a more philosophical argument, which has had among academics around, is text enough? Or even is supervised learning enough?

• 25:40Speaker 01

  Is it enough for the model just to observe the world? Or does it need to take part in the world to really understand it? For instance, would you understand the world if you read all of the internet and you watched every video on YouTube? Would you really understand it? Or do you need to actually be embodied? Be a little robot out there kicking a ball or, you know, running down the street? I actually take, I think, the less popular view, which is the internet is enough. And by observation, you can actually learn enough to be extremely, extremely compelling. I, I think that's, if we're talking about AGI and doing things as well as humans do, I think that's a no.

• 26:25Alex Kantrowitz?

  All right. I want to take a quick break, hear from our sponsor, come back, talk about ROI, and then just talk a little bit about like your journey, Aiden, from being somebody who wrote that paper to where we are today. I think it'll be interesting for listeners. So we'll be back right after this. This episode is brought to you by AVPoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data. And most teams have no single way to see it, secure it, and prove it's under control. That's exactly what AVPoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate.

• 27:09Alex Kantrowitz?

  Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. AVPoint, the unifying trust layer for AI. Head to AVPoint.com to see how enterprises deploy AI with confidence. Learn more at AVPT.co/bigtechnologypodcast. That's AVPT.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door.

• 27:54Alex Kantrowitz?

  The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a white glove enterprise security service, they protect- Protect your people with continuous personal data removal, proactive prevention tools and human-led emergency support. So when someone goes looking for your executives, Ironwall ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form and request your free risk assessment. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real world attacks. The link's in the show notes.

• 28:41Alex Kantrowitz?

  And we're back here on Big Technology Podcast with Aiden Gomez. He's the CEO of Cohere, also the author of the, co-author of the Attention Is All You Need paper that kicked this entire generative AI moment off, right? Invented the transformer. Before we get deeply into ROI, Aiden, just a personal question for you. I mean, are you, what does it feel like having seen? What does it feel like seeing your invention?

• 29:08Alex Kantrowitz?

  Being taken in all these like wild directions and sort of being this key moment in a truly like step forward for the tech field?

• 29:18Speaker 01

  Yeah, I mean, it's like beyond my wildest dreams. I think, I don't take full credit for it at all. I assign the overwhelming majority of the credit to my co-authors on the transformer paper. So it's hard for me to accept the reality of what the transformer has accomplished out in the world as my own. But it's so incredible. Like even if I step away from being one of the authors of the paper, the impact and what the architecture has been able to do for the field has been a huge shock, a colossal shock. Just the technology we have today. I thought we'd be here

• 30:02Speaker 01

  maybe in like half a century, you know, not seven years. So it's, it's really surreal and amazing.

• 30:09Alex Kantrowitz?

  Has Google effectively capitalized on it? Given that this came out of Google?

• 30:15Speaker 01

  I think Google has done super well. You know, they supported Google Brain in creating this technology and it's been integrated all over Google.

• 30:24Alex Kantrowitz?

  All right, let's talk quickly about ROI, or maybe let's go deep into ROI. We'll see how we end up here. Again, we talked about all this, all this, this money being spent on, you know, upfront costs, training the models. And you mentioned that even if the technology stopped today, there'd be so much work to do with it because there's a lot of benefit out there that isn't being realized yet. But talk a little bit about the places that you're seeing already getting a return on the investment in terms of implementing generative AI technology. Because I think in the common conversation, people don't even think those places exist, but it seems like you're seeing it on the ground.

• 31:02Speaker 01

  Yeah, I think today we're starting to see it integrated into production. In enterprise, it's much slower than in consumer. There's a much higher lift to actually get it integrated and there's a higher bar of trust necessary to drive adoption. Like I was mentioning earlier, you know, last year was very much the year of the proof of concept. But this year we've started to see it go to prod. So there's some good examples of that with our partner Oracle, which they have this suite of applications, which basically power enterprise, HR, supply chain, all of these sorts of back office functions. And we're powering over 50 different applications within those software tools.

• 31:46Speaker 01

  And so it's actually starting to get into the hands of employees and drive efficiencies.

• 31:50Alex Kantrowitz?

  Wait, hold on, talk about what like what that looks like and for an employee on the ground there. How does the software that they were working in change when you put generative AI in it? Do we go here?

• 32:01Speaker 01

  Yeah, so you're, you're automating parts of the job, little tasks within the application. You can now just push a button and the model will, will do that. You might need to provide a high level. A good example might be in writing job descriptions, right? So a manager, a hiring manager wants to hire for a specific job.

• 32:22Speaker 01

  What they want to do is just put in bullet points. I need someone who has this background, does this, et cetera. And then press go and it will generate the full job description with everything that the company needs included there. And in a way, it's actually presentable to the people applying.

• 32:39Alex Kantrowitz?

  And that's a known use case? Yeah. So let's hear something else.

• 32:43Speaker 01

  Yeah. Another good example might be in supply chain. When you're looking for an alternative supplier to one of your products, doing that search and retrieval and being able to iterate with a model and not just do a single step search where you search over suppliers, but where you give feedback. You say, actually, no, that one that you just recommended doesn't work for this reason. And you're able to refine iteratively with this assistant or agent. And these models basically touch every vertical. And so there's no particular vertical specialization. It's totally horizontal. So we were working with a legal tech startup that helps with reviewing contracts and building

• 33:27Speaker 01

  an assistant for a lawyer to help them review contracts more quickly, flag concerning terms, that type of thing. We're working with a healthcare startup that tries to use news and social media to track pandemics and are people getting sick in a particular area, reporting specific symptoms? And so using models to screen for that, it really impacts every single vertical.

• 33:51Alex Kantrowitz?

  Can I take the devil's advocate position on this? Let me see if I can channel an AI critic and see what you think about this. Basically, what they would say is job descriptions, okay, it will save you a tiny bit of time if you have AI, right? The job descriptions, if you're looking for a supplier, chances are if you work in vendor management, you're going to have a good familiarity with those suppliers anyway.

• 34:13Alex Kantrowitz?

  If you're a lawyer, like, yeah, it might be a little bit of time, but you can comb through a contract and find out what's, you know, what might be concerning about it. This is your expertise. You're like a, you know, almost like a narrow neural net trained for that one specific purpose. And now we're giving it over to AI. And they'd look at just the billions being invested in this technology and say, well, what am I really getting for that if this is effectively doing some of these things that humans are quite good at to begin with?

• 34:41Speaker 01

  I would counter and say that risk to supply chains is many trillions of dollars. I would say that lawyers are extremely, extremely expensive and you don't want them combing through your documents, no matter how efficient you think they are. Same thing with doctors. We really want them spending time with patients, not combing through hundreds of notes and filling out forms afterwards. I would say those are, maybe this stuff feels banal, maybe productivity feels boring compared to some of the hype of AI, but it is the value. This is what we're trying to build for. And so I would push back quite firmly on that.

• 35:18Alex Kantrowitz?

  I'll react to this. I think this is a thread from Benedict Evans, tech analyst. There's an interesting difference between people outside tech sneering at generative AI as chatbots that get things wrong and make crappy, quote unquote, stolen images, and people inside tech who are mostly working on using it to automate a huge number of boring back office processes inside giant corporations for billions of dollars.

• 35:42Speaker 01

  I think that's a great observation. I think that there are some very superficial critiques of generative AI that have become very popular. I think the substance is in actually doing the work and getting this technology to be productive for humanity. And a lot of people are working on that right now. It's going to take time, like I've said, but the opportunity is immense. It's the biggest in a generation.

• 36:09Alex Kantrowitz?

  Yeah, I think that's kind of the misconception and that's the interesting point about what this technology can do. So I was speaking with Flexport, again, supply chain management, and I think about writing about how the fact that like supply chain is actually like ground zero for where this technology is being applied and useful. But they're basically, they're like getting faxed things, they're getting PDFs to try to log that and comb through that. You know, the volume is crazy and they're using generative AI to read through the documents and give them actionable insights on it. And they're like, look, like it's not going to be the most exciting use case, but this is saving us a tremendous amount of time.

• 36:47Speaker 01

  Yeah, I was about to say, I'm like, it's so boring, but it is so valid. Like, people don't understand the actual scale of impact of some of these crucial banal things. And if we can scale them up, make them more accurate, more reliable. Yeah, it really is world changing.

• 37:04Alex Kantrowitz?

  Isn't it kind of crazy that like the picture of AI again is this just like, you know, I guess maybe it's because ChatGPT was the thing that started the hype cycle. But the picture, popular picture of AI is like this masterful, again, like godlike technology that, you know, can do all these things and be this friend for you, like the character AI type startups. And people talk about AI girlfriends, but then the value is really being realized in like the back office. I mean, this is pretty crazy sort of divergence there. Never, I don't think I've ever seen a technology with that type of divergence.

• 37:38Speaker 01

  Yeah, I mean, I think like the internet is a good, or like computing in general, like these general platforms for supporting new types of products and tools. Yeah, sometimes they have biases in certain ways, but it's all about diffusion, like diffusing into an economy, diffusing into our daily lives. And it takes time for that to happen. And we should remember that we're like 18 months in to that journey. And so it's still, it's really so early. But yeah, I think the internet has had huge impact, both on the commercial side, on the enterprise side, as well as with us as consumers and people. And AI will be the same.

• 38:22Speaker 01

  There will be products that are pure play AI products, targeting consumers that bring tons of joy and value to consumers. And then there will be platforms like Cohere that enable huge value within the enterprise world.

• 38:38Alex Kantrowitz?

  Yeah, and again, like talking a little bit about how impactful this is in enterprise. I think this is from Reuters. Accenture's generative AI business, which helps companies automate operations to save costs and boost productivity, recorded about a 50% jump in new bookings quarter over quarter. This has outpaced growth in Accenture's other core businesses as a go-to consultant and outsourcing service provider for companies migrating their operations to the cloud. Analysts expect slow demand for such service as enterprise spending plateaus. So basically, this is like finding ways to automate is like giving life to the consulting industry. What do you think about that?

• 39:14Speaker 01

  I think Accenture is a really good partner and there's just so much work to be done implementing this technology that that makes perfect sense. Like, there's a huge technological shift happening and the technology has unlocked a whole new set of applications. And so now we need to go out and do the work to realize it.

• 39:36Alex Kantrowitz?

  Yeah. And what type of partnerships are you having with Accenture? Is it like going into companies and again, automating back office? Or like, what is what's going on there?

• 39:44Speaker 01

  Yeah, so they're our solution integrator partner. And so, yeah, it's about taking on projects inside of enterprises to help them accomplish something like maybe it's implementing for their finance team. There's some function that they're stuck on and it takes a huge amount of their time, but it's totally non-strategic. They shouldn't be spending time on it. And so can we automate that or a big part of it using these models? It's about these strategic projects to try and unblock and automate parts of usually back office functions.

• 40:18Alex Kantrowitz?

  It's amazing how like, I wrote about this a little bit in my book, but we're like living in the knowledge economy. And even still, like we've gone from industrial economy, which is like literally like pulling levers and pushing buttons to make stuff to knowledge economy, which is all about knowledge. But even in the knowledge economy, so much of our time is like legitimately on like straight up, you know, repetitive kind of tasks that we wish we could automate to make room for us to do more knowledge stuff.

• 40:44Speaker 01

  I hope that that goes away to a large extent, but I don't think it will. Like, I think there will always be on the margin, these sorts of not good uses of our time that we spend time on. And we'll continue to push that margin back and back and back and try to automate as much of that as we can. But it's a, it's a huge, huge project. What we're focused on is kind of building from the foundation. Start by automating the biggest of those, the ones that you're wasting the most time on. And then gradually get into more niche, targeted, specific automations or applications.

• 41:19Alex Kantrowitz?

  Is anybody using your technology to replace full-time employees?

• 41:25Speaker 01

  I am not aware of that. I, I don't think I, I have any example of that happening. It's very assistive, actually. So it's, it's less about replacement. It's more about augmentation. Like, at the moment, what everyone's building are tools to augment their workforce, to make them more productive. I can't think of a single example of displacing people.

• 41:48Alex Kantrowitz?

  Okay, I know we're, we're running out of time. One more thing I want to ask you about is sort of like the role of cloud providers versus like the role of like people buying direct? And like how this is helping or what type of pressure this is putting on cloud? This is, again, we talked about this recently. So Anthropic, they just broke down, CNBC just broke down Anthropic's revenue. And third-party APIs like Amazon, and I think Microsoft Azure, if they're available there, let's say Amazon, 60 to 75% of their revenue. So how important are these cloud providers like Amazon, like Azure, in driving this forward?

• 42:24Speaker 01

  The cloud providers are great partners to Co-Year. That's where the majority of compute workloads are happening, but not all of the workloads. So Co-Year has had a long time focus on on-prem as well. Because for a lot of regulated industries like, like finance and healthcare, a lot of that data doesn't actually go on the cloud.

• 42:45Speaker 01

  But certainly for many industries that are cloud-first, that's the place that their AI workloads are going to happen. And so I think it makes sense for revenue to be coming from those sources. But for Co-Year, we support both. And so it's perhaps a little bit more balanced.

• 43:02Alex Kantrowitz?

  And so, so your technology is basically going to work Your company will basically work to integrate your technology into existing systems, or you have your own software?

• 43:11Speaker 01

  So we build our own models from scratch. And we build a platform that lets people plug in their data sources, the tools that their employees use into the models via a system called RAG, Retrieval Augmented Generation. And that's something that we're specialized in. The guy who created RAG when he was at Meta is Patrick Lewis, and he leads our RAG efforts. But it's basically the dominant architecture or system that enterprises are looking for right now. They want to customize these models with their proprietary data, and the best way to do that is with RAG. So that's something that we provide out of the box in like a super simple plug and play way.

• 43:52Alex Kantrowitz?

  Let's end with this. Can you give us your prediction for what the AI field looks like in the next two years and five years?

• 43:59Speaker 01

  Yeah, in the next two years, I think we're going to start to see really compelling assistants. It won't just be little convenience functions or small features. It'll look a lot like a partner that you do work with, someone that you interact with every single day and you view as a collaborator. Over the next five years, I think it's not a major shift, but it's an increasing incompetency. The scope of those assistants will expand. They'll be trusted with doing much more, and they'll be integrated into many more systems, so they'll be dramatically more capable. So I view it as like a continuous change over time towards much more compelling independent

• 44:43Speaker 01

  agents that we can collaborate with.

• 44:46Alex Kantrowitz?

  Well, Aidan, thank you so much for coming on. Great to see you, and thank you so much for sharing everything about the industry in general and where companies are finding the ROI. I do think that this idea that, listen, like, it may be quote unquote boring, but hey, if it's saving billions of dollars, then don't tell me that that's a boring application of technology. That's kind of my main takeaway today, and I think it's pretty fascinating stuff that you're working on. Yeah, thanks for having me on. It was great seeing you, Alex. You too. All right, everybody. Thanks so much for listening. We'll be back on Friday breaking down the news and we'll see you next time on Big Technology Podcast.

• 45:21Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.

• 45:50Speaker 03

  In this life, you can't win.

• 45:52Speaker 04

  When the world gets you down.

• 45:53Speaker 03

  No!

• 45:54Speaker 04

  Escape to Pluto TV.

• 45:56Speaker 03

  This is objectively fantastic.

• 45:58Speaker 04

  Relax into the movies and shows you already know and love, all for free.

• 46:01Speaker 03

  Shwing!

• 46:02Speaker 04

  Even our news is uplifting.

• 46:03Alex Kantrowitz?

  You stay classy, San Diego.

• 46:05Speaker 04

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 46:11Speaker 03

  We're not worthy!

• 46:13Speaker 04

  The best things on TV really are free.

• 46:15Speaker 03

  You just have to know where to look.

• 46:17Speaker 04

  Escape to Pluto TV. Stream now, pay never.