First impressions of GPT-4o

Daniel & Chris share their first impressions of OpenAI’s newest LLM: GPT-4o and Daniel tries to bring the model into the conversation with humorously mixed results. Together, they explore the implications of Omni’s new feature set - the speed, the voice interface, and the new multimodal capabilities. Sponsors: Ladder Life Insurance https://ladderlife.com/changelog – 100% digital — no doctors, no needles, no paperwork. Don’t put it off until the very last minute to get term coverage life insurance through Ladder. Find out if you’re instantly approved. They’re rated A and A plus. Life insurance costs more as you age, now’s the time to cross it off your list. Neo4j https://graphstuff.fm/episodes/2023-finale-llms-and-knowledge-graphs-throughout-the-year?&utm_campaign=UCGenAI&utm_content=AMS-SrDev-ToFuDev-UCGenAI-Audio-None-GenAI1-GenAI-NonABM&utm_medium=Audio&utm_source=PracticalAI&utm_justglobal= – Is your code getting dragged down by JOINs and long query times? The problem might be your database…Try simplifying the complex with graphs. Stop asking relational databases to do more than they were made for. Graphs work well for use cases with lots of data connections like supply chain, fraud detection, real-time analytics, and genAI. With Neo4j, you can code in your favorite programming language and against any driver. Plus, it’s easy to integrate into your tech stack. Featuring: Daniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonShow Notes: Hello GPT-4o https://openai.com/index/hello-gpt-4oAI Engineer World’s Fair https://www.ai.engineer/worldsfairAIQCON - the AI Quality Conference https://www.aiqualityconference.comBrave New Words: How AI Will Revolutionize Education (and Why That’s a Good Thing) https://www.amazon.com/Brave-New-Words-Revolutionize-Education/dp/0593656954Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:06Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related tech is changing the world, this is the show for you. Thank you to our partners at fly.io, the home of changelog.com. Fly transforms containers into micro VMs that run on their hardware in 30-plus regions on six continents, so you can launch your app near your users. Learn more at fly.io.

• 0:42Daniel Whitenack?

  Hello and welcome to another fully connected episode of the Practical AI Podcast. In these fully connected episodes, we keep you connected with everything that's happening in the AI world and help you find some resources to level up your machine learning game. My name is Daniel Whitenack. I'm founder and CEO at Prediction Guard, where we're safeguarding private AI models. And I'm joined, as always, by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? I'm doing good today, Daniel. How's it going with you? It's all good. Yeah, I got the chance last week to visit Boston and see a bunch of

• 1:25Daniel Whitenack?

  cool stuff, tour a few labs around MIT, which was a lot of fun. I toured a couple labs where they're making, or they're using AI to make proteins, like drug candidate proteins. Very cool. So the idea is, one of the companies literally named AI Proteins, hopefully we can have them on the show sometime. I requested their CEO while I was there, he was giving us the tour. But yeah, the idea being that you can use various AI-driven methodologies to explore the space of proteins for drug candidates, and they're kind of binding to certain stuff. I'm not a biologist, or...

• 2:08Daniel Whitenack?

  Anything like that, but then they take those and then synthesize them in the lab and test them and eventually hope to get them into drug, sort of drug candidates and through FDA testing and all of that stuff. So it's pretty cool.

• 2:23Chris Benson?

  I would love to have them on the show. The, I've loosely followed that field over the last couple of years, largely because someone that I used to work for is a chemistry PhD from Harvard, but had, is very familiar with biotech. So he's kind of kept me up to date on some of that. It sounds fascinating. I know that drug discovery is really all about AI these days. I think that's where all the action's happening in that field.

• 2:50Daniel Whitenack?

  Yeah, and it's pretty amazing, at least from what I've heard from a couple of those companies, just the, hopefully the speed, like the orders of magnitude faster that they'll be able to explore the solution space, I guess. So testing, you know, thousands and thousands of drug candidates very quickly, rather than maybe, you know, a postdoc or a PhD testing only a handful over the course of many weeks or even years, they're able to do things much faster, which is really interesting. And of course, they're exploring that really useful application of that technology. But I guess this is one of the reasons that

• 3:34Daniel Whitenack?

  some people might have sort of ethical concerns with some of this stuff, because it's kind of like, you can apply the technology in a really positive way and explore drug candidates. I'm sure you could also think about things that would be harmful to humans and even think about like biological weapons and that sort of thing and explore that solution space in the same way. And those sorts of things don't need FDA approval. So yeah, I imagine that there's people smarter than me that have thought more deeply about those concerns. And I know it was mentioned, I think, in our last round of interviews about Mozilla's report on AI this last year. We had an episode on that. But yeah, I was thinking about that

• 4:19Daniel Whitenack?

  while I was there. It sort of cuts both ways, I guess.

• 4:22Chris Benson?

  It does. It's just since you mentioned that, I know in kind of the defense and intelligence world with AI capabilities being the great equalizer, the idea of malignant forces in the world deciding to focus on such things, which incidentally is very illegal under international law.

• 4:40Daniel Whitenack?

  Right.

• 4:41Chris Benson?

  But certain places in the world don't care so much about that. And so we'll have to see. It's, I've had a lot of conversations about the very good and the very bad about AI with folks lately and what an uncharted world we're moving into at this point.

• 4:57Daniel Whitenack?

  Yeah, yeah. Well, I'm very happy. At least the, the people that I've run across are quite ethical and moving towards things that will hopefully, hopefully benefit us all. But speaking of benefit to many people, there was something rolled out this week that definitely caused a bit of a stir and also instantly appeared on a bunch of people's phones and devices. And that was the next GPT, GPT 4.0, standing for Omni, if I got that right. GPT 4.0 Omni. I don't know the full background of that naming, if it's meant to evoke omniscience or.

• 5:42Chris Benson?

  I think the explanations I've seen have been about multimodality.

• 5:46Daniel Whitenack?

  Yes.

• 5:46Chris Benson?

  You know, just the fact that it can photographic, video, voice, everything. And yes, it was quite a release. You know, everybody's been talking about upcoming, the expected release in the summer potentially of GPT 5. You know, interestingly enough, this came out and was quite, even though it's still part of that four family, it's had quite an impact. I know that in the last week, I will say that it's been the most open app on my phone, pretty much around the clock.

• 6:15Daniel Whitenack?

  Yeah.

• 6:16Chris Benson?

  Starting to feel like a family member because it's involved in all of our family decisions, trying to, things like getting a leak in the drywall and trying to use it to do something as mundane as figure out the plumbing concerns. And it seems to be, you know, when my wife and I are talking about household things, we now have ChatGPT as a third party in all those conversations. It seems to have supplanted my daughter. I'm not sure she likes it very much.

• 6:42Daniel Whitenack?

  Yeah, well, since you have been getting hands on and using GPT-4.0 quite a bit, for you either in the announcement or in your own use of it, what are those things that stand out as the things that have changed from, let's say, GPT-4 to GPT-4.0?

• 7:02Chris Benson?

  Well, I don't have a list of things in front of me at this moment or anything, but things that I've certainly experienced is it is much faster than just GPT-4 had been. It's able to respond very quickly in any of the modalities that we're talking about.

• 7:16Daniel Whitenack?

  And when you say modalities, you're meaning sort of text, speech, image?

• 7:21Chris Benson?

  That's correct. It seems much faster. I haven't measured it across the board, but I think the thing that's been notable in my own workflow of it is that I'm not having to wait around and kind of figure it out. And, you know, before this week, I'd kind of say, okay, I'm going to get onto GPT-4 and ask it a question. I kind of stop everything what I'm doing and do that. And I think the difference is the thing that's really impacted me the most is maybe the subtleness of no longer waiting around, being able to do it just by speaking and being spoken to. And it's no longer a stop and do something kind of activity. It's now as I'm doing it, as we're in the middle of conversation, it just becomes part of the conversation.

• 8:05Chris Benson?

  I don't tell my wife, hold on one second, I'm going to check real quick on this question with GPT-4, and let's see what it says, and then we can, you know, take that into account as we talk. Now it's just like right there at the kitchen table, we just do it. So, third party in the conversation.

• 8:19Daniel Whitenack?

  Dueling GPTs.

• 8:20Chris Benson?

  That's right. That's right.

• 8:23Daniel Whitenack?

  Yeah, I think that some of the the main features, if if people haven't been following it quite as much in the news, which I'm sure a lot of, a lot of our listeners have been following it quite closely. One thing that they focused on was speed. So in particular with the voice response, when your voice is kind of recorded in.

• 8:47Daniel Whitenack?

  Then they're talking about responding in milliseconds rather than I think before it was a few seconds, something like that, which of course is much faster. I think in general, it's a fast model in my understanding in terms of response and streaming across the modalities. Also, it's in terms of access, both account wise and cost wise. Another drop in costs as far as the cost for the performance goes. So that's a trend that I think continues. And also, one of the things I was happy to see was most of the GPT models over time

• 9:28Daniel Whitenack?

  have penalized you basically in terms of token count for putting in languages other than English, because you would get higher token counts. And if you're charged by how many tokens you put in or generate out, and let's say you're putting in Korean or something like that, then it's actually more expensive to use the tool in those other languages. So I think they, at least in my understanding from what I've read, I don't know if that's fully equitable at this time, but there was an effort to kind of correct some of those issues as they came up.

• 10:05Chris Benson?

  Yep. Have you used the video features much? I've, I have a bit and it's very good compared to things that have come before, but sometimes it seems to get amazing context and occasionally it struggles a little bit. I think it depends on how much context it's able to get out of the imagery.

• 10:21Daniel Whitenack?

  Yeah, I've mostly used the sort of image related stuff versus video. So audio and text and image is kind of what I've done. But yeah, they show a good number of things in the demo videos related to video and also even kind of combining one version of this running with another version and having interactions between the two and interview prep with the tool and all sorts of cool stuff. So if people haven't seen it, I definitely recommend that people go and check out the demos to kind of get a sense of the performance. But yeah, it's overall quite impressive.

• 11:05Chris Benson?

  The subtlety of being able to do these things with that reduced time and across modalities, you know, while it might not be whatever giant jump that upcoming GPT-5 would be, the fact that it's changing our behaviors and the way that we're using it in this last week, and enabling things that just weren't practical before, I think that really makes a difference going forward. To the point where I work with, here in the Atlanta area, I work with some of the local universities and their various computer science colleges and schools and such. And I was at one on Friday for a kind of a day-long strategic planning meeting on computer science and where they were going with it. And we were talking about this while we've been talking about AI's impact,

• 11:48Chris Benson?

  obviously, at any kind of computer science program. This may change not only what you can do, but education as well, in a, in a pretty fundamental way in terms of teaching and being able to do it in real time and stuff. And we had quite a rich conversation around, it lasted quite a while, around how we might be able to utilize these new capabilities in the classroom going forward, and also how it might change curriculum. So I think we're really starting to get to a point where I think a lot of new capabilities in education are, are right around the corner.

• 12:31Speaker 03

  If you're anything like me, you have a certain tendency to put things off until the very last minute. Seeing the dentist, going to the doctor, home improvements, that never-ending chore list of yours. And while most of the time it works out just fine, the one thing in life that you really cannot afford to wait on is setting up term coverage life insurance. You've probably seen life insurance commercials on TV and thought, "Yeah, I'll look into that later." No, later doesn't come. This really isn't something you can wait on. Choose life insurance through Ladder today. Here's what we love about Ladder and why we allow them as a sponsor. They are 100% digital. No doctors, no needles, no paperwork. When you apply for $3 million...

• 13:15Speaker 03

  And coverage or less. Just answer a few questions about your health in an application. Ladder's customers rate them 4.8 out of 5 stars on TrustPilot and they made Forbes Best Life Insurance 2021 list. You just need a few minutes and a phone or laptop to apply. Ladder's smart algorithm works in real time, so you'll find out if you're instantly approved. No hidden fees. You can cancel any time, get a full refund if you change your mind in the first 30 days. Ladder policies are issued by insurers with long, proven histories of paying claims. They're rated A and A+ by A.M. Best. Finally, since life insurance costs more as you age, now, yeah, right now, now's the time to cross it off your list.

• 14:00Speaker 03

  So go to ladderlife.com slash practical AI today to see if you're instantly approved. Again, that's ladder.com slash practical AI. L A D D E R life.com slash practical AI.

• 14:35Daniel Whitenack?

  I'm having a discussion with my podcast co-host Chris about the new GPT-4O model. What do you think we should talk about?

• 14:45Speaker 04

  That sounds like a great topic for your podcast. You could discuss how GPT-4.0 compares to previous versions, especially in terms of its capabilities and improvements. Delve into its applications across different industries like healthcare, creative arts, or customer service, and consider exploring the ethical implications, including privacy concerns and bias in AI. It might also be interesting to discuss real-world case studies or predictions about how AI will evolve in the future. What do you think your listeners would be most interested in hearing about?

• 15:17Daniel Whitenack?

  Well, a lot of that seems pretty interesting. You mentioned healthcare. What were you thinking in relation to healthcare?

• 15:25Speaker 04

  Sorry, I'm having issues right now. Our systems are experiencing heavy load. Please try again later.

• 15:31Chris Benson?

  Well, at least I'm not the only one having issues. At least I know that ChatGPT 4.0 has issues itself at times. There's something slightly satisfying about that, I must say.

• 15:42Daniel Whitenack?

  Yeah, well, it was doing pretty good there until it had volume issues or whatever they're experiencing and I got cut off.

• 15:50Chris Benson?

  I'm gonna call it a mental blank. You know, I have those from time to time. You know, I'm just gonna say.

• 15:56Daniel Whitenack?

  So what do you, Chris, our friend over in the ChatGPT world, ChatGPT 4.0, suggested some things about privacy concerns as related to AI. I probed a little bit healthcare-related things, but it wasn't able to give me an answer and got bogged down. But it also mentioned privacy concerns. Yeah, have you thought about that? As you've obviously been using the system, what changes now in terms of privacy now that we have 4.0 and not 4? How is it different, if at all?

• 16:33Chris Benson?

  I think it is. And this is a topic that has come up quite a bit this past week in various online forums. There was a particular LinkedIn post, I'll try to find it and include it in the show notes if I can, that brought it up. And with us now talking to it and receiving it back, how does that impact this? Is this recording? Is it not recording? How does this qualify under different state laws? When we were busy typing it in and getting our questions back, you know, while there were privacy concerns, it wasn't extending now to audio recording of voices, you know, which is covered under state laws of all states in the US at least, and I'm sure many countries out there. What do you think? I'm just curious. I know neither of us are attorneys,

• 17:16Chris Benson?

  but now that we're leaving our phones open to ChatGPT and capturing people, I'm sure, and I've done it in public places a bunch this week. How do you think that impacts? Do we need to tell everyone we're doing it? Okay, everyone, quiet. Everyone, okay, I'm starting ChatGPT 4.

• 17:31Daniel Whitenack?

  It's weird because it's some of the same feelings I think people had originally when they started bringing Alexas or Google Homes into their home and it was sort of always supposedly not listening, but it had to be listening at least to get the wake word, right? Yep. So there was this awkwardness there in terms of what's actually being recorded and and that sort of thing. I think the difference here, you kind of almost got there when you were talking about how you were using it in your everyday life. I think people can see that this technology, because there's a quick response. So there's, as I was playing that, like you could tell the first response that I got from ChatGPT was pretty quick. I would say it's still not quite like you and me talking. It's not natural, right? But it's,

• 18:16Daniel Whitenack?

  it's pretty quick. And so there's this tendency then to think, oh, well, I can leave this on at certain times, or like you say, have it as part of the dinner table conversation. You kind of then bring in these devices like the Meta AI glasses and like, maybe I just have ChatGPT watching what I'm watching through my Meta AI glasses and telling me about this or that. And so you've got all of these modalities coming together. It's recording in your kind of physical space, not only your voice, but potentially images and videos from your physical space. And all of that data is going over an API to OpenAI or Microsoft or however

• 18:59Daniel Whitenack?

  the Microsoft OpenAI conglomeration, I don't, that's not a word, works these days. But yeah, it's that embedding, I think, of the technology in the physical world, or the clear application of that within our sort of physical world. And like you say, not pausing to go and pull up a tab and talk to ChatGPT. It's, it could be ubiquitous and embedded in our physical world, I guess would be a good way to summarize it.

• 19:26Chris Benson?

  To extend that a little bit, Sam Altman, the OpenAI CEO, one of the comments he had made this week in an interview was, somebody was saying, when should you use it, I believe? And he said, oh, you should just have it on all the time, just listen.

• 19:39Daniel Whitenack?

  Yeah.

• 19:40Chris Benson?

  And I'm paraphrasing him, I'm not quoting him, but the gist was never have it off. I know that was one of those moments, that the privacy notion, at least right now, I'm operating under the assumption that it's coming into play when I and the people around me are familiar with it, and we've kind of made that choice to do that. But I certainly, you know, going back to the Alexa notion and stuff, I think this is going to continue to be an issue here. The Alexa stuff, we have those as well. Oddly enough, I don't find myself paying much attention to them anymore. I guess I've just gotten so used to them being part of the environment and stuff, but we'll see.

• 20:17Daniel Whitenack?

  Yeah, well, AI meeting the physical world is definitely, I think, going to become more and more a reality at the Boston Logan Airport. When I was flying out this last time, I saw they had, you know, normally they have little booths where there's a person that's like your helper at the airport, like if you have some random question about where the bathrooms are, am I at the right gate? Or how do I catch this bus? There's a helper. They just didn't have anyone there at the thing and then just relabeled it virtual assistant and just had a screen that you could push and talk to. And I know there's a good number of companies that are working on sort of interactive virtual

• 21:01Daniel Whitenack?

  agents for retail environments, that sort of thing. And then you have this crossover with the glasses and Rabbit R1 and Humane AI pen and Meta AI glasses and all this stuff. So are you becoming a cyborg, Chris? Or are you mostly just keeping it in your phone?

• 21:20Chris Benson?

  I think I've accepted the fact that it's inevitable to do that. I say that half tongue in cheek, half not. To that point, actually, it makes me think, you know, this is penetrating so far beyond people like us in this space. And I have a very good friend who is, who I don't think would identify as a technology person. And she brought up the fact that, and this isn't even specific to ChatGPT 4.0 or anything, but, but it is to your effect there. She brought up that they had pulled in, she and her daughter had pulled into a Chick-fil-A and they noticed a sign that said a robot crossing. And they didn't really know what that meant, but then they actually saw a robot delivering food.

• 22:02Chris Benson?

  And now that robot, I'm sure at this point, doesn't have very sophisticated AI capability for interactions. It's probably pretty basic. But in the conversation, I pointed out, it's inevitable. You have with so many, you know, as we pointed out a week or so ago, that we're over a million models already in Hugging Face. And with these kinds of profound releases each week, it's only a matter of a very short time before even the most mundane retail experience is going to have both robotics and AI in that. And so all of those things raise the privacy concerns that we were talking about before. And they also raise cultural and just, you know, folks getting

• 22:43Chris Benson?

  used to it, frankly. And of course, they've inevitably led to the concern over jobs and such as that, as is often coming up. But I think this is maybe the first year that it's moving so fast in terms of these capabilities that even I am trying to, I'm even struggling to, to take them in as they come out. How about yourself? Even though you're in that profession?

• 23:04Daniel Whitenack?

  Yeah, well, I have definitely, it's even, even out here in the prairie in Indiana.

• 23:10Chris Benson?

  The prairie.

• 23:11Daniel Whitenack?

  It's becoming the Silicon Prairie with Intel building their big factory in Ohio and new semiconductors coming to Lafayette, Indiana. So plug for the Silicon Prairie. Anyone out there can hashtag Silicon Prairie on Twitter and, and all the places. But even out here at, well, I'm close to a university, close to Purdue, but they've had food delivery robots here six to eight years now, where they've had food delivery robots around, around campus here, which is, is cool, but...

• 23:43Chris Benson?

  You just made me feel very parochial in that way.

• 23:47Daniel Whitenack?

  Yeah, but it's cool. I don't know. It'll be interesting to see also which brands kind of embrace this and how they embrace it and and put it into their retail environments and how that's associated with specific brands and how they create new experiences for customers around these things. It's quite interesting.

• 24:21Speaker 03

  What's up, friends? Do you remember when ChatGPT launched? I do. It felt like the LLM was this magical tool out of the box. However, the more you use it, the more you realize that's just not the case. The technology is brilliant, don't get me wrong, but it's prone to issues like hallucination on its own, but there's hope. There is still hope. Feed the LLM reliable current data, ground it in the right data and context, then and only then can it make the right connections and give the right answers. The team at Neo4j has been exploring how to get results by pairing LLMs with knowledge graphs and vector search. Check out their podcast episode about LLMs and knowledge graphs throughout 2023

• 25:03Speaker 03

  at graphstuff.fm. They share tips on retrieval methods, prompt engineering, and so much more. Don't miss it. Find a link in our show notes. Yes, check it out. graphstuff.fm, episode 23.

• 25:38Daniel Whitenack?

  Hey, so this week I'm going to have to migrate some services out of AWS ECS into a Kubernetes cluster that's hybrid between my cloud environment and a data center. What should I keep in mind?

• 25:54Speaker 04

  That sounds like a big project. Here are a few things to keep in mind when migrating services from A to S ECS to a hybrid Kubernetes cluster. One, assessment and planning. Start by evaluating your current ECS configurations and services. This will help you understand the dependencies, data flows, and necessary resources. Mapping out how these will translate into Kubernetes is crucial. Two, Kubernetes environment setup. Setup.

• 26:17Daniel Whitenack?

  Yeah, that was getting kind of boring. So maybe just tell me what cool song I should listen to while I'm doing the migration.

• 26:25Chris Benson?

  Uh oh, oh, boy.

• 26:27Daniel Whitenack?

  Are you still there? No dice. All right, well, huh, man.

• 26:32Chris Benson?

  A simple entertainment question.

• 26:35Daniel Whitenack?

  The OpenAI, you had your chance live on the Practical AI Podcast and I got skunked both times.

• 26:42Chris Benson?

  So. Yeah, well, we didn't rehearse enough yet. They did have some pretty cool videos on the ChatGPT 4.0 release. You know, they had several different ones, but one of them, they had two phones taught, you know, with ChatGPT talking to each other and they introduced them and they gave them little monikers to differentiate between them. But I will admit, I tried that at home right after it got released. I saw that video, I was like, I want to try that. And it did not, I will confess, it did not work well for my end either unrehearsed. So I guess those ChatGPT folks at OpenAI have the inside track on smooth conversations.

• 27:22Daniel Whitenack?

  I'm sure it worked at one point as most demos do, but yeah, still, still impressive. Nonetheless, I have to say, I gave it a pretty complicated question there, maybe one that I could definitely use some help with. So, yeah, I think it did pretty good at answering, of course, and was responsive. I'm wondering, Chris, what you think about now that we have GPT 4.0? What is the future of all of these different physical AI device gadgets that have come out in recent times? So there's been the Rabbit R1, there's been the Humane AI pen, there's been the Meta AI glasses and probably others that I'm not even aware of.

• 28:04Daniel Whitenack?

  What's your thought on how this influences these sort of AI gadgets?

• 28:08Chris Benson?

  While this is also a golden age of AI startups, it's also the bar keeps getting raised very rapidly and unexpectedly. So you can go from super cool to obsolete overnight. You can be one announcement away from a tough moment there for your product and our service. So, you know, for instance, now that the world has had a little time to try out the 4.0 version and it's changed the way we do it a little bit, that's set a new bar, it's set a new expectation on how you're going to interact with AI. And I will confess that this week, whereas both you and I are always big fans and supporters and advocates of

• 28:48Chris Benson?

  open models and being able to do that instead of just having a service provider, I have to confess that when I was using open source models this week, with as much as I was also using the 4.0 model, it was frustrating because my own expectation had arisen. So if I was using one of these products and the world just changed in terms of kind of standard expectation on these model capabilities, it wouldn't take much to not be able to survive that if you can't react to it quickly enough. So it's interesting times that we live in.

• 29:24Daniel Whitenack?

  So where do you think, if anywhere, those out there building AI products or maybe products that are driven by AI features, where can they capture value? Because it's certainly from my perspective, it's, you know, even with this release in GPT-4.0, unless you're already a certain ways there, it's probably not just having an LLM API, because that is essentially just a commodity now.

• 29:53Chris Benson?

  That is.

• 29:53Daniel Whitenack?

  Now, there's, you know, some are more expensive than others, but essentially the price is kind of dropping to almost zero, unless you're at a very high usage rate, which certainly some companies are, and that becomes an issue for them. But yeah, where do you think the value is to be had?

• 30:13Chris Benson?

  I still think it comes from kind of a classic Steve Jobs throwback comment is it's not just about the AI, it's not just about the LLM, it's about you're producing something of value that's trying to solve a problem and you're combining all these things together to create the right capability or experience for your customer. And I still think that's where it's at. Maybe if I give a devil's advocate to my own comments a moment ago, if you're going to have a product that has AI integrated into it, make sure the AI is really serving the capability of that product as opposed to being about the AI itself because then you can be undone by the next announcement. So I really think it's utility for the thing

• 30:58Chris Benson?

  that you're buying the device for is we're buying more and more AI-enabled devices going forward. And most of them will not have the leading edge capability via API in it.

• 31:10Daniel Whitenack?

  Yeah, I think that the space of those that are working on general purpose, serve everyone type of AI products, which definitely fits into these kind of assistant places, it's a hard road because like you say, if something could knock you off that pedestal quite easily, it's hard to compete in terms of price and the commoditization of these things. But in the enterprise, it's still very hard to utilize these tools. That report that I've referred to a number of times from Andreessen recently, they're saying there's these huge budgets in AI across enterprise companies and 75% of it has nothing to do

• 31:53Daniel Whitenack?

  with the usage of the model at all, or the hosting of any models or anything like that. It all has to do with engineering integrations around workarounds and malfunctions and making sure it's reliable and dealing with all the issues. So there's still a lot of space, I think, even if you're not vertically focused, but certainly there's also people that are vertically focused that I think will come out really well. One of the companies that I was able to interact With a little bit last week, they're doing financial workflows in the financial services sector called Far Side AI, automating things that used to take days with market

• 32:35Daniel Whitenack?

  research and creating slide decks and all of this stuff is pretty cool things, but they they're bringing their domain expertise into that field and they're applying it. And that's what really creates the value. That's why someone would would pay for that. Where whereas there's not really going to be that many people that say, no, I would rather build that from a raw L M API. And, you know, just not very many people are going to do that because it's much harder than you might expect. So yeah, I think that that in certain verticals, applying domain knowledge, creating these agents, these automations, that's a really interesting space moving forward as well.

• 33:15Chris Benson?

  One of the things that you taught us a while back was kind of that the relatively speaking smaller models and that kind of what seven, eight billion range, you know, you know, where you're able to do it on just one piece of hardware and stuff. And I think that was fantastic guidance that you gave us. This was on a previous episode, we'd look it up and we can connect back to it. But I think that that's where all the action is. I mean, I think that's whereas the the press goes to these huge model releases, the real action in creating value in a product is going to still be the smaller models that are fine tuned very well to the problem that they're solving. And I think those will continue

• 33:54Chris Benson?

  to be wow, because whereas chat GPT four O is wonderful in terms of these conversations, usually wonderful on these conversations on our iPhones. An iPhone is only one of many things I pick up in a given day. And frankly, as we go forward, I would expect all the other things I pick up are probably going to have some models associated with it just to do what that does very well.

• 34:17Daniel Whitenack?

  Maybe there's the reason why my GPT-4O isn't performing well because I'm using it on Android. Anyway, one of the other things I wanted to mention, Chris, and this is kind of tied into some of this as well, where, you know, there continues to be an advance of these closed source models. I think if you look, there's a chart that Hugging Face maintains about the sort of convergence of open models and the closed models, and the closed models are still ahead. And now, of course, GPT-4O is up there at the peak of it. But those lines are converging, so they're not just running parallel. And closed models are all the way kind of ahead to infinity. But there's a sort of crossover point, which we'll see if that actually happens.

• 35:00Daniel Whitenack?

  But that's kind of, at least as far as those graphs, it looks to be what's happening, which is interesting. There was some news out of Hugging Face this week, though, that is good news for those that aren't big foundation model builders and have big clusters of GPUs. So Hugging Face announced that they're going to be sharing $10 million worth of GPU compute. And the article that I read said to, quote, "help beat the big AI companies." So this is quite relevant to the discussion that we're having now. In my understanding, they're, they're making this compute, these GPUs, in a project called Zero GPU.

• 35:42Daniel Whitenack?

  They're making this compute available within the Hugging Face Spaces compute and application environment. And so, yeah, for those of you out there, you might be sitting around and still wanting to innovate with, with open models or try your own things and feel maybe not adequately resourced in terms of compute and particularly GPUs. So really cool to see Hugging Face take this step and provide some of that GPU resources to the, the community that's operating on Hugging Face. So, yeah, check it out. If you just search for Zero GPU, you can probably find out a little bit about that effort from Hugging Face.

• 36:25Chris Benson?

  And I love seeing that from them. You know, we've, we've long talked about That if you probably looking a little ways down the road, AI ever integrating more and more with the software around it, to the point where it'll be kind of ludicrous to have software that doesn't have some sort of AI capability in it in the future. It's feeling more and more like software in that way, that when we hit the million, the million open models on Hugging Face and then just seeing these capabilities coming up. You know, when you said that, that reminds me of like, you know, all the major cloud providers will kind of offer a limited free tier, you know, so that you can go do some stuff with it. And that's kind of how Hugging Face is offering that with open models feels to me in terms

• 37:10Chris Benson?

  of being able to go use something when you might not have the resource otherwise. So, yeah, it's good stuff, but boy, gosh, the world is changing fast here, isn't it?

• 37:19Daniel Whitenack?

  Yeah, Clem from Hugging Face, he made a quote in the Verge article that I was reading. "It's very difficult to get enough GPUs from the main cloud providers. And the way to get them, which is creating a high barrier to entry, is to commit on very big numbers for long periods of time." And of course, that's something that smaller companies or even individuals don't have the resources to do. So it's cool to see. Well, there's the zero GPU thing. So if you're out there, if you're wanting to learn, if you're wanting to run some of these models yourself, that in itself is a great learning resource and an option for you to do. But there's a couple of really cool things event-wise coming up soon.

• 38:01Daniel Whitenack?

  And actually, events where either Chris and/or I will be present physically. So wanted to mention those to everyone because there's some good things that will be streamed in terms of content and learning resources like workshops from people all across industry. So the first of these is with our good friends over at the ML Ops community. They're putting on this AI Quality Conference. It's aiqualityconference.com. And that's going to be June 25 in San Francisco. And as you all have seen, when Demetrius has been on- On the podcast, that guarantees to have some, some really great content there.

• 38:46Daniel Whitenack?

  They've got really great speakers, including people that have been on this podcast before, like Jerry Liu from Lama Index and, and others. So would definitely recommend going there and learning from people at the AI Quality Conference. Then also that same week in San Francisco. So if you wanted to, you could time this quite nicely, which is what I'm going to do. There is the AI Engineer World's Fair. So you can go to ai-engineer/WorldsFair and find out more about that. That's also going to be in San Francisco. It's going to be June 25th through the 27th. And of course, this is at least partially arranged and sponsored by some, also some good friends

• 39:31Daniel Whitenack?

  involved with the Latent Space Podcast and Discord community, which we've done collaborative episodes with and sponsored also by Microsoft. There's a day of workshops, there's speakers, and lots of really great content there. So two wonderful learning opportunities. I'm not sure if both, but I'm pretty sure some of that content will be streamed also live online. So if you aren't able to make it out to the Bay Area, you could still see some of that content and participate. Would encourage everyone to get out at least online and, and see what's going on in the, in the community and check out these great events from some of our friends across the community.

• 40:13Chris Benson?

  That sounds fantastic. I hope people join us there. And there's one other learning source that I thought I'd mention. It's one that I have not, it's a book. I haven't read it yet. I'm about to read it. And I thought I'd go ahead and share that with my fascination about AI and the way it's changing education. We talked a little bit on this show and, and some other recent shows. It's by Salman Khan. Khan is the CEO and founder of Khan Academy, which has helped millions and millions of people learn skills. And fantastic, I use it regularly. The book is Brave New Words: How AI Will Revolutionize Education and Why That's a Good Thing. It was just recommended by Bill

• 40:53Chris Benson?

  Gates of Microsoft and the Gates Foundation fame. So that's probably one of the next books I'm about to pick up. And I thought I'd mention it and maybe we'll bring it up again on a future show. So recommending that one.

• 41:06Daniel Whitenack?

  Sounds good. Well, let's have GPT-4.0 take us out. Hey, I'm about to go to sleep after this because I'm recording this podcast late with my friend Chris. I'd like to dream about an amazing future with AI. Could you give me some suggestions about maybe some thoughts I could have as I go to sleep, which might prompt some really amazing dreams about AI in the future?

• 41:32Speaker 04

  That sounds like a great way to wind down. Here are a few ideas to let your mind wander into the possibilities of AI in the future. One, imagine an AI that can predict personal health events and seamlessly coordinates with medical professionals to provide proactive care, ensuring optimal health with minimal effort from individuals. Two, think about an AI-driven global environmental system that manages resources and energy usage across the world, balancing human needs with planetary health and restoring ecological balance. Three, envision a world where AI personal assistants evolve to become lifelong learning partners, adapting to your interests and educational needs over time, making learning continuous, engaging,

• 42:17Speaker 04

  and perfectly tailored to your personal and professional growth. As you drift off, let your mind explore how these advancements could change everyday life, create new opportunities, and solve the big challenges facing humanity. Sweet dreams about the future of AI.

• 42:34Daniel Whitenack?

  You can't beat that.

• 42:42Speaker 00

  All right, that is Practical AI for this week. Subscribe now. If you haven't already, head to practicalai.fm for all the ways. And join our free Slack team, where you can hang out with Daniel, Chris, and the entire Change Log community. Sign up today at practicalai.fm/community. Thanks again to our partners at fly.io, to our beat freaking residents, Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us. That's all for now. We'll talk to you again next time.