Gemini vs OpenAI

Google has been releasing a ton of new GenAI functionality under the name “Gemini”, and they’ve officially rebranded Bard as Gemini. We take some time to talk through Gemini compared with offerings from OpenAI, Anthropic, Cohere, etc. We also discuss the recent FCC decision to ban the use of AI voices in robocalls and what the decision might mean for government involvement in AI in 2024. Sponsors: Neo4j https://www.neo4j.com/developer?utm_source=changelog&utm_medium=podcast&utm_campaign=practicalai – Is your code getting dragged down by JOINs and long query times? The problem might be your database…Try simplifying the complex with graphs. Stop asking relational databases to do more than they were made for. Graphs work well for use cases with lots of data connections like supply chain, fraud detection, real-time analytics, and genAI. With Neo4j, you can code in your favorite programming language and against any driver. Plus, it’s easy to integrate into your tech stack. Visit Neo4j.com/developer https://www.neo4j.com/developer?utm_source=changelog&utm_medium=podcast&utm_campaign=practicalai to get started. Fly.io https://fly.io/changelog – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog https://fly.io/changelog and check out the speedrun in their docs https://fly.io/docs/speedrun/. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Gemini https://gemini.google.com/appFCC decision on AI voices https://www.fcc.gov/document/fcc-makes-ai-generated-voices-robocalls-illegalFCC Bans AI Voices in Unsolicited Robocalls https://www.wsj.com/tech/ai/fcc-bans-ai-artificial-intelligence-voices-in-robocalls-texts-3ea20d9fPrompt Engineering Guide https://www.promptingguide.ai/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:06Speaker 00

  Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related tech is changing the world, this is the show for you. We just dropped Dance Party, our third full-length album on Change Log Beats. Buy it on Bandcamp and iTunes, or stream it on Spotify, Apple Music, and the rest. Link in the show notes. Thank you to our partners at fly.io. Launch your app close to your users. Find out how at fly.io.

• 0:42Daniel Whitenack?

  Welcome to another episode of Practical AI. This episode is a fully connected episode where Chris and I keep you fully connected with everything that's happening in the AI world, all the recent updates, and also share some learning resources to help you level up your AI and machine learning game. I'm Daniel Whitenack. I'm founder and CEO at Prediction Guard. And I'm joined, as always, by my co-host, Chris Benson, who's a tech strategist at Lockheed Martin. How are you doing, Chris? Doing pretty good, Daniel. Lots happened this past week. A lot has happened. It seems like, I don't know if it felt like this to you, but there's sort of a little bit of a lull around the holidays, maybe.

• 1:27Daniel Whitenack?

  Too much eggnog. Yeah, too much eggnog. But we're fully back into the AI news and interesting things happening. One of the ones that I had seen this week, Chris, was a decision. Well, I don't know how all the government stuff works, but the FCC, which regulates communication and other things in the United States, had a ruling about AI voices in robocalls. So if people don't know, robocalls are automated phone calls. Typically, when I worked back in the telecom industry, we'd call it sort of dialer traffic, right?

• 2:10Daniel Whitenack?

  You spin up a bunch of phone numbers. You can call a bunch of people. This is how you get phone calls from numbers that seem maybe local to where you're at, but they're really just automated calls. And then, you know, you pick up and realize it's spam or someone trying to sell you something or something happening. Anyway, there was an interesting one where there was an AI voice clone of President Biden, and I think they were robo-calling a bunch of people and trying to sort of change views about President Biden via this recording. Well, it wasn't a recording, it was a voice

• 2:51Daniel Whitenack?

  clone of him saying certain things which hopefully would sway people's political affiliations or sentiments leading into election season. Anyway, this was one of the things that was in the news and maybe prompted some of these decisions, or at least highlighted some of these decisions by the FCC to ban or fine people that were using AI voices in these robo-calls. So yeah, what do you think, Chris?

• 3:23Chris Benson?

  First of all, I think whoever was doing that has a serious ethical issues to contend with.

• 3:27Daniel Whitenack?

  Yeah. Well, I'm not sure that a lot of dialers are primarily motivated by their ethical concerns.

• 3:35Chris Benson?

  Yeah. I mean, I think that we've been seeing this coming for such a long time, and we've talked about it on the show, you know, with all the generative capability and the ability to commit fraud and the ability to misrepresent yourself in ways like this. So I'm glad the FCC got on top of it after something like that happened. And I think, unfortunately, I suspect we'll see quite a bit more of such things. As you pointed out, not everybody follows the law as well as maybe they should. I keep waiting for them just to ban robo-calls altogether, and it would just take the whole issue away from us. You know, we'd have AI-generated voices in other contexts, of course.

• 4:12Daniel Whitenack?

  Yeah. One interesting thing, I actually forget if this was a conversation we had on this podcast or elsewhere. Maybe someone can remember. I don't always remember all the things we've talked about on this podcast, but I saw it either in a news article or we were discussing someone on the other end of the spectrum who was using cloned voices or synthesized voices to actually spam-bait the spammers, right? So they would, they had like a script set up. Where they would get a robo call, right? Or a spam call. And actually, they had have this conversational AI that would try to keep the spammer on the line.

• 4:55Chris Benson?

  Oh, I have seen.

• 4:57Daniel Whitenack?

  As long as possible.

• 4:58Chris Benson?

  I think we did talk about that. I remember that. I remember that, yes.

• 5:01Daniel Whitenack?

  So I don't know if that's illegal. I found that one also kind of fun because that you see these people on YouTube that sort of spam bait the spammers, right? And try to keep them on the line. Because if they're talking to an AI voice, right? Then they're not scamming my grandma or something like that, right?

• 5:22Chris Benson?

  That's true.

• 5:22Daniel Whitenack?

  So, yeah, that's, that was, I think, the goal in that. But I don't know, maybe, maybe all of this is, gets in a little bit of a murky zone.

• 5:32Chris Benson?

  It does, but I would say the FCC, the Federal Communications Commission, got it right on this one. Score one for the government.

• 5:39Daniel Whitenack?

  Yeah. What I don't know, I think this would still allow, because obviously, when you call in to change your hotel reservation, or you call your airline or something, there's synthesized voices. And there have been for many, many years. Not necessarily synthesized out of a neural network, but synthesized voices. So I'm assuming that that, I haven't read the ruling in detail. I think the main thing that they're targeting is these robo calls. And so I don't think that covers these assistants. But I don't know. That's a good question.

• 6:13Chris Benson?

  I would assume it goes to intent, you know, and the representation of the voice. And if it is clearly, as in the case of the FCC ruling, is mimicking a person for the purpose of misrepresenting, you know, how they're seen or whatever, or how, you know, what their positions are and such, then I think that anything, I think that's a very reasonable thing. I think all of the types of. Circumstances we find ourselves in where people are trying to commit fraud or misrepresenting themselves in some way probably need to be addressed in this way. And but there are obviously for every one of those, there's probably a thousand legitimate use cases as well. So I agree.

• 6:51Daniel Whitenack?

  Yeah, there is probably a weird middle zone because even if you remember when I think it was originally Google did their demos at one of their Google I/O conferences. One of the things that was shown on stage is clicking and calling like your pizza place and ordering a pizza with an AI voice, right? Like, or make me a reservation at 5:00 PM at this restaurant, but you can't, there's no form on the website, right? So there was an automated way to make a call with an AI voice to make the reservation for.

• 7:26Chris Benson?

  Which seems completely legit to me because you're not, you're representing everything appropriately, you know, you're, you're not pretending, you're not getting around, uh, you know, that kind of thing. It's, you have a tool and it's a tool. And I think, and I frankly, I could use, I could use a few of those in my life, you know, and just take care of all the things, but I'm probably not going to call anyone and, uh, and have an AI model pretend to be Joe Biden or anybody else. So.

• 7:52Daniel Whitenack?

  Yeah, I think it definitely, like you were saying, it gets extremely concerning when there's a representation that this is this person and they're trying to sway your mind in one way or another, and it's not that person.

• 8:06Chris Benson?

  Yeah, pure ethical problem right there. I mean, that's so.

• 8:11Daniel Whitenack?

  Yeah. Well, I don't know. Do you think that this represents some of what we'll see this year in terms of a trend of government regulation of generated content?

• 8:22Chris Benson?

  I would not be surprised, especially, you know, when the, we talked last year about the executive order here in the US that came out. And I think that was indicative of further actions to come. I mean, they essentially laid out a strategic plan on how they were going to address AI concerns and FCC was one of the agencies, I believe, that was explicitly listed in the order, if I recall. And so I'm not surprised to see them weighing in on this at this point. So I it'll be interesting to see how it mixes across national boundaries and you know, see how various countries are addressing it and what that means for so much of this is is transnational in terms of

• 9:02Chris Benson?

  technology usage and even organization spanning. And so it will be a curious mess for all the lawyers to figure out going forward.

• 9:11Daniel Whitenack?

  Yeah, when the dialer is using Twilio or Telnext or something to spin up numbers, but they're doing it from an international account, which is probably not even in the country where they're operating. And there's all these layers. It gets into gets into some crazy stuff. I know that's always something that stands out to me. I always listen to the Darknet Diaries podcast. It's one of my favorites. So shout out to them for the great content that they produce. But yeah, that's always a piece of it, right, is putting enough of these layers in between to where, yeah, share those regulations, but.

• 9:52Chris Benson?

  We just need a blanket rule, a global blanket rule that's just do the right thing. Let's just everybody, everybody out there just do the right thing. But we may not have things to talk about on the podcast then.

• 10:04Daniel Whitenack?

  Yeah, well, the messiness of the real world will will continue. But yeah, but speaking of of Google, I mentioned the Google demos and the stuff they've done over the year with Synthesized Voices and all of that. And of course, recently, they've been promoting Gemini, which is this latest wave of AI models from from Google, which are multimodal kind of first models.

• 10:30Chris Benson?

  Yeah, there's a whole bunch of kind of related activity in there and that they took their existing chatbot Bard and they rebranded it into Gemini. And there are several, there's Gemini Pro. Very confusingly, there is the paid service now of Gemini Advanced, which is using the model called Gemini Ultra. So I know initially there was some confusion about advanced versus ultra. Well, advanced appears to be the service. Ultra is the underlying model.

• 11:01Daniel Whitenack?

  So pro represents a model size or ultra or or it represents a subscription tier.

• 11:07Chris Benson?

  Both in different ways. So pro is the free tier. There's nothing less than pro. We only start.

• 11:13Daniel Whitenack?

  Oh, obviously, yeah. We've talked about this with Apple products before. There's no low quality, anything, right?

• 11:19Chris Benson?

  Exactly. That's what I was about to say. There's no such thing as low quality. It's pro. You start with pro and that's the free version. It's the smaller model that we can all go in. You can go to Jim, just as you could go to bard.google.com. You now go to gemini.google.com and it's there and available.

• 11:37Daniel Whitenack?

  So bard is no more.

• 11:38Chris Benson?

  So bard is no more. Gemini Pro is roughly the equivalent of GPT 3.5, the free version on the OpenAI side. And now Google Advanced, which has the Google Ultra model, is competing against ChatGPT, which is hosting the GPT-4 model at the high end.

• 11:58Daniel Whitenack?

  Gotcha.

• 11:59Chris Benson?

  And there have been a billion reviews of how the two go against each other head to head.

• 12:04Daniel Whitenack?

  Have you tried the various ones or tried Gemini?

• 12:09Chris Benson?

  I've not tried Ultra yet because I haven't decided to pay for it because they're asking for 20 bucks a month. So I haven't been able to compare it directly. I've watched a whole bunch of YouTube videos more than I should have. We're a bunch of people doing side by side. And I think it's a really good model, but it generally it has met with some disappointment in that people are expecting the newest thing is always going to be the greatest thing possible. And I think we saw something with GPT-4 where when OpenAI released it and it had its initial, you know, fanfare, and then they've built a lot of infrastructure and services around it and, you know, the various plugins. They've also fixed a lot of the problems behind the scene while maintaining the actual

• 12:54Chris Benson?

  underlying model, whereas Google has not done that. They put the model out and it's comparable in many ways, but it feels very, very rough around the edges and it doesn't always give you the best output. So most of the direct head to head comparisons. Most of the various tests I've seen have had GPT-4 went out on a head-to-head thing. So my expectation on that would be that Google will start working around the issues that it has and cleaning it up. And probably within a few months, it'll probably catch up a little bit closer in that way.

• 13:29Daniel Whitenack?

  So our, our company and actually the, the last few I've been a part of have been big Google users in terms of G Suite and you know, Google Workspace and email and docs and all of that stuff. So I'm, I'm kind of embedded in that ecosystem and you know, I'm not, thankfully not having to deal with Teams or something like that, as I know many are.

• 13:55Chris Benson?

  I am at work. It's terrible. Oh, gosh.

• 13:59Daniel Whitenack?

  I feel for you and I guess I do experience that pain by, on a second order way because I have to take a lot of Teams calls. But anyway, outside of that, which is probably enough said, then, uh, so I, I'm always trying the Google stuff that comes out. And I had tried Bard and I think also before that, just the general interface to, I don't know if it was branded as, always branded as Bard. Or I remember Palm, but I think Palm was below or you know, embedded in Bard. I don't remember always what the branding was, but yeah, now there's Gemini.

• 14:37Daniel Whitenack?

  I would say my impression was similar, Chris, in that I just took literally one of their, you know how you log into any of these systems like ChatGPT or Gemini and I literally just tried one of their example prompts, like try this. I think it was like, print out how to do something in Linux or something like that. I, I think list processes or something. I just clicked the button like the example prompt and it wasn't able to respond to the to the example prompt, which I, you know, these are rough edges. I'm sure the model does a lot of things really well. And that was just like a fluke in many ways, but it I think does represent

• 15:19Daniel Whitenack?

  a lot of those rough edges that they're dealing with. And my impression, I've said this a few times on the podcast, it's like when you're a developer working directly with one of these models, it's kind of like taking your drone that's flying all great and you're controlling it. And then you take it out of autopilot mode and there's all of these things to consider that you really just didn't think about because they're taken care of by great products like Cohere, Anthropic, or OpenAI, or whatever. So I definitely feel for the developers because there's a lot of, a lot of things and a lot of behavior to take care of. But yeah, that was, this is not the best way to win me over, I think.

• 16:01Chris Benson?

  They might've done better to hold back just a little bit longer and do a little bit more. They talked about that they had roughly 100 private beta testers. And that seems to me a very small sampling of beta testers to be working on it. You mentioned another name just now, which I wanted to throw out that is very absent from this conversation out there. That is Anthropic. I don't see a lot of comparing it to Claude and stuff like that, or Claude 2 at this point.

• 16:31Daniel Whitenack?

  Or maybe, yeah, Anthropic and Cohere.

• 16:34Chris Benson?

  Yep, yep.

• 16:34Daniel Whitenack?

  Maybe some other ones.

• 16:35Chris Benson?

  Absolutely. Right now, it's been a two horse race between these two, which made me a little bit sad. I wish there had been more, a little bit more expansive and also against some of the open source models that are out there because one of the, the topics that you and I are often talking about is with the proliferation of mini models, some of which are private, some of which are, are open. It increases the, the challenges for the rest of us in the world to know what to use and when and when to switch and things like that. Something that I know you know quite a lot about.

• 17:08Daniel Whitenack?

  Yeah, it's been intriguing to see all of these. And I would say all of them are on some type of cycle, right? So we're talking about maybe GPT-4 is in the lead and here comes Gemini and then. We're mostly talking here about the closed proprietary models, that sort of ecosystem. But then I'm guessing Claude had a big release at some point and they're probably in their cycle where I have no inside knowledge of this, but it's just my own perception that Anthropic, Cohere, they're in a different release cycle, obviously, than OpenAI and Google. So we'll see something from them in the coming months, I'm sure, in terms

• 17:50Daniel Whitenack?

  of upgrades or multimodality or extra functionality like assistance or tying in more things like RAG and that sort of thing, as we've seen with OpenAI's assistance and file upload and that sort of stuff.

• 18:06Chris Benson?

  You know, if we're fair about it, when you think back to when GPT-4 came out, it didn't have all the things that, you know, the ecosystem has grown substantially since its release and it had some of the same challenges of that. And I think this might be with Gemini coming, you know, I think everyone kind of took that for granted. They were a little bit less splashy than a big giant new model coming out. And I think this is one of those moments where you kind of go, wow, there's more to this than just the model itself, you know, big new model, I got that. But there's so much to the ecosystem around a model and the various plugins, capabilities, extensions, whatever you want to call them, Google plus them extensions and at this point. But I think it really

• 18:50Chris Benson?

  goes along the lines of something we've been saying for a long time and that the software and the hardware, it's all one big system. It's not just about the model. So I suspect Google is very well positioned to make the improvements in the coming weeks. So it may be interesting to revisit some of these tests after a short while.

• 19:09Daniel Whitenack?

  Yeah. And there are other players that are kind of playing on this boundary between open and closed, either on that sort of open and restricted line. So releasing things that are open and not commercially licensed or open source, but with some other usage restrictions and that sort of thing. There's cool stuff happening in all sorts of areas. One of the ones that we've been looking at is a model from UnBabel, which is a translation service provider. They have this Tower family of models, which does all sorts of translation and grammar related tasks. But there's also a lot of multimodality stuff coming out.

• 19:52Daniel Whitenack?

  So I noticed, you know, we talked about text to speech at the beginning of this episode. And I'm just looking at the most trending model right now on Hugging Face is the MetaVoice model, which is a one billion parameter model that is text to speech. But if I'm just looking through kind of other things that are trending, we've got text to speech, image to image, image to video, semantic similarity, which are of course kind of embedding related models, text to image, automatic speech recognition or transcription.

• 20:35Daniel Whitenack?

  So there's really a lot of multimodality stuff going on as well and people releasing that. I know one that you highlighted was some stuff coming out of, I believe it was Apple, right?

• 20:47Chris Benson?

  Yes.

• 20:47Daniel Whitenack?

  Related to image modi- or what, how is it phrased? Image modification or something like that? Image editing?

• 20:54Chris Benson?

  Image editing. It's MGIE is the acronym, which I'm guessing there, I haven't heard them say this, but I'm guessing they're calling it Maggie or something like that. And it is a, where you, you'll give a source image and they have a demo that's on Hugging Face and you essentially kind of talk your way in through the editing process and gradually improve it and everything. So I think they had the bad luck of announcing this and releasing it at the same time that Google did Gemini to go head to head on GPT-4. So I think it largely got lost in the news cycle, but it looks like it might be a very interesting thing. And I think, you know, they're competing against like Adobe, you know, doing image

• 21:34Chris Benson?

  generation and all of these companies have some level of image editing model capabilities. So it will be interesting to see how Apple's plays out and how they apply it to their products.

• 21:45Daniel Whitenack?

  What I think is a differentiating or interesting element of this, which is maybe not Text to image or text to text sort of completion. But the common types of things that people are wanting to do, which are somewhat model independent, but are more workflow related. So things like RAG pipelines where you upload files and interact with them. You have kind of GPT models or the OpenAI ChatGPT interface where certainly you can upload files and chat with them or analyze them. Anthropic actually was an early one where because of their high context length window

• 22:29Daniel Whitenack?

  models had the ability to upload files and chat with those files. I don't think, at least I couldn't tell something similar in Gemini, other than uploading an image and chatting or reasoning over that image, which is sort of like the vision piece of it. But more than multimodality, there's these increasing workflows that people are developing. One of those that I think is really interesting is the data analytics use cases that are coming out. So you have, actually, I've seen a trend in a lot of these companies popping up that

• 23:06Daniel Whitenack?

  are something to the effect of new enterprise analytics driven by natural text queries. So I'm thinking of like D-Fog, I think it is.

• 23:18Chris Benson?

  Yes.

• 23:19Daniel Whitenack?

  These companies, which are a chat interface where you type in a question. Maybe your SQL database is connected and you get a data analytics answer or a chart out. And this is something that I believe if I'm, again, understanding, I don't know all the internals of ChatGPT, but it's interesting that there's different takes on this approach. And I think there's a lot of misunderstanding about how this actually happens under the hood. So I don't know, have you done much where you've like uploaded a CSV or you've done that sort of thing in ChatGPT and asked it to analyze it or something like that?

• 24:03Chris Benson?

  Ironically, that's literally something I'm playing with right now. I know you didn't know that before asking the question, but I saw a similar post about kind of analytics being used for this. And so I'm experimenting with it, but I'm still very early.

• 24:18Daniel Whitenack?

  How are your results initially?

• 24:21Chris Benson?

  They're not as good as I want, but I think that's mainly my problem. I keep running into little bumps where I'm trying to get the CSV usable very well. So I have a database that I dumped some data out of and was trying to do that. But I literally just did this today. It was today was day one and then stopped and came in for us to have this conversation. So let me let you know in another week or so how that fanned out. But it caught my eye because I saw a conversation online about this. And some of the personalities that I I've always associated with, you know, being super technically bright analytics folks were kind of saying, we're just hitting that moment where this kind of just AI driven conversational analytics is now going to be available to everyone.

• 25:06Chris Benson?

  And I was like, that's what I want. That's what I need. So I'm actually trying to do something, do something for work right now on those ones.

• 25:30Speaker 03

  What's up, friends? Is your code getting dragged down by joins and long query times? The problem might be your database. Try simplifying the complex with graphs. A graph database lets you model data the way it looks in the real world instead of forcing it into rows and columns. Stop asking relational databases to do more than what they were made for. Graphs work well for use cases with lots of data connections like supply chain, fraud detection, real time analytics and generative AI. With Neo4j, you can code in your favorite programming language and against any driver. Plus, it's easy to integrate into your tech stack. People are solving some of the world's biggest problems with graphs. And now it's your turn. Visit neo4j.com/developer to get started.

• 26:15Speaker 03

  Again, neo4j.com/developer. That's N E O four J dot com slash developer.

• 26:37Daniel Whitenack?

  Well, Chris, I was asking these questions about this data analysis stuff because this is, I've done a few customer visits recently where we've been talking about this functionality. And I've noticed as I've gone around and talked to different people, there's some general misunderstanding about how you can analyze data with a generative AI model. One, because there's something people think is going on that isn't actually going on. And two, because generally, if you ask a language model, just a chat model without uploading data, like math type of questions, usually it is really terrible at that, right?

• 27:21Daniel Whitenack?

  Even like adding things together or doing like basic aggregation is something that these models are known to fail on pretty poorly. And so the question is like, well, how am I getting anything relevant out of these systems to begin with? And again, I don't know all the internals of ChatGPT, but this is my own understanding. There's some difference if you look at maybe like an example like D-Fog or ChatGPT or Vana AI. These are some examples of this that's going on. ChatGPT takes the approach, in my understanding, where in their assistance functionality,

• 28:01Daniel Whitenack?

  so when you type, you upload a, maybe a CSV and you ask a question and you wait for seemingly forever while the little thing spends and it says it's figuring or analyzing, I think is what it says, something like that. Yep. My understanding of what's happening is more of what they used to call code interpreter. It's actually generating some Python code that then it executes under the hood to analyze your data that you uploaded and then somehow passes along the results of that code execution to you in the chat interface. So this is a very astute observation by whoever had this, that yeah, these models really stink at doing math.

• 28:45Daniel Whitenack?

  But what doesn't stink at doing math is code, right? So these models are pretty good at generating code. So why don't we just sidestep the whole math thing and generate the code and then execute it? And crunch your data and we're good to go. I think the thing that often what I've seen people struggling with like the assistance API and chat GPT is again, they have to support all sorts of random general use cases, right? Because, you know, people could upload a CSV of all sorts of different types or other file types. And so there's a lot to support and it's kind of generally slow and hard to massage and to working, right?

• 29:29Daniel Whitenack?

  What I've seen more in the enterprise use cases that we've been participating in is less a focus on code generation to do the data analysis and more of a focus on SQL generation to do analytics queries. So this is more the approach of the SQL coder, family of models, DFOG, VANA AI. We're doing very similar things to in the cases where we're implementing this similar to the VANA AI case where you connect up, let's say you have a transactional database, like your sales or something like that, or customer information or product information. And you want to ask an analytics query, right?

• 30:11Daniel Whitenack?

  Well, SQL is really good at doing aggregations and groupings and joins. Also, large language models, especially code generation models or code assistant models are really good at generating SQL because like how much SQL has been generated over time? It's, it's a very well-known language to generate, right? And so you kind of sidestep the code execution piece in that case where you're not generating Python code, but you're generating from a natural language query, a SQL query to run against a database that's connected. And you just run that SQL query in normal, good old, regular programming code to give you your answer.

• 30:54Daniel Whitenack?

  And then you send it back to the user in the chat interface. So I thought that would be worth highlighting in this episode because there does seem to be a lot of confusion of what's actually going on under the hood. Like, how can one of these models analyze my data? Well, the answer is it kind of isn't. It's just generating either code or generating SQL that is analyzing your data.

• 31:16Chris Benson?

  It still gets you there, though. It's in a sense, you know, since you're not directly having the model do it, it's sort of a workaround in a manner of speaking. But I think if you look at something like, you know, the ecosystem built around ChatGPT, there's a lot of tooling around it. And I think that's, I think this year, we're going to see more and more of that, you know, whether it be the SQL use case that you're talking about or continued with OpenAI. I think Google will do that. Well, I think Anthropic will get on that. And you'll see these kinds of tools for doing exactly that kind of thing, where you may not have a model that does a particular task super well, but it can produce an intermediate that can do something very, very well.

• 32:02Chris Benson?

  I think that's a level of, you know, we keep talking about maturity of the field. And I think part of that is recognizing maybe there's a better way to do it than just having the bigger, a better latest model. So yeah, I think that's a great way of approaching it.

• 32:15Daniel Whitenack?

  Not to self fulfill my own prophecy from our predictions from, from last year. I think in our 2024 predictions episode, one of my predictions was that we would see a lot more combination of, I think what is generally being called neuro symbolic methods, but maybe more generally just like hybrid methods between what we've been doing in data science forever and a kind of front end that is a natural language interface driven by a generative AI model. So in this case, what we have is good old fashioned data analytics, just like the way we've always done it by running SQL queries. It's just we gain flexibility in doing those data analytics

• 32:59Daniel Whitenack?

  by generating the SQL query out of a natural language prompt using a large language model. And I think we'll see other things like this, like, you know, tools and LangChain is a great example of this, where you generate good old fashioned structured input to an API and that API is called and gives you a result. But this could be applied in all sorts of ways, right? So let's say time series forecasting. I don't think right now, language models, and I've actually even tried some of this with fraud detection and forecasting and other things with large language models. Not very good at doing these tasks, but they can generate the input to what you would

• 33:44Daniel Whitenack?

  need in the kind of traditional data science tasks. So if you say, again, imagining bringing in the SQL query stuff, if you have a user and you want to enable that user to do forecasts on their own data, well, you could have them like put in, fill out a form and like in a web app and like click a button and do a bunch of work, or you could just have them say, hey, I want to forecast my sales of this product for the next six months or something. From that request, a large language model will be very good at extracting the parameters

• 34:25Daniel Whitenack?

  that are needed and possibly generating a SQL query to pull the right data that's needed to be input to a forecast. But that forecast is going to be best to best that you just use like Meta's Prophet framework or something. It's just a traditional ARIMA statistical forecasting methodology and you just like forecast it out with that input and then you get the result, right? So this is a very, it's the merging of what we've been doing in data science forever with this very flexible front end interface. And I think we'll see a lot more of that.

• 34:59Chris Benson?

  I completely agree with you. And not only that, but I think there'll be a lot more room for LLMs that are not the gigantic ones. You know, we've talked a bit and we've had guests on the show recently, you know, talking about the fact that there's room, not only for the largest, latest, greatest giant model, but there's enormous middle ground there where you can have smaller ones and combine those with tools. So it's pretty cool seeing people innovate in this way and start to recognize that not everything has to come out of the largest possible model you have available to you and add that in. So I'm really looking forward to seeing what people do this year along in the various industries and, you know, and how that spawns new thoughts. So.

• 35:42Daniel Whitenack?

  Yeah, and especially with a lot of things being able to be run locally. I've seen a lot of people using local LLMs as an interface using frameworks like Ollama and others, which is really cool to be able to use LLMs on your laptop to automate things or do these types of queries or experiment locally. So yeah, I think that even adds another element into the mix.

• 36:11Chris Benson?

  And for edge computing, for truly edge computing where it's not practical to have a cloud backing or the networking between where that model would be in the cloud and where you're trying to do it, there's a huge amount of opportunity to use them in that in that area. So yeah, I'm hoping that we see a lot of innovation. Last year was kind of the, even the year before, it was kind of the race to the biggest model. I'm kind of hoping now we see what what other branches of innovation people can can come up with to take advantage of some of that and also recognize that the midsize ones have so much utility to them that's untapped.

• 36:48Daniel Whitenack?

  Yeah, and maybe before we leave the sort of news and everything that's going on in this kind of co-pilot assistant analysis space, I did see, you know, I actually, my my wife needed help connecting to printers. Printers are not a problem that is solved by AI yet, I guess, and will continue to be continue to be a problem forever in tech. But I was noticing in the, you know, recent updates to Windows, there's the little co-pilot logo there, like even embedded within Windows. And I don't know that whoever watched the Super Bowl during, in the US, the the Super Bowl, as we record this, was the the day before we're recording this.

• 37:32Daniel Whitenack?

  But there was a co-pilot commercial during the Super Bowl. And that's another interesting thing, because this is now, it's running on people's laptops everywhere. And of course, that's connected to the OpenAI ecosystem, in my understanding, through Microsoft.

• 37:50Chris Benson?

  Right.

• 37:50Daniel Whitenack?

  But yeah, this kind of AI everywhere and also the sort of AI PC stuff that Intel's been promoting and running locally is is going to be an interesting piece of it.

• 38:02Chris Benson?

  Totally agree. As we wind up, I want to briefly switch topics here. I received some feedback a few episodes ago. From a teacher who is listening and I was so happy that to have, you know, one and maybe many teachers out there listening to us and considering this. And as we often do, people may not realize, but Daniel and I, we have a topic, but we are largely unscripted. So we are kind of shooting from the hip in terms of what we're saying. It's a very genuine and real conversation. We're not looking at a whole bunch of notes and pre-planned script. And I, I made a comment about my daughter in school and the fact that I really think schools should take advantage of models and as part of the learning process, as part

• 38:47Chris Benson?

  of the teaching to integrate it in. Whereas often, school systems right now are saying, you're not allowed to use GPT, for instance, in your homework. In that, I said, ah, that's stupid. You know, that teachers would not do that. And I, this teacher reached out and said, well, first of all, we really want to, and I'm paraphrasing her. And she said, second of all, you know, a lot of times they, it's not in their power anyways, the school system policy and stuff. And so I just wanted to apologize to anyone, especially the teachers out there that might have been offended. I'm much more cognizant now of what I'm saying on that. It was kind of a shooting from the hip, but it was insensitive. And I found that what that teacher pointed out was dead on. It was right on. And I just want to thank

• 39:31Chris Benson?

  the teachers out there, especially those who are trying to take advantage of these amazing new technologies and talk their systems into bringing them into the classroom and not make it just the bad thing not to use for homework. So thank you to the teachers for doing that. And I just wanted to call that out. It's been a really important thing from my standpoint to say. So, thank you.

• 39:52Daniel Whitenack?

  I think it represents the complexity that people are dealing with.

• 39:56Chris Benson?

  It does.

• 39:56Daniel Whitenack?

  You know, teachers want their students to thrive. I think generally, we should assume that most teachers are really actually motivated and engaged, both in culture and technology and the ecosystem, wanting their students to thrive. But sometimes, like you say, they have their own limitations in terms of what, what is the system within their, that they're working in and. You know, privacy concerns and other things. So yeah, that's a good call out, Chris. I'm glad you took time to mention it.

• 40:30Chris Benson?

  I want to say one last thing and to teachers out there who are trying to get these things into the classroom so that your students have the best available tools to do things. If you ever need someone to back you up, reach out to us. We have all our social media outlets. You can find us, find me on LinkedIn. And if I will be happy to give a whole bunch of reasons to your school systems on why they might want to use the tools. I'll be happy to work with you on that. And I thank you for fighting that fight on behalf of the students that you're serving.

• 41:01Daniel Whitenack?

  Yeah. And speaking of of learning, something that we can all learn and be better at is all the different ways of prompting these models for multimodal tasks and prompting and data analysis. And I just wanted to highlight here at the end, a learning resource for people. A while back, I had mentioned a lecture and series of slides that were was very helpful for me from DAI D A I R. Now I think that they've converted that series of slides and that prompt engineering course, I think is what they call it, into a prompt engineering guide. So if you go to a prompting guide.ai,

• 41:44Daniel Whitenack?

  they've have a really nice website that walks you through all sorts of things and also covers various models in terms of the, you know, chat, GPT, Code Llama, Gemini, Gemini Advance. We talked about those on this show and talks about actually prompting these different models. So I'd encourage you, if you're experimenting with these different models and not immediately getting the results that you're wanting, that may be a good resource to help you understand different strategies of prompting these models to get things done as you need to get them done.

• 42:17Chris Benson?

  That's a great resource. I'm looking through it as you're talking about it and it's, it's the best I've seen so far.

• 42:23Daniel Whitenack?

  Well, Chris, this was fun. I'm glad we got a chance to cover all the the fun things going on and we've complied with the FCC using our actual voices still. We'll see if how long that lasts, but it was, it was fun to talk through things, Chris. We'll see you soon.

• 42:41Chris Benson?

  Talk to you later.

• 42:49Speaker 00

  That is Practical AI for this week. Thanks for listening. Subscribe now if you haven't yet. Head to PracticalAI.fm for all the ways. And don't forget to check out our fresh changelog beats. The Dance Party album is on Spotify, Apple Music, and the rest. There's a link in the show notes for you. Thanks once again to our partners at fly.io, to our beat freakin' residents, Breakmaster Cylinder, and to you for listening. That's all for now. We'll talk to you again next time.