How ChatGPT's Search Algorithm Works, Is ChatGPT the New Distribution?, AI Recruiters vs. Humans, Does Traffic Even Matter Anymore?

In this episode #3031, Eric Siu and Neil Patel break down ChatGPT’s ranking model, the value of fresh content, and how AI is changing marketing and hiring. They also stress linking traffic to revenue and the need to build AI skills to stay ahead. TIME-STAMPED SHOW NOTES (00:00) Understanding ChatGPT's Ranking Model (03:08) The Importance of Freshness in Content (05:51) AI as a New Distribution Channel (09:11) Traffic vs. Revenue: The Real Metrics (11:51) The Role of AI in Recruitment and Hiring Practices To suggest a topic, go to https://www.marketingschool.io. For more content from Eric and Neil, check out Eric’s Leveling Up with Eric Siu YouTube channel and Neil’s Neil Patel YouTube channel. Connect with Us: Single Grain << Eric's ad agency NP Digital << Neil’s ad agency X @neilpatel, @ericosiu Instagram @neilpatel, @ericosiu Drop Us a Review If You Enjoyed the Episode!

• 0:00Speaker 00

  Finding customer information shouldn't mean digging through emails, call notes, and spreadsheets. HubSpot's Starter Plan brings all of those details together, so it's easier to track what's happening with every deal and what to do next. It pulls information from calls and emails. Then it gets to work, logging and summarizing them. Right after your call ends, HubSpot pulls next steps from your call notes and drafts suggested follow-ups. And it stays up to date for you, so you can build demand, win more deals, and spend less time on repetitive work. Your records, insights, and next steps are ready for you to act on. And you don't have to miss a beat searching for any of it. Try the Starter Plan for $10 a month. Learn more at hubspot.com.

• 0:35Speaker 00

  Why don't we move over to how to rank highly, how ChatGPT's search algorithm works? You want to know how ChatGPT's algorithm works?

• 0:42Speaker 01

  I kind of already know, but yes, let's talk about that. But before we go into that, I started integrating ads into my YouTube based on your feedback.

• 0:50Speaker 00

  How's that going?

• 0:50Speaker 01

  We were talking about our podcast a few weeks ago or a month ago, and you're just like, dude, we get a lot of leads for single grain and carrot by me just mentioning, you know, in the middle of the recording, hey, if you want us to do this for you, or you want us to do X, Y, and Z, just go to single grain. So we started doing it. I have no data yet from it because only one video, I believe the perplexity one has it.

• 1:13Speaker 00

  That was doing well.

• 1:14Speaker 01

  Yeah, it's doing well. And then the next one that comes out next week has it, of course, as well, because all of them have it now. But by the time we record next week, I should have more data for you.

• 1:24Speaker 00

  Yeah, that's that. I guarantee it'll be a good outcome. And plus, like, even if it's, even if it's a small lift, it doesn't matter because you're making the content anyway.

• 1:32Speaker 01

  Yeah, it's free. Small lift is better than no lift.

• 1:35Speaker 00

  Yeah, yeah. I'm curious to see what the deal looks like. Okay, so guys, let's, I want to talk about how ChatGPT's ranking model works. And then this, I'm going to share a screen here. So Chris Long shared this. He's actually a pretty good curator. So how ChatGPT's reranking model works. And so I think what's work worth calling out here is this says, "At the heart of ChatGPT's retrieval system is a reranking model with the cryptic name Ret RR Sky Site." And it's a sophisticated post-processing layer that takes initial search results and completely reorders them based on quality signals. So number one quality signal here is freshness. Is king. So perhaps the most significant finding for content creators is this setting.

• 2:14Speaker 00

  Use freshness scoring profile equals true, right? And so there's a bunch of other findings, but we've kind of been talking about this where I think AROF cited that 95% of chat GPT results have been refreshed in last 12 months or so. This is it's still a kind of traditional SEO, right? And then this guy um shared over here. So you can see this use freshness scoring profile is true. And then he cites some other stuff and then you have all this other stuff here down here. Configuration reveals sophisticated filtering pipeline with multiple checkpoints, enable query intent, source filtering. But I think probably the thing that's worth calling out, I'm not going to go through all this other stuff is the freshness piece. What's your take?

• 2:52Speaker 01

  Yeah, the freshness piece, we've seen it as well. Freshness is very important and people miss that with chat GPT and even Google. We're seeing freshness be a super important metric, but the problem that we're seeing with freshness is marketers are looking at it as, oh, let me just change one sentence and update the timestamp and I'm good to go. That's not what they consider fresh. What they consider fresh is you updating it and making it valuable to what a consumer would ideally want to see in today's world. But on the flip side, if you have a piece of content that's performing well and you want to update it two months later to keep it fresh,

• 3:31Speaker 01

  but you're just going to change a sentence because there's nothing really that you need changed and it's already good to go and as up to date as it can be. You don't need to update it for the sake of it, just to change the timestamp.

• 3:42Speaker 00

  Yep. So speaking of chat GPT, by the way, remember Brian Belfour from Reforge? So he did a podcast on Lenny's podcast. I was listening to it when I was, I think, at the gym last week, but the question is, is the new distribution channel chat GPT? And before we even answer that question, when you think about Facebook back in the day, organic, anything you posted on Facebook would get a lot of views, right? And then eventually you'd have to pay for it. LinkedIn, now it's kind of trending in that direction. Strong organic reach is kind of moving into paid, right? So you always get really strong organic distribution in the beginning and eventually you have to pay for it. And I remember talking to an executive at Microsoft and he even said that, hey, we're actually considering being able to pay for some results inside of like an LLM, right?

• 4:26Speaker 00

  For the moment with ChatGPT, sure, you know, with with all searches, Google's still forty two percent and ChatGPT or LLMs, I think is only like point one nine percent, but it's growing quickly. The the question is, is the new distribution channel ChatGPT? And I would argue that I think it's it's a strong new distribution channel for people to consider. And there probably is good organic reach to think about right now, but eventually you're gonna have to, it's gonna become more pay to play. And same thing with Snap in the very beginning, or when Stories launch on Instagram, they tend to favor these new features. And so I think that it's it's it's definitely worth paying attention to. And eventually, it's it's gonna it's gonna kind of whittle down, but it's in the early stages of growing really quickly. It's still small, but it's growing quickly. Go ahead, Neil.

• 5:11Speaker 01

  Yeah, I'm just curious to see what will happen over time, but yeah. And then I I think everything, just like you and I have seen, it all becomes pay for play models. These guys all try to push you because it monetizes that way. It's just like Google AI mode, they're just pushing, you know, their whole goal is to push ads through it. It's launched recently in a hundred and eighty countries. But all, anything that's good in marketing, if it keep or anything that's good from an engagement usage perspective. All these platforms will eventually figure out how to push you towards a model where it monetizes and that makes does really well for them. Doesn't mean organic stat, it just means you gotta pay for to play.

• 5:52Speaker 00

  Yeah, organic eventually becomes paid.

• 5:55Speaker 01

  Yeah, because even SEO, you know, so many people hit us up and be like, yeah, I want to do SEO because I'm not paying for it. It's free. And it's in my best benefit to be like, yeah, it's free, but I don't want to lie to people. And I'm like, technically, yes, from a perspective of not paying Google per click, but you are paying people, whether it's in-house or agency, to help you get rankings. And not gonna lie, you gotta maintain it and keep doing more also that people outrank you, and that does cost money. So even though you're not paying the platform directly, you are spending money on it.

• 6:25Speaker 00

  Anything good has a price. Okay. So, by the way, remember we talked about the 8,000 bots versus, 8,000 people versus one chatbot? So, there's a study here on how AI recruiters crush human recruiters. And this is from an AI researcher, I believe. And so, AI in HR. In an experiment with 70,000 applicants in the Philippines, an LLM voice recruiter, so this is the AI recruiter, beat humans in hiring customer service reps with 12% more offers and 18% more starts. Also, better matches. So 17% higher one-month retention, less gender discrimination, and equal satisfaction. And then it says down here,

• 7:06Speaker 00

  "The break-even point, including all software and inference costs, was 8,500 interviews." And you can see over here, we're looking at the graph right now. The AI interviewer, in the blue bar graph versus the human one in the gold, let's call it that. Fraction receiving job offer, it was higher for the AI interviewer. And then, fraction started job, higher for the AI interviewer. And then, fraction meeting one-month retention rate, higher for the AI interviewer. And, um, look, this part is the most interesting to me. So, "When offered a choice, 78% of applicants chose, they choose the AI recruiter. And we find evidence that applicants with lower test scores are more likely to choose AI." What do you think about that, William?

• 7:51Speaker 01

  Yeah.

• 7:52Speaker 00

  That's right.

• 7:53Speaker 01

  You prefer the AI or the human?

• 7:55Speaker 02

  Human.

• 7:56Speaker 01

  No, you're supposed to prefer the AI.

• 7:59Speaker 00

  That's okay. William has a heart.

• 8:03Speaker 02

  Yeah, Bugatti.

• 8:05Speaker 01

  Daddy's working right now to try to get you the Bugatti.

• 8:07Speaker 02

  Bugatti.

• 8:09Speaker 01

  I need 10 more years. But can daddy finish this? Okay, seven.

• 8:14Speaker 02

  You can finish seven cars.

• 8:16Speaker 01

  Okay, that's how I get to work. He says I can work for a Bugatti.

• 8:20Speaker 00

  Yeah, Neil's working hard for a Bugatti. There's always a purpose, guys. So yeah, what's your, what's your, what's your take on this? Cause we remember we did the one on the HR bot, the eight thousand people versus one HR bot. And in this case, AI recruiter and seventy eight percent of people, the applicants prefer the AI recruiter.

• 8:36Speaker 01

  You're talking about the IBM story, right?

• 8:38Speaker 00

  Yeah, yeah, yeah.

• 8:38Speaker 01

  In the past. That did really well on your Instagram, by the way.

• 8:41Speaker 00

  It did. It's funny how.

• 8:43Speaker 01

  I, I, I'm with you on this. I, I think AI can do a lot of this stuff better than humans. I think in some cases, AI is better and in some cases, humans, but in this case, yeah, I see the value in it. Although on your HR one with IBM, uh, people started leaving comments about the long-term effects and how the company's not looking at that and they're being more short-sighted.

• 9:05Speaker 00

  I mean, people are always gonna say stuff like that, so I, I don't, I don't know, like, yeah.

• 9:10Speaker 01

  Yeah, I take it with a grain of salt.

• 9:12Speaker 00

  Yeah. All right. So that's that. Um, we, we got one more to do and then we'll be on our way.

• 9:20Speaker 01

  So, um, let's see. Oh, uh, here's one. You know, I know we're talking about the profound traffic increase or decrease. And I know people have had similar issues for Google over time and whether it's increases or decreases. Everyone's talking about their traffic increases or decreases, but I don't know about you. Very few people start correlating it with their revenue going up or down. Like you see all these posts online, oh, look at my Google traffic, look at my chat GBD traffic. Oh.

• 9:47Speaker 00

  I don't even look at my traffic anymore. Do you?

• 9:49Speaker 01

  No, I, I, I technically my team does, so I don't have to and I get a weekly report. But, um, it's all correlated on our end. Our data analytics team correlates it to revenue. Cause I'm just like, who cares if your traffic went up double, but you didn't make any more money? And who cares if it went down half and you didn't lose any more money? I care on what traffic and what visibility is actually causing revenue to go up and I want more of that. And if it starts going down, I want to fix it, right? But I'm not talking about the traffic part. I'm talking about the revenue part. And I think marketers got it all wrong. For some reason, none of them really focus on revenue. They all wanna look at traffic and. That's cool, but if traffic in most cases, and you and I know this when it comes to content,

• 10:31Speaker 01

  that it's not a one to one. Going up doesn't mean you're gonna make more money and going down doesn't mean you're gonna lose money.

• 10:36Speaker 00

  Frictionless growth marketing hosted by Sonya Thompson is brought to you by the HubSpot Podcast Network, the audio destination for business professionals. Frictionless growth marketing hosted by Sonya Thompson shows how modern brands grow by removing the friction in their marketing and customer experience that pushes customers away every single day, often without leaders even realizing it. Each episode delivers practical insights, real brand examples, and conversations with marketing leaders showing brands how to win with the new rules of growth, identity-driven marketing, cultural relevance, and frictionless customer experiences. Listen to Frictionless Growth Marketing wherever you get your podcasts. Do you want to know what is super irritating to me that's kind of like somewhat related to this is because I've been spending a lot more time on recruiting. So when people make it over to me and I'll talk to them,

• 11:20Speaker 00

  I'll just, one of the questions I ask is, what are the, what are two of the hardest problems that you've solved? Please share numbers, right? And I don't know why, maybe this is an agency world thing, but it's always like, oh Neil, I increased the traffic by 400%, or I increased this by 200%, I increased revenue. Even if you increase revenue by 100% or whatever, but on what? Like on $1? So is it $2 now? Like, I don't know what it is. I get so bad.

• 11:45Speaker 01

  Oh, I created 364% growth. Okay, what was your baseline? Oh, we were doing $10,000 a month in MR. Alright, that's not much of a growth for me.

• 11:56Speaker 00

  I'll tell you what, guys, like you're listening to this right now, the hiring bar is gonna continue to, like if you're a founder, you're gonna, you have to continue to raise the bar more and more. And so when you're asking these people these questions, they have to report on a baseline and they have to have some type of numerical fluency. And we're constantly talking about traffic and how much percent they increase things by. You cannot afford to hire these people because they are not putting enough time to invest in themselves. And another question we asked Neil is, hey Neil, how much are you investing out of pocket in AI right now? Even if someone tells me $20 a month for ChatGPT Pro, amazing, right? But if you say $0 a month, we cannot hire you, right? Some people, by the way, remember I told you, some people in Brazil, $500 a month is a lot of money in Brazil, right?

• 12:35Speaker 01

  Yes.

• 12:36Speaker 00

  Spending $500 a month out of pocket, their money on AI stuff, right? And that's how serious they are because they're looking to enhance themselves and just do better.

• 12:44Speaker 01

  Dude, I'm with you. But yeah, you and I have the same pet peeve when it comes to interviewing and talking to candidates. And I don't care where you are, you know, like whether it's Brazil or India, if you're not investing to improve yourself, then why would you try to improve the business that you get hired by? You probably won't care.

• 13:02Speaker 00

  You know, you can't have two types of cultures in your company. Someone's either AI for it or not. So if you, if you hire a set of people, they're, and they're kind of anchors and they're not, they're not for it. And you have one group that are, they're gonna fight each other the whole time. And then you're, you're gonna keep, you're not gonna be able to move in the right direction. Whereas if everyone's putting in one direction, you're gonna move in that direction, right? I was listening to, um, uh, Brian Armstrong from Coinbase. He was, I think he was talking on the Stripe podcast. So that's actually a pretty good podcast, Cheeky Pint. Um, so you had the Stripe founders interviewing the Coinbase founder, right? And he's like, dude, um, he was like, yeah, we were very serious. We said, if you're not into AI and we don't see our usage, we're, we're gonna fire you, right? And, um, there's a group of people that didn't use it. So I think he had them come in on like a Friday or Saturday or something like that.

• 13:45Speaker 00

  And some people had legit excuses, like some people were coming back from a vacation, but then he fired the rest. And so he was just saying that that was very heavy handed. Um, but he's like, that's what's required. And keep in mind, Brian Armstrong back in 2020, he had the guts to say, hey, guys, we have one mission and we're focused on doing, we're focused on pushing crypto forward. Like whatever politics you have, whatever you have on the outside, it's already hard enough to have one mission. We cannot afford to have this other stuff. And everyone was pushing back against it. It was like, oh my God, we're gonna lose like half the company. Five percent of people quit, right? So I think sometimes you come up.

• 14:15Speaker 01

  It better. His stocks, you know, done well over the years. I know right now it's down a little bit compared to what it was a month ago. But who cares? Overall, it's growing in the right, or going in the right direction.

• 14:24Speaker 00

  Yeah, but I, I think where, where, where we're landing the plane here is, is you have to have the guts to make a call when it's unpopular. And Brian Armstrong was, was okay doing, because it's funny, you had Google make a call like maybe a year ago, so it's saying we're not a political company. But that's already like years and years after Brian Armstrong did that, right?

• 14:40Speaker 01

  And it's years and years after Google has had videos with a laugh about political stuff and the videos get leaked.

• 14:47Speaker 00

  But think about it, dude, how hard is it to already have one mission? It's, but to fight on all sides, it means like the, the, the, the people are getting distracted and maybe don't have enough to work on or whatever. But, um, if you're running a company or if you're a leader, that's what you gotta focus on, right, William?

• 15:01Speaker 02

  Yeah.

• 15:02Speaker 00

  That's right. That's right.

• 15:03Speaker 01

  You know Uncle Eric's the one who gives you those cookies? The last time you ate cookies, it was from Uncle Eric. What do you say to him?

• 15:09Speaker 02

  Thank you.

• 15:10Speaker 00

  You're welcome. Well, I got more cookies coming for you, but Neil will decide, your father will decide how many you get. So it depends on how good you are.

• 15:18Speaker 02

  And Emma too.

• 15:20Speaker 00

  All right, two. Hey, I don't control.

• 15:23Speaker 01

  He's saying Emma two, Emma two. Yeah, of course.

• 15:25Speaker 00

  Emma two. Oh, yeah, Emma two, Emma two, of course.

• 15:27Speaker 01

  Hey, thank you.

• 15:28Speaker 02

  Thank you.

• 15:30Speaker 00

  Talk to you later, William.

• 15:31Speaker 01

  Take care, everyone.

• 15:32Speaker 00

  We're done. Bye.