Pod Insights Marketing School - Digital Marketing and Online Marketing Tips I’m Dominating SEO. If You’re Struggling To Get Impressions, Watch This About this episode Growth Newsletter: https://levelingup.beehiiv.com/subscribe Need marketing help? Visit: https://www.singlegrain.com/ and https://npdigital.com/ Want to recruit great marketers? Find them here: https://marketingschool.io/hire Neil and Eric break down how SEO, AEO, and AI search are changing faster than most marketers realize. They explain why the new way to win is through APIs and agents, why AI-generated content often fails over time without a human in the loop, and how evals are replacing old school testing cycles for the best AI teams. They also get into why publishing listicles on your own site can quietly hurt organic traffic, how synthetic data could reshape marketing measurement, and why every CEO may soon be running their own internal AI agent. Key Takeaways: The future of SEO and AEO is increasingly API-first, with agents doing the work faster across content and workflows Pure AI-written content may get short-term gains, but long-term performance still depends on human editing, iteration, and strong systems AI evals are speeding up experimentation, helping teams test and improve workflows far faster than traditional approaches Chapters: 00:00 Why SEO and AEO Are Moving to APIs 01:16 Why AI Content Still Struggles to Rank 02:57 The Listicle Strategy That Can Hurt Organic Traffic 06:59 Why AI Evals Are Replacing Old Testing Loops 12:14 How Evals Change Product and Marketing Workflows 14:24 Why CEOs Will Soon Run Their Own AI Agents 𝗔𝗕𝗢𝗨𝗧 𝗧𝗛𝗘 𝗖𝗛𝗔𝗡𝗡𝗘𝗟 Welcome to Marketing School, one of the top business podcasts with over 61 million downloads. Hosted by Eric Siu and Neil Patel, recognized by Forbes as a Top 10 Marketer. 🎙️ Learn More About the Hosts Eric Siu - Leveling Up: https://www.youtube.com/@LevelingUpOfficial Neil Patel: https://www.youtube.com/@neilpatel 📩 Free Resources Ubersuggest: https://www.ubersuggest.com/ Answer The Public: https://www.answerthepublic.com/ ✅ Subscribe for Daily Marketing Insights!
Transcript 0:00 Speaker 00 Finding customer information shouldn't mean digging through emails, call notes, and spreadsheets. HubSpot's Starter Plan brings all of those details together, so it's easier to track what's happening with every deal and what to do next. It pulls information from calls and emails. Then it gets to work, logging and summarizing them. Right after your call ends, HubSpot pulls next steps from your call notes and drafts suggested follow-ups. And it stays up to date for you, so you can build demand, win more deals, and spend less time on repetitive work. Your records, insights, and next steps are ready for you to act on. And you don't have to miss a beat searching for any of it. Try the Starter Plan for $10 a month. Learn more at hubspot.com. The new way to do AEO and SEO is via API. And you know what's interesting? We had a call recently with a company that is literally, they do documentation for dev companies, okay? So let's say they're doing, just call it $10 million a year, okay?
0:45 Speaker 00 They raise a lot of money, let's just say $24 million to do this documentation, okay? And the interesting thing is, they've gotten to that, that level by having like a powered by button at the very bottom of the documentation they do for everyone, okay? So anyway, my point of saying this is that I think they use an API. So Clickflow, we have an API as well that people use to generate content with. But I think the way you're gonna do it is like, people are just gonna log into Claude Code Command Line Interface and just say, hey, can you just do this for me? Can you build this for me? They can plug it into their agents. The agents do the work, the agents could go out there and buy stuff. There's a ramp CLI now. So the agents marketing agents type of thing, so. I think most SEO companies now, or AEO companies, whatever you want to call them, the way to do it is via API, and we have the documentation for Clickflow. And we use, we use, actually use that documentation company for it.
1:29 Speaker 00 But I think it is using API because you need it to plug into your agents to get the work done faster. And if you aren't doing it that way, I feel like, you know, it's not required right now, but I do think it's, it gets work done quicker for you.
1:41 Speaker 01 Did you see that LinkedIn thread on how they know and how they mark that their content and the stuff that AI is creating? They know that it's AI-generated?
1:49 Speaker 00 You mean like Google?
1:50 Speaker 01 Yeah.
1:51 Speaker 00 I've seen, I've seen that.
1:51 Speaker 01 And I'm a big believer that they genuinely do know, you know, I was a believer even before I saw that. I was a believer that they do know that when content is AI-generated, because if you look at all the stats and the data, even though both Eric and I have our own AI content writing tools, it does not rank well. And you could get short-term traffic. But over time, we see AI content just not do as well. And it doesn't mean you shouldn't use AI to write content. It means you should use it in different parts of the content writing process.
2:20 Speaker 00 Yeah.
2:20 Speaker 01 It's great for ideation, it's great for structure, it's great for research. A lot of the writing, I highly recommend doing it manual. And I believe it's going to keep going that way because if AI doesn't care if the content's written by AI, and then they crawl it and then use it to create more content and keep learning, it becomes a self fulfilling prophecy and then it starts hallucinating and they don't want that.
2:43 Speaker 00 So what we do for ClickFlow, so what I've heard the customers say, so what I've heard from our people working with the customers, is that the content that they generate, sometimes they're just YOLO and let it publish. But similar to how like, like a Finn does it with customer service, right? For like, sometimes I have a hard time believing this and I've seen the content quality myself, I'm like, this is really good, right? But they say this is better than any other content generation tool that they've used. And that we have a human in the loop, right? So that's what helps. So when they say like, the content helps them get more citations and it ranks better, it does, but there's a lot of caveats. There's one, we have a human in the loop. Second, we spend a lot of time on iterating on it. And so that goes back to what you were just saying about Harvey, you can't just let it YOLO on its own.
3:22 Speaker 00 But I do think if we can get to a point where we have a lot of data around it and we can just release our own LLM down the road, like a vertical one, that's like down the road, but that's at least what we're seeing on the ClickFlow side.
3:32 Speaker 01 We've been seeing something interesting with content, so you know this. A lot of companies are publishing listicles on their own website. If you look at like ChatGPT and when they're doing the searches, I was with Sam from WriteSonic. A lot of the prompts now have site colon commands. So like ChatGPT will run a site colon command on your own website to see the articles you have, to see if it's relevant to what people are asking. And a lot of companies are creating articles like the 10 best web design shops, the 10 best, really well, like a year and a half. Project management shops and whatever CRMs and they're creating their own pages for the services or products they offer. And what we're seeing is even though it can help you with GEO and helping you do better on ChatGPT, within six or so months of creating a lot of those listicles, assuming you do it in quantity,
4:17 Speaker 01 we see it hurt your organic search traffic.
4:19 Speaker 00 Yeah.
4:20 Speaker 01 So it's just like you're, we think that's the wrong approach because you're trying to do better in one place at the cost of another channel. And we don't think in the long run, at least at my company NPD, that you should be publishing this content on your own website or even publishing the content because this kind of content is something where it'll help you with the algorithms now, but we don't think it'll help you with the algorithms in the long run because it's just easy to manipulate. Yeah. Real quick, if acquiring customers has been a struggle for you and you are trying to figure out how AEO works, Answer Engine Optimization or the new version of SEO, how paid works, how all this AI stuff is going to play into your customer acquisition strategy. And check out my ad agency, Single Grain. It's singlegrain.com. And if it looks like it's a good fit, we'll help you with a free marketing plan.
5:05 Speaker 01 So again, go to singlegrain.com and we'll see you on the other side.
5:07 Speaker 00 So businesses can sell directly via Meta and ChatGPT Ads. I think we're going to see a lot more of this. That's, we're not saying, by the way, get rid of your website. I think the website is still important because the more information you could give agents to crawl, the better. So documentation around your software, very important in my mind. Whatever other information you have, get it all out there. Do more podcast interviews, get your YouTube channel out there, you know, keep doing what you're doing because that stuff, they need stuff to ground on. It's not going away.
5:30 Speaker 01 This is a random note. This is a marketing related, but you want to know what my wife does to try to make us all healthy every single week?
5:36 Speaker 00 What? Make you wipe your butt with clean toilet paper and organic?
5:40 Speaker 01 No, but I don't know.
5:42 Speaker 00 I just bought a bunch of organic toilet.
5:43 Speaker 01 I don't know what my toilet paper is.
5:45 Speaker 00 Let me tell you what I bought. Go ahead.
5:46 Speaker 01 Okay. So what she does is she chops up different vegetables, like in different colors. Because you know, like radish is a different color than cabbage and all this kind of stuff. And she tries to get all the major vegetables in her body every single week, not every day, because it's too hard to get all of them in. And then that way we have all the different colors and vegetables because she saw some report on how different vegetables have different things for you and the colors are all natural. And she just chops sort of so tiny in food, like lasagna, you can't see it and you just eat it.
6:16 Speaker 00 That's good. That's smart. That's a lot of work. What does she use to chop and all?
6:20 Speaker 01 Her hand?
6:21 Speaker 00 So let me tell you this. So by the way, we'll move on. Don't worry, guys. We'll bring this back to marketing in a second. So I bought this, okay? Real three ply soft and strong bamboo toilet paper. And I just started using it yesterday, like 4.6 stars, 4,000 reviews. I saw this on Instagram and I bought it. Because someone was writing something about the toilet paper used. There's PFAS and all that. Because I was using like the Amazon one, the Charmin one. It's like, no, you gotta get, you gotta get this. If you don't want PFAS in your butt.
6:47 Speaker 01 What's PFAS?
6:47 Speaker 00 Forever chemicals.
6:48 Speaker 01 I didn't know that.
6:49 Speaker 00 Yeah, well, now you know. See? Better for your butt.
6:51 Speaker 01 Well, typically when we're at home, we don't have toilet paper.
6:54 Speaker 00 What, you use your hand?
6:56 Speaker 01 No. Our toilets are bidets.
6:59 Speaker 00 Dude, that's not enough!
7:00 Speaker 01 And then we have wet wipes.
7:01 Speaker 00 Okay, yes.
7:02 Speaker 01 But I don't know what the wet wipes are in. It's toilet paper, but...
7:05 Speaker 00 No, but the wet wipes are made from like bamboo too, or something like that. So it's like, mine's plant-based. The Dude wipes, they're expensive.
7:10 Speaker 01 We don't use Dude wipes, but we use another organic brand. I think it's organic for the wet wipes. I'm pretty sure everything in our house is organic, even though it doesn't mean much for food.
7:17 Speaker 00 Yeah.
7:18 Speaker 01 But we have bidets that clean you, and then you have the wet wipes on top of that.
7:23 Speaker 00 Dude, sometimes though, the bidets... We'll just leave it at that. I'm just gonna send you this so you can buy. Okay, guys, let's bring this back to why A/B testing is dyke. So back to marketing, guys.
7:32 Speaker 01 Yeah, we're getting close to the end, I think.
7:34 Speaker 00 Yeah, we are. We still have 14 minutes left. So A/B testing was the gold standard for product decisions for 15 years. The best AI companies abandoned it. So this is from Akash Gupta. "For a generation of PMs, experimentation meant designing a controlled test, allocating traffic, waiting two weeks for statistical significance, and hoping you had enough sample size to learn something. That loop trained an entire discipline to think in cycles of weeks. Eval's compressed the loop to minutes. Three components: a set of inputs your product needs to handle, number two, a task that generates outputs, three, and a scoring function that produces a number between 0 and 1. You run it on your laptop, no production traffic, no two-week wait, no data engineering pipeline. The math on what this changes is staggering. Teams running evals are doing 12.8 experiments per day. That's roughly 384 per month.
8:18 Speaker 00 A traditional A/B testing team runs maybe three. Over a quarter, one team has explored 1,150 variations, the other has explored not..." That learning gap compounds every single week. So the point of saying this, so he mentioned, uh, Ankur Goyal built the eval platform behind Versal, Replit, Ramp and Notion, eight hundred million dollar valuation. He ran an eval from scratch on this episode, went from a score of zero to point seven five in under twenty minutes. That's a PM shipping a measurable quality bar before writing a single line of product code. So let me, let me unpack this for people. So Andre Karpathy, super AI researcher, did, he released Auto Research. I think that these two kind of go hand in hand because Auto Research allows you to run compute experiments very quickly, right? AI experimentations. We're talking like every five minutes it's running one and he's just leaving his computer on and doing it.
9:03 Speaker 00 And he's found things that he couldn't, he's like, dude, over like twenty years of doing this, like he found a bunch of things that he didn't even know were possible. Okay. I think when it comes to evals, evals are just, it's just a simple way of saying, hey, what is defined as a successful test versus a non-successful test? That's all it is. Okay.
9:19 Speaker 01 Yes.
9:19 Speaker 00 And then you're just running it over and over, but we're going to see a lot more of this. And depending on what you're doing, if you're sending cold emails or if you're running paid traffic or whatever, you do need a good enough sample size when it comes to marketing. And you can't just, you know, say you're going to do three hundred eighty-four. So I think it depends on your data sample size. But now, when you set these agentic platforms up or your agents up, you're just going to say, hey, I want you to run these tests and I want you to, every week I want you to run evals and just so it can recursively improve over time. That's where this is all going. And frictionless growth marketing hosted by Sonya Thompson is brought to you by the HubSpot Podcast Network, the audio destination for business professionals. Frictionless growth marketing hosted by Sonya Thompson shows how modern brands grow by removing the friction in their marketing and customer experience
10:02 Speaker 00 that pushes customers away every single day, often without leaders even realizing it. Each episode delivers practical insights, real brand examples and conversations with marketing leaders, showing brands how to win with the new rules of growth, identity-driven marketing, cultural relevance, and frictionless customer experiences. Listen to frictionless growth marketing wherever you get your podcasts. That's where a lot of this stuff has already gone with AI, but not, it's kind of starting to encroach on marketing.
10:28 Speaker 01 So this way he's talking about, we've seen this happen for years. This is, it's new to a lot of people, but it's not new in the base. Area of the startup world. There's one issue with this model. It only works if you have high volume. If you don't have high volume, you can't do it. I think for most companies who are heavily venture funded startups or large enterprises, they can't leverage this because they don't have volume.
10:55 Speaker 00 Yep.
10:55 Speaker 01 And a prime example of this is if you're a new entrepreneur, you set up a D to C e-comm site, you're doing drop shipping, you're not even creating your own product, you're running Facebook ads and you're getting five sales a day and that's it. So 150 a month, you don't have enough conversion volume to run this playbook.
11:16 Speaker 00 I think what's gonna happen, so exactly what we're saying, but I think we're gonna be using a lot more synthetic data in the future.
11:23 Speaker 01 Well, i- i- if you think about, you know, MMM, MMM has been out for ages and it's what people have.
11:31 Speaker 00 What's MMM?
11:32 Speaker 01 Media mix modeling.
11:33 Speaker 00 Okay, yeah, yeah, yeah.
11:34 Speaker 01 Right?
11:34 Speaker 00 I thought you were talking about M and Ms.
11:36 Speaker 01 No.
11:36 Speaker 00 Yeah, M and M.
11:36 Speaker 01 When you think about.
11:37 Speaker 00 Yep.
11:38 Speaker 01 And some of the solutions that Google offers and a few of the others, right? Media D and the list goes on and on. When you think about it, you're taking data from what happened in the past to figure out what's most likely going to happen in the future. And it's saving a lot of time and money. And companies are also starting to look at not just, hey, where are we spending our money? And where should we? They're also looking at increment, incrementality in which was this person actually buying because of the ad? Or were they already on the check and I'm gonna buy anyways and the ad just took credit? And that's another big problem in marketing as well because we spend a lot of money on advertising,
12:18 Speaker 01 but yet was the sale actually caused by, you know, marketing or was it not? And dude, we see this all the time. And the way you see this is not what I'm talking about incrementality or, you know, causality. It's when you have the executive team just being like, what would happen if we turn off all our ads? Let's try that for a week, right? That's their way of saying, did the ad actually cause or does marketing actually cause? More sales for us or is it just taking credit for sales that we would have already gotten? And it isn't that simple because if you just turn off your advertising and your marketing for a week, a lot of what you're closing in that current week came from people who saw your brand and from advertising and marketing that happened,
13:03 Speaker 01 you know, months ago or a year ago. So it of course isn't that simple. But you know, when you think about incrementality testing and you think about MMM, because of AI, it's more easier for companies of all sizes to use this stuff, while it wasn't possible for smaller organizations in the past.
13:23 Speaker 00 Yeah, I think what people can learn from this, so what Neil is saying is true. This is not necessarily new on the marketing side, but I think on the PRD side, so a product requirements document, this guy is basically saying that evals are the new PRD. And so when you have a very specific spec-driven document and it shows like it defines what a win and what a loss is, I think this is going to be built in a lot more to whatever agents I'm building, for example, whatever workflows that I'm building. So it's just on top like, hey, I want you to have an eval here, right? That's just going to be built into whatever scaffolding that you have in the future. And that's going to allow us to do these things. Sometimes you might need synthetic data. Sometimes you can't do it, right? But I think we are, I think me, for example, I've been required to, we were doing PRDs maybe like a year, year and a half ago. And like me, PRD, right? Doesn't make sense.
14:06 Speaker 00 But now evals, when you just build it in, it's, it reduces a lot of noise. That's what I think is going to, a lot more people are going to start doing it that way.
14:13 Speaker 01 Yeah. But yeah, with all these changes that's happening with data, you know, not just synthetic data, because I know a lot of corporations are using synthetic data too. And when I'm talking about synthetic data, let's say you're having, you're doing, running Facebook ads and they want you to pass in data on your customers. Well, a lot of corporations these days are passing in synthetic data. So then that way, Facebook doesn't have their quote unquote customer data. Facebook hates it because when, when you don't pass in your real data, it makes their ecosystem not as strong and it doesn't help them for future, you know, companies who are running ads there. But because of technology,
14:52 Speaker 01 You're going to see two things. One, you're going to see easier for startups to get up and running and have the power that a lot of these enterprises had because things like MMM are much more affordable and easier to do. The second thing that you're going to see is it's also easier for companies to leverage things like synthetic data and protect their own first-party data and not pass as much through. So then you're going to have these platforms like Meta and Google not be as effective from a targeting perspective because they're not getting passed in all the data from corporations that they would like.
15:25 Speaker 00 All right. So this can be a nice final topic for us. So this is also from Akash. So every CEO will be running an open claw within 12 months. So Mark Zuckerberg, Meta CEO, is building his own in-house. So a couple of days ago, right, Jensen Huang stood in front of 30,000 people at GTC and said, every company needs an open claw strategy. So now you have Zuckerberg building a personal CEO agent and it's a Fortune 5 company, right? And so, you know, a CEO at Meta scale sees maybe 1% of information that determines whether a decision is right or wrong. The other 99% gets filtered through VPs, chiefs of staff, dashboards, and whatever made it into the pre-read. The agent replaces the filter. Think about what a CEO agent actually does. It ingests every product metric, every internal thread, every customer escalation, every competitive intelligence
16:09 Speaker 00 report across every team simultaneously. Then it surfaces the three things that actually mattered this morning. Before every one-on-one, it pulls that person's team metrics, open headcount, recent launches, and the two things they said they delivered last quarter. When the CEO asks, what happens to our glasses timeline if we move 200 engineers to AI infra? The agent gives a first pass answer in minutes instead of a two-week strategy team exercise, and it never forgets. The person who remembered why the company killed that project in 2019 left two years ago. The agent didn't. So Meta employees are already running their own versions, tools called My Claw and Second Brain. Engineering output is up 30%. Power user is up 80% year over year. Zuckerberg is doing what his employees are doing, applying it to the highest leverage seat in the company. So this is exactly what I'm doing internally right now. I have everything connected and it's proactive with
16:53 Speaker 00 telling me. It does forget. Guys, don't think this is perfect. Memory is an issue with AI right now, but I do think that if you're applying it at the highest level, like this, for example, or even like the way I'm doing it right now, it cuts the noise very quickly and I can execute a lot faster. And I think we all like to move faster.
17:11 Speaker 01 I agree with you this. I still have, I don't understand the concept why AI can't figure out the memory issue. It doesn't seem like a big deal. You tell it something once, and for some reason, it doesn't always follow. I, I
17:23 Speaker 00 Do you remember everything? You have really good memory.
17:25 Speaker 01 No, but this is a computer. Like, it should be able to like hard code and not forget unless you tell it.
17:31 Speaker 00 You know what a computer is made out of, right?
17:33 Speaker 01 Yeah.
17:33 Speaker 00 Sad. Like, we conjured it and then we put electricity. I think, I think it's gonna get better over time, and I think the memory problem should be solved, I don't know, maybe in the next year or two. But I do think just the decisions, like, man, it's so crisp now, I can move a lot faster. For example, last week, I, I, I made that CFO agent, and it's like, oh, you have $700,000 in cuts over here. I was like, wait, what?
17:54 Speaker 01 Yeah.
17:55 Speaker 00 Found money, man. Right? It's great. It's great. I love it. So, so when I, when I make a copy for you, you can use it for yourself. You can just duplicate it, and use, use it for yourself. You can use your own scaffolding on it. It's my gift to you.
18:07 Speaker 01 I was at an event a month ago, and I met someone who's working Anthropic, and they're in charge of coding up a lot of this stuff, and I asked him the memory question. And he gave me an explanation on why AI isn't perfect on recalling everything.
18:24 Speaker 00 Uh huh.
18:25 Speaker 01 And it was too complex. I didn't understand it. And I was like, when will it get solved? And, cause that's what I was really looking for, right? I'm like, I don't really care how you solve it.
18:35 Speaker 00 You don't care about the technical details.
18:36 Speaker 01 Just solve it for me.
18:37 Speaker 00 Yeah, yeah, yeah.
18:37 Speaker 01 And he's just like, he's like, it's not gonna be easy. We don't see it happening in the short run, but it'll get better over the next few years. That was his words. Not necessarily being solved in a few years, it'll get better and make the errors less often.
18:51 Speaker 00 That's what I'm really saying. Like, we expect this to compound over time. And as we talk about this too, like, this is the worst they'll ever be. So, I don't know. My, my eyes continue to burn to the ground every night, and I don't know about you.
19:03 Speaker 01 My eyes don't burn to the ground every night.
19:05 Speaker 00 But they do if you use them too much, right?
19:07 Speaker 01 No.
19:07 Speaker 00 They don't?
19:08 Speaker 01 No.
19:08 Speaker 00 You have laser, right?
19:09 Speaker 01 Yeah, I never had that issue.
19:10 Speaker 00 Yours do. Yours do. Okay, great. Noah's does. Alright, so Neil's preserving his eyes, Noah and I are being stupid. But yeah. Anyway, guys, that is it for today. Please don't forget to rate, review, subscribe, and we'll see you tomorrow.
Previous Something Is Happening To Marketing Teams That's Never Happened Before Next Nvidia’s Growth Just Reaccelerated, And That Should Scare You Page updated November 6, 2026 · transcript and summary are machine-generated
