The LM Brief: Integration gaps and budget missteps stall enterprise AI

This week’s World of DaaS LM Brief looks at new research from MIT, which finds that 95% of generative AI pilot programs inside companies are failing to deliver meaningful financial impact. The issue is not model quality but enterprise integration and a steep organizational learning gap. The report shows ROI is strongest in back-office automation even as most budgets flow to sales and marketing, and that success comes from buying specialized solutions and forming strategic partnerships rather ...

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Auren Hoffman?

  Hello, fellow data nerds. Welcome to the LM Brief, a new series from World of DAS. I'm your host, Oren Hoffman, CEO of Incubate and GP of Flex Capital. In the LM Brief, we'll bring you quick, high-value summaries of some of the most interesting DAS-themed articles, podcasts, and ideas, which is powered by Google's Notebook LM. Discover more episodes, get weekly data-as-a-service news, original content, and articles on data and more at worldofdas.com. That's worldofdas.com. Talk to you soon.

• 0:35Speaker 01

  Generative AI, it's absolutely everywhere, right? Dominating the news, boardroom discussions, there's just this massive buzz. Companies are pouring money in, making huge promises about transformation. But are those promises actually delivering real results? Okay, let's unpack this because the research we're diving into today really helps cut through that noise. We're looking at brand-new work from MIT's NANDA Initiative. That's their network for AI development and adoption. The report's called "The Gen AI Divide: State of AI in Business 2025." And our mission today, really, is to figure out why, despite all the hype, all the investment, why are most company Gen AI projects kind of, well, stuck at the starting line? What's that mean for business, for work? This deep dive should give you a really clear shortcut to understanding what's actually happening.

• 1:20Speaker 02

  It really is a surprising headline finding, isn't it? I mean, the promise of Gen AI for businesses is huge, undeniable. But yeah, this report shows that most projects aiming for that quick revenue boost, they're falling flat. And we're going to dig into some insights here that go way beyond the usual suspects, the common assumption.

• 1:38Speaker 01

  Okay, let's jump right into that headline finding, then, because it is, frankly, a bit of a bombshell. The MIT report says about 95% 95% of generative AI pilot programs in companies are basically failing, failing to speed up revenue, failing to show a real measurable impact on the P and L, the bottom line. That's a huge Gen AI divide, as they call it. It paints a picture of a near-universal struggle.

• 2:03Speaker 02

  It does. And it's important to know, this isn't just like a few stories they heard. The research is pretty rigorous. They did 150 interviews with leaders, surveyed 350 employees. Oh, and they analyzed 300 public AI deployments, too. So that gives a really comprehensive look at what's working versus what's just stuck.

• 2:20Speaker 01

  Wow, okay. So if 95% are struggling, what are companies saying is the problem? Is it the usual things, like... Regulations being too tough, or maybe the AI models themselves just aren't quite there yet.

• 2:33Speaker 02

  Yeah, those are definitely the things you hear a lot, but Aditya Talapalli, he's the lead author on this MIT report. He pushes back on that directly. He says the core issue is not the quality of the AI models. They're actually quite powerful. Instead, it's what he calls a learning gap, both for the tools and for the organizations using them. He points right at flawed enterprise integration.

• 2:54Speaker 01

  Flawed integration, so how does that play out?

• 2:56Speaker 02

  Well, think about it. Generic tools, maybe like ChatGPT, they're great for individuals because they're so flexible. But they often stall out in the company setting because they don't learn from or adapt to the specific, often complex workflows already in place.

• 3:10Speaker 01

  Ah.

• 3:11Speaker 02

  So it's less about the AI's raw intelligence and more about its, let's call it, social skills within company system. Can it actually fit in and learn the ropes?

• 3:20Speaker 01

  That's a great distinction. Individual flexibility versus enterprise integration and that learning gap. Okay. Which actually leads nicely to another maybe surprising finding you mentioned. It's about where companies are putting their AI money. The data shows a real mismatch, doesn't it? More than half the gen AI budget is going towards sales and marketing tools.

• 3:40Speaker 02

  Right. And this is where the report really kind of flips the script.

• 3:43Speaker 01

  Yeah.

• 3:44Speaker 02

  The biggest return on investment, the highest ROI. It's actually found in, let's be honest, the less flashy area. Back office automation.

• 3:53Speaker 01

  Back office, so what does that mean exactly?

• 3:55Speaker 02

  It means things like eliminating the need for business process outsourcing, cutting costs from external agencies, just generally streamlining internal operations. It's almost like, you know, everyone's rushing to deck out the front of the shop, make it look good. But the real money is being saved or made by fixing the plumbing and the wiring in the back.

• 4:14Speaker 01

  That's fascinating.

• 4:16Speaker 02

  So it really makes you wonder why the focus on sales and marketing tools, if the big, quick wins seem to be elsewhere. What does that imply for strategy going forward?

• 4:26Speaker 01

  Yeah, why do you think that is? Is it just because sales and marketing seem more directly linked to bringing money in? Easier to sell internally, perhaps?

• 4:34Speaker 02

  It's probably a mix of things. Sales and marketing applications that are often more visible, maybe easier to grasp conceptually. And yeah, maybe they seem like the most direct path to revenue growth, at least on the surface.

• 4:45Speaker 01

  Right.

• 4:46Speaker 02

  But this report clearly shows the deeper, often more immediate benefits, cost savings, efficiency are hiding in the operational backend.

• 4:54Speaker 01

  Okay, so the picture so far is, well, a bit grim. Lots of struggle, maybe misplaced focus. But what about the exceptions? That small 5% who are succeeding. This is where it gets really interesting, right? What makes that 5% different? What are they getting right?

• 5:08Speaker 02

  Yeah, definitely. And Chalapalli, the lead author, shares some clear insights on that. They saw, for instance, some really young startups, leaders, maybe only 19 or 20 years old, achieving incredible growth. Like, $0 to $20 million in revenue in just a year.

• 5:23Speaker 01

  Wow.

• 5:23Speaker 02

  Their formula seems pretty simple, deceptively so. They pick one specific pain point, just one. Then they execute really well on solving it. And crucially, they partner smartly with the companies that end up using their tools. Focus, execution, partnership.

• 5:40Speaker 01

  That's a powerful lesson in focus, for sure. But, okay, for our listeners, maybe in bigger, more established companies, how does that startup approach apply? Can large organizations adopt those same principles?

• 5:51Speaker 02

  Absolutely. And the data backs this up with some crucial insights on how companies adopt AI, regardless of size. The report found that buying AI tools from specialized vendors or building strong partnerships, that approach succeeds about 2/3 of the time, like 67%.

• 6:08Speaker 01

  Okay.

• 6:08Speaker 02

  Whereas trying to build everything internally from scratch, that only worked about 1/3 of the time.

• 6:13Speaker 01

  Only a third? That's a big difference.

• 6:14Speaker 02

  It really is. And it's super relevant for sectors like financial services or anywhere highly regulated. A lot of those firms right now, in 2025, are trying to build their own proprietary gin AI systems.

• 6:25Speaker 01

  Yeah, you hear about that a lot, control, security.

• 6:28Speaker 02

  Exactly. But this MOT research suggests that very approach, the internal build, leads to more failures. Interestingly, the report noted companies were often reluctant to share their failure rates on internal projects.

• 6:43Speaker 01

  Hmm, not surprising, I guess.

• 6:44Speaker 02

  Right. But the data leaned towards purchased or partnered solutions being more reliable. It suggests it's often smarter to buy or partner for the specialized stuff rather than trying to reinvent that wheel internally.

• 6:57Speaker 01

  So buy or partner overbuild generally. What else did this successful 5% do?

• 7:02Speaker 02

  A couple of the key things stood out. Empowering line manager, the people actually running the teams and dealing with the day to day to drive adoption, not just leaving it to some central AI ladder innovation hub.

• 7:13Speaker 01

  Get it closer to the ground.

• 7:14Speaker 02

  Exactly. And also selecting tools that can integrate deeply and actually adapt over time. Which brings us back to that learning gap idea we talked about earlier. The tool needs to evolve with the business.

• 7:24Speaker 01

  Right, it can't just be static. Okay, so that covers the pilots and what leads to success there. What about the bigger picture, like the impact on the workforce? What does the report suggest about jobs?

• 7:33Speaker 02

  Well, workforce disruption is definitely happening. No question. Particularly in areas like customer support, administrative roles.

• 7:40Speaker 01

  So layoffs.

• 7:41Speaker 02

  Not necessarily mass layoffs, interestingly. What they're seeing more is companies choosing not to backfill positions when someone leaves.

• 7:49Speaker 01

  Uh, attrition.

• 7:51Speaker 02

  Exactly. It's a quieter shift. And it seems concentrated in those jobs that were often outsourced before. Maybe seen as lower value add internally.

• 7:58Speaker 01

  Okay. And what about other challenges? You hear a lot about employees using tools like ChatGPT on the sly. Did the report touch on that, shadow AI?

• 8:07Speaker 02

  It did, yeah. That's highlighted as a real challenge. The widespread use of unsanctioned tools. Employees using things like ChatGPT without official approval or oversight.

• 8:16Speaker 01

  Which creates risks, obviously.

• 8:18Speaker 02

  Huge risks. Data security, compliance, you name it. And tied to that is the ongoing difficulty companies have in accurately measuring AI's true impact. It's hard to pin down the exact effect on productivity, on profit, especially with unsanctioned use clouding the picture.

• 8:34Speaker 01

  Makes sense. Anything about the future, what's next?

• 8:36Speaker 02

  Well, the report mentions that the really advanced organizations, the leading edge, are already experimenting with something called agentic AI systems.

• 8:44Speaker 01

  Agentic AI, what's that?

• 8:46Speaker 02

  Think of AI that doesn't just respond to prompts, but can actually learn, remember information over time, and then act independently based on goals, with inset boundaries, of course.

• 8:56Speaker 01

  Wow, like a digital team member, almost?

• 8:59Speaker 02

  Sort of, yeah. It gives us a glimpse into how the next phase of AI in the enterprise might look much more autonomous, potentially much more powerful.

• 9:07Speaker 01

  Okay, so pulling this all together, if you're listening and trying to make sense of the gen AI hype, the big takeaway from this deep dive seems to be it's not really about how powerful the AI model is in isolation. Success hinges on how companies integrate it, how they actually use it day to day. And the path seems to involve really focused problem solving, smart partnerships, and maybe looking beyond the obvious places like sales and marketing, looking towards that back office automation for some potentially bigger, quicker wins. Precisely, and I think understanding this, digging into the details like we have, helps you see past the breathless headlines. You start to grasp the real world complexities, but also the genuine opportunities of adopting AI in a business context.

• 9:51Speaker 01

  It reinforces that idea, you know, that knowledge isn't just about knowing something, it's about understanding it well enough to apply it.

• 9:57Speaker 02

  Yeah, absolutely. So as we wrap up, maybe here's a final thought to leave you with. Given this finding that a staggering 95% of current gen AI pilots are falling short of that rapid revenue acceleration goal, what does that suggest about how companies, and maybe even us as individuals interacting with these systems, might need to redefine what success even looks like for AI in business? Especially as these more advanced agetic systems start to emerge. Something to think about, keep exploring, and thanks for diving deep with us today.

• 10:29Auren Hoffman?

  Thanks for listening. If you haven't already done so, please subscribe to World of DAS wherever you consume your podcasts, YouTube, Apple, Spotify, and more. And please help us get discovered by leaving a review. And check out worldofdas.com. That's worldofdas, D-A-A-S, dot com. And of course, connect with me on Twitter, at Auren, that's A-U-R-E-N. Would absolutely love to hear from you.