(Preview) Experimenting With OpenAI’s Deep Research, Another ChatGPT Moment, Won’t Someone Think of the Entry-Level Employees?

Reactions to OpenAI's release of Deep Research, including the Deep Research contributions to Ben's Update on Tuesday, lessons from several other Deep Research experiments, and questions about the future of work, information flow, and a world in which days of work can be done in minutes.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Andrew Sharp?

  Hello and welcome to a free preview of Sharp Tech.

• 0:08Ben Thompson?

  And so my second prompt to deep research was give me an analysis of Apple's earnings. And by the way, I think these are the three most important points. Number one, just the services bit, it's worth calling out. Number two, the China problem, and it probably goes back further. And then number three, the Apple intelligence story seems overstated X, Y, Z. Analyze the earnings with sort of these points in mind. And I thought that analysis was really good. It still wasn't, you know, fortunately, I think as good as my analysis. The writing, it goes on too long. It's not really dense in information. I think one of Stratechery's strengths is a good Stratechery article is every sentence is sort of,

• 0:54Ben Thompson?

  I think, I think good writing just has its own, its own momentum. And every sentence should be pushing you forward and adding, layering on new information. And not, there shouldn't be sort of emptiness sort of in there. There's a quite a bit more emptiness sort of in, in I think these results. And I also don't think there was any novel insight. It was insightful, but that's because I infused the critical insight into it with the prompt. And even then, I think it's still got a couple of things I don't think were right. That were sort of based on more conventional wisdom. Or I told it to reference old Stratechery stuff and it would like over reference some sort of old things. That said, I thought it was good.

• 1:37Ben Thompson?

  Like it was like, if I were working in, say, you know, or I were a small scale investor or something, and I wanted an analysis along these lines. Like if I were to have sort of a research intern or a new employee, and I wanted them to generate a report, this is a solid report.

• 1:57Andrew Sharp?

  Right. If you're going into a meeting and want to be briefed on the issues and capable of holding a conversation about Apple's earnings and Apple's future in China, like this report does the job. In a pretty solid way, it's not stratechery analysis, but it was also.

• 2:14Ben Thompson?

  I'm very biased to say it's not stratechery analysis, but but but there's a few, there's a few takeaways here. And actually, your, your, let's come back to your example, like going into a meeting. Cause I think that's another one that I want to get to that I think was was really beneficial to me. The it's just the.

• 2:35Ben Thompson?

  And this is the weird thing like this takes me, it took me back a bit to the original chat GPT moment where.

• 2:44Ben Thompson?

  It was just so startling that it wrote in paragraphs and sort of like was well rounded. That it took a few days to sort of dive deeper and see like what it's actually like, is it actually making sense here? It's saying a lot of stuff sort of very confidently. There's an aspect of the quality of this output that is well formatted, is well organized. And is sort of cogent in a way that it it it can potentially play tricks on you. And I have another example of where it sort of really dropped the ball. Uh, but it matters, it's very convincing and like it and the sort of the way and I. To me, this is the reasoning aspect of all three really coming out. It's going back, it's recycling, it's going through the, you know, all its sources.

• 3:29Ben Thompson?

  But it's reorganizing itself to be very cogent and consumable in a way that's very convincing. And if you think about the concern about AI, you know, you could see how this could be a problem where it it just.

• 3:42Andrew Sharp?

  Well, hallucinations can be dangerous because it's also like very comprehensive and thorough. And so you get lulled into a false sense of security with the output. So I had to be careful.

• 3:54Ben Thompson?

  We have a heuristic in our minds that if it's well written and well organized. It's also probably all true. Yeah, there are citations and sort of all those, all those sorts of things. But, you know, I think, I think just a few takeaways on those two bits were. The quality difference between, so the, the overall writing and organization was pretty solid between both of them. The insight level between the two, I thought was pretty stark, but that insight came from the prompt. That is always been the case with L M's. The quality of your output is downstream from the prompt, but that seems dramatically expanded in this case where if you give it a really a, a clear direction, a clear thesis,

• 4:39Ben Thompson?

  and I think you put my line in here in the show notes, like the capability of deep research to fill in a thesis is really strong. It's capability to generate a thesis is still, you know, there really wasn't a thesis in the first, the first summary.

• 4:55Andrew Sharp?

  That's good. That's a relief. We don't, we don't want it generating theses going forward. That's our, our human's job, at least for another six to 12 months.

• 5:05Ben Thompson?

  But, but to your point, what does this mean sort of generally? Like, like for how, and this is a conversation that in some respects, I have more to say about it than anyone. I am basically a professional thesis generator in some respects. On the other hand, I'm heavily biased because no matter what I say or what I try to think, obviously my sort of livelihood is at stake here in a certain respect. So take whatever I say with a necessary grain of salt. But then number three, the market evidence is that I am a good thesis generator. That's why people subscribe to Stratechery.

• 5:49Ben Thompson?

  So by definition, a lot of people aren't because they're not making the profits that I'm making, right? Like I think, I feel my Midwestern sheepishness is coming out here, like, like, but it's very tangible.

• 6:03Andrew Sharp?

  I don't know where you're going with this.

• 6:04Ben Thompson?

  It's very tangible to this conversation when you think about what does this mean for work? Because.

• 6:12Ben Thompson?

  I still feel pretty secure. The, and I think to me, this only accentuates the big question about AI. And can it generate truly novel insights? Could deep research? How many versions of deep research? How many models are we going to need to get to? To have the insight? And for me, the insight was immediate. The moment I saw those China numbers, I knew I'm going to make a chart about Apple's revenue in China. That's going to show that the growth was very flat or declining. It shot up during Huawei and then it was flat and declining afterwards. Like, yeah, like now, and that, that is again, I'm the sort of hu- human L M in this regard.

• 6:57Ben Thompson?

  But there is some aspect of, I think this bit about Huawei and Apple's ships. Of course, people talk about it now, but that's because I raised it immediately. Like, like people were over the moon when Apple's results shot up. I'm like, no, this is because of, because of Huawei. And when are we going to get to the point? What's going to be necessary for that to be generated from the L M? As opposed to me infusing that into the L M? On the other hand, and this is where I come back to the market points. That's why I have a livelihood writing about this. Because that's what I do is, is think about those things. Again, I feel so sheepish saying this, but again, that's why I point to sort of the market results.

• 7:41Ben Thompson?

  Like, people clearly find my thinking about this valuable. But how many jobs and how many people who are writing reports? That's actually why they're writing reports. Are, are we looking for novel insights from the vast majority of junior analysts? And the vast majority of entry level lawyers or, or paralegals or whatever it might be? Or is there actually a huge aspect of the economy of white collar work? That is about going out, doing some research, collating some data, putting it in a digestible format for people above them, their executives or whatever it might be, to bring to bear their knowledge

• 8:26Ben Thompson?

  and sort of make a decision on it?

• 8:29Andrew Sharp?

  Yeah, no, exactly. Well, and I want to just ground people in what this product is because I would imagine most of our audience aren't subscribed to the like elite tier. I don't even know what the OpenAI tiers are, but the $200 are maxed here.

• 8:43Ben Thompson?

  ChatGPT Pro. Yeah, not plus. Plus is $20, Pro is $200 a month.

• 8:48Andrew Sharp?

  This is available to pro users right now.

• 8:53Andrew Sharp?

  You can read sample output if you're subscribed to Stratechery. It doesn't cost $200 a month to check it out. But it's you input a prompt and to your point, like the tool becomes more useful, the more detail and structure you provide. And so in that respect, deep research is like having a real employee. Like if you give an employee an assignment and it's vague and ambiguous, you're gonna get back mediocre work. And if you give an employee a very clear assignment with what you're looking for, chances are you're gonna get back better work. And that's the way deep research is operating.

• 9:30Ben Thompson?

  And this is agentic in that it's going out and surfing the web, basically. That's what it's doing. And it's finding, finding stuff.

• 9:36Andrew Sharp?

  One of the things that's remarkable to me is it's really effective at searching the web. I don't know how they distinguish between sources, but like I was recording Sharp China today, so I asked Deep Research to generate a report about the recent history of China's activity in Panama and why that has been setting off alarm bells with US leaders. And number one, the work product was a nice little overview of the shifts that have taken place since 2017, when Panama severed ties with Taiwan and established diplomatic relations with China. But it also, it took me to an in-depth article that provided even more backstory

• 10:16Andrew Sharp?

  in terms of the security concerns the US has and some of the controversy the PRC has created locally in Panama. And I honestly don't think I would have found that article on my own, in part because it was just sort of an arcane source that was based in Latin America. And most of what I learned did not make it into the show. Spoiler alert for anybody. Who hasn't listened to Sharp China, but it allowed me to relay various bullet points from that follow-up article that provided context for people who have no idea why the Panama Canal is now a source of controversy in the US-China relationship. And it also took what would have been like an hour and a half of research and reading

• 10:59Andrew Sharp?

  about the history and condensed it to about 20 minutes. And I didn't have to go like wading through the Google morass to find a couple really good articles and deep research did it for me. That's the sort of thing that could be useful to literally anyone in any industry. And it's already doing it at a really high level. And again, it provides nice little links and footnotes for all its findings, which makes the follow-up sort of a natural next step as you consume any of these results.

• 11:32Ben Thompson?

  Yep. No, no, the question though, you mentioned the Google morass. This is one of the real challenges facing deep research.

• 11:40Andrew Sharp?

  All right. And that is the end of the free preview. If you'd like to hear more from Ben and I, there are links to subscribe in the show notes, or you can also go to sharptech.fm. Either option will get you access to a personalized feed that has all the shows we do every week, plus lots more great content from Strategery and the Strategery Plus bundle. Check it out, and if you've got feedback, please email us at [email protected].