Sora Disappoints, ChatGPT Pro Tested, Inference Time Reasoning & More with Sunny Madra | E2062

Timestamps: (0:00) Jason and Sunny kick off the show (1:26) Discussing Groq's recent developments and Middle East tech inspiration (3:51) Global business challenges and Jason's 2025 announcements (7:53) Gemini app demo and ChatGPT 4 comparison (9:57) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist (11:24) Evaluating ChatGPT 4.0 Pro limitations and robo-taxi fleet costs (12:46) Differences between ChatGPT models and future improvements (19:57) OpenPhone - Get 20% off your first six months at https://www.openphone.com/twist⁠ (20:58) LLM focus shift and robo taxi cost model experimentation (23:01) Importance of prompt engineering in AI tools adoption (25:31) Workplace AI adoption challenges and generational tech differences (27:22) New productivity hacks and Gemini app's growth (30:27) Zendesk - Get six months free at https://www.zendesk.com/twist (34:06) Startup support programs and Google Gemini's research capabilities (37:08) AI model performance evaluation and simplification (45:02) Meta's Llama 3.370b launch and AI industry impact (46:37) Infrastructure costs, competitive landscape, and AI-generated content evolution (55:44) Trust and bias in language models, news analysis startup ideas * Subscribe to the TWiST500 newsletter: https://ticker.thisweekinstartups.com Check out the TWIST500: https://www.twist500.com Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp * Mentioned on the show: Check out Sora here: https://sora.com/ Check out Kling here: https://klingai.com/ Check out Groq: https://groq.com/ * Follow Sunny: X: https://x.com/sundeep LinkedIn: https://www.linkedin.com/in/sundeepm * Follow Jason: X: https://twitter.com/Jason LinkedIn: https://www.linkedin.com/in/jasoncalacanis * Thank you to our partners: (9:57) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist (19:57) OpenPhone - Get 20% off your first six months at https://www.openphone.com/twist⁠ (30:27) Zendesk - Get six months free at https://www.zendesk.com/twist * Great TWIST interviews: Will Guidara, Eoghan McCabe, Steve Huffman, Brian Chesky, Bob Moesta, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland * Check out Jason's suite of newsletters: https://substack.com/@calacanis * Follow TWiST: Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin Instagram: https://www.instagram.com/thisweekinstartups TikTok: https://www.tiktok.com/@thisweekinstartups Substack: https://twistartups.substack.com * Subscribe to the Founder University Podcast: https://www.youtube.com/@founderuniversity1916

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Jason Calacanis?

  I think OpenAI is going to lose their lawsuit. I'm saying it right now. I'm predicting it here. I think it's going to be an injunction against OpenAI and they're going to have to settle for billions. You heard it right. I think it will be the largest copyright infringement case in history. I think it will be a billion dollar settlement with the New York Times and other people are going to join it. If you are a content creator and you feel your stuff is stolen.

• 0:21Speaker 01

  Billion dollar, okay.

• 0:22Jason Calacanis?

  I think it's going to be a three comma settlement. I honestly do, or judgment.

• 0:26Speaker 01

  Trace, trace commas, oh.

• 0:29Speaker 02

  This Week in Startups is brought to you by lemon.io. Hire pre-vetted remote developers. Get 15% off your first four weeks of developer time at lemon.io/twist. Openphone. Free business phone numbers for you and your team that work through an app on your smartphone or desktop. Twist listeners can get an extra 20% off any plan for your first six months at openphone.com/twist. And Zendesk. The best customer experiences are built with Zendesk. Qualifying startups can join their startup program and get Zendesk products free for six months. Visit zendesk.com/twist today to get started.

• 1:07Jason Calacanis?

  All right, everybody. Welcome back to This Week in Startups. I am so excited because the prodigal son, my boy, my bestie, Sandeep Madra is back. You know, we were on such a tear every week. You were coming in, you were doing all these great AI demos, we were placing bets. It was completely degenerate and we loved it. And then you got a little busy. Grok has been surging. You raised a bunch of money. We saw that in the press. Yum, yum. I got a little tasty poo of your company, Definitive Intelligence, which was of course bought by Grok. Grok, of course, is the inference chips that Chamath invested in, I guess, eight, nine, 10 years ago and funded. They bought your company and here we are. Give us the latest on Grok and then let's get right to our demos.

• 1:47Speaker 01

  Yeah, no, I mean, look, apologies. I missed doing it with you, but you know, my, one of my resolutions for next year is make sure we're doing it every week. But it's been, it's been busy for us, you know, post financing. We've been doing a lot of deals, you know, spent a lot of time in the Middle East. Met some of your friends out there as well, J Cal.

• 2:03Jason Calacanis?

  Oh, great. Yes.

• 2:04Speaker 01

  Really inspired by what's happening out there, to be honest?

• 2:07Jason Calacanis?

  Why are you inspired? What, explain to people what's so inspiring. Inspiring about what's happening in Saudi, UAE, you know, and Kuwait, Doha, Bahrain, Oman, you know, there's just like a lot, Israel, a lot of going on there.

• 2:20Speaker 01

  Everywhere. Well, I, you know, if I could distill it down to kind of three things, which I find really inspiring, let's just start with a lot of young folks, right?

• 2:29Jason Calacanis?

  Yes.

• 2:29Speaker 01

  You know, they, they have a lot of young folks and that makes for, you know, a population that has a lot of energy, right? You probably even see that being in Austin, right? And so.

• 2:39Jason Calacanis?

  Big time in Austin. That's one of the things I love about it. Yeah. Get that young energy. That's number one. Got it?

• 2:43Speaker 01

  Two, you know, they have a kind of a core business. Let's call that the oil business. And there, they, they can use that to basically elevate themselves into the next industry for, for, for the country, right?

• 2:56Jason Calacanis?

  Any industry, they could participate in any industry. When you have that kind of a chip stack, you can sit at any poker table. You're invited to every game.

• 3:04Speaker 01

  And look, they've, they've made a distinct choice to, to make a big bet at the, you know, AI poker table, right? And they're doing that across the region. And then, you know, strategically, the area is central to about, you know, within a thousand kilometers, there's four billion people, right? Within 600 miles, right? There's four billion people.

• 3:25Jason Calacanis?

  Cause you have India is a hopscotch. You got all of Africa. You got all of Europe. All going into the Dubai airport or the Saudi Riyadh airport. Yeah.

• 3:34Speaker 01

  Exactly. And, you know, you've spent time out there. There's a, there's a really good energy, right? They don't deal with a lot of the stuff that, you know, we've had here, which is changing now. Good to our bunch of our friends.

• 3:45Jason Calacanis?

  You're referring to woke nonsense, regulation and an outright hatred of capitalism.

• 3:51Speaker 01

  Yes. And, you know, they, they want to win and they're making it happen. And, yeah, and they love America, which is also great too. I, I, I think it's, it's incredible.

• 3:59Jason Calacanis?

  Well said, you know, it's, if, it's, it's kind of the equivalent of if the United States was sitting there and you had. But one rival, China, right? In China, you really can't participate in that poker game. You're invited, we were invited for 20 years to that poker game.

• 4:17Speaker 01

  Yeah.

• 4:17Jason Calacanis?

  And I'm like, you know what? Can't play poker anymore. And it's like, well, whose fault is that? Why did the game break? It's like, I don't know. And whoever the host was broke the game, the game broke. Somebody stiffed the game. I don't know why the game broke. But we can't play with the Chinese anymore. Our government is stopping it. Their government is stopping it. And you can't trust the game in China. If they're going to rug pull you and take every education startup, which Xi Jinping did, and say, you know what? Education is owned by the state. All your investments go to zero. That's kind of like, you know, authoritarian behavior that makes people not trust investing in a region. Everything we do, based on some level of trust. Now, Europe is in decline. You very rarely see a company break out there. The Nordics, Berlin, sometimes London.

• 5:02Jason Calacanis?

  You do have exceptions, and I know people are trying over there. But let's face it, it's a giant retirement community, and that's why we all love going there. Period, full stop. It's like Emco Center.

• 5:10Speaker 01

  Great summer vibes.

• 5:11Jason Calacanis?

  So you, great summer vibes, love it. But doing business there is hard. So then all of a sudden, this region emerges. And, you know, it's really charming to see a group of people who are like, hey, the way you've done things in your democracy in the West has gotten the best results. Objectively, with the King Abdullah scholarships and all these great scholarships they gave people in the 80s, 90s, 2000s, I understand. They just sent all their kids to America and to Europe to get educated. They came back, now you got all these 30, 40, 50 year olds, Gen Xers, millennials, Gen Z, who have crazy degrees, perfect English. You would think they grew up in Jersey or Boston or something by their accents because

• 5:53Jason Calacanis?

  they've spent as much time in America and in Boston, going to Harvard, MIT, and, you know, NYU, whatever, than they did there. This is all like an incredible setup to, they want to do business with us. That is the right side of history, which is another reason to love doing business there. I see it, and you going there, me going there, Brad Gerson going there, all of us participating there is exciting because they want to build. Okay, great. But I also think, if we think about the larger planet. It would be very nice for India, the Middle East region, which is obviously it's a lot of different cultures.

• 6:25Speaker 01

  Yep.

• 6:25Jason Calacanis?

  And the West.

• 6:26Speaker 01

  Africa.

• 6:26Jason Calacanis?

  Europe.

• 6:27Speaker 01

  Africa too.

• 6:28Jason Calacanis?

  Yeah. And Africa too. But, you know, Africa is a frontier market, uh, which is it's emerging and, you know, there's various levels of stability and investment. So a great market. But it's just interesting that the people who are writing the checks, go build are aligned. And they're aligned against Russia, China, you know, and maybe authoritarian countries. So it's a really beautiful thing that things happening. It's easy to criticize it. There's a lot of issues, but we'll leave those aside for now. Uh, and I'm just excited that you're spending time there as well.

• 6:56Speaker 01

  Yeah, we got to get out there together. And so, uh, I had dinner with your friend Omar once as well. It was great. We'll do it. Make sure we're 2025.

• 7:03Jason Calacanis?

  I'm going to announce something in 2025 in the region. Um, yeah. So I'm going to be there.

• 7:09Speaker 01

  Okay.

• 7:10Jason Calacanis?

  Yeah, it's going to be exciting for the audience of the show, but in 2025, there'll be an announcement, two different announcements, two different regions, two different projects. And so it'll be quite nice.

• 7:20Speaker 01

  Count me in.

• 7:21Jason Calacanis?

  I'll be there twice a year.

• 7:22Speaker 01

  I'll be a little advisor on there because I'll be here quite a bit.

• 7:25Jason Calacanis?

  I, you know, the reason I want to do it too, is I want to expose my family to it, my daughters. I want them to see what's going on in this region because if you're not in America, if you're coming from another country, people are deciding, do I want to be in Riyadh, New York, the Valley, Austin, Miami, Doha.

• 7:39Speaker 01

  Yeah.

• 7:40Jason Calacanis?

  You know, Dubai, Abu Dhabi.

• 7:42Speaker 01

  Yeah.

• 7:43Jason Calacanis?

  This is the destination for smart people in the world. So it's super exciting. Okay. Let's get to demos. You haven't been here in a while. There's been stuff dropping on my head like you wouldn't believe. And I am so impressed with Gemini. I have the Gemini app on my phone. I'm just going to put it out right now. Have you been playing with the Gemini 1.5 and the deep research?

• 8:03Speaker 01

  I mean, I, I, I use it all the time. It's kind of part of my workflow.

• 8:08Jason Calacanis?

  It is at parity.

• 8:12Jason Calacanis?

  With ChatGPT 4. In my experience as a user, I'm finding it has access to some data, images, flights, other things that I'm starting to see get pulled in from the Google Suite of Services. What's your general, since we haven't talked since 1.5 and the 2.0 and all this notebook, our deep research has come out, just generally, and do we have any of those demos lined up?

• 8:34Speaker 01

  Yeah, we do. So we were actually going to do something and maybe we can expand it. So, you know, I don't know if you've done 4O Pro yet, but I have 4O Pro.

• 8:43Jason Calacanis?

  That's a $200 a month product from OpenAI.

• 8:45Speaker 01

  Exactly.

• 8:46Jason Calacanis?

  I haven't ordered it yet. Here's the reason I haven't ordered it. They don't allow you to upgrade individuals in your enterprise plan to it. I gotta like, it's a mess. Like just, hey, Sam, somebody clip this and send it to Sam Altman and see him. Just like what I'm in, I have a paid enterprise account, I have a personal account. I want to pay you 200 bucks a month and there's no upgrade button. So what's the deal?

• 9:05Speaker 01

  Yeah. Well, there's an upgrade from your individual account, but I don't know how to do it across the enterprise account. That's the thing. So I had kind of a similar issue. So I had to do it on my personal account. So one of the things I was going to say, Jay Calz, and we can maybe do like a multifaceted test here. Why don't you pull up your 4O? I'll pull up 4O Pro. And you have a very standard prompt that you come up with.

• 9:25Jason Calacanis?

  Okay.

• 9:26Speaker 01

  You know, the one when you are doing like the kind of the Uber analysis? So let's both put them in at the same time.

• 9:33Jason Calacanis?

  Okay.

• 9:33Speaker 01

  And let's compare the results. How about we do that as like kind of our first?

• 9:37Jason Calacanis?

  That would be an interesting one. I'd have to go find it. Let's do, okay, okay, here we go.

• 9:42Speaker 01

  Okay. Send that to me in the chat and then I'll drop it in Pro and then we'll do a comparison between what Pro to $200 a month, which is meant to be your PhD level versus your regular, maybe undergraduate level.

• 9:58Jason Calacanis?

  All right, founders, are you tired of doing all your own software development? Do you need help, but you can't afford all this time it takes to find great talent? Are you dreading the endless interviews and email chains just to find somebody great? It takes six months. It takes a year. Well, what you need is lemon.io. Lemon.io has Thousands. That's right, thousands of on-demand developers who can help you and they've done the work already to vet these developers, making sure that they're results-oriented and that they're super experienced. Of course, they gotta have competitive rates, so they're gonna take care of that as well. And great developers are so hard to find and integrate into your team,

• 10:40Jason Calacanis?

  unless you're using lemon.io, because they handle all that for you. They only offer handpicked developers with three years of experience at a minimum, and they have to be in the top 1% of applicants, right? Something goes wrong, don't sweat it. Lemon.io will find you a replacement developer ASAP. So many of our launch founders have worked with lemon.io and they've had great experiences. So here's your call to action. Go to lemon.io/twist and find your perfect developer or the perfect tech team in 48 hours or less. That's right. And twist listeners get 15% off the first four weeks. Stop burning money.

• 11:22Jason Calacanis?

  Hire developers smarter and faster at lemon.io/twist. So here we go. I am opening up my chat GPT window.

• 11:31Speaker 01

  Yeah, I'm using O1 Pro.

• 11:33Jason Calacanis?

  You're gonna use O1.

• 11:35Speaker 01

  Yeah.

• 11:36Jason Calacanis?

  Non-O1. Okay, not Pro. Yeah, cuz I don't have it. Here we go. All right, I just did it.

• 11:41Speaker 01

  And you can see mine, it's going through this, you know, thinking, assessing, you're seeing I have the screenshot up here, right? It's kind of going through.

• 11:49Jason Calacanis?

  I see that.

• 11:50Speaker 01

  On the right side, you're seeing how it's basically working through this problem.

• 11:54Jason Calacanis?

  Right. It says it's gathering data. I'm working through a hypothesis on US daily car trips pegged at 1.1 billion. Seems like a logical starting point. Gives you a little color commentary. We're finding trip estimates. I'm pulling data from the NHTS 2017, estimating 940 million trips daily, assessing technologies influence, calculating robot taxi fleets, et cetera, et cetera, et cetera. So you see it's doing some logic. It took my prompt and just to give people an idea of what the prompt was, was build a detailed model estimating the cost of creating a robot taxi fleet for all car trips in the USA, including Waymo Cruise as a case study the model should account for, total car trips in the USA, Uber and Lyft trips. Fleet efficiency, fleet operations, required fleet size, public transit, fleet costs, induced

• 12:37Jason Calacanis?

  demand, and the output should be provide a comprehensive model with data-backed up assumptions and link sources that estimates the total cost of building a robot taxi fleet in the entire US, yada, yada, yada. So this is kind of like a crazy, insane

• 12:48Speaker 01

  Yep.

• 12:49Jason Calacanis?

  thing that I'm asking you to do, right?

• 12:51Speaker 01

  And is yours done? So mine is still working on it, right?

• 12:54Jason Calacanis?

  So Yeah, mine has been done for a while. Yeah.

• 12:57Speaker 01

  And then

• 12:57Jason Calacanis?

  Yeah. Yeah, it just did a very quick one here and we'll see.

• 13:00Speaker 01

  Yeah.

• 13:00Jason Calacanis?

  Uh, it took 22 seconds to do mine. I can go ahead and share my screen and I'll show you.

• 13:05Speaker 01

  I'll start my share. Yep. And because mine is still working on it. And this is where we're going to assess the difference between having a $200 a month.

• 13:13Jason Calacanis?

  Yeah. Twenty. So here it is. You see mine and the output was below is a modeled estimate for scaling a robot taxi fleet, cover all US trips, plus 20% of public transportation trips with a 20% induced demand factor. What induced demand factor, if people don't know, is more people will take more rides, right? So people will take 20% more rides because they're cheap and they're available. Key data sources, total US daily trips in 2017, Uber Lyft trips, company filings, SEC filings, it tells me public transit ridership, American Public Transportation Association, robotax, the operation fleet assumptions. So US car trips, 0.95 billion a day, US car trips, 350 billion a year, which makes sense, a billion a day, uh, US

• 13:57Jason Calacanis?

  and Lyft trips, 4.5 billion a year, just over 1% of the total trips. Robotaxi trips per vehicle per day, I put it at 20 to 30 trips a day, assuming 25 a day. Maybe it's more. I don't know if Tesla can do more with their fast charging and fast turnaround cleanup. I assumed five days off for each car for maintenance of 360 days on the road per car. Pretty aggressive, I think. Uh, trips recorded. Yeah. You see, there's all my calculations. And it says here, uh, with an induced fleet size, you would, uh, have around 422 billion rides. With induced demand, you would need 46.9 billion vehicles. Now

• 14:39Jason Calacanis?

  that doesn't count peak. So, you know, this is, this estimate probably needs to be doubled to demand, to handle peak demand, right? Uh, like people leaving me Warriors game or something. Anyway, you're gonna need, uh, You need $5 trillion at 100,000 a vehicle. You need $1.5 trillion at $30,000 a vehicle to replace the entire fleet in the US. This is not to just do a ride sharing. This is to replace all rides in the US. Nobody has a car. What did you get?

• 15:08Speaker 01

  And you know, this is a good test, right? You know, you've got these projects. Yours did it in a few seconds, cost 20 bucks a month. Mine costs $200 a month.

• 15:17Jason Calacanis?

  10x the value? Give me 10x the value.

• 15:20Speaker 01

  Okay. So first of all, I think it's very important to see here, it thought for 2 minutes and 42 seconds, right? Which is incredible, right? I mean, just let's level set for a second. All of this incredible work's happening either in your case in a few seconds, in our case, you know, 2 minutes and 42 seconds here in my case. All right. First of all, I think it goes, it goes below as a step-by-step illustrative model with clear stated assumptions and references and calculations. So it's a lot more organized than yours, I think, J Cal, right? So it goes total car trips in the US. So basically, you know, walks through it and in simplicity, it gets to a billion car trips a day.

• 15:55Jason Calacanis?

  Same answer, basically.

• 15:57Speaker 01

  Yep. Okay. Annual car trips gets to 365 billion, right?

• 16:01Jason Calacanis?

  Yep.

• 16:01Speaker 01

  It says maintain more conservative. It actually does this interesting thing saying, hey, we'll go to 340 as slightly lower, still representative based on than it's a data that it has, right? Then it tries to calculate Uber and Lyft trips annually. It pulls those from the S1 filings from 2018 and 2018 of Lyft. Pretty interesting, right? I like the reference here. I don't think yours had that, right?

• 16:24Jason Calacanis?

  It did not. And it, here it actually did the time for context. It's given this context section, 5 billion rides from Uber or Lyft versus 340 billion total rides. I mean, the current ride share currently represents 1 to 1.5% of demand, which is the calculation I just did. So it's like thinking a little bit more.

• 16:39Speaker 01

  Yeah. So it's more like a J Cal. Then it's like fleet efficiency, right? So it kind of walks through this. So it's assumption is, hey, each fully autonomous robot taxi can do 20 to 30 trips a day. We'll take a midpoint. I like how it does that. Average trip duration, 15 to 30 minutes, some downtime between rides, high utilization scenarios, right? Okay. It's kind of all the stuff that you were rattling off on your own, right?

• 17:01Jason Calacanis?

  Yes.

• 17:01Speaker 01

  Then it's got fleet operations. Constraints, right? Which is, you know, six hours a day for charging, cleaning, maintenance, uh, five days a year for, for overall major maintenance shutdowns. So basically it gets, say, you know, 360 days a year it can be utilized and it gets to daily utilization. And, you know, again, I like this breakdown, right? Which is, uh, 25 trips per day, 360 days, 9,000 trips per year. So for strict with the numbers, basically we get eight, nine, seven, five, but we'll use 9,000 as a rounded figure. I like how it's kind of humanizing that bit too, making it easier for us to handle. And then it's like required fleet size to handle 100% of the US car trips, right? So just going through all the

• 17:41Speaker 01

  math, 340 billion trips, 9,000 a year, requires 37.8 million robo taxis. What did yours get to on that one, Jake? [00:01:05]

• 17:48Jason Calacanis?

  I think it said something like 46, um, because it's taking into account the induced traffic, which I put into the instructions and it put in, uh, picking up some public transport.

• 17:59Speaker 01

  Same instructions. So I should have had that.

• 18:00Jason Calacanis?

  Same instructions.

• 18:01Speaker 01

  Now, yeah, now it's like, it adds the public transport capturing 20% with FSD cars. So it starts to run through the logic here and it basically, again, still at 38.2 million rides. Now your fleet cost, it starts to calculate it. And then just for, you know, the induced demand, because you and I were talking about this, it's well known that you have these induced demands. So let's say 20% increase in total trips, you know, because of widespread capacity. And summary, after all that, yeah, the summary is like 340 billion trips, 25 a day.

• 18:36Jason Calacanis?

  I gotta tell you, it's not that different. It's not 10 times better. It feels a little more polished. Um, I wonder if it's our instruction set. So I think it'd be good to have somebody from OpenAI on the pod who worked on this project. So let's just send a little note to them. The results here are not different. The results here are 90% the same. There are some formatting. And so the UX, I think, is determining the value of LLMs right now. It doesn't feel to me like the core LLMs are getting much better because I don't think, I think they've got as much data as they're going to get, or as, you know, 80 or 90% as much data as they're going to get. So then it becomes the interface, then it becomes the instructions and how it interprets what you want and how it gets to know you in the personalization.

• 19:19Jason Calacanis?

  That's my feeling of the gains that are happening now. Formatting like Canvas, um, you know, this product.

• 19:26Speaker 01

  Artifacts, yep.

• 19:28Jason Calacanis?

  The Pro.

• 19:29Speaker 01

  Yeah.

• 19:29Jason Calacanis?

  Agents eventually, I think we're now in the fit, finish and polish of the language models and the language models themselves. Um, are they starting to plateau because they've run out of data? This is something I saw Ilya was saying, hey, we stole all this data. He didn't say steal, but let's call it what it is. We stole all this data. We got all this data. Some of it's public, some of it's not. Okay, whatever it is, it is. Putting that aside, I think that they're now stuck.

• 19:58Jason Calacanis?

  Founders know that every missed call is a missed opportunity. Customers don't want to wait. They will call someone else if you don't pick up. But if you use OpenPhone, you're never going to miss another customer call. And guess what? It's super affordable and easy to use. People used to spend tens of thousands, hundreds of thousands of dollars putting in a corporate phone system. And now for just $15 a month, OpenPhone will give you a business phone line and complete control of your destiny. Want to know who's answering customer calls and how they were handled? OpenPhone can do incredible things like sync with HubSpot and give you AI-powered call summaries, automated responses to ensure you don't miss a single ring. Well, that's all built into OpenPhone.

• 20:40Jason Calacanis?

  And if you've got an existing phone number, they will let you port it over at no extra charge. Get 20% off your first six months. What an amazing offer. At openphone.com/twist. That's O P E N P H O N E dot com slash twist for 20% off for six months. The L L Ms feel stuck to me and now it's about inference, interface, how it interprets instructions, how it personalizes you, and then proprietary data sources like Reddit, Twitter, uh, Google flights data, you know, which comes from other databases, you know, the, the data, the deeper data and the instruction set and the interface. Am I right or am I wrong?

• 21:19Speaker 01

  It's a great summary, J Cal. And it's interesting that, you know, we do this experiment on something that costs 10 times more. I would feel, you know, if I had to use the output, the output of the $200 one gives me a little bit more background on things.

• 21:34Jason Calacanis?

  Yes.

• 21:35Speaker 01

  You know, you, what you've actually done in the way you prompt is that you've, you've convinced, or you've been able to, not convinced, but you've been able to direct the model.

• 21:47Jason Calacanis?

  Yes.

• 21:47Speaker 01

  That's not as powerful to do what the more powerful model is doing.

• 21:51Jason Calacanis?

  Right. So maybe if we just said, build a model of what it would cost to replace all US rides with robo taxis with as many details as possible. Okay. And I'm going to give you the same one to do. That is just a sentence now. Now let's see if it does anything close to what I did. Because you're right. When I did this, I kept a notepad open on the side. I kept Notion on the side and I was putting in my architecture of how I would solve the problem, which is, how many rides are there? How many public transit rides are there induced? I introduced those three topics. I did, I let it know that I wanted induced in there

• 22:36Jason Calacanis?

  for 20%. I let it know I wanted to know Uber and Lyft's percentage. I wanted to know just in the US. So I did give it a framework. Yeah, this is really interesting. It came to, without my instructions, an annual operating cost of 150 billion a year and total initial deployment cost of 8, 2.8 trillion.

• 22:55Speaker 01

  Yeah.

• 22:56Jason Calacanis?

  Wow. So this came up with a totally different number and it didn't explain it very well, but it's getting them as citations, I think.

• 23:01Speaker 01

  But see, what, what you've done, Jay Cows, what you've shown our, our listeners and our watchers is that if you are willing to put the time in to create better prompts, and I think this is important for the industry, you can basically get what is the equivalent of 10 times more expense on a model just by putting the time and you know, it was good. It's, I think this is a really, really good example. I'm running mine now. So it's, these ones take a bit longer to run. So just let it run as we talk through it.

• 23:31Jason Calacanis?

  And here's the output. Look, it just found the daily passenger trips. It got that right. For trips, robo taxi a day, it picked 20 and it got it from a citation from Litman 2019. So I guess somebody had done a paper at some point. That said, 20 is the right number. We came up with the number 25 or 30.

• 23:47Speaker 01

  Yeah.

• 23:47Jason Calacanis?

  The reason we came up with that number is we gave it, we want six days off a year and we want six hours off a day to charge. So we, we gave it like, hey, five days or six days of maintenance a year, which seems, you know, like if you gotta take tires or it gets a fender bend or needs to be repainted, who knows what could go wrong on these cars for maintenance and then charging, certain amount of charging. And it came up with 55 million times 45,000. So it used the cost estimates of, I think, a robo taxi as opposed to a Waymo.

• 24:17Speaker 01

  Yeah.

• 24:17Jason Calacanis?

  Pretty fascinating. Charging infrastructure included 300 billion in charging infrastructure, which it doesn't need to include, I don't think, but maybe. Actually, maybe you do need to do that because if there were that many, you would need much more. You do actually need to include that because you would be charging every single car all day long. That's actually a really good point. Maintenance operations, software, 100 billion a year, McKinsey and Company, Rand Corporation, 2018, insurance regulatory. So people have been working on this data. So it just did a better, it just did citations. It didn't kind of use my framework. What did you get? Is it still doing yours?

• 24:47Speaker 01

  My, my, mine's still running. It's probably going to take another minute here. I had my guesses, but like.

• 24:51Jason Calacanis?

  So anyway, I don't think it's worth it. I think we learned something here, but I'm going to buy it anyway because for $2,400 a year versus $240 a year in business to spend an incremental $2,000 for an average salary in corporate America of, let's say, I don't know, $80,000. I'm including people who make 40 and people could make 150. We'll pick $75,000. You know, for $2,500, you gotta ask yourself, does an employee, an information employee, get 3% more efficient with one of these products? Pretty clearly, yes.

• 25:24Speaker 01

  Oh, yeah.

• 25:24Jason Calacanis?

  But they have to use it. And this is the thing that is making me a little mental. I can't get people to use it. I can't, I, you know, I,

• 25:31Speaker 01

  Yeah.

• 25:31Jason Calacanis?

  I know if people are using it or not, I can't get people to use it. People, it's, this is a habit. I think the world's going to bifurcate between people who use this as their default all day long and people who don't. And it's going to be really sneaky.

• 25:44Speaker 01

  Can I give you a hack there? Please? This reminds me of the time, I would say it was like just past the mid nineties when the internet, it was just making its way into the corporate world. There was a set of people, which is, you know, the our vintage, which were dying to use the internet. And there was a set of people that would not use the internet. And they fact they were didn't trust it. And so, and what was it? We were just younger and we were more kind of tuned into technology.

• 26:09Jason Calacanis?

  Tech savvy, we're had more energy, more fascinated, no kids, no bills.

• 26:14Speaker 01

  My suggestion to you is as the summer is coming up, either you do it with interns, hire two to three people that are sub twenty J Cal. And I will tell you, under twenty, everything they do, they use OpenAI because, you know, what is the one thing that you got through experience? You got mentorship, you got to work through that. So how they account for that is by using these AI tools. And so as long as you have someone with good energy, so imagine the eighteen or seventeen year old version of yourself.

• 26:43Jason Calacanis?

  Oh, yeah.

• 26:44Speaker 01

  I'm super keen, but I don't know a lot of these things. Oh, how am I going to go figure it out? I'm going to figure it out.

• 26:50Jason Calacanis?

  Yeah, I mean, you're not allowed to hire by age in the United States here, right? Unless you were like casting for a movie, I think you could actually do it there. Well, maybe not age, you'd have to be doing by look. So, but there is generational differences. So while you can't hire for age, if you do have entry level jobs, it generally will skew younger because the salary is, is in what, ten, twenty years.

• 27:11Speaker 01

  Like as internships or something.

• 27:12Jason Calacanis?

  For college credit, yeah. And there is a distinct difference between how young people use these tools and older people and getting older people to use them. It requires sometimes like a little, a change in behavior, which requires pushing. And so here's what I did. It's called new tab override. It's by Soren Henschel. And what it does is when you load a new page and in your Firefox browser, and this will work on Brave, Firefox and Brave use the same. It lets you put in an option of a URL and then you see where it says focus here? Set focus to the webpage instead of the address bar. This is clearly, this is a very important thing to do. What this does is when you open a new tab, it puts you in the URL bar.

• 27:52Speaker 01

  Right, yeah.

• 27:52Jason Calacanis?

  Here it puts you in the search box if you do this. So when you start typing, you don't have to hit tab or click your mouse to get in. Super clear, if you were for me, this needs to be on your computer. I will randomly pull it up during a Zoom meeting and say, do a new tab. I mean, I do this kind of stuff and does that make me crazy that I spot check? I spot check, I say, pull this up. So if you were for me. Be prepared.

• 28:14Speaker 01

  Yeah.

• 28:14Jason Calacanis?

  Because I want you to. Yes, chef. Yes, chef. And you know what? We're all chef in this analogy. I always tell everybody, we like a good yes chef, just acknowledging that we're making progress here. All right.

• 28:25Speaker 01

  Okay, mine finished. Let's quickly wrap this one up.

• 28:28Jason Calacanis?

  Okay.

• 28:29Speaker 01

  Mine finished. Again, uh, this was, this time it was three minutes and 18 seconds. So.

• 28:34Jason Calacanis?

  Yeah, it's pretty long.

• 28:34Speaker 01

  The prompt was shorter.

• 28:35Jason Calacanis?

  Mm hmm. Okay, this did a better job. You see, your, your premise is correct. You can get out of a standard model in the more expensive model, it's doing the sub prompting, isn't it?

• 28:45Speaker 01

  Yes, it's all correct.

• 28:47Jason Calacanis?

  Yes. What do you call this prompt generation feature where I can give it less prompting, but get more prompting because it's doing the prompting for me?

• 28:57Speaker 01

  So I think the.

• 28:58Jason Calacanis?

  Is there a term in the industry for?

• 28:59Speaker 01

  It inference time reasoning. So what it's doing is.

• 29:03Jason Calacanis?

  Inference time reasoning.

• 29:06Speaker 01

  I T R.

• 29:07Jason Calacanis?

  Iter. Okay, so it's iterating. So we're going to use that. We created an industry term. Inference time.

• 29:13Speaker 01

  Reasoning.

• 29:14Jason Calacanis?

  Reasoning. Got it. So when you do the inference, that's the query.

• 29:18Speaker 01

  Yes.

• 29:19Jason Calacanis?

  It's doing, at that time, additional reasoning, as opposed to doing the reasoning when the L L M was built on a bunch of H one hundreds and.

• 29:28Speaker 01

  Well, the, the main difference is like, say, when we started on this adventure of, you know, chat G P T. And I'll just go to the summary here, but when we started on this adventure of chat G P T, the minute the first token is predicted, every other token is already determined. And so it's just going through, you know, sort of what's going to happen. What happens in this inference time reasoning? It has the ability to kind of stop partway through and then work and say, let me go and let me do an offshoot on some of these things. Bring those answers back into my main line, right? It's a oversimplification, but that's sort of what's happening there. And so it's not a, just a standard prediction of tokens, which is what we saw in the original iterations. That's why we're seeing it. But what we've shown here, and I, I tend to agree.

• 30:11Speaker 01

  If you, if you prompt engineer with more sophisticated prompts, you don't have to do as much inference time reasoning and you can get very similar results. And so I'm fascinating here that it, that it did come to the same 2.7 trillion number that you were at and the same operating cost.

• 30:28Jason Calacanis?

  Hey startups. When you're a business, you gotta treat your customers right. Unreasonable hospitality is the standard today. But you're gonna need tools. You're gonna need a platform to help you do this. And that platform is the Zendesk Suite. The Zendesk Suite is gonna give your startup all the tools you need to deliver exceptional customer experiences so you can build stronger relationships without growing your headcount. That's key, right? Every dollar matters. You gotta control headcount. You gotta control spend. So use the tool that Shopify, Squarespace, Uber and Instacart all rely on. It's called Zendesk. Let's take a look at another customer, Unity. Very famous company. They saved $1.3 million with Zendesk automations and self-service,

• 31:12Jason Calacanis?

  and they saw an 83% increase in their first response time. These companies love Zendesk because it's so easy to set up and it scales with you as you grow. They'll also give you all the metrics to make your reporting easy, keeping you and your business agile and investor ready. And that's because you're their esteemed customer. And they've created the Zendesk for startups program just for you, where you get unlimited access to all the Zendesk products, expert insights, all the best practices, and entry into their amazing community of founders, all at no cost for the first six months. That's right, they wanna support you. Zendesk.com/twist. Get ready to scale with the best in customer support with six months free. Nothing to lose.

• 31:56Jason Calacanis?

  It's really cool when you compare this to what Gemini is doing. So I asked it the same prompt here. Build a model of what it would cost to replace all US rides with robotaxis as many details as possible. And it says here, cost of it, and it says what it's gonna do. And I didn't put these details in here, but it says in the ITR inference time reasoning, it said, build a model that it says with as many details as possible by, find the total number of rides taken in the US annually, find the average cost per ride of each model of transportation, find the estimate. So it's actually come up with its own reasoning. Analyze the reports, create a report, ready in a few minutes. Start the research. And it's doing it right now. Feel free to leave this chat. As you know, I'll let you know it's done. And it researched 69 webpages. Look at that, 69.

• 32:42Jason Calacanis?

  Not 420. And look at all these. It's showing you its worth. This is why I think this is a better product right now for me. I like to see what it's doing, you know, and it's analyzing all the results here. I guess we're about a minute into it. Uh, this is Gemini Advanced 1.5 Pro with deep research. And if you don't have the Gemini app, the Gemini app does not have deep research in it yet. It does have the other features. It's as good as ChatGPT's app. Gemini and Google have reached parity in my mind with it. Now, did you see some talk about the Gemini API? And that API, the Gemini API is gaining steam on everybody? Is that true? Are people, developers using it?

• 33:26Speaker 01

  I saw a post for that today. I, I think directionally it's correct. Like definitely there's been huge amount of growth on, on, on, uh, you know, Gemini. I'm not sure. I, I think the tweet I saw was from Open Router or something like that where they said it's, you know, greater than 50%. That may be, you know, Open Router's view of it. I still think, you know, it's probably not 50%, but happy to, happy to be proven wrong there. Google folks want to come out and share.

• 33:51Jason Calacanis?

  They said it was highest 50%, uh, which, yeah. Uh, so, you know, which by the way, uh, if you want $350,000 in Google credits, uh, you can get them at GetStartupCredits.com, GetStartupCredits.com. But, you know, I have all these startups meet with us. 20,000 people apply for launch, go to launch.co to apply for funding from our firm, join our programs, et cetera. After they apply, we have.

• 34:15Speaker 01

  Wait, is this the year you did with GCP?

• 34:16Jason Calacanis?

  Well, GCP did it for All In Summit and they did it for This Week in Startups and they did it for Accelerator. We also have Oracle provides credits. Azure, Microsoft Azure provides credits and Digital Ocean provides credits to our startups. The only one who doesn't is AWS. AWS, they have like a rack rate thing, but AWS is not very supportive of anybody but Y Combinator. They've got like a weird thing. They also don't buy ads or whatever. So, which is, but you know, AWS is great. I don't have any hard feelings towards them.

• 34:44Speaker 01

  Yeah.

• 34:44Jason Calacanis?

  But they're not supportive, um, in that way. They kind of picked YC and I think YC is very sharp elbowed sometimes. So they're like, we're Team YC, we're not. Team, everybody else. Okay, that's fine. I have half the number of applications in Y C right now. And next year, I'm gonna match them. Alright, so here we go. We've got this done. And look at this. It did a nice thing. Total rides in the U. S. Annually. It estimated it got bus rides. It included bus trips. Well, I didn't have to do that. Based on the national survey, Americans make approximately 1.1 billion trips a day. 411 billion trips annually or about 1500 trips per person. Wow, it added that. That's pretty interesting. And then it has here. Look at this. It built a table. Motor, transport, car, bus, train, freight. So included all of those and estimated rides.

• 35:28Jason Calacanis?

  Average course for ride. And it put the amount. Wow. Estimated cost of manufacturing and deploying. Early estimates up to 400,000. Maybe that's in billions or something. Tesla is projecting $25,000. They got that right for their robot taxi. Baidu 77, Waymo 180. Wow, that's interesting. Deployment cost, estimated cost of maintaining. It put that in without saying. Estimated operating cost per mile, projected average distance travel. I mean, this is incredible. Total cost.

• 36:00Speaker 01

  Yeah.

• 36:00Jason Calacanis?

  Cost per robot taxi ride, manufacturer cost, deployment cost. Yeah. And it just figured that out. Wow. This is better. Let's be honest.

• 36:08Speaker 01

  No, it is. And the superpower is that that open in docs on the top right.

• 36:13Jason Calacanis?

  Well, I mean, that is, I think, if I want to, I can just open this up in Google Docs. Creates a document for you.

• 36:20Speaker 01

  Yeah.

• 36:20Jason Calacanis?

  Which is your next step. And now you're starting to see this. Now, if I save this document in the future, it's going to know that I did that document. And it's going to be able to use your documents in your email. So if I was emailing with, I don't know, somebody running a robot taxi, or I had 10 friends who had shares in Tesla, Uber, or whatever. And we had conversations. I wonder.

• 36:39Speaker 01

  Yeah.

• 36:40Jason Calacanis?

  If on my side, it's going to take that into account and say, hey, in your email, in your Gmail, there was a conversation with Dara. Or this analyst at Warby Parker, Warburg Pincus.

• 36:51Jason Calacanis?

  And they helped you do that and they gave you some data there. Do you want me to include that data? So this could get very interesting, very quick, folks. I believe Google is the sleeping giant. Grok also doing a very good job. The other Grok, yeah, we'll get into. Okay, let's do a couple more demos here.

• 37:08Speaker 01

  Well, just quickly, I want to close out on that statement. So let's do one thing. We got to get back to our grading. O1 Pro versus O1. And then let's grade 1.5 with deep research. So three grades.

• 37:20Jason Calacanis?

  I'm going to give a B plus to Pro. I felt like it did a really good job. Yeah. And I would pay for it. And I would give an A to the new 1.5 from Gemini.

• 37:32Speaker 01

  With deep research.

• 37:33Jason Calacanis?

  With deep research, I'm giving it an A. Only because I believe the output of both of those with ITR, I feel for the average, I'm creating it on my feeling and what I think it will do for the people who work for me. The people who are not putting in the deep, thoughtful prompts, they're going to, I think, get a lot more out of Gemini 1.5 with deep research or O1 Pro, the $200 a month. Now, Google's 20 bucks a month. So I give it an A plus on a value basis.

• 38:07Speaker 01

  On a value basis.

• 38:08Jason Calacanis?

  On a value basis, I'd be like A plus and a B. So there'd be a big gap there. I'm saying in a corporate America, this pricing does not matter for the value.

• 38:16Speaker 01

  It doesn't matter. Yep. Yep.

• 38:17Jason Calacanis?

  It doesn't matter because you're spending more on people's parking. So just throw the shit in the garbage. Sorry. And let's just talk about how much it will impact my employees who use it, my team members who use it, my founders, who we invest in and partner with. I believe B and A. A B and an A. I'm not giving pluses and minuses today.

• 38:40Speaker 01

  Yeah. So my interpretation is. I actually, I'm going to give them both A's. And what I really like about what OpenAI has done is it is giving the reasoning process along the way, which I think is very powerful for people that are using this in a work context versus when it's just spit out at you. So I liked how it was sharing its reasoning along the way. So I'm, I kind of lean towards that a little bit.

• 39:06Jason Calacanis?

  Which deep research does as well. Gemini's advanced 1.5 pro with deep research. I gotta talk to Sergey and the team over there. When you're naming these things, it's Gemini. That's the product.

• 39:16Speaker 01

  Yes.

• 39:17Jason Calacanis?

  It has a version. Nobody cares about the versioning. Just it's Gemini. Don't say advanced. Don't say 1.5. Just call it Gemini. And then have Gemini with deep research and abstract out the, the version numbers for nerds. But I think this is too confusing for consumers, right?

• 39:35Speaker 01

  It's getting even worse. Like if I look at my menu here, which I'm sure you have the same choices. I have 1.5 pro, 1.5 flash, 1.5 pro with deep research, 2.0 flash experimental and 2.0 experimental advanced.

• 39:47Jason Calacanis?

  Yeah, you know, this is, okay, Google's premise was here's the box. You type in what you want, you search or you say, I'm feeling lucky. I'm feeling lucky. It's a sniper shot that takes you right to the thing. That was cute and fun. But there was just a search box. So here for Gemini, I think it should just either do a quick search, whatever the best search is, or you should have deep research. And then if you want to, you have somewhere where you can kind of tweak the model. But it's just too confusing for consumers.

• 40:14Speaker 01

  Okay.

• 40:15Jason Calacanis?

  So we make progress here.

• 40:16Speaker 01

  Last thing on this one. Okay. One last thing on this one is.

• 40:18Jason Calacanis?

  If you could only use one, which one would you pick?

• 40:20Speaker 01

  I'm going to stick with OpenAI Pro.

• 40:24Jason Calacanis?

  Ooh, I'm going to stick with Gemini deep research because I think Google has access to data that OpenAI doesn't, and I believe the gap is going to grow.

• 40:32Speaker 01

  Okay. That's, we'll, we'll come back to that one. I don't know if I bet on that, but I'm just using it now. One last thing on this one.

• 40:38Jason Calacanis?

  Okay, okay. How about this? How about this?

• 40:40Speaker 01

  You have two high school interns for the summer.

• 40:43Jason Calacanis?

  I don't do internships unless it's friends of the firm.

• 40:47Speaker 01

  Okay.

• 40:47Jason Calacanis?

  I do it as a favorite bank. You know why? Cause in those 10 weeks, they take up all your time and resources and you train them and then they're gone.

• 40:55Speaker 01

  But they're supposed to use the, these tools. That's what I think it is for the rest of the team. That's

• 40:59Jason Calacanis?

  I honestly would just rather hire, I just prefer to hire people at school. I'm going to University of Texas. Shout out to Jay Hartso, President of UT. I went to UT and I am so impressed by the UT graduates. I went to a game, Longhorns, whatever that is, uh, go Longhorns and, uh, I am all in on UT.

• 41:19Speaker 01

  We went to a football game.

• 41:20Jason Calacanis?

  I went to a football game. A Pix, well, I mean a hundred thousand people. I was on the field and it was awesome. Uh, but more awesome, there's 55,000 students at UT and they're smart. And like, I think the top one or two percent there are like Ivy Leaguers, but they're blue collar Ivy Leaguers.

• 41:39Speaker 01

  With one thing, I, I did see this thing and it was, there's UT at Austin. Then there's also University of Austin.

• 41:45Jason Calacanis?

  UT is the public Austin.

• 41:47Speaker 01

  Yeah, of course. Yep.

• 41:48Jason Calacanis?

  Big giant school. They're funded because they, my understanding is they have land and under the land they found oil. So they are super funded in Texas. If you, if you, if you're a Texas resident, UT is like eight, nine or 10,000 a year. You can get in and out of UT for 40 dime skis, which is half the price of a private school in the Bay area for one year. Cause it's 60 or 70 and you gotta give a 10K donation or else they admonish you and, and give you a hard time.

• 42:18Speaker 01

  For sure.

• 42:19Jason Calacanis?

  So one year of private school in the Bay area or New York, Dalton, whatever this nonsense is. An entire degree from UT.

• 42:26Speaker 01

  College education.

• 42:27Jason Calacanis?

  College degree, University of Austin. Now, if you're out of state, they charge you a rack rate and so it's a little bit higher.

• 42:32Speaker 01

  Yeah.

• 42:32Jason Calacanis?

  But 80% or 90% of the people go to UT are in-state. This is amazing.

• 42:38Speaker 01

  Okay.

• 42:40Jason Calacanis?

  Now, let's go over to University of Austin.

• 42:43Speaker 01

  Austin.

• 42:44Jason Calacanis?

  Joe Lonsdale and a group of these, you know, kinda free will and awesome, free thinking libertarian-ish Republican types on the right, but I would say maybe they would be considered moderates. You know, like I think Barry is probably a moderate.

• 43:00Speaker 01

  Yeah.

• 43:00Jason Calacanis?

  You know, classically, like she probably would have voted for a Clinton Democrat or, you know, a Mitt Romney as much as she would vote for a Trump or whatever. So if she did vote for Trump, I don't know if she did. Um, putting that aside, they just had their first class. It's like 50 students or something. It's a startup school. They bought some universities, so it's accredited and they want to teach from first principles, take all the woke out.

• 43:21Speaker 01

  Okay, got it.

• 43:22Jason Calacanis?

  But I mean, I'll be honest, UT doesn't have like a woke movement there. Like when they had the protests or whatever.

• 43:29Speaker 01

  Got it.

• 43:29Jason Calacanis?

  It started and ended pretty quick. But in a long way of saying, you know, I've been thinking about, you know, I have this founder university and I'm gonna bring it in person. I think in the next cohort is my plan in Texas and have.

• 43:42Speaker 01

  In Austin.

• 43:43Jason Calacanis?

  In Austin, and I want to get a space. This is the big announcement, you know, as part of what I'm doing there. And so I'm trying to figure out if I do that with a university or if I just do it in a space or if I do it remote and in person, if I do it every day for an hour a day or if I do it two hours a week and get a co-working space. So, you know, it's a lot on my plate right now, but I'm trying to figure out how I can have my own university, the founder university, teaching how to be a founder. Uh, and that's why I spent so much time getting founded at university and giving people instead of they pay to, instead of paying tuition, we give 25K to the top 10% of students to start their company at a $1 million valuation for 2.5%.

• 44:19Speaker 01

  Yeah.

• 44:19Jason Calacanis?

  Which is a good deal for us. Most people argue it's a great deal. It's kinda like the Y Combinator, Accelerator. But we expect only one out of three of those to pull through and get another round of funding. So we're taking high, high, high massively high risk bets. So if you were to net it out, if two out of three don't even make it to the next round of funding, it's really like 75K at 3 million for two and a half percent. Because you're taking into account how many people would wash out and just not make it to year two. I really enjoy that kind of part of the job and seeing a lot of good stuff. What else? We didn't like, let's go lightning round.

• 44:51Speaker 01

  Let's continue on the path. I do want to give a shout out to a couple of other things along the way. So lightning round for the next few minutes here. Okay. Just some of the, some of these are not demos, but they're important things to call out for. Recently, Meta launched Lama 3.3 70B. And what I wanted to call out on Lama 3.3 70B is just in terms of how fast things are iterating, and we won't do a demo with this because, you know, I think it's easy to call it here. But if you look at a comparison against Gemini Pro 1.5, which you were just playing with, with deep research, right? You can see that it is starting to make really good inroads against its, you know, previous competitors.

• 45:29Jason Calacanis?

  In a benchmark test.

• 45:30Speaker 01

  In benchmark tests, exactly. But this is a relatively small model, so let's not count Meta out here. That's all I'm trying to share here.

• 45:38Jason Calacanis?

  Oh no, Meta is doing a mitzvah for the industry by going open source and not trying to make money on this. And they're letting other people use it. There were some weird caps, like you couldn't have 100 million users or something. But I think Zuckerberg is going base and he looks at this like the open compute platform. He knows he's got a network effect. He'll defend it. Everybody can use his language models and he is going to be the backstop against Sam Altman's closed AI. Which is so paradoxical, by the way.

• 46:07Speaker 01

  And what I do want to call out here, right? This, this column right here is Lama 3 70B. This is Gemini's GPT-4O. And look at the pricing down here. You're talking 10 cents for a million tokens input, 40 cents for a million tokens output. Your dollar 30 and five bucks and two, $2.50 and $10. So it is getting there in being comparable, but from a price perspective, crushing. Which is something that, you know, Zuck and Meta have always been incredible at.

• 46:37Jason Calacanis?

  Well, they are obviously investing heavily in this. What do you estimate these platforms are losing? Providing services at this pricing or are they breaking even? What are they doing? Do you have any insight into their infrastructure costs? And how much they might be losing, making or breaking even.

• 46:55Speaker 01

  I fundamentally believe if you are, and look, I'm a bit skewed here, but if you're not building your own infrastructure, including chips from scratch, it's very hard to be competitive because you do have to pay an 80% margin to Nvidia along the way, right? And so fundamentally, I do not believe that anybody is losing a ton of money on these things, but they're not making a lot because the big chunk of the margin is being taken out by Nvidia along the way.

• 47:22Jason Calacanis?

  So this is the challenge for the industry and this is why Nvidia could be a short or, you know, could have topped out here because people are now realizing there's an 80% margin there, which means they're going to have to compress that margin and lower their pricing to compete with Amazon, Apple, Grox. Yeah, I mean, everybody's providing inference chips, et cetera, at greater and greater prices. Do you make custom ones with people or do you only make your own?

• 47:50Speaker 01

  No, our, our, our chip runs all models. So, you know, folks come to us and we run them and, you know, and so that, that's kind of where I think things are going to start to net out is that you see that price difference there? Let's not ignore that. Let's make sure everyone keeps watching that because I do think, I do think there's, there's pricing and there's capabilities and those things are kind of starting to go in some interesting directions. Uh, let's keep speed rounding here. Okay. So next one, let's do Sora because you had brought that one up.

• 48:17Jason Calacanis?

  Sure.

• 48:18Speaker 01

  And so have you, have you tried Sora yet, Jacob?

• 48:20Jason Calacanis?

  I haven't tried it, but I've looked at the demos. I see the demos. So.

• 48:23Speaker 01

  Yeah. So basically, you know, we've got it here. Uh, the, the generations do take a while, but like what you can see here is, you know, they've made it available. They've made a very clean UI. They've given you the capabilities to do anywhere from five, 10 seconds, right? And they can do a couple of different variations. And so.

• 48:41Jason Calacanis?

  None of it looks real.

• 48:42Speaker 01

  The interesting thing is.

• 48:43Jason Calacanis?

  Looks fake.

• 48:44Speaker 01

  I, I was, I was going to tell you that. So my overall observation is uh, cling, which we reviewed before, which is that one of the Chinese based ones, it looks the most realistic. And I fundamentally believe that's the case because it's trained, I think, on a bunch of proprietary copyrighted data. And because it's done that, it's able to do it. Now, my understanding is a lot of the folks, a lot of the training data that's provided to these is being generated via game engines. And game engines are good, but do you get a feeling this feels a little bit game engineering?

• 49:17Jason Calacanis?

  This feels like if you, if stock imagery and a video game had a baby. So we have nailed it. If you look, when you see this stuff, it looks like CGI done, you know, in the Czech Republic or Poland, Eastern European countries, South Korea, done by somebody doing like a corporate video or something. In other words, it looks professional, but not industry leading like Disney or George Lucas or, you know, J.J. Abrams would, would accept. So J.J. Abrams, George Lucas, Spielberg, Scorsese, you know, anybody making a, you know,

• 50:00Jason Calacanis?

  secession TV show, nobody would accept any of this. It's all 60% of what they would accept, 70%. So this would be great for them to storyboard and to be able to show, hey, here's what it looks like, but it's not good enough for prime time. It feels like it's a couple of years off, if ever, because, you know, while the Chinese have no problem stealing Disney's archive and doing this, and they'll have models out there that will let you do anything you want with the Marvel characters, the Disney characters soon. I think Marvel should release a model in partnership with one of these companies. And for your Disney Plus subscription, you can create Disney character models. And you can make short videos and they only exist inside the Disney app and you can send them to friends.

• 50:44Jason Calacanis?

  This would be a killer feature.

• 50:46Speaker 01

  This is one of the Chinese ones. I was gonna play, it's like a two-minute video, but this one, I want to get your reaction to this one.

• 50:52Jason Calacanis?

  I'm watching it.

• 50:55Jason Calacanis?

  That looks like stock there. That looks real. That looks pretty real. Yeah.

• 51:04Jason Calacanis?

  Like, almost like they took a George Clooney film, like a professionally shot George Clooney film in Italy, you know, The Italian Job or something. Yeah, this looks like they took, they stole from Hollywood to get this effect.

• 51:18Speaker 01

  Yeah, so.

• 51:19Jason Calacanis?

  Doesn't it look closer? It is distinctly closer in that example to a Hollywood film than a stock photography library.

• 51:26Speaker 01

  Yeah, yeah.

• 51:27Jason Calacanis?

  Well, this shows you, and I think we didn't bring this up, but the OpenAI whistleblower who apparently committed suicide or was whacked.

• 51:38Jason Calacanis?

  Uh, there's a lot at stake here. I mean, I know I sound like a conspiracy theorist, but there, I think OpenAI is gonna lose their lawsuit. I'm saying it right now. I'm predicting it here. I think it's going to be an injunction against OpenAI. And they're gonna have to settle for billions. You heard it right. I think it will be the largest copyright infringement case in history. I think it will be a billion dollar settlement with the New York Times. And other people are gonna join it. If you are a content creator and you feel your stuff is stolen.

• 52:04Speaker 01

  Billion dollar, okay.

• 52:06Jason Calacanis?

  I think it's gonna be a three comma settlement. I honestly do, or judgment.

• 52:09Speaker 01

  Trace, trace commas.

• 52:11Jason Calacanis?

  I trace commas. Well, listen, this is not unprecedented in the world. Things like this can happen. We have seen records be broken when there is serious damage. And so, and I think it's gonna, the reason, you know, this tragedy that occurred, you can look it up, folks, is a 26-year-old whistleblower inside of OpenAI who apparently is unalived and we don't know why. Um, and he was a key linchpin in this testimony. Um, I'm not saying I think he was murdered by an OpenAI employee, obviously. But this is really weird looking, very weird looking, uh, like many weird things occurring in the world.

• 52:47Speaker 01

  These things I always thought were weird and then everything that's happened in the last six months and it was like, you know what? Anything is possible now. Anything is possible.

• 52:56Jason Calacanis?

  Listen, if, if people who don't like Putin, you know, fall out of windows at an alarming rate statistically, you know, who's to say it couldn't happen here, right? I mean, I'm not gonna be so arrogant to say there couldn't be somebody, yeah.

• 53:11Speaker 01

  Uh, I'll look, very sorry for the family of the gentleman. Hopefully they figure out what happens there. Um, not, not, but quick grade on Sora.

• 53:19Jason Calacanis?

  I mean, I give Sora still a B could be much better. I'm sticking with my B there. I didn't, I'm not like, I don't know what the use case for these things is. Like, I feel like this is like ChatGPT, like 2.5 or whatever that was where you were like, or like, remember Gmail could guess the fifth word?

• 53:36Speaker 01

  Yeah.

• 53:37Jason Calacanis?

  And you're like, eh, okay.

• 53:39Speaker 01

  Okay. All right.

• 53:40Jason Calacanis?

  But the jump we saw today from

• 53:43Speaker 01

  The Chinese.

• 53:44Jason Calacanis?

  You know, no, no, the, the jump in the reasoning.

• 53:48Speaker 01

  All the reasons are, yeah.

• 53:49Jason Calacanis?

  Guess the third or fourth word.

• 53:51Speaker 01

  Yes.

• 53:51Jason Calacanis?

  Uh, would you like to?

• 53:53Speaker 01

  Yeah.

• 53:54Jason Calacanis?

  You know, it says have dinner or whatever.

• 53:56Speaker 01

  Yeah.

• 53:57Jason Calacanis?

  That guessing game that it was doing five years ago in Gmail, leading to Gemini with deep research. That jump occurred in five years.

• 54:06Speaker 01

  Boom.

• 54:08Jason Calacanis?

  If that happens with this, in five years, we'll be sitting here going, make us a Sopranos episode, and then we're going to watch it in my movie theater in Tahoe, this, you know? Uh, make us a lost episode, you know?

• 54:19Speaker 01

  Yeah.

• 54:20Jason Calacanis?

  Uh, of like this.

• 54:20Speaker 01

  That'd be fun.

• 54:21Jason Calacanis?

  Well, no, I mean, it's, it's gonna happen. And there's no reason your favorite character in the Star Wars series, you couldn't direct a film. Uh, or your daughters couldn't, you know, or your kids couldn't work together to say, you know, we always do like a little, um,

• 54:36Speaker 01

  Yeah.

• 54:37Jason Calacanis?

  Uh, what do you call it? Like, uh, you know, like, uh, um, talent show, you know, at holidays, like the little talent show with the kids? So if we do a little talent show, it's going to be like, hey, I directed this film about Ashoka from Star Wars, right? It's, it's coming.

• 54:50Speaker 01

  So I do that all the time. I do Seinfeld episodes in a modern, with a modern scenario. They're always kind of fun to do. Try, try that in, in your thing of choice.

• 54:58Jason Calacanis?

  George starts having ChatGPT, uh, do all of his communication. He's just like, I'm so bad with women. I'm just gonna.

• 55:05Speaker 01

  I wrote, I had some. He's, or he's using it at work, just.

• 55:08Jason Calacanis?

  Yeah, he just had it. Do what he thought this historical figure would say in certain situations. So instead of doing the opposite, Costanza, he does whatever Stalin would say, whatever, you know, like some lunatic would say, he says, make me like, make me into Einstein and myself. And all my responses should be like Bob Dylan and Einstein.

• 55:32Speaker 01

  Yeah. All right, two more really quickly and then we're done. Uh, you gave your grade there. Okay. This I think is interesting for folks because this comes up quite a bit. And, um, you know, I, I had this like live example, I actually tweeted about it. So I think it's fascinating. So I used in this case perplexity and I said, hey, why did Syria fall now versus in previous years? Who are the rebels and what parties are supporting them? And, you know, to be honest, you know, I actually wanted to know what happened. I, it was purely curious and I think it did a good job in terms of explaining what happened in terms of real time information. Then I did the same thing in Grok, right? The X Grok, XAI Grok. And so

• 56:10Jason Calacanis?

  GR OK.

• 56:12Speaker 01

  GR OK. And so even though these folks have access to real time information, which is great. And, you know, I think there's a huge advantage for GR OK. There is, you know, the folks that are connecting out to real time sources like perplexity. I find, and so my, my task for you here, Jason, is as things are unfolding, because I know you're always having to look up stuff either for episodes of This Week in Startups or All In. Really try this in both perplexity for real time events and in, uh, in Grok.

• 56:43Jason Calacanis?

  I find Grok is really doing a good job of catching the zeitgeist on X, but I do worry about all the anonymous accounts and the anonymous accounts at scale and their bias. You know, like I don't want King Koa, the anonymous million person account that Sax is always retweeting and is in love with Sax. Like, I get it. It's obviously some right wing person in their mom's basement. Or it could be a Russian or it could be any other chaotic actor. And they've got a million followers, like that stuff for me is really dangerous. Like an at scale account with a million followers, that's getting paid by X, right? Has revenue sharing on, but we don't know who it is.

• 57:24Jason Calacanis?

  Now I get you want to protect people and I get it's a pseudonym. So you can look at their account. And there's 20 of these accounts that have now hit prominence. Half of them have a real name, half of them don't. So with the real names, at least you know, like this is a 25 year old. Who's just a fanboy of, you know, whatever the left, the right in between this pod, that pod, Joe Rogan. You know, uh, Rachel Maddow, whatever it is, the bias is clear. It's these anonymous accounts that are hitting scale. That have thousands of other anonymous accounts. And this is where like the data at Reddit. The data at Twitter gives you an advantage in that it's consumer driven and it's fast and furious.

• 58:05Jason Calacanis?

  But now you're going to have an incentive and I don't, I've never heard anybody say this. So I'm just floating here. There is a big incentive now for a foreign actor. Or spam accounts to come in and use a hundred paid accounts. If I had a hundred paid accounts based on Twitter and Reddit and Hacker News. And I start building these pseudonyms up. And I start having conversations with myself on these platforms. Not only am I getting the value of influencing people in real time. I'm getting the value of influencing the language models. Now let's let that sink in. So a foreign actor, you know, you think about like, uh, you know, somebody who has, uh,

• 58:48Jason Calacanis?

  an axe to grind from the Middle East or an axe to grind from China or North Korea, whatever it is. And any, any political bias, you could send a hundred really smart people, you could have literally a hundred people. Where, yeah, you could have 10 smart people getting paid a hundred thousand dollars a year in a war room for a million dollars a year. With 50 different accounts each, rotating them, you know, with a turret display. And then feeding the L L Ms their biases. And now they're going to in real time, because I think OpenAI has a real time deal with Reddit and Reddit has Reddit answers. Just using Reddit as an example, you know, are you going to be able to trust the data on Reddit? I would trust it before L L Ms existed. But I don't know if I'm going to trust it after LLMs existed.

• 59:32Speaker 01

  Well, and that's kind of almost like why you need LLMs though, J Cal, right? Because you need LLMs to look at not just those hundred accounts, but maybe a thousand, right? And look at more, because that's the advantage of, of the era that we're going into is that you can have sort of a infinite number of parallel agents looking at the data and then aggregating it all, right? And then getting, trying to get to the ground source of truth.

• 59:53Jason Calacanis?

  I think it's like, there, there are dozens of accounts that reach true influence on these platforms, I think. Like it's low hundreds. So I don't think it's that difficult to shape reality.

• 1:00:04Speaker 01

  To shape the, shape the message.

• 1:00:06Jason Calacanis?

  Shape the message. Just look at Wikipedia. My Wikipedia page, other Wikipedia page, there is a small number of people who are really influential. Under a hundred who are the super editors of Wikipedia and some of them get paid covertly. And, you know, by PR firms and there's like a whole grift going on there in the backend. Just like the review systems on Amazon and other places. I don't know that these LLMs are going to figure out who the bad actors are and who are great content creators who slowly introduce bias.

• 1:00:35Speaker 01

  It's a good opportunity for a startup, though. Imagine to do that, like, you know, to look at all that and analyze that and look at real news sources. And that's an interesting idea. All right.

• 1:00:46Jason Calacanis?

  That's okay. Well, maybe we cut that out here and we make our own influence startup. Maybe we should leave this out and you and I should fund a startup, do covert intelligence operations for corporations and individuals. So we could start our own CIA. Interesting idea.

• 1:01:03Speaker 01

  Yeah.

• 1:01:04Jason Calacanis?

  All right. This has been another amazing episode of This Week in Startup.

• 1:01:08Speaker 01

  Back, J Cal.

• 1:01:09Jason Calacanis?

  It's great to have you back. Let's come back. I mean, I think every two weeks is the cadence that you need because you've got a lot going on. So I think maybe we, we could even be monthly for now, but you got to lock in because people love to get some deep modest take on everything and you're on the inside. That's what the show is all about. Insider sharing with other insiders to catch you up. If you missed this, you're going to fall behind, folks. These are the important discussions of our time. Thisweekinstartups.com. If you want to search the AI archive powered by our friends at Podcast AI, they're not a sponsor, but I am an investor. If you would like me to invest in your company and Sunny's an LP of mine, all you have to do is go to founder.university if you have an idea with two or three friends and you're in year zero or one. Theacceleratorlaunch.co/apply. Launch.co/apply.

• 1:01:53Jason Calacanis?

  I'm saving up to get the M to get the launch.com. I think our friend, uh, friends at Yahoo still own it. All right, everybody, we'll see you next time. Bye bye.