SO MANY THINGS need to go right just so you can watch a TikTok! | E2215

Register here to join Founder University Japan's kickoff! 👉🏻 https://luma.com/cm0x90mkToday's show: On TWiST, Alex is chatting with Neubird CEO Gou Rao about the intricate complexity of IT Ops, and the vast amount of layers and components that have to be delivered very quickly to your phone in order to stream a simple video clip. Neubird has developed Hawkeye, an AI "engineer" that uses "context engineering" to isolate and troubleshoot problems, while still keeping costs down. Loading up a video to show your friends may SOUND straightforward, but it's actually, as Alex notes, a miracle anything on the internet works at all. PLUS we've got Subject AI CEO Michael Vilardo stopping by to tell us how the app uses gamification to keep kids learning and engaged. NOTE TO FOUNDERS: Mike got invited on the show because he sends email updates to his "dream investors," including Jason. It's a solid best practice! Finally, we're flashing back to a remarkably prescient 2019 interview with then-Scale AI chief Alexandr Wang, who tells us why he thinks the AI Doomsday scenarios — AND the elaborate AGI promises — are both way overblown. Timestamps:(3:19) Neubird asks: What if we took AI tools and applied them to IT ops?(5:17) Why does IT ops seem to get so much more complicated over time?(6:14) Gou walks us through all the various components that need to be delivered when you post a simple TikTok video(8:41) Why agentic systems (like Hawkeye) are the ideal tool for monitoring metrics, logs, traces and alerts(10:05) CLA - Get started with CLA's CPAs, consultants, and wealth advisors now at https://claconnect.com/tech(11:17) How is IT Ops different from Site Reliability Engineering (SRE)?(12:36) Is Hawkeye an AI Agent or an "Engineer"? How we decide what to call our "co-pilot."(16:07) How "context engineering" helps Hawkeye isolate problems, and keeps inference costs down(17:27) Is Hawkeye plug and play? Or does it need to be trained on a specific system?(18:59) How Neubird measures Hawkeye's overall effectiveness(19:59) Perspective AI: Real insights, straight from your customers, and your first two months are on us. Just go to http://getperspective.ai/twist(21:47) Hawkeye can diagnose and make recommendations, or go in and fix the problem itself, if you like!(24:27) What Gou thinks about the pace of AI innovation: Has it slowed? Are we in a lull?(25:41) LLMs already have a lot of knowledge, but how do we enhance their EQ?(28:58) TIP FOR FOUNDERS: Send updates to your "dream" investors! That's how Mike got on the show!(30:20) Uber AI Solutions - Your trusted partner to get AI to work in the real world. Book a demo with them TODAY at https://Uber.com/twist⁠(31:21) The importance of branding… Why Mike pivoted to Subject.ai(33:43) How Subject helps bring teachers and students together for more 1-on-1 interaction(35:48) Mike walks us through how Subject works, and how gamification keeps kids engaged(39:56) Is anything lost when an AI grades a student's test, rather than a human?(42:09) Subject can serve as a "teacher of record" in classrooms(46:28) Why Mike and Subject AI are back in the office(47:40) TWIST FLASHBACK with Alexandr Wang of Scale AI from November 2019(53:10) Alexandr explains AI to Jason (back when it was still called "Machine Learning")(1:01:08) How AI will help people focus on "higher-value work"(1:04:48) Why Alexandr thinks apocalyptic AI scenarios are unrealistic(1:13:28) Are the big promises around AGI overblown?*Thank you to our partners:(10:05) CLA - Get started with CLA's CPAs, consultants, and wealth advisors now at https://claconnect.com/tech(19:59) Perspective AI: Real insights, straight from your customers, and your first two months are on us. Just go to http://getperspective.ai/twist(30:20) Uber AI Solutions - Your trusted partner to get AI to work in the real world. Book a demo with them TODAY at https://Uber.com/twist

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  You know what engineers do is troubleshoot, fix problems. They create problems and then they fix the problems they created. There's a lot of interest now in how to use AI, especially Gen AI, and how that can ease the burden of IT operations. The moment you get on a phone and you're looking at a TikTok video, I mean, the number of layers that are involved to deliver that, notwithstanding the physical infrastructure to deliver the audio and video content, but the infrastructure, software and physical infrastructure that it takes to store the media. It's a very complex environment and people want more of this faster.

• 0:33Speaker 01

  Just for fun, can you walk me through some of the software that goes into making a TikTok appear on my phone, for example?

• 0:39Speaker 00

  The videos have to be stored somewhere, so there's some quality control. So there are software associated with that. And then when the video gets edited and uploaded and stored, you want to make sure that there's no data loss, no integrity compromise on the video. And then the distribution aspect of it, then there's edge caching. You don't want somebody looking at the video to hit the back end server each time. And so there's what's called content distribution networks involved.

• 1:04Speaker 01

  This week in startups is brought to you by Uber. Bad data equals bad AI. Your AI is only as good as the data it learns from. Uber, that's right. Uber AI Solutions now works with enterprises around the world to source, label, evaluate and scale real world, high quality data for every industry, everywhere, so that you can focus on building the next big thing. High quality data equals smarter and faster AI. Uber.com/twist. Perspective AI. Surveys, they never capture what customers are really thinking. That's why we use Perspective AI. Real insights straight from your customers.

• 1:47Speaker 01

  And the first two months are on us. Just go to getperspective.ai/twist. And CLA. Innovation takes balance. Our CPAs, consultants and wealth advisors can help you get from startup to where you want to end up. Get started now at claconnect.com/tech.

• 2:11Speaker 01

  Hey everybody. Welcome back to Twist. And if you are in the United States tomorrow is Thanksgiving. So I presume you are listening to us on the plane, from the car, wherever you are. We're glad you're spending the day before the great holiday with us. Now, on the show today, Newbird, a Twist 500 company that is building what it calls an agentic AI SRE. What does that mean? Well, it means they're applying modern AI technology to helping companies ensure that their cloud setup actually works. As the cloud becomes more complicated, there's more wires and tubes to connect. And this company wants to make sure that you never get them crossed. Then Jason talks to Subject AI. It is a really cool company in the ed tech space. Like some others, it's helping students have more personalized educational tools, and also helping teachers get rid

• 2:55Speaker 01

  of the busy work. So that way they can spend more time one on one with the kids. Then to close off, we're doing a little bit of a flashback, all the way back to November of 2019, when Jason sat down with Alexander Wang, then the CEO of Scale AI, talking about the AI market and more. It is a fantastic time capsule and look into the prior era of AI. Let's have some fun.

• 3:19Speaker 01

  I am paying a lot of attention to applied AI. Sure, it's great to keep up with the newest models. But if you really want to understand the health of the AI market, I think you need to go and look where the rubber actually meets the road. So that's why we've talked to startups like Harvey, which is bringing AI into the legal realm. And today, we're talking to another startup working in applying AI, albeit in a very different realm. Newbird AI, that's N E U bird, asked the question, what if we took modern AI tools and applied them to IT ops? And if you want to know why that matters, well, just think back to the recent AWS outage. Well, maybe these things are very important to the running of the economy. Anyways, let's learn some more. Please join me in welcoming the show. It's Guo Rao, the CEO and co-founder of Newbird. Hey, how you doing?

• 4:03Speaker 00

  Doing well, Alex. Thanks for having me.

• 4:04Speaker 01

  Oh, my absolute pleasure. I have to say that when it comes to hands on experience in IT ops, I have precisely exactly zero. And I think a lot of folks out there, founders and all, probably are in the same boat. So maybe the best place to start, Gu, is just, why do we need an AI? I T operations engineer.

• 4:21Speaker 00

  Allow me to give you just a little bit of background. I'm an engineer. You know, what engineers do is troubleshoot, fix problems. They create problems and then they fix the problems they created. And creating and fixing the problems for some way, shape or form typically happens at two in the morning. And you don't want to wake up at two in the morning, waking up to an I T outage saying that the website's not working or the payment processing system is down and you can't take transactions. So this is I T computers are something that have moved us into the new frontier over the past, you know, 20 years and it's going to be more and more essential and critical to all of our lives. And so getting ahead of this problem and having infrastructure in place that can keep the operations running smoothly is of the utmost importance.

• 5:06Speaker 00

  So there's a lot of interest now in how to use A I, especially Gen A I, and how that can ease the burden of I T operations.

• 5:15Speaker 01

  Before we get into how you guys are doing that, though, is it correct that the state of I T operations is one of increasing ramping complexity? Because it feels like with the advent of like multi-cloud setups and so forth, there's just a lot more pipes to keep connected. And so my impression as a layman in this context is that this is probably a problem that's getting harder and harder as time goes along.

• 5:35Speaker 00

  That's exactly what's happening. As things mature and there's more demand for new and faster software, faster products, better features, complexity increases and people are moving at such a high velocity now, engineers in creating product that there are many layers of technology, many components that interact with each other to deliver the features that you like. The moment you get on a phone and you're looking at a TikTok video, I mean, the number of layers that are involved to deliver that, notwithstanding the physical infrastructure to deliver the audio and video content, but the infrastructure, software and physical infrastructure that it takes to store the media. It's a very complex environment and people want more of this faster.

• 6:14Speaker 01

  Just for fun, can you walk me through some of the software that goes into making a TikTok appear on my phone, for example? Because I think that's an interesting thought experiment to kind of talk through.

• 6:23Speaker 00

  First of all, there is the content creation and distribution aspect of things. The videos have to be stored somewhere. A lot of times this content. Is being curated by people that are brand specific, they have a tight control on their brand. They're not just putting any video up there. So there's some quality control. So there are software associated with that. And then when the video gets edited and uploaded and stored, you want to make sure that there's no data loss, no integrity compromise on the video. And then the distribution aspect of it, this now goes to people are watching this on their Verizon or T-Mobile. And audio and video format doesn't need to get delivered in. And then there's edge caching. You don't want somebody looking at the video to hit the backend server each time. And so there's what's called content distribution networks involved.

• 7:07Speaker 00

  You guys are all familiar with the CloudFlare outage that happened a couple of months ago. And so these things, there's many layers involved. A video blogger goes out there and says, I put this content out there, but my customers can't reach it. Is the problem with the cloud distribution network? Is the problem with you running out of space on your AWS account? Like, how do you even go troubleshooting these things? It's a complex problem, takes many engineers to get involved and troubleshoot these things. Whether you're an engineer at TikTok or you're supporting the blogger's website.

• 7:39Speaker 01

  Is it more surprising that the internet as we know it works today? Or is it more surprising when it breaks? Cause I feel like, given what you just described, I'm kind of shocked that everything stays online most of the time, Gu.

• 7:49Speaker 00

  It takes a lot of hard work to keep things running. It's great that the infrastructure works. Look, I'm surprised that any part of infrastructure that works, like when my train comes in on time, I'm really happy with it. Kudos to the people that are managing that. And so it's taken many years for us to perfect how to get trains running on time. And if you even look at the aircraft industry, right, or the airline industry, that's a huge logistics issue. And so people have perfected that. And now we're getting into moving away from manually keeping things online to using AI to keep these kind of systems working.

• 8:20Speaker 01

  But the complexity that we described generates so much information, logs essentially, that I presume that AI is a good tool to be a kind of a first set of eyes on all the information that's coming in. Because me, the individual human, even if I replicate myself and sit myself at 12 different desks, probably can't look at all the logs that are coming in from even a fraction of TikTok's overall. Digital footprint to stick with that one example.

• 8:42Speaker 00

  Hundred percent. That's where agentic systems come in. Let me define what an agentic system is. An agentic system is something that leverages gen AI to do what I think you called it applied AI, which is a really good term. So an agentic system takes generative AI and applies it to a very specific task domain, a problem domain. And in this case, we're talking about IT ops. And so to your point, in IT ops, what an engineer has to go through when they're debugging an issue is they're looking at a complex array of data sources. They're looking at logs, they're looking at metrics, they're looking at traces, they're looking at alerts. And each one of these have their own complex data structure complexities. And there's a lot of this data. Data has always been the Achilles heel for an enterprise. So yes, while people can do this,

• 9:26Speaker 00

  it is tedious, takes time and effort for people to go through logs, alerts, traces. AI can do this very quickly. All we do here at Newbird AI is exactly that. We leverage large language models, extract the reasoning capabilities, and interface them with the complex array of metrics, logs, alerts, traces, so that they can solve an IT issue much faster than humans can.

• 9:48Speaker 01

  And this product is called Hawkeye. You guys call it an AI IT ops engineer. I think you've also described it as an SRE in some capacity. Can you just explain to me the difference between IT ops and site reliability engineers? Why you, I think in one release said one, one release the other. This might be a small point, but I'm just intellectually curious.

• 10:04Jason Calacanis?

  One of the themes we talk about over and over again on this week in startups is making sure you do your chores. I'm no expert on these things. I have some experience. Stephen Estes from CLA is an expert. Let's talk about being cash efficient. Tell us about efficiency and what you see in the in the top tier startups in your practice.

• 10:25Speaker 03

  We're seeing kind of an interesting trend out there where companies aren't needing to raise quite as much as they had in the past. You really have to be careful as a founder to only take on as much money as you really need. You've got to do the forecast and you've got to do the modeling and you've got to dial it in and get it right. Otherwise, you're going to end up either not raising enough capital to get to where you're going and you're going to have to go get venture debt or go back, have an extender to the round, or you're going to give up too much. To the company because he just didn't recognize how much money actually needed.

• 10:54Jason Calacanis?

  Yeah, very important to get this stuff right, folks. And that's really a bummer when startups don't do things in a button up way. Always have a great partner. Good partner to have on this adventure while things change. My friend Steven over at CLA. Visit CLA connect.com slash tech and don't forget to mention that your boy J Cal sent you. That's CLA connect.com slash tech. Start today.

• 11:18Speaker 00

  Site reliability engineering was a coin turned by Google for very large enterprises where there are specific engineers focused on maintaining site operations. Now, depending on the size of the company you talk to, there are dedicated SREs that are completely focused on making sure that the IT operations are running up smooth. These are guys that respond to things like pager duty alerts, like it goes down, they get an alert, they have to go in and resolve the issue. Now, as you get into more mid-market, there's a blend. A lot of times engineers that are working on product or in DevOps. Share the role of maintaining their site operations. So they're doing double duty, so to speak. These products, agentic systems, address both use cases.

• 11:58Speaker 00

  So to talk about agentic systems in the context of site reliability engineering appeals to very large enterprises, but at the same time, we want to make sure that the same technology is helping mid-market as well.

• 12:09Speaker 01

  So essentially, I should think about SREs as very, very specialized IT ops engineers, normally at the largest enterprises. But if you're going to sell to the mid-market and up, you're going to want to speak to both the IT ops and SRE community.

• 12:20Speaker 00

  A hundred percent. A lot of times, you know, the engineering team that's writing code, they're on call duty. Like if the site goes down, this engineer, you have your project to do, you're working on your coding, but you're also on call support if something happens. And so you want this product to be able to address the needs for both.

• 12:37Speaker 01

  You guys describe Hawkeye as an engineer versus an agent. Is an engineer in this context kind of like three agents in one trench coat?

• 12:44Speaker 00

  The terms and how these things resonate with people, it's still evolving. Everybody has a different expectation on what the term is called, what it does. Is it a co-pilot? Is it an agent? Is it something autonomously? Am I interacting with it? These are all things for people to figure out how. They're going to live with agentic systems. We call it an engineer because Hawkeye initially, when it's deployed, it is autonomously and asynchronously figuring out what's going wrong with your IT ops and then figuring out what the issues are and creating remediation solutions. If somebody wants to interact with it and say, look, there's nothing going wrong now, but what if I were to do this? Can you go figure this out? People can do that, too. Do you call that an engineer? Do you call it an agent?

• 13:26Speaker 01

  Engineer is a specific term for a general category that I would say is distinct from agents. I think co-pilots, if you go back to the early days of the post-ChatGPT era, when Microsoft was talking about co-pilots, they're little assistants. They help you. I think agents to me feel more like single-shot tools that go out and do a thing like, I'm going to send my agent to Amazon to buy myself candles. The reason why I kind of like that you guys are not calling this an agent is that it does feel to be like a discrete set of systems that I can work with, but also work for me even when I'm not directly tasking them with a please go do A on site B, right? So it does feel different to me in a positive way.

• 14:01Speaker 00

  You hit the nail on the head, Alex. And look, this is a little fluid, but that's the feedback we're getting to. You're right. A co-pilot is like you have to distinctly carve out a body of work for it to go in and do, or you're asking it to cross-check the work that you've done, which is kind of not what's happening here. What we're really trying to do is streamline yours as an enterprise's IT operations where they're getting far fewer alerts, or like 80% of the alerts are being handled by Hawkeye and the remediation is done. It's automatically resolved. I'll give you a simplest use case. If somebody's wondering, like, a very large company here in the Bay Area that operates in the cloud, they run an AWS. Obviously, cost operations is a big issue

• 14:41Speaker 00

  for them. All they want Hawkeye to do is see if their today's spend is projected to be greater than 5% of yesterday's baseline, and if so, why?

• 14:52Speaker 01

  You mean of greater than 5% differential?

• 14:54Speaker 00

  Differential from yesterday's baseline. That's it. Simplest of use cases. You would think like, hmm, that sounds easy, but then you realize what engineers have to do. They have to log in, look at every instance that's launched, which instance changed size, or maybe it's Even instances, maybe somebody's consuming more storage capacity, or maybe somebody's doing a lot of egress traffic and downloading a lot of file. Like, it is not that easy. And most importantly, it's kind of boring at the end of the day when you find the issue. So you just kind of let these agents do this on their own.

• 15:27Speaker 01

  This seems to me expensive because when I think about using AI to do a lot of stuff, I just begin to think about inference costs. And you guys talk about having integrations with Elasticsearch, Prometheus, IBN, PagerDuty, MongoDB, Redis, Snowflake, everybody, which means that if I'm going to send out Hawkeye and I'm a big cloud user who has a lot of different things to keep an eye on, it feels like that Hawkeye would need to ingest so much information and process it that it would be relatively heavy or expensive to run. Am I over indexing on gross margin fears here? Or is it actually as expensive as I think?

• 15:58Speaker 00

  You are dialing in on one of the core problems that we need to solve for. Anybody can go in and download a whole bunch of logs, paste it into ChatGPT and say, what's wrong over here? What you've actually done is you've isolated the problem down to a certain area and you're asking for a solution. I'll answer your question. The term for that is called context engineering. Quite honestly, more context, while these large language models have very large context windows, then garbage out. To answer your question on cost, if a doctor were to charge you by minute, okay, just assume, and you go in unprepared and you say, I don't know what my problem is. It could be in my head, it could be in my back, it could be in my legs. Well, you're paying a lot of money. So what you will do then is before you go to the doctor, you're going to say, I'm going to ask myself, what is my biggest issue?

• 16:43Speaker 00

  Isolate it, go to the doctor and say, I have this problem in my knee right here when I touch it like this. And you'll get a very good answer from the doctor. That's called context engineering. That's all we do at Newbird. We specialize in context engineering because of two reasons. We don't want the customer to incur very large inference costs. More importantly, we don't want the LLMs to come up with garbage answers or what's called hallucinations. When you give a marketing document to a ChatGPT and say, I did this campaign last week, next week I'm going to reinvent, create a new campaign, it'll do a great job. But you can't do that with IT systems because you want Hawkeye to give you exactly what's going wrong with your website and why it's failing. Not a what if, not a possibility, but the exact.

• 17:28Speaker 01

  Does the system have to be trained on a specific company's setup? Or is it kind of something that you can plug into any cloud or IT environment? And it will automatically know how to look around and observe.

• 17:43Speaker 00

  The latter. It's a plug and play. And the reason for this is we believe these foundational models have seen so many different IT scenarios that the specific ailment you're facing is not something that hasn't ever happened before. We're not solving for that minutia of possibilities. So then all we have to do is take all this knowledge that exists in the LLMs and apply it to your context, your data. That's what we do here at Newbird. We are context engineers. We pick the right alerts, logs, metrics, traces, so that the LLMs can diagnose the problem using reasoning and we apply a different kind of reasoning methodologies. We have these models argue with each other, cross check each other, consult knowledge basis to see if the reason that it's coming up with

• 18:28Speaker 00

  is in the realm of something that's happened before. It's a complex system to build agentic systems, but it all relies on good context engineering.

• 18:35Speaker 01

  I've always felt that once we could get AI models to kind of fight amongst themselves as discrete entities, we would have some really positive friction that would yield better end results. And I think that's one reason why I'm so bullish on building up more compute capacity, because I think it would be great to be able to be wasteful with inference. Because then I think we can do quite a lot, but that's down the road. You guys launched Hawkeye, general availability like a year ago?

• 18:59Speaker 00

  Yep, a year ago. Yeah.

• 19:00Speaker 01

  So sitting here 12 months later, goo, how's it going?

• 19:02Speaker 00

  Pretty good. Look, we intentionally launched this with limited availability so we could learn with key design partners. We picked enterprises of all shapes and sizes. Some very large logistic companies, some very large financial institutions, pharmaceutical companies, manufacturing, to all the way down to low end of the mid-market. Why? Because we want to see if the product resonates well with the whole spectrum. What type of problems do you have at the very large enterprise? A lot of. Noisy alerts at the very low end, they want even more surgical because they're more tied into DevOps. What part of my code change is actually causing this issue? And so we've learned a lot. We've had a lot of good success. What do I mean by that?

• 19:44Speaker 00

  We're achieving close to 90% plus reduction in the meantime to incident response and resolution from the RCAs that we're creating.

• 19:51Speaker 01

  Meantime of incident to response. So that means that if it used to take me 10 hours to get from problem to solution, it now takes me one.

• 19:59Jason Calacanis?

  You can't make your product better without listening to your customers. But how are you supposed to actually figure out what your customers want? You could send out a survey, of course, but people just want to give the right answer in that case. They're not really giving you their honest opinion. Well, at this week in startups, we actually found the perfect answer. It's called Perspective AI. Their expertly trained AI conducts one-on-one interviews with your users based on your prompts and questions. Just tell their system whatever you want feedback on in simple language and they take care of the rest. We've learned so much about our audience since teaming up with Perspective AI. Reginald, for example, is a founder who listens and wants more quick hit, faster paced content. And an anonymous guy from Canada gave us tons of helpful feedback about some audio syncing issues we've been having and we fixed them.

• 20:43Jason Calacanis?

  This is the kind of feedback that's invaluable when you're a founder who is product obsessed like you should be. Sign up today and get started in just a few minutes at getperspective.ai/twist and you'll get your first two months free. That's getperspective.ai/twist.

• 21:01Speaker 00

  Exactly. And by the way, that's not uncommon to, and it's not just time. It takes an army of people. Look, you can't imagine that any one engineer is an expert in networking and Kubernetes and storage. It's not possible. People have expertise in different areas. And thankfully, these LLMs sort of have cross domain expertise. They have a lot of knowledge, but they need to be told what to do. And that's where the context engineering comes from. In the hardest of problems, we've been told that Hawkeye is able to save people a lot of time. And at minimum, tell them all the things that it did look at, which are something that the engineer, the on-call engineer shouldn't look at, and dial into where the problem could be, even if. If Hawkeye doesn't have access to all the information, sometimes it doesn't. Looks like I've taken care of all of this.

• 21:46Speaker 00

  This area, I don't have visibility. Focus on this.

• 21:48Speaker 01

  Does Hawkeye recommend methods of resolving the issue? Or can't actually go in there and make the changes or updates, corrections that I need to resolve it autonomously?

• 21:58Speaker 00

  Both. So it depends on the customer's comfort level. Typically, this has been the case when people deploy Hawkeye, they're looking at it from a consultation perspective. Tell me what the remediation is. Give me a root cause. They will create their remediation report, code change or corrective actions. But people may not give it right access to take the action. And it can submit a PR and somebody would have to hit approve. That's normally the case. In some areas where feature flags and things like that are involved, because that's low hanging fruit. People will just, Hawkeye, go take the feature flag, enable or disable it.

• 22:30Speaker 01

  But it sounds like maybe a good way to track how smart our overall AI intelligence is becoming is as more and more Newbird customers allow Hawkeye to make those changes. It'll imply that there's increasing base model intelligence and also increasingly sophisticated applied AI techniques that you guys decide to make this entire thing feel more like magic and less like integrated systems.

• 22:51Speaker 00

  A hundred percent, Alex. And the other thing that I'll say here for your audience, and I'm not talking specifically about Newbird. I would imagine that this is on in all facets because I think agentic systems are going to be a thing in the enterprise, not just in IT, but in all facets. Marketing, sales. And people should treat agentic systems as not, you know, off the shelf software where you have a known outcome. You should treat it like it has a persona because these things are built out of human knowledge. So what do I mean by this? You have a chance to interact with it. So it's not a binary answer. You're not evaluating an agentic system for saying it's good or bad. There's a range in between. So a lot of times these agentic systems can come back with an answer. You have time to interact with it, give it feedback and say, go do this. I like what you did, but you fell short here. Go make these changes.

• 23:36Speaker 00

  It is like another employee. You asked me how have things been going? I've had evaluations come in where an SRE manager came to me and said, your product is doing really good. Hawkeye came up with an answer, but my SRE disagreed with it. Okay. Turns out both were correct. Hawkeye found an issue, which was correct. SRE found an issue and the SRE had never thought of that. So it offers you different insights into your environment. It is bringing diversity. It's bringing a different chain of thought because it's been trained from so many different scenarios. And, you know, the manager was pleasantly really happy with that. And ultimately solving both issues improved the side ops multiple fold.

• 24:16Speaker 01

  I think it's funny how we go from, you know, deterministic computing systems to more probabilistic. We do have to do more talking to it like, great, try again. I spend a lot of time with my Chao GPT instance being like, excellent. Now please do it without being lazy this time. Thank you. Very, very fundamentally, are we still seeing the pace of AI model improvement that you were hoping to see? Because it does feel a little bit like from where I sit that we are seeing improvements in Chinese open source models. We're seeing some improvements in American and European closed source models. But it doesn't feel as fast goo as it used to. And I'm curious if I'm just wrong about that, or if you guys are also kind of noticing a bit of a.

• 24:54Speaker 00

  No, no, it's a, at some point, everything starts to plateau. And so I think you're acknowledging the fact that the rate of new features is starting to reach limits. And I think this is expected in any new system. You're evolving so fast and you're putting out features so quickly and you're training these models. And when you have billions of parameters that these models have, it takes time to train them. But then the number of parameters are limited. And at some point also having more parameters may not even make sense. I mean, you may just start getting weird answers. So in that sense, the knowledge in the models is reaching at this point, some sort of convergence or and it'll keep increasing. But the rate of increase, I think it's expected to not be at that same angle of trajectory that it was.

• 25:39Speaker 01

  Well, that's too bad. I was really hoping it was going to keep going straight up for a while there because I'm, I could use some more free intelligence to kind of layer on to myself, you know? I would love to be smarter.

• 25:48Speaker 00

  But see, the intelligence is going to come from external systems is what I'm saying. I think there's enough knowledge in these models to do a lot of good work. Now it's on external tools and external context. We have these things called MCP.

• 25:59Speaker 01

  Model context protocol. It's an anthropic project.

• 26:02Speaker 00

  The core of the brain, if it's steady, then augmenting its intelligence or sort of call it if it has enough IQ, let's get it EQ, which is the external knowledge that it would need to do awesome tasks. And that's where I think the next two or three years in AI, at least as it comes to, as you put it, applied AI is going to be around building the context and the support system around this really great set of brains that we have.

• 26:29Speaker 01

  One last question, Goo, before I let you go. We talked about there being a problem. Hawkeye notices it, figures out what's going wrong, and maybe either suggests or executes a remediation. Are you guys going to eventually go in front of the issues and begin to do like proactive scanning and checking to see what might break in someone's IT infrastructure? Because I feel like that would be a natural next step, but I may again not be not be correct.

• 26:52Speaker 00

  No, we actually do that. When somebody interfaces with Hawkeye, there's three phases to their journey. The first phase actually is retrospective, the opposite of what you said. Hey, I want to try Hawkeye. Last week I had an issue. I did what every person on the planet does. I rebooted my cluster because I don't know what the issue is. I want to get out of it. I got out of the issue by rebooting. What was the actual issue? So retrospective analysis. And then Hawkeye will say, I'm glad you got out of the issue, but this is what you should have done. Number two, real time analysis. Your how is my environment running? I have an alarm. What's the issue? And that's the primary use case. Big use case for Hawkeye is what you said, which is trends. What could happen if things are progressing the way it's progressing today, what will happen next week? A very good

• 27:36Speaker 00

  use case for that. A customer will say, I'm planning to roll out new software. Here are my changes. Is my environment ready to absorb these changes?

• 27:45Speaker 01

  Hawkeye will will tell you, hell, no, it's not. You got a lot of work to do.

• 27:48Speaker 00

  Yeah, it could be that, look, you have this much reserved memory capacity, but your pod spec is requesting more capacity than you have allocated. Your pods will fail to start. Sounds like reasonable answer to me.

• 28:00Speaker 01

  Sounds like a very reasonable answer to me. It's newbird.ai. If you want to go check it out, N E U bird.ai. And Goo, you promised to tell me when you're closing your next major round of capital before we left. So I'll leave that to you.

• 28:11Speaker 00

  We are very well funded. Fortunate to have great investors. We're funded by Mayfield, Microsoft. We have enough money in the bank that we don't need to, but then, you know, we're always talking to investors and there are strategic relationships that we're forging without disclosing anything. I think there will be some good news down the line soon.

• 28:30Speaker 01

  All right. Well, we'll see you on January 15th, give or take. That's what that sounds like to me. Go, an absolute treat. We'll talk to you soon.

• 28:35Speaker 00

  Thank you so much, Alex.

• 28:38Speaker 01

  All right. Next up on the show, we have Mike Vlado from Subject AI. Jason, this is an AI education company started off by making courses for online consumption, but has since pivoted into more tools for the classroom. We've talked to other companies in the space like Magic School, but this one is blowing up. So let's talk to Mike. Mike, welcome to the show.

• 28:55Speaker 04

  Hey, thank you so much for having me. Longtime listener, first time caller. Excited to be here today.

• 28:59Jason Calacanis?

  Great note for startup founders. Michael, for some really intelligent reason, decided to put me on his updates to investors. This is a really great technique for founders. We train people to do this at the launch accelerator, which is you make two lists. Your investors who you want to keep updated. So that's your investor update and then your dream investors. These can be the same email or slightly different with your existing investors. You might say, Alex, hey, here's where we're going and here's some challenges. This is the problem. Here's the legal issue we're dealing with. Okay, right. We're getting sued by a former employee. We got a patent dispute. Okay, great. That's for the internal, like 20 investors, right?

• 29:41Jason Calacanis?

  Then there's the prospective investors. Now you did, this is just where you shine.

• 29:45Speaker 04

  Right.

• 29:45Jason Calacanis?

  Hey, this is the third quarter in a row that we've had 20% on average growth. We are now on pace to hit 5 million in AR, 10 million in AR by this year. And we have cash in the bank and we've just reached infinite runway because we're profitable. And then somebody like me gets this email. Now I'm BCC on it. And I'm like, wait a second. I've invested in 600 companies. Did I invest in Michael's company? Oh, no. Then I do a search and there was like an email from Michael pitching me that I never got back to from four years ago or a DM. But anyway, well done, Michael. You, you've been doing this technique for some time or is this the first time you did it? With AI, like everything else, the Saying is so true garbage in garbage out without the right training data you're not going

• 30:26Jason Calacanis?

  to get great results but my pals at Uber they're now working with enterprises all over the world to source label evaluate and scale high quality real data for your startup Uber is one of my favorite companies in the planning because they demonstrated their ability to scale and to build great products first in the rideshare market then they figured out Uber Eats and then they moved on to autonomous driving and all of this was only possible because of a deep understanding of how to collect label understand and analyze data now with Uber AI solutions you can put the team and tools that turned Uber into one of the world's best companies to work for you partnering with top talent and experts from Uber's global network what better

• 31:07Jason Calacanis?

  partner help you scale your business than the company that organizes over 36 million rides every single day book a demo right now by going to uber.com slash twist that's U B E R dot com slash

• 31:23Speaker 04

  I'm doing it a little bit more often but I do have to give you a shout out I believe we were mentioned on a previous twist episode when we rebranded from a meal learning to subject and got subject.com and you mentioned it as startup of the week and seven figure domain now we didn't pay that much but I do believe on the market it could be worth that so I always appreciated the way you gave us the nod then and that's when I actually reached out via Twitter or X and we're really excited that finally we're here together

• 31:46Jason Calacanis?

  how much did you pay how much you can't you can't say not

• 31:49Speaker 04

  undisclosed amount but uh you know uh low six figures so we're really excited about that

• 31:54Jason Calacanis?

  three or 400 grand for that is a pretty good deal I have to begin right now

• 31:57Speaker 04

  less than that I can confirm so less than that and we also have subject.ai now as well so we really take a lot of pride in our branding and name recognition and you know we want to be synonymous with elite premium education for any age in any part of the world

• 32:10Jason Calacanis?

  hey so why don't you show us what you're building you can share your screen if you like and just walk us through the website or the product I don't know if we uh give you a heads up to do that but might be nice to just show the product a little bit and what's working because I know you started pre-AI and then you you wound up here with building AI tools right so yeah tell us what you're working on

• 32:27Speaker 04

  I could quickly voiceover and really give you up to speed so we were really the Netflix of I started this during my MBA experience. It was 2019. I was at UCLA, brought me to Los Angeles. 2020, I got thrown into Zoom University, the worst experience of my life. Seventy grand a year to sit on a Zoom lecture like this. Everyone, you know, had their screen blocked out, slacking each other of the answers. And, you know, in Los Angeles, it was pretty extreme lockdown. Couldn't even eat outside at a restaurant for some period. So what did you do after six, seven hours of Zoom? Well, you watched Tiger King, Last Dance, all these Netflix shows. That was the saving grace and got us through that time period. So my co-founder and I really said, hey, how can we make education feel like Netflix? How can we help students be engaged? I'm a millennial. Imagine a Gen Z learner, or now Gen Alpha, in the middle of North Dakota and Nebraska.

• 33:12Speaker 04

  They might not have nearly the support that UCLA was providing me. And so that was our original passion, premium cinematic quality. I'm in our office in Los Angeles. We have four production studios here, 10,000 square feet where we do all our filming. But then, since the launch of AI, we've really graduated past that and really providing much more adept personalized tools for students and teachers. Save them thousands of hours a year and allow all of them to feel like they're learning from their favorite teacher every single day in the classroom.

• 33:36Jason Calacanis?

  So in the same way, Waymo and Uber and Neuro are trying to make like the perfect driver. You're trying to make the perfect teacher with AI, yeah?

• 33:45Speaker 04

  A hundred percent. I'm really here to amplify the heroes of society, which are teachers and coaches. You know, I had a lot of great teachers and coaches growing up, and our goal is to give teachers the time to spend with their students in person, making magical moments happen. And, you know, we're seeing that firsthand now with Subject AI. They're able to service three times the amount of students on a daily basis by using our product. And that allows for them to have more one-on-one interactions like this, while Subject AI can be the core experience in the classroom.

• 34:09Jason Calacanis?

  This used to be called adaptive learning in the industry. It was like really hard to code, et cetera, but it's gotten easier, yeah? To do adaptive learning?

• 34:18Speaker 04

  Yeah, adaptive learning, or commonly referred to also as flipped classroom, where, you know, you walk into a classroom and every student is engaging with subject videography, personalized AI tooling. And then the teacher is doing small group or one-on-one instruction with students who are behind or way ahead, making it much more impactful. And actually, on the Andreessen Horowitz podcast, Mark and Ben have a really great episode on education, and they talk about this. It's very commonly known in the industry. The highest impact on a student's journey is how much one-on-one tutoring time they get. The problem is, most people can't afford one-on-one tutoring. But when you work within the public school system and you allow subject to be that core methodology in the classroom, now every student has a chance of getting one-on-one time with every one of their teachers.

• 34:57Jason Calacanis?

  So flip classroom is the technical term. You learn the repetitive stuff on your laptop. And then, when you're stuck or you're excelling, then the teacher comes and says, oh yeah, hey, you're way far ahead. Here's some additional coursework for you to go on. Or if you're behind, let's get, let's break you through whatever the blocker is, yeah.

• 35:16Speaker 04

  Oh, a hundred percent. I mean, think about it from a first principal standpoint. Why should a teacher be doing the same lecture three times a day, year over year? And how much variance is there? You know, depending on your previous day, anything going on in your personal life, the teacher may be performing at their best or worse on that given day. How can we give the teacher the time to focus on that one-on-one instruction where they love it? They get so much energy from talking to students in person. And the mundane, as well as the grading and feedback is all taken care of by AI. And there's no excuse in this world of AI now to not be able to provide these teachers much more supports. So they can have a much better work experience as well.

• 35:48Jason Calacanis?

  Can you just show us? Cause I got, yeah, I want to see it.

• 35:51Speaker 04

  You can see it's very much like a Spotify Netflix-esque vibe when you go through the LMS here. We really wanted to always feel like premium consumer technology or social media. Our goal is to bring the best trends from those industries into education. And you know, we did the shout out earlier about you being an Uber investor, me being an Uber, ex Uber founder. You know, that's what Travis and Emil, who are huge, huge inspirations for me. Emil's actually one of our investors. They brought the best people to work on transportation across all industries. Our goal is to do that in education and we're super excited about that. You could also turn into light mode with a lot of the teachers like when they put on a projection screen. But we'll go into dark mode here. So we're super excited about the product right now. So you could quickly see here how we're able to engage students like never before. This is all premium cinematography filmed in our LA studios.

• 36:35Speaker 04

  You know, most of the legacy players in our space are either no videography or leveraging 30 plus minute videos. Everything we do is short form, you know, especially Gen Z and Gen Alpha. They're looking for 30 seconds to a minute clip. All of our clips are around five minutes or less now, and you could see a quick 10 second clip here.

• 36:53Speaker 05

  Hello again. I'm Miss Holly and I'm back to discuss two important ideas to help us understand stories better. The central idea.

• 37:01Speaker 04

  Really engaging teachers and videography, but videos are only a small part of the course. As you can see on our left-hand panel, this is what a typical course will look like. We'll have one video and then a variety of quizzes, check for understanding and exams. That way you could be able to take the entire course asynchronous and use it in a live classroom setting to be able to get through the curriculum. One of the most exciting pieces though with AI now is now we have all of our courses have video games, which really engages the students. Right now, 96% of our students complete at least one course on subject. That's record high engagement in the industry. And so that means when a student starts at subject.com or subject.ai, 96% of the time, they complete at least one course. And these new AI video games are a huge driver of this.

• 37:39Jason Calacanis?

  All right, so you're doing a, a video game of some type to teach storytelling and fiction?

• 37:49Speaker 04

  Yes.

• 37:51Jason Calacanis?

  Got it. Wow. And these are all made with AI, I take it, or you're just?

• 37:55Speaker 04

  Everything with AI.

• 37:57Jason Calacanis?

  It's wild.

• 37:58Speaker 04

  Oh, I got it wrong. So that's not a good look for me. And I'll quickly show you one last teacher tool so that way you could get a good snippet of that before we jump back in. You know, the educator portal and everything we do for educators is absolutely critical for our success. We live to serve teachers. We live to serve teachers and students, but teachers are the core methodology we're able to get through to these students. When you work with a district, it's very uncommon that a student would, you know, revolt to a school and say, hey, if we don't get subject, I'm dropping out. Typically not a lot of agency around the student voice. The key is to have teachers say, hey, my 20 students are crushing on subject. You need to get subject memberships for your students as well. And so we're really big on empowering teachers, but this is all done with AI. And this is why we're saving so much time for teachers today. We do all the grading, lesson planning and scoring and feedback for them in our console here. And so quickly you could see which students aren't passing their course.

• 38:43Speaker 04

  Within seconds, you're able to see, oh, wow, I have five pages of students not passing. How can I do something in class today to help?

• 38:53Jason Calacanis?

  So it's really got the whole classroom information, data across all of their project progress in the adaptive learning. And you're just using a chat interface for the teacher to solve these problems.

• 39:07Speaker 04

  A hundred percent. And this is what's really powerful as well because you know, we are in a highly regulated industry. Every state has its own nuance. You could be able to go into our product and get state specific suggestions, scoring and feedback, lesson planning and worksheets within seconds. And so we're really excited about the advent of AI there and how we've been able to really accelerate our growth. So started with the premium videos, made them more short form every single year. And now everything we're really doing from here on out has been AI native. We really made the transition in early 25. We launched subject AI in June of this year. Big inflection point for subject.

• 39:37Speaker 01

  That's really, really, really interesting. I'm kind of blown away how in-depth it is. It feels like you've replaced the entire school framework with a series of videos, quizzes and so forth. That's highly automated. Does this mean that individual teachers can therefore better serve more students at once? Cause I feel like if they're just focusing on one to one, the actual kind of quote, quote, class size becomes less, less meaningful.

• 39:59Speaker 04

  Well, so what we're really excited about, and we see this in the data. One, we want our students to spend more and more time on the product because that shows they love it. And right now our students are spending around three hours a day on the product. They love subject, but teachers, we want them to spend less time on the product. We want them to be with the students in person. We want them to be hanging out, making those magical in-person experience moments happen. And so while we've seen teacher usage stay under an hour over the last two years, we've seen triple the amount of students served. That's exactly what we're going for. Higher leverage, more opportunity for teachers to have better access to student in-person experience. And let us do all the mundane admin work. Let us do the grading. Let us do the scoring and feedback. Let you do what you love.

• 40:37Speaker 01

  That's actually the only thing you've said thus far that I, that I've kind of like freaked out about a little bit is the grading element of this because. Well, actually, maybe the better question to ask is what are the underlying models powering the teacher's side of subject AI?

• 40:49Speaker 04

  We've used a variety of models right now. We're leveraging Claude, which has been fantastic for us. But you know, we've used chat GPT Gemini. We're constantly looking to be able to see what's the best and we have easily and you know, on our code base being the ability to swap in and swap out.

• 41:02Speaker 01

  Yeah.

• 41:02Speaker 04

  I think the big thing for us and you know, especially when you look at legacy competitors in our space, uh, you know, in courseware and such, a lot of them have had the thing of being clicked for credit. Uh, you know, a lot of multiple choice, true, false answers. We never wanted that adage with subject. And so previously we were having, you know, a teacher having to grade so many of these open form essay responses.

• 41:21Speaker 01

  Yeah.

• 41:21Speaker 04

  Now with AI, this is speeding things up and allows for us to have a lot of authentic assignments while keeping up math speed and delivery to these students to get instantaneous feedback.

• 41:29Speaker 01

  Is there anything lost though, when, when Claude grades my assignment versus the teacher who knows me and perhaps even the co- I can imagine a teacher's syllabus being partially verbal and not entirely written down and therefore ingestible into the AI model itself. And so like, therefore the grading system might not entirely match up with what the teacher wants. It just seems like a place where I personally, I wouldn't put as much AI versus on the, the teaching side.

• 41:53Speaker 04

  Yeah. So when we do digital curriculum, everything we do is a recommended grade. And so the teacher has the final say no matter what. When we do teacher record, we typically are the final grade, but there's still always opportunity for them to amend, edit the grade if they have some feedback for us. And so we're always working with our districts. At the end of the day, the customer is always right. And so whatever the district wants to be able to do for their students, we're here to serve them.

• 42:12Speaker 01

  Can you actually explain teacher of record to folks? Because I don't think that's actually a term of art that most of our listeners know.

• 42:17Speaker 04

  Totally. And yeah, especially any industry jargon, acronyms, et cetera. It's always like, okay, uh, let's explain it to the greater masses. You know, when I first started in this industry, I had no idea what these terms meant. So basically we're accredited by two major governing bodies, WASC, Western Association of Schools and Colleges. This is very common. You know, Stanford, USC, uh, Cal, they're all WASC accredited. There's six regional bodies across the US. And then we're also Cognia accredited, which is the international accreditation that took, you know, many years to get, and we're really proud of that. With those accreditations, we have the ability to service as teacher record for our districts. And that means that without a credential teacher in the class. Classroom, we could step in on the backend and provide a credentialed experience through subject.com or subject.ai. And so with AI, that's made it a lot more scalable and high

• 43:01Speaker 04

  leverage. And so when they do teacher record, it's typically not a credential teacher in the district. We're providing the teacher of record credential instructor on the backend.

• 43:09Speaker 01

  And this allows for AP classes and such that might have a low student count, but then they can be brought to schools that wouldn't otherwise be able to support them was my understanding of.

• 43:17Speaker 04

  Yeah, I mean, even though we were the first cryptocurrency accredited class in the country, that's exciting. Taught by Chad Copeland, Brandon Copeland, financial literacy is a big one as well. It's a high usage for us. Brandon Copeland teaches the class. He teaches that same class at UPenn. So you have an Ivy League instructor teaching the class for high school students across the US. We're super excited. But then you also see, and this is from a really passion moment. You know, I came from a small town, Cary, Illinois. A lot of small town schools, it's hard to fill all, especially STEM courses, science and math. And so when you lose a science teacher in a rural community, it's very hard to backfill that. And so we can step in and help support that district by providing a four-credit course with our teacher of record AI.

• 43:54Speaker 01

  What's the impact on student ability to learn and be college ready? There's been a lot of reporting lately about how people get into pick your brand new school with the unable to like divide fractions and very basic things that I thought we'd handled. How much can you guys help get our graduating high school students up to snuff?

• 44:11Speaker 04

  I feel very confident in our ability. I think that's the big transition we're going through with education technology in general is how can we be able to be more part of the solution and making sure all the rigor is behind it and not allowing students to easily cheat their way through platforms. And so that's why we take academic integrity to the most seriousness and make sure that students aren't just passing. We don't want to be a credit mill. And then, you know, that's again, one of the negative terminologies in our industry. We want to make sure that students are actually learning and prepare for the next step. You know, there's over 40 million adults in the US who have college debt, but no degree. We never want to be a part of that opportunity. That is super aggravating to me. It motivates me every day to make sure we're helping students either go to top colleges or go direct into the workforce. And that's what we're doing now with the career technical education product that we're debuting in 2026. So hey, you're hearing it here first.

• 44:55Speaker 01

  So you're going to help people be essentially STEM ready out the gate. Wow, that's fantastic.

• 44:58Jason Calacanis?

  How do you make the decision to go sell into schools, which is a slow, arduous process? Versus just doing what MasterClass did or just go direct, like just go direct to consumers. How do you make that decision?

• 45:09Speaker 04

  So in 2021, we were all direct to consumer and we were only AP classes and we got a lot of great momentum. Raised a great round from Kleiner Perkins, Annie Case, our partner there, love working with her. Then over the summer, we saw a huge dip in signups because naturally direct to consumer and K12 education, a lot of folks aren't active in the summer. And so we were like, hey, do we want to always engage with this? Especially if you're part of K12, B2B is going to be the stickier business model. But also, which is really important from the mission standpoint, it's not just affecting affluent families. Whereas B2C AP was almost all affluent families. When you're selling into districts, you know, we work with some of the most poverty stricken areas of the country, which means the world to me, you know, like we're in some incredible districts.

• 45:45Jason Calacanis?

  How long do they take to like actually get on board?

• 45:48Speaker 04

  When we sell the product?

• 45:49Jason Calacanis?

  Is it a year sales cycle, two years? Cause there are, you know?

• 45:52Speaker 04

  In the beginning, and so we really struggled when we pivoted and you know, it was a couple of tough years, but those years made us stronger. And now we're seeing sales cycles of under six months, which is really great. But in the beginning, learning to sell to schools, especially being outside the industry, there was so much to learn. And we readily acknowledge that and we definitely had to really learn the hard way. But now the last couple of years, we have an incredible acceleration momentum. It just took us, you know, all of really 22 and 23 to understand the sales cycle and understand who even the buyer was. What were the key stakeholders? What was ICP? And now we have great alignment on it and we're full scale mode.

• 46:21Jason Calacanis?

  Awesome. Listen, continued success. Thanks for coming on the program. Everybody go check out subject.com or subject.ai, either one, any positions you're hiring for?

• 46:30Speaker 04

  So really talented folks who are willing to be in office five days a week in either Los Angeles or Austin. We have two offices now and we're super excited about that. And the biggest thing for us, I would say, as top level engineers and data folks, we really want to continue to improve that receive our data. And we believe that's the best for unlocking more sales and more product delivery. So data and engineers, please find our way. We have a lot of job openings ready to rock. We'd love to work with you.

• 46:53Jason Calacanis?

  You're an in-person guy.

• 46:54Speaker 04

  All about in-person.

• 46:55Jason Calacanis?

  Tell people why.

• 46:57Speaker 04

  We launched in 2020, it's a pretty sensitive HR environment, say the least with COVID going on and whatnot. And so we were very flexible for the first few years and you know, we struggled. We didn't, we didn't have a lot of momentum. We weren't scaling quick. And so I really said, hey, why don't we try just running the company? Our way rather than trying to please anyone and I come from a sports background and so you know hey when you're playing high level sports everyone's training together in person so we went full hardcore five days a week in office and the number skyrocketed and it's 100% not the right fit for everyone I'm totally empathetic of that but for subject if you want to work with us we're really passionate about building in person as a team and you know one of our core values is championship sports team mentality and so we love being able to build together.

• 47:36Speaker 01

  Love it awesome continued success and we'll see you next time.

• 47:43Speaker 01

  One of my favorite parts of being here at Twist is that we get to look back at figures that are currently making news back when we had them in the studio before they were famous before the tens of billions of dollars. One of those people is Alexander Wang. Now he was in charge of Scale AI and then eventually sold about half that company to Meta and became the new god of its super intelligence team. But back in 2019 on episode 1005, Jason had Alexander in the studio to talk about all things AI and keep in mind this is several years before ChatGPT came out. Please welcome back to the show. It's Lon Harris. Lon, we're going to do another flashback and this interview is a straight banger.

• 48:20Speaker 06

  Yeah, this one was great. I think what's so interesting about this is 2020, it was like I think spring 2020 was when all of these apps like ChatGPT and all the like stable diffusion, like the first generation of image to text ones were blowing up. To me, that feels like when all of us on like the ground level became aware of the AI revolution is real and it's happening and these apps are like they exist now and you can use them. So it's fascinating to go back to a time before that and hear people talk about what we were all going to be talking about within a few years, but really like none of us understood at all. Like Alexander at some points of this is sort of explaining to Jason like what is training and how do you train a machine learning system?

• 49:05Speaker 06

  And like, how do they learn? And all of this information is now embedded in all of our minds because we've been thinking about it every day for years. But back then it was like very theoretical. You could see Jason still like wrapping his mind around it in some ways.

• 49:19Speaker 01

  Oh, absolutely. I mean, asking the difference between AI and ML and where does the data come? Come from and what you need to do with the data. Can you share data between providers? Alex was like, absolutely not. But let's jump in, Lon. Where do you want to start?

• 49:30Speaker 06

  Right away, they talk about, you know, Alexander being only 22 years old, already raising 100 million. I like that Jason felt a little bit of kinship with him as a person who had also been a young Buck founder who got asked a lot about his age and being so precocious. So let's take a look at that.

• 49:48Jason Calacanis?

  I guess the thing that most people would think is remarkable is candidly that you've raised over $100 million in your last rounds of funding. That's a lot of money.

• 49:58Speaker 07

  Quite a bit.

• 49:59Jason Calacanis?

  It's quite a bit of money from Founder's Fund that I think you're 22 years old.

• 50:03Speaker 07

  22, yeah.

• 50:04Jason Calacanis?

  22. That's annoying to be young and successful because then every interview starts with your age.

• 50:09Speaker 07

  A little bit.

• 50:10Jason Calacanis?

  It's annoying. I had it happen to me, but with like 23 year old publisher of Cyber Server, 25 year old publisher of Silicon Valley. And I was like, why does my age matter? Now, I tell you, when you hit about 35, 40, they don't mention it anymore. Cause they're like, wow, you're 40, you should be doing interesting things or be successful in the world. But you've been running this company since you were how old?

• 50:30Speaker 07

  19.

• 50:31Jason Calacanis?

  19. How did you get into the game?

• 50:32Speaker 07

  I have a fun little history. I grew up in Los Alamos, New Mexico. So both my parents are physicists and they worked at the National Lab in Los Alamos.

• 50:40Jason Calacanis?

  Yeah. Tell people about that lab.

• 50:42Speaker 07

  It was the, it was the lab where the atomic bomb was originally built. So the Manhattan Project started in Los Alamos, it was very secretive at that time.

• 50:50Jason Calacanis?

  Yeah.

• 50:50Speaker 07

  And

• 50:51Jason Calacanis?

  What do they call that lab?

• 50:52Speaker 07

  Los Alamos National Lab.

• 50:53Jason Calacanis?

  Los Alamos National.

• 50:54Speaker 07

  It's pretty boring.

• 50:54Jason Calacanis?

  It's pretty boring.

• 50:55Speaker 07

  Yeah, yeah.

• 50:55Jason Calacanis?

  It's a government sponsored lab.

• 50:58Speaker 07

  Exactly. Yeah, totally government funded. And then growing up in high school, I did a bunch of programming. I did all these coding competitions. I was getting recruiter inbounds in high school. So after high school, I actually came out here to work. I worked at this company, Quora, for a couple of years.

• 51:11Jason Calacanis?

  Yeah, we know it. They do the Q and A site.

• 51:12Speaker 07

  Yeah, Q and A site.

• 51:13Jason Calacanis?

  How'd you get that job? You just applied and they saw your code and they were like, okay.

• 51:16Speaker 07

  They recruiter inbounds because I was an anonymous person on these coding competitions.

• 51:21Jason Calacanis?

  So you could just go into a coding competition? Nobody knows your age. Do you teach yourself how to code?

• 51:25Speaker 07

  I guess the internet.

• 51:26Jason Calacanis?

  You just looked it up? You found courses online on YouTube or?

• 51:30Speaker 07

  It's hard to remember. I think I just Googled around. Anyway, I worked at Core for a couple of years doing engineering, infrastructure, et cetera.

• 51:36Jason Calacanis?

  No college?

• 51:36Speaker 07

  No. Well, then I went to college after that.

• 51:39Jason Calacanis?

  Ah.

• 51:39Speaker 07

  I went to MIT and then got basically bored after a year and started, started scale.

• 51:43Jason Calacanis?

  So you left?

• 51:44Speaker 07

  I left, yeah.

• 51:45Speaker 01

  I remember Lon back when I was young enough to be considered a rising something or other. And now I'm just a middle, middle, middle, middle person. But Alexander Wang is still quite young, so his career trajectory kept going straight up. Lon, if I had given you a hundred million dollars when you were 22, how well would you have spent that money?

• 52:01Speaker 06

  Oh, not at all. I, I mean, this is always a fascinating thing for me, looking into the tech business where everybody gets started young. And it's all these like prodigal talents who started coding when they were 13. Like I had no idea who I wanted to be or what I wanted to do when I was Alexander Wang's age in this video. Probably still not even Alexander Wang's age today, but definitely not when he made this video. I was like, I just graduated college. I was floating around. Maybe I want to go to grad school. Maybe I want to be a journalist. Maybe I want to write screenplays. I don't know. I was working in post-production in Hollywood, doing subtitles for HBO DVDs when I was age age. So no, I, I don't think I would have been in a good position to raise a hundred million dollars and decide what to do with it. One more thing I wanted to pull up there from, from this opening, though.

• 52:45Speaker 06

  I thought a really fun little tidbit was Alexander comes by being a prodigally gifted STEM guy pretty naturally. Both of his parents are physicists who worked together at the Los Alamos Laboratory. That of course, famous if you've seen the film Oppenheimer, is where the Manhattan Project was and they were doing the Trinity Test. So, you know, he, he grew up in that kind of like a hard science environment around people who were doing like fascinating, high level research.

• 53:13Speaker 01

  Next up, Alexander explains how scale provides extremely accurate data to the autonomous vehicle companies. This interview Lon really spent a lot of time talking about self-driving because, you know, in the pre-JotGPT era, we didn't think a lot about, you know, chatbots and that sort of thing. AI agents weren't even a phrase. We were trying to get cars to stay on the road. Take a listen.

• 53:32Speaker 07

  The one that has really captivated the world's attention is autonomous vehicles, right? It's a compelling example because first, nobody likes driving, but also driving is very unsafe. There's a lot of risk in driving.

• 53:43Jason Calacanis?

  Sure.

• 53:43Speaker 07

  And so the

• 53:44Jason Calacanis?

  Lot at stake.

• 53:45Speaker 07

  Yeah, exactly. And so the captivating sort of machine learning model is one that can take in all of the camera data and other sensor data from the vehicle, understand everything that's going on around it, something that's very easy for you or I. But currently, or at least before machine learning was was very difficult for machines and then can determine the best path to take and and figure out how to drive on its own, basically.

• 54:06Jason Calacanis?

  Got it. So we see the lane markers, double yellow markers, double white markers on the highway. We know, keep the car between those two lines.

• 54:16Speaker 07

  Exactly.

• 54:16Jason Calacanis?

  As smoothly as possible. We see somebody deviate from their lane into ours. We know to slow down, give them some room. Maybe they're drunk. Machine doesn't know that inherently. We have to teach it that.

• 54:26Speaker 07

  Exactly.

• 54:27Jason Calacanis?

  What does scale.com do that Tesla and Waymo don't already do? Because they're solving that problem. Do they use your software and do they need to?

• 54:35Speaker 07

  The core problem, as as you just laid out, is that machines don't know what to do unless they have data that actually tells them what they're supposed to be doing, right? And so what it means is one of the the huge bottlenecks for machine learning is is data. Ends up being like data that tells these algorithms, tell these models what they're supposed to be doing. That's where scale comes in. What we are is sort of this this data refinery, if you will. We accept a bunch of raw data from our customers. We go through and process it and we sort of we tell the machine what it should be doing. For example, given an image taken by a self-driving car, we would outline, these are where the people are, these are where the cars are, these are the lane markings, et cetera. So that over time, these algorithms can learn those things.

• 55:16Jason Calacanis?

  And I see you are highlighting cars, you're highlighting people.

• 55:20Speaker 07

  Exactly.

• 55:21Jason Calacanis?

  And the machine is figuring out, okay, that's the approximate shape of a Dodge pickup truck, that's a Toyota Prius, and these look like the silhouettes of people. But that's a human telling the machine that's what it is, for now.

• 55:33Speaker 07

  Yeah, exactly. So the the core way that our whole pipeline works is that a lot of work is done behind the scenes by machines and our own our own AI models originally. And then humans basically give input and correct mistakes to make sure that that the end data is extremely accurate. Because that that ultimately is what's important for the safety of these systems and for for low bias, etc. All these things that are needed imperative for machine learning to be.

• 55:58Jason Calacanis?

  So you would go to a customer, so they would give you videos of their cars driving? And then you would annotate it for them and put that data into a database somehow?

• 56:08Speaker 07

  That's exactly right. So for example, if they gave if they gave us a video like this, you'll see originally, the first step was a human drawing a box.

• 56:15Jason Calacanis?

  Yep.

• 56:15Speaker 07

  And then a machine learning model that's already preprocessed through all this data has determined the the path of that vehicle over time.

• 56:22Jason Calacanis?

  Right.

• 56:23Speaker 07

  And then we confirm that all this is correct and then send that data over to the customer and they train machine learning models on top of it.

• 56:27Jason Calacanis?

  Got it. And this is how I guess one of the cars got fooled. Somebody drew on the ground like an arrow turning and a car followed the arrow, which our human would do too. But they basically drew a turning signal to see if like it would fool a self-driving car. And of course it did.

• 56:44Speaker 07

  I didn't see this news. I would believe that that is how that's what would happen, basically.

• 56:47Jason Calacanis?

  Yeah.

• 56:48Speaker 07

  Yeah.

• 56:48Jason Calacanis?

  And that's what would happen to a human, by the way. So I thought that was the stupidest prank ever. They're like, look, we can fool a machine that's driving cars to make a wrong turn.

• 56:56Speaker 07

  In a lot of ways, they will have they will have some of the same challenges that humans have in driving.

• 57:00Jason Calacanis?

  I was wondering, when do you think we'll have a self-driving car in a major city like San Francisco driving a major route? What year would this be possible? 2030 and over or under 2030? When do you think we'd first see this?

• 57:15Speaker 07

  This is the million-dollar question, which is when.

• 57:17Jason Calacanis?

  It's actually a trillion-dollar question.

• 57:18Speaker 07

  Trillion-dollar question.

• 57:19Jason Calacanis?

  Let's be let's be real.

• 57:20Speaker 07

  Where we're at, like fundamentally, the technology is getting better and better every year. The algorithms that that perceive their environments are getting a lot better, like asymptotically better every year. And then the algorithms that figure out what the car is supposed to do, the planning algorithm, etc., are also getting better. So it really is. It's sort of only a matter of time before we get to the point where these kinds of routes are possible and we'll we'll live in a safer world.

• 57:41Jason Calacanis?

  So that seems to me that you're. Thinking less than 10 years from now, this will be happening with regulation, all that counted in.

• 57:48Speaker 07

  I don't think regulation is necessarily going to be that tricky.

• 57:52Jason Calacanis?

  Why?

• 57:52Speaker 07

  There is precedent for like, if you think of when autopilots first came about, I think there's precedent for how to think about a lot of these things.

• 57:59Jason Calacanis?

  You mean autopilot in the airplane sense or in the Elon sense?

• 58:02Speaker 07

  In the airplane sense.

• 58:03Jason Calacanis?

  So the FAA and whatever regulatory bodies were like, okay, we get it. Autopilot works better than a human.

• 58:09Speaker 07

  Exactly.

• 58:10Jason Calacanis?

  It's pretty obvious. Plenty of room up there to operate when you're up at 30,000 feet. Lot less room to operate when you're going through the ten line. There's six people in the middle of the street, though.

• 58:18Speaker 07

  The challenges are a bit different, but I think there's precedent for how to think about these things. I think the technology, once it gets good enough, will be clearly extremely good. And so I don't think it'll be that big of a deal.

• 58:27Jason Calacanis?

  Andrew, so you think 10 years may be less?

• 58:29Speaker 07

  I'm excited for the self-driving future.

• 58:31Speaker 06

  What's so interesting here? I mean, obviously Jason having to be sort of told or instructed about here's how we're training these AI models, you know, like the cars record the video and then they send it to us and then we annotate it with all the like helpful things that the computers need to learn and then send it back. It's interesting just that Jason needs to be sort of like, this was probably the first time he encountered, you know, the differences between what Tesla's doing with their full self-driving and what Waymo is doing with their LiDAR. And like just thinking all this stuff out. Again, it's become so second nature for us today. We've all been in Waymos. We've all, you know, tried this sort of ad, or at least everybody in the tech industry sort of been thinking about it.

• 59:09Speaker 06

  Whereas it was still so new and they were on this, you know, horizon of this really exciting new time, which they then go to talk about like Jason asks him what Alexander says is the trillion dollar question, which is basically predict the future of self-driving cars.

• 59:23Speaker 01

  And the future is many players several years down the road, all doing incredibly well. Waymo expanded their geographic footprint recently. The Tesla robotaxis product is doing well. Zooks is in Vegas. Everyone's working with everybody and it's going very quickly. So this AI thing, I think, worked out Lon, all this hype from 2019.

• 59:41Speaker 06

  There is one other very prescient moment where they're talking about over the next 10 years. Of course, this was already six years ago. How often are we going to be in self-driving cars in 10 years? Alexander didn't want to like set it exactly. Didn't want to be nailed down, but one thing that he does say is that he predicts that regulation is not going to be that tricky, that like we have some precedent for the idea of self-driving vehicles. He mentions autopilot in planes, but you could also think of like trains or other things that are on tracks or monorails or whatever. At the time, that must have seemed like a very bold prediction. Like, you think cars are going to drive themselves around cities and cities aren't going to want to like obsessively monitor that for safety and they're not going to like regulate the hell out of that. And I think he's been

• 1:00:25Speaker 06

  sort of proved right. We're doing these pilot programs. People sort of roughly believe in this technology and trust it and they're they're able to sort of roll out these tests to more and more cities all the time and expand it. And there's some pushback, but I really don't feel like the regulation of self-driving cars has been super onerous and it's stopping them from collecting the data that they need.

• 1:00:45Speaker 01

  To the contrary, regulations for self-driving cars in the US are so lax that many Chinese self-driving companies are training here. So there you go.

• 1:00:52Speaker 06

  That was a pretty remarkable thing to predict in 2019, when I think most of us would have expected cities to crack down very hard on this sort of thing.

• 1:01:00Speaker 01

  Yeah, it turns out that humans are terrible drivers. And if we're going to allow 16-year-olds to drive tanks down the highways at 100 miles an hour, maybe letting the smart computers do it's a better idea. Long, the next thing I want to grab is speaking of prescience and kind of getting things right. The two of them discussed the importance of helping humans do higher value work. Now, when we talk today about AI agents, increasing model intelligence, we're always thinking about what's this going to replace in the human labor pool? Is it blue collar work, white collar work? But often we're seeing AI today turn up in very high economic value areas, coding, legal work, that sort of thing. And so when Alexander Wang discusses where he sees the AI market going, it's really interesting. Take a listen.

• 1:01:39Speaker 07

  I think there's like helping people focus on higher and higher value work. I mean, that's sort of like the core of human progress in some sense. I strongly believe will be the actual story of AI and machine learning. And it'll have to happen more and more and more for us to be comfortable with it. A great example is like truck driving. So there's there's a lot, all these automated truck driving companies. Yeah, lots. We work with a lot of them in BARK, ICH, etc. The naive view is that That hey, they're just they're going to automate truck drivers and like if you look at the map of the states like truck driving is a top profession in a lot of states and so seems really bad. But actually, if you look at the system as a whole, there's a shortage, there's a national shortage.

• 1:02:15Jason Calacanis?

  Of truck drivers.

• 1:02:16Speaker 07

  In truck drivers in the United States.

• 1:02:17Jason Calacanis?

  And the median age is like fifty or something crazy.

• 1:02:20Speaker 07

  There's this kind of like instability in the market because of all this stuff. The automated truck driving systems, actually what they would do is automate the the long haul middles.

• 1:02:29Jason Calacanis?

  Yep.

• 1:02:29Speaker 07

  Of these truck driving.

• 1:02:30Jason Calacanis?

  The boring parts.

• 1:02:31Speaker 07

  Which are the boring parts.

• 1:02:32Jason Calacanis?

  Arduous.

• 1:02:33Speaker 07

  That displace people from wherever their homes are, etc.

• 1:02:36Jason Calacanis?

  Yep.

• 1:02:36Speaker 07

  And allow the current truck drivers to focus on these like higher value trips that are sort of like warehouse to a meeting point or whatnot.

• 1:02:42Jason Calacanis?

  Yeah, drayage to the factory or even the last mile. I mean, who knows, like, maybe these trucks will change their form factor and be half the size, be automated. And when the truck gets off the road, the same truck, instead of using eighteen wheelers, we might just use smaller midsize trucks that'll be electric and solar powered. So you have more of them. When they get off, they become the delivery truck and they just automatically start delivering. Could be a much better model.

• 1:03:06Speaker 07

  These sort of like the introduction of machine learning to improve the efficiency of the economy, it'll be slow because of how in general free market economics work. It'll take effect in areas where there's an acute problem today. It'll happen in those places first and it'll allow the current jobs that exist to become higher value, more impactful.

• 1:03:25Speaker 06

  Yeah, I feel like his truck driving example here is really good. It's a very smart illustration of the larger point that I think makes it easy to sort of visualize. Because what he's arguing is, you know, they're not going to fire all the truck drivers overnight. It's not like AI is here, AI could drive the trucks. We don't need you anymore. It's like there are different kinds of truck driving. There's the kind of truck driving where you're in a straight line down the highway for like three days in a row. And then there's the more intricate, you're going through a neighborhood, you're going around some sharp turns. You're you're the last mile where the truck needs to go to actually make the delivery or what have you. And so he's saying, you know, like, well, AI would take part of that, the like long haul boring part that's sort of uneventful.

• 1:04:08Speaker 06

  You don't really need a human for it and that's the part that makes a human drive three. Days away from their house and then be away from their family for the whole week to get that haul done. And so he's like, well, you could have the AI do that part. And then local truck drivers take care of the more intricate parts where it helps to have the human. And they get to stay in the same place. So it's like AI makes everything better and takes care of the stuff that we don't want humans to have to do. And it's like, I don't, I don't know if that's actually how all of this stuff is going to play out. But that strikes me as like the best case scenario, like the utopian vision for how AI would integrate into our lives.

• 1:04:43Speaker 01

  I'm a little more pessimistic about human labor than that, but I'll take the bullish scenario. Why not? I want to grab one more thing from this from my end lawn. And then if you have anything else, we can go to that. But they discussed the dangers of AI. And what I found very interesting in this was just how much unbothered Alexander is. I mean, that really is a completely non joking way as a real positive. I mean, I, I do think we went through a period of AI excitement, a period of AI concern. And now mostly we're talking about economic concerns, you know, is the data center built out a bubble? But we're not talking as much about like, is AI being used an existential risk to humanity? And so to hear Jason, this is back, you know, five, six years ago, taking the tack that he did surprised me because this is not how he talks today. So take a listen to this.

• 1:05:27Speaker 07

  Well, we believe the true narrative will be extremely positive, actually, versus the current narrative, which is like AI and AGI, et cetera, are going to take over the world.

• 1:05:36Jason Calacanis?

  There is a possibility that AI could get out of control at a certain point with exponential computing. That's not far fetched that it could do something crazy and stupid.

• 1:05:44Speaker 07

  You only think that's not far fetched because you watched a lot of these sci fi movies.

• 1:05:48Jason Calacanis?

  If you were to train an AI to work on a drug to kill cancer and you didn't program it properly. Could create a drug that was too aggressive because you didn't tell it. Well, in the process of killing cancer, please don't make the person blind or all these other things. So you could just forget some edge case and some general AI might think, if you said to the general AI, you should work on things that make the human species better. And it goes, okay, yeah, let's kill cancer. And then it's like, oh yeah, let's cure this communicable disease. Great. The best way to cure a communicable disease is to kill everybody who have the disease currency. So it can't be communicated. This sounds far fetched, but. There will be instances where they will make the wrong decision, right? Or will be just too slow of a ramp up for us not to catch it.

• 1:06:29Speaker 07

  The thought experience always go like, oh, you'll make an errant comment to an AI. And all of a sudden it'll take over the world and do something that you really don't want it to do. I mean, in reality, like, there is a lot of oversight over these machine learning systems right now. There's like tens, hundreds of people who look at these models. They look at all the data that comes in and out. They like analyze everything and they try to figure out, okay, was this model doing well? Was it doing poorly and how do we adjust to that, et cetera? So I think that could happen in a world where it's like we have low oversight of these systems. So oversight is always important in any new technology, right? It's like when we started having airplane autopilot, for example, it would be crazy to just say, okay, we have airplane autopilot. Just let it fly.

• 1:07:05Jason Calacanis?

  Do we put any oversights over the Facebook and social media companies?

• 1:07:08Speaker 07

  They have, to be clear, they, they do.

• 1:07:10Jason Calacanis?

  They didn't have oversight. The FCC like giving a fine and the review mirrors oversight, it's not oversight. They had no oversight. What regulation is there of AI right now? There's none. You, you're, you're acting under zero regulatory environment right now. And China's got a negative regulatory environment.

• 1:07:23Speaker 07

  It's true that like.

• 1:07:24Jason Calacanis?

  So you should be regulated to your emission.

• 1:07:27Speaker 07

  No, no, no, no, that's not what I'm saying.

• 1:07:29Jason Calacanis?

  Well, wait, you just said that you should be regulated so that we don't have problems. So which is it?

• 1:07:33Speaker 07

  I do think that there are a lot of important issues about how we deem what AI systems are appropriate. How we look at what they're supposed to be doing, et cetera. I do think governing bodies, the US government in particular, for example, has to take a deep look, understand the technology, determine what is reasonable, what is not reasonable.

• 1:07:51Jason Calacanis?

  But even in their case, they're looking at the miles driven and the accidents. But they're not looking at the code that you guys are writing. They're not looking at anybody's code. They're not looking at the AI systems. They don't even have anybody on staff who could even write an algorithm, right?

• 1:08:03Speaker 07

  That's also changing, to be clear.

• 1:08:04Jason Calacanis?

  Is it?

• 1:08:05Speaker 07

  In general.

• 1:08:06Jason Calacanis?

  Do you think they're looking at any lines of code in, at any of these systems?

• 1:08:09Speaker 07

  I'm not sure about the answer to that, but I do think they look at a large amount of data. So for example, these.

• 1:08:13Jason Calacanis?

  Okay, they do, yeah.

• 1:08:14Speaker 07

  Yeah, in Europe, for example, there are all these ADAS systems, right? So there are these driver assistance programs or driver assistance systems and, and a lot of like high-end vehicles that you buy today, right?

• 1:08:24Jason Calacanis?

  Keep you in the lane.

• 1:08:25Speaker 07

  Yeah, exactly. They keep you in the lane. If you have like stopping or traffic, you don't need to do anything.

• 1:08:29Jason Calacanis?

  Yeah, adaptive cruise control, lane change warning.

• 1:08:33Speaker 07

  Systems exist. They, people buy these systems. People rely on these systems. And in the EU, for example, where a lot of these carmakers are, where BMW, Audi, VW, et cetera are, they have a responsibility to actually both have a large data set. They have collected themselves that is able to validate that these systems are performing well, as well as pass a series of sort of trials and actual.

• 1:08:54Jason Calacanis?

  Oh, really?

• 1:08:55Speaker 07

  Yeah, yeah. Different forms of data that governing bodies place in front of them.

• 1:08:59Jason Calacanis?

  Well, that would be very interesting. Now that I think about it, we do crash tests for cars. You're required to give three cars or something to the government for them to just destroy in their crash tests. But we don't require those cars to go into a lab, get taken over by the governing body, and force them to go into real world testing environments. Because there's some real world testing environment where you do self-driving up north here, I think.

• 1:09:19Speaker 07

  A lot of these companies buy cheap real estate. They outfit them into like these mini towns so they can create these funny scenarios.

• 1:09:25Jason Calacanis?

  Have you ever been to one of those?

• 1:09:26Speaker 07

  Uh, I've never been, but I've definitely seen the.

• 1:09:28Jason Calacanis?

  The video from there?

• 1:09:29Speaker 07

  Seen the videos, yeah.

• 1:09:29Jason Calacanis?

  Yeah, it's pretty cool. They have like children come darting out, like little cardboard cutouts of children to see if it hits it. This way they can do that in private. But it's interesting that at some point the government's going to have to have people who are developers and coders actually getting into the data and understanding some portion of this.

• 1:09:47Speaker 07

  At the very least, they'll have to create like the driver's test, for example, the driver's license test for a self-driving car. I mean, that will exist.

• 1:09:53Speaker 01

  And I think we've all just become a little bit less concerned about that because when's the last time you heard Jason talk about? I mean, Terminator.

• 1:10:00Speaker 06

  Or like they're going to use the AI to make like a biological weapon.

• 1:10:03Speaker 01

  That's going to blind you, yeah.

• 1:10:05Speaker 06

  We are definitely like less concerned about all that today. And I think, yeah, Alexander was, he was a bit ahead of the time on that. I enjoy the fact that he is a little bit skeptical of science fiction. I feel like so many of the tech visionaries of the moment love science fiction and obsess over it and try to make science fiction reality. I mean, I think Bezos and Elon Musk are great examples of guys who were like, you know, they grew up reading a lot of this sci-fi and a lot of their ideas about space and robots and AI and brain implants are like, I want to bring about Star Trek and make Star Trek's future happen. And I feel like a lot of what Alexander was saying was more. Like aware of the sci fi element of it, but a little bit skeptical like he's a little bit skeptical about AGI.

• 1:10:49Speaker 06

  And like we're, we, we think we're going to get there faster than we do. And, and he sort of put forth a, a very fascinating, very sci fi feeling. But he feels the most plausible path to AGI, which is you would create an artificial life form by simulating evolution in a computer. We can't just design the final form. We would design the like the cells or the whatever was in the pond for of AI. And then the computer would like naturally select that up to be like a life form, like a human that was sufficiently evolved. And like, you could tell a guy who's throwing around ideas like that has read his sci fi,

• 1:11:30Speaker 06

  but hasn't let it like, I'm going to cloud like this is, I'm going to just try to make what Robert Heinlein was envisioning. Like he's really thinking about it in a practical way and trying to integrate it into what we really know about science and technology. I thought that was fascinating.

• 1:11:44Speaker 01

  If you want more about this, the life cycle of software objects, a short story by Ted Chiang, which was put together in his latest collection of short stories is really well worth your time. It's funny how much there is a shared language law, thanks to our science fiction heritage. And I think it's, it's fun for me because I've read all the authors that are kind of in question here. So when they, they make allusions and references, it's kind of like my, my, my personal hobby. So that's a real banger.

• 1:12:07Speaker 06

  For the sci fi nerds, we should point out Ted Liang also wrote the story that became that movie Arrival.

• 1:12:11Speaker 01

  Yes, stories of our life. But the thing that I want to just double click on before we play this general AI clip is that he talks about how just more compute is not going to get us there. And I think that that's become less of a commonly held view in the, let's get the newest Nvidia chips and get 20,000 of them in one building and then push really hard. Meta in particular, XAI in particular have been investing a lot in building up their compute power as a way to create more intelligent systems. And it's funny to hear him here say that that's not going to get us all the way to the mountaintop.

• 1:12:43Speaker 06

  What a, a helpful thing. That made me feel less crazy because I feel like sometimes there is that assumption that we don't need to like code things anymore. We just. To put enough chips in a room and then they will figure it out for us. And to me, that's always, I'm not a programmer, folks. So like, maybe that's how it'll work. But that always felt very counterintuitive to me. Like, every other piece of software ever, we, a human had to like, figure out how it was gonna work. And now we're suddenly coming with this idea that, well, if you just put enough compute power together, it'll take care of all of that for us. And I've never really bought that. So it was, it was nice to hear a technologist agree with me in that way. Like, oh, I'm not crazy.

• 1:13:27Speaker 06

  This is a big assumption for us to make.

• 1:13:29Speaker 01

  It'd be fun to hear what he has to think about this now, but here's Alexander Wang.

• 1:13:32Jason Calacanis?

  You believe in general AI that will hit that at some point, AI that is just generally smart and do anything a human can?

• 1:13:38Speaker 07

  I believe in some sense. I believe in the sense that like, for most technological things that humans can conceive of that aren't physically impossible, if humans survive, they'll happen at some point. Like, I think humans are like infinitely creative, infinitely ingenious, et cetera. I think it's very overblown, the timelines that people are talking about general AI happening. There's a lot of things that are wrong about like the common arguments. One of which is people say that if Moore's Law keeps going, then we'll have all this exponential compute. It's only a matter of time before we produce these general AI. Moore's Law is, it's gonna be dead. And then quantum computing is very far away, despite recent press releases, et cetera. I think that leg of the argument is not actually that strong. And then I also think it's not even clear that if you have infinite compute, you'll be able to produce general AI. I think that's very unclear.

• 1:14:19Jason Calacanis?

  So infinite compute helps narrow AI because you're doing a number of scenarios and playing out every scenario in Go, the game. Many more permutations than poker, many more permutations than chess, which is a finite data set. So yeah, more compute power on those things certainly gets you quicker ability. Or even just throwing people into a random video game like OpenAI is, sure.

• 1:14:42Speaker 07

  Definitely.

• 1:14:43Jason Calacanis?

  But general AI, taking somebody who've mastered chess and then saying master Go and then master impressionist painting, it's different.

• 1:14:50Speaker 07

  It's very different. One of the arguments goes that once you have enough compute, you can create artificial life by basically simulating evolution. It's one of the more vogue arguments.

• 1:14:58Jason Calacanis?

  You have so much compute power that you can say, start with this tiny Piece of bacteria, whatever, then grow it and grow and grow an entire evolutionary system to the point at which there is a human-like species and then grow that human-like species in whatever number of scenarios with a big brain into whatever comes after us.

• 1:15:16Speaker 07

  Even if you just grow the human-like species that's as intelligent as us, then you're, that's kind of good.

• 1:15:20Jason Calacanis?

  That would be general AI as well, right? Because general AI would normally, most people would define general AI as not even being smarter than us, but being as smart. But somebody would have to code that and program that and build the systems to do that. It's not just going to magically happen, right?

• 1:15:31Speaker 07

  It's very unclear if that's even possible.

• 1:15:33Jason Calacanis?

  That's an argument.

• 1:15:34Speaker 07

  I honestly think that's the most plausible argument. It's very much so science fiction in the sense that we're not close to being able to even validate the hypothesis. So I don't believe in general AI anytime soon.

• 1:15:43Speaker 01

  Lon, I just love how many people we've talked to over the last decade, two decades here on the show. So our archives are deep, we shall mine them. But that's all for Twist today. Lon, you're the best. We'll talk to everyone soon. Goodbye.