The Standup with ThePrimeagen
Original podcast
ssh terminal.shop
12,129 words · 4 speakers
Today is the stand-up episode four, where we're gonna be talking about all the things that are important to devs. My name is The Primogen, the host, and today with me, we have-
My name's CJ.
Ah, yes! Nice, and who else? Casey. Trash. Trash, you're frozen- Let's go! But you're good enough, and so that is good. Alright, frozen trash is pretty much the same thing as a normal trash. So, today's first- Wait, I'm frozen? Yeah, well, you were. Today's first hot topic is going to be a court case that was recently tweeted by Casey himself, describing that AI and the training data for AI may or may not actually be legal to use. And so what are the implications coming on down? This is with Ghibli Studio, which Casey specifically pinged in it. And so what are gonna be the outcomes? So Casey, since you open to this can of worms, why don't you guide us somewhere with this conversation?
Wow, you're going, you're going right into heavy-duty stuff here, Prime.
Yeah, yeah. This is the stand-up. This is not the mess-around, okay? We don't talk about our dogs here.
Alright, alright. So no, this was not about Studio Ghibli. This is actually from a case, it's Thomson Reuters is the, are the people who are filing this particular lawsuit, or I should say, the people who are worried that their copyrighted work is being infringed unfairly. And in this lawsuit, so just to give you sort of the, the, the big overview, I, everyone agrees that AI training is copyright infringement, at least in the sense that they are taking copyrighted works without licensing them. So, there's no argument there, like, like, AI companies aren't like going into court and saying like, "What? No, we didn't use any copyrighted work." Like, no one's saying that, right? They're
all saying, "Yes, we did," but their argument is that somehow this is okay, right? And so what happens when you go into court and try, on that footing, right? Your legal footing is, "We're, we're violating all these copyrights, but it's okay for us to do it." You have to provide what's called a fair use rationale. And just to be clear about what this is- Copyright is a worldwide thing. It's international. The US is a party to something called the Bern Convention, which is like a uniform agreement that copyright is respected everywhere, right? So it's not some weird like only US thing. It's not some like finicky thing that only Congress decided or anything like that. This is a worldwide agreement that when someone writes something, they have the copyright
for it and they have this right to exclude other people from using it in ways that they don't like, right? Now, in the Bern Convention, it allows countries to make exceptions to this on very limited grounds. So basically, a country is allowed to carve out some special uses where you can make a copy without the original author's permission. And in the US, what we have carved out, you know, over the years for that is called fair use, right? Now, this doesn't just come from the Bern Convention because copyright existed before that, but the Bern Convention is now kind of the backstop, right? It was in the 70s. There's been, people keep ratifying it so it gets, you know, more and more countries get into it till now it's basically everybody.
So in order to have a fair use exception, in order to say I can, you know, take these copyrighted works and use them without a license, you need to demonstrate that your use doesn't, you know, doesn't violate, there's basically this four-pronged test. And the most important of the prongs is essentially that you are not doing any economic damage to the rights holder, right? That's the very most important one. The Supreme Court has said it's the most important one. All the courts have consistently followed that guidance that like, if what you are doing is creating an economic negative outcome for the original rights holder, it's probably not fair use, right?
Like, it's highly unlikely that it's going to be fair use because that's the main thing that copyright was designed to do is to give money to the people who made the works so that they will continue to make them. So hopefully that all makes sense.
I think it does.
Yes.
Um, so you said the burn convention, that's the thing in the desert? That people go to every year?
Burning Man?
TJ, how long have you been planning this joke? Like, how long has it been?
He thought about it as soon as he said, I saw his light bulb just go on.
I was just waiting for applause. I couldn't make that joke, but I don't. I was gonna pretend I didn't know for real. Sorry. Prime, you blew, okay, no way.
No, that's how, that's how it works. All the countries have to send their ambassador out into the desert. They all make a really big weird thing, like sculptures and people serve drinks. Everyone's high. And then they sign this treaty. They don't know what it says. It's probably written in some foreign pictograph-like language thing that hasn't existed, Sanskrit. Uh, and then, and that's how we do copyright now. Uh, and it's fine. Sometimes they get stranded there. Uh, we hope there's not a windstorm. Uh, because then there'll be no copyrighted works the following year.
Yeah, I mean, it makes sense. That seems to be on par with government. The AI companies would think they could just steal everyone's stuff and then make money off of it, so.
So, uh, to get into, this is why I said you're opening a huge can of worms here, because like, I love studying law. It's one of the things I do in my spare time. For whatever, so, you know, I just, I'm into it. Uh, so anyway, the, the, uh, thing that I've been saying for quite some time, actually. Uh, is I don't really understand AI companies' defense. And the reason for that is their primary defense has been that this is fair use. Because the thing that comes out of the AI is not going to economically compete with the things going into it. Now, that's kind of tenuous on its own, and we don't really need to go into that to talk about this particular tweet. Uh, I think that's a pretty tenuous defense to begin with, because they rely on a thing called Authors Guild v. Google.
Which, if you read it, seems like it would have come out the other way had Google had their AI summaries at the time the case, uh, right? At that time. That case was litigated, Google just had search results, right? And the little snippet from the webpage. It didn't have the thing that basically makes it unnecessary for you to go to, say, box office mojo to find out what the box office is. So I think actually, Google would have lost Authors Guild v. Google today, and I think AI companies aren't appreciating that. So I think that defense is also very tenuous, but you can put that aside because maybe you don't think it is. Maybe you read Authors Guild v. Google differently than I do, whatever. The thing that I've been saying is, who cares about that?
The market is AI training data. We know that's a market. I know you guys have received this too. I have a YouTube channel. It's not even remotely as popular as Primes, for example. I've got emails from people offering to pay me money to license my YouTube videos for AI training. So we don't even need to hypothesize that the market exists. It already does. People are already paying money for this content. So I'm like, look, it's an open and shut case. The market is the market for AI training data. They're not paying us, and they owe us because we could be generating revenue from our content by licensing it to AI companies to put it into their algorithms. And so I've been saying that for quite some time.
That's been my general position on what this is. We don't know what the courts will find, but other people have made this argument. New York Times made this argument in court. We don't know, well, made this argument in their initial filing. Their case hasn't percolated through the courts enough to get an opinion on that yet. But this first opinion that came out in February that I missed, and I only read it yesterday, actually holds exactly that. In this opinion, finding for Thomson Reuters that their copyright was violated, and it was not fair use, the court said literally this. It said, "There is potentially a market for this training data." You are stealing their copyright, you're infringing their copyright in that market,
and that could have negative ramifications for that was literally in the opinion. And so I thought this was a very positive sign because this suggests to me that courts are thinking that way. Lawyers are making that argument, and it's very possible that it will be found to be that way as more of these cases go through. So to me, this was a very positive outcome because I think AI companies should have to pay for their training data. And that's the entire long-winded thing, so hopefully everyone now has context. Take it away, team.
Wow, that was definitely some context. I feel like TJ, TJ, I could just see you, the whole time you were bobbing, you were ready.
I missed the whole thing.
Because I already have a lot of feelings on, I'm feeling a lot of things here.
I have a funny question and a serious question, which one do you want, Prime?
Obviously, start off with TJ funny, which is the best funny.
If they trained on Trash's code, does Trash have to pay them or they have to pay Trash?
Ooh, sick bird.
No, no, no, for your code, Trash.
Yeah, yeah.
No, they pay me.
For liability, for liability.
Because you damaged them, bro.
They pay you, but you are liable, Trash. You are liable for the code, but they pay you.
Only I can debug that code, baby. Job security.
When the AI company gets sued for the output, right? Like when it's like, look, this thing deleted all of our data. And they're like, well, I mean, it's just, we can trace it back to this training set from Trash, and I don't know what to tell you, talk to him.
Trash has been selling all of his data. He just cannot wait for this, he's just, come on, get all my.
I'm trying to sabotage AI.
Okay, my serious question is though, like, so let's say codes on GitHub. Did I sign something? I mean, obviously, I read the TOS, so we can put that away. Obviously, I read it, I'm just testing you, Casey. Did I sign something to let GitHub let people do this without my permission?
So that's going to be a really good question and a really difficult question. I haven't tried to look into that because it's... The reason for that is there's such a thing as a contract of adhesion, which is what you are effectively sort of quote-unquote signing when you agree to use a service, right? In a way, it depends how you want to look at it. Terms of service and contract of adhesion are technically a little bit different. Like, a contract of adhesion is more like the thing you sign when you go to get a cellphone, you know, or something like that. But it's the same kind of principle. There are limits to what you can give away. You know, if the cellphone company had buried a little tiny clause in that giant contract that said, "And also, we can human traffic all your kids," or whatever, right? It's like, it doesn't suddenly...
I mean, that's going to be criminal as well, but I mean, you know what I'm saying? Yeah. They can't just put something arbitrarily egregious that would, you know, quote-unquote "offend the conscience" into one of these boilerplate things, and then actually expect that to get them out of jail free when they go to court. And so, I would have to say that's a much harder question. Hmm, hmm. I would say you probably, I would imagine, you wouldn't have to worry too much about Microsoft's competitors being able to do it, because the only people you probably gave an agreement to was them. So, it might be training Copilot, or maybe an OpenAI through some agreement, but Google would probably never be allowed to do it, because Microsoft wants the competitive advantage of all the source code on GitHub.
My recommendation, and I have been thinking about doing this for a while, is I think everyone should probably be going dark these days. Everyone should be pulling stuff and putting it on their own website with a little notice that says, "If you want to use this for AI training, you have to pay me." That's, I think, the best practice. You know, if, like, let's say, you gentlemen are distinguished web devs, and you know a lot of distinguished web devs. If you put up the AI-safe GitHub tomorrow, I would move everything that I have to it literally as soon as it was up. And I think a lot of other people would too. And I would also encourage everyone else to do so. So that that would be a great competitor to GitHub. If anyone wanted to do that, I would even pay a fee to do that.
Meaning I would pay for a good GitHub clone out there that would get me away from licensing Microsoft implicitly. Because, like I said, I don't know if that it will turn out to be something that they're allowed to do by slipping it in a TOS. But it might be, and until a court case, we wouldn't know. So the safer thing to do is to just not use it.
So could you use an AI, an LLM, to generate a GitHub-like interface, trained off of code on GitHub, to then launch a private AI-free service? Is that what you're trying to tell me?
It'd be an interesting thing to do, but again, based on what we have seen, you may very well be liable for copyright infringement because of the way these court cases are going. Because if you weren't Microsoft and don't have the effective, like, you know, in the GitHub Terms of Service, there's probably some clauses that say, like, you grant Microsoft a, or well GitHub, but you know, the subsidiary of Microsoft, worldwide, you know, license to reproduce this work or whatever it is, right? Because they need to for operating the service. And they'll just argue, well, Copilot is part of the service, so we train it, right? Other people don't have that out. They can't go to court and say, like, well, you agreed to it, right? And so, eh?
So that's kind of a, yeah, I mean, that's going to be a gray area, even if things go the way this court case did and the way that I think they should, which is the AI companies need a proper license. You know, who knows? Who knows what that what that looks like?
Will anyone, will anyone using AI, is there any sort of like trickle down to this where if people are who are using AI, will they become also like tangentially liable for what they have generated?
Uh, yeah, so that's very possible, but also, uh, I think that really depends on peep. So this, this will transition nicely into another thing I want to talk about, uh, and you just tell me when we're ready to transition.
Just tell me when I have it. I have it. I'm ready. Hold on, I want to say something.
Okay, so we're gonna, let's, let's get through all this first and then we'll transition. So to say like that, this.
Let me talk.
Trash.
Okay, I'm gonna, I'm gonna bring this back to the Studio Ghibli scenario, because that's what everyone's probably familiar with, right? Where that was a whole big debacle.
Yeah, Trash isn't checking out the laws, but he did see the pictures on the internet.
He saw Sam Boltman looking quite ghibli. That you want to talk to about law, okay?
No, you said, you study it. You said you study it. You, he knows what he's talking about.
I study it and I pirate all mine. That was not real.
Okay, riddle me this scenario. Okay, so we're gonna bring this back to art, and again, I think art is like where it gets really, really muddy. It's probably not as cut and dry as like this Reuters scenario, which is like a very niche kind of area, I feel like. So, riddle me this. I'm an artist. I don't really care if people use my work. I just want to be well-known. I just want people to see my work. But as a human, I have inspirations from other people. Maybe it's Studio Ghibli. Maybe it's a number of other very popular artists where it's very obvious if I'm stealing their work. So when I draw, you're probably gonna see a little bit of those details come out. All right, but I allow you to use my work for your training data, because I don't care. I have a full-time job doing something else. You know, drawing's just my hobby.
But when you're training on my data, some of those little qualities come out. Maybe I like Studio Ghibli eyes, you know? Maybe I like Dragon Ball Z abs. I don't know. So what happens if that comes out, and then someone else notices it? And you're like, wait a minute, that's Goku's six-pack, right?
Will I be in trouble? Do you get in trouble? Or is it fair use? Because? I don't know. I put like an extra shadow on one of the abs or something. I don't know.
So you're
So how'd that work?
You're basically talking about, is there a scenario where there's effectively copyright laundering? So we take Studio Ghibli, some other artist draws something sort of like it, and then that artist gives them permission to train, gives an AI permission to train. That's what we're talking about, just to be clear?
Yeah.
My assumption, A, that is the reason we have courts. In other words, for weird corner cases like this, if someone really cared and was worried about it, they would go to court and a judge or a jury would look at that evidence and say, do we think this was really against the spirit of copyright law or not? My gut feeling is that would be fine, meaning that, you know, there isn't, Studio Ghibli is not marketing educational materials for humans right now. And they are not marketing materials for AI training right now.
If things go the way I assume, they will do the latter only. They will say, okay, here is our pricing for AI training, because now that's a market and we understand that. So AI companies have to pay that if they want to. If they don't then say, also, if humans want to train on it, you have to pay us this, right? And therefore, it's, these are educational materials, then no. And also, that thing, again, all of this stuff is based on licensing. If they provided a license with their training course that says, also, you agree not to draw things too close to our style, right? If they don't do that, then there has been no, like, that causal chain is not broken.
A lot of people get confused because they think for some reason, humans and machines have to be treated identically or something like this. But actually, not only do humans and machines not have to be treated equally under copyright law, but humans and humans don't have to be treated equally under copyright law. You can sell something for one price to professionals with one license that says you can do anything you want to it, and by the way, it's $5,000, and another one to students that says it's 50 bucks and you can't use this for anything that makes money. That's completely legal. So again, you can, you can Copyright law is supposed to allow the person who made the thing to determine how it is used. And if Studio Ghibli wants to make special screenings of their movies, where a big thing comes up on the screen that says, "No one can draw like this."
If you are going to look at this movie and try to draw like this, leave the theater now, because that is not allowed and your ticket price will be refunded. They can do that.
Can they? Is that contract of adhesion that we talked about earlier?
Uh, yes.
Do you guys remember when on Facebook, like probably 10 years ago, when everyone was starting posting that Facebook cannot use my posts for like anything? And everyone just kept posting it as if it did anything and it was the funniest thing ever?
I'm pretty sure my grandparents have been doing that recently. Um, actually, these posts are not allowed to be used by AI.
My parents are just like, "You do not have my permission." I'm just like, "Oh my god."
It's too late, brother.
Your parents, your parents need to put a price tag on it. That's what they're missing. Put a price tag on there. Say like, "In order to use this for training, it costs $1,000." Put it on there. Put it on there. It might help you out someday.
Casey, here's my next question. Suppose they find out, okay, couldn't train on the GitHub stuff, or like couldn't train because people had some GPL license on their code, and it was in clear violation of the license as stated, the way that they used to train it.
Right.
Do... Are they gonna have to delete the models? What, like, what is, what is the next step? Let's say they find it out, right? Is it like, we gotta roll these back and we're starting back at GPT-2.0?
Yeah, so, again, this is, and it's important for everyone to remember, all the things we're talking about are somewhat hypothetical. We know what happened in the Reuters case. It's a district court, which means there's two more levels to go, right? Because AI companies are gonna have to fight this because, you know, they're in a lot of trouble, potentially. So, there's district court, that gets appealed to us, like an appellate circuit, right? You always feel like the Ninth Circuit or something like that. Then that gets appealed to the Supreme Court. So we've got, we don't know what we're talking about right now, right? So just important everyone's aware that we're just saying hypothetically, if this stuff happens, here's what's gonna, what's gonna go. But we don't know that. So anyway, to your hypothetical, right?
Usually, in copyright cases, the first thing you get is injunctive relief. They have to stop selling whatever the thing is that infringed, right? Right. So my assumption is that, at the very least, they would have to literally rerun the training round. They would have to pull that content, like if, let's say Studio Jubilee won it. They'd have to pull all the frames of all the Studio Jubilee things and rerun the $50 million training round. I don't know what they cost these days to run, but, you know, they're in the tens or, I don't know if they're in the hundreds ever, but they're in the tens of millions oftentimes, is my understanding.
What about like distillations? Like things that like, you know, like R1, one of its secret sauces was that, you know, it could verify some of its data against OpenAI and be like, okay, so this is like a great way to predict what's coming up next. Does that also, excluding if China were to listen to it, does that also mean that all derivative things that used O1 as part of their training set also have to then fall under the same copyright problem?
Potentially, but again, maybe it's too attenuated at that point. I don't know. And again, this is why I say like, this is going to be something courts are going to be dealing with for a while. Because it's like, that, you know, does Studio Jubilee see that, you know? Well, and the other thing too is people always say like, oh, well, you know, that'll just ruin US companies. Like, no, this will apply to everybody. If you're, China can't come into the US market and sell something they trained somewhere else. If it was based on copyrighted data, they'll get the same exact injunction slapped on them, right? They can be taken to court just like anybody else. And again, also, none of this stuff really applies to research, right? Because that is actual fair use. So if you weren't commercializing this AI, you're not providing it to the public, you're just doing it in some lab somewhere for research and publishing papers on it, that would probably be totally fine, fair use in all these scenarios.
If you're a government agency who's doing this for national security purposes, like OpenAI, I usually try to use the national security thing, it's like, as if generating Studio Jubilee images is national security.
It is.
That stuff is separate work, right? Those are separate things and they can be cabined off and they can use copyrighted works. But the instant you say, oh, hey, we're OpenAI and you can come pay to use this, Mr. General Public, that's a commercial product and you're on the hook. So, you know, if the court cases keep going the way they're going, all the things that we're talking about are plausible things that courts will be asked. Absolutely.
And the reason I asked in particular about rolling it back is my understanding is generally speaking, they are using a lot of transfer like learning, right? They're taking weights that they already did before. They're using that as the beginning of the next round of training. Now, I don't know for like new models, whatever. My understanding is though, like when you do these things, you do a lot of transfer learning. That was one of the things that they found out that was crazy about neural nets is like, yo, we can train it over here and we can train on something else later and it does better. And you're like, what the heck? We don't understand why. And they're like, okay, because of math, sure, sure. Math, like magnets are real.
It's true.
Sorry.
And TJ learned about ICP just like two weeks ago and learned about magnets. And so he's so stoked about it.
Just the magnets are, that joke's too good for me.
I love that joke. I have made it many times and everyone's like, what are you talking about? I'm like insane clown posse, anyone? No? Okay.
TJ literally just discovered it last week, so he's very stoked right now.
Um, but so that's kind of what like the transfer learning aspect is really, because I like, I wonder how far they would try and pull that back, right? Because like initially, they were just literally reading all the code on GitHub for sure. We know because they would be like, start typing out this GPL code. And it completes the rest, including source comments and saying it's GPL. Whereas it's like, okay, well, you, that's, that's definitely against the license. So that part is very, I don't know, it'll be interesting to see how that unfolds.
Well, we also have Facebook admitting in public filings that they went to like, I mean, the effect, they effectively went to Pirate Bay to download stuff for their training. Like they knowingly violated like literally international law to get their training data in the first place. And so you kind of have this situation where these AI companies are completely on the hook. So the only question is what are courts going to decide about whether any of this is fair? And Like I said, I'm optimistic because of the Reuters case. I don't know what will happen in the future. Optimistic that they won't, you know, can I get cat on stream? Can I?
There you go, yeah, cat on stream is fine.
I'm allergic to cats.
Oh, I'm sorry.
Yeah, good.
All right, trash.
But no, no, by the way, the fun part about the Facebook one is that there's an actual email from one of the like leads that's just like,
"I probably shouldn't do this on my work computer." Sweating, smiley face. It's like, oh my gosh.
It's amazing. It's nuts. I looked at that stuff and I'm like, what is happening, right? Like, they put that in an email.
They're like, yeah, Zuckerberg's like, yep, that's probably a bad idea, isn't it?
Yeah. So anyway, you know what? Can we transition now? Because now, this is the thing.
I assumed that you want to do the Shopify one, right?
No. I wanted to talk about this because, and you guys, you basically got there without me. That's the thing. I have this concept for the AI future that people haven't been thinking about. And it's all the things that you all just said, right? You were just hypothesizing. And I came up with a term for it that I want to try and make, you know, the way we sort of conceptualize this. I call it "the taint."
Okay.
I like the taint.
So React already does use this. So just so you know, the taint already does exist.
Fersell might come after you.
Yeah, Fersell, which might be the taint, will come at you for copyright.
Maybe we can work out some license agreement so I can use the taint for this.
Okay. TJ is not participating in this conversation.
It's possible that we are heading into a future where everyone is going to need to know if their AI has the taint, right? If your AI has this taint because it's had the copyrighted work into it, right? And you can have like a lot of taint or a little taint as well, right? So there's a certain amount of taint when we look at this AI that we're going to see, right?
Taint girth.
Exactly. And so it's like, will AI companies have to pivot to demonstrating that they have the least taint, effectively? Or they are unlikely to have taint when you use it, right?
I'm watching Tej. Tej has like 20 jokes lined up right now.
I'm not going to go, literally, I'm not going to say anything.
TJ wants the jokes so bad, but he's just like, not going to do it. No girth jokes, no length jokes. I'm going to miss 100% of the shots I'm not taking right now, let me tell you.
So I would like everyone to think about this possibility that there, there could be like a future where everyone's concerned about the taint of AI.
So there's a taint score.
Oh, I bet you there will be like some sort of judgment. Like if you can prove in some reasonable amount that your taint score is below 5%, it's kind of like how you could, there's like a certain amount of rat feces that can exist within edible products. That they're just like, that's unavoidable percentage. The end. There are body parts and such in it and that's it. So therefore you can also have some unknown amount of taint factor.
That's disgusting.
By the way, Casey has just disappeared on us. Apparently. He spoke a little too authoritatively on the taint and now.
Now he's out.
AI. What are you doing? I'm building it after this.
Amazing. Amazing. Can you hear me?
Yeah, now we can hear you again. We can hear you again. Hey.
The AI is fighting you. It hurt what you said.
So anyway, I'm, I'm like, like I, I want this future where everyone is inspecting the taint. Right? They're trying to find, they're chasing the taint. They're like, look. We have to figure out, because if we go to court, the last thing that I want, if I'm a CEO, I don't want to go to court and find out that suddenly I, I actually did have the taint. Right? Because I, and I didn't know, like we use some product, we license some, you know, thing. And the taint, we ended up with the taint. Right?
This reminds me of a Wheel of Time.
A coding assistant, but all we ended up with was taint.
You just do like a smell test.
Yeah, it's a sniff test.
Yeah, this reminds me of the Wheel of Time. You know, Siadine was also tainted. You know what I'm talking about, TJ?
I was thinking, I was, that, I was literally like, should we just have a 15 minute conversation about Wheel of Time in the middle of this? That would be good for stand-up, like.
Classic stand-up. No, this actually is pretty interesting because there has to be a certain level of copyrighted material that the courts are going to say is okay to be within an LLM. Because there's just like some way that like there's just gonna be things that you will never know because someone stole some work that you could not possibly ever attribute and it somehow got into it and therefore that's that and now they're saying, okay, well, you can have some percentage, but we have to prove, you have to be able to prove that you're below some threshold.
Well, it won't necessarily be a percentage, it'll just be like some number of people will sue you and you'll make licensing deals with them, right? Because what's gonna happen is it's copyright assertion is like on an individual basis, right? It's like Disney goes and sues them and then, you know, assuming that they get a similar decision to Thomson Reuters, then one of two things happens. Either all of that training data is pulled and they have to rerun the models, or they have to pay Disney. And my guess, and I could be wrong about this, but at least for the big companies like Disney, what they're gonna do is they're gonna get a little of that sweet, sweet moolah, right? So they're, you're not gonna have to pull Disney stuff, you're gonna have to pay Disney their little AI tax, right?
And OpenAI is gonna be taking some of those, you know, they won't be venture capital rounds by this point, presumably they'll be a public traded company or something like that. Once they figure out how to turn themselves out from a nonprofit or whatever, they'll have to pay some of that, you know, all of that AI income or investment money, which is what they currently are running on, investment money, they'll have to pay that to their copyright people.
By the way, I would like to say that Trash is doing the most classic stand-up behavior, clearly typing on his keyboard. I'm typing in the chat. Well, shut up. Okay, Trash, shut up, we're working here, okay? Focus up.
I'm listening.
He is Director of Community Outreach. That is his job, he's doing it.
You lost me at the taint conversation and then my head just started spiraling into like 50 memes.
Well, the good news is, is that we can move on from the taint, okay? We do not have to continue down this road to this logical conclusion. Instead, we can pivot. Alright. You guys ready for topic two? Sure. Which is kind of really just like an extension of topic one. And so that is going to be this right here. Shopify has announced from the tippity of the toppity that everybody must use AI. Toby effectively went on to say that he presented about it and called action for invitation for everyone to be able to use it. But that was not enough. And then he goes on to say that people who used AI are not just 10x the engineer,
but in fact 100x the engineer and work being done. That's right, for every single three and a half day work they've done, they've done an equivalent of a year before they started using AI, which is quite a wild amount of work. And then finally, he lists out all the things that he expects you to be doing when you're getting or when you're using AI. For those that don't know what GSD is, GSD just is a simple get shit done. That's like what it means and so.
German Shepherd dog, actually.
German Shepherd dog.
Not gonna lie, I was very confused when I saw GSD. I was very confused. It's a German Shepherd dog. If you had one, you would understand.
I do not have one, so that's probably why I'm missing a little bit of the context here. But Shopify has officially stated that you are to use it no matter what in every single one of these cases. Now, I don't, I don't know if there's any sort of, they didn't really list any sort of negative cases where you do not have to use it. But it went from a suggestion you should do this to now this is a requirement from here on out at Shopify. And one of their justifications was actually found in this previous paragraph, which is, "In a company growing 20 to 40% year over year, you must improve by at least that every year to re-qualify." So you need to improve, I'm not sure on what metric they're exactly saying, or how they're suggesting that they rate this improvement, but you must improve at the same rate
the company is in fact growing monetarily.
Which makes no sense, by the way. I literally have no idea what that is even supposed to mean in practice. I'm not sure he does either.
I'll defend that line, even though I don't agree 100%. Pretty sure they're just saying the market for Shopify-style sites is growing by this much all the time. So if we don't keep getting better, someone else is just going to snatch those up. Because those are like brand-new customers on the internet making Shopify sites. Or like in their, in their vertical, the like TAM is just growing by 20 to 40% every year. And they're like, we're gonna get surpassed by this. Right? If we're not doing this, then we're just gonna get trashed. That's what I thought they were saying.
Okay, so that you must mean the company. Because if you look at the thing, it says, uh, in a company growing 20 to 40% year-over-year, you must improve at least by the
Oh, maybe I misunderstood.
Yeah, that's what I'm saying. Like, it just doesn't make sense.
Is this you a royal you, or are you the engineer?
I was reading it as like you, like Shopify's gotta keep doing this because the market's growing there. But I see what you're saying. You're saying, yo, person, you gotta be 40% better.
Yes. Because the rest of it is all about the individual using it, getting X percent better, all this kind of stuff. My big, but TJ, I know you had some contrarian viewpoints on this. But I will say that one thing that I just really dislike about all of this kind of conversation is not that saying that AI will make you faster. For a, definitely a set of, uh, people in every single profession, AI is gonna be a huge assist to their current skillset. I don't think that's, I don't think anyone can argue that. I think the big problem is that we sell it as 100x. Because I just do not believe that three and a half days is worth a year of work, ever, for anything. Like, right? Like, you could generate shaders for a year and a half, or for three and a half days, TJ could. And I would outshader his shaders if you gave me a full year, 40 hours a week, every single week.
TJ is raising his hand. TJ?
Thank you. Thanks for calling on me. Have you worked with some of the people at these large companies?
Yes. In fact, I have. I worked with TrashDev, as you can see right below me.
Am I below you?
Well, in, in, yes, in this one, no, I'm below you. Okay, you get it. You, you're below me.
Oh, he's on the lower floor because you're in the C-suite.
I meant metaphorically.
Talk about that taint again.
Yeah, that taint.
Okay, I think 100x, I'm with you. 100x seems wild, but I will say I've worked with people that haven't gotten anything done in a year too, so.
It all depends on like what your baseline is, right? Like my baseline is not like your baseline. Like, they need to like have some metrics. Is it gonna be lines of code? Feature ships? You know what I mean? Like, you can't, I don't know. I have a lot of thoughts here, so I'm just gonna keep going.
So one of the, one of the, um, bullet points they have, and honestly, I thought I read the whole thing, but then when you started reading it, Prime, I completely missed everything you said when I was reading it. So I guess I don't know how to read.
So you in fact did not read it. Would you like me to put it back up for you?
I in fact did not read it when you started. I was like, oh, it said that? So, the, one of the bullet points was before asking for headcount, you have to prove that you need it with like your current use of AI.
Yep.
Which kind of blows my mind. So like if we go back to the water tower, right? You know, I basically came in there and like bombed your project in like 30 minutes, right?
You did, but yes.
So the way I kind of see it is like an AI is kind of more or less an intern, junior dev where they're, they're effectively a net negative. Until they onboard, get familiar with the codebase, you know, level up a little bit, right? So you spend most of your time babysitting, code reviewing, that AI slash intern. So you end up losing productivity on your side to make up, uh, to have the AI or intern do its work. So, that is like kind of counterintuitive, where that, that would be the argument I feel like to, you know, you would need headcount because you need someone to basically be a full-time prompt engineer. And basically code review the AI stuff right before it just keeps shipping more stuff. Does that make sense?
Yeah, but I mean, there's a couple flaws to all this. Like, first off, on the TJ side of things, the people that don't ship in a year, do you want them shipping to begin with? Right? Like, do you want them 100x more effective? Like, no, you don't. And so, second, even with this intern thing, people always equate it to an intern, right? AI is just like an intern. The thing about an intern is that, day one, you're like, here's what you need to do, intern, and you sink a bunch of hours into it. And then, day two, day three, day four, by, you know, by month, you're like, okay, hey, go do this project. And then, they come back in two weeks, and they have a pretty good set of project, and you're like, okay, here's a couple of different things you need to do. There is no gains with an AI. The AI is as good as it is day one as it will be on day 60 right now, unless if there's
some theoretical huge breakthrough that we have yet to see on the horizon that's going to make it from a junior engineer into some magnificent senior engineer. But, right now, barring extreme breakthrough, you get the same thing, whereas with interns, you get some sort of multiplier effect, or at least some sort of additive effect to the work, where they get to run individually because they have autonomy.
Potentially.
I think, to be fair to, the guy who wrote this is Toby, right?
Yeah, Toby.
Okay, so to be fair to Toby, if he's following his own guidelines, he probably, he might not even proofread this particular memo, right? He might just ask the AI for something that encourages the company to use AI, and then hit send. Because he was 100x, he didn't have time to read any of it, so it's possible that Toby doesn't know what it says. And frankly, that would be likely, because this is also the guy who sent out the premature optimization is the root of all evil, we should ignore tweet. So, like, you know, just, if you could go back like last year, and he was talking about how everyone needs to start thinking seriously about performance, and like, we need to use a lot of AI, and we need to think seriously about performance all the time, they don't even, it's like, what are you even talking about?
These are completely conflicting statements you're giving to your, you know, your engineers here. So, I don't know, like, it's, it almost reads like a thing where it's just like, yeah, I asked AI for a thing that would encourage AI use, I hit send, and then, you know, it leaked, oops, or something, like...
But do you think there's a world where this actually is, I mean, do you think in our world, this is actually a fine thing to mandate from the top down?
Not if you trust your employees at all, because at least for me, if I was running a company of that size, I would say, the kind of memo I would send is like, "Look, AI is a new thing. We would like people to evaluate it. If you find it useful, we'll pay for it. So go ahead and adopt AI practices if you think they're helping you. If you don't think they're helping you, you don't have to. I trust my engineers to make intelligent decisions." Or designers would be the other people presumably who are using like this for prototyping. "I trust you to figure out whether or not this is going to help you or hurt you and to act accordingly." Saying like you have to use it everywhere just sounds like you don't really believe you have any good people.
Would you say-
Because if you did, you wouldn't be saying this, because you'd trust them to make that decision.
So you would mandate the evaluation of AI? You wouldn't mandate the usage?
Exactly. You'd just say, "Look, I would like our company to look at this. Please, everyone take..." Basically, it's more of giving a permission, right? Saying, "Look, everyone, it takes some time to evaluate AI. I would like you all to try it, see how it goes, maybe write it up a little bit so we know for each of our things, like how did it go? Like who had success, who had failures? And then I'll look at it and we'll maybe make some decisions about what we want to do going forward." That seems like a sane thing. Just like, "AI is big at 20%, everyone use it." Like, I don't know, it just kind of comes off as, like, almost like frantic or like, you know, "I had too much coffee," or I don't know.
There is another interesting bullet point where it says, "AI usage will be a part of your performance review."
Exactly.
Like, how do you... Let's just say I ship all the time even without the help of AI. Will I get a bad performance score because I didn't make an AI commit? Or like, how do you even prove that I didn't use AI or use AI?
Obvious. If you would read, the other bullet point is that you're going to share your prompts in the Slack room. And so it's probably prompts shared is really your impact to the AI usage among the company?
My prompts would be so bad, just like yours, Prime. Like, can you just-
I'm gonna kill your grandmother!
You guys are not even Chaos Orb maxing, like it's pretty embarrassing. I've seen both of your prompts. That should actually, I'm gonna start doing that. That part's pretty embarrassing.
Dude, I would love to see a room full of like, shared prompts and it just gets more and more degenerate. Like as you read, you're like, oh gosh, what, what, what with their mom? Right? It just gets worse and worse.
Hey, if anyone's out there listening right now, all of my prompts, you have to pay me to train off of them. So if you're in the audience listening right now, you better close your eyes and never hear me do a prompt again, because those are mine. For a thousand dollars.
Contract of adhesion.
Yeah, got them.
Got them.
I, I've got some bad news for you guys on that because the copyright office recently released their guidance that says that anything that comes out of an AI where all you did was put in a prompt is not copyrightable. So everyone can just take that for free. You can't even-
Don't you know anything about law, idiot?
But this is the prompt he's crafting. Okay, this is not the output.
They don't need it. They can just take what it generated.
Oh, I see what you're saying. But to get the, to get the peak performance, you gotta be using Chaos Orbs and I'm copyrighting that strategy. So that's just too bad, Casey.
Fair enough.
I will say, okay, in, uh, I, I will say that I have seen people use different LLM tools quite a bit. Let's say, different levels of efficacy, right? And knowing when to use them and how to use them is not always intuitive. And also like, sometimes it breaks your expectations and they're better at some stuff than you think they were gonna be or not at other, other things. So I do think there's something of saying like what you're saying, Casey, of like, having people try it. But even like saying in extended, like, you guys really need to like, try it a lot of times to see because it is quite a bit, uh, different and like, reaching for it in unexpected places.
Like one of the ones that I used, I, I told Prime this one, like, I was doing some awesome WM, like, not it's awesome, but like, awesome is the name of it. Uh, and they have all of their docs and everything online, but it wasn't like, in a format that I could pull into my editor to get autocomplete and everything. So I just like, listed five URLs and told the LLM to generate from those, like, properly formatted stuff and put it in. And it would have taken me like- Hours and hours probably to do it, right? And it did it within two minutes, right? And then it like literally 10xed how fast I could do the rest of the things. Cause like now my editor works and I get autocomplete and I can discover everything and I'm getting squiggles and all this other stuff, right?
And I feel like if you're not trying, LLM's out for different things. You would never think to like paste a URL and just be like, fix this. Right? So there is something there where like, I don't know exactly how that's going to play out or where that's going to end. But there are things like, okay, you do actually need to try and use it if you want to get stuff out of it. I don't think, at least right now, you can just be like, I use it once a week and it really affects my like workflow. That just doesn't, doesn't, it doesn't work for anything. You know what I mean? Nothing you just use occasionally is really going to change, you know? I don't know. Except for ayahuasca, apparently that's what I heard.
Yeah, that's a, that's a one time use change right there.
Yeah, that's, yeah.
That's a vibe TJ, okay?
Yeah, exactly. That's what I heard.
Yeah, I generally, I generally agree with that. There are a few things that when you use it, you're like, man, that is actually extremely convenient. Because I would have spent like two hours trying to come up with a Vim macro to solve that problem. And then attempt a redjects to edit HTML and like go down all the same routes. Everybody else has ever gone down in the history of mankind, just to try to fix that exact problem. There are things that are fantastic. And so you don't, you definitely don't know that if you haven't AI'd hard enough.
So that's my only defense of like sort of the mandate aspect. Now, would I put it in your performance review? Nah, that ain't for me. Like I, I, I don't want to, I barely want to measure anything in performance review. I would do performance reviews mostly off vibes. Like I just hire managers who you can trust and hire manager of those managers who you can trust. However, many levels you need, I don't know how big Shopify is. And then you just like fire people if people get the bad vibes.
You know, I don't want that much. You can measure if they're doing a good job or not. You're a trillionaire. Okay, everyone desperately wants to know how to measure if you're making good software or not. And people aren't. So, there you go. That's...
Maybe we use the AIs to evaluate whether or not you have been using AI. And then we can give you a completely objective, in the sense that it's done by a machine, performance review.
There we go.
We could also take-
I'm gonna poison the data.
Yeah, we could also take an AI and throw all your Slack messages into it with all of your coworkers and then be like, "Give me his vibe score. One out of 10. How's he vibing with his teammates?" People that aren't, get them out.
Is there gonna be a thing where like, the trust and safety division of companies, which is like, you know, during the day, they're like, trying to go through all of that company's public comment boards and pull out all the like, things that are, you know, horribly demented that people are posting? And all of those are gonna just get sent to the AI team where now those will be tried as prompts to see if they encourage the AI to do better work, right? So it's just like, it's this pipeline directly from, "We had to stop 13-year-olds from seeing this", to, "This is what we feed directly to the AI, because it works really well",
when, you know, ButtStank33's comments that we had to remove from the public forum are all, are getting it set up for the day.
Dang. Well, I am just curious how this is gonna go in a year, because I'm just very... I think the performance reviews, because out of everything, it's just like mandating people use AI in some sort of capacity at your job. It's like, okay, sure, maybe you could do that, but mandating it as a measurement of performance? I don't know how that's ever going to be acceptable, or any measurable, or anything. It just seems like you should always measure people how you've measured them, and if they're getting better, relatively, then it's like, okay, this person's better. The end. It doesn't make
And if the AI was doing its job, it will show up in that metric, right? It's like if, if, if the person is using AI effectively, they should be having good output. So you wouldn't really need to also check if they used it, right?
Correct.
They were doing 100x.
You'd know.
You would know, like, it will show up somehow.
Your features, things should just get done really, really quickly because someone has 100 years worth of human work in one year of time at your job.
There's a lot of that thing too, where people keep saying these ridiculous multipliers, but their output doesn't appear to change externally. So you're just like, I don't see like Shopify producing 100x more stuff. So what happened, you know, like what's going on exactly? So it's kind of weird, like, shouldn't we see it? 100x would be like, you've rewritten all the software in the world this year, but just your company. That would be 100x, right?
Yeah, you don't have any external dependencies, right? Like you just rewrote the stack. I, I wish the like, from, in my experience, it's been much more like, instead of saying like, I'm 10x better at whatever. Oh, like AI has let me do things that I would normally not do at all. Like, because it would just take too long to like, learn some base knowledge about this thing. So I can say like, hey, do these, oh, that's not really right, kind of go here. Okay, now like summarize this, well, right? And like, I could start something and like, complete it that I would not have started otherwise. But it's not like the AI like wrote all of it or anything, you know what I mean? It's that like, it enabled me to do that, which is, there's no comparison. It's not like 100x more or less.
It's just like, oh yeah, I could do this project that I couldn't have done before. In a sense, that's a million x, but in a sense, it's plus one. Like, with, okay, so I feel like that would be such a better framing for a lot of these like CEOs. Like you can take on a more ambitious project. You can try something in a new area. We can try and like fix something that we didn't know how to fix before. Cause you can like, ask the LLM how to solve DNS problems that you couldn't figure out on your own before that were 37 replies deep in a forum. Sweet, now you can solve that. But like, it doesn't make me 1,000 or 100x better. I can't write all of my software for the year and a week. Like, that's not happening.
And it's something in which, yeah, because when we did the Towers of Mordoria stuff, there were a few things in which I could have spent a day or two just reading the love docs and just purely experimenting on how to draw, how to like do the things right in that. Or we could just generate a whole bunch of stuff and then try to fix it as we go, which in some sense just means you understand the basics of it. I understand how to use it. I don't understand how it works, which I think is a fine thing when you're attempting to learn something for the first time.
Or you want to iterate on an idea and you want to see different versions of it. Like it, like it's important to be able, it's a valuable thing to be able to try something out, throw it away, try something new, throw it away, try something new, throw it away. Oh, that one was good. Oh, that could have taken me three months to figure out before, but this way it took me three days. Okay, cool. Then maybe I delete all of the code and now I write it from scratch by hand, like a, like a Luddite or something, but uh, that iteration speed's still like important and useful, especially like, you're not going to guess product market fit right on the first one for anything, basically, right? Like if you are, then you, you are also already a trillionaire, right? If you can be the guy that always guesses everything right for PMF, congrats.
Go work at one of these countries and, or companies and deploy billions of dollars of capital and like succeed where others have failed. That's not me. I can't. I have to try it to find out if it's good.
Correct. Hey, speaking of that, um, we're going to do, because we're way over time, but we're going to just do one quick thing. There's this video going around right now on the internet.
Oh, sorry.
Which is Quake, uh, and Quake was entirely generated by an AI. This entire gameplay that you're seeing right now is generated by an AI. It was 100% trained on only Quake and now it made Quake. And you can run around and shoot bad guys, sort of. If you turn your back on them, they've never, they never existed to begin with. Sometimes they die standing up because that's just the way it goes. But you know, it's just, it is all generated. So Casey, and speaking kind of of this like fast iteration cycle, hey, we're going to be able to move really, really quickly. Is there something to this kind of technology? Let's just say it can improve by 10x or 100x or some of these big magical numbers. Do you think there's actually a real value in the gaming market
to be able to see or experience features not yet created as fast as a prompt? Just to see if they're good enough? Or is being able to just simply see something play out actually, uh, real? Until you being able to say that's a good mechanic?
So, you know, I'm really the wrong person to ask. That's a very designer-specific question. And the best I could say is, in that form, like, the form that they're doing there, probably not. Because when I've seen great designers work, and there are very few of them, but I've had the privilege of seeing like, a couple people who I would consider to be exceptionally good designers, do what they do. There's a lot to the specifics. You know, like, a great example would be, go listen to... So, I mentioned Dragon Sweeper one time on Prime Stream. There's a game called Dragon Sweeper, made by Daniel Benmer-Gui. He's also the person who made Storyteller, which was a big hit.
One of Netflix's top games on their service, actually. Trash. And he's an excellent designer, and you can go listen to... There's a podcast of him where he describes how he designed the game. And he designed it very quickly. It didn't take him very long, all things considered. But the kinds of things he had to work out to make the game fun and to experiment with are things like, exactly what is the algorithm that determines how these pieces are placed on the board, when the board is generated, or things like that? And it's just like, so, a thing that improves that iteration time would be useful. This thing that Microsoft is showing, doesn't seem like it gets at that.
That seems like a very superficial thing that they've created, that wouldn't really help any real designer, because we all kind of already know what that thing... Like, if you're just training a thing on Quake, it's like, what are the design changes I'm going to make that this thing can represent properly, that actually give me that information? There are the kinds of things that we would call a producer knob, in music. A thing where you just kind of like, pretend it does something, so that higher-ups can feel like they did something. That's what this looks like. It's like, something that some higher-up could say like, "Make the shield more green", and the AI doesn't like, "Oh, that was great." And everyone's like, "Thanks, Bob, you really turned the project around." And then they go back to doing the real work, right? So, I do think there's room for something that helps game designers iterate.
This doesn't look like it to me, but if you really wanted to know, you'd have to ask some of those top-notch game designers. Ask Jonathan Blow, ask Daniel Benmer-Gui, ask some of the people who are really excellent at doing this kind of stuff, and see what they say, because I don't really know.
Okay, yeah, because I think one of the things that's probably missing from this context is that they did this other kind of release of it, which is Copilot for your games, or I forget what they called it, it could have been Copilot.
That's, we should do, we should talk about that, that's a completely different thing.
Okay, well, they would just, it's the exact, well, in my head, it's the same thing, meaning that it takes your game records thou, you record thousands upon thousands of hours on, you know, this game, and then you create a model off of it that can generate your game, and then you're supposed to add in these features by kind of prompt maxing it into being like, okay, I want my game, but I want like surfboards. And then it makes surfboards and you go, oh, that wasn't fun looking. So then you add in new stuff. I think that's the entire goal of this project, because I believe this is along the exact same veins, which is how close can they accurately portray your game, make it playable, and then add new features that don't require any programming, just a simple prompt.
But the Copilot for Games one was the thing that's like clippy for games. Like, it comes up and you ask it like, where's the wood on this map? And they're like, oh, you have to go towards the forest to the northeast. Like, you know what I mean?
There was another one.
That's what Copilot for Gaming is.
Copilot has a lot of meetings these days. There was some Microsoft Game Studio AI that is doing what I'm describing, which is thousands of hours recording of a game and then attempting to replay and show you, like, this is your game on features.
Yes, that was a different thing, I think, not Copilot for Games, but I could be wrong about that. Maybe you're right. Uh, maybe they're both called Copilot for some reason.
I mean, everything's called Copilot for Microsoft.
Yeah, it's a good point. Maybe I've got the name wrong. Maybe it had a better name, I don't know.
They probably have three separate AI game-related things called Microsoft Copilot for Games.
You're probably right.
Or Copilot for Microsoft Games and Games for Copilot Microsoft, and so they're all different products.
Yes, by Xbox.
By Xbox.
For Windows.
You need the game pass for that one.
As many are as you can in there.
Yeah, so anyway, that's, that's all, that's all I could really say about this is, I don't, I don't, the specifics are what matter to real game designers, uh, and so the, this is just not specifics. These are more cosmetic changes that they're showing here, uh, things that the game designer would not really, like, they're thinking about exactly what will happen if I change a little bit the way we respond to like this mouse down, or what, like, that's what they're working at to feel, to get the feel for it. Everyone knows what it looks like if you walk around in 3D. It's not hard, like, that's not a prototyping problem we've ever had, right? Not for a very long time, anyway. So it just, this doesn't seem to be at the level of experimentation that they're working at typically, in my experience. But again, they'd have to say, not me. I'm the wrong one to answer.
OK, OK. So maybe in 10 years, this thing could be outstanding. But today, from your brief survey, it does not look like it necessarily fulfills the need.
It might even be outstanding today if they were training it more on the kinds of things that game designers actually work on, right? Like, the nitty gritty, right? Like I said, this is very producery, right? It's very much like, should the Valkyrie jump a little higher because it looks cool? Or something, right? Like, it's not asking the question of, like, what are the gameplay ramifications of the Valkyrie? It can't tell you, oh, the Valkyrie will win 20% more matches, right?
They, and that is a thing that they could be doing. So, you know, maybe they'll get there soon. Because that's the kind of thing, you know, just we analyze billions of matches. And then we can tell you, like, oh, if you, if you change the way that this thing worked, where you have to go to, you know, the Black King bar costs 20% more at the shop, it has this effect on games. That's something that game designers could use for, like, balancing multiplayer, like, let's say, right? This doesn't look like that to me. Maybe it does that and they just didn't say so. But it doesn't look like it can do those sorts of things. I could be wrong.
OK.
Was that a Dota 2 reference?
It was.
Oh, sick, alright, nice.
Nice.
That's TJ's love language.
Dude, we can go play that at Burn Convention. That'll be sick.
At Burn Convention. We'll all gank Roche. That's the only term I know.
Sweet, OK, cool. That's all I needed.
Alright. We'll make no further jokes about the Burn Convention, because they're all inappropriate from here on out.
Remember at React Miami? The whole, like, Burning Man thing kept coming up?
React Miami is the Burn Convention of, of developer conferences.
Josh, I don't even know what you're talking about.
You don't, oh, God, never mind.
Yeah, don't, don't.
It was funny a year ago.
No, stop talking about Burn Convention. Alright, hey, what's your sign-off?
Who? Anybody, you gotta start going.
My name's Casey. Trash. And the name is the primogen. Take us away, prim. Why would you just interrupt me like that? It was coming out, dude, that was so good. It was like bam bam bam and I was coming in and it ruined everything.
Flip will definitely get hard up in the video to take that out. If you're here watching the video now, Flip took it out, so it's good.
Flip took it out, it did not exist. Join us next week for another predictable disaster here on the standup. Next week, we have a great topic which you have suggested in the docs, Casey. Yes, we use docs by the way with this standup. I actually wanted to, I think that's just hilarious because I think I have like an internal itch that one day if I ever was forced to work for an evil company, I would do the exact same thing. And so it's just like, I have always wanted to, yeah. Anyways, what? Yeah, read the docs, loser. We'll scratch that itch, we'll scratch that itch next Wednesday. The name is the foreshadow agenda.
Page updated November 3, 2026 · transcript and summary are machine-generated