873: Standard Schema: The Universal JavaScript Data Interface

Wes and Scott talk about the standard schema for data validation, a collective effort by various library authors to create a unified interface. They discuss the benefits, how it works, and its impact on developers and libraries. Show Notes 00:00 Welcome to Syntax! 01:24 Brought to you by Sentry.io 02:44 What is Standard Schema and how does it work? Standard Schema Spec Fabian Hiller Valibot David Blass ArkType Colin McDonnell Zod 06:00 Benefits of Standard Schema 08:54 Implementation and usage 11:25 Is this primarily for end users or library authors? Hit us up on Socials! Syntax: X Instagram Tiktok LinkedIn Threads Wes: X Instagram Tiktok LinkedIn Threads Scott: X Instagram Tiktok LinkedIn Threads Randy: X Instagram YouTube Threads

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Wes Bos?

  Welcome to Syntax. Today, we're talking about standard schema. Schema, that's a word I'm gonna say wrong the whole episode, so buckle on up. This is a new effort from the devs. Fabian Hiller, he's behind Valibot, that's a JavaScript validation library. David Blas, he's behind a relatively new validation library called ArchType. And Colin McDonald, we've had him on the podcast as well. He is the creator of ZOD, TRPC. Basically, all of these validation library authors sort of got together and put together this thing called a standard schema. And today, we're going to explain to you why and what that is, and why this is good for you as the end developer.

• 0:45Wes Bos?

  My name is Wes, developer from Canada. With me as always is Scott. How are you doing today, Scott?

• 0:49Scott Tolinski?

  Doing good. Just got back from the mountain. I went skima-ing. I went to the mountain to go skiing. I actually did.

• 0:55Wes Bos?

  Skima-ing?

• 0:56Scott Tolinski?

  Yeah. Yeah, no, I'm doing good, man. Just hanging out, doing some creative coding and having fun. I'm excited to learn about this stuff a little bit, because I guess what I don't understand about this whole thing is why? I mean, I get that having a common interface amongst these libraries is great, if that's what this means. But in the same regard, then like, why not just collaborate on the library itself? Yeah.

• 1:19Wes Bos?

  Yeah, yeah, why not make that a standard library?

• 1:23Scott Tolinski?

  Yeah. But a large part about what these libraries are doing is oftentimes with data validation. Data validation, one of those big things that like, hey, if you're not validating or parsing your data correctly, you might get some errors in your code. Those errors, you might not know about them. And unless they pop up in your Sentry dashboard, because you're seeing something of property is undefined, and then all of a sudden, your users are having a bad time, because maybe your data wasn't being validated correctly, and you didn't know it was in there. So...

• 1:55Wes Bos?

  Let me tell you, let me interrupt your advert here. I got a perfect example for this. So, somebody asked the other day, how do you maintain that? Like, if you're scraping data from a website, like, what do you do when that changes, you know? Or if you're pulling... Data back from an AI is returning to you JSON. You have to still validate that that data looks like what you're expecting. And what happens when it doesn't look that way? What happens when the website you're scraping changes? Well, catch that error and then send that data off to Sentry. And you get alerts right away when that data is not looking like what you're expecting.

• 2:31Scott Tolinski?

  Yeah, that's important because hey, if the data isn't looking like what we expected, our code is going to break. So yes, check it out.

• 2:37Wes Bos?

  You're in trouble.

• 2:38Scott Tolinski?

  Yeah, sentry.io forward slash syntax. Sign up with the coupon code TASTY TREAT, get two months for free. All right, let's get into it. I'm stoked to learn a little bit about this stuff here because I primarily use ZOD. And ZOD is the only one, but this seems like a world where every other week, somebody's like, no, ZOD is out, you gotta use Valibot. Valibot is out, you gotta use archetype. Okay, what are we on today? Well, now we're on standard schemaing.

• 3:04Wes Bos?

  Yeah, so ZOD, Valibot and archetype are not being replaced. The idea with standard schema, well, let's peel it back a second. So a schema, if you haven't done this before, is you describe what your data looks like. And when you have a piece of data, you can validate it against the schema to make sure that it looks like what you're expecting. So a very simple example would be you have a schema for a person and their name is a string and their age is a number, right? It can be a lot more complicated than that. You can check that things are capitals. You can check for things that are formatted as a ZIP or postal code.

• 3:46Wes Bos?

  You can check that they don't go longer than a specific length, right? Whenever you're dealing with data in an application, you're going to need to validate that data before you go ahead and throw it in a database or before you display it to your actual user. And of course, we have ZOD, ValidBot, archetype, Yup is another popular one. There's, there's lots of them out there. So those are the schema definition and the validation libraries, right? And then on, on the other side of things, you have tools that need to validate your data before going forward. So if you think about Hano JS or Express JS, you have a middleware that's taking in a post request

• 4:31Wes Bos?

  from somebody that's filled out a form. And you want to validate that they have completed all of the fields correctly. You want to validate that they're not sending data that is not needed. You want to make sure that the length of the body that they sent is not the B movie.

• 4:47Scott Tolinski?

  Yes.

• 4:47Wes Bos?

  Because it has to be short, right? So you'll have to, to validate those, right? So Tanz stack router, Tanz stack form, Hano, Express. All of these libraries often provide a sort of a way to validate your data. We have like a little bit of a problem there because these libraries that need to validate your data, they either choose a validation library and consider it blessed. They say like, we use ZOD for validating our data. And if you use something else for describing your schemas and doing validation, then you're either out of luck or you have to write like a, like an adapter or a resolver

• 5:32Wes Bos?

  function that will sort of translate it from, from one to another. So the idea with standard schema is it's not a way to define what your data looks like. So the ZOD API, the ValidBot API, the archetype API, those are all different APIs and they've all approached it differently. They've not like, like got together and said, you know what? Let's just make one validation API to, to end it all. Those are still their own packages. So what they have agreed is a standard way to define several things. First of all, how to actually validate the data. So what's standard?

• 6:11Wes Bos?

  Schema does is anyone who implements standard schema will simply tack on this tilde standard onto each of the schemas. And that way, when you pass in a schema into something like Hanno or Tanstack Form or React Router, or anything that will support standard schema. So all these library authors will now say, okay, I don't have to, just pass me a standard schema. I don't care who made the schema. All I care about is that I can access a couple of things. First of all, I want to be able to call the validate method on your schema.

• 6:52Wes Bos?

  And I don't care who has made it. I'm just going to call a validate method on that schema. And then that validate method will be called and will be able to return data in a structured way. So the, the sort of the second standardization they've had here is that the return value from validate is always going to be in the same shape. And if you take a look at the actual standard schema specification, it's really just 58 lines of TypeScript that has very rigidly defined what the inputs and the outputs of this look like. So it's not even something library authors have to necessarily

• 7:36Wes Bos?

  npm install. It's just 60 lines of TypeScript that they just have to adhere to that will allow library authors to know that the inputs and the outputs are always going to look in a specific way. If you do take a look at the TypeScript code, you can see that the result is either going to be a successful result of whatever type you passed in, or it's going to be a failure result, right? And then the failure result is going to be an object with a property of issues. And each of the issues is going to be an array of individual issues. Each issue itself has a message property on it. You see what, you see what's happening here is that you normally have to write like a translation

• 8:20Wes Bos?

  layer between these libraries. And if you think you have five or six different large validation libraries and

• 8:29Wes Bos?

  Forty, fifty different libraries that consume schemas. Then you're talking about like five or six times fifty, right? Then you're talking about two hundred and fifty possible adapters between the two. And it just gets out of control very quickly.

• 8:45Scott Tolinski?

  Yeah, that sounds like a nightmare to support all of that stuff. Yeah, so I guess I get this now, okay.

• 8:52Wes Bos?

  Yeah, the other benefits that it has is it's a standard ways for inferring both the input and output types from it. So you can see here, I have, there's an infer input, infer output and a types value. And that will give you, if you pass it a schema, doesn't, again, doesn't matter what type of schema. Could be a ZOD schema, could be valid bot, could be archetype. It will always give you.

• 9:18Scott Tolinski?

  The right, yeah, type.

• 9:19Wes Bos?

  The right returned type, which is again, a lot of like, you think about somebody like drizzle that needs to like maintain the drizzle to ZOD adapter. Now they don't have to necessarily do that for every single validation library out there. You can simply just use the standard schema inside a drizzle. And then they haven't shipped support for this yet, but I imagine that this would be something that they will ship support for relatively soon.

• 9:45Scott Tolinski?

  Interesting, cool.

• 9:47Wes Bos?

  So last thing we have here is like a, a library that actually consumes a standard schema will simply need to look a little something like this where they can run the validate method on it. Again, they don't have to care what library is being used. And then they will either get the resolved value, which is great. Or they will get an array of those issues that I talked about. And at that point, it's not up to the validation library. It's up to the framework. What do you want to do now that you have validation errors, right? Do you want to throw them into state to display an error message? Do you want to redirect the page? Like, is it a post request where you need to redirect the user to a different page?

• 10:30Wes Bos?

  Do you need to return the error as JSON? Do you need to translate those errors into some human readable thing or something that is safe going from the server to the client? And that's up to you, right? That's, yeah. Validation is just a, a primitive. And then the actual what happens at that point is up to whatever library framework flow control you're actually using.

• 10:54Scott Tolinski?

  Wow.

• 10:55Wes Bos?

  So, yeah, that is, that is the whole idea behind standard schema. It's, it's really interesting because like, again, it's 60, 58 lines of TypeScript. But it was, it seems like a pretty big effort and a lot of thinking from all of these people to come together and agree on what this, these shapes will look like. And then this should make life, again, much easier for library authors and that need to actually implement validation.

• 11:25Scott Tolinski?

  Yeah, I, you know what I'm interested in this is, like, do you primarily see end users using this or just library authors?

• 11:33Wes Bos?

  I don't foresee end users using it a whole lot unless it is something like, the inference is really nice. So like, like one thing with drizzle is like drizzle has a property on it that's called like infer. You can, you can take the drizzle schema and you can infer your return type from the, the actual schema, right? And I could see this being a really nice way to just sort of standardize how inferring those values work. But I see most of the actual heavy lifting here, simply just being done both by the validation libraries that implement standard schema and the libraries like routers and Hanno and Express

• 12:17Wes Bos?

  that actually consume these types of things.

• 12:19Scott Tolinski?

  Interesting. Cool. Well, thank you for the rundown. I saw this pop up. It's definitely something that's in my wheelhouse, but to be honest, did not really understand what the deal was. So I really sincerely appreciate it, man.

• 12:32Wes Bos?

  At first I thought like, oh, are they coming out with a new validation library? But that, that is not the case.

• 12:37Scott Tolinski?

  Not the case. Well, glad to hear. And interesting stuff. I think this is good. Having any sort of standard connection or whatever is only good for being able to support all these different libraries that we're constantly inflicting ourselves with. So.

• 12:51Wes Bos?

  Totally. Yeah. And it, it only opens the door to like more libraries to, to be created. So if somebody has an idea for better validation libraries, like archetype came out. They, they do it entirely with TypeScript types. It's pretty nifty. You can automatically just start to use it in, in whatever library you're using. You don't have to like wait for support to be implemented.

• 13:12Scott Tolinski?

  Yeah, that's great. Fantastic. Cool, man. Well, thank you so much. This has been really interesting.

• 13:17Wes Bos?

  You're welcome. All right. Peace.