Synthetic Data with Alex Watson, Founder of Gretel AI

In this episode, Nathan interviews Alex Watson, founder and CPO of Gretel AI, about the company's work in synthetic data. They discuss why we need synthetic data, Gretel’s new pre trained tabular LLM that creates synthetic data on a zero shot basis, privacy techniques to prevent LLM memorization, and more. If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period. We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com. SPONSORS: Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive ORACLE: With the onset of AI, it’s time to upgrade to the next generation of the cloud: Oracle Cloud Infrastructure. OCI is a single platform for your infrastructure, database, application development, and AI needs. Train ML models on the cloud’s highest performing NVIDIA GPU clusters. Do more and spend less like Uber, 8x8, and Databricks Mosaic, take a FREE test drive of OCI at oracle.com/cognitive NETSUITE: NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. OMNEKY: Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. X/SOCIAL: @labenz (Nathan) @AlexWatson405 (Alex) @Gretel_AI @CogRev_Podcast TIMESTAMPS: (00:00:00) - Intro (00:03:02) - Origins of the company name Gretel AI and initial vision around enabling data sharing while protecting privacy (00:05:16) - Alex's background in data privacy and compliance from his previous startup Harvest AI, acquired by AWS (00:07:24) - Using synthetic data to create additional examples and improve detection of rare disease (00:12:50) - Why use synthetic data? (00:17:02) - Sponsors: Shopify | Omneky (00:19:00) - Training models to recreate real-world data distributions and using validators to detect unrealistic outputs (00:24:40) - Pre-training the Gretel tabular LLM on diverse internet data sets (00:30:40) - Sponsors: Oracle | Netsuite (00:34:00) - Using an agent planning architecture (00:39:41) - Example use case of adapting models with synthetic data samples for long-tail cases (00:43:00) - Using reinforcement learning to intentionally generate more diverse and representative synthetic data (00:51:16) - The potential of efficient, lightweight models compared to massive LLMs like GPT-4 (00:56:00) - Analogizing model specialization to specialized parts of the brain (01:06:04) - Using differential privacy techniques to prevent memorization and exposure of private data (01:14:37) - Adding noise during training to blur memorization (01:18:42) - Optimism that synthetic data quality issues reflect details not fully understood yet vs inherent problems

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Your data is messy, it has gaps in it. I can't create new additional examples. It's too expensive or there's no way to go back to it. So we really focused our efforts on first and foremost, helping you build better data. That's been the kind of the guiding light. That's what we're really aiming for. You know, no LLM today can generate 100,000 or a million row dataset. So the first purpose of the agent was interpreting that user query that's coming in and then figuring out how to divide it up into a set of smaller problems that the LLM can work on one problem at a time. The promise of a really lightweight model, really fast model, like shows the power that you can have of taking a domain-specific dataset you have or task and doing something meaningful without having

• 0:44Speaker 00

  to do something at the GPT-4 scale.

• 0:47Nathan Labenz?

  Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs and builders working on the frontier of artificial intelligence. Each week, we'll explore their revolutionary ideas and together, we'll build a picture of how AI technology will transform work, life and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Torenberg. Hello and welcome back to The Cognitive Revolution. Today, my guest is Alex Watson, founder and chief product officer at Gretel AI, the synthetic data platform for developers. Synthetic data is a fascinating topic. Since the early days of deep learning, it's been well-known that training computer vision models on a mix of original and programmatically altered

• 1:30Nathan Labenz?

  and degraded images ultimately improves model performance. It seems that learning the concepts through the noise boosts robustness to the random unseen oddities that models inevitably encounter in the wild. And more recently, dozens, maybe even hundreds of papers, have explored how LLM-generated data can be used to improve training sets and ultimately model performance on a wide range of problems. Yet at the same time, some research results and many observers of the evolution of the internet in general have cast doubt on just how much synthetic data the system can absorb before models begin to lose touch with their real-world origins or otherwise degrade.

• 2:11Nathan Labenz?

  With these questions in mind, I reached out to Alex, who's been building a business on synthetic tabular data generation since 2020, and who proved to be an amazing guide to this domain. While synthetic data might sound like a niche topic, I think this conversation will be of general interest. We started with a discussion of why we need synthetic data, how Gretel has trained specialist models to maintain realism while also preserving privacy in creating it, and how we can be confident that we can trust this data for analysis, testing, and, yes, AI model training. Along the way, we also explored the trade-offs between statistical realism and social manners, the impact of LLMs on Gretel's business, and the new pre-trained tabular LLM that they've recently introduced

• 2:55Nathan Labenz?

  to help create synthetic data on a zero-shot basis for a wide range of data types and scenarios. We even took a detour into AI regulation in the wake of the recent Biden White House executive order and the UK AI Safety Summit. This episode is a great example of why I love making this show. I learned a ton in the preparation and had a lot of fun with the conversation, and I think you will too. If so, I always appreciate it when listeners share the show with their friends. And of course, we invite your feedback via our email at [email protected] or via your favorite social network. For now, I hope you enjoy this conversation with Alex Watson of synthetic data company Gretel AI. Alex Watson, welcome to the cognitive revolution. Appreciate it. Thanks, Nathan. Excited to be here.

• 3:40Nathan Labenz?

  Yeah, this is gonna be great. So you are the founder and now chief product officer at this company Gretel AI. I love to hear how you come up with that name, by the way. But what you guys do is synthetic data, and I'm just so interested to learn so much more about it. It's been really eye-opening to explore the product a little bit. You do some of the best live product demos that I've seen. Your recent 12-minute YouTube short, I thought, was really good.

• 4:08Speaker 00

  I appreciate that. Thanks.

• 4:09Nathan Labenz?

  I think, yeah, I think there's going to be a ton of fun. So tell me, where did Gretel come from? Give me just a quick backstory and then let's talk about like, why do we need synthetic data?

• 4:16Speaker 00

  The original vision for Gretel was around a better way to make data that we can't make accessible accessible. And it's evolved quite a bit, like synthetic data has so much more kind of capabilities and promises that we've discovered over the past about three years of running our business. But it was a reference to the like, the digital breadcrumbs that we leave behind. And really an effort from our company with using synthetic data to make enable data sharing at a scale that hasn't been possible before. Imagine hospitals sharing medical records, research institutions, financial companies sharing data in a way that doesn't compromise consumer privacy. And that's really where we started. So you'll see that those like, you know, as we kind of go through the technology we build and talk about

• 4:59Speaker 00

  differentially private training and things like that, you'll see some of that come through in our product. You know, we've expanded that that vision, that scope quite a bit, but we really started around privacy and around the idea of protecting individual privacy, but enabling learning and data sharing at scale.

• 5:13Nathan Labenz?

  Yeah, it's awesome. So the big two value drivers today, obviously you kind of are, you know, starting the founding premise is privacy. And now there's this kind of, you know, massive takeoff in AI and, you know, so many people training stuff and trying to figure it out. And so the other big use case that we're seeing is improving the data that people are feeding into their training processes. So tell us a little bit about that one as well.

• 5:39Speaker 00

  Yeah, maybe I'll maybe start with the history of how it happened and it actually happened like incredibly early in our company. So for a brief history, prior to Gretel, I was a co-founder of a startup, security startup called Harvest AI. We built products that help customers like scan and detect important data in the cloud. We ended up getting acquired by AWS in 2016. We're going out for a series A raise and got approached around launching that service as a AWS service. So I was a GM there for about four years for Amazon Macy, which people used to scan AWS Cloud for important data and saw that even the biggest, most, you know, cloud data, incredible data companies struggled with enabling internal access to data, right? So Pinterest of the

• 6:21Speaker 00

  world, the Airbnb's of the world and things like that. So you saw what a problem this was at scale and also the power of when you can make data accessible, like AWS, we had a, at the time, 500 person compliance team that could work wonders for making data accessible. So we started out with that privacy thing and our first open source example, we released in 2020, right? Actually, I think about a week before the pandemic hit and open source ability to essentially, we used a language model. This is 2020. So we weren't using transformers at the time. We're using an LSTM where we had started to partner with the Google Tensorflow team around technology called DPSGD, which is enables you to train models with differential privacy. So you can make sure they won't memorize secrets.

• 7:06Speaker 00

  But one of the early features that we had was the ability, just like we all do today, to prompt the machine learning model and ask it to create something new. So our first real experiment was saying, can a language model like an LSTM, instead of learning a language and text, can it learn to recreate the distributions inside of a data set? So we really started focusing on tabular data around 2020. And that can be mixed numeric, categorical, text, data, anything in between. And then we had the ability to prompt the model where you could give it a subset of those features, right? Like given a zip code and a ethnicity and a date, generate the rest of this record for me. Very early in our journey. I think the first time we had this was working with some researchers at

• 7:50Speaker 00

  UCI, so University of California, Irvine. And they were doing, they were working with a rare disease data set that was highly imbalanced, right? So you have like thousands of patients, but the only people inside that data set that had the really rare disease were in the 10s to 20s.

• 8:07Speaker 00

  So the question was, can we address some of the representation bias here, first of all? So like essentially boost that minority class? And if we do that, can we improve the detection for this disease? So essentially, the idea is using synthetic data to create additional kind of labeled examples when they weren't able to go back and recreate their experiment or their their collection. And can that data set be used to improve downstream machine learning training? So the idea is it introduces new examples that weren't in the training data and that'll help the machine learning model. And we had a lot of success there. And since that point, I think we've seen more and more focus and, you know, fast forwarding to today and kind of happy to talk about where, you know, kind of bred a list today and what we're seeing today, but I'd say it's about 50 50.

• 8:50Speaker 00

  One corner, we have as a value driver, safe sharing of synthetic data where we can create data that has up to mathematically, you know, provable privacy guarantees. And the other area where we're saying, hey, how do we improve on machine learning data sets? And this can be tabular data for fraud detection, for ad recommendation systems. It can even be text data. And there's such cool kind of research coming out recently to support that where essentially using an LLM to create additional like diverse examples, like it was mentioned in the Microsoft 5155 paper.

• 9:24Nathan Labenz?

  Yeah, there's a lot of connections here. I mean, right off the bat, I'm just thinking curriculum learning. That's such a huge theme in my mind these days, you know, the the ability to get smarter in terms of what data you feed into even the pre-training stage and, you know, enhancing, filtering, enhancing, curating, boosting, you know, so many different manipulations there. But this one is probably one of the most intuitively obvious where especially you think like rare diseases, you know, it's just not in there that much. And that makes it hard for the gradient descent to to reinforce, you know, to reward the learning of it. So boost it up a little bit and, you know, next thing you know, you're getting better performance. I mean, so many, so many opportunities like that.

• 10:09Nathan Labenz?

  And that's kind of what was the light bulbs that were going off in my mind as just a, you know, application builder today, too. I was like, boy, I see just so many quick patches in my future of, you know, rare, rare cases that I want to handle better. So I think that's super interesting. Just going back to the the Amazon thing as well for a second. So basically, this is, because I do love to kind of also contrast recent history, you know, approaches. You're scanning for important data. Does that mean you're like, like I imagine, you know, this sort of five years ago version of that was just a whole Swiss Army knife of different explicit techniques, like regular expressions and like classifier, you know, you kind of

• 10:52Nathan Labenz?

  a handful of classifiers. What did that thing look like? And now today, also, I'm kind of like, maybe I'd use Claude Instant, you know, and kind of clean out a lot of that old code. What do you think? Is that a reasonable intuition?

• 11:05Speaker 00

  Yeah, that's one of the reasons I am so immensely grateful for the LLM technologies and transformers that are out is that there is a light at the tunnel for people doing traditional NLP and NER of a better, more general way to do it. So really excited about that. But you're, you're exactly right. So going back to, to Macy, it uses a combination of traditional like named entity recognition technologies, as well as, as you were saying, regular expressions and things like that to help identify any type of personal data that might exist in the cloud and label it. So you knew where it was. And it would really take a look at it and say, is this exposed to the internet? Is this shared to outside organizations and things like that? And kind of give you the visibility that you needed

• 11:47Speaker 00

  across your organization. The real goal was to enable, you know, developers to make decisions about what tools to use and use the best available tools, but also get like the enterprise visibility necessary like for that to happen. You know, I think the big challenges we faced doing this in Amazon scale was we went from a startup that had, you know, a couple amazing customers to the first week that we launched Macy, we had 6,000 customers. We were doing named entity recognition at, you know, up to petabyte scale. So so much time focused on how you make that even traditional ML technologies kind of work at the scale. Part of the reason I'm so excited about technologies today is just the amount of specialization that was required or tuning anytime your data, your data characteristics changed that

• 12:31Speaker 00

  were required. And now it's really as much as all of us probably get annoyed with the need to prompt tune and do things like that. The promise of an LLM that can understand your kind of natural language question and make that change for you automatically is really, really cool.

• 12:45Nathan Labenz?

  It's certainly a game changer in, in so many different respects. Coming back to the present and the synthetic data, yes, it's kind of unfolding today. There are a number of use cases that you guys highlight in your, your product and your demos. I'd love to hear you kind of talk through a few more beyond the boosting of the underrepresented set. One that jumped out to me and I think really highlights the challenge is insights, right? The idea that, you know, and I can just imagine, I've, you know, done a lot of data analysis in my time, right? And it's like, okay, I certainly hear why, you know, at the corporate level, you don't want to be passing around the crown jewel data set, right? I mean, did some work with Rocket Mortgage, for example, and, you know,

• 13:27Nathan Labenz?

  the care with which they, you know, maintain that, you know, their customer data, access to it, all that stuff. Like, you know, it's, it's a serious, serious effort. So you can't just be passing stuff around like crazy. That makes total sense, but then when you say, okay, well, and it's super creative, you know, concept. Well, instead of, you know, having to deal with all that, you know, we'll just make fake stuff and, you know, and use that instead. But insights, I was like, okay, boy, insights, you know, it's, I'm going to need some real theory to start to trust, you know, that you can make fake data that, you know, is enough like whatever, you know? And that's obviously something that probably most people are going to struggle to wrap their heads around. Well, how, how do you kind of define that, prove that whatever such that I can actually do my sort of pivot tables on this

• 14:12Nathan Labenz?

  and trust that what I'm, you know, getting, you know, is making any sense? I've been thinking about that a lot and I'm really, I've got some guesses, but I'm really interested to hear, you know, more about the kind of, you know, the provability of how this stuff works.

• 14:24Speaker 00

  Yeah, our approach, and I think the one that seems to be gathering a fair amount of steam in the synthetic data world, is to train a model and of course, we're like minimizing the loss function as we're training it and doing the best we can. But that doesn't tell you how the model's going to capture the real world distributions that you care about and the ability to play it. So for us, like regardless of the modality, if it's text, if it's tabular, if it's time series, it really starts with having the model kind of master the ability to recreate data matching the same distribution as the real world data was trained on. And if you can have confidence in that, you can start to alter the distribution for whatever your task is. So how we do that, we train the model at each iteration.

• 15:07Speaker 00

  And really, at the end, we sample a bunch of data from the model. About a one to one equivalent of the real world data. And then we essentially throw, from a statistical perspective, throw the kitchen sink at it. We have two ways of measuring, you know, one, I would say, is meant to be as objective as possible, and the other is meant to be kind of task specific. So we have something called our synthetic quality score. What it's doing, it's easy to walk through from a tabular perspective. We actually have similar scores for text and time series as well. But we sample a bunch of data from the model. We look at pairwise correlations and that creates part of a composite score. We look at the per field distribution. We even do like PCA distributions for each field.

• 15:50Speaker 00

  And then do a distance metric between the real world data and the synthetic data. And the idea is to give you a one through 100 score that you can look at and you can reason about and say, if this is above 80, we expect it to work well for the types of machine learning use cases that most people use data, synthetic data for. If it's below that, maybe that works for your use case. Maybe your use case is just testing or something like that. But as you were saying earlier, you don't want to create pivot tables on that. So really, we start with trying to give you that sense of confidence. We've added in the ability, and really just after seeing a lot of customers do this, to automatically test a downstream task for your data as well. So after the model's done training, we can run aggression or a classification task or things like that automatically within our platform.

• 16:33Speaker 00

  We have a lot of customers that use like Vertex or SageMaker or things like that to run this as well. So we just built it into the product, so you didn't, not everyone had to write code. But I think a mixture of that somewhat completely objective, not task specific score that is a good general indicator. And then also that like understanding of your task, like what you want to do with the data and making sure it conforms to those, those expectations. Feels like the way to get that sense of confidence you need.

• 17:01Speaker 02

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 17:05Nathan Labenz?

  Yeah, interesting. So could we unpack the, the kind of loss function a little bit more? Cause I'm kind of wondering about the relationship. That's all pretty quantitative stuff, right? It's a, it's a code base, ultimately, a, you know, sort of test suite that you can execute. On any data set that comes through and sort of say, we're going to characterize what you gave us. And then we're going to characterize what we generated and kind of show you that that hopefully lines up, you know, distribution wise. On the generation side, and it's important to probably keep in mind for folks because we're also used to one token at a time, language models. I'm very much thinking of your latest kind of, you know, tab L L M that, um, that you demoed. But, you know, it might be worth kind of distinguishing too between that latest thing and the, you know, the sort of set of more purpose specific models that you have.

• 17:50Nathan Labenz?

  But I'm kind of imagining for the main, for the, the new big one. It seems like there is a really natural and kind of an insightful thing here, maybe for people, where there's like a decoupling of the prediction, which is the distribution. And then like sampling from that distribution. I think this is something that people maybe don't conceptualize super rigorously. But like the, the task that you have sort of helped, helped me, I think, at least crystallize it a little bit. So like, most people have a general sense, right? At the end of the language model, you're putting a prediction to every token. And, you know, then you can, with your temperature setting, you know, this is the, the experience that people are most familiar with.

• 18:33Nathan Labenz?

  If they're an A I engineer or whatever, you can turn that temperature down to zero and you can get like the most likely token, or you can turn that temperature up and you can randomly select from those probability distributions. But in the kind of practical experience of it, we really only see one token. And in the training, also there is like a ground truth text document, you know, that is kind of firing one on the actual token and like zero on all the other tokens, right? So it strikes me that your situation is a bit different here where you can potentially define the target as the distribution and, and just like directly optimize and, and form the distribution of the predictions to the distribution that you've like characterized from the data.

• 19:18Nathan Labenz?

  And then the sampling from that becomes like a, you can kind of understand it in. I mean, it's the same fundamental thing, but the difference between that one token being right and the we want to generate the actual distribution seems like something that was really helpful for me to wrap my head around over the last couple of days.

• 19:39Speaker 00

  Yeah, yeah, that's probably a byproduct of starting working with tabular data where that rather than looking at a row being generated at time, we were using a variety of models when we started. We started with LSTM, we used GANs, we used diffusion models, and now, as you mentioned with our tabular LLM model, we use transformers. A byproduct of how we kind of built our product is we end up looking at the row level. So every time a row of data is generated, then we examine everything. Similar if you're generating a sequence of new LLM instructions, for example, right? Rather than looking at the per token, what we're going to look at is the, you know, per line or per record distribution. So essentially, we let the model generate everything.

• 20:23Speaker 00

  The first step during training is we're sampling and we're looking at it, but also when you're using the model for inference. So when you're asking the model for data to come out, there is the risk that the model's going to hallucinate or invent something new that no one wants to have happen. So we have a secondary level of validation. We call them, not very creative, we call them validators. But essentially, what it's doing is it's looking at all the outputs in the model and asking how different it is versus the original data that it was dropped, that it trained on. And you have the ability to filter out things that are too far outside of the distribution. And the idea there with the tabular data was to make sure that we didn't invent anybody's, you know, age that was 135 inside of a dataset, but it works really well for text data as well.

• 21:06Speaker 00

  You know, just when the model kind of goes off on a rant or invents something that's way outside of what it should be working with, you have the ability to filter that type of data out. And it helps you have more confidence in a generative model that it's going to give a usable response. Another kind of cool thing is that With so much of the focus for synthetic data really being on creating machine learning training sets, you can't have someone kind of looking at every record in a row and saying, yes, this is fine, this isn't, this is fine, right? So we've really focused on making sure that when we generate data at a thousand or a hundred thousand records, or even a million records, that you have confidence those records are kind of match your expectation. So I think that's another really kind of neat thing that I see happening to, you know,

• 21:50Speaker 00

  go back six months ago, and there were so many questions about, I want to use this model, this LLM for summarizing content on Reddit or things like that, right? And that the risk was that it would summarize something that was off base and would be an inaccurate summarization. And I think technologies like what we built for text scoring, there's been a few, open source metrics have been released recently, really help you quickly check and reason about a generative model's output in a way that would allow you to serve the results to customers without having to have necessarily like a human look at it. So a nice kind of quick AI check on data makes these models like so much more usable.

• 22:27Nathan Labenz?

  How do you do the pre-training? And how kind of big of a foundation model is this? Again, I'm so fascinated with the TAB LLM. I'm kind of imagining that like you've gone out and just assembled every public data set you can and, you know, in a sense, kind of taught a statistical world model to this thing. So it's supposed to have kind of all the right priors, basically, right? How do you go about creating and kind of validating that strong baseline?

• 22:57Speaker 00

  Yeah, so for some background, we are about to release, for listeners here, we're about to release a model, we're calling it TAB LLM, we're Tabular LLM. What it is, is a agent planning and execution architecture built to help people work with tabular data using natural language queries. And really at the core of that is both the agent that is, you know, making a decision about whether to use an LLM to generate data, or whether to use one of our tools and write code to generate data to serve your response. And what we're referring to here is the actual LLM model that we have fine tuned on data sets from across the internet. So it's one of the first examples that you'll see of an LLM that's meant to work with tabular

• 23:42Speaker 00

  data. Tabular data can be text, time series, numerical, categorical, any combination of those. The approach we took, and I think this will be a constant evolution from us. The initial approach that we took was exactly like you mentioned, Nathan. It was crawl the internet, specifically crawl GitHub. Find any accessible data sets there, Kaggle, things like that, anything with an open source license. One area we were particularly lucky with was I was noticing a lot of times like machine learning papers will reference data sets actually in the readme. So there's really great data linked inside readme's there and we could pull down the license and really kind of understand if it was usable or not. But the ideal was to train a LLM that would be used for a data generation task on what good data looks like.

• 24:28Speaker 00

  And something interesting is that while we all kind of feel that LLMs today are trained, it's mostly accurate on almost all of the content that are on the internet. If you're working with an OpenAI model or a Palm or even a Llama model. But these models really aren't trained largely yet on tabular data. And tabular data also introduces some kind of interesting challenges. In the sense that when you look at the context window that are available to LLMs today, you know, which is on a great LLM, let's say like 16,000, 16K tokens. It doesn't translate into a lot of rows in a typical tabular data set, right?

• 25:08Speaker 00

  So 16,000 tokens, assuming 50 tokens per row is going to give you about like 350 rows. So I think most of us aggressively work with data sets much bigger than that. So one of the things that we noticed as we started working with LLMs and asking them to generate tabular data. The power of asking an LLM to generate tabular data is one, they are just by a byproduct of how they work really good with time series type data. There's been some cool research about that recently. Second, it allows you to apply like a global level of knowledge to your data set. So one thing I think that's really resonated with our users on the platform is realizing that your data set is awesome. Everybody's data set is unique and really cool, but it's also like in some way limited, right?

• 25:52Speaker 00

  You don't have enough data. You never have enough of the examples, Nathan. You were mentioning kind of the long tail of data that you you deal with and finding a more systematic way to work with it. So the idea of applying a model that is seeing most of the data sets on the internet to that problem and saying, can you help me create some new kind of meaningful variations in the data to help a downstream model generalize is really powerful. So that's where we started. In the initial model, we haven't done for the Tabat Lim model, we haven't done anything like super clever with how to encode or model numeric distributions. Rather, we just treat everything as text and it goes through there. As I mentioned earlier, our first approach was crawl the internet and train it on everything.

• 26:34Speaker 00

  And I think very similar to other like research and academic work we see right now. I think a more curated, highly diverse set of high quality examples is the way to go. So we're seeing our team really kind of work on that. And some of the opportunities here is that, you know, even the GBT fours of the world, when they've seen tabular data sets, it's usually a table on Wikipedia or something like that. So it's a couple hundred rows at most. They just haven't, the L L M's have not learned that it's important sometimes that the relationships across the data set might be thousands of rows or hundreds of thousands of rows. So that's a real kind of neat application we're looking at right now is what if we train L L M's on a much larger context length and much more data?

• 27:17Speaker 00

  How good of a job they do learning the like the kind of subtle insights and distributions of the data that'll help improve ML generation when you're using the model?

• 27:27Nathan Labenz?

  Yeah, I bet quite a bit. I'd love to hear a little bit more about the agent kind of structure because I'm imagining kind of, you said, okay, you generate one row at a time. For one thing, like the order really kind of matters there. You know, I wonder if you, if you have a sort of systematic approach to kind of reordering fields, because there's been some interesting research lately that like, you know, A implies B does not imply, you know, that B implies A from the language models perspective. And then I guess like there's sort of a. Kind of sequential probabilistic evaluation, you know, where you'd be saying, okay, if I'm once kind of at least some amount of pre-training has been done, you know, if I were to give you like the zip code as the first field, then you would expect to see like reasonable demographics back

• 28:11Nathan Labenz?

  just based on that zip code. But then you'd sort of have, you know, depending on the, you know, the first variable that you predicted, you would have like a very different sort of, you know, conditional distribution for subsequent, in all sorts of varying ways, correlated variables. So you're kind of doing like a little Markov almost process randomly down the like Plinko board of of possibility and then, you know, going back and evaluating each each token, you know, for its kind of conditional accuracy, right, or conditional like real representation. Is that conceptually right?

• 28:48Speaker 02

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 28:51Speaker 00

  One small modification I would make to that is that we have found the more data you sample from a transformer L M based model for for tabular data up to the level that the L M is capable of working with. So let me give an example there. I'll start there first. Let's say you're working with Llama two or you're working with OpenAI is like 16, 16 K context window model, right? It might be capable of generating all that data, but if it's never learned that more than a couple thousand tokens are relevant to a data set, you're going to start to lose some efficiency as it generates more and more data. So what we do is we sample for from our model and our trained model up to as many tokens as we can at a time. And then we evaluate it row by row.

• 29:35Speaker 00

  And the the purpose of the agent is realizing that with current L M technologies, there's there's a couple of purposes of the agent, but the first one, the most obvious one is that, you know, no L M today can generate 100,000 or a million row data set or can go in and edit your data set, which is a really popular use case for us right now, right? If I want to add new fields, if I want to summarize product reviews, if I even want to just search for anomalies across my data, we've got to be able to process data way bigger than what an L M can handle in a single batch. So the first purpose of the agent really is to take a complex user query, for example, you know, create a demo data set with a spike in sales activity in November, I want a million rows.

• 30:19Speaker 00

  Or if you're editing data, you know, convert this unit from Celsius to Fahrenheit across my entire data warehouse or things like that. The the agent's first goal is interpreting that user query that's coming in and then figuring out how to divide it up into a set of smaller problems that the L M can work on one problem at a time. So a good analogy there would be if you asked in the N L P world, if you asked G P T four to write a book for you, you would probably get a really short book and you want a novel that's got several hundred pages. If you were able to divide that up, take that problem, what someone's asking for, and divide it up into smaller problems, like write a paragraph or a chapter at a time. You could see how an agent planning an executed base architecture that would say, okay, first step, I need to write introduction.

• 31:03Speaker 00

  Next step, I've got to have like character, you know, growth and start to work on the character arc. And finally, I need the conclusion and things like that. And it can divide those up into smaller problems. That's the approach we're taking with the data set, either editing or data set creation, where we've got something that is breaking it down into a step by step problem that a smaller, you know, in this case, our data generation L M can can work with and start generating data to high quality data for that particular window.

• 31:33Nathan Labenz?

  So is that like a it's more of kind of a instruct type model that is creating kind of code as policy outputs? And then, you know, a dedicated actual data point generation model that is kind of receiving those commands and doing the sell by sell.

• 31:52Speaker 00

  Exactly.

• 31:53Nathan Labenz?

  That allows you to put language models in too, right? I mean, I saw one of the demos was like, you know, reviews of the product. And obviously, that's a pretty different situation from the tabular data. I assume that's kind of a little more random somehow, or like it seems like it would be kind of less harder to give a representativeness guarantee on like customer reviews?

• 32:15Speaker 00

  We've got some research which I'll link over to you on how we assess the quality of the text based on what you're looking for. But so often, data sets are mixed. Like, imagine EHR data where you've got doctor's notes mixed with initial observations from patients as they come in. So I think that happens quite a bit. So we try to learn across all of them. And one of the kind of interesting things is that you don't necessarily want your LLM to do everything. And that's maybe the other part of the agent planning based architecture. If you were asking for an incrementing ID or a Fahrenheit conversion, we've got a neat example. We're doing like a maybe a high school kind of like physics level problem. The LLM will approximate, but you don't want it to actually approximate your answers.

• 32:58Speaker 00

  You want the real answer. So the other part of, I think, making synthetic data using this tabular LLM work at scale is having the LLM just recognize which areas are best to calculate or compute and doing that for you automatically. I think we all see that a little bit. If you're experimenting with GPT-4 or ChatGPT and you ask it to, you know, help you work on a data set. Sometimes it'll give you a data set back. Sometimes it will give you back code that you could use to solve a problem. And really, that's the type of stuff that we are trying to streamline. We're essentially applying the agent to realize when something should be a Fahrenheit to Celsius conversion, right? Just multiply by point six. And you get the right answer. You don't have to have an L and figure that out. So the first step is

• 33:42Speaker 00

  look at that user query, figure out, given the available tools that I have, can I solve this problem with code? If so, execute that code, get that into the data set. So you have high confidence in the answer. But for other things that require that kind of like level of knowledge or intuition that an LLM would have, you know, summarize this review as positive or negative, you know, things like that, that require you to look across fields and understand natural language text, then we use the LLM to fill in that data.

• 34:09Nathan Labenz?

  It comes together very nicely, I think, in the in the product demo. I'm definitely excited to spend a little more time with it. I do think it'll be really helpful. And it's probably a good idea also to just kind of contrast this as you started to do a little bit with trying to use GPT-4 or, you know, certainly any of the RLHF models. I think you have just like very fundamental problems here. And that's kind of where, you know, even for a project like mine. So, you know, where I'm thinking of applying this immediately is You know, we have this script writing model and its job is to write a video script for a given user who comes to us, you know, often naively and, you know, we kind of grab some content off

• 34:48Nathan Labenz?

  their website or whatever and figure out who they are. So it's extremely, extremely diverse. And you might say like extremely sparse. We have like a, you know, healthy usage, but we're not that big. It's a big world out there. So especially internationally now, different languages, you know, just kind of all sorts of long tail stuff that we have not previously put into our data set, but you know, can at any time kind of come our way. And then I think like, okay, you know, if I want to do some patching of my fine tuned, you know, 3.5, which is my, you know, kind of currently the, you know, the state of the art best thing that can, you know, best nail our task, then like one is probably not quite enough, you know, to get it to, to really learn the, the

• 35:33Nathan Labenz?

  pattern I wanted to learn. Five to 10 in my like broader mix of a few hundred samples probably is, but you know, I kind of want to create something. First of all, I, especially if it's an unfamiliar area, it's like very hard to even know what to do, right? You show some examples where it's like France, you know, and I'm like, oh God, I don't even know the fucking, excuse my language, excuse my French. I don't even know the like structure of the postal system in France, you know, let alone how to like make semi, you know, realistic examples that I would want, you know, to throw into 3.5 fine tuning. Uh, so if I'm like making this up totally on my own, I'm, you know, just seem destined for underperformance for just kind of garbage in problems. And then,

• 36:16Nathan Labenz?

  you know, if I, if I ask GPT four, I'm just like, it's going to be so RLH def, you know, to mode collapse on things like this so often that, you know, just like it answers, you know, 42 or whatever, you know, or 97, like way too many, you know, percentage of the time when you ask a random number, you know, here I just like, don't trust it at all for that sort of kind of representativeness. And, you know,

• 36:40Nathan Labenz?

  I think OpenAI would readily agree that like, yeah, you should not use it for that. Obviously, it's been trained for a very different purpose. So that, to me, is exciting. I think gives you guys a real different like position in the market that, you know, is is so distinct from the kind of mainline AI assistants. I think that's that's pretty cool.

• 37:01Speaker 00

  The flip side of RLHF, which I think is so interesting, we've done some initial work that we published on using RLHF to reduce a model's kind of propensity to talk about stuff that it shouldn't like initially. So particularly like, don't return PII and data in cars to model, which is a big enterprise use case. But I think what you talked about also is like, sometimes you want that. Sometimes you need it. We have customers, for example, that are generating, you know, no joke here, they're generating spear phishing emails to test their own spear phishing detection system, right? So there are times that you need to go against what the model has been trained to do. And another interesting piece that came into my mind, you're talking about RLHF. So I think like, the enterprise use cases are a little different than consumer, where in some

• 37:45Speaker 00

  cases, like, you need the ability to turn off some of the guardrails, because you need to create a level of diversity or talk about things that a RLHF model probably just doesn't want to talk about. Big use case for us, I'll give you another example, would be healthcare companies, right, that are looking to create synthetic versions of patient medical records. And a lot of models, when you're trying to augment your examples, will just refuse to do it, because they think you're talking about something that could be potentially harmful, but it's really being used for a good use case. So I think there is definitely a case for times where you want to, you know, take off some of the the guardrails or the set of expectations that organizations might have are just much different than what or teams, developers might have are much different

• 38:26Speaker 00

  than what a consumer might have if they're using chat GPT or something like that. One that I think is so cool, that really, you really start to notice with tabular data, big application, and it's the most basic thing that we would see with synthetic data, but people always start generating that mock data set they've had to generate for a demo, or for their UI or something like that. And inside that data set, you're going to have names, you're going to have addresses and things like that. And genders, things like that, a lot of protected class type stuff. And what you'll notice is the models have a, you know, tendency to return like one type of data. So you've got names that seem very consistent. You have probably represented the training set, you know, demographics that are across the United States or things like that.

• 39:11Speaker 00

  One cool application of RLHF that we've done, that we've been experimenting with is actually training the model to be more diverse in the results that it gives back. So if I ask for a set of demographics for a particular zip code or city or things like that, having the model want to return a more diverse and kind of aligned set of demographics than, than, than I think, than what a model might do off the shelf is pretty powerful. But you want the ability to be able to control that. Sometimes you want real data. Sometimes you want ethically aligned data. Those are both really important. I think the irony of the whole thing is RLHF can be really good for both of them. It's just like what direction, it's a tool, right? And it's kind of what direction you point the, the algorithm and the other loss function it's solving.

• 39:55Nathan Labenz?

  This goes back to my kind of decoupling of the distribution prediction and then the sampling from it. It seems like you could achieve that largely with just temperature. Like if you said, we're going to make our, under, you know, the core model and its, you know, logits or, you know, percentage weights, outputs as true to real world data as we can. Then you could slide your temperature slider from zero and be like, at zero, you get the modal prediction. And at, you know, one or whatever, you get the sort of normal, you know, represented, you know, the real world distribution. And then at two, you get like the minority server represented, you know, on any and all dimensions version. But it sounds like you're approaching that in a different way.

• 40:38Nathan Labenz?

  So is what I'm saying like not workable for some reason? And is there, why is there more complexity?

• 40:44Speaker 00

  Even a temperature distribution, when you're editing it, is going to, when you turn it up relatively high and you start to get into even kind of crazy data for a really imbalanced data set, even then the temperature isn't going to introduce something that is like 1% of data very often, or at least to the level that you're wanting it to. So this is a technique we can use to, to kind of force that to happen.

• 41:06Nathan Labenz?

  This kind of reinforcement learning is really a tool for saying. We really want you to be, you know, to create a more diverse representative data set. This is like the stock photography thing where it's like, we're going to be intentional about this. But to do that, you have to, you know, apply even a more than a standard technique.

• 41:22Speaker 00

  I love their engineering blog at Pinterest and they had a really neat example on how the search results, for example, if you're searching for pictures of wedding rings would bring back a picture of wedding ring with like very diverse skin colors or things like that, which I think is a nice feature to have. Once again, you don't always want that. Often what I found particularly in the machine learning or the kind of the tabular data space is that the classes are incredibly imbalanced. Then give you another example, even for things at scale, right? So we work with a major social media company. They were impacted just like, you know, everybody else by the, the kind of changes to the ad, third-party ad tracking and things like that. So really, you know, they are trying to make the best possible use out of their data that they have.

• 42:04Speaker 00

  And when you look at the ad recommendation problem, it's massively imbalanced, right? For every thousand people that you present an ad to, maybe one or two click, right? Hopefully better than that, but often that's, I think, the kind of case that you're looking for. So you're trying to make the absolute most out of that data, but that data represents point zero one, something, you know, in that range. And I'm making these numbers up, but it's meant to be illustrative like of the data. So you need the ability to tell the model, this is really important. I want to learn from these particular features, this very, you know, kind of imbalanced class, and then generate meaningful new variations to improve detection with this. And so that's where, you know, I'm coming from for like the different very strong techniques outside of like altering temperature where you want very explicit control over the model output

• 42:48Speaker 00

  to make sure it meets your expectation because you've got a task at hand.

• 42:51Nathan Labenz?

  Well, control over models is definitely something we should all be striving for. So in all aspects of AI, you know, definitely in the big picture kind of worries about, you know, are we going to keep this whole AI technology wave under control in any number of ways? I think this your situation here is one of the more compelling things I've ever heard also for, you know, the need for kind of the raw model, you know, that that has the more accurate world model, even if it is, you know, sometimes not so pretty to look at. So I wonder how you, and I'm not like super bent out of shape about these issues, but I guess a lot of times I kind of view the model developer's ability to control things as kind of a canary in the coal mine.

• 43:35Nathan Labenz?

  You know, like if they can't, you know, prevent it from like being offensive today, are they going to prevent it from, you know, following build a bioweapon command, you know, tomorrow is kind of the, you know, the most credibly alarming scenario, I think. But this is definitely, you know, I certainly see your point about like, hey, we want to have all these different dimensions of control. And we've got, you know, even just to build stuff to test our, our detection systems, you know, we got to have data that's going to set off alarms, right? So that is, that is all super interesting. I wonder what role does like, you know, kind of conventional AI ethics have in the company, given all these use cases that you want to enable?

• 44:20Speaker 00

  Have a very optimistic view of AI. And I think a lot of times I view synthetic data as a tool that could be used to improve, like the alignment or the ethics by someone that wanted to do it. I think to your point, it could also be used to do stuff that is, that's harmful, right? So I think that's a real question that we're going to be, you know, kind of wrestling with as a, as a community over the next couple of years. I'm a, you know, a big fan of having the alignment checks and the warning flags everywhere. But to the extent possible, giving people control over, over what the model does. And, you know, generally speaking, curious to hear your, your opinion on this one as well, too. I think.

• 45:00Speaker 00

  I'm a more of a fan of the like the open model where it can be kind of adapted for whatever particular use case that you might have. Versus moving to a more kind of closed space where a very small group of of very powerful companies have control over. What the models can and can't return. So. There is no, you know, perfect answer there, but I would say like, you know, I want to believe that people want to do the right thing. AI is going advancing so incredibly fast that, you know, from what we're seeing, I think that, you know, for example, the White House executive order that came out this week is. It is a sign that people are paying attention to the right things. I'm also, you know, maybe this is a byproduct of being one of the smaller companies out there, you know, we've got about sixty five people at Grotto right now.

• 45:43Speaker 00

  You know, I'm curious to see how regulation will kind of play into this this world. Where, you know, smaller companies that are innovating may not have. Hundred people on a regulatory or compliance team to to help work on this yet. So. Love the direction. I think something really important that we do as we kind of move forward is like thinking about. Enabling competition and things like that and enabling innovation. While protecting. People's privacy while protecting that. The use of AI. Across our our ecosystem.

• 46:13Nathan Labenz?

  Yeah, no simple answers on all that and the, you know, hundred. Plus page. Order. You know, that seems to mostly be ordering another ten thousand plus pages of reports. Is certainly reflective of that. I mean. I also thought it was a pretty good first step and, you know, at the highest level, I've been kind of saying a lot lately that. As someone who does take kind of big picture AI risks pretty seriously. It's hard for me to imagine a much better situation for the kind of overall game board to be in today than the one we actually have. You know, at a minimum, we can say all the people at the big companies that are developing the most powerful systems are like. Pretty serious minded.

• 46:53Nathan Labenz?

  And, you know, the most kind of rogue one is meta and, you know, they're still like, you know. More responsible than, you know, you could easily imagine people being if they were just like. Didn't care or thought the whole thing was totally ridiculous. So. I think that's a, that's all a pretty good start. I like the flop thing pretty well. I mean, I, I'm imagining that you're like, you had said 500 billion tokens, right? As the pre-training base. So I mean, if I, you know, if my intel is correct, GPT-4 is 10 trillion tokens. So kind of 20 times as many tokens, however many more parameters.

• 47:33Nathan Labenz?

  It feels like you probably have three orders of magnitude between kind of what your compute budget was for this. And where the, even just the reporting threshold, you know, would kick in. So it seems like you have plenty of room to run as a small company before, you know, you would hit any, you know, any odorous regulation.

• 47:53Speaker 00

  And that's one of the neat things is, you know, I think that we don't have to, and we're not trying to compete with GPT-4, you know, to create a modular data model at that scale. I think that the promise of, for other people that are building, you know, AI-powered applications right now. The promise of a really lightweight model, really fast model, you know, at Microsoft's textbooks are all you need paper using like a 1 billion parameter model. That, you know, is super fast on difference, super low on training costs, trained on it like a relatively diverse but small set of examples. Like shows the power that you can have of taking a domain-specific data set you have or task and doing something meaningful without having to do something at the GPT-4 scale.

• 48:39Speaker 00

  So personally, like, so, you know, excited about that because I think it's going to enable innovation from the, you know, life sciences companies, from fintech companies, you know, you name it. AI video content creation companies, things like that, that can create small, efficient, fast models that, that do something really cool and really unique that the big models haven't, you know, or can't do at the same, at same level. So personally excited about that. And then the combination of those two, like still leverage that big model where you need it is, you know, like we use it for, for the intent parsing, really understanding what type of query a user wants.

• 49:13Speaker 00

  Then we use our small model for, for speed. I'm excited about that because I think it, it enables people to experiment without needing, as you were saying, like, you know, to train on, on, you know, 10 trillion tokens or something like. Just so big that it, it, it becomes a barrier to entry.

• 49:30Nathan Labenz?

  Yeah, I think if we give it a little time, you know, there are some really positive natural trends. Because and there, you know, there are some ways where kind of everybody's interests can be aligned. I'm, you know, generally speaking, the systems that worry me the most are the super general ones. Things that are designed and, you know, engineered for kind of narrow purpose seem inherently just, you know, a lot easier to keep under control because like, you know, Alpha Fold may be, you know, a world changer, but, you know, up until at least this week, you know, I think it does a couple of things now, but it previously did, you know, kind of one main thing, and that's, you know, predictive protein structure. And you got to fit that into a broader system.

• 50:13Nathan Labenz?

  You know, lots of awesome examples like that, you know, Alpha Go can play Go better than any human, but that's all it does. So I think that's all really good. And, you know, there is kind of a vision for long-term AI safety. That's kind of like the ecology of small models that I think Eric Drexler was, you know, has, has a manuscript on this. He calls comprehensive AI services that is like a good early articulation of it. Kind of pretty prescient, actually, given, you know, it's like five years old, I think, already. And his idea is just that let's have narrow, superhuman AI in everything. And then we don't necessarily need, you know, superhuman, superhuman general AI, which might be hard for us to control.

• 50:55Nathan Labenz?

  So I, you know, but right now we're just like still figuring out all these techniques and how to make things work and like what the curriculum is supposed to be and what the learning dynamics are. And the one thing that it kind of is working, you know, without question. I mean, a lot of things are working without question, but, you know, it's so tempting in the meantime to be like, well, why don't we go, you know, what happens at 10 to the 27, 10 to the 28, 10 to the 29? And there I'm like, uh, yeah, I actually would like to see us be a little bit more cautious before we, you know, just race through like however many more orders of magnitude. Cause I have no idea, you know, really what comes out the other hand of, you know, 10 to the 30 at this point. You know, all bets feel off. Does that feel safe to you? I mean, if, if somebody were to come today and be like, hey, great news, everybody.

• 51:37Nathan Labenz?

  You know, all my H100s just warmed up and we're, we're going 10 to the 30 right now. You know, we'll see in a hundred days with my 50,000 H100 cluster. It should take, you know, 100 or probably a little more than that to go 10 to 30, but you know, whatever.

• 51:53Speaker 00

  I don't think there's any way to stop someone from doing that.

• 51:56Nathan Labenz?

  Well, you are working at like small city size electricity consumption at that point. So I mean, that is the kind of thing that the state can currently intervene. Now, there may be algorithmic breakthroughs in the future that make, you know, that's sort of impossible to, you know, to stop. But.

• 52:10Speaker 00

  Well, getting a little bit meta here, maybe just kind of talking a little bit of theory. It feels like these advancements we made have really been kind of modeling how the human brain works. And that's neural networks, right? At a go and at some point, nature stopped saying we should have a bigger and bigger brain. And started to say, we're going to start like having parts of your brain that are specialized for certain things. So that's where like, I don't think there's anything we can do to stop someone from training, uh, something on every, you know, token that can be found across the entire internet. But I actually think this idea of there is a quantity of smart enough. Where for general reasoning, you've got good stuff there.

• 52:49Speaker 00

  And then it's the like, the, the task specific, the code generation, you know, L L M's or the synthetic data generation L M's or, um, you know, generation, uh, for, um, alpha fold. Like what they're doing like that, like a task specific model that is really great at what it does as a tool that's available to the others. That both I think kind of helps us reason a little bit about more what's going on in a way we couldn't do if the model got bigger. But I also kind of believe is actually probably a smarter and more efficient way to, um, to build out that A G I. So I would, I would see like the, you know, personally to the future as hopefully, you know, cause I, I like the idea of the auditability and the understandability of these small kind of expert models.

• 53:31Speaker 00

  Of, of a world where you've got a lot of mild, uh, models that are trained on small amounts of domain specific, really special data. And then orchestrated by a larger, you know, smart enough L L M without creating the uber intelligence that no one understands how it works. Curious how, how you've, uh, you've thought about this as well.

• 53:50Nathan Labenz?

  Yeah, I think largely similarly with maybe just a little more tinge of fear in my, um, you know, in my affect. But yeah, you know, safety and narrowness, again, I think is super important. It would be.

• 54:05Nathan Labenz?

  I see a case, you know, I guess if I were to, you know, try to summarize the case there, it would be like beyond a certain point, scaling isn't necessarily economical anymore because you're good enough, you know, to do a good job at tasks and, you know, that's kind of what needs to exist. There I do, now I kind of want to revise maybe my earlier statement on like being, you know, pumped about the state of the game board because I do think we look at some of the leading developers and it's like there isn't a, in some of them, you know, one maybe in particular, there is a sort of borderline ideological, you know, sense that like, we're going to keep scaling and we're going to make something like, that's

• 54:48Nathan Labenz?

  like the most powerful thing we can make, you know, and we're going to try to do it safely, but like, we are going to make the most powerful thing we can make, seems to be kind of a prevailing notion and I'm like, that is the part that doesn't seem super wise to me and it does seem like the kind of thing that the state can do at least something to control for a while. So I would, I would again, like to see a little caution there, but I'm, it's funny, we, I just did an episode today with my friend, the CEO of Lindy, and we were kind of running down all the places and ways in which we are both EAC, which are many, you know, and then there's like, just this one little corner of the, you know, of the world where we're like, yeah, maybe, um, let's not rush to 10 to the 30, you know, not knowing what kind of alien pops out the other end of that.

• 55:31Nathan Labenz?

  But I, I think this is, you know, I really appreciate your perspective. It is, it is so interesting. You're just such a, such a different kind of angle on so many of these kind of core fundamentals. I'd love to hear how that plays out for you in terms of your sense of understanding on the part of the language models.

• 55:53Nathan Labenz?

  You know, you've got the stochastic paradigm, obviously, you've got the reasoning engine, characterization. What do you make of that, you know, as somebody who's kind of focused much more on the representativeness side of the challenge?

• 56:08Speaker 00

  Our number one focus as we've built out our service is, and I think it's helped keep us kind of grounded, is helping data scientists, developers with the problems they have with data today. Your data is messy, it has gaps in it. I can't create new additional examples, it's too expensive, or there's no way to go back to it. So we really focused our efforts on, first and foremost, like, helping you build better data. That better data is either more accurate, or is more private than the existing data. That's been the kind of the guiding light, that's what we're really aiming for out of the gate, and learning as we go.

• 56:49Speaker 00

  One of the areas, we're about to release a very early version of our service, and to see and really learn from users for what they're able to share and feedback with us for how they use it, and use that to guide development. So instead of, you know, starting, making a set of assumptions that prove it to be incorrect, I think one of the areas that we've been successful as a startup is like getting code out there really fast, getting examples out there people can iterate with, asking for feedback and iterating on that feedback. So, you know, I'm super curious to see where this use of generative AI for working with, you know, our big focus here is working with tabular or mixed modality, tabular text and time series data goes. We'll use that to drive our own investments on, how much time do we spend working on a better agent?

• 57:34Speaker 00

  For example, so a better agent tooling. If you wanted to create a time series or something like that at the scale of a million rows, right? Like, how do you take in our knowledge around building time series that works, and then combine it with the other technologies? And where we see it being successful, that's where we're going to double down. One of the areas I feel is so kind of neat and de-risking in this space right now is there are so many potential tools you can bring to the problem, whether it's retrieval augmented generations of bringing in example data sets into the LLM memory just to help it, whether it's, you know, kind of the React or the agent approaches for breaking things down in smaller problems, and the LLM training itself. So we've got a bunch of different dials we can use to solve the problem.

• 58:16Speaker 00

  We're hoping to learn from how people use the service and see which areas we really need to double down on. But I'm psyched for that, I think it, in the sense that, you know, I really don't, I have some ideas on where it's going to go, but I don't really know where things are going to go. But this, like... Let me think about tabular data as a resource and how much of the data we work with every day is like organizations is in some sort of tabular format. It's pretty unique space to be in, you know, I'd say most organizations, like eighty, eighty-five percent of data. And some levels in some sort of structured, semi-structured format. So being able to like work with that and leverage it is a kind of a niche like, but really cool space to be, to be playing in right now.

• 58:58Speaker 00

  And, you know, to your point earlier, I'm sure it will be just a matter of time until we've got, you know, competition from the Open AIs of the world, the anthropics of the world and things like that. But right now, you know, we've, we've got a great set of users we've been building on it. And I think this kind of combined approach across advancing L L Ms to the point we need them to do without trying to build the Uber L L M. And then, and then also combining other, you know, cool technologies that are happening in our space to solve a problem. It's working out pretty well.

• 59:27Nathan Labenz?

  Yeah, it kind of strikes me that there's maybe another, you know, Pareto curve between these two modalities, you know. I'm trying, I'm trying to find a synthesis for the stochastic parrot reasoning engine debate. And in your architecture, I'm sort of seeing maybe they can be both, you know. You very much are training the core L L M here that generates the data to be like a stochastic parrot in a sort of, you know, highly principled way. But nevertheless, like you want that randomness, right? That, that's kind of a big part of the value driver. And then you also need this like planner that has to be much more like reliable.

• 1:00:11Nathan Labenz?

  And probably a lot of the models we use today are kind of at some sort of, you know, outer part of the curve on the production possibility frontier. But maybe there's like a bifurcation that happens there too, where like you're really pioneering the sort of high integrity stochastic parrot, you know, side. And then other people are like, we are really pushing, pushing the reasoning side.

• 1:00:32Speaker 00

  That's such an interesting idea and it kind of mimics, we went to a conference, there's a major health organization called H L seven. And they have a fire, it's called FIRE, it's the most popular medical data record format in existence today. And they ran a whole conference on synthetic data. And the feedback we heard at that conference was the exact opposite of probably every customer conversation I've ever had, but it was so interesting. In the synthetic data world for healthcare, there's a few projects. There's an open source project called from MITRE called Cynthia that allows you to generate medical format record data that you can use for testing systems and things like that in the healthcare space. It's been under development for like four or five years, purely statistical based approach.

• 1:01:17Speaker 00

  And what they called out was that for many of the use cases they want, particularly around AI or machine learning, that data from Cynthia is just too clean. I'd never heard in my entire like professional career up until this point, but what they were saying is like, you do want a little bit of that variability. You want that, you know, slight variation that still has the city that gets introduced, but you don't want crazy, right? So it really is about kind of finding that balance, like just enough within the scope that you need. And then also thinking about it at scale, right? You can't evaluate these things one at a time. You need to be able to reason about 50,000 examples you create for an LM training set or a million examples you create to boost a ad recommendation data set, something like that.

• 1:02:01Speaker 00

  So you really have to think about it at scale. And that's just been starting with tabular data where it's so easy to look at it and say, this is right or this is wrong. I think is, is maybe had us thinking about this as a company a little bit earlier than the rest of the industry that is now like, wow, we can generate really amazing text or images for a, like can we use the train and machine learning pipeline? But how do we know that all thousand images that I created are kind of meet my expectations? So I personally really like this idea of letting an LM be an LM, like let a machine learning neural network generate whatever it wants to. But then examine the outputs at each step and build some, you know, some controls that if it goes too far off the reservation, turn the temperature up too high, something doesn't make sense anymore with real world data, you can detect it and kind of filter it out.

• 1:02:46Nathan Labenz?

  That kind of recalls how in a lot of like image tasks, there's kind of training on systematic corruptions of. The image as well, like you want to make your stuff, you know, robust, you add a little noise here and you kind of, you know, distort this way and, you know, change aspect ratio. And if it can work across all those different things, then, you know, you're going to be much better off in a real world situation. And there's kind of a similar problem, I'm sure, for a lot of like, medical things where, you know, stuff is whatever, anything from illegible to incomplete to, you know, contradictory to, I just saw a funny story about like, a person who had the same name and birthday, you know, in the same hospital as another person and like, spent their whole life, you know, trying to be, you know, disambiguated

• 1:03:30Nathan Labenz?

  and like struggles. So yeah, I mean, it's just so many kind of crazy things out there. I know we don't have too much time left and I've really enjoyed kind of the digressions in this conversation. I did want to ask a little bit more about how you go about kind of training for privacy protection specifically. We've talked a lot about how you train for, you know, representation, you know, and super representation. But I understand there's like probably a whole different technique for just making sure that like, you don't spit at somebody's real email address or whatever. That is interesting. You know, when I find, one of the experiment I ran, maybe can help me understand this a little bit better. This has been a topic of discussion lately too, is I ran an experiment on OpenAI fine tuning with a bunch of my writing, you know, and kind of my resume, my data, whatever.

• 1:04:15Nathan Labenz?

  And I was like, you, and I need to do it again with 3.5. This is a little while ago. But it was the OO2 generation of fine tuning, which they never launched publicly, but I had like an opportunity to test a version of it. And anyway, it kind of like, sort of moved in my direction. It did not know who I was though. It was, you know, I was like, trying to turn up the epochs a bunch, you know, and it's still like, never learned that it was supposed to answer as like, I am Nathan Levens. It, you know, it was like Nathan something else sometimes and, you know, it was kind of vaguely similar to me, but definitely not like memorizing those facts. So I'm very confused about kind of memorization in general. Jeremy Howard recently had a thing where it's like they, LLMs can memorize from one example. It definitely hasn't been my experience. So maybe for, you

• 1:04:58Nathan Labenz?

  know, kind of background, like what do you observe about this sort of LLM memorization?

• 1:05:05Nathan Labenz?

  You know, in my case, I was trying to get it to do it. You're trying to prevent it. But what is happening there? And then, you know, what's the technique that you're using to really make sure that it's not happening for your product?

• 1:05:15Speaker 00

  Where we started was training language models from random weights on a data set from a customer, right? In that sense, the model learns from the data that it sees, and it has a very high propensity to memorize and replay secrets in data. There was a great paper that came out, and this is like towards the beginning of our company in 2020. It came out from Cal Berkeley called The Secret Share Paper. Don Song's team and several others kind of working on it. And what they were highlighting was that, you know, given training a language model on a data, like how quickly it starts to memorize, even rare occurrences in the data and the chance it'll play it back. When you're training, it's an interesting example you gave where you're training, you know, GPT 3.5,

• 1:05:58Speaker 00

  fine tuning it on an example, because I haven't seen kind of written up exactly how their fine tuning works. If it's actually, you know, updating all middle weights or if it's using a path to base approach or something like that, just adapting the model on top of it. But it gets a little harder to detect when you have this massive pre-trained corpus and you're making very small changes to only a percentage of the model weights, for example, across the entire model. But it still happens. One of the really areas that we see, you know, customers doing a lot is fine tuning a model and then just running a series of tests. Like we call them canaries, but essentially trying to get a model to auto complete a credit card number or things like that. Here's what I've seen work, starting with the removal of PII or personal data. It's the first thing you can use an LM to do it.

• 1:06:43Speaker 00

  You can use NER to do it, whatever you want to. The first step is really kind of removing the data you never want to have show up inside of your model. The second risk, and this is the really the risk that trips trips people up, particularly like when you're working with, you know, patient medical data or things like that, is that some combination of attributes, really easy to imagine in a tabular work, you know, example, right? You might get rid of a name, but you have a height and you've got a zip code and you've got some sort of disease or something like that. And just that combination of attributes can very quickly become identifying. So none of them identify by themselves, put a few attributes together and you have a real problem from privacy perspective. Same thing with text, you know, the types of styles that people have for writing and things like that,

• 1:07:26Speaker 00

  as well as the, you know, data that you're training it on, that combination of attributes can become identifying. I suspect that the OpenAI approach, as you trained on more and more data, would become more likely to start to have things like that, where the combination of attributes, writing styles, anything like that can become identifying. The answer to that across both tabular and text is actually the same type of approach works. There is an approach called differential privacy. Everyone's heard of it. No one really seems to really know how it works and always try to find a simpler and simpler way to describe it. What differential privacy does is it inserts a quantitative like level of noise into your data. And so, you know, when you're training a machine learning model, if you're training

• 1:08:10Speaker 00

  an LLM in this case, you're training with differentially private fine tuning, for example, it's inserting noise into the optimizer and clipping gradients on the way out. And what that's making sure is that like some rare combination of, you know, words inside of your data like, hey, my name's Alex, I'm six feet, I live in, you know, whatever, Southern California, something like that, doesn't become memorized and replayed by the model. So essentially, you can guarantee either a training set example, right? So per example or per entity inside of the data, it could be a set of examples about an individual user. You can guarantee that none of the tokens inside of that data set will be replayed directly by the model.

• 1:08:53Speaker 00

  And that's so important when you are training on compliance control data. So we've got how many different customers we have going on with different like healthcare organizations that are trying to train on doctors' notes or, you know, customer support records or things like that, where you need to make sure the model does not memorize a customer name or replay it, or a combination of attributes there. So things like differential privacy give you a tool that you can say, it's no longer that I think the model didn't memorize it and I haven't been able to extract it. You can actually say like with a level of confidence that, given the way we look at an individual training example or record, I can guarantee that the model will not be able to replay that, or will not have memorized to be able to replay that example.

• 1:09:36Speaker 00

  So in the tabular world, this has really opened the doors up for us. We've got a couple of national level healthcare organizations that have been able to get approvals to share data between hospitals by training, not just on de-identify data, so not just removing names from patient medical records, but in this case, creating a synthetic version of those patient medical records where you know that the model did not memorize my combination of zip code and height and gender or things like that that could become identifying. With those kind of like actual mathematical guarantees, it becomes possible. So, super excited about differentially private fine tuning, particularly in the LLM space. When you look at small companies

• 1:10:18Speaker 00

  that are trying to train those models on their demand-specific data, but they hit compliance or privacy issues, it gives you a tool that it's not just a best guess, or we think that it's going to be fine. You can actually convince yourself that the model's not going to return something that it shouldn't.

• 1:10:32Nathan Labenz?

  As you do the training, you've taken the gradients and you're working your way through backpropagation. You are literally adding a noise factor to the updates to the weights.

• 1:10:47Speaker 00

  For each subsequent token generation.

• 1:10:51Nathan Labenz?

  And that basically allows you to say, we've kind of essentially blurred the picture. In aggregate, there's probably, I guess, like a trade-off there where you probably can, the model converges more slowly, I guess, by, almost by definition, but without learning this stuff.

• 1:11:07Speaker 00

  Yeah, that's such an interesting thing because research has been coming out recently. We had a conference we ran on synthetic data. We had some folks from Google come in and talk about some of the research they're doing. And that's exactly right. When you're introducing a little level of noise into the data, it requires more training time to get down to the same level of accuracy. One of the things that's really interesting with this approach is that, and this increases compute requirements, but there's a theory that by really increasing the batch size, bigger and bigger and bigger, that you're sending into the model at any given point. So, it's going to increase your computational complexity here. But you can use differentially private techniques with increasing larger batch sizes and approach

• 1:11:48Speaker 00

  the same level of accuracy as real-world data. So, in this sense, you're kind of getting privacy without real, a real hit on, on, on utility of the data. But essentially, by more compute budget and more data, it becomes possible to reach the same level of accuracy that you would if you just trained on, on the data itself, which is pretty exciting. So that's pretty new. From what we've seen using Gretel today, like you're going to have a utility hit using differential privacy, especially on small data sets. It makes a lot more sense when you get into a data set where you've got, you know, 100,000 or more examples. Essentially, that level of noise it has to introduce to blur out, you know, somebody of a particular zip code is a

• 1:12:31Speaker 00

  lot lower. That's why you've seen differential privacy. For example, the US Census Bureau uses it. Google and Apple use it on the next keyword prediction or emoji prediction when you're typing a text. At that scale, then differential private really starts to work. But I am personally really excited about this, like, public, you know, LM's trained on public data, you're fine tuning it on a private data set, and you're introducing differential privacy as you do that. Large batch sizes, plus like being able to interleave public examples will help a model converge really quickly. And I think in a lot of cases, we got into the weeds there a little bit, but it is like the key to unlocking, you know, AI for regulated industries that are going to have to convince a

• 1:13:14Speaker 00

  regulator that there's no way that this patient who is part of this data set, their identities can be compromised, right? Like, I always love this term, like, you want the model to learn about a disease, but not about a patient. This is a really great technique to make sure that you have that separation.

• 1:13:30Nathan Labenz?

  That's cool. It's really, I've learned a lot by going down this rabbit hole. So always excited for a journey into the weeds. One last thing I wanted to just get your take on a little bit is there's obviously a ton of activity going on in synthetic data, right? And I would kind of flag Anthropic's constitutional AI is kind of interesting version of this, where they're constantly, you know, iterating on this HHH basis to make things more helpful and honest and harmless. And then, you know, and so that seems to work, like, you know, the clause is really good. So that's great. And then you see the, I think you even mentioned earlier, the, you know,

• 1:14:10Nathan Labenz?

  synthetic textbooks project at Microsoft, which also seems to be like, a great proof point for the value of synthetic data. Then you see these kind of weird stories, like self-consuming generative models go mad, which I think most people, you know, if they listen to this show, they probably at least saw that, you know, blurb whenever it came out of, not too long ago. And there, they sort of say, you know, if you do this like generation after generation, things get weird. Do you think there's anything inherently about synthetic data that is kind of a long-term problem? Or do you think that this is all just, these kind of weirdnesses are just reflections of, you know, not having figured out some of the details yet?

• 1:14:53Speaker 00

  I'm pretty strongly in the not in the details category.

• 1:14:57Nathan Labenz?

  I could have guessed that.

• 1:14:58Speaker 00

  I've also heard another like, kind of a story that I've heard as well as like, well, if, you know, GPT-4 and Anthropic and other LLMs are like creating so much content on the internet, is the next cycle of LLMs, you know, gonna have a regression because it's just operating on, you know, data that's already created from previous generation LLMs? I think that's an interesting question. We're gonna see how that kind of plays out over time, but I would newly posit that in a lot of cases, LLMs for where we are today can generate and often do generate, which is why we do it, like a higher quality version of the data that was fed than it originally started with. I think so many people use this today.

• 1:15:43Speaker 00

  We use Grammarly to improve our text. Sometimes we run an email through an LLM and kind of ask it to help us make some improvements or things like that. So I think the signal inside of there, and that's kind of what came out in that textbooks are all you need paper, is a very promising thing. I don't think this is fully understood yet, but the idea that synthetic data can be kind of a cleaner, more diverse version of, you know, the limited data you might be starting with is a really powerful idea that I think we're gonna see from. So I'm optimistic that these models, and I would say that maybe the math example is just an example of an opportunity to configure things or kind of work with them better, that we aren't moving towards some sort of mode collapse or anything like that with

• 1:16:28Speaker 00

  synthetic data, feeding synthetic data. As long as the data that we're generating is high quality and ideally improving on the data that you have, then I think we'll be in a good spot. That's gonna be playing out, so I'm really curious to see how that works out.

• 1:16:40Nathan Labenz?

  Yeah, the dynamics certainly of the future of the internet and, you know, a changing mix of content being published there is definitely gonna be another just fascinating society-scale story.

• 1:16:53Nathan Labenz?

  Anything else that you wanted to touch on that we didn't get to?

• 1:16:56Speaker 00

  No, I think it's been an awesome conversation. I was just kind of laughing about the last topic. And, you know, as long as like every LM generation doesn't start with, you know, I'm a helpful AI assistant, like, how can I help you? Or let me explain this for you. The things that we see coming out of LMs all the time, I think that we'll be moving in the right direction. So definitely enjoyed the conversation today. And thanks for inviting me on.

• 1:17:17Nathan Labenz?

  Alex Watson, founder and chief product officer at Gritle AI. Thank you for being part of the cognitive revolution. It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please, don't hesitate to reach out via email at [email protected], or you can DM me on the social media platform of your choice.