Cooking up synthetic data with Gretel

John Myers of Gretel https://gretel.ai puts on his apron and rolls up his sleeves to show Dan and Chris how to cook up some synthetic data for automated data labeling, differential privacy, and other purposes. His military and intelligence community background give him an interesting perspective that piqued the interest of our intrepid hosts. Sponsors: Code-ish by Heroku https://www.heroku.com/podcasts/codeish – A podcast from the team at Heroku, exploring code, technology, tools, tips, and the life of the developer. Check out episode 101 https://www.heroku.com/podcasts/codeish/101-cloud-native-applications for a deep dive with Cornelia Davis (CTO of Weaveworks) on cloud native, cloud native patterns, and what is really means to be a cloud native application. Subscribe on Apple Podcasts https://itunes.apple.com/us/podcast/code-ish/id1457879893 and Spotify https://open.spotify.com/show/5VvkHRhd9E9SQf7i9HsPMd. Knowable https://www.knowable.fyi – Learn from the world’s best minds, anytime, anywhere, and at your own pace through audio. Get unlimited access to every Knowable audio course right now. Click here https://www.knowable.fyi to check it out and use code CHANGELOG for 20% off! The Brave Browser https://brave.com/cha913 – Browse the web up to 8x faster than Chrome and Safari, block ads and trackers by default, and reward your favorite creators with the built-in Basic Attention Token https://basicattentiontoken.org. Download Brave https://brave.com/cha913 for free and give tipping a try right here on changelog.com https://changelog.com. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: John Myers – LinkedIn https://www.linkedin.com/in/john-myers-98b07b7Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Gretel | Website https://gretel.aiGretel | LinkedIn https://www.linkedin.com/company/gretelaiGretel | Twitter https://twitter.com/gretel_aiGretel | Slack https://gretel.ai/slackinviteGretel Synthetics | GitHub https://github.com/gretelai/gretel-syntheticsGretel Blueprints | GitHub https://github.com/gretelai/gretel-blueprintsGretel | Improving massively imbalanced datasets in machine learning with synthetic data https://gretel.ai/blog/improving-massively-imbalanced-datasets-in-machine-learning-with-synthetic-dataGretel | Deep dive on generating synthetic data for Healthcare https://gretel.ai/blog/deep-dive-on-generating-synthetic-data-for-healthcareUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Synthetic data is the ability to generate data that is almost indistinguishably recognizable from some type of source data set. And it has all the granular elements of the original data set that you would want. However, if you combine those granular elements, you don't have a one for one matching to a record in the source data set. And, you know, it heavily relies on machine learning and artificial intelligence to learn the semantics of the source data set. And at that point, once you learn those semantics and that model is built, you could just continue to generate records that in aggregate tell the same story as the source data, which is kind of like one of the key elements that we always like to talk about is you could still run the same types of aggregate queries and get the same story.

• 0:44Speaker 00

  It's not about just being able to use the individual records that you synthesize.

• 0:50Speaker 01

  Being with for change log is provided by Fastly. Learn more at fastly.com. Our feature flags are powered by Launch Darkly. Check them out at launchdarkly.com and we're hosted on Leno cloud servers. Get a hundred dollars in hosting credit at leno.com slash change log. Hey friends, this episode of Practical AI is brought to you by Codish, a podcast from the team at Heroku that explores code, technology, tools, tips, and developer life. There's tons of great conversations on the Codish podcast, so I would encourage you to check it out and subscribe. But in particular, I wanted to bring to your attention two episodes, episode 98 and 99, where Hulian Duque explores the ethical and technical sides of deep fakes.

• 1:31Speaker 01

  The rise of manipulated pictures and videos and other forms of computer generated media are able to cause uncertainty and doubt in what we see in here online. So how are we able to use these tools for good, if at all? Here's a sneak peek.

• 1:45Speaker 02

  Let's say we want to do a deep fake of my voice and we train the model and we have enough data and everything. This will be also able to imitate my accent, for example, like how I pronounce English and the strong pieces of my accent, or is not there yet?

• 2:07Speaker 03

  It really depends. If there would be a person with similar accent on the input, then it would be fine, but it's kind of cheating. You can think it's cheating because we're reusing accent of a different person that's similar to your accent. But if it would be like an American native speaker or a British, a person with a British accent.

• 2:28Speaker 03

  Or like whatever other accent, then it will kind of be a mixture on the output. So we're not there yet in terms of converting accents. It's a little bit more difficult than we initially anticipated because like when we started the company, we thought it would be, you know, we'll kind of solve it in a year or something. But then it turned out that, whoa, no, we're here for, we're here for much longer.

• 2:54Speaker 01

  Check these episodes out. Links are in the show notes to both episodes, or head to heroku.com/podcasts to listen and subscribe. Again, check the show notes for links or go to heroku.com/podcasts.

• 3:20Speaker 04

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.

• 3:46Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International. And I'm joined as always by my co-host, Chris Benson, who is a principal emerging technology strategist at Lockheed Martin. How are you doing, Chris?

• 4:03Chris Benson?

  I am doing very well. How's it going today, Daniel?

• 4:06Daniel Whitenack?

  It's going good. It's a nice cold day here in Indiana. We'll have a few more of those before this is all done.

• 4:12Chris Benson?

  Probably so. It's not quite as cold down here in the sunny south.

• 4:16Daniel Whitenack?

  Yeah.

• 4:17Chris Benson?

  Although it's not sunny today, actually, so.

• 4:20Daniel Whitenack?

  Yeah, and we had a new project funded this year related to some AI work for local languages and trying to get all of that spun up and infrastructure in place and community website up for some shared AI tasks and that sort of thing. So it's a lot of structuring and setup right now for me, I would say. I don't know what big things you've got going, but yeah, it seems like that time of year for me.

• 4:46Chris Benson?

  Absolutely. You know, from my standpoint, I've just been enjoying, you know, just getting out there this weekend, flew around a little bit. I had my first night flight.

• 4:55Daniel Whitenack?

  Yeah.

• 4:56Chris Benson?

  Pleasant to get back to work here this week and it's good.

• 4:58Daniel Whitenack?

  Yeah, Chris is getting his pilot's license for any of those out there that are wondering what he's talking about. But yeah, that'll be exciting.

• 5:05Chris Benson?

  I didn't run into anything last night. It was a good story.

• 5:09Daniel Whitenack?

  Yeah, I'm close by Purdue University and we have an airport at the university. So I told Chris, as soon as he gets his license, he can fly up here and then we can do our recordings in person.

• 5:20Chris Benson?

  There you go.

• 5:22Daniel Whitenack?

  Today, I'm really excited by the topic that we have going. Chris, a lot of times in our past conversations, we've made reference to synthetic data or augmented data or data augmentation methods. We've also talked in various forms about privacy, but I don't think we've really had a, an episode that has combined those in the great way that we're about to. So today we have with us John Myers, who is CTO and co-founder at Gretel. Welcome, John.

• 5:54Speaker 00

  Hey, good morning. Happy to be here.

• 5:56Daniel Whitenack?

  Yeah. First, before we dive into all of that good data related stuff and practical goodness, could you just give us a little bit of an idea about your background and how you ended up working with Gretel?

• 6:09Speaker 00

  Yeah, absolutely. So I think the way I kind of ended up at Gretel is somewhat accidentally, which I think a lot of folks who ended up in this field ended up that way. So my background is computer science by education. And then I did about 14 years in the Air Force. When I joined the Air Force, I came in as a communications and informations officer, which is kind of a fancy word for kind of like network IT leader. And I did a couple of years working in space launch communications out in California at Vandenberg Air Force Base. And then I got a interesting application to go to the National Security

• 6:51Speaker 00

  Agency and do some really cool, hands-on engineering and development work. Sounded really awesome. At that point, you know, my knowledge of the NSA was basically having seen Enemy of the State with Will Smith. And I was like, that sounds.

• 7:06Daniel Whitenack?

  Wait, you mean it's not like that?

• 7:08Speaker 00

  I wouldn't say it is a documentary of sorts, but I was like, well, it's, it sounds really cool.

• 7:14Daniel Whitenack?

  Loosely based on reality.

• 7:16Speaker 00

  Loosely based on reality. I didn't meet Will Smith or anything, so that was a bummer, but it was over in Maryland. I'm from Philly originally. I was like, it's close to home. I can get out there and like, kind of just experience something new. When I got there, I got immersed into the intelligence community and I'm working at NSA and I still wear the Air Force uniform, but I was kind of in an offshoot program there and I got to work on really cool stuff. Working on like low level operating system engineering, you know, building exploits, stuff like that. And then I kind of pivoted into doing the big data analysis, which was kind of up and coming field at that time. And then I left there, did one more stint with the Air Force, doing a different set of things out of Las Vegas.

• 7:59Speaker 00

  And at that point, I was kind of at a critical point if I wanted to stay in full time or do something else. And at that point I was so hooked on building, I just wanted to build and engineer and building engineering teams. That I decided to leave active duty. I joined the reserves and then when I got out, I did the complete opposite thing you can do when you're part of a 300,000 person organization. And I launched a startup in cybersecurity with like three other people. And then we did an enterprise security startup and we did that for three years. It was called Flux Systems. And then we were acquired by a company called NetScout that kind of is, you know, one of the leaders in network performance monitoring and wanted to utilize some of what we had in cybersecurity for some of their upcoming products.

• 8:43Speaker 00

  When I got there, I was a principal architect and among a lot of projects I worked on, I, I worked a lot on a lot of their cloud infrastructure and they build capabilities that also help enterprises and service providers detect and stop DDoS attacks. And a lot of those devices collect a ton of telemetry from the customer environment and to send it up to a cloud repository where it's securely kept. And we wanted to look at how we can do analysis of that data and. Start pouring through that data and you start to realize there's a lot of sensitive information in this data and we should probably pre-process it so we can kind of work on it safely. And I think I spent a lot of time doing that, probably more time than I wanted.

• 9:27Speaker 00

  And that was kind of like this pivotal transition point where I got into doing kind of engineering to enable me to do engineering. So at the same time, you know, back in my head, it's kind of like one of those like Shark Tank pitches. I'm like, surely I'm not the only person with this problem. Like, what could we do about it? With some other close colleagues of mine, as we started talking about these stories and we all kind of shared very similar but different pain points, we kind of orbited around this idea of like, what if we could just make data anonymization and being able to make data safe to use, kind of just like a general purpose thing that like engineers everywhere can use and yada, yada, yada.

• 10:05Speaker 00

  We launched Gretel. So we launched Gretel in the fall of 2019 with myself and our CEO and my other co-founder Alex Watson, who has, you know, a very heavy machine learning and data science background and was previously the jungle manager at AWS Masey, which is another product that is a very, very successful AWS for detecting sensitive information in S3 buckets. So he has a whole different slew of stories. Some call them nightmares about data anonymization. And yeah, we've been doing it ever, ever since. And, you know, really, our mission is to make data anonymization and creating safe data generally available to engineers everywhere, not just the resourced organizations like the Facebooks

• 10:47Speaker 00

  and the Googles who have massive resources to, to kind of experiment with all the techniques to do it.

• 10:53Daniel Whitenack?

  I am kind of struck, I was thinking while you were talking, especially with your background in the Air Force and at the National Security Agency and other places, I was sort of remembering back to our conversations with the founders of Amuta, which is another company that does sort of, I guess they're more focused on sort of the combination of law and data and governance and all of that sort of thing. But it seems like there's this really strong Whatever it is about, you know, people coming from that sort of background, they had a sort of similar background, it sounded like, you know, really creates some deep thinking around these problems of data anonymization, privacy, governance. I don't know, John, what's your perspective from that side?

• 11:36Daniel Whitenack?

  How do you think like your background with these sorts of agencies or like the military has sort of shaped how you think about data maybe differently than maybe someone like me who's just always sort of started in startups and just sort of got what data I could and have used it and yeah.

• 11:55Chris Benson?

  And Daniel, that was exactly what I was going to ask next too, just so that you know.

• 11:58Daniel Whitenack?

  Yeah. Well, also, you know, Chris has some experience in that world as well.

• 12:04Speaker 00

  Gotcha. Yeah. So I think one of the things that I learned a lot about doing intelligence work in the military and working with data is that I learned a lot about the chain of custody of data. And a lot of times when I meet folks that are like yourself, like a data scientist or they're in the analytics space, a lot of times they are kind of just given data and given some task and say like, here, go make magic happen. And I don't know how often the kind of like the chain of custody or like how that data was actually generated is thought about. But for me, I always think about like, where was that data? Like, where was it? Where was it born? Like at the moment that data was collected, something happened and it was written into a database.

• 12:46Speaker 00

  And I think that way a lot. And so my other co-founders also have a background in intelligence community. And so it was something we're all aware of. And so when we started talking about Gretel, really, we wanted to make consumers safer, right? So to make their personal data not used as, as the way it is today. Because often when you think about big companies like Google, Facebook, like they build products, but they also look at their users as a product. So we kind of backed into saying like, what if we could enable engineers to make the data safe at the moment that it's created. So right at that inception of the data, and that's something that we were just really aware of, of where we came from, because that chain of custody of the data is so important. And it's not as much as a governance thing versus more of an engineering problem, because

• 13:30Speaker 00

  as, as an engineer, a data engineer, when I'm writing my data into my production database, can I at the same time create a safe version of that data and write into a staging database that anyone can access with privacy guarantees? So I don't have to go through this whole like repetitive process of snapshotting my production database, combing over it, writing some, you know, bespoke script to sanitize it. Can we just make it part of the entire pipeline at the point where the data is created?

• 13:54Chris Benson?

  I would like to go back for one second. When you kind of got to that moment where you realized, am I the only one that's dealing with this issue? And you kind of had maybe an aha moment or something there where it was kind of, you, you kind of realized that. What got you to that? I'm kind of curious about that moment of recognition, because I think other engineers and other data scientists wonder, you know, are they going to have something similar of, you know, as they're out there creating, what was it that made you suddenly realize this is something that I'm recognizing not only impacts me, but probably impacts the broader community and I had, as well as I have something to contribute toward that solution? And as part of that, was any of the background, you know, we talked about your, your intelligence background there, did any of that contribute to that moment and that recognition?

• 14:37Chris Benson?

  If you hadn't had any of those experiences, might you have missed that altogether?

• 14:41Speaker 00

  I think there's like a two big things to answer that, and I'll start with the, the former. So like when I hit the aha moment, like I had a small team at my previous company and we were kind of analyzing the data and, you know, we realized that we needed right ways to kind of detect the sensitive information that's in it. And the sensitive information is an information that like is the fact of the sensitive information. It's like, it's like names, it's company names, it was email addresses, it was IP addresses, it was things that were identified, can identify our customers.

• 15:07Chris Benson?

  PPI.

• 15:08Speaker 00

  PPI. And so we were really, okay, well, let's just write some detectors for it. We can use a lot of reg exes, we could write a custom rules. And then we're like, okay, now we need a way to write a rule really quickly. Okay, now we need a framework to put the rule into. And I was like, I can't be the only person who's trying to figure out if an email address slipped into a data stream. And, um, you know, it's not like some communities have like really specific data structures that are really specific to them, like in healthcare and stuff. Like this was just things that are PPI and to a degree PII that identify organizations and people. And there are generic ways to do that where like, can you just like bring a regular expression to the table and like, just like some framework kicks in and can scale for you? That was one of the kind of things that kind of turned that was like, what we were doing was fairly a repeatable process.

• 15:53Speaker 00

  And we assumed it was a repeatable process in many industries. And the second part of that was where to apply that detection and where to apply whatever type of transformation or synthesis we want to do. It was kind of a no brainer that you want to do it as close to the source where the source of the data is itself. You know, you were talking about systems that can collect private information, like, can you do it on the system before you even think about transmitting it to the cloud? So there's no risk there. Can you do it or as the on the edge as you know, people would say these days? That's not even a question for me based off of kind of our backgrounds and being like so kind of in tune with data custody.

• 16:29Daniel Whitenack?

  So I'm curious, as you've built out the set of products, which we'll, we'll definitely get into the details of those in a bit and talk a lot more about the practicalities of synthetic data and all that. But you kind of mentioned that this was like doing engineering so that you could do engineering.

• 16:46Speaker 00

  Yeah.

• 16:47Daniel Whitenack?

  As you've like engaged with various companies that are using your product, has that story been getting sort of like they sort of immediately understand what you're after? Because I remember like, you know, when I was first getting into data science, there wasn't a lot of talk about this sort of, you know, rigor and, and the way we were treating data. And probably, you know, people might've seen something like this as maybe a little bit burdensome, like something they have to do before they actually get into the work that they really want to do. But how have people been like feeling that need in the industry and been, you know, accepting this sort of solution from your perspective?

• 17:33Speaker 00

  Yeah, I think it's been received really well and it's kind of a classic build versus buy problem. And a lot of folks are just willing to buy. But what they don't want to buy is some type of really difficult to install appliance or virtual appliance that kind of breaks their workflow. And so the way that we're targeting this is making it so it is. Our end user, our developers that can easily integrate it into what they're doing already. So making just another API call in their, in their stack of what they're executing on versus saying like, yeah, sure, we can do this, but we have to kind of come and install a virtual appliance and you have to reroute your entire data pipeline through it. And so as soon as we kind of explain that, it works right into their existing infrastructure. And, you know, we take care of kind of the scale for them where they could just kind

• 18:15Speaker 00

  of bring, you know, their predictors or bring what they want to actually detect on to the, to the table. They'd much rather just kind of buy versus, versus build it because it eats up a ton of cycles for them to kind of build this thing. And once, it's not a build once and deploy type of thing either. It's not like they're building a framework that they can deploy once. It requires care and feeding because you're constantly adjusting what type of information you're processing and what types of things you want to anonymize on. And so we can kind of go on that journey with you and enable you to kind of care and feed a lot faster.

• 18:50Speaker 04

  Have you heard about Knowable? It is an awesome new platform for learning from the world's best minds anytime, anywhere, at your own pace through audio. Learn about the performance benefits of a plant-based lifestyle from NBA All-Star Chris Paul, or how to launch a startup from Reddit co-founder Alexis Ohanian. There's even a 10 lesson course from astronaut Scott Kelly. Here's a sneak peek.

• 19:16Speaker 07

  We learned a lot up there, but what can you learn from a life in space? The answers might surprise you. In this Knowable course, I want to share some of the things I've learned that you might not expect. Lessons about leadership on a dark night on an aircraft carrier in the middle of a churning sea. Lessons about the fear you feel with seven million pounds of thrust exploding underneath you. And most of all, there's an idea out there that astronauts are always perfect. Failure is not an option, right? That's why I want to take you through some of my life experiences to show you how that's just not true. I believe everyday, regular, human failure, if we handle it right,

• 19:58Speaker 07

  can be one of our greatest opportunities to learn, grow, and succeed.

• 20:03Speaker 04

  Knowable is accessible on your phone and on the web, and each audio course is broken out into individual lessons, usually around 15 minutes long. As a Change Log listener, you can get an annual membership to Knowable for 20% off. Get unlimited access to every Knowable audio course right now. Just download the Knowable app or visit knowable.fyi and use code CHANGELOG for that 20% discount. We put a link in your show notes for easy click-ins. Check out Knowable today and start learning from hundreds of top experts from around the world. Once again, that's knowable.fyi, code CHANGELOG.

• 20:58Chris Benson?

  So I'm kind of curious, I know in the beginning of the conversation, when Daniel was introducing you, John, he talked about synthetic data. Could you start off by kind of telling us what is synthetic data? And kind of give us a little bit of a background before we dive into the specifics of what Gretel does? And how it gets there, but kind of give us the terms that we need to know to be able to follow?

• 21:20Speaker 00

  Sure. So we were at a happy hour or something. I'll kind of give you that level of definition.

• 21:25Chris Benson?

  Perfect.

• 21:27Speaker 00

  So.

• 21:28Daniel Whitenack?

  Our podcast is always the happiest of hours. Our listeners week, I'm sure.

• 21:35Speaker 04

  That was funny.

• 21:37Speaker 00

  So, you know, I would say synthetic data is the ability to generate data that is.

• 21:44Speaker 00

  Almost indistinguishably recognizable from some type of source data set. And it has all the granular elements of the original data set that you would want. However, if you combine those granular elements, you don't have a one for one matching to a record in the source data set. And it is a, you know, it heavily relies on machine learning and artificial intelligence to learn the semantics of the source data set. And at that point, once you learn those semantics and that model is built, you could just continue to generate records that in aggregate tell the same story as the source data, which is kind of like one of the key elements that we always like to talk about is you could still run the same types of aggregate queries and get the same story.

• 22:30Speaker 00

  It's not about just being able to use the individual records that you synthesize. Now, I will say there are use cases for using those individual records, like if you have like a development environment and you're, you're kind of building, building a system and you want to look and see how the records fit into like your layouts and stuff, but for the most part, the idea is to be able to use those records in some type of, set of aggregate feature.

• 22:50Daniel Whitenack?

  There's a whole lot of jargon, of course, in, in our industry and you've already mentioned as well, like anonymizing data. How does like synthetic data complement anonymization techniques, or maybe like it's an alternative to it, or how do those two things fit together in terms of anonymization and synthetic data?

• 23:10Speaker 00

  Yeah, I would say it all starts with the core use case, but it could either be a complement, it could be totally separate, or it could support, and they could support each other. So we kind of have two large buckets that we kind of focus on at Gretel. And one of them is being able to kind of detect PII, detect PPI, and then apply different transformation techniques to the data in place so that like your data is essentially the same, but this, there's like typical redactions or like character replacements or whatever. And that kind of falls in line with a lot of the existing solutions that are out there that fall under kind of like a data loss prevention capability. And you'll see a lot of the cloud providers like Azure, AWS, Google, they all kind of have a DLP set of APIs you can apply, except that usually requires like to be bought into their ecosystem and already have your data sitting there.

• 23:55Speaker 00

  In our mind, that's table stakes, just to even have a conversation about privacy. And we offer a set of APIs that allow you to kind of detect and do those typical transforms. And synthetic data for us is a way to kind of take the data set, build a model, and kind of let the model generate new records that that you could just accumulate and use. However, and it doesn't necessarily require you to funnel each record through a certain type of detector and look for PII, because we're just going to learn the semantics of the entire data set and generate new records. But those records should not be the original records that you had. And they play hand in hand. It's just like, for one example, let's say you have really sensitive PII, let's say, social security numbers in the source data

• 24:38Speaker 00

  set. If you could detect that a certain column or social security numbers, we might go ahead and recommend that you generate new randomized social security numbers, which is very deterministic. And then you can have that new column in that data set, then send it into our synthetic capability. And that'll just help guarantee that we don't memorize any of the tokens or replay any of those social security numbers, because that is always a risk with synthetic data is that you might memorize and replay some secrets. And that's kind of where like, that whole field of differential privacy is coming in to address that situation as well.

• 25:11Chris Benson?

  So does the synthetic data that's being generated, does it always kind of start with being a replacement for, you know, the actual PII that you're contending with at the time? Is it always kind of starting as a replacement factor? Or is there ever a use case where you're generating it? Maybe like, what if you're starting with no data and you wanted to generate it entirely synthetic? And just because you don't have something to start with, is that within that? Or is that something? Would that be a separate type of use case, separate product?

• 25:39Speaker 00

  I would say that the synthetic data generation is not just based on of doing anonymization, because you can kind of do that type of anonymization without the underlying need for machine learning and AI.

• 25:50Chris Benson?

  I see.

• 25:51Speaker 00

  I think the issue that comes up is that you have a lot of different attacks, like re-identification attacks, that are completely plausible and possible on data that has been just anonymized in place, right? So just because you're anonymizing names and addresses of phone numbers and email addresses. Well, let's say, just for argument's sake, you have a bag of customer data, and you have a bunch of records. You know, I live in Baltimore, and let's say I'm your only customer, who is a male in his mid-30s in Baltimore. Even if you take all my personal information out, you might be able to join the fact that you have a customer like me in Baltimore, and I'm the only one. Well, now you've re-identified me. So with synthetic data is how can we actually generate a lot of those other risky fields that are really risky in aggregate, right?

• 26:36Speaker 00

  So you look at categorical fields like ages, genders, locations. How do you actually generate those records so that they can't be recombined to re-identify someone, but they're still useful? And when you want to look up, you know, the average amount of revenue you get from people in Baltimore or some type of aggregate question like that. And then on the second question, for us to generate synthetic data, you do need some type of training input to learn the underlying semantics. And then, once you have that model, you can generate any number of records. It doesn't have to be like a one-on-one. Like if I have like 5,000 training records, you can generate five synthetic records. You can generate 20,000. And so, but once you learn that semantics and the fact that you can generate any number,

• 27:20Speaker 00

  you can do a lot of interesting things. You can do enforcement on what you're generating. So let's say I want to generate records, but I only want to accept records that are of a certain category, like a certain age or a gender group. So then you can use that to synthesize new records to help balance the data set that might be otherwise biased and not have enough samples of something that you're trying to predict on, for example. And so, once you have a core model built, you can kind of generate records to meet a lot of those needs.

• 27:46Daniel Whitenack?

  We've mostly been talking about use cases around private data and privacy sort of aspects, but is this synthetic data generation capability, does it also help people who are working in sort of data scarce scenarios or like imbalanced data set scenarios? So let's say that we don't have any sort of personally identifying data in our, in our data set. We're not, you know, at least to our knowledge, we're not dealing with that issue, but we do either have an imbalanced data set. Or maybe we're just working in a sort of data scarce domain where we do have some data.

• 28:26Daniel Whitenack?

  Like you say, maybe we have 5,000 records, but we really need 25,000 records for our model. Is it viable to use synthetic data in that type of scenario?

• 28:35Speaker 00

  Yes, and that is actually one of the core use cases that we have experienced where there might be already a situation where data is deemed safe to use. I'll use like fraud as an example because fraud is a really good one where you have so many records that are not fraud. You're usually trying to predict the opposite, right? So like the, the field you're trying to predict is, is an actual fraudulent event, but you might have just not enough records of that fraudulent event. And so what we're able to do is kind of guide you through how to synthesize more records that fit into the fraud category. So that when you go and you build your actual machine learning algorithm, that there is enough of those fraudulent records there that it could actually, you know, create a proper decision boundary or whatever.

• 29:18Speaker 00

  So you have a net better model at the end.

• 29:22Daniel Whitenack?

  I was just wondering like, because sometimes I'm familiar with like imbalanced data sets where you do some type of maybe interpolation to get some extra points or something like that. But you're talking about really like training a neural network model or some type of machine learning model to actually generate this data. So maybe this is just too much of a simplification. I'm sure your methods are pretty advanced, but is the basic idea that you would have some of the fields of a record be input to the model during inference? And then it's trying to predict another of the fields of a record and similar to like predicting like the next word after a sequence of other words or, or something like that.

• 30:04Daniel Whitenack?

  Is that some of the basic idea or can you give us any sort of intuition and in terms of how the data is set up and that works?

• 30:12Speaker 00

  Yeah, that's actually very close. So our core engine for training and generating synthetic data is completely open source. It's on GitHub under Gretel AI. And our initial implementation with the open source package is it's kind of a wrapper framework and you can have a bunch of pluggable backends. And so right now, our first pluggable backend is a LSTM on TensorFlow. So it is kind of a sequential model, which is a lot different than a lot of the other techniques that are out there. And what we do is we have the ability to kind of focus on text input. And then in the open source package, we also have another module that kind of wraps that entire thing inside of like a data frame. And

• 30:53Speaker 00

  then we can infer different field delimeters and essentially it will reconstruct those records as a sequence. And then exactly that be, we can do it one of two ways. One, you can just say, just keep generating records with no input. Or you could specify what we call a seed, where it's like, okay, I only want you to create records that start with, you know, this age group, this gender, and then it'll complete those records. And that allows you to more efficiently increase a record of a certain type, based off of what your requirements are. And then what we actually ship as the product is we have a bunch of different layers that work on top of that to do data validation. We have separate models that we build that learn,

• 31:34Speaker 00

  that learn and enforce the semantics of individual fields to make sure that when records are generated, they still fit within the constraints that you had before, whether it's the right character sequences, the right structure of the fields, if you have like daytimes, making sure that like categorical fields are always recreated. So if you have like, you know, we don't want to invent a new state, right? So if there's 50 states that are in your, in a certain column, that we'll make sure we're only generating valid states. Those are things that we provide inside of the product. But the open source package lets you kind of just jump right in to build and train on structured or unstructured data.

• 32:20Speaker 04

  We deserve a better internet and the Brave team has the recipe for bringing it to us. Start with Google Chrome, keep the extensions, the dev tools and the rendering engine that make Chrome great. Rip out the Google bits, we don't need them. Mix in ad and tracker blocking by default, quick access to the Tor network for true private browsing and an opt-in reward system so you can get paid to view privacy respecting ads. Then turn around and use those rewards to support your favorite web creators like us. Download Brave today using the link in the show notes and give Tipping a try on changelog.com.

• 33:02Daniel Whitenack?

  John, I'm curious if some types of data are easier to synthesize than other types of data. So you've mentioned like dates, you know, categorical variables, categories, labels, you know, numbers. But then like we also have things like, you know, audio and imagery and like other things like that. What's the sort of current state of the art in terms of synthesized data? And what data types or like domains of data maybe are sort of the bread and butter right now? And maybe which ones have some challenges in terms of synthesizing data in certain scenarios?

• 33:41Speaker 00

  Yeah, so right now what Gretel ships is really focused around structured and unstructured text. So I think about like, you know, records from a database or any type of text input. Audio and video and imagery is next that we would probably see in a future iteration of the product. And it's something that we're working on now. You know, a lot of the state of the art around that is not kind of in our wheelhouse now because we were able to kind of just back into our customer problems via like structured records. And

• 34:16Speaker 00

  Yeah, I mean, that's just right now. It's just we kind of have to pick our battles. And right now, that's kind of the main one that we're focused on is being able to enable people to kind of synthesize new versions of database tables or static data sets so they can more safely share them.

• 34:28Chris Benson?

  So I'm curious, we've talked a little bit about kind of the product side of things. And also, you've made reference to the open source as well. Could you differentiate a little bit between kind of what each side of that has to offer? So to kind of give people a framework in their head about what they would go to for each? And where do they maybe step up from open source to your paid product services? That kind of thing?

• 34:50Speaker 00

  Yeah, absolutely. So right now, the open source packages, and there's two of them. One of them allows you to get started with synthetic data. And the other one that allows you to get started with kind of our traditional transformers to kind of mutate data in place. Those are Python libraries that are available to anyone licensed under Apache 2.0. And obviously, you go in using those, knowing that it's Python. And that already is kind of a qualification for a lot of our customers. And which isn't a problem. We have a ton of researchers and data scientists that live and breathe in Jupiter notebooks. And so they're able to plug that right in. Last August, we launched a beta of our more of our premium features. And that beta basically allows you to use our cloud service to test out our labeling capabilities.

• 35:35Speaker 00

  And then what we ended up doing was packaging up a lot of our premium capabilities, which kind of include automatic data validation. It does a lot of analysis to make sure that correlations across all your data are held and distributions across your data are held properly. We also released those available as an SDK that you can download through our authenticated API. We had a great about, you know, several months of going through that beta, getting a ton of feedback from users. And then, you know, what we walked away from that knowing is that what we really want to do is make this available to engineers everywhere. And engineers everywhere can't necessarily just download a Python SDK and incorporate it in their pipeline if, let's say, your entire backend is written in Java.

• 36:19Speaker 00

  And so how do we drastically simplify what these premium SDKs do? And so what we're building now is the ability to kind of launch Gretel services as kind of containerized capabilities that are backed by REST APIs. So now you can interact with our services purely through a REST API, which is completely language agnostic. Every engineer at some point has gone through the process of making API calls to a remote service. And so now that is kind of the qualification factor. We wouldn't have learned everything that we learned if we didn't have kind of that granular level capability out there through the beta. So now the entry point will kind of either be you could run Gretel services in your environment. We're also building a hosted service where we can run and kind of scale these capabilities for you. What it should be as easy as taking your data set or taking some records,

• 37:04Speaker 00

  a lightweight configuration, pushing it to an endpoint. And that endpoint will then trigger a whole bunch of backend work to learn, build a model, generate data for you. And at that point, we really just want to be a bump in the line in your entire data workflow to be able to call into these APIs. And so that's what we're working on now is just to really simplify that down.

• 37:22Daniel Whitenack?

  Based on what you've seen with your current users and customers, like if I'm a data scientist working on a new project, getting into some new data, do you have any recommendations in terms of workflow with your tools? Like, you know, when I get the data, I'm profiling it, I'm doing some exploratory analysis. Where and when should I be thinking about fitting in some of these, you know, REST calls or Python SDK elements into my workflow? So that I can make sure that I'm dealing with maybe both sides of things anonymity and creating synthetic examples? Are you seeing that done?

• 38:05Daniel Whitenack?

  Maybe more specifically, my question would be, are you seeing that done in a workflow? Like upfront and they do this on data? And then they use that data moving forward always? Or you seeing this as sort of a ongoing part of people's workflow?

• 38:19Speaker 00

  I would say that upfront is not the usual case that we recommend. And we recommend that there's usually a little bit of data cleaning you want to do, not down to the granularity of doing a ton of like all the exact feature engineering you would do to build a model. But at a minimum, and we have blueprints that help folks go through this process as well, is that you want to identify, you know, for example, columns that you probably don't need to worry about synthesizing because there's not something that your model's going to grab on to, right? So if you have records that have maybe like people names in them, typically those people names aren't going to be correlated to a lot of your continuous variables and your other variables in the data set. And if you can drop those columns first, you're going to save a lot of time on being able to train a synthetic model for that.

• 39:02Speaker 00

  And so other examples would be, you know, there's a lot of data sets we get from customers that, you know, are highly dimensional, you know, several hundred columns. And they're trying to train a model, you know, maybe like XGBoost model on that to predict something. And a lot of times, you know, what we recommend is like, look, if you can kind of train your model first, and then you identify what the algorithm deems are the most valuable columns, just drop a lot of the other columns because then you're going to get way better performance out of maintaining the correlations on different subsets of the data set. And so we do kind of recommend it's like at that point, like maybe right before you would actually think about actually training your model is like, once your data is pretty much in that good state around that ballpark.

• 39:45Speaker 00

  But it completely varies based off of, off of use case. And we have some customers that the first stop is coming to Gretel because they want immediately to take advantage of the PII that they could remove. So I'd say it definitely varies.

• 39:56Daniel Whitenack?

  Yeah, and I found that Gretel's blueprints repo, which is, seems pretty interesting. So you, I see a bunch of these examples, boost massively in balanced data set, create synthetic data from CSV or data frame, all sorts of examples. So if our listeners are interested in that, it looks like there's some notebooks and things in there that they can look at. We'll link that in our show notes for sure for people to take a look at. But maybe one thing to kind of start us thinking about, you know, things into the future. Where do you see our, like, the current challenges that are unsolved right now in terms of privacy and maybe data augmentation or synthetic data?

• 40:37Daniel Whitenack?

  What are some of those problems out there that you still see as, as open problems that need to be addressed?

• 40:44Speaker 00

  Yeah, I'd say there's a couple of problems and there are some semi-related. So one of them is it's still a very nascent field. And there's a lot of tools out there and there's no magic bullet, right? There's no way just to magically take a data set and completely create a version of it that is, you know, perfect and doesn't violate privacy, right? There's always going to be a trade off between utility and privacy. And helping people understand that, I think is going to be a really big challenge. And there's a ton of great research out there into how to kind of do that trade off between utility and privacy. And that's one of the things that we want to figure out is how to make that more obvious to, to engineers when they want to anonymize data or make data safe to share.

• 41:30Speaker 00

  It's like all these knobs you can tune, right? And like ideally, you don't want to go to a software engineer who's maybe a full stack engineer, and they have access to a production table and they want to make a safe version of that data. You don't want to ask them to tune a bunch of like hyper parameters for TensorFlow LSTM, because they're going to be like, whoa, I don't know what's going on here. But you might want to ask them to say like, look, what is the trade off in utility and privacy that you should have here? Like, are you sharing this externally? Are you sharing this internally? Ask them those levels are, and then how can we infer what all those really nitty gritty knobs are that need to be turned for the underlying model that's being built? Which kind of segues into the second problem I see is that making these tools generally available

• 42:13Speaker 00

  to software engineers everywhere is, is going to be a massive challenge, right? And you can't ask every engineer to download a Python SDK and have like a crash course in machine learning to ask them to kind of build a safe version of their data set. And so how do we kind of bundle and package these capabilities in a way that engineers everywhere want to use as part of their day to day workflow, right? If you look at companies that made things like dead simple, like Stripe made payments less scary because they have a ton of language bindings. It's really easy to integrate into your app. It's just like another API call that you make and you don't think about it. And they're doing all this heavy lifting of like processing payments, which is a very complex thing. Uh, how do we kind of generalize down to that level? And that's kind of like.

• 42:58Speaker 00

  Definitely one of the big visions and missions that we have here at Gretel.

• 43:01Chris Benson?

  So I'm kind of curious, as you're describing that and like going back to the beginning of that second, you know, challenge that you're looking at in terms of, it really strikes me the scale of what needs to happen here. So kind of beyond the specific challenges that maybe need to be solved and that maybe Gretel wants to address, the scale of this is definitely holding a lot of engineers back, but you know, that are contending with this and can't get where they want to go. And if you're looking out over the next few years, it kind of where this has to go as an industry and the, the need to broadly at scale, be able to increase productivity in AI ML in general, and this being such a core tenet of that, where do you see the industry going with

• 43:44Chris Benson?

  that? You know, what needs to happen in the large to enable, you know, 10 times, 100 times as many engineers to be able to overcome these kinds of problems and get productive with the problems they're trying to solve? You really got me thinking as you were answering those last two about how to get there from here. How do you get there?

• 44:03Speaker 00

  That is a great question. I think in my minds, and this is something that we even do inside of Gretel, is that I think one of the key things that has to get us there is that we just have more of a free form exchange of, I guess, ideas and talent among different types of developers and engineers that are out there. And when you look at like a lot of organizations, there's still, I guess, always a lot of segregation between like your platform engineers and your software engineers and your data engineers, and you have your machine learning engineers and your data scientists. And really, I think everyone needs to be able to do a little bit of everything. And like, how do you kind of build toolsets that allow, you know, a software engineer

• 44:47Speaker 00

  to easily take a look at the data, even though it's using some complex machine learning capabilities without having to go and request, you know, a machine learning engineer spend tons of time doing it when, you know, that MLE should be maybe researching other parts that are like more vital to what the, the core mission is of that, of that organization. And you know, you see that there's been a lot of acceleration in like micro frameworks for building REST APIs. And so like that is, you know, a really good example for how that allowed a lot of people to operationalize things, right? Even as a data scientist, you could fire up, you know, model training and predictions and back it with a REST API and make it generally available. Like, what's like the machine learning version of that, like micro framework for REST API that allows software engineers to kind of quickly take use of all the capabilities that are out

• 45:30Speaker 00

  there that are up and coming with synthetic data. And we do that at our company now. We have a completely blend of backgrounds and really it's, it's, we don't want the whole like sequential motion of like, well, you know, this person builds the model and then hands the model over to this person who builds this. Like, we just want everyone to be able to kind of plug in and build. And so how do organizations move to that kind of methodology?

• 45:52Chris Benson?

  Tearing down the walls there, so to speak, of those distinctions.

• 45:56Daniel Whitenack?

  Yeah, for sure. Well, John, I'm super excited about what Gretel is doing. And I really appreciate your detailed description of why these things are important and how you're solving some of these problems. I think it's really important. We'll make sure and link, like I mentioned, some of these links that we talked about in our show notes for people to check out. Please go and check these out. Try to generate some synthetic data with their tools and check out their platform. And yeah, thank you so much, John. Appreciate you taking time to talk to us.

• 46:26Speaker 00

  Awesome. It's been a pleasure to be on the show and thanks for having me.

• 46:33Speaker 04

  Thank you for listening to Practical AI. If this is your first time, make sure you subscribe so you don't miss a thing. Head to practicalai.fm to subscribe or find us in Apple Podcasts, Spotify, or wherever you listen to podcasts. And if you get value from the show, please do share it with a friend or a colleague. We appreciate you spreading the word. Practical AI is hosted by Daniel Whiteneck and Chris Benson. It's produced by Jared Santo and our music is provided by Breakmaster Cylinder. We are brought to you by some awesome sponsors. Shout out to Fastly, Linode, and Launch Darkly. That is our show. We hope you enjoyed it and we'll talk to you again next week.