Building a data team

Inspired by a recent article from Erik Bernhardsson titled “Building a data team at a mid-stage startup: a short story https://erikbern.com/2021/07/07/the-data-team-a-short-story.html”, Chris and Daniel discuss all things AI/data team building. They share some stories from their experiences kick starting AI efforts at various organizations and weight the pro and cons of things like centralized data management, prototype development, and a focus on engineering skills. Sponsors: PSSC Labs https://pssclabs.com/practicalai – Solutions from PSSC Labs provide a cost effective, highly secure, and performance guarantee that organizations need to reach their AI and Machine Learning Goals. Learn more and and get a FREE consultation today at pssclabs.com/practicalai https://pssclabs.com/practicalaiThe Brave Browser https://brave.com/cha913 – Browse the web up to 8x faster than Chrome and Safari, block ads and trackers by default, and reward your favorite creators with the built-in Basic Attention Token https://basicattentiontoken.org. Download Brave https://brave.com/cha913 for free and give tipping a try right here on changelog.com https://changelog.com. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: “Building a data team at a mid-stage startup: a short story” by Erik Bernhardsson https://erikbern.com/2021/07/07/the-data-team-a-short-story.htmlBooks “Cracking The Data Code” by Mike Bugembe https://www.amazon.com/dp/1781333335“Data Driven” by DJ Patil and Hilary Mason https://www.amazon.com/dp/B00SXHFTASUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  I created a little bit of a nightmare myself. I was at a previous employer. It was a large company that everyone's heard of. I was hired in to create the first AI team in that organization. And I made the mistake of hiring a team of data scientists. And in this case, they were true data scientists, but I ended up making assumptions about what those capabilities those individuals had. They were all probably better at the mathematics of deep learning than I was. But then when we got to the point where we needed to do DevOps and deployment and all that, there was absolutely no understanding. Everything from SQL to what is a container, all these things. It was an interesting ramp up experience,

• 0:42Chris Benson?

  and I had to make some course corrections late on after hiring several people in that capacity. I had to specifically recognize that there were other skills that I had not addressed at all and go hire people for those themselves.

• 0:56Speaker 01

  Big thanks to our partners, Leno, Fastly, and Launch Darkly. We love Leno. They keep it fast and simple. Check them out at leno.com/changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by Launch Darkly. Get a demo at launchdarkly.com.

• 1:14Speaker 02

  With advancements in AI and deep learning evolving at lightning pace, it's more important now than ever to research the best options suited to your unique needs. This is particularly true when building custom systems and those systems that are GPU-heavy. Not only do the applications running on the system matter, but your AI infrastructure and budget constraints need to be front of mind as well. PSSC Labs, which is an HPC and AI custom solutions provider based in California, has been creating high-performance computing systems to meet their clients' unique enterprise computing challenges for more than 25 years. And with cloud computing costs growing at astronomical rates, plus companies increasingly losing control of

• 1:57Speaker 02

  their data security, it is no wonder that enterprises and government agencies need to continually look for ways to take back control of their data. Solutions from PSSC Labs provide a cost-effective, highly secure,

• 2:10Speaker 02

  And performance guaranteed that organizations need to reach their AI and machine learning goals. For more information and a free consultation, please visit pssclabs.com/practicalai. Once again, that's pssclabs.com/practicalai.

• 2:35Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at kingdaw.com/community and follow us on Twitter. We are at Practical AI FM.

• 3:00Daniel Whitenack?

  Welcome to another Fully Connected episode where Chris and I keep you fully connected with everything that's happening in the AI community. We'll take some time to discuss the latest AI news and we'll dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitenack, data scientist with S I L International, and I'm joined as always by my co-host Chris Benson, who is a strategist at Lockheed Martin. How you doing, Chris?

• 3:27Chris Benson?

  I'm doing very well, Daniel. How's it going?

• 3:29Daniel Whitenack?

  Oh, it's going great. I hear almost congrats are in order. You're about to be an official pilot with your pilot's license. Is that right?

• 3:39Chris Benson?

  I actually got the license last Sunday.

• 3:41Daniel Whitenack?

  Okay, so you are actually official.

• 3:44Chris Benson?

  Frighteningly, I am a licensed pilot, and I know that has nothing to do with AI, but I appreciate the congrats. So

• 3:50Daniel Whitenack?

  I'm sure there's some type of AI system that images the Earth or airplanes or manages air traffic control that will find you at some point.

• 4:00Chris Benson?

  At some point, yes. Something like that might exist. And you know, I do work for an aerospace company too. So aside from what I'm doing, there might be things like that out there.

• 4:10Daniel Whitenack?

  I'm trying to remember the exact conversation we had, but there was like a conversation we had where we talked about this. I think it's persistent surveillance that is being promoted, I think, by at least one company where essentially you just Record video of everything all the time, but in low quality.

• 4:29Chris Benson?

  Yeah.

• 4:29Daniel Whitenack?

  So low enough quality to where you can't like personally identify all the people walking around and that sort of thing. But high enough quality to where if something goes down, then you can sort of backtrack and figure out what what happened. So I don't know if that's happening above my head now. I don't think Lafayette, Indiana is maybe the highest priority target for that sort of system.

• 4:53Chris Benson?

  If I knew about something like that, I couldn't talk about it. Except.

• 4:55Daniel Whitenack?

  Oh, yeah, yeah.

• 4:56Chris Benson?

  Except, except, wait, wait a minute. Except my wife is English and, you know, so I spend a fair amount of time in the UK. And, you know, they do have cameras pretty, you know, just kind of all over the place there.

• 5:08Daniel Whitenack?

  Sure.

• 5:08Chris Benson?

  You cannot walk through London without being filmed all over the place, you know.

• 5:13Daniel Whitenack?

  Yeah.

• 5:14Chris Benson?

  That's just normal life. I think we're, I think you're in the US. That's becoming more and more normal life as well, because I don't know about you, but everybody has a ring or a nest on their door at this point.

• 5:23Daniel Whitenack?

  Yeah, yeah, definitely a lot. And we have, you know, cameras in her, my wife's business that all around the business and all sorts of that that stuff. The network video recorders now, you can record so much for so cheap because you just load them with solid state drives or or even spinning discs and, you know, you can get a lot, a lot of data. So.

• 5:46Chris Benson?

  Yeah, it's kind of funny on on our Nest Cam, we, you know, how only like postal workers, you know, and we are talking a little bit about, you know, AI related stuff. But just on our Nest Cam, we found an Amazon driver who decided to to put a package into our mailbox, which isn't legal and stuff. But I was kind of amazed in an age where not only are there cameras pretty much everywhere now, but now there's all of this automation. There's tons of deep learning analysis, super cheap, deployed all over the place. You have folks be careful what you do. There's somebody watching.

• 6:17Daniel Whitenack?

  The building that my wife's business is in, and then we own a second building, um, which her business will expand into. And right now there's not a lot in there and there's no internet over there.

• 6:28Chris Benson?

  Right.

• 6:28Daniel Whitenack?

  So what I, as a sort of. quasi-security camera solution. Now they have like deer cams that people use or like wildlife cams that people put out in the

• 6:41Chris Benson?

  I have tons of this.

• 6:42Daniel Whitenack?

  Yeah, people put them out in the forest, but now they have the ones where like when they see an event, they work off the cell signal, they can ping your phone, but also they have like built-in pre-trained models for like different animals, you know, deer or turkeys or whatever. I don't think there was a people one in the one that I got. Oh, like hopefully not that many people are hunting for for people. But I mean, we're not using them for for hunting. We're using them for for this purpose. But, uh, yeah.

• 7:12Chris Benson?

  You're raising a really good point, and that is AI is just everywhere these days. The, you know, these convolutional models are so cheap to deploy. And like aside from us talking about, you know, AI from the AI industry perspective, the animal protection nonprofit that my wife and I run, we have tons of those cameras. And and yes, you are seeing these pre-trained models, you know, in these common tools and these consumers have no idea.

• 7:37Daniel Whitenack?

  With a lot of data to sift through.

• 7:38Chris Benson?

  It is. That is starting to happen right now. And and I got to tell you, that's, I would have thought that's about as far from AI as I could possibly be. We're seeing models now turn up in the most unlikely of places, it seems.

• 7:51Daniel Whitenack?

  I guess we can turn to what we had talked about discussing today, which I think is a really interesting topic. It's something I've been thinking a lot about recently is building a data team in a company. There is a really interesting article I saw, yeah, last week, very recent from Erik Bernhudson about building a data team at a mid-stage startup, a short story. And the format of the article is quite interesting. It's almost like a parable. So he's sort of taken a bunch of his experiences over the years, building data teams and wrapped them up in this sort of parable about a data scientist coming into a new company, charged

• 8:34Daniel Whitenack?

  with building a data team and the experiences that such a person might encounter, which I think is quite intriguing and refreshing to. To read.

• 8:45Chris Benson?

  You had shared the article with me before the episode and he puts it into the reader's perspective there. You know, you're the one going through it. You notice a lot of code starts to do that kind of thing. It was an interesting perspective shift because it puts it onto the reader. I liked it.

• 9:00Daniel Whitenack?

  It definitely got me thinking about a lot of the experiences I've had in the past. And I don't know about you, you've been at several places, I'm sure.

• 9:08Chris Benson?

  Both big and small.

• 9:09Daniel Whitenack?

  So as part of those experiences, I'm sure you've been charged at one time or another with maybe not always starting the first data team at a company, but building a data team at a company. Is that, is that part of your, your experiences in the past?

• 9:23Chris Benson?

  It is indeed. And actually I have started the data team more than once, just mainly because I've been working for a long time. And, or, you know, a while back, companies didn't even have data teams. They didn't exist. And if, and ironically for a while, you just had a DBA, a database administrator. And that person was expected to do anything that had anything to do with data for a long time. The, the idea of data teams is a relatively recent thing in terms of mainstream. And when I say that, meaning a number of years now, but you know, it hasn't been decades.

• 9:55Daniel Whitenack?

  Yeah. And by data team, it's probably worth us discussing even that, even that term. In my mind, I have sort of this vision of a team of mostly data sciencey people, but maybe with some kind of towards the side of infrastructure more, and some maybe more towards the side of experimentation and research or prototyping type of people. When you think of data team, what, what comes to your mind? I'm curious.

• 10:28Chris Benson?

  Usually it's a, it's a little bit different. It's still very unique. There's not a standard concept for what a data team is. And I've had some misfires in that. There are different roles and those roles are as distinct as the software development world has become. With developers and, you know, different DevStack Ops people and or DevOps people, it's maturing is what's really happening. It's been maturing rapidly because this AI thing, this deep learning taking over the world has been happening so fast. Before this era, not a lot of companies had dedicated data teams. And so we're still in that process of figuring it out. We have lots of them at my employer and they don't all, even in the, in the organization, they don't all think of themselves in the same construct.

• 11:12Chris Benson?

  They, you know, you can ask different data teams, what is a data team? And they'll give you different answers.

• 11:15Daniel Whitenack?

  Yeah, one anecdote that I actually was discussing with someone recently, a friend from, from college, which I won't share the details because I didn't get his permission. But basically the, the story was he has a engineering background, is working in the industry and he had an opportunity within the company to, because they knew he had some coding skills and some modeling skills and that sort of thing. They basically wanted him to sort of become a data scientist within the company because he had a lot of the industry knowledge and they knew he was sort of gifted on the coding and programming and modeling side.

• 11:58Daniel Whitenack?

  So they said, you know, hey, why don't you sort of start our in-house data science team? And I think what he's found is there are people throughout the company that are doing data sciency things. They're not really coordinated maybe well yet. There's not a lot of like ML Ops and sort of good operational and deployment strategies yet. And so that's a lot of what he's parsing through is what are, what are those best practices? And they're not giving him, you know, a full data engineering team to solve all those things for him. They're saying, hey, you know, you figure out a way to do it with these people that

• 12:39Daniel Whitenack?

  have been identified as data scientists or data analysts throughout the company and saying, you know, hey, figure out how to deploy models. Such that people can use them, but you're going to have to figure it out. And we don't really have a lot of pre-built infrastructure for that. So yeah, I mean, it's a big, it's a big challenge.

• 12:59Chris Benson?

  That actually happens at companies, large and small. I've seen that at both and it's interesting as you try to learn how, how people have moved in. I think a lot of early people have come in, which certainly includes me. And I think to some degree, to some degree includes you from the software development side with less experience on kind of the pure data science-y world and that there's a ramp up for people that are, that are moving into those kinds of roles. Because just because you've coded doesn't mean that you understand statistics deeply and understand the various mathematical constructs that you need to apply to this. And so I know for me, some of which I had in school a long time ago, but there was definitely

• 13:40Chris Benson?

  a ramp up for me to be able to be productive, especially four or five, six years ago. You know, when before things were quite so popular and when individuals were kind of doing everything end to end.

• 13:52Daniel Whitenack?

  Yeah, one of the points that was sort of brought out in this article is that sometimes companies hire in or promote people to do AI or whatever and figure out where it should be done. And come to find out, maybe the immediate needs aren't that sort of machine learning and AI stuff. Or maybe you can't even get to those yet, but it's sort of equal type things or like people that say, you know, I wish I could figure out this number or this metric. And it's totally accessible. They just can't translate that, those words into either SQL or scripting or whatever it takes

• 14:34Daniel Whitenack?

  to pull, extract that data out from its various sources and get that in front of them. That's sort of one of the things that is needed most often first.

• 14:44Chris Benson?

  I created a little bit of a nightmare in that area myself. I was at a previous employer. I was hired in, there was a large company that everyone's heard of. I was hired in to create the first AI team in that organization. And it was early enough to where I had come into that from generally smaller operations in that. And I made the mistake of hiring a team of data scientists. And in this case, they were true data scientists, but I ended up making assumptions about what those capabilities could do, you know, what, what those capabilities those individuals had. They were all probably better at the mathematics of deep learning than I, than I was.

• 15:27Chris Benson?

  I had self taught and self studied and I could, I could hold my own, but you know, that's what all of their formal education had been. They had all recently, for the most part, come out of college, out of university. And, but then when we got to the point where we needed to do DevOps and deployment and all that, there was absolutely no understanding. Everything from SQL to what is a container, all these things. It was an interesting ramp up experience and I had to make some course corrections late on after hiring several people in that capacity. I had to specifically recognize that there were other skills that I had not addressed at all and go hire people for those themselves.

• 16:15Speaker 02

  We deserve a better internet and the brave team has the recipe for bringing it to us. Start with Google Chrome, keep the extensions, the dev tools and the rendering engine that make Chrome great. Rip out the Google bits. We don't need them. Mix in ad and tracker blocking by default. Quick access to the Tor network for true private browsing and an opt-in reward system so you can get paid to view privacy respecting ads. Then turn around and use those rewards to support your favorite web creators like us. Download Brave today using the link in the show notes and give Tipping a try on change.org.com.

• 16:57Daniel Whitenack?

  Chris, you brought up a really interesting point which crosses over into the hiring side of things, which is, there's a few scenarios that could happen, right? You could hire in like machine learning, AI, data scientists who are expecting to do machine learning, AI, advanced type things. And if the immediate needs are, hey, take this disparate data and assemble it together and get it in front of people, there could be some job satisfaction issues that you might, that you might run into, just because, you know, I'm not saying that all people like that would think that that sort of thing is beneath them. You know, I think a lot of people enjoy dipping into that at some point.

• 17:41Chris Benson?

  Or they might.

• 17:42Daniel Whitenack?

  Yeah, they might. I mean, if a year goes by, two years go by, and like, there's, you know, the only thing they've done is write SQL, there's a mismatch. At the same time, like you were saying, you know, you could, you could hire in the other way and then have trouble advancing into the more, you know, sophisticated analyses and that sort of thing. So what is your take on how to parse out that, that hiring bit and how to think about who, who you should really be bringing in because that could create a lot of issues?

• 18:17Chris Benson?

  I have an answer for that, but I think it's a little bit of a cheat on your question because I think the question assumes that you don't necessarily know all the things. But I've had the benefit of a little bit of experience at going through this process several times now in several different organizations. And so for me, I would hire in to reflect the entire workflow. So I know what a good data science, including deep learning workflow, looks like these days from beginning to end, all of those things that have to happen, from understanding the problem, to identifying what kinds of models need to be there, to how you would implement them, what kind of equipment you need for those.

• 19:02Chris Benson?

  Models, the software, how you do the DevOps or DevSecOps to get those all the way out to deployment to production. And so there's the, from the early conception, almost at the business level, all the way through those various steps to the end. And you have something out there. It's a model that's wrapped in software and it's doing something productive in the world. At this point, I would catalog those. I know how much effort roughly would go into each of those areas, ballpark. And I hire against that kind of those levels of effort in those different stages to try to get a complete team. And depending on what the budget is and how many people, you know, I will kind of group some of those tasks together or whatever and figure out what that is.

• 19:45Chris Benson?

  And also depends on the candidates I talk to. I may make a change. If I get a particularly capable candidate, then that can change how I'm thinking about things on a tight budget.

• 19:54Daniel Whitenack?

  What if the CEO that hires you in is expecting some cool AI data science, machine learning type things? But what you find out very quickly is that that actually isn't the most immediate need. The most immediate need is data aggregation and getting some metrics in front of people. How do you handle that situation with your, with your leadership? Any, any thoughts there?

• 20:25Chris Benson?

  First of all, they always expect that. That's not like what if, that is every time. Cause the people responsible for that, even, even people who are usually supposed to be technical leadership are beyond the details. At the point, they're making the decision, they're far enough along in their career because they may not be handling those technical details on a day to day basis. And therefore they don't really understand anymore, even if they think they do. And so there is a, there is a gentle education process and there is a discussion of what happens based on, you know, if you, if you just run forward and do, try to do deep learning when you're not set to be able to do it effectively.

• 21:06Chris Benson?

  You're running into a wall and the harder you try to do it, the faster you're running into that, that brick wall. So there's a, there's a bit of an education process.

• 21:14Daniel Whitenack?

  And also there's, even if there is an opera, a clear opportunity to move into some AI related work and machine learning stuff.

• 21:24Daniel Whitenack?

  My experience is the right data that you need is usually very hard to get, or it's very fragmented. This article also talks about fragmented data and that sort of thing. So I've been in places where you want to ask the question like, okay, to train this model, I need all of this type of data. But that's a sort of anti-pattern because maybe people are used to like, let's say, the example is in a financial institution or something like that. The previous maybe support people or customer service people,

• 22:05Daniel Whitenack?

  or even analysts or whatever, they're maybe used to looking at a very small set of transactions, or at a single transaction, or a single user and all the things that's gone on for that user. And so when you ask a question, give me all the transactions of this type, it's sort of an anti-pattern for how they've been looking at the data. And their systems aren't really set up for that sort of query even. So it may be that you have to push the infrastructure, or rethink how you're getting the data, or the patterns that people are pulling data in order for you to even set up your problem and have success in doing any type of modeling.

• 22:46Chris Benson?

  I have had that same experience. And ironically, just because it's at the top of my mind, I'm thinking about that same previous employer, large company with a well-known name, and lots of physical hardware products that come out of that organization. They collect a fair amount of telemetry from those various products. But what we discovered was based on the things that we wanted deep learning models to do, and ways of improving that product's capabilities and the user experience, that most of the telemetry was absolutely useless for our purposes. It was, it was great for figuring out what went wrong with the product after the fact, but it didn't actually, it couldn't be used to teach a model

• 23:31Chris Benson?

  how to more effectively do the capability. And I think that's a common, I, in my experience, we saw that across products. And I think that that would probably hold true across many organizations where you, where you may collect a lot of data, but that doesn't mean it's the right data. And it's not the data that's going to help you get where you want to go.

• 23:49Daniel Whitenack?

  When you enter an organization and you're building a data team, You start interacting with product teams and customer support and that sort of thing. If those teams aren't yet data-driven, what are some of the things that you think motivate those teams? If it's not data, how are they making their decisions in a non-data-driven way? Because that's often what I've I've seen is like, I start interacting with a product team or something like that, and they aren't making data-driven decisions. One of the ways to think about how to change that culture is to think about what what is motivating them. What has been your experience in the past in terms of the culture of the teams that you start interacting with when you build a data team?

• 24:33Chris Benson?

  I think that's a huge issue, meaning the word being culture. And because when, if in general, to generalize and based on at least what I've seen, when teams aren't using data to derive their decisions, you know, in an explicit, objective manner, then they're usually relying on experts, or at least self-proclaimed experts. And there those decisions often are somewhat arbitrary and oftentimes not consistent, even even with that person's other decisions across time and across similar situations. So, and in doing that, there is a belief because they've built a business on it. So this is one of those, this is one of those kind of political, cultural things

• 25:17Chris Benson?

  that is deeply entrenched in an organization and that you, as as the new leader of a data science team, are forced to contend with, and it's a really hard problem. It's a it's a hard nut to crack. They may have run years or even decades on that approach. So you have to find a way to convince them that there is a better way and that they'll get better results from that. Because they've gotten, in their opinion, they've got good results, which is why they're still doing it. If they weren't getting some level of result, it would have already passed. But it's your job, usually in the very early days, to figure out how to address those perceptions.

• 25:55Daniel Whitenack?

  I'm struck by the scenario that's talked about in this article from Erik. He talks about a sort of mid-stage startup around 10 million. And so that's about the size that that my wife's business is. And looking at her marketing and and sales customer service department.

• 26:14Daniel Whitenack?

  If you think about that early stage, like you were talking about, it was basically her, she built up a ton of expertise and internal knowledge in terms of what was working and what was driving sales. And that basically boosted the company to mostly where it's at. But then you start thinking, okay, well, it's out of size where we're hiring in marketing people or people that are supposed to be driving sales. Is it reasonable to assume that each of those people are going to have both the sort of ownership over the business and the drive to like build up that, you know,

• 26:55Daniel Whitenack?

  level of internal knowledge and, you know, there's going to be appropriate knowledge transfer between all of these people coming in. It's just not the case. But like you say, you hit this wall where now how do we be creative? How do we try new things and how do we make sure that we're driving new sales and growing? It has to be data driven at that point, but the culture, it wasn't sort of set up that way organically, not because they weren't wanting to be that way, but because it just sort of organically grew into this department where they're doing the things that like you say, they know work to some degree and they, they felt like we're still working. And so I think now in her company, they're doing a lot of thinking about, yeah,

• 27:40Daniel Whitenack?

  how do they, how do they drive that data-driven culture in marketing? And some of it's just the very simple stuff that even Eric talked about in his article, like, do people understand how UTM codes and website traffic works? Like, there needs to be some knowledge sharing there. And then there needs to be common, you know, data gathering. Like, okay, we've got this stuff over here in Facebook Pixel and this stuff over here in Google Analytics and this stuff over here in Shopify and this stuff over here in these sort of random places. No one can like really coalesce around anything if, if all of that's fragmented out. And so there needs to be data aggregation together. There needs to be a common way to look at it.

• 28:23Daniel Whitenack?

  And then, you know, building that culture, like it, it's also about people's motivation. You have to think about, if I'm gonna show something to this marketing person, how are they motivated by that? I mean, it could be like commissions or something, right? Like if you, if you make this much off of Facebook ads. Then you you get this commission or this incentive, right? Well, pretty quickly, they're going to want to know how much they're making off of Facebook ads. And if they're not setting up their UTMs, right? And they're not using the the common systems where data is coming in, then they're not going to be able to know. Right?

• 28:59Chris Benson?

  You know, it is you're telling me that and I'm listening to kind of the just these normal struggles of of your wife's business going through this. And I'm struck with the fact that you have a brilliant wife who is really good at what she does. And you are you are really good at what you do. And and

• 29:20Daniel Whitenack?

  I'll give her the I'll give her

• 29:22Chris Benson?

  At least she has the benefit. She is a brilliant business person and love talking to her and love learning from her. But she also does have the benefit of being married to you and you're able to put these things in front of her. Most business people, as smart as they are, are don't have such an intimate fountain of knowledge about these particular topics. They know their business, but they don't necessarily understand, you know, have someone who can inform them all the all these data points. They can hire people to do that. But to your point, those people may not be quite as motivated as you are as a as a business owner or the spouse of a business owner.

• 29:59Daniel Whitenack?

  It kind of brings a lot of weight to this whole building a data team side of things. Because I think about, like, let's say I'm not in the picture and she hires a data person to figure out, you know, figure out how to make our company, you know, data driven and using modeling and all this stuff. And she hires that person and that person spends all their time on, you know, fancy deep learning stuff, but doesn't address these basic issues of like, how does the marketing team operate? What's the culture? What numbers do they need to see in front of them? That actually could. I mean, I'm not saying it's gonna, it would take down the business, but it's gonna make a significant negative impact.

• 30:43Chris Benson?

  Absolutely.

• 30:45Daniel Whitenack?

  Because it's, it's not what's needed, right? So I think that people coming into these sorts of positions need to be sort of not scared, but sober minded in the sense of, you know, really having the perspective of what, what are the needs of the business? Rather than what's the coolest project that I can work on?

• 31:05Chris Benson?

  Or what's the way I build my resume? Or what's the way I get the training class that I really would love to do, but isn't necessarily directly in line with what we're trying to accomplish at the organizational level? So yeah, there's a huge risk. You, you're able to bring a purity because all you care about is the success of the organization, but that's not always the case.

• 31:26Daniel Whitenack?

  One of the other points that Eric brings up, which I think is really interesting is executive support for ML AI type things. And this sentiment that sometimes comes up when you're an AI or machine learning person, maybe you've dealt with all of the sort of getting metrics in front of people thing. There is a really important problem that you think is solved really well by machine learning and AI. You've, you know, trained a model or whatever it is. And like, you're genuinely convinced that this is a meaningful thing that you've done that has great benefit for the company.

• 32:09Daniel Whitenack?

  But you try and try to like build support for this and you get nothing. What are maybe in your experience, what maybe is going on in that situation where you're trying? You have this solution, but you're having trouble like either helping people in the organization understand it or understand the value or understand the benefit. And you know, buy into it and support it.

• 32:33Chris Benson?

  It's your job to communicate that as the leader of the effort or as the visionary who understands what's possible. You have to be able to explain it, but without diving into all the technical details, you have to be able to. Maybe you're either not using data science, or maybe you're using more traditional mechanisms in data science. And you know that a convolutional neural network or a natural language processing model has a particular strength in a certain area. You've got to find a way to communicate that and doing that by, to some degree, dumbing it down. And I don't mean that in a derogatory way. I mean that of your audience is not as technical as you are. And so you have to get that communication at the level that they get the value.

• 33:17Chris Benson?

  You have to abstract it to a point where they can, where they're going to understand that. But that's, it's really on you. It's not on them. It's your job to show them, if I look at other similar fields and they used to do it this way. And there's, here's a paper or an article about that. And this other company who has something similar to ours, or maybe has a similar interest in this particular task, did something and they've gone all in after testing it. This is why, these are the basics of this, that's the basic of that. There's a definite advantage, we should invest in that. And also know when not to invest, because we've come through an age the last few years where so many people are just wanting to do AI so they can say they're doing AI. And there's a lot of things that deep learning is, is not the best thing for.

• 34:02Chris Benson?

  Or, or at least it's way more expensive than other options that are equally as good or nearly as good. So it's, it's finding somebody who understands that to lead your effort and who can communicate that effectively to all the stakeholders.

• 34:16Daniel Whitenack?

  I think that's why maybe certain tools that have come out recently, like, let's say a Streamlit or something like that, can actually be incredibly powerful to solve this problem. Because would you want to develop your whole product in Streamlit? Maybe, maybe not, depending on what it is.

• 34:38Chris Benson?

  But would you use that tool in order to prototype something out and demonstrably show the value of what you're doing?

• 34:48Daniel Whitenack?

  Yeah, definitely. Prototypes and sort of minimal viable things are, I think, really valuable in this case. And often, the way that Eric puts this is, sometimes the data team just doesn't take it upon themselves to get the work into a place where it demonstrates value and is reasonably easy to ship. So you could have a Jupiter notebook, right? Or you could run a model on your GPU server, right? And then do your test set on your GPU server and show that you get, you know, 90% accuracy or whatever it is. And then your executive team's like, oh, that's great. You know, let's ship it. How do we get it into this product?

• 35:33Daniel Whitenack?

  And if your answer is, oh, I haven't thought about that yet, or it only runs on the GPU server, or I don't know how to like extract it, we have to figure out that problem. Do you want to invest in that? It doesn't really inspire a lot of confidence, right?

• 35:47Chris Benson?

  It doesn't.

• 35:48Daniel Whitenack?

  So I think just that one step of, do we expect data teams or data scientists to actually build robust products? Maybe in certain cases, they they're part of that, like in smaller startups and that sort of thing. In larger companies, maybe not. But should they be expected to maybe go that little extra mile to create a prototype that demonstrates value and gets things in in front of people in a meaningful way, even if it doesn't scale all the way up? I think that's like, that's a huge, huge point that I don't see emphasized that much, you know, because you see a lot emphasized about getting your models training well and,

• 36:30Daniel Whitenack?

  you know, evaluating well, but not this sort of prototyping bit of the problem.

• 36:35Chris Benson?

  I agree with that completely. I think that people, you know, overtalk it a little bit early on and don't recognize the value of lightweight prototyping to help you figure your way through it, figure out what it is that you need. And prove out that what you're thinking is actually accurate because if you think of how many organizations build things that actually are not very useful in the end, or don't have the audience that they originally expected, that you can solve that in part with prototyping and help yourself hone in on. And I think that a lot of organizations are, they interpret that as scary in a way, or potentially as long and expensive. And so they make the mistake of trying to talk their way through it. And I've seen that through my

• 37:18Chris Benson?

  entire career. And in my earliest days, there was no such thing as agile. That didn't happen for a while, actually. And eventually, this agile movement around the beginning of the 2000s came about. And it's taken the next 20 years for that mindset to really take hold in a broad sense. And so you need to try stuff out. And you need to be ready to go off and do a coding spike on something and figure out with a simple model, whether or not this is doable, whether it could be, or if it's doable, can you deploy it? Is it deployable in a reasonable way? Do you have resources where you need to deploy it? So your points are well made. These are all things that you need to be thinking about when you're building these teams and you're looking

• 38:02Chris Benson?

  for the people with the right mindset and the right skill sets so that you can be successful.

• 38:08Daniel Whitenack?

  And on your data teams that you've built in the past, in terms of the communication between the data team and maybe like organizational units that that data team is serving, let's say, marketing, or supply chain, or whatever it is, how does the communication work between those sort of external or internal, but other organizational units and the data team work? In your experience, often, does that flow through the management of the data team? Or does that flow directly through the individual? Data scientists working on various products or?

• 38:43Chris Benson?

  So I don't think there's any standardized way and I've seen that happen in all sorts of different ways. And some of them are formal and some of them are informal, just because people are talking. But I've rarely seen great integration in that capacity. I've rarely seen that kind of inter-functional communication and the translation required for them to understand each other to, to happen well and seamlessly and consistently. But it definitely helps. And this goes back to that thing we talked about at the very beginning is culture. If you don't evolve your growing organization with the right culture to take advantage of this, it's really, really hard to make that change down the road. Even, and you may have to, but you're really going to have to consciously set some things aside

• 39:28Chris Benson?

  that maybe were long time valued processes.

• 39:31Daniel Whitenack?

  And I think this is something, this is like a growth area for me. I think that I've seen recently is oftentimes when you're building a team, if you're the one building the team, it's most natural for sort of all communication and sort of project communication and all of that to sort of flow through you as the management of the data team. And at a certain point, so I think that's probably good in the beginning because you're setting some standards, you're setting workflows, all of this stuff. And you sort of need to guard that a little bit, I think. That's probably reasonable. But as the team grows and as the number of project grows, at some point you become the bottleneck, right?

• 40:16Chris Benson?

  You do.

• 40:17Daniel Whitenack?

  If, you know, all the communication from all of these different projects and the different organizational units are flowing through you, then pretty quickly the queue builds up and you know, stuff is falling through the cracks. So I feel that right now in terms of the teams that I'm helping build, there needs to be this transition to more sort of embedded communication where the people working on different projects are, are feeling the freedom to have.

• 40:51Daniel Whitenack?

  Those communication while still, you know, trying to maintain standards and all of that, of course, and make sure that products are done well, there still needs to be data management, but I think things need to get more decentralized over time.

• 41:06Chris Benson?

  It does. And that data management that you're talking about has to become human management. It has to be recognizing your individuals for what they are and what their capabilities are and understanding that they're all different and developing a good understanding of what each of those individuals can and can do well on a, along a spectrum. And then being able to, to give them those responsibilities where they both can be successful, but also have room to grow. And that is really hard to do. It's super easy for me to say that and it's super hard to execute that. That's what you have to do if you're going to grow past that, those early stages of putting it all together.

• 41:44Daniel Whitenack?

  I was really inspired by this article, just in terms of like, understanding where I've been at at the past and how, how I've been able to grow into certain data teams. But also I think it's, it's a really great way to frame some of this up in a creative way. So thank you, Eric, for writing this and putting in the time to, to do it. You're welcome on the show anytime to talk through other things along with this. So if you're out there, feel free to join us sometime. In these fully connected episodes, we usually also try to give a couple of learning resources related to the topic that we're, that we're talking about. This week I wanted to mention a book from one of our past guests, Mike

• 42:26Daniel Whitenack?

  Bugginbe, Cracking the Data Code, which is a great, a great book where he talks a lot about data culture and creating a data-driven culture in your business. And then also there's a, there's a book by also one of our previous guests, Hilary Mason and DJ Patel called Data Driven: Creating a Data Culture. It's more like a booklet. I think you can get it in some cases for free, like on your Kindle or something like that. It's a good, good little read. So maybe, maybe check those out.

• 42:55Chris Benson?

  There's great choices right there.

• 42:56Daniel Whitenack?

  Yeah. Yeah. There's definitely resources out there and you know, Although you and I have had experiences, I definitely respect the opinions of people like Eric and Hillary and DJ and Mike Bucanve and others who really have been able to scale things like that up. Because yeah, like you said, the human problem is maybe the main thing that you're dealing with as you're building a data team is not so much your ability as a data team to do things, but how you relate to other teams in your organization, how you can be gracious and clear and tenacious and creative and all those things kind of

• 43:35Daniel Whitenack?

  together and not burn a bunch of bridges and end up in a bad situation.

• 43:42Chris Benson?

  Yeah, that's really important. And you really have to respect all of those people, including the diversity of their differences, because they did not have your experiences. They did not grow up thinking that data was the thing they were going to spend their time on. You have to position the value in a way that they understand and that they can also value. And it's crossing that chasm that is the key to success there.

• 44:06Daniel Whitenack?

  Well, thanks for having this chat. I enjoyed hearing your stories, Chris.

• 44:10Chris Benson?

  I enjoy hearing yours as well. We have good ones.

• 44:12Daniel Whitenack?

  All right. I'll see you next week.

• 44:14Chris Benson?

  Okay, take care.

• 44:18Speaker 02

  Thank you for listening to Practical AI. We have a bundle of awesome podcasts for you at changelog.com, including our brand new show, Ship It with Gerhard Lazu, a podcast about getting your best ideas into the world and seeing what happens. It's about the code, the ops, the infra, and the people that make it happen. Yes, we focus on the people because everything else is an implementation detail. Subscribe now at changelog.com/shipit or simply search for Ship It in your favorite podcast app. You'll find it. Of course, the Galaxy Brain move is to subscribe to our master feed. It's all changelog podcasts, including Practical AI and Ship It in one place. Search changelog master feed or head to changelog.com/master and subscribe today. Practical AI is hosted by Daniel Whitenack

• 45:03Speaker 02

  and Chris Benson with music by Breakmaster Cylinder. We're brought to you by Fastly, OnStarfleet, and Linote. That's all for now. We'll talk to you again next week.

• 45:39Speaker 02

  Hey, guys.