DJ Patil - Data Science pioneer on AI, healthcare, and defense

DJ Patil served as the first U. S. Chief Data Scientist under President Obama. He's currently a GP at GreatPoint Ventures, on the Board of Devoted Health, and co-chairs the taskforce on GenAi for the Pentagon. In this episode of World of DaaS, DJ and Auren discuss: AI's impact on data scienceHealthcare data transformationInnovation, bureaucracy and efficiency in National SecurityThe best higher education choices in terms of ROI Looking for more tech, data and venture capital intel? Head...

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Auren Hoffman?

  Hello, data nerds. Welcome to World of DAS. I'm your host, Orin Hoffman, CEO of Safegraph and GP of Flex Capital. Discover more episodes, get weekly data as a service news at worldofdas.com. That's spelled world of D-A-A-S dot com. Hello, fellow data nerds. My guest today is DJ Patel. DJ was the first US Chief Data Scientist under President Obama. He's currently a General Partner at Great Point Ventures. He's on the Board of Devoted Health. He co-chairs the Task Force on Gen AI at the Pentagon. DJ, welcome to World of DAS.

• 0:39Speaker 01

  Awesome. Finally, I get to be here and hang out with you.

• 0:42Auren Hoffman?

  I am very excited. We've had so many conversations over the years, so now we have one that is taped for eternity. So anyone, any AI can be trained on this going forward. Now, you actually coined the term "data scientist," and now we've got an interesting juxtaposition in the data science world where AI is transforming that field. What percentage of today's data science work do you think is going to be automated over the next five years?

• 1:09Speaker 01

  We were trying to figure out what we should call ourselves when we do this weird thing with data. We didn't have a labeling approach. The interesting thing that I have constantly surprised by is why is the term taken off? I think it's because it's kind of a catch-all phrase for people who play with data, and that it gives you a lot of space to go do things. If you were the BI person, you're not allowed to go build models.

• 1:34Auren Hoffman?

  My major in college was Operations Research, which today would be called Data Science, but Operations Research is just a terrible name for a major.

• 1:42Speaker 01

  And the statistics was too narrow, doesn't allow the physicists to be in there or the mathematicians, and so it works better. And so I think because it's like a term that has always been a big term to allow people in there, like the data engineers used to be data scientists also. They weren't the data engineers. What we're now seeing is a question of, well, what's this next phase of data look like in terms of roles, responsibilities? What do you do with these different jobs? I would expect there's increased specialization. And at the same time, we're starting to see this new disruption that's starting to take place because of the use of these technologies. We're starting to see the AI-ification in there.

• 2:25Speaker 01

  But there's also a ton more that's the old school is coming back. The ability to design tables and structure data more carefully, more thoughtfully with an eye towards a problem. That's a thing. Making sure you kind of have good governance. The data steward idea is coming back.

• 2:41Auren Hoffman?

  So much of the traditional data science job is being attacked by AI. Seems like you could do so much of it today, way more than most other jobs are being attacked. But maybe I'm wrong.

• 2:53Speaker 01

  Parts of it, yes. Parts of it, we're just not seeing it happen. At least from where I sit, we haven't seen it happen. The biggest limiter for AI that we see in the enterprise right now is that still people don't have their data in the good place where they can do something with the data.

• 3:10Auren Hoffman?

  So it's still that data-munching part of it has to happen.

• 3:13Speaker 01

  Data cleaning is 80% of a data scientist's job. I feel like it's kind of gone up to like 87%. Sometimes 90-plus percent. You've got more data in, these heterogeneous data types that start showing up. So stuff gets really messy and then you're like, how do I do something with this? And so you're spending all this time munching that data. You've been living this your whole career, doing the exact same thing.

• 3:37Auren Hoffman?

  Oh my God, I've been like only data-munching. That's all I do. It's a terrible thing to do, but the fun stuff is all the other stuff you get to do after you've munched it.

• 3:45Speaker 01

  Totally. And you've seen this a ton also. Like you interview people and they just don't understand how important that is for having context of doing the job. And then you also see like what you've done with everything from rap leaf on and also you're taking advantage of the fact that others are either lazy, incompetent, or it's just technically hard to do. You kind of go across those three dimensions and you get a lot of defensibility.

• 4:11Auren Hoffman?

  AI systems, we've been using them quite well to actually do a lot of the data bunging. It takes a lot of the work out of it. There's a lot of things where sometimes you need like a low price labor somewhere and AI can really do most of that. So maybe it could take on 80% of the low price labor you were doing before. So AI is attacking a lot of this data cleaning, too.

• 4:34Speaker 01

  The stuff that you might be sending to a back office that's typically gone out to mechanical turf or offshore to like organizations in India or the Philippines. This often happens in healthcare. You have these BPO shops, these business process shops that do this. A lot of that data is really ripe for it because it's not real good data cleaning. It's well, let's change this from comma delimited to semicolon or put it in the right structure.

• 4:59Auren Hoffman?

  How do you think AI is going to attack different economies? From my standpoint, I don't know that it hurts the China economy because it's so manufacturing based or the Japan economy is so manufacturing based. But if you think of the India economy or the Philippines economy, there's so much of that is just back office to the US. It seems like those economies are going to get hurt very, very, very quickly through AI. I don't know if you agree with me or not.

• 5:20Speaker 01

  We had the opportunity to write the first national strategy on AI under President Obama, and that included autonomy and a whole bunch of other things. What's interesting is what we got right and what we got wrong. In there, what we got wrong is that we thought there was going to be a lot of job displacement when the classic self-driving cars. We realized dexterity is hard. Because of transformers and all that other technology, you get the ability to code. So the companies like Infosys and others where that coding layer in India typically, where that coding layer is not high quality. It's just laying down tons of lines of code. That stuff, I think, is starting to get impacted. I expect it over the next five-ish years to really get impacted.

• 6:03Speaker 01

  A lot of that stuff in India and the Philippines and the back office, a lot of that's healthcare based. Robotic process automation was already starting to make inroads on that. A lot of places, because of the way healthcare works, is that you have these artificial barriers that allow you to do something with the data. So as AI comes in, my belief is that that is going to be where the health systems get margin from. And so they're looking at that and saying, hey, how do I take out that margin? And that margin is directly attributed to dollars that are outside the United States. And so that impacts the Philippines and India predominantly. How big of that of a market is it? I just haven't checked or stayed up on top of that.

• 6:42Auren Hoffman?

  It seems like it's going to impact different groups in different ways, but I would not be long India, Philippines right now unless they make their own pivots or they invest more on go more upstream, which they may be doing.

• 6:55Speaker 01

  Well, we are starting to see more of those people in those BPO shops saying, hey, why am I not building a company that's just doing this? They have deep, deep expertise.

• 7:06Auren Hoffman?

  That's right, in the process. So maybe the company does extremely well, but instead of having a thousand employees, they have 200 or something.

• 7:14Speaker 01

  What we're seeing, which is stunning, is how many companies out of Israel, out of India, who are healthcare focused, focusing only on US healthcare. And then when I say healthcare, I mean the back office, the claims engine, all these components. And so they understand healthcare better than many of us in the United States.

• 7:34Auren Hoffman?

  That's a really good point. My experience is that the average software engineer at a top company is super high quality, but the average data scientist at that same company is often fairly mediocre. Would you disagree with that?

• 7:48Speaker 01

  I think it's super lumpy all over the place. You can go to some places and you find data scientists who punch way above their weights and you find others where you're just like, it's very frankly, weak sauce. I think there's a few things that are going on. The first is that the places where you see really good data scientists doing stuff, they're not going to hang out if they're not valued. And most of the time, even though you have good data engineering, it doesn't mean you value the data scientist.

• 8:14Auren Hoffman?

  Good point. So at Google, you're just going to value the software engineer more than the data scientist. A great data scientist might leave and you're just left with the good data scientist.

• 8:23Speaker 01

  That's right. And so what I see a lot of times is the data scientists are in sufficient demand and they go to good places. Let's split the data science camp into two, which is traditional, just business intelligence analysis and those that are building models that would be typically machine learning algorithms all the way into the AI layer. And so if you get to the business intelligence layer component, those people a lot of times pick up other jobs. You see a number of data scientists who are really good. Now, people forget Kevin Weal, who's head of product for OpenAI, started off as a data scientist at Cooliris, then went to Twitter as a data scientist and then moved up. I would argue the similar thing happened to me is on the technical side is data science, playing with

• 9:08Speaker 01

  data, doing stuff, and eventually moved into product roles. And so I think the data scientists often have the ability to take on other roles versus engineering is like you're in the engineering box, you're technical there.

• 9:20Auren Hoffman?

  And it's a good point. If a company doesn't value engineering, the great engineers are probably going somewhere else anyway.

• 9:26Speaker 01

  That's right. Here's another thing that we see, and Orin, I think you fit squarely in this bucket, is a lot of the people we see who are data scientists are just multi-dimensional by the nature. I think that's why we get attracted to this. Could you have been a Silicon Valley guy? Could you be a policy guy? Could you be another dozen type of roles? You're equally adept at all of them. The physicists that we saw coming into Wall Street early on, playing with experimental data, they were data scientists too, whatever we were going to call them.

• 9:55Auren Hoffman?

  That was the main data science role was that physicist.

• 9:58Speaker 01

  Exactly. You're a quant. And so what I think we have now that we see across data science, especially places like Berkeley, where we have an incredible data science major program, is those students are multi-dimensional. If you want to do economics and you want to play a ton with data, where's your home? If you want to do anthropology plus data, there's no direct home or social sciences. There's this kind of new horizontal across the verticals of traditional academia and data science ends up being that layer.

• 10:32Auren Hoffman?

  Often, the person who's most successful in those fields, the Raj Chetty's of the world, they're also data scientists as well. They happen to be an economist. They really have that extra data science skill that's there.

• 10:45Speaker 01

  I think Raj Chetty is a perfect example of what do you call him?

• 10:48Auren Hoffman?

  He would call himself an economist, but he's clearly also a data scientist. He would not be successful if he didn't have that skill as well.

• 10:56Speaker 01

  For the last 10 years, departments like economics have not given the course load or kept up with the world as data gets messier. And so what you see from a lot of those students is they need the skills and techniques to work with messy data. So where are they getting that? They're getting it out of the data science shops, out of the AI parts of the computer science departments, other places. That's the shift I think we're also seeing now with full-blown AI. Where are you going to get that training? Where is that happening? It's oftentimes not in your home department. There's a little, small cadre of people who've really got that. We haven't figured out how to get that to scale yet.

• 11:36Auren Hoffman?

  Back in the day, when I used to interview people for like data science-related roles, I would care a lot about their proficiency in things like probability and statistics. Does that even matter anymore?

• 11:49Speaker 01

  Yes, I'm testing for a few things. My default question back at LinkedIn was, here, we gave you all of LinkedIn data. What do you want to know? What questions would you go?

• 11:57Auren Hoffman?

  So that's more creativity, too.

• 11:59Speaker 01

  Creativity, curiosity. So I'm first looking for that. I'm looking for the long pole because there's a lot of people with the technical skills. I want that early decision point rather than the technical capacity first. Second, then, what I want to see is that you're not just doing a cookbook approach to solve problems. You're thinking about the different parts of the messiness of data, the other issues. The classic used to be you give them a bunch of data and they're like, well, I'm just going to do K-means. And then they're like, okay, then I'm going to do SVD. And they're just kind of going through this recipe book of things. And you're like, did you ever ask where did this data come from? Did you ask some of the other characteristics? And then the probabilistic and the very technical is, do you have a real touch and

• 12:43Speaker 01

  feel for the nuances? Of data because you can understand probabilities. But there's a difference in being able to appreciate the probabilities may be lying to you or confusing or there may be other intrinsic things. The depth of understanding of probability or statistics is really important for me.

• 13:05Auren Hoffman?

  What are we like struggling most with data science today?

• 13:09Speaker 01

  We're struggling with a few things. One is, and I think everyone is going to struggle with this, the world's changing. The technology landscape is changing so incredibly fast. What should you do? What should you study? Where should you specialize in? I get asked all the time, I'm sure like you do, is what should I major in? What are those things? And I tell people, well, I would major in something where you're also deeply passionate in several other areas. You can't just be only computer science or only data science. I think you've got to be like an E. You've got to be deep in three areas and then horizontal. If you're going to be interacting in the physical world, mechanical engineering or robotics or something and data, that's great. That's like a super dynamic place to be. That's awesome, but you've got to have that.

• 13:53Auren Hoffman?

  One thing I'm surprised about is just the value of data, raw data. I would have thought that with just the rise of data science, the rise of AI, that just selling data would be a better business today than it is. The market for raw data is not growing. It's growing a little bit, but it's not nearly growing as exponentially as I would have predicted five, 10 years ago. Why is that the case?

• 14:20Speaker 01

  The first I would point people to is I think you had a really good thread about walking through the Rapleaf acquisition all the way through what that looked like and the value creation and how companies think about. That's a really important take that I think people don't appreciate. Two is, I think when we see the value of the data ecosystem, there's a few things going on. One is, there's a set of data that I think remains very polarized, and I certainly stall in this camp around certain types of data brokers and the information that has been captured, which really aligns with the FTC report that was published almost a decade ago, and Congress has yet to act on it.

• 15:02Speaker 01

  The other form that I think is there is, people have yet to see the unlock of data. One of the versions of the big data collection components that's in there is biobanks. And so, we have all this data that's been collected around people, but we haven't yet gotten the pharmaceuticals out of it. So, people are asking, hey, what's the value there? In the collection of data, the type that Safegraph and you've been working on, one of the things that's always been surprising to me is, we saw incredible value of what could be done with this data during COVID response on this front. And we said, look, look at all this is being done, but people haven't been able to appreciate it. I wonder a lot of times, and I think healthcare is a little bit more of the way to look at this, which is, it's not just that it's you're selling the data.

• 15:46Speaker 01

  You're selling a very crisp product. And so, you don't realize that the secret sauce really is data. So, it kind of looks a little bit more like LinkedIn, maybe?

• 15:55Auren Hoffman?

  You're selling the application has been a lot easier than selling the data itself.

• 16:00Speaker 01

  That's right. And to be fair also, there's a lot of data acquisition that you can do. You can buy lots of data, and it does not move the needle. It's not that good. I had to do this for Devoted Health in the early days to try to build out our mailing list, physical mailing list to people and figure out who might be eligible in our markets that could benefit from our service. We did a ton of this, looking through the data that we were buying from vendors, and we were like, this is not good at all.

• 16:28Auren Hoffman?

  Well, marketing data historically is not good. In marketing data, if you're like, a bit better than throwing darts, it has some value. So, even gender is like 75% and maybe 80% correct in most marketing data.

• 16:42Speaker 01

  We thought we were clever because we were like, well, we know this isn't that great. What happens if we bring this data together with other data sets? Could we do something more?

• 16:51Auren Hoffman?

  That's the problem. Yeah. Just compile the errors. It's like point eight times point eight. It's not the other way.

• 16:58Speaker 01

  Exactly. It turned out not to be as a quality that we wanted.

• 17:02Auren Hoffman?

  You are also involved very much in the DOD world. What are some non-obvious issues the Department of Defense is grappling with when it comes to AI?

• 17:11Speaker 01

  So there's a whole bunch of things that I think are worth thinking about. So the first is, how do you use these technologies? Where do you use them? People's heads jump off into just like autonomous weapon systems, or the nuclear security footprint is going to go to AI. The way to think about this is, think of it as a hand-wavy impact slash scale. And then think of the X-axis as consequences. So if you're high on the consequences side, that's where your weapons world are. In the medical clinical world, that's like there. Surgery, diagnosis, that's high consequences. High impact, low consequences is like travel systems, paperwork being filled

• 17:52Speaker 01

  out, things like that. Logistics, moving batteries, fuel. Perfect. Amazing opportunities to do that. You have a lot of people, over 3 million people who work at the DOD. A lot of them are doing jobs that they don't want to do because you got some enlisted person who's just recopying a form. It's like a bank. So that's the place where it is. You also think about the data sets that the DOD has overall. And what could be unleashed if you bring all that data together, both for the national security apparatus, for research and development purposes, healthcare? There's incredible opportunity. One of the ones that I think AI is our fastest path to figuring out how to get synthetic blood

• 18:36Speaker 01

  developed, which would be amazing, not only in the battlefield, but people have it in the emergency vehicle, those types of situations. Synthetic blood could be a game changer for the country. Tissue regeneration, these type of things, these technologies to get advances in that are material. Sciences on a path to fusion, that is going to be AI driven. That to me is some of the most exciting opportunities. And so the question is, how to do that? When people talk about using AI for planning systems or everything, this is the same thing that we've seen in data. The reason why you don't see data used in some organizations or in basketball teams or a lot of other places is it's cultural. There's a very big cultural element to this. And if you can't get a notion of trust.

• 19:20Speaker 01

  So when people talk about trusted systems, a lot of times they're thinking transparent, fair, fat, these, all these kinds of things. And what I would encourage people to think about is how do you develop trust with these systems? Trust includes all of those things, but you have to put this in a commander's hand or somebody who's a decision maker and say, here's the output. Do you believe it? Are you willing to take action based on this? That is a different component of how we build products, rather than just the technical ability to build the product.

• 19:53Auren Hoffman?

  The Defense Department writ large is one of the most bureaucratic institutions in the world. And so on one hand, the great AI can attack the bureaucracy and make things way more efficient, make things move faster, make decisions get made quicker, et cetera, because it can flatten things out. On the other hand, the hardest thing to attack sometimes is bureaucracy because the bureaucracy will stifle it. They are smart. They don't want to lose their own jobs. They'll make things really slow. Everyone I know in the Defense Department who's tried to implement whatever it is, some transformer, some other type of large language models. There has been STEMI that's had 20,000 pieces of red tape thrown at them and not being able to do something.

• 20:39Auren Hoffman?

  How do we actually make progress in Defense Department? Or are we just 30 years from now, we're still going to have a first-rate bureaucracy and a third-rate fighting force?

• 20:50Speaker 01

  My belief is the fighting force that is AI native is the one that wins. You have to be AI native to win.

• 21:00Auren Hoffman?

  Everyone you talk to in the Defense Department believes that.

• 21:03Speaker 01

  Not five years ago.

• 21:05Auren Hoffman?

  Today, I would say 100% of people that I have ever talked to believe that. And they all want it to happen. They just all think it won't happen because of the bureaucracy.

• 21:14Speaker 01

  The Air Force adamantly used to refuse the idea that there could be autonomous drones in use. That was an active stance. The Navy was there. I use that as an intentional example that we can create a thinking mind shift. So we've created the thinking mind shift. Now, how do we get the actual work to take place? And that's the other thing. So for example, one of the programs that we had to run right over, thanks to Ash Carter, was the Hack the Pentagon program, which is allowing the Pentagon to use bug bounty programs. And everyone said, you're going to tell China and all these other countries exactly where the bugs are and where the holes in the security. And you're just like, well, that means they already know where the holes are. Only we're finding out about it.

• 21:57Speaker 01

  Once we got that in, that became the de facto way of doing things. So you have to show people the way to do it. What we do need is leadership, right from Secretary of Defense, all the way through to say, this is a way we have to reshape this. Same thing happened when the Navy decided to move to nuclear ships. The other aspect that's not talked about and usually attributed to bureaucracy is Congress has to get on board. DOD says, we don't need a new aircraft carrier. We need AI. And Congress says, well, my district produces this part. You get an aircraft carrier. I remember working under President Bush on these problems after 9/11. Rumsfeld was back then saying, we got to transform this stuff to be more agile,

• 22:43Speaker 01

  to figure these things out. And yet, we're still stuck and mired in this.

• 22:47Auren Hoffman?

  So what do we do about it? I mean, everyone knows, you're right, that's Congress's problem. We're spending all this money on aircraft carriers and on tanks that are just going to get easily defeated in the next war. We all know all these problems. We've got these fighter jets that cost a billion dollars each that are just not going to be effective. What do we do about it? Congress is in the way, the bureaucracy is in the way. Is there anything we can do? Or do we just have to accept defeat?

• 23:10Speaker 01

  There's zero chance we can accept defeat. First off, I think what I would tell everyone is it's very easy to be cynical about the DOD. It's very easy to be cynical about the government. Our job is to fight cynicism. We have to fight the cynicism because too much is at stake. The world is not a safe place. When we look around, the attacks, the way people want to approach things, that's happening. Other countries want not only our technology, but they want to hit us with the technology so that we're hurt by it. So what are concrete things we can do? So the first is, and I think this is the first tiniest of a step, is the Biden National Security Council issued an executive order just a couple of days ago that says, go faster with AI.

• 23:53Speaker 01

  National security teams, everyone else, you need to go faster. Two is you need programs like the Defense Innovation Unit, things like Doug Beck is leading, to get funded. Constituents need to support that. The executive office has to support that, to say, we have to have DIU, Defense Innovation Unit, to scale at literally 10x. We need that much more to happen because you need new ideas to come in. Because a lot of the traditional Beltway bandits, they're doing what they're supposed to do, which is not helpful. And then the third thing that is there that we often forget about is we need the right talent to show up. Those of us that have the skills have to go in and take the roles.

• 24:36Speaker 01

  And not just the roles at the coding layer. We need people who are running major components, who are the AI officers, the Craig Martells, who used to be at Lyft and at one of the shared sharing companies. He was also at LinkedIn and then went in to be the chief AI officer for the DOD and got a bunch of stuff off the ground. You need a deputy secretary who understands this. And you need secretaries of defenses who think about this the right way and are going to drive it down from the top. And you need that bottoms up. There's no easy solution when this is 20% of G. It's kind of like the same thing with people like, why can't we just fix healthcare? It's so big, you got to attack from all the sides.

• 25:15Auren Hoffman?

  Even if we do all those things, I feel like they all help on the margins and stuff. I'm glad we did DIUX and stuff like, I feel like it's helped on the margins and it's good. But the primes don't care. They're still as fat as ever. All the services companies, Booz Allen, Khaki, all those guys that are out there, they don't care. And then the DOD itself, they just have this massive bureaucracy. It's six layers of paperwork to go do anything. It's so hard to like actually do anything. And I feel like they're all like good ideas. They're all things we should do for sure, but it doesn't attack the main problem. And I don't see like the main problem getting any better. The main problem still gets worse. All those things help in the margin, but if the main problem gets worse, it doesn't help.

• 25:59Speaker 01

  Part of this is you're the, one of the best conveners on this, of people, bringing people together. It's people like you and others that have to convene people.

• 26:09Auren Hoffman?

  I'll convene top generals in DOD and all they do is they all know the problem, but they're all like, my hands are tied. Congress has tied it, or I can't do anything. And these are literally like three and four star people. You think people have power, but they're like, I have no power. I can't do anything. I'm a three star, I'm a four star, but I literally have no power to go change really anything beyond this little thing that I'm working on. And then there I can try to do it and I can try to grow people and mentor people and all the other things.

• 26:36Speaker 01

  There's early indications of positivity, like what we're seeing out of the war colleges with people who are just coming out and they get this. We see that new leadership there. You're absolutely right that there is this paralysis feeling across a lot of people. I look at what CQ Brown is doing, what General Smith is doing, coming down of the Marines who worked with us a ton on a bunch of these things with Ash Carter. They get it. The question that is at hand is how do we create a collective movement for what that is? And that really has to be thought of as major work by Congress. There just needs to be a new study. What do we want the Pentagon to be?

• 27:18Speaker 01

  The problem there is that everyone is going to lobby the heck out of it. And we have to just hold true to that because part of this, what I hate to see happen is the way it was said to me, which I think is the worst example of this during the Gulf wars was the country's at war, but the Pentagon isn't. And I think that is a version that we still see today. And the version that I hate to see happen, which is kind of said is we have to see an existential crisis moment for us to get our act together.

• 27:50Auren Hoffman?

  If we have an existential crisis, I believe we'll figure it out. But ideally, we should be able to figure out without that.

• 27:57Speaker 01

  We should wait. We have multiple existential crisis already.

• 28:01Auren Hoffman?

  Correct. They're just slow moving crises.

• 28:03Speaker 01

  I'm an unpaid volunteer to work on these problems at the Pentagon. I would tell everyone, I'm doing my part to the max ability that I am legally allowed to by the number of days I'm allowed to spend working on this under congressional rules, laws, I am maxing out all the time I can.

• 28:22Auren Hoffman?

  That's another crazy thing. I have friends like you who are volunteering in national security. They want to even help more, but they're not allowed to, which makes no sense. So like, I don't even want to get paid. Let me help. They're like, no, sorry, we're capping you at the number and you have to like log your hours. It's crazy.

• 28:38Speaker 01

  If you look at my paperwork, it's amazing.

• 28:40Auren Hoffman?

  It's so silly. It's like, oh, DJ Patel, like the guy who invented the word data science wants to help out more and this guy is crazy expensive and he's willing to help you for free. Oh, no, no, sorry, we can't let him help. It's bureaucracy.

• 28:55Speaker 01

  This gets into the weeds of something, but there is a real problem we have. You and I lived this during COVID, no one cared about the rules.

• 29:02Auren Hoffman?

  COVID was awesome. Oh, COVID itself was terrible, but March through November, let's just say of that just like intense six months, everyone was working on it. People were like, we're going to break the rules. We're going to, who cares about this transactional authority?

• 29:17Speaker 01

  We may not be here tomorrow. Let's just go.

• 29:20Auren Hoffman?

  Exactly. Let's move fast and things got done.

• 29:24Speaker 01

  That's right. And I think one of the key lessons there. People like you, all the people that were written in Michael Lewis's books and all the other stories, all of us, we had deep operating experience and so we were allowed to come in. Normally, if there's another problem, we're not allowed to come in because of ethics rules or other things. And there's important reasons for the ethics rules, but we need to rethink them. For example, one of the challenges that the Biden team really has faced is they don't have great operators. And why don't they have great operators is fundamentally because the ethics rules prevent operators from being in those jobs because of stock ownership. But if you help build a startup, but there's no liquidity, there's

• 30:07Speaker 01

  no public market, you can't get free of it. Or maybe your spouse works in an area or and your spouse is doing an incredible amount as a civil servant and you've lived this too. We make it really tough.

• 30:21Auren Hoffman?

  It's so silly. Imagine if you're like a billionaire and you own $1,000 worth of General Motors stock. You probably don't even know it's in your portfolio. Certainly it's not going to affect your decisions. Now, I can understand if you own like a billion dollars of General Motors stock and your net worth is 1.1 billion, you should be recruiting yourself for anything dealing with the automotive industry. That makes sense. But these rules are just so archaic.

• 30:47Speaker 01

  My soapbox version is, I think there's lots of places that I would call theaters. You go through the airport, there's a security theater. There's a bunch of good stuff, but there's a lot of stuff that you're like, this isn't real security. There's a bunch around these things around ethics, theater, and it's not really addressing the real problems. There's a bunch of this other stuff. You pick your versions of this. And what I think we would do far better about is just asking, how do we actually solve real problems? As we've learned at Big Data People, taking a probabilistic approach on things is a really healthy constructive way to actually get maximal forward progress on things.

• 31:27Auren Hoffman?

  Did you read Unit X, the Chris Kirchhoff, Raj Shah book?

• 31:31Speaker 01

  I have it right here. Ash Carter is the one that made me figure out how we were going to reboot it. And I was very, very grateful that Raj and Chris decided to jump in the role and do it. And so I owe them a debt of gratitude for taking on that role and driving it.

• 31:50Auren Hoffman?

  Those guys are great guys, great patriots. I love the book, but the book is so frustrating. It's like all the red tape, all the ethics things that they threw at. There's so many rules about it, all this ethics stuff that they threw people. It's just like a whole series of terrible anecdotes about how both the Congress and the Defense Department bureaucracy gets in the muck of any type of even like minor innovation.

• 32:16Speaker 01

  I will say this other one, that Ash Carter, Inside the Five-Sided Box, it shows the picture of how it's really brutal to do this, but also some of the tactics and techniques that Ash Carter personally used to make the progress that he did. Measured over 30 years, really substantial progress. The thing is, is we can't wait 30 years. We can barely wait three years.

• 32:42Auren Hoffman?

  Carter, he was a very smart infighter, very smart about moving things, very smart about getting stuff done. He had a lot of imagination and innovation that we just don't see from most people in that role.

• 32:55Speaker 01

  He was a physicist.

• 32:57Auren Hoffman?

  Somehow he also had the strength and the courage to do a lot of stuff. And there's a lot of blowback that could have happened. And there's a lot of people who try to investigate him and try to tarnish him. And there's a lot of primes you didn't like him. We need more people like that in there. No offense to Lloyd Austin, I've never met him, he seems like a very nice man, but it just doesn't seem like he's attacking it in the way of, okay, we're in a crisis, we have to do the right thing. He's just more managing the Pentagon. These are people like from the same party, you would assume there are some, but they're just very, very different operators.

• 33:32Speaker 01

  We could have this exact same parallel conversation about Health and Human Services, HHS. Why doesn't FDA collaborate more with NIH? Where's the CDC in this? And we saw abject failures from the CDC during COVID. And I think Mandy, who's the current director of CDC, is really working hard to fix some of those things. But it really pains me to think about the CDC not being the data place that we needed it to be. And we saw what the power of data could be. We saw what you were doing. We saw what COVID tracking project

• 34:09Auren Hoffman?

  And 30 years ago, the Atlanta piece of the CDC, that was amazing. That was the pinnacle. And then somehow it might have been the bureaucracy as well, but just slowly attacked it. They're just doing less and less. They still do some good work, but there's so many things that could be doing better.

• 34:26Speaker 01

  I'll give you an example. Rob Califf and I, who's a current FDA administrator and was FDA administrator when I was working with him and Francis Collins at NIH. We were really focused on asking the questions. Why does somebody have to go through phase one trials if something that is effectively the same thing already did? Why should we waste that money and go through all the human trials again? We're wasting a lot of time to get there.

• 34:54Auren Hoffman?

  We know it doesn't harm in specific ways. So now it's really just about, well, solve this particular problem.

• 35:01Speaker 01

  That's right. And then also, how do you start to get a better level of trust with devices, especially that are AI enabled? All these questions. So Rob is trying to get that restarted. A lot of that fell apart just because of COVID. I think it's a very bipartisan approach. Everyone is on board with this. The same time, the reason I got so involved in the creation of Devoted Health was we said, who's going to walk the talk of delivering healthcare? Who's going to figure out new models to do this? Who's actually going to do it? Our version was, well, we did it on the policy side. Now we need to do it on the entrepreneurial side. And the reason Devoted Health can do so much and is able to deliver care at the price we can't,

• 35:45Speaker 01

  given the margin structure of healthcare, is we are leveraging a ton of data. We are leveraging not just the data, but we have machine learning. We have AI technologies that are used to augment things, mostly the back office. And that's where we get leverage. And that's where we can keep teams smaller. We can keep things cheaper and we can have better outcomes because we're able to use all of that. But it starts honestly with a really good environment and data hygiene and quality with data as a first-class citizen.

• 36:18Auren Hoffman?

  Healthcare data is so siloed and it's in all these legacy systems and it has to be interoperable. But what else do we need to do to make that better?

• 36:27Speaker 01

  This was my first big aha with healthcare data in that service of healthcare, not the research side, not pharmaceutical, those, just the service. So think hospital data, working with your primary care physician, et cetera, insurance payers. That data is not big data. It is very sparse data. It's repetitive data. It's error prone. It's all over the place. The infrastructure that we have typically brought to this, which has been consumer internet technology, doesn't work really well. Do you need Kafka moving data around super fast? No, you need more airflow plus plus kind of technologies to do this.

• 37:08Auren Hoffman?

  And ideally it's somewhat persistent. Why do you have to answer the same question 15 times? I should just say like, hey, this is what you answered last time. You want to update it? You answered this about your mother last time and your grandmother.

• 37:21Speaker 01

  You know how hard it is to train someone on a fax machine these days who just graduated from college?

• 37:27Auren Hoffman?

  I have no idea how to use one anymore.

• 37:29Speaker 01

  They're like, what's a fax machine? I'm like, you're not supposed to know. It's like trying to teach somebody like what an old school typewriter is or an eight-track or something.

• 37:37Auren Hoffman?

  Oh my God, yeah. Faxing the prescription. I mean, it's just silly.

• 37:42Speaker 01

  It's ridiculous. And so what we have to do is, and this is where it's so important on policy for these things, is you need technologists in the room. Policy people are saying, we need healthcare interoperability. Your data should go from one provider to another. It can't just be locked up. But there's nobody in there who's like, what's the exact API? How do we think about this? Or these other things? This is actually why Mickey Tripathy, who is the national coordinator on this, for the first time ever, the national coordinator is not a physician. He's a data guy. He's a technologist. Well, who else would you want? And I can't tell you how much of a fight this was. This is one of my big throwdowns during the transition,

• 38:24Speaker 01

  which I'm now somewhat allowed to talk about, is saying, this is the kind of person you need. You don't just want a physician. This is a technical problem. There's enough technologists who understand medicine who can help you here.

• 38:39Auren Hoffman?

  I'm with you. Sometimes we take this air of expertise that someone's supposed to have. They're a good heart surgeon or something, which is a very, very important job. And then we assume, well, then they could fix the medical system. You're a great heart surgeon. That's what you do. You may not know anything more than anyone else about fixing the IT systems that go through. It doesn't transfer from one to another.

• 39:02Speaker 01

  Well, let me address the other side of this, which is on the side of health records and using it to discover new drugs, to find new treatment policies. This is what I was really tasked with under the Precision Medicine Initiative, was until we were really working on this during the Obama administration, hospital systems thought it was their data, not your data. That had to be a major change. In there, you have the health system who's saying, well, sorry, it's the EHR company's fault. And the EHR is saying, no, no, it's the hospital's fault. And they're just pointing at each other and you're like, well, the patient is suffering. What can we do here? Even though we have interoperability today, we have still made it too difficult

• 39:43Speaker 01

  for large data sets to be brought together for clinical research use. And then when we do bring them together, everyone finds a reason why somebody shouldn't have access to the data.

• 39:55Auren Hoffman?

  And today, if you use things like homomorphic encryption, you can join data sets and be able to query, ask questions of the data sets without actually seeing the underlying data. You can have your cake and eat it too. You can get rid of the whole privacy issue and be able to get all the benefits of being able to join these really valuable data sets.

• 40:14Speaker 01

  You talk to most people who have rare diseases, other issues, they're posting all their data online anyway.

• 40:20Auren Hoffman?

  Correct. Yeah. Because what do you care about your privacy if you're going to die in the next year? You want to get fixed.

• 40:26Speaker 01

  And people think that this is just a data. Perspective. People forget how clinical trials got revamped by a bunch of incredibly courageous women who had stage three, stage four breast cancer, drove a car into the middle of the campus courtyard at Genentech, handcuffed them to themselves to the car until Genentech said, "Okay, we're going to listen to your demands about giving them access to treatments." They're not going to live until that moment. They know they're going to die. Let's give them a chance now.

• 40:59Auren Hoffman?

  There is a feeling amongst many people I know in the health data world, health tech world, that Epic Systems is the evil empire. How true is that?

• 41:09Speaker 01

  Here's a challenge. Epic is pervasive. It's like in all these systems. You talk to a lot of physicians, there's a pocket of them, they're like, "This is amazing. Look at all the things I can do." There's another pocket, and I fall into this, and I think a lot of other people say, "My doctor is spending more time typing than talking to me." And because of that, you need a scribe, and you need all this other systems. And you're like, "Well, who's practicing medicine?" And so, we, on the data side, have really done, in many ways, a disservice, because we said, "We could collect all this data. We could do all this stuff with electronic medical records." And yet, we have not yielded that result back to the public.

• 41:51Auren Hoffman?

  Even when the data's in Epic, they don't really have easy APIs. They're like, "Oh, sorry. It's here. It's ours." It's like, "Hey, we need a court order to get your teeth out or something."

• 42:01Speaker 01

  It's not just Epic. It's the whole system. I worked with the Trump administration closely to work on a rule that says hospitals must post their data about pricing online.

• 42:12Auren Hoffman?

  I love that rule. Why did it take till the Trump administration? I mean, it's so obvious. And people have been talking about that for 30 years before that. The Clinton administration talked about it, then the Bush administration, then the Obama administration, finally the Trump administration, finally. I'm like, "Why did this take so long? Like, it's so obvious. We should have done that."

• 42:30Speaker 01

  I can tell you concretely because every step of the way, we eroded their ability to fight back. It took us first to get people to believe it was their own data. It took Castlight becoming a company to try this out. And then when this even happened, when the Trump rule went into effect, the Trump administration's rule went into effect on the healthcare data, those hospital systems put up the data in file formats that you can't even read it.

• 42:56Auren Hoffman?

  They did everything possible to not comply.

• 42:59Speaker 01

  That's right. It's like passive aggressive to the point of absurdity until like you have to start enforcing it. So this is something that most people forget about data, technology and making change happen is you have to have an enforcement side. Somebody has to be there to either shame people or other things. I think the data community collectively came along and shamed the hospital systems into saying this. The big problem still to this day, and one of the things I didn't know about this, I don't think you probably knew about this, but when we were building our companies and doing stuff in Silicon Valley, the government was saying, we are going to make a move to electronic health records. We need companies to show up.

• 43:44Speaker 01

  And guess what? None of us showed up. No one in Silicon Valley showed up. Who showed up? Epic's like, well, we're the billing layer. I guess we'll do that. We're like grumpy with them. We're like, well, why doesn't have Silicon Valley tech behind it? Well, we didn't show up. We didn't compete.

• 44:00Auren Hoffman?

  I don't know the story, but my guess is these things are rigged from the beginning to make it hard to compete. You have to have these 572 requirements, most of which are not needed. The government is very good at specifically requesting software. They're really good at requesting features that will A, make things be at least two or three times more expensive and B, those features will actually degrade things two to three times as well. They request things all the time that add costs and that make the product worse. Why would you want to build a bad product if you're a great tech company? Like, I don't want to build a piece of crap. I want to build something beautiful and something that works.

• 44:38Speaker 01

  There's another problem, which is a startup is like crossing this desert. The desert's short, the desert's long, until you get your first customer. Government makes it really long, and then longer, and then longer, and then longer. And you're like, there's no oasis. And that's why we started the Defense Innovation Unit. And I can't tell you the number of times I jumped in to kind of say like, these requirements make no sense or are unnecessary. The problem also in this is, to be fair, this is one of my hopes around AI that we're starting to see is people to kind of go in and say, where is this stuff screwed up? Where is friction? The number of times general counsel would say, well, it turns out there's this one small law that was passed over here.

• 45:21Speaker 01

  And you're just like, that's what's going to break everything? This one tiny thing is going to stop the entire fixing of the healthcare system? That's insane. And then you have to go to Congress with this one small thing? We need a big refactoring. Oddly enough, Macron is the one who just said this in France, is that we have to ask ourselves really what outcomes we want, and then have we the right protections, the right barriers on it? In many cases, we have areas where we have too much regulation. And then we have other areas where the regulations do not take into account today's world. This is where I think SB 1047 did not make sense.

• 46:01Auren Hoffman?

  What is that? I've never heard of that. What's SB 1047?

• 46:04Speaker 01

  There were 18 laws that were signed by Governor Newsom here in California around AI. And those 18 were very common sense. You can't use AI to create child porn. You can't use it to create revenge porn. You have to disclose it if you're using this in clinical settings. You can't use it for hurting people with denial of claims. Duh, that stuff shouldn't happen anyway. Those are typically at the application layer. There's one at the modeling layer. Models above a certain size must do certain... There's like a whole bunch of other requirements. And you're like, well, that's a yesterday approach to the problem. And two is it really hurts the opportunity to leverage open source. And then three is where's the enforcement?

• 46:48Speaker 01

  And four is it's going to cost a ton to stand up a new government organization to look at this stuff. And then it's going to hurt our ability to actually use these systems for good purposes. Then it means that we lose this opportunity to actually have this technology. The laws aren't going to get changed for 30 years. My view of that is we'll kick it back down to a collection of people from broad basic around the community. Let's figure out what the common sense approach for this is and then go.

• 47:18Auren Hoffman?

  So there are so many data founders and CEOs that are listening to this. What is the one data set you wish you could get your hands on that they could create for you?

• 47:27Speaker 01

  My greatest one that I want to see is the precision medicine data set, like a giant data set of healthcare data, clinical data, payment data, environmental condition data. Because I want to ask questions, just like we do with an LLM, where we say something like, hey, process this, or find me interesting recipes, or build a recipe for me for a group of 10 people, two of which don't eat fish, three of which are vegan. I want to look at that data set and I want to say, show me all the common characteristics of people who have Lyme disease. Show me all the people who have this rare disease that people like to say is one of

• 48:10Speaker 01

  N, and it turns out there's 30 of them, and what we might have an off-label use for that drug. Find me all the places where healthcare costs make no sense, because they're outlier costs and we're paying for it, but it's fraud, or it's abuse of use of systems and it doesn't make sense. I want to reevaluate why we give certain therapies or treatments and that we look at them and we say, is that really makes sense? Or are we over treating people? Or maybe we're under treating in the case of maternal health, like with black mothers, for example, people of color, where the mortality rates are too high for a first world society like the United States.

• 48:50Auren Hoffman?

  Now, a few personal questions. When is that? You're obviously a smart person, you have a PhD for math, etc. But I think you've told me before, in high school, you didn't have like the best grades. You started out at a junior community college. So you were a little bit more of a late bloomer. My guess is there's millions of people that are late bloomers, maybe tens of millions of people that are late bloomers, didn't have the most amazing grades in high school or something. How do we tap into that potential?

• 49:16Speaker 01

  I got super lucky. Despite growing up in Cupertino, the hotbed of technology in the early 80s, the school system just didn't work for me. That style of education just didn't work for me. I needed something else. I would have loved it if I had Wikipedia and Khan Academy and these other things. That assumes I would have had broadband, a laptop and access to tech. But I think one of the things that's there is I got so lucky because I had an institution that was effectively an alternative safety net. You didn't get into regular college? Fine, come hang out here for two years and grow up. I can't emphasize enough how lucky I was because I had lecturers and teachers who were more interested in teaching than research.

• 50:01Auren Hoffman?

  This is when you were at De Enza College?

• 50:03Speaker 01

  This is at De Enza College. Anyone can go. It's a deal. It's like unbelievable economic value. And so it's why I'm such an advocate for two years of college education should be free for everybody, not only for retraining, but for all the late bloomers that are out there. I met women in study groups.

• 50:22Auren Hoffman?

  But it's essentially free. I mean, it's so cheap. Even today, it's super cheap.

• 50:26Speaker 01

  Super cheap. And you get to mix with people. You're not just dealing with people in the ivory tower. You're dealing with people who they're juggling like single mother, single father, they have another job and they're studying for this. Real hard trade offs. Those people taught me work ethic in a very different way. So when I got ready and I finally moved to University of California, San Diego, I was like very focused. I was very determined. But I don't think I would have found my love for mathematics without that, or definitely not my love for data, without those professors and those teachers who were there. It's also one of the reasons I think liberal arts is so important. I think too often we ignore the liberal arts.

• 51:08Speaker 01

  I know, or I'm always envious of your ability to read both breadth and depth. That ability to think critically and learn those things. I think the liberal arts schools are phenomenal for this and helping foster what you want to be.

• 51:20Auren Hoffman?

  I'm somewhat skeptical of that. I think most people I know who are really into liberal arts stuff today, it's almost like school was not good for them on that. I don't know if school is always the best place for the liberal arts. And a lot of people develop the passion almost in spite of school. And many people I know who like history majors or poly science majors, they don't even read anymore 20 years later. They're not even reading those types of things. I'm not sure that it actually fosters it. I feel like it can almost take out the passion.

• 51:51Speaker 01

  So for me, my minors are in theater and psych. We call program a concentration. They're effectively minors. Those things taught me so much more about the world. And maybe it's because also I was coming from a technical field. So I was like, this is how I got to see what the world was like versus, you know, staring at this math proof for four hours and being like, how does this work? The thing that I think about these days is we're so good at customization of apps and technology and ads, yet we're so poor at being able to customize our curriculum for students to figure out what works for them.

• 52:25Auren Hoffman?

  Well, that's for sure. I was an engineer at UC Berkeley. Most of my classes, at least in my early years, were taught by very well-known professors. Many of them had Nobel prizes and they were on average terrible. They were not Richard Feynman, who's maybe the one Nobel Prize winner who's like amazing at teaching. I couldn't even comprehend them. I couldn't understand them. They're probably great for postdocs, but not great for an 18-year-old. And then one year I took a summer class and I had a professor from San Jose State. And he was the best teacher I had had in two years. And that's because his profession was teaching. He knew how to teach.

• 53:07Speaker 01

  And he's at San Jose State because he cares about that.

• 53:11Auren Hoffman?

  He cares about teaching. He was amazing and he brought it to life and he really helped us all understand the material. All of us were so much better at the end of that class than we were at any other class that we took.

• 53:23Speaker 01

  That's right. And that's the same thing for me at the junior college level. I had it also in versions at UC San Diego. I had this professor who basically it was a late undergrad class. So it's upper level, but he basically treated us like grad students. He handed us our ass daily, crushed us. His kind of approach was this is what it means to really understand things. We're so lucky when we get to be with an educator that is like that. One message I would have for people out there that in the corporate world also is I think, or and I know you have been this kind of person is we often get so myopic in our own world.

• 54:04Speaker 01

  We're so narrow in our, the amount of things our hand, we forget to lend a hand to somebody else.

• 54:10Auren Hoffman?

  That's true. Sometimes we forget how many people lend a hand to us.

• 54:14Speaker 01

  I got lucky to meet you early on when I moved out here in Silicon Valley. You massively informed a lot of my thinking.

• 54:20Auren Hoffman?

  Well, I got lucky to be you. We got lucky to meet each other.

• 54:23Speaker 01

  You could always pick up the phone and call. There was never, what are you going to do for me? There was always like a community vibe. I've always tried to say data science is a team sport. However, you want to define data science. I think it's a team sport. And so if you're suffering in silence, you're doing it wrong. You need to raise your hand and let people know that you need help. If you're out there and you have the capacity and the ability to help somebody, you should be helping somebody else. Be that teacher that I had or you had at San Jose State. Be that person because when you do that, you're going to get so much more out of it. That's how we create systemic change.

• 55:01Auren Hoffman?

  Oh, this is great. Last question we ask all of our guests. What conventional wisdom or advice do you think is generally bad advice?

• 55:07Speaker 01

  Follow your passion. It's so easy just to say, follow your passion without doing the hard work. I think it's really about not just passion. It's about finding experiences, figuring out that and knowing that your interests are constantly in flux and changing and you should be having so many ideas and working with so many different people. People always say, what do you like, what you don't like? Find what gives you energy. Figure out what takes away your energy. Get rid of the things that take away your energy and focus on the things that give you energy and inspire you.

• 55:39Auren Hoffman?

  It's interesting. So I've asked this question now to over 100 people on this podcast. Conventional wisdom or advice is generally bad advice? And the number one thing people say that's bad is follow your passion.

• 55:51Speaker 01

  I did not do that just because I listened to the podcast, but it is true now that you say it.

• 55:55Auren Hoffman?

  It's probably a third of the people have said that exact thing. Either now that might be the conventional wisdom already that it's bad. It's already moved to the other side. I'm not sure because like everyone now thinks it's bad. I want someone to be like, follow your passion is the right thing to do.

• 56:11Speaker 01

  You have to have one of the classic Orin Hoffman graphs that show this.

• 56:15Auren Hoffman?

  Totally. Yeah. Now it's clearly conventional wisdom at this point. This has been amazing. Thank you, DJ Patel, for joining us at World of DAS. I follow you at D Patel on X. I definitely encourage our listeners to engage you there. This has actually been a ton of fun. We're old friends, so I really enjoyed this conversation.

• 56:33Speaker 01

  Always, brother.

• 56:36Auren Hoffman?

  If you're a super data nerd, go to worldofdas.com. That's D A A S, worldofdas.com. And sign up for our weekly data as a service roundup newsletter.

• 56:49Auren Hoffman?

  Thanks for listening. If you enjoyed the show, consider reading this podcast and leaving a review. For more World of DAS, and DAS is D-A-A-S, you can subscribe on Spotify, or Apple Podcasts, or anywhere you get your podcasts. And also check out YouTube for videos. You can find me at Twitter, at Auren, that's A-U-R-E-N, Auren, and we'd love to hear from you.

• 57:13Auren Hoffman?

  World of DAS is brought to you by Safegraph. Safegraph is geospatial data for physical places. Check it out at safegraph.com. And by Flex Capital. Flex Capital invests in data companies like those we talk about at World of DAS. Check it out at flexcapital.com.