R, Data Science, & Computational Biology

We’re partnering with the upcoming R Conference https://rstats.ai/, because the R Conference is well… amazing! Tons of great AI content, and they were nice enough to connect us to Daniel Chen for this episode. He discusses data science in Computational Biology and his perspective on data science project organization. Sponsors: DigitalOcean https://do.co/changelog – DigitalOcean’s developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99% uptime SLA, and 24/7/365 world-class support to back that up. Get your $100 credit at do.co/changelog https://do.co/changelog. Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: Daniel Chen – Website https://daniel.rbind.io, GitHub https://github.com/chendaniely, X https://x.com/chendanielyChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: R Conference: Website https://rstats.ai/gov/?utm_source=practicalaiTwitter: @rstatsdc https://twitter.com/rstatsdcTickets https://feeds.transistor.fm/rstats.ai/tickets?utm_source=practicalaiDiscount code PRACTICALAI20 is good for 20% off every ticket type, including the conference & all workshopsLinks relevant to the show: William Stafford Noble 2009 - A Quick Guide to Organizing Computational Biology Projects https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000424Greg Wilson, et al. 2014: “Best Practices for Scientific Computing” https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001745Greg Wilson, et al. 2017: “Good enough practices in scientific computing” https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005510Jenny Bryan’s code smells: link 1 https://www.youtube.com/watch?v=7oyiPBjLAWY and link2 https://speakerdeck.com/jennybc/code-smells-and-feels?slide=23Jenny Bryan on naming things https://speakerdeck.com/jennybc/how-to-name-filesJD Long’s talk at rstudio::conf this year about being empathetic https://resources.rstudio.com/rstudio-conf-2019/putting-empathy-in-action-building-a-community-of-practice-for-analytics-in-a-global-corporationpython’s version of pyprojroot https://github.com/chendaniely/pyprojroot“Be kind: all else is details”. – Greg Wilson, Teaching Teach Together – The Rules http://teachtogether.tech/Books “Pandas for Everyone” by Daniel Chen https://www.amazon.com/dp/0137891156“Advanced R” by Hadley Wickham https://www.amazon.com/dp/0815384572Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  As far as a first language goes, it doesn't really matter. If you're coming in from a data science point of view, what's the most important thing is when you see a data set that is quote unquote messy, can you like in your head, write the general sequence of steps to make it clean again? I borrow all the terminology from the R world, which is like the concept of tidy data. And so if you can see a data set and know the steps on making it tidy, at that point, it really doesn't matter what language you use because you can literally just look up like in the R world now in tidy verse, it's like pivot longer or wider. So you would just Google like pivot longer wider tidy R. And then on the Python side, it'll be like pivot longer wider like Python. One of those words will show up some search result.

• 0:43Speaker 00

  And I think that's probably the more important thing is just knowing the steps on processing data and then just treating programming as the thing to get you there.

• 0:55Speaker 01

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at ChangeLaw because of Rollbar. Check them out at rollbar.com and we're hosted on Linode cloud servers. Head to linode.com/changelog. This episode is brought to you by Digital Ocean. Droplets, managed Kubernetes, managed databases, spaces, object storage, volume block storage, advanced networking like virtual private clouds and cloud firewalls, developer tool like the robust API and CLI to make sure you can interact with your infrastructure the way you want to. Digital Ocean is designed for developers and built for businesses.

• 1:35Speaker 01

  Join over 150,000 businesses that develop, manage and scale their applications with Digital Ocean. Head to do.co/changelog to get started with a $100 credit. Again, do.co/changelog.

• 2:06Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.

• 2:33Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International, and I'm joined as always by my co-host, Chris Benson, who is a Principal Emerging Technology Strategist at Lockheed Martin. How are you doing, Chris?

• 2:52Chris Benson?

  I'm doing very well today, Daniel. How's it going?

• 2:54Daniel Whitenack?

  It's going great. Over the weekend, on Saturday morning, my time, I gave a workshop at ODSC Europe, Open Data Science Conference Europe, and that was a good time. Virtual conferences are kind of fun because they get to connect. There were people joining from all over the world, kind of, so that was cool to get, you know, people joining into the workshop from sort of all over and get to discuss some fun things with them. We did some transfer learning and reinforcement learning and GANs with TensorFlow, which was fun. Transfer learning is very much the bedrock of a lot of, you know, industry work.

• 3:34Chris Benson?

  Sure is.

• 3:35Daniel Whitenack?

  Reinforcement learning and GANs is like, for me, it's a really fun topic to play around with and have some fun with. It definitely has some practical application, but it's just kind of fun to get into. So that was a good time. It was a fun weekend in that sense. What about yourself?

• 3:52Chris Benson?

  Well, over the weekend, just enjoying cooler weather. It's been nice. We're getting into, you know, about to get into the fall here, and that was pretty pleasant. Last week, I know that we may talk a little bit about public orientation, government-type stuff, potentially today, in terms of conferences and stuff, and was doing some work with the Air Force. It's amazing to see, you know, government, you know, following industry into advanced technologies, things that support AI, doing software development better. That was the last week for me, and it's just a pleasure.

• 4:25Daniel Whitenack?

  Yeah, yeah, cool. Yeah, it's definitely good to get some cooler weather, I'm sure for you too. I can't imagine, we've mentioned this before on the podcast, I just don't know. I don't know if I could survive Georgia.

• 4:37Chris Benson?

  It's actually chilly outside right now.

• 4:40Daniel Whitenack?

  Yeah, it's awesome.

• 4:41Chris Benson?

  It's amazing when you get through a Georgia summer, which is all very humid and hot. And you get to this first inkling of spring and everything is like, ah.

• 4:52Daniel Whitenack?

  Yeah, yeah. Yeah, well, as you mentioned in talking about sort of government and that vertical, which I know you're involved a lot in, recently got connected to the R conference, which I've spoken at before, the New York City version, which was a lot of fun. I think that was a two or three years ago I was there. But now they do, of course, they're doing virtual conferences this year. I attended the one online recently, it was super high quality and a lot of fun. They had a great system for it, but they're doing several of these that are related to different verticals. There's going to be one coming up that is related to government and related sectors.

• 5:36Daniel Whitenack?

  And what used to be the sort of DC conference, but is now, of course, virtual. And got connected to that. So we're actually gonna, as the podcast, we're gonna kind of do a media partnership with that conference. And that's going to be really cool. And as part of that, they helped arrange today for us to have a conversation with Daniel Chen, who's with us, who is a Lander Analytics data scientist. He's a PhD candidate as well at Virginia Tech and a former R studio intern. So, welcome, Daniel. It's great to have you on the show.

• 6:11Speaker 00

  Hi, thanks for having me.

• 6:12Daniel Whitenack?

  Yeah, we're gonna have to navigate this to Daniel situation. Chris, best of luck to you.

• 6:20Speaker 00

  If it makes it easier, you can call me Dan.

• 6:24Daniel Whitenack?

  I may still respond to that accidentally, but we'll navigate this. I think we'll, I think we'll get through. I don't know about you guys, but it always bothered me when like, when there's two people named the same. Sometimes people want to call that like Daniel squared or something, but that always really bothered me because it, it seems to me to be actually two Daniel, not Daniel squared. Like there's two of them. It's two Daniel. It's not like I'm multiplied by Daniel Chen.

• 6:50Chris Benson?

  You're not multiplied by yourself there.

• 6:52Daniel Whitenack?

  Right. I don't know if that's just my own peculiarity.

• 6:58Chris Benson?

  No, you know, my name's Chris. I had, there were Chris's all over the place growing up, so I, I understand. You feel like you're melting into the background when there's like five Chris's in your class. I get it.

• 7:08Daniel Whitenack?

  Yeah. Well, on that note, Daniel, Daniel Chen, if you want to just let us know a little bit about yourself, about your background, how you got into doing what you're doing now, the path that you took to there. I'd love to hear a little bit about it.

• 7:25Speaker 00

  Yeah. So I grew up in New York City and I guess I've, my dad is a software engineer. My parents, when they came to America, they both studied computer science in college. My mom doesn't do computer science related stuff anymore, but I've always had a computer at home since, you know, when I was a kid. So like back in the day, these were just like old company hand me downs. And around high school, I went to one of the math and science high schools in New York City. And it was interesting because sophomore year, it was a requirement for all the sophomores

• 8:07Speaker 00

  to take one semester of computer science and one semester of technical drawing. And then the people who liked computer science at the time could take the AP class, et cetera, et cetera. But it was interesting in the fact that like every single sophomore student in high school was exposed to programming some way he should perform.

• 8:24Daniel Whitenack?

  Yeah, that's super interesting.

• 8:26Speaker 00

  Yeah, we actually, looking back on it, I have no idea how. The instructors handled, like, got all the material in, because we covered in like one semester, we covered, we first went through NetLogo, which is like drawing turtles, uh, and I made like a little blackjack game for that small project. Then we went through uh Scheme, which is like a Lisp language to talk about like lists and functions. And then like towards the end, we like got introduced to Python, where it was like make your own prisoner dilemma kind of um algorithm and we'll compete it in the class. And that was all in one semester. So like as an educator, like I start teaching data science now, I'm like always baffled when I think back, like, how did they make that work?

• 9:09Speaker 00

  Because like, there's no way, like, I wouldn't be able to teach all of that stuff, even in like a semester. So that was super interesting.

• 9:17Daniel Whitenack?

  Yeah, what did, did it seem overwhelming at the time, or did it just seem like new and uh like exciting stuff or or different stuff?

• 9:25Speaker 00

  Yeah, so it was new and exciting, but as I started teaching more, I didn't realize it at the time, but yes, there's always at the, because of it was a math and science high school, like clearly there were people who have done this stuff before in the past.

• 9:42Daniel Whitenack?

  Right.

• 9:42Speaker 00

  And then there are the people who like have seen this for the first time. And so I was in a camp of like, I've actually never programmed before, but then there's all these kids who like knew the answer as the questions being written on the board and I'm staring at a blank like piece of paper, like, how do I do this? And so it was actually, that was like one of the like, I don't think I'm ever gonna do this for a living uh moments.

• 10:05Daniel Whitenack?

  Yeah, it pushed you into that place rather than sort of uh um further inspiration, at least at the time.

• 10:12Speaker 00

  Yeah, well, like, I've, I've always been interested about like just tech and things, but yeah, programming definitely at the time seemed this is not for me kind of ordeal. Um, yeah, and then fast forward a little bit to my undergraduate years. I ended up getting a computer science minor, uh, just because I was like, you know what, I'm just gonna go, go do it, like, just learn how to program like formally. And that's when I realized, like, looking back, like, the whole people who have seen it before versus not seen it before, like, my intro classes, they were relatively easy for me. Even though it was like, for example, the one C++ class, I've never actually programmed in C++ before, but I didn't have to think about like

• 10:55Speaker 00

  You know, print statement debugging, like, that's a, that's like, not a brand new concept at the time.

• 11:00Chris Benson?

  Right.

• 11:01Speaker 00

  Or, um, you know, if statements and loops are no longer something I need to struggle with because I've seen it before in the past. And then I actually felt bad for some of my students. I picked up my computer science minor like junior, senior year of my undergraduate career. And then I felt bad for like the freshmen coming in who were like, they've wanted computer science as like their degree, but they've, like, but they've never seen it before and like they actually struggled really hard. Uh, so like, that's when I like had those, uh, those feelings back in high school again.

• 11:34Chris Benson?

  Gives you empathy, doesn't it?

• 11:35Speaker 00

  Yeah, yeah, yeah. Um, that's when I actually started realizing like, hey, wait, I've actually seen this before. And that's why it's easy for me. So, um, and in some way that's like carried forward. So like after, after that, I got my master's in public health in epidemiology, which is somewhat relevant these days. And it was a two-year program and the second year I ended up taking a intro to data science class with, uh, some of my MPH friends. And that's where I met Jared. Uh, so that'll eventually tie in somehow. And so like, it was during that intro to data science class where I sort of really understood what like,

• 12:19Speaker 00

  data science and like, what could you actually do? Um, like during the time when I was doing my masters, it was, we talked a lot about like linear regression, logistic regression, survival analysis, and all of the Epi concepts associated with that. But it never, like, I never knew what like random forest was or like clustering, um, and all of that stuff. Until I took the data science class. And that's when it was sort of like, oh, if you can just think of something, uh, something already exists to make that happen in some way, shape or form. So it was really eye opening in that sense that like. Whatever you can imagine, you can probably make it happen in some. So that was great.

• 13:02Speaker 00

  And then from my MPH, I entered my current PhD program. Fast forward like till today, since I started, I am now doing my dissertation topic on data science education in the medical and biomedical sciences. So.

• 13:19Daniel Whitenack?

  Do you think that those, I mean, it sounds like that those experiences of, you know, in high school, when you were introduced to computer science and then when you're introduced and kind of your vision was expanded to see all these different methods and the possibilities later in your education. Do you think that like pushed you to this specific interest in data science education? Or like, what is it you feel about data science education that like, I know there's a lot of gaps out there and a lot to be addressed, but how did your specific interest in that develop and what are you kind of hoping to learn and contribute through what you're doing now? And in your current PhD work?

• 13:58Speaker 00

  Yeah. So I guess like in terms of like pivotal moments in my life, it would definitely be taking that data science class during my master's program. And part of it was Jared was a inspiring teacher. Jared, Kair and Rachel, they taught the class and it was a very, actually it was a very difficult class, but like if you struggle through it, there was so much that you learned from it. And also during that class, it was the first time I attended a software carpentry workshop. And so those two things put together sort of put me on the road where I am now.

• 14:40Speaker 00

  So during that software carpentry workshop, so a little background about software carpentry, which is now the Carpentries, they are a nonprofit organization focused on teaching scientists the computing skills that they sort of never were learned, uh, never taught. And so. I attended that first workshop. I sort of knew a little bit of Python from like undergrad and high school years and had been sort of like playing around in Bash and Git because for some weird reason, I decided to install Linux on a computer where no one that I work with uses Linux. And so the stuff that they taught during that workshop were like all of those pieces. It was a little bit of Python, some Bash and some Git. And I thought to

• 15:25Speaker 00

  myself like, hey, like, I can actually do this. It's like not that much of a jump from what I currently know. So that's how I got into the education area. So like the following semester, I signed up to be a carpentry instructor. This was back in like 2014 or so. And that's where I met Greg Wilson, who was the instructor trainer at the time. He currently works at our studio. But that's sort of where I picked up all of the fundamental parts of like, teaching this stuff. Didn't know that like, this would actually turn into like a career or like a dissertation topic. But that's sort of when I realized, or started

• 16:06Speaker 00

  thinking about like, what makes a good teacher, thinking about students, like how to convey topics in some coherent way for people who are new to this. And I did that for enough times, or over a large enough period of time, that I eventually wrote it all down into a book called Pandas for Everyone. And so that is my attempt of teaching Python from a data science perspective. Yeah, using Python.

• 16:35Daniel Whitenack?

  Yeah, that's awesome. So you mentioned Jared a couple of times. It's Jared Lander. He is very involved in the R world. So if you're listening and you're here in the part of the R community, you probably know that name already. But he's also involved in the R conferences that I mentioned, like the one that's coming up later this fall, the podcast is involved with as well. And he actually was a previous guest on the podcast as well, all the way back on episode number seven, which seems like another age ago.

• 17:05Speaker 00

  It does.

• 17:05Daniel Whitenack?

  And you know, I don't know if, as part of that Data science class with him, Daniel, if this was part of it, but I remember him just talking this really great sort of giving a really great overview of the landscape of like machine learning or AI techniques and like where certain things fit in and like how to kind of orient yourself in terms of how, for example, deep learning fits into like the spectrum of other techniques. So that was very, very useful.

• 17:35Chris Benson?

  So I got a question for you, Dan, and it's something that really caught me when you said it a few minutes ago. You were you were referring to that first data science class when you were taking your masters as a pivotal moment for you. I'm kind of wondering, we have other students out there listening and they're kind of trying to figure out where they want to go. What was it about that class that you found inspiring? You talked a little bit about the fact that if you get through it and when you could, you, you know, it would help you. But what was it that really grabbed you about that? What was it that you found beautiful about data science at that particular moment?

• 18:07Speaker 00

  Yeah, so there's two parts to it. One was the people and then the actual and then second was the actual data science material. And so this was a a class. So the people that you're interacting with are probably the going to be more important than anything else. And what I've also learned, this doesn't apply to actual Jared's case, but one of the things I learned over the years, like what makes a good teacher doesn't necessarily mean you have to master the material, like being a good teacher is different from knowing the material. But it was sort of like the way the whole entire class was taught. Jared taught the technical lab component, and he was also a carpentry instructor at the time. And so it was sort of that style of actually live coding in the class to go through the lab material that was really good as a student to see because one, like, it just slows you down.

• 19:02Speaker 00

  Instead of flipping through slide decks, it literally will just slow you down. And you see the like typo error process and stuff like that, which is a lot to take in when you're a student, seeing it all for the first time, but.

• 19:21Speaker 00

  I want to believe that subconsciously it does really help a lot just seeing the error process. And then Kair Patel and Rager Shutt, they taught like the general data science landscape portion of it. And that's where I learned about like, how does this apply to like, everything else? Like, there are so many techniques and methods outside of what I was learning in my epidemiology classes that I just didn't know existed. And so just learning about those methods and just understanding, or not really understanding at the time, but just seeing how they, what they are, how they work, just understanding the heuristics of how they function under the hood.

• 20:03Speaker 00

  I saw so much, it was eye opening for me just to see how this could just be applied in the health space. Granted, I was doing a master's, so like, a lot of the stuff that we were learning in the data science class, you know, I believe that if I were to do a PhD in epidemiology, I would have seen some of that stuff eventually, but it was more just like, I was doing a master's, there was so much new information about a field already coming in, and then you just threw in this analytics component, and it was just like, whoa, we can just do this for everything. So it was sort of like that eye opening moment for me, where it was just, the teachers were great, so it kept me vote motivated. And then the material itself, I just was able to make so

• 20:46Speaker 00

  many more connections to what I was currently learning. And so that sort of just kept pushing me forward.

• 21:08Speaker 02

  Change log plus plus is the best way for you to directly support practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal, and help sustain our production of practical AI into the future. Simply follow the change log plus plus link in your show notes, or point your favorite web browser to changelog.com/ plus plus. Once again, that's changelog.com/ plus plus.

• 21:59Daniel Whitenack?

  Yeah, so I was really interested to hear that you kind of, as you were going through that data science class, you saw kind of a new world open to you in terms of how these techniques could be applied specifically like in the medical space or in epidemiology like you were talking about. Do you feel like that those communities now are aware of those methodologies and like data science and AI is really like taking a foothold in those industries? Or do you still kind of see it as like maybe a bit of an uncomfortable mixing right now and people like still learning where things are being applied? What do you think is like the kind of current state of those things and how do you see it progressing

• 22:44Daniel Whitenack?

  forward?

• 22:45Speaker 00

  So it's definitely been more adopted in the medical space, especially with like deep learning stuff being so good at image recognition. Like that's a prime case for, you know, looking at medical imaging. But it's tricky for other parts of medicine because a lot of what we learned in epidemiology courses and biostats courses is trying to do inference on our like data. And so EPI as a field is one way you can think about it is it is the field of setting up all of your observational experiments. So when you do the stats, you're a little bit more comfortable with like what is actually like a cause and effect.

• 23:29Speaker 00

  And so if you take that part in mind, it gets a little tricky because there's so many machine learning methods that are really just black boxes that really don't give you like any sort of inference. It's really just made for a prediction. And so you have to be careful using these methods in a medical context if they are like these black box methods, because if it predicts something wrong, it becomes harder to figure out why did the model predict this wrong. And usually at the other end of this is someone's life on the line.

• 24:02Daniel Whitenack?

  Yeah, the consequences are high.

• 24:04Speaker 00

  Yeah. So yes, there is a place for all of the, you know, AI ML stuff in medicine. And you just have to be more careful when you're trying to put a model into production, I guess, than like, you know, your regular company, I guess. That's the other way to put it. It's, it's at the end of the day, like in health, like the end of that model is going to affect someone's life versus, you know, some bottom line, I guess.

• 24:31Daniel Whitenack?

  Right. And I imagine that, uh, that, that kind of ties into some of your feelings about, you know, good, uh, good code practices and like the carpentry stuff that you were talking about as well in terms of understanding like the implications of the code you're writing and how, how to test it and how to deal with, um, you know, debugging models and, and all of those things.

• 24:53Speaker 00

  Yeah. So that the next part question or problem is like, not everyone was as fortunate as me. Like I went into a public health program or a medically related program and then got thrown into data science. Um, and then went down that track. So a lot of people who are actually practitioners or physicians on the medical end, when they want to do like research, they typically are just doing research from, you know, like Excel sheets because that's what they know, or that's what they went through school with doing. Um, they weren't taught like all of the techniques and methods and skills from like computer

• 25:36Speaker 00

  science or data science or just programming, uh, in general. And so.

• 25:42Speaker 00

  Yeah, that's sort of where the carpentry stuff comes in, where, you know, now it's our time to teach all of the researchers, like the skills that they haven't like actually formally learned. And they just went through their life, you know, patching stuff together because, you know, programming was the means to get their work done. And it was just, you know, they just had to program something or do some kind of analysis just to get the result that they needed. They just struggled with a tool because they never really had formal training. And so that's eventually how I came to my dissertation topic, which was, I've been teaching for so long. You know, I read like education books like for fun and I've always had this interest in the medical space.

• 26:25Speaker 00

  So find an advisor who will let me mash those two things together. And I got super lucky at tech. So my current advisor is Anne Brown. And yeah, I got super lucky just getting to meet her through the Virginia Tech library. So if you are a student, definitely go defend a librarian. And, you know, because if you think about what the people in the libraries do, like they've been doing data science like since libraries were a thing.

• 26:53Chris Benson?

  That's a great point right there. So one thing I'm curious about is you, you have a lot of experience in both Python and R. You know, on the, on the Python side, you wrote pandas for everyone to, to, to share that learning. On the R side, you're, you know, you're giving a talk at the R conference focusing on government and public sector. And I'm wondering, you know, those are two different tools within the data science toolkit, if you will. And how do you see those? At what point do you turn to R and say, that's, you know, the, the particular problem I'm trying to solve right now lends itself better to R in your view? Versus when would you turn to Python? How do you, you hit, since you have them both and often those two communities, you know, people kind of doing either or.

• 27:38Chris Benson?

  But, you know, for the benefit of someone who might want to consider both, how do you see that? Where, where is each one stronger for you personally?

• 27:46Speaker 00

  Yeah, so currently, like today, the way I pick the language is like, who am I working with? So if I'm working with my advisor, I'm probably working in Python. If I'm working with someone else who does R, I'll probably use R. That's today. If you're currently an R user and you go through my book, there was a tweet like a couple of weeks ago that was actually like, this book is great if you're an R user because like I make so many references to R things in the Python book. It's not super explicit, but it's one of those like if you know, you know, kind of moments. And what's actually like interesting these days or now and like it really doesn't matter which language. If it's your first language, it doesn't matter.

• 28:28Speaker 00

  Eventually like you're you're gonna end up learning both just it's I I almost feel like it's the nature of just doing data science.

• 28:36Chris Benson?

  It's the nature of programming.

• 28:37Speaker 00

  It's the nature of programming.

• 28:38Chris Benson?

  Lots of languages for different things.

• 28:40Speaker 00

  Yeah, yeah. And so as far as a first language goes, it doesn't really matter. If you're coming in from a data science point of view, like I I always make the distinctions between like data science and computer science. But if you're coming in from a data science point of view, what's the most important thing is when you see a data set that is quote unquote messy, can you like in your head write the general sequence of steps to make it clean again? And I borrow all the terminology from the R world, which is like the concept of tidy data. And so if you can see a data set and know the steps on making it tidy, then at that point, it really doesn't matter what language you use because you can literally just look up like

• 29:25Speaker 00

  in the R world now in tidy verse, it's like pivot longer or wider. So you would just Google like pivot longer wider tidy R. And then on the Python side, it'll be like pivot longer wider like Python. But in Python, it's melt and pivot. So one of those words will show up like some search result. And I think that's probably the more important thing is just like knowing the steps on processing data versus and then just treating programming as like the the thing to get you there. Because if you struggle, if if you're just starting off, you don't know the steps, you don't know like the terminology or how to clean data. And then you're also trying to learn a brand new language. So when something goes wrong, you don't know if it's like

• 30:07Speaker 00

  your that the sequence like the overall sequence was wrong or was it like an actual like programming type of mistake? And that's that's what you wanna like.

• 30:18Speaker 00

  Separate as much as possible. So like just pick one, learn how to manipulate data. And once you're comfortable with that, it becomes super easy to transition to another language. Like when I did my data science course in my master's program, it was actually all done in R. It was actually all done pre tidy verse was like formalized as a thing. But I worked with processing data for like a good year or two. And then that's when I like sort of when I like actually understood what like tidying data meant. That transition into Python was super easy. And then that's sort of why like the ordering of the book that I put together or like, there was a lot of stuff in the book that was sort of like this is, I learned all this from my transition to R.

• 31:03Speaker 00

  And so that's why there's so many like random R things in the Python.

• 31:07Daniel Whitenack?

  Yeah, that's awesome. And I, it sounds like we were talking a little bit before the show about kind of your, your personal data processing pipeline, like when, when any of us kind of go into a project, somehow we have to set up a set of, you know, scripts, programs, folders, files, config, you know, data sources, whatever that is to, to define our, our project and the structure of like the pipeline that we're using. It sounded like that's something that you think about quite a bit. What, what is your, as you're kind of now also thinking about data science education a lot?

• 31:48Daniel Whitenack?

  What are your thoughts as, as far as when you're talking to students, when you're thinking about how to educate them around your project structure? What are some of the main things that, you know, really can benefit you as you set up a project, a new project? Whether that be just something that's analytics or whether that be a machine learning project? What are some ways that you can help yourself down the line when you start out a project?

• 32:13Speaker 00

  Yeah, so I am in academia. So there's three papers that like sort of talk through this entire process. The first one that I read that sort of introduced me to all of this is by William Noble. And the title is called A Quick Guide to Organizing Computational Biology Projects. And that was probably the first time I've seen. In academic writing, literally, how do you set up the folders in a project? So you have like an output or an output folder. You have like a scripts folder, you have a docs folder. You have a readme file on the top level.

• 32:55Speaker 00

  Stuff like that. And then the two other papers that sort of like expand on this from the carpentries folks is there's a paper called Best Practices for Scientific Computing. That was written in 2014 and then in 2017, there was another one that said, "Good enough practices for scientific computing." So you can see how like doing good or best practices is actually pretty difficult. But it really all does stem from, one of the core pieces is really having a folder structure. So that your scripts can find the data that you're working with. And it's focused around the idea of,

• 33:35Speaker 00

  "Yes, it works on my machine, but it needs to work on someone else's machine or another one of your machines or the cloud." As well, without having to, you know, change a whole bunch of like file paths. In an ideal world, it runs on your computer with a, with like, a command and it will run on another computer with a, the same exact command without you having to change anything. So that's sort of the overall like overview of what I focus a lot on. And then there's, you know, then there's like the super technical parts of like, yes. Git is a thing, version control is a thing that you have to know when you're trying to collaborate. That's just sort of the nature of the beast.

• 34:21Speaker 00

  The good thing is, the carpentries has a Git lesson. So if you want to learn it on your own, it is written down somewhere. And I've, this past summer, put together a few workshops that are on the carpentries like YouTube page on like, the actual super like complicated collaboration aspects of.

• 34:43Speaker 00

  Using Git and GitHub. But yeah, so most of my stuff really does focus around, you know, project organization is the actual like cornerstone or centerpiece to managing like a project.

• 34:56Daniel Whitenack?

  Yeah, it's interesting. Like you talked about the one paper being like best practices and then they went to like good enough practices. That that concept definitely resonates with me. I was wondering your own, because there is like software engineering sort of best practices in industry where like I think now, like if you're working on a project, you have a GitHub repo. If it's not connected to some sort of C I C D and you don't have some sort of portable way of deploying this thing, maybe with Docker or something like that, that's kind of like what people are doing a lot. But that's like a, that's a lot of things for a like someone in academia or like a new data scientist to learn. It can be rather burdensome.

• 35:41Daniel Whitenack?

  So like, yeah, I guess daunting, yeah, daunting is a good word. So like in terms of like people that are starting out as data scientists, do you think that's something like as they're embedded in an organization, should, should they sort of strive for, you know, eventually kind of learning all of those software engineering best practices and like adding that to their workflow? Or do you think there is a sort of in between where, you know, the workflow of a data scientist is, it is different, right? There's different data concerns and all of those things. So how much of a software engineer does a data scientist have to be, I guess, is the

• 36:25Daniel Whitenack?

  in, in question that I'm, that I'm going for?

• 36:28Speaker 00

  Yeah, so that's the other big dilemma is a lot of workflows from data science are actually like anti-patterns from software engineering.

• 36:39Daniel Whitenack?

  Right.

• 36:40Speaker 00

  As a data scientist, we primarily work in like scripts that execute from top to bottom. Very rarely do we end up writing classes or. Things like like using those software engineering tools in a data science analysis. We will write functions, that's good, but we don't necessarily create packages like that is considered like maybe a a best practice, but it's it's a lot more stuff. Uh, so like just writing a function is good enough, but then what happens when you have fifty functions like? Yeah, there's this likes tension between well, not tension, but there there is like the way you program things and from a data science perspective is going to be different from software engineering. That's just going to happen. It's

• 37:25Speaker 00

  kind of interesting when you hear stories about data scientists working with the engineers and then like when their code bases need to mesh and that becomes a a different question and problem on its own. But from at least from what I'm working on now, which is the data science perspective, but it's like cater towards the biomedical sciences and those people. We even need to go like an even step further back from that. From like we're thinking about best practices and that stuff, because these are the people who are so new to this field that if you talk about like Docker and C I C D integrations like those are like letters that they've never seen put together before in that order. And so so one of the, I guess,

• 38:10Speaker 00

  byproducts of my dissertation is this, uh, I guess you can call it a book slash lesson plan. That's called DS for Biomed, so Data Science for the Biomedical Sciences. Literally, the first thing I talk about is like, we're just gonna talk about spreadsheets for now. Like, because it is probably something that they're most familiar with in terms of a data perspective. Spreadsheets are one way you can think about spreadsheets. It is a GUI for your data set. And so people like looking at things and being able to click on thing. And so how do we go from spreadsheets to data science? Pipeline is sort of where I'm focusing more of my time uh these days.

• 38:52Speaker 00

  And so, yeah, like, I just finished like the first like spreadsheet module, so I can actually talk about this. And putting that part together, I sort of realized that, like, yes, we can actually introduce like those tidy data concepts, like, in the spreadsheet section, which is like, you know, if you've ever loaded up like an Excel sheet, like, first of all, when I'm, as a data scientist, when I see an Excel sheet, I'm already preparing myself over like if I get a CSV file. And so, like, why do I, like, you know, cringe when I see an Excel file? Well, it's because like, you know, sometimes, you know, we have multiple tables like in the same sheet. And it's like from A to M is like one table and from like P to Z is like another table.

• 39:35Speaker 00

  And you have to load those tables separately, like those are like data issues that happen when you're loading in data into R, Python, or whatever language. But from a lot of people who don't actually work with programming languages, that's great. They get to see everything at the same time. So it's sort of like identifying those bad habits and trying to show them why they're not conducive. If you want to load them into a programming language is sort of, like, that's where I'm at right now. And so it all also comes down to the whole mantra of you want to have empathy for the people who are learning this stuff.

• 40:14Speaker 00

  And, you know, if all they get away from like that first workshop is like structuring their spreadsheets better, I'll be okay with that. I'll be happy with that. So it's always about making like these small incremental improvements every time you start a new project. And that that happens if you're like a full-blown data scientist scientist as well, like, yeah, maybe you have the whole project structure thing working for you and you can have your computer, you know, all your code work on whatever machine that your code base is deployed on. What would be the next step for you? That might be trying to learn like one of the continuous integration services or like. Using Docker or something. So there's always something that you can do to improve

• 40:55Speaker 00

  like your workflow. And I guess like that that does take a lot of effort on like one's end, uh, cause you do have to do a lot of like introspection of like what can be improved. And like, um, I guess like the way I've always seen it for me, it was easier for me to do it because it's always like, oh, what is the best practice? And then I read about it and it's like, that is way too complicated. And then like six months later, it's like, that seems doable now because I've like learned all the other stuff in the middle. That's like, that gets me there. So there's different entry points towards like picking up practices from software engineering. But at the end of the day, like data science pipelines or workflows really don't mesh with software engineering stuff.

• 41:38Speaker 00

  Like in software engineering, your end product is probably like a library or like this big program thing versus like in data science, it's really like this pipeline of scripts that like create this model. And then this model gets handed off to like the software engineers to implement somewhere else. Uh, so like those things are just going to be different. And it makes sense that the best practices on both sides aren't going to be the same. But you know, if you make incremental progress, you know, you'll eventually get to a good spot.

• 42:21Daniel Whitenack?

  So you're really, um, thinking a lot about trying to get people working in these areas, um, bioinformatics and other related areas to think about using data science techniques. If you were to look into the future and let's say that you've accomplished your goals of getting these people to use these sorts of techniques in their workflows. What are some of the example things that you envision them being able to do with data science techniques? That maybe they wouldn't have been able to do if they kind of followed the, the same workflows that they have been using for quite some time?

• 42:56Speaker 00

  So one of the main takeaways would be just working with multiple sources of data at the same time. We have a system where, you know, every local department, uh, organization, government at the government level, et cetera. They're doing reports of

• 43:19Speaker 00

  Case counts, for example, on a daily basis. And they don't necessarily all come in as like one. They're not all combined together for you. In this current pandemic, yes, you can find data sources that are doing the aggregation for you. Back in 2014 during the Ebola outbreak, that wasn't necessarily the case. We were getting daily reports from different countries as like PDF files, for example. And so being able to work with multiple data sources is going to be one of those like skills that are going to be super important. And how it all ties back into like, why use a data science approach?

• 44:01Speaker 00

  And when I say that, like, why use a programming language to do that kind of analysis over something like spreadsheets is that goes into like one of the most important things when you're working with data is like you always want to keep your raw data completely intact. And so this way, if there is an improvement or something in your actual data science code, in an ideal world, you just rerun your code over a new set of data and then you get your updated results right away. That's probably the the most important like idea that we have, even in like today's COVID world. That's sort of the reason why like you'll hear like recommendations changing over the

• 44:46Speaker 00

  past couple of months is because in the beginning, we weren't, the data itself didn't show conclusion, but as more data came in, if you were to rerun your analysis over and over again over new courses of data, you might actually find a new outcome. And so like currently, we are in real time living the scientific process. And part of that process is, you know, making sure that if you do have new data sources coming in, you can still rerun your analysis and. You know, that part is reproducible. And then, you know, as more data comes in, your conclusions may change. So that's sort of how that all ties into current times. But it really is something that is like really just fundamental to data science as a whole.

• 45:31Speaker 00

  Since, you know, we're always querying data from the world and we want our pipeline to be there. So that like as new data comes in, we can have an updated model at the other end.

• 45:42Chris Benson?

  That makes perfect sense. Um, so, you know, we talked a little bit a while ago about the fact that you're doing a talk at the R conference. And I was wondering if you could share a little bit about, you know, what you're talking about? And what is your message to the R community? Uh, and you give us a little insight into that. Would love to hear what's of interest to you.

• 46:02Speaker 00

  So my previous R conference talks have always been around the topic about like this data pipelining, uh, part of data science. So like, you know, I think the last talk I gave, uh, last year was, you know, something around like, we're, I'm going to teach you how to make a make file, uh, so you can like make your reports.

• 46:24Chris Benson?

  Super useful.

• 46:25Speaker 00

  Yeah.

• 46:25Chris Benson?

  It is.

• 46:26Speaker 00

  So last year, so 2019, I was one of the interns at our studio. And I worked on a package called Great This, which is the auto grader system for R code. So it can, yes, tell you if like you're an instructor, like here's the correct answer. And then the student can type in some R code and compare the results. That's like the easier way you can grade code. The more complicated way you can grade code is like looking at the code itself. So like in more technical terms, it's looking at the abstract syntax tree. And so you're literally comparing like if the student put in, for example, log of three

• 47:07Speaker 00

  and your solution is log of two, you want a sentence that's essentially says, you put in three where the answer should have been two. So creating that sentence is a lot more complicated than it may or may not seem if you didn't think it was a hard problem. So during that process, I learned a lot about R's way of.

• 47:29Speaker 00

  Handling code expressions. And so this year, I am trying to teach that to regular people. How that ties into the general, to the greater our ecosystem is if you've worked with Tidyverse packages, you'll notice that you are allowed to pass in like column names without having them like quoted in strings, but they look like regular variable names, uh, which is like terrifying from like a Python user's perspective because lazy eval is not a thing in the Python world, but it's like there in the R world. So it's sort of trying to introduce those topics is sort of my goal for like the next series of talks.

• 48:14Speaker 00

  Why does this help? This is sort of the transition of if you want to write your own Tidyverse compatible packages for your own work, this is sort of like what you need to know to make that happen. So yes, um, the talk is more towards like the software engineering side of things, but it's one of those like, hey, if you want to have your own work plug into this whole ecosystem, like how would you go about doing it? So, uh, this is my part of trying to make an incremental improvement for myself and like for the greater community.

• 48:47Daniel Whitenack?

  Do you think as you dug into like the underlying mechanics of how like our processes, expressions, do you think that's influenced how you write your R in terms of like just your general programming and like, has it made you more sympathetic in terms of like how you, how you write your R with that kind of better underlying understanding?

• 49:11Speaker 00

  So as far as like, like regular day to day, like if you were to just tell me to run some type of analysis on a data set now, it doesn't affect that part of it. If anything, I have a lot more sympathy for the people who develop these packages. Like last year, I literally read. The advanced art book like three times over and over again without understanding like what I was doing during my internship. And then like after seeing it and having it like mesh in my brain for like a year, like I'm reading it like again, rereading it again and it like makes total sense now. So as far as like a day to day thing, it doesn't affect it that much.

• 49:53Speaker 00

  But when I am writing functions and things that might need to end up like a collection of functions. If I start writing a collection of functions, whether they make it into a package or not, I am more mindful of certain things. Mainly around like dependencies is what I'm really mindful about. One of the things that sort of surprised me last year was, you know, if I wanted to, you know, do like some kind of grep search for a string, you know, in tidy first world, I would just instinctively like use something from like the string R package or something like that.

• 50:29Speaker 00

  But you don't need the entire string R dependency, which if all you're doing is like a simple grep call, just use like the regular built in grep L. It's like, fine, like I've sort of realized that like, yes, there's like, when you are a package developer, all of the engineering like hurdles are now like your problem. Like your, your job is to make like the end user's life easy. Uh, and then you deal with all of the engineering burden on your end. And so I, I definitely appreciate that a lot more. But like, as far as like my day to day, it's mainly just like, you know, just write more functions and like try to keep working on like the best practices and stuff like that.

• 51:08Speaker 00

  And then when I have a, a new student that I'm working with, that's sort of like grounds me back like, okay, this is where I once was. And then so how do I get them to like some other point, uh, the next like level in their life and programming? Like, how do I make that transition like less violent for them?

• 51:28Daniel Whitenack?

  Yeah, no, I, I, I really appreciate what, what you just said, actually about dependencies. Like you can reduce, I don't know what the right word is, I guess your liability or your, your potential debugging issues in the future. If you only need for, like, I've, I've done this sometimes where it's like, oh, I need like a sigmoid function or something like, well, I could like import any number of packages where I could call like sigmoid parentheses, like give my thing, but I could also just write that in a couple of lines of code and just embed the function in my own code so that like it's super clear what's going on. I don't have an external dependency.

• 52:11Daniel Whitenack?

  I think that's something that like is underrated a lot. So I really appreciate you bringing that point to the surface. And I'm super interested to hear like the other insights you have from your talk at the R conference. We'll definitely look forward to that. And we'll also link in our show notes to a bunch of the things you've mentioned, the carpentry courses, your book, various packages you've mentioned and all of that. We'll also link to Jared Landers episode if people want to go back and listen to that. But yeah, we really appreciate you joining us. I think if I'm right, that people can find out more about the R conference at rstats.ai is the correct website. And we'll of course link

• 52:54Daniel Whitenack?

  to that as well. But yeah, it's been a huge pleasure to get to chat with you, Daniel. Looking forward to spending some more, more time together at the conference.

• 53:04Speaker 00

  Yeah, thanks for having me. It's been really fun.

• 53:10Speaker 02

  The R conference is where R enthusiasts and data scientists gather to explore, share, and inspire ideas. It's happening December 2nd through 4th, and we have a discount code for you. If you're interested, use code PRACTICALAI20. That's good for 20% off every ticket type, including the conference and all workshops. This episode was hosted by Daniel Whitenack with Chris Benson. Our special guest, Daniel Chen. Thanks for coming on the show, Daniel. Our music is produced by the one and only Breakmaster Cylinder, and we are brought to you by amazing sponsors. Thanks again to Fastly, Linode, and Rollbar for their continued support. That's all for now. We'll talk to you again next week.