Building the world's most popular data science platform

Everyone working in data science and AI knows about Anaconda and has probably “conda” installed something. But how did Anaconda get started and what are they working on now? Peter Wang, CEO of Anaconda and creator of PyData and popular packages like Bokeh and DataShader, joins us to discuss that and much more. Peter gives some great insights on the Python AI ecosystem and very practical advice for scaling up your data science operation. Sponsors: DigitalOcean https://do.co/changelog – DigitalOcean’s developer cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99% uptime SLA, and 24/7/365 world-class support to back that up. Get your $100 credit at do.co/changelog https://do.co/changelog. Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Featuring: Peter Wang – Website https://www.anaconda.com, X https://x.com/pwangChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Anaconda https://www.anaconda.com/PyData https://pydata.org/NumFOCUS https://numfocus.org/Jupyter https://jupyter.org/Numba http://numba.pydata.org/JAX https://github.com/google/jaxMasakhane https://www.masakhane.io/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  And ultimately, we don't need Docker. Actually, it turns out if you build things in the right way, you can have incredibly portable, side-by-side installable native executables that work just fine. And that's essentially what the Konda system is about.

• 0:13Speaker 01

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at Change Law because of Rollbar. Check them out at rollbar.com. And we're hosted on Linode cloud servers. Head to linode.com/changelog.

• 0:31Speaker 01

  This episode is brought to you by Digital Ocean. Droplets, managed Kubernetes, managed databases, spaces, object storage, volume block storage, advanced networking like virtual private clouds and cloud firewalls, developer tool with a robust API and CLI to make sure you can interact with your infrastructure the way you want to. Digital Ocean is designed for developers and built for businesses. Join over 150,000 businesses that develop, manage, and scale their applications with Digital Ocean. Head to do.co/changelog to get started with a $100 credit. Again, do.co/changelog.

• 1:18Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We're at Practical AI FM.

• 1:44Daniel Whitenack?

  Well, welcome to another episode of Practical AI. This is Daniel Whiteknecht. I am a data scientist with SIL International, and I'm joined as always by my co-host, Chris Benson, who is a principal AI strategist at Lockheed Martin. How are you doing, Chris?

• 2:01Chris Benson?

  I am doing very well. How's it going today, Daniel?

• 2:03Daniel Whitenack?

  It's going pretty good. Yeah, it's got a chance to get outside a bit over the weekend, even if it was just in my yard, doing, doing mowing and yard work and that sort of thing. So good to get away from the screen a little bit. And what about yourself?

• 2:19Chris Benson?

  I tried to do some yard work myself and for anyone who who listened to us a couple of weeks ago, I had the broken rib and I discovered it is not as healed as I was hoping it was. And so I stopped doing yard work.

• 2:30Daniel Whitenack?

  That's no good.

• 2:31Chris Benson?

  And took lots of medication and and thus I'm here and everything is fine.

• 2:34Daniel Whitenack?

  Okay, good. Yeah. The other thing I did was actually some people asked me about my AI workstation build in our Slack channel. So that's up and going. I have that pretty much running 24/7 with some type of model training. But I have a colleague who there's two GPUs on it. So he's running some stuff on the other one. And I tried to set up like I got a new router for my I intended to like put in this new router at my house and set my like other router and bridge mode and like I had this really nice plan and you know, it's VPN access and all of this stuff and all of that completely failed. So.

• 3:12Chris Benson?

  Of course.

• 3:13Daniel Whitenack?

  You know, more network things in the in the future. But yeah.

• 3:17Chris Benson?

  Never works the first time.

• 3:18Daniel Whitenack?

  Exactly. Exactly. Well, speaking of very practical things and also, you know, setting up environments and all of those things, we're really excited today because, you know, Chris, we've we've mentioned so many times on the podcast, we've mentioned Anaconda or Conda in all sorts of contexts, because, you know, it's just a pillar of the sort of data science and AI world ever since, you know, I've known about data science, I've known about Anaconda. So I'm really excited that today we have Peter Wang, who is the CEO at Anaconda, joining us today. Welcome, Peter.

• 3:57Speaker 00

  Thank you. Thank you for having me on the podcast.

• 3:59Daniel Whitenack?

  Yeah, great to have you. Before we jump into all things Anaconda, it'd be great if you could just tell us a little bit about your background and how you eventually crossed paths with this data science world and ended up helping found Anaconda and all of those things.

• 4:15Speaker 00

  Uh, yeah, I'll try to give a somewhat of an abbreviated version of the story.

• 4:18Daniel Whitenack?

  Sure, that'd be great. It's a lot to cover all at once.

• 4:21Speaker 00

  It's a lot to cover. It's about twenty something years here. But uh, I started, actually, my academic background is in physics. And so I was doing-

• 4:27Daniel Whitenack?

  Ah, same here.

• 4:28Speaker 00

  Oh, great, yeah. Quantum information and quantum computing and physics. And uh, when I graduated, I decided to go into the software industry, uh, join a startup. I've always been coding. I've been coding since I was a very young child and I've always loved it. But, you know, just given it was the dot com era, I thought that would be, you know, a good time to try my luck at that whole thing. But anyway, uh, long story short, I ended up in Austin, Texas, working at a Python-based scientific software consultancy. And there, and that was kind of the early to mid two thousands. And it was there that I really, I'd always been, I guess since ninety-nine, it was when I first started getting into Python. But then, over the course of the two thousands, when I started doing a lot of work in the scientific Python, scientific numerical computing with Python.

• 5:12Speaker 00

  That was, you know, basically I was using Python before NumPy existed, right? It was still numeric or numarray and those kinds of things. But basically, over that time, I started seeing through more and more of my work and my consulting jobs that Python was being adopted in places that were outside of like what you would consider scientific computing. And so we were initially thinking, oh, this is like a cooler alternative to MATLAB. And then we started seeing it in business environments, started seeing it, you know, going to like, uh, investment banks and just see it being used everywhere to do financial modeling. And I'm like, okay, well, this is very interesting, right? And so around just the turn of the decade, I guess, the last decade, the twenty teens, you know, I think many of us in the community were having this realization. But I personally, as an entrepreneur, you know, I started realizing,

• 5:54Speaker 00

  you know, hey, with the joint evolution or transformational things of cloud computing as well as big data, the big data creates a demand to do bigger analysis, if you will. And traditional SQL is not going to cut it when you have that much messy, you know, the four Vs of messy big data. SQL doesn't cut it. And then if you look at what hedge funds do, and hedge funds are usually kind of the leading edge of numerical modeling technology, they're all doing very sophisticated kinds of predictive analytics. And Python, they were choosing Python all the time. So- There was that. And then I realized with with cloud computing, it meant that every company on the planet would be able to rent computers to do their data would end up in the cloud and they would rent computers to do massive scale, supercomputing scale jobs.

• 6:35Speaker 00

  That would have been inconceivable before because you'd have to ask IT, wait five years and they might build you a data center, right? And so with these two combined transformational forces happening, I realized, aha, Python is actually a thing we should be pushing for data analysis. And so in 2012, when I founded Anaconda, of course, we at the time it was called Continuum Analytics. I also started the PyData kind of community movement and kind of that branding, if you will, it's kind of a branding exercise. Because all the tools are basically SciPy, right? And I mean, of course, you have pandas in there and a few others, you know, stats models, things like that. But ultimately, it was basically SciPy rebranded for a business audience. But that pushing of Python for data analysis was something that me and a few other people, I'd say we were the pioneers, we're on the vanguard.

• 7:17Speaker 00

  And everyone else was looking at us like we were weird. It was either Hadoop, or maybe it was going to be R as the Scion for, you know, the successor to SAS. But we came on the field and I was very vocal at that time, like, hey, Python's the thing. Python's awesome. And then people will be looking at you weird. But now I think we've proven that Python is a thing and it's a good and useful thing. Of course, it has its warts. But anyway, that's kind of how I came to founding the company. And we created actually the Anaconda distribution as sort of like a thing we had to do. We were interested in creating distributed computing and interactive visualization and compilers and optimizing all this and that and the other. But we couldn't get any of that into people's hands because they were still struggling to install SciPy or Matplotlib. So we built a distribution called Anaconda to make that problem, particularly nasty problem, even to this day, easier.

• 8:02Speaker 00

  And so that's just kind of continued to be a thing. So, yeah.

• 8:05Chris Benson?

  So to take you back just a moment, I'm kind of curious, just for a perspective, kind of historical perspective. As a big advocate for Python at the time, as you're looking at this and you're kind of saying, hey, it's going this way. I'm just curious. If you can take yourself back, what was it about Python then that was really driving that passion? Why was that passion not with MATLAB or with R or with some of the others? What was it that really grabbed you? I'm just curious how you got motivated on that.

• 8:33Speaker 00

  So I am kind of a, like I said, I've been programming for a long time. I know a number of languages. I started with BASIC and Logo, as so many people do, you know, children of the 80s. But I learned a lot of languages. In fact, my professional work had been in C++ and I was a huge performance nerd and all that stuff. When I first included Python, it was, I think, version 1.5.2 and it was a slash dot post. And I was like, you know, I'm tired of all these slash dot posts on Python. Let me finally take a look at what it is because Perl 6 looks like it's going to take a little while to get here and I might as well play with Python, right? But when I started using it, I realized, oh my gosh, this is very nice. This is executable pseudocode and furthermore, I can use Python to script my low-level C++ graphics engine and be way faster than sitting there beating my head into like C++ templates,

• 9:15Speaker 00

  which back at that time, they were not very, like, they were not very well supported by any of the compilers. So I could get more abstraction, I could prototype very fast, and it was just pleasant to work in. I felt like I could do stuff without like slogging through a pile of like syntax. So that kind of ease of use and that friendliness, even for a seasoned programmer, professional programmer like myself, was very nice. And what ended up happening over the course of the 2000s was that you could see people who are not traditional programmers, and this is going to be a very important thing as we talk about practical AI and kind of talk about the demographics of the next generation of practitioners. The people who made Python good for science and data science and all this, they're not professional software developers, right?

• 9:57Speaker 00

  So Jupiter was born as iPython, created by, well, I mean Fernando, for instance, is an applied physicist. There's a lot of physicists actually banging around in the Python ecosystem.

• 10:08Chris Benson?

  It's a trend.

• 10:09Speaker 00

  It's a trend, it's a thing. And then you've got like, you know, Jake Vander Plaats, another contributor to like scikit-learn and creator of Voltaire and whatnot, he was an astronomer. You've got like Travis Oliphant, creator of NumPy, my co-founder at Anaconda. He was a double E, you know, he was an assistant professor when he made NumPy. So the tools that were built in Python for doing data science and analysis and things like that were built by people who could take what was there in the Python ecosystem, modify it so it's fit for purpose, so it was pleasant to use for what they wanted to use a computing system to do. That's very different. That's sort of the product development is coming inside the, it's inside the head, right? It's coming from within the person making the thing. So like, for instance, Wes McKinney, the creator of pandas, developed when he was at a hedge fund, right? And he's

• 10:54Speaker 00

  like, this is the thing I need to like, to like bang around my data frames. So that's a motif in the Python ecosystem that took it from just kind of this cool, fun, easy to learn language that was, you know, embodying Guido's vision of computer programming for everyone and a nicer scripting system than Bash. We really, I think the scientific Python ecosystem took it to this next level of like, okay, this is the numerical quantitative computing system that we all wish we had, right? And it's unencumbered by like 30 years of legacy crap. It's on a language that's very nice to use, easy and approachable, but under the hood, you open it up, you know, this like nice little like approachable little Honda Accord. You open up

• 11:35Speaker 00

  under the hood, it's this incredibly modular like warp drive unit, right? You can bolt on things like SWIG, really weird pieces of software that do incredible things, automatically generates wrappers for any of these other languages, right? So you can bolt onto Fortran, C++. We have a just-in-time compiler we built for it. It's a really interesting, upgradable piece of kit, right? To use a British sort of term. So that's, I think, what's given it's kind of the sticking power that has now kind of, you know, now we've seen that evolve more and more. And so now it's this really cool community, almost like a standard language that's very modular. And those are some of the

• 12:09Daniel Whitenack?

  Yeah.

• 12:10Speaker 00

  key aspects of it.

• 12:11Daniel Whitenack?

  Well said.

• 12:11Speaker 00

  Sorry, that was a very long, long-winded answer, but you get me talking about

• 12:14Daniel Whitenack?

  No, it's great.

• 12:15Speaker 00

  true Python evangelism and I could just, you know, I can't stop.

• 12:17Daniel Whitenack?

  No, I totally relate. I remember, yeah, it was late in my PhD when I was working on a piece of scientific computing code in Fortran. And I remember that was the first time that I was introduced to Python because I went to go see some of our collaborators. I spent like three weeks with them working on some, some experiments and I was like pair programming with this guy and he's like, oh, you know, I'm just gonna run a few things. And he had this, you know, Python script around, you know, running all of this Fortran stuff. And I remember just being like stunned. I'm like, oh, why haven't I been doing this? This guy has like a superpower of some kind.

• 12:54Speaker 00

  Right.

• 12:55Daniel Whitenack?

  But, uh, yeah, I'm wondering, you know, maybe moving from there, like you talked about how Like the numerical and scientific and data science tooling that was developed in Python, that was kind of developed by this sort of, these groups of whether it be scientists or people in industry doing data analysis, or not really maybe the traditional kind of programmer types. Do you feel like that contributed to some of the maybe struggle or inconsistency around like managing environments and installs and like how people like managed all of their stuff? Was it just because people like had a bunch of different views on those things? Or what's your perspective on like?

• 13:38Speaker 00

  Why is packaging so terrible in Python, right?

• 13:40Daniel Whitenack?

  Yeah, yeah, exactly.

• 13:42Speaker 00

  Well, so I can answer that question, but first I would just have sort of a meta critique of the question, which is any system which does the number of things that Python does, I would assert, has similar kinds of software dependency issues.

• 13:54Daniel Whitenack?

  Yeah.

• 13:55Speaker 00

  Okay? And even systems that don't. So JavaScript, which is all interpretable, right, has created an absolutely nightmarish scenario of packaging. And even though they have a vendor of the world, everything sits in the subdirectory, vendor of the world kind of approach, even though, you know, it's all pure, just pure text, there's no compilation, you can't go and gripe about Fortran ABI specifications. I mean, it's just text, right? But even JavaScript, everyone knows packaging is a nightmare. So even under the most non-compiled sort of scenario, you end up with a packaging morass because you have an ecosystem that is able to build on each other sort of things.

• 14:30Daniel Whitenack?

  Yeah, can do so many things.

• 14:33Speaker 00

  Right. But in the case of Python, think about what we're talking about, gluing to Fortran code. All right, well, you know, maybe Perl has nicer packaging. Where's the Perl glued to Fortran code? And where is that being used on a million nodes, right? It's not. So I think there's a thing to recognize, like the reason why your duct tape has all this crap stuck to the back of it is because it's sticky, it's duct tape. So Python is a glue language, of course gets a bunch of cruft glued into it, and now we have a much harder problem to solve because we do speak to so many different things.

• 14:59Daniel Whitenack?

  Yeah.

• 14:59Speaker 00

  Okay, now that being said, I think fundamentally, one of the reasons why Python packaging is difficult, has been that packaging was treated as a second-class concern by the BDFL, right? So Guido clearly admitted very early on, he just didn't think about it that much and didn't really, you know, it wasn't a big problem for him, it wasn't exciting or interesting for him. And he's not really apologizing for it, he's just saying this is kind of the way it was. It was someone else's problem to go and figure out, whether it was the fill up EB with step tools, or you know, the disutilities, whatever it might be, it was really not a thing he was super interested in. And then when we came along, actually, there's a the very first PiData workshop I put together in March of 2012, three months after we started the company. And Guido was working for Google at the time. He came by and stopped by. We're all very excited to see him.

• 15:42Speaker 00

  People gave him a lot of crap about like, hey, when are we going to get a matrix multiplication operator? But then we also asked him about packaging. We're like, hey, can you help us get the core packaging folks in the core dev to work with us on packaging stuff, you know, in the scientific Python ecosystem? Because it's just been a mess for a very long time. And his answer was, look, it's possible that your needs are so exotic that you should just go do your own thing. Like, don't worry about it. Just solve your own problem. So we're like, OK. So we did. You have to understand, by that time, the people in the SciPy ecosystem had been fighting various multiple genealogies and multiple generations of Python packaging tools for 10 years, right? It had never really been great. And so by that time, we were like, OK, let's just solve this once and for all. And we realized something very fundamental.

• 16:26Speaker 00

  And this is true for any system. This has nothing to do with Conda. This is true for for any system that touches compiled code. And this is now the second part of my question, the second part of my answer. The first part of my answer was that Guido didn't really care, so it was kind of festering. The second part of it is that software development, basically every single operating system that is a PC-based operating system sucks. And so we have inherited the long shadow of 1970s technical debt, right? If you're on Linux, any of you guys heard of a thing called Docker or use a thing called Docker? Maybe just a little thing, right? Love Docker. Why did Docker come about? Docker, what's that? Because you even have RPM. I mean, what Linux distro doesn't have a package manager? And yet you use Docker?

• 17:07Speaker 00

  Because the concept of having static linkage or dynamic linkage between software libraries, even though we have a tried and true robust dynamic linker system, it's terrible, right? You go to Macintosh, OK? So maybe you use Homebrew, right? That's kind of the preferred kind of package manager on there. And of course, eventually, Apple with App Store will kill all of that third-party stuff. But but for now, we have Homebrew. Well, you start building stuff with Homebrew. What's what if you want to have multiple different versions of libraries? And you have to build framework builds of them. And those framework builds are incompatible with each other. And so they need access to like the raw GL context. God forbid you need to do any of that stuff, right? So every single system in Windows, notoriously DLL hell, right? Literally is the Windows DLL format, right?

• 17:49Speaker 00

  And they do Windows side by side, have all sorts of things that they do. It's an unsolved problem because we've inherited the C linker and loader. And ultimately, that is part of the core ABI spec of the underlying operating system. So now, if you're just, you know, a couple of like physics turned software nerds like me and Travis, you're like, okay, holy crap, how do we ship LVM with our dynamic compiler? How do I ship a pile of JavaScript dependencies with like Bokeh and our interactive, you know, web this stuff? How do I do any of these things? How do we ship? For instance, you know, people, do you guys use Jupyter Notebooks? Or have you, you're familiar with Jupyter Notebooks, right?

• 18:23Daniel Whitenack?

  Yeah.

• 18:23Speaker 00

  And have you ever converted them? Have you ever used MB convert to convert from a notebook to like a PDF or something like that?

• 18:28Daniel Whitenack?

  Yeah, yeah.

• 18:29Speaker 00

  Did you know, fun thing, that it uses a thing called Pandoc underneath? Which has an embedded Haskell compiler. Okay, so like, in order to ship MB convert on Windows, you have to go and build a Haskell runtime on Windows.

• 18:42Daniel Whitenack?

  What a web of things.

• 18:43Speaker 00

  What a web of things, right? But this is the kind of nonsense that we end up having to deal with. Um, I call it nonsense, but really it's just, we realized that it was more than just the C ABI linkage, whatever, and this was back in 2012, pre Docker. But even Docker then, on Windows, not a really great story, right? And ultimately, we don't need Docker. Actually, turns out if you build things in the right way, you can have incredibly portable, side by side installable, native executables that work just fine. And that's essentially what the Condor system is about. It's about, rather than wrapping everything up in a hermetic sort of Docker environment, we create a very simple specification of what packages do you want? And then we have a recipe system that then has a build system behind it

• 19:26Speaker 00

  that is able to build native binaries for every single platform, optimized for every single kind of hardware version that we support. And that's ultimately, you know, when it works well, it works well. When it falls over, then it can be a little hard to untangle exactly what the problem was. And we're working on that, of course. But that's ultimately the motivation there and why it's terrible. I think we inherited the long shadow of the 70s. C, you know, linker loader, I'm looking at you. And then also we inherited some of Guido's, you know, preferences in language development. And I'm not blaming him, I love him and I'm so grateful for what he's done with the language. But anyway, you guys asked, so that's the honest answer.

• 20:14Speaker 02

  What up nerds? Jared Santo here, your humble producer. I'd like to tell you about something new we're beta testing around Practical AI. It's a membership program, which we think could be really valuable for the whole community. We call it Changelog++ and it's the best way to directly support Practical AI and all of the podcasts, videos, and other stuff we create here at Changelog. We have big plans and ambitions for this, but we are experimenting for now to make sure there's interest. That means when you sign up today, you get Practical AI and whatever Changelog shows you listen to now, except no ads. I guess that means this part you're listening to right now, it'll be gone. We also have some extended episodes planned, bonus content, merch store discounts, and a lot of ideas.

• 20:55Speaker 02

  But since it's such early days, we are offering memberships at a 40% discount for early adopters. That disappears at the end of August, so head to changelog.com/plusplus to join today. Lock in that discount, get closer to the metal, and make the ads disappear. Once again, that's changelog.com/plusplus. We'd love to have you supporting us as a member.

• 21:27Chris Benson?

  Okay, so I'm going to start off the next section as we're starting to dive into Anaconda itself. With the obvious question you've probably been asked a billion times, I get Python, Snake, and Anaconda, but I am curious, why specifically Anaconda? And if there were any alternative names that might have been fun that you could share with us?

• 21:46Speaker 00

  Yeah, well, it was. So I will give you the origin story. It was at a moment at that PyData workshop when we were looking at trying to promote a Pythonic alternative to Hadoop. At the time, there was a distributed MapReduce system built around Erlang that had come out of Nokia. It was called Disco. And so- I was like, wow, this is nice. It's a nice Pythonic MapReduce if people really want MapReduce. And we should get this to people. And then I realized I had this sort of like moment of of truth. I was like, wait, we can't even ship SciPy to people after 10 years, 12 years. How are we going to ship Erlang runtimes to people, right? And so I turned around and I looked at Travis, we're sitting in the back of the room and I looked at him, I said, we need to create, if we're going to really do this, we have to, you know, a bunch of stuff is going to have to go rolled in. We need to create a new distribution of Python for big data.

• 22:30Speaker 00

  So let's call it Anaconda because it's a big snake, right? It was no, it was like, literally, it happened in a flash. It was just, there was no great deliberation about this.

• 22:39Daniel Whitenack?

  And when you're saying, so just for listeners who are maybe, maybe a little bit newer to this world, when you're saying a specific distribution of Python, so what does that specifically change about your local environment? That you're using this distribution of, is it just around the, like, usingconda instead of pip? Or is it a whole different Python interpreter? What are the sort of specifics when you say specific distribution?

• 23:03Speaker 00

  Yeah, so the specifics are that we build the Python interpreter itself. And we build the installer around it and then all the packages. Because when you build packages that have binary or C extension module or C++, they have to be built with a compatible compiler set as what you built the interpreter with. Otherwise you segfault, right? And then every subsequent package that has C dependencies needs to be built with the same compiler set. Otherwise you end up with runtime segfaults, which is no fun for anybody. So we basically have created a normalized build system. It's like a Lego ground base plate that has equal spaced studs and is level. And then you can put everything on it, but you need that first base plate. So that's what the Anaconda runtime really is. So it's a Python interpreter that we've built.

• 23:47Speaker 00

  And then you can either get Miniconda, which is just that Python interpreter with the standard library and theconda package manager. Or you can get Anaconda, which comes with like 250 or something, 220 packages pre-built as well. So pre-populated base plate. But the idea of the Anaconda system is that usingconda, you can then install packages into this that fit on that base plate and fit with each other. That being said, there's nothing stopping you from using pip. I mean, I use pip, right? You can pip install other modules in, but if they have C dependencies or pre-compiled binary components, it's better to install those withconda, because then you know that those are compatible. And especially as we're talking about in the context of AI and ML,

• 24:29Speaker 00

  these like that, obviously accelerated hardware is a deeply important topic, right? And so you want to get the version of the package that's built for your piece of hardware. And sometimes when people go and build binaries and they make pip wheels available, they have to build those with the lowest common denominator of hardware flags. So you might be paying $3,000 for Xeon processor, but only getting basically a $500 or $200 Xeon processor worth of capability because certain flags are not turned on, right? And this is the kind of thing that is not important for like a huge number of users, but really important for some users. And of course, as time rolls on, more and more important for everyone. So that's what that means. We can install Anaconda into a user land directory. You don't need admin permissions, and you can install stuff. It's all self-contained in there. If you don't like it, you can blow it away with one directory remove command.

• 25:14Speaker 00

  And it stands separate than your system Python, than anything else. So that's really what... So people useconda with Docker all the time. It's a very common pattern.

• 25:22Daniel Whitenack?

  Yeah, that makes sense. So a lot of these packages, like you're talking about, you know, scikit-learn, or pandas, or NumPy, or these other things, generally people think of this sort of landscape of open source data science tools. So is what portions of the sort of Anaconda system are open source? And how did you, when you kind of entered into this journey, how did you go about navigating that open source landscape where, you know, there's all sorts of things that have all sorts of licenses and, you know, building a product around open source is kind of an interesting thing, especially, I think it's, you know, it seems to be a lot of things like that are trending now, but I think, you

• 26:05Daniel Whitenack?

  know, when this whole field was getting started, you know, there probably weren't as many examples of businesses built around like open source. So yeah, how do you all think about that? And how did that grow as you were, as you were starting out?

• 26:20Speaker 00

  Yeah. So Travis and I are both ardent supporters of open source, because I think it leads to open innovation. So for me, that's the, almost like open source is almost a means to an end. It's not an end unto itself. So if something's open source, but it leads to... Single vendor innovation, that's still bad. And, and we have examples of that now, actually, in the burgeoning AI space. And I would encourage people to look beyond merely the kind of astroturf like, oh, well, this is open source. Well, all the contributors come from one company, whether it's a small one or a big one. It's like, who's going to get in there? Are you going to accept patches? Can we really fork it? And you, you're not going to sue us for some thing, you know, like there's, for me and Travis has been watching what the open scientific software ecosystem was able to produce. That collaboration was so generative and so amazing.

• 27:05Speaker 00

  We want to ensure that that would endure. So in building a business and building, trying to build business models around open source, that was part of our entrepreneurial exploration of that. Can we build a company and have a good one that fosters and, and, and sustains open source innovation? So everything in Anaconda is open source. The recipes are open source. I mean, the, the condomat package manager is open source. That's always been the case from the very beginning, right? So we don't make our money by holding back any of that stuff. We started with a support and consulting kind of model because again, Python for data science is kind of a new thing. And there were plenty of our projects were, were nascent, right? Whether it was Numba or whether it was, you know, any of these other projects. We had a lot of consulting demands for those kinds of things, so we did that.

• 27:46Speaker 00

  As we shifted into a product oriented mode, what we realized was that enterprise pains, especially enterprise pains addressed by software, they're not really about proprietary closed source features. They're about enterprises wanting to have roadmap transparency, having a vendor, a throat to throttle when something goes wrong. You know, all these other kinds of things. So Red Hat demonstrated how you can do this in a fairly sustainable way. So we, our first product that we actually shipped was a package server that, you know, you could have, an IT guy could say, you know what? I don't want GPL packages coming in here because legal, right? So I want to have all the data scientists internally be able to point to my internal mirror of the Anaconda ecosystem. But I get to blacklist all of these, or zero out all of these GPL packages.

• 28:29Speaker 00

  I get to set which versions are available in various channels. So the prod cluster that I manage will only ever get package updates from the prod channels. And I get to flip the bid on that one. But the devs, you know, the data scientists who have the sandbox and they want the latest bleeding edge version of something or the other, they can knock themselves out, right? And now they're not complaining that I'm holding them back from their work. So that package repository server, I mean, we still sell that today. It's a very popular product. It's a, it addresses a deep need that enterprises have. So that's kind of how we think about the product development. We also do have an enterprise machine learning platform, just as like a domino or an H2O, whatever kind of thing. So we've had that that we've been selling for a while. But I think that for us, looking at kind of the growth of the packaging demand and the ecosystem, you know, the package server for us is really

• 29:14Speaker 00

  kind of a no brainer enterprise offering.

• 29:15Chris Benson?

  I'd like to actually follow up on that exact thing. And is

• 29:18Speaker 00

  Sure.

• 29:18Chris Benson?

  as you're looking at organizations, companies, businesses out there that are trying to find their way into data science and Anaconda being one of the major avenues on doing that. What is the value proposition that a CIO at some company is, it should be looking at when they're thinking about, do I go Anaconda? Or do I go, you know, some other route? Do we mix and match? You know, because that is a question that companies are dealing with every day right now.

• 29:43Speaker 00

  Right.

• 29:44Chris Benson?

  What should that CIO be thinking about when they're trying to decide whether they want to go with Anaconda or not?

• 29:49Speaker 00

  Yeah. And so there's actually not very much that's competitive with our package, sort of a package E commercial, we'll call the commercial license, you know,

• 29:56Chris Benson?

  Yeah.

• 29:57Speaker 00

  of Anaconda, because ultimately what we're solving is a very unique but important problem, which is the software supply chain, right? Now you see a lot of companies, it's actually shocking to me, because I've been involved in open source since like 95, right? In the early Linux days. But even today, there are companies that are just starting to understand like, oh yeah, maybe we should figure out how to use open source in a governed way, right? They're starting to have that conversation at the, the CIO and the IT leader level. And when it comes to ML and AI, you know what, it's the ecosystem moves so fast, it's a whole Wild West of things out there. Anaconda is basically like the only company that is out there as your last outfitter between, you know, civilization and the Wild

• 30:37Speaker 00

  West. And so if you actually want a build of NumPy or of scikit-learn to go and run on your customer sensitive PII HIPAA data and not have it just come from some like grad, grad students like, you know, server somewhere under their desk. You have to talk to a vendor. You have to have someone who will actually talk to your legal people, you know, sign us some lines. We're that. We're it. So we actually are compatible with a whole host of other, you know, I talk about how our ML platform is, you know, competitive with things like Domino or maybe a SageMaker, some of these other things. But our package server and our commercial license for the Anaconda distribution is not competitive with those things. It goes hand in hand with those things. So in fact, we have a partnership with Red Hat, a partnership with IBM that we've just announced earlier this year, where our

• 31:20Speaker 00

  package server and those commercial license packages, that is going out to the world via those channels. Because again, there's not much competitive with that. So what the CIO should be thinking about is, how do I govern the software bits that actually run? You know, like this Docker that, you know, this like three gigabyte opaque binary that my data scientist like intern just handed me? How do I, how good do I feel about running that in production, right? If I want actually some transparency into it, if I want repeatability? Like, an aerospace manufacturer was talking to me at PyCon a couple of years ago, and the data scientist and aerospace engineer there, he said, "We have to demonstrate to the FAA that we can run these wing models 50 years. 50 years after the last plane rolls off the line."

• 32:00Daniel Whitenack?

  That's crazy.

• 32:00Speaker 00

  So, so it can't be some Docker file with a run command npm install this, or sudo pip update that, right? You've got to have something that you can point to, and everybody up and down the governance chain feels good about. And right now, that topic of open source governance for ML AI is not a broadly discussed topic, but of course, you guys, you know, as practitioners, understand the importance of that, especially as predictions and predictive models come under more regulatory scrutiny. So, yeah, we have, I think, a relatively distinguished and unique offering in that regard.

• 32:32Daniel Whitenack?

  Yeah, I'm curious. You mentioned a couple of times, and I guess this is a product of kind of this shift in, you know, the first, the hype around data science, and everybody's doing data science. Now, we're kind of all shifting. We all want to do AI, right? And if we, if we do AI, then we get bigger salaries-

• 32:51Speaker 00

  Yep.

• 32:52Daniel Whitenack?

  And that sort of thing. So how have you seen this shift towards AI and wanting to do AI things as, as opposed to sort of maybe just data science? How has that influenced the way that you're interacting with clients and the things that they want to do and the open source projects that you're wanting to support within the Anaconda ecosystem? How has that shifted things and made you think about things differently, or the same? Or what does that look like?

• 33:20Speaker 00

  Well, it's ma- well, without getting too snarky, it's made me very cynical about the tech business press. But maybe I was already at maximum, I already maxed out my cynicism there already, so maybe.

• 33:29Daniel Whitenack?

  Yeah, you're just riding a plateau.

• 33:31Speaker 00

  Now it's now it's flipped around, now it's now flipped around, right? Because I used a sign in to share, now I'm at negative cynicism. And the challenge is this, that, you know, as someone who kind of understands the technology in the space, I think we're very close to some things that are really very close to what we could call AI, right? So there's a real, there's a substance there. But the second order, third order, far field wave of interest and hype is way bigger than what's justified, I think. But at the same time, if we, you know, I mean, GPT-3 is jaw dropping, right? I mean, usually some of these things, it's like, holy cow. And then we, and there's deeper questions there, like, we have to think about, this technology is like, at an intellectual level, it's like a nuclear age revolution. We can't just put this in everybody's hands, there's, we gotta be serious about how we use this stuff, right?

• 34:16Speaker 00

  But, but, but all that aside, I think that what people are maybe sometimes miss in that business level, kind of up-leveling of like AI and all this kind of stuff, is that it is a ladder of needs, or a Maslow hierarchy, right? If your business sucks at basic data management, if you can't even run SQL queries over your stuff, you're not going to get to do data science. And if you can't do data science, your AI is going to be kind of crap. Like, you're not going to do any real AI, you're going to spend a lot of money. No one's going to stop taking your dollars just because of bad data infrastructure. Consultants will take your dollars all day long. They're going to produce some like clickable BI chart and, and charge you a couple million dollars for it. You won't have gotten the value out of it, right? But then, you know, whoever has ever gotten fired for having a bad IT project, it's just somebody else's dollars, right?

• 35:00Speaker 00

  So, again, like I said, I have a lot of cynicism about how this stuff works. But I do think that legitimately, what, so to your, but in a more serious tone to answer your question, in steering the ship in Anaconda and looking at where we make our investments, you know, we support the development of some of the fundamental tools. So we invest in things like Dask, which are the next generation distributed computing in Python. We support things like Numba, which give us more performant, just across the board. It makes the low-level libraries very performant on next-gen hardware, right? So hardware manufacturers like Intel and Nvidia partner with us to, you know, add improvements to the compiler. Pandas, we support fundamental development of things like pandas. So I think my thing is this, I'm fixated on empowering the practitioners and helping them up-level their data literacy across the organization.

• 35:44Speaker 00

  So my investment is not going to be at the cutting edge of the hype. The way I want to steer the community, the way I steer my friends who are movers and shakers in the community is to really think about this. If we are to have this technology be something that's transformative for humanity as a whole, then it cannot become an ivory tower where there's a few acolytes who know how to use a few privileged proprietary systems to go and tell the rest of us what the predictions are. This cannot be how this works. It has to be a democratized transformation of how every business, every person thinks about it. In fact, an underlying thing at Anaconda is that we want to make sure that everyone gets data literacy. That's why we will always have this free capability.

• 36:26Speaker 00

  We don't charge for like a few hundred extra rows on this library or something like that. It's always free and unfettered access because I want every school kid in Bangladesh to be able to model quantitatively in a Jupiter notebook, why some hotshot politician enacted some policy. Everybody, everywhere, math is empowering for everyone. And this is just computational math. So from that perspective, there's a deep moral aspect to my mission and to the mission at Anaconda. Now for AI, ML and data science, the transformation I've seen in the field is that, yeah, everyone talks about AI, but then when you get all the practitioners together, we all know to put the stuff on the side. Like we put the MBA speak to the side and then we all just talk about the real stuff. And it's like, usually data engineering, it's usually, well, software bits, right? Who's setting up the working environment? What version of pandas are you using, et cetera, et cetera?

• 37:11Speaker 00

  GPUs, so people are actually now starting to really model the hardware footprint as they're approaching data jobs, which I think is fantastic. It's what should have always been done, right? It's actually a practice that IT has left behind for like 15, 20 years in the Java era. Now hardware matters again, vectorization matters again, and it's a beautiful thing. So maybe I sort of lost my point here. I'm just like, no, ranting.

• 37:33Daniel Whitenack?

  No, it's really good. Yeah. I mean, I think some of those things that I see, like you're talking about, definitely resonate with me. I definitely think, like, when I first got into data science, like, the hardware wasn't really, I wasn't thinking a ton about it. And I was also, like you're saying, shipping off that, like, three gigabyte container to, you know... DevOps Doug and he was, you know, figuring out and, you know, hating me because it took however long to build and, and all of those things. So it's good to hear you, hear you talk about some of those things. And I definitely see how, you know, a lot of these main components that you're supporting are really fundamental. I was just doing some speech recognition stuff end of last

• 38:15Daniel Whitenack?

  week, over the weekend on Vidya Nemo and they were like, well, you need to install Numba to, you know, speed up some of these like data augmentation things for the, for the speech files, which is like a main component of the thing. It's really driving that. And, you know, that, that really influences the actual, you know, AI training and the quality of that. So these sort of pre-processing things and all of those are really, you know, really fundamental. I don't know if this factors into your thoughts around packaging and distribution and that sort of thing. I know a lot of people are talking about like these model hubs now and like, you know, TensorFlow hub, PyTorch hub, hugging faces, model hub, you know, in addition to the code,

• 38:57Daniel Whitenack?

  there's sort of like, there's the data, there's the code, and then there's these like, these things that are kind of weird and maybe in the software engineering world that are like these different types of data, which are the serialized models and, and influence how this code runs. How do you think about that at Anaconda? And has that, you know, conversation been going on about, you know, packaging and distributing these things? Or are you mostly focused on the, on the code at this point?

• 39:22Speaker 00

  We're still focused on the software supply chain, but of course, I'm very exposed to the kinds of dynamics you're talking about, right? Because I see these conversations happening in the practitioner ecosystem and in the conversations. There's a really important dynamic that's happening here that, without getting too like hyperbolic and biblical about it, that data science and we'll call AI or machine learning, this represents essentially the transformation of the software industry. So for the last probably 40 years, since the dawn of like the PC era, but even prior to that, even a little bit, software developers and software engineers have been able to think of themselves as a distinguished class. Like, we do software. The harder people, they're writing Verilog and taping things out, whatever that means,

• 40:05Speaker 00

  and making chips, they plug it in, power it on. Now then we come in and we do our jobs, right? And then, of course, that's separate from the DBAs, where this other weird class of like Oracle licensed whatevers, right? They just sit there speaking a bunch of weird SQL all day long. This deconstruction of the information system into hardware, software and data management is deeply unnatural. And it's actually something that was not the case. If you go back to the Norbert Wiener and to like the early cybernetics era, no one thought about it that way, right? Do you listen to any of these, like the founding fathers and mothers of the space? It wasn't like, oh yeah, well, I'm going to focus on hardware, right? But the PC era and then everything that came afterwards and the server side on microprocessors,

• 40:46Speaker 00

  all these things led to this deconstruction of an information system into these three primary axes. Yeah, I guess it's a decomposition into three axes. And what we're seeing again now with data science and certainly with ML and AI is that we now have a synthesis again. We're forced to do a synthesis. We have to understand the runtime. And actually, the runtime characteristics of your software is data dependent, right? That's, how weird is that? You imagine talking to Java head 15 years ago to say, well, okay, Mr. Java architect, check this out. If I pass in certain values out of this database, your code runs 10 times slower, right? That doesn't happen because you write a crud system, it pulls a row, does some crap and pushes a row and it's done, right?

• 41:28Speaker 00

  So this idea that, okay, runtime performance, so the hardware footprint is dependent on data. And additionally, correctness is value dependent. Can you imagine writing a unit test? And you, now we have these for models, we have model tests, right? Prior to models, it was all just code. Can you imagine a unit test where it's like, well, one plus one is two. And so this function, my add function works, but it only works for even integers, right? Like, that's weird, right? And yet we know that when we build these AI systems, these models, their, their performance, their correctness is actually value dependent. And this is a point that no, I don't hear anyone else making, maybe because I come at it from a physicist's perspective and I think about deconstructing everything to fundamentals,

• 42:10Speaker 00

  but deconstructing the computation concept into fundamentals. For the last 40 years, we've had value independent processing, right? And Jim Gray's written papers about this and people have talked about this, but like your average coder nerd doesn't think about this at all. They're like, I'm a software dev. I learned this thing, I'm going to learn, go this year, I'm going to do something else next year. But the whole field of software is going away. It's melding into, we might call it model development, we might call it something else, but I call it value dependent or value sensitive computing. And now your management of your upstream data is as important as managing the upstream code. Right? So the previous approaches to data management don't work anymore. But of course, checking in every row of a database into Git doesn't work either. So we have to develop an entire new set of practices for this new industry.

• 42:54Speaker 00

  All of the previous components, those axes are important, but they can be no longer be seen as separable, right? They're now integrated. So anyway, that's, that's my welcome. Thank you for coming to my TED Talk about that. But I think that really, that's the lens I look at all this stuff through. So it's no wonder that we have model hubs emerging. But I think the management of those things and how we talk about versioning of data, the model performance and characterizing it, all of that is a nascent and emerging area. That'll be fascinating to watch how that really goes as it meets production in the real world.

• 43:47Chris Benson?

  Okay, so a few minutes ago, we were touching on deployment. And I know Daniel made his DevOps Doug allusion there. And so I actually want to go back to that for a moment.

• 43:58Daniel Whitenack?

  Someday, I really hope DevOps Doug listens to this.

• 44:00Chris Benson?

  He's gonna get hate mail.

• 44:01Daniel Whitenack?

  Just so you know, Peter, Doug was my first like DevOps engineer that I worked with when I first started out at a startup. And, you know, he taught me all sorts of great things. Anyway, go ahead.

• 44:11Chris Benson?

  Yeah, we're, we're gonna get our first practical AI hate mail, you know, from, from every Doug.

• 44:15Daniel Whitenack?

  I should contact them and have them on the show.

• 44:18Chris Benson?

  Okay, so here's my question. I'm thinking about Anaconda, kind of again, in the organizational structure and thinking about, you know, having to put software together and move on to deployment. And, you know, one of the things that we're seeing a lot now is using Python in doing the data science and doing the modeling and stuff. But we're seeing a kind of a move toward deploying in other languages, you know, where you may take a model and do that. And I'm kind of wondering what your thoughts are on how are you thinking? Like in a world where someone may, for performance reasons, maybe once upon a time, or maybe still now, they were deploying in Java or C++, but maybe they're thinking of Go or Rust just for pure performance issues. How do you see it fitting into that in terms of pipeline? Do you think that's not necessary?

• 45:03Chris Benson?

  What are your opinions?

• 45:04Speaker 00

  Wow, yeah, that is a complex topic. So depending on which framework you're using and what you're doing with that. Compiling down is always something that's going to be a part of the Python ecosystem, right? I mean, Daniel mentioned Numba before, like, you don't have to convince me about compile down, right? There are times when you just need to go lower. Right. What pains me, though, is this idea that there are things that, because Numba goes not down to C, it goes to machine code. Like, we literally are generating, we're skipping a level. We're going from Python to machine code, right? So in other places, I think in cases like TensorFlow, and there's tools like Jax, there's a whole bunch of stuff coming out now where you can go from high-level Python to much lower-level runtime primitives. And I think that's fine. I think when people are doing rewriting, so compile down is something different from a translational perspective than rewriting,

• 45:49Speaker 00

  right? And I know that, for instance, when people, just the other day, someone was complaining to me about the fact that they built some models in Torch, and then they have to basically to go to TensorFlow server, and they have to rewrite everything in TensorFlow. That's deeply inefficient. It shouldn't have to be done that way. And so I think that, and the problems to solve there are not monumental, I think. I think they're mostly ecosystem tooling and some of these kinds of top things that we will solve in time. So I hope that Python compiling down is not an issue, and we'll keep doing that. Where people do feel a need to rewrite, I'm not sure what all they're doing in the Python. I think one of the problems with the Python's explosive growth over the last how many years is that there's simply not been enough instruction about idiomatic, how to think vectorially,

• 46:31Speaker 00

  how to do idiomatic things in Python, or do things in Python in an idiomatic way that's faster. I mean, I just see all sorts of code in the wild. That's just like, oh, you know, that's, you don't have to do it that way, but you don't have time to educate everyone. But maybe we should, right? Because then what happens is you end up in organizations, businesses move at their own cadence. And now you've got a data science team that's all relatively green. They write some code, it's slow. The IT person, the IT team, the software dev team is like, well, that's Python. We know it's slow. Let us rewrite it in Go, because I kind of like, really, really like to use Go. And they do it, and they don't have to. And here's the cost of doing that. Now your iteration, your cycle time is way slower, right? Now when something goes wrong in production, you need to get two people on the line, and then where'd the translation go wrong, right?

• 47:17Speaker 00

  I think I still, back to that point about what is the mission? The mission is to make data science literacy widespread and to empower everyone to ask questions of their world and to be able to use all this powerful infrastructure. If to do that, they have to go and hire a dev team to rewrite their stuff from Python and to go or Rust or God knows what, then we failed in that mission. Right? So I do, I don't have a language bigotry of like, it must be Python everywhere all the time, but it's like, I would like to make sure that, you know, Brad Victor has this concept of immediate connection, right? I want the data scientists, when they're in a Jupiter notebook or in a dev environment, when they're doing data exploration, I want them to be able to feel like they can round trip and that's on their own terms. That's a really important thing.

• 48:00Speaker 00

  So I think, I think hopefully in time, we'll get, we'll make sure that remains a possibility for most cases.

• 48:05Chris Benson?

  Great explanation, by the way. Thank you.

• 48:07Daniel Whitenack?

  And I think that kind of leads into something I wanted to ask. You kind of started talking about some of the pain points that still exist between, you know, maybe data scientists and engineers, or maybe the, this sort of gap in data literacy and these things. I was wondering, as you, and I also know that Anaconda does, you know, a state of data science survey, and of course, you deal with all sorts of people throughout the industry. I was wondering if you could talk a little bit about maybe certain things, looking back from maybe 2012 till now, that you see as really encouraging things in terms of data science tooling and where we've come, and then maybe a few things that are still really

• 48:50Daniel Whitenack?

  open challenges that we haven't been able to solve yet.

• 48:53Speaker 00

  You mean specifically in the tooling?

• 48:55Daniel Whitenack?

  Yeah, or just in data science workflows, I guess, that you see. Yeah.

• 49:00Speaker 00

  Yeah, let's see here. In data science workflows, I think a lot of like some table stakes things since 2012 have been resolved. We have much more capable, just in terms of like, you know, input handling and just like a lot of the basic day-to-day quality of life stuff for data scientists, I think, has improved. People have settled on some sorts of tools as standard and they're good. So like using Jupiter notebooks, which I know gets a, there's a, there's a mixed feeling about notebooks, right? And I can go on a length about them, but I think in general.

• 49:34Daniel Whitenack?

  We had Joel Gross on the show.

• 49:36Speaker 00

  Oh, yeah, yeah.

• 49:37Daniel Whitenack?

  We've had all perspectives.

• 49:38Speaker 00

  All perspectives. You know, I like to talk about threes and I think that the notebook rolls up three different things into one and in doing so unfortunately confuses the crap out of everyone. Cause everyone thinks it's something else, you know?

• 49:48Daniel Whitenack?

  Right, right.

• 49:49Speaker 00

  But basically I think that at least with notebooks, the idea that people can do somewhat literate programming is way better than if it was a bunch of opaque code and then a PowerPoint. Right? So for all of the hate and all of the like, oh, line number is out of sequence and like this, like, I don't know, you just gave me a notebook. I don't know what to do with it. For all of that hate aside, I think notebooks are a net good because they show people, ultimately, again, back to that data literacy thing, it gets people excited and interested. And here, here's another thing about Python. It's good that it's Python that has, that is in these notebooks. Python is very accessible and readable, even if you don't know how to write it. So if you're a business analyst who only does Excel and someone gives you a notebook with a little bit of Python code, you can actually still inspect that Python code and say, hey, it looks like you're pulling

• 50:32Speaker 00

  last quarter's data instead of this quarter's data in this particular thing where you sliced it here, but you use zeroed out all of this zip code. Why did you do that? That bit of accessibility and communication collaboration, collaboration, kind of vernacular power should not be underestimated. Imagine it was a pile of JavaScript. How many nested braces and functional embedded callbacks and some other crap? I mean, your, your business analyst has no hope of understanding what the heck you're doing, right? They don't know these 20 lines are just to try to strip out some ending semicolons.

• 51:01Daniel Whitenack?

  I, I totally agree. Yeah. There, there's actually, I've mentioned it a couple of times on the show, this group called Masakane, who is promoting new baselines for machine translation for African languages. And they're involving local communities in that and. They have a whole paper about this, how they like develop the community and all of that.

• 51:18Speaker 00

  Oh, it's wonderful.

• 51:19Daniel Whitenack?

  But a really central, a central piece of that was Jupiter notebooks because they wanted to involve, you know, local communities in the work. And of course, like you say, it's not like you're just gonna go up to a new group of people and say, hey, you know, clone this GitHub repo and like, you know,

• 51:33Speaker 00

  Yeah.

• 51:34Daniel Whitenack?

  run this bash script and like all this stuff. And so they were able to, you know, utilize notebooks and specifically like hosted notebooks or like Colab and, and all of these things to really like get people going and like, hey, you just opened the notebook. There's like notes in there. There's explanation and you can just like go. And of course, you know, you want people to, to advance from there and to, you know, really dig into things when things are, you know, there's weird behavior or something. Maybe you kind of learn some new things. But yeah, I was really impressed with their usage of that. And I think that resonates with, with what you're saying.

• 52:07Speaker 00

  Well, it's, the web has become unwritable, right? Let's just be very clear. Like, I don't know, last time you guys set up a website from scratch? I mean, you have a website for the podcast, obviously. Maybe you had a web dev for it, I don't know. But to set up a website from scratch, forget putting widgets in there, forget embedding, you know, interactive graphics, just a website from scratch. Most data scientists won't be able to do that.

• 52:27Daniel Whitenack?

  That's true.

• 52:27Speaker 00

  I mean, just configuring NGINX or setting up Apache, uh, you know, getting SSL cert in there, doing all this other, you know, happy, like whatever, like that's impossible, right? And even for a dev like myself, it's really just annoying. So what the Jupiter notebook did, believe it or not, a lot of the value is simply making the web writable, making a writable web technology accessible for people who were not even programmers, so are not familiar or comfortable with the shell. That's the other thing that's true about data scientists. A lot of them are not comfortable with the terminal at all, right? A lot of them are on Windows. And now they can build websites on Windows with interactive widgets, running massive skill computation. Holy crap. Let's like, you know, complain about. The Jupiter Notebook at that point is like complaining about, I don't know,

• 53:09Speaker 00

  the cup holders in the Starship Enterprise or something. It's like, stop it. You're moving at warp nine. Just shut up, right? So I think that's the thing that devs like myself who have a dev background, we look at the space of technology and it's a relatively flat landscape. Yeah, I can go learn this language. I can do that thing. I can go grab a search, spin up that AWS credential. No problem. For the average person, every single one of these things is a cliff that's insurmountable. And this is kind of to the point of Anaconda as well, right? Advanced users, actually a lot of advanced users do use MiniConda as their deployment technology, like I said, within Dockers and whatnot. But there's also others who are like, well, I can just, I can do my own thing. I don't really want to use this big package manager thing. But there's so many people out there learning how to do this stuff. They're on Windows. They have no idea how to, what a compiler is. You know, they just want to do their jobs for

• 53:53Speaker 00

  their poli sci homework, right? And so that is where accessibility, again, kind of building on the Python motif of being accessible as a language, want the tooling around this to be accessible. And this is now, to answer the second part of your question, where have we fallen short? I think that as the technical space of data science, machine learning has, you know, grown up, put on a suit, got a real job, we've got software developers coming in saying, hey, I'm going to retool as an ML engineer. I can learn the stats. And they can, they're smart people. They can do all these things. Part of what's being lost nowadays that I see in the more modern tooling is there's the kind of a taste is lost and the taste of making this accessible for people who are not ops geeks. I think some of that drive has gone away. So when I go and look at the augmentation for any kind of ML framework, and this is not

• 54:37Speaker 00

  to put a ding on them. I mean, this is, you're doing complex orchestration. There's going to be some work involved. But just in general, that sense of like, how do we make this dirt simple for that poor atmospheric scientist trying to model hurricanes, trying to understand climate change? How to make it simple for these guys trying to build better translation, machine translation for African languages? And they're out in an African village, trying to get the locals involved. How to make stuff really easy to use for them? That kind of thing is, I think, being crowded out a little bit, right? That kind of sensibility and taste is gone, I think. And it's unfortunate. There's also willingness to embrace a really big corporate open source, which is not, again, I salute the companies for open sourcing their technologies. But I think the open source

• 55:19Speaker 00

  Ecosystem around SciPy and whatnot never really had some of the corporate open source like hegemony kind of thing happen to them that happened in the Linux space. So they've never seen weaponized open source. And I'm one of the few people trying to go in there and like raise the banner and be like, hey, guys, recognize there's like community grassroots kind of, you know, open source community innovation. And then there's like, big company saying, here's our big old toolkit, a million man hours, and it's all yours. Use it, please. And by the way, it runs best in our cloud framework. Like, that's, you know, I don't know how I feel about that exactly.

• 55:50Chris Benson?

  It's more than a license, right?

• 55:52Speaker 00

  It's more than the license. It's about community innovation and open standards. Yes, absolutely.

• 55:55Chris Benson?

  So as we finish up, I just kind of wanted to get your sense of what does the future of Anaconda look like? What's in your mind that you'd like to do that you haven't gotten to yet? What should your users be looking forward to at this point?

• 56:08Speaker 00

  Yeah, so that's a great, thank you. Great question. So we've been working a lot of infrastructure technologies for the last several years, trying to help get the commercial adoption of data science and open data science successful, right? Which is why we sell like a package server, things like that. We don't want IT raising the exception saying, you cannot use, thou shall not use Python, because we're a Java shop, right? I wanted to put that argument to rest. So I think we've been pretty successful with some of the things we've done to help smooth those things out. Going forward, though, I want us to kind of lean more into the practitioner community, help the community and thought leaders and practitioners, diverse voices, right, across culture, across background, and whatnot, to surface organically in the community and really drive a conversation about the practice

• 56:52Speaker 00

  of data science and quantitative, like quantifying our world, modeling our world, predicting our world, but doing it in an open way, in an ethical way, and in an intentional way. I don't want us to do, I don't want data science to end up where social media has, where it's like, oh, we accidentally destroyed democracy. Crap. You know, like, you want, you want data science predictive analytics to be sort of like, hey, we know exactly, well, we know we're going into it eyes wide open. So I want to create a community, community tools, so tools for ethical practice of data science, as well as then facilities, site, you know, some of the next generation capabilities that are practitioner facing. So we're going to be, you know, look forward to a lot more of that. We're going to try to engage with the user community a lot more. We'll be revving our product offerings there as well as some of the things that will be

• 57:35Speaker 00

  standing up on the website itself, on the Anaconda Cloud. So really excited about those kinds of things. It's beyond tools at this point. It's about people. It's always been about people, but now the emphasis is kind of coming back around to being about the practitioners and the community.

• 57:46Daniel Whitenack?

  Awesome. Well, thank you so much. I think that's a great way to end. We'll of course link to a bunch of Anaconda things in the show notes, so make sure and check those out. Connect with us on Slack and LinkedIn and Twitter. Let us know how much you appreciate Kondo over the years and you know what they're doing. Thank you so much, Peter, for joining us. It's been a pleasure.

• 58:04Speaker 00

  Thanks, Daniel. Thanks, Chris. This has been a lot of fun. Thanks for listening to my rants.

• 58:08Daniel Whitenack?

  It was fun. Thanks.

• 58:09Speaker 00

  Thank you.

• 58:13Speaker 02

  Don't forget, we have a giveaway going on in celebration of episode 100. Enter for your chance to win some awesome AI hardware from Nvidia, Intel, and Google, plus Practical AI and PackyDerm swag. We're giving away three bundles, so you have a good shot at them. Check your show notes for details on entering. You have until the end of the month. Speaking of PackyDerm, a little birdie told me they have a big announcement coming soon, and you should join their Slack channel to stay tuned. Learn more about that at PackyDerm.com. Thanks to our longtime sponsors for their continued support. Shout out to Fastly, Linode, and Rollbar, and to the mysterious BrakeMaster Cylinder for these awesome beats. That's all for now. We'll talk to you again next week.