AI-generated code with OpenAI Codex

Recently, GitHub released Copilot https://copilot.github.com/, which is an amazing AI pair programmer powered by OpenAI’s Codex model. In this episode, Natalie Pistunovich tells us all about Codex and helps us understand where it fits in our development workflow. We also discuss MLOps and how AI is influencing software engineering more generally. Sponsors: RudderStack https://rudderstack.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog – Smart customer data pipeline made for developers. RudderStack is the smart customer data pipeline. Connect your whole customer data stack. Warehouse-first, open source Segment alternative. iMerit ML DataOps Summit https://imerit.net/dataops – A FREE virtual event on December 2nd, 2021! iMerit is excited to gather 1000+ senior leaders at the forefront of deploying data solutions that power machine learning and artificial intelligence. Learn more and register to attend at imerit.net/dataops https://imerit.net/dataopsFastly https://fastly.com/podcast – Compute@Edge free for 3 months — plus up to $100k a month in credit for an additional 6 months. Fastly’s Edge cloud network and modern approach to serverless computing allows you to deploy and run complex logic at the edge with unparalleled security and blazing fast computational speed. Head to fastly.com/podcast https://fastly.com/podcast to take advantage of this limited time promotion! Featuring: Natalie Pistunovich – GitHub https://github.com/Pisush, X https://x.com/nataliepisChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: GitHub Copilot https://copilot.github.com/OpenAI Codex Beta http://beta.openai.com/codex-waitlistPrevious GopherCon Workshop: Infrastructure for a Fraud Detection ML Application https://github.com/Pisush/gc-ml-infra-workshopUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  These models, for them to be any good, for them to be productive, they've got to end up in software. And there's a reason that so much of the modern internet infrastructure has been written and go. It's a fairly small language. There's kind of one way of doing everything, which is a struggle for people when they move into it. But after you accept that, it's just wonderful. So you can remember everything. And that means you're more productive and you're less likely to create errors. I know, as I've watched over the last few years, the modeling get out and be productive in all these different use cases, I think Go is a really good language for doing that in, in terms of being able to get to productivity quickly with your models.

• 0:38Speaker 01

  Big thanks to our partners, Leno, Fastly, and Launch Darkly. We love Leno. They keep it fast and simple. Check them out at leno.com/changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by Launch Darkly. Get a demo at launchdarkly.com. This episode is brought to you by our friends at RudderStack. And we're calling all data engineers to check out RudderStack Cloud and start building smart customer data pipelines. RudderStack is warehouse-first, no more silos. RudderStack, build your customer data lake on your data warehouse, not theirs, enabling all functionality of a CDP with more security and retaining full ownership of your data. It's open source and API first. RudderStack can be easily integrated into your existing development processes. And

• 1:22Speaker 01

  because they're open source, you can see all their code. So you don't have to worry about vendor lock-in or black boxes. And best of all, they have transparent pricing. Stop paying your CDP a premium to store your data. RudderStack is free up to 500,000 events and pricing scales transparently from there. Learn more and get started at rudderstack.com. Again, rudderstack.com. That's R-U-D-D-E-R-S-T-A-C-K.com.

• 1:56Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at kingdog.com/community and follow us on Twitter. We are at Practical AI.

• 2:22Daniel Whitenack?

  Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist at S I L International, and I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

• 2:37Chris Benson?

  Doing very well. It is Thanksgiving week as we record this, and so I'm chilling out. No stress, man.

• 2:44Daniel Whitenack?

  Yes, getting the Tofurkey ready.

• 2:47Chris Benson?

  Tofurkey, absolutely.

• 2:48Daniel Whitenack?

  Yes, I'm super pumped this week because Natalie Pastunovic is with us. Welcome, Natalie.

• 2:56Speaker 04

  Thank you. Hi.

• 2:57Daniel Whitenack?

  Hey, so I met Natalie, I don't remember what year it was actually, but we were in Siberia. I do remember that.

• 3:05Speaker 04

  2016, I think.

• 3:06Daniel Whitenack?

  Yes, so at a conference, 2016, in Siberia, we discussed a lot of Go and AI-related things, and I'm glad this has kind of come full circle. But also Natalie is a distinguished co-host of our sister podcast, Go Time, which is all about the Go programming language.

• 3:25Chris Benson?

  There you go.

• 3:26Daniel Whitenack?

  Yeah, check that out. It's great to have some cross-pollination. But yeah, Natalie, you're doing so much. You're a developer advocate at Aerospike, OpenAI ambassador, Google Developers Expert, founder of various things. So very honored to have you on the show and excited to talk about all that. Maybe to start out, what's it like to be an OpenAI ambassador? What's involved in that and what do you get to do?

• 3:54Chris Benson?

  That's what I was wondering, too.

• 3:56Speaker 04

  I'm sorry, I signed an NDA. I cannot answer that.

• 3:58Daniel Whitenack?

  Okay, right, right.

• 3:58Speaker 04

  No, I'm kidding. I'm getting to have a weekly sync with the other six developer ambassadors. So there's seven of us. And what we get to do is offer office hours for people who get access to GPT-3 and to Codex and to all the other engines. And I get to hear all sorts of interesting ideas that people think and try to use the engines for, and I get to offer them some help. And then we have a weekly think that we chat between us, kind of interesting ideas that we saw, maybe tips that we can give to each other as ambassadors, and we get to hear from the OpenAI team about what's fun and upcoming. And we get to sign NDAs and we get to try all sorts of engines before

• 4:40Speaker 04

  they come out. So we got to play with Codex a while before it came out.

• 4:45Chris Benson?

  Yeah, that's awesome. I have a follow up even to that, where I'm diving into follow ups early on here. So without giving away someone's, you know, special sauce or super secret idea, what's a really cool thing that you've heard through that, or that you've seen or anything? I mean, just what?

• 5:00Daniel Whitenack?

  Interesting use case or something.

• 5:01Chris Benson?

  Yeah. She's having to think NDAs.

• 5:06Daniel Whitenack?

  From the usage of GPT-3 or Codex, like someone said, hey, can we do this? And maybe it was like surprising to you or something. Anything like that that comes to mind?

• 5:16Speaker 04

  Yeah, definitely. So one of the things that is less familiar, or GPT-3 is kind of less in the titles for, is that it has some endpoints that you can create your small little world of knowledge base and then ask it all sorts of questions and kind of maybe it will label things for you and so on. So one team wanted to use that to help them understand and kind of analyze cities and kind of like give them properties with that database. So they kind of create their own, this is the information that I have about this city and now kind of give me back a description of it.

• 5:52Daniel Whitenack?

  Very interesting. So I have to reveal, I am a total biased user of GPT-3, but also I recently got access to the Copilot Codex stuff and have been trying it out. But at S I L, we had this problem where, so we translate a lot of content, right, into various languages. And we're working on technology that will help us estimate the quality of those translations. The problem is like, if we're trying to work on that, we don't have a ton of data with like really bad translations, right? Because everything we've published have been quality Checked good translations of whatever material. And so we we use GPT three to like help us generate like bad contradicting translations.

• 6:37Daniel Whitenack?

  So like we said, hey, here's like the source language. And here's like, what would be like a contradicting statement to this to this? Or what would like follow this pattern of like removing like clauses or certain types of information or something? So that was actually very useful for us in kind of creating that labeled data set. Yeah, I don't know. What what's the sort of after you've seen a lot of use cases? What are the kinds of the things that people want to do most with GPT three? Is it like creating like movie scripts? Or is it like, what's the thing that people want to do most?

• 7:15Speaker 04

  Variations of marketing. So writing content or adjusting content to different audiences. Copy AI, I think, was one of the very early companies to have raised money that are built very much on top of GPT three. And many tried to reproduce what they did.

• 7:33Daniel Whitenack?

  Yeah.

• 7:34Chris Benson?

  Not surprised by that.

• 7:35Daniel Whitenack?

  Yeah.

• 7:37Daniel Whitenack?

  You also mentioned Codex. So we talked a little bit about GPT three, and we have mentioned that before on the show. But we haven't talked as much about Codex. Could you give us just like a brief sketch of what Codex is? And what what the intention is that someone might do with it?

• 7:56Speaker 04

  Definitely. Codex is the engine that drives GitHub's Copilot. This is one of OpenAI's engines. And it is designed to not just perform general language tasks or natural language tasks, but it's specifically trained to translate language to code. And it's trained on some languages, so mostly 10 or 15 languages that it's really good at. But it performs great even in other languages that it was not meant to be trained for. So Python, it's a language that it performs best at. But as you mentioned, I'm a big fan of Go. And it's also one of the first languages on the list that it's supporting. Some unexpected languages that it supports, for example, is Shell.

• 8:41Daniel Whitenack?

  Yeah, so like bash scripts.

• 8:42Speaker 04

  All the bash magic.

• 8:43Daniel Whitenack?

  Yeah.

• 8:43Speaker 04

  Yeah, exactly. The data scientist jumped, yes, finally. All the commands, all the step commands, someone, somebody's gonna do that for me.

• 8:51Daniel Whitenack?

  Yeah, I mean, it's probably better than my endless stream of copying commands from, from Stack Overflow, which sometimes do strange things on my, my local system, but yeah. So you mentioned Copilot. How do people use Codex in practice? So you kind of explained what it does a bit. How are people using it? Or how is it intended to like, impact people's workflow or development?

• 9:18Speaker 04

  So probably the most used way of Codex is through Copilot, which is available as a VS Code plugin. So anytime you use Copilot, you use Codex, whether you're aware or not. And if you have applied to the Codex waitlist and you got your access, then you see the exact same interface that you see when you use any GPT three engine. And you get to just kind of syntax highlighting, which is nice. And then kind of makes everything familiar for you. But you basically kind of give the same type of commands, write this for me or complete that for me. And you know that this will perform like particularly good in code.

• 9:56Speaker 04

  And some fun things that I like showing when I'm saying, what, what can it do is just mention casually kind of a name of a library, so no direct path or anything like that. And say, add a unit test or write a function and here's the signature of it. And if we're talking as two developers and I'm saying, hey, Daniel, can you take this client? Go client for Aerospike and kind of like do some example of crowd operations. You're like, okay, let me Google Aerospike go client. Uh, let me look at the documentation. Let me copy paste some examples. Let me change one thing. Oh, it broke. Wait, let me try again. Copilot like three seconds. Okay, here's everything for you. Kind of mind blowing.

• 10:39Speaker 04

  It's so much faster than us and it does not make mistakes in the sense of using the library, but also in the syntax of the language. It's performing like it knows to do those two things in a few seconds with a little context that you're not even. By the time you finish Googling, it already finished doing what you asked it to do.

• 10:56Daniel Whitenack?

  Wow. So aside from speeding that process up, does it change your workflow in any other ways? Or is it just the time compression of getting to something productive?

• 11:06Speaker 04

  It's changing absolutely. So some people like reading the documentation and then going and trying things. Some people like just kind of finding some quick guide or even just copy pasting from Stack Overflow and seeing how it goes. Or maybe taking an example and playing with that. So sometimes the examples are not covering everything you need. So you can say, generate this example for me and then you can start playing with that. That's one thing it does. Some of the nice things that you can say is add a unit test, right? And then maybe it will do something that you did not think of. It can propose a different solution for the problem that you have. And then, you know, usually you make a pull request and then you start discussing, ah, this is one way of doing that. But your colleague tells you, no, you can also do it this way. And then you have this conversation.

• 11:49Speaker 04

  So this is becoming a lot faster because you're having this conversation with a AI. Or like, give me some proposals, some, something else, some, another way to implement this. Or here's a function signature and tell me, how would you implement that? And so it's sort of like pair programming with a computer.

• 12:05Daniel Whitenack?

  Okay.

• 12:06Speaker 04

  Write some unit tests, whether if it's the boring part of just automate like for all the different types, do that.

• 12:12Daniel Whitenack?

  I'm sure there's a lot of data scientists out there who are like, oh, I, you mean, I don't have to write all my unit tests? I've had a lot of interactions with data scientists who really want to write that code, but are pretty hesitant to write, to write tests.

• 12:26Speaker 04

  Yeah, you can also say refactor this for me.

• 12:28Daniel Whitenack?

  Oh, yeah.

• 12:29Chris Benson?

  And I was gonna ask, I mean, and it really goes with what you're kind of saying there, Daniel, is there's a point here where no code, you know, it's already kind of coming to, to realization in various places. Where does this go down the road? Do you think that, uh, are we going to be writing our own go pretty soon? Or do you think we're just going to be giving kind of use cases and getting to that? Or Python, or, or pick your language, whatever it is.

• 12:52Speaker 04

  I think that in the short term, we will be using this in the IDE and getting all the help and different proposals and so on. But I do think that no code will go even one step further. So not just to your IDE, because for having an IDE, you need to be, you need to have it set up, you need to, you know, whatever, language binding set up and so on. I do see that the world of no code is developing even further to the place that with just an interface, you can already play that. So kind of somebody will put a GUI on top of that and then it'll be a few clicks and then it translates to that call to the API of Codex that then generates that code for you.

• 13:31Daniel Whitenack?

  Gotcha. So I'm curious, I mean, you mentioned the different languages that are supported. I'm assuming, and it would be interesting to dive into all of these details, which would take a whole nother episode, but I'm assuming that like a lot of code from GitHub was used to to train this model. I've always been curious, like you mentioned how it sort of doesn't make mistakes in terms of the syntax of the language and following those prompts. Is that because there's sort of like the prediction of like generated code plus an analysis of like, is this like valid syntax or not? Or is that like a rule-based analysis of the syntax like behind that? Because I know like with generating natural

• 14:13Daniel Whitenack?

  text, you can certainly generate things that are like not grammatically correct, right? And I assume it's similar on the code side, generating things that are not syntactically valid or something like that. Any insights there in terms of how that that happens?

• 14:29Speaker 04

  Yes, this is a hot topic, especially around the time that the Copilot was released first. There were lots of questions on what type of license had the content that it was trained on. So obviously, it was only trained on the open code, right? Not anything that is private on GitHub and only on the like a specific set of licensed code. But some code is not licensed. And even for some of the license types, there was also a discussion whether it's okay or not. And the code is generated correct, like whatever grammatically

• 15:02Speaker 04

  Equivalent is for code. But what's interesting is that if you tell it something like generate an SSH key for me, it will generate a key for you. It will have the right syntax. So it will be the right length. It will not use like forbidden characters, but it will not be a valid one. So this is also an interesting point, how it automates development, but not just yet develops infrastructure in this world. Because all the world of configurations, it will not do correct for you. You still have to put all the keys. You still have to set up all those values, you know, because it can generate a value in the right range. But does that value make sense? That's still to come.

• 15:53Speaker 01

  This episode is brought to you by iMerit and their upcoming ML Data Ops Summit in partnership with TechCrunch. It's a virtual event happening December 2nd, 2021. Check out the speakers and register at imerit.net/dataops. The event is gathering more than 700 attendees from top AI and ML companies and feature major speakers, including Facebook AI, Cruise, Zoox, GE Healthcare and more. And I'm here with Ivan Lee, the founder and CEO of DataSor, who's also speaking at the event. Ivan, I know you'll be speaking at the conference on this subject, but can you share a teaser of what's happening right now in the NLP space?

• 16:29Speaker 05

  If we look at the advances in NLP over the last few years, there have been some really exciting developments. Perhaps most notably, OpenAI's GPT-3 and their ability to just really start mimicking humans in generating snippets of English language. What we've noticed is that perhaps of all the branches of AI, NLP is one of the most mature. And there were some obvious use cases when we were starting out. There's things like the ability to handle customer support, improve upon chatbots. These were very clear verticals that we wanted to go after. But as we learned more, it turns out there's applications in the legal industry, in healthcare, in financial.

• 17:10Speaker 05

  There were a number of nonprofit organizations using us to label COVID-19 research and be able to just make sense of all the abundance of research that was coming out. We were kind of astounded by the creativity and the ways in which NLP could be produced.

• 17:25Speaker 01

  All right, learn more and register to attend for this free virtual event at imerit.net/dataops. Again, you'll hear from top AI and ML speakers who've successfully deployed machine learning data operations in their organizations. Again, this event is free and it's virtual. Learn more and register at imerit.net/dataops.

• 18:08Daniel Whitenack?

  Okay, so I did get access to the GitHub Copilot, the trial, or whatever they're calling it. And I've tried out a few things. For a while, I didn't try it out because I'm one of those annoying people that use Vim. And so I was like, I don't, I don't really want to install VS Code. But I did because I wanted to try it. And I think actually, I could be wrong about this, but when I looked at the documentation last on Copilot, there was a little bullet that said neovim. So maybe I need to look a little bit deeper into that and play around with it. And I don't know if you have any info there, but I did try just like a whole bunch of things.

• 18:49Daniel Whitenack?

  And what I found interesting was there's sort of like tab complete kind of operations, like you might find in Gmail. And then there's the like prompt type of thing. So it does seem like there's, am I correct on that Natalie, that there's sort of like, you can start typing something that you're about to do, like a loop or a function or something, and it will kind of tab complete a lot of things for you? Or you could like in natural text or a comment, tell it to do something for you. Do I have that right?

• 19:19Speaker 04

  Yeah, those both are two ways of using it. Yeah.

• 19:22Daniel Whitenack?

  Okay.

• 19:22Speaker 04

  I saw a tweet the other day, a funny one that says, "My next startup is going to be auto-generated from scratch by Codex. I'm just gonna press Tab until I have a software, and then let's see how it goes."

• 19:33Daniel Whitenack?

  I did think like three years ago, and I probably should have done it, that I I should have just created a script that bought, so took like real words and converted them into something dot AI and just like bought up a bunch of domains and then like sold them. But alas, I I didn't.

• 19:53Chris Benson?

  You missed that opportunity.

• 19:54Daniel Whitenack?

  Oh, yeah, I know, right? But yeah, like some of the things I did, just to like, give people a sense. And this is like, I'm sure it's not as complicated as the things that you just mentioned. But as a simple example, I like started like a Python list like data equals, you know, bracket one, comma, two, and then it just like filled in the rest of the numbers for me up to a certain point, right? And then it was cool because then I was like, well, I want to create a data frame now, right? And that's going to be one of the columns. And I said, like, labels equal bracket A, B, and then it filled in the same number of things, but with the A, B pattern. And then, you know, data frame, create data frame, it did that for me.

• 20:36Daniel Whitenack?

  And then I just said, you know, comment, you know, save data frame to CSV and it did the thing. So that's just a very simple example, but it was pretty cool to see like for a non VS code user who is also learning how to use VS code at the time, I wrote things like very, very quickly, which maybe is blasphemous from from a V M user, but it's very exciting. I want to get back maybe to one thing you mentioned, Natalie, around the the open source code and and, you know, how it was was trained and such. I wonder if you have any thoughts. One of the things that I've thought about this whole time with the OpenAI Codex stuff is that like open source code is really like, a lot of it's really bad.

• 21:21Daniel Whitenack?

  And there's a lot of closed source code that's like really good. Was there like, and if you don't know, it's fine, but outside of like the licensing issue in terms of code quality, like what code might be there? And also, there's all sorts of stylistic like opinions that people have. So is that part of that, like providing ways to have people have alternates or something like that?

• 21:43Speaker 04

  So the wonderful thing about Go is that there's no different styles. There is go font, which is the formatting function, and then we all use that. So not in my universe.

• 21:56Speaker 04

  I think you probably can see different styles. I think if you use Copilot, it will stick to the style that you started. If you use a different language. And I do think that if you use the prompts or kind of a clean way of interacting with it, you will see sometimes one style and sometimes another style.

• 22:16Daniel Whitenack?

  Gotcha.

• 22:16Speaker 04

  And training on good versus bad code, that's a wonderful question. It's a model that's trained on hundreds of millions of parameters. Certainly, there's good and bad code. I don't know if it represents the ratio of good, bad code in open and closed source. I don't know if such a number exists.

• 22:33Daniel Whitenack?

  Yeah, I'm just thinking that there's code out there that is in the open that I've written, that I wouldn't want anything emulating.

• 22:43Daniel Whitenack?

  So maybe I should deal with that on my own.

• 22:46Chris Benson?

  I'll take a stab at that.

• 22:47Daniel Whitenack?

  At my bad code or?

• 22:49Chris Benson?

  No, no, no, no. Just the general, because I mean, this has come up before. I think open source tends to be better than closed source. Because open source, people are going to see, if it's out there on GitHub or any other place, people will see it and people are embarrassed to publish code that's just not very good. But in closed source, they do it all the time.

• 23:09Daniel Whitenack?

  I have no shame, I guess.

• 23:11Chris Benson?

  Yeah.

• 23:14Chris Benson?

  There's open source, there's closed source, and there's Daniel's code.

• 23:17Daniel Whitenack?

  Yeah, yeah. It's a whole nother category.

• 23:19Speaker 04

  That adds up to OCD.

• 23:23Daniel Whitenack?

  Right, yeah, that works out, which is also a part of my life. So we've mentioned Go quite a bit. There's probably a good portion of our audience that might not even be aware of what Go is. Could you just like, maybe give a couple of minutes of like, if I don't know what Go is, what is it? And how is it sort of like used in things that I might be familiar with?

• 23:50Speaker 04

  It's a statically typed language that just celebrated its 12th birthday. It came from Google, but it's used everywhere. So not private to Google in any way. And there's lots of community contributions, obviously more as time goes. So the closest language to it is Pascal or Pascal, as I heard the American pronunciation for it.

• 24:12Daniel Whitenack?

  Right, I like yours better.

• 24:16Speaker 04

  It's a pretty close to C, but it has all sorts of benefits like a built-in concurrency and parallelism. It has safety and it's used in all sorts of tools that you know. So it obviously in the beginning, it was used mostly for web development, so lots of websites. And it kind of bubbled into the world of DevOps and infrastructure. So Docker is written in Go, Kubernetes is written in Go. If you do monitoring, you might have heard of Prometheus or Jager, both are also written in Go. SpaceX is unofficially shared they're using Go, so it will be on Mars soon.

• 24:56Daniel Whitenack?

  Exciting. Yeah, I know also that Go is used at CERN. I have a friend there that's using Go for high-energy physics. It's a whole, whole range of things. Yeah.

• 25:07Speaker 04

  Yeah, yeah. It's mostly backend. It's not exactly great for frontend, but surely you can do something. Also, you probably can do something with mobile if you really want to. I personally see it not just a great fit for, for web and for tooling, but also for things like machine learning. And I think we've both been preaching the idea of using Go in the infrastructure of systems that do machine learning, so serve AI models, because it's wonderful with parallelism and concurrency and it's super fast. And it has this really nice feature that it's very easily cross-compileable. So one CLI command, add the flag and you're

• 25:47Speaker 04

  good. So that's also why teams that do tooling internally really love that, because you have five people or five types of operating systems and whatnot and architectures, you just add all flags, done, your binaries run everywhere.

• 26:01Daniel Whitenack?

  Yeah, and I definitely am really excited that this messaging is getting out on, on this channel, because I, I definitely believe in, in what you're saying. We, on, on our team, and it's sort of a mix, right? I think it's still a mix, because you, maybe you can comment on your workflow with this, Natalie, but like, there's so much tooling in the sort of like experience. Experimentation phase of AI around training and testing models that's like really great in Python. But our team has has found that like as you sort of transition that out, right? Like you've done your experimentation, you know, like this model architecture and this sort of data is like what we want to do. And it's useful in these ways. This is like the access pattern and how we want to run inference.

• 26:47Daniel Whitenack?

  Then like transferring to running that behind an API that's written in Go or a streaming server that's written in Go or some type of like batch processing infrastructure that's written in Go that sort of integrates with that model is what we've found to be a really useful pattern in in my own work. I don't know if you have any comment about that, Natalie, if if I'm off or if you have other thoughts.

• 27:11Speaker 04

  No, I definitely share this view that this is a great choice, although it's not the trivial one. And I do think that as AI kind of bubbles into any field like legal tech or agri tech or whatnot. Like this is all becoming things of how do you serve your AI model? One thing is kind of have it trained and running, but then comes everything else. And there is a wonderful paper by a group of Googlers from 2015 about the technical debt of AI systems. And it's kind of a what happens when you patch up a system that does ML and quickly upload it to production. And it lists over 12 or so pages of everything that can go bad and all the considerations you should have.

• 27:54Speaker 04

  And, you know, it's things that in the past you may have not thought of them in the context of systems that serve AI, but you just thought of them as kind of general system, so any website, but monitoring, security, all the additional stuff that like making sure that things are working, all these systems. It's the wonderful ecosystem of infrastructure that already has the perfect ecosystem inside Go. Everything you need is kind of already there. So plugging those two together is even a better reason. For using Go, because Go is really fast for serving your model and also really useful for monitoring everything.

• 28:31Chris Benson?

  There's a thing, and I know we've brought it up before, and that is that that these models, for them to be any good, for them to be productive, they've got to end up in software. And there's a reason that so much of the modern internet infrastructure has been written in Go. It gives you all those things that you just talked about. It gives you two other things that we alluded to earlier. I just wanted to bring back up was it's a fairly small language. And part of that is, as going back to what you were saying, is there's kind of one way of doing everything, which is a struggle for people when they move into it. But after you accept that, it's just wonderful. So you can remember everything fairly easily. And that means you're more productive and you're less likely to to create errors. And so I know as I've watched over the last few years,

• 29:14Chris Benson?

  the modeling get out and be productive in all these different use cases, I think Go is a really good language for doing that in in terms of being able to get to productivity quickly with your models.

• 29:23Speaker 04

  And one other benefit for this characteristic of Go is that if you have the AI generate a huge chunk of Go, it will look exactly like the code that you wrote, because there's only one way of doing that. So you know that uncanny valley of robots? You're not going to see the uncanny valley of code.

• 29:42Daniel Whitenack?

  If it looks too human, it's too creepy.

• 29:44Speaker 04

  But once you cannot tell whether if it's a whether it's a robot or a human, you're happy with it again. And because there's only one way of writing Go code visually, you will not be able to know who generated that. So you're not falling into that uncanny valley, unlike with many other languages that have multiple formats.

• 30:02Chris Benson?

  True.

• 30:02Speaker 04

  So that's why it's a great language for AI.

• 30:04Daniel Whitenack?

  So like sort of bringing this back to the codex world. So if we're generating code and there's like a lot of variability in like style or how you might do something, or even like things like, like there's a lot of Python 2 code out there and Python 3 code out there, right? You know, Go sort of is more consistent like that. So would you say that like it's maybe a bit easier to get bigger chunks of code generated in Go than like just kind of the small helps that I was alluding to earlier in Python where like I was doing small-ish things?

• 30:41Speaker 04

  Yeah, and you will feel easier reading it and integrating that into your codebase. Or creating that to be your codebase and you integrate into that.

• 31:07Speaker 06

  We are going to ship it in 3, 2, 1.

• 31:11Speaker 07

  I'm Karhal Azou, host of Ship It, a show with weekly episodes about getting your best ideas into the world and seeing what happens. We talk about code, ops, infrastructure, and the people that make it happen, like charity majors from Honeycomb.

• 31:26Speaker 08

  We act like great engineers make great teams, and it's exactly the opposite in fact. It is great teams that make great engineers.

• 31:34Speaker 07

  And they farly were the founders of continuous delivery.

• 31:38Speaker 06

  Start off assuming that we're wrong rather than assuming that we're right. Test our ideas, try and falsify our ideas. Those are better ways of doing work, and it doesn't really matter what work it is that you're doing. That stuff just works better.

• 31:50Speaker 07

  We even experiment on our own open source podcasting platform, so that you can see how we implement specific tools and services within changelog.com, what works and what fails.

• 32:02Speaker 02

  It's like there's a brand new hammer and we grab hold of it and everyone gathers around. We put our hand out and we strike it right on our thumb. And then everybody knows that hammer really hurts. When you strike it on your thumb, I'm glad those guys did it. I've learned something instead, yeah.

• 32:17Speaker 06

  I think that's a very interesting perspective, but I don't see it that way.

• 32:22Speaker 02

  Okay.

• 32:22Speaker 06

  It's an amazing analogy, but I'm not sure that applies here.

• 32:26Speaker 07

  Listen to an episode that seems interesting or helpful. And if you like it, subscribe today. We'd love to have you with us.

• 32:46Daniel Whitenack?

  So we were talking just before we started recording about this whole world of ML Ops or AI Ops and how that is like so, so much of the conversation in the AI world is centered around this. We've got sort of various takes on this over time on the podcast and I'm curious, like when you're thinking of ML Ops or like when you're thinking of a project that is coming up in your pipeline and involves machine learning or AI in some way, like what is it that you're, you have in mind in terms of like the ML Ops things that you need to be thinking about?

• 33:25Speaker 04

  So kind of what is a checklist for your ML Ops project?

• 33:29Daniel Whitenack?

  Like what are the first things that are like, yeah, the necessities of ML Ops that are just crucial to making things work in production?

• 33:38Speaker 04

  It's a great question and I think it's getting this bare minimum, it's getting more complex or like the longer, every year this list gets a bit longer. So you definitely need to have some sort of data processing, right? Data governance variation, you'll call that. You need something for serving the model. You need something for the feedback loop of retraining it online or offline. So a growing trend that I noticed over the last year or two is about feature extraction, feature engineering, feature stores and so on. So you almost cannot ignore this anymore in the context of ML Ops. I guess because it's an AI podcast, we don't need to talk what is a feature, which is fun. We get to talk instead

• 34:19Speaker 04

  about what is Go. But the reason that I think that Go is a good choice for that, for things about features is again, it's that it's really fast in all the different benchmarks, partly because the good concurrency and it's also easy to utilize this feature there.

• 34:34Daniel Whitenack?

  Yeah, we were also talking before the show about both, how both of us have been trying to promote this within the Go community and I'm really excited because you are, you are giving a talk at the upcoming GopherCon. So this will be December of 2021, for those listening in. So if you if you can find it on the Internet after that. What are you hoping that sort of your audience takes away from that talk at GopherCon? Who may be an audience that isn't as familiar with with AI, but is into all of these infrastructure and monitoring things? What are some of the takeaways that you you hope they'll have?

• 35:09Speaker 04

  So for the good practices of ML Ops, there's a some short checklist of things that you should have in mind, like be aware of all the things that ML Ops mean and why Go is a great choice for that. And the more general cause that I have, and I hope that everybody who comes to the talk eventually will have, is kind of a heads up of this is how AI is likely to integrate into your developer flow, whatever it is you're doing with Go or even in another language, really. And, you know, you can be cynical about this. You can say, nah, I'll keep I'll stick to what I know. But you can also utilize all the different benefits that we said, like you can automate boring parts. So, you know, we all love Kubernetes for doing that for us,

• 35:53Speaker 04

  and not just because it's boring, but also it actually prevents us from making mistakes. You get inspired for all the autocompletions or the alternatives that it offers for you. And a huge thing that we didn't give enough focus that it can do is that documentation, but like both ways documentation, right? It can help you document your own code and it can help you read somebody else's code if the documentation is bad slash not existent or just a language that you don't know.

• 36:22Daniel Whitenack?

  Can Codex add like comments? Like I can just say add comments to this text or comments to this code? Will it do that yet?

• 36:30Speaker 04

  Yeah, explain this code.

• 36:31Daniel Whitenack?

  Awesome. That's awesome.

• 36:33Speaker 04

  It can be something like, help me understand this codebase that I wrote last year because I forgot what I did there.

• 36:38Daniel Whitenack?

  That hits home.

• 36:39Chris Benson?

  I was just thinking that, exactly. I always look back on my old code and go, oh.

• 36:45Speaker 04

  Who did that? Who hacked into my computer and changed everything? And then also changed the history of commits and the timestamp? How dare they?

• 36:51Chris Benson?

  Exactly.

• 36:52Speaker 04

  So sometimes you have to understand what somebody else did. Like, for example, I'm really not good in front-end technologies. I am generalizing. I'm I've barely experienced working with any of these. But sometimes you have to understand what's happening. So I can go and try to understand it. I can go and find a developer to explain that to me. Or I can just highlight this and say, hey, AI, please explain this for me.

• 37:13Chris Benson?

  So I have a question for you. I actually want to take the same question Daniel asked you about kind of, you know, bringing the Go folks kind of into AI. And for some of the folks that are listening to this podcast right now, and they've been hearing us talking about Go, they may not have been exposed to it very much in the past, maybe not at all. And but they're now kind of curious, having heard the three of us chit chatting about it for a few minutes. How would they incorporate that or start to? What would be a reasonable way of them doing that to get that into their flow right now? Maybe alongside their Python? What's a good first step for them?

• 37:47Speaker 04

  If you have access to Copilot or Codex, just take any code in Python, highlight it and says, rewrite that into Go. Run that, see what happens.

• 37:58Chris Benson?

  That's fantastic.

• 37:58Speaker 04

  That's one one thing I would do right away. Because I first like to play with that. And then I'm going to go and read the docs and kind of go through the tutorial. So once you've played with that a little bit and got excited, Google for the tour of Go. This is a fun little sandbox that you can like, it's a guided, step by step of all the things that you need to know. It's not specific for AI Ops. It's not specific for Ops in general, but it does cover most of the interesting features of Go and kind of helps you understand how it's built and what's happening under the surface and how can you utilize the fun features that it has? It's been around for a while. You can definitely find some code examples. If you want to see, Daniel, is there a workshop that you're giving about this topic, about infrastructure in Go for ML stuff?

• 38:42Speaker 04

  Is it on, is it available anywhere?

• 38:45Daniel Whitenack?

  Yeah, so I'm giving an updated workshop at the upcoming GopherCon. So thanks for bringing up there. There will be the GopherCon event in early December. And there are workshops. Also, the main event is is free for everyone. That's my understanding this year. So the sort of main talks you'll be able to join. But then, yeah, I'm going to post an updated version of my workshop this, you know, around the time of the workshop. So if you're listening to this and it's early December, you can check out my GitHub and see some updated things there. But yeah, I think it's a great idea, Natalie, to like rewrite some Python stuff and go to see how it might look.

• 39:25Daniel Whitenack?

  It's fun to see that codecs can be like a productivity thing, but maybe it can also be a learning thing. Chris, I don't know if you remember, one of the early shows that we had was like, there was an app that helped African farmers identify disease spots on cassava plants.

• 39:43Chris Benson?

  I do remember.

• 39:44Daniel Whitenack?

  The model was completely capable of saying like, this is diseased or not, right? But the intentionality behind the app was that it wanted to sort of help infuse that knowledge in the community. Right? So it it pointed out things and sort of explained why they might be indicative of what, you know? And by doing that, it sort of freed the community from dependence on the app and that knowledge was sort of infused. So it's cool to see that, like, you know, these tools can also, you know, automate things for us, but also help us kind of gain knowledge in new areas a lot quicker. So, yeah.

• 40:20Chris Benson?

  Kind of cool when AI brings humanity into focus there.

• 40:23Daniel Whitenack?

  Yeah, yeah. Natalie, I know I know you do a lot of community related things, whether it's the OpenAI stuff or it's related to Google developers or with GopherCon Europe or GopherCon. When you're sort of talking about AI and machine learning, I'm just curious how you've seen those conversations shift over time? Because I think early on, like, I heard people talking about like Terminator and sentient things. I've seen some of those conversations shift over time to other, maybe more useful subjects. But as you're interacting with those communities, have you noticed any trends in terms of people's interest in AI

• 41:06Daniel Whitenack?

  or their fear around AI or their desire to do this or that? Or any thoughts on that?

• 41:12Speaker 04

  Yeah, I think in 2017, I talked and shared some code that I wrote on how to use Go with TensorFlow, which seemed as unrelated as using Go with mobile. And Go with mobile has never really picked up. It's still under the experimental path of the Go project. But the TensorFlow API actually is available in Go. And I guess it came not just from the side of the software of like, here's how you can use that and use that API and so on, but also from the side of the infrastructure. And here is how the entire monitoring and DevOps ecosystem that is already in Go mostly,

• 41:51Speaker 04

  just fitting itself nicely with ML, SML and AI bubble into the average tech stack. So now you have front end, you have back end and you have AI. Everything is getting automated and we obviously have to support that as DevOps people, as infrastructure people, as backend people, as a tech team.

• 42:09Chris Benson?

  That's a great point. I love that point. I'm going to steal your point going forward is that we find ourselves commonly talking about how AI is in software. It's a part of software now and stuff. But the way that you just put it in terms of a front end and back end, now there's AI. And there's a reason for it's not just for data scientists. It's for developers and there are so many tools like what you're working on that are making it accessible to people. How do you think that will change the software development world going forward in terms of how non-specialists in AI are engaging now that these new tools like Copilot and Codex are becoming part of the workflow?

• 42:48Speaker 04

  I think it will make things faster and efficient in the sense of how, you know, in the past, you developed code by writing everything. And then IDEs came and got all those fun plugins and useful things like, "show me the signature, remind, help me find quickly where the function is defined" and so on. And it became a part of your workflow, although it depends how back you go for the developer's history. This would be considered a cheat and like, "Oh, I don't need that. I'm just, I just need the clear notepad." So I see this as the next step of something that fits the world of developer productivity. It also helps boost the world of no-code, which kind of introduces into the community of people who create.

• 43:32Speaker 04

  Tech, people who don't know code, just because they will have the tools that will translate their English commands into tech or into code, right? So we as developers, we kind of in our mind, we do this translation from our instructions in English in the GitHub issue from the product manager, whatever, you translate that into code. And then the machine goes and translates that into compiled binaries for you. So in addition to this being, this flow being more efficient, it's going one level up of abstraction that I will just write in English what I need, I will play with the GUI and the AI will tell the computer to compile the code that it generated for us. So I see that as we go forward, there'll be like these two branches of developer productivity.

• 44:16Speaker 04

  But I also see that things like infrastructure and monitoring not necessarily going to be as affected by that because you cannot have it, like you cannot say to the AI, create a config file for me and you can just go ahead and use that because the numbers will be funny and the keys will be funny as we talked in the beginning. So this is something that people will still have to manually do for a while.

• 44:39Daniel Whitenack?

  Yeah, I, I hope that maybe that comes someday, but I think even those other things that you mentioned are just like an amazing new kind of development and how we write code and how we think about the things that we automate and the code that we generate. So I'm super happy that you joined us on the other podcast in Changelog and we will include a number of links that are relevant to this discussion, including the waitlist for the codec system, along with the signup for Copilot. I will also include links to GopherCon and all the cool Go stuff that we've mentioned. So if you're interested in either of those things, definitely check out those links

• 45:21Daniel Whitenack?

  and make sure and listen to Natalie's upcoming talk at GopherCon. I know I will be so, thanks so much Natalie, appreciate you taking time to join us.

• 45:30Speaker 04

  Thanks, Daniel. Thanks, Chris.

• 45:35Speaker 02

  That's our show. Thanks for listening. For more like this, check out our master feed. It is all Changelog podcasts in one easy to consume place. Let your podcast app snack everything we produce and then pick and choose which ones to listen to. Subscribe today at changelog.com/master or just search for Changelog Master in your podcast app of choice. You'll find it. Special thanks to Brake Master Cylinder for providing our music and to our longtime sponsors, Facely, Launch Darkly, and Linode. That's all for this week. We'll talk to you again next time.