GoogleX Chief Scientist On Imposter Syndrome, Career Growth, Project Taste

Carey Nachenberg was a Chief Scientist at a GoogleX moonshot, a Fellow (senior most eng at Symantec) and a professor at UCLA. I interviewed him about his career story and we discussed: • Story behind his growth to IC10 (VP equivalent) • How high-level IC recruiting works • How imposter syndrome held him back • How to develop “project taste” • How AI is affecting his students Timestamps: (00:00) Intro (00:54) Growth to Fellow at Symantec (13:13) The most complex malware (16:13) Why C was faster than assembly (17:17) Imposter syndrome (21:28) What matters more than intelligence (28:03) Experience at GoogleX (34:24) Leaving GoogleX (37:43) Experience at Lyft (43:40) Getting credit on collaborative projects (46:53) Becoming a professor at UCLA (49:13) How to speak well (53:23) How AI affected his students (1:03:53) Career regrets (1:07:16) Finding work you enjoy (1:09:03) Advice for younger self (1:11:04) Outro Where to find Carey: • LinkedIn: https://www.linkedin.com/in/carey-nachenberg-14bbb03/ Where to find Ryan: • Newsletter: https://www.developing.dev/ • X/Twitter: https://x.com/ryanlpeterman • LinkedIn: https://www.linkedin.com/in/ryanlpeterman/ • Threads: https://www.threads.com/@ryanlpeterman • Instagram: https://www.instagram.com/ryanlpeterman

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  I got to work on whatever I wanted for my entire career.

• 0:03Ryan Peterman?

  This is Kerry Notchenberg. He's a cybersecurity expert who is a fellow at Symantec, which is four levels higher than staff.

• 0:10Speaker 00

  I look for things with big business impact. I look where there were gaps.

• 0:13Ryan Peterman?

  From there, he joined Google X as a principal engineer and fought imposter syndrome.

• 0:18Speaker 00

  And what if, like, I'm not good enough for Google or Meta or something?

• 0:22Ryan Peterman?

  As a professor at UCLA, he's seeing firsthand how AI affects the classroom.

• 0:26Speaker 00

  I allowed students to use LLMs. In retrospect, you know, that was a bad idea because I think they were using it in ways that hindered learning.

• 0:33Ryan Peterman?

  How do you even tell if they're using LLMs, though?

• 0:37Speaker 00

  One way you can tell is...

• 0:39Ryan Peterman?

  I learned a lot from his career stories and hope you do, too.

• 0:42Speaker 00

  I get this call, a frantic call from UCLA. Can you still teach? Because our lecturer bailed on us. I have PTSD from that experience, I have to say.

• 0:54Ryan Peterman?

  Before we get into all the juicy parts of your career, like working at Google X or autonomous vehicles with Lyft, I want to start by kind of laying the high-level groundwork. So you worked at Symantec for a long time and became their senior most, you know, similar to a chief scientist type of role. Kind of wanted to go over that and all the lessons you might have learned there and, you know, anything interesting that came up there. So could you share the high-level story arc of you working at Symantec and how you grew to fellow?

• 1:27Speaker 00

  Totally. So I actually started back in, I think, 1992 as an intern at Peter Norton Group. And so Peter Norton Group was later an acquisition by Symantec. But some of your viewers are going to remember Norton antivirus, from Norton and Norton Utilities and so on. And so I think I was the first intern at Peter Norton Computing, and they didn't even have a desk for me. So I literally worked in a QA lab with the QA lab manager, who was a guy that used to sell knives for a living. Because like back then in 1982, they weren't professionally trained software people, right? So basically, they would just get whoever knew something about computers, and this guy knew something about computers to run the lab, and I would work on one of the test computers.

• 2:13Speaker 00

  And so that was my first, you know, summer internship. And then, fast forward... When I left in 2016, I had become the senior most engineer of the company. So from the junior most person, the first intern, to the senior most person at Symantec, which had acquired Norton. So it was a long ride, but that's the high of all.

• 2:34Ryan Peterman?

  Was cybersecurity something that really you were focused and really wanted a job in cybersecurity, or it just by chance?

• 2:43Speaker 00

  Yeah, so at the time, when I was working at Peter Norton Group, I wasn't working in cybersecurity, I was working on something called Norton Commander, which was like a file utility manager. So I didn't actually work on cybersecurity until my third year, my third year of internships. At this point at Symantec, where they had acquired a product called, I don't remember what it was, but it was an antivirus product that they renamed Norton Antivirus. So that product was acquired and rebranded, and they had a team of people analyzing computer viruses. So my third year of internship, that's when I got into cybersecurity. I had no experience. I just needed an intern and they threw me on it.

• 3:21Ryan Peterman?

  Like, what does the career ladder look like? Because I think Google and those companies came in at some point and said, hey, this is L3, this is L4, this is L5. And then kind of a lot of companies copied that. At Symantec, what did career progression look like?

• 3:36Speaker 00

  I would say the levels generally track to companies like Google and Facebook or Meta, going all the way from like a junior software engineer through software engineer, senior software engineer, staff and so on. So the levels are very similar. The difference was, for me, at the very high levels of distinguished engineer and fellow at Symantec, you know, when I went to Google, they basically said, you know, we can't just hire fellows, you know, we haven't experienced you, we don't know if you're going to do well here. So effectively, I was downgraded to a principal engineer, a level eight, from what would have been probably a level 10 at Symantec. It was a vice president role at Symantec.

• 4:21Ryan Peterman?

  What does the hiring process even look like for people so high level? Was that a tailored bespoke process?

• 4:29Speaker 00

  When I went to Google, basically, the former chief operating officer of Symantec, a guy named Steven Gillette, had recently transferred into Google X. And they were starting a stealth project in Google X, which we could talk about. He had known me from my time at Symantec and basically, you know, said, hey, why don't you talk to the team and see if there's a good fit? And so I didn't have to apply. I was just, you know, brought in and we had one meet and greet with the company, the team's founders in Venice. And then we had an interview, a day's worth of interviews, probably about eight interviews. And, you know, six weeks later, I had my offer letter, so.

• 5:10Ryan Peterman?

  You know, for a lot of software engineers that are in the lower levels, the interviews are lead code, they're system design, and there's, you know, some generic behavioral interview. For at the high levels, what does that loop even look like? Is it, is it mostly behavioral? Are they asking you lead code stuff too?

• 5:28Speaker 00

  So I don't believe that I had any coding problems during that interview. And if I recall correctly, there were some, there was one design problem. But mostly, these were leadership-like questions, like, you know, how would you solve a hard problem? How did you solve a hard problem in your career? How do you deal with conflicts? You know, tell me your thoughts on, you know, where the field is going. You know, Google X is a forward-looking, or X, rather, is a forward-looking organization. And so some of the discussions were focused on that.

• 6:04Ryan Peterman?

  Right.

• 6:04Speaker 00

  Some of the discussions were literally sales pitches. Like, I didn't really have to say much. They went and talked to me about where they thought the world was going to try to get me excited.

• 6:12Ryan Peterman?

  Right.

• 6:13Speaker 00

  I think it probably depends on the person, but in my case, I think they thought there was a good fit. And it was really about sort of going through the motions of, you know, having interviews and selling me on the role.

• 6:26Ryan Peterman?

  Okay, so then going back to Symantec, I mean, you know, getting promoted to fellow, what did those highest level promos look like?

• 6:34Speaker 00

  At least at Symantec, most promotions up through what was called technical director or senior technical director, which would be the equivalent of, let's say, staff engineer or senior staff engineer, probably at other companies. Most of that was done just within the organization. So as long as a vice president or senior vice president was okay with a promotion, it was allowed. At higher levels for distinguished engineer and fellow, it was, there was a basically core group of technologists led by the CTO of the company. We would meet once a quarter. We would get applications from those people and we would then review their applications

• 7:15Speaker 00

  and actually have them come talk to us about their work and then make a decision based on that. And so the reason that we did it that way is that we'd have consistency across all divisions, all teams of what it meant to be a distinguished engineer or a fellow for the organization. So that's the process we went through and it wasn't just per division, it was for the company.

• 7:39Ryan Peterman?

  So when you got promoted to fellow, was that one of the happiest moments of your career? Or was it something you expected and not a big deal?

• 7:46Speaker 00

  You know, it was very interesting. So I did not have to go through that process to get promoted to fellow. So I was a distinguished engineer at Symantec and they had that process. I wasn't part of it because I wasn't a distinguished engineer at the time. That team is made of, the team of, that does promotions is made up of distinguished engineers at the time. We had acquired a company called Veritas and Veritas had a notion of a fellow, which Symantec didn't. So basically after the merger between the two companies, they said, okay, we need to level set the role because the people at Symantec have never had this kind of role, who would belong there? And I was nominated without my knowledge and it just happened and I got a congratulations.

• 8:30Speaker 00

  Or maybe my boss filled me in and said, hey, there's some discussions going on, you know, and then you're a fellow. So they basically took my portfolio of stuff and gave it to the team that did the evaluations and that was the end of that.

• 8:43Ryan Peterman?

  I mean, with your growth to fellow, something about the way you work sets you apart from other engineers. What do you think are those things for you?

• 8:54Speaker 00

  I think what helped me get to fellow was working on really impactful projects for the business. Not necessarily always the most technically difficult projects, although many of them were, but more impactful. Like they moved the needle for the company. I had so many of them under my belt at that time that they just said, you know, the criteria are this many projects should be done of this scope. And I had plenty more. And so they basically said, you're, you know, you're above the bar. So what was the, what was the key insight to doing those projects is finding things where there, where there were gaps, things the company needed, where people weren't stepping up to do them. And, you know, I did them.

• 9:40Speaker 00

  Maybe a quick step back. At Symantec, it was sort of like the wild west. There wasn't an engineering culture per se. There was just people working on stuff and like projects would often run really late because, you know, it was a little bit more of a wild west. We didn't have an agile process. We didn't really do our own integration tasks. Like we'd throw over our code to QA and they would worry about it and we'd work on the next thing. And in fact, I had very unusual circumstances. I didn't rise to a path where I was given a small project and somebody said, okay, do this. Here's, here's a box around the project. They basically said, Kerry, you already sort of know the technology because you've been working on it as an intern.

• 10:22Speaker 00

  Go figure out what to do and do it, which was amazing because here I am like a junior software engineer and I got to work on whatever I wanted for my entire career. At Symantec, never assigned work to do. Yeah, it was crazy, which was great. And I, you know, and so I got to just pick projects that I thought would be impactful. And I picked well, I got, I guess I picked well because like, you know, project after project landed, they were integrated. We did tech transfers into the product, they shipped. So that's how I got there. It wasn't like, oh, I did increasingly bigger scope projects that somebody gave me.

• 10:54Ryan Peterman?

  So how'd you train that that project taste? Because impact typically leads to career growth everywhere.

• 11:00Speaker 00

  Yeah, yeah, yeah. It's a good question and it's, I look for things with big business impact. I look where there were gaps. So like way back when, you know, this is like old news now, but it's sort of interesting. Like, we had these computer viruses which are emerging, which were called polymorphic viruses. They were self mutating malware. And they were self mutating in a way that there could be like literally quadrillions of variants. And the way that the teams were working on these things, back when I was an intern even, is they would basically write handwritten assembly language to go look for telltale signs of a variant, you know, in the mutation. Okay?

• 11:41Speaker 00

  And the problem with that is it worked really great, except it took six months because we shipped a new product every six months. And so it took six months to discover to handle a virus. And then the next day, somebody released three new viruses, which were slightly, you know, slight variants or differences, different viruses, but mostly the same. And then all of that work would not work anymore. And so, you know, I would look at that problem and say, oh, there's a need to be able to move more rapidly in covering new malware. And by the way, detecting these self mutating threats is not like using a reg X. You can't just, you know, search for a string to find these things. So I'm like, oh, that seems like a really interesting hard problem. So I picked it and then I started working on it. That was my master thesis and then eventually transferred to the product.

• 12:24Ryan Peterman?

  If you were to think about the things that you took on, they were a series of, I guess, side projects or, you know, whatever you wanted to take on, where you'd take on this new thing, maybe something else would come in, you take that on. Is that kind of how you were working?

• 12:39Speaker 00

  I would say there were probably six or seven times in my career where I'm like, oh, the company needs this type of thing. Let me go spend six weeks, two months. Five months figuring out what that looks like, building prototypes, talking to engineers and figuring out what they need. And then, you know, building that. And there were other times, which is probably 80% of my career, where I was just sort of tweaking those things. In other words, we built them. We were trying to either tech transfer it, so I was helping with that, fixing bugs, improving. Those were sort of incremental improvements on those systems. But that, that is one of those two things generally.

• 13:13Ryan Peterman?

  So you mentioned a little bit about viruses. And when I was doing some research, I saw that you had done some storytelling on top of Stuxnet and kind of compiled. I think that's such an interesting story. Can you tell me a little bit about Stuxnet? Maybe we can go into that.

• 13:30Speaker 00

  Sure. Stuxnet was at the time, just unfathomable. It was just a very complex piece of malware, which was multi-platform. So it didn't just infect like Mac machines or Windows machines. It infected, I think, you know, Windows machines, but also like microcontrollers that would actually run like centrifuges and so on. And so it was probably the first multi-platform piece of malware we had discovered. Use zero days in order to break into systems that, you know, that, you know, basically vulnerabilities, exploiting vulnerabilities that hadn't been patched. Because they weren't even known about. And it didn't use just one of those or two of those. I think it used like six different vulnerabilities to spread, many of which were zero days.

• 14:12Ryan Peterman?

  Oh my God.

• 14:13Speaker 00

  It would literally stealth itself. So on your computer, if you were to look at a, at a thumb drive, which had Stuxnet on it. And look in your, your, your finder application or your Windows, you know, file system application. You would see, you know, nothing there, but it was there. You'd stick that in your computer, it would auto-launch. It actually had a payload to auto-launch. If you were to look at the logic that was running on a centrifuge, or rather the controller that ran the frequency converters. You would not see any of the Stuxnet logic. It was in that controller. But if you downloaded the logic from that controller onto a Windows machine, it would stealth and remove the logic from Stuxnet as it pulls it off.

• 14:56Speaker 00

  And then if you updated that logic, for instance, it would reinsert itself into that logic to reinfect it as it went back. So it would actually like sort of piggyback on back and forth, stealth itself. It was just amazing. And then, of course, how it disrupted the centrifuges is super interesting as well.

• 15:11Ryan Peterman?

  Yeah. It's so complicated and sophisticated that it makes me wonder who wrote it. And I saw something like it was, you know, 50 times bigger than the average virus, incredibly complicated software. And I was reading into Wikipedia a little bit before we kind of, it said, no one has claimed credit for who wrote this thing. Who do you think wrote this thing?

• 15:34Speaker 00

  I think it's pretty good. You'd be pretty safe to say it was Israelis and the American government. You know, my understanding or recollection is that there are water, not watermarks, but sort of, you know, coding styles or things in there that sort of implicate both governments.

• 15:50Ryan Peterman?

  Have you ever looked at the source code or?

• 15:52Speaker 00

  I have not. I didn't do any analysis on Stuxnet. My career was focused early on analyzing malware, like literally looking at the machine language and disassembling and so on. But later on in my career, it was mostly about detecting. So it was how could I build algorithms to detect that malware rather than hands on analyzing the malware myself? So I never looked at Stuxnet, yeah.

• 16:13Ryan Peterman?

  You mentioned a little bit about assembly code. Did you ever write assembly code when you were working at Symantec?

• 16:19Speaker 00

  I did. Yeah, I wrote assembly code as an intern. And although back in those days, it was mostly C.

• 16:26Ryan Peterman?

  Yeah.

• 16:27Speaker 00

  But some assembly as well. And I remember the first antivirus engines were written in assembly for speed. And one of my first tasks as I joined full time was I said, you know, this really needs to be a C, so it's more maintainable. So we ported the thing to C and actually made it faster. Because the people back then, people didn't know algorithms. They didn't understand what a big O was or how to, you know, they would do linear searches. And so we were able to go and take something in assembly language, move it over to C, have less code, and it would be, you know, five times faster, so.

• 16:56Ryan Peterman?

  I see. So the the speed ups moving from assembly to C was due to better algorithms and

• 17:02Speaker 00

  That's right.

• 17:03Ryan Peterman?

  things like that. It wasn't because of a compiler or something.

• 17:05Speaker 00

  Well, the compilers weren't that great back then. But even without an optimized compiler, if you use a hash table versus or binary search versus a linear search over 60,000 signatures, you know.

• 17:16Ryan Peterman?

  I I saw that you worked at Symantec for a long time. And, you know, I think in the tech industry, it's common for people to move around here and there. What do you think kept you at Symantec as long as you were?

• 17:29Speaker 00

  You know, that's a great question. If I have to be perfectly honest, I would say imposter syndrome.

• 17:36Ryan Peterman?

  Really?

• 17:36Speaker 00

  So, well, yes and no. So at Symantec, I didn't really have imposter syndrome because I had done a lot of stuff and I was well regarded. You know, I was known in the company. And so I had a good, safe place. But I always worried, what if it just is because I'm at Symantec and I grew up here and I learned the stuff here? What if I went somewhere else and I wouldn't be able to learn the stuff? Or what if people had different standards? And what if, like, I'm not good enough for Google or Meta or something? And so I stayed because it was comfortable. And I complained. I complained all the time. I wasn't happy later on in my career. I have to be honest with you. I wasn't doing things that made me happy. More when you get more senior, you do a lot more BS, right? And and and

• 18:20Speaker 00

  you also have the opportunity not to do as much BS, but you have to push yourself not to do it because it's very easy to, you know, go to meetings and, you know, have broad discussions and it's not really that necessarily fun.

• 18:32Ryan Peterman?

  Right.

• 18:33Speaker 00

  And so I wasn't happy near the end of my tenure at Symantec, but I was afraid that I wouldn't be able to do well or had failed the interview process. And so I just stayed. And it was comfortable.

• 18:43Ryan Peterman?

  Throughout your career, there were so many promotions and you had so much impact. For someone like you to have imposter syndrome, you know, I feel like that shows that a lot of people, you know, it's it's a very natural feeling for a lot of people. Did you eventually you did leave Symantec? So was there anything that helped you overcome imposter syndrome?

• 19:05Speaker 00

  You know? The thing that helped me was that somebody said, hey, we want to interview you. We think you'd be a good fit. And so I said, you know, I'm probably going to fail this interview. I'm sure I'm not good enough, but I'm going to do it. And so I just did it. And so that, you know, I needed an external pull or push. I don't know what you would call it, but in order to get me to take the chance. And then it worked out. But for me, like in my head, I was, you know, I wasn't competent to do that job, you know?

• 19:32Ryan Peterman?

  You, you mentioned also that at the highest levels, there's, you know, a lot of BS. And, you know, I guess it sounds like meetings and things like that. Do you have any, I guess, tips on how to be less involved in the BS? Because I think that's a natural pull, pull for anyone.

• 19:50Speaker 00

  Yeah, it's sort of natural. Definitely. It depends what you're doing. I mean, some, some tech senior technical directors and distinguished engineers, even fellows, were working day to day and building code and, and working with their teams. It just depended. I was an individual contributor, vice president. So I was an IC through my entire time at Symantec. Other people would actually manage teams and work, uh, close, close to man projects. You know, it's just, it's inevitable, right? In other words, you're having more strategic meetings. And then the problem is you're having a strategic, strategic meeting with a bunch of people, many of which, many of whom don't necessarily know that much, but they have an opinion because everybody has an opinion. And there's a lot of debating and a lot of arguing and a lot of like, you know, posturing for, you know, for power.

• 20:35Speaker 00

  And, you know, it's just, there's a, there's a lot of garbage that comes with being more senior. Unfortunately, like there was some, some joy, especially for me, when I got to pick my own projects, to be able to just sit down and literally go two months with nobody asking me, what, what are you doing? And, you know, and I'm just like cranking and trying things that doesn't work, but that does. And super excited, right?

• 20:54Ryan Peterman?

  Right.

• 20:55Speaker 00

  Then you get into a room with seven people and you're like, we've agreed that this is our new company strategy. One of my last rule, uh, things of the company I did was actually define the company technology strategy for the whole company. And everybody agreed to it and the CEO agreed to it. Then we get in a room and everybody would say, oh, sure. But, you know, we have to make money on our projects or products. And so you're adding those features to align with the technology strategy. That's gonna set us back and we've been told we have to make, you know, this much top line revenue. And so, you know, you end up having debates and discussions and it's like very draining.

• 21:28Ryan Peterman?

  So you said you were pulled into Google X and you ended up taking the interview and doing well. I'm curious, what was it like entering, you know, Google or this like Feng style big tech? And were there any cultural differences that stood out to you?

• 21:44Speaker 00

  You know, fewer than you would think. I would say the biggest difference that I saw there was there were really, really, really smart people. Like Symantec had some smart people, but again, it didn't have an engineering culture. Even when I left in 2016, it was starting to develop one, but it was really, you know, it was more a little lucier, gooseier than a Google for sure. But the quality of the people in Google X and X were really very high quality in terms of intelligence. Now, what seemed about the same was that many people in X, as there were many people in Symantec, didn't have good taste, research taste or project taste. And so a lot of people were really smart, but it wasn't clear that they were picking projects that would be, that would land

• 22:28Speaker 00

  or that were feasible, you know, at least in my opinion. So I think that's a, that is an attribute of engineers, no matter what company, no matter how intelligent people are. But it was, you know, like it was startling how much brilliance there was. And I do remember like there was one guy who was clearly like over 200 IQ. The guy was just, he talked to him and he was just astounding brilliant and he was still in L4. Why was he in L4? Because, you know, he had lack of communication skills, you know, worked on really interesting stuff that was interesting to him, but not initially had business impact, didn't collaborate well, apparently, you know, like there were things, whatever it was.

• 23:11Speaker 00

  And it didn't matter that he was brilliant. Like he was twice as smart as I was, but you know, just because you have intelligence doesn't mean you're going to be successful. And so that was, you know, I saw the same thing there.

• 23:21Ryan Peterman?

  If I'm understanding correctly, if you're very ambitious and you really want a career growth, intelligence is not that important. It sounds like there are some things that are much more important. You cited communication, soft skills, project taste, picking things that actually matter.

• 23:38Speaker 00

  Yeah.

• 23:38Ryan Peterman?

  Is there anything else that you, that comes to mind?

• 23:41Speaker 00

  There are definitely people who are less intelligent. You're not gonna like at Google, we didn't have too many of those people. Like there were people that were really, really, you know, most people were really quite smart. So I would say a baseline is you need to have a baseline level of intelligence. But I, for instance, don't think I'm a really intelligent person. I take it forever to learn new things. I have like this ramp, which is like this, you know, at least internally, that's how I feel. You don't need that much intelligence to be successful, but enough. But so communication skills, collaboration skills are really important. Like knowing how to work with somebody and not just piss them off. Because you're saying you're wrong, but figuring out how to, you know, you know, how to, how to give them what they need in order to get what you want. Which, by the way, I haven't really mastered yet. I've screwed that up a bunch of times too, but that's one.

• 24:25Speaker 00

  We talk about business outcomes, I'll generalize that. I would think something that's really important to move up is focusing on outcomes. So this is like a really important thing. And I probably did some of it subconsciously or unconsciously and some of it after I learned about it more consciously. It's very easy for people to focus on their own outcomes. In other words, they know what they'd like to do. They know about the technology they want to build. They know that they want to make it 10% faster or whatever it is or. But often the outcomes of the company or the outcomes that somebody else is trying to meet are different than your outcomes. And if you don't project yourselves into their shoes or the company's shoes and identify what the company's outcomes or divisional outcomes are, or the other team's outcomes are, people are not going to be interested in what you have to do,

• 25:09Speaker 00

  even if it's really complex and hard and interesting for you. And so I think what really helps, far more than intelligence, is focusing on what outcomes need to be solved for a project or for the company. What metrics matter for those outcomes? Because often there are things that don't really matter that much. Like, you know, there are like requirements that are unimportant and requirements that are super important. So focusing on the most important requirements and. Doing that work and focusing on the most important outcomes for your division or company, and then focusing on only the most important requirements is going to get you far, much farther than being intelligent, I would say.

• 25:50Ryan Peterman?

  I think you have an interesting perspective because you're in academia because you're lecturing at UCLA, but you've also had a lot of success in the industry as well. When I was in school, everything was very obviously intelligence-based. Maybe unless there's a, oh, aside from hard work, but you know, there's a test. You either get it right or not, aside from group projects and things like that. Obviously, coming from that place, you know, intelligence feels like everything. And then you get to industry, and I agree with you 100%. Intelligence is not everything in industry. But because in college, it's very obviously meritocratic.

• 26:34Ryan Peterman?

  Whereas in industry, there's other things too, like do people like you? Or, you know, other things like that. Would you say that career growth is meritocratic in the industry?

• 26:46Speaker 00

  You know, in my experience, it was. There were cases where people were being promoted because a vice president basically pushed really hard or SVP pushed really hard and said, you know, they need to be promoted, period. Otherwise, we're not going to be able to keep them. And that's never a good reason to promote somebody, right? Because you don't uphold standards that forever people will still look at. And then you end up with a bunch of people that are not great. And everybody's like, well, why shouldn't I be promoted? Because that clown is promoted, right? But by and large, I would say it was meritocratic. You know, I remember when I was on these committees, we would look at the accomplishments and the complexity of the accomplishments, the impact that they made for the company. We'd look at the communication skills. We looked at, for instance, patent portfolios.

• 27:28Speaker 00

  Were they helping the company with intellectual property, which was important back then? I don't know how important it is now, but so it was generally pretty fair. And I saw Google too, it was very fair. There were, you know, very reasoned discussions about each person. So I think it is, I think it is. It's just, it's not just, you know, you ever see those charts where they have like a circle? And then they have like little, like a sort of a polygon inside.

• 27:52Ryan Peterman?

  Right, right.

• 27:52Speaker 00

  And it shows like how your intelligence is versus how your, your technical work is. And it had to be pretty, you know, pretty well rounded for the senior levels or, or at least be really good in some areas.

• 28:03Ryan Peterman?

  So then at Google, you went to Google X working in a new cybersecurity division. And then a company was spun out of that, right? Chronicle, if I recall correctly, but it's still under the alphabet umbrella of companies.

• 28:17Speaker 00

  That's right.

• 28:17Ryan Peterman?

  Which was then reacquired by Google Cloud. Could you share a little bit more about that story?

• 28:24Speaker 00

  Yeah, sure. So we started as a stealth project. Nobody knew we were doing cybersecurity initially. The project name was Project Lantern, and this is inside of X. And we literally started from zero. We didn't know what we wanted to build. There were lots of debates and we knew generally what we wanted to do, but we spent, I think, a good six months trying to just figure out what we were going to build. We then basically converged on an idea. We started hiring a bigger team, started working on, you know, building prototypes of the product out. Finally got to an MVP, started, you know, shipping, working with partners, which was great. You know, actually seeing real customers use it was super useful and seeing that we would, we would actually go into customer sites before we had the product and just watch how they did their work and saw where they struggled, which was super useful.

• 29:07Speaker 00

  Really interesting, actually, like watching cybersecurity teams work. Some of the people were like stoned, you know, they're clearly out of it, you know, like these are the people that are using your product, you know, for better or worse.

• 29:18Ryan Peterman?

  So the product was a product that cybersecurity engineers would use.

• 29:22Speaker 00

  Yeah, so best way to think about, in a nutshell, about Chronicle's product, which was called backstory, was basically cybersecurity today is a big data game. Okay. And what you, what you want to do, especially if you're trying to discover attacks in your environment or investigate attacks, which is a big part of cybersecurity. Some of it's proactive, some of it's blocking the attacks before they come in, but a lot of it is. They're going to get in and we have to know where they are, when they got in, what assets they accessed and so on. And so, as it turns out, virtually all software and hardware that's used in corporations today generates huge amounts of logs. A firewall will have every connection, the source IP, the target IP, what protocol.

• 30:03Speaker 00

  Web proxies will tell you what websites were visited, what, again, what machine visited them. You have telemetry like DHCP that tells you like what machine, what machine's IP is associated with a MAC address, associated with a machine name. You have email logs, you have client logs, what software was installed, right? All that data is super valuable for identifying attacks. Okay. But there's such high volume of that data that people couldn't really process it. And so, the customers that we were starting to work with would use a competing product, which I won't name, but they would literally go, they would ingest a certain fraction of that data, very little of it, because it was so expensive to maintain it. And they would go for coffee for 30 minutes while waiting for a query to finish to look

• 30:47Speaker 00

  up just one piece of information in that data. And so, we said, look, we have Google planet-sized compute, you know, and storage. How could we totally turn this around? And what we did was we built a product that would ingest all that data, petabytes of data, like literally some for some companies, you know, a petabyte a week or a petabyte a month, a huge amount of data of, you know, every device, every connection, every file installation, every settings change, like all that kind of stuff. And then we indexed it, and that's what I worked on. That was my sort of addition, like, so that it would be more like at the speed of a Google search than a 30-minute, let's go get some coffee. What would be a use case? Let's imagine that you discovered a piece of malware on a computer.

• 31:31Speaker 00

  You might have a hash for that malware, you might know the IP address where it came down from or was downloaded, you might have the file name, you might know the directory it was installed. Our product would allow you to take any of those artifacts. And plug it in and then we would instantly tell you which devices also have that artifact on them. What related artifacts there were to that, you know? So you could say, oh, well, this file had a different name, even though it had the same hash, let's say. And so we'd better check for this name because this might be on some other computers so you could pivot.

• 32:03Speaker 00

  We could tell you how many devices were impacted in your environment, whose those devices were. When was the first infiltration? When's the last infiltration? Is it still active? And do that in like two seconds, you know, that kind of those kind of use cases.

• 32:14Ryan Peterman?

  Wow.

• 32:14Speaker 00

  As opposed to 30 minutes where, you know, hopefully it gives you an idea.

• 32:18Ryan Peterman?

  You know, when it comes to Chronicle, even just doing the research, I got a little confused. Sounds like it spun out and then it came back in. What is the benefit of doing that stuff? Why not just do it, you know, within Google and that's kind of that?

• 32:33Speaker 00

  That's a great question. So I think X's initial goal was to create viable businesses, you know, that could spin out and, you know, be world changing. Okay. And so we were following that playbook, which was, hey, let's incubate it. Let's get really good people to work on it. We have really smart people. Let's not encumber the team as much as we might a normal Google team. Let them work fast, let them do what they need to do, and then let's spin it out. You know, I can't really talk about why they required it, partly because I only have hypotheses and I don't know. But let's just say that it was a tight fit with Google Cloud.

• 33:17Speaker 00

  In other words, Google Cloud offers services to customers. This was a cloud hosted service. It used a lot of storage and a lot of compute, which by the way, Google Cloud had and Google Cloud could bill for, right? And so I think that for various reasons, which I can't talk about, after we spun it out, we were still an Alphabet company. Like, wait a moment. Okay, so we were still like under the Alphabet umbrella. We were the C in Alphabet for Chronicle. But I think they thought, you know, they had competing products. They wanted to integrate them. There are a bunch of reasons that they brought it back in and sort of integrated it. So we were for a time not part of Google. You know, for instance, Google, I don't know if you've ever heard, but Google performance

• 33:58Speaker 00

  are notoriously lengthy and time consuming and you have to write pages of stuff about yourself and what you did and PRs that you did and all the, all the stuff, right? At Chronicle, we're like, we don't want to waste time on that. We want to build the project. So as soon as we spun out, we had one-page performance reviews. And I think they were in a Google slide. It wasn't even like a big page of written stuff. So, so we could move more quickly, you know? And so that was great while it lasted.

• 34:23Ryan Peterman?

  Why is it that you eventually left Google?

• 34:27Speaker 00

  That's another one which has to do in part with imposter syndrome. It's a regular thing in my career. And in part, it has to do with wanting to work at a startup as opposed to in a, a bigger, stodgier sort of Google environment, whereas things are more slower moving and there's more regulations and things you have to deal with. Not that we didn't have to deal with those things at Chronicle, but more so. Part of me didn't feel like, like there were interesting problems I could have done. For instance, the storage architecture that we came up with, part of which I built. And I think my code's still in there. I feel very proud of that. You know, six, seven years later, whatever it is. Part of that was an architecture which was very expensive. And there were ways to basically move that into different file formats and get off of

• 35:10Speaker 00

  things like spanner, which was, you know, very heavyweight. Where I could have dived in and tried to solve those problems and they'd be nice, big juicy problems. I didn't have the confidence in myself to do that. And I, you know, I was afraid, especially when you get to senior levels, there are very high expectations for you, right? And so like, if you go off and try to do something and then people are like, well, what have you been doing the last couple of months? And it didn't land and like, you're like, oh, I tried, you know? I didn't feel the confidence in our leadership that I could go off and take those risks and to have them have my back. At Symantec, I did, like, I knew my bosses for years and so they, they just knew that if I were going to go do something, I'd either be successful or if I failed, it was for a good reason and they'd just give me the rope, okay? But at Google, I didn't feel like I had that.

• 35:54Speaker 00

  And they offered me other roles too. They said, oh, you want to work on secure databases. There were some really interesting things there of like, how do you compute and do database queries entirely in encrypted space, rather than decrypting and doing it, you know, basically taking private data and exposing it where malware or other attacks could get to it. There were interesting things. I really wanted to try something, you know, sort of over cybersecurity. And while I was at X, I was talking to all the Waymo guys, especially before Waymo even became Waymo. And I was learning all about self-driving cars. And so that was what I really wanted to do. And that's why I decided to leave.

• 36:29Ryan Peterman?

  Yeah, before we get into the self-driving cars, I'm curious because I talked to someone who felt the high expectations of the highest levels was a little bit limiting or kind of put too much pressure on them. And they requested a demotion. Did you ever consider something like that?

• 36:51Speaker 00

  Not semantics. It's semantic. I could do whatever I wanted to, and it didn't really matter.

• 36:54Ryan Peterman?

  Right, right.

• 36:54Speaker 00

  You know, yeah. But at Google, I had thought about those types of things. Actually, you know, absolutely came to mind. The problem is that even at lower levels, there are certain expectations that I probably wouldn't have met if I want to go off for a couple of months and just think about something, which is the way I've been most successful in my career. And so, you know, I got to be honest with you. I'm a loosey-goosey kind of guy. Like, I can produce prototypes and optimize algorithms and do relatively interesting stuff. But when it comes to dotting every i, crossing every t, making sure I test every edge condition in my unit tests, that's not my thing. I don't enjoy that. But at Google, that's just like, that's what you do. And so for me, that wouldn't have necessarily made a difference

• 37:39Speaker 00

  because I would have had to do the things I didn't like to do in order to do the things that I wanted to do.

• 37:43Ryan Peterman?

  Okay, going into autonomous vehicles. So you went to Lyft. It sounds like that was mostly because of personal interests in the space. Can you talk about how you were hired and the story behind that?

• 37:54Speaker 00

  Sure. So at Lyft, I actually had a former student from UCLA that was working at Lyft. And he said, oh, you should come work here. It's really interesting. And I thought, they'll never hire me because I actually tried to apply for Waymo when I was in Google X. And, you know, again, this is another problem with being very senior. When you're very senior and you don't have domain expertise in a new space, they're less likely to say, oh, we'll take a, we'll, we'll try you because you're very expensive, right? And, you know, you might not have the skills that they need and, you know, they're paying somebody that they can't use. So Waymo didn't want me instead of Google or Alphabet. So I said, well, they're probably not going to want me because I don't have any experience here, but why not? And so my former student arranged a meeting with their president.

• 38:39Speaker 00

  We had had coffee and then I went through a set of interviews again, seven, eight interviews. Those did have some coding and design problems. That went fine. Fortunately, there was no dynamic programming because I can't do that. I mean, just can't like, you know, maybe certain cases I can, but. You know, I'll fail every dynamic programming interview question. And I was hired.

• 38:59Ryan Peterman?

  What were the projects like or what was the thing you were most interested in working on that you did?

• 39:03Speaker 00

  So for me, the big project that I sort of was proud of was transforming the architecture

• 39:12Speaker 00

  from one which was a classic robotics architecture, like you would have seen in the, you know, even at Waymo until probably the mid, mid little 2010s, which was largely handwritten algorithms and, you know, sort of not expert systems, but handwritten algorithms that would make decisions like, oh, it's time to do a left turn. Let's run the left turn decision-making system and figure out, is it safe? Do we initiate? Do we not initiate? Those systems, you know, in the mid 2010s were using neural networks, but they were only using it for vision. So in other words, you know, recognizing vehicles, pedestrians and so on, maybe their, you know, their, you know, angle and so on and their, and their velocity and acceleration.

• 39:56Speaker 00

  But at Lyft, they were using that sort of earlier approach, which I'm sure came up through places like Carnegie Mellon where a lot of it was hand-coded. And to me, I looked at that, especially during my time at X, seeing neural networks and semantic even, seeing neural networks and said, there's got to be a better way. Because if you start hard coding an algorithm to figure out how to do a lane change and then somebody swerves in front of you. Now you're doing an avoidance maneuver, right? Now, avoidance maneuver, do you switch out of your lane change algorithm in order to do an avoidance algorithm? Or do you stay in lane change and handle avoidance? It just made no sense. And so, you know, the team was also, the SAP was also hand parameterized. So literally, you know, they're tweaking, like, how close do we want to get to the curve? You know?

• 40:41Speaker 00

  Okay, how close are we willing to get to pedestrians? What about bicyclists? What about, you know? Okay, so if we get too close to a bicyclist today, let's tweak that and make that bigger. But wait a second, now it's going to hurt our curb distance. And so, like, there were, you know, 100 parameters that all had to be tweaked, and it was a really difficult problem. And by the way, companies like Tesla were doing this until recently, too. That approach, and Waymo was doing that for a long time, is a dead end. It's a dead end. And so, the approach that, you know, I think that probably Waymo is using now, I don't know, but my guess is, and I know that Tesla is now using, they've announced it, is, you know, an end-to-end neural network-based perception and behavior planner system and prediction.

• 41:26Speaker 00

  You know, basically, all, you know, there might be multiple heads on these neural networks, and there are multiple functions that they're, they're performing, but effectively, it's a single or small number of networks working together to figure out the movement. And so, figuring out, doing a lane change is not necessarily a lane change algorithm, although there may be a little bit of that, but it's mostly about, you know, we know we need to go there, we know there's a left turn lane, let's, you know, let's start maneuvering into the left turn lane and turning on the signal. And so, my project, the project that I was most proud of there was basically designing with the head robotics of the team, who are old school, by the way, an architecture which would accommodate, basically, an end-to-end neural network-based driver, but put guardrails

• 42:11Speaker 00

  on it. In other words, have a safety layer that would just ensure that if the network went off the rails and told it to go through a red light, we would slam the brakes. The safety layer would take precedence over the system. But the safety layer was there only for basically ensuring, you know, that collisions never happened, basically bringing it to a safe stop, or basically, you know, following legal rules that the neural network may not do perfectly. Does that make sense?

• 42:36Ryan Peterman?

  Right, that makes sense.

• 42:37Speaker 00

  And so, that was, I was very proud of that project. Unfortunately, this is a, this is like a bit painful for me. I didn't get as much credit for that as I would have liked, given the work I did. Which is like, one of the things I learned is like, you really have to toot your own horn. You have to really talk about what you're doing. Somebody else took credit for a lot of that. But the hard and really great part of that was, this was no coding, really. This was all about working with roboticists who did not want to change the approach. And getting them to consider a new approach, working together to define the approach together, rather than telling them how it should be. Cause I didn't quite know, but by the way, I thought I had a better idea, but they didn't want to hear that because I had no degree in robotics.

• 43:19Speaker 00

  And then eventually coming up with a product that was a collaboration where they were able to buy in and push it themselves. If that makes sense.

• 43:27Ryan Peterman?

  Right, right.

• 43:28Speaker 00

  And that was all like, you know, influence and not, cause like, and influence, but not based on my skill. Cause I don't, they didn't have any Thomas vehicle skill.

• 43:37Ryan Peterman?

  I think that's a common topic that people wonder when they're doing shared projects is how do you make sure you get credit for what you worked on? And so maybe you could talk about that.

• 43:46Speaker 00

  I wish I had a good recipe for that. I, you know, in general, when you're, when you're more junior, it really makes a lot of sense. In the moment, when you finish your project or you're, you're almost done with it, to take notes on what you did and what the big accomplishments were and what the metrics were. So that when it comes time to write up a promo packet, or even your sort of yearly review packet, you have all those details, which you're going to forget later on. And even like two years later, when you're going up for a more senior promo, right? So having that, I think is useful. To be honest with you, I never did that, but like in retrospect, that would have been very useful for me. Cause I would come out of it from my head and then ask people like, what was that? How much faster was that? Like what, what, what now was able to, able to detect that we could have detected before? But I think that does help a lot.

• 44:31Speaker 00

  And you have to toot your own horn, like, you know, when you're, when you're writing your performance review, without embellishing, I think embellishment is really bad, actually. But, you know, really state what you did and the value added and the sections that you worked on. Don't claim credit for everything. Claim credit for the parts that you worked on, because I guarantee you, when a committee is reviewing your packet, they're going to be like, wait, why are they taking credit for all this? When we know that so and so did all of this great work. And then you lose credibility. So, um, cause I've been on those committees, right? I've seen that, I've seen that happen. So you'll be very specific and granular about what you did, what the benefits were, how you collaborated. I think that helps a lot. When you're more senior.

• 45:13Speaker 00

  It's more difficult because it's it's a lot of soft power. It's a lot of, you know, influence. I was actually reluctant. Like, I didn't even try to take credit because I didn't want to alienate my co collaborators who also were part of this. And so I didn't go around and start talking to the president saying, we've come up with a new approach. I let them talk about it. I let their boss talk about it. And that actually hurt me because I never got, you know, when it came to review time, they said, oh, their manager initiated this project. I'm like, what? Really? This is news to me.

• 45:47Ryan Peterman?

  Oh, no.

• 45:48Speaker 00

  Yeah, yeah. So that was very, I had PTSD from that experience, I have to say.

• 45:53Ryan Peterman?

  Before we leave Lyft, I'm kind of curious, because that space is super competitive. There's like, I don't know, a billion different self-driving companies, especially back then. What did it look like for Lyft to kind of win in that space?

• 46:09Speaker 00

  You know, it wasn't clear that we had a strategy to win in that space. We were, definitely a nascent organization. I think they'd been around a couple of years when I first started versus like 10 years for for Google and, you know, X working on that technology and Waymo. So I didn't go in, for instance, thinking that I would be building the next generation of self-driving car that would actually overtake a Waymo. I went in thinking this is an opportunity to learn something new and work with really smart people. And that was my outcome that I was trying to achieve. So I don't know if there was a plan necessarily. And the division was eventually sold to a subsidiary. So they got the IP. That was great for them. And, you know, at that point, I said, okay, I'm retiring.

• 46:53Ryan Peterman?

  You know, I'm kind of curious because how did you get into actually becoming a professor at UCLA and lecturing there? I know you get a lot of joy out of it. What's the story behind going back and lecturing?

• 47:06Speaker 00

  So back when I was my

• 47:10Speaker 00

  Late teens, early twenties, I was teaching programming in a place called Learning Tree, which you probably have never heard of, but Learning Tree is a for-profit school where they will teach you gardening, guitar, knitting, and back then, they started teaching programming. And it wasn't very easy to find people who could teach programming because it was early. That was probably in 1990, 1991. So I applied because one of my friends was doing it and I really enjoyed it. And I was teaching people from DeVry. You ever heard of DeVry?

• 47:43Ryan Peterman?

  Oh, I've heard of that.

• 47:43Speaker 00

  They send those commercials, right? Well, they were trying to learn from me at Learning Tree so they could teach at DeVry back then because they didn't even have a programming class there.

• 47:51Ryan Peterman?

  Right.

• 47:51Speaker 00

  So I was teaching people and really enjoyed it and it felt really good to get up in front of people and explain things and try to be really clear. And so I had had some experience doing that. At UCLA, when I was an undergrad, I would teach little classes. We'd find a room in the evening and I'd just invite people who had problems with some of the material and we'd just go over it on the whiteboard together, or blackboard. So I enjoyed that kind of stuff. And when I was at Symantec, one of my colleagues was a guy who was a part-time lecturer at UCLA. We were having lunch and I said, oh, you know, I really enjoy teaching. And he's like, oh, you should apply. And I'm like, UCLA would never hire me. I don't have a PhD. It'll never happen. And he said, you know, let me bring you an application just in case.

• 48:36Speaker 00

  And I said, okay, whatever you want. And a couple of days later, he brings his application. He's like, here, filled it out and gave it back to him. Didn't hear anything for months. I don't remember if it was six months, a year. I don't remember how long it was. And two weeks before fall quarter of 2001, so like December of 2000, I get this call, a frantic call from UCLA. Can you still teach? Because our lecturer bailed on us. And I said, of course. So I had two weeks to plan a curriculum and basically teach undergrad course at UCLA. And, you know, it's 25 years later now, so.

• 49:09Ryan Peterman?

  I'm very glad that they did end up picking you because you are one of the best lecturers. I think so.

• 49:15Speaker 00

  Appreciate that.

• 49:16Ryan Peterman?

  A lot of my peers think so as well. And that's why I kind of wanted to ask you. How do you make these CS lectures so engaging and interesting?

• 49:25Speaker 00

  Well, first of all, it's very kind of you to say that. So there are a couple of things that go through my mind. So first thing is, remember I told you I didn't think I was that smart? So I think whatever intelligence I have and whatever that is, whatever level that is, helps me write better lectures because I feel like unless I could understand something myself, being, I think, sort of slow, other people can't understand it too. And so I try to design lectures for what I think is one of the lower common denominators, which is myself. And I don't try to teach to the top 5% of the class. Maybe that's a problem for some people, but I try to teach for the, maybe the 30th percentile or 50th percentile. And I think like, what would, what, what would I want to know if I were being taught this for the first time? I try to have empathy

• 50:09Speaker 00

  I, for the student. I think like, where are they coming from? What have they learned about? Have they do, do they even know this concept? Should I introduce this first before I do that? So I think a lot about like, I try to put myself in their shoes and ask, what would they know? What concepts were they going to be fuzzy at? Versus concepts all pretty, have pretty solid where I can just use the concept and explain it. And so that's a lot of what goes into my lectures. And so just for now, like I'm, I'm trying to prove some slides. I'm literally spending days back and forth with ChatGPT-03 discussing concepts and trying to simplify it so much, but still get the essence right. And then I'll go to Gemini and verify, see, figure out where they have differences and then all, because there's a lot, not a lot of materials on the internet that are actually really good, to be honest with you.

• 50:53Speaker 00

  Um, and the textbooks all suck too, I have to say. We talked about outcomes earlier, and for me, an important outcome is that students not only learn something, but enjoy the process. They don't, I want them to have a good time. And so I'm always thinking when I'm making slides, can I make them funny? Can I make some, some joke or something silly? Can I make them like sort of colorful or, you know, add like an emoji or something? To make it a little bit more fun so that they, there's just a little bit of surprise when they come to class. They never know what they're going to see. It might be a little inappropriate. You know, might be a little silly. Because I don't just want them to learn, I want them to learn and enjoy, they want to come to class.

• 51:31Ryan Peterman?

  One thing I'm also curious, because I think a lot of people are scared of public speaking, but I, you know, you're very good at it. Do you have any tips on speaking well?

• 51:41Speaker 00

  Practice, just practice a lot, the more you do, the more fluent you're gonna get. I even find, like, when I'm not teaching, because I'm part-time teaching right now, I'm retired, I'm taking my dog for walks and working on side projects, playing with LLMs and stuff. And I find that my speaking deteriorates over time when I'm not actively using it, even over the course of a year. And that might just because I'm getting older or whatever, but in general, I found it too, like earlier in my career. So if you want to get better at presenting, if you want to get better at communicating, you have to practice a lot. And so that might mean getting a lunchroom and giving a talk about a project you're working on. So that you can explain to other people what you're doing, even if you don't need to, you're not doing it because it's required to transfer over your project or to integrate some technology, just to do it.

• 52:26Speaker 00

  And people love that, by the way, and you'll probably screw it up the first couple of times, you'll get better and better at it. And eventually, you'll find that people will listen to you and take you more seriously if you're a great presenter, a great communicator. And in fact, story really quickly, back to my time at Google X. I remember, I actually had a talk that I used to give at Symantec, and I got permission, it was on Stuxnet, I think, and on maybe malware detection, I had another one. And I got permission from Symantec to give that talk privately inside of X. And I remember people coming up to me afterwards and saying, wow, you know, you're one of the smartest people I've ever met. I'm like, little, you really don't know. But people will think you're intelligent, and they will give you more credit, and they'll introduce you to other opportunities

• 53:09Speaker 00

  based on your ability to communicate, because people associate that with intelligence, if that makes sense. And like, I can go into endlessly about times in my career where communicating effectively helped my career. So it's super important, but yeah, practice.

• 53:23Ryan Peterman?

  You know, with all the LLMs and, you know, AI coming in, are you seeing people cheat more often with LLMs? Are people learning less or more?

• 53:31Speaker 00

  Uh, last year. In fall, I allowed students to use LLMs not to write the whole project, but for autocomplete or to write simple, you know, parts of the project which weren't really relevant to the material that we were covering. And I think in retrospect, you know, that was a bad idea because I think people were auto completing a lot more than just like this function uppercase a string or something, you know, like that kind of thing, you know, they were, they were using it in ways that hindered learning. I've recently been reading a bunch of papers about how it helps and hinders learning. And it actually, I don't know if you saw this recently, a study at MIT that like synaptic connections are down from like 70% to 50% or something. I forget the numbers,

• 54:12Speaker 00

  but like I just saw a blurb. When people use LLMs to solve a problem rather than working through it themselves. So I'm actually, this coming fall, I'm going to allow LLMs for learning, like clarifying concepts, asking for what, what does this mean? But I will not allow them ideally for projects because I believe that it did hurt understanding. And we saw that in exams. Like if you look at the exam understanding versus project scores, there's a big delta.

• 54:38Ryan Peterman?

  Perfect project scores, but getting everything wrong.

• 54:42Speaker 00

  Although I have to say, the projects were not solvable entirely with LLMs, but you, you know, with the latest models now, you could probably get a 95% on, on, on them with really bad code, but we were, we were evaluating correctness, not code.

• 54:54Ryan Peterman?

  How do you even tell if they're using LLMs, though?

• 54:59Speaker 00

  One way you can tell is that when they autocomplete, often they'll autocomplete, the LLMs will generate error checking, right? The error checking will typically have an exception with an error, a message. And the messages will be very consistent across different implementations. And so in fact, we have cheat checking software that checks N squared different, you know, everybody's project against every else's project. And we will have flags where it's like 30% of the code is similar. And a lot of it is like word for word, similar error messages and similar variable names. And, you know, like similar idioms. And so like, pretty obvious, people are using these things extensively.

• 55:37Ryan Peterman?

  You know, a lot of people are worried about LLMs kind of automating software engineering and, you know, should they even get a software engineering degree anymore? You know, what do you think about that?

• 55:47Speaker 00

  It's a great question. And I, and I think that it depends on what these models evolve into. Imagine you could literally give a project to an LLM, just like a junior engineer.

• 56:01Speaker 00

  And it would produce a component which is largely correct with tests, with security factored in, with proper modularity, DROI, all the other good stuff you want. If we get to that point where bigger and bigger tasks of more complexity are solved correctly, with good style and not no code smell and all the other good stuff, that that is, let's call that like AGI programming for a time being, for the time being. Contrast that with what we have today, which is a models that can actually solve tightly specified sub problems pretty well, build tests for them, but you still need some, you know, some supervision. You know, did it use a good algorithm? Did it like do a deep copy when it should have done a shallow copy of a data structure? Like all this stuff like this, right?

• 56:44Speaker 00

  I think if we stay in a world where we get really good, but not AGI good, based on that definition I gave you earlier, I think software engineers, software, you know, software engineering will still be a great field to get into because someone is going to have to go and look at that code and understand the mission of the company and understand the standards and so on, and then make sure that it's doing the right thing. And that requires real thinking and introspection and corrections and, you know, and even if you have the model fix things, you still have to know what to have to fix. And so I do think that having that degree and having the skill of being able to write code and read code is super valuable. And we could talk about if you want, like, but what about all the jobs going down right now?

• 57:28Speaker 00

  You know, if that, which may be caused by the, by LNs probably in part. So that's what, that's one situation. The other situation, if you truly have an AGI where you can go and delegate something to it and it will do like a L5 job, it will do a really good job. It might need a little bit of couple tweaks, but they're minor tweaks. Takes on projects that would take weeks, just gets them done. Um, I think the world's a lot different and that, and. In that world where literally all the software of whatever complexity can be written correctly, securely with right tasks, I think all bets are off and I think it's a different, a different set of skills that are necessary personally. And I can tell you what I think those are, but I think it's different.

• 58:09Ryan Peterman?

  What are those skills? Project management, soft skills, those things?

• 58:12Speaker 00

  So in a world where software can be written, like arbitrarily complex software can be written, tested, you know, good style, everything is, you know, good stuff like you'd expect of a senior engineer. To me, the world changes into a place where you're, where engineers are going to be focused, and maybe they're not even engineers anymore, on what we build, not how we build it. Okay? So what problem are we solving? Why are we solving it? I think of, in that world, the greatest engineers will be people who really understand a problem that they're trying to solve for a customer. That might be an internal customer, it might be a customer that you're like a consumer, might be a business. And you know exactly what pains they're facing and how they miss their success and what gets them

• 58:55Speaker 00

  really pissed and what is hard for them to do now and you want to make it easy? And then figuring out how to really clearly communicate to an L M those requirements to get, to do all that hard programming work that you would have had to do over weeks and months. And that is a really hard problem in its own right. So if people who have really great clarification skills, really great sort of outcome analysis skills, like what outcomes is the customer trying to achieve, what are the metrics by which the customer measures success, what's important to them, what's not important to them in those outcomes? How can we communicate to a model in a way that tells a model what we needed to do and to meet those requirements? Because models will get some of that wrong.

• 59:36Speaker 00

  Those are the people who are going to be successful. And in that world, I think you have big companies like Google and Meta and so on, Amazon. But you're going to have 10,000 smaller companies that are going to now be able to tackle problems like building software for pet sitters that never was, you know, tackled before, you know? Or building software for like doggy daycare. I think about that because our dog goes to a doggy daycare and the software just sucks that they use, right? It's really bad. If you have a bunch of domain experts who can use these tools, now you have a million small businesses each solving a problem in a way that's really perfect for those customers and not having to worry about the engineering. Does that make sense? So it's a different world. Still a lot of software engineers, but different skills.

• 1:00:18Ryan Peterman?

  What you said, it sounds like, and I don't know the product management function description too well, but it sounds like it. It sounds like someone who's understanding the customer, the business, communicating well. It's almost like the LLM is like a software engineering team, but it's, you know, a little query engine.

• 1:00:36Speaker 00

  It would be like a competent product manager. And I gotta say, like in my life, in my lifetime, I've met very few competent product managers. But yes, you could call it, product manager would be a good name for it.

• 1:00:45Ryan Peterman?

  Yeah.

• 1:00:45Speaker 00

  If program, product managers were actually competent. Both of them are not, I gotta say.

• 1:00:50Ryan Peterman?

  What, what makes a competent product manager, by the way?

• 1:00:53Speaker 00

  You know, I think a competent product manager, a lot of them are good at communicating, although many are not. But a competent product manager in my mind is somebody who really understands, again, customer outcomes and customer metrics. So like, for instance, if you've heard of jobs to be done or outcome-driven innovation, these are methodologies which are more deterministic and, and than just touchy-feely. So they actually have methodologies where they say, this is how you discover a customer's outcomes. Like what, what jobs are they trying to get done during their day? Where are they struggling? How do they measure success? Like what metrics are important to them? Like, you know, maybe, you know, like for instance, when I'm brushing my teeth, I drool, I don't know about you. Do you drool when you brush your teeth?

• 1:01:33Ryan Peterman?

  Sometimes, yes.

• 1:01:34Speaker 00

  Yeah, I drool a lot. Like, I guess like I got a lot of saliva in it. Okay.

• 1:01:37Ryan Peterman?

  Yeah, yeah.

• 1:01:38Speaker 00

  And for me, like, one of the really annoying things about brushing my teeth is it like, like the drool, do I all down my arm? And I have to rinse my arm after I brush and it's like going everywhere. And like, that's, that's a metric by which I judge whether my toothbrush is great. Of course, I haven't found a good one. Maybe I should design a toothbrush. But basically, you really need to understand the pains that customers go through and what they care about. And what's really not that important, because a lot of things that you might think are important internally, customer doesn't care about. And so I think most, most product managers are touchy-feely. They're like, well, I think I know what the customer wants. And I think I saw this really cool feature in this product here. So I'm going to go and do what they did, but do it a little bit better without really understanding what the customer is struggling with. How often they struggle with that thing? Is it important to fix?

• 1:02:22Speaker 00

  Or maybe it's not like, you know? Maybe it seems cool to you. And maybe they had that feature because they have some extra team members that could do it. But that's not necessarily the right thing. And so, yes, it would be a very competent product manager who really understands the customer. Maybe worked in that environment, knows the problems, has suffered through the problems and can then basically tell LLM how to solve that problem.

• 1:02:42Ryan Peterman?

  I see. So competence, if I'm understanding correctly, is user empathy. It's knowing what actually matters for them and building for that and communicating for that and measuring that.

• 1:02:55Speaker 00

  That's right. But a lot of people think that's a touchy feely skill. Like, it's something you just sort of develop and you sort of have intuition about the customer. I think it is a repeatable process done through interviews, done through observing the customer. It's not something that you just sort of get better at by feeling it. It can be repeated, is what I'm saying. And very few product managers will do that.

• 1:03:15Ryan Peterman?

  I see.

• 1:03:15Speaker 00

  Most of them just say, oh, I know the product. I've been working on it for five years. I know the customer. I talked to Joe last week, you know, a customer name. And, you know, this is what they really need. You really know that? Like, you think you know that, but what are they trying to get accomplished?

• 1:03:27Ryan Peterman?

  Right, right.

• 1:03:27Speaker 00

  Product managers typically think in terms of features, not in terms of customer pain points and what they're trying to get done.

• 1:03:33Ryan Peterman?

  I see.

• 1:03:34Speaker 00

  In my experience.

• 1:03:35Ryan Peterman?

  Right, right.

• 1:03:36Speaker 00

  I'm sure there are exceptions.

• 1:03:37Ryan Peterman?

  Yeah, yeah, yeah. I'm sure there are.

• 1:03:38Speaker 00

  I've just never met any.

• 1:03:39Ryan Peterman?

  Yeah.

• 1:03:41Ryan Peterman?

  Okay. You know, coming to the end of the interview, I always love reflecting back on the careers. I think you've had a full career at this point, or retired at this point. So I'm curious, like, looking back on your career, is there anything that you regret or something that you wish you would have changed that maybe others can learn from?

• 1:04:01Speaker 00

  Yeah, I mean, a bunch of things. I would say, like, first of all, I should have left my job at Symantec earlier. I feel like you should stay in a job as long as you... Or learning new things and building new skills. And as long as you feel empowered to grow and try things that might be uncomfortable for you and not have to worry about your, you know, what if I fail occasionally? Like have the space to try things and get them wrong, but to be to be able to create great things. And I had that during a lot of my time at Symantec. But there was a point where I just wasn't growing anymore and I was stagnant. And I didn't want to leave, not because, you know, it was interesting, but because I just didn't have the confidence to go somewhere else. And so I would say, there is a lot of value in staying in a job as long as you're growing. And that means maybe switching teams and staying in the same company.

• 1:04:46Speaker 00

  There's a lot of value of having that institutional knowledge of a platform that you're working on or set of systems, of your reputation. Reputation goes a long way. If you're really good in one in one area, you switch through to another team. You can use that reputation to help you in the other area often. And so I think staying in a job for five years, even 10 years can be, as long as you're learning, can be really great. I don't advise people to switch every couple of years necessarily. On the other hand, you know, staying staying too long, you know, you can get stale. It gets easier to just say, you know, I'm comfortable. I'm not really having to be challenged. So I can do my job. I can wake up and have some nice coffee in the morning and I do whatever I do. And it doesn't really, you know, when you get more senior, often you can just do that.

• 1:05:29Speaker 00

  And so when you get to that point, it's time to leave and and challenge yourself for more. And the problem with that is when you do leave, you do start over. Like I found that I had a pretty great career at Symantec. If you asked anybody at Symantec from the 21 years I was there, who I was, would know me. They'd say hi to me in the hallway. I even had people come up to me on the street because I would be on TV talking about Stuxnet and stuff. Like, you know, I was on Fox and MSNBC and Wall Street Journal, New York Times. And we were all that stuff and it was really exciting. But then you go to Google and they're like, what have you done for us? We don't care that you did those things. It only matters what you've done for us. And so that requires a lot of rebuilding up trust and building up, you know, sort of a reputation and doing a lot of good work.

• 1:06:13Speaker 00

  And, you know, that's stressful and it takes a lot more work than staying where you are. So. If you do that, you gotta choose wisely.

• 1:06:21Ryan Peterman?

  I hear, I'll probably misattribute who said it, but there's some quote about you should either be learning or you should be earning in your career. What are your thoughts on you're golden handcuffed somewhere, you're no longer learning, but you're earning a ton? Do you, would you still advise that that person leaves?

• 1:06:39Speaker 00

  I would never say, I would never say leave if there's a good package. And you know, we have to optimize for multiple things in our life, right? Learning is definitely important, but also being financially stable. And different people need different amounts of money to feel comfortable about being safe in their life and having enough to take care of their family or themselves. I think there is a place for both and it can be okay to be comfortable and making a lot of money for a while, but I think that if you're solving for enjoyment and fun and hard problem solving, which a lot of people are, that can be toxic over too much time.

• 1:07:14Ryan Peterman?

  Definitely. It sounds like early in your career, you were lucky to find what you enjoyed. A lot of those early problems were super interesting. Do you have any advice for people who want to find what they enjoy in software engineering?

• 1:07:29Speaker 00

  Yeah, for people who are in college now, graduating soon, I would say like try lots of internships if you can. And that might be difficult now, given the way things are going with jobs right now. Everything's cyclical, but right now, jobs are tough. I found cybersecurity entirely randomly. Yeah, wasn't something I was like, I want to be in cybersecurity. It's like, I had an internship, my first year was doing file management. My second year was doing blah, blah, blah, third year, oh, viruses. The more you can explore, the more you're gonna potentially discover a passion. And like, I think there are many jobs where you would think it's gonna be totally boring. But if you go and start working on the problem, you're gonna realize there's really interesting problems to solve and you might find a passion for a field that you would never expect that you would enjoy.

• 1:08:14Speaker 00

  And then you're like, wow, this is really interesting. There's really fun problems, like the people I'm working with. Customers are interesting, or maybe they're a pain in the ass, but that's interesting to deal with. Like. I would say just try things. Don't try to wait and find the perfect job. Find a job that where you have a good manager. There are some hard problems to solve. And you don't know anything about it and you'll learn. Like that's, you might find your dream job and that lasts to be 25 years.

• 1:08:42Ryan Peterman?

  So before you know what you enjoy, you're saying, you know, search with breath. And then, once you find it, then you can kind of go deeper.

• 1:08:50Speaker 00

  That's right. I mean, look, you might get lucky. If your first year internship is really interesting and you're doing well, I would stick with that. You could try breath there, but I would say like a greedy breath.

• 1:09:02Ryan Peterman?

  The last question I have for you is if you were going to go to yourself right when you were graduating from UCLA and give yourself some advice, knowing what you know now, what would you say?

• 1:09:13Speaker 00

  It wouldn't be one thing. I mean, the biggest thing would be don't let fear of failure hold you back. You probably can do more than you think you can do. I might have had a richer career had I listened to that. I would say focus on the outcome. So whenever you're working on a project, think about who's going to use it. What do they care about? What, how do they measure success? And then just optimize for that and don't like try to make the perfect thing or don't try to make a very, you know, round thing if all you can use a little square piece here. Like often problems can be solved without having to be perfect and still be really good. So like focusing on people's outcomes. Big one is when you're, when you're presenting, for example, like this had to learn. I used to go into presentations and talk about the technology to senior leaders.

• 1:09:58Speaker 00

  And like, in retrospect, those senior leaders don't care how the algorithm works. They don't care. They just want to know, is it going to be faster? How much faster? Is it going to generate more revenue? How much revenue? Is it going to, you know, fix a problem that currently takes 20 people to do it, make it 10 people to do it? And so you really need to get in people's heads and think about what they're solving for. Again, this whole idea of outcomes and then speak to their needs, not your own. So that's a big one. I said, find a good manager. I'll just say that again. Manager can make or break your career and your life and your like. Happiness. So, you know, a good manager that trusts you and gives you some rope is super valuable. Learn how to collaborate with people, just like, don't assume that you're

• 1:10:41Speaker 00

  right and just tell people they're wrong. You have to really learn to work with people and understand their point of view. I still struggle at that, but you know, it's something, it's a work in progress. Those would be big things.

• 1:10:52Ryan Peterman?

  Yeah, yeah. Well, thanks so much, Kerry, for your time. I really appreciate it. I was really looking forward to it. I think there's a lot of stuff in here that people are going to benefit from. So thanks so much for your time.

• 1:11:02Speaker 00

  It was my pleasure.

• 1:11:04Ryan Peterman?

  Hey, thanks for watching the show. I don't sell anything or do sponsorships, but if you want to support, you can subscribe on YouTube or you can leave a review on Spotify. And I'm always looking for new guests to interview. So if anyone comes up who you think you really want to hear their career story, let me know and I'll try to reach out to them and get them on the show. Thanks for listening as always and I'll see you next time.