Latent Space: The AI Engineer Podcast
Original podcast
Watch the full show on YouTube: https://www.youtube.com/watch?v=HQucglw-4t8
10,097 words · 2 speakers
Okay, so here we are in the LinSpace studio with Vivek Ravisankar of HackerMan. Welcome.
Hey, hey. Thank you. Thank you for having me in the podcast. How are you?
Uh, good. I mean, I am here in Hawaii. I'm on a offsite, but you know, the show must go on. And we've been trying to schedule this podcast for a while. So I was like, you know, I'm not gonna make you reschedule again. Like, you're a busy guy. You've got lots of things to do. But like, um, you know, like, I think the state of hiring is always like very interesting. People are cheating at exams and then people are using AI to also like evaluate candidates. It's just a mess. So, um, I guess like, what's the state of like engineer hiring these days? What are you seeing?
Wow. Yeah. Let's, let's start right there. Hey, first off, love your commitment. You're on a vacation, looks like, in Hawaii or wherever you are.
Offsite, offsite.
This is the cognition company offsite, yeah.
All right, okay. So then you're working, um, but still love your commitment on this. What's the, what's the state of hiring? The state of hiring is in a very, very interesting, interesting spot. So maybe I could just like take a couple of dimensions in terms of, in terms of looking at it. So I would say, if I have to think about the overall jobs perspective, I think Indeed, uh, Indeed has this, Indeed.com is the number one job site. Reasonable barometer for developer jobs, not, I wouldn't say it's like the perfect one. But if you look at the, if you look at the job board index, you would find that the developer jobs index has flattened year over year, 25. Actually, I do have some stats. Do you want me to share?
Go ahead.
All right. Hey, so, so this is the Indeed's tech job posting index. Um, again, Indeed is the number one job site, reasonable barometer for developer jobs, I would say. The blue or the purple is the, is the one for tech and the green is, is the overall. You could see, I think they reset the index to 120. So you could see a very, very sharp rise in 2021, uh, early 2022. This is the ZIRP era. I think people were hiring like crazy. And then you can see like the get fit era, it declined pretty rapidly. So I think the amplitude of the swing of tech jobs is way higher than the overall one.
But if you zoom in a little bit, you would actually find that 2025 job index is flattening relative to 2024. So the decline has stopped, but it's nowhere close to the peak, nowhere close to 2021, not even like close to 2020. But on the other hand, if I were to just like, take a look at the AI specific jobs. This is from a, this is from data from a, from a website called TrueUp, which, which does a pretty good job in like AI jobs and others. If you actually take a look at it, the AI jobs tell a completely different story. This is all of the AI jobs that are posted on the site are very heavily focused in, in, in Bay Area. A tell a, tell a very, very different story.
Then there's another analysis done by, I'm blanking on name of the, this is like a data-driven VC firm. I think Signalfire, yeah. Yeah. They did a pretty, pretty good analysis on this one, where this is another dimension of, hey, what is happening in the seniority of hires? And what you're starting to see is the big tech companies as well as startups tending to hire more towards senior compared to, compared to new grads. This is like probably broad-based things of what we are seeing in the market, which is, hey, the overall tech jobs is, the decline has stopped, which is good. The AI jobs are on a rise and companies starting to move more and more towards like senior hires.
That's what the market data says. One of the interesting aspects that we have is, I mean, like we work with like, we work with close to 2,500 companies across the globe, across sizes. This is all the way from a seed stage company to Fortune 500 organization. So we see a lot of interesting data and we're starting to see companies hiring more aggressively in both AI and probably the traditional application developer roles. I think the slope is starting to shift. We're also starting to see companies hired much more on early talent and new grads. So we would see a very different chart. I predict that you're going to see a very different chart on both the slope of the hiring curve
as well as the demographic of the type of hires in the next six-ish months.
Oh, what happened? Why, everything you said until that point was expected and then this is new. Like, why, it seems like the trend is reversing. Maybe it's good news, but if we don't have any explanation for it, I'm a bit wary, you know?
Yeah, yeah, yeah. I have to say something controversial if I'm going on a podcast, right? Or something new, so this may be. So I'll talk about this. We talk to a lot of customers. I talk to a lot of customers. It's everything from OpenAI, Nvidia, Amazon, Salesforce, all of these. We work with all of the customers. I think over the last maybe 12 to 18 months, there was this notion of, hey, AI is just going to continue to get better at programming, which it is, or coding or software development. And most of the work that junior developers do is essentially just like coding. So we're just going to have AI do the job and you're just going to hire only senior people. And now people are starting to realize two things. One is, well, you always need a human in the loop. AI is really good, but like you certainly need human in the loop. There's a lot more work that you need.
And second, if you really want to become AI native as an organization, I mean, guess, guess, who? Guess the people who are actually AI native? They are like new grads. They are people actually coming out of colleges. And so if you want to change the culture of your organization, you need to infuse that talent. You cannot, it's actually a little bit of a boneheaded move to like, you know, completely stop hiring people there. And so I think like a lot of companies are starting to realize that. And that's a shift that we're seeing. Now, again, it's not bounced back to where it was, but the slope is changing. And I'm actually quite bullish that you're going to see a much different trajectory for new grad hiring over the next six to nine months.
Yeah, one of the most consistent things I hear from like OpenAI and like the coders inside the big labs is that the existing coders don't change their mindset about how to use coding agents as quickly as the people who are, as the Gen Z interns that they hire, who come in and like, just like, you know, trust the model. Most of the time it fails, but like twenty five percent of the time it works. And it works in a surprising ways. And then they are like, oh, like, let's, let's double down on that. And that's very interesting to me.
Yeah, yeah, hundred percent. I mean, like, I would actually say the, even in our own company, it's been way easier to get new grads and early talent just like be much more AI native. Then I have to like push really hard on the senior engineers and others like, guys, like, why aren't we using this? Yeah, we're, we're starting to see that come up more and more. And there are like, and, and the new grads who are coming up, they're not like, you know, okay, I just like graduate, I just took these courses, I got a four GPA and I'm, and I need, need a job. They are like real ten X people who can like shift the trajectory of your organization. We have a few of them as well. I'm not going to name names because you're going to hire them. But like, you know, we, we, we have some, some of them in our company as well.
Okay, so I skipped over introducing HackerRank. I think a lot of people have heard of you, but maybe like, maybe not like the most recent version of you. What are the surface areas of things that you built? All I know is interview platform. You have a community, you have like presumably like an employer side that like, and I also saw on like your website, you have some kind of internal skills discovery ranking type thing. So what, what are all the things you have?
Yeah, yeah, yeah. Maybe we should have started here. That's okay. Maybe you can edit this.
No, no, no, it's, I like, I like it's starting at the top and then, and then, you know, we'll weave in things as they come up.
Yeah. So yeah, maybe I'll give a little bit of background. I'm, I'm, I'm the one of the founders and CEO of HackerRank. Started the company about twelve years ago. I grew up mostly in India and I moved here to start HackerRank. And we were the first Indian based company to get into Y Combinator. I really wanted to be in the Bay area, I really wanted to get into Y C. So prior to, prior to starting the company, I used to work at Amazon in India as a developer. But I used to interview a lot of people. And found that resumes didn't have a great correlation to your skills. I'd interview people who were really good on paper, but completely bound in the interview and vice versa. And that was the motivation for us to start the company.
So I started this company with my co-founder Hari, who I went with, who, who I studied together with in college. And he's also one of the, one of the, one of the, uh, one of the guys who, who has quite like very low GPA, well, like the best hacker in, in town, right? Like, and so he'll always like get rejected by all of these stupid application filters that would just like filter by your GPA and all of that. So we started with the notion that skills matter more than pedigree. That was the core ethos of the company that is still the core ethos of the organization. And the first product that we started was like take home assessments, which is instead of looking at a resume, companies could actually create their own take home assessment, give it to candidates. Candidates can solve that and depending upon how well they solve, they can decide if they want to move forward or not.
This was a much better way of showcasing skills over pedigree. And then we realized that there is an aspect of using this ethos across the entire lifecycle journey of a developer, which is in terms of how you prepare for a job, how you get hired and how you upskill yourself within the organization. So today we have five products that span across the entire lifecycle journey. We have a community that's now over 28 million developers where developers, it's free. I mean, there is a paid version, but it's predominantly free where you can come in, learn, practice, hone your skills on badges and, and others. We have a product called Engage, which our customers host hackathons in our community to attract developers. We have Screen, which is the take home assessments, which is where we started.
We have Interview, which is a paid programming interview session that you can do. And Skill Up, which uh, helps you, your existing engineers to upskill themselves to be the next gen developers. So these are the five products. We have um, close to tw- uh, between 2,500 to 3,000 paying customers, 28 million developers. And um, yeah, we're, we're, we're, we're continuing to grow and we see, we see a lot of data.
I mean, that, that is a humongous amount of data. So, okay, what are your strong beliefs now on what is next engineer versus a regular engineer? Like, how do you detect for that?
Yeah, no, it's a great question. I think here is how we think about the next gen developer and we distill it down to four attributes. The first one is you need to be very strong in the fundamentals of software engineering. Yes, AI is going to do a bunch of things, but you still need to know the fundamentals. I mean, just like the way we think about it is, you know, 90% of your flight is flown by AI, but you still need the pilot to know and article engineering. You wouldn't jump on a plane if your pilot is a white pilot, right? Like, I'm sure like you'll get off. So you still need to know the fundamentals of software engineering. And that doesn't mean like, you know, trick questions and things of the signs. You need to know code complexity. You need to take a look at the code and say, well, it's often or often squared. You need to know the basic data structures. You need to know the building blocks and all of those things.
Second attribute is you need to be able to use AI across all aspects of SDLC. Well, that was supposed to be the job of a developer, but we got so bogged down with just like writing code and all of those things. And we started to have like specializations across each of them. So I think like you should be, now that AI agents are getting better in each of the aspects, you should be able to work across the SDLC using AI. The third attribute is every new technology comes with a set of concepts that you need to like learn. And specifically, so you need to have a very strong knowledge of AI concepts. And this is everything from prompt engineering to fine tuning models to setting up MCPs. And like, there's just going to be more and more of this that's going to come up. And finally,
you need to have a lot of taste and good taste and business acumen whenever it comes to building products. So this is how we define the next gen developer or the 10x developer. You're strong in fundamentals of software engineering. You can use AI across all phases of SDLC. You have very good knowledge of AI concepts and you build with good taste and business acumen. And this is how we've structured our company. This is how we're structuring our products to help developers become the next gen developers, to help companies hire the next gen developers and to upskill the existing people to become the next gen developers across these attributes.
Yeah, I mean, I think that's, I mean, the AI piece must be new, right? Like, how did you, you know, develop this over the last three years?
Yeah, yeah, it is new. So, um, so one thing that we, we, we realized and we did was our goal has always been like, hey, whenever it comes to evaluating a developer, we need to make sure that it can mirror your daily job, the day to day job. And if you think about the evolution of sort of technical interviews in general, I think, I guess, like maybe in the 1990s, people used to ask questions like, how many golf balls fit in a seven four seven? Um, you know, things on those lines like to, to just like evaluate your computational thinking and stuff. It has got nothing to do with, uh, programming. And then, and then we started to, uh, ask people to write code on a piece of paper. For what it's worth, I got my job at Amazon. This was like, certainly I'm updating myself over a decade of writing code on a piece of paper.
And then, and then we shifted, but it was still like data structures and algorithms. And then we shifted the same writing code on a piece of paper to online. Then we started doing tie projects. And now where we see companies going is towards evaluating them on real world tasks on code repositories. That's where like the world is going. And when you, when you think about like evaluating real world tasks on code repositories, then your environment, which is the IDE that you use and all of those things also need to mimic the real world. So we took a lot of inspiration from the cursors of the world or the wind surfers of the world, whatever, whatever it is, like in terms of like IDE, Copilot next to it, building code repositories and making this as real as possible. So that's been the evolution for us. Uh, so we've always believed in it.
We also think the next evolution is going to be beyond code repositories, um, because again, as much as, as much as IDE and AI assistant and code repositories, like the home for, for, for developers, that's not the only place that they work in. It's always like the prod is down. Uh, okay. So you go to, uh, Sentry, you look at logs, you, you look at a bunch, bunch of things, then you go back to your IDE, you try to fix. So the next evolution of interview might be, might even be beyond, uh, just like a code repository set of tasks, uh, with an IDE. So we think the, uh, we're pushing the bar, we're pushing the industry to make your interview process mirror your real world work as much as possible.
Anything on coding agents, like the cloud codes and all that? Or I guess Copilots or I don't know what the, what era we're in.
Yeah, yeah. Well, you must be knowing more than me. You're not close to it.
This is my whole life right now. Yeah, not only do I work at Cognition and Windsurf and then I also, you know, am like organizing this A I E code thing, which, you know, you're definitely invited to. But like, it's like, yeah, everyone's launching a terminal. Everyone has like the cloud thing. Codex launched the cloud thing, Cloud Code launched the cloud thing. And like, I don't know if it's like part of the skill set to like try everything or just pick one.
The way we think about that is, I mean, like, the way we think about that and we define our hiring process also like that is there are two major work types of workflows in today's world that developers use. One is you're going to tell an agent or actually a bunch of agents go do some stuff. And once it's done, you're going to review the P R. That is one workflow. The second workflow is you're going to use A I as a co-pilot to actually do the task. These are like the two workflows. So the way we test for the usage of agent stuff is less about can you prompt the agent to do the work? But more in terms of, hey, I've actually generated a bunch of P R's from Devin or Cloud Code or Gemini or like Codex or whatever. Can you go ahead and like review and how good is your reviewing capabilities on that?
And then, of course, on the A I as a co-pilot or an assistant, it is here is a code repository, here is a, here is a feature that you need to build, go build it alongside co-pilot. Of course, you can also use an agent, but like, you know, that's, that's, that's kind of how. But those are the two predominant modes that we see. It's also likely the reason why Cognition also believes that way, right? Like you guys acquired Windsurf, which is more of an I D E co-pilot as well as like Devin is more of an agent. I think, I think we're going to find these two as the predominant workflows. Will one supersede another? I don't know. We'll have to wait and see, but as of now, I see both of these workflows existing hand in hand.
Yeah, I, I, I think so too. What I've been calling this like the sort of async value of depth. Like basically you want either like the the sort of super local. You're working on the hardest problems, you need the the the agent to help the human. Or the other side where it's like super async, you don't really care. It's like just changing a button, something that's super commoditized.
Yeah.
So you could set it up to Devon or whatever.
Yeah, yeah, yeah. Anyway, like even internally, we're we're getting to in our company, we're getting to a point of it's not just like developers, but solution engineers, support people, being able to tag something like a Devon to say like, hey, there's a typo in this edit message, go go fix it. I mean, like you don't want to bother engineers for that. It's it's always in the low priority task, never gets done. So, yeah, so we're we're we're starting to move in that direction.
Yeah, it's really like it's great for the marketers, the PMs, the CEOs who like, you know, should not own the code, but at least can like prototype the PR and say.
The CEO shouldn't just like misuse and just like cause a bunch of tech debt which engineers have to go and fix. So I'm very I'm very calm in that of that.
Yeah. Well, so, you know, you have five products. Would you still so say like the the interview stuff is the the core business?
Yeah, I would say the screen and the interview, the two of the five products is about like eighty to eighty five percent. And then I'd say skill up is starting to be like close to tenish percent, engages five percent. Community, we just introduced pricing plan, so it's a very, very small percentage.
How do you price community?
Yeah, so we are so so a bunch of all of the challenges are free. And now we are introducing two plans. One is a plus and the other is infinity, where you could just have AI tutor for for all of the challenges, what you can actually learn from it. The plus has limited limited amount of tutor that you can use and infinity as like unlimited, you can just like use it for as much as you like. So that's that's our that's our pricing plan.
Okay, so you're you're killing lead code, basically.
I guess Clouly can lead code.
Okay, okay, let's start the Clouly discussion, right? Because you're dancing around it.
Yeah, yeah.
Yeah, Clouly came on, see, was there a major AI cheating tool before Clouly? Or was it Clouly that just drew a lot of attention to it?
Interestingly, I mean, like Roy and I used to chat, like, you know, when he launched this, this whole thing. I guess now he's become a big celebrity. He doesn't respond to my DMs much, but anyway.
Actually, Clouly is not the one. It's actually Interview Coder. It's just that he got popular with Clouly. Interview Coder is the one. Were there other tools before that? I'm sure it was, but I think like Interview Coder just like caught the wave. And then there were like a bunch of other tools that started to, that started to come through. By the way, it's not as invisible as, as the website claims. Yeah, especially on Mac and things. And there are like ways that you can detect. We can, we can talk, we can talk about that. Yeah, it's, it's not, it's not as clean as it claims.
So like, okay, that, just for everyone who is like not as clued in as us. The idea is that it's like an overlay on top of your screen while you're on Zoom with somebody. And one, while you're talking is transcribing and it's trying to feed you answers, right?
Correct, yeah, yeah.
And then like people, people complain like, oh, like, you know, I see the guys like looking at, looking at this corner of the screen. Therefore it must be cheating. Maybe, maybe not. It's just like, you know, on the spectrum, I don't know, but like.
Yeah, so let's, let's maybe talk about how, let's maybe talk about all the ways people cheat. Okay, so I've done a PhD on this, so we'll talk about. Or, or, or maybe, maybe I'll, maybe I'll phrase it in a slightly different way. Talk about all the ways that impact the integrity in your hiring process. So I think like major, I'll bucket it into three things. One is. You leak the questions online and people actually see it and, and you know, you know, the answer beforehand. The second is you use these unauthorized tools, whether it's interview coder, Clouly, there's one called UltraCode, a bunch of, bunch of things that, that, that, that come up. The third is impersonation.
I ask you to take the interview or the test on my behalf. So these are the three ways in terms of how it actually does. On the, on the, and let's talk about like, you know, what is our stance? The way we define integrity is it's not about whether you use AI or not, it's about whether you follow the rules. So we, we think about like integrity, when we think about like, when we think about building integrity, Jen is, hey, how do we, how do we make it very clear to the candidate that these are the rules? And how do we make sure that you can prevent the candidate from sort of, sort of going away from the rule? And third is if the candidate does exhibit suspicious behavior, how do we actually flag that both at a candidate as well as to the company immediately? I mean, that's, that's, those are some of the principles that we follow whenever it comes to the ensuring integrity of the process.
So question leak, we actually run scraping across a lot of the different websites, a lot of different forums to actually see whether questions have gotten leaked. Can you, Chegg is the biggest offender, but I love Chegg as a company and things on the site. I know they're going through hard times, but rooting for, for them. I hope they come back.
Oh, no.
But, but Chegg is like right up there. There are other forums, StudyX. It is simply leak, we, we actually don't see a lot of questions getting leaked on Leadcode. I mean, there, there are, but, but it's not, it's not in the top three or the top five. So we actually run our scraping mechanisms across a lot of forums, a lot of them, a lot of these sites to pull them up. We then do DMCA, sometimes they, sometimes they take it down. Actually, most of the times they do take it down, they respect and take it down.
Do you do this for free or what? You're, you're, you're running your, your customers' questions on?
Yeah, customers' questions are on created like your questions.
So as part of your service, you run DMCA takedowns?
Oh, okay.
Yeah, yeah, yeah, we do that, yeah.
And they all respect it?
Setting?
They all respect it? They all respect the DMCA?
Pretty much. I think like, you know, I'd say like ninety-five percent plus actually respect it. And we are also very cognizant of, you know, sometimes people actually upload this on like some GitHub repository. And, um, and, you know, when you put a DMCA, like, you know, you want to shut down the entire GitHub repository. We got caught once like that in Hacker News. Uh, boy, like, you know, you get, you get like that on Hacker News, it's just like, uh, it's, it's just like the giving like a very softball to, to a bunch of hungry lions, like, you know, they just like come and pounce you. Oh, you guys suck, you guys suck, blah, blah, blah. So we, we had to like tweak and make sure that we're very confident that it is the leak question. And like, you know, when, when they go in DNC, there's like a human in the loop before you actually submit the request and all of those things.
So we've done a lot of work in making sure, I mean, ultimately, ultimately, we're striving to make sure that we have like a fair process. Uh, you know, and as much as customers wanted, it's also the candidates wanted. Like I've seen like candidates DM me on LinkedIn and Twitter, uh, or X, saying like, hey, I want you guys to up your, up the bar in terms of how much you think about like caring about the hiring process and integrity. Because I'm doing it in an honest way, but the guy next to me is not, and I don't want like, you know, lose the opportunity. So it's as much a big deal for the candidate as much as for the company. So that's the first pillar of like the question leakage. The second one is all the unauthorized tools that you use. So a few things that we do here.
One is, um, we have trained our own plagiarism, uh, model, which can detect, uh, whether you're using an external tool to write code or not with about like eighty-five to ninety percent precision.
So we have actually trained, so which means like whether it's whether it's Clouly or any other mechanism that you actually use, there are some coding patterns. Everything from uh how quickly you solve to how many times you edit certain things. I'm sure like now people are watching this as though like, oh, Vivek is open sourcing the algorithm. Let me try to like game this like, no, it's going to be very hard for you guys to game it, but I'm okay to like, you know, just like tell it out. But like, so we we've trained our own sort of sort of plagiarism model to detect that. And on top of it, we have like varying degrees of protection that we give for our customers. Like, so we have introduced this thing called as a Proctor mode, which which asks you to share your screen. It can look for applications. It can take screenshots on a regular basis.
We also have an app which which is the next level, Proctor mode plus plus, which now can access your operating system. So it can actually shut down apps like Clouly's of the world as well. So we have like different mechanisms from right from basic plagiarism detection of coding patterns. I say basic in the sense of like, you know, the first level all the way to, yeah, download an app and it can actually shut down other things. The third one is impersonation. We do a lot of image verification. We also like suddenly you would find like two people coming on the frame. Suddenly you find a different different person in the frame in the middle of the test. So we we see we see a lot of them. So we actually like, you know, look at images. We we parse through to make sure that it's
the same person. So we do. We do all of this. I'll say one more thing. I know I've been on a big monologue. Above all of this, we are embracing AI. I think it may have been Steve Jobs, but I but I don't know. This might be one of the quotes that always gets attributed to somebody, but they never said it, right? Like, I don't ever think Henry Ford said, if I ask people what they want, they would have they want a faster horses.
It's not him? Oh, shit.
I don't think Henry Ford said that. It's widely attributed to him.
Clearly, it's Albert Einstein said it.
Sorry?
I'm joking. I'm joking. Everything leaves. Yeah, yeah. Everything leaves Albert Einstein.
I missed that. I missed that. Yeah, yeah. So I don't know. I don't know. This might this may fall in one of those things. I think it was like. Hey, the best way to embrace piety is to make it official or something on those lines. And I think he was referring to Nap Services versus the iTunes Store. You know, yeah, you're illegally downloading and stuff like, why don't they just like make it official for you to make, for you to just like download it. And so, hey, if you want to prevent people from using all of these AI tools and stuff, why don't you just like give AI as part of your, part of your assessment, as part of the hiring process?
So that's, so yes, we're going to do all of these things, but more importantly, at the top, we want to make sure that like people get access to AI assistant, which also means you also need to think about like a different kind of problem. It cannot be the standard lead code, road stuff that you want to do, which will mean it's going to be more real world, more code repositories. It's also something that developers really love and enjoy much more than than the normal one. So that's the direction in which we're, we're heading.
Yeah, amazing. I mean, I, I think that's like, that's a really encouraging, I think that's, um, I guess enlightened or inevitable, actually, almost is like the best word for it. Like you can't avoid it. Like, you know, you might as well get with the trend.
Yeah, I think like, does it defer for more senior versus more junior now? Like, you're seeing that, I guess, there's a lot of fear that there is, that there's a massive decline in junior hiring. But we just talked about how junior might be bouncing back, they may have more skills in the agenda coding.
Yeah, any, any differences on seniority side?
So first of all, I, it's, it's bouncing, it is, the early signs are very promising, just so you know. Because we look at how many people are companies sending assessments to junior versus senior, how many people they are interviewing. Grown about twenty to twenty-five percent year over year, um, relative to all of the other, other news, news items that you see on social media, where it's like, hey, declining, everybody's going to be out of jobs and stuff. And I actually expect it to continue. In terms of the hiring process, yes, it's certainly different. I do think the fundamentals aspect remain the same. But like how deep you actually dig into varies a lot. I think for a senior, it's much more around like system design and other aspects of it. So I do think, I do think that changes and.
And I also think the more senior that you go, I think you you have a bigger ego. So anytime you send a take home assessment, you're probably not going to not going to attempt one. So I I guess like, you know, you reach a different kind of a stratosphere for like principal engineer and others where you're not even going to attempt to take home assessment, you're going to directly go into the interview. So from a rubric perspective, I would say new grads, fundamentals and like much more hands on coding with AI assistant. For senior, it is still fundamentals, but but much more of code review and architecture and system design and and other aspects of it. And they're like, whatever, super senior or so, the principal engineer is mostly going to be directly on interview. And yeah, and and if you have the meta label, you you got to probably pay five million dollars.
So so so so that is there as well.
Are you seeing a lot of meta AI candidates because they just got laid off?
We haven't, yeah, the the the the recent one, we haven't seen much, but I also think they're like AI researchers. So we don't see a lot of AI researchers yet coming through. I actually don't don't know how their interview process is. My guess is like, there's such a dearth, they're just like, oh, you're an AI researchers, come here, here's a million dollars. I I don't know. That's kind of like what is happening. I think it's probably getting pretty crazy.
I've talked to some researchers, so Yee Tei is a friend of mine and he's leading thinking at DeepMind. No, mostly you can tell when you ask them training questions if they have the right intuition. If you're like, what if I move this hyperparameter this way, what what would happen to the model? If they don't have an intuitive sense, if they're like, I don't know, I have to check. Probably they don't actually have the hands on experience.
Interesting, interesting.
So like, at at some point, if you're really in ML, and there's like, you know, two hundred, three hundred of these people in the world, they all know each other. And so like, yeah, I mean, if you if you know, if you've been through that kind of training or that that kind of practical hands on experience, then then you know. And and like everyone else, you know, has has more traditional ML work. I mean, it's not trading a frontier model and that's that's a different job market. And there's there's also a market for that. I think like the the interesting thing also, I was I was kind of curious, like, so you know, some of your clients are like the the big labs and I think like, you know, there's a lot of people that want to join the big labs. You don't have to name anything in particular, but like, you know, what are they looking for? If I'm preparing for an interview, like, what do I do?
I'm going to say something, I don't know if it's controversial or not. The AI native or AI forward companies
are the least AI forward when it comes to interviewing and hiring process. I think Anthropic is not our customer yet, so I'm not going to talk about all about our customers.
I don't want you to get in trouble.
I might get in trouble, but Anthropic has, if you go into their career space, I don't know if they still have it. They had it like about a couple of months back. They make it very clear that you should not use AI when you're applying to the job.
Actually, they had, let me see if I can pull that out. They had this honor code. It's about a few months back, so I don't know if you still have. Hey, while we encourage people to use AI systems during their role, please do not use AI assistance during the application process. We want to understand your personal interest in Anthropic without mediation through an AI system. And we also want to evaluate your non-AI assisted communication skills.
It's very interesting, right, in terms of
in terms of what they do. So, so, so what is, what, yeah, so that's probably an interesting one. The most AI forward companies, if I, if I think about it, are not as AI forward as you would expect in the hiring process. The companies that are pushing AI hard, but it comes a lot from services organizations. These are very large services organizations, consulting services organizations. I think I can understand why, because it's, it's like a margin business. You are selling human labor and if you can actually make it more productive is actually good for the, for the team, for the company, for the organization. They are actually much more AI forward and pushing through to get this. So that's what we see.
Although like I, I know every company is working towards becoming much more AI first, which is, and when I say AI first in the hiring process, it's going beyond the traditional algorithm, data structure stuff to code repositories, having AI assistance built into it. So, yes, so that's a shift. It's, it's much more early in the AI native versus the large Fortune 500 organizations or the large consulting kind of, kind of services organizations that we work with.
Yeah, that's surprising, but maybe also the way, you know, they have a very high bar and you know, they they they want to make sure that you jump through the same hoops that they went through when they they were hired. So, exit, you know, like you don't have to focus too much on hiring, you know, I know you're you're sort of very broad, general, you know, you've a lot of opinions on the AI landscape. Also, just kind of like stepping out of your role at HackerRank, like, what what what else is sort of like top of mind that you know, that you really wanted to riff on?
There are a bunch of things that that I that I think about, that I think through. One is, why is why is AI not as helpful as it's as as it's projected at work for non-developer roles?
Okay.
It's it's not a, yeah, so it's always puzzling. It still doesn't create good presentation, still doesn't create good financial models, still writes like pretty poor docs. Even in the developer world, I think there's like mixed thing in terms of productivity gains, like people say, oh, my God, like, you know, this is, that's that's what you get on social media, right? Like, oh, this would have taken me a week. I did it in two hours. Sure.
The the consultant, yeah.
Yeah, I mean, especially the presentation one. Everyone seems to love the slop. You know, there's all these startups that are putting out presentations and like, and it's like AI-generated image, this is like kind of doesn't mean anything and just like takes up space. And I think like, I think my my my learning from that is, actually, people want slop. People deal with all sorts of human-originated slop slop all day long. So replacing with AI slop doesn't really matter. And, you know, you you your standards are high. You think that, oh, I would never do AI slop presentations. But I tell you, like, there's millions of people out there who, like, their standards are not that high.
Yeah, you know, one interesting thing that I've observed is people, I mean, humans are just like wonderful. I have a new found respect for humanity after all of the AI. We are adapting so quickly that now humans can easily, pretty easily distinguish whether something is done by AI or human. Or at least like something is done entirely by AI or human. I've seen like people react to, oh, this doc seems like it was written by AI. There was a guy who actually came and did a presentation for us as part of the interview. It was very obvious and was done by AI. But that's not, that's not the interesting thing. The interesting thing is not only are people able to discern how much of AI was used or not. But if you used a lot more AI than human element, you start to ascribe a lower value to it.
Oh, this was actually mostly done by AI. I'm not very impressed. What did you really do?
And that is, that is starting to be an interesting aspect that we're figuring out. So, so, so I actually think the more AI becomes powerful, the more valuable human craft and other elements are going to become. Because people are going to easily determine, oh, this is all done by AI and just going to attribute a lower value.
Yeah. Oh, yeah, I think it's interesting. I'm actually a little bit more open to it in the sense that if it's helpful, it doesn't really matter if it came from AI or humans. But when the human submits it to you, they, they sign off on the whole thing. It means they, they stamped that they review it. So if I come across slop in there, I don't care if it's AI slop or human slop, I blame you. Right? So you better come to me, you better come to me with like good stuff. If your judgment is poor, whether or not you wrote the thing, I don't care. If your judgment is poor, if you cannot see, if you cannot review for me that you are, you have like slop in there that should be taken out or edited by human hands, then you probably shouldn't work for me.
Yeah, I mean, in the sense that can you use AI to, can you use Midjourney to create beautiful images?
Yes, sure. It's a different kind of brush. Yeah, totally.
Yeah, but you can, you can, I don't know, maybe I can discern somebody who just said, go create a mountain or something versus like.
Putting in the effort of like, hey, this style kind of painting and like oil pastels or colors and things on those lines.
Oh, yeah?
There's like a diff.
At least see it, or at least I'm being spoiled by my, by our design, design team. Where it's very evident that I would have just done some very high level things versus like you're actually creating prompts that make sense. Like, for example, like, we're, we're in the process of even like evolving our brand and all of those things. And one of the things that we are leaning in is the idea of, I think, human craft is going to start to get more and more valued as AI becomes more powerful, which is the irony. And so all of our branding is going to be much more analog, much more about like blueprints and things which you would associate with humans. And even if you're using the journey, it would have this sort of oil painting kind of a feel or a texture to it. Because like, you know, humans like paint.
So all of our branding styles and images need to showcase the human element in it. And so what are all the, what are all the things that humans are known for, craft, like, you know, painting, blueprints and analog stuff? That's, that's all we're doing. And so it's important to not just generate any random image just because it looks good. But how do you actually generate with much more thought? So maybe, maybe, I mean, those are the things, right? Like, you're easily able to differentiate from somebody who actually used the tool to get what you want as opposed to just like, oh, I'm just going to put a few prompts and see what comes out of it.
I think that is a really interesting art. We're actually thinking about creating a second show for latent space called High Taste Testers. That where basically it's like a small little competition. And one of them would be prompting image and video models on like getting good output. And then each, it'll be filmed in a restaurant. Every course will have a different prompt and then everyone just like competes.
Nice.
And we will, we'll read it. I think it would be kind of fun. But yeah, that's cool. I, I mean, like, yeah, I, I think like the automation and knowledge work, you know, like the number one thing that's at PMF, which is recording sales calls, which is Gong, which is, what's the other one?
Uh, you didn't want to get all that?
Good on us. That's the one that, that's sort of like the, with a more like AI native one. But like, they all do the same thing. Notion has it. But I, you know, I think like AI for non-coding use cases, you know, coming back to the original topic, is starting to percolate out. And I think it's just, it just takes a long time. I think Arc, uh, sorry, Atlas from OpenAI is like very interesting. It's a browser that does work for you. Obviously super limited, but like it, it is a huge progress since the first operator. Which which is only in January.
So like.
Wait, they operate in January? Wow, I thought it was like. So you know that, wow.
Cloud Code was launched in March.
That's crazy. Yeah, yeah, yeah, yeah. Our timelines are all getting compressed. What do you what do you think about Atlas?
Atlas, I like it. As far as OpenAI launches go, it is one of the better ones. Because a lot of them are like very vibe coded. This one is less so. But I am very spoiled by Ark and Dia, right? Ark, actually, because of the tab groups and the vertical tabs and all that.
My god, I I was about to say the same thing. Like, I cannot believe they launched the Dia without the side tabs and without the functionality of side tabs. Like, it's it's still on Ark, by the way. It's not agent taking and stuff, only because it has the side tabs and tab tab grouping.
Orion, I've tried Firefox. This is the best one, right? The the problem is, I think, so and and then, you know, just to close the topic on Atlas. The head of Atlas has tweeted about, oh, like, people asked him, like, can you please have the vertical tabs from Ark and the spaces and the tab groups and all that? And he's like, well, have you seen our side scrolling tabs? Like, we put a lot of work into that one. And I'm like, OK, this guy's never shipping it. He doesn't care. So like, I think there's a difference between like the general population, which only, you know, is just like. Which I think that's what OpenAI is going for. And then the pro users, and the pro users, we're just going to use probably the browser company's stuff, and that's not surprising. I think like, what's interesting is actually, what is Google going to do? Because now, oh, you're coming for Chrome.
Like, screw you. Like, I'm going to change my license to ban you. Like, probably they should at least do that.
You know what's funny?
Google open, quote unquote, open source transformers, Google open source Chromium. And literally, like, you know, those are like the two technologies that all of these companies are being built on. Yeah.
But like, you know, this is open source as it should be, right? You know, like, okay, you know, like, haha, they like, didn't, didn't like capitalize amazingly. But they're still Google, they're still, they're still huge. They're still catching up, like, uh, you know, and in some ways arguably number one on, on depending what metrics you look at. And they have like an enormous runway and like infinite money, like, I'm not crying for them. They open sourced it and, and people are using open source as intended. This probably accelerates humanity because they made the decisions to open source. If they did not, we will be stuck in a Google monopoly for like the next hundred years.
Um, so like, we should just be grateful and not make fun of them. Because, because like, you know, they, they just have to catch up and, uh, you know, on their own merits instead of, uh, using lawyers. Right. Which I think like is the right way.
Yeah, I think like, uh, the other thing I'm, I'm really interested in is like on the Arc Dia side, the founder side, right? A lot of founders are going through AI native reinventions, basically, right? You have a company pre-AI and then I know now everything is AI and you're now you have to decide. Whether or not you like have to sort of pivot the products. And that's exactly what the browser company did. They have Dia, they have Art. Uh, and I think the problem with their customer base is the customers are forcing them back to the pre-AI version. But they're like, no, no, no, I don't want the new thing, just give me the old thing. Uh, and that's very scary for a founder because you are trying to do a brave new thing and your customers don't let you. So you're, you're, you're trapped.
Yeah, yeah, that, that, yeah, what you're describing is innovator's dilemma, right? Like, um, you know, this is the future, but your existing customers are like saying like, I'm satisfied with my current thing. And the more you satisfy your existing customers, the higher the probability of you dying, which is like the irony of the innovator's dilemma.
Yeah, no, I mean, whether or not that actually applies to Arc and Dia, I don't know, because it's just a feature. It's not a, it's not a product, it's just a feature.
Yep, yep.
But like, I don't know, you know?
I mean, I would literally change the Dia to brought back the vertical tabs and, uh, it's literally that one thing that's just like keeping me in Arc.
Yeah, yeah, yeah. So, uh, the, the head of Eng is coming to speak at AIE and he's, he said, yeah, they're working on it. They just, they, they tried to say no, they, they, you know, like part of the product job is to say no, even though people say they want it. They, you have to simplify, simplify, simplify. But I think the, the demand is too strong. Oh, uh, well, thanks so much for, uh, you know, your time and like sharing your wisdom and, and stuff. Do you have any like conception, like philosophically or company wise?
I'd say, I think there is, there is a lot of, there are a lot of questions around, hey, should I become a developer? Is the developer job going to go away and things along those lines? And as much as it may look like it's going to be self-serving for Hacker Naver for me and things, I actually think you're going to see a proliferation of developers. I think you're going to see way more developers in, in the future. I think the job role will change for sure. I think it's not going to stay the same, but I think you're going to see way more. In fact, we're actually starting to see developers are not, right now, they are not even like contained into the typical R and D role. They're now starting to spread across the entire company.
The hottest role right now, other than AI researcher, is probably forward to plant engineers who are now like in, in like go to market. And then the probably the second hottest role, uh, or whatever, third hottest or whatever is the go to market engineer who are like really like orchestrating all of these tools, whether it's clay or, or, or any of them to be much more personalized outbound emails and stuff. We, we do, we do quite a bit of that as well. So you're actually seeing the flip of it, which is engineering is actually entering across all of them, not just R and D. So I would actually say you, we are going to see proliferation of more vibe coders, more production engineers,
more developers across the code market or other non-technical orgs, uh, or at least what are classified as non-technical orgs. And I, I think it's, it's a shame that some of them are giving advisors to don't, don't learn to code and all of those things. I also think like coding is a way of sharpening your thinking. I think it's helped a lot, even like when I think about like building a business, it's always like an ambiguous problem. You try to make it much more concrete, you take it into sub-problems, you go ahead and solve that. That's a very effective skill for you to go ahead and like learn even in business. So not only is it just going to sharpen your thinking, it's like writing, just like helps you to make your thing clear. Um, I also think you're going to see like way more developers in the future. So that would be my closing thought, go learn to code.
Yeah, I strongly endorse this. It's one of those weird things where like if you believe a little bit in AI in AI encoding, you do what you say. If you believe too much, then you're like, oh, it's gonna take away the job, so I don't bother. And I think there's like a happy middle. It's weird. I'm trying to get, I'm trying to get Singapore government to encourage people to to learn to code. Like, I wanna change our national second language from Chinese to Python.
That is cool.
Yeah, haha, but like, no, like in schools, elementary schools, replace every single like, like slash S slash Chinese slash Python, right? Like, just search and replace and you will, in in thirty years, you will have the strongest national coders you ever see.
Yeah, yeah, I agree, agree. Let me know if there's ways that you and I can partner on that. I'm working with the government of Rwanda.
What is Rwanda doing?
Who wants to want to like create a million coders from Africa.
That sounds base.
High schools and others, so.
So like, but I mean, their their practical, their practical thing is like, they're gonna train all these coders, you're gonna spend all this time, and where are the jobs? Right, like, so I think like, we have to like, think about what you just said. I think it's the first time someone has explained to me that like, yeah, actually, basically, every other function in the company is turning into engineering.
So that's where the jobs come from, right? It it it's just not the traditional sweet job. It is F D E that can code. And like, I I used to be in finance. And guess what? When I, when I was a trader and I was young, I'm no longer young, but when I was young, and all the older guys are like, hey, this thing requires some coding, you look young, can you figure it out? And I did. And so I became a trader who coded. And then that became, it eventually got called the quant. And and then that's that started my coding journey. But I I never had, I don't have a software, I don't have a C S degree, nothing, like, I just, you just learn it, yeah.
Yeah, skill over pedigree, right? That's that's what it matters.
Uh, cool, it's a real pleasure, uh, and I'm sure we'll catch up, uh, in person and and we'll we'll we'll, uh, get you on the the A I U stage at some point, but, uh, thanks for your time.
Yeah, yeah, absolutely, absolutely. Continue your great work. Thank you so much for the invite and, uh, yeah, looking forward to getting this live.
Page updated October 24, 2026 · transcript and summary are machine-generated
