Latent Space: The AI Engineer Podcast
Watch the full show on YouTube: https://www.youtube.com/watch?v=-rIAVuaRjOg
7,540 words · 2 speakers
Okay, we are here in the remote studio with Ahmad Weis. How are you?
I'm doing great. Thanks a lot for having me.
Yeah, you and I have known each other since before AI. You were active in the WordPress community. I don't know how I first came across you beyond that. I think just general web stuff.
Maybe Devrel, maybe Devrel. I think before you had joined Netlify.
Yeah, but were you ever a professional, Devrel?
Yeah, I think I used to lead, like, I was VP of Devrel at Rapid API for a while.
Oh, that's right. That's right. Yes, yes, yes, yes.
That is as professional as one gets, right?
No, no, no, because I always see you as like an independent creator type person.
No, I like worked at a bunch of different places with Google, with Airbnb, mostly mostly been like this open source guy where I have like published like 300 plus open source repositories.
Yeah.
Everything is semantics after that, you know, my open source work took me places in, you know, like, you know, like you, I create a lot of content, so, Devrel.
Yeah, okay, so tell us about the path into command code and then, you know, we're going to highlight some of the work you did recently.
I think the story kind of starts at COVID, you know, I basically did a Corona CLI thing that went viral. So COVID was at its peak. I was traveling a lot being in Devrel and whatnot. And I think Greg Brookman and Sam Altman ended up giving me access to GPT-3 early. So since then, I've been an AI engineer, right? And the first thing I did, in fact, I was just looking at it. It was July 2020. And Greg sent me a message like, what is the use case? What are you going to use this API for? And I told him, I'm going to suggest the next line of code, like a code snippet, right? This is year and three, more than a year before GitHub Copilot was a thing, right? So I started building this thing called CLAI. I've always been a big fan of building CLI.
So, you know, silly side project, CLY, right?
I think that has eventually became command code now, like, you know, a short history is we ended up building in the AI cloud, now everything is in AI cloud. It was called Langbase, it grew quite big, 1.2 billion agent runs a month. Ended up building a memory infrastructure or whatnot. And now we've sort of like pivoted into this feeling that there is only one type of agent and that is a coding agent, it can do it all, right? So why hide that capability behind some memory system or some primitive or whatnot? In that, a six-year-old codebase has eventually turned into this thing we call command code, right? And command code actually started with this feeling that I was using command code a lot more than other coding agents personally. And then a couple of team members started adopting it.
And we started ending up building this meta-neuro-symbolic model called, you know, Taste1. The thing it does is, like, I have a lot of experience with code. Like, I've been, I think, writing code for 27 years or something, right? After publishing 300-plus open source projects, you get to have a lot of opinions on things. And I mostly find myself working on things that are super cuttingy, so there are no docs that an AI agent can go read or whatnot, right? So at that time, I feel like, you know, my opinions matter more than what NLLM can actually find or what you can do with RAG or whatever. So I ended up encoding this behavior in meta-neuro-symbolics, a neuro-symbolic architecture, where if you learn something from me, documented for me like a skill, right?
And we started calling it Taste, right? If you see me prefer PNPM a lot, but publish or link my local CLI with NPM, globally, right? Did learn that I prefer PNPM for installing packages in almost every other thing, but when I'm linking my CLI locally, I'm using NPM. So these type of things and learnings. And that eventually ended up becoming Taste files, which are very similar to skill files. You can think of it like command code automatically learns from you on a per-repository basis. So your team, right? And it builds a library of skills, which is quite less verbose, right? They're not like, you know, everything is not in there.
It's like things that it sees as like repeated preferences and patterns across your work, right? And it could be coming from so many different coding agents or whatnot. When we, when you merge something into main, that is when we can trigger what were your accepts, edits and rejects overall. But that's not what we are here to talk about. You know how the silly thing about anything in tech is? The thing that you think will go viral or will get adopted? Never does. And the thing you basically just, you know, off the cuff ignore like, who cares about open models? It becomes a thing, right? So it is twenty fifth of May today and about literally twenty five days ago. Everybody was talking about how DeepSeek is so good.
And there were a lot of people talking about how DeepSeek is so bad and why are people saying that it is so good? So I was in the, in the camp of, I, I need to make a decision on if DeepSeek V four Pro is actually as good as Opus, right? And we are doing anywhere from like, you know, a couple billion tokens a day. At that time, we are like hundred times more than that now. I ended up discovering a thing that I call tool confusion, which is very odd. We, we have been able to figure out how to deterministically fix tool calling for open source models or open models. It's been such an amazing thing that a lot of people have been trying it out through a command code. And I also made a bunch of that like completely open. You can, you can go and implement this in any coding harness.
Because you know, me being me, this is not that important, right? So yeah, let me know how, how deep you want me to go into that topic and I'm very excited to talk about it.
Yeah, I, I think we can get dive right into it. I think you have a viral post that you want to share. And also I think frame the problem for people who maybe are only used to OpenAI, right? They did, they don't know what a tool calling problem is.
Think of it like this, from our whole vibe is command code, right? You, you should be able to command your L L Ms to do whatever you want them to do, right? That is, that is the vibe we are going for, right? First thing was to build taste so we can steer models in the right direction. And then the next best thing happened, you know, programmers stop writing code. So who are we going to learn taste from, right? So we started learning taste from models because models are writing code. If you spend a billion tokens on Claude, on G P T, on DeepSeek, you discover a lot of discrepancies across the stack and different full stack systems. Like DeepSeek is not that good at code code for some reason. And all of those, I started calling them variations.
And we discovered a deterministic pattern across those variations. So when you're running a coding agent, tool call on a very basic thing is like, you know, if you're trying to run a bash or shell tool to discover something like you're trying to list what the directories are, you're trying to read through a bunch of different files, user has asked about a question about, you know, maybe your, how is authentication built in this repository? And your coding agent is right now trying to figure out with a bunch of tools that are pre-cooked in that I need to list all the files, read a bunch of them, explore them, and then answer the user, right? So what of, what, what a lot of open models are doing is, I think they somehow suck at
tool calling, right? And the pattern is not like super broad, right? It is like, you know, when, this is very specific to DeepSeek. So DeepSeek V4 Pro has this weird alpha maleness where whatever it sends you, it thinks that that is the right thing to do. And if you, if it is sending you wrong schema of the tool calls and you send back a ZOD error, it doesn't listen to you. It would repeat that same thing for like fifty-six times on average in a billion tokens, where you'll be like, why are you, why is it doing that, right?
Why don't, why don't they listen to the errors? I, I thought that's like a very common thing, like instructor used to do that.
I have no idea, like, you know, as a programmer, I would think that, you know, if my tool call is failing, all I have to do is send some schema back, like ZOD error and an LLM that is smart enough, right? And it will just do it, right? My gut feeling is, it, it might be a hot egg because they are, I think a lot of these open models are built or trained in a system where they are learning from data, which they consider really high quality, like they are learning from a better model than them or something like that, right? And their training is whatever you are being told is right, it's correct. So their entire nature is whatever I am telling you is also correct, so don't try to correct me. This is, again, quite a lot of this is vibe based.
And I discovered this because like it's it's a silly thing like for example, I shared it in my post as well. Do you mind if I share it like share the screen or something?
No, go ahead. I think you have the ability. We tried tested this before we recorded.
Yep.
It's easier to for people to look at. They can look up the
Yes.
while we're talking as well. Uh just make sure you zoom in, obviously.
Yes. So this was the post, right? So this was me being like, you know, I just discovered why and how DeepSeek can outperform OPUS four point seven. So this is OPUS four point seven, not four point six. Four point six is a better model, right? So in the thing that we discovered was like it was sending in wherever you have some tool like a shell tool and it has some parameters or arguments that should be optional. It would send some weird thing on that like an empty object. Or or a null where it is, uh it doesn't belong. And ZOD being super strict would just strip up and send back that error.
And it will get back the same result again and again, same tool call, right? So instead of sending back that error, I ended up repairing that, right? It started with just like three thirty two hundred lines of four repairs. Think of it this repair logic like, you know, database migrations, you know, you have one migration per file. So I ended up creating repair files. Like if you see something like this where you know it is emitting, you know, JSON strings type of data when I actually wanted an array. I can determinately deterministically fix that to an array, right? And when I do that, I will not only just send back the result. I will also send back a note, a hint, a repair hint
that you know, you should have sent me this type of data, but here is the result anyway. Think of it like you're teaching somebody how to drive a car. And they are about to, you know, hit another car instead of telling them what to do correctly. You will first try to save them and then you will explain like why you save them, right? Like how what they should have done in the first place. I think a lot of these models. Really like this repair logic because what we saw is the moment you send the result with the repair logic, right after that, the third tool call is fixed. Instead of, you know, it all of a sudden becomes super smart. It understands like, okay, I got the result what I was looking for and I'm I'm gonna do this.
And it shows up in so many different places as well. Like for example, it is trying to read a file. And it is not giving you the offset of are you trying to read the hundred lines from the top or at the bottom of this file? So I just make a judgment call like, okay, it's the first time it's going to read that file. So let's give it the first hundred files of first hundred lines of that file, right? And then it realizes very quickly, oh, I was actually trying to read a log file and I needed the last hundred. And it can very easily understand instead of, you know, those fifty plus on average tool call failures. And something like this should be very obvious to developers, but what's happening is a lot of developers actually use Claude code.
They hack their, you know, change the base API endpoint in the, you know, API key. And they try to use Claude code as the harness for open models. In Claude code, uh, you know, you know, they hide a lot of the errors behind Control O, right? So you don't even know that, you know, you have like fifty plus tool call failures plus per session.
You're just sitting there and you're like, oh, why is DeepSeek so slow? And they're, it's in their vested interest to not fix that, like they don't care about open models. They didn't build their coding agent for open models. So it works out really well for them, but the common thing that the common theme across Twitter is, oh, it's so good. And oh, it's so bad, it's super slow.
So I feel like this always ends up being a tool call, a harness issue. Then, you know, an actual model issue, like it can be as silly as something like this. When it's sending the read file path, right? It would create some markdown link in it for no reason at all. And this is super deterministically fixable, right? So you don't have to like waste your tokens on it or anything like that, right?
Yeah, I think that's.
Go ahead.
Well, I mean, I, I think the, reading the data and actually looking at like, what you actually send, I think it's, it's really helpful. Being helpful, like, I wonder if it's only DeepSeek that's like showing this kind of stuff, or is it just a general open models trick?
Yeah, I think the first thing, again, me being me, I thought this is just DeepSeek. Then I looked at our, you know, logs for last thirty days and Kimi is doing exactly the same thing. Then we fixed the, uh, Kimi models and we fixed Minimax models and now we have like sixteen thousand different repair. You know, variations across hundreds of billions of tokens. We are doing anywhere from six hundred billion tokens right now. And the data on failure of those tool call is super, super important. In overall, this goes from a model that was practically completely not useful, like DeepSeek, uh, V four flash
to something that can actually compete with this, right? Like, I, I, and this is like more of a vibe check when I pushed out this update. One of our investors, uh, Tom Preston, when his fund, uh, PW, their G P, he was like, what did you do? Why is DeepSeek V flash? V four flash super solid now. Like the vibe of the model completely changes. It starts doing things in a different way. I don't know if you've seen this, but like if you run any coding agent with permissions on, the models are actually number. And if you run them without, uh, you know, the complete bypass of permissions, they do much better. Even if you like sit through those yes, yes, yes, accept or whatnot, you will see that, you know, the model ends up
getting steered in the wrong direction because of the slowness of permission blocks or whatnot. Maybe that's not how the models are trained or whatnot. But it's, it's the same thing with models. Like if they are seeing a lot less tool call errors, they are much more creative. They are mu- they can explore a lot and they can continue a lot longer. Like one of our users has actually done like, I think seventy billion tokens on DeepSeek. I was looking at his data. He broke our usage page. That's how we discovered him. And he's like, I constantly run deep secret command code for 12 plus hour long sessions. I personally have not done that, right? So I mean, I think a lot less tool calls are happening. Tool letters, tool confusion is happening.
This repair logic kind of blows you away, like, you know, how good open models can be overall.
Yeah, that's a really fascinating. I mean, I guess. What else have you found? Is there, is there ongoing work? And it's been, it's been a little bit since you've done this. Uh, you've generalized a bit. Uh, is there ongoing work in, in this area or, uh, you know, are you sort of basically constantly spinning out insights like this from command code?
Yeah, it's like, so one of the first things that happened was I think we might be the most used coding agent out there right now for DeepSeek. So we are doing so much inference on DeepSeek. So a lot of people actually tagged a bunch of people on DeepSeek research team. And to prove this, actually, we launched a go plan with just dollar one per month. To, where you can do like 600 million tokens of D C P for pro in it. Just to prove like open models are actually really, really good and they are catching up, right? And I think that kind of correlated to, I, I, I, I, I think I would, it wouldn't be too far to say that DeepSeek saw that
they can discount their prices and do the same, ensure people that their models are actually really, really good. But also I saw a lot of people from different coding agent harness companies being tagged, uh, in the street, like, you know, like, is, is this thing doing that, right? There's like also Py where people can easily just put this entire thing into a prompt and get Py to fix itself, right? So things like that. But one of the most interesting things that just recently happened was we have been able to apply the same thing to design slop.
You know, like, you know, that indigo purple gradient thing that all L L M's do?
I think Mario Zechner has a current viral post about it right now. Basically, if you, if, are you familiar with Mario Zechner?
Yeah, yeah, yeah.
Yeah, yeah, if you go to his profile, he has an example of design slop.
There you go, right? I think everybody just knows. So I love the purple color. And I would like to point out that it's the indigo slop that is happening, not the purple slop. Right? But, but what we have found is we can deploy the same model to fixing design slop. Across the hundreds of billions of tokens that we have done. We looked at the numbers and we found that obviously we chatted with a bunch of amazing designers with amazing design taste. And we found out that it is also very similar a problem. There's a finite set of things that most L L M's do.
And if you can give them like a compositional framework of sorts, like you can repair their design thinking, the same thing applies. Right? Like, for example, these are like probably ten big rules, you can put them in a skill file or anything. And you will see your design get better. We've seen this repeated across different L L M's, commercial or open whatsoever. And it's the same repair tool logic of sorts, right? Like you are basically guiding your coding agent that, you know, you're not going to do this, this, this, this and this. And if it does, you are deterministically fixing it. Like, one of the things that you might find really good here is, first one was this.
The work of pattern first composition where when you ask a model, go and design me this dashboard, they generally do not think about the intention behind that design. And they just slop you with that, you know, three cards in a row in, you know, one border on the left side or top side or whatnot, a pretty common thing.
If you give them a very simple framework of, you know, what type of surface area are we looking for? Which is literally just these seven patterns, they do really, really well.
And things like.
Where, where do you seven patterns from this? Did you just come up with it?
Yeah, I talked to a bunch of different designers.
Yeah, I'm just wondering if there's like a book or some, you know, yeah, this is like lore that people just know.
People, I think you could, you could think of it like, you know, when a designer is designing a dashboard, they're thinking about this is a monitor surface, right? Like we are trying to monitor, that is the intention behind this, right? And this is across the chats we had with a bunch of amazing designers. This is how they work. And we thought if we could turn this into a slash design skill, see how L L M's will repair themselves. We also discovered something really, really weird. Like I personally don't use OK L C H, but apparently L L M's are really good at it. And if you see them using H S L or something, they are, they, they, they, they don't actually
are able to control the lightness in H S L very quickly. But onto human eye, it's very, very easy to see like this color and this color do not look the same. But if you force an L L M to use OK L C S, they can control the color palette really, really well compared to any of other things. And this is what a lot of my designer friends do as well. They love.
Yeah, I mean, I think they, that's the reason we invented OK L C H, right? Because they did, we realized, I mean, every time color theory advances, so that we also have to change the C S S functions, which is mildly annoying for learning C S S.
So, yeah, it, it, I can't shake this feeling, but it's like, I, I actually wrote about this just last night. This is from last night. Like, I think they're, they're, the design slop thing somewhere here, like this is new. So I'm struggling to find where that is. But we only have like twenty four reference documents, ten, you know, design smells and seven patterns that we saw across different designers. And when we designed a bunch of different landing pages and got those designers to look at them, it takes about one point five second for them to spot, oh, this is from A I and this is somebody human looked at it, right? So the differences that they spotted when we wrote them down, they are just
deterministically fixable patterns. Like you can convert H S L into OK L C H, or you can force an L L M to do that, right? And it's like a, feels like you can fix ninety percent of a design slop.
Which is not a capability gap. It's more like a contract gap in what your harness is telling NLLM to do. Versus what your user is saying. User is always going to see, fix my design. Make it prettier, make it pop or something like this. If you can give them a framework of this is what the design taste of a really good designer is like. They will pick this type of color scheme, they will pick. And they will think about intent before starting to implement that landing page. What is the intent here, right? It's a short contract, but it's. It makes your design slop really, really minimal, I would probably add, right?
Yeah, I think this is a really good overview. And for those who don't know, like Ahmad actually has a lot of authority also on the design side.
I actually don't even super know everything that you've done, but you know, I definitely will listen. When you have thoughts on the intersection of design and code.
Thanks a lot. I feel like I wanted to be a designer. I was never really that good at it, you know, like I couldn't sketch or anything. But I think this is, this is a feeling that I have, like a lot of people are right now able to build just about anything. And we are now differentiating between good work and bad work based on their design. So enabling every builder using a coding agent to be able to design like a designer. Is something that is very close to my heart, right?
Yeah, I think that's some, I think we're all a lot becoming more generalist. And there's definitely things that I would not dare to do previously. That I, that I'll now try with AI.
Like.
Yeah, it looks like you want to show us command code. Yeah.
Yeah, it's like, for example, this is the landing page of our documentation, right? As a developer, I would not have gone through the trouble of creating all of this, right? But when I fed that to our design scale, so command code now comes up with a. Comes bundled with a slash design scale, which has like these, all of these references in there. Cooked, pre-cooked in, like I was just last, last night fixing our deals that we have, like Q and two point six is, you know, fifty percent off. So I fed that a very basic screenshot of, you know, all of this mess and this is what it converted into.
And it, to me, it looks really, really good. I'm like, okay, this is, this feels like, you know, like a ticket that you could print up, you know, on a, I don't know, like a cinematic or whatnot. And from deal that it basically understood the intention behind this thing and tried to recreate that design. When I, as a programmer, only told it, here's a deal, here's the data, just add that to our docs, right?
So giving that framework of, you know, this is not like tool calling or anything. It's like a really nice framework of thinking that L L M's have this capability cooked in. They can organize their thoughts really, really well. They can design really, really well if you just give them the right way to think about those things, right? Same thing, uh, we did with, uh, tool calling, same, same concept basically applies to, uh, you know, design. And one of our team members right now is working with a community, uh, of security people. They think that the same problem can be applied to security as well. Like you look through the logs and you figure out that this is the most common pattern.
Security wise, this is what is, uh, brewing up and they can apply automatic fixes to your packages or whatnot and make L L M's really, really kind of like coherent on following your guidelines of never write bad secure or bad or like poor secure code.
Yeah, I think like that's something that we all want to like get smarter on. Can you give us like an overview of just like the differentiators that, that, uh, people should be aware of? Because I think like, this is one of them that's like, that you've been working on, um, you know, taste is something that you've, you're, you're definitely focusing on, on owning. Are you still pursuing your own models? Or are you mostly just gonna be like the best CLI for open models?
Yeah, I think there are like two directions right now that we sort of own. Uh, the thing is, again, this is like a CLI, right? It is like full-fledged coding agent does everything that you can expect any coding agent to do. It has both, uh, you know, commercial models or open models. It's just that that we have found our PMF of source in the open models market. More than, you know, Claude is actually really, really lenient for tool calls. So even if, you know, your coding agent harness messes up, it can figure out that, oh, I'm, I'm being sent this error and can fix itself. Not the case with, uh, you know, uh, open walls. But the taste thing is, uh, still there, like, uh, it basically sort of like works like this.
This is a very common, uh, site that I have and I'm using something, please use TypeScript. So using TSE, I, I want you to use TSA, uh, Tsub, right? It's using some different framework of testing. I prefer Vtest. And it's like a lot of, you know, back and forth in getting your coding agent to do what you wanted done in the first place. But with taste, this taste is like a meta-neuro symbolic model. Like I have a lot of, uh, I tried to like, based on the feedback, I've hidden it very well in docs somewhere. So, you know, developers don't have to go and read the, you know, silly, silly things. Right?
The entire KL divergence loop that we look at, like if NLLM already knows about something, it should not end up in your, you know, skill or taste file. That is absolutely useless context, right? So it basically does all of this weird thing where once you actually go through, where is that link? Once you actually just go through all of this, once you have built a CLI or an API or a front-end project with command code, it actually ends up learning a bunch of those rules that are automatically managed for you. So for example, if you are using commander for building CLIs and now in this particular project, you start using Miao, it will replace that for you.
The entire idea is that your skills are being automatically learned and automatically managed. And they are absolutely transparent. They are in your repository, not in our model. So you're reviewing it in every PR, you're looking at like, yeah, I don't want to. You know, say something like this or whatnot, and they're never still. A lot of issues that I feel a lot of people face are because their agents dot M D or Claude dot M D has some wrong information when they sat down. Uh, this is like one thing I discussed uh when we launched, we announced our, you know, five million dollars seed. That a lot of people, when they sit down, they think of the rules or whatnot, they think in the terms of grandiose things, right?
Rules are like, you know, let me actually zoom in. So this is like the difference between our skill or rules file or our T S file, right? This is what you are writing down. In T S S, what is continuously being learned from your prompts or your edits? And they are being stored in the same markdown file for you, right? And updates are when you remember to do it. In most of the time, I've seen we, we, as humans, we sit down and we do this grandiose thing. Like, you know, use this, I always prefer that or whatnot. And tastes are a lot of micro decisions, not too broad. When you're doing this, when you're running up slash P R Q command.
Sometimes Emmet prefers that, you know, you basically fetch the latest from main. Rebase it, send the P R on a branch and then go back to the main branch. That is what my P R workflow is, that is not what my P R file says. So it automatically goes and fixes it, right? And I, it has seen me go back to the main branch again and again, it's like, okay, it's just one liner for it. But improves my workflow without me having to take care of it. And over, you know, over the time, it basically compounds a lot. Like if you have, one of the guys I was looking at his code, he built one Android app with command code. And now he's like, you know, I'm able to build more Android apps very, very quickly.
It's, it's the same thing, but it is learning the skill from you as you build those things, right? And over the time, we have seen like, you know, uh, it becomes really helpful.
Like somewhere down there, we ran a study with like seventy plus developers. In the number of times that they had to go edit files because their L L M. Made a different. You know, the scene took a different turn or steer their L L M like, yeah, don't do this, don't use this. Don't do use T R P C or something and use Hono or whatever for this part of A P I. They found that the number of edits or steers went down.
So, so let me ask some clarifying questions.
Yeah, yeah.
You, you, you, on the website, it says a model, but then you also say it's like portable. Uh, between systems and stuff like, is it a, is it, is there a system of files? Like, obviously, this is a, a form of memory. How does it compare with skills? Do I, should I prefer skills? Can I use them side by side? There's all these questions.
Yeah, yeah, yeah. So one thing is, I actually got this question so many times. I have like a written like, like a blog post on it. Uh, skills versus taste, right? What is the difference? The, at the very basic layer, T S is the highest order bit, which is managing your skills and rules. Skill T S is this automatic engine of sorts that, that is creating skills for you. Making sure they are not stale and you can obviously go edit them yourself as well. It, and overall, it actually looks like this, right? The, after looking at. I think I probably have built more than seventy C L I's with command code so far. And this is the entirety of my taste for building C L I's.
This little thing, right? So it knows that, for example, uh, I. Use P N P M only build dependencies thing. It knows that I always prefer starting from zero point, zero point one. It's like those silly little things. And uh, this is like as, all I have to do is I have to do this N P X taste pull. We have the taste package as well, by the way, right?
Nice.
Yep, it's, and it just pulls this particular file, puts it in your. Uh, repository and then all you have to do is ask any coding agent and follow my taste of building C L I's. And build me a CLI that does this, this, this and that. And then by the end, show the T S compliance. And it would, any L L M would just go through all this list and be able to figure out. I, I, I was supposed to use this. I was supposed to use CLAC for interactive things or whatnot. It works out really well. And then when you start continuously building on top of it. And you change every project, every CLI is different from other CLIs. This taste of sorts changes in that particular repository, right? So it lives in your Git repo anywhere, right?
And all of this was automatically created, but you can, yeah, you can come in and edit it to yourself as well. It's a Markon file.
Then the confidence, I guess, is L L M generated as well, right? So if you
Yeah, but like, but like if you are
Always do it. It's one, you know? Yeah.
Something like this.
Like the, the problem we actually hit was the funny story there, we actually tried to hide this early on. We're like, if we don't show you the learnings and we just made you compare Claude code versus command code. And developers had like this wow moment like, oh wow, this is so good. Um, they didn't know what was being learned. But the problem we had is we don't know how to merge different taste files. It is extremely a, a human endeavor. Think of it like fifteen different engineers working on a GitHub repository with thousand different GitHub branches. And taste files are so different in all those taste, different branches. And when you're merging that back into the main, we cannot judge what you want to keep or what you don't want to keep, right?
So we made it really transparent. And this is before their, uh, skills existed, by the way, this is from last October, right? There was no concept of skills. There was APIs, CLI's and MCPs. And we ended up creating this and then, you know, the skills thing came around. So if you as a human are writing something, I think you, you can just probably write confidence score of one, you know what you are talking about, right? If something is being learned from you, some of it is really good, some of it is not. Like for example, I always have this local option, which I don't want to show up in my help commands. Because this is a option for debugging local. Uh you know, setup of my APIs.
So very little thing, I will never personally go and write this learning in a skill file. It's just too minor, right? Uh for me, but it's really good that, you know, uh it picks up on small things like this. And works it out, right? Uh one of the things that I've seen in our Discord community, which is blowing up uh after all of the, you know, tool confusion fixer. Is a lot of people, what they are doing is they're building one project with a really high quality L L M, like Opus or G P T five point five, right? They are building a T S file and then they are using, you know, super cheap models to continuously build on that more, you know, project with that T S file.
Yeah, I mean, I think, I think that's a, that's a system that kind of works, makes sense for me. Cool. I mean, I think it's a great overview, you know, if people are interested, they can, they can obviously look you up and uh chat more. Uh you guys, you guys have been building for a good while in, in this space and like, it's really cool to see that you're sort of honing in on uh this coding use case. I, I have also obviously become more interested in coding uh A I agents for my, for myself as well. Any final words, any other things that people can look forward to?
I can probably share just a little bit about what uh our roadmap looks like, where we are headed next, right? So we are going to open source command code very, very soon. I'm hoping we can announce that on the AI engineering conference, N S F. Uh if I can work out the quirks of six or six years old, you know, repository. The idea behind that is
Six years old? Wait, hang on, six years old? Oh, because you're, it was a C L I before A I and stuff, yeah.
Yeah, like I've been building it for like since twenty twenty, right? Uh it became a product last year. And I've told everybody in there, you know, mothers and brothers and that this will never be my product, right? So funny how uh you know, things work out. Uh anyway, so we are open sourcing it. The idea behind that is I want to make command code completely hackable. Uh I've been working very, very hard on that. My background is also in this thing, like I've spent like thirteen years in WordPress core. Uh Matt Mulnwick is actually one of our angels now. When he heard that we are open sourcing command code, he reached out. So. It's it's the idea is you should be able to modify any part of command code.
Irrespective of where our business model is headed, right? And the other idea that I'm super subscribed to right now is that we will not turn this into a soup of, you know, fifteen hundred, you know, models that you decide what you do with them, right? I think of it like, you know, so there are there are like three different philosophies, right? One is it's like Windows and every game works with it. You know, I think open code is like that. Every model works with open code. One is like Linux where you build your own drivers like Py, you can build anything with Py, right? Uh, with command code, what I'm thinking and what the team kind of is going forward towards is,
we're gonna build it like Apple. It will have the best of the best models, both open and closed. It will not have every model, but it will be hackable in any way. So you would be able to put in your local model if you wanted to or whatever, right? So that's where we are headed, like we're about to open source it very, very quickly, uh, very, very soon. So yeah, pretty excited about that.
Okay, excellent. Looking forward to that. Open source is always good. It's all a competition, I would say. Uh, you know, I I I was reminded that actually DeepSeek announced that they're gonna do DeepSeek code at some point. So, uh, but you know, obviously they'll only do DeepSeek stuff.
So many of the comments on this particular thing were about like, you know, uh, they were tagging DeepSeek people, why are you not building your own coding?
They are, they are, yeah, they announced.
And then they, yeah, they did after that, like, you know, maybe a week later or something, they announced that we are hiring for that or whatnot. So it makes sense, right? But they will only be for DeepSeek. When to, uh, three point seven max is the second most used model on command code right now. It's just two or three days old, right? So, yeah.
Cool.
We'll see how it goes.
We'll see how it goes. Okay. Well, all the best. It's great to catch up and get a sense of like, you know, everything going on. I definitely think there's some like good ideas here that people can take regardless of what project they work on. So I think that's why we do the podcasts is learning from everyone and try to have a common, I guess, shared knowledge of all these things.
Most definitely. Yeah, I'll continue sharing more and more. I think everybody can learn. It's not just, you know, you don't have to use this ID and command code. You can just use it in any encoding harness that you want, right? As far as we are all improving, right?
Okay, that's it.
Awesome.
Page updated October 24, 2026 · transcript and summary are machine-generated
