Latent Space: The AI Engineer Podcast
Watch the full show on YouTube: https://www.youtube.com/watch?v=hLUGXO5DSpo
4,660 words · 2 speakers
You're here at EAI Engineer with Gavir Boy, founder of NanoClaw. How's it been? It's been amazing. This has been such a great event. Yeah. Congrats for putting it on. Yeah, it's great to see you in person. I can see you online as well. And it sounds like you had a very eventful trip to Singapore. You met the Minister of Foreign Affairs. Yeah, it's just like, what did you do? What happened? They tell the story of the Singapore trip. Yeah, so this really goes back maybe about a month now. I was actually on vacation, like taking my first weekend off since putting NanoClaw out there. And finally took two days off at a nice hotel with my wife.
And then, I think it was Friday night, I was just scrolling on X and I see this. Somebody reposting a Facebook post from Minister of Foreign Affairs of Singapore. Where he did this whole write-up of how he's using NanoClaw, his whole setup. Memory system, indexing, deployment, running it on Raspberry Pi. And he called me out by name, which is like, I was like, okay, that's... My name isn't even on the repost. He's clearly nerding out. And he went deep.
And then I went and I found this post and he had done a Facebook post. And he posted this whole GitHub gist of his whole setup. Or whole write-up of his whole NanoClaw setup. And he's got this kind of second brain type of setup where he's got a Karpatti LLM Wiki. With embeddings and some memory system. Which, did you have that in your original design though, right? No, no, that's all stuff that he's brought in. The memory system that he's used, NiMo, which has kind of gotten a little... Started to get a little bit popular now that he's posted about it. Find that. So it's like, there's 141 GitHub stars. I checked it. I was like, where is this from? When I first checked it, I think it had like forty-three GitHub stars.
So, and that was after he started using it. So he found this system. He liked it. He found NanoClaw. And he put it all together himself. He, he codes, um, and he, he created this setup. He did a whole write-up. So apparently he was migrating from the first version of NanoClaw to the second version, and he created this whole uh, write-up of his setup as part of the migration, and then he just decided to post it. Yeah. And it started to go a bit viral on X, and I did a retweet of it. And then he retweeted my retweet, and we started to talk. My your best bros. Best bros. And, and he said, you know, when, if you're ever in Singapore, we should, uh, you know, I'd love to kind of have you over and, and host you. And then I think it was two, three days later, you reached out.
And you said, we're doing this event. And I'm like, it just, it's, it's all perfect. I checked that he'd be around and, and he said yes. And then you managed to get him to come down and, and talk here, which is wild. See, uh, I think it, well, it did help that you were here. So you wanted to be. And I think also, no, my intentions are part pure in the sense that we just want to promote the, the industry in Singapore. I'm Singaporean. Uh, I live in, I live in the US, but like, I think the government wants to support. We then, it's on the, we basically, basically, basically this. Um, I'm curious, we guys talked for two hours. Anything you can share about like surprising, uh, use cases or whatever. It's like, you, you talk about the fix that. But just like, how does he think about it?
And maybe open your mind is sort of, you know, you should, can you let about? I think for me, his use case really helped for me to kind of crystallize what direction we want to build in. Going back, I would say a month ago, there were these two overlapping, but kind of distinct directions. When we're thinking about adoption of Claw type of agents, autonomous agents in a business setting in a company. Uh, so there's the one where it's, the team manages agents. So it's agents that you build. An agent factory or, or agents that automate workflows. I gave a talk about that, the agent factory that we built, and we had started to build that already for ourselves. And we've been building it for a while. It's still kind of under construction.
So that's one distinct factory. It's the team as a team working together to build the agents and managing them as a team. And then you've got the other side, which is personal agents in, in a work setting, in a business setting, but individual people who have their agent, their assistant that's helping them do their job, and it's more one-to-one. And we were going, working on both of those use cases, both internally, we were using agents in both ways within our team. Uh, and started to work with design partners that were interested in both use cases. So we were working with a team where they wanted to give each person in their legal team, their own personal assistant. And then we're working with another, uh, design partner that wanted to give, build agents that automate a workflow for their legal team. So automate drafting first contracts.
With the Foreign Minister, Ministry of Foreign Affairs, Minister of Foreign Affairs, a use case, it really helped to crystallize that. From my perspective, the way to start in a business, introducing agents to a company is to give each person their own agent. Uh, because there's a certain knowledge and expertise and learning curve on how do you work with agents? What are they good at? What are they not good at? How do you prompt them? How do you, uh, build the instructions, skills? Adjust those over time, manage the, uh, context window.
And it takes some time to learn that. And initially, one of the biggest mistakes that people make is that they just want to throw something in an agent. And then walk away and expect to get a finished result at the end. You gotta, you gotta put in the effort, right? It's the dream that we all have and, and that's a natural instinct, right? It's good to be, to kind of be ambitious, but you have to work on it. You have to fine tune, not fine tune a model, but, but fine tune the output by adjusting instructions, adjusting skills, adjusting what you put in. And, and also iterating with the agent. From my perspective now, especially seeing his use case, the killer use case for claw type of agents, autonomous agents today. Is a second brain use case where you're just kind of dumping in information.
And you're not expecting it to give you ready-made output, but it's just collecting that information, building up its internal. Memory or knowledge graph or Wiki, L L M Wiki, Wikipedia view. And then able to give you useful outputs at the end.
Yeah, I've been using Town as a stand for that. My last in A I Europe, I talked about how I'm using Devon from L H R. A Town assistance, even a little bit of, uh, Victor as well. These are all like some of the, the alternative personal assistants. I use Open Call personally, but, uh, I, I think like, there's this general field of like knowledge management that needs to be capitals. That I think you guys have to do very well. And like, I think you have the. Simplicity and a privacy policy that that drew him, right? Like, I think that was, he mentioned that he looked at over God first. Uh, did he talk about the the security side or like, well, you know, the something Falcon cock a person?
Yeah, the security side was the key thing that that that brought him in. There are a lot of people like him. I actually started to use Open Claw and we were using it for our business at the time. I was building an AI native marketing agency. We had some customers, we were ramping up. And then we set up Open Claw and it started to manage our sales process. And very quickly, within like two days, it was doing the work of an employee, of a sales manager, just managing our whole pipeline. But then I started to dig in and I started to see the size of the code base and the number of dependencies. And some other things like logging all messages in plain text that just made me a bit apprehensive to use it for like production. Use cases to build a business on it.
So I felt fine using it personally, but when I was saying I'm going to connect it to my customers' data and I'm going to start to build all these workflows that are running and I'm trying to build my whole company on top of it. I didn't feel like it was stable enough for me. So the key changes, first of all, made a really minimal code base with Nano Claw. So I didn't clone Open Claw and change it, just started from the ground up from scratch. Use a lot more components out of the box. So use Agent SDK instead of instead of building our own agent, uh, like. Open Claw uses Pi, so then you've got to build a lot of session management and compaction and a lot of other things because Pi is really minimal. Instead, we use Agent SDK, which comes with a lot of that stuff out of the box. Um, only supported one model initially, one agent initially, so saves a lot on that.
Uh, used, integrated with things for connections to messaging channels, like we integrated with, uh, Versale's chat SDK library that just gives you all of the messaging. Different messaging outside of the box. So really minimal, uh, fewer dependencies. Uh, but then the biggest thing is the isolation model. So first of all. You run your whole Nano Claw in a VM or on its own Mac Mini and that's great, but you still need to isolate the agent. In its own agent runtime and separate that from the messaging bridge that's connecting to Slack or Discord or whatever. And the router and the other pieces that decide what gets pushed into the agent and what gets done with the agent's output. So you need to have isolation of that. So we run each agent in its own container. And then.
Make sure that the agent doesn't have any credentials in its environment. And that's really big. If it's got credentials and it's touching unsanitized input, which for most useful use cases is gonna be, it's gonna be, you know, reviewing a PR. It's it's got the whole PR input. Anybody could open a pull request to an open source repo. So making sure that there are no credentials in the agent's environment. So even if it gets prompt injected, it can't leak credentials, it can't leak API keys. And then the third key thing is separating. So the agent can use credentials. All requests coming out of the agent's environment get proxied through a vault. And the vault adds credentials if the agent is supposed to have access. And then the third thing is having access policies and access control, including human in the loop approval.
So you can give your agent access to emails, maybe set a policy that it can read emails without any approval process. But if it wants to send an email, you get a message in whatever channel you've connected. So in Slack and you've got buttons, approve, reject. You can see what it's trying to send and you decide if you should let it through or not.
Yeah, I feel like he may not have like the fine brains understanding that you have, but like their messaging does come across with your DOS. That is what is. Yeah, I mean, like, what else have you found in this trip to Singapore? Like you met other users, like if you met other builders that spread interesting conversations?
I've met a lot of really interesting people out here. You know, I in my talk, I put my agent out there to book, uh, book coffee chats.
I think there's some kind of VPN proxy thing. I didn't think I was in Singapore because I have my US home away. So I was doing something weird where it didn't let me.
The talks are streamed live on YouTube, right? So yeah, I just wanted to make sure that people who are here will be able to access and it didn't get taken down by people. You know, massive people connecting around. So I did set it that it's geo-blocked to Singapore. Sorry about that. I'll open it up for you later. So yeah, so I I put my kind of agent out there and people chatted with it and then booked coffee chats and chat with really interesting people building in especially memory. Students out here who are building really interesting things.
Actually, I did want to ask, do you have your own memory solution that you use?
Personally, my. What I'm using personally is is a kind of LLM Wiki type of solution. I just point my agents at Carpathia's posts and I say, read this, look at this. Let's chat about it for a minute and then uh. But NanoCo has built in a kind of simplified version of that where it just has some basic instructions to your agent saying, create markdown files, anything, anything substantive that the user shares with you. Make sure to save it somewhere. Either it goes in your cloud DND or it goes in a markdown file or it goes in some other file, but it's gotta be saved somewhere.
Yeah, I think my problem with those things is just that even using Olbest, it will create a lot of duplicate files, which is like not know that it already has a similar thing and then you just create another duplicate set of files and now I have like two sorts of true, right?
Yeah, you gotta have other instructions on top of that saying, create an index of all your files and save that index in your in your instructions, see what files you have, then go and save it. And then you do need to have like a background process that runs daily or every few days, looking over your your memory files and finding the duplicates and flagging them.
Yeah, there's definitely a lot more to do there that we're not doing yet. I know some of the other kind of claws.
Leaving off the show, so let's say, it sounds like you just wrote your.
I'm not sure that like retrieval based solutions are are ideal for like personal assistant. Because it's very specific and it isn't necessarily something that retrieval is really good at. Like if you ask your assistant, what are the most important things I should be focusing on this week? There's no retrieval based search, semantic search, keyword matching search that's going to be able to give your agent that information. But if you have a good LLM Wiki that has like, here are the projects you're working on, here's the timeline, here are the, you know, here's a log of the different calls you've had this week. It can go through a few different files and collect the things and give you that list.
Yeah, amazing. Um, yeah, it's really cool. I think last question is about your company, right? When I first messaged you, you were still, we still had a the marketing stuff. But now you've started NanoCo. What is NanoCo OS division? Is it gonna be a VC style startup or?
NanoCo is.
We shifted gears. We were, when I first spoke to you, when I first created OpenClaw, we were building an AI-native marketing agency. We had some customers, we were ramping up, it was going well. I built NanoClaw just for my own use as a side project on a weekend, launched it on Hacker News MIT license.
That's why I first found it, I tweeted about it.
Yeah, yeah, immediately, within like hours, I think, after it came out.
Okay, like I can read this codebase, I cannot read OpenClaw.
That was the idea behind it, because I said I'm gonna build something that I have an idea of the right way to structure it, to make it kind of sane and secure. But I don't quite trust myself with this, so I want to make it readable and let other people look at it and validate it. And since then, I've had a lot of security experts review it and critique it and give feedback and point out small things. But what's come out of all of that is that the core approach seems to be sound, because nobody's pointed out an issue with a general approach. So we were building that agency, I built NanoClaw for myself. It started to get a ton of traction and attention. And then Karpati tweeted about it and it went to just a whole nother level.
And for a little while, we were debating if we should build an agency or build NanoClaw. And then after Karpati, it just became clear, like, there's such a big community, so much energy behind this. So we went all in on it. We have a company now, we've got 10 people on the team. And what happened was the early adopters, I think not just of NanoClaw, but OpenClaw, Hermes, all the Claws, have been a lot of VCs, CEOs, executives who are using it themselves and get really excited about it. And we've had tons of CEOs and other executives approaching us and saying, "Hey, I built out this great setup for myself, I've got these three different agents, this memory system", just like the Minister of Foreign Affairs here. Uh, and they say, I want to roll this out to everybody in my team.
But I don't want to become the IT guy who's now like fixing their agents and debugging the memory issues. Can you guys come and help me set this up for my team?
Yeah, so you're not, you're not in your lab, like you're rolling it out, it's hosted and all.
Yeah, so we're gonna do deployments, uh, and for a lot of companies.
Seeing as they might talk, it's gonna be very relevant beer.
I'm gonna go back and catch up on some of the talks.
It's good that there's dev journey, but like more focused on knowledge work rather than co.
Yeah, I'm gonna get, catch up on, uh, and your talk and a few others. So, yeah, the, uh, gonna do the deployments, uh, for a lot of companies, it's gonna mean deploying on their infrastructure to their cloud.
Take, take notes from my thing. It is exactly those, like, people are gonna tell you about, like, SSO and, like, uh, uh, BBC peering and like on-prem deployments, all that stuff.
Yeah, uh, we're gonna have to hook up to their credential management system, like,
getting their credentials from their vault.
Yeah, it's actually a lot of it is not yet.
It's not, but you have to understand the AI. I think that's what's missing for a lot of these companies. They have good engineers, they've got good DevOps. I spoke to a lot of these companies, um, they've got good security teams. But if you don't have the piece of the AI engineering, it's hard to put together the different pieces and to have confidence that what you built is secure, it works and, and it's, uh, it's the right setup. So, having, I, I, I think of it more as a partnership between AI engineers coming in and working with the, their DevOps, their security team, their IT department, and helping them get set up, the initial setup, and then afterwards helping them manage it over time. Uh, initially, you gotta connect it to their credential management, observability, their other security systems.
Um, and then help them get set up with their initial integration with their internal data sources. Um, but then, over time, their IT team, their security team can pick up those pieces and when there's a new request for integrating with a new data source, they can take that on their own. But there's still management to do in terms of upgrading and maintaining it. Agents are different than normal software in that normal enterprise software, you can deploy it, put it on some server and let it run for like five years. And as long as you never touch it, it just works. Agents don't really work that way. The core thing that you're building on is constantly changing over time. You can't just run four, six for the next three years and just leave it. You gotta upgrade to four, eight, four, nine, five. And every one of those upgrades changes things.
And, and the labs are putting out new features, new capabilities. Memory is getting baked into the L L M, getting baked into the agents. And you have to be constantly updating in order to just stay kind of at the, at the front. So that's what we're going to be doing. We started to do our first deployments. Uh, we've got over a hundred companies that have approached us, interested in rolling out agents. Um, and yeah, big announcements coming up.
Two for closing questions. One, uh, for your agent factory, do you think that Git and GitHub will last?
I think it's gonna have to last for us because we're an open source project. And you've got the culture and the community.
I'm trying to see what comes after Gitmo.
Yeah.
I don't have it yet.
There, there is a weirdness there, right? Because the whole agent factory should be on GitHub and not happening in Slack. But because it's an open source project, like if it was closed source, we would just have the agents leaving responses, doing the reviews, test results. It would just all be in the, in the pull requests on GitHub. Because it's open source, I don't want to have an internal discussion back and forth, leaving feedback from my agent meta, you know, feedback on the factory itself, happening in a public GitHub thread. It could be interesting, but it seems a bit off. Uh, but if this was internal, I would want it to be on GitHub. And then I think we'd be running into the issues where GitHub probably still isn't quite the right format of that.
My, my people do all their stuff in Slack and then they pay me to stamp the, the P R A in GitHub. But like, why do I need to, why do I need GitHub at all? You know, like, I should just be the Slack.
Okay, so maybe our approach is the right way. So we have the whole thing in Slack and then.
That's not running I E.
And all the way till, you know, merge and then you hit approve and it gets merged. And I think there's a lot to do. Slack has good, pretty good UI. You can, you can build out some good user experiences just in Slack.
So Slack over WhatsApp, he uses WhatsApp, you use Slack. There's like a lot of Telegram. I like, I like Telegrams. Like there's so much easier to unbox it, I guess. But you can't do, you know.
I can do my, uh, my tiers of messaging apps. So I think S tier would be Slack. Uh, and then Discord, Teams. And then goes like Telegram and then WhatsApp. They're very, very limited in what, what you can do. And it's hard, like you have to have your own phone number if you want like a decent experience. Uh, chatting, putting the bot on your own number. It makes sense if you wanted to just see your messages, read only. But to chat with the bot on your own number, it's. I mess, yeah, I haven't experimented with it.
Oh my God, it's terrible. Uh, you can't, I can't message for itself because I needed some number to, to receive it. So I just give up.
Okay, so that's below WhatsApp on the tier list. We filled out our whole tier list.
See, but what you know, basically you have to buy, you have to buy on over. But I, I think there's some sort of, I do that. Anyway, uh, last, last question. What can people help you on? What are you looking for? Al, what do you look like? What is the question that you want answered?
Managing open source projects today is, is a huge challenge. So like, I, I would say, I, I won't even say for, for, for AI engineers, I won't even say, uh, contribute a pull request. It's like, it's part of the problem. Yeah, it's like triage, help us figure out how to manage a, a open source project in a better way. Um, it's a big open challenge. It's like this, uh, it's an arms race where coding agents have just made it exponentially easier for people to open pull requests. And then it's so hard to, to triage, to review, to understand if it's aligned with.
The same as, uh, what Pete Sandberg is saying, right? No more pull requests, only prompt requests. Like, don't give me your code, give me your, you're like, what, what your use case is. And then, uh, something that comes to my mind after having this discussion is you have all these like, oh, you know, my bug here, my feature request here, whatever. You should just copy all going to a Wiki of future developments. And then you pull from the Wiki. So like the Wiki is kind of a buffer. All the changes come in. As a continuous stream of stuff, but it becomes a wiki and then you pull from the wiki as you as you leave.
Yeah, we have started to build a wiki for our agent factory. So as we're developing it, we're adding to the wiki, kind of on each merge, on each commit. And then I think that is going to become a standard part of building the open source project. So what you guys are doing with, is it DeepWiki? That's awesome. I think though that it's going to become a standard for teams to build out the wiki as they're doing the development in real time. And then, yeah, the the the future development is in the wiki linked to the different parts of the project that it's related to. And the bugs are part of that wiki. And then as you're developing it, each time you start working, you pull information from the wiki to get context.
And then you finish the PR, finish the commit, and you push back to the wiki some more context.
Yeah, great, great. Well, thanks for coming. I'm so glad to meet you. I'm sure this is not the last time we will see you, but definitely it's been a nice trip, support.
It's been awesome, yeah.
Great.
You too. Thanks.
Page updated October 24, 2026 · transcript and summary are machine-generated
