AURA and Open-Source Agents for Production Operations

AI agents have transformed how software gets written, but the operational side of running software in production has not yet experienced a similar revolution. The same teams responsible for keeping systems healthy, investigating incidents, and managing reliability are still doing much of that work manually. Mezmo https://www.mezmo.com/ is a Production AI company that makes autonomous operations fast, efficient, and safe. Their open source project, AURA, is a declarative agent framework specifically designed for SRE and platform engineering workflows. It takes a Kubernetes-inspired approach where teams define what they want agents to do rather than scripting every step of how to do it. Andre Elizondo https://x.com/consumptive_ai is the head of product at Mezmo, and he has a background in systems engineering, SRE, and observability. In this episode, Andre joins Kevin Ball to discuss what makes SRE agent workflows fundamentally different from coding agents, how AURA handles context engineering, AURA’s declarative configuration model, the spectrum of agent autonomy, and where the role of the SRE is headed as agents take on more of the operational work. Full Disclosure: This episode is sponsored by Mezmo. Kevin Ball https://www.kball.llc/ or KBall, is the vice president of engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript meetup, and organizes the AI inaction discussion group through Latent Space. Please click here to see the transcript of this episode. https://softwareengineeringdaily.com/wp-content/uploads/2026/07/SED1945-Andre-Elizondo.txt Sponsorship inquiries: [email protected] mailto:[email protected] The post AURA and Open-Source Agents for Production Operations https://softwareengineeringdaily.com/podcasts/aura-and-open-source-agents-for-production-operations/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.

• 0:00Speaker 00

  AI agents have transformed how software gets written, but the operational side of running software in production has not yet experienced a similar revolution. The same teams responsible for keeping systems healthy, investigating incidents, and managing reliability are still doing much of that work manually. Mesmo is a production AI company that makes autonomous operations fast, efficient, and safe. Their open source project, Aura, is a declarative agent framework specifically designed for SRE and platform engineering workflows. It takes a Kubernetes-inspired approach, where teams define what they want agents to do rather than scripting every step of how to do it.

• 0:42Speaker 00

  Andre Elizondo is the head of product at Mesmo, and he has a background in systems engineering, SRE, and observability. In this episode, Andre joins Kevin Ball to discuss what makes SRE agent workflows fundamentally different from coding agents, how Aura handles context engineering, Aura's declarative configuration model, the spectrum of agent autonomy, and where the role of the SRE is headed as agents take on more of the operational work. Kevin Ball, or K-Ball, is the vice president of engineering at Mentos and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action Discussion Group

• 1:27Speaker 00

  through Leighton Space. Check out the show notes to follow K-Ball on Twitter or LinkedIn, or visit his website, kball.llc.

• 1:46Speaker 01

  Andre, welcome to the show.

• 1:49Speaker 02

  Thanks for having me.

• 1:50Speaker 01

  Yeah, I'm excited to dig in. So let's start with you. Can you give us a little bit of your background and how you came to be at Mesmo?

• 1:57Speaker 02

  Yeah, absolutely. So my background at my core is, you know, I started my career running systems in a lot of different forms, both at enterprises, small companies, both as a practitioner as well as in the ecosystem. At various different companies across cloud, infrastructure, observability, security. And so my, my view on the space is primarily from an operator or an SRE. Back in the day, we called them sysadmins, right? And over the last few years, have been heavily involved in the AI ecosystem, especially around agents. And so, you know, here at Mesmo, what I'm focused on, I head up our product team and we're a small but mighty team. That's very focused on building, you know, really the, the ecosystem around how we enable SREs,

• 2:41Speaker 02

  platform engineers, really everybody in, in operations to gain the same benefits that we've seen with coding agents over the past few years, rather, for how we better use them to reinvent how we build, maintain, troubleshoot, everything around production.

• 2:57Speaker 01

  Nice. So can you give us actually just that like quick, high level, what is Mesmo? And we're talking a lot about agents and we'll talk a lot about agents, but like Mesmo as a company, what do you do?

• 3:07Speaker 02

  Yeah. So when you think about what the software factory is that a lot of folks are talking about right now, usually that's around, hey, how do I use something like a coding agent to build software faster, test it faster, and get it to production faster? What Mesmo is focused on as a company, and we have a, you know, a, a long past around observability, making observability really useful, really kind of like toned in on making it kind of like a next iteration of how we use observability, where we've really focused our time within the last six to eight months or so, is really around how we take it a next level further, right? So when we think about how things like observability as a, as a good anchoring point,

• 3:47Speaker 02

  change in the land of agents, well, not only do you need agents that know how to handle that observability, but the observability systems look a lot different, right? Instead of an agent looking at a dashboard, it's going to prefer the raw data, it's going to prefer insights, it's going to prefer, you know, like specific tools around how it uses that, that data. And so, you know, when you think about the mission of Mesmo, we're really looking at the latter half of the software factory, right? Like creating that software factory and ultimately the dark factory for operations, not just how we get software into production, but how do we really? Really change the entire lifecycle around what does it mean as a, you know, as an operator to make sure that that software is healthy, to make sure that that software, I can optimize performance, I can optimize cost.

• 4:32Speaker 02

  It's a big topic that I bring up a lot, which is like, you know, what if you had an SRE with an empty Jira queue? What would they do? And there's a lot of things that fill that, that particular topic. And so, you know, at Masmo, we're really focused at, on not only building the agents, but building the operating environment, building the ecosystem for how we enable teams to do that today.

• 4:51Speaker 01

  Totally. I feel like, yeah, as we automate more and more of software delivery, that makes things like observability that much more important because you need to be able to understand, hey, my agent said this was fine, but is it actually behaving? How is it going?

• 5:03Speaker 02

  Totally. Yeah.

• 5:04Speaker 01

  So I could see a whole company focused around agent legible observability. You went more than that. So how did Aura come into, and maybe just a very high level? What is Aura and how did it come out of what you're working on?

• 5:17Speaker 02

  Yeah, great question. You know, when we first started to go down this path of the next iteration of observability, right? When you think about like, how do you distill patterns and logs and traces and metrics? We focused on initially, like most folks, like exposing our MCP server in a way that people could consume it through agents. And, you know, that meant that instead of your agent consuming an entire, say, like hour of traces or week of traces, which would immediately blow the context window. You know, we wanted to make sure that that was tailor made for agents being the primary consumers more than humans in the long term. Now, where we saw some problems internally, as well as how folks were consuming that MCP server, is that, you know, SRE workflows look a little bit different, right?

• 5:59Speaker 02

  So when you think about like the types of workflows, the types of context engineering, the types of human in the loop, and like what that looks like for things like coding agents and things that are, you know, just building and testing software locally or in a controlled environment. That looks a little bit different than some of those problems. Like I mentioned that, you know, you have an SRE handling a large swath of data in front of them or her to, to figure out, hey, what's going on? And there's a lot of, you know, kind of like looking at graphs, looking at dashboards, looking at all these different pieces of information. And what we saw. Again, from our own experience as well as in various different partners in the space is that, you know, what folks typically were doing is they were taking a very general purpose, very unopinionated agent framework, right?

• 6:42Speaker 02

  And like trying to put together that framework in a way that, you know, hey, I can do some pattern recognition on logs. I could do some, you know, pattern recognition or RCA on traces or metrics or something like that. And really what's interesting about why we went down that path initially is, you know, we really just wanted the outcome of getting better RCAs, faster RCAs, be able to troubleshoot our own systems better, use that internally better. And what we found was like our experience of going through that journey and some of the folks that we initially partnered with, were very similar to a lot of the things that we've heard from various different SRE teams and platform teams in the space is like, you know, it's like the building of agents, especially for managing production is very much like the iceberg analogy

• 7:25Speaker 02

  that we've seen for a lot of different things like Kubernetes, like cloud, et cetera, where, you know, everybody's thinking about the tip of the iceberg and like, you know, hey, how do I just take a model, wrap it in a simple agent framework and give it a prompt? And in reality, there's, you know, a lot of things that are needed to actually make that reliable, right? Like, how do you make sure that the model can consistently call tools and that those tools don't just like randomly fail? Or how do you orchestrate something, you know, fairly complex, like a multi-agent workflow without making somebody kind of get dragged through glass to get to that point where they're able to actually realize that outcome of solving that use case? And again, when we saw that with operations workflows specifically, we just saw more and more of that kind of like upfront effort

• 8:08Speaker 02

  in order to make that reliable workflow and is really like hampering people's ability to be successful with those frameworks. And so, you know, we really just wanted to kind of like shorten the circuit, right? And so that's really, you know, going into where Aura came into play is like, it was our own learnings. We looked at, you know, a lot of different frameworks and harnesses at the time in the space. And, you know, a lot of them were focused on either being insanely, you know, general purpose, kind of like one harness for everything, one framework for everything. And they typically didn't make those hard things, like I just mentioned, very Easy. There were like a lot of Lego blocks that you had to figure out what the Death Star looked like at the end of it. And what we wanted to do with Aura was, you know, kind of like what, what the experience that a lot of, you know, SRE teams or platform teams are familiar with, which is like,

• 8:52Speaker 02

  hey, if I'm deploying something in Kubernetes and I do a Kubernetes deployment, well, you know, I tell Kubernetes like what the outcome should look like. I don't necessarily need to tell it imperatively every single step that it needs to do to manage those workflows, scale out those workflows, ensure that it's reliable at the end of it. And we wanted to create that same type of experience for how ops teams built and manage and maintains agents. So again, like shortening that process to getting to the outcome and ultimately making it so that, you know, the tools that we expose, like our own MCP server and others, just become much more valuable because you can get, you know, you can get your outcome closer to day one than if you were to kind of go through that process of building all the individual pieces that are frail and kind of like have the tendency to fail on their own.

• 9:37Speaker 02

  You know, we wanted to make sure that that was a, not only a shortened workflow, but a very reliable workflow to get to that outcome.

• 9:43Speaker 01

  Yeah. So that's, there's a bunch in there and I'd love to dig in a little bit, but let me, let me pinpoint one of those things because you mentioned, you know, SREs are used to working in a, something like Kubernetes, which is essentially like a declarative environment, right? I declare my relation, my, my objects, I declare their relationships and the system is responsible for bringing them up, making sure that they get into that state and kind of getting there. And talking about that with regards to agent definition is fascinating to me because like, I think most of us are familiar with like coding agents of different sorts. It's probably the most widely spread agent harnesses out there are things like Claude Code and that sort of thing. And there, I think a lot of us are still being pretty declarative or imperative. We say, I want this, I want that, you know? So what does it look like to have a declarative agent definition?

• 10:26Speaker 02

  Yeah, I think the proof is really, you know, the, the benefit of us open sourcing Aura is that you can look at the repo, you can look at exactly what the examples are and like, we operated very much on like a lot of the, the Linux fundamentals or philosophies of like having things like same defaults, you know, where hey, if I don't need to tune every single part of the process, you know, I can operate on the defaults built into the system. But at the same time, I have the ability to tune the reasoning budget, tune the quality score, whereas the agent is maybe executing its tools or going to an end state and trying to solve a problem like, how does it self-evaluate itself and self-correct itself as part of that loop? Also enforcing the loop itself, right? Like ensuring that we were

• 11:08Speaker 02

  kind of opinionated on those things underneath, but also gave the folks that were using Aura the ability to customize them if they saw fit. Again, it's like the same thing with Kubernetes manifest, right? Like, I may just want to deploy Nginx or Redis, and that may just be all I need to do. Or I may need to scale that out. I mean, to do like a bunch of different pieces along the way in order for that to be very tailored to what I'm working on internally, to my workflow. And so we wanted to do the same thing for agents. And so again, when we looked at the ecosystem, you know, there were different ways of like defining the agent in a framework. There were different runtimes. There were different you know, kind of pieces along the way that were like very unopinionated and very much like, hey, you can do this, you know, you do a little bit of anything.

• 11:53Speaker 02

  And again, with with the types of problems that we were seeing with the industry, we wanted to really just operate on the path of like, hey, you can you can start using Aura, you can pick up our container tomorrow, you can run it in your cluster or even on your laptop and give it a simple config file. That's also the beauty of Aura is when you look at, you know, how you build these agents declaratively, again, going on the on the manifest ideology here. We use a simple TOML file, and in that TOML file, right, I define the agents, whether I'm saying, hey, here's a a simple agent, maybe with one tool, and maybe that tool is my observability tool, maybe that tool is my cloud provider. And a simple system prompt that says, hey, how should, you know, what should you do? Or what should you focus on?

• 12:33Speaker 02

  But I don't need to define, you know, again, the reasoning loop. I don't need to define the self-correction loop. I don't need to define you know, anything along the way. But as I'm starting to get into some of these more kind of complex things, right? Where, hey, maybe I'm dealing with a a tool behind that agent that I know is going to return a lot of data, right? Like, maybe there's like different MCP servers, like, for example, the Prometheus MCP server. I'm sure a lot of people have tried to use that MCP server. And you just end up getting this context. Load of like, you know, every possible thing that's returned by that MCP server. And so, you know, again, where we wanted to make things easy was we introduced different concepts along the way. Like, for example, Scratchpad. Scratchpad is something that's built into Aura natively.

• 13:14Speaker 02

  And it's something that you could very easily turn on with a single, you know, a single line in that config file and say, "Hey, start using Scratchpad." And basically what it does is it acts as kind of a layer in between the tool and the model to give the model the ability to interact with the outputs of that tool via like a simple file on disk. And then ensures that, you know, it'll kind of like look at whether or not that tool needs to be recalled, whether or not you can kind of respect the previous tool call, like, you know, how you can kind of share that tool call across different agents. And so, again, it was really operated on that idea of sane defaults on day one. Make it really, really easy to get it started, but also get it running and ultimately achieve the outcome. But also not alienate folks that really, you know, cared about tuning every part of the process along the way.

• 13:59Speaker 02

  That's actually something probably worth calling out is like, you know, when we also looked at folks in the space, especially a lot of like commercial solutions or things that are like built into, you know, the various different ops tools that people are using today. Like there's an, you know, an embedded agent inside of anything. A lot of those were black boxes, right? And so, one of the really important things for us in like our day one, you know, opinions on how we've built and released and continue to develop on Aura is really that we wanted to make sure that all of those things that we're doing are fully transparent to the user. You can not only see what they're doing, you can, you know, interact with it, you can, you can tune it. And so that's, you know, transparency on day one was also an incredibly important thing for us along the way.

• 14:40Speaker 01

  Yeah, that's super cool. And what, I mean, even just the tiny example of Scratchpad is really nice because I feel like I operate a lot in coding agent land. And it's like, you learn these patterns of, oh, let's dump this to a file and let's interact with it. Or, oh, let's dump this into a SQLite database and then query it rather than just raw putting everything in context. And when you find those patterns, if you make it easy declaratively, just say, "Hey, agent, here you go. You know how to use it? Run with it. It's, it's really powerful." Target with Aura is this is an agent framework for SREs for you know, kind of production use, which is once again like different than most of us first encounter agents nowadays and coding agents or things like that. And maybe we're building our own somewhere.

• 15:24Speaker 01

  But let's like flesh out a little bit more of those differences. So we talked some about the declarative versus imperative. What are the other big differences in building an agent framework for a production environment for SRE? As compared to I'm running this as a coding agent on my laptop?

• 15:39Speaker 02

  Yeah, there's, there's really like three big things that I look at across the, across the landscape of like, what does it take to have a really successful SRE agent for any team? You know, a big pillar across this is, is how we, how we handle context engineering. And so again, like when, when teams are just kind of picking up a framework and just putting a model and putting a simple prompt like oftentimes they're not thinking about context engineering on day one. And you know, for folks that maybe aren't super familiar with context engineering. Right? We always have the limitation of any model, you know, big or small is, is the context window, right? And so we're always operating within that context. And I'm sure many of your listeners know about this, but just to be verbose on that topic.

• 16:21Speaker 02

  And you know, in reality, operating within the context window, it can be very simple for very simple use cases, right? And it looks very different for things like coding agents where, hey, the context window is just maybe the code or the understanding of the code base. It's like right in front of me. But for SREs, right, that, that context engineering looks very different, right? Like, you know, when you think about the amount of runtime data, everything, you know, understanding of the code, understanding of the infrastructure, understanding of a lot of the observability data, right? So like traces, metrics, logs, et cetera. You know, you can't just say, hey, go give me like the last week of traces and logs and metrics, and I'm just going to shove it into the context window and hope that the model not only has the ability to have all that within its context window, but also doesn't like hallucinate along the way.

• 17:06Speaker 02

  Right? And so the thing about context engineering for these types of workflows is they automatically become like at scale. Right? Like typically because of the, the nature of, you know, us operating this in production, we're dealing With scale and those don't present themselves on day one. And so context engineering is a really big pillar of that. The other piece is really like how we handle things like, you know, multi-agent orchestration. And so when you think about like some of the advanced patterns that are starting to pop up that are becoming, you know, really, you know, useful in this whole agent ecosystem is like, hey, how do I have, you know, as part of any different type of agent, maybe that's like a, an agent that's doing, you know, root cause analysis or whatever. Like I may want different agents focused on different tasks that, you know, can collaborate with each other.

• 17:51Speaker 02

  And that's going to look very different in different types of use cases. Maybe I want a security reviewer, maybe I want somebody who's, you know, just focused on investigating the cluster, just focused on investigating the code base. And that helps me to, you know, do this in parallel. So it's actually a lot faster to do that. But it also makes sure that like each one of those agents, you know, isn't, isn't dealing with again, like a single context window as its limitation across the way, right? And so like orchestrating multi-agent workflows is, you know, typically a non-trivial thing in pretty much any harness or any framework. And so again, that's like usually a day three, if, you know, if anything, maybe a day four type of problem that not everybody's thinking about on day one. And so it's really limiting them the ability to gain those,

• 18:34Speaker 02

  those benefits until they've actually hit the problems with like, you know, what they're doing in the simple agent or maybe the simple agent plus one or whatever. But we just wanted to make that again, like very easy to solve on day one because we believe that the nature of these types of production SRE workflows benefit a lot from this orchestrated multi-agent workflow. And then the last one is, you know, how do we handle things like human in the loop, memory, things that are opinionated to the task? And again, that's where, you know, we think about like what memory looks like in coding agents, what memory looks like for pretty much anything else. And it's very general, it's very like, you can kind of have the agent provide memory.

• 19:14Speaker 02

  And then the memory system is like either just a simple graph or maybe it's just a simple, you know, kind of collection of like summarizations across the context window. Whereas, you know, what you would want in, you know, an incident investigation is really a memory system that's looking at the context. Of, hey, what are the previous incidents that I looked at? Like, what were the previous resolutions to those incidents? What happened along the way, right? And so like, this is also another problem that like is unique to these S3 workflows is really how we handle those longer horizon tasks, those longer horizon things that the agent is dealing with. And then the last piece I'll just kind of highlight as as part of that is things like human in the loop. You know, how we have agents that we can, you know, give a little bit of autonomy or a lot of autonomy to, right?

• 19:58Speaker 02

  Like, we believe that should be not a either or, but really like a graduated process, right? Where I maybe want to start with something that looks more like a copilot and then be able to very easily grant some more autonomy where, you know, it looks a little bit more like an assistant. And then, once I truly trust that assistant and I actually want to, you know, change the way that I'm running production to really be agentic and really be autonomous. Well, then I want to graduate to more of that autonomy that looks like, you know, really that like dark factory when we feel like we're comfortable with turning off the lights. And so we wanted to make sure that Aura kind of took in all those different concepts that I mentioned and just make them kind of like available on day one and something that, you know, we have our opinions baked into the harness.

• 20:40Speaker 02

  But at the end of the day, you know, you can plug that harness into, you know, our tools. You could plug it into your own tools, your own, uh, you know, observability systems, cloud providers, et cetera, behind the scenes. And, you know, really kind of like provide that for the community to build on as a common set of practices that we believe will enable this like industry to move forward.

• 21:01Speaker 01

  You have highlighted like some of the key things. So if it's okay, I'd love to dive another level of detail deeper on probably each of these. The one I want to start with is orchestration. Because as you know, like, I feel like this is a problem everybody's trying to figure out right now in different forms or another. And you have people building their own orchestration frameworks. You have it being built into different things. So first, from a developer experience standpoint, what does orchestration or multi-agent look like in Aura? And then maybe we can peel back the hood a little bit and be like, and just sort of look at what are the primitives? How is it built? How would, if I wanted to customize it, what would I plug into?

• 21:36Speaker 02

  Yeah, so the way that we handle orchestration is meant to be like as simple as possible. Right? So you know, there's a lot of different things that folks can customize. Again, if I just picked up like a general purpose framework like Langchain and said, hey, I want to do, you know, multi-agent orchestration while I'm thinking about the type of reasoning loop. I'm trying to think of like the type of retry as a resiliency across the way. And the way that it looks like with an aura is again, that simple Tomo file that's driving the whole process. I define an orchestrator or a top level agent. That agent is very simple. It's you know, anywhere from like five to 20 lines of like, you know, how detailed I want to be on my system prompt. Basically, that's like the majority of of the context there. And then I basically just kind of like note that as my orchestrator agent. And then what I can do is I can define various different workers.

• 22:21Speaker 02

  And so again, like just another subsection of the Tomo file where I'm saying, hey, this is a worker that is that is named this, and I'm going to give it this description in this preamble. And I can also limit its tools so I can give it access to everything. Or I can say, hey, you know, these are the tools that I just want this worker to have. Thus, again, like limiting some of the potential of of context bloat or, you know, kind of limiting the ability for that agent to be really good at one task. And like, again, it can look basically as simple as that where I define the orchestrator. I define the L L M that I want to use. I define the workers. And you know, everything behind the scenes is actually driven by aura out of the box. Right? So I'm not having to think about how the orchestrator is going to handle its planning loop. Right? So like the first thing that the orchestrator will do is kind of

• 23:05Speaker 02

  create a plan, create a go forward, like what am I going to do? Right? Like, how am I going to fan this out? What workers are available to me? It's going to go ahead and provide that to the workers where it's basically going to go into kind of an execution mode. And then it's going to go ahead and kind of like self evaluate, right? So like as I get the results from those workers, hey, do I have enough of a confidence that will actually sell for itself along the way? And say, hey, you know, this is a confidence I feel comfortable with moving on to the next step. Or, you know, you can kind of make it either very high of a of a quality threshold or very low, depending on kind of what you want to do with it. Typically, we see like, you know, 70% confidence is what what we see a lot of folks kind of define initially there. But, you know, I'm not having To think about like how the tools are provided to those sub workers.

• 23:49Speaker 02

  I'm not having to think about how that planning and self correction loop is defined for the orchestrator. I'm not having to think about like how the orchestrator provides the relevant context to just the workers that it's currently.

• 24:01Speaker 01

  That was kind of what I was gonna ask about is like, what is the communication path back, forward and back? Like, is it just dumping files in place for them? Is it, you know, standard in and standard out? Like, can it hand off more complex things? Like, if I have a sub agent that pulls together a database for me, can I hand that to a different sub? Like, what are the the tools available in the tool chest?

• 24:21Speaker 02

  Yeah, so we we kind of go for like a single path in, single path out for right now. That'll change as the harness develops, but you know, it's meant to be, hey, the orchestrator does the planning and then hands it off to the workers and the workers hand it back. And then the orchestrator can hand it to different workers if it determines that, hey, I've built some context, but I need to clarify it. I need to get a little bit more context, execute another tool, or maybe execute the same tool again because it didn't give me the result I was looking for. And so the orchestrator becomes kind of the important part of that process. But the thing that we're starting to see more and more of is like, you know, these multi-agent workflows, you know, they have a lot of tendency to become really complex really quickly. And so we wanted to make sure that was simple on day one, but like, as we see more and more of these patterns emerge,

• 25:05Speaker 02

  we definitely want to make sure that Aura kind of enables you to like gain the benefits of either like performance or reliability out of what that like orchestration flow looks like. One thing I'll also call out with with orchestration, which is pretty interesting, is that, you know, when you think about like how you are gaining better economics out of the tokens that you're using in some of these workflows, again, you're dealing with scale. And so like, you want maybe the bigger model, maybe you want an opus on how you're doing the planning and the reasoning in the top level orchestrator. But then on those sub workers, you can actually declare a different L M, a different model behind the scenes that, you know, operates as more of a delegated task worker, right? Like, hey, maybe I want to use Haiku, maybe I want to use an open source model.

• 25:45Speaker 02

  And so that actually is also something that Aura makes insanely easy to do using our declarative format. So that you can say, hey, I want to not only use a multi-agent workflow, but I also. Want to use smaller agents for the workers and ultimately get better, kind of like token economics out of the whole process. And that's kind of like one of the things that we solved on day one that we've, we've seen some pretty surprising, like pretty exciting results out of just enabling that to happen and kind of embracing the idea of a multi-agent system with a lot of different models in the mix.

• 26:17Speaker 01

  Totally. Yeah, there's a lot of like classic like data categorization, extraction, et cetera, that is trivial to do with a small model and you really don't need to throw up a set it, but you want that as a sub piece of your broader thing. So yeah, making that easy is is super helpful. So to playback, making sure I understand, you're sort of right now, operating on like sort of single orchestrator, set of available workers, the orchestrator is deciding what and just sort of spawning things out and then it can dynamically decide, hey, I got this from here, but actually I want to send it over there, doing things, but it's kind of single level of delegation. Is that?

• 26:50Speaker 02

  Exactly. Yeah, exactly. And we're to your earlier point too, we're, we're making it as simple as possible. A lot of our interactions are just utilizing flat files on the system. And so, you know, that's something that pretty much every L M, every agent is good at interacting with is a file on disk. And so, you know, we're not introducing a lot of like, you know, custom proceeds. We're kind of operating on the, the simple, most reliable thing that when breaks, you know why it breaks, but ideally it just doesn't break because it's a simple enough mechanism.

• 27:17Speaker 01

  I want to come back to talking about memory and human loop because those are interesting, but that that points in another direction that I think is really interesting to explore, which is like the observability of the agents themselves. Like we've been talking about, you know, how you're using these to tackle observability data and things like that. But like, what tools does Aura have for making it self observable? So that, as you said, you know, when it breaks, you can tell why it broke and how to fix it?

• 27:41Speaker 02

  Yeah, being an observability company, this is like one of the things that we made a very hard requirement for us to, to solve. Basically, you know, even when we didn't GA Aura, when we're kind of in a pre-release, like an early access type of thing, we instrumented everything in Aura natively with open telemetry. So we're heavily invested in like, open telemetry for the agents. We define everything again in the readme, you can see exactly how we're instrumenting that, what Pattern we're using for open telemetry, which were built on top of open inference for open telemetry. And so you can see exactly in the spans of that trace of the agent interaction, you know, hey, how is it doing the reasoning for the next step? Was the tool call successful, et cetera? So we organize that automatically and we build on top of open telemetry to make that happen.

• 28:26Speaker 02

  And the other thing that we have natively built into Aura is a really rich set of server side events that can be returned back to the client that can really show you like what's happening live as part of the investigation process or like what is the agent doing behind the scenes? So, you know, definitely wanted to, didn't want to get to the point where it's like, hey, I give this really complex task to maybe a multi-agent workflow and I just kind of like wait on the spinner or anything like that, like as that's executing. We have a CLI built into Aura, so you can very easily just like start interacting with the CLI. And that is, you know, it'll very easily show you all those different events that I just mentioned, right? So like you can do slash stream, you can do slash expand and it'll show you like, hey, what are all the activities that I can see from the agent happening, you know, behind

• 29:10Speaker 02

  the scenes as I'm interacting with it? And then, you know, because it's built on top of open telemetry, I can send that to my observability system and that makes it really easy for me to understand, you know, the auditability of like the interactions from the agent, what did it actually do and why, right? So like the reasoning steps across each process are also captured in that trace. And so we just wanted to make sure that, you know, one, it's built on open standards. You can send it to anything that supports open telemetry and it becomes a great way for you to start building trust in your agents, right? Again, like this is a big barrier in the industry, I'm saying, is that, you know, for SREs, like anybody who's dealing with like a production use case, you really have to understand exactly what's happening behind that agent in order to trust it.

• 29:53Speaker 02

  Otherwise, you know, you're not going to put it into production, you're not going to, you know, offload more tasks to it, you're not going to really integrate it into your workflow. And so the, the observability or the transparency of what's happening behind the scenes, like that was a insanely important piece that we wanted to solve on day one.

• 30:09Speaker 01

  One of the big patterns I'm seeing right now, once again, doing the metaphor to coding agents, because that's where I live a lot of my time, is like looking at traces essentially of how are things going and using that to iteratively improve. And, you know, if you're talking about an observability agent or an SRE agent, it is the same thing, but you can't just be looking at files on your laptop anymore. You've got to have those deep observability traces.

• 30:30Speaker 02

  Totally. Yeah, exactly. Because we're built as an observability tool, you know, our platform enables you to send those traces directly to us. So obviously, we have the ability to pull those in, visualize those, show you the different, you know, service mappings and the interactions. Most people have an observability tool in place. And so it's easy for you to take Aura and just point to, you know, whichever open telemetry provider you decide to use. You know, the visibility that you're going to get into those workflows is not going to be limited to, you know, you using our commercial product.

• 30:59Speaker 01

  Let's come back and talk a little bit about memory because I think this is also a really interesting domain. And I'd love to get, once again, one or two levels deeper on how you approach it. As you commented, a lot of memory implementations out there are pretty naive. I'm going to use an even simpler example. You look at ChatGPT memory or something and it remembers everything. It doesn't give me any visibility into what it remembers or why. And, you know, when I'm working on something for my son's project, it says, hey, because you're a director or you're a VP of engineering, you should be thinking about it this way. And it's, it's just this one big undifferentiated mess. Obviously, that's not what you want in a production SRE context.

• 31:42Speaker 01

  It's, you mentioned examples of exposing incident logs or incident histories or things like that. So like, how does your memory architecture work? Once again, first from a user perspective and then like diving in under the hood.

• 31:54Speaker 02

  It's a good question. And, you know, a lot of this, you can actually see in our PR is like how we're developing the next iteration and what we're, what we're already starting to improve within Aura. But the way that we look at it is like, there's different levels of memory. And so, you know, there's memory like more turn-based or session-based, basically, like that you want the LM to be able to, to recall, you know, what was the last tool call that happened on the last couple of seconds? Right? Like, or what was the previous run of this agent able to come to as a conclusion that maybe is, is right in front of the agent at that moment? Right? And that's more of the turn-based or session-based memory, which is kind of like the lower levels of where we want that really fast. Kind of like when you think about memory architectures with like, you know, having DDR is like, you know, my, my RAM versus a hard drive

• 32:38Speaker 02

  versus, you know, something that's like a. Offsite storage, we kind of took the same approach with Aura where there's very simple memory that's built for turn-based and session-based that again, you can tune along the way. And then what we're looking at, like, the really interesting problem is when we start to get into some of those like longer horizon memories. And this is actually where like, you know, we are working on this within the platform itself, within the Mesmo platform, to offer this as something that, you know, is, is a little heavy to do in line, but it's actually great to do kind of as a, as an async or background process that just gets exposed to the agent that it doesn't have to waste tokens on. And so this is where we're looking at like, again, more of the kind of needle in the haystack, incident over incident investigation, but like tailored to the types of workflows that

• 33:22Speaker 02

  SRIs would deal with, where those memories are not just about the last tool call, or, you know, the last iteration of this agent running, but it's like, hey, what happened in the last six months? What happened in the last 12 months? What happened like with this service that I started to see maybe a little bit of service degradation and also a change that somebody pushed from GitHub and also, you know, started to see some funky stuff in the logs? Like, that all becomes a collective memory that we're kind of creating as a simple briefing to provide to the agent that just makes it easier for that agent to either solve the next thing, you know, the next time better, or potentially just short-circuit the process entirely and say like, hey, you know, normally I would kind of come to this conclusion of doing this next thing, but I can see the last time I did this, this thing. And I can kind of compare that to ground truth in the background. So

• 34:07Speaker 02

  I can kind of validate whether or not that had the intended impact that I needed. And maybe that'll change what I do the next time that this happens, because now I've actually evaluated that in the background. And so that's kind of like where the longer horizon memory is like something that we're very interested in, because again, if you look at like, you know, just taking either the graph, you know, memory MCP server that most people are familiar with, that you can just grab off of a, kind of the standard MCP servers or something like Memzero or Mempalace, or any of the kind of very general MCP, you know, memory servers. We wanted to make sure that that was, again, shortening the circuit to the outcome or shortening the path to the outcome where we know the types of tasks that SRIs need to solve for. And we know how memory becomes really important to some

• 34:50Speaker 02

  of those longer horizon tasks. And so we're just going to provide that on day one.

• 34:54Speaker 01

  What does that end up looking like technically then? Is that a kind of rag-like implementation? Is it exposed via another, an MCP? Or it's a tool that, a separate tool that you can query? Or like, how does that end up actually playing out in the agent?

• 35:09Speaker 02

  Yeah, so the way that we're exposing it today is just taking like the simplest, again, lowest common denominator across the board, which typically ends up being another tool. And so, you know, the way that we, we look at it, like, we call them investigations. And so as I'm handling either a new investigation or looking at previous investigations, that's just another tool available to the agent that we expose through our MCP server. How that practically looks, right, is we're providing kind of a briefing as part of that tool call. So it's kind of like the summary and then like where the agent can go for the next steps. Something that again, like, because we're kind of focused on building on top of these, you know, lowest common denominators in the space, which a lot of this looks like, you know, just a, an MCP tool. You could plug what we're doing in the background for memory or what we're doing in the background for context engineering,

• 35:54Speaker 02

  like longer horizon things into Aura. And that's, you know, what, what we see is a, a great path to consume that. But you can also plug it into, if you've already invested in that agent framework, that's, you know, you've already gone through the glass to get to that outcome that we were just talking about before. Or you want to plug it into Claude, you want to plug it into wherever, you know, you can easily do that. We did experiment with a little bit, and again, you could see this in our PRs, doing things like providing a virtual file system to the agent where we can kind of like, seed in those memories without the agent actually having to do the tool call. And some of that's in a kind of an experimental phase where we're looking at like, how can we better expose these memories that don't actually require the agent to do a tool call? And is rather just like transparent to the agent where it's not wasting tokens. But some of that stuff is like, kind of in a research preview.

• 36:39Speaker 02

  And again, you can see that in our PRs and kind of follow along with that.

• 36:42Speaker 01

  Yeah, no, it's, it's super interesting. And maybe this connects a little bit to the context engineering question, right? Of like, what sets of things do you want to embed in your initial context? So it just kind of knows it versus make it on demand, go and call it versus, yeah, have it in an explorable database or find, you know, some other way of, of finding it.

• 37:02Speaker 02

  And there's also one last thing that I, I want to make sure I also. Layer on here that's related to memory, we don't always call it memory, but it's like on the same exact path, which is things like institutional knowledge, right? So like more of the institutional memory or institutional knowledge, right? Like where, for example, every SRE team, right, has a set of runbooks that they have internally. Maybe that lives in Confluence, maybe that lives in GitHub. Maybe that lives in, you know, insert your favorite place to store runbooks, right? And so that becomes incredibly useful grounding for the agent. And so, you know, we also expose a really easy way for you to either connect us to where those runbooks are currently stored. If they expose something like a vector database, it's really easy to plug that in and be use that as grounding

• 37:46Speaker 02

  for the agent. So the agent will always kind of ground and respect itself based on or respected the workflow based on what you've defined as the the source of truth being your runbooks. We've also actually published a blog post on like, hey, if the agent has the ability to not only consume those runbooks, but as it starts to see things during its investigation process, how can it actually contribute back? And we've published a blog post on this, how we're actually doing this internally, where we have our runbooks stored in GitHub. We actually use Aura for a lot of our own SRE workflows internally. And as we identify things that like, hey, this workflow has changed based on what's in the runbook, instead of that becoming a a human process where somebody's maintaining the runbooks and it's only a one directional, you know, like consumption by

• 38:30Speaker 02

  the agent, we've actually seen a ton of really positive success with providing the agent a feedback mechanism to basically iterate on those runbooks, open a PR and say, hey, I did this investigation and I need to update the institutional knowledge. And that's just like a really good example of kind of the opinionated memory where like, everybody has institutional knowledge in some way or form.

• 38:51Speaker 01

  Yeah, well, I love that because it closes the feedback loop. And it gets us to the last sort of pillar that you talked about that maybe we want to dive into, which is like, the amount of human in the loop you have. And sort of you described here, a version of human in the loop, which is like, I did a whole bunch of stuff, but here, let me suggest to a human an improvement. But yeah, how do you think about, you you mentioned, which I I love this idea of having the sort of gradient from Almost entirely human with Copilot going on to completely automated. But like, what does that end up looking like? How do you end up configuring it in Aura? And where do you see different types of SRE tasks typically falling out in terms of that level of autonomy versus human run?

• 39:35Speaker 02

  It's interesting because, you know, the way that people are looking at human in the loop today is like very dependent on the types of systems behind what the agent is interacting with, right? So that's kind of like one parameter, one dynamic. There's also like, how confident are you in the workflow itself? And then there's also like, how confident are you in the data that has been returned through the workflow, right? So there's like an aspect of how you provide the ability for somebody to say like, hey, these types of tools, which maybe are read only, or sorry, write, you know, they have the ability to write, or they have the ability to do something that, you know, it's kind of a one-way door, right? Like, I want to make sure that I provide a very hard kind of barrier like for the agent

• 40:17Speaker 02

  to be able to know that that is something that's insanely sensitive and it needs to ensure that it asks for help before continuing on. Like, that's kind of one piece that, you know, we see everybody with some level of like, hey, there's remediation, but like, I need to make sure that there's always a human that hits the go button before it does remediation because I know that that's a, you know, a one-way door that's really hard to come back from. That's kind of like one aspect of like, limiting or providing human in the loop on the tools themselves. Then there's like, you know, how far do you want the agent to go in the process before, you know, ask for a person to kind of let it continue on with go, right? So like, maybe you're not limiting access to the tools themselves, but you're limiting access to like, how far in the process the agent can get to, right?

• 41:02Speaker 02

  Like, you can basically act as an assistant, but at the end of the day, the person who is implementing the remediation to what you've investigated is always going to be a person. I'm not going to provide you those types of tools and it's always going to be, you know, you're, you're read only, you're only getting me up to the point where you're creating a nice package for an SRE to go execute a script, remediate a problem, but you're coming to exactly what needs to happen before, you know, that point becomes like- A simple place for the agent to either determine that it needs to continue on or you know, just completely hand off the process. And then the last one is, you know, where we start to really get into, I think, the like, the true benefits that people are looking for, which is when I'm thinking about like a truly autonomous workflow, right? Where, hey, instead of S3 is handling this

• 41:47Speaker 02

  intent, it's actually an agent handling this intent. That's really where we start to get into, you know, like an aspect of, hey, how do you use some of the data that we were talking about before with the observability data, right? And like, grade that over time, where even if you think that that's a pretty good workflow and you have the right data and you have the right workflow to find, you want to make sure that it is tested a certain amount of times before you give that the full like thumbs up, continue on, operate autonomously, right? That's kind of like the graduation process. So like, there's, there's those like, three different areas there. Now, how that looks, you know, specifically with an aura, it's all driven through the config file. Where it looks a little bit unique is like, as soon as we start to get into those like

• 42:29Speaker 02

  longer horizon things, again, where it's like, hey, I'm not just limiting access to the tool itself, but I'm telling the agent where it needs to go ask for help. That's where we're exposing a tool within our MCP server to basically provide to the agent. So like, the agent doesn't need to go figure out, hey, you've told me that I need to go ask for help here, but like, I gotta go figure out who's on call. I gotta go figure out how to get access to Slack. I gotta go figure out how to do, you know, X, Y, Z. Instead, what we're doing is like, that's a process that we're seeing across, you know, a lot of S3 workflows that needs to happen. So like, we're just exposing a simple tool for the agent to be able to ask for help. And then what we're doing behind the scenes is like, some of that correlation that I mentioned, hey, who's on call? Who do I need to go ask? Trigger a notification via Slack? Somebody give me the thumbs up and then provide that to the agent versus the agent, again, having to do that itself.

• 43:14Speaker 02

  So that's kind of like how we're not only defining the different types of human in the loop in the process, like governed autonomy is kind of like what I, what I think about it, but also make it so that our platform kind of provides the simple mechanisms for the agent to kind of like, ask for help from a single place without having to bloat a bunch of context to go figure out all the possible places that it could go ask for help. And so, like I said, the way that we operate With doing that, right? Like Aura is open source. We always expect people to be able to use Aura with or without Mesmo. And so, you know, you can definitely point Aura at a different tool that's, you know, a different human in the loop process. And we publish kind of the spec there of like, if you wanted a different human in the loop process, here's how you can do it.

• 43:56Speaker 02

  Here's how you can define it. And then we kind of like operate on, hey, we opened that spec, we've defined that spec. But then for the actual implementation, okay, great, we're going to make the simplest path for you to realize that implementation, which ends up being just, hey, our MCP server is going to naturally provide this to you on day one. So you can consume it right off the bat.

• 44:15Speaker 01

  Yeah. Well, and I think that's a really nice way to, to have a pairing between a commercial product and open source. So if you say like, look, all the doors are open, you can do all the different things. And here's some things that we know you're probably going to want to solve a lot of times that we solve well for you. So here you go. That kind of gets into an interesting question around sort of the future of Aura. And in particular around like governance, right? So is this, right now, this is a Mesmo run project and you all are doing most of the work? Are you looking to get it into third party foundation? Or how, how are you thinking about governance of Aura going forward?

• 44:49Speaker 02

  Yeah. So right now Aura is a small but mighty project. And I say that because, you know, the primary contributors are from Mesmo. We're starting to see more and more people interested in the space. And the fact that we've gone with the Apache license makes that a lot easier for folks to be able to contribute without having to jump through a bunch of hoops. We're a member of the AAF, so the agent foundation that's underneath the Linux foundation. And we're active contributors there. And so, you know, there's definitely a future here where I think as we start to get more and more interest in Aura. And, you know, we, we have no intent of this just being like, you know, the Mesmo way to run agents. We want this to be as like general purpose of a best practice standard in the industry. And so I think like, you know, not to put the cart before the horse, because we, we still want to get as much proof

• 45:32Speaker 02

  and partner with as people as closely as possible, where we can very quickly, you know, make iterations where it makes sense. But I think as it starts to mature as a project, definitely makes sense for, you know, potentially contribution to. I think CNCF is probably pretty crowded, but like the great thing about AAF is it's, it's, it's fairly new, fairly green filled. And, you know, there's a lot of folks that are interested in kind of moving the space forward through that. So definitely looking at that as a, as a good option potentially in the future here.

• 45:59Speaker 01

  Yeah, totally. Well, and I think it makes a ton of sense to keep it tightly controlled to start with, right? Like, I've definitely seen people try to open things up a lot too early, right? And it leads to chaos oftentimes, right? Like, the benevolent dictator for life, even if the dictator is a company, like, that's a good way to start a project and keep it moving. If folks have not used Aura, they are curious, they want to get started, maybe they want to try contributing or tweaking it, what is your sort of on-ramp? What do you recommend?

• 46:29Speaker 02

  Yeah, so one, we have a Slack community. So, you know, everybody from Mesmo is there, you can kind of get help. If you have an idea, we're very interested in kind of like fielding, you know, what people want Aura to look like in the future and what ideas they have in the space. And you can definitely open an issue on the GitHub repo like you can for many things. PRs are definitely welcome as well if you're interested in donating your tokens, right? Like, and we've tried to make that as coding agent friendly as possible, where we're, you know, heavily leaning on like the Claude MD and Agents MD and things like that. And so we're trying to make it as easy for folks to contribute to. But then again, that Slack community is a great place where like, if you don't know where to get started, or you're having some problems, or you just need a little bit of help or have some questions,

• 47:10Speaker 02

  you know, pop into that community and very interested in helping, you know, helping where we can. And we're also, you know, very interested in partnering with people. Like, if you're looking at how you want to change the future of SRE and you're aligned to this conversation that we just had, and, you know, you want to partner with a company like Mesmo that's kind of like looking at building this, you know, firsthand, and we have engineers actively working on this, you know, definitely reach out and we're like happy to provide any kind of support or like implementation, things like that to make sure you're successful. So, a lot of different paths based on, you know, just community contribution versus like true commercial partnership and design partnership there.

• 47:47Speaker 01

  Yeah, I love that you're doing an agent-friendly open source project. I know a lot of open source maintainers have been kind of resistant to agentic coding. But having that first and figuring out how do you design your workflows and your processes and your automations and everything to make that friendly? I mean, that's, to me, that feels like what makes sense in this world we're now in.

• 48:09Speaker 02

  Totally. Now, we're in this point where the code is cheap, but the ideas are expensive, right? And so it's like, the easiest thing for us to do is make that process, you know, as streamlined as possible, right? Like, we know that coding agents are the future and they're here to stay. And, you know, it actually like lowers the barrier, right? There was a while where like, in order to contribute to an open source project, you had to really be in depth in that project and like actually understand what's happening, maybe be an expert in Go or Rust or whatever. You know, because Aura is written in Rust, not everybody knows Rust, and we totally understand that. But every agent knows Rust, right? Like, anybody who has access to Opus or any of the coding agents, right? You know, you have an expert in front of you and you have your tokens. And so,

• 48:51Speaker 02

  you know, now the donation to open source becomes, hey, if you want to donate your tokens, these projects like ourselves and other folks in the space, we just need to make it as easy as possible for you to do that.

• 49:01Speaker 01

  Well, I realized we're getting close to the end of our time here. Are there any things we haven't talked about yet that you think would be good to talk about before we wrap?

• 49:09Speaker 02

  You know, there's a concept that we're really keen on advancing, uh, kind of in the industry and kind of how we operate as Mesmo, which is, you know, there are two kind of like axes on this, in the space, right? There's what production AI looks like as far as like AI in production and like all these things that we're talking about, right? Like multi-agent workflows, there's, uh, you know, how you reliably tell tools, there's memory, there's things like that. And that's all like, how does an agent operate well in production? And then there's another axis, which is like AI for production, right? So like, that's where you think about like, what are the types of tasks? What are the types of skills needed? What are the types of tools needed that really enable these agents to

• 49:49Speaker 02

  handle these types of tasks that we see high value and something that's, you know, inherently hard to solve. And so we think about production AI and like how we're trying to shape the industry around like this is a problem. It's really the intersection of those two things where we want to enable. SREs to not only own their their own agents and like kind of go through this process safely and reduce the risk to get to the outcome. But like what we see as a healthy side effect, which I personally, I'm really excited about just from being a practitioner earlier on in my career, is really like this is our mechanism to also elevate the whole role of what SRE means, right? Where we're more, we're architecting reliability more so than responding to things that we've implemented.

• 50:31Speaker 02

  Like, like, we're taking more of a design approach for how we can make systems reliable, more so than being firefighters all the time. And so I think that's a really interesting opportunity that we have to move the space forward. And like I said, I'm just, we're really excited about like where this is already starting to show success and where we see this kind of advancing in the next months and years and things like that, that really bring all the SREs today that are used to automation, they're used to, you know, handling these really high risk scenarios and just giving them the tools and the ecosystem to make that ready for the agent era.

• 51:04Speaker 01

  I love that. It fits really well into something I've seen in a, in a bunch of AI spaces where like a lot of people are trying to delegate their thinking to AI, but where it really shines is when you can use AI to elevate your level of thinking and really do something that's more impactful, deeper, more just leveraged, all of those different things. So can you like maybe pull one example out of how using this at Mesmo has helped you elevate your, your SRE approach?

• 51:31Speaker 02

  Yeah. So one is, you know, again, like we've taken the approach of making it less like a firefighter and more like a reliability architect. So all of our SREs, right? Like we have all of our engineers on call as well, right? Like we take the approach of like the SREs are the ones building the guardrails and building the systems, building the workflows, but that makes it easier for other folks to also take that pager, right? Like be, be on call for the systems where, you know, all that tribal knowledge that typically was trapped in an SRE's head, like agents are a really good mechanism to make that kind of a next iteration of like as code, right? Where like we get that out of people's heads and make it more automated and make it more standard practices. And so that's really helped us to really iterate internally on what's the next iteration of.

• 52:15Speaker 02

  And really use our own ability to, you know, capitalize on our own harness and like our ability to kind of like incorporate our own learnings there. And we really just like elevated our SREs internally so much to the point where like it's made the role really exciting and, and, and really given people internally the ability to kind of see what the feature looks like, which then makes them also excited to talk to other SRE teams at other companies, which has been kind of a cool sight to see is like we're getting to the point where we can actually share knowledge, build a community, build, you know, kind of like these standard practices that extend outside of Mesmo as well.