AI Agents Fixing Your IT Before You Even Know Something Broke | Erhan Giral & Ryan Manning, BMC Helix

Most enterprise IT teams spend the majority of their time fighting the same fires repeatedly. BMC Helix is building the AI system that handles those fires automatically, detecting anomalies, tracing root cause through millions of asset relationships, generating remediation plans, and learning from every incident it resolves. Craig Smith sits down with Erhan Giral, VP of AI Strategy and Innovation at BMC Helix, and Ryan Manning, Chief Product Officer at BMC Helix, to explain how agentic AI is transforming IT service management from a reactive, human-driven process into something closer to a self-healing system, and why doing that at enterprise scale requires a fundamentally different architecture than most AI deployments attempt. The most technically interesting part of this conversation is where BMC Helix is headed: building "gyms", synthetic data center environments where AI agents deliberately break things and learn to fix them overnight, 24 hours a day, generating the bespoke operational training data that text-based foundation models can no longer provide. Erhan describes an architecture of specialized sub-agents, anomaly detection, log analysis, root cause analysis, remediation planning, that work in a hierarchy, passing hypotheses between each other until they converge on an answer, fine-tuned to reason the way a specific enterprise's best IT engineer would rather than the way a generic documentation page reads. For customers, the results are measurable: 25 to 50% cost reduction, fewer recurring outages, and IT staff who can finally go home at a predictable time rather than spending their nights firefighting problems that could have been prevented. Subscribe to Eye on A. I. for weekly conversations with the people building and deploying the future of AI.

• 0:00Speaker 00

  What's that gonna do to IT staff? It sounds like it's gonna get increasingly automated.

• 0:05Speaker 01

  That space is changing pretty quickly and that workflow is changing. Agentic AI pretty dramatically.

• 0:11Speaker 02

  Right now, AI, if you think about AI is only learning through someone else's description, right? People will start exposing these models to the real world more and more so that they can create, they can gain that first-person view of things. Once we detect something, we always ask the data, why, why, why? Until we get to a point where we can take an actionable step to mitigate or perhaps even remedy that issue.

• 0:34Speaker 00

  Why don't you introduce yourself to listeners?

• 0:37Speaker 02

  My name is Arhan. I run the AI office for BMC Helix now. I've worked with Ryan on applying AI to various different problems in service ops space. So service ops is essentially, you can think of that as any digital business these days is really a conglomeration of IT, IT services you can imagine. And service ops is essentially an attempt to automate operational activities around these services as much as possible. So that we look at enterprises as a combination of software, infrastructure and human beings operating on these, on these assets.

• 1:22Speaker 02

  And we provide various products and nowadays agents to automate different workflows around these, around these services. My personal background is in monitoring space. I've, I've done early work on monitoring applications and infrastructure and then combinations of them, which means, you know, you have to process a lot of machine data. You have to reason, be able to reason about high throughput, you know, streams of data that these data centers and these machines are constantly emitting.

• 1:59Speaker 02

  So yeah, that's what I do in Helix.

• 2:00Speaker 00

  We're gonna talk about Helix and, and IT service management. Can you begin, one of you, by talking about how the landscape has changed with the introduction of AI agents? What the process for service management was before? And, and then what it's migrated to today?

• 3:06Speaker 00

  Yeah. And, and, you know, just thinking back in the old days, it was Jira, right? You would, you'd open up a Jira ticket. It is, it is, does your solution work alongside Jira or replace it? Just for people that aren't that familiar with, yeah.

• 4:23Speaker 00

  Right. And, and, and you guys, we'll give the background of BMC first. We were talking about that before we started.

• 4:50Speaker 01

  Yeah. Yeah. So I mean, service management, the discipline is, is complicated. The, the explanation of what they do is pretty simple. It's the front door to IT. So if, you know, I'm a new employee trying to get my laptop provisioned or if my, if I'm having an issue with an application, I might go to a portal, send an email, call the service desk and try to get that remediated. And that, that, that space is changing pretty quickly and that, that workflow is changing with agentic AI pretty dramatically.

• 5:54Speaker 01

  Yeah, no, that's, Jira does have it. So our, our competitors, you know, go beyond ServiceNow into Atlassian, into Freshworks. The market is a lot of players. The interesting thing about the service management market is that unlike the CRM market, customer relationship management, it's exceptionally difficult to service small, medium, large, and extra large customers in the same platform, not just from a product perspective, but from a go-to-market perspective.

• 6:24Speaker 00

  Enterprise and the economy generally.

• 6:26Speaker 01

  And so in the high enterprise where we operate, we'll say the Fortune 2000, you submit a ticket, you're either submitting it to ServiceNow or to BMC Helix. Down market, Jira is a very popular tool. Freshworks is a very popular tool. I've got a few others. Yeah.

• 6:58Speaker 01

  Yeah, I like to think about BMC in four eras. Era one, Remedy Rules. A lot of folks know Remedy. That was an acquisition that BMC made years ago and was the ServiceNow and service management before ServiceNow was the on-premise ServiceNow.

• 7:17Speaker 01

  Era two, about 2010, the music began to slow a little bit. Lost our way a little bit, a little bit in terms of, you know, hopping on the cloud train. And the company ended up getting bought by Bain and taken private. It sort of continued the same mission.

• 7:39Speaker 00

  Yeah.

• 7:40Speaker 01

  Then era, era, era three, KKR came in and said, hey, ServiceNow still only has one competitor, it's BMC Helix. What if we took a different approach? What if we brought in a technical team?

• 7:52Speaker 00

  In the operations of those companies.

• 7:55Speaker 01

  Just recently turned around CA and sold to Broadcom. I'm in Saad Ali Siddiqui or on.

• 8:01Speaker 00

  To their.

• 8:02Speaker 01

  Were asked to come in and rebuild the platform from the ground up. You know, AI first. And that, that journey, you know, took us to close to where we are today, where we, you know, went through the, the natural progression of the heartaches for our customers and for ourself and rebuilding a platform and still servicing some of the biggest companies in the world to now being able to innovate and separate ourselves from the competition with the innovation that we got access or permission to build through KKR's ownership.

• 8:51Speaker 01

  Yeah, yeah, and there's the service management being the front door to IT can encompass a lot of different requests that need to get fulfilled. Some of them are very simple. You know, I need a new iPhone. Some of them are, hey, the quoting tool is down. It's the end of a quarter. We're in trouble. And there's all these workflows that sit behind that request, some of them more complex. When those more complex issues come in.

• 9:18Speaker 02

  Generative models have amazing capabilities in the human language space.

• 9:21Speaker 01

  The front door team talks to the engine room.

• 9:23Speaker 02

  I think everyone grasps.

• 9:25Speaker 01

  Keeping those services up to date.

• 9:26Speaker 02

  What we've been trying to do is can we first get them to create structured out.

• 9:32Speaker 01

  And so what we did in that transformation I talked about is we brought those two disciplines together, service and operations.

• 9:40Speaker 02

  The machines naturally emit.

• 9:41Speaker 01

  Category service ops.

• 9:43Speaker 02

  Because when you want to reason about that massive graph Ryan was talking about, you have to be able to quickly process all the telemetry data.

• 9:52Speaker 01

  The CIO doesn't have to worry about lots of outages taking too long to restore that outage.

• 9:58Speaker 02

  Understand that graph and those relationships.

• 10:00Speaker 01

  And certainly like less regression to the same thing happening again.

• 10:03Speaker 02

  And in IT, it's very common that you have to constantly deal with deep relationships because something that runs on a computer turns out it's actually not a real computer. It's a visual computer that is resourced by a physical computer somewhere else. So there is all these resource dependencies, resource and transactional dependencies in the fabric that you have to understand before you can be, you know, you can produce anything actionable and useful. Because on the on the operations side, what we want to create is not.

• 10:34Speaker 01

  The complicated thing, so on the service management side, a lot of disruption with AI.

• 10:38Speaker 02

  There's a lot, any language-based workflow is right for disruption.

• 10:42Speaker 01

  When I used to have to go to a portal and call someone, now I can just go to Microsoft Teams, type something in a virtual agent.

• 10:51Speaker 02

  And that does require to be able to do that in a reliable sense. You have to create repeatable, predictable and structured out from the machine, from the models.

• 11:02Speaker 01

  But when those technical infrastructures come in, there is so many relationships between corporations and infrastructure.

• 11:08Speaker 02

  It's amazing. We have a customer with only 200,000 assets in there. Together and viable.

• 11:14Speaker 01

  Their CMDB where they store all their assets.

• 11:16Speaker 00

  Can you walk us through a use case or case study?

• 11:21Speaker 01

  And so go into that space, it's not like, hey, let's just plug an LLM in and run. Hopefully you can expand on this and you get your answer or you get that remediated.

• 11:30Speaker 02

  From outside in, it's actually very simple. So you first need to understand, hey, do I have a problem?

• 11:36Speaker 01

  To get to root cause, why is that happening?

• 11:38Speaker 02

  Should I mobilize anything at all? Like, do I have a problem? Awareness and identifications of problems is one use case, you know, we constantly deal with. And that requires our agents to stream this telemetry information. This can be your machine logs, time series information, alarm information that they emit. So something needs to be there and constantly monitor these things. And we have a funnel that deals with that. Let's take the observable data and run that through various machine learning models so that we can understand, oh, you know, that spike that you see right now at 8:30 AM on a Monday on the bank's ATM network is actually kind of unusual.

• 12:22Speaker 02

  That you don't expect that spike to appear at 8:30, perhaps maybe it's supposed to come in later. So something needs to essentially constantly reason about, hey, you know, what's unusual? Or given the time we are in, or given the workload we are dealing with, given the circumstances in general we are. So that's one part of the big use case we try to do. And of course, once you detect an anomaly, once you detect an availability problem, you then first ask, okay, so why? You know, why? What is the root cause of this issue? You know, why am I, why is this router so busy that, you know, where it should be, you know, close to idle state, for instance? So then that we call that the root cause analysis.

• 13:04Speaker 02

  So it's actually something that needs to dig into all that telemetry data, dig into all that observability data to say, ah, it's, you know, it's actually, it's not that, that's a symptom. What's really going on is, you know, such and such file system or such and such, you know, network queue, such and, you know, in broker somewhere is now full and, you know, dropping the traffic or what have you. So, you know, you always ask, you know, once we detect something, we always ask the data, why? You know, why, why, why? Until we, until we get to a point where you can take an actionable step to mitigate or remedy that, or perhaps even remedy that issue.

• 13:45Speaker 02

  And while doing that, of course, you know, data centers and IT is very target rich. All kinds of stuff happens all the time. So you need to be able to do this by sitting away all the noise, all the background noise that typically happens in that environment. That's, it's like, you know, there's always this cosmic background noise that you have to essentially first detect and then push away to then focus on the real signals in the data to understand what the root cause is. And then, and then we also do impact analysis, like, okay, so is this a big problem? Because sometimes technically challenging problems are just that, you know, no one, no

• 14:28Speaker 02

  one cares about, maybe it's a QA environment, maybe it's a staging environment that's being, you know, modified at the moment. So essentially from economic point, economist's point of view, we always ask, okay, we have a problem, but is this an impactful problem? Should we mobilize human beings? Should we spend more resources in mitigating this issue? So that, so our system constantly makes these decisions in the background by looking at real, real time status of systems while also constantly comparing the state to its past self. And we also make, you know, plans of action based on plans of actions that were executed

• 15:11Speaker 02

  in the past by human beings or by bots.

• 15:14Speaker 00

  And this, well, first of all, this is all being done within an agentic layer. It's not, and with reasoning models to look at the problems and try and figure out, it's not kicking questions out to the, the human team who then sit around and try and figure it out. It's, it's doing this internally, right?

• 15:43Speaker 02

  That's right. So some parts of what I just described, that it's actually taking that, you know, gigabytes of gigabytes of monitoring data and then reducing and then essentially pushing that noise away. That's typically, we employ proprietary technologies to, to, to, to, to do that because there's so much data to process. We couldn't really hope to expose all of that to the generated models all the time. So we take all that sparse data and run through various machine learning and statistical

• 16:22Speaker 02

  analyzers first to, to, to, to basically find the needles in the haystack, if you will. But once those needles are found, they are collated, combined, correlated, causally analyzed, and then the LLM, you know, the LLM is given a pretty comprehensive, causally described like a, like a, like a patient chart, like a, like a set of X-rays of the system. And then we ask, we ask the LLM, okay, I mean, we ask various questions to LLM, but most importantly, you know, we ask the question, why?

• 17:05Speaker 02

  And, you know, what needs to be done? And that level is completely agentic, just like you said. And there we use reasoning models, just like you said, maybe with a twist. We, because we want our models to reason just like one of the employees of our customers. Because like, for instance, if you go to an open AI model or a generic off the shelf model, they'll always give you plausible responses, you know, accurate responses based on the documentation, based on what they know about the space. But it's often the case that when you go to a big organization, the rules of engagement around these systems are very different.

• 17:47Speaker 02

  Meaning you can say, oh, you know, go to this file and make these edits and everything should be fine. But then the, that bank's knock, you know, network operation personnel will probably say something like, well, you know, that sounds plausible, but that's not how we do things. You know, I need to first, you know, run these additional tests. Maybe I need to talk to some other team, get their approval. You know, I need to make sure, you know, there's a backup procedure. I need to estimate this. There's, there's lots of, lots of things that they need to account for. And then that's often specific to the enterprise, to the, to the, to the, they work. So what we do is we take our, we take the reasoning capabilities of these models and

• 18:31Speaker 02

  we fine tune them so that they, they can, they can reason just like that employee of that particular enterprise. So we found that when you tune these models, especially the planning aspects of these models around the subject matter experts' practices, they become extremely, extremely capable. They, they really produce actionable, actionable and relatable results for the users. So just to point to your question about reasoning, yes, we, we rely on reasoning, but we, we try to guide, we try to guide that reasoning towards economic ways of solving that problem. That's another thing, great.

• 19:15Speaker 02

  So a lot of times when you ask complex IT questions to these models, you know, they'll, they'll come up with, you know, plausible, you know, possible plans, but then is the, is the plan actionable? Essentially, if the solution is, you know what? Tear down everything, rebuild everything, this is a rewrite situation, you know, then the customers won't be happy. You know, they'll, they'll say, that's too expensive. I can't really do that. So then we, we train our models and we reward them in ways such that more economical, more, more expedient ways of lower risk solutions are preferred over others and what else. So there's a lot goes into the guidance, guidance of that reasoning.

• 19:57Speaker 02

  But at the core, just like you said, we do rely on LLM's reasoning capabilities to figure, figure things out, to do the orchestration especially.

• 20:07Speaker 00

  Yeah, and so that whole reasoning workflow, that's, is that one module? I mean, we were talking before we started recording that you guys build your modular, so you have these microservices that work together. That presumably then you can

• 20:33Speaker 00

  configure or pick and choose among them, depending on the use case. Is all of that reasoning one module?

• 20:43Speaker 02

  Yeah, yeah. So we follow the similar philosophy of divide and conquer, you know, create good encapsulations around capabilities. So we follow an architecture pattern called mixture of experts.

• 20:57Speaker 00

  Yeah.

• 20:58Speaker 02

  What this is, is it allows you to, you know, these pre-trained models are trained on piles of piles of publicly available textual data, I mean, if you're talking about language models.

• 21:11Speaker 02

  And what we found was, actually, what we not, we didn't find this, but Meta sort of led the way into this architecture in our, in our adaption. They, they described, they had this wonderful paper, paper to talk, where they talked about, you know, how, how these models can be further tuned for different purposes while encapsulating that training in a specific set of weights and biases called experts. So what that is, is you take one of the open weight models. Open weight model means a model that you can basically download to your computer and run as much as you want,

• 21:54Speaker 02

  based on, of course, depending on the licensing agreements. So what you do is you take one of these open source models and then you identify a problem data sets and a reward function. So you essentially, you, you basically devise a clever way of signaling the network. Hey, you know, here is, if, when you, when, when you give me a good response, here, you know, I'll give you cookies. If you don't, if you, if you, if you fail, you know, I'll, you know, you'll, you'll get the stick. So you basically train, train these models on additional data sets that allow you to focus on that problem. And then you capture these as what we call experts.

• 22:36Speaker 02

  So experts are just like you're saying, they, they're mini, I mean, they're not, I wouldn't say they're services, but they're encapsulations of that training, that they are the artifact of that training cycle. And within this mixture of expert architecture, you also specify gate activation values during your training, which means the model then knows when, when it's prompted with a question, whether to activate that training set or not. Like, for instance, if you ask our model, you know, what's the, what's the weather like in France today? It will give you an answer, it will do some tool calls and figure, figure out the answer. But in that prompt, nothing will signal that, oh, you know, they're asking me a question about, you know,

• 23:21Speaker 02

  like a root cause analysis on a mainframe Z system. So, so it will detect that and it won't activate that part of its training.

• 23:30Speaker 00

  Right.

• 23:31Speaker 02

  But if, if the question is, hey, you know, I'm in big trouble, you know, how do I restart, you know, how do I reinitialize this L bar on my mainframe? Then that, that activation layer kicks in and says, ah, okay, so looks like they're talking about this other thing that I was trained on and it loads those weights and biases that you create during your training. And then those weights and biases sway the network just enough so that it, whatever it generates, plan or output summarization or what have you, is now affected by, affected by, by that training. We found, and it's not just us, but also this is throughout academia and now also in the industry,

• 24:15Speaker 02

  the best way to generalize a data set, a training data set is really

• 24:24Speaker 02

  to, to go through these trainings so that the model can, can reason about the native in, in a model, model, model native way. So that's why, you know, we are, we've been pursuing this for, for some time now.

• 24:37Speaker 00

  Yeah. And then, so once the reasoning is done and

• 24:43Speaker 00

  and the system has identified a problem and, and designed a, a plan of action, is that then passed on to an agent that executes that plan of action?

• 24:58Speaker 02

  Yeah. So we, when we first started, we just, we started with a textual recipe that we would hand off to the user and say, okay, so here's what you need to do, this is the recipe I need you to follow. And maybe there's like eight to 12 steps in there and then we called it a day. But of course, you quickly realize, well, okay, some, first of all, some of these actions are very automatable, like, once we break it down to a problem, a problem into little steps, it already, you know, automatically suggests automation. So in that regard, what we first said was, okay, so a lot of these are, first, let's verify the problem. Let's do additional diagnostics, essentially a lot of read-only operations.

• 25:41Speaker 02

  So we categorized, we, we thought, you know, we train our models so that it knows how to categorize and label these actions as, well, these are diagnostical, so you'll never break anything by just doing these. So we started automating those of, as you can imagine. Some of the actions are on the remediation side of things, which means, you know, oh, you need to go, go to this configuration and actually change that port number or, or go to this machine and allocate more file system space. So essentially, things that touches the system, we still require a human, a human in the loop

• 26:20Speaker 02

  for them to approve the plan before any of, any, any such automation, automation is actually called upon.

• 26:31Speaker 02

  So we, we, right now, essentially, we are in an assistive capacity, which means these human beings are now, you know, fed these recommendations and AI-generated plans of action. And then as they do things and as they, as they adhere to the plan or deviate from the plan, we actually track that lifecycle, Craig, so that if, if any of our, you know, recipes are executed and everything is fine, that's great, that's great feedback for us. But we actually learn from our mistakes more. Meaning if, let's say, for instance, we tell them, hey, you know, you need to do A, B, and C, but then if they do A, B, and X,

• 27:12Speaker 02

  then we actually learn that once they, that, once that ticket is closed. So that we can, even if that subject matter expert is not really donating and writing a lot of descriptions of things, we want to essentially understand their intent and the real, the real way they, they fix the issues. And we, we train our models as that fly, fly of turns, as they, you know, respond to tickets and, and, and, and leave little, little digital traces of, of their actions.

• 27:46Speaker 00

  The human in the loop, with a lot of these autonomous systems, you know, there is something called automation bias, where people become accustomed to accepting the, the machine's recommendation because it's, it's worked the last 10 times. And so the 11th time, you, you don't even think about it, you just say, go ahead. I mean, is, is there a point at which

• 28:18Speaker 00

  at, at which this will be fully automated? And, and, yeah, and the, the human users will, will sort of be monitoring out, monitoring outcomes,

• 28:34Speaker 00

  but not necessarily, you know, approving every action.

• 28:39Speaker 02

  Yeah, that's certainly where it's going. There's definitely huge economic pressure in automation, of course. One thing that you said is automation bias is very true.

• 28:54Speaker 02

  And to, to address that, we, we have built various quite interesting fingerprinting techniques of incidents and issues. Essentially, when something goes wrong, we keep a very detailed record of what the machine signals were, what the human beings have done, and locality of that issue, how that issue transpired, like, what was the first domino that toppled over, what was the second domino? So we create these causal traces of incidents and issues in the enterprise so that we can say, oh, you know, what's happening right now is something that just happened last week.

• 29:37Speaker 02

  And it seems like every Friday you guys are going through this. Hence, it's highly likely that this is, it is that issue. So it's actually to, to combat that and also give confidence people that, you know, the, this is something they can actually tackle with automation. We, we have done a lot of work on fingerprinting, fingerprinting issues.

• 30:03Speaker 02

  And that, it's actually, in IT, in this, in this sort of problem space, it's very important to know whether you're dealing with a brand new problem versus something that happens with some, some frequency. There's, there's tremendous difference, difference between them. Essentially, being able to say, you know what, I don't know, I'm throwing the towel in, you really need to pay attention to this, is, is, is a great asset. That's not a failure of the product. And when it does that, we are extremely happy that we've been able to do that. Because, you know, one thing that you've noticed is probably LLMs will never tell you, oh, you know, I don't know something, right? They, they're very positive there.

• 30:48Speaker 02

  They're they're they they they love to generate. So it's actually to get them to say something like, well, you know, this is really out of my bounds. Is is difficult, and that's something we've been working on for for ever since we we we we uh we we started. We started this, but of course, is if you if you are sufficiently.

• 31:12Speaker 02

  If you can sufficiently.

• 31:15Speaker 02

  Fingerprint these incidents and issues and can sufficiently confident to say, hey, you know, this is actually something we dealt with. And here is how we solve these problems. Then there is a lot of economy economies there. So you can imagine that the the pressure in the market is to always towards. OK, so can we fix these reoccurring problems, which tend to be like eighty eighty five percent of the problem space? Can we what can we do to automate these as quickly as possible? That's that's actually what, you know, people like Ryan and me think about all day.

• 31:45Speaker 00

  Yeah, well, one of the things that people have been doing is they build.

• 31:51Speaker 00

  This kind of a committee of. L L M's that that check each other's work. And then come up with a consensus about, you know, which what is the optimal?

• 32:08Speaker 00

  Output or play, whether that's a plan and action or or. Yeah, are you doing anything like that?

• 32:18Speaker 02

  Yeah, we absolutely do, which is interesting.

• 32:23Speaker 02

  So we so we have this new agent called Deep Root Cause Analysis Agent. So this is something that you unleash on big. Problems in a postmortem sense, so meaning problem already happened and. You're trying to understand what why why that was, and you have a lot of time. So you have a lot of research research time. It's actually it's not a it's you don't have to be, you know, counting milliseconds necessarily. So when this when this thing is launched.

• 32:54Speaker 02

  It it quickly formulates an opinion, a theory about what it is. Based on all the available data, all the snapshot data that was handed to it. But it has agency. So in fact, you know, when we say agentic AI, that what what what that really means is, hey, you know, do you have a system that can take initiative? To dig into to dig into data and come up come up with a come up with an outcome, come up with an outcome. That is towards that is in the trajectory that you think it it it should do. So for us, that's root cause analysis, meaning we task our agents to say is, hey, you know, here's all the telemetry data that we have.

• 33:35Speaker 02

  Here are all the data connectors that you might want to pull in, you know, in an agentic manner, if you want. And I need you to. Get to the bottom of this and do a deep root cause analysis on this. And when you test this agent, it it, like I said, it formulates a. A hypothesis. And then this hypothesis is then passed to other subagents in the system. We have an agent that is expert in log analysis and analysis of log streams and log log data in general. So then when this hypothesis is passed down to that agent with the with the task, because the.

• 34:16Speaker 02

  Our planner says, OK, so I suspect it's going to be the load balancer again. So, you know, somebody needs to go study the load balancer logs. And it hands it off that to that agent. And that agent says, OK, I guess this is what I do now. It goes to the it goes to the log source and then starts analyzing the data. But we give that agent enough enough creative space such that if it finds a refuting evidence or supporting or refuting evidence in the data, it can signal that back to the other agents and say, hey, you know, what you told me was.

• 34:57Speaker 02

  Was to look for this, but I didn't find that, but I I found this other thing, which suggests your root cause analysis was misinformed. Meaning I found more evidence for you, perhaps you should reconsider. So we don't have a peer to peer to peer. Agent forum like maybe you're imagining, but we have a hierarchy of. Like a principal researcher versus individual clerks that are very good at researching different types of. Machine machine machine data like logs and metrics and topologies like graphs of things. So those are individual areas of expertise.

• 35:36Speaker 02

  And they test the hypo they know how to test the hypothesis and also create new insights that the master planner. Can then reason reason about, and they constantly back and forth, you know, do this until they converge on on a on a response like everybody's happy. Or exhausted.

• 35:58Speaker 00

  The. We were talking about how you're containerized and these are built as microservices. And you have a list of of these services. From the users, the customers' point of view, do how does that work? Do they choose to use some and not others? Or is that just.

• 36:26Speaker 00

  Architected that way of for because it's easy for you to swap modules out as you improve things.

• 36:37Speaker 02

  Yeah, so when when the platform was first designed, it was just like that. It was it had that à la carte feeling of, OK, so. Here is what I need here is, let's say, I'm I run a S R E team, site reliability engineering team. So for my team, I need a workflow like this, I'm going to need this service to be modeled, and then I need this workflow to be defined. So it it it certainly was like that, so that you could pick cherry pick the services that the platform offers you. But in the agentic world, you know, we are quickly migrating, you know, everything towards agents nowadays.

• 37:17Speaker 02

  This actually is a lot smoother and transparent from the point of view of the user, because they just naturally conversate. They say, hey, you know, I'm trying to do this, I'm trying to do that. And there is sufficient intelligence baked into the platform baked into the agentic layer of the platform. That it knows how to delegate these to the all the other. Bots and models and what have you. So we have a big.

• 37:48Speaker 02

  Umbrella term, you know, we call it Helix G P T, which is essentially all the generative, all the reasoning, all the analysis tasks that the platform is generating is then routed to to either a model provider of the user or one of the models we host for the user, either on their on premises or on the or in their private clouds versus maybe a cloud offering of choice. So this is the flexibility Ryan was talking about. We actually, you know, we are able to deploy the components pretty much everywhere, including on on on premise.

• 38:27Speaker 02

  So we have a rather sophisticated routing layer that knows how to, you know, where goes, you know, what what workload goes where. And it does all the metering and all the compliance and and whatnot.

• 38:40Speaker 00

  There's something about the about what you guys have built with Helix that he was saying. Service now isn't able to do that because I that we didn't get to the why.

• 38:53Speaker 02

  I I would say we have, you know, you you can run us in any. Cloud or hybrid or on premise scenario. So from that perspective, we do have a unique unique stance, I suppose. Meaning a lot of organizations are pretty nervous about, you know, sending their most intricate I P and and data to third party A I vendors.

• 39:23Speaker 02

  Because they see how capable this plan and models are, they can, you know, quick to learn things. And so so there is a lot of interest in being able to run these this this level of intelligence in an air gap manner, if you will. So that so that nothing leaves the the that organization's boundaries. So for that reason, we have, you know, we built a lot of scaffolding around around these models. So that A they they run very efficiently, you know, we don't. We actually, we, we look for parameter efficient models as much as possible, meaning

• 40:02Speaker 02

  I need to be able to run all this intelligence on ha- highly available, not so cutting-edge hardware. And that does require some work on, like I said, being able to,

• 40:18Speaker 02

  you know, the, to train the model so that you don't essentially activate necessarily the entirety of that model for every single token. Also being able to do, um, um, uh, su- support uh, multiple tenants perhaps on one installation, one, one single, single tenant. Um, so we've done a lot of work to, uh, to make uh, generative AI not just possible and exciting, but also economic to run. Um, so from that perspective, I think we do have, uh, like I said, I don't do a lot of competitive analysis as part of my job. Um, but from that perspective, we know, I know, um, we have pretty unique, um, features, um, and advantages.

• 41:02Speaker 00

  Yeah. And, and if somebody wants to, to run this on-premise or a part, if it's a hybrid solution, uh, you know, uh, on-premise and in the cloud, uh, how do you deliver the model or the modules to them? Is that just they download it and install it on their side?

• 41:24Speaker 02

  Pretty much, yeah. Just like, uh, so, uh, so you, uh, you know, we talked about containers and containerization. Uh, so this, this happens to be, um, yet another set of containers, uh, you, you deploy as part of the, uh, the product. Uh, the only, uh, perhaps maybe slight difference is uh, the, the machine that, that's going to run this container will now need access to a, a decent GPU. Um, and, uh, we support uh, pretty old hardware, all the way from, uh, Nvidia's, uh, L-Force uh, to A, A one hundreds. And nowadays our TX six thousands are, are pretty good.

• 42:08Speaker 02

  So we, um, so we just require them to, uh, procure or lease or, you know, find, find a GPU. Uh, and then, um, we give them the software architect that basically is part of our platform really. Uh, and that thing, uh, turns that, uh, turns our model into an inference server that all the other things in the, in the system can then, uh, start utilizing. Um, the, uh, and also the training, we do, we have, we have this optional uh, tenant-specific training pipeline also, uh, which, what, what that is, is, you know, they could just go to our administrative UI and tell us where their sources are, where their, uh, ticket sources are, or their knowledge articles, rombooks and what have you.

• 42:52Speaker 02

  And what we do is, uh, we harvest all that and then we analyze the data to see, um, what can be learned from them. Um, so that's also pretty, uh, I would say interesting, uh, because that allows us to, uh, uh, to keep up with the organization because no IT organization is constant, right? So they deploy something. Maybe their network is all Cisco hardware, uh, but all of a sudden somebody introduces new Juniper hardware somewhere. So, so they, they shift, right? And as they adopt new technologies and new, um, you know, new components, all the failure modes change. Um, they, they, they bring their own problems. They're bringing their own comparability issues and whatnot. So there's always movement in, uh, that we have to deal with.

• 43:36Speaker 02

  Uh, so for that reason, we tap into these, uh, sources and periodically pull the data out to see, um, what needs to be, uh, generalized and what's, what's learnable from them. Um, so anyway, so that's also another, uh, component that goes with this, um, uh, generative, uh, AI, uh, inference server. Uh, and once you, once you do that, you have, uh, you, uh, you periodically, let's, let's say every 24 hours, your model revs up. So you're, um, it, it, it might decide to learn new things and, uh, and update, update itself. Uh, that way you're, uh, it's, it's never behind. Um, so yeah.

• 44:14Speaker 00

  Yeah. Um, the, um, do you have, uh, metrics of, uh, you know, that, that demonstrate how, um, the, the, this Helix performs, uh, in a large organization? I mean, these are big decisions by, uh, by organizations. I imagine that much of your customer base has been with you for a very long time. But to get somebody, uh, Fortune 2000 to switch, I mean, they've already got a solution. Um, to switch to, uh, Helix, uh, is, um, is, I would imagine a long sales cycle.

• 44:59Speaker 00

  So, uh, do you, what, what do you present to them as, uh, evidence that, that you guys save them time and money?

• 45:09Speaker 02

  Yeah, yeah, yeah. So, uh, uh, yes, we always, uh, um, you know, we, we have to, uh, refer to, um, uh, some, some measured numbers, some customers. But what happens, Craig, is at the end of the day, they, it always comes down to a bake-off of sorts. So they, um, uh, just like you said, these, these organizations, um, don't buy these, uh, tools just to use for a year, right? They typically plan for three to five years out. Uh, and it's often the, uh, case that they ask us, okay, so, uh, you know, what can you do now? But we also need to understand where you are going, you know, to see if, if our goals align. Uh, so just like you said, these are actually very, uh, complex, long, uh, sales, sales cycles.

• 45:54Speaker 02

  What we, uh, what we tell them is, uh, so we do have a portfolio of, um, uh, uh, savings that measured, measured savings that we share with them. So this typically, based on the use case, this typically anywhere from, uh, 25 to 40, sometimes 50%. So, you know, we, we, we, we give them these numbers. Uh, but of course, at the end of, and we, we have a lot of, uh, benchmarking information, uh, both, uh, synthetic and open source, uh, and third party, you know, we also, uh, we have done, uh, some, uh, we have competed in some competitions to, to show people, you know, what, what this, what these models can, can, can do, uh, in isolation.

• 46:37Speaker 02

  Uh, so we have a lot of, uh, proof point, uh, proof data, but at the end of the day, um, like I said, it, it's often what happens is you get deployed in a, um, pre-production environment or some, some QA environment that they're, they're choosing, sometimes together with your competition with, with, with other tools. And then they just measure. They just say, okay, so, uh, uh, you know, uh, how, uh, given an operator and this tool, uh, and the day maybe, you know, they sometimes, uh, you know, they inject a problem and then they see, okay, did you find it? Did the operator, uh, did the operator, uh, gain any insights from you or, or so on and so forth?

• 47:14Speaker 02

  So it's actually, uh, uh, this, this sort of, uh, uh, software, I've never seen, you know, be, is being sold unless you, you actually do a proof of concept. Uh, so that you, you call, you collect some, uh, you collect some data yourself.

• 47:28Speaker 00

  And, and on where you're going, I mean, what do you, what, what is on the roadmap?

• 47:34Speaker 02

  Um, so, uh, you, I mean, uh, you, you see how software engineering got disrupted, right? Software engineering, uh, I would say within the last six to eight months,

• 47:48Speaker 02

  is now different, fundamentally different, right? We had tectonic shifts in software engineering. Uh, and, uh, we, we believe a similar disruption will also happen in operations, in ops, uh, ops. Um, and the reason is, you know, software engineering is a very verifiable problem, right? So you can generate a piece of code and then compile that code and say, okay, does it compile or compiles? Does it run? You can run some tests against it to see. So it's like, uh, oh, you can imagine you can create uh, training flywheels for software pretty easily. And that, that's why that was the first thing that got heavily, heavily automated now.

• 48:29Speaker 02

  Uh, we believe ops are also close to this. I mean, it's the, uh, verify, verification cycle of software is maybe measured in minutes. For ops, that's typically hours. Um, and may, maybe per- perhaps days, but at the end of the day, it is still a verifiable problem because these are digital systems in, at the end of the day, and IT system is, uh, it might look complex. I mean, it's a complex system, but it's not a chaotic system, right? So at the end of the day, these are still computers running software and. You know, it is a deterministic, it's a deterministic system.

• 49:09Speaker 02

  But the question is, can you verify this in quickly enough to be able to train your model? And that's exactly what we are doing right now, Craig. So right now, most of our research effort is going through building gyms or labs, if you will, where the agents can go and get trained on.

• 49:31Speaker 02

  So large language models were a function of text data that was available on the internet or offline, right? So they're, they gone through piles of piles of text data. But they are probably at the limit of that data. So nowadays, in the future, what's gonna happen is people will start exposing these models to the real world more and more so that they can create, they can gain that first person, first person view of things. Because right now, AI, if you think about AI, is only learning through someone else's description, right? Text means, you know, someone wrote something and that's their interpretation of reality, right?

• 50:10Speaker 02

  So, so now everybody's trying to cut that off and expose the AI directly to the reality of whatever domain they are in, so that they can, you know, they can do things and learn from that. And in our space, that means creation of these, what we call gyms. So gyms are essentially, imagine a data center that just runs whole bunch of different applications, but maybe nothing is sensitive or created for, for the, for the purpose of training in the sense that I can let my AI go there and break things. I can say, okay, so create like a chaos monkey agent that just goes and randomly breaks things and say, ah, you know,

• 50:51Speaker 02

  I took, I, I, I messed up your application, which then my other AI comes in and says, okay, so let me first verify what's going on, try to fix this, and it tries different things and I log all of this. I, I keep a record of how, how things are progressing and whether the remediation works. Is it expensive? Is there a better one? So I can do all these in a 24/7 lights out manner in a data center so that I can create a bespoke agent that knows all the intricacies of whatever application it's responsible for.

• 51:28Speaker 02

  So that's, I think that's where, that's where, obviously, that's where we are trying to go. I think that's where the industry is also going towards. So you'll see more and more, you'll hear about these, you know, world models and whatnot. That's also sort of philosophically aligned with what I, what I just said.

• 51:44Speaker 02

  But what I can, I mean, what I can confidently tell you is these things are a function of their training data. And good training data, we exhaust textual space. So most of the best data will probably come from environments that are created so that these, these AI agents can just go and experience themselves and learn from the, learn from those mistakes.

• 52:12Speaker 00

  Yeah, yeah, and that's fascinating. I mean, I've talked to a, a number of people about world models and, and direct

• 52:23Speaker 00

  learning from reality as opposed from text. So it's exciting to hear that that research is going on. Are, are you developing your own models in, in, for that? Or, or do you use, you know, models from one of the foundation labs and then find?

• 52:50Speaker 02

  Yes, so we have always used a foundational model, both in our research and in our shipping functions. And because they are just marvelous and good enough on their own. They, and also, you know, training a foundational model is pretty, pretty expensive and slow operation. So we always started from foundation models, but we have a, a pretty advanced workflow now, I would say, that, that allows us to try out new ones all the time. So I, I have a radar that sort of, you know, we constantly try, you know, different, you know, new,

• 53:34Speaker 02

  new foundational models from different vendors, benchmark them, train them, and, and to evaluate them. These days, the, the, we, we are really fond of the Quinn family from Alibaba.

• 53:51Speaker 02

  They're, they, they do great, great frontier work.

• 53:56Speaker 02

  And the, also Gamma 4 from Google is, is other, so we have two editions of the same model. So we have a, a Quinn based model and then a Google, Google based one. And we offer them both. And, and the foundation model is very, the way our, we are architect is swappable. So, so as these things, you know, develop and advance, we are not really married to, to them very, very tightly. So we can easily swap them out in and out.

• 54:34Speaker 02

  It's a, it's an, so it's interesting, Craig, that it's, it's a very empirical space. So you can't just look at the architecture, you can't read the paper and say, oh, that's a better model. You really, at the end of the day, you have to sit down and, and test these, try these. There's all kinds of reward hacking models due, you know, when you're trying to test. I mean, some, sometimes they detect that they're, you know, oh, it looks like I'm being tested right now. So it's, it's, it's, it takes a different rigor to formulate an opinion about a foundational model. But that's, we are all learning that, you know. I think everybody in the industry is learning some techniques around being able to evaluate this and essentially right-size them.

• 55:18Speaker 02

  Because you'll, you'll see that one model comes in whole bunch of different sizes and resolutions and quantizations and what have you.

• 55:27Speaker 02

  So, so, you know, we did have to tool ourselves and so that we can make these judgments, you know, data-driven, data-driven way. But like I said, we, we, we offer two foundational models today.

• 55:40Speaker 00

  So this is the future of IT service management? And, and it sounds like it's gonna get increasingly automated. Is what's that gonna do to, to IT staff? Does it make them change what they need, the skills that they need? Or is it, is, are they gonna become agentic managers? Or managers of agent systems as, as opposed to having their fingers in the code?

• 56:19Speaker 02

  So the good news is, I think what's gonna happen is the quality will go up. Because what we, what these systems, these agents really tackle, target is repetitious things that shouldn't, shouldn't have happened for the first, first place. You know, there's a lot of, you know, we expend a lot of calories just to fix the same problem again and again, or things that could have been prevented to begin with. And when we, when we work with our customers or our own IT personnel, you know, who are, you can imagine they're quickly adapting all these technologies also. They actually, so right now, my reading is they're quite happy to, to have all this assistance.

• 57:03Speaker 02

  They say, well, you know, this saves me a lot of time. I can go, you know, go home at a predictive, predictable time now. I'm not swamped of all this garbage firefight. I can now concentrate on actually making the organization better or making my such and such projects better. So, so, so this, you know, this sort of technology buys you additional digital capacity, as you know, Ryan calls it, to then invest in other, other new things. It's actually, I think the smart, growing organizations will take this digital capacity and then redeploy it in solving the customers or end users' problems better

• 57:46Speaker 02

  or offer them, you know, more services and more, more, more digital goods.

• 57:54Speaker 02

  And what, what does this, what means for this individual is people who have agency is now able to move a lot faster. So we see this, you know, the, a certain cohort in every organization is sort of really shining, you know, with, with the help of, with the help of these technologies. So I think some people will move up in the chain, some people will be disillusioned and will say, oh, you know what, I'm done, I'm retiring. It's certainly a big economic labor, labor shift, you know, it's happening in front of our eyes.

• 58:34Speaker 02

  But my, like I said, I don't, my, my personal opinion of this is, this is great for the society. I think everything will get higher quality and will become cheaper because of this. And for the people who are involved, I think they'll, they'll just, they'll just tackle higher order, higher value adding, adding problems as these technologies get bigger and become more ubiquitous, as they will.

• 59:02Speaker 00

  Yeah, okay.