Orchestrating agents, APIs, and MCP servers

In this episode, Daniel sits down with Pavel Veller, EPAM’s Chief Technologist, to explore the practical challenges of orchestrating many AI agents and managing connections to disparate systems/tools. Pavel shares insights from his hands-on work with agentic architectures and internal tools like "DIAL". Pavel also helps us understand things like MCP servers and why connecting assistants via APIs is easy—but making them useful is hard. Featuring: Pavel Veller – LinkedIn https://www.linkedin.com/in/pavelveller/, X https://x.com/pvellerDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaLinks: EPAM https://www.epam.com/DIAL https://epam-rail.com/platformSWE-bench results https://www.swebench.com/#verified

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:03Speaker 00

  Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, and accessible to all. If you like this show, you will love The Change Log. It's news on Mondays, deep, technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts. Thanks to our partners at fly.io. Launch your AI apps in five minutes or less. Learn how at fly.io.

• 0:44Daniel Whitenack?

  Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I'm CEO of Prediction Guard, and I'm really excited today to dig a little bit more into Gen AI orchestration, agents, coding assistance, all of those things with with my guest Pavel Veller, who is Chief Technologist at EPAM Systems. Welcome Pavel. Great to have you here.

• 1:10Speaker 02

  Thank you. Hello, hello.

• 1:12Daniel Whitenack?

  Yeah, yeah. Well, I mean, there's a lot of topics. Even before we kicked off the show, we were chatting in the background about some really interesting things. I'm wondering if you could just kind of level set us of, people may or may not have heard of EPAM. I think one of the things that I saw that you all were working on was this Gen AI orchestration platform dial. Maybe before we get into some of the specifics about that and other things that you're interested in, maybe just give us a background of what EPAM is. I know you mentioned even in our discussions that some of what you're doing right now maybe wouldn't have even been possible a couple of years ago. And so things are developing rapidly. Just level

• 1:56Daniel Whitenack?

  set the kind of background of this area where you're working.

• 2:00Speaker 02

  Sure. Yeah. So EPAM is a professional services organization. We're global. We're in 50 something countries, 50,000 people globally work with clients. We have been for, I think, 32 years to date.

• 2:16Speaker 02

  So we do a lot of different things, as you can imagine. And what I was mentioning about doing things that would not be possible is doing things with Gen AI today. We do a lot of work for our own clients. We also do work for ourselves, applying the same technology. Because EPAM historically as a company has been running on software that we ourselves built. The philosophy has always been that things that do not differentiate you, like an accounting software or like CRM, you would go and buy off the shelf. Things that differentiate you, how we actually work, how we operate, how we execute projects, how we hire people, how we create teams, how we deploy teams.

• 3:00Speaker 02

  All of that software has always been our own since as early as late 90s. And we keep iterating on that software for ourselves. And that software today is very much AI first. And a lot of things we do, we do with AI and really do because AI in its current form exists.

• 3:17Daniel Whitenack?

  Interesting, yeah. And how, how I guess does, you know, I think when we initially kind of was, were prompted to, to reach out to you, part of it was around this, this orchestration platform. So talk a little bit, maybe generally, not necessarily about the platform per se, although we'll get into that. But just Gen AI orchestration generally. So you talked about, you know, some of these things that are becoming possible. Where does orchestration fit in that? And what, what do you mean by orchestration?

• 3:49Speaker 02

  You probably think of Dial. You can Google it. We do a lot of applied innovation in general as a company. This is one of the good examples of applied innovation to AI. The best way to think of Dial would be, you guys all know ChatGPT, right? ChatGPT isn't an LLM. It's an application that connects to an LLM and gives you certain functionalities. It can be as simple as just chatting and asking questions. It can be a little more complex, uploading documents and speaking to them, like talk to my documents. It can be even more complex when you start connecting your own tools to it, right? We see our clients.

• 4:31Speaker 02

  Not only do this, but also want something like this for their own business processes. And this orchestration engine becomes, how do I make it so that I don't have twenty different teams doing the same similar things over and over again in their own silos? How do I connect my teams and their AIs and their, you know, thoughts and results into a consolidated ecosystem? And it's likely because of Gen AI and because of what we can do with conversation and text becomes sort of conversation first. You can think of conversation first application mashups almost, right? Like you talk, express a problem.

• 5:12Speaker 02

  What comes back is not just the answer, maybe what comes back is UI elements. Buttons you can click, forms you can fill out, things you can do, as well as things that are done for you by agents automatically. So Dial, in that sense, is, well, by the way, it is open source. You guys can also go look, you know, download and play with it. But it, it is a, this like ChatGPT-like conversational application that has many capabilities that go beyond. We have Dial apps. They predate MCP, but the idea is that you, so Dial itself has a contract, an API that you implement. You basically come back with a streaming API that can receive a user prompt.

• 5:57Speaker 02

  And whatever you do, you do, and you come back to Dial with not just text. It's a much, much more powerful payload with UI elements, interactive elements, and things that Dial will display for me, the user, to continue my interaction. And Dial becomes this sort of center mass of how your company can build, implement, integrate AI into this single point of entry. And then Dial goes, well, from day one, Dial was a load-balancing model-agnostic proxy, right? So every model, every deployment has limits, you know, tokens per minute, tokens per day, whatever, request per minute. You'll likely, if you're a large organization with large, different workflows,

• 6:41Speaker 02

  your AI appetite will go well beyond a single model deployment. You'd like to load balance across multiple, and then you'd like to try different models. Ideally with the same API for you, the consumer. So Dial started like that, started like load balancing, model agnostic proxy, single point of entry. We can log everything that is prompted in the organization. We can do analysis on that separately because that's very helpful to know what kind of problems your teams are trying to solve. And then it evolved into this application hosting ecosystem. Now it's evolving clearly towards what MCP can bring in because now you can connect a

• 7:20Speaker 02

  lot more things to it through MCP. So I think it's frightening it like 20 something clients by now.

• 7:28Daniel Whitenack?

  So just a couple of follow up questions. It's been in the news a lot, but just, just so people understand if that maybe they haven't seen it. What are you referring to with MCP and kind of how that relates to some of this API interface that you're enabling?

• 7:45Speaker 02

  Well, the easiest is to Google it. You could Google it. You're gonna find it. It's on Claude. I, let me tell you how I think about this because not what it actually like.

• 7:54Daniel Whitenack?

  That's helpful. Yeah.

• 7:54Speaker 02

  Yeah, I think what in a very simple term. So MCP allows to connect the existing software world to LLMs. In a way, think like, and I'm gonna, I don't want to hype it too much because it's not yet a global standard or anything. It's very early, early, early days. It's been months, right? But what, let's say HTML and browsers and HTTP, they enabled to connect, well, us, people to software all over the world. MCP does that, but for LLMs. So today, if I want, if I today want to be able to prompt my application that is in front of an LLM to do

• 8:37Speaker 02

  things with additional tools. Let's say I wanted to be able to search file system based on what I prompted and find a file and something in that file, right? So my application needs to be able to do that. My option is what? I can write that function. I can then tell my L M. Hey, here's this function you can call if you want to. Call it. I'm going to call it for you. Great, that's one function. What if I need to do something else? I want to go talk to my C R M system and get something out of there. I'm going to write that function. If I'm going to write all the functions I can think of, it's going to take me years, probably hundreds of years. So instead, what I can do today, I can say, hey, my L M application, can you talk a protocol? Because there's a protocol called M C P.

• 9:21Speaker 02

  I'm going to bring you M C P servers that other people have built for my C R M system, for my file system, for my C L I. There are M C P servers for everything. IntelliJ exposes itself as an M C P server to do things that it I D E can do. Now you can orchestrate those things through L M. So you connect all these M C P servers through an M C P client, this application in front of L M, to L M, expose the tools to L M. L M can now ask the client to call a tool. And through this M C P protocol, the client calls the server. The server does the function that has been written in that server. And boom, L M gets results. So it's this connective tissue that did not exist three months ago. Three months ago, everybody was writing their own.

• 10:05Speaker 02

  And right now, everybody, as far as I can tell, writing M C P servers. And those who talk to L M's, they consume M C P servers.

• 10:13Daniel Whitenack?

  Yeah. And maybe just give even. So I like the example that you gave of sort of searching file systems. What are, just to kind of expand people's understanding of some of the possibilities, what are some of the things that you've seen maybe implemented in in dial as things that are being orchestrated, you know, in in general terms? What are kind of some some of these things?

• 10:36Speaker 02

  Let me give you a higher level and much more sort of fruitful example.

• 10:43Daniel Whitenack?

  Okay?

• 10:44Speaker 02

  We have our own agentic developer. It's called AI slash run Code Me. Cause AI slash run has multiple different agentic systems. Code Me is specifically coding oriented. We have others oriented at other parts of SDLC workflow. By the way, you guys can go to SWE bench and look at verified list. I believe Code Me as of now is takes fifth. It's number five on the list of all the agents to compete for solving open source defects and stuff. So Code Me as an agentic system has many different assistants in it.

• 11:23Speaker 02

  Dial as a generic front door as a chat GPT would like to be able to run those assistants for you as you talk to Dial. And until MCP, it really couldn't other than, hey, Code Me, implement an API for all of your assistants. Let me learn to call all of your APIs. Now the story is, hey, Code Me, give me an MCP server for you, which what they have done. Dial as an MCP client can now connect to all Code Me features, all the assistants, expose them as tools to an LLM and orchestrate them for me.

• 12:05Speaker 02

  So I come into the chat, I ask for something that something includes reading a code base and making architecture sketches or proposals or evaluation, right? And LLM will ask Code Me assistants to go and read that code base because that there is a feature in Code Me that does it. And Dial needs to only orchestrate, but doesn't need to rebuild. Or build from scratch. That's the idea. So this is the example.

• 12:34Daniel Whitenack?

  Yeah. Could you talk a little bit? I'm asking selfish questions because sometimes I get these asked of me. And I'm always curious how people, how people answer this. So one of the questions that I get asked a lot in respect to this topic is, okay, I have, you know, tool or function or assistant one. And then I have assistant two, and then I kind of have a few, right? And it's fairly easy to route between them because they're very distinct, right? But now, if you imagine, okay, well, now I could call one of a thousand assistants or functions or something, or, or, you know, later on, 10,000, right? How does the sort of scaling and routing kind of actually?

• 13:19Daniel Whitenack?

  How is that affected as you kind of expand the, or the, the space of things that you can do?

• 13:25Speaker 02

  So that, I think, and again, I can't know, and I don't know, but I think that is still the secret sauce. In a way, that is still why there is all of, all of this coding agents in SWE bench. All of them work with, let's say Claude Sonnet 3.5 or Claude Sonnet 3.7 or GPT-4O. LLM is the same. And yet, results are clearly different. Some score 10 points higher than the other. You go to cursor, ID cursor, you ask it something, it does something. You switch the mode to Max. They've been introduced very recently. Cursor on, on Sonnet 3.7 and now on Gemini 2.0, I think.

• 14:09Speaker 02

  They have a Max mode, which is per use versus their normal monthly plans. Because Max will do moiterations, will spend more tokens, will be more expensive, will likely run through more complex orchestrations of prompts and tools and whatnot to give you better results. So how you build the pyramid of choices for your LLM? How you, because yeah, you will not ask LLM, you will not give it a thousand tools. If you as a human look a thousand options and you lose yourself, you know, 100 options in it. I, again, I don't know. I expect LLM to have the same sort of oops, overwhelmed effect. You don't want to give it a thousand tools. You want to give it groups.

• 14:53Speaker 02

  You want to say, hey, you know, pick a group and then within that. So you want to do this basically like a pyramid, like a tree. But how you build it and how you prompt it and how you, how you do this? Now that's still on you. This is the application that connects the MCPs, the tools that it itself has, the prompt that the user has given, the system instructions, and building the, some of the chain of thought LLM can build. And this is going to be a very interesting balance. What do you ask LLM to build? How much of this sequencing of steps will be on you in your hands versus how much you're going to delegate to a L M and ask a L M to come up with a sequence of steps?

• 15:36Speaker 02

  And from what I've seen over the last year, you're better off delegating more total limbs because they get better at it. So the more you control the sequence yourself, the less, the more sort of inflexible it becomes, you're better off delegating to a L M, but you don't expect it to just figure out from one prompt. Daniel, I can give you that example that I gave in the beginning, if you want, about the failure. Yeah, go for it. So I use A I, so I built with A I, right? But I also use A I as a developer. So I'm on cursor as my primary I D these days. I use the A I slash run code me that I mentioned. I play around with other things like as they come up, like Claude Claude and things.

• 16:21Speaker 02

  But I also record what I do. Little snippets, 50 minutes videos for my engineering audience at E P A M for the guys to just look what it is that I'm doing, learn from how I do it, try to think the same way, try to replicate, get on board with using A I. So I started out to do a task. I wanted to record, I wanted to on record, get a productivity increase with a timer. My plan was I'm going to estimate how long it will take me announce, let's say two hours, do it with an agent. And I always pause my video when the agent is thinking because that's a boring step. But the timer is going to get ticking. And at the end, I'm going to arrive at, let's say, an hour, maybe 40 minutes out of two, boom, that's the productivity gains.

• 17:05Speaker 02

  And 30 minutes in, I completely failed. I had to scrap everything that D L M and agents wrote for me and start from scratch. And my problem was I overprompted. I thought I knew what I wanted agent to do. There were like, there was three steps, like copy this, write this, refactor this, and you're done. And it did it. It iterated for 10 minutes. It was the Code Me agent developer that we have. When I scrapped it and started doing it myself, I did half of it, stopped and realized that the other half was not needed. It was stupid of me to ask. So the correct approach would have been to iterate,

• 17:46Speaker 02

  do the first half, stop, rethink, and then decide what to do next. But the agent was given the instruction to go all the way. So it went all the way. And this is the other thing with Thousand Instructions, right? You don't want an agent to be asked to do something that you think you know, but you only really will know as you iterate through.

• 18:12Daniel Whitenack?

  In these cases as well, so like I find your experience with the, you know, balancing how you how you prompt it, you know, how far the agent goes. All of this is intuition that that you're kind of learning. One of the things that was interesting, we just had Kyle, the CEO of GitHub on, and we were talking about agents and coding assistants. One of his thoughts was also around the orchestration after you have generated some code, right? It's one thing to create a project, create something new, but you know, most of software development kind of happens past that point, right? And I'm curious, as someone who is really trialing these tools day in and day

• 18:56Daniel Whitenack?

  out, kind of as your daily driver and utilizing these things, I think that's on people's mind is, oh, cool. Like I can go into this, you know, tool, generate, you know, a new project that maybe whatever it is, you know, you always see the demo of creating a new video game or whatever the thing is, right? But ultimately, like, I have a codebase that is very massive, right? I'm maintaining it over time. I'm, uh, you know, most of the work is more on that operational side. So in your experience with this set of tooling, what has been your learning? You know, any insights there? Any, any thoughts on kind of where that side of things is heading?

• 19:38Daniel Whitenack?

  Especially for, you know, you're dealing with, I'm sure, real-world use cases with your customers who have large codebases, right? So.

• 19:47Speaker 02

  Well, that's great that I'm, I'm so glad that you asked. Cause what I do is actually that latter aspect. I have a mono repo of like 20 different things in it that could have been separate repos of their own. So I have a large codebase that I work with. And I actually saw our own developer agent occasionally choke because it attempts to read too much. And it just chokes on like tokens and limits and things that it can do per minute or per hour or something. So that's one thing. But what I find myself doing with Cursor, for example, I actually pinpointed very actively, very often, because I wanted to work with these files when it's something specific.

• 20:28Speaker 02

  I'll just point the files at it and I'm going to ask, I'm going to prompt it in context of this three or four files and that limits how much it's going to go out. But really, back to your question, to me, it's not about codebases that much. I don't think it's going to be, well, maybe if I do something greenfield and funny, it's going to write it, I'm going to run it. And if it works, it's all I need. Like it's correct, it works, great. Today, and it's still a mental shift, it's still early, I'm still looking and thinking of the codebase that I write with my agents as codebase that will be supported by other people, likely with agents, but people still. So correct by itself is not good enough.

• 21:11Speaker 02

  I want it to be aesthetically the same. I want it to follow the same patterns. I want it to make sense for my other developers who will come in after me. I want it to be as if it's the code that I have written, or at least more or less that I have written. And that slows me down a little bit, clearly, I'm sure. But the other thing is, I am the bottleneck. An agent will take minutes, small digit, like, single-digit minutes, if not less, to spit out whatever it spits out. And oftentimes, in code bases, it's not a single file. It's edits in multiple places. Then I have to come in and read it. Here's the difference. When I write myself, my brain has a timeline.

• 21:55Speaker 02

  I was thinking, as I was typing, as I was thinking, as I was... I know how I arrived at what I have arrived at. I may decide that it's bull... You know, scrap, let's start over. That happens, we're all developers. But I know how I arrived at where I am. When I look at what agents produced for me, I have no idea how it arrived at where I am. I need to reverse engineer, like, why? What did it do? It takes time. I tried recording it, and I can't, because I can't speak as I think at the same time. Yeah. This is the bottleneck, literally. So this is the bottleneck. The other thing is, when I was doing that video with a timer, I sort of, I expected certain outcomes, but I also knew that if it works, I'm gonna say this at the end, I'm gonna say, guys, look,

• 22:41Speaker 02

  it took me 20 minutes, let's say 30 minutes out of an hour. So it's 2X, right? Literally 2X productivity improvement. Amazing, isn't it? But here's the thing. Within the 30 minutes that I've spent, the percentage of time I spent critically thinking was much higher than normal. Percentage of time I spent doing boilerplate is much lower, because the agents did this. I really critically thought about what to ask, how to prompt, and then analyzing what it did, thinking what to do next. Do I edit? Do I re-prompt? Can I sustain the same higher percentage of critical thinking for the full day to get 2X in a day? Probably I can't. So what's probably gonna happen, I'm gonna get 2X,

• 23:26Speaker 02

  but I'm gonna use the time in between as agent work to do something else. My day will likely get broken down into more smaller sections. My overall daily productivity is likely to increase. I'm likely to do more things in parallel. Maybe I'll do some research, maybe I'll answer more emails, right? But it's gonna be more chaotic. Also likely more taxing. I don't think we've learned yet. I don't think we've had enough experience yet. I don't think many people talk about this yet. People talk about this, oh my God, look what I've built with agent.

• 23:59Speaker 02

  I will, I wonder how they're going to talk about how they've worked for like six months with agents. And how six months that they've done with agents is better than six months without. And how they feel at the end of the day. And think, think about in the zone, right? Well, I hope like as engineers like to be like, you know, disconnect all emails, whatever. Get the music on, IDE in front of you, you, you're in it for like two hours. With agents, you just can't. You prompt an agent, it goes off doing something. What do you do?

• 24:33Daniel Whitenack?

  Yeah.

• 24:34Speaker 02

  Do you like pull up your phone? And then your productivity increases one way, your screen time increases the other way. It's not a good idea. What can you do? Like, what do you do in this minute and a half or three? And you don't know how long, right? You, well, you can see the outcomes coming up, but the agent is still spinning, it's still spinning like. So I'm sorry, it's a long answer to a question.

• 24:54Daniel Whitenack?

  Yeah.

• 24:55Speaker 02

  But that's what I'm thinking about, like constantly. And that's what I don't yet have answers for.

• 25:01Daniel Whitenack?

  Yeah.

• 25:01Speaker 02

  But I really hope to eventually through experiments and recording and thinking arrive at at least what it means for me. Cause I cannot even tell you what it means for me yet.

• 25:10Daniel Whitenack?

  Yeah. Yeah, I mean, um, I, I experienced this yesterday too, because I'm, I'm preparing various things for, for investors. You know, do updating some competitive analysis and that sort of thing. And, you know, I, I just, when you have whatever it is, I think it was 116 companies. And I like, oh, I'm going to update all of these things for all of these companies. Like, um, you know, obviously I'm going to use an AI agent to do this. This is not something I want to do manually, uh, is put in all of these things and search websites. So, so I did that, but to your point was like, I could, I could figure out how to do a piece of that and get it running.

• 25:54Daniel Whitenack?

  And then I see it running and I, you know, I realized that this will take however long it is, right? 10 minutes or whatever the, the timeframe is. And then you context switch out of that. To something else, which for me, I think was email or whatever. I'm like, oh, this is gonna run. I'm gonna go answer some emails or something like that, which in one way was productive. But then I had to context switch back.

• 26:19Speaker 02

  Yeah.

• 26:19Daniel Whitenack?

  Like, oh, why did I output all these things? Or, you know, it happened to be that I wasn't watching the output, right? And in one case, when I ran it, I was like, oh, well, I really should have had this. It output this column or this field, but I didn't think of that before. And I wasn't looking because I turned away from the agent back to my email, right? So, yeah, I think this is a really interesting set of problems that is more of like a new. Yeah, it's a new way of working that hasn't been parsed out yet.

• 26:52Speaker 02

  And I tried not to do it, like I tried, but then you sit idle. Like you literally sit idle. It's like, and it doesn't feel good. It feels, it feels like, oh my God, why am I not doing anything?

• 27:01Daniel Whitenack?

  Yeah, it's, it's an interesting dynamic. That's, that's for sure. And I've definitely seen people that show, you know, having multiple agents working on different projects at the same time. And that, when I see someone with two screens and things like popping up all the place, I, you know, there's no way I could in my brain sort of monitor all of that that's going on, right?

• 27:25Speaker 02

  It must be very taxing first. And second, half of those merge requests, pull requests from the agents will be, let's say, subpar.

• 27:34Daniel Whitenack?

  Yeah.

• 27:35Speaker 02

  Frustration and you will rise too. Like you would think, man, I would have done it already myself much better. Like, what is this? Like emotionally, it is a very different way of working emotionally.

• 27:47Daniel Whitenack?

  Yes.

• 27:48Speaker 02

  And I really, I can't, well, I keep thinking, I can't forget. I advise people also to think, not just think about productivity gains. I just think about delegating to agents and enjoying the results. Think about how it changes the dynamic of your day and how you think about it afterwards. Right?

• 28:09Daniel Whitenack?

  Yeah, yeah, that's interesting. So I know we're circling kind of way back. That is an interesting discussion, but I do want to make sure people can kind of find some of what you're doing with Dial. You mentioned kind of the open source piece of this. What's sort of needed from the user perspective to kind of spin this up and start testing it? And any for those of those that are out there that are interested in like trying things, some things with the project? What would you kind of tell them as a as a starting point? And like what the process is like to kind of get a system like this up and running?

• 28:45Speaker 02

  I actually not sure I can tell for Dial specifically. Nobody is running local Dials. It's not something you run locally.

• 28:56Daniel Whitenack?

  Gotcha.

• 28:57Speaker 02

  It's something that you run sort of centrally in organization of size can be different, but you expose it to your people through like a URL that they all can go to and use like sort of use AI through Dial and do things through Dial.

• 29:15Daniel Whitenack?

  Interesting.

• 29:15Speaker 02

  One of the apps we built as an example earlier, it was last year, was like Talk to a Data. But if you look at analytics like Snowflakes over the world, they all have something like this today, like semantic layer, which you work on, and then through semantic layer, through prompting and through some query conversions and connectors to data warehouses and data lakes, you get yourself a chat with your data, like analytical reports, graphs, tables. So we built that that was built into Dial. So you go to Dial and then again, imagine ChatGPT, imagine ChatGPT that allows you to choose what model you talk to. Right? Not just OpenAI models, but all the all the other models that exist as well as applications.

• 30:01Speaker 02

  So go to this ChatGPT, which is now case Dial. You select this Data Heart AI, we call it, which is our talk to data. And you start talking to it. And this is still your Dial experience, but you're really talking to an app that then talks to semantic layer. Then it's, you know, builds queries based on your questions, runs them, gets data back, visualizes it in Dial, because Dial has all these visualizations capabilities to explain how it's not just text coming back. Build your charts and you can interact with it. So, but again, you don't run dial locally.

• 30:33Daniel Whitenack?

  Yeah.

• 30:34Speaker 02

  If, if you want to explore what it is, I hope, I expect that if you go to, I think it's rail-epam.com.

• 30:42Daniel Whitenack?

  Epam-rail.

• 30:44Speaker 02

  Yeah. Epam-rail.com, thank you. And you're going to read about what it is and you're going to find all the links to hopefully documentation, how to, you know. But also, most, most companies who we work with, they want more than just, hey, how do we install it? They want, and now we want to build with it. And that's where we come in with professional services. We can build them things for their dial. So that they can do the AI that matters to them in their context, with their data, with their workflows, with their restrictions, um, things they can and cannot do and yada, yada, yada.

• 31:25Daniel Whitenack?

  Yeah. And I'm wondering for this kind of, if you think about this zoo of underlying applications or assistance. I'm wondering, because you've obviously been working in this area for some time. Do you have any insights or learning around kind of easy wins for, you know, underlying functions or agents that can be tied into this sort of orchestration layer or maybe like more challenging ones? Things that you've learned over time and in developing and working with these things in terms of, you know, things that you could highlight as, you know, easy types of, of wins and things that,

• 32:09Daniel Whitenack?

  I mean, you mentioned the workflow stuff around some of what isn't yet kind of figured out, but more on the orchestration layer and the function calling, you know, what, what are some areas of, of challenge or things that might not be figured out yet that are, that you think are interesting to, to explore in the future?

• 32:27Speaker 02

  Let me think. So, because my first thought was to, so you're asking about connecting tools and functions

• 32:33Daniel Whitenack?

  Yeah.

• 32:34Speaker 02

  to an L L M and which of the functions or what, what type of connectivity sort of is easier and

• 32:39Daniel Whitenack?

  Yeah, yeah. Is there anything that's out of scope or more of a challenge currently? Or is it fair game for kind of, you know, I guess it's whatever you can build in that function in the assistant. But yeah, what limitations are there, challenges in that kind of mode of development of developing these underlying functions or tools?

• 32:59Speaker 02

  I see. So it's kind of a two-fold answer. If you take the technicality aspect, like how do I build a tool that does X? The complexity is really in X. Like, if you want to go and query a database, how hard is that? Well, not hard, right? I mean, connectivity to the database. If you have a query, you run it, you get results back. So it's not hard to do the technicality of querying a database. Making it useful and making the result useful in context of users' prompt and conversation is a lot more challenging. I had this, so I'm running a service.

• 33:41Speaker 02

  You can, it's actually has a public webpage called api.epm.com. It's our own, so you will not really go past the front page, but you'll, you'll understand what it is. It's a collection of APIs that we built, my team has built, that exposes a lot of data. Remember I said epm runs on internal software? So all of those applications, they stream their data and their events out into a global data hub. Think big, big, big Kafka cluster. But that's Kafka, so you can read data out of it as a Kafka consumer. But if you want to have like more modern, you know, API, search, lookup, this, that. So we have an API service, all of that data. And somebody came to me today and said, hey, have you heard of MCP? I'm like, yes, of course I have.

• 34:25Speaker 02

  Why don't you guys build MCP for api.epm.com?

• 34:29Speaker 02

  My answer is it is easy to build. APIRibbon.com speaks RSQL. I can build a server that will take your query, create RSQL. LLM will be able to do that easily, run it, give back the data. But I said it's not going to be useful because this is single data set APIs. Your questions are likely analytical. You likely want to ask something that expects me to do summary by month, this, this, this, and give you like a, which like, that's a very different question. So you ask me about MCP to an API, easy to do. Make it useful for your actual use case, much harder to do.

• 35:10Speaker 02

  I likely need to do a lot more than just connectivity of tool to an LLM. I need to understand what you're asking, figure out the orchestration that is required, maybe custom apps, maybe something else. And then, and then you start hitting authentication, legacy apps, all the other roadblocks. And in a way, the TalkToData is an amazing prototype that we built, and I have a video about this. But we sort of stopped because we clearly sensed how steep the curve is to get it to like actual, because what we wanted to do, what we envisioned we could do was analytics democratized. So you don't have to go to analytical team, ask them to build you a new Power BI report,

• 35:55Speaker 02

  and them spending a week doing so. You can just come into Dial and say, hey, you know, show me this, this, this, and this. And yes, we technically can do it. But to be able to do this for all kinds of questions you can ask about our data, that's a much harder thing to do.

• 36:11Daniel Whitenack?

  Yeah. Yeah. And it also, yeah, to your point, underlying systems might have limitations. I think in analytics related use cases that we've encountered with our customers, you know, often I'll just ask the question around, hey, if you, if you gave this database schema or whatever it is to, you know, a reasonably educated, you know, college intern or whatever that is, and you ask, you know, What columns would be relevant to query based on this, you know, based on this natural language query, you know, you can pretty easily tease out, well, I look at all these

• 36:54Daniel Whitenack?

  columns, I have field 157 and custom underscore new underscore field, you know, there's no way for for just someone off yet, you know, to to know anything about that. And so it's not really a limitation of what's possible in terms of the technicality, like you said, it's more of, you know, you're not always set up for success in terms of utility, like you mentioned.

• 37:19Speaker 02

  And for data, that's where semantic layer comes in. So if you have descriptions of your columns, of your tables with business meaning, then connecting that semantic layer with some data samples to LLM will allow it to write the query that you thought was impossible to write, because it is impossible without the semantic layer sort of can explain the data that you have in business terms, in the language that the questions will be asked of your assistant. And that's what allows us allows us to do this talk to a data analytics.

• 37:58Daniel Whitenack?

  Yeah. Well, I know that we've talked about a lot of things. I think you you are probably seeing a good number of use cases across your clients at EPAM and also your own experiments with Dial and other things. I'm wondering, as you, as you kind of lay lay in bed at night, or whenever you're thinking about the future of AI, or maybe it's all the time, or maybe it's maybe it's not at night. But yeah, as you kind of see what is to your point, just bringing it all the way back to the beginning, you see what is possible to do now, which even six months, a year ago, whatever it was, you know, was not possible.

• 38:41Daniel Whitenack?

  What kind of is most exciting for you, or most interesting for you to see how it plays out in the next, you know, six to 12 months? What is kind of constantly on your mind of where things are going? Sounds like, you know, the how we work with these tools is one of those things. We already talked about that a little bit. But what else is, you know, exciting for you or encouraging in terms of how you see these things developing?

• 39:09Speaker 02

  My answer may Surprise you. When I think about it, I don't, you know, think or anticipate any new greatness to come. I actually mostly worry. And I worry because I know that my thinking is linear. Like most of us, even though, looking back, we know that technology has been evolving rather exponentially. Our ability to project into the future and think what's coming next is linear. So I am unlikely to properly anticipate and get ready for and then expect, right? And wait for what's to come. I am sure to be surprised.

• 39:52Speaker 02

  And I guess as everybody else, I'll be doing my best to hold on to not fall off. So I worry seeing how the entry barriers rise. It's harder for more junior people to get in today. When I'm asked about skills I recommend that people focus on as far as trying to be better prepared for the future. I always answer it with the same things. I always say fundamentals and then critical system thinking. And fundamentals, you can read about a lot, but you really master them when you work with them yourself. Not when someone else works with them for you.

• 40:34Speaker 02

  And not having them is likely going to constrain you from being able to properly curate and orchestrate all these powerful AI agents. And when they get so powerful that they don't need you to curate and orchestrate them, then what does it do to you as an engineer? And maybe that's not the right thinking, but this is what I think about at night, like you asked. When I think about AI and what's coming. I am excited as an engineer. I like using all of this. I just don't know how it's going to reshape the industry and how it's going to change my work, you know, in years to come.

• 41:13Daniel Whitenack?

  Yeah, well, I think it's something even in talking through with you, kind of some of the work that you and I have been doing with agents and how that really has triggered a lot of questions in our own mind of what is the proper way of working around this? And I think there is going to be a, you know, that is a widespread issue that people are going to have to navigate. So yeah, I think it's, I think it's very valid and we will be interested to see how it develops and would love to have you back on the show to have your learnings again in six or 12 months of how it's shaking out, shaking out for you. Really appreciate you joining. It's been a great conversation. Thank you very much. It's been a pleasure.

• 42:04Speaker 00

  All right. That is our show for this week. If you haven't checked out our change log newsletter, head to changelog.com/news. There you'll find 29 reasons. Yes. 29 reasons why you should subscribe. I'll tell you reason number 17, you might actually start looking forward to Mondays.

• 42:25Speaker 03

  Sounds like somebody's got a case of the Mondays.

• 42:28Speaker 00

  28 more reasons are waiting for you at changelog.com/news. Thanks again to our partners at fly.io, to Breakmaster Cylinder for the Beats, and to you for listening. That is all for now, but we'll talk to you again next time.