Giving New Life to Unstructured Data with LLMs and Agents

Instabase founder and CEO Anant Bhardwaj joins a16z Infra partner Guido Appenzeller to discuss the revolutionary impact of LLMs on analyzing unstructured data and documents (like letting banks verify identity and approve loans via WhatsApp) and shares his vision for how AI agents could take things even further (by automating actions based on those documents). In more detail, they discuss: - Why legacy robotic process automation (RPA) struggles with unstructured inputs. - How Instabase developed layout-aware models to extract insights from PDFs and complex documents. - Why predictability, not perfection, is the key metric for generative AI in the enterprise. - The growing role of AI agents at compile time (not runtime). - A vision for decentralized, federated AI systems that scale automation across complex workflows. Follow everyone on X: - Anant Bhardwaj - https://x.com/anantpb - Guido Appenzeller - https://x.com/appenz Check out everything a16z is doing with artificial intelligence, including articles, projects, and more podcasts, here: https://a16z.com/ai/ Timestamps: 00:00 Introduction 00:43 What is Unstructured Data and How Can The UD Problem Be Solved? 07:26 Use Cases: How Enterprises Can Use Unstructured Data 13:07 The Shift In How Enterprise and Consumers View AI Capabilities 15:03 Documents: The Role of Humans vs AI in Unstructured Data 16:06 The Most Interesting Use Case, Lending Over WhatsApp 19:13 Main Barriers For Enterprise AI Adoption 21:25 AI Agents + Enterprise Workflow 25:23 Future Of AI As Decentralized, Federated, Execution 28:05 Technical Advances Of AI Impacted User Experience 32:35 Three Reasons Why Enterprise Adoption Of AI Is Necessary

• 0:00Speaker 0

  So rebody process automation is literally if human had to do something, you basically open some browser or whatever, take some data, put into some other system, click some button and all that stuff. So it records that human clicks on that desktop and tries to keep repeating it. So you kind of like get that automated. And the hard part that they had is you can't do robotic process for unstructured data because it's not fixed. So they change it. So anything will be very, very brutal. The bet that we are taking is that AI will drive automation in a significant way. RPA would be fully eaten by AI automation. And the future is likely going to be more of decentralized federated execution.

• 0:43Speaker 0

  I'll just give you the little bit of history when I was doing research at MIT. And I think big data was a big thing in 2015. Everybody was doing this. And so first, let me just define unstructured data because people have different definition of unstructured data. So my definition is very simple. Anything that cannot be put into nice database tables where you can run SQL. Anything that is not that is unstructured data. So like a PDF document or an image or anything. Yeah, anything that cannot be put in a nice table that you can run query. And we already knew how to answer questions when data is nicely in structured format. So at MIT, the question that they were trying to ask was how do you answer questions when data is not in that format? So and that's very heterogeneous, that basically didn't have any schema. You don't even know what questions are relevant or not.

• 1:30Speaker 0

  So that was the key sort of hypothesis. And we were building this product called Data Hub. And this has the ability to mount different kinds of things. So you could mount file systems, you could mount databases, and you can mount something called application node. So because some data also lives in random applications. And can you ask any question? So that was a big research project. I was like, this could be very valuable. So I dropped out. I didn't solve the whole problem and came here in Silicon Valley. And then I started talking to a bunch of companies. Tell me what is your unstructured data problem, because we have to figure out business or the where to sell and where is the real real value for the for the organizations, especially enterprises. And we got pulled into this gnarly problem, which is here are all my images and documents in Excel and PowerPoint.

• 2:17Speaker 0

  And can you help me answer questions? So my first question was, why do you even care? What question you want to answer? We need to understand that. And they're like, we do a bunch of processes that receives a ton of unstructured data, and we have to make a decision. Like, for example, if immigration, when somebody applies for immigration, they submit a bunch of things and they have to make a decision whether they should give you VIJA or not. Or you apply for loan. You submit a bunch of things and they have to make a decision whether you should get a loan or not. So we were like, sounds interesting. So let's let's think about how to solve it. And you won't believe it. The techniques at that time were very rudimentary. So there were four common techniques that people use. Number one, they call this like templates, where they will simply say, here is your template for passport.

• 3:04Speaker 0

  And if you want a passport number, go look 10 pixels below and 10 pixels from the right and draw a 20 pixel long box, and whatever you find is your passport number. Good luck with that. Yeah. It's very, very brittle, right? Because as soon as you scan differently and think things will break. Second techniques were basically people writing different kind of rules, like go and look for the keyword period beginning and anything right of that is a start date or something. Doesn't work. Like it just breaks. A third technique, people were trying to train these ML models by writing features for a specific document type. And what feature do you write for like base stuff? It's just very, very hard. So those also didn't work. So we basically at that time started doing research, which we killed in two years, what's called program synthesis, which is we were basically like if I had access to amazingly intelligent people, how do I solve unstructured data problem?

• 3:55Speaker 0

  I will ask them to write code for you know on the fly. So can I basically ask computer to synthesize a program on the fly? It's very, very hard for computer to write program at that time, LLM was in the thing. But we would like most of the data can be extracted from documents and all that by writing some form of regular expression and those kind of things. So let's do the synthesis of these regular expression based on what input-output combination that you give, and that is the answer. And that worked reasonably well as long as your input is in the similar kind of structure. Because the problem with program is it's deterministic. So if your input changes, it will break. So but it still produced reliable results, but not good enough that we could solve many problems, but we could solve some part of the problems. So this was 2017. And transformer paper came.

• 4:42Speaker 0

  And I think with transformer paper, they also released a model called BERT at that time. So we were super excited. We were like, because that was like state of the art and like best sort of model to understand natural language. So we basically applied BERT on these unstructured documents. We took a bunch of those tokens and put that, and that produced really bad results. Really really bad results. So we would like at that time, actually I sent a note which Martins would have a copy of, which we we were like, seems like this problem is not solvable. Unless somebody solves AI complete problems, which they call uh you know AGI that you can. But we were like, there is nothing else that is promising enough. So what do we do? So we basically tried to use some creative approach that if you look at the BERT language model, they were encoding the token as the position of the word in the sentence.

• 5:30Speaker 0

  And that's how the attention uh mechanism would work and and and and would do the fill mask problem. Yeah. So we were like, what if we also start encoding in addition to the position of the word in the sentence, X and Y coordinate too. So we basically took 110 million documents, took every single word or the token and encoded with the position in the sentence, but more importantly, X and Y coordinate, and then tried to basically solve fill mask problem by basically blocking and see that if that box can be filled by the by the model and trained a model, which is similar to BERT, we call this instaLM. And that produced great results because the attention is now not just looking at the sequence of tokens, but also XY coordinate in the two-dimensional space, which is really really cool from the perspective of the document layout understanding.

• 6:16Speaker 1

  And I think it's a fair to say that has become, I mean, this is much later today, but today is sort of a standard technique almost, right? If you're looking at two-dimensional data, you have some rotary encoding of X and Y or something like that.

• 6:25Speaker 0

  Yeah, yeah, yeah. So at that time that was not that was not the case. So actually, Rafal, who was one of our ML engineers, he you will see uh those two or three papers being in the top of the arena during during that time. So we were very happy. We started winning a lot of deals. We triple our revenue that year, 2021 to 2022. But then OpenAI launched Chat GPT, which is November 2022.

• 6:45Speaker 1

  It turns out the the bitter lesson held uh size matters.

• 6:49Speaker 0

  So and we were like, oh man, like that is a better basically that you could actually pass the documents. And at that time they didn't support documents in the in the first release, but you could basically take the text with the positions like reserve and pass, and it could it did a reasonably good good job. And we were like, is this end of insta base? Like, like you can now seem slightly seems like solve solve this whole problem. And then we realized that there is just a ton of things, and I think there's a paper by Databricks, which is compound AI system that LLMs are very good, but you need a bunch of systems before and after this for it to be reliable, and we can get into the details, but that is the history of how we are where we are today.

• 7:26Speaker 1

  Yeah, amazing. Very small personal anecdote. I have a lot of PDF files, everything, a piece of paper I get, I just scan and dump into a folder. And I recently wrote myself a little tool that basically first I asked an LM to come up with a hierarchy of documents. You know, we're a family of five, you know, here's some things about our about our family, and then give me give me a document hierarchy. And then basically by taking taking a document and just giving taking the summary of the document, given to an LM say, which which folder should this go into? That's an amazingly efficient sorting algorithm. It's really impressive what what LMs can do. So today you have a solution that basically allows enterprises or uh you know companies to work with with unstructured data. Can you talk a little bit about what this does or what some of the use cases are?

• 8:09Speaker 0

  So the use case is pretty simple, which is let's say I'll take a simple example of a bank that wants to do lending or an insurance company that wants to basically process your claims. So let's take one of one of these two use cases. So when when people apply for, let's say, application of home loan. It's like literally a hundred page long packet, and you don't even know where is what. It could be that first 10 pages is their bank statement. Here's a shoebox of documents. And in between, there might be cat's picture. In between, there might be some random letter from somebody. And so the issue, I think, is there is no one structure. What bank says, hey, I need to do something that can verify your income. I need something that verifies your identity. So it's not that they tell you here is my passport and here is my driver license, here is application packet and go and process it.

• 8:55Speaker 0

  You have to do this reliably because you cannot make a single error. You can just think about like how do you solve this? So there are two techniques, and that's what uh, you know, one of the I think paper that we wrote is LLMs is not all you need, because one thing that you can do is put that into some stuff and ask LLM the question. But the problem is if it goes beyond the context window, then that's a problem. You can do RAG because RAG is a technique where you put that into some vector database, figure out for what question what are the relevant chunks that could be uh useful and then produce that. But how do you know something you did not miss? You might get precision, but if you miss something, then that's a problem. And LLMs are great, but they make surprising errors. So for example, let's say you have 10-page long bank statement with tables. Somehow they will get a lot of things right, but miss like four random cells with the value, and you don't even know that they missed it.

• 9:47Speaker 0

  And that just changes the whole thing. So these are the very surprising kind of errors. So we looked at like how do you solve this reliably? Because reliability part is important because these are. Complex decisions that banks or insurance companies or immigration make. So the right way to solve this is how do you know how to split this particular packet into a bunch of things we care about? So you have to analyze every single thing in detail. Once you have done this, then how do you get all of these structures that we care about? Like, for example, we run separate table detection algorithm rather than passing the whole thing to LLM because how do you know you didn't miss four things? How do you make sure all the cells are correct? Similar thing for checkboxes and the signatures and other things that basically matter. Once you have classified, then what are the relevant schema that we need?

• 10:33Speaker 0

  Then you basically go and do those things. How do you validate that each of those things are correct? Then write validations and then do cross-validate validation because is the pay stuff saying the same thing that W2 does? Because if not, then that's so basically the what we provide is this interface where people can build all of those things without writing a single single line of code. And then you build this application. And now you can run this application as part of deployment, which will integrate with your upstream and downstream. So now you can do lending in like less than five seconds rather than earlier that would have taken several, several weeks. The one very interesting use case is intelligence use case, for example. So let's say that's where I talk about why the approaches are critically important. So let's say you are a country and you want to collect a bunch of intelligence, you collect a bunch of intelligence data, and you want to answer if there is any threat to the country.

• 11:26Speaker 0

  So and you receive like, let's say millions of documents per day. One way to dump that all into some RAG system and ask a question. How do you know you didn't miss anything? Because they care about that. And maybe the right way to answer that question is not putting all the documents into a search. Rather, looking at every single page of the document. Look for the things that you care about, like which is terrorism threat or money laundering or whatever, and then extract that, put that into database, run SQL query. Once the things that match, then go and do the deeper analysis, because now you guarantee completeness. So I think that what we have seen is that while RAG is good for casual search, you need a complex workflow under the hood that is explainable, that is auditable, that is guaranteed to be accurate and correct, is important for solving many of these enterprise problems.

• 12:12Speaker 0

  So that's what we do. We help basically enterprises take any kind of unstructured data and make decisions from it for reliable, 100% complete and accurate use case. There are cases where we can make error. And in that case, we have to pass to humans, like, hey, seems like something is wrong. Can you go and look at it? Totally. And

• 12:30Speaker 1

  look, I mean, I think this is the trend with current AI systems, right? I've not encountered an AI system yet that is perfect. And by some metric, I think we never will, right? I think what we what you need is finding things with reasonable error rates and then a good escalation path to humans to deal with those, right? Uh

• 12:45Speaker 0

  exactly.

• 12:46Speaker 1

  And even humans, humans are not 100% correct,

• 12:48Speaker 0

  right? So you have to build the right processes to catch it. So that's why I think sometimes when people say this AI doesn't work, it's just that AI is not supposed to work reliably 100% of the time. You have to build a system around it. That's right. Yeah. So that, and that is going to be a lot of investment that you will see across the board, which is how do we build the right systems around AI and LLMs that solves the problem.

• 13:06Speaker 1

  Is there a shift in how enterprises or general, I think consumers of AI think about reliability. I mean, look, look, classically, if I'm a chief compliance officer in a bank or so, I have a new piece of software, and my, you know, my take is this software can never do X because that puts us out of compliance. I recently spoke to a bank that basically said, like, well, we tried that. It doesn't work with AI, right? So now we're saying, you know, the the a well-trained human gets us out of compliance about X times uh, you know, every X hours or so, right? And so the AI has to be 10x better, and then we're gonna sign off on it. Uh, you know, so so you you you cannot have absolute perfection. Um, so so you know, we have to sort of change the acceptance criteria. Is that something you're seeing as well? I

• 13:50Speaker 0

  think more important is predictability. I think people are fine with errors as long as errors are predictable. When errors are not predictable, that's where the problem is. So when basically somebody makes an error and you don't even know that it was made, that that's when in humans, you know, they will make three, four percent error, you know. But if you put the second human, by default, the chance of that is low and with AI, the issue is that they're pretty accurate. They're very good. And but they make mistakes in a surprisingly unpredictable way. And that that is a bigger problem. And that's where I think the tooling and systems around it to detect them, to be able to explain when the error was made, to be able to figure out how to catch them, or building system that allows you to minimize that effect, that is the critical part.

• 14:36Speaker 0

  So I think in general, what we have seen is enterprises are fine using AI as long as we show them predictability. They don't they don't care about you know 99% accuracy. You can be 90% accurate or even 80% accurate, but just tell us which 20% need to be reviewed or which 20% need to go somewhere. And that requires a lot of systems around these tools to get there. So I think we sometimes misunderstand the what enterprises want. They don't want 100% accuracy. They want something.

• 15:03Speaker 1

  That makes sense. Is this the future that essentially, you know, in the future if an organization receives a document that typically humans will no longer see the document, but will primarily look at an IA generated summary, or you know, I will pre-parse it and you know, I can reason about it at a higher layer.

• 15:19Speaker 0

  Whenever unstructured data like documents come in, humans will still see some kind of dashboard with like whatever stuff is, and only the thing of interest they will go and double click on. And AI will do a lot of things to minimize their time to get to that thing of interest very, very quickly. Like Google is a great example. When you search, you don't read every single thing. Google gives you like, hey, here is maybe three or four things of interest that you want to double click and do research on. And I think AI will play similar important role where in many cases gets rid of the boilerplate, it reduces the thing to the absolute essential.

• 15:52Speaker 1

  Are we looking at a world where my system will take uh you know my my couple of key key points or key phrases and generate a PDF document, then your system will take the PDF document and reduce it back on the couple of key points and phrases?

• 16:03Speaker 0

  Exactly. That's

• 16:05Speaker 1

  uh I guess not a better way to operate uh uh in the future. What is the most interesting use case you you've seen for your technologies of anything sort of obviously ordinary?

• 16:15Speaker 0

  I think I think uh what we are seeing, customers being a lot more creative than we had ever imagined. So just think of I was working with a bank in India. And now, given that AI is has become reasonably reliable, they are offering entire lending over WhatsApp. So you go to WhatsApp, you say, like, hey, I'm a business and I want a loan. And then on WhatsApp, you get a response back saying, hey, can you upload these things your last 30 days of like all the you know your PNL statement and whatever those things look, and you basically piecemeal you submit these three ports, like, oh, this looks good. Can you also do this? And I've never seen like lending being done conversationally over WhatsApp. This is insane. Like the customer experience is like fundamentally very different. And I think that I do believe that over the coming years, it is going to change the user experience in a very, very significant way.

• 17:05Speaker 0

  Currently, I think a lot of people think AI is a technology and you know how we can use this inside software. I think that the biggest impact would be with the degree of affordance that it gives you, you can completely build a new class of interaction with your customers that would never have been possible. Uh, and and we are seeing more and more of those currently, like all of these processes, like insurance client and all, it's pretty painful process, right? Very much. And I think America is slightly more conservative in those things. But if you go to developing world where digitization is more of a new thing and people are already using all the stuff on phone, things are just moving in a way where, you know, because AI makes you feel like you're talking to humans. Nobody loved chatbars before, but now you feel good because they they basically are conversing with you in pretty similar to human-like behavior.

• 17:56Speaker 0

  And that interface coupled with all the customer interaction that they have. Of course, one of the big use cases that everybody's trying to go after is a call center. But just think of every other things too. Like how do you create open an account? How do you do lending? How do you do processing? It will have significant impact on how the user experience is going to change in a very, very significant way.

• 18:19Speaker 1

  Yeah, totally. And I think there's even, I think, an opportunity here to take some processes, which currently were very, you know, I I I take a lot of documents that throw them over wall and back comes a response to really turn to something something more interactive, right? Where it's like, hey, we don't uh, you know, tell me more about your space use case. Okay, then I need these documents, and I send them. It's like, well, that document is missing something, and you know, you can do this interactive with very virtual latency.

• 18:40Speaker 0

  Everything, even immigration, right? Like you send the stuff and you don't even know. Like, you know, two months later you hear like your stuff is rejected, or we need something like this. Okay, all of those things can fundamentally be

• 18:48Speaker 1

  changed. I just got a letter back from the ARS that submitted a long uh you know application with lots of supporting documents, and I got a form that about saying the documentation is not complete without any mention of what is not complete. And so you're saying that's like what does this mean? This

• 19:00Speaker 0

  can be just much more interactive. Yeah.

• 19:01Speaker 1

  And because now you can do things in real

• 19:02Speaker 0

  time.

• 19:03Speaker 1

  Yeah, and so I'm pretty

• 19:04Speaker 0

  optimistic on the impact of this on every single business uh on how they interact with their customers. That makes

• 19:12Speaker 1

  no

• 19:13Speaker 0

  sense. What do you

• 19:14Speaker 1

  see as the main barriers for companies to adopt this? It's like, you know, I mean, we I've seen I've seen many classic enterprise adopting AI. There's discussions around compliance and legal and where does my data go? And then, you know, like like a long list of of um sort of uh concerns that are being expressed. Well, what are the top sort of uh items that you've seen? These enterprises are not historically known for moving very quickly. So that's just number one.

• 19:41Speaker 0

  So I think expecting that like you know,

• 19:42Speaker 1

  I would say they're moving a little quicker in the AI revolution than they did previously. So

• 19:46Speaker 0

  exactly. In general, I think each of these large enterprises have to get approval from their compliance committee and the regulations committee, you know, and they all basically and not none of them really understand things. And sometimes you get regulations that might or questions that might. Might not even be applicable. Like, for example, tell me every time you change the feature, how do you LLMs like we LLM developers don't change features, right? And but when you get like all of those things that that basically is a massive time sync. But I think the two key things that they care about is how do you guarantee that my data is safe and secure. So that's number one. And second is how do you give me auditability and predictability? That's the two more like if you boil down to all their questions, they eventually boil down to those two things.

• 20:32Speaker 0

  Like nobody wants like AI making a decision, even if it is correct, if they cannot explain here are the set of steps that it took. Because if if something wrong happened, they have to explain. Because in the human world, you can explain. Something came, this went to these five different teams where they did this part and this particular error was made, and that's why which we will collect in future so that this kind of mistake would not happen. If AI becomes a black box with no instrumentation of how things get done internally, that that basically has hard time, especially for customer-centric use cases. For simple casual search and those kind of things, it's fine. But the runtime has to be something that is auditable. And you should be able to find if something went wrong, where it went wrong. And they don't tell you directly.

• 21:18Speaker 0

  They they ask like question that eventually boils down to this. But that's what we have seen as the major requirement.

• 21:24Speaker 1

  Makes sense. Let me switch tax here a little bit. We've seen, you know, one of the, I think what is buzzwords at the moment are agents, right? And so if you know they're it's an overused term, it's sometimes used as a marketing term for you know a glorified set of prompts, basically, right? But we're also seeing it as a essentially different user interface paradigm, right? Where I no longer walk through a transaction step by step, but basically I give a high-level instruction to agent agent acts autonomously. We're even seeing yourself as a software design paradigm where now have multiple agents that work together and you know make decisions more autonomously. How do you think this will change with you know how enterprises process data, how we work with unstructured data and then this entire space?

• 22:05Speaker 0

  So let's look at like what we already know that has worked well. So what we already know that has worked well is enterprises already know how to run some workflow that is created by some developer and they define a bunch of steps using some workflow management tool and you you can run it. So people already know how to run this. Argument you can make is can we just tell the agent, like give me the answer, and they do it. The problem with currently the agents are if you just give them the same goal and same set of tools, and they might choose different paths two different times. So they are not guaranteed to deterministically always go in one path. So in general, people don't like runtime inconsistencies. So runtime has to be consistent. So I think the where I have seen things work well within enterprises during build time when somebody has to define the control path and the logic and all of those kind of things.

• 22:52Speaker 0

  You can maybe have agent produce the first draft, which is like, hey, this is how I plan to execute, this is what it looks like because otherwise human might have taken like a long period of time. Pretty similar to cursor, right? If you want to build something, they can write the first draft of the code, then human can look, make some minor edits, but then you run that code deterministically. Yeah, exactly. So my point is I think it's the same way. So I do not believe that autonomous agent would be a runtime phenomena. However, there would be a build time or compile time phenomena, which basically means that during build phase, they can do the 90% of the work. Humans make some changes, and that's a huge, huge, huge value because the reason why things don't scale at the enterprises is because there is either lack of enough developers or skills or drive or whatever. If AI agents can do things and make it so easy that you can build those, and then once it is approved, then we know what is running.

• 23:43Speaker 0

  Then there is auditable, and you can also add steps and checkpoints, whatever that is needed, uh like for example, cursor generated code, but you want more logging, so you can add logs in between, whatever those things could be. So once you have that deterministic artifact that can run in production, so that's where I think the world is going to move towards, which is a compile time phenomena and the runtime phenomena. Runtime phenomena has to be deterministic, something that is auditable, debuggable. You exactly know what is happening. You should be able to see the logs and all that kind of stuff. At compile time, agent can play an important role because they can help with the reasoning and create the first draft where human can participate with the agent to produce the artifact that is going to run.

• 24:24Speaker 1

  Yeah, it's a lot of sense. I mean, this is a super hot debate at the moment, right? And I think we've everything from this AGI vision where it's like, no, this is going to be a fully agentic loop and it decides when it wants to terminate, decides what tools to use. And it's just, you know, you give your credit card and let it run, right? And I think I personally agree. I don't think we're there yet, right? The the these most freeform agantic systems that we've seen, they they typically don't work yet. This approach of saying let the LLM generate the flow, but then freeze the flow once it works. I think at least in the short term, it's a much more problematic vision.

• 24:54Speaker 0

  Also, basically, I think we can take a lot of lessons from what works in the human world. Let's assume every human is an agent. You don't allow every single employee in your company make autonomous decision. No, some person at the top says, here is the set of things that we are going to do, you can only do these set of processes. And then so basically the runtime is pretty deterministic. Most of the reasoning and agency and all that cool

• 25:15Speaker 1

  stuff is used. So

• 25:16Speaker 0

  getting so LLM process re-engineering is a

• 25:18Speaker 1

  thing now. Yeah, I guess. That's that's fantastic. So what are you excited about looking forward? What are you excited about in your space? You know, where I mean, AI at the moment is hard to predict what's happening in six months, right? But if you you know try to stretch your crystal ball to the absolute limits here, what what what what things do you think we'll you will see our 12 months out, two years out in your space?

• 25:40Speaker 0

  So we've been, we have been um we've been debating and reasoning on this for for quite a period of time, and maybe my answer would be slightly controversial, uh, because you know different people have different view of what would be the future. So I do believe that AI will continue to improve and the capabilities, and I think they will play an important role in compile time, building things, reasoning and all that, although runtime is going to be much more deterministic and predictable and controllable. Now the question is what is going to be execution pattern? There are two different views of the world. One is that does it make my data management problem easier, that it allows compile time, move all the things into one place and be able to answer and do things?

• 26:29Speaker 0

  Or you basically keep the tooling in the world the way it is, siloed everywhere, and AI would become smart enough to have multi-agent communication where each agent can do things and figure out how to how to, you know, if one makes an error and affects how to do the communication. So uh we have been uh working on this idea of federated AI execution, where how you can, as an organization, you can define these thousands of agents in a very federated way, but dynamically are able to discover other agents through some platform or whatever that that could be, and then able to communicate. So you if you give a bigger goal, somehow they basically you don't need a one central person to decide everything. Dynamically, all the agents can discover, they all can say the capabilities, then you can figure out the control path, they can then you can figure out how to run.

• 27:23Speaker 0

  So we are trying to build federated decentralized automation framework, which basically means that can I take any process in any organization and figure out the federated decentralized execution framework and that can run. And that's what I believe that automation world would move. There are still a lot of open questions, a lot of unknowns. A lot of work to do, yeah. Uh, but the bet that we are taking is that AI will drive automation in a significant way. RPA would be fully eaten by AI automation. And the future is likely going to be more of decentralized federated execution.

• 28:00Speaker 1

  Yeah.

• 28:00Speaker 0

  And so that's that that's that's that's what's one hell of a vision there. I'm

• 28:04Speaker 1

  uh I'm excited about it. So AI is progressing very rapidly. How have sort of the technical advances of AI impacted what you can deliver to your end customers? I mean, that there must be changing business constantly, is that right?

• 28:18Speaker 0

  Yeah, yeah. So I think the earlier we focused primarily on the unstructured data problem as part of the automation, because that's one of the long 10 in the poll is how do you even understand them? Because once you get data in the structured format, you know how to do next steps. So we primarily focused until now, which is if you get a bunch of unstructured data, how to get you the things that you need to make the next step of the decision. We did not touch the next step of the decision. Like, let's say you are a lending company or you are an insurance company. Once you get all the data, you might have to, you know, trigger some other tool like their lending system or some sort of fraud system or whatever, the risk system and things like that, because that requires knowing about those systems, how to interpret the results and all that kind of stuff.

• 29:04Speaker 0

  So we are like data in, we will do everything, give you valuable data out, and after that, you are responsible for all the other integrations. And the way these guys solve those problems was by using this technology called RPA, you might have a robotic process automation. So robotic process automation is literally if human had to do something, you basically open some stuff, browser or whatever, take some data, put into some other system, click some button and all that stuff. So it records that human clicks on that desktop and tries to keep repeating it. So you kind of like get that automated. And the hard part that they had is you can't do robotic process for unstructured data because it's not it's not fixed. So they change it. So anything will be very, very brutal. But if the things are exactly the same after that, you can actually like record the screen and and and replay it.

• 29:51Speaker 0

  It has been very, very brittle. Like the problem with the RPH, even though they add value, uh, I think there are some big players there, UI path, automation anywhere, and many of them have reasonable, you know, massive market. Market cap. Now with AI, the argument that we make, and we might be wrong, uh, is once the data comes out, which we are very, very good at at that until that point, can we also start operating those other systems? Now, this makes a massive jump in, which is AI will help us operate those systems. And uh there are some interesting protocols that has come, which is model context protocol that allows you to dynamically discover capabilities, call those functions. It has a ton of problems still, which is does all the system even support MCP? What if they don't?

• 30:34Speaker 1

  And uh they they sort of punted on authentication, but but we'll figure that out over time. Yeah.

• 30:38Speaker 0

  Then authentication. Then how do you know if something breaks? One of the arguments that we're making is that maybe in future, as we basically go broader, can we do entire end-to-end workflow? So once data comes out, do we have a way to plan and regen during the compile time, which AI agent can do, how to operate those systems, how to call them, how to get the data, then call some other system if something gets wrong, how to involve humans. So create all that stuff during with the AI agent during compile time, and then extend our offering to do this entire thing end to end. Like can RPA be fully replaced with AI automation. RPA had some you know stuff that is easier to solve because some user logs in. So it always runs in the context of the user if you're clicking on desktop and all that. One of the hacks that we believe might work is called identity pass through.

• 31:26Speaker 0

  That can we assume the user identity that can be provided during runtime and then let that user identity get passed to all of the

• 31:33Speaker 1

  MCP. I mean, do I always want an agent to have the same capabilities that I have? You know, like I mean an agent is like um today like a good intern, right? So I I trust the intern up to a point. I don't necessarily want unlimited spending on my credit card, but I want to keep that at $50 or something like that.

• 31:51Speaker 0

  And you can decide that during compile time. So basically you can say that, like, hey, even if like let's say this this user context is with this, but as soon as it gets to this operating tool, maybe like we create some like user divided by half fake identity that will have less permissions or things like that. So the good thing is, and that's why I said the AI agent should only be you during compile time so that it gives humans all the control that's what the runtime behavior should be. Yeah, makes sense. This problems come when AI agent is making runtime decisions because then you have no control where things are going. Uh so the separation is critically important. So during the initial build, you can choose like if you want to curb what agency they have and what limits and constraints that they have, and that's what it will it will go and do during the runtime. Right,

• 32:36Speaker 1

  Anon. Thanks for being here today. That was absolutely amazing. I think we're we're on a very exciting journey, you know, with with AI. And you know, if looking back, I think the last big wave that I was a part of uh, you know, was was probably the.com boom. And I think if there's one lesson learned for enterprises in general back in those days, is that these big technological shifts happen, you have to jump on the wave early. It may be complicated, maybe you know, still a little a little weird, a little little uh, you know, your compliance, your legal folks, they don't know how to deal with it. But you know, if you don't, you may end up like Mars and Noble, right? The the downside is is substantial. And I think it's it is clear today that that there's a huge opportunity for enterprises here to both have more efficient workflows for themselves, but also to have a much, much better end customer experience and and and partner experience.

• 33:25Speaker 0

  And in addition, it does three things, which is it saves you a lot of cost. It does. It allows you to do things much, much faster. And the third one is fundamentally changes customer experience in a very significant way. So I think I think there are there are all the business reasons for enterprises to adopt these things. Now it's just about how to make this work. I don't think I have any question on whether this, you know, whether this will work or is this the right decision? It's about how to make it work. That that is the bigger question, I think.