Using Agents for Long-Horizon Financial Work — Aman Singh, Arca

Arca head of AI Aman Singh joins Latent Space head of editorial Richard MacManus at AI Engineer New York, to discuss how Arca is building an AI-native wealth management firm. Singh explains Arca's architecture, which includes a custom harness and persi...

• 0:00Speaker 00

  We're actually not a SaaS company. We are, um, we are the firm ourselves. Agents prompt people more than people prompt agents. And it is a proactive kind of, um, service that is running for them twenty four seven.

• 0:13Speaker 01

  We're here at the AI Engineer New York Conference twenty twenty six, um, and I'm here with Aman Singh, uh, from Arca. And, uh, Arca's recently raised sixty four million dollars for an AI native wealth management firm. So first of all, Aman, can you tell us a bit about your role at Arca and, um, the financial services you provide? And, um, how you're using AI agents to run them?

• 0:38Speaker 00

  Yeah, for sure. Um, thanks for having me.

• 0:42Speaker 00

  My role at Arca is I lead the AI platform team. So we're building the underlying sort of, um, harness and our capabilities to kind of run the business with AI, which is our ultimate goal. In terms of the services that we provide to our clients, um, with, this is a broad spectrum of financial services. So that includes, um, sort of financial advice, financial planning for, um, for their assets. And sort of, um, uh, tax planning and insurance and estate planning as, as people get wealthier. Maybe you are, um, someone who just sold a business, for example. And there's lots of complexities with all the different partners, et cetera, right? So, uh, we're gonna help you think through that.

• 1:23Speaker 00

  We also have clients that are, you know, uh, really large technology executives that, uh, have, uh, are trying to figure out a real plan for retiring, retiring early, right? And also small families that, um, uh, that are trying to buy their first home, for example.

• 1:39Speaker 01

  Yeah.

• 1:39Speaker 00

  Yeah.

• 1:40Speaker 01

  So the, um, the clients that you have so far, are they, um, do you have to kind of talk them through using agents? And, and why you're using agents and tell them a little bit about the technology behind it?

• 1:51Speaker 00

  Yeah, so, um, the role of AI at Arca is not to replace the advisor, it is to replace all of the work around the advisor. So our clients don't really interface with AI directly. All they experience is a level of service that has not been, been, been possible in this industry before, delivered through the empathy and care of a human advisor. So we're building the machinery behind, um, every client that's constantly thinking about, you know, um, what's, what is the market doing and how does it impact Richard? You know, um, what, what, what else is going on in your life? Like, what are you, what are the, the textures of your life that impact your finances?

• 2:31Speaker 00

  And, uh, we're able to be way more proactive, way more responsive, um, and just deliver a better, um, kind of, uh, level of client service using AI that does all of the, the back office stuff.

• 2:41Speaker 01

  Okay, so let's get into the, some of the technical aspects of it. Which models are you using to power your agents? And, um, how do you decide which model is best for, for which action?

• 2:51Speaker 00

  Yeah, um, so today we're using, uh, the frontier, uh, models from, um, from, uh, the frontier labs like Anthropic and, and OpenAI. Claude is kind of our primary, um, um, model has been, but, um, we've gone through several iterations of models over the years. We used to be in Gemini way back when, um, and then OpenAI and then, and then Anthropic. And what's guiding all of our decisions are, um, our benchmarks that we have internally for, um, all of the various, um, capabilities of our general harness, uh, that it has to express in, in running our business. And so, uh, just recently, for example, um, despite the external benchmarks claiming, um,

• 3:33Speaker 00

  great things for Luna, it didn't really quite work out for us as, as well as we had expected. It was way too slow or, um, and it was costing a lot. And so, um, our internal benchmarks kind of guide our model decisions. And, uh, as a principle, we, we want to be, um, model agnostic. And, and sort of not, um, uh, you know, um, take a dependency on, on a, a certain provider or a certain harness from one of these labs and kind of keep that, that optionality open for us.

• 4:01Speaker 01

  Okay, so, uh, you just mentioned, uh, harnesses. So you're using different harnesses depending on which model you're using?

• 4:08Speaker 00

  No, we have, uh, we have our, our own, uh, kind of built internally that is more model agnostic.

• 4:13Speaker 01

  Okay, um, so, um, you know, when, when you're, when a financial advisor is doing something like a taxation report or something like that, how does the harness decide which model to use? Is that automated?

• 4:25Speaker 00

  Well, yes, so we have, um, certain tiers of models based on different complexities of, of tasks. So a task like assisting with a tax plan or doing a tax analysis would get our like heaviest tier of models. Inside of that tier, it is a little bit flexible. So you could use, um, so like an Opus tier model or a Sonar tier model, uh, or a, a, a heavy GPT model. But, um, based on, uh, there's a lot of like background activity that our agents are participating in, right? That is using the, a lower tier of model, um, based on the complexity of the task. So that's kind of how we, how we make decisions, uh, on, on which model serves which task.

• 5:04Speaker 01

  Okay, so is it the, um, uh, the financial advisor that's actually making that decision while they're using the product?

• 5:11Speaker 00

  Not really, um, they're, they're, they're not aware of, uh, of the happening. We've abstracted this away, uh, from them entirely. They kind of just interact with our product, uh, as they would, um, and those decisions are made internally, uh, by us.

• 5:24Speaker 01

  Okay, so, um, in your presentation at, at the conference earlier today, you talked about a textbook, uh, example. Uh, I just wondered if you could maybe walk us through, uh, an example of adjusting a client's, uh, portfolio, investment portfolio. You know, from identifying a need to maybe rebalance the portfolio to executing trades. Can you talk us through, you know, what, what parts of that process the agent does and, and what parts the human advisor does?

• 5:52Speaker 00

  Right, um, so just like in the talk, this is a thing that will span ten days. Um, and so agents are ultimately, um, responsible for delivering that outcome. But there are, are, are certain parts that they do and certain parts that they do not. So in this example, um, when, uh, when the client emails us, uh, with this whole tax bill, there's a, there's a piece of the analysis of like, where is this client's money allocated today? And what is the most optimal way, um, to liquidate some of that to, to produce a hundred thousand dollars? That is best suited for agents

• 6:25Speaker 00

  to take on as an analysis task, so that they do exactly that.

• 6:28Speaker 01

  Do they have certain limits though, that they can't, you know, they can't execute a trade at X amount?

• 6:33Speaker 00

  So the analysis part is fully automated, but, um, they're unable to actually execute, uh, any sort of money movement or liquidation automatically. We have a very deep integration with our custodian, uh, that actually gives us APIs to be able to do that. But we have strict guardrails around human in the loop for things that would either move money or liquidate assets or even directly communicate with our, with our end clients, uh, because there's a level of service that, that, uh, service and quality that we expect. So all of the like internal analysis will be done by agents, but the, the, they will also pull in the right person to be able to execute the trade when needed.

• 7:13Speaker 00

  And they'll give it, give the person a plan that they, they're able to review and kind of like adjust, um, and yeah, go from there.

• 7:19Speaker 01

  Yeah. Yeah, so you mentioned that, um, these processes tend to be, you know, ten days or at least a longer sort of, uh, running time than most sort of agent stuff these days. So, um, when an agent kind of resumes a task maybe days later than they, you know, than they started it, how do you restore, how do you check that the plan still fits the client's, uh, you know, needs?

• 7:42Speaker 00

  Yeah, um, so this is an interesting, interesting one to think about is we have tons of these signals like I described in the talk. Some are, for example, you can imagine there's an email thread that is receiving responses and there's chatter happening, right? You can imagine that, that any additional response to that goes to the same agent that's kind of like reading that, that email thread and reacting to it, right? But a, a fresh signal of, it's like a, like a phone call that was made in the middle. That is a brand new signal that isn't attached with any other ongoing process. And so there's like a fresh agent that wakes up to figure out what this means, right? And the way they, way they understand what is happening right now is our internal memory systems.

• 8:25Speaker 00

  So we have, um, we have this, this persistence of for every client, um, what is happening right now, what are we trying to service, who owns what, what has happened so far? What are we tracking for next, right? What, what needs to happen immediately in the next couple of days? What are we tracking for the long term, right? Which individual is responsible for which action? All of this is being tracked by our memory systems that our agents contribute to. But also there is a, a kind of daily sort of dreaming process that reflects on the state of the world as it relates to Richard, the client, and is able to kind of like update these memory systems to, to persist longer horizon kind of tasks.

• 9:04Speaker 00

  So that agent that received a new signal quickly ramps up on what, uh, the situation is with the client. And he'll know, okay, we're, we're actually executing a big tax bill. Um, and we'll be able to see, okay, my signal relates to that tax bill. So let me kind of contribute to the, to, to the state of that world.

• 9:22Speaker 01

  Can you talk a little bit about the the safety sort of aspects that you've maybe built into your product or harness to to make sure obviously that no money is lost along the way?

• 9:32Speaker 00

  Yeah, so all of our safety restrictions are are fully deterministic and they they take the decision making off the the agent's hands. So for example, certain tools let the let the agent kind of execute the tool itself. So if it's like a simple record keeping kind of tool, it will execute it. We also with every time a tool is called that writes data, we have L L M based verification running on on that action. These like fast L L M calls that that analyze every single right to our database. So that's another layer of verification we're running. And then we also ingest a lot of like external information. So there's emails coming in,

• 10:13Speaker 00

  documents coming in, phone calls being made, which are targets for prompt injection. So we run sort of verifications at the at the edges that detect presence of certain, you know, flavors of prompt injection. And then anytime there is a there's a case where data leaves our system, for example, in like a web search in in the most obvious case, but also if we're sending a calendar invite to Richard, right? That's an external kind of data where we have both deterministic checks on presence of certain information that we know is P I I, but also a consensus of small models that that ensure that there isn't any kind of data exfil being attempted. So we have tons of different different kind of guardrails built into the

• 10:56Speaker 00

  harness with the principle that we want to kind of contain them at the edges of capability and not earlier than that. Yes.

• 11:02Speaker 01

  And and because you're in the financial, you know, industry, are there certain kind of regulations that are required that you have to sort of keep to in terms of the technology that you're using? Or is that is it just more about outcomes and the way you sort of deal with clients?

• 11:18Speaker 00

  Yeah, the the industry cares about outcomes, but compliance cares about audibility, right? And so our our system is is such that audibility is by default. We have every single action that that an agent takes produces a change log of what changed, who changed it, from which part of the system and when. And this applies not just to agents, by the way, it applies to human actors as well. So if you change something in the UI that affects the client, we'll log that action. So it lets us both get audibility by default, but also power some of our agentic capabilities where the same thing that that we tracks like a UI change also

• 12:02Speaker 00

  could be used to like inform the agent that that happened.

• 12:05Speaker 01

  Right.

• 12:05Speaker 00

  So audibility has been a big part of our our story, both for obviously compliance and regulation, but also for our agentic loop.

• 12:14Speaker 01

  Yes. In your presentation, you had a nice line, agents prompt people more than people prompt agents. So how do the agents decide who to involve and, you know, what sort of how much of the context that they need to provide that person in order to do what they need to do?

• 12:32Speaker 00

  Yeah. Another interesting one. So this is similar to how humans work in in in in our industry is there is a process that we're tracking over 10 days. There's multiple actors involved. The the actor at step seven may not need to know about this just yet, right? They're pulled in at the right time. And so our agents with our kind of like long memory persistence plus the daily reflection on what is happening, reflect on where the the state of the world is at the moment. And they're able to pull in the right person at that time when it is when when there's a signal that that implies that that person must be pulled in. And all of this is

• 13:14Speaker 00

  work that we have done to operationalize the context of how this industry works. But not just that, but how does Arca want it to work? And what are our principles of of how work happens in this industry? And that is another advantage of us being sort of a a not just a SaaS company, we're actually not a SaaS company, we are, we are the firm ourselves. So we can standardize the way this work even happens, so that our agents can participate in standard ways that that work must happen.

• 13:45Speaker 01

  So earlier in the conference, Anthropic did a workshop about their managed agents infrastructure, and a lot of it, even the orchestration is becoming more automated now. So do you foresee kind of using that type of functionality? Because obviously it presents a danger in that the human is not necessarily always in the loop anymore. So have you had any thoughts about using that technology to automate even more of the process?

• 14:11Speaker 00

  I think we are very bullish on the models from from Anthropic and labs like OpenAI, but less so on on taking a dependency on a product like managed agents. Because managed agents, one of their their key things is how the models kind of persist and and contribute memory as to what is happening in the world and all the operational knowledge that they need to be need to use to to be successful. And so letting them own that part of the business is kind of like a a risk because owning that is owning your intelligence is kind of gives you the ability to impact the business. If you if you hand off the intelligence to another

• 14:56Speaker 00

  provider, you lose some of that control that you need to kind of like build something that is unique and tailored to your business to such a great degree that it can actually impact your business.

• 15:07Speaker 01

  And just on Anthropic, they've also recently released financial services. So they're partnering with a whole bunch of different firms. Do you think what they're doing is kind of competitive to what you're trying to do? Or do you see them as different things?

• 15:20Speaker 00

  Yeah, I think I think Anthropic getting into our kind of industry is is very validating for for our industry. But the way I think about it is that it is very different to be an a services firm that has access to AI tools versus a firm that was built to be AI native. The firm with AI tools requires their users to prompt the use the the agents kind of to to produce work or to do work. But here, like I said, in at Arca, agents prompt people more than people prompt agents. And it is a proactive kind of service that is running for them 24/7 and it's kind of like running the business in the most standard way that we that we expect at the level of service that we that we that we aspire to, which looks very different if

• 16:05Speaker 00

  you're if you're an AI, sorry, a company with AI tools.

• 16:09Speaker 01

  Yes, yes, okay. So a little bit more about on the security side. So can you explain like what in your processes runs inside a sandbox? And how do you verify like a change to a financial plan, you know, before anyone acts on it?

• 16:27Speaker 00

  Yeah, so the the way we use sandboxes is that the the sandbox is one component of our harness. Our agent isn't actually inside of the sandbox. It uses the sandbox to offload computation. And we've we've built systems to make it safe for code inside of that sandbox to access our data. So we kind of like have this thing which translates the code it wrote into a strict format that we expect, throw away the code and you kind of instill guardrails on that format before it reaches your backend. So we allow the agents kind of to use code to query our data, which means that they can like aggregate it, they can munch it, they can kind

• 17:11Speaker 00

  of like work with it as they as they see fit to produce such a thing like a like a financial plan. Inside of the same sandbox, they produce this, you know, beautiful artifact that we we can share with clients that has like it's a PDF of like, here's your financial plan and here's how we're doing planning to kind of like approach your life financially. And once that is produced is where we run our verifiers. So it's a certain type of artifact that is produced from the sandbox. The moment that is produced, we run a bunch of different different verifications on it. Some deterministic that check for presence of like, hey, did you just put like a raw account number in there? You should not do that. So deterministically, we

• 17:53Speaker 00

  can check that. But also, fast models like like the the smaller models or Jev to analyze for presence of things that might make this artifact noncompliant. Like, for example, you really should not be like promising returns in in that artifact. So that's another thing that we will check before it is it is kind of publishable. And the last thing I'll say on that is that doing this is actually makes our kind of compliance better than if humans were doing it. Because AI is tireless at at at finding even the smallest mention of a noncompliant piece of writing in there. So yeah, we'll continue to invest in in in more

• 18:37Speaker 00

  of that and kind of like constrain at the capability edges.

• 18:40Speaker 01

  Yes. Yeah, you mentioned Jev there. So you you've started to use decision models as part of your process. What other products other than Jev are you sort of looking at?

• 18:50Speaker 00

  Yeah. So what we've noticed is when when models are working with financial data, right? When they're answering questions that require financial knowledge, they are they're very confident in in in the knowledge that they have. So a small example is that models will very happily use the tax brackets from 2025 to to come up with an analysis and be like, all right, that is it. But that is that is obviously incorrect. And so that is one of the ways in which we're using decision models in in our harness is to to check when the the model is about to make a financial recommendation or or anything, any action that requires financial rigor will detect and will nudge the model to kind of use our

• 19:33Speaker 00

  knowledge base of verified facts about the world that is true today and or use their kind of web search. So it kind of makes it, this is kind of a a point about the the harness and and owning the harness that lets you do things like this that that help your your models be be accurate where it matters.

• 19:51Speaker 01

  And do you use decision models in, for example, recovery if like a trade fails to execute? Is there a decision model involved there? Or is there another path to to get the agent back on track?

• 20:02Speaker 00

  No, not today. That is not a use case that we're exploring today.

• 20:04Speaker 01

  Okay. So, you know, if a step, a trade does fail, for example, how does the agent recover at the moment?

• 20:12Speaker 00

  So, at the moment, we are not doing autonomous trading with agents. It is very much a, the agents will propose that we make trades.

• 20:23Speaker 01

  I see.

• 20:24Speaker 00

  And it is fully up to kind of our, our experts to figure out if that is the right trade to be making and to execute the trade.

• 20:31Speaker 01

  Okay. So on the, on the same sort of line of that, your slides say that every staff edit or cancellation comes with a why and come, it becomes an eval. So how does that feedback get turned back into the system as a measurable improvement?

• 20:50Speaker 00

  Yeah, that is another one of the advantages of us being the business and not a SaaS company. Is that we have, we have the ability to have, to require them to give us feedback. And yeah, the way that we loop the feedback back into improvements is something that you would, you would expect is all of that feedback gets piped into a data store that, that couples that feedback with the trace of the agent that produced it. And so we're able to kind of like take that, take that trace, put it, put it next to the, the feedback. And every night we run this process where we take, collect traces from the real world, from our agents in the wild.

• 21:31Speaker 00

  And run, run a meta agent on top of that suggests changes to our harness. So maybe tools that were misleading or things that were, were errored or there's missing knowledge in our knowledge base. And they'll surface this information to us and then we'll, we'll be able to kind of improve it. Today, we still have a, a human in, in, like a human engineer in the loop before we actually ship a prompt change. I've noticed that, that agents will, will make really nasty prompt changes if you leave them, leave them to it. Every little fix becomes the whole prompt. And they're, they're unable to stay meta and kind of generalize the fix into, into a prompt that generalizes.

• 22:13Speaker 00

  So, but yeah, we have, we have feedback loops from our product that we collect directly as first class product features to collect evals and turn them into improvements.

• 22:21Speaker 01

  Okay. And you also mentioned in your presentation about turning knowledge of how a firm works into agent context, which is actually something a lot of forward deployed engineers do, try to do is turn an organization's kind of internal knowledge into actionable things for agents. What does that look like for your company technically and how do you sort of keep it up to date?

• 22:46Speaker 00

  Yeah, so we have a knowledge base of, of files that is represented as, as, as files. And the way we keep that up to date is we, we've handed it to our experts to curate and update. So it is a, a first class feature in our product. So on the left hand side, there's a tab called knowledge, which has the full operational context that our agents are using. And as they, they work on, work on things, they're able to cite knowledge. And so it kind of like creates this, this loop of like, oh, we noticed that the knowledge is incorrect. And we, that gets surfaced to us in our trace analyses.

• 23:29Speaker 00

  And because it is a, a part of the product itself, people are able to go in and like edit the, edit the scale it directly. That's how it kind of works.

• 23:39Speaker 01

  All right, okay. So, you know, you mentioned you're, you're actually offering these services to, to clients through, you know, I presume you have a staff of financial advisors in your firm. So can you maybe give us an example of some, you know, a client that you've had that's had a measurable improvement in their investments or taxation or whatever through the agent system and maybe some sort of details about what in the agent architecture kind of helped that client, you know, get such a good return?

• 24:12Speaker 00

  Yeah, I'll give you a recent example. This is, this person wasn't a client yet. But they were, they were at the, the point of becoming a client. So we had all of their, their documents, their statements, their current investments, et cetera. And this person was already working with an advisor. And so they meet with us, they tell us that, hey, I have this advisor already. They're charging me fifteen hundred dollars for, for their services. And they're not very responsive to me. So I'm, I'm chatting with you. And so our agents have kind of like, you know, read through all of those documents. They've analyzed them. They've analyzed what the person's goals are that they state, right?

• 24:55Speaker 00

  They've also analyzed the statement of like who this current advisor is, right? And how much they're, they're charging them. And they discover that, that there is, there are fees that this person was charging them that they weren't aware of and surfaces it to them. And kind of like makes them not only aware of, of the fact that they're being overcharged, but also aware that that person wasn't even fiduciary. So they're not obligated to act in their interests. So it is, this is the sort of like attention to detail that our agents bring about in, in when they're analyzing a financial picture. That, that sort of would not have been possible, it's possible if we didn't have, have these.

• 25:39Speaker 00

  There's several examples of proactivity that our agents kind of like produce of like someone has a deadline coming up. Or they're at risk of missing their RMD or that kind of thing. And so this is difficult at scale for humans to keep track of, but it is the, the default behavior of, of our agents at, at Orca.

• 25:58Speaker 01

  Yeah, wonderful. Well, that's probably a good point to stop at. So thank you very much, Aman, for joining us.

• 26:04Speaker 00

  Thank you for having me.