Autonomy under control – How banks can build trust in AI

Across the world’s financial institutions, the appetite for AI is growing fast and hunger for change is real. But few, if any, banks trust AI to act unsupervised, and with good reason.

Trust in AI needs careful engineering through evidence, measurement and governance. Autonomous AI demands tight discipline, not blind faith. So, if banks are to build trust in AI, they need to prove they’re in control of their models.

Keeping humans in the loop isn’t always the answer

To date, many financial institutions have aimed to control AI-driven processes by requiring a person to approve every step. Although this “human-in-the-loop” approach may feel safe, it often creates rubber stamps that increase bottlenecks without adding true oversight. And as AI becomes more autonomous, in-the-loop simply breaks.

A more forward-thinking answer is to move humans from inside the loop to above it; that is, from doing and reviewing the work to designing the system that does the work. In the “human-above-the-loop” model, people set the standards, the guardrails and the verification regime, rather than personally checking every output. Essentially, the human contribution shifts upstream.

Autonomy concentrates accountability

When humans work above the loop, they own the standards by which AI systems run. They’re thinking about controls and outcomes when each process is being built, not inspecting it after the fact. That way, people retain genuine control, while the machines down below take care of the volume work. AI executes autonomously but inside people-prescribed boundaries, with humans monitoring from above, verifying continuously and intervening by exception.

Ultimately, accountability for AI should always reside with a named human owner. The more an agent can do on its own, the more precisely you need to know who owns its behavior. Accountability can’t sit with an AI model; it stays with the person who deployed it and the standards they signed up for. If you can’t name the owner, you shouldn’t run the agent.

Proof creates trust

You can only build trust in AI when you’re able to show your workings. You need to demonstrate how you’ve tested, measured and monitored a system, what happens when it fails, and how you put that right. But you also need to capture the model's reasoning itself in a “trace”; that is, a record of how the system arrived at an output, not just the output alone. In a trusted design, the trace is evidence you can inspect, challenge and stand behind.

My rule is simple: no measurement, no governance, and no governance, no deployment. If you can’t say how you’ll measure whether a model is working and safe, then it isn’t ready to run. Governance, however, shouldn’t be seen as the brake on AI. Done properly, it’s an enabler that lets you go fast without careering off the edge.

Data sovereignty and security are top priorities

For financial institutions, an ever-growing focus of governance is safe movement of data. Every model, integration and embedded AI capability is a new place where data can move, leak or leave a jurisdiction it shouldn’t.

Banks operate under heavy constraints on where data lives and who can touch it, and AI makes those flows harder to see. As a result, sovereignty and security have stopped being back-office concerns and are becoming front-of-mind design decisions. If you don’t know where your data is and who can reach it, you don’t have a governable AI estate.

Control comes from architecture

Potentially, the ongoing abstraction of AI – its translation of complex technical details into simple user-friendly concepts – could make it harder to govern. Teams are more likely to treat an abstracted model as a black box they don’t have to understand, which may breed complacency.

The best defense is not only to stay model-independent, so you’re not tied to one AI provider, but also to concentrate on the underlying IT infrastructure that will power – and, critically, control – your models. It’s there that a leading financial technology provider like FIS can help you establish the guardrails, the controls and the institutional credibility that AI models need to fulfil their potential.

Trust won’t arrive on its own

Technology providers sit in a privileged spot when it comes to engineering trust in AI. We can build the guardrails once into a unified, orchestrated infrastructure and let many financial institutions benefit, rather than every firm reinventing controls badly on their own. Then banks can adopt AI with control built in from the start, not bolted on after.

The winners in the AI arms race will be the institutions that recognize they can’t build trust in AI alone. They will act decisively, but inside discipline. They will treat AI adoption as a company-wide change, not an IT program. And above all, they will balance the growing autonomy of AI with unfailing human oversight in a technology architecture that makes trust a foundational requirement.