The Regulator's AI Tightrope, From Hallucinations to High-Confidence Decisions

CTRL AI DEL is Dailoqa's weekly, no-nonsense look at what's actually happening with AI in financial services. This issue looks at AI governance in financial services, why regulators remain cautious, and what it actually takes to earn their confidence.

AI governance in financial services is not about winning permission to use AI, it is about earning the credibility to use it at scale. Regulators remain cautious because current models can be opaque and inconsistent. Deterministic, agentic AI offers a more auditable alternative, but governance, accountability and continuous monitoring still have to be designed in from the start, not added afterwards.

The Regulator's Dilemma

Financial regulators globally remain understandably cautious about embracing AI, especially for anything that involves actual clients. Their reluctance largely stems from lingering trust issues with current Large Language Models. These systems, according to this view, have a habit of lacking clarity, occasionally hallucinating, producing confidently incorrect information, and offering inconsistent answers without explaining why. Black box decisions do not sit comfortably alongside people's money, and that leads to legitimate concerns about financial harm, legal exposure and reputational damage. It is a fair concern, and one worth taking seriously rather than dismissing as regulatory caution for its own sake. The institutions that treat this scepticism as a design constraint, rather than an obstacle to route around, tend to end up with systems that are genuinely easier to defend later.

This caution is not just anecdotal. The UK's Financial Conduct Authority has been explicit that its approach to AI remains principles based and outcomes focused, and it has said publicly that it does not plan to introduce a separate set of AI specific rules, choosing instead to apply the frameworks that already cover the risks AI introduces¹. That stance sounds reassuring until you consider what it actually means in practice. A firm's obligations do not change simply because a decision was made by an algorithm rather than a person, and the same standards of fairness, suitability and treating customers fairly still apply in full. For risk and compliance teams, that expectation carries the same weight it always has, now applied to a considerably less familiar kind of decision maker.

The Case for Deterministic, Agentic AI

There is, however, a more disciplined alternative taking shape in the form of Agentic AI, particularly the deterministic kind. These are systems built to give the same answer to the same question every time, rather than generating a plausible-sounding response on the fly. Four things make this approach genuinely more workable in a regulated environment.

No more surprises. Deterministic agents significantly reduce the odds of an AI going off script and inventing conclusions, leading to far more accurate and dependable outputs.

Better explainability. Unlike more opaque systems, deterministic agents make it possible to see why a decision was made, which matters considerably when regulators come asking.

Compliance built in. These agents can have rules, regulations and updates woven directly into their code, making compliance close to automatic rather than a separate check performed afterwards.

Consistent automation. They can handle a large volume of repetitive tasks with consistency, freeing up human professionals for the more complex and less predictable work that genuinely needs judgement.

None of this makes deterministic agents a silver bullet. They are simply easier to reason about after the fact, which is precisely what regulators are asking for. UK regulators have already flagged audit trails and human in the loop protocols as live issues, with further guidance expected during 2026. A system that can show its working, step by step, is easier to defend under scrutiny than one that produces a confident answer with no visible reasoning behind it. That is not a small distinction. It is the difference between a decision a firm can stand behind and one it can only hope was right.

What It Takes to Win Regulators Over

To convince regulators to genuinely embrace AI, a few things need to happen at the same time, not in sequence.

Solid governance and ethics. Clear frameworks for accountability and ethical AI use, not a policy document that exists mainly to satisfy an audit.

Explanations that actually explain. Tools that make AI decisions genuinely understandable, not just to technical specialists, but to anyone who needs to grasp the why behind a decision.

AI literacy across the board. Regulators and financial professionals both need to become more fluent in how these systems actually work. Familiarity builds trust faster than reassurance does.

Regulation that keeps pace. Guidance that adapts as quickly as the technology itself, ideally with a degree of international coordination to avoid a fragmented patchwork of rules.

The clock on this is not theoretical. Under the EU AI Act, credit scoring and creditworthiness assessment are explicitly classified as high-risk activities, which means the full weight of risk management, human oversight and conformity assessment obligations applies from 2 August 2026. Any bank or lender running an AI-assisted credit decision in an EU market is already working against that date, not planning for some distant future one. The practical effect is that governance can no longer be a project that starts once a system is built. It has to be part of how the system is designed and documented from the first line of code, because retrofitting audit trails and bias testing onto a live production model is a far harder job than building them in from day one.

The Hurdles That Remain

Even with strong governance and deterministic design in place, a few more pieces of the puzzle need to fall into place before the more cautious voices in the room are genuinely satisfied.

Who is accountable when AI gets it wrong

There is still the open question of legal liability. When an AI system makes a financial error, pinpointing responsibility, whether that sits with the developer, the deployer, or the human who signed off on the process, remains a live legal question rather than a settled one. In the UK, this question already has a partial answer, and it is not a comfortable one for individual executives. Under the Senior Managers and Certification Regime, authorised firms are expected to be able to show what level of assurance a senior manager had over an AI system before it caused harm, and regulators have signalled that guidance clarifying exactly what that assurance should look like is coming by the end of 2026. Until that guidance lands, the safest assumption for any bank deploying AI in a client-facing process is that the accountable executive will be asked to explain, in detail, what they knew, when they knew it, and what they did about it. Vague answers will not satisfy a regulator, and they should not satisfy a board either.

Regulators are building their own AI capability

For regulators to genuinely trust AI, many are concluding they need to start using it themselves. Supervisory technology, commonly shortened to SupTech, is the term for regulators using the same categories of tools, AI, machine learning and large scale data analytics, to monitor the institutions they oversee. According to the World Economic Forum, SupTech adoption has accelerated globally as supervisors look to close the oversight lag, spot systemic risk earlier, and move from reacting after the fact to monitoring in something closer to real time. There is a useful symmetry in that. A regulator that understands agentic AI because it runs agentic AI of its own is better placed to ask a bank the right questions, and less likely to be persuaded by an explanation that sounds sophisticated but does not actually hold up.

Models drift, so oversight has to be continuous

Even the most deterministic agent needs a watchful eye over time. This is not a hypothetical risk. A model trained on one economic environment can start producing subtly different outputs as market conditions shift, transaction patterns change, or the population being scored evolves, and none of that shows up as an obvious failure. It shows up as a slow drift in accuracy that is easy to miss without deliberate monitoring in place. Treating model risk management as a one-time certification exercise, rather than a continuous discipline, is one of the more common gaps regulators are now looking for when they review AI governance frameworks.

Guardrails for the guardrails

AI and human errors and limitations are not mutually exclusive, and one can help offset the other to produce more consistent and accurate results. We have made a related point earlier in this series, using the idea of a dial rather than a switch for how much autonomy an agent is given. The same principle applies at the level of the whole governance system. Full automation without a human check is not a viable answer, and neither is full manual review of everything an AI system does, since that simply reintroduces the fatigue and inconsistency problems humans were meant to help solve in the first place. The answer sits somewhere in between, calibrated to the risk of the task, and revisited as evidence accumulates about where each system tends to get things right and where it does not.

None of this is an argument for slowing down. It is an argument for building AI systems that regulators, boards and customers can all interrogate without needing a translator in the room. The institutions that treat governance as core design work, rather than paperwork bolted on afterwards, will be the ones that move fastest once regulators are satisfied, because they will not need to go back and rebuild anything to prove it. That is the real prize on offer here, not permission to use AI, but the credibility to use it at scale.

6. FAQ

What is AI governance in financial services?

It is the combined set of frameworks, controls and accountability structures that ensure AI systems used in banking, lending or insurance operate fairly, transparently and within regulatory expectations.

Does the FCA have AI-specific rules?

Not currently. The FCA applies its existing principles-based, outcomes-focused framework to AI rather than introducing a bespoke AI rulebook, though guidance on specific issues such as audit trails continues to develop.

Who is accountable for AI under SM&CR?

The senior manager responsible for the relevant business area is expected to be able to demonstrate the level of assurance and oversight they held over an AI system, particularly where it caused harm.

How do you make AI auditable in banking?

By using deterministic agents where possible, logging the reasoning and data behind each decision, and building explainability tools that let both technical and non-technical reviewers understand why a decision was made.

What is SupTech and why does it matter?

SupTech is supervisory technology, AI and data tools used by regulators themselves to monitor financial institutions. It matters because a regulator that understands AI first-hand is better equipped to supervise it in others.

What does the EU AI Act mean for banks using AI?

Credit scoring and creditworthiness assessment are classified as high-risk under the Act, triggering mandatory risk management, human oversight and conformity assessment obligations from 2 August 2026.

What is model drift, and why does it matter for AI governance?

Model drift is the gradual change in an AI system's accuracy as real-world data shifts away from what it was trained on. It matters because it can quietly erode reliability long after a system has passed its initial review.

Glossary

• AI Governance, the frameworks, controls and accountability structures that keep an organisation's AI use fair, transparent and compliant.

• Deterministic AI, AI components built to produce consistent, rule-bound outputs rather than open-ended generation.

• Explainable AI, AI designed so that the reasoning behind a decision can be understood and communicated, not just the output itself.

• SupTech, Supervisory Technology, the use of AI, machine learning and data analytics by regulators to monitor the institutions they oversee.

• Model Drift, the gradual decline in an AI model's accuracy as the data it encounters in production diverges from its training data.

• SM&CR, the Senior Managers and Certification Regime, the UK framework that holds named senior individuals accountable for specific areas of a regulated firm's conduct.

• High-Risk AI (EU AI Act), a formal classification under the EU AI Act that triggers mandatory risk management, oversight and documentation obligations for specific AI use cases, including credit scoring.

References

1. Financial Conduct Authority, AI and the FCA, our approach, 2026.

2. https://www.fca.org.uk/firms/innovation/ai-approach

3. FluxForce, EU AI Act Article 6, High-Risk AI Requirements and Penalties, 2026. https://www.fluxforce.ai/regulations/eu-ai-act-article-6-high-risk

4. RR Compliance, AI Governance Policy for FCA Authorised Firms, 2026 Guide. https://www.rrcompliance.com/post/ai-governance-policy-fca-authorised-firms

5. World Economic Forum, Suptech Can Boost Resilience, Transparency and Accountability, December 2025. https://www.weforum.org/stories/2025/12/financial-regulators-technology-protect-strengthen-financial/