Why Banks Are Struggling to Scale Generative AI and How to Fix It

Why Banks Are Struggling to Scale Generative AI and How to Fix It

Generative AI continues to dominate boardroom discussions across the financial sector, yet many banks remain stuck in experimentation mode. While proof-of-concept pilots are widespread, few institutions have translated those early efforts into enterprise-wide systems that deliver measurable value.

According to Fergonn Fernandez of New Rocket, the issue is not the technology itself. It is how organizations approach it.

The Pilot Trap

“Few organizations have scaled AI, because they ask ‘where can we use AI?’ rather than ‘what business problems are we solving’.”

Fernandez points to a common pattern across banks. AI initiatives often begin as isolated pilots that test feasibility in controlled environments. These projects rely on limited data, systems, and processes. They help organizations understand what is technically possible, but they rarely lead to meaningful business outcomes on their own.

“AI experimentation does not lead to measurable value without a vision, and that is what organizations should articulate first.”

The difference between pilot-stage efforts and enterprise-wide production is significant. Pilots are designed to explore possibilities within constrained conditions. Enterprise deployment reflects a broader organizational commitment.

“Pilot-stage AI efforts are materially different models from enterprise-wide production, as it answers first what is possible and not possible in a highly controlled and limited environment, sample sizing everything from data, systems and processes.”

By contrast, scaling AI requires alignment with a clear vision that guides execution.

“Whereas, enterprise-wide production should mean that the organization is executing on their AI vision, informed by successful AI pilots.”

Fernandez emphasizes that this progression must be intentional.

“In other words, the organization’s vision should inform that path to execution which means taking a measured pilot approach to prove out use case viabilities (people, process and technology) and then transition the ownership from proof of concept to full business ownership.”

Centralization and Speed

The debate over centralized versus decentralized AI models continues to shape how banks structure their efforts. Research from McKinsey suggests that centrally led approaches are more effective for scaling, but some worry that centralization can slow innovation.

Fernandez frames the issue differently.

“It’s not about building the best AI models, it’s about who can operationalize AI the fastest”

He notes that centralized models can accelerate scaling when they are designed to support teams rather than restrict them.

“Organizations scale the fastest through centralized AI models mostly because it is designed to help empower teams to execute their work autonomously, not from removing oversight.”

In this structure, central teams provide governance, tools, and standards, while business units retain the ability to move quickly within that framework.

Building Trust in Banking AI

For banks, scaling generative AI also requires addressing regulatory and risk considerations. Moving from experimentation to production depends on establishing trust in how AI systems operate and are governed.

Fernandez highlights accountability as the first priority.

“Clear AI accountability and ownership should be established first.”

He adds that transparency in decision-making is essential.

“Banks need explicit accountability to drive decisions and traceability from model outputs to business actions.”

Risk management must also be embedded directly into system design rather than added later.

“Second, design AI models with risk in mind.”

This includes safeguards that reduce misinterpretation and ensure human oversight.

“Embed risk controls to mitigate interpretation risk, implement human-in-the-loop controls and develop escalation paths upfront, while looking to continuously improve the governance process to drive a more fortified and trusted AI revenue-generating system.”

The Path Forward

As banks continue investing in generative AI, the gap between experimentation and execution remains a central challenge. Fernandez’s perspective suggests that success depends less on the sophistication of individual models and more on the ability to operationalize AI across the enterprise.

Organizations that define a clear vision, validate it through pilots, and commit to structured execution will be better positioned to unlock the full value of AI in banking.