Banks are discovering that their biggest artificial intelligence expense may not be software licenses or server racks, but the tiny units of computation—known as tokens—that power AI agents. According to a new analysis from McKinsey, those token costs could eventually consume as much as 20% of what banks spend on labor, forcing lenders to rethink how they deploy the technology.
Tokens are the chunks of text that AI models process to generate responses. Every query, every step in a multi-step workflow, and every conversation between two AI agents consumes tokens. For banks, which are increasingly using AI agents to handle customer service, fraud detection, and back-office tasks, the bills can add up quickly.
Why token costs are spiraling
McKinsey, the management consulting firm, says controlling AI spending is becoming a job in itself. Some banks are forming dedicated “AI FinOps” teams—a play on the “FinOps” discipline used to manage cloud computing costs—to track token usage and rein it in. The challenge is that AI agents don’t just answer a single question. They run complex, multi-step workflows, and they can even interact with other agents, so token consumption piles up fast.
That complexity breaks simple cost-control measures like “route every request to the cheapest model.” A task that starts on a low-cost model might escalate to a more expensive one as it grows more complex, or an agent might loop back and forth with another agent, each exchange burning more tokens.
Banks are responding by being choosier about where agents run. They are approving projects with clearer payoffs, and they are asking not just whether the technology saves time, but whether it can lift revenue enough to justify an ongoing token bill. The pressure is rising as customer behavior shifts: McKinsey expects retail-bank agents to be widely used within three years, and its survey of 46,000 retail banking customers found many already use AI to compare banks, hunt for higher-yield savings accounts, and explore refinancing or debt consolidation.
What it means for investors
For investors, the bigger market story might be banks’ funding costs, not their tech budgets. If customers use AI to scan for better savings rates, “shopping around” gets easier and deposits become less sticky. That can push up deposit beta—how quickly banks have to raise the rates they pay savers when competitors move—squeezing net interest margins even if loan yields don’t change much.
McKinsey found that 72% of AI users would let an agent complete one banking task, but many would cap autonomous approvals at $500. That suggests trust may build step by step. As banks gradually loosen today’s blocks on agent-authorized transactions, deposit pricing could start to matter more like a fast-moving product market, reshaping which retail-heavy banks look advantaged from quarter to quarter.
For now, many banks still block outside agents from authorizing transactions, which limits how fast these tools can move money and squeeze profits. But the trend is clear: AI is not just a cost center—it’s a competitive force that could change how banks price deposits and compete for customers.
Investors should watch how banks manage their AI budgets, as those costs could eat into earnings if left unchecked. At the same time, the shift toward AI-driven deposit shopping could benefit banks that are quick to adapt their pricing strategies, while hurting those that are slow to respond.
Related: SoftBank's AI data center costs show how quickly AI spending can balloon. And Italy's banks and bonds are colliding as UniCredit nears a Commerzbank deal, a reminder that banking margins are under pressure from multiple angles.
