The AI Unit-Economics Ledger: Why Cheaper AI Costs More

Your AI got cheaper. Much cheaper. And your bill did not go down.

That is not a billing error, and — this is the part most banks get wrong — it is not a cost problem either. It is a forecasting problem. And a forecasting problem is worse, because a cost problem can be budgeted. You can decide to spend more. A forecasting problem cannot be budgeted, because there is no number to put in the budget.

Why this matters now

Something changed in 2026 that makes this urgent rather than interesting. Gartner forecasts that worldwide spending on inference will reach twenty-three point three billion dollars in 2026 and surpass spending on training, at nineteen billion dollars — the firm's own framing is that inference workload spending overtakes training spending in 2026, and that fifty-five percent of AI-optimised infrastructure spend goes to inference this year, rising to fifty-nine percent next year. AI-optimised infrastructure-as-a-service is forecast to grow ninety-six percent to forty-two billion dollars.

That crossover is the moment unit economics start to bind. While the spend was concentrated in training, AI cost was a capital question asked once. Now it is an operating question asked every day, by a bank that approved its pilots on a per-call estimate and is moving agents into production.

Most banks fail here for a single reason: the business case they approved described a price, and the invoice they receive describes a loop.

Start with the part that is true

Frontier model prices really have collapsed. On the BenchLM Token Price Index — a third-party tracker of forty-one models across three tiers, blending input and output at a published three-to-one ratio in favour of input, which is the side of the ledger that actually drives an agentic bill — the frontier sub-index reads 18.7 for September 2026 against a March 2023 base of 100, roughly eighty percent below where it started (data from BenchLM.ai).

Note what the same index also shows. The frontier sub-index bottomed out at 6.4 last winter and has risen since. The cheap era arrived, and then the frontier moved: each new generation of capability enters the basket at a higher price than the one it replaced, which is Gartner's paradox restated in market data rather than in a forecast.

So the cheap-AI story is real. The collapse happened, it is large, and this article is not going to argue with it. What it disputes is the inference a business case draws from it.

Here is where it breaks. That figure describes the price of a token. A bank does not buy tokens. It buys completed work: a reconciliation finished, a case resolved, a document checked, an exception cleared. And those two numbers have come apart.

Gartner named the mechanism in August 2026 and called it the Inference Paradox — better unit economics subsidise more complex workflows, so total cost climbs even as each unit gets cheaper. Their forecast is that inference cost per agentic workflow rises more than fivefold through 2028. Their senior director analyst Will Sommer put the consequence plainly: "Product leaders cannot rely on more efficient token economics to rationalize AI costs."

This matters for how you read the rest of this piece. The claim here is not ours alone and not contrarian for its own sake — a Tier-1 analyst house has already stated it. What follows is the operational half: the four lines a bank has to forecast, and the refusal rule that makes them stick.

The loop, and why the two numbers separated

A chatbot receives a question and writes an answer. An agent does something structurally different. It plans. It calls a tool. It reads the result. It checks its own work. It retries. And on every single turn of that loop, it re-reads everything it has already done.

That last sentence is the whole story, and it inverts the intuition. Researchers including Erik Brynjolfsson and Alex Pentland measured this directly in April 2026: agentic workloads consumed roughly one thousand times more tokens than a code-chat or code-reasoning task, and — critically — input tokens rather than output tokens drove the overall cost (Bai et al., arXiv:2604.22750).

The agent is not expensive because of what it writes. It is expensive because of what it keeps re-reading. And the same paper severs spend from value directly: higher token usage does not translate into higher accuracy — accuracy peaks at intermediate cost and saturates above it. Paying more does not buy a better answer.

A second study asked where inside the work those tokens actually go. Measuring token consumption across the phases of agentic software engineering, it found the iterative review stage accounted for an average of fifty-nine point four percent of tokens — the machine checking and re-checking its own output (Salim et al., Tokenomics, arXiv:2601.14470). That is a preliminary finding, averaged across thirty software-development tasks in a single multi-agent framework on a single model, so read the magnitude as a direction rather than a constant. What the same paper also does is corroborate the mechanism from an independent direction: it measures input tokens at fifty-three point nine percent of consumption — a different framework, a different model, a different research group, and the same finding that the cost sits in what the agent reads. Generation was comparatively cheap. Verification was not.

So most of the money is not spent doing the work. It is spent confirming the work is acceptable.

Now ask what your last approved AI business case forecast. Almost certainly a price per call multiplied by expected calls. Neither number has any awareness that the loop exists.

The finding that actually matters

The multiplier is not the worst result in that research. This is.

Return to the token-consumption study for the finding that matters most to an approver. Bai and colleagues ran the same task more than once — not variations, the identical task repeated. Token consumption varied by up to thirty times.

Sit with what that means for a bank. The invoice for a given piece of work is not a figure. It is a distribution. The same reconciliation, run on Tuesday and again on Thursday, can differ by a factor of thirty, and nothing in your change log will explain why.

And before reaching for the obvious answer: you cannot ask the system. The same researchers tested whether frontier models can predict their own token consumption before running a task. The correlations were weak to moderate — up to zero point three nine — and the models systematically underestimated their real costs. The model does not know what it is about to spend either, and it errs in the direction that breaks a budget.

This is why the usual instinct fails. The instinct is to negotiate a better rate. But the rate was never the issue; the collapse in the rate already happened, and it did not fix the bill. What a bank cannot currently do is predict the quantity.

The standard that defines only half the problem

There is a standard for AI cost data, and it is worth understanding precisely because of what it does not do.

The FinOps Open Cost and Usage Specification — FOCUS — standardises billing and usage dataset schemas across AI, cloud, SaaS and data-centre providers. It defines columns, not key performance indicators. AI consumption travels through generic fields for consumed quantity and unit, so a token and a request and a seat all become comparable line items whoever billed them.

What it does not define is cost per completed task. It never claimed to: unit-cost analysis is explicitly a practitioner exercise built on top of the normalised data. The specification's own public repository carries an open issue recording that practitioners still lack worked examples for mapping token consumption into those columns — a documented gap inside the standard rather than an absence of evidence.

The formulation worth remembering: the industry standardised the numerator. The denominator is yours to build.

That is the entire reason a ledger is necessary. No supplier and no standards body is going to hand a bank its cost per resolved case.

The AI Unit-Economics Ledger

Four lines. Not six — four, because a chief financial officer can hold four lines in their head, and a template nobody completes is not a control.

They are ordered by when a bank finds out about them: four gates, in the order an institution actually passes through them. Approval, integration, production, audit. That ordering is the framework's diagnostic value — you can read your own maturity off how far down the list you have got.

L1 · Inference

The model call itself: input, output, cached input, reasoning tokens where they are billed separately, and the difference between list, committed-use and negotiated pricing.

This is the line your business case already contains. It is also the line whose unit price has fallen furthest, which is exactly why it flatters the forecast. Found at approval.

The control is routing: assign each task to the cheapest model that can actually complete it, rather than defaulting to the most capable one.

L2 · Retrieval and context

Embedding generation, re-embedding when documents change, vector storage and query cost, re-ranking, and the context payload resent on every single call.

Here is why this line hides so well: it scales with the size and churn of your corpus, not with the number of users. Double your indexed documents and this line doubles with no new users at all — so it appears nowhere in a per-user or per-seat business case, which is the shape almost every case takes. Found at integration.

The control is treating corpus growth as a cost decision with an owner, and measuring re-embedding as a recurring event rather than a one-off migration.

L3 · Orchestration

The multiplier. Planning steps, tool calls, retries, self-checks, handoffs between agents.

Everything in the first half of this article lands here: the thousandfold consumption, the concentration of spend in review, and the thirty-times variance. This is where non-linearity actually lives, and it is the least modelled line in the ledger. Found in production, which is the worst possible place to find it.

The controls are real and they work — prompt caching has measured support in the literature (arXiv:2601.06007), and model routing is Gartner's own recommended remedy. Both need funding as engineering disciplines with named owners, not goodwill.

L4 · Run, assure and review

Three limbs, one budget conversation.

Run is observability, tracing, drift and quality monitoring, incident response. Assure is the judge models, groundedness scoring, and the regression and red-team suites re-run on every release — a recurring cost, not a build cost. Review is the human part: exception handling, escalation, disputes and appeals.

Review is named deliberately, and it is the limb most often deleted from a business case. Because review is where the headcount saving your case promised either survives or quietly disappears. Found at audit.

For a Vietnamese bank there is a compliance dimension that lands squarely on this line. Vietnam's AI Law 134/2025/QH15 — passed 10 December 2025, in force 1 March 2026 — carries duties of transparency and of incident detection and remediation, and, for systems classified high-risk, duties of human oversight and human intervention. Article 35(1)(a) gives an eighteen-month transition, to 1 September 2027, only to AI systems already in operation before 1 March 2026 in finance, healthcare and education — a new deployment does not inherit it. The impact-assessment duty a bank actually carries is the personal-data regime's, not the AI Law's: PDPL Article 21(1) requires a personal-data processing impact assessment filed within sixty days of first processing, unconditionally, and updated every six months on change. All of these are recurring operating costs rather than a one-off compliance project, and human oversight is simultaneously a legal duty and the Review limb. That coincidence is the clearest argument for naming the limb rather than absorbing it into infrastructure.

Note also what the transition does and does not defer. It defers compliance obligations on systems already running — not the cost of building the capability, and not anything at all for a system a bank stands up today. Conflating a transition with a holiday is a category error worth avoiding.

Separately, PDPL 91/2025/QH15 and its implementing decree govern cross-border data flows, and those provisions are the clearest driver of topology cost we can identify: they can force a more expensive deployment than a public API. That is an L1 and L2 architecture cost, and it is the one place where Vietnamese law changes the bill rather than the paperwork.

Did the headcount saving arrive?

This is the question a chief financial officer actually asks, and it is where the evidence is thinnest and loudest at the same time.

The widely-cited cases — the ones about replacing hundreds of service agents — trace back to the deploying companies' own announcements and to their technology suppliers, presented as forward-looking projections rather than audited results. Where those projections have later been revisited, the revision has run downward, and none has been restated as an audited reduction in a regulated process's cost base.

The public record does not support a sustained net reduction in the cost base of a regulated banking process through headcount substitution.

The mechanism explains why, and it is more useful than any single case. When you automate the straightforward path, what remains is the hard residue: the disputes, the exceptions, the appeals. That work needs more expensive people than the entry-level triage it replaced. The cost did not vanish. It moved into line four, where nobody forecast it.

The forecast audit: a page you can fill in this week

Take one approved AI business case. Any one.

1. Find L1. It will be there.
2. Ask whether L2 was forecast against corpus growth, not user growth. If the case contains a per-user number, it was not.
3. Ask for the L3 multiplier — model calls per user request, measured rather than estimated. If nobody can produce it, the case has no denominator.
4. Ask what L4 costs per release and per exception, with the three limbs separated. If Review is missing, the headcount saving is unverified.
5. Ask for the variance, not the average. A single number for a workload with thirty-times spread is the defect, not the starting point.
6. Assign each line a named owner who answers for the forecast.

When only line one is forecast, leaders should send the case back. Not because the bill is too large, but because it is not a forecast at all — it is an estimate wearing a forecast's clothes.

Risks and caveats

Where this thesis does not hold. A bank with genuine engineering maturity — disciplined inference tiering, cache-aware agent architecture — can make falling per-unit prices outrun orchestration bloat. The controls work. If you can operate them at production scale, the ledger becomes a monitoring instrument rather than a survival tool.

So the binding constraint may be talent, not price. Very few traditional commercial banks currently run dynamic routing gateways and cache-aware architectures in production. That is the honest limit of the argument — and it is also the strongest reason to adopt the ledger, because a bank that cannot yet engineer the optimisation must at minimum measure the exposure.

On scope. The strongest measurements available — the thousandfold multiplier, the review concentration, the thirty-times variance — were taken on agentic coding tasks, and narrower than that: the thousandfold multiplier and the thirty-times variance were measured on eight frontier models against a software-engineering benchmark, and the fifty-nine point four percent figure is a preliminary result from a single multi-agent framework on a single model. Treat it as a direction, not a constant. The mechanism generalises; the specific magnitudes should not be assumed to transfer unchanged to a credit file or a reconciliation. Treat them as the documented shape of the problem, not as your bank's numbers.

On what a bank can know. Of the four lines, only inference and retrieval storage can be estimated from public evidence today. Orchestration, assurance and human review are knowable only from your own telemetry. That is not a gap in this article; it is the finding.

Key takeaways

• Model prices fell roughly eighty percent from their March 2023 base. Bank AI bills did not follow, because a bank buys completed work rather than tokens.
• Gartner calls this the Inference Paradox and forecasts inference cost per agentic workflow rising more than fivefold through 2028.
• Agents are expensive because of what they re-read, not what they write: roughly a thousandfold token increase over a code-chat or code-reasoning baseline, input-driven — and higher token usage does not buy higher accuracy.
• Nearly sixty percent of agentic token spend goes to the machine reviewing its own output.
• Runs on the same task can differ by up to thirty times in total tokens — so the invoice is a distribution, and the models cannot predict their own consumption either.
• FOCUS standardises the numerator across vendors. The denominator is the bank's to build.
• Four lines, four gates: inference at approval, retrieval at integration, orchestration in production, run-assure-review at audit.
• Review is the limb everyone deletes, and the one that decides whether the headcount saving was real.
• One refusal rule: approve nothing that forecasts only line one.

Figures in this article, and where each one comes from

Deliberately absent. This article carries no bank technology-budget figure, no vector-database or observability unit pricing, no accounting-standard citation, no transaction-threshold figures from banking circulars, and no named bank. Each was offered by the research behind this piece, each was checked against a primary source, and each was excluded because the source would not carry the claim. A figure that cannot carry its conditions does not appear.

No customer, employee or institution-level data of any kind appears in this article, no bank is named and no model is named at a price, and the ledger is the author's proposed decomposition rather than a ratified standard. This is independent thought leadership, not affiliated with any current or past employer, prepared in line with Vietnam's AI Law 134/2025 and the personal-data regime, and not a substitute for your own legal, compliance, finance and data-protection review.