The token paradox

During one of my first experiments with agentic AI, an afternoon spent running agents in loops over real code, I consumed five hundred euros' worth of tokens. The figure made me wince at first. In hindsight, that afternoon foreshadowed on a small scale what finance departments are now seeing on their bills: an agent at work consumes on a scale that bears no comparison with what a human consumes by conversing, and that work is exactly what is being bought.

At equal performance, the price of inference is collapsing

The paradox involves two opposing movements. The first is the fall in the price of inference, meaning the cost for a model of producing an answer. That price is counted in tokens, the word fragments into which the model splits the text it reads and writes, and which providers bill per million. At a given performance level, it falls very fast. Epoch AI, an independent institute that measures the quantitative trajectories of AI, put this decline in September 2026 at about 47% per quarter since 2023, a factor of thirteen per year^[1].

The pace varies with how old the targeted performance level is. The decline reaches a factor of seventy-five per year when a performance level first becomes state of the art, then slows to a factor of 4.7 two years later^[1]. The price of the most recent capabilities is therefore the one that falls fastest.

The MIT FutureTech team led by Neil Thompson reaches a similar order of magnitude in the revised version of its study, published in March 2026: five to ten times per year for frontier models, of which about a factor of three comes from algorithmic progress alone^[2]. The phenomenon had been named in late 2024, when Guido Appenzeller, a partner at Andreessen Horowitz, coined “LLMflation”, a portmanteau of LLM (large language model) and “inflation”^[3]. Summer 2026 even prompted talk of a price war: at the end of July, OpenAI cut the price of GPT-5.6 Luna, the cheapest model in its range, by 80%, and that of GPT-5.6 Terra, its mid-tier model, by 20%; in August, Anthropic dropped the increase it had announced for Sonnet 5, which stayed at $2 per million input tokens^[4][5].

A deflation with blind spots

These measurements hold performance fixed. Companies, however, want the best model available, and that tier costs more: according to the MIT team, the price of running frontier models rises by three to eighteen times per year, driven by larger models and longer reasoning^[2]. At Anthropic, Fable 5.1 is priced at $10 per million input tokens, against $4 for Opus 5.5^[5]. The way tokens are counted has changed as well. Since Claude 4.7, the tokenizer, the tool that splits text into tokens, produces about 30% more of them for the same text^[5], and OpenRouter measured an effective cost increase of 12 to 27% on real requests, excluding very short ones^[6]. The listed price per million tokens is therefore a poor guide to the price of a task.

And yet the bills keep rising

The second movement concerns budgets. If the cost at equal performance falls five to thirteen times per year, budgets should shrink. They do the opposite. In September 2026, Gartner forecast worldwide AI spending of $2.7 trillion for the year, up 49.5%, with growth of 117% for spending on generative AI models alone^[7].

Bain quantified the mechanism over twelve months: between December 2024 and December 2025, the cost of tokens was cut in half and their consumption multiplied by 4.5, which more than doubles the bill^[8]. In its survey of large enterprises published in January 2026, Andreessen Horowitz found that their average spending on language models was expected to grow by a further 65% over the year^[9].

The economist William Stanley Jevons had described this mechanism as early as 1865. As the steam engine became more economical with coal, England burned more of it, because the lower cost of use opened up applications that had been too expensive. In January 2025, when DeepSeek shook the markets by claiming to have trained a high-performing model for a fraction of the accepted cost, Satya Nadella, Microsoft's chief executive, invoked the same paradox to predict a surge in usage^[10]; twenty months later, the 2026 figures prove him right.

On OpenRouter, agents now consume more than humans

The growth in volumes has an identifiable engine: agents. According to OpenRouter, an aggregator that gives access to the models of many providers, tokens consumed by agentic systems overtook those consumed by humans around February 1, 2026, based on traffic of more than 450 trillion tokens observed between January and mid-June^[11]. According to Peter Walker of OpenRouter, agentic consumption has grown fourteenfold since that crossover, against a factor of 2.8 for human use, and about 70% of agentic tokens come from cached contexts, that is, contexts kept by the provider so that they can be reread at lower cost^[12]. Falling prices made agents affordable, and their consumption grows much faster than prices fall, even though caching absorbs a large part of it.

The agent rereads everything at every step

An agent consumes so much because it rereads its context at every iteration: the open files, its tools' results, the session history. That context grows as the task advances, and each turn rereads all of it. The documentation for Claude Code, Anthropic's agentic coding assistant, states that a team of agents consumes about seven times more tokens than a standard session when the teammates work in plan mode, each maintaining its own context window^[13]. For development tasks, a study published in April 2026 by researchers from the University of Michigan and Stanford, including Erik Brynjolfsson, measures consumption on the order of a thousand times higher, on average, than a conversational exchange about code^[14].

The same study reports three findings that bear directly on budgets. On an identical task, the total number of tokens consumed by the same agent can vary by up to a factor of thirty from one run to another, the most expensive run typically costing twice the cheapest. Models predict their own consumption poorly and systematically underestimate it. Finally, accuracy often peaks at intermediate cost, beyond which additional tokens no longer improve it. An employee whose cost could double, or even be multiplied by thirty, for the same task, with no way of predicting it, would quickly worry a finance department.

The study covers only eight models and a single code-repair benchmark (SWE-bench Verified), but it matches what teams observe: some of the tokens improve the result, others add nothing, and it falls to the system's architecture to tell them apart.

The budget becomes an architecture parameter

If token volume affects both quality and cost, deciding how much an agentic system may consume, and where, amounts to designing its architecture. Several settings have to be fixed: the number of adversarial checks per deliverable (reviews entrusted to a second agent whose job is to challenge the first one's work), the number of iterations after which the agent hands control back to a human, the model assigned to each tier of the system, and the share of the context that is reread in full or summarized in a memory. Each of these choices shifts quality and the bill at the same time.

The price gap between models gives these choices weight. BCG estimates that a frontier model costs five to twenty-five times more per token than a simpler one, and sorts optimization levers into five families: dropping unnecessary calls, routing each request to the suitable model, caching, capping consumption and training employees to use AI in relevant ways^[15]. Caching shows how large the possible gains are. At Anthropic, rereading a context that is already cached costs a tenth of the normal price, and up to one-fortieth on some recent models^[16]. These gains depend on system design, over which the purchasing department has no influence.

Cost management has taken up the subject within two years. According to the State of FinOps survey published in February 2026 by the FinOps Foundation, which brings together cloud cost management practitioners, 98% of the teams surveyed now manage AI spending, against 63% in 2025 and 31% in 2024^[17]. In July 2026, Gartner observed that enterprise AI budgets were coming “under greater scrutiny, with increased focus on usage efficiency, cost control and measurable outcomes”^[18].

What I apply in my own harness

For several months I have been designing an agent harness for pre-sales, meaning the set of rules, tools and specialized agents that frame Claude Code's work on commercial proposals, including costings and presentations. The rules that have weighed most on the bill concern what the agent reads. Each deliverable has its own session, so that the context does not carry the exchanges of another topic. Bulky reading, whether a client's reference documents or web monitoring, is handed to a secondary agent, and only its summary comes back to the main session. Any file longer than twenty thousand characters is produced by a script that the model writes once and then runs, which avoids writing the whole file into the session context. The visual check of a presentation covers a sample of pages.

The choice of model follows the same concern: concentrate spending where an error would cost the most. The review agents work with a high reasoning effort (the setting that determines how many tokens the model spends thinking before it answers), and only the adversarial reviewer, whose role is to challenge every figure by cross-checking it against its source before a deliverable is handed over, uses the maximum setting. To put orders of magnitude on this, Anthropic reports, across its enterprise deployments, an average cost of about $13 per developer per active day of Claude Code use, and under $30 per day for 90% of users^[13]. These amounts remain modest: on average, a day of Claude Code use costs less than an hour of a consultant's time, and the bill depends mostly on where the tokens are placed.

What to take away

I draw three convictions for the leaders who budget agentic AI.

The AI budget is managed like production capacity. The useful indicator is the cost per verified outcome, which relates consumption to a deliverable whose quality has been checked. When the ambition of the outcome rises, consumption rises with it, and the monthly bill alone does not show whether the increase is justified.

The token budget belongs in architecture documents, alongside latency or availability. A team that cannot say where its system's tokens go cannot steer it, nor can it rely on the agent, which, as the study cited above shows, predicts its own consumption poorly.

Postponing a use case to save money is paid for in learning delay. Falling prices make waiting tempting: a capability set aside today will cost about ten times less in a year. Yet the competitors who have learned in the meantime how to place their tokens will keep their lead, because that know-how is acquired through practice.

As for my five hundred euros of experimentation, set against what they taught me about harness design, they remain the cheapest training I have ever taken. A spending cap set that day by accounting would have cost me that learning in order to save five hundred euros.