Models come and go, the harness remains

On September 1, 2026, Anthropic released Claude Fable 5.1. The next day Google shipped Gemini 3.8 Flash, and the day after that OpenAI unveiled Astra. xAI followed on the 21st with Grok 4.7^[1][2]. On the 22nd Anthropic was back with Opus 5.5, which matches Fable 5.1 on most work at 40% less than the cost of Opus 5^[3]. Five frontier models in three weeks, not counting the smaller releases in between.

Against that backdrop, the question I hear most often in workshops sounds slightly out of step: “which model should we choose?” I understand where it comes from. For twenty years we picked a database, a language or a cloud for a decade, and it feels natural to do the same here. A model, though, is not chosen like a foundation. It wears out, it gets replaced, and its lifespan is counted in months. The question worth asking is what we build around it, and what will be left once it is gone.

A year, give or take

There is no need to take commentators at their word: the providers publish their own retirement schedules. At Anthropic, Claude Sonnet 3.7, launched in February 2025, was retired twelve months later to the day. Opus 4 and Sonnet 4 lasted thirteen months, Opus 4.1 exactly one year. For its current models, the company now lists a minimum retirement date set one year after release^[4]. At OpenAI, GPT-5, released in August 2025, was deprecated ten months later and will be switched off on December 11, 2026; o3 goes the same day, after twenty months in service. The extreme case is still GPT-4.5 preview, retired less than five months after launch^[5].

And the pace is picking up. Anthropic shipped six versions of Opus between November 2025 and September 2026^[4][3]. At Google, Gemini 3.6 Flash, introduced on July 21, had a successor six weeks later^[6][1]. Notice periods range from sixty days at Anthropic to roughly six months at OpenAI. In late 2025 Anthropic even made formal commitments to preserve the weights of retired models, which amounts to admitting that forced migrations are part of the job and that they cost money^[7]: a central component of our systems now changes about once a year, whether we like it or not.

The 95% misunderstanding

That schedule is the right lens for rereading the most quoted statistic of 2025. According to MIT's NANDA project, roughly 95% of enterprise generative AI pilots had no measurable effect on the P&L^[8]. The figure made the front page of Fortune and briefly rattled the markets^[9]. It needs some qualifying. Critics rightly noted that any pilot not rolled out at scale counts as a failure, and that the report mostly describes organizations poorly equipped to evaluate, buy and integrate^[10]. I find that corrected reading more interesting than the raw number, because other studies back it up. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, over runaway costs, value that never materializes or poorly controlled risk^[11]. MIT also found that solutions built with a specialist partner succeed twice as often as improvised in-house builds^[8].

McKinsey provides the most telling measure, because it comes back every year. In 2025, 88% of organizations used AI somewhere, but only 39% saw its effect on their earnings. Of all the factors studied, the one that best explained value was the redesign of work processes, which only 21% of adopters had undertaken^[12]. The August 2026 edition has 37% of companies attributing some of their earnings to AI, and a share of “high performers” still around 6%. Yet eight respondents in ten say AI makes them more productive personally^[13]. Between the two surveys, models were replaced several times over and improved considerably; the effect on the bottom line did not move. The difference lies around the model, in the organization and in the tooling.

Capability and reliability

The surroundings matter more than the model because a model's capability and a system's reliability are two separate quantities, and they do not grow at the same rate. Sierra's τ-bench, now a reference for customer service agents, was the first to show it. The best agents succeeded on fewer than half of the tasks, and that rate dropped to around 25% when they had to get the same task right eight times in a row^[14]. A Princeton team led by Arvind Narayanan has since given the gap a name, the “capability-reliability gap”. Across twelve reliability metrics applied to fifteen models, nearly two years of spectacular capability gains produced only modest gains in reliability^[15].

Waiting for the next model will not be enough. The gap narrows with what gets built around the model: rules, checks, guardrails, error recovery. That is the harness, in the literal sense of the word: the gear that lets you hitch a powerful but unpredictable force and turn it into useful work.

What gains value as the model loses it

The first of these assets is evaluation. Greg Brockman, OpenAI's president, wrote as early as 2023 that “evals are surprisingly often all you need”^[16]. Drew Breunig went further, in a line I am happy to borrow: your eval is the most valuable AI asset you own, more than your model and more than your prompts^[17]. An evaluation suite that captures what your business calls good work puts you in charge of the calendar. Every new model becomes a candidate you can audition in a few hours, and a migration forced by a provider becomes routine. Hamel Husain, whose work on the subject is a reference, observes that failing AI products nearly all share the same root cause: the lack of a solid evaluation system^[18]. The Berkeley researchers who studied “criteria drift” add a further argument. Evaluation criteria do not exist before use; you discover them by reading what the system produces^[19]. Every organization therefore forges its own, and nobody can sell them off the shelf.

Those evals still have to be truly yours. On June 3, 2026, OpenAI announced it was closing its hosted evaluation platform, its Agent Builder and its reusable prompts API, all three shutting down on November 30. The company recommends moving evals to Promptfoo, an open-source tool that runs locally or in continuous integration^[20]. Teams that had entrusted their test sets to the model provider now have to move them at the same time as the model: a vendor's tooling lives on the vendor's schedule, and yours has to be able to do without it.

The second asset is context, and with it the company's memory. Andrej Karpathy popularized the term “context engineering”, which he describes as “the delicate art and science of filling the context window with just the right information for the next step”^[21]. Anthropic turned it into an engineering discipline of its own, one that treats context as “a finite resource with diminishing marginal returns”^[22]. What that context holds is mostly what the company knows: its business rules, its conventions, its living documentation, the history of its decisions. Once structured to reach agents at the right moment, that memory serves today's model and every model that follows.

The third asset is integration, which standardized in under two years. The Model Context Protocol, launched by Anthropic in late 2024 and adopted since by OpenAI, Google and Microsoft, was handed in December 2025 to a neutral foundation under the Linux Foundation. There were more than ten thousand public servers at the time^[23]. Meanwhile, the LLMOps tooling market is growing by more than 20% a year^[24]: connectors, too, outlive the models that pass through them.

Research is starting to put numbers on what practitioners sensed. On SWE-bench Mobile, a benchmark built on a production iOS codebase, twenty-two agent configurations were compared. With the model held strictly constant, a factor of six separates the best harness from the worst^[25]. The model's surroundings now weigh more than the move from one model generation to the next. Stanford went a step further by treating the harness as something to optimize. Its Meta-Harness loop has a coding agent inspect, diagnose and rewrite the harness itself. It beats the state of the art in context management by nearly eight points while using four times fewer tokens^[26]. A harness can be measured, designed and improved, like any other engineering asset.

The agentic factory manifesto

I draw four rules from all this. I apply them in the projects I support, and they make up the manifesto of what we call an agentic factory.

First, never tie yourself to a model. Every architecture should pass what I call the replacement test. If the provider retires your model on Monday, you should be able to switch within days, with evals to prove nothing has regressed. If the switch takes a month or more, the architecture has a problem.

Second, budget evals like infrastructure, sheltered from schedule trade-offs. It is the only AI investment whose value rises mechanically over time, since every failure met in production feeds the test suite.

Third, treat memory and context as intellectual property. What your agents know about your company belongs to you. That knowledge should be versioned, audited and portable from one model to the next and from one provider to another. Evals and prompts fall within that scope, since, as we have just seen, they can disappear along with the platform that hosts them.

Fourth, industrialize the loop, beyond the prompt alone. The loop is what turns an impressive demo into a system you dare to depend on.

To anyone who has lived through a forced migration, the title of this paper is stating the obvious. It does carry a budget consequence that few organizations have drawn, though: if the model is a consumable and the harness an asset, the split of investment should flip. Less energy spent chasing this week's leaderboard, more spent building what lets you get the most out of any model. September 2026 made the point better than any argument could: five new models in three weeks, and no reason for any of them to stay on top for long. Models will come and go, that is what they do, and nobody knows which one will win the next round. When it arrives, what will count is already having in hand the one thing it cannot bring you: the machine for turning it into value.