Loop engineering: beyond the prompt

There is something ironic about the fortunes of the phrase “prompt engineering”: just as it was entering the boardroom vocabulary, practitioners were already leaving it behind. In our hackathons, a POD produces a working demonstrator in under three hours of delivery, evaluated against a no-AI baseline — and it is never the prompt that accounts for that performance. It is everything built around it: the prepared environment, the workflows, the tools, the checks, and the discipline of the loop. Andrej Karpathy gave part of this reality a name — “context engineering”, “the delicate art and science of filling the context window with just the right information for the next step”^[1]. I would widen the claim: what makes an agentic system reliable is the engineering of the entire loop. I call this loop engineering.

Why the loop, and not the model

The reason comes down to two curves that refuse to track each other. The first is spectacular: according to METR, the length of the tasks agents can complete doubles roughly every seven months — and the pace has been accelerating since 2024. The best models of 2026 sustain tasks lasting several hours where those of 2023 ran out of breath in minutes^[2]. The second curve is nearly flat: across twelve reliability metrics applied to fifteen models, Arvind Narayanan's team at Princeton measures that two years of capability gains have yielded only modest gains in reliability^[3]; and the τ-bench benchmark shows agents that succeed at a task once but collapse when asked for eight consecutive successes^[4]. Capability exploding, reliability stagnating: the gap between the two is precisely the working space of loop engineering. The field confirms it — 90% of developers now use AI, yet 30% do not trust the code it generates, and adoption improves delivery throughput while degrading its stability when nothing frames it^[5].

Anatomy of a loop: four organs

The harness — what the agent can do

The first organ is the harness: the tools the agent can call, the rules that frame it, the guardrails that stop it. Three lessons from the teams building these at scale. First, frugality: “more tools don't always lead to better outcomes” — better a few high-impact tools with carefully crafted descriptions, since “small refinements in tool descriptions can yield dramatic improvements”^[6]. Second, architectural simplicity: the implementations that succeed rest on “simple, composable patterns” — chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer — rather than on opaque frameworks^[7]. Third, layered defense: a guardrail in isolation rarely protects; it is the combination — relevance and safety filters, tool safeguards graded by the reversibility of the action, output validation — that makes an agent resilient, with two non-negotiable triggers for human escalation: exceeding a failure threshold, and any high-risk or irreversible action^[8].

The context — what the agent knows right now

The second organ is the context window, and the discipline that goes with it: treating context “as a finite resource with diminishing marginal returns”^[9]. It is an economic constraint as much as a cognitive one: in agentic loops, cost is dominated by input tokens, because the agent rereads its context at every iteration — the “snowball” documented by the Stanford Digital Economy Lab, which measures consumption up to a thousand times higher than a simple chat on coding tasks^[10]. The loop engineer is therefore permanently deciding what enters the window, what gets summarized, what gets reread on demand — and what will never enter it at all.

The memory — what the system learns

The third organ outlives the session: it is what the system retains from one run to the next. Self-enforcing project rules, learned conventions, decision registers, lessons converted into guardrails — every lesson encountered in production should end up encoded somewhere, rather than in the memory of a person or an agent. It is the most neglected organ and the most profitable: it is what makes a loop improve with use instead of repeating its mistakes, and it is what turns individual know-how into an organizational asset.

The evaluation — what says whether it's good

The fourth organ judges the other three. Without evaluation, a loop produces the plausible with perfect confidence; the AI products that fail almost all share this root cause^[11]. The good news is that the price of admission is low: “20 to 50 simple tasks drawn from real failures make an excellent starting point”, with three families of graders — programmatic, fast but rigid; model-based, flexible but in need of calibration; human, the reference but costly^[12]. And because evaluation criteria emerge from use rather than pre-existing it^[13], the evaluation suite is an asset every organization forges for itself — one that, incidentally, makes model migrations painless.

The patterns that keep the loop standing

Four patterns recur in every one of my systems that has held up in production. The adversarial second pass: after every high-volume production run, a contradictory review cross-checks the strong claims against their sources — it is what intercepts overclaims before they reach the client, and it is planned as a step in the workflow, not as an afterthought. Routing by cognitive load: the budget model for mechanical tasks, the mid-tier for reading and drafting, the most capable reserved for high-stakes judgment calls — all the more rational given that token volume alone explains 80% of the variance in an agentic system's performance: you place your tokens where they produce^[14]. Bounded loop depth: METR's horizons provide the sizing rule — beyond the task duration the model sustains at 50%, you do not pray, you decompose, you place a verification gate, or you escalate to a human^[2]. And finally, governed fan-out: massive parallelization for homogeneous batches with low unit stakes, ordered sequencing for critical or interdependent batches — never the other way around.

The anti-patterns that derail it

Failures have their classics too. The confident infinite loop: an agent with no depth bound and no failure threshold, pressing on with assurance into a dead end — the modern variant of the “retry storm”. The snowball context, which degrades cost and quality simultaneously as the window swells. Premature multi-agents: orchestration complexity added before exhausting what a single, well-harnessed agent can do — the two reference labs give the same advice here, start simple^[7][8]. And evaluation postponed, which condemns you to steering by impressions — when we know, measurements in hand, that impressions lie: seasoned developers believe they are 20% faster when they are in fact 19% slower^[15].

From the loop to the production system

A well-designed loop is still just a component; in production, it slots into a framework. The capability model of the DORA report says it in its own way — “strengthen your safety nets” is one of the seven capabilities that make AI amplify the best rather than the worst^[5]; and for organizations that must formalize, the GenAI profile of NIST's risk management framework provides the full grammar — twelve risk categories and two hundred actions spread across govern, map, measure and manage^[16]. My advice is simpler: every agentic system should have, written down somewhere, its loop contract — maximum depth, token budget, verification gates, escalation conditions, evaluation suite. A system whose contract you cannot write is not ready for production.

I am sometimes asked whether loop engineering will survive the next generation of models — after all, more reliable models will need fewer guardrails. Probably, and so much the better: the loop will get lighter. But it will not disappear, because it encodes what the model will never know on its own: what your organization calls a job well done, what it forbids, what it tolerates, what it has learned from its failures. A prompt can be copied in ten seconds. A loop has to be built — and that is precisely why it is worth something.