Two agents, one night, 158 million tokens: what it cost and what we are fixing

Draft for publication · 16 September 2026 · Owner: Aashish Bhandari (Max) · Author: Claude ("Naruto") · Measurements: Codex ("Goku") · Status: base case, before fixes.

Executive summary

A design review of a web service had raised twelve findings: a missing transaction guard, retry and credential-rotation gaps, a fragile handoff route, thin diagnostics, portability problems and similar. On the evening of 15 September one AI coding agent was asked to fix all twelve, test them, integrate the work and produce a release package. It did. A second agent, from a different vendor, rebuilt the result from scratch the next morning and verified it independently.

The builder ran as one coordinating "parent" model with sixteen helper "workers" on cheaper models. Nobody typed code overnight. The verifier approved closure with no serious finding. Delivery at this pace is now routine for us. The cost is not yet acceptable, and this article is about why.

One number to hold on to: the models read the equivalent of about two hundred copies of War and Peace in order to write less than one. Almost all of that reading was the same conversation, sent again and again. That is the lever.

A caution before the details. Reading the telemetry of a run like this is itself a model task, and our first attempt at it cost 60% of the run it was measuring. This time it cost 4.7%. Our target is under 2% of the run being assessed, measured the same way, or the analysis stops and asks for a budget.

Where the numbers come from

Every figure here comes from a lifecycle hook that runs when a session pauses or ends. It reads the session transcript with a plain script, writes counters to a JSON file and regenerates a CSV and a summary. No model reads a transcript to do this. It costs zero model tokens and cannot be forgotten. The primer on this, and on why the cost of an agent is roughly activations × context size, is in How an agent's cost adds up and Zero-token telemetry. The full fact sheet for this run is in Base case fact sheet.

What is already in place, and what each mechanism did

Since early September both agents run under the same set of rules, installed in their global instruction files (AGENTS.md for Codex, CLAUDE.md for Claude Code), enforced where possible by hooks, and shared through a small set of skills. Here is each mechanism, and what this run says about it.

Zero-token telemetry capture. Installed as Stop and SessionEnd hooks for both agents. Before it, a model was asked to summarise its own usage at the end of a task, which cost one full-context activation and could not see its own final answer. Measured effect: capture of 55 components, 1,505 activations and 1,299 tool calls in this run for zero tokens. It is the reason the rest of this article exists.

Routing by work shape. The parent is the strongest model. Workers default to the cheapest model that fits the job: mechanical and high-volume work to the smallest, broad reading to the mid-tier, demanding reasoning only to the top tier. Measured effect at frozen list prices: this run's named-model usage prices at $99.80 against $218.81 had every token run on the parent's model, a 54% reduction. The earlier, smaller build measured 43%. This is price substitution. The workers still processed 110 million tokens.

Delegation floor and one-shot worker packets. A worker is dispatched only for a bounded workstream, with the objective, file paths, permissions, output shape and budget in one short packet, and told to return once. Measured effect: 70% of the tokens moved off the expensive parent. Partial failure: the parent reused four workers through 28 follow-up assignments, and those four long-lived workers carried 49% of the entire run, because a reused worker re-sends its accumulated history on every step.

Event-driven completion, no polling. The rule says: dispatch, then do independent work or wait once; never ask "are you done?" on a timer. Measured effect: it held while the parent had work of its own, and failed the moment it went idle. The parent issued 89 waits, 88 of them at sixty seconds, and 66 timed out. Each of those checks re-sent the whole conversation: 10.3 million tokens, 6.5% of the run and 12.7% of its list-price cost, for the information "not yet". The mechanism is explained in The idle-parent trap. On Claude Code the harness wakes the parent when a worker finishes, so the trap does not open there. On Codex today a finished worker cannot wake an idle parent, so a declarative rule alone is not enough.

Activation budgets and batching. Claude counts per human prompt (target four, ledger at eight, deliver at twelve). Codex counts per task (ledger at eight, deliver at twelve). Both must batch independent reads and run one validation gate. Measured effect this run: 342 parent activations across twelve human turns, about 28 per turn. Not yet the target, and now measurable.

Compaction. The parent's context was compacted five times, three of them by hand. Each cut the next step's input by 36% to 84%. It is a relief valve, not a controller: the context regrew, and the longest post-compaction turn still spent 20 million tokens and issued 62 waits.

Analysis discipline. Performance work lives in a separate workspace, reads the derived telemetry first, extracts missing fields with deterministic tools, and never assigns a model to parse raw logs. Measured effect: analysis overhead fell from 60% of the assessed run to 4.7%. Still above the 2% target, and the target is now written down.

Shared decision record and winning criteria. Ten decisions and ten criteria, one file each, readable by both agents. A criterion is "won" only after three consecutive qualifying sessions per agent, judged from telemetry alone. This run is the before row for every one of them.

Work in progress

Delivery was excellent. Orchestration was not. The two are separable, and separating them is the whole project. Six fixes are queued, each with a criterion that decides, from telemetry alone, whether it worked:

1. Five-minute waits instead of sixty-second ones, with one progress line before the wait.
2. A zero-token watchdog: a plain script that reads worker counters every minute and wakes the parent only on completion, budget breach or a stuck signature.
3. One worker, one stage: a follow-up and activation gate per worker, with a fresh minimal-context worker for the next stage.
4. Telemetry extensions so waits, follow-ups, compaction tokens and terminal errors are counted by the hook, not reconstructed by hand.
5. Mechanical work to the cheapest model, and parser failure as a stop condition, with the analysis budget checked before an audit starts.
6. Automatic approval reviewers, 7.5% of this run at an unknown price, checked to see whether they are a setting rather than a design.

The same task shape will run again once these land, and the same hook will count it. If the numbers move, the criteria will say so. If they do not, this article stays the base case and we try the next six.

Further reading in this series: the primers linked above, the zero-token watchdog explainer, and Goku's full runtime analysis from which every measurement here is taken.