Cut agent tool call cost: GitHub's 20% fix was a prompt rewrite

TL;DR

Agent tool call cost is dominated by what each call returns, not by how many calls the agent makes — every result stays in context and is re-billed on every later turn. GitHub hit this on Copilot code review: swapping bespoke search tools for the generic grep, glob and view trio from its CLI harness raised average review cost while catching fewer useful issues. The tools were fine, and rewriting the system prompt to be diff-anchored bought back roughly 20% of average review cost at unchanged quality.

The regression: better tools, worse reviews

On 10 July 2026, GitHub engineer Napalys Klicius published an account of the migration. The change looked like pure cleanup. Three bespoke tools were retired in favour of three shared Unix-shaped ones:

The same harness powers the GitHub Copilot cloud agent, so this was consolidation onto a proven surface. The result was the opposite of the intent: average review cost went up, and the agent caught fewer useful issues.

Worth naming, because it will happen to you too:

Tool-surface regression is when replacing a tool with a better-designed one degrades the agent, because the new tool carries conventions and instructions shaped for a different job.

Why generic tools drag an agent off-task

The grep/glob/view trio is genuinely better than what it replaced. It is also the toolset of an interactive coding assistant, and it arrived with the guidance that makes that job work: explore, follow leads, build a picture of the codebase.

Code review is not that job. A review has a bounded starting artifact — the diff — and a bounded question: is this change correct. GitHub's diagnosis was that the generic coding-assistant instructions made the agent behave like "a broad coding assistant instead of a reviewer." It widened searches, guessed at paths, and accumulated context it never needed.

The failure is easy to miss because nothing errors. The agent still returns a review. It just spent a lot to write it, and the noise crowded out the evidence that would have caught the real bug.

What agent tool call cost is actually made of

Here is the part that makes tool design a cost problem rather than a taste problem, and the reason agent tool call cost is so consistently underestimated. As GitHub puts it: "Every tool result becomes part of the agent's working context. Extra file contents can be carried forward into later reasoning, increasing cost."

A tool call is not billed once. Its result is appended to the context and re-sent as input on every subsequent turn of the loop.

Run the arithmetic on a single stray read. Assume a 10-turn review and one unnecessary 4,000-token file read, with no prompt caching and no compaction:

• Read it on turn 2 → it sits in the context for turns 3 through 10.
• That is eight further turns × 4,000 tokens = 32,000 extra input tokens, on top of the 4,000 the read itself returned.

One file, read early, in a loop of ordinary length, at nine times its apparent price. Three of them and you have added six figures of input tokens to a task whose visible output is a handful of review comments. (That figure is arithmetic from the stated assumptions, not a GitHub measurement — the mechanism is theirs, the illustration is mine.)

This is the same economics that makes context compaction a lossy, load-bearing decision rather than a housekeeping detail. Early junk is the most expensive junk, because it is re-billed the most times and it is the first thing a compaction pass has to decide about.

The fix: instructions shaped like the job, not the tool

GitHub changed neither the tools nor the harness. It rewrote the system instructions to describe review, replacing the exploration pattern with a narrowing one.

The reported outcome: roughly 20% lower average review cost, with no quality signal that could block shipping. Same model, same tools, different instructions.

None of this is GitHub-specific. Anthropic's guidance on writing effective tools for agents lands in the same place from the tool author's side: "You can directly encourage agents to pursue more token-efficient strategies, like making many small and targeted searches instead of a single, broad search for a knowledge retrieval task," and "Even small refinements to tool descriptions can yield dramatic improvements." Narrow-then-read is not a Copilot trick; it is the cheap default that broad instructions keep overriding.

If you are writing that layer yourself, the mechanics of what belongs in a tool description versus the system prompt matter more than how many tools you expose. And the "batch independent discovery" rule is the same insight that makes parallel tool calls worth a DAG in the harness — independent greps have no reason to be serial.

How to see this in your own agent

You cannot fix what your evals do not record. GitHub's internal benchmarks were the thing that made the regression legible, because they surfaced behaviour rather than just verdicts: the tool call sequences and paths taken, the output quantities, where errors occurred, whether the agent narrowed or widened, and how context accumulated.

That turns into three questions you can ask of any recorded run:

1. Did it narrow first, or read broadly? Count file-read tokens before the first grep/glob.
2. Did it batch independent searches? Serial discovery on independent queries is pure latency and pure re-billing.
3. Did it call the read tool with a justified exact range? A whole-file read where a 20-line range would do is the unit of waste.

Output-only scoring cannot see any of this — the review still reads fine. This is exactly the gap that trace-level eval harnesses exist to close, and it is why "the model got worse" is so often the wrong diagnosis. Log the trace, diff two traces on the same input, and the browsing pattern is obvious in seconds.

Four rules that generalise

1. Tool surface is product surface. GitHub's own framing is that tool surfaces are a product experience layer, not an implementation detail — "a small wording change can affect cost, quality, and the shape of the investigation because it changes how the agent spends its attention."
2. Instructions belong to the job, not to the tools. Share the harness across agents; do not share the system prompt. A reviewer, a triager and a migration agent want opposite defaults from the same three tools.
3. Price the read, not the call. Budget an agent in tokens-carried-forward, not in number-of-calls. A cheap call that returns 4,000 tokens is an expensive call.
4. Evaluate traces, not just outputs. If your eval cannot tell you whether the agent narrowed or widened, it cannot tell you why the bill moved.

Common mistakes

• Blaming the model and upgrading it. A model change is the most expensive way to fix a prompt bug, and it hides the regression rather than removing it.
• Adding tools to compensate. More surface means more ways to browse. GitHub's fix went the other direction — same three tools, tighter rules about when each is allowed.
• Copying a harness's system prompt wholesale. The prompt encodes the original job. Inheriting it is how a reviewer learns to explore.
• Scoring only the final answer. The failure mode here is invisible to output-only evals until it shows up on the invoice, which is the slowest possible feedback loop. If you are choosing between assistants, this is also the axis most head-to-head comparisons never measure.

The takeaway is small and annoying: the highest-leverage lever on agent tool call cost is usually a paragraph of instructions, not a tool, a model, or a framework. GitHub found 20% of theirs in a rewrite. Go read your agent's traces before you go shopping.

Frequently asked questions

Why did better tools make GitHub's Copilot code review worse?

The tools themselves were not the regression. GitHub swapped Copilot code review's bespoke search tools for the generic grep, glob and view tools from its CLI harness, and those tools arrived with instructions written for open-ended interactive coding. Under that guidance the agent explored the repository broadly instead of staying anchored to the diff, which raised average review cost while catching fewer useful issues.

What is agent tool call cost actually made of?

It is dominated by what the tool returns, not by the call itself. Every tool result is appended to the agent's working context and re-sent as input on every subsequent turn of the loop, so one unnecessary file read is billed many times over the life of a single task. That is why an agent that reads three extra files early can cost noticeably more than one that reads a single exact line range late.

How did GitHub cut roughly 20% of review cost?

By rewriting the system instructions rather than changing the tools or the model. The new instructions anchor the agent to the diff, tell it to narrow first with grep and glob and to call view only when it already knows the exact file and line range, and tell it to retry with a simpler search on a miss instead of widening the exploration.

Should I reuse my coding agent's harness prompt for other agent tasks?

Reuse the harness, not the prompt. A CLI harness is built for broad interactive coding, so its instructions reward exploration; a reviewer, a triager or a migration agent all start from a bounded artifact and want narrowing behaviour instead. Keep the shared tool implementations and write task-specific instructions on top of them.

How do I tell whether my agent is browsing instead of working?

Evaluate tool traces, not just final answers. Record the call sequence, the size of every result, and whether each step narrowed or widened, then ask three questions of each run: did it narrow before reading, did it batch independent discovery calls, and did it read a file only with a justified exact range. Output-only scoring hides this failure completely, because the answer still looks acceptable while costing far more than it should.

Does prompt caching remove this problem?

It reduces the price of the re-sent prefix but does not remove the incentive. Cached input tokens are cheaper, not free, and a bloated context still crowds out the evidence the agent actually needs and pushes long tasks toward truncation or compaction. Cheaper repetition is not a substitute for not accumulating the junk in the first place.

Sources

• Napalys Klicius, GitHub Engineering — Better tools made Copilot code review worse. Here's how we actually improved it., 10 July 2026 (the tool swap, the regression, the rewritten instructions, and the ~20% cost figure)
• Anthropic — Writing effective tools for AI agents (token-efficient search strategies and the leverage of tool descriptions)

The 32,000-token penalty and the two illustrative traces are my own arithmetic and illustration from the mechanism GitHub describes, under the assumptions stated on each diagram — not measurements from GitHub's benchmarks.

Originally published at umesh-malik.com

Keep reading on umesh-malik.com:

• Rust LLM Policy: Use AI to Review, Not to Create
• Build your research spike as running code: 8 blockers a doc missed
• Fix your agent tool instructions: GitHub's 20% review-cost cut