If you use an agentic coding tool — Claude Code, or anything with a persistent instructions file — you've probably hit this. You start a CLAUDE.md (or .cursorrules, or AGENTS.md) with a few conventions. It's useful. So you add the architecture. Then the gotchas. Then the three things it keeps getting wrong.
Six months later it's 800 lines, and you are paying for all 800 of them on every single message, in every session, forever.
That's the part people miss. An instructions file isn't loaded once per session. It's part of the context for each turn. A file that doubles doesn't cost you twice once — it costs you twice per message, for the life of the project.
Why the two obvious fixes both fail
Delete most of it. Now the agent doesn't know your conventions and you're correcting it by hand again. You traded token cost for your own time, which is usually a worse trade.
Keep it and eat the cost. It grows monotonically, because nobody ever opens a working instructions file to prune it. There's no forcing function.
Both fail for the same reason: they treat context as one undifferentiated blob that's either loaded or not. But the facts in that file don't have equal value on any given turn. "We use tabs" matters every time. "The payment reconciler retries with exponential backoff because the provider's 429 has no Retry-After" matters only when the agent opens that file.
Three tiers
Split the context by when it's needed, and load each tier accordingly.
1. Always-on: a bounded index
The instructions file holds only what's true for every turn — and, critically, has a size ceiling that doesn't move as the codebase grows.
Not a summary of everything. An index: what the modules are, one line each, and where to look for more.
## Project memory (auto-maintained)
- **api** — HTTP layer, request validation, rate limiting
modules: routes, middleware, schemas
- **worker** — background jobs, retry policy, dead-letter handling
modules: queue, handlers
- lessons: 53 learned (injected when relevant)
- detail on demand: `recall "<topic>"`
That's the entire always-on cost. In claudectl I cap it around 250 tokens. The number matters less than the fact that there is one, and that adding the fiftieth module doesn't raise it — new detail goes into tier 2, not here.
2. On demand: path-scoped detail
Claude Code reads .claude/rules/*.md files that can be scoped to paths. A file scoped to api/** enters context only when the agent touches something under api/.
# api — HTTP layer
- Validation happens in schemas.py, never in routes. A route that validates
inline is a bug: the OpenAPI spec is generated from the schemas.
- Rate limiting is per-account, not per-IP; per-IP broke customers behind
corporate NAT.
This is where the expensive, specific knowledge lives. You can have a hundred of these. On a turn that touches two modules you pay for two.
If your tool has no path-scoping, approximate it: keep the detail in files the agent can read on request and reference them from the index. You lose the automatic trigger but keep the cost shape.
3. Per-task: inject only the relevant subgraph
The last tier is optional and the most involved. On each prompt, look at what was actually asked, pull the related facts, and inject just those — under an explicit budget (I use ~600 tokens).
The trap: this is a per-turn code path, so anything it does, it does forever. My first version incremented usage counters and wrote two files on every prompt. Two sessions in the same project promptly overwrote each other's counts. Whatever you do here, make it append-only and cheap, and fold the bookkeeping in later.
What keeps it bounded
Tiering alone doesn't help if each tier grows without limit. Three rules:
- Consolidate. Periodically roll related facts into fewer, denser ones.
- Invalidate rather than append. When something is superseded, remove the old fact. An instructions file that accretes contradictions is worse than a small one — the agent now has to guess which is current.
- Evict. Track what actually gets used and drop what doesn't.
Eviction needs a real signal. Mine reads how often a fact was pulled into a prompt, not how recently it was written — recency told me almost nothing.
It isn't free
Building this memory costs tokens: something has to read your code and write those files. Two things make it affordable:
- It's amortised. Extraction runs occasionally; the savings are on every message.
- Route it to a cheap model. Summarising a module is not a frontier-model task. Internal calls in claudectl default to a cheap model while your actual coding sessions use whatever you chose.
The same asymmetry works for execution: have the expensive model write a plan, approve it, then let a cheap model carry it out. Planning is the part that needs the good model.
Limitations, honestly
- If your project is small and your instructions file is 40 lines, this is over-engineering. Keep the 40 lines.
- Automatically extracted memory is sometimes wrong. Review it. Mine puts generated content behind an approval gate for that reason.
- Path-scoped loading depends on tool support. Without it, tier 2 is manual.
- The always-on index is a real constraint: sometimes something genuinely important doesn't fit, and you have to decide it's tier 2 instead.
Takeaway
Ask of each fact: does the agent need this on every turn, or only when it touches a specific thing? Almost everything is the second kind, and almost everyone stores it as the first.
I build claudectl, an open-source workspace layer for Claude Code where this is implemented — bounded index, path-scoped rules, budgeted injection. MIT, Python standard library only. The approach works without it, which is why I wrote it up as an approach. Docs: https://babarmuhammad.github.io/claudectl/
