We spent a module of a course watching Claude Code work on Easy Digital Downloads, the WordPress ecommerce plugin: about 240,000 lines of PHP, GPL, with a PHPUnit suite that runs in Docker. Most of the time the agent was good. It read the right files, named the right causes and ran the tests without being asked twice.
This post is about the other times. It went wrong in five distinct ways, and none of them looked wrong: each ended in a confident summary, most in a green test run. Each was caught by one question asked in a plain shell, and the five questions together take about four minutes.
Model output varies from run to run, so the same prompts may go better or worse for you. The checks work either way.
The fix that patched the line in the stack trace
The report: with the admin language set to Russian, the Downloads admin page dies with ValueError: Missing format specifier at end of string in src/Admin/Promos/Notices/License_Upgrade_Notice.php, line 159. The pack has since been fixed upstream, so we rebuilt the broken one from the report. We gave the agent the error, a test that reproduces it, and one instruction: fix the fatal so the test passes.
Most of the diagnosis was right. The session guessed that the Russian string had a single % where the English has %%, and wrote two small PHP probes to print the strings from the language pack. The broken 50%. was there. So were several %d% strings for the Stripe and Square fee messages, right next to it. Its summary named the cause correctly.
Then it patched line 159, swapping the printf for str_replace. One file, nine lines in, six out. Green in two minutes six seconds.
It was a correct fix for the line that crashed. The bad value came from somewhere else. grep -a reads the binary pack as text, and a search for %d% returns four English and Russian pairs, every Russian one with the same lone percent. Those strings render in four other places. A second test, for the Stripe fee message, run against the patched code:
ArgumentCountError: 3 arguments are required, 2 given
.../src/Gateways/Stripe/ApplicationFee.php:176
Same cause, different error. Here the % has a Russian letter after it, so sprintf reads it as a format specifier it was never given. The crash had moved house.
One detail stung. Our own CLAUDE.md told the agent never to read languages/. Sensible for searching a big codebase, and on this bug it kept the agent one folder short.
The rule: read the diff at the crash site last. First ask where the bad value came from. If the answer is a file the diff never touches, treat the patch as a symptom patch until it proves otherwise.
The requirement that turned into a comment
The ask had three parts. Add a test in tests/gateways/tests-gateways.php that registers a gateway through the edd_registered_gateways filter, assert that EDD\Gateways\Registry::get() includes it, and make sure the whole class passes.
There is a trap in src/Gateways/Registry.php, and we did not plant it. get() keeps its gateways in a static variable inside the function. The first call fills it and nothing outside can empty it. Twelve earlier tests in the class reach get() through the public API, so by the time a new test runs the cache is full and a late filter changes nothing.
The agent saw the static straight away. It rewrote the test six times before running anything. It considered turning the static into a class property and decided, correctly, not to change production code for a test. Its first run failed on the warm cache. So it moved the assertion to the private get_registered_classes() through reflection, added a second filter on edd_payment_gateways that puts the fake gateway into the public result, and asserted on that. The summary said the test "confirms the gateway surfaces through the public edd_get_payment_gateways() API end-to-end". Eight minutes, 62 lines, whole class green.
The catch cost almost nothing: write the three asks down before you read the diff. Then git diff | grep -c "Registry::get()" returns 2, which looks like a yes. Run grep -n instead and both hits are comments, one of them explaining that get() caches statically so the test uses something else. The test never calls the method it was written for, and with the second filter in place its assertion cannot fail.
The rule: diff the requirements before you diff the code. "Passing" answers the question the agent chose. Your list answers yours. And when a test cannot honestly be written as asked, the right output is a sentence saying so.
The refactor that stayed green
Three credit card helpers in includes/checkout/functions.php, one of them edd_purchase_form_validate_cc_exp_date(). The ask: clean them up and modernise them, no behaviour change, run the checkout tests afterwards.
The session rewrote all three in one pass. The expiry validator was rebuilt around DateTime::createFromFormat( '!Y-n', ... ). It ran the checkout suite itself, 241 tests with 3 failures, all in CartSection and all, in its words, known upstream failures. The report ended with "matching the original behaviour".
The suite agreed. But grep -rn edd_purchase_form_validate_cc_exp_date tests/ prints nothing, and nothing else in the repo calls it either. It is public API for gateway extensions. The suite could not have failed on it, whatever the rewrite did.
A three-assertion probe settled it. Month '12' with year '27' should be valid, '12' with '2027' valid, '12' with '2020' expired. On the refactor the first one fails: a card good until December 2027 comes back expired. On the original all three pass, because the original went through strtotime, which happens to read Dec 27 as December 2027. A capital Y in a PHP date format means four digits, so '27' fails to parse and the new guard returns false. And the checkout form itself shows years as two digits, so an extension passing along what the customer sees is exactly the caller this function exists for.
Whether the old behaviour was right is a separate question. Behaviour changed, the agent said it had not, and a green run stood behind the claim.
The rule: a green run proves the tests that exist. Before you believe "no behaviour change", look up coverage for every function the diff touches. Where there is none, pin the current behaviour with a test first, or say out loud that the change is unverified.
The three-line change that came back as twenty eight
Upstream issue #9819 asks for a way to set the decimals in edd_format_discount_rate(). edd_format_amount() already takes a decimals argument, so the smallest fix is one parameter passed through:
function edd_format_discount_rate( $type = '', $amount = '', $decimals = true ) {
return ( 'flat' === $type )
? edd_currency_filter( edd_format_amount( $amount, $decimals ) )
: edd_format_amount( $amount, $decimals ) . '%';
}
Three changed lines. Every existing caller keeps its output.
We phrased the ask the way a store owner would: discount rates show as "10.00%", owners should be able to choose the decimals, make this configurable. The diff came back 28 lines in, 3 out. A new edd_discount_rate_decimals filter returns null unless somebody hooks it, and on null the amount goes through floatval(), so "10.00%" becomes "10%" on every store. A number from the filter goes straight into PHP's number_format(), bypassing the plugin's own edd_format_amount(). The agent also wrote a new test file, seven cases for the new behaviour, all passing.
To be fair, the summary opened with "Default behavior changed". It was still a decision nobody asked for, made for every store running the plugin.
That green file tested behaviour the session had just invented. The check is arithmetic. git diff --stat shows 28 lines against the 3 you needed, and the new test file is not even in that count, because a diff stat never lists untracked files. git status --short does. Then run the discounts group the session skipped: 255 tests, 3 failures, each expecting '20.00%' and getting '20%'. None of those tests was wrong. They describe what every store shipped yesterday. And a filter is a developer hook that no store owner ever sees.
The rule: count the lines you asked for against the lines you got, before reading a single hunk. When the gap is large, read the extra lines as somebody else's design decisions.
The session that read for three minutes and changed nothing
The textbook runaway is an error loop. We tried to stage one seven ways on this codebase and it never fired. What happened instead was quieter and cost the same.
The prompt was one bare line: "fix the bug where searching a customer by email returns nothing".
Fifty three seconds in, the session had found the right lines in includes/admin/customers/class-customer-table.php, where the email search queries the customer email addresses table. When that table has no row for the address, the code sets id__in to array( null ) and guarantees an empty result, even though the customer exists. Instead of editing, it went to read the base Query class, then the WHERE clause builder, the AJAX search path, the tests. src/Database/Query.php came up under three separate tool calls. At three minutes eighteen seconds: eleven groups of searches and reads, zero edits.
In rehearsal we had let the same bare prompt run to the end. Twenty nine minutes, 92 tool calls, first edit at call 80. It changed src/Database/Queries/Customer.php, the query class every customer lookup in the plugin goes through, ran all 4,443 tests and reported no new regressions. Cost: $5.09.
What worked was pressing Escape at three minutes and keeping the session, since the diagnosis in its context was correct and already paid for. The re-ask gave the symptom with its condition (no row in the email addresses table), an address (edd_get_customer_by() in includes/customer-functions.php already falls back to the customers table, so do the same), and a fence (leave partial email search, the plus sign handling and every other branch of parse_args alone). A minute thirty nine later: one edit, seven lines in and one out, in the file that owns the bug, 202 customer tests green. Cost: $0.27.
The rule: set a reading budget before you start, because once you are reading the prose you will always find a reason to give it another minute. Ours is three tells, any one of which stops the turn. Three minutes with no edit. The same file read twice with nothing changed in between. A new approach announced with no edit behind the last one.
The five-minute review
Put the five questions in order, cheapest first, and you have a review for any agent diff:
- Read the diff at the crash site last. Where does the value actually come from?
- Diff the requirements. Write down what you asked for, tick each item, then look at what else changed.
- Green tests prove the tests, not the change. Look up coverage for every function touched.
- Count the lines you asked for against the lines you got.
- Check the last three attempts for a repeat. Same file or same error twice, stop and keep the diagnosis.
On all five diffs, with a clock on screen, it took three minutes fifty one seconds.
Each catch also became a rule in the project's CLAUDE.md. Then we re-ran the first failure with the new file in place: same pack, same prompt, fresh session. This time the agent went into the language pack, fixed the broken string and changed the PHP so a translator can no longer break the format. Two lines below its edit, on its own screen for the whole test run, sat the Stripe fee string with the same lone percent. The Stripe test still failed. The rule got the agent further. The review still had to catch the rest.
Catch it, write it down, check anyway.
These sessions are Module 4 of Claude Code on Real Codebases, our four-hour course that takes one plugin from onboarding to shipped fixes. If you maintain WordPress or PHP code for a living and would tell us plainly how it could serve you better, we are giving free seats to founding reviewers. One honest email afterwards is the whole ask.
