Browser automation has a standard reflex: if the action throws, retry it. CodeMirror 6 broke that reflex for us in the most expensive way possible — the throw and the success were the same event.
This is a short field note on the failure, the mechanism as far as we could actually measure it, and the fix that made it structurally impossible to hit again.
The symptom
We were automating a post editor built on CodeMirror 6, filling a long article body with Playwright's fill().
-
fill()threw a timeout error - Our handler treated the throw as "nothing happened" and retried
- The saved article came out with the entire body duplicated
The first fill() had thrown and inserted the content. The retry inserted it again. Every layer of that pipeline behaved reasonably, and the output was still wrong.
What we could verify, and what we could not
CodeMirror 6 manages a contenteditable surface through its own view model — the DOM you see and the editor state behind it are separate structures. fill() inserts text, then verifies the result against its own expectation of the element's value. On a CodeMirror surface, that verification can fail even though the insertion event was fully processed.
That is as far as we measured. We did not chase the internals further, because the useful lesson is not the specific bug — it is that "exception means not executed" is an assumption, and some editors break it.
The fix: one path, then count
We stopped trying to make the retry smarter and instead made double-insertion structurally impossible.
1. Insert through one path that replaces instead of appending.
await editor.click();
await page.keyboard.press('Meta+A');
await page.evaluate(text => {
document.execCommand('insertText', false, text);
}, body);
Select-all before insert means a retry overwrites the previous attempt instead of stacking on top of it. execCommand is deprecated, but it goes through the browser's editing pipeline — which is what contenteditable frameworks actually listen to — and in practice it is still the most stable way in.
2. Verify by counting, after reload.
We plant a unique marker string at the end of the body, save, reload, and count occurrences:
const count = await page.evaluate(m =>
document.body.innerText.split(m).length - 1, MARKER);
if (count !== 1) throw new Error(`marker x${count} — duplicated insert`);
The success check moved from "did the call return cleanly" to "does the saved artifact contain exactly one copy." Those are different questions, and only the second one is about reality.
The general shape
- For write operations, an exception is not proof of non-execution
- Before any retry of a write, re-read the current state — the previous attempt may have partially landed
- Non-idempotent operations (append, insert, submit) deserve a marker you can count afterward
- Judge success from the saved artifact, not from return values or toasts
The same trap exists anywhere a "submit" or "add row" can throw after the side effect landed. When a write throws, the first question is not why did it fail — it is did it actually fail.
Field note from building untactit, a control plane for the skills, rules, and memory AI agents run on — currently pre-launch. The product applies the same rule: a write counts as done only after the target is read back.
