The problem
Last year I sat in on a loop for a senior backend role. The candidate solved a graph traversal problem in eleven minutes — clean code, right complexity, no hints needed. On paper, a hire. Ninety days later, on the job, the same person couldn't work out why a service was leaking connections under load. Not a knowledge gap — a reasoning gap. They'd never had to debug something they didn't already understand end to end.
That disconnect isn't rare anymore. It's the default outcome of running a 2019-era interview loop in 2026.
Why it happens
The classic algorithmic interview measures one thing well: can you recognize a known pattern (two-pointer, DP, BFS, sliding window) and implement it correctly under time pressure. That's a recall-and-execute task. It was never a great proxy for engineering judgment, but for a decade it correlated well enough with "smart, prepared, can code" to be useful.
AI broke the correlation, not the test. Any candidate who's spent even a few months pairing with an AI assistant has effectively memorized the pattern library through repetition, whether or not they touch a model during the interview itself. The rehearsal loop got faster, so the "recall a pattern under pressure" signal compressed toward everyone scoring well. You're no longer measuring engineering ability — you're measuring how many hours someone spent grinding a fixed problem set, which is a much weaker signal and correlates poorly with what the job actually requires.
Meanwhile the skill that was always the real differentiator — debugging code you didn't write, inside a system whose invariants you don't fully know, with incomplete information and someone waiting on you — never showed up in the interview at all. It's roughly 70-80% of senior engineering work by time spent, and it was zero percent of the assessment.
What to do about it
We rebuilt the loop around three changes, and the signal quality difference was immediate and obvious to every interviewer on the panel within the first week of running it.
1. Replace "implement X" with "here's a broken 150-200 line service, find it." We handed candidates real code (sanitized) with a bug that only reproduced under concurrency — a race between a cache invalidation and a read path. No algorithm to recall. Just: read unfamiliar code, form a hypothesis, test it, narrow it down. This is the actual daily loop of debugging in production, and it can't be shortcut by pattern memorization.
2. Allow AI assistance explicitly, and interview the usage, not just the output. We stopped pretending we could detect or prevent AI use and started treating it as a tool candidates would obviously have on the job. The question shifted from "did you use AI" to "when the AI suggested that fix, why did you accept it — and here's a case where its suggestion is subtly wrong, catch it." Candidates who understood the system could catch the wrong suggestion in seconds. Candidates who were pattern-matching couldn't, even with the AI's help, because they didn't have the mental model to evaluate what they were looking at.
3. Ask "why" mid-task, repeatedly, and watch the pause before the answer. Not gotcha questions — just "why that line, why not the other approach." The hesitation pattern between someone reasoning live versus someone recalling a rehearsed justification is very distinguishable once you're listening for it. It's a soft signal, but paired with the debugging task it stopped being noisy.
None of this eliminates false positives. It doesn't need to — it just needs to raise the correlation between loop performance and 90-day performance back above where the old format had fallen to.
Key takeaways
- Algorithmic interviews measure pattern recall under time pressure — AI collapsed that signal by making rehearsal cheap and universal.
- The skill that predicts job performance — debugging unfamiliar code under partial information — was never directly tested by the old format, AI or not.
- Allowing AI explicitly and interviewing the judgment behind its use produces more signal than trying to detect or ban it.
- Watching how a candidate evaluates a wrong suggestion tells you more than watching them produce a correct one.
