AI-generated video detectors fail on exactly the footage that matters most. A benchmark released this week called RA-Bench tested seven traditional detectors, ten off-the-shelf multimodal models and two purpose-built detectors against 17,886 videos of wars, disasters and public emergencies, and found that none of the three families held up. Passing the same clips through an ordinary resharing chain -- re-encode, shrink, drop the frame rate -- pushed the best fine-tuned detectors from catching 46 percent of the fakes down to 1.4 percent.
Key facts
- The benchmark holds 17,886 videos: 1,830 real anchor clips drawn from 675 public crisis videos across 10 social-risk categories, paired with 16,056 generated clips from four open-source and five closed-source video generators.
- After a simulated last-mile resharing chain, fake-recall for the fine-tuned detectors fell from 46.0 percent to 1.4 percent.
- A 633-clip subset fooled all five human reviewers independently; on that subset the traditional detectors averaged 47.5 percent AUC, which is slightly worse than a coin flip.
- Posted to arXiv on August 17, 2026 as arXiv:2608.14391, and ranked the #1 paper of the day on Hugging Face.
The reason this benchmark is different from the deepfake test sets that came before it is in its name. RA stands for "real videos as anchors." Instead of generating synthetic clips from scratch and asking a classifier to sort them from a pile of unrelated real footage, the researchers started from 675 genuine crisis videos, curated 1,830 clean anchor segments out of them, and then asked nine different video generators to continue each anchor from its own first frame using a shared caption. Every generated clip is length-matched to the real one, re-encoded the same way, and stripped of audio. That removes almost every shortcut a detector could take. It cannot win by noticing that the fake clips are shorter, or brighter, or in a different container format. It has to actually tell that the pixels were dreamed rather than filmed.
Under those conditions, the detectors that look strong in the literature stop looking strong. Detectors that report accuracy in the high 60s to high 90s on public reference sets fall to source-level averages in the mid-40s to high-50s on RA-Bench. Worse for anyone hoping to just pick the best one, the ranking itself becomes unstable: which detector wins changes depending on which generator made the clip, so a leaderboard built on one generator tells you little about the next. The authors state the result plainly in the paper: "none of the three detector families generalizes consistently across RA-Bench instances."
The most useful way to think about what breaks is to picture a forger who is caught not by the quality of the signature but by the paper. Current detectors mostly work like a document examiner who has memorized the exact fiber pattern of one brand of stock. Change suppliers -- switch generators -- and the examiner is guessing. Now run the document through a photocopier three times, which is what resharing does, and even the real fibers are gone.
That photocopier is the second half of the study, and it is the part with operational consequences. The researchers built a subset called RA-Bench-LastMile that simulates what happens to a video after it is posted: transcoding to H.264, spatial downsampling, a frame-rate cut to 8 frames per second, and a synthetic news badge slapped on top, applied alone and in combination. This is a controlled approximation, not live platform telemetry, and the paper is explicit about that. But the effect is stark. Under the full chain, traditional detector accuracy slides and the fine-tuned detectors -- the ones specifically trained for this job -- almost entirely stop calling anything fake, with recall collapsing to 1.4 percent. A detector that only works on the pristine original file is not a detector for the internet, where nobody sees the original file.
The human half of the study lands in the same place. Reviewers labeled 68.6 percent of open-source generated videos as fake but only 52.9 percent of the closed-source ones, meaning the better commercial generators are already at roughly coin-flip believability for people. The 633 clips that fooled every reviewer became the HumanProof subset, and detectors did not rescue them: the traditional detectors averaged 47.5 percent AUC there, which is to say no useful signal at all. The paper's summary of that overlap is short: "videos that mislead people are also difficult for current detectors, and that social dissemination makes detection harder."
There is one more finding worth flagging because it is a warning about how these systems get evaluated. One of the fine-tuned detectors turned out to be reading the timestamp format rather than the video. Under the official timestamps it looked meaningfully better than chance, and clips that happened to be exactly five seconds long were disproportionately flagged as fake. Replace the absolute timestamps with plain frame indices while feeding the model the identical 16 frames, and its performance fell back to roughly chance and the five-second effect vanished. That is a textbook case of shortcut learning, and it is a reminder that a good score on a detection benchmark can be an artifact of formatting rather than evidence of vision.
The honest caveat is the one the authors put in their own limitations section. RA-Bench is visual-only. It does not model audio, manipulated captions, selective editing of real footage, repeated rounds of platform processing, or how long a real moderation queue takes. It proves that detection is fragile under a realistic set of transformations. It does not prove that every detector will always fail everywhere. But for anyone treating automated detection as the plan for AI-generated crisis misinformation, the result argues for a different plan -- provenance and signing at the point of capture, which is the approach behind content watermarking and the code-watermarking work we covered earlier this month. The benchmark, the project page and the GitHub repository are all public, so the claims can be checked.
Originally published on Ground Truth, where every claim is checked against the primary source.
