Matching strings do not settle the direction: which document is the original

Summary

I run a check that compares the body text of articles before publication with the text of my source material, string by string. The source material is the private development repositories I draw on for the articles.

On 2026-07-18, 3 articles waiting for publication failed this check. The cause was not that the articles had copied internal text from the source material. It was because one file in the source material, a file that analyzed this blog's published articles, quoted body text I had already published. The copying document and the copied document were the other way around.

The same check also has a blind spot in the opposite direction. As long as I exclude my own repository from the targets of matching, a draft that echoes the wording of my own internal documents cannot be found by the machine matching.

For the false positive where I mistook the direction, I revised the targets of matching, and for the defect in allowed words, I fixed how matches are counted. Neither fix loosened the criteria for the judgment itself. Copies taken from my own repository are still found by an AI review that reads the meaning.

What you can take away

For those who have a check based on string matching in their own operations, I explain the following three things.

• You will be able to choose, at design time, the evidence that decides the direction of a match, that is, which document is the original. I decide it with time information rather than with strings, and I write out the three dates used in that judgment
• You will be able to estimate how large a blind spot remains in a check that excludes itself from the targets of matching. I describe a real case that the machine matching could not find and only an AI reading the meaning found, and why I set aside the idea of putting my repository back among the targets
• You will be able to plan ahead for how the way you record already-judged matches can force you to judge them again. I describe a real case where I had to judge the same position a second time

This article is a discussion based on primary records from operations I continue as individual research.

What a string match does not settle

I start with the conclusion. Even if strings match between two documents, that does not settle which one is the original. Nor can the match itself tell you whether one copied the other, or whether both simply used the same public identifiers or naming habits.

I made three errors with this check: a false positive from mistaking the direction, a false negative from excluding myself from the targets, and a defect in the handling of allowed words. A false positive is the error of judging something that was not copied to be a copy, and a false negative is the error of missing a copy. All three come from one and the same property: I tried to decide the direction of the original, and whether a text was mine or someone else's, from the matching strings alone.

I judge the direction from time information, not from string matches. I compare three dates queried with git. They are the date the article body was last changed, the date of the last full matching run that found 0 matches, and the date that text entered the source material.

External plagiarism detection rests on the same premise. I confirmed the following three sources in their original text on 2026-09-16. According to the English Wikipedia entry Content similarity detection, an external detection system is defined as one that compares a suspicious document with a reference collection. That collection is a set of documents assumed to be genuine. In the generated plagiarism detection task of PAN at CLEF 2025 as well, suspicious documents and source documents are separated in advance as part of how the data is constructed. According to Reception Reader, a study dealing with text reuse in historical materials, pairwise similarity comparison is reported not to explain the direction of reuse over time. The same study lists, as future development, giving the direction by a process that links text fragments back to their earliest occurrence.

How the check that detects copies is built

Before going into the errors, I show what this check matches and how. The following table is the configuration as of 2026-09-16.

The minimum match length of 16 characters is the result of measurement. At 8 characters, so many common technical Japanese phrases and command strings matched by chance that the results were useless for judgment. The check does not output the matching strings themselves because the check's output would itself become a new path for leaks.

This platform itself is excluded from the targets because the article files exist in its working tree. Without the exclusion, an article always matches itself. Documents in the source material that contain quotations from this platform can also be excluded from matching by specifying their paths. But only documents that quote content originating here may be excluded. If the source material's own internal documents were put on that exclusion list, no match that ought to be stopped would be detected at all.

A reverse-direction false alarm caused by quotations in a source document

The first false alarm was on 2026-07-18. In a bulk matching run, 3 articles in the stock waiting for publication failed. The match lengths were 16, 24, and 16 characters, and the same single file also matched 8 published article directories.

The other side of the match was a file in one of the source materials, the repository for research drafts and research operations. This file analyzed the published articles of this blog. On 2026-09-16 I checked it again, read-only. This file was added on 2026-07-16 and has not changed since that day. I had not copied internal text of the source material into the articles; the source document quoted body text already published here.

At that time I added a mechanism that specifies, per source material, paths to exclude from matching. I also wrote into the rule documents the restriction that only documents quoting content originating from this platform may be excluded.

This false alarm was reproduced on 2026-09-16 as well. I ran the reproduction in a copied working directory set to the same state as production, and referred to the original repository read-only. Removing just one exclusion produces 29 matches against that same single file from 9 published articles. Of these, 21 are body text of 6 articles published between 2026-07-07 and 2026-07-11. Those were published before 2026-07-16, when that single file was created, so they are reverse-direction matches where prose from here was taken into the source document. The longest was 65 characters. The remaining 8 are fragments of 3 articles published between 2026-08-06 and 2026-09-03. Their content was the paths of published article directories and the names of skills I have published. Removing all 9 exclusions gives 58. These counts of 29 and 58 are under the reproduction condition with exclusions removed, so they cannot be compared side by side with the counts for the production configuration I describe later.

The blind spot left by excluding my own repository

While writing one draft, I had echoed the wording of other documents in two places in my draft. The documents I echoed were this repository's own rule documents and the implementation of the reader-facing chat. The lengths were 34 and 19 characters respectively. The machine matching could not detect them, and only an AI reading the meaning found the echoed wording. As long as I exclude my own repository from the targets, wording echoed from my own internal documents cannot be found by the machine matching.

I considered simply putting it back among the targets and set that idea aside. This repository's rule documents quote the sentences of articles themselves as examples of review comments. The handoff records and operations data contain summaries of the articles' content. A machine cannot distinguish a quotation that went from an article into a document from a copy that went from a document into an article. Putting the repository back among the targets would make the list of exclusions too broad, and the check would become a formality.

The approach I am now moving toward is self-matching against only those places that, by their structure, contain no quotations of articles. The code of server-side processing, the site implementation, and scripts fall into those places. There is nowhere in the code that quotes an article, so the ambiguity of direction does not arise. For now, copies in this range are found by the AI review that reads the meaning.

The defect in handling allowed words

The third error, the defect in handling allowed words, was a defect in how the content of a match was counted. This check has an allow list of general names registered as approved for publication and widely known technical terms. What remains after removing those from a match is the specific content. On 2026-08-13, I fixed a defect where an allowed word could not be removed when a match boundary fell inside the word. Before the fix, fragments of the product names of public AI models were counted as specific content. The fix changes the counting so that it first finds the ranges the allowed words occupy on the full text of the line and counts only the characters not covered. I ran the check over all published articles and drafts before and after the change, and confirmed that matches already judged continued to be detected.

Recording matches already judged, and judging the direction again

Once I finish judging the direction of a match, I record the result with the reason and the date, and later runs pass it. Below, I call registering a match in this record "acceptance." I created the acceptance record on 2026-08-11. As of 2026-09-16 it has 26 lines. The articles with an acceptance are 17 published articles. Matches in drafts and in articles waiting for publication cannot be accepted.

The grounds for the direction written on the 26 lines are all time information. Not one was decided by the string match itself. The 26 lines fall into four kinds. Of these, 20 lines are reverse-direction overlaps where wording from here was later taken into a source document. Another 3 lines are matches where both sides shared field names of a public API or path names of a repository structure. Just 1 line is a case where both sides quoted the same sentence from a published technical article. The remaining 2 lines are matches where common phrasings containing particles lined up by chance. There is not a single match in the direction that ought to be stopped, that is, where an article echoed internal text of the source material.

Judging the direction does not end in one pass. On 2026-09-16, a full matching run in the production configuration reported 7 failing matches across 3 published articles. All 7 were the same type of sentence, a set phrasing that states the typesetting rule for mixing Japanese and Latin text. The 3 failed articles all took that rule itself as their subject. I traced the origin of this rule in the article Who decided on the half-width space between Japanese text and alphanumerics?. Querying the direction with git showed the reverse direction: the same phrasing had been taken into source documents on 2026-09-16 and 2026-09-02.

I had judged the direction for these 7 positions on 2026-08-26 and 2026-09-07 and accepted them. They failed again because at that time I recorded only one matched source material. On this day the same positions also matched another source material. Each acceptance pins down not only the text of the fragment but also the set of identifiers of the source materials that matched at that time. Each time the same text is later taken into another source material, a match already judged goes back to failing.

The count grows each time the source material moves forward. When I ran full matching on 2026-09-07, new matches appeared across 8 published articles. The count ranged from 17 to 24 depending on when the run happened. That is because the working trees of the source material move with every run. After judging the direction of all of them, the number in the direction that ought to be stopped was 0.

For drafts I fix things differently. The matches on 2026-09-08 and 2026-09-11 were both reverse-direction, where a source document later overlapped. Even so, I resolved them by rephrasing the relevant sentence in the draft without changing its meaning. The list of facts I made to write this article also produced 7 matches when I matched it right after finishing it. Of these, 6 were the connective words inside English quotations taken from external primary sources, the parts that only join the phrasing together. The core words of the quotations matched no source material. The remaining 1 was the name of the directory where withdrawn drafts are kept. The source material side had also named the same kind of directory with the same word. So I had not copied text from the source material; both sides simply used the same naming habit. Matches in drafts cannot be accepted, so I shortened the English quotations to only their core words. I also rewrote the directory name into a Japanese description without giving the name, and resolved the matches. When I matched again after rewriting, it came to 0. In a full matching run in the production configuration on the same day, the other 4 draft articles, 8 files, also passed with 0.

A design that revises the targets of matching without loosening the check

Since I created the script for this matching check on 2026-07-12, I have rewritten it 18 times in two months and one week. The latest change was on 2026-09-07. In the rewrites I added allowed words, added reverse-direction matches to what can be excluded, created the acceptance record, and fixed the defect in allowed words. Not one change loosened the criteria for the judgment itself. For the mistaken direction and the defect in allowed words, the check stopped, and its design of not passing what it cannot judge stayed as it was. I confirmed the cause first, and then fixed the targets of matching and how matches are counted. For the blind spot from excluding my own repository, the check did not stop, and an AI reading the meaning found the echoed wording. The design of not passing when it cannot judge is not unique to this check. I wrote about how I built the same design separately in two places in I built evaluators that do not pass what they cannot judge, in two separate places.

For those who put the same check into their own operations, I write three key points. Decide the direction from time-based evidence, not from string matches. If you exclude yourself from matching, decide from the start that copies in that range are found by an AI review that reads the meaning. And because each acceptance also pins down the set of matched source materials, you will judge the direction again each time the source material grows.

I set the limits of my claims myself. In this operation, my articles, which do the observing, and the source documents, which are observed, form a loop connected by quotation. The generalization that this configuration always produces false positives was not supported by a search on 2026-09-16. What external primary sources support goes only as far as two points. One is that external detection assumes the reference collection is genuine. The other is that a match does not explain the direction in time. I write about the loop as a discussion confined to measurements from my own operations.

Research I referred to

• A study dealing with text reuse in historical materials. On 2026-09-16 I confirmed in the original text that pairwise similarity comparison does not explain the direction of reuse over time, and that the study states, as future development, giving the direction by a process that links text fragments back to their earliest occurrence. Reception Reader: Exploring Text Reuse in Early Modern British Publications (David Rosson, Eetu Mäkelä, Ville Vaara, Ananth Mahadevan, Yann Ryan, Mikko Tolonen. Journal of Open Humanities Data 9, 2023-04-17. DOI 10.5334/johd.101) https://openhumanitiesdata.metajnl.com/articles/10.5334/johd.101

Materials I referred to

• On 2026-09-16 I confirmed that an external detection system is defined as one that compares a suspicious document with a reference collection assumed to be genuine. Content similarity detection (English Wikipedia) https://en.wikipedia.org/wiki/Content_similarity_detection
• On 2026-09-16 I confirmed that, in how the task is constructed, the roles of documents are separated in advance into suspicious documents that may contain reuse and source documents that are the other side of matching. Generative Plagiarism Detection (PAN at CLEF 2025) https://pan.webis.de/clef25/pan25-web/generated-plagiarism-detection.html

Originally published at The Future of Humans, AI, and the Web, a site where my research and development is recorded and analyzed by a human and an AI.