Why is my word count different from Microsoft Word's?

The Korean translation came back with result_words: 0. I had the file open in front of me while I read that.

Nothing was wrong with the file. Correct Korean, nothing dropped, the job had run start to finish. The counter just said the thing contained no words.

I was going down our CLI's subcommands one at a time against the live test environment, mostly checking that nothing had rotted since the API changed shape four times in a single day. Every command exited 0. Every output file opened. I only noticed the zero because it happened to be printed on the screen next to a filename.

That number decides what a job costs, because we bill on words. I work on a tool that rewrites prose inside .docx files and charges by the word, so read the rest as somebody with an interest in the answer. So a 1,700 word Korean document had run the whole pipeline, called the model, produced the file, and charged nothing. Russian went the same way, and so did Japanese written only in kana. Korean, Russian and Japanese are all on our public API's advertised list.

Here is the ruler that did it. Two regexes, added at different times by people solving different problems:

HAN = re.compile(r'[\u3400-\u4dbf\u4e00-\u9fff\uf900-\ufaff]')
ASCII = re.compile(r"[A-Za-z0-9]+(?:[._'-][A-Za-z0-9]+)*")

def count(text):
    return len(ASCII.findall(text)) + len(HAN.findall(text))

Hangul is not [A-Za-z0-9] and it is not a Han ideograph. Neither is Cyrillic, neither is Greek, neither is a hiragana. They match nothing, they contribute nothing, the answer is zero.

Now the part I actually want to talk about, which is why that lasted.

Feed the same function English and it is right. Feed it Chinese and it is right. Same function, same afternoon:

Across the twelve languages I checked, that ruler gets three of them right.

A counter that returns nonsense for everything gets found in an afternoon. This one is correct on plain English and on Chinese, which is what every fixture in the repo happened to be written in, so nothing ever went red. The zeros were not hiding. Nobody had asked.

The only reusable thing in the whole story is what should have happened next. When a count comes back zero, run the same code over something you are certain contains the thing you are counting. If it still says zero, your counter is broken and the measurement is void. If it comes back right, the zero is about your input and you can act on it. It costs about a minute either way. I hadn't done it, because I wasn't measuring anything, I was reading exit codes.

The same regex was wrong in the other direction wherever an accent turns up. [A-Za-z0-9]+ stops dead at é, so le développement is three tokens instead of two, and O avanço rápido da inteligência artificial is six words that the ruler calls nine. French, German, Spanish, Portuguese, Polish and Italian documents were all billed above their length, which is worse than free. Free is only embarrassing.

The fix looks obvious once you see it, since [^\W_] means "letter or digit in any script":

ANY = re.compile(r"[^\W_]+(?:[._'-][^\W_]+)*")

Twelve out of twelve. Still wrong, though, in a way that shows up on real files and not on test sentences, because it splits on punctuation inside a token. Three that came out of actual documents: Fiona’s thesis counts 3, because the joiner set has the straight ' in it and not the curly ’ that Word autocorrects to. a grant of $300,000 counts 5, on the comma. the clip runs 1:07-1:49 counts 6, on the colon. Word says 2, 4 and 4, and so does anybody reading the page.

So we stopped writing a regex for it. Word's ruler is much closer to str.split() than to anything clever. Whitespace separates words, punctuation inside a token belongs to the token.

That held until a 1,387 word document came back counted as 1,383. Four short. It had four em dashes with no spaces around them, joining word pairs like telling—amongst. Microsoft Word breaks a word at an em dash or an en dash but not at a hyphen, so cost—benefit counts as two words and well-known counts as one. Four dashes, four missing words, exactly.

I didn't work that asymmetry out from our own files alone. The clearest demonstration I found is on UT Austin's legal writing blog, from people who care because briefs have hard word limits: "With a hyphen, Microsoft Word counts this as one word: 343-44", and "With an en dash, it counts it as two: 343–44". Same digits, half the count. So you can't throw every dash-shaped codepoint at the problem and go home.

The one we looked at and deliberately left alone is the slash. Word splits mg/day into two words, so splitting on / looks like the same class of fix. It isn't, because Word does not split https://example.com/a/b or 12/07/2026. It knows what a URL and a date look like, and a character class doesn't. We tried it on a references-heavy document and gained more than 160 phantom words, essentially all of them URL fragments in the bibliography, which is about what you'd expect once a single DOI turns into five or six tokens. Leaving the slash out undercounts our documents by less than 0.2%. Overcounting is somebody's money, so it stays out until someone writes a tokenizer that knows the difference.

Twelve languages are pinned in a test file now, one case each, with the count a reader of that script would give. Put the old two regexes back and 11 of the 17 cases in that file go red. That number is the one I care about, more than the 17 green ones, because the six that stay green under the broken ruler are English, Italian, Chinese, a mixed-script line, hyphenation, and dashes. Which is to say, exactly the cases somebody would have thought to write by hand. Italian only slipped through because dell'intelligenza uses a straight apostrophe.

Paste this to get the table above:

import re

HAN = re.compile(r'[\u3400-\u4dbf\u4e00-\u9fff\uf900-\ufaff]')
ASCII = re.compile(r"[A-Za-z0-9]+(?:[._'-][A-Za-z0-9]+)*")
NOSPACE = re.compile(r'[\u3400-\u4dbf\u4e00-\u9fff\uf900-\ufaff\u3040-\u309f\u30a0-\u30ff]')
DASHES = re.compile(r'[\u2012-\u2015\u2190-\u21ff]')

def broken(t):
    return len(ASCII.findall(t)) + len(HAN.findall(t))

def current(t):
    spaced = DASHES.sub(' ', NOSPACE.sub(' ', t)).split()
    return len(spaced) + len(NOSPACE.findall(t))

CASES = [
    ('The rapid advancement of artificial intelligence', 6),
    ('人工智能改变了行业', 9),
    ('인공지능의 급속한 발전은 수많은 산업을 변화시켰습니다', 6),
    ('Стремительное развитие искусственного интеллекта', 4),
    ('Η ραγδαία ανάπτυξη της τεχνητής νοημοσύνης', 6),
    ('これはたいせつだ', 8),
    ('Fiona’s thesis', 2),
    ('a grant of $300,000', 4),
    ('the clip runs 1:07-1:49', 4),
    ('a cost—benefit analysis', 4),
    ('a well-known result', 3),
]
for text, want in CASES:
    print(f'want {want:2}  broken {broken(text):2}  current {current(text):2}  {text}')

The ranges are written as \uXXXX escapes on purpose, and I found out why while drafting this. I had them in as literal characters, and something in my editing chain NFC-normalized the file, which turned the compatibility ideograph U+F900 at the end of the Han range into U+8C48. That widens the range enough to swallow Hangul, so the broken ruler stopped returning 0 for the Korean line and started returning 24, which is the syllable count. My example of a counter being wrong was itself wrong, in a new way, for one draft. Escapes are ASCII and survive normalization.

What I still don't have is Word's real rule for the slash. Those three cases come out of our own notes on what Word did with real files, not from any spec, and "it knows what a URL is" is a guess at the shape rather than something I can point at. If you have matched Word's count on a corpus with URLs and dates in it, I'd like to read how.

The tool, if you want to look at it, is HumanPen.