You paste a paragraph from an AI chat into a job application. It looks clean. You submit it.
What you cannot see is that a handful of characters came along for the ride: a zero-width space wedged between two words, a non-breaking space where a normal space should be, curly quotes instead of straight ones, and a dash that is technically a different character than the one on your keyboard. None of them show up on screen. All of them are really there.
Most of the time this is harmless. Sometimes it quietly breaks things: a search that will not match, a copy-paste that renders a little box instead of a letter, a form that rejects your input for no reason you can see, or an applicant tracking system that parses your resume into nonsense. And lately it has a second life in the news, because those same invisible characters are showing up in conversations about AI watermarking and hidden text.
I hit this problem enough while building my own job-search tool that I built a cleaner for it. Here is what is actually going on, and what I do about it.
The part everyone should know
Text on a computer is not just the letters you see. It is a stream of characters, and plenty of characters are designed to be invisible or nearly so. They exist for good reasons: to hold words together, to control spacing, to support languages that need them. The trouble starts when they get into your text without you knowing.
A few common ways that happens:
- Copy and paste. Copying from a web page, a PDF, or a chat window often brings along formatting characters that were never meant to leave that page.
- Smart formatting. Many tools "helpfully" turn straight quotes into curly ones and two hyphens into a long dash. Nice on a printed page. Not always welcome in a form field or a code box.
- AI writing tools. Text generated by an AI assistant tends to carry its own house style: certain spacing, certain punctuation, the occasional invisible spacer.
Why should a normal person care? Because the places where your words matter most are often the pickiest:
- Applicant tracking systems read resumes as plain text. A stray invisible character can split a word, hide a keyword, or garble a line, and you never find out why you did not hear back.
- Search and forms match on exact characters. A curly quote is not the same character as a straight quote, so a search for your name or a title can silently miss.
- Rendering. Send text with an odd character into an app that does not support it and your reader sees a little box or a question mark where a letter should be.
This is not exotic. It is the digital equivalent of showing up to an interview with a price tag still on your sleeve. Small, invisible to you, and worth removing before anyone else notices.
The news angle, and where I draw the line
If you follow AI news at all, you have probably seen two related stories.
One is watermarking: the idea that AI-generated text can carry hidden signals, sometimes as invisible characters, so it can be identified later. The other is hidden text used to smuggle instructions, where invisible characters are tucked into a document or a web page to quietly influence an AI system that reads it. Both are real, and both are worth understanding.
I want to be clear about what this article is not. It is not a guide to defeating watermarks or dodging AI detectors. I think that is the wrong goal and, honestly, a losing game. My argument is simpler and older than any of this: you should be able to see everything that is in your own writing, and you should be able to ship it clean. Clean text is portable, professional, and predictable. Whether an invisible character came from a chat tool, a copy-paste, or a watermark, the fix is the same, and the reason is the same. You are the author. You get to decide what is in the file.
Stripping invisible characters does not defeat a real watermark either. What actually reads as machine-written is cadence, every sentence the same shape and length, and no character cleanup fixes that. This is about clean, honest text, not disguise.
Call it text hygiene. Same spirit as spell-check, just for the characters you cannot see.
For the technically curious
Everything above is the whole point for most readers. If you want the mechanics, here they are.
The characters worth watching fall into a few buckets:
-
Zero-width characters. Zero-width space (
U+200B), zero-width non-joiner (U+200C), zero-width joiner (U+200D), and the word joiner (U+2060). These take up no visible space at all, which is exactly what makes them easy to miss. -
The byte order mark / zero-width no-break space (
U+FEFF). Frequently hitchhikes at the start of copied text. -
Non-breaking and exotic spaces. The non-breaking space (
U+00A0) is the common one. There is also a whole family of unusual spaces (thin, hair, figure, and so on) in theU+2000toU+200Arange. -
Smart punctuation. Curly single and double quotes (
U+2018,U+2019,U+201C,U+201D), plus the en dash (U+2013) and em dash (U+2014) that autoformatters love to insert. -
Tag characters (
U+E0000toU+E007F). An obscure block that has become the vehicle for so-called ASCII smuggling, where readable-looking text hides an invisible payload.
The cleaning logic is not complicated in spirit: strip the characters that should never be in plain text, normalize the ones that have an obvious plain equivalent, and leave everything legitimate untouched. The library exposes a few tiers so you can match how aggressive the cleaning is to where the text is going. Install it with npm install ai-text-hygiene, then:
import { clean, stripInvisible, cleanConservative } from 'ai-text-hygiene';
// Full house-style clean for prose: strips invisibles, folds curly
// quotes to straight, em dash to a comma, ellipsis to three dots.
clean('We "delivered" results — on time…');
// => 'We "delivered" results, on time...'
// Strip-only tier, safe for any language (no punctuation changes).
const zwsp = String.fromCharCode(0x200B); // a zero-width space, invisible in input
stripInvisible('in' + zwsp + 'visible');
// => 'invisible'
// Length-stable tier for capped fields: 1:1 swaps, no growth.
cleanConservative('curly "quotes" to straight');
The important design choice is what you do not strip. Normalizing is a judgment call: turning a curly quote into a straight one is safe and almost always what you want in a form field, but you would not want to flatten legitimate content in a language that depends on characters an overeager filter might catch. That is why the strip-only tier is the one that is safe for any language, while the full clean targets English house style. The real work is in choosing the right set to remove versus the right set to normalize, and in doing it predictably every time.
In my project this lives in a single module (text-hygiene-core) so the same rules apply everywhere text leaves the app, and it is published on its own as an open-source library under the MIT license. If you want to read the actual code or drop it into your own tools, the repo is here:
Why I built this
I did not set out to write a Unicode cleaner. I built trajecktory, a tool to run my own job search like a pipeline, and text hygiene turned out to be one of those small, unglamorous problems that quietly matters. Every resume, every cover letter, every message that leaves the app should be clean, plain, and exactly what I wrote. Nothing hidden, nothing I cannot see.
If you have ever wondered why a form rejected text that looked perfectly fine, this is often the reason. Now you can see it, and clean it.
trajecktory is open source and still evolving. If this was useful, the code is on GitHub and I write about building it as I go.
