Your access log already knows whether ChatGPT is citing you

There's a lot of guessing about whether AI systems are citing your site. Analytics won't tell you — most of these referrals arrive with no referrer at all.

Your access log will, and it's more specific than people realize: two different classes of user-agent mean two completely different things, and only one of them is evidence that you were actually cited.

I've been logging this on a small site since launch. Here's the mechanism, the distinction that makes it useful, and the trap that will make you over-count.

The distinction that matters

Group the AI user-agents you see into two buckets:

Bucket 1 — crawlers. GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Claude-SearchBot, Applebot-Extended, CCBot. These are building or refreshing an index. A hit here means you're in a crawl queue. That's necessary, it's not an outcome, and it says nothing about whether anyone ever saw your content in an answer.

Bucket 2 — user-triggered fetches. ChatGPT-User, Perplexity-User, Claude-User. Different job entirely: a person was in a conversation, and the assistant fetched your page to answer them — either browsing live or following a citation the user clicked.

That second bucket is the closest thing to direct evidence that your page participated in an answer. Naming convention across vendors is conveniently consistent: -Bot and -SearchBot suffixes are infrastructure, -User suffix means a human is on the other end.

Most write-ups on this topic lump all of these into one "AI traffic" number, which throws away the only distinction you actually care about.

Capturing it without a log pipeline

My stack has no nginx in front of the app, so there's no access log on disk to grep. If you're in the same position, you don't need to add one — the edge proxy can log the hit itself.

The whole mechanism is a regex and a console.log in the request path:

const AI_BOT_RE =
  /GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-User|Claude-SearchBot|anthropic-ai|PerplexityBot|Perplexity-User|Google-Extended|Applebot-Extended/i

const match = (request.headers.get('user-agent') ?? '').match(AI_BOT_RE)
if (match) {
  console.log(`[ai-bot] ${new Date().toISOString()} ${match[0]} ${request.method} ${pathname}`)
}

Then whatever collects your stdout is your dataset:

# which agents, how often
awk '{print $3}' archive.log | sort | uniq -c | sort -rn

# which pages they took
awk '{print $3, $5}' archive.log | sort | uniq -c | sort -rn

# the evidence bucket — usually few enough to read line by line
grep -E 'ChatGPT-User|Perplexity-User|Claude-User' archive.log

Two things worth doing before you trust the numbers:

Archive out of the log buffer. Process managers rotate, restart, and flush. A -User hit may appear exactly once, ever — if it lands in a buffer that gets cleared, it's gone and you'll never know it happened. A small script on a periodic cron that appends new lines into an append-only file, deduped, is enough. Make it idempotent so re-running is free.

Check what your matcher excludes. Mine initially skipped llms.txt, robots.txt, and sitemap.xml, which are precisely the files an AI crawler hits first. Those counts read as zero for two days and the zero was an artifact. If your proxy has a path matcher, audit it against the paths bots actually want.

The trap: fake AI user-agents

Here's the part I hadn't seen written up anywhere, and it will corrupt your data if you don't handle it.

Some of your "AI crawler" traffic is a vulnerability scanner wearing a costume. One 83-second burst on my site: 60 requests rotating through 6 different AI user-agent strings, hitting paths like /aws-credentials and /id_rsa, plus query strings carrying shell-injection probes.

Spoofing a user-agent is trivial, and AI crawler strings are a good disguise precisely because everyone has decided to allow them.

If you count those as AI interest, you'll conclude the robots love you. Tag them instead:

const SCANNER_RE =
  /\/(?:etc\/passwd|proc\/self|cgi-bin\/|id_(?:rsa|dsa|ed25519)|private-key|credentials|secrets|api-keys|wp-(?:admin|login)|phpmyadmin|xmlrpc)|[?&](?:cmd|exec|command)=[^&]*(?:%3B|%60|%7C|;|`|\|)/i

const tag = SCANNER_RE.test(`${pathname}${search}`) ? 'ai-bot-scan' : 'ai-bot'

Two deliberate choices there. Blacklist suspicious paths rather than whitelisting your real ones — if a blacklist misses something you get a few junk lines, but if a whitelist misses a page you just shipped, it misclassifies a genuine citation as a scan. One failure mode is noise; the other is silently destroying your only copy of a signal that occurs once.

And keep the scan lines, just tagged differently. They're still useful for security review; they just don't belong in the citation count.

Five weeks later, the trap was most of my data

I wrote the section above from a single 83-second burst. Then I left the log running and came back on 2026-09-12 — 3,392 lines, five and a half weeks. The naive count:

1,487 *-User hits. Read straight off the log, that says a human-driven assistant fetched my pages fifteen hundred times. It says nothing of the kind.

Roughly 1,330 of those — about 89% — are one machine on a timer. The tell is in the path distribution: / 665 times and /en 664 times — near-perfectly paired, and together dwarfing every real page on the site by two orders of magnitude. Pulling the timestamps for the root hits alone: 661 of them since 2026-08-11, median gap 49 minutes, 84% of gaps falling in a 25–95 minute band, with the / and /en requests fired 0.1–0.7 seconds apart. People clicking citations in conversations do not arrive on a metronome, in pairs, on two URLs.

The rest of the costume is more familiar. Scanner probes wearing *-User strings went after cloud credentials across all three major providers, plus the usual file-read attempts — the AWS link-local metadata address for IAM info, the Azure OAuth token endpoint, the GCP service-account token path, ..%2F..%2F.env, and file:///proc/self/environ. Claude-User asked for /Dockerfile and /metrics. Perplexity-User asked for /log-viewer.

Strip the metronome, the scanner paths, robots.txt and the login probes, and what remains is on the order of 80 *-User hits that actually landed on content — spread across the true-solar-time explainer, the day-master and how-to-calculate pages, the method page in both languages, the case studies, and the conformance test.

Eighty is a real number and a much better one. It is also about 5% of the raw count. That is the size of the error you inherit by trusting the user-agent string: not a rounding difference, a factor of nineteen — and it points the wrong way, flattering you. Had I shipped a dashboard in August that plotted *-User hits without filtering, it would have shown a beautiful curve, and every point on it would have been a lie.

The lesson isn't that logs don't work. It's that the filtering is the measurement. The regex that finds AI user-agents is the easy half; the half that decides which of those hits are real is where the signal actually gets made.

What it actually looked like

Small numbers, and I'm going to keep them small rather than dress them up. Three readings from one site:

Three things showed up in that data that I would not have gotten any other way.

Crawlers behave differently from each other. On launch day one crawler hit all ten new pages exactly once each, no repeats — walking a list. Another arrived two days later and only touched the homepage and one section, which reads like discovery rather than indexing.

The first -User hits are legible individually. With two of them, you read the lines:

Both landed on the methodology page — the one documenting how the product computes what it computes. Neither hit a content page. With n=2 I'm not going to build a theory on it, but it's a specific, checkable observation about which page an answer engine went to get facts from, and it pointed me at something I'd have deprioritized.

Zero is information, if you took the baseline first. The pre-launch row exists because I recorded it before shipping. Without it I couldn't distinguish "no one is citing us" from "we've always looked like this."

The honest limits

• Spoofable, and not conservatively so. Everything above is a self-reported header. The scanner section and the metronome are the proof. It's tempting to assume spoofing means you're under-counting — the five-week data says the opposite: the raw number was ~19x the filtered one, inflated by traffic pretending to be an assistant. An unfiltered count is not a floor. It isn't a ceiling either. It's just not a measurement.
• A -User hit is not a citation you can quote. It means a page was fetched to serve a conversation. You don't get the question, the answer, or whether you were quoted or contradicted.
• Small n stays small — and big n is usually small wearing a costume. Two hits is two hits. Fifteen hundred hits was eighty hits and a monitoring bot. Neither number gets to become a trend just because it's convenient.
• Your CDN may absorb it. If a cached response never reaches your app, your proxy never logs it. Worth checking whether your edge is answering on your behalf.

Why bother

Because the alternative is inferring your way to a conclusion you like. The AEO advice market is full of tactics with no measurement attached, and most of them can't be evaluated by the person selling them either.

This costs one regex, one console.log, and a cron job. It won't tell you what the model said about you. It will tell you, with dates, whether anything on the other side ever came to your server to find out — and which page it wanted.

If you set it up, take the baseline reading before your next launch. That's the row you can't reconstruct later.

The page those first two fetches went for is the method page — the computation, the constants, and where the numbers come from. The product is auspiceoracle.com.