If you've ever asked ChatGPT or Perplexity "what's a good [category] company" and gotten three confident names back, you've watched a ranking algorithm work — just not the one most of us spent the last decade optimizing for.
This post is a technical look at what's actually happening between the prompt and the answer, why most sites are invisible to it without realizing, and what we at SoftWin now build into client projects by default to fix that.
The problem
Traditional SEO optimizes for position in a list of links. AI answer engines optimize for which two to seven entities get named in a generated sentence. Those are fundamentally different retrieval and ranking problems, and most technical SEO work doesn't touch the second one at all.
If your site is invisible to an LLM's retrieval layer — or your brand has no verifiable footprint outside your own domain — you don't rank lower. You just don't exist in the answer.
How it actually works, mechanically
Most consumer AI search/answer tools (ChatGPT with browsing, Perplexity, Gemini, Copilot) follow a variant of this pipeline:
- Query interpretation — the user's natural-language question gets parsed into intent + entities ("fintech" + "software development company" + implicit "reliable/reputable").
- Retrieval — the system pulls candidate documents/passages via a mix of a live web index and, for browsing-enabled tools, real-time search (retrieval-augmented generation, or RAG). This is passage-level, not page-level — the model retrieves chunks, not whole documents.
- Cross-source aggregation — the model checks how consistently an entity (your company) is associated with the query topic across multiple independent sources, not just your own site. This is where third-party lists, directories, and review platforms carry outsized weight.
- Trust/authority weighting — signals resembling classic SEO authority (backlinks, domain age, structured data validity) combine with newer signals: review platform ratings, award/certification mentions, and social sentiment.
- Generation with citation limiting — the model composes a natural-language answer and typically cites a small, bounded set of sources (often 2–7), not an exhaustive list.
The practical implication: you're optimizing for passage-level retrievability and cross-source entity consistency, not just page rank.
Why this matters for the business, not just the SEO team
Some context worth internalizing if you're building the technical case for prioritizing this: AI-powered answer surfaces (Google AI Overviews, ChatGPT, Copilot, Perplexity) now collectively reach well over a billion monthly interactions, and analysts have been projecting a meaningful decline in traditional search click-through behavior as more of that "who should I use for X" research moves into a single AI conversation instead of ten browser tabs.
If your engineering and content teams aren't deliberately making the site legible to retrieval systems and AI crawlers, you're ceding that surface to whoever happens to be structured correctly — regardless of whether they're actually the better product.
The signals that actually move the needle
Based on available research tracking how ChatGPT, Gemini, Perplexity, and Claude generate business recommendations, here's roughly what correlates with getting cited:
| Signal | Why it matters |
|---|---|
| Repeated mentions on third-party "best of" lists / directories | Strongest and most consistent factor across tools — cross-source consensus is easier for a model to trust than a single self-published claim |
| Awards, certifications, verified partnerships | Cheap trust shortcuts for the model to cite |
| Review platform presence (G2, Clutch, Trustpilot, Capterra) | Volume + rating threshold (generally 3.5–4★+) |
| Domain/content authority | Classic SEO fundamentals still feed the trust score |
| Structured, extractable content | Passage-level clarity — see below |
| Social sentiment (Reddit, forums, news) | Smaller today, growing fast, especially for ChatGPT |
| Case studies / named clients / usage data | Gives the model concrete, citable facts instead of marketing copy |
Notably absent: keyword density, meta tag stuffing, or anything from the classic on-page SEO gaming playbook. These signals correlate with model confidence, not keyword match.
What we actually implement — the https://softwin.io/ checklist
A few concrete, engineering-level things we now treat as standard, not optional:
1. Structured data that actually validates. Organization, FAQPage, and Article schema, not just present but validated (Google's Rich Results Test / Schema.org validator). A minimal example we drop into most client sites:
{
"@context": "https://schema.org",
"@type": "Organization",
"name": "Your Company",
"url": "https://example.com",
"sameAs": [
"https://www.linkedin.com/company/example",
"https://www.g2.com/products/example",
"https://www.crunchbase.com/organization/example"
],
"description": "One clear, factual sentence describing what you do and for whom.",
"award": "Clutch Top B2B Company 2026"
}
2. robots.txt audits — specifically for AI crawlers. This is the single most common invisible failure we find. Check whether you're blocking:
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: Google-Extended
Disallow: /
If any of these are set to Disallow: / (often inherited from a boilerplate template years ago), the corresponding AI tool cannot see your site at all, regardless of content quality. Flip these deliberately, not by accident.
3. Passage-level content structure. Since retrieval pulls chunks, not full pages, we make sure every meaningful section — a service description, a case study result, a pricing tier — can stand alone: clear heading, direct answer in the first sentence, no dependency on surrounding paragraphs to make sense.
4. An llms.txt file where relevant. Still an emerging, non-standardized convention, but for content-heavy sites we're starting to add a plain-language /llms.txt summarizing what the site is and linking to the most citation-worthy pages — cheap to add, low downside.
5. Off-domain entity consistency. Consistent NAP (name/description/positioning) across LinkedIn, Crunchbase, G2, industry directories, and press mentions. Cross-source consistency is a retrieval trust signal; contradictory descriptions across platforms actively hurt.
Common mistakes (that we keep finding in audits)
Blocking AI crawlers unintentionally via inherited robots.txt rules — by far the most common and most invisible issue. Publishing long-form content with no clear structural hierarchy, so there's no clean passage for a model to extract. Zero footprint on third-party platforms — a technically perfect site with no external validation is easy for a model to skip. Letting schema markup go stale or fail validation silently (nobody monitors this the way we monitor uptime, but they arguably should). Optimizing purely for search rank while the rest of the entity's web presence is inconsistent or thin.
FAQ
Does GEO replace technical SEO?
No — it depends on it. Domain authority, crawlability, and structured data are prerequisites, not alternatives. Think of GEO/AEO as an additional ranking layer built on top of solid technical SEO.
Which crawlers should we actually allow?
At minimum: GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, and Google-Extended (feeds Gemini/AI Overviews), unless you have a specific reason to keep your content out of AI training/retrieval entirely — that's a legitimate business decision too, just make it deliberately.
How do we measure whether this is working?
Track citation frequency by running a recurring set of representative prompts against ChatGPT/Gemini/Perplexity, monitor referral traffic tagged from AI assistants where available, and track schema validation status like any other technical health metric.
Is there a risk to exposing more structured content to AI crawlers?
It's a real tradeoff worth an explicit decision — allowing crawlers means your content can be used in AI-generated answers (with or without traffic back to you). For most companies trying to build visibility, the tradeoff favors allowing it; for others with different business models, it may not.
How long until we see results?
Faster than traditional link-building SEO in most cases we've run, because so few competitors are doing this deliberately yet — often weeks to a few months for measurable movement in citation frequency.
Wrapping up
AI answer engines aren't a black box you can't reason about — they're a retrieval and ranking system like any other, just optimizing for different signals than the search engines most of us built our SEO habits around. The technical fixes are usually small (robots.txt, schema validation, content structure); the harder part is building a genuine, verifiable footprint across the web that the model can cross-reference.
We've been running these audits for client projects at https://softwin.io/ and happy to share what a real one turns up. If you want to see what GPTBot and friends can currently see (or can't) on your site, reach out to SoftWin — happy to walk through a free technical pass.
