A security disclosure dubbed Dark Sourcery alleges that scammers are manipulating the web pages retrieved by AI search products so answers repeat fake customer-support contacts. The mechanism is credible and immediately useful to understand: the attacker does not need to jailbreak the chatbot if they can make a lie look repeatedly corroborated in search results. But the disclosure’s claims of 374 affected companies and tens of thousands of pages have not been independently audited.
Key facts
- Ariel Simon published the primary disclosure under the title “Dark Sourcery.”
- Simon says his team tested answers from ChatGPT, Gemini and Google AI Overview.
- The post reports 374 affected companies and tens of thousands of malicious pages; it supplies no public dataset or reproducible query log.
- The claimed target is false phone numbers, URLs, email addresses and update instructions that look like official support information.
The hook is familiar to anyone who has called a fake help line from a search result, but AI answer engines can make the mistake feel more authoritative. A conventional search page exposes a crowded list of links. A chatbot condenses that list into a confident-looking instruction and may place the number directly beside a brand name. The user sees a citation and may infer that the fact has been verified. It has not.
Simon describes attackers placing the same or similar false contacts across user-generated posts, PDFs, hosted pages, reviews, fundraising or job platforms, and Q&A-like content. The details are tuned for retrieval: “24/7,” “updated 2026,” repeated formatting, numbers split by spaces, dots, emoji, Unicode characters or words, and mixtures of genuine and fraudulent details. The goal is not one viral page. It is the appearance of independent corroboration. Think of an answer engine as a hurried researcher: if ten apparently unrelated notes make the same false claim, it may summarize the pattern without realizing one campaign wrote all ten.
The post says its measurement system analyzed answer sources, fetched content and counted an incident when a false phone number, URL or email address appeared in an answer. It also says results were statistical and did not reproduce every time a query ran. That admission is important. Search rankings, model versions, location, timing and query wording change; an observed answer is not a stable property of every user’s session.
A short attributed phrase captures the risk: Simon calls the campaign “how hackers manipulate AI to scam you.” The title is vivid, but readers should separate the observation from the scale claim. The author names a research team and a Vigilance contact, yet the public company footprint is thin. There is no released corpus of malicious pages, no exact transcribed query-to-number pairs for independent replay, and no public response from Google or OpenAI confirming the alleged reports. The Hacker News discussion includes plausible anecdotes and requests for reproduction, but no independent validation of the 374-company figure.
The strongest counterargument is therefore evidentiary, not conceptual. SEO spam, phishing and fake support pages predate generative AI; retrieval systems have always had to decide which web sources deserve trust. Dark Sourcery may be a sharp name for an extension of that old problem rather than proof of a wholly novel compromise. Still, AI summaries change the distribution channel. They can turn a low-ranking poisoned page into a short, direct recommendation with far less visible friction.
Platform guidance supports the cautious behavior even though it does not validate this specific campaign. OpenAI’s search guidance says web results can be incomplete, out of date or incorrect and tells users to inspect sources. Google likewise asks Gemini users to review sources. The consumer rule is stronger for high-risk requests: never call a bank, airline, software-support or refund number solely because an AI response displayed it. Type the company’s official domain yourself, open the official app, or use the number on a card or statement.
This is retrieval-side poisoning, related to but distinct from prompt injection and data poisoning. Prompt injection attacks the instruction channel. Training-data poisoning attacks material a model learns before deployment. This campaign, if accurately described, attacks the open web after deployment and relies on the system’s appetite for current sources. The caveat should travel with every retelling: the mechanism is well grounded; the disclosure’s reach, victims and exact examples are not independently established.
Originally published on Ground Truth, where every claim is checked against the primary source.
