Crawled but not cited: closing the gap between AI bots reading you and engines quoting you

Here's a pattern we see constantly: a site's server logs are full of GPTBot, ClaudeBot, and PerplexityBot — thousands of AI-crawler hits a month — and yet when you actually ask ChatGPT or Perplexity the questions your buyers ask, your domain is nowhere in the answer. The bots are reading you. The engines aren't quoting you. That's the crawl-to-cite gap, and it's the single most important thing to measure in AI search right now.

Crawling and citing are two different events. A crawl means an AI company fetched your HTML — into a training corpus, a retrieval index, or an on-demand answer. A citation means that, at answer time, the model decided your specific URL was the best thing to quote for a specific question. You can be crawled ten thousand times and cited zero times, because getting crawled is about access and getting cited is about being the most quotable answer.

Two things have to be true to get cited

• Access: AI bots can actually reach your pages. If robots.txt blocks GPTBot or your content is rendered client-side where crawlers can't see it, you never enter the pool of candidates. This is table stakes — necessary, not sufficient.
• Quotability: when your page is a candidate, the model finds a clean, self-contained answer it can lift. This is where almost everyone loses — the page ranks, gets fetched, and then offers nothing an engine can quote in one line.

The playbook that closes the gap

1. Answer the question in the first sentence. Open each page with one self-contained sentence that answers the prompt outright. Engines quote short, standalone claims; an answer buried below the fold rarely gets pulled.
2. Structure it for extraction. Phrase headings as the questions people actually ask, keep paragraphs tight, and add an FAQ block. Clean Q→A structure is what a model can lift verbatim.
3. Build off-site brand signals. Brand mentions — a YouTube channel above all — are the strongest correlate of AI citations in the data, above backlinks and Domain Rating. Get named on the sites and videos models already read.
4. Keep it fresh. A visible 'updated' date and current stats matter: engines lean on recent pages for time-sensitive topics.
5. Cover the exact prompts you track. Make sure a page answers each prompt head-on, then look at who's cited instead of you and beat their coverage.

See it in one place

IndexerNow's AI score view puts the whole chain on one screen for a chosen site: whether AI-bot tracking is live, your 0–100 discoverability score, and whether engines actually cite you — with the playbook above surfaced exactly when you're crawled but not cited. It's the fastest way to tell an access problem from a quotability one and act on it.

For the mechanics of what each engine looks at, read how to get cited by ChatGPT and Perplexity, and why YouTube and brand mentions beat backlinks for AI visibility.