llms.txt
weight 18
The markdown map AI systems look for when they want an overview of your site rather than a scrape.
robots.txt AI bots
weight 18
Whether GPTBot, ClaudeBot, PerplexityBot and Google-Extended are allowed to read you at all.
Schema.org JSON-LD
weight 14
The strongest machine-readable statement of who you are — and what tells you apart from similarly named products.
Sitemap
weight 12
How crawlers find pages beyond your homepage. We follow the URL declared in robots.txt, not just the root.
Title & meta description
weight 10
Still the primary snippet source for classic results and a common source for AI citations.
Canonical URL
weight 8
Stops duplicate and parameterised URLs from splitting whatever authority a page earns.
llms-full.txt
weight 8
The extended context file, for agents that want detail before answering a question about you.
Open Graph
weight 6
Controls how pages appear when shared, and read by several AI crawlers as supplementary metadata.
Indexable page
weight 6
A stray noindex that escaped staging is the most damaging thing on this list. We look for it.
ai.txt
informational
An emerging convention for declaring AI usage policy. Reported, never scored.
Content-Signal
informational
Per-purpose permissions in robots.txt: yes to being cited, no to being trained on.