AI Readiness
Check your site's readiness for AI crawlers
What is AI Readiness?
Language models (ChatGPT, Claude, Perplexity, Gemini) index websites and cite them in responses. The better a site is optimized for AI, the more often it will be cited as a source — translating into traffic and authority.
robots.txt and AI bots
GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot and Google-Extended are AI crawlers. You can block them (Disallow: /) or explicitly allow them (Allow: /). No rule = bots decide themselves.
llms.txt — new standard
Like robots.txt, the /llms.txt file is a proposed standard (llmstxt.org) for describing site content for AI models in a concise Markdown format.
schema.org and structured data
JSON-LD with Schema.org helps AI models understand the site context: organization type, author, article topic. Sites with schema are more often cited with metadata preserved.