# llms.txt - AI crawler access policy # Version: 1.0 # Last updated: 2025-07-11 # NOTE: This file is for informational and signaling purposes only. # To control crawler access, refer to /robots.txt # Major LLM Crawlers (confirmed user-agents) User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: Edgebot Allow: / User-agent: NeevaBot Allow: / # International AI or Search Crawlers User-agent: Baiduspider Allow: / User-agent: YandexBot Allow: / # Placeholder entries for unlisted or emerging LLMs # These do not have official public user-agents (as of now) # Keep them for future tracking or log correlation # User-agent: WritesonicBot # Allow: / # User-agent: NimbleBot # Allow: / # User-agent: OutriderBot # Allow: / # User-agent: MistralBot # Allow: / # User-agent: CohereBot # Allow: / # User-agent: GrokBot # (xAI – Elon Musk's LLM company) # Allow: / # Catch-all rule User-agent: * Allow: /