Is Claude-User blocked on your site?
Enter a domain to see whether its robots.txt lets GPTBot, OpenAI's training crawler, or any of 14 other AI crawlers read the homepage.
Ask about another crawler:
- OAI-SearchBot
- ChatGPT-User
- GPTBot
- Claude-SearchBot
- Claude-User
- ClaudeBot
- PerplexityBot
- Perplexity-User
- Google-Extended
- Applebot-Extended
- Amazonbot
- DuckAssistBot
- Meta-ExternalAgent
- CCBot
- Bytespider
About Claude-User
Fetches pages when people ask Claude questions. Anthropic says blocking it prevents Claude retrieving your content in response to a user query. Read the operator's documentation.
How to block or allow it
robots.txt rules are read per crawler: Claude-User follows a group that names it, or else the User-agent: * group. To block it everywhere:
User-agent: Claude-User
Disallow: /The AI crawler robots.txt generator makes these lines for any mix of crawlers. The check reads only your homepage path, robots.txt, llms.txt and a HEAD of llms-full.txt; see exactly what is fetched.