pigfox.ai

Other Management Consulting Services: websites and AI crawlers

455 websites measured, compared with all 562,003.

Measured by Triremes between 4 November 2026 and 8 November 2026 across 562,003 business websites; groups under 100 sites are not shown.

Share of sites whose robots.txt blocks each AI crawler

GPTBot4% ClaudeBot3.5% PerplexityBot0% OAI-SearchBot0% ChatGPT-User0.2% Google-Extended3.7% Applebot-Extended3.5% CCBot3.3% Bytespider3.3% Meta-ExternalAgent3.5%

Share of sites with each AI search signal

No robots.txt19.3% Has llms.txt12.5% Organization JSON-LD13.4% WebSite JSON-LD23.7% Title 30 to 60 characters24.2%

The numbers

MeasureOther Management Consulting ServicesAll sites
Blocks GPTBot4%7.8%
Blocks ClaudeBot3.5%7.5%
Blocks PerplexityBot0%0.4%
Blocks OAI-SearchBot0%0.3%
Blocks ChatGPT-User0.2%0.5%
Blocks Google-Extended3.7%7.3%
Blocks Applebot-Extended3.5%7.3%
Blocks CCBot3.3%7.6%
Blocks Bytespider3.3%7.8%
Blocks Meta-ExternalAgent3.5%7.4%
No robots.txt19.3%18%
Has llms.txt12.5%6.5%
Organization JSON-LD13.4%11.4%
WebSite JSON-LD23.7%13.8%
Title 30 to 60 characters24.2%29.9%

By industry

By state or territory

About these numbers

Each figure is the share of measured sites in the group. Measured by Triremes between 4 November 2026 and 8 November 2026 across 562,003 business websites; groups under 100 sites are not shown. Lists are in alphabetical order. These figures describe what sites do, not what they should do.

Data: Triremes. Check your own site with the AI search readiness checker.