Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology): websites and AI crawlers
1,573 websites measured, compared with all 562,003.
Measured by Triremes between 4 November 2026 and 8 November 2026 across 562,003 business websites; groups under 100 sites are not shown.
Share of sites whose robots.txt blocks each AI crawler
Share of sites with each AI search signal
The numbers
| Measure | Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology) | All sites |
|---|---|---|
| Blocks GPTBot | 4.6% | 7.8% |
| Blocks ClaudeBot | 4.6% | 7.5% |
| Blocks PerplexityBot | 0.4% | 0.4% |
| Blocks OAI-SearchBot | 0.1% | 0.3% |
| Blocks ChatGPT-User | 0.4% | 0.5% |
| Blocks Google-Extended | 4.2% | 7.3% |
| Blocks Applebot-Extended | 4.1% | 7.3% |
| Blocks CCBot | 4.5% | 7.6% |
| Blocks Bytespider | 4.3% | 7.8% |
| Blocks Meta-ExternalAgent | 4.4% | 7.4% |
| No robots.txt | 29.6% | 18% |
| Has llms.txt | 7.1% | 6.5% |
| Organization JSON-LD | 11.2% | 11.4% |
| WebSite JSON-LD | 16.5% | 13.8% |
| Title 30 to 60 characters | 31.3% | 29.9% |
By industry
- Administrative Management and General Management Consulting Services (3,119 sites)
- All Other Professional, Scientific, and Technical Services (894 sites)
- All Other Specialty Trade Contractors (530 sites)
- Architectural Services (523 sites)
- Child and Youth Services (589 sites)
- Civic and Social Organizations (667 sites)
- Commercial and Institutional Building Construction (1,608 sites)
- Computer Systems Design Services (1,845 sites)
- Computer and Computer Peripheral Equipment and Software Merchant Wholesalers (388 sites)
- Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services (429 sites)
- Corporations (474 sites)
- Custom Computer Programming Services (2,631 sites)
- Educational Support Services (1,132 sites)
- Educational organizations (551 sites)
- Electrical Contractors and Other Wiring Installation Contractors (1,093 sites)
- Elementary and Secondary Schools (845 sites)
- Engineering Services (2,562 sites)
- Environmental Consulting Services (585 sites)
- Facilities Support Services (388 sites)
- Highway, Street, and Bridge Construction (497 sites)
- Home and construction businesses (522 sites)
- Janitorial Services (633 sites)
- Landscaping Services (415 sites)
- Machine Shops (905 sites)
- Medical, Dental, and Hospital Equipment and Supplies Merchant Wholesalers (506 sites)
- Other Computer Related Services (1,127 sites)
- Other General Government Support (919 sites)
- Other Individual and Family Services (836 sites)
- Other Management Consulting Services (455 sites)
- Other Scientific and Technical Consulting Services (882 sites)
- Other Social Advocacy Organizations (439 sites)
- Plumbing, Heating, and Air-Conditioning Contractors (694 sites)
- Professional Organizations (453 sites)
- Professional and Management Development Training (1,124 sites)
- Professional services (1,046 sites)
- Research and Development in Biotechnology (except Nanobiotechnology) (562 sites)
- Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology) (1,573 sites)
- Restaurants (847 sites)
- Services for the Elderly and Persons with Disabilities (480 sites)
- Software Publishers (710 sites)
By state or territory
- Alabama (1,544 sites)
- Alaska (589 sites)
- Arizona (1,919 sites)
- Arkansas (679 sites)
- California (11,483 sites)
- Colorado (2,466 sites)
- Connecticut (1,365 sites)
- Delaware (471 sites)
- District of Columbia (1,086 sites)
- Florida (6,724 sites)
- Georgia (3,468 sites)
- Guam (119 sites)
- Hawaii (569 sites)
- Idaho (765 sites)
- Illinois (4,492 sites)
- Indiana (1,562 sites)
- Iowa (1,004 sites)
- Kansas (933 sites)
- Kentucky (1,033 sites)
- Louisiana (1,332 sites)
- Maine (559 sites)
- Maryland (4,202 sites)
- Massachusetts (2,457 sites)
- Michigan (3,057 sites)
- Minnesota (1,556 sites)
- Mississippi (687 sites)
- Missouri (1,715 sites)
- Montana (674 sites)
- Nebraska (827 sites)
- Nevada (890 sites)
- New Hampshire (613 sites)
- New Jersey (2,751 sites)
- New Mexico (950 sites)
- New York (5,192 sites)
- North Carolina (2,894 sites)
- North Dakota (351 sites)
- Ohio (3,043 sites)
- Oklahoma (1,160 sites)
- Oregon (1,464 sites)
- Pennsylvania (4,230 sites)
- Puerto Rico (812 sites)
- Rhode Island (425 sites)
- South Carolina (1,259 sites)
- South Dakota (387 sites)
- Tennessee (1,806 sites)
- Texas (7,804 sites)
- U.S. Virgin Islands (115 sites)
- Utah (1,117 sites)
- Vermont (444 sites)
- Virginia (5,421 sites)
- Washington (2,678 sites)
- West Virginia (518 sites)
- Wisconsin (1,705 sites)
- Wyoming (381 sites)
About these numbers
Each figure is the share of measured sites in the group. Measured by Triremes between 4 November 2026 and 8 November 2026 across 562,003 business websites; groups under 100 sites are not shown. Lists are in alphabetical order. These figures describe what sites do, not what they should do.
Data: Triremes. Check your own site with the AI search readiness checker.