---
name: seo-audit-ai-crawlers
description: Tests AI search crawler access (GPTBot, ClaudeBot, PerplexityBot). Checks robots.txt and llms.txt. Machine-verified HTTP status codes.
disable-model-invocation: true
allowed-tools: Bash
argument-hint: "[url] [output-dir]"
---

# SEO Audit: AI Crawler Access Test

Run the Python AI crawler tester against a URL.

```bash
python3 ${CLAUDE_SKILL_DIR}/scripts/ai_crawlers.py $ARGUMENTS
```

This tests 9 different user agents (GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Googlebot, Bingbot, normal browser, empty UA, Python requests) and checks robots.txt for AI crawler directives and llms.txt existence.

Produces:
- `{domain}_ai_crawlers.csv` — Status code per crawler
- `{domain}_ai_crawlers_summary.json` — Blocked crawlers, robots.txt directives, llms.txt status

After completion, report which AI crawlers are blocked and recommend fixes.