Skip to main content

Can AI search actually read your site?

Build a read-only checkup skill for Claude Code that shows exactly which AI crawlers can reach your pages, then fix what it finds in one afternoon.

23 min read

Can AI search actually read your site?

More and more of your traffic starts with a question typed into ChatGPT, Claude, or Perplexity instead of Google. Those tools can only cite pages their crawlers are allowed to fetch and can actually read. A lot of creator sites quietly fail that test: a robots.txt line copied from a forum years ago, a firewall rule that blocks "bots," a page that is empty until JavaScript runs, or an llms.txt that turns out to be the site's HTML shell.

In this guide we build a small checkup you can run on any site in about twenty seconds. It's one Python file with no dependencies, wrapped in a Claude Code skill. You type /ai-visibility https://yoursite.com, it reports what each AI crawler is allowed to see, and Claude drafts a fix list plus ready-to-upload robots.txt and llms.txt files. It never touches your live site. You decide what ships.

The rest of this guide is for Pro members

4,822 more words. Creators Toolbox Pro includes every paid guide. $5/mo or $45/yr. Cancel anytime.

Become a Pro member