Technical SEO and AI search
AI search readiness checker
Enter a page address to score how well AI search and answer tools can reach, read and cite it. Each check explains its weight and why it matters, so you can fix the biggest gaps first.
Runs on our edge server; we fetch the public page once and do not store it beyond a 10 minute cache.
Method
How it works
The checker fetches the page, then robots.txt and llms.txt from the same site. It reads the HTML the server returns, because many AI crawlers do not run JavaScript. Each check is worth a fixed number of points. A pass earns the full points, a warning earns half, and a fail earns none.
The bot table applies the robots.txt rules to the page path for each AI crawler, grouped by purpose. Search and answer crawlers can cite or fetch the page. Training crawlers collect content for model training, so blocking them is a separate policy decision.
Structured data is read from JSON-LD blocks and summarised by type. Use the schema validator for property-level checks.
FAQ
Questions people ask
What does the AI search readiness score measure?
It measures whether search and AI answer tools can reach, read and cite the page. The checklist covers crawler access, a readable robots.txt, llms.txt, the first HTML response, indexing directives, titles, descriptions, headings, structured data and entity markup. Each check shows its weight and why it matters.
Do AI crawlers run JavaScript?
AI crawlers often do not run JavaScript, so treat the first HTML response as the content they see. If the main copy only appears after scripts run, it can be missing from their view. Server rendering or static HTML keeps the key text in that first response.
Does llms.txt improve AI search results?
llms.txt is an emerging convention from llmstxt.org that gives tools a short Markdown map of the site. Major AI search providers have not confirmed that they rely on it. The check gives it a small weight, and a missing file is a warning, not a failure.
Why does a training block still pass the check?
The score separates search and answer crawlers from training crawlers. Blocking a training crawler such as GPTBot or Google-Extended is a policy choice, so it does not lower the score. Blocking a search or answer crawler does, because those crawlers decide whether the page can appear in AI answers.
How does the checker decide whether a bot can read a path?
It uses the RFC 9309 longest-match rule, the same as the robots.txt validator on our generator page. A missing robots.txt means no rules. A server error means crawlers may treat the whole site as blocked until the file is reachable again.
Does a high score guarantee AI citations?
No. This is a technical readiness check. Citations depend on content quality, authority and each platform's own choices. Use the score to remove technical blockers first, then work on the content itself.
Related tools
- Technical SEO & AI Search robots.txt Generator Build a robots.txt with AI crawler controls, then test any file with the RFC 9309 longest-match rule.
- Technical SEO & AI Search llms.txt Generator Write an llms.txt file in the llmstxt.org format, with a live preview and validation.
- Technical SEO & AI Search Schema Validator Validate JSON-LD against required and recommended properties for Organization, Article, Product, FAQPage, LocalBusiness and more.
Next step
Want this done properly across your whole stack?
Tracking, search, automation and reporting, engineered and operated by subimpact network. Start with a free audit.
Get Free Audit