What it does
The server checks four parts of a website that affect how agents can access it: robots.txt, sitemap.xml, llms.txt and the homepage. The check is bounded, so it covers only these items rather than a full crawl.
It suits anyone who needs to see how a site presents these files to agents before agents access it. The publisher's documentation provides no further details on setup or running requirements.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- Official MCP Registry first seen 13 Nov 2026 · last seen 13 Nov 2026 · its page on the source
Agent Crawl Readiness
Bounded crawl-readiness check for agent access: robots.txt, sitemap.xml, llms.txt, homepage.
v1.0.0As shown on registry.modelcontextprotocol.io when we last read it