What it does
The server crawls websites and converts their pages to Markdown. Crawling can be guided by sitemaps, glob patterns and depth limits, and the publisher states that it obeys robots rules and crawl-delay settings.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- Official MCP Registry first seen 13 Nov 2026 · last seen 13 Nov 2026 · its page on the source
Website Crawler — Markdown Pages, Sitemaps & RAG Provenance
Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay
v1.0.4As shown on registry.modelcontextprotocol.io when we last read it