MCP servers · unitedideas

ActableSite AI Crawler Monitor

MCP server that checks AI crawler robots.txt policy and monitors changes to a public site's policy, sitemap and llms.txt file.

This text was machine-translated from the publisher's own information and has not been checked by a human editor. The publisher's original text is the reference.

This resource was imported automatically from the sources listed below and has not been reviewed by the AI Gate team yet. The description is the publisher's own, as published. Check the repository before installing.

What it does

The server checks the robots.txt policy that applies to AI crawlers and monitors public-site policy, sitemap and llms.txt files for changes. It suits teams that want to track how a site's AI crawler rules and related files change over time.

Safety

Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.

  • Runs code or changes an assistant's behaviour

Sources

  • Official MCP Registry first seen 13 Nov 2026 · last seen 13 Nov 2026 · its page on the source
    ActableSite AI Crawler Monitor

    Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

    v1.4.2
    As shown on registry.modelcontextprotocol.io when we last read it
MCP servers
hashicorp

Terraform MCP Server (HashiCorp)

HashiCorp's MCP server that gives an assistant current Terraform Registry information on providers, modules and policies, and can manage HCP Terraform workspaces.

1.6K GitHub starsCloud & DevOpsSoftware development
Updated 17 Nov 2026
MCP servers
modelcontextprotocol

MCP reference servers

Seven small example servers maintained by the Model Context Protocol project: files, Git, web fetching, memory, time, step-by-step thinking and a test server.

92K GitHub starsSoftware development
Updated 18 Nov 2026
MCP servers
redis

Redis MCP Server

Redis's official MCP server that lets an AI assistant store, read and search data in a Redis database, including streams, JSON documents and vector indexes.

642 GitHub starsData & databasesSoftware development
Updated 17 Nov 2026
MCP servers
firecrawl

Firecrawl MCP Server (web search and scraping)

Firecrawl's MCP server that lets an AI assistant search the web, turn pages and PDFs into clean Markdown, crawl sites and interact with dynamic pages.

7.8K GitHub starsWeb, search & browserProductivity & automation
Updated 17 Nov 2026
ActableSite AI Crawler Monitor | AI Gate Discover