What it does
Each test is a single English sentence in a markdown file. An agent opens a real browser, reads the page through its accessibility tree and returns one of five verdicts: passed, failed, stale, errored or flaky. A passing run is recorded and later runs replay that recording with no model call. The agent runs again only when the recording stops fitting the app. Only a failed verdict is a claim about the app. Stale means the recording no longer fits, and errored means the runner could not finish.
The CLI can run a folder of tests on each pull request and leave one comment, updated on every push, that names what broke, the file most likely responsible and the evidence for that guess. Exit codes are 0 for clean, 1 when a test fails and 2 when the runner cannot finish. In an editor, the MCP server exposes three tools: run_tests, can_i_ship and list_tests. A nightly GitHub Action runs the project's own suite against two live sites and publishes every verdict.
The CLI runs on your machine or CI runner using your own ANTHROPIC_API_KEY. A first run of a test costs roughly one agent conversation, and a replay costs nothing. The hosted service is $19 per month with 100 tested pull requests included, after a 14-day trial that needs no card.
Limitations and requirements
CLI runs need your own ANTHROPIC_API_KEY, and a first run of each test incurs model cost. The hosted service is $19 per month with 100 tested pull requests included, after a 14-day trial. This is commercial software, not open source.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- Official MCP Registry first seen 13 Nov 2026 · last seen 13 Nov 2026 · its page on the source
io.github.Arjun0606/smolanalytics
End-to-end tests from one sentence: an agent drives a real browser, and passing runs replay free.
v0.92.1As shown on registry.modelcontextprotocol.io when we last read it