inspector-assessment
An experimental fork of the MCP Inspector that tried to answer one question: can you find real vulnerabilities in an MCP server by automatically firing injection payloads at every tool it declares? It is archived, and the answer turned out to be mostly no. That answer is the reason it is still worth reading.
Archived, and deliberately so. This was the research phase of a line of work, not a product. It is public because a negative result you can clone and re-run is more useful than one you have to take on trust — and because the approach that replaced it is only interesting once you know what it replaced.
What it does
The upstream Inspector gives you a console for poking at an MCP server by hand. This fork adds a pass that runs on its own: it connects to a live server, enumerates every tool the server declares, invokes each one with generated inputs including command injection, SQL injection, path traversal, SSRF and code-execution payloads, and then reads the responses looking for evidence that something actually executed rather than that a string was reflected back safely.
Around that sit a set of assessor modules, each responsible for one judgement — whether the manifest validates, whether the declared tool annotations match observed behaviour, whether documentation covers what the server exposes, whether dependencies carry known vulnerabilities, whether the tool surface changes between runs. There is a CLI for running it in CI alongside the web UI.
It is a monorepo — client (React), server (Express), cli — and it is published to npm, so it can be run against your own server without cloning anything.
What it found, which was not what it set out to find
The hypothesis was that automated injection testing would catch real MCP server vulnerabilities. Testing it properly meant building a deliberately vulnerable server to test against. The result was four findings, and they are recorded in the repository's own README rather than only here:
- The attack surface is the model, not the input handler. MCP servers get exploited through what the model reads — instructions hidden in a tool description, context poisoned by a tool's output, metadata written to mislead. Sending whoami to a tool tests none of that.
- Injection testing finds what it was built to find. The vulnerable testbed was vulnerable in exactly the ways the payloads probed for — eval(), shell=True. Real servers rarely contain those patterns, and the ones that do are visible to static analysis without invoking anything.
- Reading beats probing, in this domain. A model given the tool descriptions, the source and the manifest, and asked to judge them as a whole, catches a wider class of problem — acceptable-use violations, descriptions that misrepresent what a tool does, undisclosed telemetry, social engineering — none of which an automated invocation can reach by construction.
- The behavioural evidence was confirmatory, not additive. Against a 59-tool testbed, every tool the injection pass flagged was already identifiable from its source. The runtime proof was worth having, but it did not find anything new.
A tool that only confirms what a cheaper method already told you is a tool you should stop maintaining, which is why this one is archived rather than iterated on.
Where it led
The fourth finding is the one that changed things. If a model reading a server the way a model will actually encounter it outperforms a harness firing payloads at it, then review should be built around the reading and use behavioural testing to corroborate — not the other way round. That is the shape of the production work described on pipelines.
There is a gap between the two, and it is worth being plain about it rather than letting the arc imply something you can go and look at: the prototype that sat between this experiment and the production pipeline is not public, and there is no repository to link. This page and the pipeline description are the two ends of a line whose middle is private.
Look at it yourself
Everything above is a claim about a public repository, so none of it needs to be taken on trust.
There are no test counts or pass rates on this page. The suite is large and, on an archived project, does not run clean — that is what archived means, and quoting a figure from the last day it did would be quoting something that has stopped being true. The counts this site does publish are on the record, where CI re-derives every one of them and a claim that stops holding fails the build.
The repository is at github.com/bryankthompson/inspector-assessment, derived from modelcontextprotocol/inspector, which remains the right tool for actually debugging an MCP server. The rest of the public code is indexed on repositories.