I started 38 MCP servers. 17 of them never say what their tools do.

Efe Çakıcı Efe Çakıcı #mcp#security#measurement

584 tools across 38 servers, 392 with annotations, 17 servers with none at all. The ones that stay silent are the browser controllers and the database drivers.

If an agent is going to run a tool on your machine without asking you every time, it has to know what that tool does. MCP gives every tool a place to declare it: annotations. readOnlyHint, destructiveHint, idempotentHint, openWorldHint.

I wanted to see whether anyone fills them in.

So I started 38 MCP servers on my laptop, asked each one for its tool list, and wrote down what came back. 584 tools. 392 of them carry annotations. 17 of the 38 servers declare nothing at all, on any tool. The scan ran on 2026-09-18, and I re-checked three of the servers today and got the same result.

What these four annotations mean

Annotation Meaning
readOnlyHint the tool does not modify anything
destructiveHint the tool may destroy data
idempotentHint calling it twice is the same as calling it once
openWorldHint it talks to systems outside the local environment

A client reads these to decide what to auto-approve. A tool with no annotations leaves the client two options: ask the user about every call, or guess.

What I found

Servers scanned 38
Tools seen 584
Tools with annotations 392 (67%)
Servers declaring nothing at all 17
Servers declaring annotations on every tool 20
Servers declaring annotations on some tools 1
Tools with readOnlyHint: true 199
Tools with destructiveHint: true 68
Tools with openWorldHint: true 109

The 17 that declare nothing

Bitwarden MCP Server 59 tools sqlite 6 tools
playwright-mcp 33 tools sqlite-manager 5 tools
github-mcp-server 26 tools tavily-mcp 5 tools
argocd-mcp 16 tools elasticsearch-mcp 4 tools
Browser MCP 12 tools redis 4 tools
puppeteer 7 tools ddg-search 3 tools
Browserbase MCP Server 6 tools airbnb 2 tools
mcp-server-commands 1 tool mcp-fetch 1 tool
example-servers/postgres 1 tool

Two groups sit inside that list. Four of the seventeen drive a browser: playwright-mcp, Browser MCP, puppeteer, Browserbase. Five talk to a database: sqlite, sqlite-manager, elasticsearch, redis, postgres. Those are the tools that touch the most sensitive surfaces in a typical setup, and they are the ones telling the client the least about themselves. I am reporting the pattern, not a cause.

What this is not

This is not a vulnerability list. A server without annotations is not insecure. Annotations are self-reported, nothing verifies them, and a wrong annotation is worse than a missing one. What the numbers measure is narrower: how much information a client has when it decides whether to run a tool on its own.

Limits

  • One snapshot, 2026-09-18, on one machine, over stdio. Remote and HTTP servers are not in the sample.
  • tools/list only. This measures what servers declare, not what their tools do when called.
  • 17 of the servers I tried failed to scan, mostly for missing credentials (slack, stripe, gitlab, sentry, supabase, cloudflare, heroku, browserstack, asana, apify, mysql, postgrest and others). The sample is 38 of the servers I attempted, not a random sample of the ecosystem. The two seventeens are a coincidence.
  • Versions are omitted on purpose. Most servers do not report one, and I will not publish a version I could not read from the server itself.

Findings I killed before publishing

The number above is small and boring on purpose. What follows is the part I think is worth more.

I wrote five reports in the last month and did not publish any of them. Three were claims about a cloud provider, a password manager and a browser tool. In each case the behaviour turned out to be the product’s documented purpose, not a control failure. One was a prompt-injection experiment with a payload so obvious it proved nothing at n=2. One was a measurement where every task was a read, so “zero deviations in 16 calls” was a tautology: there was nothing to deviate from.

Today I added a sixth. A verification script of mine promoted a finding to “verified” while the reproduction had failed, because the browser never launched. The test printed two no lines and the pipeline recorded a confirmation anyway. The gate now refuses to promote unless the assertion itself passed, and I re-ran the test before trusting it.

The rule I work by: a measurement is publishable, a finding is not, until it reproduces on demand. A false positive costs more than a missed bug, because the correction is public.

Reproduce it

The scanner is mcp-scope: it starts a server over stdio, calls tools/list, and records the annotations verbatim. The tool and the dataset are public: https://github.com/aliefe04/mcp-scope. The dataset is 39 JSON files, one per server plus a run summary, and it is the same data the table above is computed from.

Next measurement: whether the declarations match behaviour. A tool that says readOnlyHint: true and writes anyway is a different kind of problem than a tool that says nothing.