What the scanner read, and what it did not

A source scanner that reports what it found, without reporting what it looked at, is asking to be believed. This page publishes the other half for five widely used open-source repositories: how many files were there, how many were read, and why each unread file went unread.

Every number here is a claim about reading: how many of the files present the instrument opened. It is not a claim about what was found in them — for that a control is needed, and none was held.

None of the five reaches full coverage, and that is a statement about this tool’s declared scope before it is anything about these repositories. A scanner claiming 32 file extensions cannot reach 100 % against a tree holding 79 kinds; what the figure measures is the overlap between what the tool claims and what the corpus is made of. The highest is OpenSSH at 76.17 %, and the number falls from there — but a lower number here means more file types this tool does not claim, not more cryptography left undiscovered. Vault at 55.72 % is 4110 unread files of which 2708 are .txt.

One run, 2026-09-11, committed to this repository and rendered from it. The page makes no request while rendering and reads no clock; the percentages below are computed from the file counts rather than stored, so the table and the downloads cannot disagree.

The measurement

The corpus, pinned

A coverage figure without a pinned corpus cannot be recomputed by anyone, including the person who produced it, because repositories move. Each row below is the exact commit the figure was measured against, in full: an abbreviated hash is not fetchable and collides as history grows.

When each repository was read, and for how long, is in the two files rather than in this table: it is a property of the machine that ran the scan and not of the estate it read.

A pinned commit makes a figure reproducible: a reader with the pin re-runs it and gets the number again. It does not make it interpretable — the same scanner over a tree of Go and over a tree of Ruby reports different coverage with nothing in the scanner changing. That is why the two files carry what the corpus is made of and not only how big it is.

The instrument, pinned

The tool is qrp-mcp 0.6.0, carrying 43 algorithm patterns, 2 infrastructure patterns and 6 signing-command patterns, and claiming 32 file extensions.

It also ran from 7ec14eb14f554b20907dda71f6f67d8b41dfb7d5 with a clean working tree. That commit is here because the version string turned out not to identify the emitter: two runs during the preparation of this page both reported 0.6.0 from different code. A version is a claim about the instrument; the commit is the instrument.

The instrument now carries three rules it did not have: three families its own table declared and no rule could reach. A declared boundary that the rules cannot honour is the same defect as a rule that invents a finding.

Before anything was counted, these directories were removed from every repository: .git, .mypy_cache, .pytest_cache, .ruff_cache, .tox, .venv, __pycache__, build, dist, node_modules, target, vendor, venv. That list filters the denominator, so it is stated rather than left implicit.

Why a file went unread

There are two reasons and they are kept apart. Collapsing them would rebuild the word “unknown”, which stands both for “it is not there” and for “nobody looked” — and a reader cannot tell those apart after the fact.

• type_not_claimed — the tool does not claim this file type. A declared boundary, not a failure. This is where nearly every unread file in this run falls.
• unreadable — opened and could not be read. Attempted and failed. Across all five repositories this run recorded 0. It is printed as a zero rather than omitted: a column that disappears when it is empty cannot be checked.

Both files now also say where the unread mass sits, because a total does not say it. Three file kinds account for between 51.71 % and 89.37 % of everything not read, in every one of the five:

Vault is the case. .txt alone is 2708 of the 4110 files not read. So Vault at 55.72 % coverage largely means Vault holds 2708 text files this tool does not claim, and there is no reading of 55.72 % that tells you that.

The denominator is concentrated too, and more evenly than the gap. The two largest file kinds present are between 42.61 % and 54.54 % of everything counted, in every one of the five:

Both figures are partitioned by file extension, which is one partition among several: the same trees split by directory or by language give different shares with nothing changing on disk. The block names the partition beside every share — files by extension on this side and unread files by extension on the other — and the CSV carries it for the unread side, which is the side it flattens.

The kind count is in the table for a reason: it is the baseline the share is read against. Three of 24, by extension, at 68.93 % is five and a half times a flat split; three of 61, by extension, at 67.95 % is fourteen times one. The same share means different things, and without the count it cannot be read as high or low at all.

What this page does not claim

Control: none held. No corpus with independently established contents exists for this instrument; nothing here claims the second axis — whether what was present in a file was found in it.

Both downloads carry this as a field rather than as prose. The block records claims.axis as reached and claims.control.held as false, with none_held as the reason; the CSV repeats all three in claims_axis, control_held and control_absent_reason. The sentence above is the file’s, so it is checked rather than taken.

TNO 2025 P11921, Cryptographic Asset Discovery and Inventory, by Sijpesteijn, van Leuken and Kerling, 23 March 2025, TLP:CLEAR, 64 pages, funded by CIO Rijk, the NCSC and the Netherlands Ministry of Economic Affairs, states in section 4.2.7 that the precision of such a tool cannot be verified in practice, because calculating it requires knowledge of the total cryptography present. In that study’s comparison table, the accuracy cells for two commercial providers read, verbatim, no clear data, as there is no ground truth. Chapter 8 recommends that establishing one be a national action. Eighteen months later it does not exist.

The report is public: publications.tno.nl, TNO-2025-P11921-GB.

It does not rank the five repositories. Vault having lower coverage than OpenSSH says something about the scanner, not about the repository — a codebase with more file types the tool does not claim will score lower while being no different in itself.

Five open-source repositories are not a corporate estate. Nothing here generalises to one.

All five are shallow clones. The same tool at the same commit counts a different denominator inside a working directory on a developer’s machine, because that directory has node_modules and build output in it. Measured on these five, the exclusion list hides between zero and one file — so the number does not move, but what follows from it does. The dirty flag will not warn you either: build output is usually in .gitignore, and an ignored file leaves the tree reporting itself clean. Tested: 42 files on disk, 2 in the commit, git status --porcelain empty.

Whether a commercial CBOM tool solves any of this has not been tested. This page does not claim it does not.

A defect this measurement found in the scanner

While these numbers were being produced, the scanner was returning Classic McEliece as an elliptic curve. The token EC matched inside the name, and the match was accepted. The file had been read, parsed, understood and counted — and reported confidently wrong.

It is fixed. It is on this page because it is half the argument: a file inside the examined count is not thereby a file the tool got right, and coverage is the measure that makes the question askable at all. An instrument that publishes neither its coverage nor its own commit gives a reader no way to find this class of defect from outside.

A second one, found the same way and pointing the other direction. The rules matched the dotted form of a Python import and not the spaced one — asymmetric.rsa yes, from cryptography.hazmat.primitives.asymmetric import rsa no, which is how the language is actually written. Three families the tool’s own published table declared — ECDH, Ed448 and EdDSA — had no rule that could reach them at all, so it would answer that it knew Ed448 while no scan could ever report one. And ECDHE, the spelling that appears in a cipher list, was excluded by a word boundary.

Measured with the tool rather than with a text search: qrp-mcp at 7ec14eb14f554b20907dda71f6f67d8b41dfb7d5, scanning the OpenSSL, OpenSSH and Certbot checkouts at the commits in the table above, counting findings that carry a file and a line — 1 380 ECDH, 643 Ed448 and 280 EdDSA. Two thousand three hundred and three findings the scanner reports now and reported none of before — 19 586 located findings against 17 283 before the rules were fixed, an increase of 13.3 %.

All five of those figures are a different measurement from the coverage block and are not in the two files below: the block reports what was read, not what was found. They are reproducible from the tool at the commit named above against the same public checkouts, and they are stated here rather than left for a reader to assume the artefacts carry them.

Every file carrying them was reached, was read, and was counted in the numerator, and the coverage figures did not move for it — that correction changed what the rules detect, not which files were opened.

A third, and the first to move a number. Thirteen OpenSSL files — twelve .H and one Makefile.PL — were counted under type_not_claimed, which states that the tool does not claim the file type, while .h and .pl are both in the claimed list. The membership test was case-sensitive against a lower-case set, so an upper-case extension never matched and the file was skipped; the report then lower-cased the suffix before counting it, so the files surfaced as types the tool does claim. Three defects in one line, and the third is the worst: the lower-casing in the report concealed the case-sensitivity in the match, so the count stated the opposite of what had happened.

It was found by running one comparison nobody had run: the intersection of claimed_types with the unread kinds. Empty in four repositories, {.h: 12, .pl: 1} in OpenSSL. That intersection is now an invariant in the test suite rather than a test for this case, because the cause was incidental and the property is not.

This page has said five times that the coverage figures did not move. That was true of what it described and had been allowed to harden into a property of the number, which it is not. A detection rule does not move coverage. A reading rule does. The corrections before this one were detection rules and the figures held; this one is a reading rule, and OpenSSL moved from 61.38 % to 61.60 % — 3763 files read where 3 750 were read before. Exactly the thirteen, and no new findings among them.

A rule that invents a finding and a rule that hides one are the same defect, and coverage reports neither. It is a claim about reading. It is not a claim about finding.

The data

Both files are one run and are checked against each other before this page renders.

• scan_coverage_2026_09_11.csv — the five rows.
• pokritie-blok-2026-09-11.json — the full coverage block as the scanner emits it, including the per-extension breakdown of every unread file.