A missed vulnerability costs you one finding. A false positive costs you the report — once a scanner has sent you chasing something that was never wrong, you have no way to tell which of the other findings to believe. So the checks get pointed at sites whose answers are already known, and every finding that survives is read by hand.
Google, GitHub, Cloudflare and Mozilla run their security properly. On targets like those a correct scan returns few findings and zero false ones, which makes every extra finding a lead. Each one is checked against the live response by hand before anything is changed — including the ones that turn out to be right.
These were live. Each one is fixed, with a regression test, and the reasoning is recorded so the fix is not quietly undone later.
A flattened Set-Cookie header was re-split on commas. The comma inside expires=Tue, 09-Mar-2027 matched, shearing the cookie so every attribute after the date became invisible — including the HttpOnly the cookie actually carried.
Any cookie whose expires= precedes its flags. That is most of them.
The check matched any * anywhere in the directive, so ordinary allowlist entries like https://*.onetrust.com tripped it. Nearly every policy that loads analytics has one.
Reported High. The better the policy, the more likely it fired.
A header-absence check. GitHub enforces rate limits at the edge and exposes no rate-limit headers on HTML responses.
Every site that rate-limits without advertising it.
The finding was right — the file is not RFC 9116. The evidence was hardcoded to say "not found" regardless, so anyone who checked saw a file returning HTTP 200 and concluded the scanner was broken.
A correct finding made to look like a bug.
Three never reached anyone, because a new detector gets pointed at known-good targets before it is committed. The third is the one worth reading: it is the detector written to fix the second, making the same class of mistake in a new place.
The probe gated on whether a port was reachable. Cloudflare's MX completes the TCP handshake on 587 and then resets before answering EHLO, so the probe learned nothing — and silence was being scored as a failure.
It fired on some runs and not others, depending on when the reset landed. A finding the reporter cannot reproduce is worse than a consistent one: re-running the scan appears to fix it.
All of it true — of the Cloudflare "Just a moment…" interstitial those sites return to a scanner. None of it true of the sites themselves.
It is not one bad check. When an edge answers instead of the origin, every content-derived check describes the interstitial — CSP, cookies, headers, fingerprint — and the whole report reads as a scan of the customer.
The detector written to fix the problem above made the same mistake: it matched x-datadome by presence, and DataDome sets that header on allowed traffic. The real 1.3 MB homepage was sitting in the response body.
Header signals now match on value, and the pattern is mandatory on every one of them, so the shortcut cannot be taken again.
Half of a false-positive audit is deciding what to leave alone. These were each investigated, confirmed correct, and written down as correct — because the obvious way to verify them produces the wrong answer, and without the note someone eventually "fixes" a working check.
The next run was meant to produce a published baseline, not another audit. It produced both. Every entry was verified against the live site before anything was changed, and the first is the one that would have cost us the most.
GOV.UK publishes v=spf1 -all at gov.uk and p=reject at _dmarc.gov.uk — close to the strongest email authentication a UK domain can have. We looked for both on the wrong name. The registrable domain was derived by counting label lengths: "uk" is two characters and "gov" is three, so it took the .co.uk branch and returned the hostname unchanged, querying www.gov.uk for SPF and _dmarc.www.gov.uk for DMARC. Neither exists. www.bbc.co.uk escaped only by having a fourth label.
One finding, three separate bugs. The severity had also been escalated from Medium to High because the MX lookup counted two CNAME records pointing at Fastly as mail servers — a DNS answer carries the whole resolution chain, not just the type you asked for. And the evidence line read "Status: NOERROR (domain exists)" about a name that does not exist, because that string was hardcoded.
DNSKEY records live at the zone apex, and we asked the scanned hostname. www.nasa.gov answers with a single CNAME and no DNSKEY, while nasa.gov is properly signed.
This one we introduced ourselves, with the fix above. Until answers were matched against the type requested, that CNAME had been counted as a DNSKEY and the check passed for entirely the wrong reason — a hidden false negative that the fix converted into a visible false positive. Across the corpus, DNSSEC findings went from 13 of 30 to 26 of 29 in a single run, which is what made it obvious. The higher figure is the correct one: the old number was under-reporting.
stripe.com sends no rate-limit headers at all and unquestionably rate-limits. Rate limiting lives on login and API routes, not the homepage, and most CDNs throttle silently. The check could see whether rate limiting was advertised, and nothing more.
The infrastructure allowlist behind it had already been patched twice — once for Google server tokens, once for GitHub's edge — which is the shape of a check chasing an unobservable property one vendor at a time. It now says only what it saw, at Info, with no weight in the grade.
A survey widget's apiKey inside Google Tag Manager, with a payload carrying a company id and an issued-at and nothing else. Vendor SDKs ship exactly this kind of token publicly, to identify which account a widget belongs to.
Tokens now split on whether the payload carries a principal claim — sub, user_id, email, role, scope. With one, it is a credential leak. With none, it is an account identifier, reported at Info so the signal survives without the accusation.
JSEncrypt, an open-source RSA library loaded on countless bank login forms, contains one BEGIN RSA PRIVATE KEY literal used to format its output. There is no key material anywhere near it. The pattern matched the bare header, so every site loading the library earned a maximum-severity Critical.
Verified against the real 57 KB minified library: one header occurrence, zero key blocks. The pattern now requires a base64 key body to follow the header, and still matches a key genuinely embedded in source.
The cookie is __Host-js_csrf — Secure, SameSite=None and deliberately not HttpOnly, while every other cookie Dropbox sets carries it. A double-submit CSRF token works precisely because the page can read it and echo it back in a request header.
Marking it HttpOnly hardens nothing; it breaks the defence, because the page can no longer read the value either. We were telling Dropbox to disable their CSRF protection. The exemption applies to HttpOnly alone — a CSRF cookie still needs Secure and SameSite.
When a bot-protection interstitial answers instead of the origin, every check that reads the response describes that page. Etsy's report said the site was missing CSP and served a wildcard CORS policy. Both were true only of Cloudflare's challenge page.
Twelve of eighteen findings described a page Etsy never served. Naming them as untrusted in a coverage note was not enough. Findings are now withheld unless their source never read the intercepted response — DNS, email authentication, mail transport. Etsy goes from 18 findings to 4, all four genuinely about Etsy.
Four of these, and two from the earlier round, are the same error in different places: treating the presence of a response as its meaning. A reachable port read as a STARTTLS answer. A vendor header's presence read as "blocked", when it is set on allowed traffic. A CNAME counted as a mail server. NXDOMAIN read as "unknown", when it is the ordinary way of saying "no record". A header saying a key would go here, read as a key. Twice we committed that error while fixing it.
Fewer findings is the goal here. Every one that came off was something the site had not done wrong.
| Target | Before | After |
|---|---|---|
| google.com | 15 | 13 |
| github.com | 12 | 11 |
| cloudflare.com | 9 | 8 |
| mozilla.org | 8 | 7 |
The newest checks were then verified across 15 sites — including Wikipedia, the BBC, Amazon, Vercel, Shopify and GOV.UK — with zero findings above Info, both bot-protection interstitials correctly identified, and every real page left alone.
The second round is measured the same way, across the 29 sites common to both runs: actionable findings fell from 168 to 117. GOV.UK went from 9 findings and 3 actionable to 7 and 0, etsy.com from 17 and 10 to 4 and 1, dropbox.com from 13 and 5 to 12 and 4. Not all 51 were deletions: some were findings withheld because they described a bot-protection page rather than the customer, and one was a check reclassified to Info because it cannot observe what it was asserting. That one still appears in every report — it simply no longer counts against the grade.