Evidence
What we tested, what we missed, and what this doesn't prove
A scanner that reports nothing has a perfect false-positive rate, and a scanner that reports everything has a perfect detection rate. So SecScan is measured two ways: against an app whose faults are known exactly, and against real sites where the right answer is few findings and none of them wrong. The results below include the ones that count against us.
A vulnerable app with 74 planted faults, three scanners
SecScan, OWASP ZAP and Nuclei were run against the same deliberately vulnerable app. Three separate AI judges decided, without knowing which tool was which, whether each tool's findings showed it had found each fault. They agreed on 213 of 222 decisions (95.9%). The range for ZAP and Nuclei is the gap between what all three judges agreed on and what a majority did.
| Faults found | SecScan | OWASP ZAP | Nuclei |
|---|---|---|---|
| All 74 planted faults | 68 (91.9%) | 18–19 (24–26%) | 7–8 (10–11%) |
| Headers and policy (13) | 13 | 10 | 2 |
| Software versions (4) | 4 | 2 | 0 |
| Exposed files and endpoints (26) | 26 | 3 | 5 |
| Secrets in code (13) | 13 | 0 | 1 |
| Injection (10) | 9 | 4 | 0 |
| Access control and API (8) | 3 | 0 | 0 |
The six SecScan missed
- An SSRF case (needs an out-of-band collector)
- An API endpoint with no authentication
- Function-level access control on an API
- Cross-account access to another user’s data
- A page-level insecure direct object reference
- Guessable, enumerable identifiers
These are the tests that need two logins, an API description or an out-of-band collector. That is where the gap is, and it is why the coverage page lists broken access control as needing a second account.
Read this before quoting the table. The vulnerable app was built to contain the faults SecScan checks for, so it is an inherently favourable test. A lower score for ZAP or Nuclei shows they overlap less with SecScan's checklist, not that they are worse tools. ZAP was run in full-scan mode but without its forced-browse wordlist, its API import or logins, so its results on hidden paths and access control mean “not set up for it”. The judges are AI models and share whatever blind spots such models have. The test app and the raw outputs are not published yet, so this is our result, not one anyone else has reproduced.
The benchmark also caught its own bias. ZAP never requested three pages because nothing on the home page linked to them, while SecScan found them from a built-in path list, which made the first scores look better than they were. Those pages are now linked like real features. Earlier, three rounds of adjusting match patterns moved ZAP's score from 59.5% to 47.3% to 31.1% on identical data, which is why scoring moved to blind judges.
Thirty well-run sites, and every high finding checked by hand
On 24 September 2026 we scanned 30 well-known sites, passively: page requests and DNS lookups, nothing an ordinary visit would not send. Eight answered with a bot-protection page instead of their own, so they are reported as unseen and not graded. Of the 22 that were read, 3 graded A, 9 B, 9 C and 1 D. There were 282 findings in total, 98 above Info, no critical, and 16 high.
All 16 high findings were checked against the live sites the same day and all 16 were real. The 82 medium and low findings in that run were not individually re-checked; earlier rounds verified the same finding types on the same kinds of sites. Two of the highs deserve a caveat: the archive.org source map is a copy of a public library, so what it exposes is that library and not archive.org's own code, and “no Content-Security-Policy” as a high on a site like Wikipedia is a judgment about severity that reasonable people rate lower.
And the sweep found a false positive
amazon.co.uk: five findings, one high, about a page Amazon never
served. Amazon answered with a two-kilobyte bot-check page and a
“continue shopping” robot check, and our detector recognised
neither, so it read the interstitial as Amazon's home page and reported
its missing headers. It is the same mistake this project has recorded
before: treating a response the target never sent as evidence about the
target. It was found by checking every high by hand, and fixed the same
day, with tests that also make sure a real page mentioning those words is
left alone. A second small bug came out of the same result: an Akamai
server called AkamaiGHost was being read as the Ghost blogging
platform.
The longer history — false positives that shipped, ones caught before release, and findings that look wrong but are right — is on the false positives page. Before today's sweep, 111 runs across 39 hosts were on record, and the latest full round on 19 September found 80 actionable findings on 20 sites, eight of them high, every one verified true, and none false.
What runs before every release
- 1,267 automated tests pass, covering both directions: a finding appears when the condition is present and is suppressed when it is not.
- A 19-flaw quick check runs against a local vulnerable site and fails the build if any planted flaw goes undetected. It passes.
- End-to-end scripts drive the MCP server the way an editor does (28 checks on the tools alone), and confirm that an API token cannot reach any account route.
- Four guarded sites are scanned before scanner changes and their finding counts must not move unexplained.
- Every scan records each check as ran, skipped (with the reason) or failed, so a clean report cannot quietly mean “we did not look”.
What none of this proves
- No outside party has reproduced any of these results.
- The benchmark measures known, planted faults. It says nothing about unknown flaws in real applications.
- SecScan is an automated scanner. It cannot reason about business logic, and it is not a penetration test.
- Still open: a site's own CDN subdomain is treated like a stranger's script by the subresource-integrity check; one hard-coded token can produce three separate high findings; and server-side request forgery has not been swept for false positives on real sites.
SecScan is built by one person, and none of this replaces a review by a security professional. It is meant to catch the common, reliably detectable problems early and say plainly what it did not test.
Check your own app
Every account gets 3 free scans every month. Each report lists what ran and what was skipped.
Run a scan →