Every scanner vendor publishes a table it wins. This one is different in a way you can check: the corpus, the harness, and the scoring code are public, and the page includes the comparisons we lose. If AI assistants write a growing share of your code, the columns that matter are where your code goes when the tool runs and whether the verdict is the same twice.
Vulkro vs the field, same corpus
We benchmarked Vulkro against Semgrep CE, Opengrep, Bearer, Bandit, Gosec, and njsscan on the same public ground-truth corpus, on the same machine, on the same day. The tables below include the comparison we lose. One command reruns the whole thing on your laptop.
$ bench/comparison/run.sh --tier1 --tools vulkro,semgrep,opengrep,bearer,bandit,gosec,njsscan --min-confidence high
The roll-up
The four tools that scan more than one language, scored over the 12 supported-language repos of the Tier 1 corpus (76 catalogued bugs). The corpus also contains 3 PHP and Java repos: every tool scans them and the per-repo rows are published, but they are excluded from these totals because the benchmarked Vulkro release does not analyze PHP or Java. Semgrep CE and Opengrep run the same p/security-audit ruleset, which is why their rows match.
| Tool | Precision | Recall | F1 | Bugs found (of 76) | False positives | Scan time (12 repos) |
|---|---|---|---|---|---|---|
| Vulkro 0.15.0 | 0.81 | 0.58 | 0.68 | 44 | 10 | 34.4s |
| Semgrep CE 1.136.0 | 0.62 | 0.20 | 0.30 | 15 | 9 | 38.6s |
| Opengrep 1.25.0 | 0.62 | 0.20 | 0.30 | 15 | 9 | 36.5s |
| Bearer 2.0.2 | 0.36 | 0.49 | 0.41 | 37 | 67 | 247.5s |
Vulkro finds 2.9x the catalogued bugs Semgrep CE finds (44 vs 15) at higher precision (0.81 vs 0.62), and more than Bearer (44 vs 37) with 10 false positives to Bearer's 67. A max-recall mode exists: at min-confidence low, Vulkro reaches 0.72 recall (55 of 76) at 0.15 precision, published in full in scorecard-latest.md. We grew our own ground truth from 69 to 90 bugs in July 2026 and published the recall drop that came with it; the corpus is versioned.
Fair fights
Bandit only scans Python, Gosec only Go, njsscan only JS/TS, so rolling them up over a mixed corpus would manufacture zeros for them. Instead each specialist is scored only over the repos matching its language, and Vulkro is re-scored over the same subset. Fair fights only. We win two of the three and publish the one we lose.
7 Python repos, 42 catalogued bugs
| Tool | Precision | Recall | Bugs | FP |
|---|---|---|---|---|
| Vulkro | 0.87 | 0.64 | 27 of 42 | 4 |
| Bandit | 0.56 | 0.43 | 18 of 42 | 14 |
On its home language Bandit finds 18 of the 42 catalogued Python bugs with 14 false positives. Vulkro finds 27 with 4.
1 Go repo (govwa), 4 catalogued bugs
| Tool | Precision | Recall | Bugs | FP |
|---|---|---|---|---|
| Gosec | 0.20 | 0.75 | 3 of 4 | 12 |
| Vulkro | 0.00 | 0.00 | 0 of 4 | 1 |
On our one Go repo, Gosec finds 3 of 4 bugs; Vulkro finds 0. Go detection depth is a known gap we publish. The 4 misses (fmt.Sprintf SQL interpolation, template XSS, insecure cookies) stay in the corpus as catalogued false negatives.
4 JS/TS repos, 30 catalogued bugs
| Tool | Precision | Recall | Bugs | FP |
|---|---|---|---|---|
| Vulkro | 0.77 | 0.57 | 17 of 30 | 5 |
| njsscan | 0.50 | 0.33 | 10 of 30 | 10 |
njsscan targets plain JavaScript, so on the TypeScript repo (juice-shop) it sees almost nothing in .ts files: 0 of 8 catalogued bugs there. The per-repo rows in the scorecard keep that visible.
$ cat bench/comparison/scorecard-high.md # section "Per-language specialists"
The concessions
Every card ends with the concession. If a competitor is the better tool for your stack, the honest line below says so.
Semgrep CE / Opengrep
The free rules stop where the hard bugs start.
Bearer
The closest competitor on recall, at 6.7x the false alarms.
Single-language specialists
Free, tiny, and on Go, better than us.
Not benchmarked
Snyk Code has no column above: it requires an authenticated account, and publishing benchmark results may be restricted by its Terms of Service (a check we have not completed). The harness keeps a working Snyk runner: add snyk to --tools, authenticate, and produce the numbers locally for yourself after checking the current ToS. Akto and Pynt are DAST tools; they cannot scan a static code directory and exist only as documented stub lanes.
$ bench/comparison/REPRODUCE.md # section "Tools not included"
Reproduce it
The harness, the pinned corpus, the ground truth, and the scoring code are public. One command reruns the whole comparison from a fresh machine; the walkthrough is bench/comparison/REPRODUCE.md.
# zero-to-scorecard on a fresh machine (needs vulkro, git, python) VULKRO="$(command -v vulkro)" bench/comparison/run.sh --tier1 # the published headline setting VULKRO="$(command -v vulkro)" bench/comparison/run.sh --tier1 --min-confidence high # read the result cat bench/comparison/results-*/scorecard.md
Same binary, same corpus SHAs, same scoring code: same TP/FP/FN counts on any machine. Competitor counts can drift because the Semgrep registry ruleset is not pinned upstream; the scorecard records the versions it ran. Full methodology and every table.
Beyond the scorecard
A score tells you who caught more bugs. This tells you what Vulkro ships that the others do not. Every cell is a real capability, checked against each tool. Snyk, Semgrep, SonarQube, and Socket are the reference set.
| Capability | Vulkro | Snyk | Semgrep | SonarQube | Socket |
|---|---|---|---|---|---|
| Reproducible benchmark ships with the toolThe harness, pinned repos, ground truth, and scoring code are public. You reproduce the exact number on your own laptop. | Yes | No | No | No | No |
| Cross-file taint dataflowSource-to-sink data flow, 3 passes, cross-file depth 4. Free tools mostly ship regex, not a dataflow ladder. | Yes | Yes | Pro tier only | Paid only | No |
| Reachability-ranked SCA (KEV / EPSS)SCA across npm, PyPI, Go, Cargo, Maven. Local CVE bundle of OSV + NVD + CISA KEV + FIRST.org EPSS. Ranks reachable code first. | Yes | Partial | No | No | Partial |
| In-loop AI write guardvulkro guard checks AI-written code as it lands, before it reaches your working tree. | Yes | No | No | No | No |
| Hallucinated-package gatevulkro slopcheck refuses install of packages an AI invented (slopsquatting) before they hit your lockfile. | Yes | No | No | No | Partial |
| Per-tool AI-origin auditTracks which code an AI wrote, with reviewer attestation. Incumbents do not ship this. | Yes | No | No | No | No |
| Reachability-backed VEX + CBOMOpenVEX and CycloneDX-VEX with not_affected backed by reachability, plus CycloneDX CBOM (crypto BOM). | Yes | Partial | No | No | No |
| Scans run locally / no code telemetryYour code is never uploaded. VULKRO_OFFLINE=1 with a license file hard-refuses every outbound call; the account check sends six usage fields (product, install id, version, operating system, timestamp, and a scan counter), never code or findings. | Yes | No | Partial | Partial | No |
| Deterministic (same code, same result)No AI in the engine, so the same input always yields the same findings. Diffable in CI. | Yes | No | Yes | Yes | Yes |
| Local desktop console from one scan34 analysis views available over a single offline scan. The binary is also an MCP server, an LSP server, and the console backend. | Yes | No (cloud UI) | No | Server dashboard | No (cloud UI) |
| No surprise billingNo self-serve auto-charge. Licenses are issued directly by our team with explicit terms; nothing bills you automatically. | Yes | No | No | No | No |
Qualitative comparison: every Vulkro cell is shipping behaviour (vulkro --help); competitor cells reflect each vendor's public product documentation as of July 2026.
Beyond the table: 150+ core detectors, OWASP API Top 10 (all 10) and LLM Top 10, a 13-class DAST probe (authorization-gated), reachability-gated SCA across 5 ecosystems, and 23 machine-readable output formats including SARIF, CycloneDX SBOM, CycloneDX CBOM, SPDX 2.3 and 3.0.1, OpenVEX, and GDPR RoPA.
Migration
Pick your current tool. Get a one-line replacement for each common workflow.
Same workflow, no upload.
| Snyk Code workflow | Vulkro equivalent |
|---|---|
snyk code test | vulkro scan . |
Snyk PR check | vulkro scan . --since main --format gh-pr |
Snyk SARIF export | vulkro scan . --format sarif |
Snyk container scan | vulkro container <image> |
Head-to-heads
Long-form, one-on-one head-to-heads: the numbers, the workflow mapping, and where each tool genuinely wins.
Vulkro vs Snyk · Vulkro vs Semgrep · Vulkro vs Bearer · Vulkro vs Trivy · Vulkro for Salesforce vs CodeScan · Vulkro for Salesforce vs DigitSec · Best offline SAST