Confidence model

Every finding Vulkro emits carries a confidence: High | Medium | Low value plus a confidence_reason: String that explains why the engine fired and what would mark this a false positive. This is the heart of how we keep a high-recall scanner usable.

Levels

Level	Meaning	When you'd see it
High	Strong, often runtime-confirmed evidence.	Provider-format secrets, taint-confirmed injection paths, KEV-listed CVEs, vulkro probe confirmations.
Medium	Pattern matches but with credible false-positive shape.	Heuristic IDOR (path takes an :id and no auth check is visible, but might be wrapped). Generic secret patterns without provider format.
Low	Heuristic. Often fires on test/example/migration code.	Pattern-match-only checks where context confirms little.

vulkro scan defaults to --min-confidence high. To see the rest:

vulkro scan . --min-confidence medium
vulkro scan . --all

Strict-confidence filter

For CI gates that want maximal precision, vulkro scan --strict-confidence (or VULKRO_STRICT_CONFIDENCE=1) layers an additional filter on top of the --min-confidence cut: any High finding whose evidence bag is empty is dropped at output time. A finding survives the strict cut only when it carries at least one populated Evidence item, which means at least one corroborating signal (a taint-flow source-to-sink, an exact-match invariant such as a SUP-COMPROMISE catalog row, a schema-vs-code contradiction, or a pair of pattern signals whose weights sum past the cumulative threshold).

Off by default; behaviour-preserving for callers who don't opt in. See the strict-confidence section on vulkro scan for the flag's surface and the env-var equivalent.

Per-detector-category tiers

The generic High | Medium | Low table above tells you the shape of a finding's confidence. What each level specifically means is detector-specific. The table below is the authoritative per-category reading; every rule page (under /docs/rules and siblings) links back here.

Detector category	High means	Medium means	Low means
Secrets (secrets, git_history)	Provider-format match (AWS AKIA, Stripe sk_live, GitHub ghp_, etc.) on a non-test path. The string is shaped like a real credential.	Generic high-entropy literal that matches no provider format, but assigned to a *_KEY | _TOKEN | _PASSWORD identifier. Could be a placeholder.	Long hex / base64 string with no naming signal. Frequently a hash, content checksum, or test fixture.
Auth (auth_dataflow, authz)	Route is reachable AND no auth middleware, dependency, decorator, or guard is visible in the import scope or call graph.	Route looks unprotected at the file level, but a global mount (app.use('/api', ...)) or framework convention (APP_GUARD) could be applying auth at runtime.	Route name pattern-matches admin/internal/private but no behavioural signal supports the claim.
IDOR / BOLA (idor)	Handler reads a path/query param like :id and resolves the object without a where(user=...), .filter(owner=...), or equivalent scoping clause; and taint flows from the param into the lookup.	Handler takes an :id and the lookup is unscoped, but no taint analysis confirmation (the param might be derived).	Heuristic only: an :id route exists with no visible check, used for forensic mode.
Mass assignment (mass_assignment)	Body-to-model bind via Model(**req.json), Object.assign(user, req.body), setattr loop over body, etc., on a model whose schema has fields the API surface does not allowlist.	Body-to-model bind on a model whose schema we cannot fully resolve (cross-file fragment, dynamic ORM).	Pattern match alone, no model resolution.
Injection (injection, injection_extra, taint)	End-to-end taint chain from request source to a SQL / shell / template / eval sink without intervening sanitiser.	Sink call with a likely-tainted argument but no full chain confirmation.	Sink call with no taint corroboration; pattern-only.
Crypto (crypto_weakness, crypto_new)	Exact-match invariant: md5(buf) for a password, MODE_ECB for confidentiality, verify=False on TLS. The construct is provably wrong regardless of context.	Weak primitive in use, but might be for a non-security purpose (e.g. md5 for cache key).	Reference to a weak primitive in a comment, docstring, or test name.
SCA / CVE (package_risk, slopsquat_known)	Direct dependency matches a KEV-listed advisory at the exact pinned version.	Transitive dependency matches an advisory; or the version match is a range hit not an exact one.	Heuristic typosquat / new-package signal without a CVE backing it.
SSRF (security::ssrf, taint)	Tainted URL flows into an HTTP client call without an allowlist check or hostname normalization between source and sink.	Tainted URL flows into a sink but the chain crosses a function we couldn't fully resolve.	Pattern only: URL-shaped argument to an HTTP client.
LLM (llm_security)	Tainted user input flows into a model SDK call (openai.chat, anthropic.messages, langchain) without isolation or structured-tool constraint. Or: an env var known to hold a secret flows into the prompt.	Tainted input near an LLM call but no full chain.	LLM call with any string-template usage that could in principle contain user data.
CSRF / cookies (csrf, cookies)	Session-cookie middleware is mounted AND CSRF middleware is absent or commented out; or cookie is set with SameSite: 'none' without Secure.	Session middleware present, CSRF middleware unclear; or cookie missing one of Secure/HttpOnly/SameSite but not all.	Pattern-only: a session-like cookie name with no other signal.

If a finding's tier surprises you, look up the category here first; the per-category meaning above is what the rule actually asserts. The matching rule page at /docs/rules walks through the OWASP category and its detectors in detail.

confidence_reason

Every finding has a one-line explanation:

confidence_reason = "taint flowed from req.body to db.query without sanitiser"
confidence_reason = "auth helper not found in import scope"
confidence_reason = "runtime-confirmed via active probe"
confidence_reason = "AKIA-prefix matches AWS access-key format"

Surfaced in JSON, SARIF (properties.confidence_reason), and the desktop console. Designed to be readable without re-reading source.

Calibration table

Vulkro holds a static calibration table that downgrades High -> Medium for detector categories scoring under 30% TP-rate on the 13-repo benchmark. The intent is that if a category is 70%+ false-positive in the wild, callers shouldn't be told "High confidence" by default.

The table is updated when benchmark numbers change. This keeps the default-mode signal-to-noise ratio honest as new detectors land.

Where the bar is

The published benchmark on 13 deliberately-vulnerable repos:

	vulkro default	vulkro --min-confidence high	semgrep CE	bearer 2.0
precision	0.21	0.62	0.60	0.45
recall	0.91	0.76	0.22	0.45
F1	0.34	0.68	0.32	0.45

--min-confidence high is now the production-recommended cut: it leads Semgrep CE on precision (0.62 vs 0.60) and Bearer on recall (0.76 vs 0.45) on the same corpus. F1 0.68 beats both major competitors outright.

The lift came from three changes shipped on top of the Phase 4 AST- confirmation engine: the cumulative-weight High threshold was calibrated from 1.5 to 1.1 to match what shipped detectors actually emit (Phase 4 pairs at 0.7 + 0.5 = 1.2); four taint / template / autoescape emit sites were retrofitted to attach Evidence so the aggregator can survive the OWASP-category calibration downgrade; and a (file, line, message) dedup pass removes duplicate emissions from overlapping intra-/inter-procedural taint engines.

See Benchmark for the full methodology and per-repo breakdown.