A signal is a lead. A finding is proven.
The difference between an autonomous pentest you can trust and one you can't is not what it notices — it is what it does next. A tool result or a model's assertion is a candidate. It becomes a finding only when it is reproduced from a clean session and the impact is supported by evidence. Here is exactly how that happens, and why the unproven is held back.
The short version: every result starts as a candidate — a tool hit, a model assertion, a suspicious response. A candidate becomes a finding only when it reproduces from a clean session with the same outcome and the impact is backed by evidence. The request, the control, the replays and the impact travel with it, hash-chained. Anything that does not reproduce is held back, not shipped for you to sort. That discipline is the product.
The trust problem with autonomous testing
An autonomous tester that reports everything it notices is just a scanner with better prose — and a model that reports everything it suspects is worse, because it is confident. The hard question a security team asks is not "did it find things" but "can I trust the things it found enough to spend engineering time on them?" The answer has to be built into how a result is promoted, not bolted on as a confidence score.
From candidate to finding
Every result passes the same gate:
- Business rule — the specific thing the application is supposed to enforce (an invoice is visible only to its owner; an order total is never negative).
- Controlled test — a deliberate experiment against that rule, run as the identities you supply, with a control to compare against.
- Clean-session proof — the result is reproduced from a fresh context before it counts.
- Impact — what a confirmed break means for the business, so the finding is actionable, not academic.
A critic step actively challenges thin evidence and holds back the unproven, so the bar to promotion is adversarial, not permissive.
Why a clean session is the pivot
Most false positives survive because nobody tried to make the issue disappear. A single success can be luck, leftover state from an earlier step, or an artefact of the testing context — all of which vanish on a second look. Reproducing from a clean session — fresh context, no conversation history, the conditions reset — removes those explanations. If the issue reappears from a clean start, it is a property of the application, not of the run. If it doesn't, it was never a finding.
What a confirmed finding carries
A finding is only as good as what you can do with it. Each one travels with the exact request, the control it was compared against, the replays that reproduced it, and a direction for the fix — captured in a hash-chained, append-only ledger so the evidence cannot drift from the claim. Where the class calls for it, the proof is two-sided: a control that is correctly refused alongside a test that is wrongly allowed, so a single ambiguous response is never enough. See a full worked finding for exactly what that looks like.
What gets held back — and why that matters
Just as important as what is promoted is what is not. Unstable signals, one-off results that don't reproduce, and anything the evidence doesn't support are held back rather than reported. This is the opposite of a scanner's incentive, which is to raise everything and let you triage. It also means our honesty runs in both directions: no finding is not a clean bill of health, and coverage is reported as evidence — tested, no eligible surface, or blocked — so you always know what the absence of a finding actually means.
FAQ
What makes a finding a finding and not a candidate?
Independent reproduction. A signal is a candidate; it becomes a finding only when it reproduces from a clean session with the same outcome and the impact is supported by evidence. A signal that doesn't reproduce is held back.
Why reproduce from a clean session?
A single success can be luck, leftover state or a testing artefact. A clean session removes those explanations — if the issue reappears from a fresh start, it is a property of the application, not the run.
How do you keep false positives low?
By refusing to promote anything not reproduced and corroborated, and by requiring two-sided proof where the class calls for it. The unproven is held back, not shipped for you to triage.
See the discipline in action. Read a worked finding, the canary method, or book a demo.