A Vulnerability Is a Hypothesis. An Exploit Is the Proof.

Until exploited, all findings are just hypotheses. For years, security teams struggled with noise from scanners flagging patterns, now it comes from frontier models producing plausible findings. But plausible is not proof.

Every security team runs on one scarce resource: the attention of the people who can actually fix things. Each finding you put in front of them consumes that attention, so you want to spend it only where it is warranted. Only an exploitable bug is worth interrupting an engineer for.

A bug is a hypothesis until you prove it

Whether a weakness can be exploited depends on whether an attacker can reach it, and what they can accomplish once they do.

Take server-side request forgery. The same SSRF, byte for byte, is a curiosity in one deployment and a full compromise in another. It’s informational behind a strict egress policy with nothing sensitive on the internal network. On a host whose metadata service hands out cloud credentials, it reads those credentials and pivots into the account, using the same bug and the same source. The difference is in the metadata service, the reachable hosts, and the egress policy, none of which a model or scanner reading the source can see. Looking only at the source, they appear identical.

A finding that sounds plausible but has not been reproduced is only a hypothesis, and treating it as a result clogs the queue. Worse, it lets the dangerous ones hide: a weakness that looks minor on its own is often the first step of a chain that ends in a serious compromise.

The worst cases never even show up in the code. Business logic and authorization flaws such as IDORs live between the source code implementation and the design documentation and PRDs. Looking only at the implementation is not enough to tell if the code is doing what it’s supposed to do. An endpoint returns a record by its ID, parses the input, checks that the caller is logged in, and returns the data with no injection, no memory bug, and nothing to flag. It just never checks whether this user is allowed to see that record, so anyone can read anyone else's data by changing a number in the URL. You find the flaw by trying the thing that should not work and watching it work.

AI made the pile bigger

AI has changed the scale of vulnerability research. A model produces candidates: things that look like they might be wrong. It doesn’t tell you whether the vulnerability is reachable, whether a mitigation addresses it, or if it’s a real violation of intent, or just unusual code. A flood of candidates only scales up the noise problem.

Proof is the cut, and proof has to be autonomous

There are only two ways to decide whether something is reachable and exploitable. You can predict it by reasoning about the code, or you can demonstrate it, by reaching the weakness in the running system and proving impact.

But AI now produces findings faster than any human can prove them. When candidates arrive by the thousands, a person validating each one moves the backlog to the most expensive team in the company. So the proving has to be autonomous, but autonomy is hard. A model on its own gives up after the first promising lead, wanders out of scope, repeats work it has already done, and sometimes reports a plausible result as a real one. Turning it into a system that proves exploitability takes persistence across long, branching paths; coordination so work is not repeated; strict scope; and independent validation at every step. The hardest part is deciding what to do next, because every response is new evidence.

An autonomous system has to form a hypothesis from that evidence, then choose the next experiment to test it. Consider a request to /actuator/env. A web application firewall blocks it, but an autonomous system asks why it was blocked. If the WAF is matching the literal path, /%61ctuator/env may slip through. If it does, and the application decodes %61 back to a, the mitigation doesn’t hold.

Can you safely let an autonomous system make those decisions against a production application? The answer is yes, but not because a human is watching every step. Safety comes from governance: scope you control, validation that proves exploitability without disrupting the application, and every action logged and replayable.

Earn the interruption

Offensive security today has to determine what an attacker can actually reach and exploit in the running system, and then prove it. They then need to produce a short, demonstrated set of findings a team can act on, each one carrying the proof that earned the interruption.

The Right to Interrupt: A New AppSec Operating Model for AI-Generated Findings details how to adapt AppSec programs to accommodate this shift.