The scanner-versus-pentest gap
Two kinds of security testing live in your pipeline today, and both of them fail you at CI cadence. Fast scanners spray noise: severity 8 on a code path that never executes, or a CVE in a vendored library your build strips out. Slow pentests give you proof, then hand it over three weeks after the release you were asking about shipped. In between is a gap the size of every deploy you make on a Tuesday afternoon.
Mehdi Boutayeb, co-founder of ASC-IT and a former Airbus engineer, made that gap the whole point of an August 12 essay on devops.com. His argument, condensed: teams ship dozens of times a day, security testing has not kept up, and the fix is autonomous agents that operate the way an attacker would. Enumerate. Form a hypothesis. Try to exploit it. Report only after proving it worked.
He then plants a flag on the second half of the piece: those agents belong on your side of the network boundary, not on somebody else's.
What "proof-based" actually buys you
Boutayeb frames autonomous agents as reasoning loops that "enumerate, form a hypothesis, try to exploit it, and only report the finding once they have proven it." If you have ever spent a sprint triaging generic scanner output, you already know why that matters. The current CI scanner UX is a lottery ticket that costs you an engineer-hour to redeem, and most tickets do not pay out.
There is a second reason proof-based agents belong in CI. Real breaches usually chain several bugs together. A leaked token that only reads a bucket becomes catastrophic once you notice the bucket serves the CI runner. Static analyzers, by design, cannot see the chain. They see one file at a time. An agent that queries your infrastructure, remembers what it found, and tries a next step can. That is a real capability jump, and it is why the vendor pitch is going to arrive in every CISO inbox this quarter.
The catch, obviously, is that "an agent that queries your infrastructure" is a phrase that should make a supply-chain person go very quiet.
The bill for sending it off-site
Here is the part of the article worth pinning above your CI config. If the agent doing the reasoning lives in someone else's cloud, then, per the essay: "Your internal IP addresses, your hostnames, sometimes your credentials and your source code travel to a cloud endpoint." That is not paranoia. That is the request shape of every hosted model tool that "just needs read access."
Boutayeb lists concrete casualties. Compliance regimes and customer contracts that forbid external data transmission. Critical-infrastructure operators who cannot legally ship internal topology to a third party. Any team, honestly, whose secrets committed to git history are among the first things a competent pentest would surface. You are being asked to send the exact material an attacker would exfiltrate to a SaaS endpoint, so that the SaaS endpoint can tell you an attacker could exfiltrate it. Efficient.
The vendor answer to that objection is usually a page about SOC 2 and a data-processing agreement. Fine. That is not the same thing as the data never leaving.
Tokens at the boundary
The concrete alternative in the piece is not "just self-host it," which we all know is where reasonable proposals go to die. It is a design pattern worth naming, because it will show up in RFP after RFP over the next year:
Run the reasoning model locally on open-weight weights. Before anything sensitive reaches the model, do "tokenization at the boundary": real hostnames, real IPs, real credentials get swapped for deterministic placeholders. The model reasons over the tokens. When a tool call actually needs to execute, the placeholders are swapped back locally, inside your trust boundary. The model sees an abstract shape of your infrastructure. The tools see the real thing. The wire between you and the vendor sees neither.
In pseudo-config, the swap looks something like this:
tokenization:
boundary: pre-model
rules:
- kind: hostname
pattern: "*.internal.$ORG"
placeholder: "<host-{n}>"
- kind: secret
source: $SECRETS_BACKEND
placeholder: "<cred-{n}>"
- kind: source-file
pattern: "src/**"
placeholder: "<file-{n}>"
reinject: at-tool-call # placeholders resolved locally, never sent upstream
The piece points to Darkmoon as an example: an open-source autonomous penetration testing platform. Take that as a data point, not an endorsement; the pattern is what matters. Any vendor pitching an autonomous testing agent should be able to draw you the same diagram, or explain why they cannot.
A short checklist before you sign anything
Boutayeb closes on four evaluation questions worth stealing verbatim into your procurement doc. Paraphrased for the on-call:
- Does a finding come with a reproducible proof-of-exploit, or only a severity number?
- Where does your target data live during a run, and can the agent complete a job with zero outbound calls to the vendor?
- Can a developer re-run the finding on their laptop from what the tool handed them?
- Is the methodology auditable, or is "trust the model" the whole answer?
The moment a sales engineer flinches on question two is the moment you stop taking notes.
Autonomous testing is coming to CI whether you buy it this quarter or next. Skip the debate about scanner versus agent. The design fight worth having is local versus remote. Pick wrong and you have paid a vendor to build the exfil channel for you.
