The BOLA bug was fixed. What proves it stays fixed?

A pentest report can prove that user B can read user A's object. The team fixes
the authorization check, closes the ticket, and moves on.

Six months later, a refactor removes that check.

The original report still exists, but the proof is no longer running anywhere.
An ordinary happy-path API test usually cannot distinguish “the owner can read
the object” from “every authenticated user can read the object.” A scanner may
rediscover the bug eventually, but that is not the same as preserving the exact
security property the team already learned.

I built ExploitSpec around a narrow
idea: once a human has proven an HTTP exploit, turn that proof into a small,
reviewable regression test that lives beside the code.

It is deliberately not a scanner. It does not decide whether an endpoint is
vulnerable. The human supplies the meaningful invariant; the runner makes that
invariant deterministic enough for CI.

The invariant, not just the request

Consider a BOLA/IDOR finding:

1. an owner creates a private invoice;
2. the test captures the runtime invoice ID and payment reference;
3. another authenticated tenant requests that exact invoice;
4. the response must be 403 or 404;
5. the body must not contain the captured payment reference or sensitive JSON fields.

The important part is the relationship between two actors and a dynamic object,
not a hard-coded URL copied from a proxy history.

version: "1"
id: SEC-BOLA-001
name: Cross-tenant invoice access is denied

actors:
  owner:
    headers:
      Authorization: "Bearer ${OWNER_TOKEN}"
  attacker:
    headers:
      Authorization: "Bearer ${ATTACKER_TOKEN}"

steps:
  - name: Owner creates a private invoice
    actor: owner
    request:
      method: POST
      path: /api/invoices
      json:
        customer_email: alice@tenant-a.example
        amount_cents: 4200
    expect:
      status: 201
    capture:
      invoice_id:
        json_path: $.id
      payment_reference:
        json_path: $.payment_reference

  - name: Another tenant cannot read it
    actor: attacker
    request:
      method: GET
      path: "/api/invoices/{{ invoice_id }}"
    expect:
      status: [403, 404]
      body:
        not_contains:
          - "{{ payment_reference }}"
      json:
        not_exists:
          - $.customer_email
          - $.payment_reference

Each actor receives an independent cookie jar and header set. Values captured by
one step can be used by later steps, so a test does not depend on fixture IDs.

RED, GREEN, and STABLE

A test that passes against the fixed version is not yet trustworthy. It could
also pass against the vulnerable version, or succeed only because a fixture is
missing.

ExploitSpec therefore calibrates the same invariant against two known states:

exploitspec calibrate \
  --vulnerable-url http://127.0.0.1:18080 \
  --fixed-url http://127.0.0.1:18081 \
  --runs 3 \
  finding.exploit.yaml

ExploitSpec calibration  SEC-BOLA-001
  + RED    invariant fails on the vulnerable baseline
  + GREEN  invariant passes after the fix
  + STABLE fix passes 3/3 runs

Calibration accepted. This spec can now guard the fix in CI.

Here, RED does not mean “critical severity.” It means the invariant correctly
detects the known-vulnerable state. GREEN proves the fixed state satisfies the
invariant. STABLE repeats the fixed-state check to expose flaky tests before the
team trusts them.

You can run the complete local demonstration with only Go:

git clone https://github.com/pazent/exploitspec.git
cd exploitspec
make demo

There is also a
minimal consumer repository that
runs the released GitHub Action and publishes JUnit output.

Why plain YAML?

The format is intentionally boring:

• the security invariant can be reviewed in a pull request;
• the test stays with the application instead of a SaaS account;
• captured evidence and assertions have explicit names;
• deterministic exit codes and JUnit/JSON reports fit existing CI;
• there is no model call in the execution path.

The runner is local-first, has no account or telemetry, and is Apache-2.0
licensed.

Importing evidence without committing secrets

ExploitSpec can turn one supported cURL command into a starter file, but treats
the input as hostile data. It never invokes a shell or sends the request.

The importer conservatively replaces query and form values, JSON string leaves,
and non-allowlisted header values with required environment variables. It
rejects malformed URLs, shell control syntax, unsupported opaque bodies, and an
existing output path.

That is intentionally noisier than guessing which value “looks like a secret.”
The generated file is a starting point for a human review, not a magic finding
converter.

Current limits

The project is pre-1.0 and deliberately narrow:

• it executes HTTP invariants; it does not discover vulnerabilities;
• authors must control cleanup for mutating requests;
• explicitly allowed hostnames are not DNS-pinned;
• proxy behavior follows Go's environment configuration and still needs a wider compatibility matrix;
• imported evidence still requires review before commit or execution;
• a regression test is only as strong as its assertions and test data.

Those limits are documented because a security tool should make its trust
boundary inspectable.

The question I want help answering

Which confirmed AppSec finding is hardest to encode as a deterministic HTTP
regression test: authorization boundaries, business-logic races, signed
requests, or something else?

The repository, runnable demo, specification, and open roadmap are at
github.com/pazent/exploitspec.

Disclosure: I created and maintain ExploitSpec. I used an AI coding assistant
to help edit this article; every command and technical claim above was checked
against the public repository.