ravage

Ravage logo

Ravage

Ravage is an open-source, evidence-first research workspace for autonomous web application security evaluation on controlled, authorized targets. It combines a localhost-first evaluated base, deterministic exploit closure, an opt-in bounded agent graph, an explicitly authorized HTTP-only remote lane, deliberately vulnerable labs, audit trails, and benchmark/referee tooling.

Ravage is a research alpha, not a production safety guarantee or permission to test public systems.

Audit the current evidence · Run the local quickstart · Read the documentation · Contribute

Demo

https://github.com/user-attachments/assets/2f777a15-7bc0-4a8e-919d-fd874b157da1

Current Evidence

The frozen 2026-07-12 run completed all 104 public XBEN cases once under a description-only black-box contract:

Metric	Result
Exact randomized flags	85 / 104
Solve rate	81.73%
Retained terminal outcomes	16 failures, 1 error, 2 timeouts
Accounted model replies	1,151 / 1,151
Provider-usage list-price model cost	$55.758722
Cost per valid flag	$0.655985

The run used gpt-5.4-2026-03-05 through Ravage’s high route tier. Exact flag scoring, complete terminal outcomes, model usage, provenance, audit data, and SHA-256 manifests are retained. Start with the reviewer guide, not the raw archives.

The bounded agent graph, graduated graph budgets, deterministic session projection, and remote HTTP route were added after this frozen run. They are not part of the 85/104 result and must earn their own benchmark evidence.

What This Result Does Not Establish

It is one frozen full run, not a distribution. Earlier full runs used different development states; there is no repeat set under the identical Run 3 commit and configuration.
XBEN is public and saturated. Randomized flags still test live execution and extraction, but they do not restore task novelty.
No external-agent leaderboard is published yet. The competitor harness is implemented, but the current evidence is Ravage-only.
The result is specific to the complete GPT-5.4-based system configuration. It does not predict local-model or future-model performance.
The cost is reconstructed from recorded provider usage and frozen list prices. It is not an invoice and excludes compute, storage, engineering, and review time.
Exact-flag CTF performance is narrower than production pentest quality. It does not measure organization-wide coverage, remediation quality, or safe operation on arbitrary applications.

XBOW describes the public suite as outdated and saturated. Treat this release as auditable public-regression and harness-integrity evidence—not an unseen-task, state-of-the-art, production-efficacy, or head-to-head claim.

What Ravage Does

ravage attack: runs the model-driven web agent with scoped tools, evidence gates, resumable state, audit events, reports, and an attachable observer. An explicit agent-graph route can continue eligible unsolved local runs or operate directly through the restricted remote HTTP lane.
ravage scan: runs deterministic in-tree probes without model calls.
ravage lab: manages deliberately vulnerable local applications for safe, repeatable testing.
ravage audit verify: verifies the run database’s hash chain.
ravage xben: runs controlled XBEN selections with declared input and scoring contracts.
ravage competitors: preflights and executes configured external-agent adapters, adapts Ravage results, and produces comparison reports.
ravage code-bug: routes selected Agent Skills-style guidance into Ravage attack or XBEN workflows.

Normal operation is black-box-first: Ravage starts from an authorized brief, an in-scope target URL, a target description, rules of engagement, and live observations. Local targets use the frozen base first. A non-local target requires an explicit authorization acknowledgement and the HTTP-only graph route; that flag never expands the scope declared in the brief. Source code can support explicitly labelled white-box work, but source hypotheses are never proof by themselves.

No-Paid-Model Local Quickstart

Requirements:

Python 3.12;
Docker for the included lab;
no model provider key for this deterministic scan.

git clone https://github.com/duriantaco/ravage.git
cd ravage

scripts/bootstrap.sh
source .venv/bin/activate

ravage lab up ravage-acme-box --labs-dir examples/labs
curl --retry 20 --retry-connrefused --retry-delay 1 \
  http://127.0.0.1:8088/ >/dev/null

ravage scan examples/labs/ravage-acme-box/brief.yaml \
  --probe surface_map \
  --run-dir runs/quickstart-acme-scan \
  --report

ravage audit verify runs/quickstart-acme-scan
ravage lab down ravage-acme-box --labs-dir examples/labs

Expected artifacts:

runs/quickstart-acme-scan/
  audit.db
  run.json
  scan-summary.json
  report.md
  report.json
  workspace/events.jsonl

The bootstrap script creates a Python 3.12 virtual environment and installs the source workspace. Use python -m pip, not bare pip, for manual installs.

For the model-driven agent, model providers, your own localhost application, and troubleshooting, continue with Setup and How To Use.

How It Works

authorized brief + explicit scope + hard budgets
              |
       +------+------------------------+
       |                               |
       v                               v
 local target                  authorized remote target
 frozen evaluated base         direct bounded agent graph
       |                               |
 eligible unsolved stop                |
       v                               v
 opt-in bounded graph           structured HTTP only
       |                               |
 local scoped tools             serial scoped requests
       +---------------+---------------+
                       v
          typed observation + proof gate
                       |
                       v
       durable state, audit, receipts, report

Core boundaries:

Briefs define authorization, scope, rules, objectives, and budgets.
Scope is checked before HTTP, browser, redirect, and tool activity.
Model statements are hypotheses; findings require live target evidence.
Runs retain actions, observations, rejection reasons, proofs, and cost fields.
Graph-wide node, concurrency, model, tool, wall-time, cost, repeat, and proof reserve limits are enforced independently of model instructions.
Benchmark scoring is evaluator-owned and preserves failures, errors, and timeouts rather than replacing them with better retries.

The remote low-noise profile serializes traffic, obeys the brief’s rate cap, adds bounded deterministic jitter, keeps a stable identity, and records each hop. It is traffic restraint—not stealth, WAF bypass, or an assurance of non-detection.

Read Architecture for the evaluated/current boundary, graph controls, remote HTTP contract, and known gaps.

Command Map

Goal	Command
Create starter files	`ravage init --target-url http://127.0.0.1:8080`
Generate a brief	`ravage brief template --target-url http://127.0.0.1:8080`
Check setup	`ravage setup check --brief brief.yaml`
Inspect tools	`ravage tools list` / `ravage tools check`
Manage labs	`ravage lab {list,show,up,down}`
Run deterministic probes	`ravage scan BRIEF.yaml`
Run the model agent	`ravage attack BRIEF.yaml`
Continue an eligible local miss with the graph	`ravage attack BRIEF.yaml --autonomous-route --autonomous-route-engine agent-graph`
Run the authorized remote HTTP graph	`ravage attack BRIEF.yaml --authorized-remote-target --autonomous-route --autonomous-route-engine agent-graph --operational-profile low-noise`
Attach the live observer	`ravage observe RUN_DIR`
Render a report	`ravage report RUN_DIR --brief BRIEF.yaml`
Verify audit integrity	`ravage audit verify RUN_DIR`
Run XBEN	`ravage xben ...`
Compare configured agents	`ravage competitors {preflight,run,report,adapt-ravage}`
Route a knowledge pack	`ravage code-bug ... --skills PATH`

Run ravage --help and the relevant subcommand’s --help before following an old artifact or archived plan.

Run Artifacts

A model-driven run normally contains:

runs/<brief>-attack-<timestamp>/
  audit.db
  run.json
  workspace/
    events.jsonl
    transcript.jsonl
    working_state.json
    artifacts/
    terminal/
  report.md       # when --report is used
  report.json

Attack directories are stateful. Use a fresh directory for a new run; use --resume only to continue that same run. The observer masks captured flag values by default while showing proof state, progress, and command history.

Included Labs

Lab	URL	Flags	Primary focus
`ravage-acme-box`	`127.0.0.1:8088`	4	Support portal, authorization, injection, JWT, SSRF
`ravage-forgeops-box`	`127.0.0.1:8090`	6	Release operations, traversal, injection, internal services
`ravage-node-market-box`	`127.0.0.1:8092`	5	Market APIs, authorization, JWT, JSON state, internal pivoting
`ravage-perimeter-box`	`127.0.0.1:8094`	5	Multi-port recon, authentication, injection, traversal
`ravage-session-boundary-box`	`127.0.0.1:8096`	3	Session, CSRF, CORS, framing, WebSocket, browser storage

Read the lab index and each lab’s README for its operator contract. OPERATOR_NOTES.md, manifests, expected flags, and source are harness material; do not give them to a strict black-box agent.

Safety And Scope

Only run Ravage against systems you own or have explicit written authorization to test, within the agreed rules of engagement. Do not run it against third-party systems or infrastructure outside that authorization.

ravage scan, the default attack path, and broad shell/browser/tool capabilities remain localhost-first. The only non-local path is the explicitly acknowledged agent-graph HTTP route. It rechecks every request and redirect against the brief, pins DNS identity, persists its request ceiling, and exposes no shell, browser, process runtime, or external scanner. Included labs contain serious intentional vulnerabilities and must stay on localhost or an isolated private network.

Read DISCLAIMER.md before use. Report vulnerabilities in Ravage itself through SECURITY.md, never through a public issue containing exploit details or secrets.

Documentation

Use:

Setup: Python, tools, model routes, briefs, and setup checks.
How To Use: labs, scans, attacks, observation, reports, and troubleshooting.
AI Web Operator Guide: model-driven operation and scoped tools.
Model Providers: local and hosted routes.

Evaluate:

Understand and build:

Contributing

Contributions are welcome. Start with CONTRIBUTING.md, which covers safe development, test tiers, probe expectations, benchmark evidence, commit titles, and pull-request requirements.

Use the issue forms for reproducible bugs, feature proposals, and benchmark reproductions or disputes. Never upload real credentials, provider keys, private target output, cookies, or unreviewed raw run archives.

Quick development setup:

scripts/bootstrap.sh
source .venv/bin/activate
.venv/bin/python -m pytest path/to/relevant_test.py -q

Release Status And License

Ravage is currently source-checkout-first. The ravage==0.0.1 and ravage-schemas==0.0.1 PyPI packages are legacy previews, not the canonical state of this checkout. Until a newer tagged release is published, install from this repository and record the exact commit when reporting results.

The core CLI, schemas, labs, audit trail, benchmark harness, and Markdown/JSON reports are licensed under the Apache License 2.0. Professional PDF and DOCX report formats remain a separately licensed extension point.

This site is open source. Improve this page.