mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
docs(execution-grounded-detection): corrective brief that supersedes the project
Written after five evolve rounds produced zero decisions distinguishable from noise, and after reading Clearwing, Big Sleep, and the AIxCC 2025 finals. The premise we got wrong: OpenUltraSAST treats the LLM as a noisy oracle to average (K runs), floor (noise budget), gate (sign test) and slowly improve (evolve loop). 120 runs produced 120 distinct trajectories -- the variance is the loop's freedom to search, not the model's uncertainty. And the premise is empirically false as a claim about LLMs: Big Sleep found a real SQLite zero-day in Nov 2024; AIxCC finalists uncovered 18 real 0-days across 54M LOC, with the 2nd-place system using non-reasoning LLMs at 90% accuracy and $181/point. The lever is architecture, not prompt optimization. What the winners share, and Clearwing is the open instance of: the LLM is a judgement layer on an execution substrate, and truth comes from the sandbox -- a crash, a sanitizer report, a patch that stops the crash -- not from a score against a labeled corpus. Clearwing's poc_runner (apply diff, recompile, re-run, does-the-crash-still-happen) replaces our entire noise architecture. We built K-runs because we had no ground truth; they generate their own. The course: adopt Clearwing as the substrate (the maintainer already forks it), delete the noise architecture rather than fix it, and contribute the one thing Clearwing lacks -- the web/logic detection path. Our IR candidate enumerator (12.1% ruleset vs 96.6% IR, measured) and our obligation/absence-bug checker reach the dangerous web bugs that never crash a sandbox, which Clearwing's crash-oracle model cannot. The open research problem is a differential oracle for those classes: the vulnerable-parent/fix-commit pair as ground truth, with a request the fixed version rejects and the vulnerable one serves. Recommendation is explicitly not to rebuild: eighteen defects this session, all ours, against a 19k-line battle-tested MIT substrate. Not approved. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
66a51c674c
commit
81a63cf5ac
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "constrained-detector",
|
||||
"created_at": "2026-09-06T14:30:00Z",
|
||||
"updated_at": "2026-09-06T17:00:00Z",
|
||||
"updated_at": "2026-09-08T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "tasks",
|
||||
"approvals": {
|
||||
@@ -22,5 +22,6 @@
|
||||
"depends_on": [
|
||||
"learning-harness"
|
||||
],
|
||||
"approval_note": "2026-09-06 brief drafted at the maintainer's direction ('I'd rather spend time to design the exact DSPy style interface and evolutionary / statistical prompt optimization than fixing piece by piece'). Not approved; requirements not generated.; 2026-09-06 renamed from detector-optimizer to constrained-detector after the trajectory measurement: the brief's own conclusion is that the optimiser is not the next thing to build, and the spec should be named for what it delivers; 2026-09-06 requirements generated at the maintainer's direction, with the maintainer's emphasis recorded as Requirement 3 ('don't forget the emphasis on the real world bugs from the larger projects to avoid overfitting. we otherwise benchmark for cases that are not relevant'); 2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase'); 2026-09-06 design approved by the maintainer in session ('approve the design and generate the tasks')"
|
||||
"approval_note": "2026-09-06 brief drafted at the maintainer's direction ('I'd rather spend time to design the exact DSPy style interface and evolutionary / statistical prompt optimization than fixing piece by piece'). Not approved; requirements not generated.; 2026-09-06 renamed from detector-optimizer to constrained-detector after the trajectory measurement: the brief's own conclusion is that the optimiser is not the next thing to build, and the spec should be named for what it delivers; 2026-09-06 requirements generated at the maintainer's direction, with the maintainer's emphasis recorded as Requirement 3 ('don't forget the emphasis on the real world bugs from the larger projects to avoid overfitting. we otherwise benchmark for cases that are not relevant'); 2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase'); 2026-09-06 design approved by the maintainer in session ('approve the design and generate the tasks'); 2026-09-08 SUPERSEDED by execution-grounded-detection: this session's measurements showed the noise architecture (K-runs, per-family floor, regression budget, sign test, evolve loop) is a statistical treatment for a structural problem, and that the field (Big Sleep, AIxCC finals, Clearwing) grounds detection in execution rather than in scoring against a labeled corpus. The taxonomy, the IR candidate enumerator (96.6% ceiling) and the obligation checker survive as the web/logic detection path to contribute to a Clearwing-based substrate.",
|
||||
"superseded_by": "execution-grounded-detection"
|
||||
}
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
# Corrective brief: execution-grounded detection
|
||||
|
||||
**Status:** course-setting. Supersedes `learning-harness` and `constrained-detector`. Not a requirements
|
||||
document — a decision about what to build, what to delete, and what to adopt, written after the
|
||||
learning-harness line produced numbers its own measurements say to discard.
|
||||
|
||||
## 1. The premise we got wrong
|
||||
|
||||
OpenUltraSAST was built on an unstated assumption: that an LLM is a *noisy oracle* whose judgement must be
|
||||
averaged (K runs), floored (per-family noise budget), gated (sign test), and slowly improved (evolve loop).
|
||||
Every mechanism in `learning-harness` follows from that assumption. This session measured all of it and it
|
||||
does not hold:
|
||||
|
||||
- Five evolve rounds, $2.17, **zero decisions distinguishable from chance** — because the acceptance rule
|
||||
compares two ensemble votes over a detector that never runs the same investigation twice.
|
||||
- 120 runs of one detector produced **120 distinct trajectories**. The variance is the loop's freedom to
|
||||
search, not the model's uncertainty about an answer.
|
||||
- The noise floor a regression budget is derived from measures *how often an unconstrained search happens
|
||||
to reach the sink* — so the worse the search wanders, the more collateral damage a round is licensed to do.
|
||||
|
||||
The assumption is also empirically false as a claim about LLMs, and the evidence is now overwhelming:
|
||||
|
||||
- **Google Big Sleep** (Nov 2024): the first LLM-discovered zero-day in widely-used real-world software — a
|
||||
stack buffer underflow in SQLite — found by *variant analysis*, a sandboxed test case, and an
|
||||
auto-generated root-cause report. Not pattern-matching; reproduction.
|
||||
- **DARPA AIxCC finals** (DEF CON 33, Aug 2025): seven autonomous systems processed 54M lines of code,
|
||||
patched 43 of 54 synthetic vulnerabilities and **uncovered 18 previously-unknown real 0-days**. Buttercup
|
||||
(Trail of Bits, 2nd place) found **28 vulnerabilities across 20 CWEs at 90% accuracy, $181 per point,
|
||||
using non-reasoning LLMs only**, ~$152 per task, 45 minutes each.
|
||||
|
||||
The lever in every one of those systems is **architecture, not prompt optimization**. Buttercup won with
|
||||
non-reasoning models. Nobody evolved a prompt with a genetic loop. The question "can LLMs find security
|
||||
bugs" was answered *yes* by 2024; we spent this project answering a question the field had closed, with the
|
||||
wrong architecture.
|
||||
|
||||
## 2. What the successful systems share, and Clearwing is the open one
|
||||
|
||||
Big Sleep, Atlantis (AIxCC 1st), Buttercup (2nd), and **Clearwing** (`/home/mc/Source/clearwing`,
|
||||
Lazarus-AI, MIT, an open reproduction of Anthropic's Glasswing) share one shape:
|
||||
|
||||
> **The LLM is a judgement layer on top of a deterministic + execution substrate. Truth comes from the
|
||||
> sandbox — a crash, a sanitizer report, a patch that stops the crash — not from a score against a labeled
|
||||
> corpus.**
|
||||
|
||||
Clearwing's evidence ladder is the whole thesis in one type:
|
||||
|
||||
```
|
||||
suspicion → static_corroboration → crash_reproduced → root_cause_explained
|
||||
→ exploit_demonstrated → patch_validated
|
||||
```
|
||||
|
||||
The load-bearing gate is `poc_runner.py`: apply a candidate diff inside the sandbox, recompile, re-run the
|
||||
PoC, and return a boolean for *does the crash still happen*. That single primitive replaces our entire
|
||||
noise architecture. You do not average five noisy runs when a container can tell you whether the bug is
|
||||
real. **We built K-runs and noise floors because we had no ground truth. They generate their own.**
|
||||
|
||||
## 3. What Clearwing does better, mapped to what failed here
|
||||
|
||||
| Our failure this session | What Clearwing does instead |
|
||||
|---|---|
|
||||
| K-runs / noise floor / sign test / regression budget — a statistical treatment for a structural problem | **Execution is the arbiter.** `poc_runner` reproduces or it doesn't. `stability.py` runs the PoC across N fresh containers (≥90% = stable) to kill flakiness at the source instead of averaging it. |
|
||||
| Evolve loop: 5 rounds, 0 reliable decisions, $2.17 to learn nothing | **A fixed specialist roster** dispatched by file tags — `memory_safety`, `kernel_syscall`, `crypto_primitive`, `web_framework`, `logic_auth`, `general` — plus **variant analysis** (verify a bug, generate a grep+semantic pattern, find its structural twins, iterate to fixpoint). Density compounds *within a run*, no optimizer. |
|
||||
| Second judge / fusion, gesturing at independence | **4-axis adversarial validator** (REAL / TRIGGERABLE / IMPACTFUL / GENERAL) that sees *only* the finding, PoC and exploit — never the discovery transcript — steel-mans both sides, and **must attempt reproduction**. |
|
||||
| Corpus: 12 hand-harvested function excerpts per family, scored by text-inside-function, capped by twins/straddles/truncation defects we spent days repairing | **Corpus is real repos at vulnerable-parent commits + OSS-Fuzz** (7000 targets in `full` mode), run blind, with the fix commit used only as an *after-the-fact oracle*. Truth is generated by the sandbox, not annotated by us. `CampaignConfig` scales this to hundreds of repos with shared budget, checkpoint/resume, and diminishing-returns detection. |
|
||||
| Detector chooses what to look at (555 grep calls vs 71 entry-point lookups) | **Deterministic preprocessing feeds candidates:** callgraph → intra-procedural taint (source→sink) → 3-axis ranker → hunter. The model judges; it does not search for the sink. This is exactly the `constrained-detector` design we had started — Clearwing is the mature version of it. |
|
||||
|
||||
## 4. The one thing OpenUltraSAST proved that Clearwing lacks
|
||||
|
||||
This is not a total loss, and the course is not "throw everything away." Two results here are real,
|
||||
measured, and *absent from Clearwing*:
|
||||
|
||||
1. **IR-driven candidate enumeration with a published ceiling.** We measured that the pattern ruleset caps
|
||||
recall at 12.1% while call-sites-and-bindings from the semantic IR reach **96.6%** across the web slices,
|
||||
a median of ten sites per function. Clearwing's candidate feeder is C-first (taint tables for
|
||||
`memcpy`-shaped bugs); its `web_framework` specialist is a *single unconstrained prompt* with no
|
||||
equivalent candidate enumerator for logic/web classes.
|
||||
|
||||
2. **Absence-bug detection.** Our `obligations` work — an access-control bug is the *absence* of a guard
|
||||
around an operation its siblings protect — has no analogue in Clearwing, whose ladder is built around a
|
||||
*crash*. An IDOR or a missing authorization check never crashes a sandbox. Clearwing's execution
|
||||
ground-truth model does not reach these classes at all, and they are exactly the "dangerous but
|
||||
low-hanging" web bugs this project was chartered to catch.
|
||||
|
||||
So the gap Clearwing leaves open is **the web/logic taxonomy without a crash oracle** — which is where our
|
||||
IR candidate enumerator, our family taxonomy, and our obligation checker are genuinely ahead.
|
||||
|
||||
## 5. The course
|
||||
|
||||
**Adopt Clearwing as the substrate; contribute the web/logic detection path it lacks; delete our noise
|
||||
architecture entirely.**
|
||||
|
||||
Concretely, superseding both prior specs:
|
||||
|
||||
1. **Delete, do not fix.** The K-run machinery, the per-family noise floor, the regression budget, the sign
|
||||
test, the evolve loop, the acceptance rule, the mechanism-lever optimizer. All of it was a response to
|
||||
having no ground truth. `learning-harness` Req 3.5, 8.2, 9.5 and the whole of the evolve tasks are
|
||||
retired, not amended.
|
||||
|
||||
2. **Keep and port** the three assets that stand on their own: the closed family **taxonomy**, the
|
||||
IR-driven **candidate enumerator** (96.6% ceiling), and the **obligation / absence-bug checker**. These
|
||||
become a Clearwing `logic_auth` + `web_framework` detection path that enumerates candidates
|
||||
deterministically and judges them once, rather than searching.
|
||||
|
||||
3. **Ground truth for the web/logic classes that cannot crash a sandbox.** This is the hard research
|
||||
problem and the real contribution. Options, to be decided in requirements:
|
||||
- a *differential* oracle — the vulnerable-parent vs fix-commit pair is the ground truth, exactly as our
|
||||
pair corpus intended, but the "reproduction" is a constructed request that the fixed version rejects and
|
||||
the vulnerable version serves (a sandboxed HTTP harness, the web analogue of `poc_runner`);
|
||||
- failing a runnable oracle, the fix-commit-as-blind-oracle discipline from Clearwing's FFmpeg protocol —
|
||||
run blind on the parent, score against the fix *after the fact*, never in context.
|
||||
|
||||
4. **Corpus is harvested, not hand-labeled.** Replace the pair-excerpt corpus with vulnerable-parent
|
||||
commits at repository scale (our existing recipes are a seed; the CVE-history and OSS-Fuzz harvesters are
|
||||
the model), and let the oracle — sandbox crash for memory, differential request for web — decide truth.
|
||||
|
||||
5. **Reporting keeps the one honesty discipline we got right:** per-slice numbers with the real-world slice
|
||||
deciding, the overfitting gap published, and the evidence level on every finding. Clearwing already emits
|
||||
an evidence ladder and SARIF; our per-slice + overfitting-gap discipline is additive.
|
||||
|
||||
## 6. Build vs. adopt — the honest recommendation
|
||||
|
||||
The instinct after a project like this is to rebuild. Resist it. This session logged **eighteen defects,
|
||||
every one ours**, three of them the same shape of fixing an instance and missing the class. A codebase we
|
||||
wrote from scratch to match Clearwing would carry that defect rate against a 19,000-line, MIT-licensed,
|
||||
battle-tested substrate that the maintainer *already forks* (`norandom/deepwing`).
|
||||
|
||||
**Recommendation:** do not rebuild. Adopt Clearwing (via the existing fork) as the execution substrate and
|
||||
contribute the web/logic detection path — taxonomy + IR candidate enumeration + obligation checker + the
|
||||
differential web oracle — as the piece it genuinely lacks. OpenUltraSAST's lasting output is then not a
|
||||
competing tool but the answer to Clearwing's open question: *how do you ground-truth the dangerous web bugs
|
||||
that never crash?*
|
||||
|
||||
The alternative — keep OpenUltraSAST standalone and bolt a sandbox onto it — means reimplementing
|
||||
Clearwing's sandbox lifecycle, container pooling, sanitizer images, PoC stability, campaign orchestration
|
||||
and disclosure workflow, which are the parts this session proved we get wrong. That is the expensive path
|
||||
and it competes where we are weakest.
|
||||
|
||||
## Open questions for the requirements phase
|
||||
|
||||
- **The differential web oracle is unproven.** Can a sandboxed request harness reliably distinguish a
|
||||
vulnerable handler from its fixed twin for the access-control / injection / SSRF classes, at corpus scale,
|
||||
without per-repo setup? If not, the fix-commit-as-blind-oracle is the fallback, and it is weaker.
|
||||
- **Adoption mechanics.** Contribute upstream to `Lazarus-AI/clearwing`, or maintain the web/logic path in
|
||||
the `norandom/deepwing` fork? The `Upstream.md` sync discipline already exists.
|
||||
- **What of the existing corpus survives** as seed for the harvest, given the twin/straddle/tier defects the
|
||||
learning-harness reviews surfaced?
|
||||
- **Cost.** Buttercup ran at ~$152/task; our web slice ran at $0.03–0.14 per correct pair. The differential
|
||||
oracle's sandbox cost is unmeasured and is the number that decides whether campaign-scale is affordable.
|
||||
|
||||
Sources: [Big Sleep / SQLite zero-day](https://thehackernews.com/2024/11/googles-ai-tool-big-sleep-finds-zero.html) ·
|
||||
[AIxCC finals — Buttercup, 28 vulns / 20 CWEs / non-reasoning LLMs](https://blog.trailofbits.com/2025/08/09/trail-of-bits-buttercup-wins-2nd-place-in-aixcc-challenge/) ·
|
||||
[AIxCC results — 18 real 0-days, 43/54 patched](https://semiengineering.com/aixcc-2025-what-it-means-for-device-security/) ·
|
||||
[Clearwing](https://github.com/Lazarus-AI/clearwing) · local: `/home/mc/Source/clearwing`, fork `norandom/deepwing`.
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"feature_name": "execution-grounded-detection",
|
||||
"created_at": "2026-09-08T00:00:00Z",
|
||||
"updated_at": "2026-09-08T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "brief",
|
||||
"approvals": {
|
||||
"requirements": { "generated": false, "approved": false },
|
||||
"design": { "generated": false, "approved": false },
|
||||
"tasks": { "generated": false, "approved": false }
|
||||
},
|
||||
"ready_for_implementation": false,
|
||||
"supersedes": ["learning-harness", "constrained-detector"],
|
||||
"depends_on": [],
|
||||
"approval_note": "2026-09-08 corrective brief drafted at the maintainer's direction ('can you produce a corrective spec that supersedes the project and sets a course'). Written after the learning-harness evolve rounds produced zero decisions distinguishable from noise, and after researching Clearwing (/home/mc/Source/clearwing, fork norandom/deepwing), Google Big Sleep, and the DARPA AIxCC 2025 finals. Not approved; requirements not generated. The central recommendation is to adopt Clearwing as the execution substrate rather than rebuild, and to contribute the web/logic detection path (taxonomy + IR candidate enumeration + obligation checker + a differential web oracle) that Clearwing's crash-oracle model does not reach."
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "learning-harness",
|
||||
"created_at": "2026-09-06T06:32:41Z",
|
||||
"updated_at": "2026-09-06T13:00:00Z",
|
||||
"updated_at": "2026-09-08T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -26,5 +26,6 @@
|
||||
"tree-sitter-overlay-extra",
|
||||
"harnessx-self-improving-rulesets"
|
||||
],
|
||||
"approval_note": "2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase'); 2026-09-06 design approved by the maintainer in session ('approve the design and generate the tasks'); 2026-09-06 tasks approved by the maintainer in session ('approve the tasks and go on with the implementation'); 2026-09-06 requirements amended by the maintainer in session ('amend the requirements to untrusted_destination and note the flat taxonomy'): Req 1.1 names untrusted destination instead of server-side request forgery, and Req 2.7 and 3.7 require every agreement and family figure to state that hierarchical partial credit could not apply while the taxonomy is flat; 2026-09-06 Req 6.1 amended by the maintainer in session ('drop the skills set from the requirement'): a detector configuration has no skills set, because the skill router belongs to the tool's earlier shape and a family's prompt, checklist, tools and counterexamples already carry what a round evolves; 2026-09-06 design amended during task 4.1 (recorded in design.md): a recipe may name cross-file registration documents, because connexion registers by operationId in an OpenAPI file and express in a router module, so Req 10.2 is unreachable from a single-file excerpt for those rows; 2026-09-06 task 5.3 amended by the maintainer in session (asked and answered): the first evolve rounds target `injection` rather than `access_control`, because under the split and review-tier rules access_control has one train teacher and four holdout pairs and teaches zero obligation shapes, while injection has twelve teachers and ten holdout pairs"
|
||||
"approval_note": "2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase'); 2026-09-06 design approved by the maintainer in session ('approve the design and generate the tasks'); 2026-09-06 tasks approved by the maintainer in session ('approve the tasks and go on with the implementation'); 2026-09-06 requirements amended by the maintainer in session ('amend the requirements to untrusted_destination and note the flat taxonomy'): Req 1.1 names untrusted destination instead of server-side request forgery, and Req 2.7 and 3.7 require every agreement and family figure to state that hierarchical partial credit could not apply while the taxonomy is flat; 2026-09-06 Req 6.1 amended by the maintainer in session ('drop the skills set from the requirement'): a detector configuration has no skills set, because the skill router belongs to the tool's earlier shape and a family's prompt, checklist, tools and counterexamples already carry what a round evolves; 2026-09-06 design amended during task 4.1 (recorded in design.md): a recipe may name cross-file registration documents, because connexion registers by operationId in an OpenAPI file and express in a router module, so Req 10.2 is unreachable from a single-file excerpt for those rows; 2026-09-06 task 5.3 amended by the maintainer in session (asked and answered): the first evolve rounds target `injection` rather than `access_control`, because under the split and review-tier rules access_control has one train teacher and four holdout pairs and teaches zero obligation shapes, while injection has twelve teachers and ten holdout pairs; 2026-09-08 SUPERSEDED by execution-grounded-detection: this session's measurements showed the noise architecture (K-runs, per-family floor, regression budget, sign test, evolve loop) is a statistical treatment for a structural problem, and that the field (Big Sleep, AIxCC finals, Clearwing) grounds detection in execution rather than in scoring against a labeled corpus. The taxonomy, the IR candidate enumerator (96.6% ceiling) and the obligation checker survive as the web/logic detection path to contribute to a Clearwing-based substrate.",
|
||||
"superseded_by": "execution-grounded-detection"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user