Open_Ultra_SAST/.kiro/specs/contributor-scan/brief.md

# Brief: contributor-scan, make the model layer a tool someone runs

**Status:** brief only, unapproved. Successor to `model-grounded-detection`, which is complete (16/16 tasks)
and is not reopened by this.

## The gap this exists to close

`model-grounded-detection` built and measured an arbiter. It works: 41.3% of the corpus reaches
`model_entailed` against the flat IR's 2.2%, and the wired pipeline scores 86.7% pair-correct at 6.7% leak
against the LLM hunter's 76.7% / 12.3%, at a fifth of the model calls and ~100× cheaper.

**And none of it is reachable by a user.** `ousast scan` still runs
`preprocess → map → overlay → obligations → regress → report` and touches the model layer nowhere;
`model/pipeline`, `judge`, `config_value`, `report`, `calibrate` and `execution` have zero production callers
outside `model/`. Every number quoted above came from a measurement harness in a scratch directory.

That is a scope gap in the predecessor spec, not an oversight in its execution: it had no task saying *make
this reachable*, and its Req 11.2 (the gates stay byte-identical) is satisfied precisely **because** nothing
is wired. The spec is complete and the feature is unshipped, both at once.

## What the goal actually is

> "a real world tool that helps developers when they want to contribute"

That reframes the work. A contributor does not have a labelled function and a known weakness class; they
have a repository they are about to touch and a question about what is risky in it. Everything measured so
far is function-level pairs: a labelled function, a declared family, a known target. None of it exercises the
questions a whole repository asks.

## What is missing, in the order it bites

1. **Scan integration.** Wire `model/pipeline` into `ousast scan` so a finding with a rung reaches a report.
   Without this nothing else matters.
2. **Whole-repo scanning.** Region selection from entry points instead of a labelled function; ranking across
   thousands of candidates rather than one; a budget policy (the enumerator caps at 8 candidates per region,
   which is meaningless at repo scale); and the performance envelope: a CPG build on a 100k-line tree, not a
   40-line excerpt. **This has never been run.**
3. **A known-vulnerable repository as the target.** The 176 real-world CVE pairs already in the corpus are
   single-function excerpts and do not exercise any of the above. Running against a real checkout (curl,
   libpng, a WordPress plugin) is the only way to find what breaks at scale.
4. **The memory decision.** 173 of those 176 pairs are `memory` family, routed to a deferred execution tier
   that is a stub. Either adopt Clearwing for them or scope C memory-safety out explicitly. Right now it is
   neither, and a sixth of the corpus sits idle.
5. **Retire `evidence_level`.** It duplicates `rung` in intent; noted as a follow-up in group 3 and never
   tracked.

## Already built and unrecorded

These were delivered during `model-grounded-detection` without being in any spec. They belong here so the
record is not scattered:

- The pipeline itself (`model/pipeline.py`): Req 8 existed, task 3.2 built `judge()`, and nothing ever
  said to connect an enumerator to it. Built, tested, measured, unspecified.
- Ops: a single image carrying tool and engine, a compose file with read-only target and tmpfs scratch,
  bash/zsh/PowerShell wrappers that make the container invisible, and a pyinfra deploy for host installs.
  The Docker path is written and syntax-validated but **the image has never been built**.
- Corpus: the `owasp` slice (40 validated xss pairs; `output_encoding` went from 1 pair to 41) and
  authored sink/sanitizer facts for Java and JavaScript.

## Known limits to carry forward

- `permissive_values` is a flat literal list with no keyword association, so it holds both `True` and `False`.
  `debug=True` is dangerous where `httponly=True` is safe and the model cannot tell them apart.
- Interprocedural resolution is the most repeated miss cause across every experiment: OWASP's helper classes,
  `angular-http-server`'s `resolveUrl`. Joern supports it; our queries may not be configured for it.
- Five bugs in `model-grounded-detection` shared one shape: a taint-specific assumption applied to every
  family, each found by a measurement rather than a check. Any new family-specific path should be audited for
  siblings before it is measured, not after.

## What the evidence says so far (appended 2026-09-09, after group 2)

This is a research project whose findings get taught, so a negative result is a deliverable. But the standing
test the work has to pass is blunter than that:

> "it must find the bugs, otherwise it's SAST noise. most devs know codeql / semgrep have dataflow
> capabilities but never use it"

That sentence settles what the differentiator is, and it is **not the dataflow engine**. CodeQL and Semgrep
already have one and developers do not turn it on. What they will not do is write queries, tune a ruleset, or
read two hundred rows to find the one that matters. So the product is the **rung** -- a claim the graph
decided, reported without configuration -- and everything below `model_entailed` is on trial.

### The scoreboard, honestly

| checkout | known in-scope | found | rung | false positives |
|---|---|---|---|---|
| vampi (python, 520 lines) | 3 | **3** | all `model_entailed` | 2, both endpoints the OpenAPI spec declares public |
| libpng (c, 89k lines) | 0 (its CVE is out of family) | — | — | 30 over the session, now 1 |

vampi is the result worth teaching: SQL injection across two modules, a broken object-level authorization,
and a second BOLA **the engine found before it was in the recipe**. libpng is the result worth being honest
about: 198 rows, zero of them valid, and the reason is structural rather than a tuning problem -- the library
never calls a sink the `c/memory` family models.

### What the session actually demonstrated

Nine defects, each found by a measurement and none by review. Three of them were one bug class in three
matchers (`resolve` in `resolveUrl`, `user` in `users`, `gets` in `fgets`); two were capability present but
unreachable (the model layer itself, then `query_batch`); two were a paid budget stopping unpaid work and a
question scoped to a function when the bug crosses modules.

The teachable finding is not any one of them. It is that **a corpus of function-level pairs cannot see any of
them**, and every one appeared within minutes of pointing the tool at a real repository. The gates stayed
byte-identical through all nine fixes, which is the same statement from the other side.

### What this changes about priorities

* **2.11 (the suspicion band) is the critical path**, not a cleanup task. libpng produced 163 rows that
  assert an API is present rather than misused. That is precisely the noise the quote above is about, and no
  amount of arbiter accuracy survives shipping it.
* **A family that cannot decide its class must say so** (2.16). A silent C scan must not read as a clean bill
  of health; that is worse than noise because it is quiet.
* **Negative results get recorded as measurements, not as failures.** `c/memory` scoring zero on libpng is a
  finding about the abstraction, and the artifact says which abstraction would reach it (size arithmetic,
  i.e. a constraint problem) rather than leaving it as a gap to be tuned at.

### The final objective, and what it needs

> "that + an LLM way of diving in without being forced to rethink the code"

Two halves, and they are complementary. The **rung** says *this was decided*; the second half says the
developer must not have to re-derive *why*. The material for that already exists and is called the
**witness**: `username -> db.session.execute(text(user_query)) in models/user_model.py (line 73, 8 steps)` is
the reasoning, already written down, already attached to the finding.

So the objective is largely a plumbing question, and the session's recurring defect shape applies to it.
Checked rather than assumed: `openultrasast.get_finding` returns the whole record including the witness, but
`openultrasast.findings` was dropping the rung -- so an LLM listing libpng's 198 rows could not tell the one
the graph decided from the 197 an enumerator proposed without fetching all 198 individually. Fixed: the list
carries the rung, leads with what was decided, and returns a per-rung tally.

What still stands between here and the objective:

* **The witness is one line.** It names the source, the sink, the file and the step count, but not the steps.
  For "dive in without rethinking", the path elements are the thing worth having, and the query already walks
  them (`flow.elements`) before discarding all but the first.
* **The suspicion band drowns the signal** (2.11). An LLM given 163 "this API is present" rows will reason
  about them, at length, and that is worse than a human ignoring them.
* **A family that cannot decide must say so** (2.16), or the LLM will confidently explain a silence.