Open_Ultra_SAST/.kiro/specs/contributor-scan/tasks.md

Implementation Plan: contributor-scan

Guidance for the implementing agent

• The predecessor's lesson, in one line. Five bugs in model-grounded-detection shared one shape: a taint-specific assumption applied to every family, each caught by a measurement rather than a check. Before you finish any change to a routing rule, a matching rule, or a per-family question, grep for its siblings and fix them in the same commit. The most expensive instance cost a 19-minute run that returned an identical number, and only the identity revealed the bug.
• Run the whole CI list before pushing, not the parts you remember: ruff format --check ., ruff check ., mypy src/openultrasast, pytest, the three gates, compileall src tests, and pytest -m semantic. CI was red for most of a session because ruff check passing was taken as evidence that ruff format --check would.
• The gate baseline is the safety net. benchmarks/measurements/2026-09-06-gate-baseline.txt. Run all three gates before and after every task. The model layer is additive: if a gate moves, you changed something that was never meant to change.
• RED first, one commit per task, selective git add.
• Zero-dependency core. Copy the capability-probe pattern in semantic/extra.py and cpg/capability.py.
• Do not modify model/pipeline, taint, dominance, config_value, specs or candidates. They are built and measured. If a change looks necessary, it belongs in its own commit with its own measurement.

Group 1 — Wire it (gate-protected)

• 1.1 Regions from entry points

  • model/regions.py: ScanRegion(path, function, families, rank, source) and regions_for(entries, targets, *, taxonomy, language_of). Rank from what EntryPointRecord already carries (access_level, trust_boundary) — do not invent a new signal. File-level fallback when a target has no entry point, with source="file_fallback". Families from region shape and language, via the existing taint_specs/dominance_specs/config_specs.
  • Sibling audit (Req 8.5): family routing now exists in three places — ceiling.py's harness, pipeline._arbitrate, and here. Check all three agree before finishing.
  • Observable: a region is produced per entry point; a target with no entry point yields a file region marked as such; no region carries a family whose spec is absent for its language.
  • Requirements: 2.1, 2.2, 2.3, 2.4, 8.5

• 1.2 The repository driver

  • model/scan.py: ScanBudget, ModelScanResult, scan_repository(...). One CPG per run, reused across regions. Regions in rank order; budget decremented per model call and checked before each; the remainder counted into regions_unjudged. Findings ordered by rung, then rank.
  • Observable: one backend.build call for many regions; a budget of N yields at most N model calls and a non-zero regions_unjudged; ordering puts model_entailed above suspicion; a failed CPG build yields an empty result with cpg_build_failed, never an exception.
  • Redaction (Req 8.3) is inherited from model/pipeline, which redacts every prompt; a test asserts the driver adds no unredacted path of its own.
  • Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 8.3
  • Depends: 1.1

• 1.3 The MAP stage

  • Wire scan_repository into cli.py inside Stage.MAP, mirroring _check_obligations: takes entry_points and targets, returns findings merged additively beside the pattern, overlay and obligation findings. A [model] config block (enabled, budget, max regions), zero-dep, conservative defaults. Payload into the manifest with by_rung, cost and elapsed.
  • Observable: a scan on a repo with Joern present reports findings carrying rungs; the same scan with OPENULTRASAST_JOERN_PROBE=off is identical to today's output plus a cpg_unavailable degradation; the three gates are byte-identical.
  • The [model] block is capability-detected like every other optional plane (Req 8.4): core stays dependencies = [], and the engine, endpoint and execution tier each degrade to a recorded reason.
  • Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 8.4
  • Depends: 1.2

Group 2 — Scale it (the phase that can reshape the rest)

• 2.1 A pinned known-vulnerable checkout

  • benchmarks/repos/<name>.toml: url, commit, the known CVE, the file and function it lives in, licence. Start with one small enough to iterate on. Fetch is explicit and offline-by-default, like the pointer pairs.
  • Observable: the recipe resolves to a checkout; CI never fetches it.
  • Requirements: 4.1
  • Depends: 1.3

• 2.2 The first repository measurement

  • Run the full scan against that checkout and commit the artifact: CPG build time, scan wall-clock, peak memory, findings by rung, cost, regions_unjudged, and whether the known CVE was found and at which rung. If the repository cannot be processed in its envelope, record the stage it failed at.
  • This is the go/no-go for the shape of the rest: if one CPG per repository is infeasible, the region and budget design changes and phases 3–6 should not be built on it first.
  • Observable: the artifact exists with every field; the reading states go or no-go and why.
  • Requirements: 4.2, 4.3, 4.4, 4.5
  • Depends: 2.1

Group 3 — Ship it

• 3.1 Contributor output

  • model/report.py: a scan summary — counts per rung, the top findings with witness and path:line, and a plain statement of what suspicion means (the model could not confirm it). Findings to the writable output only.
  • Observable: a scan produces a summary readable without opening JSON; every finding shows rung, witness and location; nothing is written into the analysed tree.
  • Keeps the disciplines already built into model/report.py (Req 8.1, 8.2): per-slice rows with the real-world slice deciding, a refused bare aggregate, and the overfitting gap published beside any headline — absent rather than zero when a slice is missing.
  • Requirements: 5.1, 5.2, 5.3, 5.4, 8.1, 8.2
  • Depends: 1.3

• 3.2 Build and verify the image

  • docker compose build, then run a real scan through it and through the shell wrapper. The Dockerfile and compose exist and are syntax-valid; the image has never been built, so nothing about it is proven.
  • Observable: the image builds; TARGET=<repo> docker compose run --rm ousast scan /target produces the same findings as a native run; the target mount is read-only; the network is off by default.
  • Requirements: 6.1, 6.2, 6.3, 6.4, 6.5
  • Depends: 3.1

Group 4 — Close the deferred decisions

• 4.1 The memory-family decision

  • Either reach memory through an adopted execution tier, or record explicitly that C memory-safety is out of scope. 173 of the 176 real-world CVE pairs sit idle behind a stub; the decision gates Req 9.2.
  • Observable: the decision and its reason are recorded where a later reader finds them; if out of scope, the corpus says so per slice.
  • Requirements: 7.1, 7.3

• 4.2 Retire evidence_level

  • It duplicates rung in intent. Retire it, or record why both are kept. Note the legacy value static_corroboration comes from the flat-IR overlay, which Req 4.4 of the predecessor demotes to the suspicion band — so a straight rename would overstate it.
  • Observable: one vocabulary, or a recorded reason for two; SARIF and JSON consumers considered.
  • Requirements: 7.2, 7.3

Group 5 — Grow the corpus

• 5.1 PHP/WordPress slice

  • Plugin and core weaknesses. php2cpg needs a PHP interpreter, which the image provides. Validate on a sample that the model can read the pairs before vendoring — the OWASP harvest found 238 and vendored 40 for exactly this reason.
  • Observable: a sample measurement precedes vendoring; only categories the model can arbitrate are vendored; the slice reports as its own row; every pair states a licence.
  • Requirements: 9.1, 9.4, 9.5, 9.6, 9.7
  • Depends: 3.2

• 5.2 More vibe-code pairs

  • The slice where the stated audience works, currently 35 pairs. Same read-check discipline.
  • Requirements: 9.3, 9.4, 9.5, 9.6, 9.7

• 5.3 Real-world C library CVEs

  • libpng and the image libraries, OpenSSH, libpam, kernel modules. Gated on 4.1: until the memory decision closes these would join 173 idle pairs.
  • Requirements: 9.2, 9.4, 9.5, 9.6
  • Depends: 4.1

Group 6 — Regression baselines

• 6.1 Repository baseline and delta
  • A committed baseline per pinned checkout; a delta report on fact-table or query changes showing findings gained, lost and rung movements. An unexplained loss of a previously-entailed finding is a regression to investigate, not a number to accept. The baseline records the engine version, because a CPG engine change moves every verdict.
  • Requirements: 10.1, 10.2, 10.3, 10.4
  • Depends: 2.2

Task-plan review notes

• Group 1 must land and pass review before group 2: measuring an unwired pipeline is what produced a session's worth of harness-only numbers.
• Task 2.2 is a go/no-go. If one CPG per repository is infeasible, stop and redesign the region/budget model rather than building groups 3–6 on an assumption.
• Group 5 is deliberately last. The corpus is not the binding constraint today — reachability is — and 198 of 238 OWASP pairs were left unvendored precisely because volume without readability makes every number worse.