Open_Ultra_SAST/.kiro/specs/contributor-scan/tasks.md

Implementation Plan: contributor-scan

Guidance for the implementing agent

• The predecessor's lesson, in one line. Five bugs in model-grounded-detection shared one shape: a taint-specific assumption applied to every family, each caught by a measurement rather than a check. Before you finish any change to a routing rule, a matching rule, or a per-family question, grep for its siblings and fix them in the same commit. The most expensive instance cost a 19-minute run that returned an identical number, and only the identity revealed the bug.
• Run the whole CI list before pushing, not the parts you remember: ruff format --check ., ruff check ., mypy src/openultrasast, pytest, the three gates, compileall src tests, and pytest -m semantic. CI was red for most of a session because ruff check passing was taken as evidence that ruff format --check would.
• The gate baseline is the safety net. benchmarks/measurements/2026-09-06-gate-baseline.txt. Run all three gates before and after every task. The model layer is additive: if a gate moves, you changed something that was never meant to change.
• RED first, one commit per task, selective git add.
• Zero-dependency core. Copy the capability-probe pattern in semantic/extra.py and cpg/capability.py.
• Do not modify model/pipeline, taint, dominance, config_value, specs or candidates. They are built and measured. If a change looks necessary, it belongs in its own commit with its own measurement.

Group 1 — Wire it (gate-protected)

• 1.1 Regions from entry points

  • model/regions.py: ScanRegion(path, function, families, rank, source) and regions_for(entries, targets, *, taxonomy, language_of). Rank from what EntryPointRecord already carries (access_level, trust_boundary) — do not invent a new signal. File-level fallback when a target has no entry point, with source="file_fallback". Families from region shape and language, via the existing taint_specs/dominance_specs/config_specs.
  • Sibling audit (Req 8.5): family routing now exists in three places — ceiling.py's harness, pipeline._arbitrate, and here. Check all three agree before finishing.
  • Observable: a region is produced per entry point; a target with no entry point yields a file region marked as such; no region carries a family whose spec is absent for its language.
  • Requirements: 2.1, 2.2, 2.3, 2.4, 8.5

• 1.2 The repository driver

  • model/scan.py: ScanBudget, ModelScanResult, scan_repository(...). One CPG per run, reused across regions. Regions in rank order; budget decremented per model call and checked before each; the remainder counted into regions_unjudged. Findings ordered by rung, then rank.
  • Observable: one backend.build call for many regions; a budget of N yields at most N model calls and a non-zero regions_unjudged; ordering puts model_entailed above suspicion; a failed CPG build yields an empty result with cpg_build_failed, never an exception.
  • Redaction (Req 8.3) is inherited from model/pipeline, which redacts every prompt; a test asserts the driver adds no unredacted path of its own.
  • Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 8.3
  • Depends: 1.1

• 1.3 The MAP stage

  • Wire scan_repository into cli.py inside Stage.MAP, mirroring _check_obligations: takes entry_points and targets, returns findings merged additively beside the pattern, overlay and obligation findings. A [model] config block (enabled, budget, max regions), zero-dep, conservative defaults. Payload into the manifest with by_rung, cost and elapsed.
  • Observable: a scan on a repo with Joern present reports findings carrying rungs; the same scan with OPENULTRASAST_JOERN_PROBE=off is identical to today's output plus a cpg_unavailable degradation; the three gates are byte-identical.
  • The [model] block is capability-detected like every other optional plane (Req 8.4): core stays dependencies = [], and the engine, endpoint and execution tier each degrade to a recorded reason.
  • Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 8.4
  • Depends: 1.2

• 1.4 Batch the queries — one Joern invocation per scan, not one per region and family

  • Why this is in group 1 rather than group 2. Wiring 1.3 exposed that the cost model in the design was half right. One CPG per scan was the right call, but every query is its own joern --script JVM launch of roughly 30 seconds, and the driver issues one per region per family. A single ten-line Python file admits six families and therefore takes six launches and over three minutes. At 100 regions that is ~5 hours; at 1000, ~50. Task 2.2's go/no-go would not have been a measurement, only arithmetic.
  • Extend cpg/backend.py with a batched call: one invocation carrying many (region, family, spec) requests and returning rows keyed back to each. The .sc scripts already take parameter lists; they take a list of requests instead. model/scan.py collects the work first, issues one call, then arbitrates from the rows.
  • Keep the seam. The subprocess boundary stays the only Joern touch-point (predecessor Req 4.5). Server mode would keep a JVM warm but adds a process model and weakens that promise; batching needs neither.
  • The arbiters do not change. taint, dominance, config_value and pipeline take rows and are already measured; if this task needs to edit one, that is a signal the batching is leaking into them.
  • Observable: a scan over N regions issues one joern --script invocation, not N×families; the same fixture produces the same findings and rungs as the unbatched path, asserted against a recorded baseline; the ten-line scan drops from minutes to seconds.
  • Requirements: 3.5, 4.2, 4.3
  • Depends: 1.3

Group 2 — Scale it (the phase that can reshape the rest)

• 2.1 A pinned known-vulnerable checkout

  • benchmarks/repos/<name>.toml: url, commit, the known CVE, the file and function it lives in, licence. Start with one small enough to iterate on. Fetch is explicit and offline-by-default, like the pointer pairs.
  • Observable: the recipe resolves to a checkout; CI never fetches it.
  • Requirements: 4.1
  • Depends: 1.3

• 2.2 The first repository measurement

  • Run the full scan against that checkout and commit the artifact: CPG build time, scan wall-clock, peak memory, findings by rung, cost, regions_unjudged, and whether the known CVE was found and at which rung. If the repository cannot be processed in its envelope, record the stage it failed at.
  • This is the go/no-go for the shape of the rest: if one CPG per repository is infeasible, the region and budget design changes and phases 3–6 should not be built on it first.
  • Observable: the artifact exists with every field; the reading states go or no-go and why.
  • Requirements: 4.2, 4.3, 4.4, 4.5
  • Depends: 2.1

Group 3 — Ship it

• 3.1 Contributor output

  • model/report.py: a scan summary — counts per rung, the top findings with witness and path:line, and a plain statement of what suspicion means (the model could not confirm it). Findings to the writable output only.
  • Observable: a scan produces a summary readable without opening JSON; every finding shows rung, witness and location; nothing is written into the analysed tree.
  • Keeps the disciplines already built into model/report.py (Req 8.1, 8.2): per-slice rows with the real-world slice deciding, a refused bare aggregate, and the overfitting gap published beside any headline — absent rather than zero when a slice is missing.
  • Requirements: 5.1, 5.2, 5.3, 5.4, 8.1, 8.2
  • Depends: 1.3

• 3.2 Build and verify the image

  • docker compose build, then run a real scan through it and through the shell wrapper. The Dockerfile and compose exist and are syntax-valid; the image has never been built, so nothing about it is proven.
  • Observable: the image builds; TARGET=<repo> docker compose run --rm ousast scan /target produces the same findings as a native run; the target mount is read-only; the network is off by default.
  • Requirements: 6.1, 6.2, 6.3, 6.4, 6.5
  • Depends: 3.1

Group 4 — Close the deferred decisions

• 4.1 The memory-family decision

  • Either reach memory through an adopted execution tier, or record explicitly that C memory-safety is out of scope. 173 of the 176 real-world CVE pairs sit idle behind a stub; the decision gates Req 9.2.
  • Observable: the decision and its reason are recorded where a later reader finds them; if out of scope, the corpus says so per slice.
  • Requirements: 7.1, 7.3

• 4.2 Retire evidence_level

  • It duplicates rung in intent. Retire it, or record why both are kept. Note the legacy value static_corroboration comes from the flat-IR overlay, which Req 4.4 of the predecessor demotes to the suspicion band — so a straight rename would overstate it.
  • Observable: one vocabulary, or a recorded reason for two; SARIF and JSON consumers considered.
  • Requirements: 7.2, 7.3

Group 5 — Grow the corpus

• 5.1 PHP/WordPress slice

  • Plugin and core weaknesses. php2cpg needs a PHP interpreter, which the image provides. Validate on a sample that the model can read the pairs before vendoring — the OWASP harvest found 238 and vendored 40 for exactly this reason.
  • Observable: a sample measurement precedes vendoring; only categories the model can arbitrate are vendored; the slice reports as its own row; every pair states a licence.
  • Requirements: 9.1, 9.4, 9.5, 9.6, 9.7
  • Depends: 3.2

• 5.2 More vibe-code pairs

  • The slice where the stated audience works, currently 35 pairs. Same read-check discipline.
  • Requirements: 9.3, 9.4, 9.5, 9.6, 9.7

• 5.3 Real-world C library CVEs

  • libpng and the image libraries, OpenSSH, libpam, kernel modules. Gated on 4.1: until the memory decision closes these would join 173 idle pairs.
  • Requirements: 9.2, 9.4, 9.5, 9.6
  • Depends: 4.1

Group 6 — Regression baselines

• 6.1 Repository baseline and delta
  • A committed baseline per pinned checkout; a delta report on fact-table or query changes showing findings gained, lost and rung movements. An unexplained loss of a previously-entailed finding is a regression to investigate, not a number to accept. The baseline records the engine version, because a CPG engine change moves every verdict.
  • Requirements: 10.1, 10.2, 10.3, 10.4
  • Depends: 2.2

Task-plan review notes

• Group 1 must land and pass review before group 2: measuring an unwired pipeline is what produced a session's worth of harness-only numbers.
• Task 1.4 was added after 1.3 was wired, when a ten-line scan took three minutes. It belongs in group 1 because group 1's promise is a working scan, and a scan that cannot finish on a real repository is not one. Finding it here rather than in 2.2 is the argument for wiring before measuring.
• Task 2.2 is a go/no-go. If one CPG per repository is infeasible, stop and redesign the region/budget model rather than building groups 3–6 on an assumption.
• Group 5 is deliberately last. The corpus is not the binding constraint today — reachability is — and 198 of 238 OWASP pairs were left unvendored precisely because volume without readability makes every number worse.