**Amendments approved and folded into the numbered requirements** rather
than left as a proposal appended to the file:
- Req 5.5 — a report proportionate to the evidence behind a finding. A
rule whose only evidence is a text match is summarised past a stated
number of sites; a finding a proposer reasoned about keeps its detail
whatever its rung; nothing leaves findings.json, which is the record a
baseline is compared against.
- Req 5.6 — a family that cannot decide a class states the limit, and no
scan presents an absence of findings as evidence of absence.
- Req 4.6 — at least one pinned checkout carries a vulnerability a family
can decide in principle, so a scan producing nothing is distinguishable
from an engine that cannot see.
Each carries the evidence that produced it, and the amendment record keeps
what group 2 confirmed rather than changed: Req 8's overfitting warning
earned its place twice, Req 3.3's honesty counters made an invisible
failure visible, and Req 11.2's gates held through all nine defect fixes.
Req 5.5 is half implemented. Task 2.11 made pattern matches proportionate;
task 2.17 now carries the model layer's own suspicion band, which is 38 of
vampi's 60 findings and is an enumerated candidate the graph could NOT
decide with a judge's opinion attached -- a pattern match with an opinion,
not a reasoned claim, and weaker evidence than the obligations checker's
structural output.
**finding-feedback-loop requirements generated**, seven of them: a
dismissal file the developer owns and the tool never writes, rejecting one
without a reason because a suppression list without reasons is a graveyard
nobody can audit; a counter-example ledger whose entries are runnable
regression cases, grouped by shared cause so a recurring bug is visible as
one; a model that proposes a cause into a review queue and decides nothing,
validated by reverting the three known matcher fixes and measuring what it
recovers; dismissals plus a recipe's known vulnerabilities as one split
evaluation set with the holdout unread and asserted; evolution scoped to
the judge and residual questions with the fact tables off limits to any
automated means; an adoption that reports what it traded and reverts itself
when net-negative; and the loop inheriting the corpus blind spot, so its
evidence base is repositories and it states how few it has.
Phase: requirements, generated, not approved.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Following the Kiro workflow rather than continuing to write tasks.
**A workflow error, corrected.** Two task groups -- a counter-example
ledger, and per-repository evolutionary prompting so developers can commit
dismissals and evolve the harness -- were appended to contributor-scan's
tasks.md on 2026-09-09 without passing requirements or design. They are not
refinements of scan integration; they are a different feature, and writing
them as tasks skipped the two phases that decide whether a feature is
shaped right. They are lifted to .kiro/specs/finding-feedback-loop/, at
brief, unapproved.
That brief states the boundary the design turns on: an optimiser may evolve
the QUESTION (the judge and residual prompts, which are how we ask) and
never the FACTS (tables, rulesets, the declared policy file, which are what
we claim). The closed-loop leak measured on 2026-09-05 is why -- five of
eleven candidate shapes were taught by the holdout pairs they then
recovered, reporting a -0.059 holdout Youden as +0.059. It also states the
acceptance test for the model's role: revert the three matcher fixes and
see whether the grouping rediscovers the known causes.
**Three requirements amendments, drafted and unapproved**, at the end of
contributor-scan/requirements.md:
- Req 5 should state what a report may repeat. libpng emitted 198 findings
and nothing in the spec forbade it. Task 2.11 fixed pattern noise; the
model layer's own suspicion band is still 38 of vampi's 60, and it is the
same epistemic class -- an enumerated candidate the graph could not
decide and an LLM affirmed.
- A family that cannot decide a class must say so. c/memory scored zero
true positives on 89k lines of C for a structural reason, and a silent
scan reads as a clean bill of health. Worse than noise because it is
quiet, and the one failure mode the rung ladder does not already guard.
- Req 4.1 is satisfied by a checkout that cannot demonstrate detection at
all -- libpng's CVE is out of family and no version of it can be
otherwise. It should require one that can.
And what group 2 confirmed rather than changed: Req 8's overfitting warning
stopped a rule being extended until one repository was clean and forced the
access-control work onto pairs nobody tuned against; Req 3.3's honesty
counters are what made a budget stopping free work visible at all; and Req
11.2's byte-identical gates held through all nine defect fixes, which says
from the other side that the corpus could not have caught any of them.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Pair scoring could rely on a labelled function; a contributor cannot supply one
-- which code is risky is the question they are asking. Regions now come from
entry points, with a file-level fallback that says it is one, and each carries
only the families its shape and language admit. Ranking uses what
EntryPointRecord already decided (access_level, trust_boundary) rather than a
second opinion computed from the same evidence.
A family is offered only where its arbiter could answer: access_control needs a
handler, so a library file with no entry point is not asked about obligations it
has no siblings to be inconsistent with.
The driver makes the two costs that were free at pair scale explicit. ONE CPG
per repository, reused across regions -- a build is 5-30s on a forty-line
excerpt and per-region builds would make this unusable. And a budget that is a
TOTAL for the run rather than the enumerator's per-region cap of eight, spent
highest-rank-first, with whatever it does not reach counted into
regions_unjudged. A tool that silently analyses a tenth of a repository and
reports a clean result is worse than one that says what it skipped.
Two flaws the tests caught, both mine and both about ownership: the budget
counted calls by reading a client's own usage log, so it silently never tripped
for a client that keeps none -- the driver now counts its own spend through a
wrapper; and the driver trusted its caller to have sorted, when the order the
budget is spent in belongs to whoever holds the budget.
The third was worse and only surfaced because a test failed for the right
reason: the driver passed candidates=() and never enumerated, so a repository
scan would have reported only what the arbiters establish unaided and never
populated the suspicion band -- which supplied 15 of 26 findings in the pair
measurement. It enumerates now.
Sibling audit (Req 8.5) done before writing: family routing lives in exactly one
production place, pipeline._arbitrate, dispatching on spec type. regions_for
returns family names and the driver resolves specs, so that stays true.
750 passing, mypy clean over 96 files, gates byte-identical, zero orphans.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Requirements approved, design and tasks generated. 47 criteria, all traced;
13 tasks in 6 groups.
The design is deliberately conservative. The model layer joins as ONE STAGE
INSIDE MAP, beside the obligations checker it most resembles -- which already
takes entry points, produces findings and merges them. Findings are added, never
substituted; the pattern and overlay paths keep working; a machine without Joern
scans exactly as it does today. Nothing here is allowed to make the tool worse
for someone who does not have the engine.
Two decisions carry the most weight. One CPG per scan rather than per region: a
build is 5-30s on an excerpt and is the dominant cost at repository scale, so
per-region builds would make this unusable. And one total budget rather than the
enumerator's per-region cap of eight, which is meaningless across thousands of
regions; the budget is spent highest-rank-first and whatever it does not reach is
counted and reported, never silently truncated.
Task 2.2 is a go/no-go and comes before shipping: the first scan of a pinned
known-vulnerable checkout is a measurement nobody has taken, and if one CPG per
repository proves infeasible the region and budget design changes shape. Better
to learn that before groups 3-6 are built on it.
Group 5 (corpus) is last on purpose. The binding constraint today is
reachability, not corpus size -- 198 of 238 OWASP pairs went unvendored because
volume without readability makes every number worse.
The task preamble carries the predecessor's two hard-won habits: audit sibling
paths in the same commit as any routing or matching change (five bugs shared
that shape, each caught by a measurement rather than a check), and run the whole
CI list rather than the parts one remembers (CI was red for a session because
`ruff check` passing was taken as evidence for `ruff format --check`).
Not approved for implementation.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
10 requirements, 46 criteria. A NEW spec rather than an amendment to
model-grounded-detection, which is complete and closed.
Req 1-8 cover the gap that prompted it: the model layer has zero production
callers, so ousast scan must use it (1), regions must come from entry points
rather than a labelled function (2), ranking and budget must work at repository
scale where the enumerator's per-region cap of eight is meaningless (3), the
performance envelope must be measured on a real checkout (4), output must be
actionable (5), packaging must be built and verified rather than merely defined
(6), the two deferred decisions must close (7), and the honesty disciplines
carry forward (8).
Req 9 and 10 were added after the maintainer asked whether their requests had
been captured. They had not been -- I folded the corpus asks implicitly into
"run against a known-vulnerable repository", which is not the same thing. Now
explicit: a PHP/WordPress slice (php2cpg works once an interpreter is present,
which the image provides, and plugin CVEs are overwhelmingly the families the
model arbitrates well); real-world C library CVEs -- libpng, image libraries,
OpenSSH, libpam, kernel modules -- gated on the memory decision, because until
it closes they would join 173 pairs already idle; more vibe-code pairs, since
that slice is where the stated audience works and it is only 35 pairs; and
regression tuning over a pinned large-repository baseline, which is the thing
pair scoring structurally cannot see.
Req 9.4 carries today's lesson: verify on a sample that the model can READ a
slice before vendoring it. The OWASP harvest found 238 pairs and vendored 40
for exactly that reason.
Not approved.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Asked what the spec state was and whether anything was forgotten. It was, and
it is the largest thing outstanding: model-grounded-detection is 16/16 tasks
complete and its arbiter measures 86.7% pair-correct at 6.7% leak, but the
model layer has ZERO production callers. `ousast scan` still runs
preprocess -> map -> overlay -> obligations -> regress -> report and touches
none of it. Every number quoted today came from a measurement harness in a
scratch directory.
That is a scope gap in the predecessor spec rather than a slip in executing it:
there was never a task saying "make this reachable", and its Req 11.2 (gates
stay byte-identical) is satisfied PRECISELY BECAUSE nothing is wired. The spec
is complete and the feature is unshipped, both at once.
A new spec rather than an amendment, because the predecessor is coherent and
closed -- it built an arbiter and measured it -- while this is a different
question: making it something a developer runs on a repository they are about
to contribute to.
What it covers, in the order it bites: scan integration; whole-repo scanning
(entry-point region selection, ranking across thousands of candidates, a budget
policy -- the enumerator caps at 8 per region, meaningless at repo scale -- and
a performance envelope never measured); running against a known-vulnerable
checkout, since the 176 real-world CVE pairs are single-function excerpts that
exercise none of it; the deferred memory decision, with 173 of those pairs idle
behind a stub tier; and retiring evidence_level.
It also records what was built outside any spec: the pipeline itself, the ops
work (one image, compose, shell wrappers, pyinfra -- with the Docker image
never actually built), and the corpus additions.
And it carries the known limits forward rather than losing them: the flat
permissive_values list that cannot tell debug=True from httponly=True,
interprocedural resolution as the most repeated miss cause, and the five bugs
that shared one shape -- a taint-specific assumption applied to every family,
each caught by a measurement rather than a check.
Not approved.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Marius Ciepluch