docs: plain-English pass over the active specs

Ran the humanizer rules over 79 prose files in the active specs (brief,
design, requirements, research and narrative docs). Removed em dashes,
significance inflation, forced rule-of-three, bold-label list items and
filler. Skipped tasks.md checklists and the retired specs (harnessx-*,
openrouter-sast-harness, unseen-repo-evaluation).

Every number, path, command, hash and identifier was preserved: checked
each file against HEAD for drift in numbers, inline code and code fences,
zero drift across all 79 files.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-12 18:16:41 +02:00
co-authored by Claude Opus 4.8
parent b08f372022
commit 39a1873e5b
79 changed files with 1103 additions and 1103 deletions
+8 -8
View File
@@ -1,4 +1,4 @@
# Brief: finding-feedback-loop — what happens after a finding is judged wrong
# Brief: finding-feedback-loop, what happens after a finding is judged wrong
**Status:** brief only, unapproved. Successor scope lifted out of `contributor-scan`, which is mid-implementation
and does not cover this.
@@ -7,13 +7,13 @@ and does not cover this.
Two task groups were drafted into `contributor-scan/tasks.md` on 2026-09-09 without passing through
requirements or design. That was a workflow error: they are not refinements of scan integration, they are a
different feature — a loop that learns from findings a human rejected — and writing them as tasks skipped the
different feature (a loop that learns from findings a human rejected) and writing them as tasks skipped the
two phases that decide whether the feature is shaped right. They are lifted here to be specified properly.
## The gap this exists to close
Nine engine defects were found in one session against two repositories. **Three of them were the same bug in
three different matchers**: a spec token matched as a substring rather than as a word — `resolve` inside
three different matchers**: a spec token matched as a substring rather than as a word: `resolve` inside
`resolveUrl` in the sanitizer matcher, `user` inside `users` in the discharger matcher, `gets` inside `fgets`
in the sink matcher. Each was found by a measurement, fixed locally, and left no trace. The third survived
until a 89k-line C library made it visible as 26 identical false positives.
@@ -60,17 +60,17 @@ sound. The same machinery writing sink tables is the thing that produced the lea
## What the shape looks like
* **Per-repository, declarative, committed by the developer.** A dismissal file in the *target* repository —
site, rung, reason, author — read by the tool and never written by it, the same rule the declared policy
* **Per-repository, declarative, committed by the developer.** A dismissal file in the *target* repository (
site, rung, reason, author), read by the tool and never written by it, the same rule the declared policy
file already follows. Reviewed like code, diffable, and a dismissal without a reason is how a suppression
list becomes a graveyard nobody can audit.
* **Dismissals are labels, not suppressions.** With the known vulnerabilities a `benchmarks/repos/` recipe
already declares, a dismissal file gives a per-repository evaluation set with *both* classes in it — which
already declares, a dismissal file gives a per-repository evaluation set with *both* classes in it, which
is what an optimiser needs and what a suppression list normally throws away.
* **A ledger for the maintainer.** Counter-examples grouped by what they share (same token, same matcher
clause, same witness shape), so a recurring cause is visible as a cause rather than as five separate bugs.
* **Split-respecting adoption.** Train split only, holdout never read, the three gates byte-identical, and
precision *and* recall reported on both sides — an adoption that quietly trades recall is worse than none.
precision *and* recall reported on both sides; an adoption that quietly trades recall is worse than none.
## The acceptance test for the model's role
@@ -80,7 +80,7 @@ recover a cause we already know should not be trusted to propose one we do not.
## Known limits to carry forward
* The pair corpus could not see any of the nine defects; whole-repository measurement is what found them, and
this loop inherits that. Its evaluation set has to include repositories, not only pairs.
this loop inherits that. Its evaluation set has to include repositories as well as pairs.
* Only two repositories exist today (one Python, one C), and the C one has never produced a true positive.
An optimiser tuned on that evidence base would be tuned on very little.
* `contributor-scan` Req 8 already warns about the +16% overfitting gap between the slice a change was
+10 -10
View File
@@ -10,7 +10,7 @@ anything.
The design turns on one asymmetry: **two of those three bugs needed no judgement at all.** Once the first was
understood, the other two were the same fix carried across. So the automation that pays is *propagation and
memory*, and the model is confined to the one step that genuinely needs inference — naming what a group has in
memory*, and the model is confined to the one step that genuinely needs inference: naming what a group has in
common.
### Goals
@@ -22,7 +22,7 @@ common.
### Non-Goals
- **Automated fact-table edits.** Not deferred — prohibited. See Boundary Commitments.
- **Automated fact-table edits.** Prohibited rather than deferred. See Boundary Commitments.
- Suppressing findings silently. A dismissal makes a finding *labelled*, never invisible.
- A general-purpose optimiser. The only surfaces evolved are two prompt constants.
- Replacing the pair corpus. It is the holdout that keeps per-repository tuning honest.
@@ -38,10 +38,10 @@ common.
### Out of Boundary
- `semantic/facts`, `semantic/obligations`, `ruleset/`, the declared policy file — **no automated writer, by
- `semantic/facts`, `semantic/obligations`, `ruleset/`, the declared policy file: **no automated writer, by
any path, at any time.** This is the boundary the whole design exists to respect.
- The three gates. They are the invariant an adoption is measured against, not a thing this spec changes.
- `model/taint.py`, `dominance.py`, `config_value.py` — arbiters are fixed here; a proposal that would change
- `model/taint.py`, `dominance.py`, `config_value.py`: arbiters are fixed here; a proposal that would change
one is a proposal for a human, not an edit.
### Allowed Dependencies
@@ -64,7 +64,7 @@ Three pieces already exist and are reused rather than rebuilt:
- **`benchmarks/repos/*.toml`** declares known vulnerabilities per checkout, with `in_scope` stating whether a
family can decide one at all. That is the positive half of an evaluation set, already committed.
- **`pairs.py`** already carries a train/holdout split and the machinery that respects it. The leak of
2026-09-05 was in the *lever*, not the splitter — `semantic/loo.evaluate_loo` was honest throughout.
2026-09-05 was in the *lever*, not the splitter; `semantic/loo.evaluate_loo` was honest throughout.
- **`redaction.py`** is applied to every prompt and persisted trajectory, and applies here unchanged.
### Boundary Map
@@ -87,21 +87,21 @@ benchmarks/repos/*.toml ── read ────▶ evaluation set
**A dismissal is a label, not a suppression.** A suppressed finding disappears and its reason rots; a
labelled one is reported as dismissed with its reason, and a dismissal that stops matching is reported too.
This is what turns a suppression list into the negative half of an evaluation set — the thing that makes the
This is what turns a suppression list into the negative half of an evaluation set, the thing that makes the
rest of the feature possible.
**The tool never writes the dismissal file.** Same rule as the declared policy file, for the same reason: a
file the tool edits is a file no reviewer trusts. It is committed by a developer and reviewed like code.
**Grouping is deterministic; only the hypothesis is not.** Entries are grouped by shared spec token, matcher
clause, family and witness shape — all mechanical. The model is asked one question about a group it did not
clause, family and witness shape; all mechanical. The model is asked one question about a group it did not
choose, and its answer goes to a queue. This keeps the inference confined to the step that needs it.
**Evolution targets the question, never the facts.** A prompt is how we ask; a fact table is what we claim.
`CANDIDATE_QUESTION` and `residual_question` are already the LLM-facing surface and are the only mutable
targets. Enforced by a test that fails if any writer touches the fact modules, not by convention.
**The holdout is never read.** Not "should not" — the evaluation-set API has no path to it during
**The holdout is never read.** The evaluation-set API has no path to it during
optimisation, and a test asserts the absence. The 2026-09-05 leak was possible because reading it was merely
discouraged.
@@ -123,9 +123,9 @@ benchmarks/ledger/
### Changed
- `cli.py` — a `feedback` subcommand: `ledger`, `groups`, `propose`, `evolve`. Maintainer surface; `scan`
- `cli.py`: a `feedback` subcommand: `ledger`, `groups`, `propose`, `evolve`. Maintainer surface; `scan`
gains only the dismissal read.
- `reports.py` — a dismissed finding is rendered as dismissed with its reason.
- `reports.py`: a dismissed finding is rendered as dismissed with its reason.
### Unchanged and load-bearing
@@ -5,22 +5,22 @@
## Introduction
Nine engine defects were found in one session against two repositories. **Three were the same bug in three
matchers** — a spec token matched as a substring rather than a word: `resolve` inside `resolveUrl` in the
matchers**: a spec token matched as a substring rather than a word: `resolve` inside `resolveUrl` in the
sanitizer matcher, `user` inside `users` in the discharger matcher, `gets` inside `fgets` in the sink matcher.
Each was found by a measurement, fixed locally, and left no trace; the third survived until an 89k-line C
library made it visible as 26 identical false positives.
Nothing in the system remembers that a finding was wrong. There is no artifact saying *this site, at this
rung, is not a defect, and here is why* — so a class recurs, and each recurrence costs another measurement.
rung, is not a defect, and here is why*, so a class recurs, and each recurrence costs another measurement.
Two audiences want that artifact for different reasons. A maintainer wants the *propagation* automated: two of
the three matcher bugs needed no judgement at all, only the fix carried across. A developer wants to dismiss a
finding in their own repository, have it stay dismissed, and have the reason reviewable beside the code — a
finding in their own repository, have it stay dismissed, and have the reason reviewable beside the code; a
tool that cannot be told it is wrong is one they stop running.
This feature is bounded by a rule the project has held from the start and a leak that shows why. An optimiser
may evolve **the question** — the judge and residual prompts, which are how we ask. It may never author **the
facts** — tables, rulesets, the declared policy file, which are what we claim. On 2026-09-05 the improve lever
may evolve **the question**: the judge and residual prompts, which are how we ask. It may never author **the
facts**: tables, rulesets, the declared policy file, which are what we claim. On 2026-09-05 the improve lever
admitted candidate shapes because they "recover a currently missed holdout pair"; five of eleven were taught by
the holdout pairs they then recovered, and a holdout Youden of **−0.059 was reported as +0.059**.
@@ -38,11 +38,11 @@ recorded and my colleagues can see why.
#### Acceptance Criteria
1. The system shall read dismissals from a declarative file in the **target** repository, and shall never
write to it — the same rule the declared policy file already follows.
write to it, the same rule the declared policy file already follows.
2. A dismissal shall carry the site it concerns, the rung it was dismissed at, a reason, and an author.
3. The system shall reject a dismissal that carries no reason, because a suppression list without reasons is
a graveyard nobody can audit.
4. The system shall report a dismissed finding as dismissed, with its reason, rather than omitting it — so a
4. The system shall report a dismissed finding as dismissed, with its reason, rather than omitting it, so a
stale dismissal is visible rather than silent.
5. When a dismissal no longer matches any finding, the system shall say so, because a dismissal that has
stopped applying is either a fixed bug or a moved target and both are worth knowing.
@@ -54,20 +54,20 @@ cause is visible as a cause rather than as five separate bugs.
#### Acceptance Criteria
1. When a finding is contradicted — by a dismissal, by the judge with coverage, or by a declared contract —
1. When a finding is contradicted (by a dismissal, by the judge with coverage, or by a declared contract)
the system shall record the site, the rung, the witness, the spec tokens that produced it, and the commit.
2. Each ledger entry shall be runnable as a regression case: this site, at this commit, shall not be reported
at that rung.
3. The ledger's regression cases shall run offline in CI, so that re-introducing a fixed cause fails a test
rather than a repository scan four weeks later.
4. The system shall group ledger entries by what they share — spec token, matcher clause, family, witness
shape — and report the groups by size.
4. The system shall group ledger entries by what they share (spec token, matcher clause, family, witness
shape) and report the groups by size.
5. The ledger shall be committed evidence, not a local artifact.
### Requirement 3: A model may propose a cause, and only propose it
**Objective:** As a maintainer, I want the model to do the part that needs judgement — naming what a group of
counter-examples has in common — without it deciding anything.
**Objective:** As a maintainer, I want the model to do the part that needs judgement (naming what a group of
counter-examples has in common) without it deciding anything.
#### Acceptance Criteria
@@ -132,7 +132,7 @@ quietly loses recall is worse than none.
1. The evaluation set shall include whole repositories, not pairs alone: the pair corpus could not see any of
the nine defects that motivate this feature, and the gates stayed byte-identical through every fix.
2. The system shall state how many repositories its evidence base contains, because two — one of which has
never produced a true positive — is very little to tune against.
2. The system shall state how many repositories its evidence base contains, because two (one of which has
never produced a true positive) is very little to tune against.
3. Where an adoption is measured on one repository only, the system shall label it as such rather than
reporting it as a general improvement.