Commit Graph
9 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 5 1843ef9a02 fix(learning-harness): the round-3 review findings, remediated RED-first
Eight of nine group-3 tasks were rejected. Each defect was hiding a number,
and the numbers are why the loop was not learning:

- The fixed side was judged against the *vulnerable* side's function
  ranges. 191 of 227 pairs with a resolvable labeled span (84%) have
  different spans on the two sides, so a leak was recorded as silence,
  always in the direction that flatters Youden.
- Round zero measured every family with the generalist. `family:unknown`
  earns no credit against any real family, so every family but `unknown`
  baselined at a structural zero and every noise floor with it.
- The cross-family sweep counted absolute failures instead of negative
  flips. On a corpus where most pairs fail at round zero that rejects
  every round for damage it did not do. Unscorable rows voted too.
- K runs was declared and never threaded; `pairs --k-runs` was parsed and
  discarded, and its only test grepped `--help`.
- The meta-agent proposer was a stub, so `learning round` could never
  propose anything. It now composes MetaAgent.evolve against a scratch
  copy of the family directory, and `--proposals FILE` runs a recorded
  sequence offline.
- `refused_holdout` was a vocabulary word no code path could produce, and
  the covering test accepted either outcome.
- Cost was metered once, after everything had been paid for; a stage that
  raised left no record at all.
- The corrected lever number was published beside the earlier one rather
  than in place of it, and the test named for that property never checked.
- Refusals were `continue` statements: `Refusal.degradation()` existed and
  nothing called it. A train-split identical twin was reported as
  `split:train`, which is untrue.

Test repairs that each changed a number: two split tests measured an
`x = 1` fixture that teaches nothing whatever the rule says; two CLI
scoreboard tests asserted against all-zero output produced by a stub that
answers twice and puts its finding outside the labeled function.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-06 13:03:06 +02:00
Marius CiepluchandClaude Fable 5.1 a87755830a docs(specs): kick off authorization-obligations (brief + requirements, unapproved)
Absence bugs as obligations rather than flows: obligated operations and
discharger kinds as closed facts, obligation shapes learned from trusted
pairs, the application's policy recovered from sibling-handler
consistency and an optional declared policy file, path-aware evaluation
with a function-local degraded mode, honest rungs below proof, an
`obligations` improve lever under the holdout gate. Depends on
reachability-flow-model and guard-dominance-regime; the roadmap marks
reachability as the critical path for two consumers.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 16:31:49 +02:00
Marius CiepluchandClaude Fable 5.1 381ea7b92d docs(corpus-seeded-mechanisms): close the spec after six review rounds
- tasks.md: all tasks checked; rounds 1-6 recorded with RED evidence,
  measured leave-one-out and admission numbers, and the recorded design
  deviations.
- spec.json: completed 2026-09-05.
- roadmap: corpus-seeded-mechanisms marked completed; the loop is closed
  end to end (export -> improve admits under the holdout clause -> scan
  searches -> reports name the mechanism).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 15:37:32 +02:00
Marius CiepluchandClaude Fable 5.1 f11cf183de docs(corpus-seeded-mechanisms): leave-one-out table in the roadmap
Task 3.2 (Req 4.4, 7.2): per-slice leave-one-out next to the pair
baselines; reported, never a gate.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 14:55:00 +02:00
Marius CiepluchandClaude Fable 5.1 8c6598b2e8 docs(pair-corpus-honesty): record round 12 and the key-literal history rewrite
- tasks.md: round-12 verdict and the decision to assemble the redaction
  test's example key at runtime and rewrite the unpushed series so no
  commit carries a contiguous key literal; two dated notes marked as
  superseded by tasks 8.1 and 8.4.
- roadmap: the folded inventory-vs-overlay Youden item is checked.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 10:47:14 +02:00
Marius CiepluchandClaude Fable 5.1 27f9ec6a92 docs(pair-corpus-honesty): close the spec; escalations resolved, roadmap carries the measured state
- tasks.md: all tasks checked; the two escalations are closed (5.1 by the
  Req 10 pointer rows, 5.3 by the Req 9 review tiers); round 11 recorded
  with the carry-forwards for corpus-seeded-mechanisms and
  overlay-ir-completeness.
- roadmap: measured state after tiers and pointer slices (vibe-py 115,
  agent-vfc 59, gates byte-identical to pre-spec); pair-corpus-honesty
  marked completed.
- spec.json: completed 2026-09-05 after eleven review rounds.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 10:45:43 +02:00
Marius CiepluchandClaude Fable 5.1 3eba6e10a0 docs(pair-corpus-honesty): spec, review record, roadmap; close real-world-vfc-slice as superseded
Task 7.2 and the spec itself (brief, requirements incl. Req 9/10
amendment, research, design, tasks with eight review rounds recorded).

- real-world-vfc-slice: spec.json closed, superseded_by
  pair-corpus-honesty; brief carries the closure note.
- roadmap: Direction section, per-slice baselines (sast 4/11, vibe-py
  13/35, vfc-js 5/17, vfc 0/176, agent-vfc 0/29 title tier), spec order
  pair-corpus-honesty -> corpus-seeded-mechanisms -> reachability-flow-model
  -> overlay-ir-completeness -> guard-dominance-regime.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
2026-09-05 10:45:43 +02:00
Marius Ciepluch 0116f9bea6 feat(real-world-vfc-slice): add labeled OpenSSL/Firefox/Chromium/curl pair corpus
Harvest 176 isolated vuln-vs-fix functions (109 CVEs, 352 training rows)
into benchmarks/pairs/vfc with provenance and a JSONL training manifest.
Score the slice as an honesty dashboard (ousast pairs --slice vfc);
keep the pair gate on the local fixtures only.
2026-09-04 16:45:38 +02:00
Marius Ciepluch e9f185da59 feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra
Add openultrasast[semantic] with official grammar wheels and one CST walker
into FileIR. quick stays extra-free. Host CLI is not overlay-ready. Default CI
stays extra-free; a semantic job runs pytest -m semantic.
2026-09-04 15:07:35 +02:00