Marius CiepluchandClaude Opus 5 506bbe5f5c fix(learning-harness): round zero writes down every pair it has paid for ...

The 176-pair vfc baseline hit its four-hour timeout and wrote nothing. Scores
were accumulated in memory and persisted once at the end, so roughly four
hours of paid API calls were lost. I fixed the exception path this morning --
a provider hiccup costs one pair -- and missed that a timeout, a kill or a
machine going away cost all of them.

Each pair is now appended to `scores.jsonl` and flushed as it is measured, and
`--resume` keeps what a previous run of that exact baseline already bought.
This also gives the per-pair outcomes the round-3 review asked for, which make
the noise floor auditable after the fact instead of only in aggregate.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG

2026-09-06 19:53:49 +02:00

..

golden

feat(gate): Phase 4 task 8 — lock the detection gate into CI

2026-06-22 17:13:02 +02:00

conftest.py

feat(pair-corpus-honesty): score non-vendored pointer pairs through a local cache

2026-09-05 10:45:43 +02:00

test_benchmark.py

feat(three-stage-scan): add aliased-sink mutation the regex scan misses

2026-09-04 11:50:44 +02:00

test_calibration.py

ci: add quality gates and release workflows

2026-06-18 10:06:21 +02:00

test_candidate_enumeration.py

feat(constrained-detector): the ceiling, measured offline across every slice

2026-09-06 18:44:19 +02:00

test_cli_hx_dispatch.py

feat(three-stage-scan): attach tool hunter to map hotspots

2026-09-04 00:04:36 +02:00

test_cli_improve.py

feat(pair-corpus-honesty): reject improve rounds that regress any provenance profile on holdout

2026-09-05 10:45:43 +02:00

test_complexity_hints.py

feat(three-stage-scan): attach test-gap hints to hotspots

2026-09-03 20:33:48 +02:00

test_complexity_ledger.py

feat(three-stage-scan): report map and honor fail-on worth-fixing

2026-09-04 00:18:33 +02:00

test_complexity_map.py

feat(three-stage-scan): attach test-gap hints to hotspots

2026-09-03 20:33:48 +02:00

test_complexity_signals.py

feat(three-stage-scan): compute complexity signals for hotspot ranking

2026-09-03 20:05:13 +02:00

test_config.py

feat: load OpenRouter settings from a local .env

2026-09-04 14:06:18 +02:00

test_degradation_matrix.py

test(learning-harness): the gate baseline and the degradation matrix

2026-09-06 13:20:23 +02:00

test_detection_benchmarks.py

feat(gate): Phase 4 task 8 — lock the detection gate into CI

2026-06-22 17:13:02 +02:00

test_findings.py

feat(ruleset): Phase 2 (task 3) — rules as governed data, severity from policy

2026-06-22 15:27:16 +02:00

test_fixture_difficulty.py

feat(learning-harness): declared twins and a fixture-difficulty check

2026-09-06 13:09:56 +02:00

test_fusion.py

feat(fusion): Phase 13 — two-panel fusion adjudication

2026-06-22 18:57:03 +02:00

test_gate.py

feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs

2026-06-22 20:02:34 +02:00

test_hardening_pipeline.py

feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs

2026-06-22 20:02:34 +02:00

test_harness_ext.py

feat(harnessx): configurable provider + per-task budget caps

2026-06-22 17:59:44 +02:00

test_harness.py

ci: add quality gates and release workflows

2026-06-18 10:06:21 +02:00

test_hunter_curated_tools.py

fix(learning-harness): a hunt that ends in prose is asked to format it

2026-09-06 14:18:09 +02:00

test_hunter_harness.py

feat(harnessx): configurable provider + per-task budget caps

2026-06-22 17:59:44 +02:00

test_hunter_tools.py

feat(three-stage-scan): add path-clamped hunter read grep and refs

2026-09-03 21:01:57 +02:00

test_hunter.py

feat(hunter): add standard-mode scheduling

2026-06-18 10:59:25 +02:00

test_improve.py

feat(pair-corpus-honesty): score non-vendored pointer pairs through a local cache

2026-09-05 10:45:43 +02:00

test_index.py

ci: add quality gates and release workflows

2026-06-18 10:06:21 +02:00

test_learning_acceptance.py

feat(learning-harness): the acceptance rule and the directory revert

2026-09-06 11:46:44 +02:00

test_learning_aggregation.py

docs(learning-harness): amend the taxonomy name and carry the flat-taxonomy caveat

2026-09-06 10:22:33 +02:00

test_learning_canaries.py

fix(learning-harness): the three defects the group 2 review found

2026-09-06 10:32:02 +02:00

test_learning_classifier_report.py

feat(learning-harness): absence rows travel with their registration

2026-09-06 12:39:55 +02:00

test_learning_classify.py

feat(learning-harness): three-tier auto-classifier

2026-09-06 09:25:45 +02:00

test_learning_cli.py

fix(learning-harness): the run reads the meter it publishes

2026-09-06 14:12:14 +02:00

test_learning_cutoff.py

fix(learning-harness): a slice is part of what identifies a baseline

2026-09-06 14:39:43 +02:00

test_learning_detectors.py

fix(learning-harness): the three defects the group 2 review found

2026-09-06 10:32:02 +02:00

test_learning_endpoint.py

fix(learning-harness): logprobs are on the choice, not the message

2026-09-06 15:05:56 +02:00

test_learning_families.py

test(learning-harness): apply the group 1 review suggestions

2026-09-06 09:40:24 +02:00

test_learning_journal.py

feat(learning-harness): the round journal, the archive and the rejected buffer

2026-09-06 10:10:01 +02:00

test_learning_pairs_evidence.py

fix(learning-harness): the round-3 review findings, remediated RED-first

2026-09-06 13:03:06 +02:00

test_learning_pairs_path.py

feat(learning-harness): family scoring on the pairs hunter path

2026-09-06 10:41:06 +02:00

test_learning_proposer.py

fix(learning-harness): the round-3 review findings, remediated RED-first

2026-09-06 13:03:06 +02:00

test_learning_publish.py

fix(learning-harness): the round-3 review findings, remediated RED-first

2026-09-06 13:03:06 +02:00

test_learning_refusals.py

fix(learning-harness): the round-4 review findings

2026-09-06 14:03:10 +02:00

test_learning_reliability.py

feat(learning-harness): thinking is a measured condition, and every round

2026-09-06 15:12:09 +02:00

test_learning_round4_evidence.py

fix(learning-harness): the round-4 review findings

2026-09-06 14:03:10 +02:00

test_learning_round_evidence.py

fix(learning-harness): the round-3 review findings, remediated RED-first

2026-09-06 13:03:06 +02:00

test_learning_round_loop.py

fix(learning-harness): the round-3 review findings, remediated RED-first

2026-09-06 13:03:06 +02:00

test_learning_rounds.py

fix(learning-harness): round zero writes down every pair it has paid for

2026-09-06 19:53:49 +02:00

test_learning_scan.py

fix(learning-harness): the round-3 review findings, remediated RED-first

2026-09-06 13:03:06 +02:00

test_learning_scoring.py

fix(learning-harness): the three defects the group 2 review found

2026-09-06 10:32:02 +02:00

test_learning_split.py

fix(learning-harness): the round-3 review findings, remediated RED-first

2026-09-06 13:03:06 +02:00

test_learning_thinking.py

feat(learning-harness): thinking is a measured condition, and every round

2026-09-06 15:12:09 +02:00

test_learning_verifiers.py

fix(learning-harness): the three defects the group 2 review found

2026-09-06 10:32:02 +02:00

test_loo.py

fix(learning-harness): the round-4 review findings

2026-09-06 14:03:10 +02:00

test_map_gate.py

feat(three-stage-scan): gate stage-2 maps against split-sink ranking

2026-09-04 11:53:06 +02:00

test_mapping.py

feat(authorization-obligations): name handlers registered by call or convention

2026-09-06 07:31:48 +02:00

test_mcp.py

feat(three-stage-scan): add fail-on worth-fixing

2026-09-03 19:44:04 +02:00

test_mechanism_export.py

feat(corpus-seeded-mechanisms): corpus writer and additive mechanism fields

2026-09-05 14:27:33 +02:00

test_mechanism_lever.py

feat(corpus-seeded-mechanisms): the mechanisms lever runs from ousast improve

2026-09-05 15:35:08 +02:00

test_mechanism_loop.py

feat(learning-harness): one teacher rule across every learning path

2026-09-06 10:15:44 +02:00

test_mechanism_reports.py

feat(corpus-seeded-mechanisms): reports name the mechanism, its known fix guard and pair provenance

2026-09-05 15:35:08 +02:00

test_mechanism_seed.py

feat(corpus-seeded-mechanisms): export trusted pairs as mechanism records

2026-09-05 14:40:15 +02:00

test_mutation_fixtures.py

feat(three-stage-scan): add aliased-sink mutation the regex scan misses

2026-09-04 11:50:44 +02:00

test_obligation_check.py

feat(authorization-obligations): the obligation checker, path-aware and function-local

2026-09-06 07:24:13 +02:00

test_obligation_dominance.py

feat(authorization-obligations): Dominance protocol with an order-based default

2026-09-05 19:38:04 +02:00

test_obligation_export.py

fix(learning-harness): the round-4 review findings

2026-09-06 14:03:10 +02:00

test_obligation_facts.py

feat(authorization-obligations): group sibling handlers and find consistency anomalies

2026-09-06 07:24:13 +02:00

test_obligation_intent.py

feat(authorization-obligations): hunter adjudicates intent without raising evidence

2026-09-06 07:24:13 +02:00

test_obligation_operations.py

feat(learning-harness): declared twins and a fixture-difficulty check

2026-09-06 13:09:56 +02:00

test_obligation_policy.py

feat(authorization-obligations): group sibling handlers and find consistency anomalies

2026-09-06 07:24:13 +02:00

test_obligation_reports.py

feat(authorization-obligations): reports and ranking name the obligation

2026-09-06 07:24:13 +02:00

test_obligation_scan.py

feat(authorization-obligations): run the checker in MAP with config and manifest counters

2026-09-06 07:24:13 +02:00

test_obligation_shapes.py

feat(authorization-obligations): obligation shapes derived from trusted pairs

2026-09-05 19:38:04 +02:00

test_obligation_siblings.py

feat(authorization-obligations): group sibling handlers and find consistency anomalies

2026-09-06 07:24:13 +02:00

test_opencode_skills.py

chore: add kiro spec-driven tooling and steering docs

2026-06-21 14:28:19 +02:00

test_pair_corpus.py

feat(learning-harness): family scoring on the pairs hunter path

2026-09-06 10:41:06 +02:00

test_pair_harvest.py

feat(authorization-obligations): harvest mode handler_context and the obligation label

2026-09-06 07:24:13 +02:00

test_pair_hygiene.py

fix(learning-harness): the round-4 review findings

2026-09-06 14:03:10 +02:00

test_pair_registration_context.py

feat(constrained-detector): the ceiling, measured offline across every slice

2026-09-06 18:44:19 +02:00

test_pipeline_calibration.py

feat(calibration): automatic in-pipeline false-positive feedback

2026-06-21 20:44:08 +02:00

test_policy.py

feat(governance): Phase 1 — CWE policy store + project score (zero-dep)

2026-06-22 12:31:35 +02:00

test_preprocess.py

feat(scan): add initial SAST pipeline

2026-06-17 21:13:51 +02:00

test_provenance.py

feat(pair-corpus-honesty): deterministic provenance fingerprint at scan time

2026-09-05 10:45:43 +02:00

test_provider_retry.py

feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs

2026-06-22 20:02:34 +02:00

test_rank.py

feat(harness): add runtime and reachability mapping

2026-06-18 07:57:06 +02:00

test_redaction.py

feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs

2026-06-22 20:02:34 +02:00

test_regress_candidates.py

feat(three-stage-scan): select bounded regress candidates with sev-5 extras

2026-09-03 22:36:02 +02:00

test_regress_recipes.py

feat(three-stage-scan): add language recipes with inconclusive fallback

2026-09-03 22:48:10 +02:00

test_regress_safety.py

feat(three-stage-scan): reject unsafe regression snippets before execution

2026-09-03 22:12:45 +02:00

test_regress_verdicts.py

feat(three-stage-scan): map sandbox outcomes to worth-fixing verdicts

2026-09-03 22:59:01 +02:00

test_reports.py

feat(three-stage-scan): report map and honor fail-on worth-fixing

2026-09-04 00:18:33 +02:00

test_ruleset.py

feat(benchmark): Phase 2 (task 4) — actionable per-rule signals, ledger, FP knob

2026-06-22 15:41:17 +02:00

test_run.py

feat(scan): add initial SAST pipeline

2026-06-17 21:13:51 +02:00

test_sandbox_probe.py

feat(three-stage-scan): run or skip deep regression with visible degradation

2026-09-03 23:43:15 +02:00

test_sandbox_runner.py

feat(three-stage-scan): run or skip deep regression with visible degradation

2026-09-03 23:43:15 +02:00

test_scan_deep.py

feat(propose-adjudicate-prove): overlay adjudicates inventory before prove

2026-09-04 13:57:33 +02:00

test_scan_pipeline_integrity.py

feat(three-stage-scan): attach tool hunter to map hotspots

2026-09-04 00:04:36 +02:00

test_score_pipeline.py

feat(ruleset): Phase 2 (task 3) — rules as governed data, severity from policy

2026-06-22 15:27:16 +02:00

test_scoring.py

feat(benchmark): Phase 2 (task 4) — actionable per-rule signals, ledger, FP knob

2026-06-22 15:41:17 +02:00

test_semantic_cst.py

feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra

2026-09-04 15:07:35 +02:00

test_semantic_extra.py

feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra

2026-09-04 15:07:35 +02:00

test_semantic_overlay.py

feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra

2026-09-04 15:07:35 +02:00

test_semantic_pipeline.py

feat(propose-adjudicate-prove): overlay adjudicates inventory before prove

2026-09-04 13:57:33 +02:00

test_skills.py

feat(skills): Phase 14 — Trail of Bits skill index + router

2026-06-22 19:51:04 +02:00

test_slot_contract.py

feat(slots): Phase 3 task 6.3 — model-disabled, slot-contracted deterministic stages

2026-06-22 18:39:09 +02:00

test_split_sink_fixtures.py

feat(three-stage-scan): add split-sink fixtures the regex scan misses

2026-09-04 11:49:44 +02:00

test_split_sink_regress.py

feat(three-stage-scan): prove split-sink triggerable in sandbox

2026-09-04 11:54:01 +02:00

test_stages.py

feat(three-stage-scan): keep quick as stage-1 inventory

2026-09-03 23:14:00 +02:00

test_tool_hunter.py

fix(learning-harness): a hunt that ends in prose is asked to format it

2026-09-06 14:18:09 +02:00

test_typescript_grammar.py

feat(learning-harness): absence rows travel with their registration

2026-09-06 12:39:55 +02:00

test_variant_scan.py

feat(corpus-seeded-mechanisms): reports name the mechanism, its known fix guard and pair provenance

2026-09-05 15:35:08 +02:00

test_variant_search.py

refactor(corpus-seeded-mechanisms): apply review round 2 suggestions

2026-09-05 14:55:00 +02:00

test_variants.py

feat(corpus-seeded-mechanisms): structural shapes derived from labeled pairs

2026-09-05 14:27:33 +02:00

test_verification.py

ci: add quality gates and release workflows

2026-06-18 10:06:21 +02:00

test_verify_judge.py

feat(harnessx): configurable provider + per-task budget caps

2026-06-22 17:59:44 +02:00