Marius CiepluchandClaude Fable 5.1 041d87199a perf(hunter): cacheable message order; adaptive step budget in the sink verifier ...

Token efficiency, measured on 180 verified pgAdmin candidates: stage 2 costs
$0.0139 per candidate, 90% of the spend, because the conversation is re-sent on
every turn and grows with each tool result.

- Context files now precede the question in the hunter's first message. DeepSeek
  bills a cached prefix at 1/30 of a fresh one, by prefix; with the question first,
  every candidate asked about the same file (about 5 per file) broke the prefix on
  its first line and paid for the file again. A test pins the order.
- The verifier runs each candidate at 3 steps and repeats it once at 6 only when
  the hunt spent every step on tool calls without a verdict. On the seven declared
  v2 sites, 3 steps found 5 and 8 steps found 6; most candidates conclude in 3.
- Per-candidate token usage (prompt, cached, completion, calls) and escalation
  are recorded, and each case reports its cache share, so the next efficiency
  decision is made on numbers.

Validation on the 180 pgAdmin candidates plus the 22 known sites (about $1)
waits for account credit: DeepSeek returns 402 at the moment.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

2026-09-29 09:52:23 +02:00

..

golden

feat(gate): Phase 4 task 8 — lock the detection gate into CI

2026-06-22 17:13:02 +02:00

conftest.py

fix(cpg): the real-request cost, attributed and removed -- overlays once, seeds scoped, data tables out

2026-09-11 01:16:00 +02:00

test_benchmark.py

feat(three-stage-scan): add aliased-sink mutation the regex scan misses

2026-09-04 11:50:44 +02:00

test_calibration.py

ci: add quality gates and release workflows

2026-06-18 10:06:21 +02:00

test_candidate_enumeration.py

feat(model-grounded-detection): group 1, subtract the noise architecture

2026-09-08 12:15:54 +02:00

test_cli_hx_dispatch.py

feat(contributor-scan): 1.3 the model layer reaches a user

2026-09-09 07:42:44 +02:00

test_cli_improve.py

feat(pair-corpus-honesty): reject improve rounds that regress any provenance profile on holdout

2026-09-05 10:45:43 +02:00

test_complexity_hints.py

feat(three-stage-scan): attach test-gap hints to hotspots

2026-09-03 20:33:48 +02:00

test_complexity_ledger.py

feat(three-stage-scan): report map and honor fail-on worth-fixing

2026-09-04 00:18:33 +02:00

test_complexity_map.py

feat(three-stage-scan): attach test-gap hints to hotspots

2026-09-03 20:33:48 +02:00

test_complexity_signals.py

feat(three-stage-scan): compute complexity signals for hotspot ranking

2026-09-03 20:05:13 +02:00

test_config.py

feat: load OpenRouter settings from a local .env

2026-09-04 14:06:18 +02:00

test_cpg_backend.py

fix(scan): report requests the reserve held back as unasked, not failed

2026-09-23 08:51:12 +02:00

test_cpg_budget.py

test(cpg-budget): a process vanishing mid-read is the success case too

2026-09-26 07:44:33 +02:00

test_cpg_queries_compile.py

Revert "feat(taint): project sink wrappers, one level and confirmed by dataflow"

2026-09-26 15:36:25 +02:00

test_cpg_session.py

fix(cpg): stop a failed session start from consuming the transaction budget

2026-09-17 08:35:57 +02:00

test_degradation_matrix.py

feat(model-grounded-detection): group 1, subtract the noise architecture

2026-09-08 12:15:54 +02:00

test_detection_benchmarks.py

feat(gate): Phase 4 task 8 — lock the detection gate into CI

2026-06-22 17:13:02 +02:00

test_dispatch_value.py

fix(mapping): record every callback of a register_rest_route call as a route

2026-09-24 11:08:22 +02:00

test_fact_memo.py

perf(push): find the Python-side scale walls all at once, and remove them

2026-09-23 19:36:32 +02:00

test_findings.py

feat(ruleset): Phase 2 (task 3) — rules as governed data, severity from policy

2026-06-22 15:27:16 +02:00

test_frontend_retention.py

fix(contributor-scan): retain declared Gruntfile and report census gaps honestly

2026-09-14 08:07:06 +02:00

test_fusion.py

feat(fusion): Phase 13 — two-panel fusion adjudication

2026-06-22 18:57:03 +02:00

test_gate.py

feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs

2026-06-22 20:02:34 +02:00

test_gitignore_semantics.py

fix(preprocess): read .gitignore with git's semantics; never ignore a tracked file

2026-09-25 15:29:43 +02:00

test_graph_reuse.py

feat(pre-push-safety-net): reuse compatible artifacts throughout replay

2026-09-13 17:08:49 +02:00

test_hardening_pipeline.py

feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs

2026-06-22 20:02:34 +02:00

test_harness_ext.py

feat(harnessx): configurable provider + per-task budget caps

2026-06-22 17:59:44 +02:00

test_harness.py

ci: add quality gates and release workflows

2026-06-18 10:06:21 +02:00

test_hunter_curated_tools.py

fix(hunter): ask for the final answer in plain text; DeepSeek's JSON mode lost it

2026-09-28 20:10:57 +02:00

test_hunter_harness.py

feat(harnessx): configurable provider + per-task budget caps

2026-06-22 17:59:44 +02:00

test_hunter_tools.py

fix(hunter): bound every tool result; show the provider's reason for an HTTP error

2026-09-28 20:15:14 +02:00

test_hunter.py

feat(hunter): add standard-mode scheduling

2026-06-18 10:59:25 +02:00

test_improve.py

feat(pair-corpus-honesty): score non-vendored pointer pairs through a local cache

2026-09-05 10:45:43 +02:00

test_independent_population.py

spec(pre-push-safety-net): reserve the independent population v2 (17 cases, unscanned)

2026-09-26 17:53:51 +02:00

test_index.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_layout_facts.py

feat(pre-push-safety-net): exclude vendor inputs and partition frontend coverage

2026-09-13 11:53:29 +02:00

test_map_gate.py

feat(three-stage-scan): gate stage-2 maps against split-sink ranking

2026-09-04 11:53:06 +02:00

test_mapping.py

fix(mapping): a PHP method with a visibility modifier is a region

2026-09-09 17:33:36 +02:00

test_mcp.py

feat(mcp): the findings list carries the rung, and leads with what was decided

2026-09-09 13:15:54 +02:00

test_model_calibrate.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_model_change_context.py

fix(scan): symlink, unsupported-language and unparsed-file gaps are owned by what they name

2026-09-25 14:30:36 +02:00

test_model_config_value.py

fix(config): report every permissive setting in a file, not the first one

2026-09-20 16:30:04 +02:00

test_model_dominance.py

feat(model): an access-control finding names a line, and what the PDG does not give us

2026-09-09 13:09:47 +02:00

test_model_endpoint.py

spec(pre-push-safety-net): pre-register the hunter experiment on population v2

2026-09-28 09:10:02 +02:00

test_model_evidence.py

feat(model): shipped joins the evidence vector as a within-tier weight; v5 baseline

2026-09-11 13:45:00 +02:00

test_model_execution.py

feat(model-grounded-detection): group 6, the execution tier as a seam not a build

2026-09-08 16:58:03 +02:00

test_model_judge.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_model_ladder.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_model_partitions.py

fix(contributor-scan): retain declared Gruntfile and report census gaps honestly

2026-09-14 08:07:06 +02:00

test_model_pipeline.py

feat(model): a region that spans the flow, not just a function

2026-09-09 09:27:19 +02:00

test_model_regions.py

fix(ranker): scope a region to the file that defines its handler (M1b 12.4)

2026-09-17 14:04:39 +02:00

test_model_report.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_model_scan_stage.py

feat(contributor-scan): 1.3 the model layer reaches a user

2026-09-09 07:42:44 +02:00

test_model_scan.py

fix(contributor-scan): retain declared Gruntfile and report census gaps honestly

2026-09-14 08:07:06 +02:00

test_model_scope.py

fix(scan,taint): TypeScript's limit is owned by TypeScript; a source pattern is a token

2026-09-25 15:13:30 +02:00

test_model_shipped.py

feat(model): let the project say what it ships, and rank that first

2026-09-09 12:45:21 +02:00

test_model_specs.py

feat(repos): a WordPress plugin with a verified CVE, and the WordPress DB API

2026-09-09 16:13:17 +02:00

test_model_taint.py

feat(model): one finding per sink site, and WordPress's registration model

2026-09-09 14:30:48 +02:00

test_model_taxonomy.py

feat(reports): say what was looked for, and what cannot be decided

2026-09-09 15:08:01 +02:00

test_module_audit.py

feat(model-grounded-detection): group 1, subtract the noise architecture

2026-09-08 12:15:54 +02:00

test_mutation_fixtures.py

feat(three-stage-scan): add aliased-sink mutation the regex scan misses

2026-09-04 11:50:44 +02:00

test_obligation_check.py

feat(model-grounded-detection): group 1, subtract the noise architecture

2026-09-08 12:15:54 +02:00

test_obligation_dominance.py

feat(authorization-obligations): Dominance protocol with an order-based default

2026-09-05 19:38:04 +02:00

test_obligation_facts.py

feat(authorization-obligations): group sibling handlers and find consistency anomalies

2026-09-06 07:24:13 +02:00

test_obligation_intent.py

feat(authorization-obligations): hunter adjudicates intent without raising evidence

2026-09-06 07:24:13 +02:00

test_obligation_operations.py

feat(learning-harness): declared twins and a fixture-difficulty check

2026-09-06 13:09:56 +02:00

test_obligation_policy.py

feat(authorization-obligations): group sibling handlers and find consistency anomalies

2026-09-06 07:24:13 +02:00

test_obligation_reports.py

feat(authorization-obligations): reports and ranking name the obligation

2026-09-06 07:24:13 +02:00

test_obligation_scan.py

feat(authorization-obligations): run the checker in MAP with config and manifest counters

2026-09-06 07:24:13 +02:00

test_obligation_shapes.py

feat(authorization-obligations): obligation shapes derived from trusted pairs

2026-09-05 19:38:04 +02:00

test_obligation_siblings.py

feat(authorization-obligations): group sibling handlers and find consistency anomalies

2026-09-06 07:24:13 +02:00

test_opencode_skills.py

chore: add kiro spec-driven tooling and steering docs

2026-06-21 14:28:19 +02:00

test_pair_corpus.py

feat(model-grounded-detection): group 1, subtract the noise architecture

2026-09-08 12:15:54 +02:00

test_pair_harvest.py

feat(authorization-obligations): harvest mode handler_context and the obligation label

2026-09-06 07:24:13 +02:00

test_pair_hygiene.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_pair_registration_context.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_pipeline_calibration.py

feat(calibration): automatic in-pipeline false-positive feedback

2026-06-21 20:44:08 +02:00

test_policy.py

feat(governance): Phase 1 — CWE policy store + project score (zero-dep)

2026-06-22 12:31:35 +02:00

test_preprocess.py

feat(scan): add initial SAST pipeline

2026-06-17 21:13:51 +02:00

test_profiling.py

perf(push): drop per-entry cache fsync; let long runs profile themselves

2026-09-23 19:05:08 +02:00

test_provenance.py

feat(pair-corpus-honesty): deterministic provenance fingerprint at scan time

2026-09-05 10:45:43 +02:00

test_provider_retry.py

fix: a body that stops mid-read is transient, and a round that dies still says so

2026-09-07 07:37:41 +02:00

test_push_admission.py

fix(pre-push-safety-net): take the operation symbol from the call (M1b 12.1)

2026-09-17 09:56:59 +02:00

test_push_assistance.py

feat(pre-push-safety-net): bound optional assistance to admitted witnesses

2026-09-13 18:03:31 +02:00

test_push_cache_semantics.py

feat(pre-push-safety-net): key evidence reuse by compatible semantics

2026-09-13 16:16:19 +02:00

test_push_cache.py

fix(push): stop rescanning the whole cache on every publication

2026-09-23 14:12:47 +02:00

test_push_contracts.py

feat(pre-push-safety-net): retain bounded change and rename context

2026-09-12 21:12:45 +02:00

test_push_eligibility.py

feat(pre-push-safety-net): enforce evidence eligibility and record blocked qualification

2026-09-13 19:32:43 +02:00

test_push_experiment.py

feat(pre-push-safety-net): measure frozen PMPro and Node replay feasibility

2026-09-13 15:24:06 +02:00

test_push_feasibility.py

feat(pre-push-safety-net): measure frozen runtime feasibility and retain no-go

2026-09-13 19:06:02 +02:00

test_push_hook.py

feat(pre-push-safety-net): preserve existing hooks during explicit integration

2026-09-13 17:51:28 +02:00

test_push_inputs.py

feat(pre-push-safety-net): pin labeled evaluation inputs

2026-09-12 18:45:54 +02:00

test_push_interface.py

feat(pre-push-safety-net): execute full Git push transactions

2026-09-13 17:45:16 +02:00

test_push_policy.py

fix(push): a gap from another language partition no longer blocks admission

2026-09-21 13:50:38 +02:00

test_push_regression_integration.py

feat(pre-push-safety-net): cover combined push failure paths

2026-09-13 18:43:13 +02:00

test_push_report.py

feat(pre-push-safety-net): render bounded reports and preserve full evidence

2026-09-13 13:25:44 +02:00

test_push_reuse.py

feat(scan): size portions from measured per-answer cost and isolate the kill culprit

2026-09-23 07:55:33 +02:00

test_push_runner.py

feat(pre-push-safety-net): reuse compatible artifacts throughout replay

2026-09-13 17:08:49 +02:00

test_push_scoring.py

feat(pre-push-safety-net): preserve uncertainty in benchmark scoring

2026-09-12 19:04:20 +02:00

test_push_snapshot.py

feat(pre-push-safety-net): retain bounded change and rename context

2026-09-12 21:12:45 +02:00

test_push_vetoes.py

feat(pre-push-safety-net): render the experimental evaluation explanation (M1b 10.10)

2026-09-16 19:59:12 +02:00

test_rank.py

feat(harness): add runtime and reachability mapping

2026-06-18 07:57:06 +02:00

test_redaction.py

feat(hardening): Phase 15 — redaction, retry/backoff, budgets, threat-model docs

2026-06-22 20:02:34 +02:00

test_regress_candidates.py

feat(three-stage-scan): select bounded regress candidates with sev-5 extras

2026-09-03 22:36:02 +02:00

test_regress_recipes.py

feat(three-stage-scan): add language recipes with inconclusive fallback

2026-09-03 22:48:10 +02:00

test_regress_safety.py

feat(three-stage-scan): reject unsafe regression snippets before execution

2026-09-03 22:12:45 +02:00

test_regress_verdicts.py

feat(three-stage-scan): map sandbox outcomes to worth-fixing verdicts

2026-09-03 22:59:01 +02:00

test_reports.py

feat(cpg): give the files a frontend refuses a CPG of their own

2026-09-10 07:32:20 +02:00

test_repos.py

feat(repos): pinned known-vulnerable checkouts, offline by default

2026-09-09 08:10:59 +02:00

test_ruleset.py

feat(benchmark): Phase 2 (task 4) — actionable per-rule signals, ledger, FP knob

2026-06-22 15:41:17 +02:00

test_run.py

feat(scan): add initial SAST pipeline

2026-06-17 21:13:51 +02:00

test_sandbox_probe.py

feat(three-stage-scan): run or skip deep regression with visible degradation

2026-09-03 23:43:15 +02:00

test_sandbox_runner.py

feat(three-stage-scan): run or skip deep regression with visible degradation

2026-09-03 23:43:15 +02:00

test_scan_deep.py

feat(propose-adjudicate-prove): overlay adjudicates inventory before prove

2026-09-04 13:57:33 +02:00

test_scan_pipeline_integrity.py

feat(contributor-scan): 1.3 the model layer reaches a user

2026-09-09 07:42:44 +02:00

test_score_pipeline.py

feat(ruleset): Phase 2 (task 3) — rules as governed data, severity from policy

2026-06-22 15:27:16 +02:00

test_scoring.py

feat(benchmark): Phase 2 (task 4) — actionable per-rule signals, ledger, FP knob

2026-06-22 15:41:17 +02:00

test_semantic_cst.py

feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra

2026-09-04 15:07:35 +02:00

test_semantic_extra.py

feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra

2026-09-04 15:07:35 +02:00

test_semantic_overlay.py

feat(tree-sitter-overlay-extra): parse JS/C/Java overlay via optional extra

2026-09-04 15:07:35 +02:00

test_semantic_pipeline.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_skills.py

feat(skills): Phase 14 — Trail of Bits skill index + router

2026-06-22 19:51:04 +02:00

test_slot_contract.py

feat(slots): Phase 3 task 6.3 — model-disabled, slot-contracted deterministic stages

2026-06-22 18:39:09 +02:00

test_split_sink_fixtures.py

feat(three-stage-scan): add split-sink fixtures the regex scan misses

2026-09-04 11:49:44 +02:00

test_split_sink_regress.py

feat(three-stage-scan): prove split-sink triggerable in sandbox

2026-09-04 11:54:01 +02:00

test_stages.py

feat(three-stage-scan): keep quick as stage-1 inventory

2026-09-03 23:14:00 +02:00

test_subtraction_complete.py

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

test_tool_hunter.py

perf(hunter): cacheable message order; adaptive step budget in the sink verifier

2026-09-29 09:52:23 +02:00

test_typescript_grammar.py

feat(learning-harness): absence rows travel with their registration

2026-09-06 12:39:55 +02:00

test_verification.py

ci: add quality gates and release workflows

2026-06-18 10:06:21 +02:00

test_verify_judge.py

feat(harnessx): configurable provider + per-task budget caps

2026-06-22 17:59:44 +02:00