Marius CiepluchandClaude Fable 5.1 48510fd5c2 bench(learn): exp-002 result -- retrieval ensemble vs temperature sampling ...

Reject B. 160 paired units (injection 119, access_control 41; 56 repositories),
both arms, two replicates each, no early stop. Canary agreement A 0.875 -> B
0.906 (CI [-0.03, +0.10], spans; B meets 0.90). Within-pair AUC A 0.815 -> B
0.741 (CI [-0.13, -0.02], against B). AUC -0.02 (against), ADVISORY recall
-0.06 and precision +0.01 (span), usd per candidate A $0.0021 -> B $0.0051
(against). Priced usage A $0.68 / B $1.63; balance $16.72 -> $16.04 at the
run's end (lags). Counts only; task 7.4 marked done.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo

2026-10-02 19:05:34 +02:00

..

2026-09-30-decision-engine-labels

feat(learn): labels from ground truth with provenance (decision-engine task 4)

2026-09-30 19:54:26 +02:00

2026-09-30-harnessx-removal-baseline

chore: .dockerignore; drop the committed baseline .sarif

2026-10-02 08:28:57 +02:00

2026-09-30-harnessx-removal-equality

bench(harnessx-removal): deterministic outputs byte-identical after the removal (task 7)

2026-09-30 12:40:57 +02:00

2026-09-30-harnessx-removal-loop

feat(plane): the improvement loop as a plane Run (harnessx-removal task 4)

2026-09-30 11:59:50 +02:00

2026-09-30-php-engine-alerts

feat(plane): honest coverage in alerts; PHP alerts from the engine on the host

2026-09-30 13:44:32 +02:00

2026-09-30-php-quick-rules

feat(ruleset): PHP quick-mode rules from the engine's vocabulary, measured on the development corpus

2026-09-30 15:05:47 +02:00

2026-10-01-decision-engine-harvest

bench(learn): record the finished engine harvest (decision-engine task 5)

2026-10-01 16:54:27 +02:00

2026-10-01-decision-engine-injection-slice

bench(learn): injection first slice -- compiled AI classifier, out-of-repo evaluation (decision-engine 6.2, 6.6)

2026-10-01 20:40:41 +02:00

2026-10-01-decision-engine-labels

fix(learn): fixed-side negatives only where the fixed side declares the function

2026-10-01 11:03:54 +02:00

2026-10-01-decision-engine-labels-advisory-fixes

bench(pairs): real fix-commit pairs for the short families

2026-10-01 11:51:28 +02:00

2026-10-01-decision-engine-leak-audit

fix(learn): withhold label-tracking signals from the v1 classifier; leak audit

2026-10-01 19:40:58 +02:00

2026-10-02-decision-engine-harvest-advisory

bench(learn): harvest the advisory-fix pairs

2026-10-02 09:40:00 +02:00

2026-10-02-decision-engine-negatives-gap

bench(learn): where the positives-only groups come from

2026-10-02 16:00:37 +02:00

2026-10-02-decision-engine-negatives-rebuilt

bench(learn): labels, features and memory rebuilt with the recovered negatives

2026-10-02 16:31:30 +02:00

2026-10-02-decision-engine-slice-2

bench(learn): slice 2 -- six families out-of-repo

2026-10-02 11:33:16 +02:00

2026-10-02-exp-002

bench(learn): exp-002 result -- retrieval ensemble vs temperature sampling

2026-10-02 19:05:34 +02:00

2026-09-06-candidate-ceiling-all-slices.json

feat(constrained-detector): the ceiling, measured offline across every slice

2026-09-06 18:44:19 +02:00

2026-09-06-candidate-ceiling.json

measure: the candidate generator must be the IR, not the ruleset

2026-09-06 16:04:07 +02:00

2026-09-06-corpus-hygiene.json

feat(corpus): vendor 40 OWASP xss pairs as the owasp slice

2026-09-08 17:28:11 +02:00

2026-09-06-fixture-difficulty.json

feat(learning-harness): declared twins and a fixture-difficulty check

2026-09-06 13:09:56 +02:00

2026-09-06-gate-baseline.txt

test(learning-harness): the gate baseline and the degradation matrix

2026-09-06 13:20:23 +02:00

2026-09-06-learning-families.json

fix(learning-harness): the published table lost a whole slice

2026-09-06 20:44:56 +02:00

2026-09-06-mechanism-lever-split.json

feat(learning-harness): one teacher rule across every learning path

2026-09-06 10:15:44 +02:00

2026-09-06-trajectory-divergence.json

measure(detector-optimizer): the detector never runs the same investigation twice

2026-09-06 15:49:54 +02:00

2026-09-06-vibe-py-holdout-hunter-findings.json

docs(learning-harness): brief for a classified, self-improving detector set; fold closed-loop-integrity

2026-09-06 08:08:01 +02:00

2026-09-06-vibe-py-holdout-hunter-sonnet5.json

docs(learning-harness): brief for a classified, self-improving detector set; fold closed-loop-integrity

2026-09-06 08:08:01 +02:00

2026-09-07-evolve-rounds-injection.json

docs(learning-harness): round 5, and what it says about the experiment itself

2026-09-07 08:09:50 +02:00

2026-09-08-joern-go-no-go.json

measure(model-grounded-detection): task 2.3 reads GO on Joern

2026-09-08 13:26:59 +02:00

2026-09-08-model-entailment-ceiling-cpg.json

feat(model-grounded-detection): Java web sinks, and the post-closure-fix ceiling

2026-09-08 17:16:00 +02:00

2026-09-08-model-entailment-ceiling.json

measure(model-grounded-detection): the entailment ceiling is 2.2%, and it is depth not a wall

2026-09-08 11:10:47 +02:00

2026-09-08-module-audit.json

feat(learn): A/B experiment runner -- registered manifests, paired arms, bootstrap and McNemar (decision-engine task 7)

2026-10-02 16:04:33 +02:00

2026-09-08-pipeline-vibe-py.json

fix(ci): run ruff format, and record run 5 at 90% pair-correct

2026-09-08 19:46:58 +02:00

2026-09-09-declared-public.json

feat(model): an endpoint its contract declares open carries no obligation

2026-09-09 15:39:58 +02:00

2026-09-09-php-facts.json

feat(facts): PHP semantic facts, validated against a real PHP CPG

2026-09-09 14:18:57 +02:00

2026-09-09-php-regions.json

fix(specs): the php2cpg "blocker" was the shim, not the frontend

2026-09-09 14:45:36 +02:00

2026-09-09-pmpro-cve-found.json

feat: the first real CVE, found in production plugin code

2026-09-09 17:24:37 +02:00

2026-09-09-repo-libpng-bounds.json

feat(model): a bound on the data lowers the rung, it does not clear it

2026-09-09 13:01:30 +02:00

2026-09-09-repo-libpng-envelope.json

feat(model): the envelope checkout, and no to Joern server mode

2026-09-09 11:30:02 +02:00

2026-09-09-repo-libpng-shipped-scope.json

feat(model): let the project say what it ships, and rank that first

2026-09-09 12:45:21 +02:00

2026-09-09-repo-pmpro.json

fix(cpg): a query that could not be asked is not an empty answer

2026-09-09 17:15:51 +02:00

2026-09-09-repo-vampi-authz-precision.json

fix(model): a guard discharges the obligation it is a guard of

2026-09-09 11:02:48 +02:00

2026-09-09-repo-vampi-crossfunction.json

feat(model): a region that spans the flow, not just a function

2026-09-09 09:27:19 +02:00

2026-09-09-repo-vampi-module-scope.json

feat(model): a region for the code that is in no function

2026-09-09 09:43:53 +02:00

2026-09-09-repo-vampi.json

feat(model): first repository measurement, and the four things it found

2026-09-09 08:46:21 +02:00

2026-09-09-report-proportionality-model-band.json

feat(reports): the model's own suspicion band is a proposal too

2026-09-09 15:23:57 +02:00

2026-09-09-report-proportionality.json

feat(reports): a report proportionate to what the engine can say

2026-09-09 13:29:26 +02:00

2026-09-12-engine-runtime-smoke.json

feat(pre-push-safety-net): reproduce evaluated Joern runtime

2026-09-12 18:32:18 +02:00

2026-09-12-packaged-snapshot-smoke.json

feat(pre-push-safety-net): retain bounded change and rename context

2026-09-12 21:12:45 +02:00

2026-09-12-snapshot-materialization.json

feat(pre-push-safety-net): materialize bounded isolated snapshots

2026-09-12 20:55:09 +02:00

2026-09-13-admission-policy-smoke.json

feat(pre-push-safety-net): admit only supported actionable changes

2026-09-13 12:56:55 +02:00

2026-09-13-change-context-smoke.json

feat(pre-push-safety-net): connect change context to ranked questions

2026-09-13 10:41:32 +02:00

2026-09-13-compact-report-smoke.json

feat(pre-push-safety-net): render bounded reports and preserve full evidence

2026-09-13 13:25:44 +02:00

2026-09-13-delta-comparison-smoke.json

feat(pre-push-safety-net): compare targeted base and head evidence

2026-09-13 12:42:15 +02:00

2026-09-13-engine-budget-smoke.json

feat(pre-push-safety-net): bound backend execution by one deadline

2026-09-13 10:03:03 +02:00

2026-09-13-first-replay-experiment.json

feat(pre-push-safety-net): measure frozen PMPro and Node replay feasibility

2026-09-13 15:24:06 +02:00

2026-09-13-first-replay-final-frozen-run.json.gz

feat(pre-push-safety-net): measure frozen PMPro and Node replay feasibility

2026-09-13 15:24:06 +02:00

2026-09-13-first-replay-initial-instrument-trial.json.gz

feat(pre-push-safety-net): measure frozen PMPro and Node replay feasibility

2026-09-13 15:24:06 +02:00

2026-09-13-frontend-retention-smoke.json

feat(contributor-scan): retain first-party JavaScript tests in graphs

2026-09-13 11:29:48 +02:00

2026-09-13-graph-lease-control.json

feat(pre-push-safety-net): validate graph identity and isolated leases

2026-09-13 16:05:51 +02:00

2026-09-13-nodegoat-feasibility.json

feat(pre-push-safety-net): measure frozen runtime feasibility and retain no-go

2026-09-13 19:06:02 +02:00

2026-09-13-nodegoat-feasibility.json.gz

feat(pre-push-safety-net): measure frozen runtime feasibility and retain no-go

2026-09-13 19:06:02 +02:00

2026-09-13-partition-retention-blocker.json

docs(pre-push-safety-net): record frontend retention prerequisite

2026-09-13 11:05:44 +02:00

2026-09-13-partition-scope-smoke.json

feat(pre-push-safety-net): exclude vendor inputs and partition frontend coverage

2026-09-13 11:53:29 +02:00

2026-09-13-push-packaged-validation.json

test(pre-push-safety-net): validate packaged outcome and retain rollout no-go

2026-09-13 19:39:19 +02:00

2026-09-13-push-packaged-validation.json.gz

test(pre-push-safety-net): validate packaged outcome and retain rollout no-go

2026-09-13 19:39:19 +02:00

2026-09-13-push-qualification.json

feat(pre-push-safety-net): enforce evidence eligibility and record blocked qualification

2026-09-13 19:32:43 +02:00

2026-09-13-push-qualification.json.gz

feat(pre-push-safety-net): enforce evidence eligibility and record blocked qualification

2026-09-13 19:32:43 +02:00

2026-09-13-ranker-scope-smoke.json

feat(pre-push-safety-net): preserve exact ranker scope and outcomes

2026-09-13 10:16:22 +02:00

2026-09-13-replay-deadline-repair.json

fix(pre-push-safety-net): stop semantic planning at the shared deadline

2026-09-13 15:11:42 +02:00

2026-09-13-replay-smoke.json

feat(pre-push-safety-net): replay immutable revisions through the full transaction

2026-09-13 14:50:25 +02:00

2026-09-13-reuse-control.json

feat(pre-push-safety-net): reuse compatible artifacts throughout replay

2026-09-13 17:08:49 +02:00

2026-09-13-reuse-final-control.json.gz

feat(pre-push-safety-net): reuse compatible artifacts throughout replay

2026-09-13 17:08:49 +02:00

2026-09-13-reuse-initial-control.json.gz

feat(pre-push-safety-net): reuse compatible artifacts throughout replay

2026-09-13 17:08:49 +02:00

2026-09-14-joern-session-lifecycle.json

test(release): measure repaired reuse and local Joern lifecycle

2026-09-14 13:18:51 +02:00

2026-09-14-node-census-repair.json

fix(contributor-scan): retain declared Gruntfile and report census gaps honestly

2026-09-14 08:07:06 +02:00

2026-09-14-nodegoat-post-census-feasibility.json

test(release): measure repaired reuse and local Joern lifecycle

2026-09-14 13:18:51 +02:00

2026-09-14-nodegoat-post-census-feasibility.json.gz

test(release): measure repaired reuse and local Joern lifecycle

2026-09-14 13:18:51 +02:00

2026-09-16-nodegoat-m1a-recorded-vetoes.json

feat(pre-push-safety-net): record admission vetoes and meet M1a on NodeGoat

2026-09-16 15:31:34 +02:00

2026-09-16-nodegoat-m1a-recorded-vetoes.json.gz

feat(pre-push-safety-net): record admission vetoes and meet M1a on NodeGoat

2026-09-16 15:31:34 +02:00

2026-09-16-nodegoat-m1b-exit.json

test(pre-push-safety-net): record the M1b exit run (task 10.9)

2026-09-16 19:11:03 +02:00

2026-09-16-nodegoat-m1b-exit.json.gz

test(pre-push-safety-net): record the M1b exit run (task 10.9)

2026-09-16 19:11:03 +02:00

2026-09-16-nodegoat-m1b-explained.json

feat(cpg): add a transaction-owned engine session and route it opt-in (M2 11.1, 11.3)

2026-09-16 20:24:34 +02:00

2026-09-16-nodegoat-m1b-explained.json.gz

feat(cpg): add a transaction-owned engine session and route it opt-in (M2 11.1, 11.3)

2026-09-16 20:24:34 +02:00

2026-09-16-nodegoat-m1b-production.json

test(pre-push-safety-net): record the M1b production comparison on NodeGoat

2026-09-16 18:06:39 +02:00

2026-09-16-nodegoat-m1b-production.json.gz

test(pre-push-safety-net): record the M1b production comparison on NodeGoat

2026-09-16 18:06:39 +02:00

2026-09-16-nodegoat-m1b-step1.json

feat(pre-push-safety-net): measure M1b step 1 and record the inventory prerequisite

2026-09-16 16:03:16 +02:00

2026-09-16-nodegoat-m1b-step1.json.gz

feat(pre-push-safety-net): measure M1b step 1 and record the inventory prerequisite

2026-09-16 16:03:16 +02:00

2026-09-16-operation-inventory-smoke.json

feat(pre-push-safety-net): stop declared exclusion vetoing first-party answers (M1b 10.6)

2026-09-16 17:35:00 +02:00

2026-09-16-session-script-execution.json

feat(cpg): add a transaction-owned engine session and route it opt-in (M2 11.1, 11.3)

2026-09-16 20:24:34 +02:00

2026-09-16-session-transport-equivalence.json

test(cpg): prove transport equivalence for every shipped query kind (M2 11.2)

2026-09-16 20:37:47 +02:00

2026-09-17-group-12-final.json

test(contributor-scan): record the group 12 end state (M1b 12.1-12.3)

2026-09-17 13:44:20 +02:00

2026-09-17-group-12-final.json.gz

test(contributor-scan): record the group 12 end state (M1b 12.1-12.3)

2026-09-17 13:44:20 +02:00

2026-09-17-javascript-build-path.json

revert: JavaScript one-start build did not deliver in the push path (M2 11.6)

2026-09-17 09:04:44 +02:00

2026-09-17-m3-php-transfer.json

test(m3): PHP still not established; blocker is the dataflow execution pass

2026-09-18 13:22:05 +02:00

2026-09-17-m3-php-transfer.json.gz

test(m3): PHP transfer not established; the taint batch cannot answer PMPro

2026-09-17 21:01:51 +02:00

2026-09-17-m3-python-transfer.json

test(m3): Python transfer detects all three declared targets (M3 13.1)

2026-09-17 18:17:31 +02:00

2026-09-17-m3-python-transfer.json.gz

test(m3): Python transfer detects all three declared targets (M3 13.1)

2026-09-17 18:17:31 +02:00

2026-09-17-nodegoat-session-transport.json

test(cpg): confirm the session stays alive and halves query time (M2 11.4)

2026-09-17 07:50:04 +02:00

2026-09-17-nodegoat-session-transport.json.gz

test(cpg): confirm the session stays alive and halves query time (M2 11.4)

2026-09-17 07:50:04 +02:00

2026-09-17-per-family-context-effect.json

test(contributor-scan): measure the per-family context effect (M1b 12.2)

2026-09-17 11:36:51 +02:00

2026-09-17-per-family-context-effect.json.gz

test(contributor-scan): measure the per-family context effect (M1b 12.2)

2026-09-17 11:36:51 +02:00

2026-09-17-per-family-context.json

feat(contributor-scan): give each family its own change context (M1b 12.2)

2026-09-17 11:19:30 +02:00

2026-09-17-region-attribution.json

test(ranker): every question now completes on the real case (M1b 12.4)

2026-09-17 14:17:51 +02:00

2026-09-17-region-attribution.json.gz

test(ranker): every question now completes on the real case (M1b 12.4)

2026-09-17 14:17:51 +02:00

2026-09-17-session-runtime-profile-after-build-change.json

revert: JavaScript one-start build did not deliver in the push path (M2 11.6)

2026-09-17 09:04:44 +02:00

2026-09-17-session-runtime-profile-after-build-change.json.gz

revert: JavaScript one-start build did not deliver in the push path (M2 11.6)

2026-09-17 09:04:44 +02:00

2026-09-17-session-runtime-profile.json

test(runtime): measure the seven-class profile with the session; NO-GO (M2 11.5)

2026-09-17 08:12:37 +02:00

2026-09-17-session-runtime-profile.json.gz

test(runtime): measure the seven-class profile with the session; NO-GO (M2 11.5)

2026-09-17 08:12:37 +02:00

2026-09-18-php-batch-diagnosis.json

test(m3): PHP still not established; blocker is the dataflow execution pass

2026-09-18 13:22:05 +02:00

2026-09-19-family-detection-census.json

feat(pre-push-safety-net): census the declared families and fix a sink matched in a comment

2026-09-19 17:39:29 +02:00

2026-09-19-output-encoding-family.json

fix(output-encoding): ask about the server half, and say what the family cannot decide

2026-09-19 20:38:39 +02:00

2026-09-19-php-blocker-open.json

test(m3): record the PHP blocker as open, six hypotheses eliminated

2026-09-19 13:24:26 +02:00

2026-09-19-untrusted-destination-vocabulary.json

fix(semantic-facts): give untrusted destination the clients a Node server calls

2026-09-19 17:55:31 +02:00

2026-09-20-access-control-module-split.json

feat(access-control): carry the obligation across the route-and-DAO split

2026-09-20 08:57:32 +02:00

2026-09-20-collection-read-obligation.json

feat(access-control): model the collection read, by argument shape rather than by name

2026-09-20 12:10:27 +02:00

2026-09-20-every-permissive-setting.json

fix(config): report every permissive setting in a file, not the first one

2026-09-20 16:30:04 +02:00

2026-09-20-php-detection-corrected.json

fix(census): stop hiding findings behind a question-outcome count, and correct the PHP record

2026-09-20 19:22:04 +02:00

2026-09-20-php-runtime-after-graph-fix.json

measure(php): quantify what remains once the malformed-CPG bug is excluded

2026-09-20 18:13:25 +02:00

2026-09-20-sibling-scope-and-identity.json

fix(access-control): decide the sibling scope, and see identity a destructuring spelled away

2026-09-20 15:16:03 +02:00

2026-09-21-ontology-coverage-nodegoat.json

measure(contributor-scan): first ontology coverage run, and it found a family we do not have

2026-09-21 17:19:59 +02:00

2026-09-21-partition-gap-blast-radius.json

fix(scan): carry the partition in both census passes, and PHP completes 120 of 127

2026-09-21 13:45:00 +02:00

2026-09-21-two-languages-defeat-completion.json

measure(m3): two languages in a repository defeat completion, and so defeat admission

2026-09-21 10:57:19 +02:00

2026-09-22-dataflow-batch-cost-curve.json

measure(php): batch size is not the cost, and the curve is sublinear

2026-09-22 08:46:23 +02:00

2026-09-22-php-declared-target-in-scope.json

measure(m3): the declared PHP target is detected, and the replay budget is sized

2026-09-22 19:34:49 +02:00

2026-09-22-php-query-cost-identified.json

measure(php): the query cost is the matched sink count, and the blocker is identified

2026-09-22 09:53:52 +02:00

2026-09-22-sink-weighted-batching.json

measure(contributor-scan): confirm 7.7 costs the small population nothing

2026-09-22 13:56:00 +02:00

2026-09-22-tier-zero-pruning.json

fix(scan): prune a tier-zero question unless ITS partition is the incomplete one

2026-09-22 08:06:15 +02:00

2026-09-23-measured-portioning.json

measure(contributor-scan): 7.9 measured portioning completes 281 of 406, up from 153

2026-09-23 08:51:43 +02:00

2026-09-23-pmpro-sqli-adjudication.json

measure(sqli): adjudicate a 1-in-5 sample of PMPro SQL injection findings

2026-09-23 20:57:02 +02:00

2026-09-23-portioning-vs-hard-cap.json

measure(contributor-scan): 7.8 portioning matches the cap on findings, not on completion

2026-09-23 00:54:13 +02:00

2026-09-24-pmpro-sqli-adjudication-after-path-rules.json

measure(sqli): PMPro precision sample after the path rules -- 2/5, six of seven sampled false positives gone

2026-09-24 03:29:57 +02:00

2026-09-24-pmpro-sqli-adjudication-round3.json

measure(sqli): PMPro 1-in-5 sample after the fifth fix -- 1/5, recall pins intact

2026-09-24 13:36:36 +02:00