measure: access-control feature table and separation screen (952 rows, 74 positive rows, 5 repositories); first ablation manifest registered

Three builds: the first two read 150 rows as empty (a producer bug, fcb43fd), the third is clean. The diff
producer was blind to removed lines (537b61e). Screen: deletion_count 0.89 is the reversed-fix construction;
on rows whose file lost lines entry_point_count 0.83 and caller_count 0.73 are the class signal; semantic covers
262 rows (TypeScript only). Repository folds assigned, leak audit recorded, manifest ac-ablation-1 registered
for decision (five arms, about 35 dollars).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

benchmarks/measurements/2026-10-13-access-control-screen/NOTES.md

@@ -0,0 +1,97 @@
+# Access-control feature table and separation screen, no model calls (2026-10-13/14)
+
+## What was built
+
+`table.py` checks out the head of every change of the frozen access-control training pool (digest 53f9f364:
+5 fix pushes with 48 verified locations, 100 ordinary pushes, 5 repositories), enumerates the candidate rows
+(functions the push touched, plus the verified fix sites of a positive) and runs the four producers over them.
+Rows are labelled 1 at a verified fix site, 0 on an ordinary push. The screen scores every engineered feature alone
+against the label (ROC-AUC) and names the strongest feature per model annotation. Repositories are the clusters,
+and there are five, so the screen is a direction, not a result.
+
+The final table: 952 rows, 74 positive rows (62 functions; a function with two verified spans counts twice),
+878 negative rows, digest 02774edc (152a72ce with folds). Producer coverage: diff 952, source 933, semantic 262
+(the semantic producer handles TypeScript and JavaScript only; Java has no obligations support and PHP is
+unsupported), taxonomy 952 (a constant: it renders the family list, it is not a row feature).
+
+## It took three builds, and the first two printed plausible numbers
+
+1. **Build 1** (952 rows, source ok 783): the screen printed AUCs. 150 rows, 16%, every positive of trigger.dev
+   among them, had read an empty file although git holds the blobs. The gate `source == 0` did not fire because
+   most rows were fine.
+2. **Build 2**, after adding a checkout proof (`git status` clean after the sparse checkout is undone) and an
+   unread-row gate (every unread row checked against `git cat-file -s`): the gate refused the table, the same
+   150 rows. The checkout proof never fired, so the checkout was not the cause.
+3. The cause was a producer bug: the entry-point scan read every product file of the tree, one empty file raised
+   `empty_source`, the exception escaped the cached property and failed the source producer on every row of
+   the change, re-reading the whole tree per row (fcb43fd). **Build 3**: 0 unread rows, source ok 933; the 19
+   remaining failures are files the push deleted (FileNotFoundError, 16) and three unparseable tsx files.
+
+A producer that fails on every row of whole changes while the same files read fine by hand is a producer bug,
+not disk or concurrency. The memory note carries it.
+
+## The diff producer was blind to the shape of this class
+
+A positive push here is the fix replayed backwards (base = fix commit, head = its parent). A fix that adds a
+permission check becomes a push that only removes lines, and the diff producer kept only hunks with added lines
+at head. In build 1, 38 of 74 positive rows had no hunk in their file at all and 69 of 74 were "not on a changed
+line", while 60% of ordinary rows were: the hunks annotation separated the labels by construction and showed
+the model nothing at the fix site. The producer (version 2, 537b61e) now records the head lines after which the
+push removed code; distance and on_changed_line count them; the model's hunks input renders them as
+`removed_after_line`; `diff.deletion_count` is an engineered feature. In build 3, 66 of 74 positive rows have a
+deletion anchor in their file and 12 are on a changed line.
+
+## Screen (build 3, `separation.csv`, `screen.csv`)
+
+| Annotation | Strongest feature | AUC | rows |
+|---|---|---|---|
+| hunks | diff.deletion_count | 0.89 | 952 |
+| obligations | semantic.obligations_undischarged | 0.76 | 262 |
+| callers | source.caller_count | 0.72 | 933 |
+| entry_points | source.entry_point_count | 0.71 | 933 |
+| roles | semantic.sinks | 0.17 | 46 |
+| function_text | source.span_lines | 0.36 | 933 |
+| change_summary | source.is_module | 0.36 | 933 |
+| def_use | semantic.binds | 0.36 | 262 |
+
+Read with care:
+
+- **deletion_count (0.89) and the low hunk_lines (0.15) are the construction, not the class.** Reversed fixes are
+  deletion-heavy pushes with few added lines; ordinary pushes are not. Any feature that says "this push removed
+  code and added little" separates these labels. `separation_deletion_rows.csv` repeats the screen on the 235
+  rows whose file lost lines (66 positive, 169 negative, all 5 repositories): there deletion_count drops to 0.71,
+  hunk_lines and hunk_count stay at 0.12 and 0.14, and **entry_point_count rises to 0.83, caller_count stays at
+  0.73**. Those two are the class: the verified access-control sites are web entry points with callers.
+- **obligations_undischarged (0.76) rests on 8 positive rows from one repository** (trigger.dev; the semantic
+  producer covers no Java or PHP). Not evidence yet.
+- **roles** scored 46 rows. The semantic annotation is absent for 80% of this pool.
+- `leak_audit.csv` is the same screen over the folded table (`ablation.leak_audit`), kept for the ablation record.
+
+## Folds and the first ablation manifest (registered, not run)
+
+Folds are repository-grouped (`ablation.assign_folds`, k=5, seed 0), one repository per fold: 0 p4-plugin (6
+positive rows), 1 dependency-track (4), 2 job-import (2), 3 shopper (54), 4 trigger.dev (8).
+`ablation.json` registers the first manifest in `~/ousast-results/hunt/ac-ablation-1/` for a decision: arms
+full, no-hunks, no-entry_points, no-callers, no-semantic; compile folds 0-2 (the Jenkins plugins), holdout 3-4
+(shopper and trigger.dev, 62 of the 74 positive rows); metric search_outcome at tolerance 10; one repeat; 8
+dollars and 1,500 calls per arm. The baseline cost 6.13 dollars for one arm over this pool, so this manifest is
+about 35 dollars. With 5 positive pushes `analyse` will report `dry_run`; the location-level numbers are what it
+can say.
+
+## Open questions for the maintainer
+
+- Whether the hunks annotation should be ablated at all while positives are reversed fixes, or whether the
+  ordinary draw should be stratified by "push removed lines" so the construction signal cancels.
+- Whether to extend the semantic producer to Java (obligations) and PHP before the next class round; 80% of
+  this pool gets no roles, def_use or obligations today.
+- The 21 access-control sources dropped by the disk guard and batch 2 (27 changes, 69 locations, awaiting the
+  spot-check) would take positives past 40.
+
+## Files
+
+- `table.py`: the build (worktree per change, full checkout, producers, labels, gates, screen).
+- `summary.json`, `separation.csv`, `screen.csv`, `separation_deletion_rows.csv`, `leak_audit.csv`, `log.json`
+  (per-change seconds and rows; the slowest ordinary change of shopper took 2,407 s in the repository-wide
+  caller scan), `ablation.json`.
+- Table: `~/ousast-results/valid/train/work/ac-screen/table.jsonl` (94 MB) and `table_folds.jsonl`; builds 1 and
+  2 kept beside it as `ac-screen-v1` and `ac-screen-v2-refused`.

benchmarks/measurements/2026-10-13-access-control-screen/leak_audit.csv

@@ -0,0 +1,18 @@
+column,auc,n,note,annotation
+diff.distance,0.7519618922057947,935,,hunks
+diff.on_changed_line,0.26616080773256173,952,,hunks
+diff.hunk_count,0.26391368589546266,952,,hunks
+diff.hunk_lines,0.1529581973773318,952,,hunks
+diff.deletion_count,0.8854275687988672,952,,hunks
+source.span_lines,0.36404681748104334,933,,function_text
+source.is_module,0.3577462794575717,933,,change_summary
+source.caller_count,0.7206525501054023,933,,callers
+source.caller_files,0.7166567032690432,933,,callers
+source.entry_point_count,0.7070918415505144,933,,entry_points
+semantic.sources,0.6976744186046512,46,,roles
+semantic.sinks,0.17441860465116277,46,,roles
+semantic.sanitizers,0.4534883720930233,46,,roles
+semantic.binds,0.35999015748031493,262,,def_use
+semantic.calls,0.5642224409448819,262,,def_use
+semantic.obligations,0.7541830708661417,262,,obligations
+semantic.obligations_undischarged,0.7620570866141732,262,,obligations

benchmarks/measurements/2026-10-13-access-control-screen/log.json

@@ -0,0 +1,737 @@
+[
+ {
+  "id": "jenkinsci-dependency-track-plugin-673cc48b4f75ebf5",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "vulnerability",
+  "seconds": 0.79,
+  "rows": 4
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-0d167208df6d423f5170101165abbc9deb005841",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 3.55,
+  "rows": 9
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-123a5dc0150fc0e45e6e27cb36c55108dd4b2c26",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.29,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-16f7f643651266f2e0d10901152c81261ee44225",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.99,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-1efa1410b701872981b903d7857d2406913d5b97",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.64,
+  "rows": 4
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-2c7d0738ad3e55626021a48d7e23d4aa18163ac0",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.35,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-3b1a2ea9082a3a0e7a8cfaa9a448ae9ad23e528b",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.56,
+  "rows": 4
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-432f1df244aaa3a6ca02c1412a597d2dc97db5aa",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.49,
+  "rows": 3
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-50848c0edeeb99d3803bed63324a48754228a159",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.39,
+  "rows": 3
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-670f5988f7939f18343c3547d6d363e1600cc70b",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.79,
+  "rows": 6
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-694379976b192eaaf0275e5cbfe95e8ca898d06b",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.25,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-6d66c56feb5dc23ecc5f8f73562e06ec0c8e65ff",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.44,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-9203f702005c090bff7898ce91cafac9a26a8b7d",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.7,
+  "rows": 3
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-9f5ddda50d037d1185953377896c80def8d41e57",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.53,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-bbc913e3c9878ae3a56712f28cdeb6464bf5daee",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.9,
+  "rows": 3
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-bbe8342f2d8f4cdf4566f71cc7ed87b695f3a5c4",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 2.07,
+  "rows": 7
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-bcfa2ae89b8627657c1d2e9ddd71d5b285f49f25",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 3.92,
+  "rows": 16
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-def475c90bc85f709dc22841ffcb265c48afc5fd",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.42,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-e74042b706b640355643be4c9ab8aa0f4dddb5c4",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.89,
+  "rows": 4
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-e7e4409570ee0d2eb31ba7ade5ecedb5b817f436",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 0.86,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-dependency-track-plugin-ordinary-fd311a6cef414e39b08f528608b9f1753b0b9a77",
+  "repo": "jenkinsci/dependency-track-plugin",
+  "kind": "ordinary",
+  "seconds": 1.92,
+  "rows": 11
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-07e0e7023a46cf4d",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "vulnerability",
+  "seconds": 0.3,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-044a2e819b271dade0106de10291b2fed45836df",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.54,
+  "rows": 6
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-1bcce383bc2977e8ed5170eea2fdab408edbe682",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.44,
+  "rows": 7
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-23feb819c2e2778f60e8a7a8096b92d4086ff3ea",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.12,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-445573104f2dda90a133308d574c2fed0828d081",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.97,
+  "rows": 11
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-4e2cb93e8de3b3ffacddbf72bd157cae1ef67f46",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.41,
+  "rows": 7
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-4f97016985ec2a5080e0868261f60a644604f483",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 3.0,
+  "rows": 58
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-5032d8339ef734759812af9ee37b01823c0de3f4",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.12,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-66b17d16ecd2f108912351ddb2e15a637520b0a8",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.41,
+  "rows": 8
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-6be5a096043a669dfa535e22302c8d98d7ce5233",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.39,
+  "rows": 5
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-7e5f489c88805d8e25ade0d1c81d4fc7149abc7e",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.26,
+  "rows": 6
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-b5334bc4a6be08abd0e44e443a340d1b6f135c72",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.7,
+  "rows": 8
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-bb511a1d2e44a7c206d09e43daec7af1d9a78b2e",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.12,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-dc0abc56f335aa048192b1ff173da6a036d54322",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.11,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-e29917d2c7eb818e4ffda664a932b942e2aee2b0",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 1.27,
+  "rows": 25
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-e52d913aad5f83cb6c35c6494f15d77c7bce72db",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.13,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-ec1711f1bd0ab146225bca3057a17ae7a4154828",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.25,
+  "rows": 4
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-ec2103f6375fadb68375fd782e4de6d3daf1fccb",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 1.04,
+  "rows": 15
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-f10bc2f0443d6ae6d49217a4c0e6271451fd0ce5",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.27,
+  "rows": 4
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-f10f5a83cc4070533bdbe2be4e476e2220165b9d",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.19,
+  "rows": 3
+ },
+ {
+  "id": "jenkinsci-job-import-plugin-ordinary-ffdeb8e959d01e145dd1c970595f929cb562f360",
+  "repo": "jenkinsci/job-import-plugin",
+  "kind": "ordinary",
+  "seconds": 0.14,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-5022afedadfe8449",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "vulnerability",
+  "seconds": 6.27,
+  "rows": 6
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-0cfd50a9fdc86cfe3e9a770038d273ea382f59bb",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 3.38,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-171dde20917615e63a6120251d7cb367f9d52992",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 2.0,
+  "rows": 3
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-18e4708722279a3a07eafb35f1419dfed6c7dbd1",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 1.37,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-1a38001c4dc8e2424479d44f95f6e25e25df4959",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 3.75,
+  "rows": 3
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-1d19356cb9f2d8e9b72a129e5719cca45d89821f",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 1.0,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-2ec2dccaa9e6ff318ccada47f2af691170aaa7be",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 1.83,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-30fb98f8e26adaa33762b526cdd443aa6122ac5b",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 1.06,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-46fb68bfea093c51f841bc6bdec9676bc4d468c7",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 10.1,
+  "rows": 14
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-4cb6dab4456e1afb3de1230cfacdce3e162fe8ea",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 1.8,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-65c18b225338e93837ef6882ad79cd9773e6e2c4",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 2.04,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-65fbed70350a4435ec091390d92722bbf7cc1c68",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 1.81,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-7b512b08c584a76d57f4401634f8e9736a365223",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 1.05,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-8a67359f1a69264da35c9e837273a649b3345b9d",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 0.83,
+  "rows": 1
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-a33e38e16ececd6e093eb213a9b8a3ab8352f937",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 28.32,
+  "rows": 58
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-aa910d56936a308600bdad568e2ab23ada636b7d",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 2.21,
+  "rows": 6
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-b2f7b1ba3eea1a81370dec019ebff29931875182",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 9.2,
+  "rows": 23
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-b42c75d6823b9e67bd968679494d1090f5d77941",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 2.31,
+  "rows": 5
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-dca0f55c69c487611689e72230e211ef4b760d9e",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 2.13,
+  "rows": 2
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-f4c24f7964bd1de847ace7b43546be81e51d2ccc",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 13.41,
+  "rows": 23
+ },
+ {
+  "id": "jenkinsci-p4-plugin-ordinary-f74fb378f0e7ce258aae449dc70f35820a1b11ed",
+  "repo": "jenkinsci/p4-plugin",
+  "kind": "ordinary",
+  "seconds": 11.4,
+  "rows": 12
+ },
+ {
+  "id": "shopperlabs-shopper-02254beed509364c",
+  "repo": "shopperlabs/shopper",
+  "kind": "vulnerability",
+  "seconds": 182.45,
+  "rows": 54
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-0db4faf747f7bc4d9eadacbfa0dcbd2620251ca5",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 1.14,
+  "rows": 1
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-13037bfd039904801dd8103dec3d0e720c547b94",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 5.82,
+  "rows": 2
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-17778cdd2c24a50d2fb29dae4a3aab5390b1b996",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 2.2,
+  "rows": 1
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-1aa9a4c2f471f00914a230b8168b3926f39837a7",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 1.4,
+  "rows": 1
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-31f33b1631254639836ce06ad6b078e6cfb0b13a",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 1.48,
+  "rows": 1
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-44809499ef406c9e199965fff6d124434857a36b",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 13.72,
+  "rows": 8
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-4628430c0a70506e15fba209557606f860566598",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 55.13,
+  "rows": 37
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-5394b647023599eb9d9a40f7b9ccc434cd8136a3",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 76.69,
+  "rows": 63
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-5a079da974a3d7f6e7ca0de3c6ca6b0849e304a2",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 30.55,
+  "rows": 3
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-7e56ccd817b0dc835919fac74db792e15ee62c1d",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 4.2,
+  "rows": 4
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-7eaab898afd90140f2162149585faf2642e18987",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 6.63,
+  "rows": 4
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-7f7e6bdb67509a50fae95f13a829494a8a3ab7b4",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 2.28,
+  "rows": 6
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-886e671270b21aed8ab851b5b82a53fc6e87fbc6",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 10.53,
+  "rows": 6
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-9313ed20032199988f571d7e655255819f7f6f04",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 11.97,
+  "rows": 10
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-a0c5bd3a05a0fef58b9043273d6062124a3ecf8e",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 48.67,
+  "rows": 29
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-a5a2760dc0c63872b55afec0e3d452df7989393b",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 2444.46,
+  "rows": 27
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-b178fe37da1c31f9f0057f35fdf4145936a25138",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 7.55,
+  "rows": 3
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-ceaea6a078025d4a7634d2646d27dde3f13b6f58",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 6.82,
+  "rows": 2
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-fada4250aaf3350c1db92698259c54f9ae6b4bb0",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 280.17,
+  "rows": 2
+ },
+ {
+  "id": "shopperlabs-shopper-ordinary-fcc3888151b9c43843c01f631870dfe7e8fd6329",
+  "repo": "shopperlabs/shopper",
+  "kind": "ordinary",
+  "seconds": 2.04,
+  "rows": 1
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-0eb72665259b13c2",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "vulnerability",
+  "seconds": 122.27,
+  "rows": 8
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-0e63f8317e7c8b472d8c9c7bb36ad3986d0238c7",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 91.01,
+  "rows": 19
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-22a7e33df74729c0a2dbd4b12d30c60378ec677f",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 25.82,
+  "rows": 2
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-2d569f64f8027bd2dde80ce14042542200f8fc7d",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 172.36,
+  "rows": 10
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-35298ac357d4717203cc181986d29109ae90e6cd",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 88.91,
+  "rows": 7
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-409388365eaaa9bb61eb4a9060e1fb46c8acc6a2",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 40.13,
+  "rows": 7
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-417ab876e3177ced4a4aa46c3de664b10950c6fe",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 46.5,
+  "rows": 7
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-478adb5d74d36f860f6f22d28ee18a299d692f45",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 41.79,
+  "rows": 3
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-4c42f6cc0b183a1435ec39d831892ca58b415008",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 357.99,
+  "rows": 33
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-55d85d0b2360cdeaecbf94f9fea8404cc5853acb",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 33.66,
+  "rows": 5
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-6be6c1816c2dd2dc09fc361f40be6af2a07c2de2",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 385.36,
+  "rows": 22
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-771937adf5d6b72943a006ec86b5bfe1057804e7",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 35.35,
+  "rows": 4
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-7ab437c8ad8ef1db66132157056cb59d94e763a8",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 741.52,
+  "rows": 39
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-87448ccaf23b7f99f505f953f2ae6fd6c3f3975c",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 27.53,
+  "rows": 3
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-9f29eb209042c4682f3fab90b4036dba300ea892",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 123.69,
+  "rows": 10
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-bfa902bd1822812e8f0aaf0d6831414dc70c13aa",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 354.28,
+  "rows": 45
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-cd518683c8c7830176bb1f9efdcbfe4b03d79457",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 13.76,
+  "rows": 1
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-cf4aa7e918eb6c3634b7dc8827b21be246519591",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 75.25,
+  "rows": 7
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-efcb89ac26984aac8e98f9c6ccb9a902c0483881",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 10.8,
+  "rows": 1
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-f78261a606362cf5480679e7286585367d744d08",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 327.86,
+  "rows": 18
+ },
+ {
+  "id": "triggerdotdev-trigger.dev-ordinary-fdec5a09a72fae962f3db9b3dcf8af08d34d06ad",
+  "repo": "triggerdotdev/trigger.dev",
+  "kind": "ordinary",
+  "seconds": 30.7,
+  "rows": 1
+ }
+]

benchmarks/measurements/2026-10-13-access-control-screen/screen.csv

@@ -0,0 +1,9 @@
+annotation,strongest_feature,auc,n
+callers,source.caller_count,0.7206525501054023,933
+change_summary,source.is_module,0.3577462794575717,933
+def_use,semantic.binds,0.35999015748031493,262
+entry_points,source.entry_point_count,0.7070918415505144,933
+function_text,source.span_lines,0.36404681748104334,933
+hunks,diff.deletion_count,0.8854275687988672,952
+obligations,semantic.obligations_undischarged,0.7620570866141732,262
+roles,semantic.sinks,0.17441860465116277,46

benchmarks/measurements/2026-10-13-access-control-screen/separation.csv

@@ -0,0 +1,18 @@
+column,auc,n,note,annotation
+diff.distance,0.7519618922057947,935,,hunks
+diff.on_changed_line,0.26616080773256173,952,,hunks
+diff.hunk_count,0.26391368589546266,952,,hunks
+diff.hunk_lines,0.1529581973773318,952,,hunks
+diff.deletion_count,0.8854275687988672,952,,hunks
+source.span_lines,0.36404681748104334,933,,function_text
+source.is_module,0.3577462794575717,933,,change_summary
+source.caller_count,0.7206525501054023,933,,callers
+source.caller_files,0.7166567032690432,933,,callers
+source.entry_point_count,0.7070918415505144,933,,entry_points
+semantic.sources,0.6976744186046512,46,,roles
+semantic.sinks,0.17441860465116277,46,,roles
+semantic.sanitizers,0.4534883720930233,46,,roles
+semantic.binds,0.35999015748031493,262,,def_use
+semantic.calls,0.5642224409448819,262,,def_use
+semantic.obligations,0.7541830708661417,262,,obligations
+semantic.obligations_undischarged,0.7620570866141732,262,,obligations

benchmarks/measurements/2026-10-13-access-control-screen/separation_deletion_rows.csv

@@ -0,0 +1,18 @@
+column,auc,n,note,annotation
+diff.distance,0.7289761520530752,235,,hunks
+diff.on_changed_line,0.31410256410256415,235,,hunks
+diff.hunk_count,0.1386049847588309,235,,hunks
+diff.hunk_lines,0.12058454366146672,235,,hunks
+diff.deletion_count,0.708086785009862,235,,hunks
+source.span_lines,0.3603639949793796,235,,function_text
+source.is_module,0.41420118343195267,235,,change_summary
+source.caller_count,0.7297382105074413,235,,callers
+source.caller_files,0.7217141832526447,235,,callers
+source.entry_point_count,0.8280437511206741,235,,entry_points
+semantic.sources,,7,one class or empty,roles
+semantic.sinks,,7,one class or empty,roles
+semantic.sanitizers,,7,one class or empty,roles
+semantic.binds,,24,one class or empty,def_use
+semantic.calls,,24,one class or empty,def_use
+semantic.obligations,,24,one class or empty,obligations
+semantic.obligations_undischarged,,24,one class or empty,obligations

benchmarks/measurements/2026-10-13-access-control-screen/summary.json

@@ -0,0 +1,24 @@
+{
+ "producer_rows_ok": {
+  "diff": 952,
+  "source": 933,
+  "semantic": 262,
+  "taxonomy": 952
+ },
+ "table_digest": "02774edc4c07dcc92b2fcf04ed342e7d0bc8afe761c808fef2d11b473bd99271",
+ "rows": 952,
+ "labelled": {
+  "positive": 74,
+  "negative": 878
+ },
+ "repositories": 5,
+ "changes": 105,
+ "positive_repositories": 5,
+ "source_failures": {
+  "FileNotFoundError": 16,
+  "ValueError": 3
+ },
+ "unread_rows": 0,
+ "positive_rows_on_changed_line": 12,
+ "positive_rows_with_deletions": 66
+}