Author SHA1 Message Date
Marius CiepluchandClaude Opus 5.5 e13ce35a75 engine: admit TypeScript (jssrc2cpg, JavaScript taint specs via one alias), .mts/.cts, mixed JS/TS builds one CPG
Implemented with Codex. Not merged: wall-clock deadline tests failed at host
load 47; to be re-checked against main at normal load.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-04 10:34:50 +02:00
Marius CiepluchandClaude Opus 5.5 20777d8863 chore(audit): module audit gains learn.slice
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 17:04:33 +02:00
Marius Ciepluch 77e47ac7d3 Merge branch 'graph-slice' 2026-10-03 16:58:37 +02:00
Marius CiepluchandClaude Opus 5.5 094f6d226c learn: exp-005 graph slice -- inconclusive (within-pair AUC 0.762 -> 0.746, CI [-0.063, 0.032]); unsure rate 0.29 -> 0.24; fill \$1.59
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 16:58:37 +02:00
Marius CiepluchandClaude Opus 5.5 0acbc7ec41 learn: exp-005 graph slice -- slice in the prompt (opt-in), program-arm fill, manifest and units
Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 15:06:08 +02:00
Marius CiepluchandClaude Opus 5.5 1b2c7993a7 learn: in-function def-use slice from the tree-sitter layer + leak audit (\$0)
slice.py renders source -> step -> sink lines with fixed wording for flow,
no-flow and not-analysable; the prompt option is default-off (byte-identical
default prompt). semantic IR: JS/C declarations count as assignments, augmented
assignment keeps its target (gate, map_gate, pair_gate outputs unchanged).
Audit over exp-004's 501 units: 59 flow, 161 no-flow, 281 not analysable;
has-flow within-pair accuracy 0.52, pooled AUC 0.55 -- no leak. Fixes a
numbering bug that rejected every excerpt with a blank line. With Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 14:56:52 +02:00
Marius CiepluchandClaude Opus 5.5 463fe1503a learn: build_features reads v2 _a/_b scans and TypeScript repository-scan witnesses; --dry-run
Population v2 scans are named <case>--<side>_a/_b.json; the script looked only
for <case>--<side>.json and recorded those examples as engine not-run. The two
repeats combine symmetrically for both sides. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 14:41:55 +02:00
Marius CiepluchandClaude Opus 5.5 2836b78819 merge(learn): exp-004 -- inconclusive, pooled top-30 precision 0.90; experiment fill command; registration fix
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 14:29:22 +02:00
Marius CiepluchandClaude Opus 5.5 6d9850578e learn: dry-run estimate from measured cached-response cost; docs: exp-004 result; tidy instrument check
The estimator priced all input at the full rate (\$5.58 estimated vs \$1.24
spent); it now uses the family mean billed cost of cached responses when usage
is recorded. Implemented with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 14:23:36 +02:00
Marius CiepluchandClaude Opus 5.5 b50f7afccb learn: exp-004 run -- inconclusive, no pooled t_block; fill \$1.24 of a \$2 ceiling, replay \$0; instrument check recorded
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 14:18:17 +02:00
Marius CiepluchandClaude Opus 5.5 90a2642019 learn: registered() returns the frozen registration, not the first row
A pending-then-frozen registration left two rows and rows[0] could be the stale
pending one, so fill and run refused a properly frozen experiment. Implemented
with Codex.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 12:42:39 +02:00
Marius CiepluchandClaude Opus 5.5 8aeca972c0 learn: freeze exp-004 units (501 units, 390 paired, 220 groups) after the negatives harvest; audit gains learn.fill
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 12:36:02 +02:00
Marius CiepluchandClaude Opus 5.5 8d14b0c13f learn: experiment fill -- fill a rule experiment's response cache with a hard ceiling, dry-run estimate
Implemented with Codex; review fix: the bounded client keeps the configured
thinking setting so a fill answers as the run would.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 12:35:27 +02:00
Marius CiepluchandClaude Opus 5.5 212a391872 merge(learn): cache fill and exp-004 registration -- pooled block gate with family floor, Req 6.4
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 09:53:10 +02:00
Marius CiepluchandClaude Opus 5.5 da91ef0dd9 exp-003: supplementary all-units replay after the cache fill; exp-004 registered
Supplementary analysis, not a new decision: exp-003's verdict stands.
The 142 units excluded for an uncached response were filled with the same
six incumbent programs, parameters and request keys, on exp-003's
937-example memory (the harvest's 196 rows and its 2026-10-03 embeddings
left out; a zero-cost replay first reproduced exp-003 exactly: 211
decided, 1056 replayed, $0.582876). Fill: MeteredClient, ceiling $0.99
under the approved $1.00, 697 client calls, $0.331759, no provider error.

Replay on all 353 units (244 paired, 146 groups): 0 client calls, $0.0,
1768 responses replayed; no arm flags any unit, the registered rule reads
inconclusive. Calibration holds only for untrusted destination (ECE
0.0454) and arm A still finds no BLOCK point there.
benchmarks/experiments/exp-003-precision-bound-blocking/result-complete-cache.json

exp-004 registered in s3://sast-memory (manifest 87fceb65..., pin
2ad9c1d, units pending): benchmarks/experiments/exp-004-pooled-block-gate/
registration.json. docs/decision-engine.md: both, briefly.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 09:47:20 +02:00
Marius CiepluchandClaude Opus 5.5 2ad9c1db7a learn(exp-004): Req 6.4 pooled BLOCK gate with a family floor; manifest before units
Maintainer decision 2026-10-03: BLOCK precision is shown pooled across
families. Req 6.4 (amended; 7.4 annotated): one t_block on a per-family
Platt-normalised score (calibrate.fit_map, training folds, inner
cross-fit), Wilson 95% LB >= 0.95 over all families' training flags; a
family blocks only with >= F = 10 held-out flags and zero errors, else
advisory; t_quiet per family keeps >= 90% of training paired positives.

evaluate: normalised, pooled_threshold, quiet_threshold,
pooled_block_gate, family_floor, apply_floor. experiments: rule
pooled_block_gate (normalise: platt, floor, recall), decide_pooled on the
pre-floor pooled held-out LB, floor table, quiet_recall and after-floor
totals in analyse_rule; exp-003's analysis is unchanged.

plane/experiments/exp-004-pooled-block-gate.yaml: procedure and decision
rule registered now; units frozen at run time from the post-harvest
memory (units_status pending until then).

Tests (tests/test_learn_rules.py): pooled threshold where no family alone
reaches the bound, no held-out label moves a threshold or the map, a
family with one held-out error cannot block, t_quiet and no blind
silencing, the pooled verdict, the exp-004 manifest.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 09:40:58 +02:00
Marius CiepluchandClaude Opus 5.5 5b2bd9bb91 merge(learn): negatives harvest -- 98 real fix pairs with surviving functions (79 post-cutoff), paired groups 151 -> 249
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 08:29:32 +02:00
Marius CiepluchandClaude Opus 5.5 50c93edfb8 learn: 98 advisory-fix pairs whose function survives the fix (advisory-fixes-2)
Real (vulnerable function, fixed function) pointer pairs for the six compiled
families, mined at zero model cost from reviewed GHSA advisories (pip, npm,
maven, composer), newest first: 79 of 98 published after 2026-06-01
(post_cutoff = true). Each pair: one linked one-parent fix commit, <= 4 source
files / 120 lines, the function holding the changed code lines declared once on
both sides with a changed body; every function diff read and kept only when the
named function holds the fix (145 reviewed, 100 kept); 2 dropped by the
reserved-population test; permissive licences only, no repository named
elsewhere under benchmarks/.

New label source advisory-fixes-2; the advisory-fix tests run over both
batches. Memory 937 -> 1133 static examples (98 paired groups added);
positives-only groups stay 99 (vibe-py/dev-php/v2 data gaps), their share of
positive groups in the six families 0.40 -> 0.29. Embeddings: 7 calls,
$0.0014. No chat-model call. Record:
benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 08:23:42 +02:00
Marius CiepluchandClaude Opus 5.5 6ecf0a304e merge(learn): exp-003 precision-bound blocking -- inconclusive; the Wilson bound needs 73 clean flags, injection has 38 paired positives
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 08:21:27 +02:00
Marius CiepluchandClaude Opus 5.5 b31c85678f exp-003: result -- inconclusive, no BLOCK rule flags a held-out unit
Zero-cost replay of the six slice-2 programs: meter 0 client calls, $0.0;
211 of 353 units scored, 142 excluded for an uncached response (path lost
all 39). Arm A (calibrated) fails calibration (ECE 0.077 access control,
0.1067 injection); arm B (Wilson LB >= 0.95, nested) chooses no threshold on
any fold, as the registration's power note predicted (73 clean flags needed,
38 paired injection positives); secondary arm C flags nothing either. The
instrument check shows real scores; the zero is the rule's.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 08:15:22 +02:00
Marius CiepluchandClaude Opus 5.5 7bb97ad0bc exp-003: registration, before the run
Registered in s3://sast-memory at dd7ebc1: manifest 5917f9a1, units 338de998
(353 units, 146 groups). Primary metric, decision rule and the threshold
selection are fixed in the manifest; the power note says arm B cannot reach
a Wilson LB of 0.95 on this data (73 clean flags needed, 38 paired
positives at most).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 08:03:22 +02:00
Marius CiepluchandClaude Opus 5.5 dd7ebc119b learn(exp-003): decision-rule experiments and the precision-bound BLOCK arm
kind: rule experiments score once from the response cache behind a
zero-ceiling meter (a miss is excluded and counted, never asked) and apply
each arm's BLOCK rule to the same held-out scores. Arm B chooses a per-family
raw-score threshold on the other outer folds only (lowest t with Wilson 95%
LB >= 0.95); a test fails if a fold's own labels move its threshold.
Manifest and frozen units (353 units, 146 groups, 244 paired) for exp-003.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 08:02:51 +02:00
Marius CiepluchandClaude Opus 5.5 b24eeb15f3 merge(plane-on-kubernetes): groups 1-2 -- cluster profile, no host literals, delivery through the S3 store
Kind proofs: records 01 and 02. Task 2.6 (receiver removal) is paused while the
safety net and the decision engine come first. deployment.md conflicts resolved
in the plain-language style; corrects the RuntimeClass claim (Substrate runs
runsc inside its worker pod).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 07:57:13 +02:00
Marius CiepluchandClaude Opus 5.5 c8b418c34c plane(2.5): kind proof and record 02 -- artifacts through the store, one TLS rule
Live on kind with the runner image that carries 2.2
(ousast-runner@sha256:d2a55293...; templates and validation-46 Tasks repinned,
128 lines; kind-images.json updated): the smoke Run on store delivery finished
in 26 s and 24 s; the tar was PUT to runs/ax-e2e-smoke/facts/output.tar
(10240 B) and collected 2 s later; read back from the store its members
(facts.json 451 B, summary.json 238 B, units.jsonl 291 B) equal the extracted
files; facts.json, summary.json and done.json (149 B) re-put, state.json
(350 B) mirrored; a rerun from an empty results root seeded state.json from the
store and finished in 2 s without resubmitting. The actor's egress policy read
back live: one tlsPassthrough rule on 443 naming the store host, nothing else.
`ousast plane status ax-e2e-smoke` printed the table each time. Record:
benchmarks/measurements/2026-10-02-k8s-02-store-delivery/record.json with the
smoke outputs, store read-back and policy beside it. Task 2.6 (receiver
removal) is paused by the maintainer's reprioritisation.

Gates: pytest 2031 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0. (The first run, at 0.97 GiB free disk, failed 11 tests and errored 5
on the FileStore's 1 GiB free-space refusal; rerun after the disk was freed.)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 07:45:19 +02:00
Marius CiepluchandClaude Opus 5.5 432fd2947e merge(pre-push): operational readiness increment 1 -- never block, never hide, quick tier, new files, background engine, install
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-03 07:29:17 +02:00
Marius CiepluchandClaude Opus 5.5 2ec8071c26 pre-push(17.7): prove it on the audit's three repositories
Rerun records, serve-static and unrolled/secure (same planted commits) with
this build installed from its wheel: 6 replays and 10 hook pushes, all exit 0
in 0.69-1.21 s against the 30 s deadline, input read proven per run. The
planted JS eval in a new file surfaces in the quick tier on the host and, in
one Docker engine run (190.9 s, deadline 300), as two advisory engine findings
classified new/file_added_in_head. The planted Python %-formatted SQL is still
missed (no quick rule, no host engine): recorded as a coverage gap, no rule
edited. Go is named as not covered. The run caught a wrong install hint in
the hook (the package is not on PyPI); corrected here.
Record: benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 23:02:32 +02:00
Marius CiepluchandClaude Opus 5.5 591818d5fc pre-push(17.6): install, Docker wrapper and first-run docs
`ousast pre-push install|uninstall [REPO] [--force] [--docker]` ships in the
wheel with the hook as package data; it honours core.hooksPath, refuses an
existing hook unless --force (which chains it as pre-push.before-ousast), and
uninstall restores the chained hook. ops/ousast-docker is a hook-callable
script: the repository read-only and the artifact directory at their own
paths, stdin passed through, no network, --memory 3g, the GHCR 2.0.1 image by
default, advisory exit unless blocking. Help text for every pre-push flag; a
first-run guide, settings, skip-reason glossary, CI recipe and full --help in
docs/scanning.md; README tiers 1 and 2 pull the published image. The image
was not rebuilt: 1.1 GB free, below the 4 GB threshold. Requirements 7.4, 9.6.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:51:22 +02:00
Marius CiepluchandClaude Opus 5.5 66357ffea1 pre-push(17.5): say when the engine cannot finish, and offer it in the background
--engine inline|background|off (hook: OUSAST_PUSH_ENGINE). background returns
after the quick tier and starts one detached explicit replay of the same base
and head (push/background.py), one at a time per artifact directory; its
artifact lands under <artifact dir>/engine/<repo key>/<head>.json and the next
run prints "Engine result for <head> from the previous push" once. A deadline
miss says so in one line naming the deadline and the background option. No
engine speed work. Requirement 9.5.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:31:14 +02:00
Marius CiepluchandClaude Opus 5.5 511de4b75f pre-push(17.4): a file new in head is new, not comparison unknown
ChangeContext records the paths Git reports as added. An entailed,
undischarged operation in an added file, with complete head evidence and no
identical operation in any base answer, is classified new
(file_added_in_head) instead of base_incomplete; admission accepts that reason
as change evidence. Unpaired add/delete pushes and every other unknown
comparison keep their vetoes. An engine finding whose novelty is established
and whose only vetoes are capability vetoes is shown as an advisory engine
finding, not an alert. Requirement 9.4.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:24:24 +02:00
Marius CiepluchandClaude Opus 5.5 15999a09e1 pre-push(17.3): quick rules as the fast first tier
Before the engine and inside the same deadline, the shipped quick rules run
over the head side of the changed files (push/quick.py) and keep matches on
changed lines or in changed functions. They print as advisory pattern matches,
labeled apart from admitted alerts, are kept in the artifact's quick_tier with
the files and bytes read, and never admit or block. A new branch with no
remote base runs the quick tier over its last commit and says so. The planted
eval in serve-static's new debug.js now surfaces; the planted Python
%-formatted SQL in records does not (a quick-rule coverage gap, no rule
edited). Requirement 9.3.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:18:14 +02:00
Marius CiepluchandClaude Opus 5.5 f555dee640 pre-push(17.2): name every skipped check in plain words
One terminal line per distinct coverage reason (push/explain.py): hex-encoded
paths decoded, per-question sha256 tokens dropped and counted, unknown reasons
still printed; the JSON artifact keeps the raw values. A changed language with
no engine frontend and no quick rules is recorded and printed as "language not
covered: <lang> (...)", in pre-push and in `ousast scan`. The scan report
renders the eight degradations cli.py recorded but never showed, and any
reason without a sentence. The hook forwards OUSAST_COMPARISON_BASE.
Requirement 9.2.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 22:10:33 +02:00
Marius CiepluchandClaude Opus 5.5 bf07cd4d1d pre-push(17.1): never block a push the user did not choose to block
The installed hook exits 0 with one line when ousast (or OUSAST_COMMAND) is
not on PATH, still chaining a prior hook. In Git hook mode without an explicit
--mode blocking, argument/usage errors (an invalid OUSAST_PUSH_DEADLINE, a cache
directory inside the repository, a refused experimental flag) and unexpected
exceptions print one "check did not run" line and exit 0. Blocking mode keeps
its usage errors. Requirement 9.1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:57:42 +02:00
Marius CiepluchandClaude Fable 5.1 605b7af5ba plane(2.4): reconciler on Delivery -- the start body carries the URLs, the store completes the task
_execute builds the task's `delivery {put, inputs}` from Delivery, passes it to
router.starter, grants the store's host in the egress policy, awaits
Delivery.delivered (the tar's exact key in the store), then collect()s:
download, extract with the member checks, re-put the declared artifacts, write
done.json; the outcome is the collected summary.json. Three findings from the
tests: a rerun of an unfinished task completed at once on the previous
attempt's tar, so Delivery.reset(task) deletes output.tar and done.json before
every attempt; a worker thread that raised left the run blocked on its queue,
so the worker now turns any exception into a failed task; a consumer may
declare a producer's summary.json without the producer listing it (the loop
Run does), so declared(run) publishes outputs[] plus every artifact a consumer
names. The receiver thread survives behind OUSAST_DELIVERY=receiver for this
group only (no presigned URLs, the http rule, OUSAST_ARTIFACT_URL in the env);
the default path renders no receiver address into any Task. The fake ax PUTs
to the start body's URL and GETs its inputs from theirs; the reconciler tests
run through the S3-shaped fake store and assert the objects, done.json, the
seeded rerun and the receiver mode. reconciler.py: 549 lines (bound 550).

Gates: pytest 2031 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:49:58 +02:00
Marius CiepluchandClaude Opus 5.5 a656fda2f4 pre-push(17): operational-readiness task group and Requirement 9
Append task group 17 "Operational readiness (2026-10-02)" with one task per
maintainer-approved item (never block, name every skip, quick tier, new-file
classification, engine honesty and background mode, install/Docker/docs, proof),
add Requirement 9 (9.1-9.6, marked added 2026-10-02, maintainer-approved), and
check 13.4 and the parent of 14 as the audit found them done.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:46:25 +02:00
Marius CiepluchandClaude Fable 5.1 c49eae4f30 plane(2.3): egress -- one TLS rule over store host, Git hosts and Model hosts; no http rule
policy_for(task, workspaces, model, store_host, receiver_host=None) emits one
tlsPassthrough rule on 443 over the sorted union of the memory store's host,
the bound Workspaces' https Git hosts and the bound Model's egress hosts. There
is no http rule: an http Git repo is refused by name (it could not be reached),
a store given as an address is refused naming S3_ENDPOINT (the gateway allows
hostnames only). receiver_host adds the old receiver's http rule for the path
group 2 keeps until 2.6; the reconciler passes it by keyword (passing the
receiver's address positionally would have raised outside the worker's try and
hung the run on the queue -- 2.4 hardens the worker). Tests: the store-only
policy, the Model-bound policy, the two refusals, the receiver exception, and
every committed template under plane/tasks/ rendering exactly the store plus,
for verify and roles, the provider host.

Gates: pytest 2027 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:29:32 +02:00
Marius CiepluchandClaude Opus 5.5 49e11e0252 docs(readme): one paragraph for the published site and local preview
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:17:44 +02:00
Marius CiepluchandClaude Opus 5.5 f0fcb6686f docs: publish the MkDocs site to GitHub Pages
A Docs workflow builds the site strictly with the docs extra and deploys it to
https://norandom.github.io/OpenUltraSAST/ on pushes to main that touch the docs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:16:49 +02:00
Marius CiepluchandClaude Fable 5.1 9506a18b56 plane(2.2): runner -- delivery {put, inputs} in the start body, PUT to the store
StartGate.offer validates an optional `delivery` beside the credentials: https
URLs to a hostname (never an address, no userinfo; AX_RUNNER_INSECURE_DELIVERY=1
admits http in unit tests only), names ^[A-Z][A-Z0-9_]*$, a 400 that names the
field and echoes no URL. The runner PUTs its output tar to delivery.put and
GETs each declared input from delivery.inputs[NAME] (deliver() gains a method;
log lines and DeliveryError carry the URL without its query, so no presigned
signature reaches a log, a marker or a summary). OUSAST_ARTIFACT_URL/DIAL stay
honoured until task 2.6 so both paths run in the same build; the completion
marker and the retry-only boot are unchanged, and a re-start brings a fresh
URL for the retry. Router.start_task carries `delivery` in the body and
redacts its URLs like credential values. Tests: the gate's acceptance and
refusals, an end-to-end PUT/GET against the S3-shaped fake with no URL in any
log or file, the failed-PUT-then-retry-only boot, deliver() with PUT.

Gates: pytest 2024 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:12:19 +02:00
Marius CiepluchandClaude Fable 5.1 35d8032c0a plane(2.1): Delivery -- presigned store URLs, collection, state mirrored to the store
src/openultrasast/plane/delivery.py: Delivery(store, run, base) mints a presigned
PUT for runs/<run>/<task>/output.tar and a GET per declared input
(runs/<run>/<producer>/<artifact>), expiry OUSAST_TASK_TIMEOUT + 600 s;
delivered() asks the store for the exact key; collect() downloads, extracts
under the run directory with the receiver's member checks (no symlink, hard
link or escape; nothing written on a rejection), re-puts the declared outputs
as single objects and writes done.json; put_input() for inputs no task
produces; save_state()/load_state() mirror state.json. The reconciler opens the
store first (no fallback; the CLI reports a store that cannot open and submits
nothing), mirrors state.json after every change and seeds a rerun with no local
state from the store (the 2.1 bullet's state mirroring is what gives the new
module its importer). The reconciler crossed 500 lines here (517) rather than at
2.4 as 1.6 projected: the bound moves to 550 with the reason in the assertion,
as 1.6 decided. tests/plane_fake_store.py is the S3-shaped fake (objects in a
dict, presigned URLs on a local server); tests/test_plane_delivery.py covers the
body, expiry, polling, rejection, declared-only re-put, state seeding and that
no URL reaches a log. Module-audit manifest regenerated (148/5, no orphan).

Gates: pytest 2020 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:05:48 +02:00
Marius Ciepluch 9833571e94 Merge branch 'worktree-agent-af2e4a098d650d0a2' 2026-10-02 21:03:41 +02:00
Marius Ciepluch c217d1d1d3 Merge branch 'worktree-agent-aed844222b4750ed7' 2026-10-02 21:03:41 +02:00
Marius CiepluchandClaude Opus 5.5 352f8c07aa docs: plain-language pass -- detection techniques, architecture, evaluation, decision engine
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:03:24 +02:00
Marius CiepluchandClaude Opus 5.5 552671ca6a docs: plain-language pass -- README, index, where we stand, scanning, examples, threat model
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:01:10 +02:00
Marius CiepluchandClaude Opus 5.5 7e23379e96 docs: plain-language pass -- deployment, plane, memory, RustFS, token economics
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 21:00:37 +02:00
Marius CiepluchandClaude Fable 5.1 79296bb4a4 plane(1.8): kind proof and record 01 -- profile, literals, doctor, probe, bound
benchmarks/measurements/2026-10-02-k8s-01-profile/record.json with the live
evidence of group 1: `ousast plane doctor --profile kind` passes every line
(exit 0, 1.4 s; the store check reaches the bucket: verify 0.62 s, probe object
24 B); `--profile k3s` fails on the unreachable context and on the images file
the release has not published yet (two expected lines; the namespaces are not
checked behind an unreachable context); the literal scan has 0 offenders (23
before 1.3) and fails naming workspaces.py:37 when a literal is reinserted;
up.sh reran idempotently in 38 s and the smoke Run finished in 24 s on the
rebuilt image (facts.json 451 B, summary.json 238 B); the placement probe says
`any`; the reconciler bound numbers of 1.6. Doctor, literal, smoke and up.sh
outputs beside it.

Gates: pytest full suite on this tree's src/ and tests/ content: 2010 passed,
42 skipped, exit 0 (the 1.5 run; the 1.8 run reported two module-audit
failures caused by the uncommitted 2.1 module present in the working tree);
ruff check 0; ruff format 0; mypy 0; mkdocs build --strict 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:55:57 +02:00
Marius CiepluchandClaude Fable 5.1 2cca03c1c0 plane(1.7): gvisor tarball reachability documented as planned (manual, k3s)
docs/deployment.md B.1: Substrate's ateom-gvisor worker runs runsc from the
release tarball named in manifests/ate-install/sandboxconfig-gvisor.yaml
(gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd, sha256-pinned
per architecture), fetched by the atelet on the node; the node must reach it or
a copied SandboxConfig must name a mirror with the same sha256. The check is the
atelet's fetch line in the worker pod's logs, run by hand on the day k3s exists;
labelled planned until then (design open question 3 stays open).

Gates: docs only; mkdocs build --strict passed with this paragraph present in
the 1.4 gate; no file a test or linter reads changed since the 1.5 gate.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:50:14 +02:00
Marius CiepluchandClaude Fable 5.1 2afa20e2ea plane(1.6): reconciler bound recorded -- 489 lines, bound moves to 550 in 2.4
reconciler.py: 477 lines at 6bd44ca, 489 after 1.2 (the profile parameter of
run() and its docstring). Task 2.4 adds the Delivery wiring (about +25) while
the receiver path is kept for group 2, and 2.6 removes the receiver wiring
(about -20): projected 514 during group 2, 494 after 2.6. The estimate crosses
500 while both paths coexist, so 2.4 moves tests/test_plane_reconciler.py's
bound to 550 with the reason in the assertion message; 2.6 records the final
count. Numbers in benchmarks/measurements/2026-10-02-k8s-01-profile/record.json
(record 01, drafted here, completed by 1.8).

Gates: no file a test or linter reads changed since the 1.5 gate (pytest 2010
passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:49:45 +02:00
Marius CiepluchandClaude Fable 5.1 86ec237483 plane(1.5): placement probe -- Substrate places by any free worker, not by fit
ops/ax/probe-placement.sh scales ousast-pool to one worker, applies probe-pool
(1 x 2 CPU / 3Gi, the already-built ateom-gvisor image, no ko build), runs one
model-free repo-facts Task requesting 2560Mi (limit 3Gi) and one requesting
256Mi, records `kubectl ate get workers -o json` after each resume, and removes
everything it created. Live on kind: both actors landed on the 1536Mi
ousast-pool worker (status.allocated.actors=1 there, 0 on probe-pool) ->
placement: any. design.md section 4 records the outcome and the fallback group
5 must take (an engine atespace with engine-pool as its only pool if Substrate
scopes pools by atespace; else a plain Kubernetes Job on k3s). Evidence under
benchmarks/measurements/2026-10-02-k8s-01-profile/probe/.

Gates (tree identical to 1.4 in src/ and tests/; ops/ gains the probe script,
which the literal scan covers): pytest 2010 passed, 42 skipped, exit 0; ruff
check 0; ruff format 0; mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:49:03 +02:00
Marius CiepluchandClaude Fable 5.1 8099b29deb plane(1.4): up.sh by profile -- context, registry, pool and images file from kind.toml
ops/ax/up.sh reads kube_context, registry, pools.default and the images path
with `python -m openultrasast.plane.profile --print`, writes the runner digest to
ops/k8s/profiles/kind-images.json instead of ~/.cache/.../runner-image, renders
ops/ax/workerpool.yaml.tmpl from the profile's pool (POOL_NAME/REPLICAS/CPU/
MEMORY, WORKER_IMAGE), and keeps an existing pool and an ax CLI newer than its
checkout instead of rebuilding them (ko and go install would refill several GB
of Go cache on this disk). smoke-run.sh takes the runner from `--print
images.runner` and runs the checkout's code with OUSAST_PYTHON. repin_templates
reads the JSON images file (runner key; the one-line pin still reads). `--print
images` returns the file path (the first live rerun fell over this). Templates
and the generated validation-46 Tasks re-pinned to the rebuilt image (128
lines, one digest).

Live: up.sh reran idempotently on kind in 38 s (every step kept except the
runner build); smoke-run.sh done in 24 s on the new image, facts.json 451 B,
summary.json 238 B delivered (record: benchmarks/measurements/2026-10-02-k8s-01-profile/).

Gates: pytest 2010 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0; mkdocs build --strict 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:43:40 +02:00
Marius CiepluchandClaude Fable 5.1 36f5927999 plane(1.3): host-literal scan test over src, plane, ops and docs
tests/test_plane_literals.py scans src/, plane/, ops/ (minus ops/ax/up.sh and
ops/k8s/profiles/) and docs/ for the kind context, the kind registry, the kind
network's gateway, a loopback address with a port and port-forward; router.py
keeps the last two, a line tagged ax-tunnel or loopback is allowed, and a
committed template's digest-pinned image is accepted only while it names the
kind profile's registry (checked by a third test). A self-check reinserts each
literal into a fixture and sees it caught. The runner's health URL and the Joern
session's query URL are tagged loopback; docs/deployment.md and ops/ax/README.md
stop naming this host and describe the kube context and the registry as the
profile's (what 1.2 made configurable).

Gates: pytest 2009 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0; mkdocs build --strict 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:32:51 +02:00
Marius CiepluchandClaude Fable 5.1 77a03bad61 plane(1.2): literals out of doctor, egress, router, generate -- the profile is handed in
doctor(profile) checks the profile's kube context, the two namespaces behind a
reachable context only, the images file's runner reference in the profile's
registry (a local registry over its v2 API, a remote one via docker manifest
inspect) and opens the memory store; its first line reports every configured
address (Req 1.2). Egress(executable, context), open_router(context, url=...),
receiver_cluster_ip(context) and Receiver(..., context) take the profile's
context and build none; generate.increment/render and harvest take the profile's
atespace. `ousast plane doctor|run|workspaces|harvest --profile`. The test fakes
(kubectl-ate, kubectl, ax) record the context they received and the tests assert
it is the profile's. grep "kind-" src/openultrasast/plane: no match.

Gates: pytest 2006 passed, 42 skipped, exit 0; ruff check 0; ruff format 0; mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:26:46 +02:00
Marius CiepluchandClaude Fable 5.1 8eac1b2c69 plane(1.1): PlaneProfile -- one TOML per cluster, env-overridable, no secrets
ops/k8s/profiles/kind.toml and k3s.toml hold the values that differ per cluster
(exec, kube_context, registry, images, router_url, ax_server, memory, atespace,
snapshots_bucket, image_pull_secret, pools); kind-images.json carries the digest
pin ops/ax/up.sh wrote. load_profile() reads OUSAST_PLANE_PROFILE (name or path,
default kind), applies OUSAST_<FIELD> and OUSAST_POOL_<NAME>_<FIELD>, and refuses
an unknown key, a credential-shaped key or value, an exec outside local|remote
and an images file without digest pins. `python -m openultrasast.plane.profile
--print FIELD` serves shell scripts. The module-audit manifest is regenerated
for the new module (counts 147/5, no orphan).

Gates: pytest full suite 2002 passed, 1 failed (the module-audit manifest, fixed
by regenerating it; tests/test_module_audit.py re-run exit 0), ruff check 0,
ruff format 0, mypy 0.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 20:16:23 +02:00
155 changed files with 27449 additions and 2259 deletions
+43
View File
@@ -0,0 +1,43 @@
name: Docs
# Builds the MkDocs site and publishes it to GitHub Pages on every push to main that touches it.
on:
push:
branches: [main]
paths: ["docs/**", "mkdocs.yml", "README.md", ".github/workflows/docs.yml"]
workflow_dispatch:
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- uses: actions/setup-python@v6
with:
python-version-file: .python-version
- run: uv sync --extra docs
- run: uv run mkdocs build --strict
- uses: actions/upload-pages-artifact@v3
with:
path: site
deploy:
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4
@@ -119,6 +119,19 @@ rule hits.
2. Without a trained model available the tools fall back to today's behaviour and say so in the report.
3. Hand-written rules remain as signal producers; changing their enabled/shadow status no longer changes what is
reported unless the engine's decision changes.
4. *(Amended 2026-10-03, maintainer decision; refines the BLOCK rule of 7.3 and 7.4. Tested by exp-004.)* BLOCK
precision is demonstrated pooled across families, not per family:
- **Normalisation**, declared before any run: each family's raw score is put on one scale by a per-family Platt
map (`learn.calibrate.fit_map` with its floors) fitted on training folds only. A family the training folds
cannot map does not block on that fold.
- **t_block**: one threshold on the normalised score for all families, chosen on training folds as the lowest
one whose Wilson 95% lower bound on precision, over the training flags of all families together, is >= 0.95.
It is reported by the Wilson 95% lower bound over all held-out flags of all families, before the floor.
- **Family floor**: a family may block only if its own held-out flags have zero errors and number at least F
(F = 10, reasoned in the exp-004 registration); otherwise that family stays ADVISORY.
- **t_quiet**: per family, the highest threshold keeping >= 90% of the paired positives at or above it (chosen
on training folds, its held-out recall reported); findings below it are silent.
- No held-out label moves a threshold or a normalisation map; candidates whose verdict is `unsure` never block.
### Requirement 7: Built for unseen repositories -- a repo safety net
@@ -136,7 +149,8 @@ rule hits.
ADVISORY, trading precision for recall; both reported with out-of-repository intervals. The decision unit is a
candidate in changed code (`pre-push` delta), with whole-repository scans as a secondary mode.
4. Calibration is checked on held-out repositories; where it does not hold, only ADVISORY is offered for that
family.
family. *(Amended 2026-10-03: once exp-004 adopts the pooled gate of 6.4, BLOCK precision is shown by its held-out
bound and family floor, and calibration is reported, not required.)*
5. Any adaptation to a user's own repository (facts reuse, their dismissals) is opt-in, separate from the
generalisation model, and never contributes to its reported numbers.
@@ -254,6 +254,13 @@ containerd; `Dockerfile:1-7` already argues against it).
a 3 Gi actor lands only on `engine-pool`. If Substrate does not place by resource fit, the fallback is a
separate atespace with its own pool if Substrate scopes pools that way; failing that, the engine stays
`alerts-engine` on a Docker host and the k3s engine becomes a plain Kubernetes Job outside ax, recorded as such.
**Probe outcome (task 1.5, 2026-10-02, `benchmarks/measurements/2026-10-02-k8s-01-profile/`): `placement: any`.**
With `ousast-pool` at one 1536Mi worker and a `probe-pool` of one 3Gi worker both ACTIVE, a `repo-facts` actor
requesting 2560Mi (limit 3Gi) was placed on the 1536Mi `ousast-pool` worker (`kubectl ate get workers`: ACTORS
1/1000 there, 0/1000 on `probe-pool`), and so was the 256Mi actor. Substrate does not place by resource fit, so
group 5 takes the fallback: the engine pool must be the only pool its actors can land on. Before 5.2, check whether
Substrate scopes pools by atespace (an `engine` atespace with `engine-pool` as its only pool); if it does not, the
k3s engine runs as a plain Kubernetes Job outside ax and `alerts-engine` stays the kind path.
- **`alerts-engine` stays** for development on a host with Docker (kind profile), its help text and
`docs/plane.md` row marked "development only; the plane task is `engine`" (Req 4.3). Generated Runs choose
`engine` when the profile's `images` file has an engine image, else leave `alerts` for `alerts-engine`.
+13 -13
View File
@@ -10,7 +10,7 @@ its kind proof and the record it writes under `benchmarks/measurements/<date>-k8
go-ahead** with a ceiling; steps that need a hand on a cluster that does not exist yet are marked **manual**.
- [ ] 1. Profile and literals (design section 1; open-question checks 2, 3, 7)
- [ ] 1.1 `PlaneProfile`
- [x] 1.1 `PlaneProfile`
- `src/openultrasast/plane/profile.py`: frozen dataclass with the fields of design section 1; `load_profile()`
reads `ops/k8s/profiles/<OUSAST_PLANE_PROFILE>.toml` (a name or a path; default `kind`), then applies
`OUSAST_<FIELD>` overrides; rejects an unknown key, a secret-looking key (`*_KEY`, `*_SECRET`), an `exec`
@@ -19,7 +19,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_profile.py`: defaults; file then env precedence; each rejection names the field; both
committed profiles load; no field holds a value matching a credential pattern.
- _Requirements: 1.1_
- [ ] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
- [x] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
- `doctor(profile)`: context from `profile.kube_context`; the registry check resolves the `images` file's runner
reference (`docker manifest inspect` for a remote registry, `/v2/_catalog` for a local one); `open_store`
verifies `profile.memory`; prints every configured address (Req 1.2). `Egress(context=...)`,
@@ -29,46 +29,46 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
profile's context, never a built one.
- Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile.
- _Requirements: 1.1, 1.2_
- [ ] 1.3 Host-literal scan test
- [x] 1.3 Host-literal scan test
- `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does)
over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`,
`localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged
`ax-tunnel`; fails listing `path:line`.
- Evidence: the test passes on this commit and fails when a literal is reinserted (shown once in the record).
- _Requirements: 1.3_
- [ ] 1.4 `up.sh` by profile
- [x] 1.4 `up.sh` by profile
- `ops/ax/up.sh`: `KO_DOCKER_REPO` and the context from `kind.toml` (read with `python -m
openultrasast.plane.profile --print registry|kube_context`); writes `kind-images.json` instead of
`runner-image`; `generate.repin_templates` accepts that file (runner key only until task 4.3).
- Evidence: `ops/ax/up.sh` reruns idempotently on kind; `ops/ax/smoke-run.sh` passes.
- _Requirements: 1.1, 2.4_
- [ ] 1.5 Placement probe (open question 2, decided before the engine work)
- [x] 1.5 Placement probe (open question 2, decided before the engine work) -- result: `placement: any`
- `ops/ax/probe-placement.sh`: applies a second WorkerPool `probe-pool` (1 replica, 3 Gi) from the design's
engine template, then one model-free `repo-facts` Task with `resources.requests.memory: 2560Mi`, and records
with `kubectl ate get workers` which worker took the actor; then the same Task with 256Mi. Removed after.
- Evidence: `record.json` says whether Substrate places by resource fit (`placement: fit|any`). `any` selects
the design's fallback for group 5 and is written into `design.md` section 4 before group 5 starts.
- _Requirements: 4.1_
- [ ] 1.6 Reconciler bound (open question 7)
- [x] 1.6 Reconciler bound (open question 7) -- 489 lines after 1.2; projected 514 during group 2: the bound moves to 550 in 2.4
- Record `wc -l src/openultrasast/plane/reconciler.py` (477 at 17ac7ad) and the lines task 2.4 may add
(receiver wiring removed: `-12`, `Delivery` calls: `+8` estimated). If the estimate crosses 500, the bound
in `tests/test_plane_reconciler.py:803` moves to 550 in task 2.4 with the reason in the assertion message.
- Evidence: the numbers in `record.json`.
- _Requirements: 2.1_
- [ ] 1.7 gvisor tarball reachability (open question 3) **manual**
- [x] 1.7 gvisor tarball reachability (open question 3) **manual** -- documented as planned in `docs/deployment.md` B.1; the live check waits for k3s
- `docs/deployment.md` (B.1, planned): the k3s node must fetch
`gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd` (`substrate/manifests/ate-install/sandboxconfig-gvisor.yaml`)
or a mirrored copy named in a copied SandboxConfig; the check is `kubectl -n ate-system logs` of the worker
pod's atelet fetch line, run by hand on the day k3s exists. Labelled planned until then.
- _Requirements: 7.2_
- [ ] 1.8 Kind proof and record 01
- [x] 1.8 Kind proof and record 01
- `ousast plane doctor --profile kind` passes every line; `ousast plane doctor --profile k3s` fails only on the
unreachable context (the expected line); `test_plane_literals` 0 offenders; the probe of 1.5; the bound of 1.6.
- `benchmarks/measurements/<date>-k8s-01-profile/record.json`.
- _Requirements: 1.1, 1.2, 1.3, 7.1_
- [ ] 2. Store delivery, receiver-less egress (design section 3)
- [ ] 2.1 Presigned URLs and the store run directory
- [x] 2.1 Presigned URLs and the store run directory
- `src/openultrasast/plane/delivery.py`: `Delivery(store, run)` mints presigned PUT for
`runs/<run>/<task>/output.tar` and GET for `runs/<run>/<producer>/<artifact>` (boto3 `generate_presigned_url`
on `S3Client.sdk`, expiry `OUSAST_TASK_TIMEOUT + 600`); `delivered(task)` HEADs the tar; `collect(task)`
@@ -78,7 +78,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_delivery.py`: against a local S3-shaped fake (PUT/GET/HEAD by key); URLs never appear in
logs; expiry; member rejection; declared-only re-put; state seeding.
- _Requirements: 3.1, 2.1_
- [ ] 2.2 Runner: `delivery` in the start body
- [x] 2.2 Runner: `delivery` in the start body
- `src/openultrasast/plane/runner.py`: `StartGate.offer` validates an optional `delivery {put, inputs}` (https,
hostname, no echo); `_deliver_output` PUTs the tar to `delivery.put` (same retry and `DeliveryError`
semantics); `fetch_inputs` GETs from `delivery.inputs[name]`; `OUSAST_ARTIFACT_URL` and `OUSAST_ARTIFACT_DIAL`
@@ -86,14 +86,14 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_runner.py`: accepted body with `delivery`; 400 on an `http` or IP URL; PUT to a local
server; inputs fetched from presigned-shaped URLs; the marker and retry-only boot unchanged.
- _Requirements: 3.1_
- [ ] 2.3 Egress without the receiver
- [x] 2.3 Egress without the receiver
- `src/openultrasast/plane/egress.py`: `policy_for(task, workspaces, model, store_host)` emits one
`tlsPassthrough` rule on 443 over the sorted union of store host, Git hosts, Model hosts; no `http` rule; an
IP store host raises naming `S3_ENDPOINT`.
- `tests/test_plane_egress.py`: for every committed template under `plane/tasks/` the rendered policy is the
expected set; the IP refusal.
- _Requirements: 3.2_
- [ ] 2.4 Reconciler on `Delivery`
- [x] 2.4 Reconciler on `Delivery`
- `src/openultrasast/plane/reconciler.py`: `run()` opens the store first (no fallback), builds `Delivery`,
passes `delivery` to `router.starter`/`start_task` (body extended in `router.py`), `_await` on
`Delivery.delivered`, outcome from the collected `summary.json`; the receiver thread is kept behind
@@ -101,7 +101,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
- `tests/test_plane_reconciler.py`: the ax fake run completes through the delivery fake; a rerun from an empty
results root resumes from the store's `state.json`.
- _Requirements: 2.1, 3.1_
- [ ] 2.5 Kind proof and record 02
- [x] 2.5 Kind proof and record 02
- `ops/ax/smoke-run.sh` with `OUSAST_DELIVERY=store`: `facts.json` and `summary.json` arrive through the store;
the actor's policy read back has one TLS rule naming the store host; `ousast plane status ax-e2e-smoke`
prints the table. Instrument: the tar's byte size from the store equals the extracted files' total.
@@ -147,6 +147,43 @@ with Express-style routes and middleware, including vulnerable, fixed and benign
frontend/framework adaptations and new abstract capabilities, and shall report C/C++ envelope or
unsupported-property results separately from demonstrated vulnerability detection.
## Requirement 9: Operate as an advisory safety net on any repository
Added 2026-10-02, maintainer-approved, from the operational-readiness audit. The maintainer's
definition: a user installs the hook on ANY repository and gets honest, advisory-only results;
nothing ever blocks a push unless blocking was explicitly chosen.
9.1 (added 2026-10-02, maintainer-approved) When the installed hook cannot run the check, because the
`ousast` executable is missing or an argument or setting is invalid, the hook shall allow the push and
print one line saying the check did not run and how to fix it; only an explicit `--mode blocking`
opt-in may turn an analysis result into a rejected push.
9.2 (added 2026-10-02, maintainer-approved) When a check is skipped or incomplete, the terminal output
shall name each distinct reason in plain words, one line per reason, with repository paths decoded and
no opaque hash tokens; a language with no engine frontend and no quick rules shall be named with its
file count; a first push of a new branch shall say there was no base to compare against and what was
checked instead. The JSON artifact keeps the raw reason values.
9.3 (added 2026-10-02, maintainer-approved) Before the engine, within the same deadline, the Safety Net
shall run the shipped quick pattern rules over the head side of the changed lines and show their
findings as advisory pattern matches, labeled separately from admitted alerts. This amends 3.2 for this
tier only: quick-rule findings are shown, never admitted, and never block.
9.4 (added 2026-10-02, maintainer-approved) When a finding's operation lies in a file that is new in the
head revision, the comparison shall classify it as new rather than unknown; an engine finding whose
novelty is established but whose capability is not yet qualified shall be shown as an advisory engine
finding, not an admitted alert. Genuinely unknown comparisons keep their vetoes.
9.5 (added 2026-10-02, maintainer-approved) When the engine cannot finish within the deadline, one line
shall say so; on explicit request (`--engine background`) the Safety Net shall start the engine
detached after the push, record its result in the artifact directory, and show it on the next run as
the engine result for that head from the previous push.
9.6 (added 2026-10-02, maintainer-approved) The hook installer and remover shall ship in the wheel
(`ousast pre-push install|uninstall`), refuse to overwrite an existing hook unless forced, and a
hook-callable container wrapper shall pass the push input and artifact path through; the documentation
shall give a first-run guide, a glossary of skip reasons, a CI recipe and help text for every flag.
## Accepted experimental acceptance profile
These values are the initial implementation/evaluation profile accepted for task generation, not
+51 -2
View File
@@ -475,7 +475,7 @@ Preserve all completed task evidence below.
- _Depends: 13.1_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4)
- [x] 14. Ask every declared family whether it establishes anything (precondition for M4)
- Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing.
- [x] 14.1 Census the seven declared families on NodeGoat and VAmPI
- Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only.
@@ -575,7 +575,7 @@ Preserve all completed task evidence below.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds
- [x] 13.4 Let a question inside one partition complete, whatever else the repository holds
- **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan.
- The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote.
- **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it.
@@ -587,6 +587,7 @@ Preserve all completed task evidence below.
- _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical.
- _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`.
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
- _Checked 2026-10-02 per the operational-readiness audit:_ implemented and verified on PMPro (120 of 127, above). 13.3 stays open on its own blocker; this box was left unchecked only because it waits on it.
- _Depends: 13.3_
- _Requirements: 2.1, 2.2, 6.3, 8.1_
@@ -619,6 +620,54 @@ Preserve all completed task evidence below.
- _Requirements: 8.1, 8.2_
- [x] 17. Operational readiness (2026-10-02)
- **Named by the 2026-10-02 operational-readiness audit**, maintainer-approved as the first increment. The maintainer's definition: a user installs the hook on ANY repository and gets honest, advisory-only results; nothing ever blocks a push. The audit found the plumbing sound (snapshots, resolution, deadline accounting, advisory exit 0, the JSON artifact) and five things in the way: the hook can block a push, quick rules are not part of pre-push, the engine cannot finish a 26-file repository inside 30 s, the terminal hides every skip reason, and there is no supported install path without the source checkout. Before numbers for every item are the audit's live runs on records (Python), serve-static (JavaScript) and unrolled/secure (Go), each with one planted commit.
- This group is plumbing and honesty. It enables no capability, edits no rule for a planted sample, and leaves graph reuse (the engine speed work decided in 11.6) as a later task.
- _Done 2026-10-02:_ all seven subtasks done. The engine still does not fit the 30 s hook deadline (graph reuse, the 11.6 direction, stays a later task), and plane verdicts after the push were not in this increment.
- _Requirements: 9.1–9.6_
- [x] 17.1 Never block a push the user did not choose to block
- The installed hook exits 0 with one line when `ousast` is missing (the check did not run, and how to install). In hook (stdin) mode, an argument or usage error (an invalid `OUSAST_PUSH_DEADLINE`, a cache directory inside the repository) is advisory: one line and exit 0. `--mode blocking` stays the explicit opt-in that can reject a push, and keeps its usage errors. The prior-hook input-retention failure keeps exit 1, because it is the chained hook's verdict that cannot be obtained, not ours.
- Files: `ops/pre-push`, `src/openultrasast/cli.py`. Tests: `tests/test_push_hook.py` (a push with `ousast` absent from PATH succeeds and prints the line; a bad deadline in hook mode exits 0; blocking mode keeps exit 2).
- Evidence: the audit's `hook.sh` push 1 (rejected before) and `fail.sh` (rc=2 before) rerun with exit 0.
- _Done 2026-10-02:_ the hook prints one line and exits 0 (chaining a prior hook if present) when `ousast` (or `OUSAST_COMMAND`) is missing; in hook mode without `--mode blocking`, argparse/usage errors and unexpected exceptions print one line and exit 0. Tests: `test_missing_ousast_never_blocks_the_push`, `test_hook_mode_usage_errors_are_advisory_unless_blocking` (blocking keeps exit 2), and `test_cli_refuses_the_flag_outside_explicit_replay` updated (the experimental flag is still refused in hook mode, now advisory). Gates: 1987 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.1, 5.3, 5.4_
- [x] 17.2 Name every skipped check in plain words
- One terminal line per distinct coverage reason: hex-encoded paths decoded, sha256-suffixed reasons grouped with a count, JSON unchanged. "language not covered: <lang> (<n> files)" for a language with no engine frontend and no quick rules, in both `pre-push` and `scan`. A first push of a new branch says there is no base and what was checked instead. `reports.py` gains lines for the eight scan degradations `cli.py` records and the report did not render (`learning_endpoint_unavailable`, `cpg_unavailable`, `facts_unavailable`, `policy_invalid`, `model_withheld_execution_budget`, `max_findings_exceeded`, `hunter_model_unavailable`, `sandbox_unavailable`).
- Files: `src/openultrasast/push/report.py`, `src/openultrasast/push/explain.py` (new), `src/openultrasast/reports.py`, `src/openultrasast/cli.py`. Tests: `tests/test_push_report.py`, `tests/test_reports.py`.
- Evidence: the three repositories' terminal output names engine-missing, Go not covered and the new branch.
- _Done 2026-10-02:_ new `push/explain.py` turns every raw coverage reason into one plain `Skipped:` line per distinct kind (hex paths decoded, sha256 tokens dropped and counted, unknown reasons still printed); the existing one-line `Notice:` keeps its next action and gains an engine-missing variant. `language_not_covered:<lang>:<changed>:<total>` is recorded for a changed language with no engine frontend and no quick rules; `ousast scan` prints `notice: language not covered: go (N files)` and its report says so. `reports.py` renders the eight previously dropped degradations and prints any reason it has no sentence for. The hook passes `OUSAST_COMPARISON_BASE`. The new-branch line names the base it lacks; what was checked instead lands with the quick tier (17.3). Tests: `tests/test_push_explain.py` (6), two in `tests/test_reports.py`; module-audit manifest regenerated for the new module. Gates: 1995 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.2, 5.2, 8.2_
- [x] 17.3 Run the quick rules as the fast first tier
- Inside the deadline and before the engine, the shipped quick rules run over the head side of the changed files and keep the findings on changed lines or in changed functions. They print as advisory pattern matches, labeled apart from admitted alerts, recorded in the artifact's `quick_tier`, and never block.
- Files: `src/openultrasast/push/quick.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_quick.py` (new).
- Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it.
- _Done 2026-10-02:_ new `push/quick.py` runs the shipped quick rules (no shadow rules, no model) over the changed head files inside `_prepare` under the shared deadline, before the engine, keeping matches on a changed line or inside a changed function (entry-point ranges); it records `files`/`bytes_read` as proof of input. The terminal shows them under "Quick rules: N pattern match(es) on changed code ... Advisory: not verified by the engine."; the artifact keeps `quick_tier`; admission and push disposition are untouched. A new branch runs the quick tier over its last commit's changes and says so. `_analyze` gained `engine="off"`. On the audit's serve-static planted commit the `eval` in the new `debug.js` is now shown (js-eval, js-xss-response at debug.js:2, ~1.1 s on the host); the planted `%`-formatted SQL in records is NOT matched by any quick rule (coverage gap, no rule edited). Tests: `tests/test_push_quick.py` (3). Gates: 1998 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.3_
- [x] 17.4 Classify a file new in head as new, not unknown
- `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto.
- Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`.
- Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB).
- _Done 2026-10-02:_ `ChangeContext.added_paths` records Git status-A paths (rename targets excluded). In `_compare_evidence`, after every existing context/head/deadline check, an operation in an added file is `new`/`file_added_in_head` when the claim is entailed, undischarged and has change evidence, and no base answer holds the same operation (else `operation_moved_within_change`); an unpaired add/delete still blocks as `change_context_unresolved`. Admission accepts `(new, file_added_in_head)`; with the empty registry such a candidate's only veto is `capability_unavailable`, and the terminal now shows it under "Engine: N finding(s) this push introduced. Advisory ... not an alert." Any comparison, evidence or location veto keeps a candidate out of that line. Tests: `tests/test_push_new_file.py` (9). Live engine evidence depends on host memory (17.7). Gates: 2007 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.4, 3.4_
- [x] 17.5 Say when the engine cannot finish, and offer it in the background
- One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for <head> from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work.
- Files: `src/openultrasast/push/background.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/cli.py`, `ops/pre-push`. Tests: `tests/test_push_background.py` (new).
- Evidence: a hook push with `OUSAST_PUSH_ENGINE=background` followed by a second push showing the previous result.
- _Done 2026-10-02:_ `--engine inline|background|off` (hook: `OUSAST_PUSH_ENGINE`) and `--background-deadline` (default 900 s). `off` and `background` run the quick tier and record `engine_off`/`engine_background` with a plain skip line; the deadline line names the deadline and the background option. New `push/background.py` starts one detached explicit replay (`start_new_session`, stdin closed, output to a log) per artifact directory and repository (lock file with a live-pid check: a second start reports busy), writes its artifact under `<artifact dir>/engine/<repo key>/<head>.json`, and the next run prints "Engine result for <head> from the previous push: ..." once, then marks it shown. Not usable through the Docker wrapper (the container exits with the hook); documented in 17.6. Tests: `tests/test_push_background.py` (4, one launching a real detached run). Gates: 2011 passed, 42 skipped; ruff, format, mypy clean.
- _Requirements: 9.5, 4.1, 4.2_
- [x] 17.6 Install, Docker and documentation for a first-time user
- `ousast pre-push install|uninstall [--force] [--docker] [REPO]` ships the hook as package data, honours `core.hooksPath`, refuses an existing hook unless `--force` (which chains it as `pre-push.before-ousast`), and works from a pip install. `ops/ousast-docker` is a hook-callable script (not a shell function) that passes stdin and the artifact path through. The `openultrasast:dev` image is rebuilt only if `df -h /` shows at least 4 GB free; otherwise the rebuild command is documented and the record says it was not rebuilt. `docs/scanning.md` gains a first-run guide, a glossary of skip reasons, a GitHub Actions recipe and full `--help` text; README tiers 1 and 2 updated, pulling `ghcr.io/norandom/openultrasast:2.0.1` where the wrapper allows.
- Files: `src/openultrasast/push/hook/pre-push`, `src/openultrasast/push/install.py` (new), `src/openultrasast/cli.py`, `ops/ousast-docker`, `docs/scanning.md`, `README.md`, `ops/README.md`. Tests: `tests/test_push_install.py` (new), `mkdocs build --strict`.
- Evidence: install from the package into a scratch clone, push through it, uninstall restores the prior hook.
- _Done 2026-10-02:_ `ousast pre-push install|uninstall [REPO] [--force] [--docker]` (`push/install.py`) installs the hook shipped as package data (`openultrasast/push/hook/pre-push`, byte-identical to `ops/pre-push`, verified in the built wheel), honours `core.hooksPath` without editing it, refuses an existing hook unless `--force` (ours is replaced; a foreign hook moves to `pre-push.before-ousast` and is chained), and uninstall removes only a hook it wrote and restores the chained one. `ops/ousast-docker` (also package data; `--docker` installs it beside the hook, which then uses it) is a script that mounts the repository read-only and the artifact directory at their own absolute paths, passes stdin with `-i`, runs as the user with no network and `--memory 3g`, defaults to `ghcr.io/norandom/openultrasast:2.0.1`, never pulls inside a hook, and exits 0 unless `OUSAST_PUSH_MODE=blocking`. The hook passes `--engine`/`--comparison-base` only when set, so an older ousast or image still runs. Every pre-push flag has help text; `docs/scanning.md` has the first-run guide, settings table, new-branch and background notes, the artifact keys, a skip-reason glossary (a test keeps it in sync with `push/explain.py`), a GitHub Actions recipe and the full `--help` text; README tiers 1 and 2 and `ops/README.md` updated (pull and tag the GHCR image instead of building). **Image not rebuilt:** `df -h /` showed 1.1 GB free (< 4 GB); rebuild with `docker build -t openultrasast:dev .`. Tests: `tests/test_push_install.py` (11) and the glossary test. Gates: 2022 passed, 42 skipped; ruff, format, mypy clean; `mkdocs build --strict` exit 0.
- _Requirements: 9.6, 7.4_
- [x] 17.7 Prove it on the audit's three repositories
- Rerun records, serve-static and unrolled/secure (same planted commits) with the new build in quick mode and with the hook installed, plus one Docker engine run on serve-static if free memory >= 3 GB and free disk >= 1 GB. Prove each run read its input (files, bytes).
- Files: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json`. Evidence: before (the audit) and after: exit codes, wall time against the deadline, findings shown, every skip line printed. A planted sample still missed is a coverage gap, recorded, never a rule edit.
- _Done 2026-10-02:_ `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json` (before = the audit, after = this build installed from its wheel into a scratch venv). 10 hook pushes and 6 replays, all exit 0, 0.69-1.21 s against 30 s on the host; the quick tier read exactly the changed file and every head snapshot read the full tree (records 43,577 B, serve-static 61,525 B, secure 103,023 B). The planted JS `eval` in the new `debug.js` surfaces: quick tier on the host, and as two advisory engine findings classified `new`/`file_added_in_head` in one Docker engine run (190.9 s against a 300 s deadline; MemAvailable 3.28 GB, 1.16 GB disk). **The planted Python `%`-formatted SQL is still missed**: no quick rule matches it and the host has no engine. Recorded as a coverage gap, no rule edited. Go is named as not covered. The proof caught a wrong install hint (`pip install openultrasast`, not on PyPI), corrected in the hook in this commit.
- _Requirements: 9.1–9.6, 6.3_
## Implementation Notes
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.
+258 -145
View File
@@ -1,38 +1,53 @@
# OpenUltraSAST
OpenUltraSAST is an experimental static security analyser meant to work as a safety net on
repositories it has never seen: it scans a checkout, or the commits a `git push` would publish,
and separates what it can prove from what it merely suspects. A scan combines language-scoped
pattern rules, a tree-sitter overlay, a Joern code property graph and, optionally, a model that is
asked only the question the graph cannot settle; every finding carries the evidence that earned
its place. A scan that could not look at something records a degradation instead of reporting a
clean result. `ousast` is the interface; model work over whole repositories runs on a separate
agentic plane (google/ax), and a learned decision engine is in development. Neither is needed
for a scan.
OpenUltraSAST is an experimental static security analyser. It is meant to be a safety net on
repositories it has never seen. It scans a checkout, or the commits a `git push` would publish. It
keeps what it can prove apart from what it only suspects.
This file is the front door; each section links to its page of the documentation site (`docs/`,
`mkdocs.yml`; `uv sync --extra docs && uv run mkdocs serve`).
A scan combines up to four parts:
- language-scoped pattern rules;
- a tree-sitter overlay (a syntax tree of each file, built by the tree-sitter parser);
- a Joern code property graph (a graph of the program's syntax, control flow and data flow);
- optionally, a model. It is asked only the question the graph cannot settle.
Every finding carries the evidence that earned its place. When a scan could not look at
something, it records a degradation. It does not report a clean result.
`ousast` is the interface. Model work over whole repositories runs on a separate agentic plane
(google/ax). A learned decision engine is in development. A scan needs neither of them.
This file is the front door. Each section links to its page of the documentation site.
The site is published at <https://norandom.github.io/OpenUltraSAST/>. Its sources are in `docs/` and
`mkdocs.yml`; to preview it locally, run `uv sync --extra docs && uv run mkdocs serve`.
## Status
**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate:
no hook capability is enabled, and the two independent populations spent so far gave engine recall
1 of 11 and then 0 of 17. The learned decision engine ranks but may not block: BLOCK is offered for
no family. The project has stopped trying to reach the gates by hand; every detection change now
has to win a controlled measurement on held-out repositories. The one page that states all of this
with the record behind each figure is [docs/where-we-stand.md](docs/where-we-stand.md); the
release history is in [RELEASE_NOTES.md](RELEASE_NOTES.md).
**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate.
- No hook capability is enabled.
- Two independent populations (case sets frozen before any scan) have been spent so far. Engine
recall was 1 of 11 and then 0 of 17.
- The learned decision engine ranks but may not block. BLOCK is offered for no family.
The project has stopped trying to reach the gates by hand. Every detection change now has to win
a controlled measurement on held-out repositories. One page states all of this, with the record
behind each figure: [docs/where-we-stand.md](docs/where-we-stand.md). The release history is in
[RELEASE_NOTES.md](RELEASE_NOTES.md).
## Install and first scan
Which tier you need: a deterministic scan of a checkout or a CI gate is tier 1; the Joern engine
(`standard`, `deep`) is tier 2; running model work over whole repositories on the plane is tier 3,
for maintainers; tier 4 does not exist yet.
Pick the tier you need:
1. Tier 1: a deterministic scan of a checkout, or a CI gate.
2. Tier 2: the Joern engine (`standard`, `deep`).
3. Tier 3, for maintainers: model work over whole repositories on the plane.
4. Tier 4 does not exist yet.
### Tier 1: quick scan, no Docker
The core install depends only on PyYAML. From a checkout, `uv sync` (or `pip install .`) installs
the `ousast` entry point; `uv sync --group dev` adds the test and lint tools.
the `ousast` entry point. `uv sync --group dev` adds the test and lint tools.
```bash
uv sync
@@ -41,19 +56,41 @@ uv run ousast scan . --mode quick --fail-on verified # CI: exit 1 on any evide
```
This runs the language-scoped pattern rules, entry-point reachability hints, ranking,
verification and scoring: deterministic, reproducible, no network, no keys. What it cannot do:
there is no engine, so nothing follows a value across statements or files, and the PHP quick rules
are a low-precision, in-sample development set (per-rule precision lower bounds from 0.03 to 1.0 on
the real pins, `benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). The optional
`semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for the overlay.
verification and scoring. It is deterministic and reproducible. It needs no network and no keys.
The same install gives you the advisory pre-push hook for any repository. It runs the quick rules
on the lines a push changes, names every check it could not run, and never blocks a push:
```bash
cd /path/to/your/repository && ousast pre-push install # remove: ousast pre-push uninstall
```
What it cannot do:
- There is no engine. Nothing follows a value across statements or files.
- The PHP quick rules are a low-precision, in-sample development set. Per-rule precision lower
bounds run from 0.03 to 1.0 on the real pins
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`).
The optional `semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for
the overlay.
### Tier 2: standard and deep scans (Docker)
`standard` and `deep` need Joern. The Docker image `openultrasast:dev` ships the tool, Joern
v4.0.625 and `php-cli` (needed by the PHP frontend) in one image; it is built locally from the
`Dockerfile` (no published image yet), and it is big: a JRE plus about 2 GB of Joern. The shell
wrapper builds it on first use and then runs `ousast` in the container with your directory mounted
read-only and the container network off:
`standard` and `deep` need Joern. The Docker image ships three things in one image: the tool,
Joern v4.0.625 and `php-cli` (the PHP frontend needs it). Each release publishes it to GHCR. It is
big: a JRE plus about 2 GB of Joern. Pull it instead of building, and tag it with the name the
shell wrapper uses:
```bash
docker pull ghcr.io/norandom/openultrasast:2.0.1
docker tag ghcr.io/norandom/openultrasast:2.0.1 openultrasast:dev # or build: docker compose build
```
The shell wrapper runs `ousast` in the container. It builds the image only if `openultrasast:dev`
is missing. Your directory is mounted read-only and the container network is off. For the pre-push
hook, use `ousast pre-push install --docker`: Git cannot call a shell function, so it installs the
script `ousast-docker` beside the hook.
```bash
source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1
@@ -62,90 +99,132 @@ ousast-with-judge scan . --mode standard # network on, the model is a
```
`docker compose build` and `TARGET=/path/to/repo docker compose run --rm ousast scan /target` do
the same without the wrapper. `deep` adds REGRESS, which runs each reproduction snippet with
`docker run` from wherever the scan runs; the engine container has no Docker socket, so run `deep`
from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
verified) and a working `docker`, or accept the recorded `sandbox_unavailable` degradation.
the same without the wrapper.
`deep` adds REGRESS. REGRESS runs each reproduction snippet with `docker run` from wherever the
scan runs. The engine container has no Docker socket. So run `deep` in one of two ways:
- from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
verified) and a working `docker`;
- or accept the recorded `sandbox_unavailable` degradation.
Details, including the host install, are in [ops/README.md](ops/README.md).
Keys and store settings live in `.env` in the working directory (gitignored; `.env.example` lists
the names). `DEEPSEEK_API_KEY` serves every LLM call; `OPENROUTER_API_KEY` is used for embeddings
only (`ousast learn memory embed`). `.env` never overrides a variable already exported in your
shell, so `unset` a stale export when a key change seems to have no effect. The plane's memory
store defaults to a local file store under `~/ousast-results/plane/memory`; an S3-compatible bucket
is optional (`uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT` and the
AWS credentials in `.env`). Nothing in a scan needs it.
Keys and store settings live in `.env` in the working directory. The file is gitignored;
`.env.example` lists the names.
- `DEEPSEEK_API_KEY` serves every LLM call.
- `OPENROUTER_API_KEY` is used for embeddings only (`ousast learn memory embed`).
- `.env` never overrides a variable already exported in your shell. If a key change seems to have
no effect, `unset` the stale export.
- The plane's memory store defaults to a local file store under `~/ousast-results/plane/memory`.
- An S3-compatible bucket is optional: `uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>`
with `S3_ENDPOINT` and the AWS credentials in `.env`. Nothing in a scan needs it.
### Tier 3: the plane on ax (maintainers)
None of this is needed for tier 1 or 2. Today's profile is a single-node kind cluster on the
maintainer's host. `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the
PATH (it refuses to start when one is missing) and builds Agent Substrate and ax from their
checkouts under `~/.cache/ousast/ax-src/`. Measured idle footprint with two workers: the kind node
takes 1.7 GiB of memory (27% of the 7.7 GiB host) and 4.9 GB of disk for images, with several GB
more of Go caches during the build; the worker pool is sized for that 7 GB host. The verify and
roles tasks need `DEEPSEEK_API_KEY`, and memory lives in the local file store or the S3 store
(RustFS is the tested server). Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
Tier 1 and 2 need none of this. Today's profile is a single-node kind cluster on the maintainer's
host.
- `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the PATH. It refuses to
start when one is missing.
- It builds Agent Substrate and ax from their checkouts under `~/.cache/ousast/ax-src/`.
- Measured idle footprint with two workers: the kind node takes 1.7 GiB of memory (27% of the
7.7 GiB host) and 4.9 GB of disk for images. The build uses several GB more for Go caches.
- The worker pool is sized for that 7 GB host.
- The verify and roles tasks need `DEEPSEEK_API_KEY`.
- Memory lives in the local file store or the S3 store. RustFS is the tested server.
Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
### Tier 4: Kubernetes (k3s), planned
No deployment to any cluster other than that kind cluster has been made. The target is two
profiles of one code base, `kind` locally and `k3s` in production, where the same `ousast plane run`
submits a Run in a remote execution mode: it needs a kubeconfig context or the cluster's ingress
(and an `imagePullSecret` on the worker ServiceAccount for a private package), runner and engine
images published to GHCR and pinned by digest, and the S3 store as the artifact path, with no
receiver on a laptop. The plan, and the code changes it needs with file paths, are in
Nothing has been deployed to any cluster other than that kind cluster. The target is two profiles
of one code base: `kind` locally and `k3s` in production. The same `ousast plane run` would
submit a Run in a remote execution mode. It needs:
- a kubeconfig context or the cluster's ingress (and an `imagePullSecret` on the worker
ServiceAccount for a private package);
- runner and engine images published to GHCR and pinned by digest;
- the S3 store as the artifact path, with no receiver on a laptop.
The plan, and the code changes it needs with file paths, are in
[docs/deployment.md](docs/deployment.md#production-topology-ax-on-a-kubernetes-cluster).
## Scan modes
`quick` runs the pattern rules, hints, ranking, verification and score and needs nothing.
`standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and the
Joern model layer, which builds one code property graph per repository and decides taint,
guard-dominance and configuration questions on it; a model is asked only the residual question
(without a key it is skipped and only the graph's entailed findings are reported). `deep` adds
REGRESS: promoted candidates are loaded by a small snippet in a Docker sandbox and get a
`triggerable` or `not_triggerable` verdict. A missing engine is a recorded degradation
(`cpg_unavailable`, `sandbox_unavailable`), never a clean result.
There are three modes. Each adds to the one before.
- `quick` runs the pattern rules, hints, ranking, verification and score. It needs nothing.
- `standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and
the Joern model layer. The model layer builds one code property graph per repository. It decides
taint, guard-dominance and configuration questions on that graph. A model is asked only the
residual question. Without a key, that question is skipped and only the graph's entailed
findings are reported.
- `deep` adds REGRESS. Promoted candidates are loaded by a small snippet in a Docker sandbox. Each
gets a `triggerable` or `not_triggerable` verdict.
A missing engine is a recorded degradation (`cpg_unavailable`, `sandbox_unavailable`), never a
clean result.
`--fail-on never|findings|verified|worth-fixing` sets the exit code. `--config openultrasast.toml`
loads settings. Every run writes these files under `<target>/.openultrasast/runs/<scan-id>/`:
`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`, `report.sarif`
and `trace/events.jsonl`.
`ousast pre-push` (experimental) analyses the commits a push would publish against their base.
First, the quick rules run on the changed lines. Then the engine compares head with base, and only
new or worsened defects become candidates. It is advisory by default, and it prints one line per
check it could not run. Its capability registry is empty, so it emits no normal alert today. An
engine finding the push introduced is shown as advisory. Install, settings, skip reasons and a CI
recipe: [docs/scanning.md](docs/scanning.md#first-run-install-the-hook-on-any-repository).
`--fail-on never|findings|verified|worth-fixing` sets the exit code and `--config openultrasast.toml`
loads settings. Every run writes `manifest.json`, `findings.json`, `verification.json`,
`score.json`, `report.md`, `report.sarif` and `trace/events.jsonl` under
`<target>/.openultrasast/runs/<scan-id>/`. `ousast pre-push` (experimental) analyses the commits a
push would publish against their base so that only new or worsened defects are candidates; it is
advisory by default and its capability registry is empty, so no normal alert is emitted today.
Flows and the per-language rule table: [docs/scanning.md](docs/scanning.md).
## How detection works
Three representations, from cheap to expensive: text (regex quick rules, one statement at a time),
tree (the tree-sitter CST and an IR per function, for the overlay and the obligations) and graph
(the Joern CPG, the only one that follows a value across statements, calls and files). Taint runs
as fact tables, queries and verdicts on the graph; absence bugs (an operation that should have
been guarded) are found as obligations, dominance and configuration census; a model answers only
the residual question. [docs/detection-techniques.md](docs/detection-techniques.md) explains each
technique with the code it comes from and what it misses;
[docs/architecture.md](docs/architecture.md) gives the stage order, the evidence ladder (a state
machine, not a label a model may assign), the central CWE policy and the project score, and the
self-improving loops (`ousast improve`, with `--memory` for proposals from plane rows).
The tool uses three representations of code, from cheap to expensive:
Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP, and
run only against files of their language. The detection gate (`uv run python -m openultrasast.gate`)
enforces at least 90% recall and under 10% false positives per language on the bundled cheat-sheet
corpora; on this tree it reports 93.62% recall (44/47) and 0.0% false positives. Those corpora show
that a rule change did not break a known detection; they are not a recall estimate on real code,
and PHP is not in the gate. Framework knowledge (WordPress, Flask, Django, Express, Spring) is
tagged as an optional prior in `src/openultrasast/ruleset/frameworks.toml`, so its contribution can
be measured and switched off.
1. Text: regex quick rules, one statement at a time.
2. Tree: the tree-sitter CST (concrete syntax tree) and an IR (intermediate representation) per
function. The overlay and the obligations use it.
3. Graph: the Joern CPG. It is the only one that follows a value across statements, calls and
files.
Taint runs as fact tables, queries and verdicts on the graph. Absence bugs are operations that
should have been guarded. They are found as obligations, dominance and configuration census. A
model answers only the residual question.
- [docs/detection-techniques.md](docs/detection-techniques.md) explains each technique, with the
code it comes from and what it misses.
- [docs/architecture.md](docs/architecture.md) gives the stage order and the evidence ladder (a
state machine, not a label a model may assign). It also covers the central CWE policy, the
project score and the self-improving loops (`ousast improve`, with `--memory` for proposals from
plane rows).
Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP. Each
rule runs only against files of its language.
The detection gate (`uv run python -m openultrasast.gate`) enforces at least 90% recall and under
10% false positives per language on the bundled cheat-sheet corpora. On this tree it reports
93.62% recall (44/47) and 0.0% false positives. Those corpora show that a rule change did not
break a known detection. They are not a recall estimate on real code, and PHP is not in the gate.
Framework knowledge (WordPress, Flask, Django, Express, Spring) is tagged as an optional prior in
`src/openultrasast/ruleset/frameworks.toml`. So its contribution can be measured and switched off.
## The plane on ax
Model-driven work over whole repositories runs as a Run manifest, a DAG of ax Tasks, each in its
own gVisor actor on Agent Substrate. ax is the only agentic executor; there is no local
subprocess path. Each Task binds at most one Model and its own `usd`/`calls` budget, egress is
deny-by-default per Task (hostnames only), and the provider key travels only in the Task's start
request, never in a manifest or an image. Measured on the first increment: the plane agreed on 16
of 20 declared sites at $0.0205 per candidate.
Model-driven work over whole repositories runs as a Run manifest. A Run is a DAG of ax Tasks. Each
Task runs in its own gVisor actor (an isolated sandbox) on Agent Substrate. ax is the only agentic
executor; there is no local subprocess path.
- Each Task binds at most one Model and its own `usd`/`calls` budget.
- Egress is deny-by-default per Task, by hostname only.
- The provider key travels only in the Task's start request, never in a manifest or an image.
Measured on the first increment: the plane agreed on 16 of 20 declared sites at $0.0205 per
candidate.
```bash
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
@@ -155,42 +234,61 @@ uv run ousast plane remember validation-46 # ingest the run's rows
```
`workspaces`, `harvest`, `alerts-engine` and `memory-normalise` generate Runs, their inputs and
the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md); what runs
where today and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md). What runs
where today, and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
HarnessX, the earlier agentic extra, was retired 2026-09-30: a `[harnessx]` section is ignored
with one warning, and `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail
naming the plane replacement, as do config keys nothing ever read. Fusion is deterministic.
HarnessX, the earlier agentic extra, was retired 2026-09-30. A `[harnessx]` section is ignored
with one warning. `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail and
name the plane replacement. Config keys that nothing ever read fail the same way. Fusion is
deterministic.
## Memory
One store, keyed by repository and pin, holds what plane runs and the decision engine learned:
rows (facts, verdicts, unit costs, alerts, coverage, feature records) and content-addressed blobs
(excerpts, embeddings, cached responses, compiled programs). `OUSAST_MEMORY` selects the backend:
a local file store, or `s3://<bucket>` on any S3-compatible server with versioning, lifecycle
rules, object tags and S3 Select; RustFS is the tested server. An admin sets the bucket up once;
the store verifies it at every start and refuses to run, naming each missing piece, rather than
fall back. Layout and the read and write paths: [docs/memory.md](docs/memory.md); the bucket
policy and admin steps: [docs/rustfs.md](docs/rustfs.md); how stored rows are meant to turn into
better decisions: [docs/memory-and-detection.md](docs/memory-and-detection.md).
One store holds what plane runs and the decision engine learned. It is keyed by repository and pin.
It holds two kinds of data:
- rows: facts, verdicts, unit costs, alerts, coverage and feature records;
- content-addressed blobs: excerpts, embeddings, cached responses and compiled programs.
`OUSAST_MEMORY` selects the backend. It is either a local file store or `s3://<bucket>` on any
S3-compatible server. The S3 server must support versioning, lifecycle rules, object tags and S3
Select. RustFS is the tested server.
An admin sets the bucket up once. The store verifies it at every start. If anything is missing, it
refuses to run and names each missing piece. It does not fall back.
- Layout and the read and write paths: [docs/memory.md](docs/memory.md).
- The bucket policy and admin steps: [docs/rustfs.md](docs/rustfs.md).
- How stored rows are meant to turn into better decisions:
[docs/memory-and-detection.md](docs/memory-and-detection.md).
## Decision engine and evaluation
A learned decision engine is being built to replace hand-tuned detection edits: every instrument
produces signals, never verdicts; labels come from ground truth only; a compiled AI classifier with
local memory turns retrieved similar cases and the candidate's signals into a calibrated
probability with BLOCK and ADVISORY operating points. It is **in development, not adopted**: no
scan, pre-push check or report uses it. Measured on six families, on repositories the compiled
program was not shown, the pooled AUC of its score runs from 0.77 (injection) to 0.96 (access
control); the stricter within-pair AUC is 0.72 to 0.93; BLOCK is offered for no family
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`). Maintainer surface:
`ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`. Design, slices and
the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
A learned decision engine is being built. It is meant to replace hand-tuned detection edits. It
follows three rules:
How anything is measured on code the tool was not tuned on, the pair corpora, the independent
populations and the M4 gates, and the fold discipline that keeps learned decisions out of their
own training data: [docs/evaluation.md](docs/evaluation.md). Population v3 (PHP, 15 cases) is
frozen as the one-time final check and nothing in the tree may read it.
- Every instrument produces signals, never verdicts.
- Labels come from ground truth only.
- A compiled AI classifier with local memory turns retrieved similar cases and the candidate's
signals into a calibrated probability. It has BLOCK and ADVISORY operating points.
It is **in development, not adopted**. No scan, pre-push check or report uses it.
It was measured on six families, on repositories the compiled program was not shown:
- The pooled AUC (area under the ROC curve) of its score runs from 0.77 (injection) to 0.96
(access control).
- The stricter within-pair AUC is 0.72 to 0.93 across the same families.
- BLOCK is offered for no family
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
Maintainer surface: `ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`.
Design, slices and the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
[docs/evaluation.md](docs/evaluation.md) explains how anything is measured on code the tool was not
tuned on. It covers the pair corpora, the independent populations and the M4 gates. It also covers
the fold discipline that keeps learned decisions out of their own training data. Population v3
(PHP, 15 cases) is frozen as the one-time final check. Nothing in the tree may read it.
```bash
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
@@ -201,19 +299,26 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dr
## Token economics
Model tokens are the one running cost. At runtime the plane routes them: one Model and one budget
per Task, a metered client that refuses the next call at the ceiling, repository facts computed
model-free and reused, blobs replayed at $0, triage before the hunt and a tie-break only on
disputed candidates. In development, 85.5% of the maintainers' coding-session tokens were tool
input and output rather than reasoning (down from 90%), and the remedies in `benchmarks/dev/`
target that. Mechanisms,
the measured costs and the before-and-after table: [docs/token-ergonomics.md](docs/token-ergonomics.md).
Model tokens are the one running cost. At runtime the plane routes them:
- one Model and one budget per Task;
- a metered client that refuses the next call at the ceiling;
- repository facts computed without a model and reused;
- blobs replayed at $0, with no new model call;
- triage before the hunt;
- a tie-break only on disputed candidates.
In development, 85.5% of the maintainers' coding-session tokens were tool input and output rather
than reasoning. That is down from 90%. The remedies in `benchmarks/dev/` target that share.
Mechanisms, the measured costs and the before-and-after table:
[docs/token-ergonomics.md](docs/token-ergonomics.md).
## Examples, threat model
End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
What the tool trusts and what it does not (scanned code is untrusted input, secret redaction,
per-task egress, budgets, the `deep` sandbox): [docs/threat-model.md](docs/threat-model.md).
- End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
- What the tool trusts and what it does not: [docs/threat-model.md](docs/threat-model.md). This
covers scanned code as untrusted input, secret redaction, per-task egress, budgets and the
`deep` sandbox.
## Contributing and maintainer tooling
@@ -228,25 +333,33 @@ uv run python dagger/ci.py # containerized CI pipeline
```
An agent that can run shell commands needs nothing but the CLI. For [OpenCode](https://opencode.ai)
there are two project skills in `.agents/skills/`: `openultrasast-scan` (modes and their
prerequisites, `--fail-on`, the evidence ladder, recorded degradations, run artifacts) and
`openultrasast-triage` (false-positive reasons, what an adjudication may feed, fixing and
re-scanning). `ousast mcp` runs a narrow MCP server over stdio with ten tools (`openultrasast.scan`,
`status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`, `explain`,
`propose_patch`, `export_report`); no tool runs a shell, Docker or a free-form command:
there are two project skills in `.agents/skills/`:
- `openultrasast-scan`: modes and their prerequisites, `--fail-on`, the evidence ladder, recorded
degradations and run artifacts.
- `openultrasast-triage`: false-positive reasons, what an adjudication may feed, and fixing and
re-scanning.
`ousast mcp` runs a narrow MCP (Model Context Protocol) server over stdio. It has ten tools:
`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`,
`explain`, `propose_patch` and `export_report`. No tool runs a shell, Docker or a free-form
command.
```jsonc
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
```
The `kiro-*` skills, `AGENTS.md` and the Kiro-style specifications under `.kiro/` are the
maintainer's development tooling and can be ignored by users. Other maintainer commands:
`ousast repos` (pinned known-vulnerable checkouts) and `ousast model candidates` (what the
candidate enumerator can reach).
maintainer's development tooling. Users can ignore them. Other maintainer commands:
- `ousast repos`: pinned known-vulnerable checkouts.
- `ousast model candidates`: what the candidate enumerator can reach.
## Licence
OpenUltraSAST is licensed under the [Apache License 2.0](LICENSE). Third-party material keeps its
own terms: Joern (Apache-2.0) is downloaded at image build time, the tree-sitter wheels of the
`semantic` extra are MIT, and the published benchmark cases were licence-checked one by one (v1's
OpenCVE case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md)).
own terms:
- Joern (Apache-2.0) is downloaded at image build time.
- The tree-sitter wheels of the `semantic` extra are MIT.
- The published benchmark cases were licence-checked one by one. v1's OpenCVE case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md).
@@ -0,0 +1,39 @@
{
"experiment": "exp-003-precision-bound-blocking",
"registered": "2026-10-03",
"store": "s3://sast-memory",
"manifest": "plane/experiments/exp-003-precision-bound-blocking.yaml",
"manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
"pin": "dd7ebc119b5ad8e77c05d418eed9cbd46e193587",
"units_status": "frozen",
"units_file": "plane/experiments/exp-003-precision-bound-blocking.units.jsonl",
"units_digest": "338de998b1cfa36857cd793dbc4cf7fd0a13ab517a91f6790335881e941b3006",
"units": {
"total": 353,
"paired": 244,
"groups": 146,
"by_family": {
"access_control": {"units": 41, "positives": 25, "paired": 32, "paired_positives": 16, "groups": 24},
"deserialization": {"units": 48, "positives": 31, "paired": 34, "paired_positives": 17, "groups": 31},
"injection": {"units": 119, "positives": 63, "paired": 76, "paired_positives": 38, "groups": 34},
"output_encoding": {"units": 45, "positives": 29, "paired": 32, "paired_positives": 16, "groups": 27},
"path": {"units": 39, "positives": 23, "paired": 32, "paired_positives": 16, "groups": 23},
"untrusted_destination": {"units": 61, "positives": 37, "paired": 38, "paired_positives": 19, "groups": 36}
},
"note": "frozen model-free on the 937-example static memory; the outer folds (k 5, seed 0, compile fraction 0.25) are the incumbent evaluation's; injection and access_control equal exp-002's 160 units"
},
"kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
"zero_cost": "responses replayed from the response cache only, behind a meter with zero USD and call ceilings around a refusing client; a unit with any uncached response is excluded and counted; the run is void unless the meter reads 0 calls and $0",
"arms": {
"A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds (slope CI contains 1, intercept CI contains 0, ECE <= 0.05); BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
"B": "selective blocking: per family and outer fold, threshold chosen on the family's paired units in the other four folds as the lowest raw score t with Wilson 95% LB of flagged precision >= 0.95 (unsure verdicts never flag); none -> no BLOCK on that fold",
"C": "secondary, non-deciding: as B with point precision >= 0.95 over at least 10 training flags"
},
"primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision, per family, arm B, paired population",
"populations": {"primary": "paired (both sides of the candidate present; excludes every positives-only group and any unpaired unit)", "secondary": ["pooled (every unit)"]},
"secondary_metrics": ["flagged_precision", "coverage (flagged true positives / positives)", "flagged", "repository-cluster bootstrap 95% CI of flagged precision"],
"decision_rule": "REJECT B if in any family B flags held-out paired units at a point precision < 0.95; else ADOPT B if in some family B's held-out paired Wilson LB >= 0.95; else INCONCLUSIVE (the incumbent rule stays)",
"m4_reading": "a family reaches the M4 precision bar on held-out repositories when its held-out paired Wilson LB >= 0.95; point precision reported beside it",
"power": "a Wilson LB of 0.95 needs >= 73 flagged units with no false flag; the largest family holds 38 paired positives over all five folds, so arm B cannot offer a BLOCK in any family by construction; expected verdict inconclusive",
"status": "run and analysed 2026-10-03: inconclusive -- no arm flags any held-out unit (result.json); meter 0 client calls, $0.0; 142 of 353 units excluded for an uncached response"
}
@@ -0,0 +1,981 @@
{
"arms": {
"A": {
"label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
"rule": {
"name": "calibrated_block"
}
},
"B": {
"label": "selective blocking -- per-family raw-score threshold from the other folds' paired units, lowest t with Wilson 95% LB of flagged precision >= 0.95",
"rule": {
"bound": "wilson",
"name": "precision_bound_block",
"precision": 0.95,
"select_on": "paired"
}
},
"C": {
"label": "secondary, non-deciding -- as B with point precision >= 0.95 over at least 10 training flags",
"rule": {
"bound": "point",
"min_flagged": 10,
"name": "precision_bound_block",
"precision": 0.95,
"select_on": "paired"
}
}
},
"code_commit": "7bb97ad0bca62b5f986c5768297d8b8fca50fae4",
"decision": "inconclusive",
"experiment": "exp-003-precision-bound-blocking",
"families": {
"access_control": {
"A": {
"calibration_holds": false,
"ece": 0.077,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 16,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 16,
"true_flagged": 0,
"units": 32
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 24,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 25,
"true_flagged": 0,
"units": 41
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 16,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 16,
"true_flagged": 0,
"units": 32
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 24,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 25,
"true_flagged": 0,
"units": 41
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"C": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 16,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 16,
"true_flagged": 0,
"units": 32
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 24,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 25,
"true_flagged": 0,
"units": 41
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"capacity": {
"evaluated": 41,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 12,
"outer-1": 13,
"outer-2": 13,
"outer-3": 13,
"outer-4": 13
}
}
},
"deserialization": {
"A": {
"calibration_holds": false,
"ece": null,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 3,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 2,
"true_flagged": 0,
"units": 5
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 12,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 11,
"true_flagged": 0,
"units": 14
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 3,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 2,
"true_flagged": 0,
"units": 5
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 12,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 11,
"true_flagged": 0,
"units": 14
},
"thresholds_by_fold": {
"outer-0": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"C": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 3,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 2,
"true_flagged": 0,
"units": 5
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 12,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 11,
"true_flagged": 0,
"units": 14
},
"thresholds_by_fold": {
"outer-0": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"capacity": {
"evaluated": 14,
"excluded_replay_miss": 34,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 0,
"outer-2": 2,
"outer-3": 2,
"outer-4": 2
}
}
},
"injection": {
"A": {
"calibration_holds": false,
"ece": 0.1067,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 26,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 38,
"true_flagged": 0,
"units": 76
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 34,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 63,
"true_flagged": 0,
"units": 119
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 26,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 38,
"true_flagged": 0,
"units": 76
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 34,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 63,
"true_flagged": 0,
"units": 119
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"C": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 26,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 38,
"true_flagged": 0,
"units": 76
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 34,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 63,
"true_flagged": 0,
"units": 119
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"capacity": {
"evaluated": 119,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 29,
"outer-1": 33,
"outer-2": 33,
"outer-3": 26,
"outer-4": 31
}
}
},
"output_encoding": {
"A": {
"calibration_holds": false,
"ece": null,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 8,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 8,
"true_flagged": 0,
"units": 16
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 15,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 16,
"true_flagged": 0,
"units": 24
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 8,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 8,
"true_flagged": 0,
"units": 16
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 15,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 16,
"true_flagged": 0,
"units": 24
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"C": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 8,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 8,
"true_flagged": 0,
"units": 16
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 15,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 16,
"true_flagged": 0,
"units": 24
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"capacity": {
"evaluated": 24,
"excluded_replay_miss": 21,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 7,
"outer-1": 6,
"outer-2": 6,
"outer-3": 7,
"outer-4": 6
}
}
},
"path": {
"A": {
"paired": {
"coverage": null,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 0,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 0,
"true_flagged": 0,
"units": 0
},
"pooled": {
"coverage": null,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 0,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 0,
"true_flagged": 0,
"units": 0
}
},
"B": {
"paired": {
"coverage": null,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 0,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 0,
"true_flagged": 0,
"units": 0
},
"pooled": {
"coverage": null,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 0,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 0,
"true_flagged": 0,
"units": 0
}
},
"C": {
"paired": {
"coverage": null,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 0,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 0,
"true_flagged": 0,
"units": 0
},
"pooled": {
"coverage": null,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 0,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 0,
"true_flagged": 0,
"units": 0
}
},
"capacity": {
"evaluated": 0,
"excluded_replay_miss": 39,
"not_run": 0,
"paired_positives_by_training_folds": {}
}
},
"untrusted_destination": {
"A": {
"calibration_holds": false,
"ece": null,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 6,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 4,
"true_flagged": 0,
"units": 9
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 9,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 7,
"true_flagged": 0,
"units": 13
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null
}
},
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 6,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 4,
"true_flagged": 0,
"units": 9
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 9,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 7,
"true_flagged": 0,
"units": 13
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null
}
},
"C": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 6,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 4,
"true_flagged": 0,
"units": 9
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 9,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 7,
"true_flagged": 0,
"units": 13
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null
}
},
"capacity": {
"evaluated": 13,
"excluded_replay_miss": 48,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 4,
"outer-1": 3,
"outer-2": 3,
"outer-3": 2
}
}
}
},
"hypothesis": "The calibrated-probability BLOCK rule (Platt map + calibration gate, ECE <= 0.05) is the wrong instrument for M4, which asks for precision on what is flagged. Selective blocking -- a per-family raw-score threshold chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95 -- reaches the M4 precision bar on held-out repositories where the calibrated rule offers no BLOCK.\n",
"instrument_check": {
"families": {
"access_control": {
"distinct_s": 30,
"paired": {
"blockable": 28,
"positives_before_first_negative": 10
},
"pooled": {
"blockable": 37,
"positives_before_first_negative": 19
},
"verdicts": {
"not_vulnerable": 19,
"unsure": 4,
"vulnerable": 18
}
},
"deserialization": {
"distinct_s": 5,
"paired": {
"blockable": 4,
"positives_before_first_negative": 1
},
"pooled": {
"blockable": 13,
"positives_before_first_negative": 10
},
"verdicts": {
"not_vulnerable": 2,
"unsure": 1,
"vulnerable": 11
}
},
"injection": {
"distinct_s": 48,
"paired": {
"blockable": 50,
"positives_before_first_negative": 3
},
"pooled": {
"blockable": 77,
"positives_before_first_negative": 11
},
"verdicts": {
"not_vulnerable": 46,
"unsure": 42,
"vulnerable": 31
}
},
"output_encoding": {
"distinct_s": 14,
"paired": {
"blockable": 11,
"positives_before_first_negative": 0
},
"pooled": {
"blockable": 19,
"positives_before_first_negative": 8
},
"verdicts": {
"not_vulnerable": 9,
"unsure": 5,
"vulnerable": 10
}
},
"untrusted_destination": {
"distinct_s": 9,
"paired": {
"blockable": 4,
"positives_before_first_negative": 1
},
"pooled": {
"blockable": 8,
"positives_before_first_negative": 4
},
"verdicts": {
"not_vulnerable": 3,
"unsure": 5,
"vulnerable": 5
}
}
},
"note": "descriptive, all folds together, ranking by raw score s among non-unsure verdicts; not nested, chooses nothing. It shows the scores are real and the zero is the rule's, not an unread input."
},
"manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
"meter": {
"client_calls": 0,
"decided": 211,
"excluded_replay_miss": 142,
"folds_digest": "a1f78e141a572a4adcc9cd57eb8a784c440c177f0b60d5ef5bed9e94c8753c5f",
"metered_usd": 0.0,
"replay_priced_usd_not_spent": 0.582876,
"replayed_responses": 1056
},
"min_flagged_for_bound": 73,
"overall": {
"A": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 59,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 68,
"true_flagged": 0,
"units": 138
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 89,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 122,
"true_flagged": 0,
"units": 211
}
},
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 59,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 68,
"true_flagged": 0,
"units": 138
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 89,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 122,
"true_flagged": 0,
"units": 211
}
},
"C": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 59,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 68,
"true_flagged": 0,
"units": 138
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 89,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 122,
"true_flagged": 0,
"units": 211
}
}
},
"populations": {
"primary": "paired",
"secondary": [
"pooled"
]
},
"primary": [
{
"name": "flagged_precision_lb",
"target": 0.95
}
],
"reason": "B offers no BLOCK on held-out paired units in any family; the incumbent stays",
"resamples": 2000,
"secondary": [
{
"name": "flagged_precision"
},
{
"name": "coverage"
},
{
"name": "flagged"
}
],
"spend_usd": {
"score": 0.0
},
"target_precision": 0.95,
"units": {
"by_label": {
"0": 89,
"1": 122
},
"evaluated": 211,
"excluded_replay_miss": 142,
"frozen": 353,
"groups": 89,
"not_run": 0,
"paired_evaluated": 138,
"unsure": 57
}
}
@@ -0,0 +1,53 @@
"""Instrument check for exp-004: are the scores real, and how clean is the pooled top of the ranking?
Reads the run's arm_outcome rows from the memory store (OUSAST_MEMORY) and the frozen units file; makes no model call.
usage: python benchmarks/experiments/exp-004-pooled-block-gate/instrument_check.py (from the repository root)
"""
from __future__ import annotations
import collections
import json
from pathlib import Path
from openultrasast.config import load_dotenv
from openultrasast.learn import experiments as ex
from openultrasast.plane.memory import open_store
EXPERIMENT = "exp-004-pooled-block-gate"
UNITS = Path(f"plane/experiments/{EXPERIMENT}.units.jsonl")
def main() -> None:
load_dotenv(Path(".env"))
rows = [r.row for r in open_store(None).rows(repo=ex.experiment_repo(EXPERIMENT), kind="arm_outcome")]
with UNITS.open() as handle:
units = {u["unit"]: u for u in (json.loads(line) for line in handle)}
print("arm_outcome rows", len(rows), "units file", len(units))
by_family: dict[str, list[tuple[float | None, int, str | None, str]]] = collections.defaultdict(list)
for row in rows:
unit = units.get(row.get("unit"))
if unit:
by_family[unit["family"]].append((row.get("s"), unit["label"], unit.get("pair"), str(row.get("verdict") or "")))
for family, xs in sorted(by_family.items()):
scored = [x for x in xs if x[0] is not None]
pairs: dict[str, dict[int, float]] = collections.defaultdict(dict)
for score, label, pair, _ in scored:
if pair:
pairs[pair][label] = score
full = [p for p in pairs.values() if 0 in p and 1 in p]
wins = sum(1 for p in full if p[1] > p[0])
ties = sum(1 for p in full if p[1] == p[0])
top = sorted((x for x in scored if x[2]), key=lambda x: -(x[0] or 0.0))
first_negative = next((i for i, x in enumerate(top) if x[1] == 0), len(top))
print(
f"{family}: n={len(xs)} distinct_scores={len({round(x[0] or 0.0, 4) for x in scored})} "
f"unsure={sum(1 for x in xs if x[3] == 'unsure')} pairs={len(full)} within_pair_win={wins} tie={ties} "
f"pos_above_first_neg={first_negative}"
)
pooled = sorted((x for xs in by_family.values() for x in xs if x[0] is not None and x[2]), key=lambda x: -(x[0] or 0.0))
print("pooled raw top-30 labels:", "".join(str(x[1]) for x in pooled[:30]))
if __name__ == "__main__":
main()
@@ -0,0 +1,81 @@
{
"experiment": "exp-004-pooled-block-gate",
"date": "2026-10-03",
"fill": {
"client_calls": 2073,
"usd": 1.238223,
"ceiling_usd": 2.0,
"still_missing": 0,
"dry_run_estimate_usd": 5.582,
"note": "the dry-run estimator over-priced 4.5x (no prompt-cache pricing); measured spend is the meter's"
},
"run": {
"units": 501,
"client_calls": 0,
"usd": 0.0,
"replayed": 2509,
"replay_miss": 0
},
"verdict": "inconclusive",
"reason": "no t_block on any outer fold, pooled or per family; B offers no BLOCK",
"instrument_check": {
"per_family": {
"access_control": {
"n": 66,
"distinct_scores": 41,
"unsure": 5,
"pairs": 28,
"within_pair_wins": 22,
"ties": 1,
"positives_above_first_negative": 12
},
"deserialization": {
"n": 64,
"distinct_scores": 38,
"unsure": 7,
"pairs": 25,
"within_pair_wins": 18,
"ties": 4,
"positives_above_first_negative": 5
},
"injection": {
"n": 149,
"distinct_scores": 52,
"unsure": 54,
"pairs": 53,
"within_pair_wins": 30,
"ties": 17,
"positives_above_first_negative": 6
},
"output_encoding": {
"n": 69,
"distinct_scores": 31,
"unsure": 32,
"pairs": 28,
"within_pair_wins": 18,
"ties": 6,
"positives_above_first_negative": 2
},
"path": {
"n": 66,
"distinct_scores": 43,
"unsure": 15,
"pairs": 29,
"within_pair_wins": 25,
"ties": 3,
"positives_above_first_negative": 13
},
"untrusted_destination": {
"n": 87,
"distinct_scores": 36,
"unsure": 33,
"pairs": 32,
"within_pair_wins": 18,
"ties": 4,
"positives_above_first_negative": 3
}
},
"pooled_raw_top30_labels": "111111111110111110111111110111",
"reading": "scores are real (31-52 distinct values per family); pooled raw top 30 holds 27 positives (0.90), first negative at rank 12; the Wilson bound needs >= 73 flags with no error, so precision at the top, not data volume alone, now binds"
}
}
@@ -0,0 +1,35 @@
{
"experiment": "exp-004-pooled-block-gate",
"registered": "2026-10-03",
"store": "s3://sast-memory",
"manifest": "plane/experiments/exp-004-pooled-block-gate.yaml",
"manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
"pin": "2ad9c1db7ae673c2a2e48a0afadcaa795dfb1ced",
"units_status": "pending",
"units_file": "plane/experiments/exp-004-pooled-block-gate.units.jsonl",
"units_digest": "",
"units": "frozen at run time by `ousast learn experiment units` from the memory after the negatives harvest (merged in 5b2bd9b, 1133 static examples), committed, and registered again before the run; until then the run refuses (check_registered: the units file does not exist)",
"requirement": "learned-decision-engine Req 6.4, amended 2026-10-03 by maintainer decision (refines the BLOCK rule of Req 7.3 and 7.4)",
"kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
"zero_cost": "the run replays the response cache behind a zero-ceiling meter; a unit with any uncached response is excluded and counted. The harvest changed the memory the programs retrieve from, so a paid cache fill, approved separately with its own ceiling, must precede the run",
"normalisation": {
"choice": "per-family Platt scaling: learn.calibrate.fit_map with its existing floors (own slope and intercept per family at >= 20 positive and negative repository groups, a pooled slope with a per-family intercept at >= 10, no map below that and so no BLOCK for that family on that fold), fitted on the training folds' paired units only",
"why": "a rank-to-quantile map gives every family the same flag rate whatever its precision, so one pooled threshold would flag the top slice of a weak family as readily as of a strong one; Platt puts each family on the scale of its own training precision, which a pooled precision bound needs. The map is a normaliser only: precision is shown by the held-out bound, not by calibration (exp-003: ECE <= 0.05 does not hold per family)"
},
"arms": {
"A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds; BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
"B": "pooled gate: per outer fold j, training units normalised by an inner cross-fit; t_block = the lowest normalised score whose Wilson 95% LB of precision over the training paired flags of all families together is >= 0.95 (unsure never flags); t_quiet per family = the highest normalised score keeping >= 90% of that family's training paired positives; fold j's units BLOCK at p >= t_block, ADVISORY at p >= t_quiet (or without a map or t_quiet), silent below"
},
"family_floor": {
"F": 10,
"rule": "a family may block only if its own held-out paired flags (all five folds) number >= 10 with zero errors; otherwise its flags are reported as ADVISORY. The floor uses held-out labels to decide which families block and moves no threshold",
"why": "zero errors in 10 flags is unlikely for a family much worse than the bar (probability 0.056 at precision 0.75, 0.107 at 0.80); a larger F is out of reach of the smaller families (16-19 paired positives in exp-003's units over all folds); 10 is exp-003 arm C's training minimum"
},
"primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision over arm B's paired flags of all families together, before the family floor (so dropping a failing family cannot raise it)",
"populations": {"primary": "paired (both sides of the candidate present)", "secondary": ["pooled (every unit)"]},
"secondary_metrics": ["per-family flagged precision and Wilson LB", "the family-floor table", "quiet_recall per family (held-out positives at or above t_quiet / positives) against 0.90", "after-floor totals (what users would get)", "repository-cluster bootstrap 95% CI of flagged precision", "coverage", "flagged"],
"decision_rule": "on arm B's held-out paired flags of all families together, before the floor: REJECT B if their point precision < 0.95; else ADOPT B if their Wilson 95% LB >= 0.95 and at least one family passes the floor (>= 10 held-out paired flags, zero errors) -- those families block, every other family stays advisory; else INCONCLUSIVE (the incumbent rule stays)",
"power": "the pooled bound needs >= 73 flags with no error (110 with one, 142 with two). On exp-003's units (all 353 scored, result-complete-cache.json) few paired positives rank above the first negative per family on the raw score (access control 10, deserialization 7, injection 2, output encoding 0, path 8, untrusted destination 5); normalisation keeps each family's order, so the pooled top set reaches 73 only if the harvest's pairs separate better. An inconclusive verdict is a plausible outcome",
"tests": "tests/test_learn_rules.py: test_one_pooled_threshold_blocks_where_no_family_alone_reaches_the_bound, test_no_held_out_label_moves_a_threshold_or_the_map, test_a_family_with_one_held_out_error_cannot_block, test_quiet_threshold_keeps_ninety_percent_of_positives_and_nothing_is_silenced_blind, test_the_pooled_verdict_follows_the_registered_rule, test_the_exp004_manifest_registers_the_procedure_before_its_units",
"status": "registered 2026-10-03, units pending, not run"
}
@@ -0,0 +1,822 @@
{
"arms": {
"A": {
"label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
"rule": {
"name": "calibrated_block"
}
},
"B": {
"label": "pooled gate -- one t_block over all families on a per-family Platt score (training folds, inner cross-fit), t_quiet per family at 90% recall, family floor 10 clean held-out flags",
"rule": {
"floor": 10,
"name": "pooled_block_gate",
"normalise": "platt",
"precision": 0.95,
"recall": 0.9,
"select_on": "paired"
}
}
},
"code_commit": "90a2642019df414cb67b5dcea6e4ec235f35650a",
"decision": "inconclusive",
"experiment": "exp-004-pooled-block-gate",
"families": {
"access_control": {
"A": {
"calibration_holds": false,
"ece": 0.0715,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 28,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 28,
"true_flagged": 0,
"units": 56
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 37,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 38,
"true_flagged": 0,
"units": 66
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"floor": {
"errors": 0,
"flagged": 0,
"floor": 10,
"may_block": false
},
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 28,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 28,
"true_flagged": 0,
"units": 56
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 37,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 38,
"true_flagged": 0,
"units": 66
},
"quiet_recall": {
"paired": 0.8929,
"pooled": 0.9211
},
"t_block_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
},
"t_quiet_by_fold": {
"outer-0": 0.44900641988143913,
"outer-1": 0.3723384028044276,
"outer-2": 0.4202772526614184,
"outer-3": 0.4481849709797749,
"outer-4": 0.36346174463707154
}
},
"capacity": {
"evaluated": 66,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 22,
"outer-1": 22,
"outer-2": 22,
"outer-3": 23,
"outer-4": 23
}
}
},
"deserialization": {
"A": {
"calibration_holds": false,
"ece": 0.0725,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 25,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 25,
"true_flagged": 0,
"units": 50
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 39,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 39,
"true_flagged": 0,
"units": 64
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"floor": {
"errors": 0,
"flagged": 0,
"floor": 10,
"may_block": false
},
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 25,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 25,
"true_flagged": 0,
"units": 50
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 39,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 39,
"true_flagged": 0,
"units": 64
},
"quiet_recall": {
"paired": 0.88,
"pooled": 0.9231
},
"t_block_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
},
"t_quiet_by_fold": {
"outer-0": 0.17507194490500488,
"outer-1": 0.22766488322270723,
"outer-2": 0.2163745870743959,
"outer-3": 0.3112201878893598,
"outer-4": 0.363243722721776
}
},
"capacity": {
"evaluated": 64,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 20,
"outer-1": 20,
"outer-2": 20,
"outer-3": 20,
"outer-4": 20
}
}
},
"injection": {
"A": {
"calibration_holds": false,
"ece": 0.1042,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 41,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 53,
"true_flagged": 0,
"units": 106
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 49,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 78,
"true_flagged": 0,
"units": 149
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"floor": {
"errors": 0,
"flagged": 0,
"floor": 10,
"may_block": false
},
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 41,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 53,
"true_flagged": 0,
"units": 106
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 49,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 78,
"true_flagged": 0,
"units": 149
},
"quiet_recall": {
"paired": 0.8679,
"pooled": 0.8974
},
"t_block_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
},
"t_quiet_by_fold": {
"outer-0": 0.287983504097999,
"outer-1": 0.17524866355153343,
"outer-2": 0.12918182665453967,
"outer-3": 0.14521040470628938,
"outer-4": 0.23005714687697673
}
},
"capacity": {
"evaluated": 149,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 38,
"outer-1": 44,
"outer-2": 45,
"outer-3": 43,
"outer-4": 42
}
}
},
"output_encoding": {
"A": {
"calibration_holds": false,
"ece": 0.0736,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 28,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 28,
"true_flagged": 0,
"units": 56
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 39,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 41,
"true_flagged": 0,
"units": 69
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"floor": {
"errors": 0,
"flagged": 0,
"floor": 10,
"may_block": false
},
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 28,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 28,
"true_flagged": 0,
"units": 56
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 39,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 41,
"true_flagged": 0,
"units": 69
},
"quiet_recall": {
"paired": 0.8571,
"pooled": 0.9024
},
"t_block_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
},
"t_quiet_by_fold": {
"outer-0": 0.3723166587726901,
"outer-1": 0.5329072757249385,
"outer-2": 0.3785066578507705,
"outer-3": 0.4389011844848602,
"outer-4": 0.43855966643440264
}
},
"capacity": {
"evaluated": 69,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 23,
"outer-1": 22,
"outer-2": 22,
"outer-3": 22,
"outer-4": 23
}
}
},
"path": {
"A": {
"calibration_holds": false,
"ece": 0.0621,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 29,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 29,
"true_flagged": 0,
"units": 58
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 37,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 37,
"true_flagged": 0,
"units": 66
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"floor": {
"errors": 0,
"flagged": 0,
"floor": 10,
"may_block": false
},
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 29,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 29,
"true_flagged": 0,
"units": 58
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 37,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 37,
"true_flagged": 0,
"units": 66
},
"quiet_recall": {
"paired": 0.8621,
"pooled": 0.8919
},
"t_block_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
},
"t_quiet_by_fold": {
"outer-0": 0.4680507207548809,
"outer-1": 0.3955717127313657,
"outer-2": 0.43694703358760895,
"outer-3": 0.4450583743716199,
"outer-4": 0.46396657833309796
}
},
"capacity": {
"evaluated": 66,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 23,
"outer-1": 23,
"outer-2": 24,
"outer-3": 23,
"outer-4": 23
}
}
},
"untrusted_destination": {
"A": {
"calibration_holds": false,
"ece": 0.1143,
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 31,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 32,
"true_flagged": 0,
"units": 64
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 49,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 50,
"true_flagged": 0,
"units": 87
},
"reason_by_fold": {
"outer-0": "calibration",
"outer-1": "calibration",
"outer-2": "calibration",
"outer-3": "calibration",
"outer-4": "calibration"
},
"thresholds_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
}
},
"B": {
"floor": {
"errors": 0,
"flagged": 0,
"floor": 10,
"may_block": false
},
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 31,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 32,
"true_flagged": 0,
"units": 64
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 49,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 50,
"true_flagged": 0,
"units": 87
},
"quiet_recall": {
"paired": 1.0,
"pooled": 1.0
},
"t_block_by_fold": {
"outer-0": null,
"outer-1": null,
"outer-2": null,
"outer-3": null,
"outer-4": null
},
"t_quiet_by_fold": {
"outer-0": 0.13676746287025585,
"outer-1": 0.11255775703980551,
"outer-2": 0.13763698150768486,
"outer-3": 0.09787938647374639,
"outer-4": 0.13763698150768486
}
},
"capacity": {
"evaluated": 87,
"excluded_replay_miss": 0,
"not_run": 0,
"paired_positives_by_training_folds": {
"outer-0": 26,
"outer-1": 26,
"outer-2": 24,
"outer-3": 26,
"outer-4": 26
}
}
}
},
"hypothesis": "One BLOCK threshold pooled across families on a per-family Platt-normalised score, chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95, reaches that bound on held-out repositories' paired units, where per-family thresholds (exp-003) could not for lack of flags; a per-family floor (>= 10 held-out flags, none wrong) keeps a family that the pooling masks from blocking.\n",
"manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
"min_flagged_for_bound": 73,
"overall": {
"A": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 182,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 195,
"true_flagged": 0,
"units": 390
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 220,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 283,
"true_flagged": 0,
"units": 501
}
},
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 182,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 195,
"true_flagged": 0,
"units": 390
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 220,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 283,
"true_flagged": 0,
"units": 501
}
}
},
"overall_after_floor": {
"B": {
"paired": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 182,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 195,
"true_flagged": 0,
"units": 390
},
"pooled": {
"coverage": 0.0,
"flagged": 0,
"flagged_precision": null,
"flagged_precision_ci": null,
"flagged_precision_lb": null,
"groups": 220,
"meets_target_lb": false,
"meets_target_point": false,
"positives": 283,
"true_flagged": 0,
"units": 501
}
}
},
"populations": {
"primary": "paired",
"secondary": [
"pooled"
]
},
"primary": [
{
"name": "flagged_precision_lb",
"target": 0.95
}
],
"reason": "B offers no BLOCK on held-out paired units; the incumbent stays",
"resamples": 2000,
"secondary": [
{
"name": "flagged_precision"
},
{
"name": "coverage"
},
{
"name": "flagged"
},
{
"name": "quiet_recall"
}
],
"spend_usd": {
"score": 0.0
},
"target_precision": 0.95,
"units": {
"by_label": {
"0": 218,
"1": 283
},
"evaluated": 501,
"excluded_replay_miss": 0,
"frozen": 501,
"groups": 220,
"not_run": 0,
"paired_evaluated": 390,
"unsure": 146
}
}
@@ -0,0 +1,84 @@
{
"arms": [
"B"
],
"client_calls": 2394,
"experiment": "exp-005-graph-slice",
"families": {
"access_control": {
"cached": 0,
"conditional_retries": 330,
"estimate_method": "estimated",
"estimated_output_tokens": 300,
"estimated_usd": 0.9956892000000007,
"filled": 330,
"missing": 330,
"requests": 330,
"still_missing": 0,
"units": 66
},
"deserialization": {
"cached": 0,
"conditional_retries": 310,
"estimate_method": "estimated",
"estimated_output_tokens": 300,
"estimated_usd": 0.8843295999999988,
"filled": 310,
"missing": 310,
"requests": 310,
"still_missing": 0,
"units": 64
},
"injection": {
"cached": 0,
"conditional_retries": 675,
"estimate_method": "estimated",
"estimated_output_tokens": 300,
"estimated_usd": 2.270278999999999,
"filled": 675,
"missing": 675,
"requests": 675,
"still_missing": 0,
"units": 149
},
"output_encoding": {
"cached": 0,
"conditional_retries": 325,
"estimate_method": "estimated",
"estimated_output_tokens": 300,
"estimated_usd": 1.0014180000000004,
"filled": 328,
"missing": 325,
"requests": 328,
"still_missing": 0,
"units": 69
},
"path": {
"cached": 0,
"conditional_retries": 320,
"estimate_method": "estimated",
"estimated_output_tokens": 300,
"estimated_usd": 0.9399192000000005,
"filled": 325,
"missing": 320,
"requests": 325,
"still_missing": 0,
"units": 66
},
"untrusted_destination": {
"cached": 0,
"conditional_retries": 425,
"estimate_method": "estimated",
"estimated_output_tokens": 300,
"estimated_usd": 1.174316,
"filled": 426,
"missing": 425,
"requests": 426,
"still_missing": 0,
"units": 87
}
},
"reason": "",
"status": "done",
"usd": 1.5890382240000016
}
@@ -0,0 +1,165 @@
{
"alpha": 0.05,
"arms": {
"A": {
"budget_usd": 0.0,
"label": "six slice-2 incumbents unchanged, replaying exp-004's response cache",
"repeats": 1,
"replay_only": true
},
"B": {
"budget_usd": 6.0,
"label": "the same six programs with the in-function slice added to their prompts",
"repeats": 1,
"replay_only": false
}
},
"code_commit": "0acbc7ec41dcaeca805c4720bc3f6d02cb940ee7",
"decision": "inconclusive",
"experiment": "exp-005-graph-slice",
"hypothesis": "Adding the in-function def-use slice to the prompt improves within-pair ranking of the six slice-2 incumbent programs on held-out repositories, at no more than 1.5 times the cost per candidate.\n",
"looks": [
{
"boundary": 1.959964,
"crossed": false,
"estimate": {
"a": 0.7615384615384615,
"b": 0.7461538461538462,
"ci": [
-0.0628019323671497,
0.03155339805825241
],
"diff": -0.01538461538461533,
"groups": 220,
"metric": "within_pair_auc",
"resamples": 10000,
"se": 0.024030780235707972,
"units": 501
},
"fraction": 1.0,
"groups": 220,
"z": -0.6402045723740141
}
],
"manifest_digest": "1b1ec814cde12801491376d2086c8c2d6db70f4a86fc308c0adc5a530398e554",
"mde": 0.0,
"primary": [
{
"name": "within_pair_auc"
}
],
"reason": "the CI of within_pair_auc spans 0; the incumbent stays",
"resamples": 10000,
"secondary": [
{
"name": "top_30_precision"
},
{
"name": "unsure_rate"
},
{
"name": "usd_per_candidate"
}
],
"spend_usd": {
"A": 1.444503,
"B": 1.589038
},
"tests": {
"top_30_precision": {
"ci_vs_zero": "spans",
"direction": 1,
"estimate": {
"a": 1.0,
"b": 1.0,
"ci": [
0.0,
0.0
],
"diff": 0.0,
"groups": 220,
"metric": "top_30_precision",
"resamples": 10000,
"se": 0.0006666666666666664,
"units": 501
},
"mcnemar": null,
"target": null
},
"unsure_rate": {
"ci_vs_zero": "favour",
"direction": -1,
"estimate": {
"a": 0.29141716566866266,
"b": 0.23952095808383234,
"ci": [
-0.08583690987124465,
-0.01996007984031939
],
"diff": -0.05189620758483032,
"groups": 220,
"metric": "unsure_rate",
"resamples": 10000,
"se": 0.01685991406098136,
"units": 501
},
"mcnemar": {
"b": 47,
"c": 21,
"concordant": 433,
"p_value": 0.0021862431776861705
},
"target": null
},
"usd_per_candidate": {
"ci_vs_zero": "against",
"direction": -1,
"estimate": {
"a": 0.002883239520958083,
"b": 0.0031717325349301397,
"ci": [
0.0002458205128205199,
0.0003319800399201574
],
"diff": 0.0002884930139720565,
"groups": 220,
"metric": "usd_per_candidate",
"resamples": 10000,
"se": 2.1746161270931252e-05,
"units": 501
},
"mcnemar": null,
"target": null
},
"within_pair_auc": {
"ci_vs_zero": "spans",
"direction": 1,
"estimate": {
"a": 0.7615384615384615,
"b": 0.7461538461538462,
"ci": [
-0.0628019323671497,
0.03155339805825241
],
"diff": -0.01538461538461533,
"groups": 220,
"metric": "within_pair_auc",
"resamples": 10000,
"se": 0.024030780235707972,
"units": 501
},
"mcnemar": null,
"target": null
}
},
"tost": null,
"units": {
"by_label": {
"0": 218,
"1": 283
},
"groups": 220,
"outcomes": 1002,
"paired": 501
}
}
+131 -17
View File
@@ -7,7 +7,7 @@ from:
- quick rules and deterministic roles, run here on the unit's tree: a pair side materialised from its excerpt, a
population or recipe pin exported from its clone or checkout (one at a time, deleted after, disk-guarded);
- the engine: ``benchmarks/learn/engine_pairs.py`` records for pairs, the recorded protocol scans
(``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>.json``) for populations;
(``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>[_a|_b].json``) for populations;
- verify passes a/b and ``agree`` from the harvest Run (``harvest-verify``) and, for population v2, the recorded
``validation-46`` Run; model roles from ``harvest-roles``;
- callers and entry-point distance on a repository (``not_applicable`` on a pair excerpt: no repository exists).
@@ -21,6 +21,11 @@ An instrument that has not run for a unit yet (the engine job still running, a R
memory store (``OUSAST_MEMORY``, default ``~/ousast-results/plane/memory``) as kind ``features``; a rerun replaces
them by id. ``--counts`` writes the counts-only coverage record (no candidate, path or repository name).
``--dry-run`` instead compares existing local store examples with the corrected engine features, printing
per-source transitions from not-run/none to ran (and recorded failures separately). It reads only the spent
v1/v2 manifests, saved scans and local Git objects; it writes neither the store nor ``--counts`` and does not
rebuild other instruments. No label snapshot, harvest units or scratch directory is needed in this mode.
Usage::
python benchmarks/learn/build_features.py --labels labels-<sha>.jsonl --units <harvest plane>/units.json --counts counts.json
@@ -30,9 +35,11 @@ from __future__ import annotations
import argparse
import json
import os
import shutil
import subprocess
import tempfile
import tomllib
from collections import Counter, defaultdict
from collections.abc import Mapping
from pathlib import Path
@@ -61,6 +68,7 @@ from openultrasast.learn.features import (
source_part,
verify_part,
)
from openultrasast.learn.labels import Clone
from openultrasast.learn.roles import RoleSet, from_model_roles, infer_for_checkout
from openultrasast.pairs import DEFAULT_CATALOG, _materialize_side, _targets, load_pair_catalog
from openultrasast.plane import memory
@@ -89,6 +97,109 @@ def read_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8")) if path.is_file() else None
def resolve_scan(scans: Path, case: str, side: str) -> dict[str, Any] | None:
"""Read the v1 single scan, otherwise combine the v2 repeats for this side."""
single = read_json(scans / f"{case}--{side}.json")
if single is not None:
return dict(single)
repeats = [read_json(scans / f"{case}--{side}_{repeat}.json") for repeat in ("a", "b")]
available = [r for r in repeats if r is not None]
if not available:
return None
# Identical rule for vulnerable and fixed: a finding in either repeat counts;
# exact duplicates count once. engine_part takes the highest rung and minimum
# witness steps across the union. Completion is sum(completed)/sum(questions).
# A successful repeat survives a failed/missing peer, which marks degradation.
ran = [r for r in available if engine_for(r) is None]
findings = {json.dumps(f, sort_keys=True): f for r in ran for f in r.get("findings", [])}
degradations = sorted({d for r in available for d in r.get("degradations", [])})
if len(ran) != len(repeats):
degradations.append("repeat_missing_or_failed")
return {
"state": "ran" if ran else "none" if all(engine_for(r) == NONE for r in available) else "failed",
"questions": sum(int(r.get("questions") or 0) for r in available),
"completed": sum(int(r.get("completed") or 0) for r in available),
"findings": list(findings.values()),
"degradations": degradations,
"image": "+".join(sorted({str(r.get("image") or "recorded") for r in available})),
}
def recorded_engine(language: str, findings: list, result: Mapping | None, repository: bool) -> Part:
# Recorded findings take precedence over the shared engine language coverage.
if language not in ENGINE_LANGUAGES and not findings:
return NONE
state = engine_for(result)
if state is not None:
return state
result = result or {}
questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
return engine_part(
findings, completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
) # fmt: skip
class PinSourceIndex(SourceIndex):
"""Resolve findings with the same matcher as a checkout, without exporting it."""
def __init__(self, clone: Path, pin: str) -> None:
super().__init__(clone)
self.clone, self.pin = Clone(clone), pin
def lines(self, path: str) -> list[str]:
if path not in self._lines:
self._lines[path] = self.clone.git("show", f"{self.pin}:{path}").splitlines()
return self._lines[path] or []
def dry_run(store: memory.FileStore, results: Path, cache: Path) -> dict[str, Any]:
"""Count existing examples whose engine becomes available; never write rows or blobs.
Only spent v1/v2 populations are read. Counts are examples, not the two
static/plane feature records. Other sources are reported with zero changes.
"""
root = Path(__file__).resolve().parents[2]
cases = {}
for population in ("population-v1", "population-v2"):
manifest = root / "benchmarks" / "independent" / f"{population}.toml"
for case in tomllib.loads(manifest.read_text(encoding="utf-8"))["case"]:
for side in ("vulnerable", "fixed"):
cases[(population, memory.repo_key(case["repo"]), case[side], side)] = case["id"]
rows = [r.row for r in store.rows(kind="example")]
if not rows:
raise ValueError("dry run read no examples from the store")
counts: dict[str, Counter[str]] = defaultdict(Counter)
for row in rows:
count = counts[row["source"]]
count["examples"] += 1
old = row["instruments"]["engine"]
reason = "not_run" if old.get("version") == "missing:not-run" else "none" if old["state"] == "none" else None
if reason is None:
continue
key = (row["split"], row["repo"], row["pin"], row["pin_role"])
case_id = cases.get(key)
if case_id is None:
continue
scans = results / row["split"].replace("population", "independent") / "scans"
result = resolve_scan(scans, case_id, row["pin_role"])
path, _, function = row["candidate"].partition("::")
engine = {}
if engine_for(result) is None:
index = PinSourceIndex(cache / "independent" / case_id, row["pin"])
lines = index.lines(path) # fail loudly if the candidate's input cannot be read
count["source_bytes_read"] += len("\n".join(lines).encode("utf-8"))
selected = dict(result or {})
selected["findings"] = [f for f in selected.get("findings", []) if f["site"].partition(":")[0] == path]
_, engine = _signals(index, [], selected, {})
part = recorded_engine(row["language"], engine.get((path, function, row["family"]), []), result, True)
if part.state == "ran":
count[f"{reason}_to_ran"] += 1
elif result is not None and part.state == "failed":
count[f"{reason}_to_recorded_failed"] += 1
return {source: {**{"not_run_to_ran": 0, "none_to_ran": 0}, **dict(count)} for source, count in sorted(counts.items())}
def done(run: Path, entry: str) -> bool:
state = (read_json(run / "state.json") or {}).get("tasks", {})
return state.get(entry, {}).get("status") == "done"
@@ -188,17 +299,7 @@ def static_parts(root: Path, index: SourceIndex, quick: Mapping, engine: Mapping
key = (path, function, family)
parts: dict[str, Part] = {}
parts["quick"] = quick_part(quick.get(key, []), rules, quick_version) if covers(QUICK_LANGUAGES, language) else NONE
if language not in ENGINE_LANGUAGES:
parts["engine"] = NONE
elif engine_state is not None:
parts["engine"] = engine_state
else:
result = engine_result or {}
questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
parts["engine"] = engine_part(
engine.get(key, []), completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
) # fmt: skip
parts["engine"] = recorded_engine(language, engine.get(key, []), engine_result, repository)
lines = index.lines(path)
parts["source"] = source_part(lines, language, function)
parts["roles"] = roles_part(function_of(lines, path, language, function), inferred, language)
@@ -237,21 +338,34 @@ def export(clone: Path, pin: str, target: Path) -> bool:
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--labels", type=Path, required=True)
parser.add_argument("--units", type=Path, required=True, help="units.json of the harvest plane")
parser.add_argument("--labels", type=Path)
parser.add_argument("--units", type=Path, help="units.json of the harvest plane")
parser.add_argument("--catalog", type=Path, default=DEFAULT_CATALOG)
parser.add_argument("--cache", type=Path, default=Path.home() / ".cache" / "openultrasast")
parser.add_argument("--engine", type=Path, default=RESULTS / "plane" / "harvest-engine")
parser.add_argument("--verify-run", type=Path, default=RESULTS / "plane" / "harvest-verify")
parser.add_argument("--roles-run", type=Path, default=RESULTS / "plane" / "harvest-roles")
parser.add_argument("--v46", type=Path, default=RESULTS / "plane" / "validation-46")
parser.add_argument("--scratch", type=Path, required=True)
parser.add_argument("--scratch", type=Path)
parser.add_argument("--min-free-gb", type=float, default=1.2)
parser.add_argument("--counts", type=Path, required=True)
parser.add_argument("--counts", type=Path)
parser.add_argument("--store", help="OUSAST_MEMORY spec (default: the environment's)")
parser.add_argument("--limit", type=int, default=0, help="stop after this many targets (a test)")
parser.add_argument("--kinds", default="pairs,pin", help="which targets: pairs, pin (population and recipe pins)")
parser.add_argument("--results-root", type=Path, default=RESULTS)
parser.add_argument("--dry-run", action="store_true", help="read local stored examples and print engine changes by source; no writes")
args = parser.parse_args()
if args.dry_run:
spec = args.store or os.environ.get("OUSAST_MEMORY") or ""
if spec and not spec.startswith("file://"):
parser.error("--dry-run requires a local file:// store (no network)")
store = memory.open_store(spec)
if not isinstance(store, memory.FileStore):
parser.error("--dry-run requires a local file:// store (no network)")
print(json.dumps({"dry_run": True, "by_source": dry_run(store, args.results_root, args.cache)}, sort_keys=True))
return 0
if any(getattr(args, name) is None for name in ("labels", "units", "scratch", "counts")):
parser.error("--labels, --units, --scratch and --counts are required unless --dry-run is used")
labels = [r for r in jsonl(args.labels) if r["unit"] == "pin" and r["source"] not in CONDITIONAL_SOURCES and not r.get("conditional")]
units = json.loads(args.units.read_text(encoding="utf-8"))
@@ -314,7 +428,7 @@ def main() -> int:
continue
split = {r["split"] for r in rows} | {unit["source"]}
population = "independent-v1" if "population-v1" in split else "independent-v2"
engine_result = read_json(RESULTS / population / "scans" / f"{case_id}--{rows[0]['pin_role']}.json")
engine_result = resolve_scan(args.results_root / population / "scans", case_id, rows[0]["pin_role"])
stems = [(args.verify_run, n) for n in names] + [(args.v46, unit["ref"])]
owner = next(
(n for n in names if f"{n}-roles" in {e for e in (read_json(args.roles_run / "state.json") or {}).get("tasks", {})}),
+175
View File
@@ -0,0 +1,175 @@
"""Read-only, zero-model slice audit. Run: env -u OPENROUTER_API_KEY python benchmarks/learn/slice_leak_audit.py.
Only sources.toml itself is read, never its referenced corpora. Stored source IDs
are the reporting strata (the store merges advisory pair catalogs into `pairs`).
Ties get half credit. Both signed and best-direction scores are recorded; like
learn.leaks, a reversed label shortcut is still a leak. Missing classes/pairs
produce null metrics, not plausible zeros. Unsupported slices score zero for both
indicators and remain separately counted in coverage.
"""
from __future__ import annotations
import hashlib
import json
import os
from collections import Counter, defaultdict
from collections.abc import Mapping, Sequence
from pathlib import Path
from typing import Any
from openultrasast.config import load_dotenv
from openultrasast.learn.examples import load_examples
from openultrasast.learn.labels import load_sources
from openultrasast.learn.slice import function_slice
from openultrasast.plane.memory import MemoryStore, open_store
UNITS = Path("plane/experiments/exp-004-pooled-block-gate.units.jsonl")
OUTPUT = Path("benchmarks/measurements/2026-10-03-graph-slice-leak-audit/record.json")
THRESHOLD = 0.70
def complete_pairs(rows: Sequence[Mapping[str, Any]]) -> list[tuple[Mapping[str, Any], Mapping[str, Any]]]:
groups: dict[tuple[str, str], list[Mapping[str, Any]]] = defaultdict(list)
for row in rows:
if row.get("pair"):
groups[(row["family"], row["pair"])].append(row)
pairs = []
for members in groups.values():
if len(members) == 2 and {m["label"] for m in members} == {0, 1}:
v, f = sorted(members, key=lambda m: -m["label"])
if v["group"] != f["group"] or v["fold"] != f["fold"]:
raise ValueError("Pair crosses group or fold")
pairs.append((v, f))
elif len(members) != 1:
raise ValueError("Ambiguous pair")
return pairs
def metrics(rows: Sequence[Mapping[str, Any]], signal: str) -> dict[str, Any]:
pairs = complete_pairs(rows)
higher = sum(v[signal] > f[signal] for v, f in pairs)
lower = sum(v[signal] < f[signal] for v, f in pairs)
ties = len(pairs) - higher - lower
pair_score = (higher + 0.5 * ties) / len(pairs) if pairs else None
positive = [r[signal] for r in rows if r["label"] == 1]
negative = [r[signal] for r in rows if r["label"] == 0]
auc = (
sum((v > f) + 0.5 * (v == f) for v in positive for f in negative) / (len(positive) * len(negative))
if positive and negative
else None
)
best_pair = max(pair_score, 1 - pair_score) if pair_score is not None else None
best_auc = max(auc, 1 - auc) if auc is not None else None
return {
"rows": len(rows),
"pairs": len(pairs),
"positive": len(positive),
"negative": len(negative),
"higher_on_vulnerable": higher,
"higher_on_fixed": lower,
"ties": ties,
"within_pair_accuracy": pair_score,
"pooled_auc": auc,
"best_direction_within_pair_accuracy": best_pair,
"best_direction_pooled_auc": best_auc,
"flagged": any(v is not None and v >= THRESHOLD for v in (best_pair, best_auc)),
}
def audit(store: MemoryStore, units: Sequence[Mapping[str, Any]], *, expected_rows: int = 501, expected_pairs: int = 195) -> dict[str, Any]:
if len(units) != expected_rows or len({u["unit"] for u in units}) != expected_rows:
raise ValueError(f"Expected {expected_rows} unique units, read {len(units)}")
if len(complete_pairs(units)) != expected_pairs:
raise ValueError(f"Expected {expected_pairs} complete pairs")
examples = load_examples(store)
index = {e.id: e for e in examples}
if len(index) != len(examples):
raise ValueError("Duplicate example IDs in memory")
sources = {s.id for s in load_sources().sources}
coverage: dict[str, Counter[str]] = defaultdict(Counter)
rows = []
excerpt_bytes = 0
blobs: dict[str, str] = {}
for unit in units:
e = index[unit["unit"]] # missing input is fatal, never counted as no-flow
if (e.family, e.label, e.group) != (unit["family"], unit["label"], unit["group"]):
raise ValueError(f"Unit metadata differs from store: {e.id}")
if e.source not in sources:
raise ValueError(f"Source absent from sources.toml: {e.source}")
if e.excerpt_sha not in blobs:
# MemoryStore.get_blob(verify=True) DELETES corrupt blobs: do not use it.
data = store.get_blob("excerpts", e.excerpt_sha, verify=False)
if not data or hashlib.sha256(data).hexdigest() != e.excerpt_sha:
raise ValueError(f"Missing, empty or corrupt excerpt: {e.id}")
excerpt_bytes += len(data)
blobs[e.excerpt_sha] = data.decode("utf-8")
result = function_slice(blobs[e.excerpt_sha], e.language, e.family)
status = "not-analysable" if result is None else "flow" if result.steps else "no-flow"
coverage[e.family][status] += 1
rows.append(
{
**unit,
"source": e.source,
"status": status,
"has_flow": int(status == "flow"),
"slice_length": len(result.steps) if result else 0,
}
)
pairs = complete_pairs(rows)
asymmetry = Counter(
"both" if v["has_flow"] and f["has_flow"] else "vulnerable_only" if v["has_flow"] else "fixed_only" if f["has_flow"] else "neither"
for v, f in pairs
)
scopes = {"all": rows, **{f"source:{s}": [r for r in rows if r["source"] == s] for s in sorted({r["source"] for r in rows})}}
return {
"cost_usd": 0,
"threshold": THRESHOLD,
"unit_rows_read": len(units),
"example_rows_read": len(examples),
"matched_rows": len(rows),
"excerpts_read": len(blobs),
"excerpt_units": len(rows),
"excerpt_bytes": excerpt_bytes,
"complete_pairs": len(pairs),
"coverage": {f: {s: c[s] for s in ("flow", "no-flow", "not-analysable")} for f, c in sorted(coverage.items())},
"pair_asymmetry": {k: asymmetry[k] for k in ("vulnerable_only", "fixed_only", "both", "neither")},
"signals": {
signal: {scope: metrics(members, signal) for scope, members in scopes.items()} for signal in ("has_flow", "slice_length")
},
"source_strata": "Stored source IDs from sources.toml; advisory pair catalogs are merged into pairs by the example builder.",
"limitations": (
"Flat IR: no dominance/guards, textual branch order, potential parameter inputs, conservative call returns; no alias analysis."
),
"units": rows,
}
def main() -> None:
load_dotenv(Path(".env"))
os.environ.pop("OPENROUTER_API_KEY", None) # dotenv must not restore the removed key
raw = UNITS.read_bytes()
units = [json.loads(line) for line in raw.decode().splitlines() if line.strip()]
report = audit(open_store(), units)
report["units_bytes"] = len(raw)
report["units_sha256"] = hashlib.sha256(raw).hexdigest()
OUTPUT.parent.mkdir(parents=True, exist_ok=True)
OUTPUT.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n")
totals = Counter({s: sum(c[s] for c in report["coverage"].values()) for s in ("flow", "no-flow", "not-analysable")})
lines = [
f"Slice leak audit: $0; units bytes={len(raw)}",
f"Rows read: units={len(units)}, examples={report['example_rows_read']}, matched={report['matched_rows']}",
f"Excerpts read: {report['excerpts_read']}; units={report['excerpt_units']}; bytes={report['excerpt_bytes']}",
f"Coverage: {dict(totals)}",
f"Complete pairs: {report['complete_pairs']}",
f"Pair asymmetry: {report['pair_asymmetry']}",
f"Has flow: {report['signals']['has_flow']['all']}",
f"Slice length: {report['signals']['slice_length']['all']}",
f"Source strata: {', '.join(report['signals']['has_flow'])}; threshold={THRESHOLD}",
f"Report: {OUTPUT}",
]
print("\n".join(lines))
if __name__ == "__main__":
main()
@@ -1,7 +1,7 @@
{
"modules": 151,
"modules": 159,
"counts": {
"load_bearing": 146,
"load_bearing": 154,
"standalone_capability": 5
},
"orphaned": [],
@@ -229,6 +229,8 @@
"model.report",
"pairs",
"plane.tasks.alerts",
"push.quick",
"push.runner",
"rank",
"regress.candidate",
"reports",
@@ -356,12 +358,14 @@
"learn.excerpt",
"learn.experiments",
"learn.features",
"learn.fill",
"learn.folds",
"learn.labels",
"learn.leaks",
"learn.program",
"learn.retrieve",
"learn.roles",
"learn.slice",
"plane.harvest",
"plane.tasks.features",
"plane.tasks.remember",
@@ -410,7 +414,8 @@
"importers": [
"learn.cli",
"learn.compile",
"learn.experiments"
"learn.experiments",
"learn.fill"
]
},
{
@@ -422,6 +427,7 @@
"learn.compile",
"learn.evaluate",
"learn.experiments",
"learn.fill",
"learn.folds",
"learn.program",
"learn.retrieve"
@@ -435,6 +441,7 @@
"learn.embeddings",
"learn.examples",
"learn.labels",
"learn.slice",
"plane.harvest"
]
},
@@ -443,7 +450,8 @@
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"learn.cli"
"learn.cli",
"learn.fill"
]
},
{
@@ -455,6 +463,14 @@
"plane.tasks.features"
]
},
{
"module": "learn.fill",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"learn.cli"
]
},
{
"module": "learn.folds",
"classification": "load_bearing",
@@ -500,7 +516,8 @@
"learn.cli",
"learn.compile",
"learn.evaluate",
"learn.experiments"
"learn.experiments",
"learn.fill"
]
},
{
@@ -539,6 +556,14 @@
"plane.tasks.roles"
]
},
{
"module": "learn.slice",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"learn.program"
]
},
{
"module": "map_gate",
"classification": "load_bearing",
@@ -560,6 +585,7 @@
"pairs",
"plane.tasks.alerts",
"preprocess",
"push.quick",
"push.runner"
]
},
@@ -579,9 +605,12 @@
"cli",
"cpg.backend",
"cpg.session",
"findings",
"learn.cli",
"learn.features",
"learn.fill",
"learn.labels",
"learn.slice",
"model.candidates",
"model.config_value",
"model.dominance",
@@ -602,6 +631,7 @@
"push.cache",
"push.contracts",
"push.policy",
"push.quick",
"push.report",
"push.reuse",
"push.runner",
@@ -649,6 +679,7 @@
"push.cache",
"push.contracts",
"push.policy",
"push.quick",
"push.report",
"push.reuse",
"push.runner",
@@ -672,6 +703,7 @@
"importers": [
"cli",
"learn.cli",
"learn.fill",
"plane.budget",
"plane.generate",
"push.assistance",
@@ -716,7 +748,9 @@
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"model.scan"
"findings",
"model.scan",
"push.runner"
]
},
{
@@ -775,6 +809,7 @@
"classification": "load_bearing",
"reason": "model-grounded-detection Req 7.2-7.4: the security vocabularies the CPG queries are parameterised by",
"importers": [
"learn.slice",
"model.config_value",
"model.dominance",
"model.pipeline",
@@ -846,7 +881,10 @@
"learn.embeddings",
"learn.examples",
"learn.experiments",
"learn.fill",
"learn.program",
"plane.delivery",
"plane.doctor",
"plane.egress",
"plane.engine_alerts",
"plane.generate",
@@ -871,12 +909,21 @@
"learn.compile",
"learn.embeddings",
"learn.experiments",
"learn.fill",
"learn.program",
"plane.generate",
"plane.tasks.roles",
"plane.tasks.verify"
]
},
{
"module": "plane.delivery",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"plane.reconciler"
]
},
{
"module": "plane.doctor",
"classification": "load_bearing",
@@ -924,6 +971,7 @@
"reason": "ai-service-plane Req 1.1-1.4: ax-shaped Task/Workspace/Model and the Run kind, validated before anything runs",
"importers": [
"learn.cli",
"plane.delivery",
"plane.egress",
"plane.generate",
"plane.harvest",
@@ -945,12 +993,27 @@
"learn.embeddings",
"learn.examples",
"learn.experiments",
"learn.fill",
"learn.program",
"plane.delivery",
"plane.doctor",
"plane.generate",
"plane.reconciler",
"plane.tasks.loop",
"plane.tasks.remember"
]
},
{
"module": "plane.profile",
"classification": "load_bearing",
"reason": "python -m openultrasast.plane.profile",
"importers": [
"cli",
"plane.doctor",
"plane.generate",
"plane.reconciler"
]
},
{
"module": "plane.reconciler",
"classification": "load_bearing",
@@ -1093,6 +1156,7 @@
"improve.evolve",
"improve.validator",
"plane.tasks.loop",
"push.quick",
"scoring.project_score",
"semantic.overlay"
]
@@ -1132,6 +1196,7 @@
"plane.tasks.alerts",
"plane.tasks.features",
"plane.tasks.repo_facts",
"push.quick",
"push.runner",
"rank",
"regress.candidate",
@@ -1186,6 +1251,7 @@
"importers": [
"cli",
"push.assistance",
"push.background",
"push.eligibility",
"push.policy",
"push.report",
@@ -1203,6 +1269,14 @@
"push.runner"
]
},
{
"module": "push.background",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"push.runner"
]
},
{
"module": "push.cache",
"classification": "load_bearing",
@@ -1234,6 +1308,23 @@
"push.runner"
]
},
{
"module": "push.explain",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"push.report",
"push.runner"
]
},
{
"module": "push.install",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"cli"
]
},
{
"module": "push.policy",
"classification": "load_bearing",
@@ -1245,12 +1336,21 @@
"push.vetoes"
]
},
{
"module": "push.quick",
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"push.runner"
]
},
{
"module": "push.report",
"classification": "load_bearing",
"reason": "pre-push-safety-net Req 5.1/5.2/5.5/7.3: compact output and complete artifacts; runner integration pending",
"importers": [
"push.assistance",
"push.background",
"push.runner"
]
},
@@ -1315,7 +1415,8 @@
"improve.evolve",
"map_gate",
"pairs",
"plane.tasks.alerts"
"plane.tasks.alerts",
"push.quick"
]
},
{
@@ -1501,6 +1602,7 @@
"hunter_tools",
"learn.features",
"learn.roles",
"learn.slice",
"mapping",
"model.candidates",
"model.layout",
@@ -1557,6 +1659,7 @@
"hunter_tools",
"learn.features",
"learn.roles",
"learn.slice",
"mapping",
"model.candidates",
"model.layout",
@@ -1590,6 +1693,7 @@
"importers": [
"cli",
"hunter_tools",
"learn.slice",
"model.candidates",
"pairs",
"semantic.cst",
@@ -1723,6 +1827,7 @@
"classification": "load_bearing",
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"learn.slice",
"semantic.overlay"
]
},
@@ -1748,6 +1853,8 @@
"reason": "reachable from gate, map_gate, pair_gate, cli",
"importers": [
"cli",
"learn.cli",
"learn.fill",
"model.endpoint",
"plane.budget",
"plane.tasks.roles",
@@ -0,0 +1,5 @@
[ok] profile k3s: exec=remote kube_context=ousast-k3s registry=ghcr.io/norandom images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=ghcr-pull
[FAIL] cluster ousast-k3s: context ousast-k3s unreachable (error: context "ousast-k3s" does not exist); docs/deployment.md lists the prerequisites
[FAIL] runner image in ghcr.io/norandom: images file /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json: No such file or directory; docs/deployment.md lists the prerequisites
[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
exit=1 seconds=1.1
@@ -0,0 +1,7 @@
[ok] profile kind: exec=local kube_context=kind-ousast registry=localhost:5001 images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)
[ok] cluster kind-ousast: reachable
[ok] agent substrate (ate-system): 9 pods ready
[ok] ax controller (ax-system): 4 pods ready
[ok] runner image in localhost:5001: ousast-runner@sha256:95edc7ed5621... present in localhost:5001
[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
exit=0 seconds=1.4
@@ -0,0 +1,19 @@
F.. [100%]
=================================== FAILURES ===================================
________ test_no_host_literal_outside_the_profile_and_the_kind_bring_up ________
def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None:
kind = load_profile("kind", environ={})
found = offenders(_files(), kind.registry)
> assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found)
E AssertionError: host-only literals (Req 1.3):
E src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration
E assert ['src/openult...emonstration'] == []
E
E Left contains one more item: 'src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration'
E Use -v to get more diff
tests/test_plane_literals.py:81: AssertionError
=========================== short test summary info ============================
FAILED tests/test_plane_literals.py::test_no_host_literal_outside_the_profile_and_the_kind_bring_up
1 failed, 2 passed in 0.26s
@@ -0,0 +1,2 @@
... [100%]
3 passed in 0.23s
@@ -0,0 +1,16 @@
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: ousast-probe-big-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: ousast-probe-big, atespace: default}
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "2560Mi"}, limits: {cpu: "1", memory: "3Gi"}}
workspaces: [{name: ousast-probe-big-ws, path: "/workspace"}]
@@ -0,0 +1,16 @@
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: ousast-probe-small-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: ousast-probe-small, atespace: default}
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "256Mi"}, limits: {cpu: "1", memory: "512Mi"}}
workspaces: [{name: ousast-probe-small-ws, path: "/workspace"}]
@@ -0,0 +1,26 @@
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
apiVersion: ate.dev/v1alpha1
kind: WorkerPool
metadata:
name: probe-pool
namespace: ax-system
labels:
workload: ousast
spec:
replicas: 1
workerImage: localhost:5001/ateom-gvisor-715889664656de67e44382a8d6ab981d@sha256:220f1cf13ea485b73a2e4270fa544901a18ae826d8f22bd54d42761944a41c5c
template:
nodeSelector:
ate.dev/substrate-version: "4f9c293"
resources:
limits:
cpu: "2"
memory: 3Gi
requests:
cpu: 250m
memory: 3Gi
@@ -0,0 +1,78 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T05:24:59.069304031Z",
"version": "1758"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:29:30.551668391Z",
"version": "2"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
@@ -0,0 +1,81 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T18:29:45.281958347Z",
"version": "1761"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"allocated": {
"actors": 1
},
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:29:30.551668391Z",
"version": "2"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
@@ -0,0 +1,81 @@
{
"workers": [
{
"ip": "10.244.0.22",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-09-29T16:32:08.594663703Z",
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
"updateTime": "2026-10-02T18:29:53.001457323Z",
"version": "1763"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"allocated": {
"actors": 1
},
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "1"
},
{
"name": "memory",
"quantity": "1536Mi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
"workerPool": "ousast-pool"
},
{
"ip": "10.244.0.28",
"labels": {
"workload": "ousast"
},
"metadata": {
"createTime": "2026-10-02T18:29:30.486026311Z",
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
"updateTime": "2026-10-02T18:30:04.825058843Z",
"version": "4"
},
"nodeName": "ousast-control-plane",
"sandboxClass": "gvisor",
"status": {
"capacity": {
"actors": 1000,
"resources": {
"limits": [
{
"name": "cpu",
"quantity": "2"
},
{
"name": "memory",
"quantity": "3Gi"
}
]
}
},
"state": "WORKER_STATE_ACTIVE"
},
"workerNamespace": "ax-system",
"workerPod": "probe-pool-5dffd74bdc-dqlww",
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
"workerPool": "probe-pool"
}
]
}
@@ -0,0 +1,132 @@
{
"record": "k8s-01-profile",
"spec": "plane-on-kubernetes",
"tasks": ["1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8"],
"date": "2026-10-02",
"host": "the maintainer's laptop: 7.9 GiB, single-node kind cluster, RustFS store reachable over https",
"commits": {
"1.1": "8eac1b2",
"1.2": "77a03ba",
"1.3": "36f5927",
"1.4": "8099b29",
"1.5": "86ec237",
"1.6": "2afa20e",
"1.7": "2cca03c",
"1.8": "the commit that adds this line (git log: plane(1.8))"
},
"images": {
"runner_before": "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884",
"runner_after_up_sh": "localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb",
"pin_file": "ops/k8s/profiles/kind-images.json (written by ops/ax/up.sh; the templates re-pinned to it, 128 image lines)"
},
"instrument_checks": {
"code_under_test": "PYTHONPATH=<worktree>/src first; `python -c 'import openultrasast; print(openultrasast.__file__)'` printed the worktree path before every gate",
"store_reached": {
"how": "scratch store_probe.py: open_store('s3://sast-memory') through the worktree code (verify_bucket runs), then read the probe object",
"open_and_verify_seconds": 0.62,
"probe_key": "_probe/select.jsonl",
"probe_bytes": 24,
"probe_version_id_present": true
},
"smoke_delivery_sizes_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291, "task.yaml": 1148},
"registry_resolution": "doctor HEADs /v2/ousast-runner/manifests/<digest> on the kind registry: HTTP 2xx for the new digest"
},
"doctor": {
"kind": {
"command": "ousast plane doctor --profile kind",
"exit": 0,
"seconds": 1.4,
"lines": [
"[ok] profile kind: exec=local kube_context=... registry=... images=... router_url=(tunnel) ax_server=(tunnel) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)",
"[ok] cluster <kind context>: reachable",
"[ok] agent substrate (ate-system): 9 pods ready",
"[ok] ax controller (ax-system): 4 pods ready",
"[ok] runner image in <kind registry>: ousast-runner@sha256:95edc7ed5621... present",
"[ok] memory store s3://sast-memory: opens (an S3 bucket is verified)"
],
"file": "doctor-kind.txt"
},
"k3s": {
"command": "ousast plane doctor --profile k3s",
"exit": 1,
"seconds": 1.1,
"failing_lines": [
"cluster ousast-k3s: context does not exist (expected: the cluster does not exist yet)",
"runner image in ghcr.io/norandom: images file ops/k8s/profiles/images.json missing (expected: the release asset is published by task 4.1)"
],
"passing_lines": ["profile k3s: every address reported", "memory store s3://sast-memory: opens"],
"note": "two expected failures, not one: the images file is the release asset and does not exist before group 4; the namespaces are not checked behind an unreachable context, so no third line fails for the same cause",
"file": "doctor-k3s.txt"
}
},
"literal_scan": {
"test": "tests/test_plane_literals.py",
"scope": "src/, plane/, ops/ (minus ops/ax/up.sh, ops/k8s/profiles/), docs/",
"patterns": ["kind context", "kind registry", "kind gateway 172.19.", "loopback with port", "port-forward"],
"exemptions": [
"router.py for loopback-with-port and port-forward",
"lines tagged ax-tunnel or loopback (two: the runner's own health URL, the Joern session's query URL)",
"digest-pinned image: lines under plane/tasks/ whose registry equals the kind profile's (a third test asserts every one does)"
],
"offenders_at_1_3": 0,
"offenders_before_1_3": 23,
"reinsertion_demo": {
"inserted": "a kind-context literal appended to src/openultrasast/plane/workspaces.py",
"pytest_exit_with_literal": 1,
"named": "src/openultrasast/plane/workspaces.py:37: [kind context]",
"restored": "byte for byte (cmp)",
"pytest_exit_restored": 0,
"files": ["literal-reinserted-fail.txt", "literal-restored-pass.txt"]
}
},
"up_sh_rerun": {
"command": "OUSAST_PYTHON=<main venv python> ops/ax/up.sh",
"exit": 0,
"seconds": 38,
"kept": ["cluster", "substrate", "egress gateway", "ax control plane", "worker pool ousast-pool (2/2 ready)", "ax CLI (newer than the checkout)"],
"done": ["receiver Service re-applied", "runner image rebuilt and pushed (new digest above)", "kind-images.json written", "smoke Task applied and resumed"],
"first_attempt_failure": "`profile --print images` answered `no image named ''`; fixed in profile._lookup (an images path with no sub-key is the file path) and covered by test_plane_profile",
"disk_gb_free_before_after": [1.7, 1.5],
"file": "up-sh-rerun.txt"
},
"smoke_run_1_4": {
"command": "ops/ax/smoke-run.sh (receiver delivery: the path group 2 replaces)",
"exit": 0,
"seconds": 24,
"status": "done",
"delivered": "facts.json 451 B, summary.json 238 B, units.jsonl 291 B; summary.image = the new digest, units 2/2",
"file": "smoke-run-1.4.txt"
},
"placement_probe_1_5": {
"script": "ops/ax/probe-placement.sh",
"pools": {"ousast-pool": "scaled to 1 worker, 1 CPU / 1536Mi", "probe-pool": "1 worker, 2 CPU / 3Gi, the already-built ateom-gvisor image (no ko build)"},
"big_actor": {"requests_memory": "2560Mi", "limits_memory": "3Gi", "landed_on": "ousast-pool (1536Mi worker): status.allocated.actors=1; probe-pool 0"},
"small_actor": {"requests_memory": "256Mi", "limits_memory": "512Mi", "landed_on": "ousast-pool: allocated.actors=1; probe-pool 0"},
"placement": "any",
"consequence": "design.md section 4: group 5 takes the fallback (an engine atespace with engine-pool as its only pool if Substrate scopes pools by atespace; else a plain Kubernetes Job on k3s and alerts-engine on kind)",
"resume_latency_seconds": "first resume of each new template succeeded at once (golden snapshot built within the resume deadline); whole probe 46 s",
"cleanup": "probe tasks deleted, probe-pool deleted, ousast-pool back to 2/2 (verified by kubectl get workerpool)",
"files": ["probe/probe.log", "probe/workers-before.json", "probe/workers-ousast-probe-big.json", "probe/workers-ousast-probe-small.json", "probe/ousast-probe-big.yaml", "probe/ousast-probe-small.yaml", "probe/probe-pool.yaml"]
},
"reconciler_bound_1_6": {
"lines_at_spec_commit_6bd44ca": 477,
"lines_after_1_2": 489,
"added_by_1_2": "the profile parameter of run() and its docstring (+12)",
"estimate_2_4": "Delivery wiring: open the store, build Delivery, pass the delivery body to the starter, delivered() and collect() in the await/execute path, state mirroring, the OUSAST_DELIVERY=receiver switch: about +25 while the receiver is kept (group 2 keeps both paths), about -20 when 2.6 removes the receiver wiring",
"projected_during_group_2": 514,
"projected_after_2_6": 494,
"decision": "the estimate crosses 500 while both paths coexist: task 2.4 moves the bound in tests/test_plane_reconciler.py to 550 with the reason in the assertion message, and 2.6 records the final count"
},
"gvisor_tarball_1_7": {
"status": "planned, manual",
"where": "docs/deployment.md B.1",
"url": "gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd (sha256-pinned in substrate/manifests/ate-install/sandboxconfig-gvisor.yaml)",
"check": "the atelet's fetch line in `kubectl -n ate-system logs` of a worker pod on the k3s node; labelled planned until read there"
},
"gates": {
"1.1": "pytest 2002 passed + module-audit manifest regenerated (test_module_audit 6 passed), ruff 0, format 0, mypy 0",
"1.2": "pytest 2006 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"1.3": "pytest 2009 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0; mkdocs --strict 0",
"1.4": "see the commit message"
}
}
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - 0 n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=24
@@ -0,0 +1,58 @@
== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (20:35:35)
cluster exists; kept
== agent substrate (ate-system) (20:35:35)
substrate ready; kept
NAME READY STATUS RESTARTS AGE
ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (12h ago) 3d
ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (12h ago) 3d
ate-controller-c95586684-g7nvn 1/1 Running 0 3d2h
atelet-4f9c293-9zpfg 1/1 Running 0 3d2h
atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d
atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (12h ago) 3d2h
postgres-0 2/2 Running 3 (3d2h ago) 3d2h
rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d2h ago) 3d2h
rustfs-bucket-init-nvfr6 0/1 Completed 0 3d2h
== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (20:35:35)
gateway present; kept
== ax control plane (ax-system) (20:35:35)
ax ready; kept
NAME READY STATUS RESTARTS AGE
ax-redis-86df786c87-pz5tp 1/1 Running 0 3d2h
ax-server-b479bdc76-hwkqb 1/1 Running 0 3d2h
ousast-pool-6656b55f5b-5brkn 1/1 Running 0 5m23s
ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d2h
== artifact receiver Service (ousast-receiver.ax-system -> this host) (20:35:35)
service/ousast-receiver unchanged
endpointslice.discovery.k8s.io/ousast-receiver-host configured
== worker pool (gVisor): pools.default of the profile (20:35:36)
pool ousast-pool exists (2/2 ready); kept
pod/ousast-pool-6656b55f5b-5brkn condition met
pod/ousast-pool-6656b55f5b-9wbrw condition met
== runner image localhost:5001/ousast-runner (20:35:37)
sha256:f004a5ef8f71b1cb8a5880c509389c0569181af7b19170b574958cc3ef75cb1c
localhost:5001/ousast-runner:dev
runner image localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
== ax CLI from the deployed checkout (a release CLI skews from the server) (20:36:05)
ax CLI newer than the checkout; kept
== smoke task (20:36:05)
task.ax.io/ousast-smoke created
workspace.ax.io/ousast-smoke-ws unchanged
task.ax.io/ousast-smoke resumed
NAME ATESPACE PHASE ACTOR WORKER-IP AGE
ousast-smoke default Running ousast-smoke 10.244.0.29 5s
== footprint (20:36:11)
total used free shared buff/cache available
Mem: 7877 4797 225 63 3464 3079
kind-registry 53.77MiB / 7.693GiB 0.00%
up.sh exit 0
@@ -0,0 +1,24 @@
{
"metadata": {
"atespace": "default",
"createTime": "2026-10-02T19:39:19.428045847Z",
"name": "default",
"uid": "741e52b7-aaf1-4a91-b4bf-fc3b20ba4a36",
"updateTime": "2026-10-02T19:39:19.428045847Z",
"version": "1"
},
"rules": [
{
"tlsPassthrough": {
"hostnames": [
"files.because-security.com"
],
"ports": {
"numbers": [
443
]
}
}
}
]
}
@@ -0,0 +1,83 @@
{
"record": "k8s-02-store-delivery",
"spec": "plane-on-kubernetes",
"tasks": ["2.1", "2.2", "2.3", "2.4", "2.5", "2.6"],
"date": "2026-10-02",
"commits": {
"2.1": "35d8032",
"2.2": "9506a18",
"2.3": "c49eae4",
"2.4": "605b7af",
"2.5": "the commit that adds this file (git log: plane(2.5))",
"receiver_removed": "pending: task 2.6 (paused 2026-10-03 by the maintainer's reprioritisation; the receiver path still exists behind OUSAST_DELIVERY=receiver)"
},
"images": {
"runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019",
"built_by": "ops/ax/up.sh rerun (file up-sh-rebuild.txt): every step kept but the runner build; the image carries the 2.2 runner",
"disk_gb_free_after_build": 1.3
},
"instrument_checks": {
"code_under_test": "the worktree's src first on PYTHONPATH; `ousast` run as `python -m openultrasast.cli` from the worktree (ops/ax/smoke-run.sh with OUSAST_PYTHON)",
"store_read_back": {
"how": "scratch store_tar_check.py opens the store through the worktree code (verify_bucket runs), lists runs/ax-e2e-smoke/, downloads output.tar, compares its members with the files the reconciler extracted (file store-check.json)",
"tar_bytes": 10240,
"tar_members_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
"extracted_files_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
"members_equal_extracted": true,
"member_bytes_total": 980,
"note": "a tar is 512-byte blocks: 10240 B of tar carry 980 B of files; done.json records the tar's byte size"
},
"no_receiver_in_the_task": "the rendered task.yaml is 996 B against 1148 B on the receiver path: no OUSAST_ARTIFACT_URL/DIAL in the Task's env; the start body carries the URLs"
},
"smoke_runs": {
"store_1": {
"command": "ops/ax/smoke-run.sh (store delivery, the default after 2.4; OUSAST_DELIVERY unset)",
"exit": 0,
"seconds_submit_to_done": 26,
"tar_put_at": "2026-10-02T19:37:24+00:00 (the object's LastModified)",
"collected_at": "2026-10-02T19:37:26+00:00 (done.json finished)",
"put_to_collect_seconds": 2,
"poll_interval_seconds": 3,
"file": "smoke-run-store-1.txt"
},
"seeded_rerun": {
"command": "the same, with the local results root deleted and the store's runs/ax-e2e-smoke/ intact",
"exit": 0,
"seconds": 2,
"what_happened": "state.json seeded from the store: every task done, nothing submitted to ax (plane-on-kubernetes 2.1, design section 2: a rerun from another machine resumes from the store)",
"file": "smoke-run-seeded-rerun.txt"
},
"store_2": {
"command": "the same, after deleting runs/ax-e2e-smoke/ from the store (5 objects) so the task executes again, with the egress policy polled",
"exit": 0,
"seconds_submit_to_done": 24,
"tar_put_at": "2026-10-02T19:39:38+00:00",
"collected_at": "2026-10-02T19:39:40+00:00",
"put_to_collect_seconds": 2,
"file": "smoke-run-store-2.txt"
}
},
"objects_delivered_to_the_store": {
"runs/ax-e2e-smoke/facts/output.tar": 10240,
"runs/ax-e2e-smoke/facts/facts.json": 451,
"runs/ax-e2e-smoke/facts/summary.json": 238,
"runs/ax-e2e-smoke/facts/done.json": 149,
"runs/ax-e2e-smoke/state.json": 350,
"note": "units.jsonl is extracted locally but not a declared output of the smoke Run, so it is not re-put"
},
"egress_policy_read_back": {
"how": "kubectl ate get egress-policy ax-e2e-smoke-facts -a default -o json, polled while the actor lived (file egress-policy-read-back.json)",
"rules": [{"tlsPassthrough": {"hostnames": ["files.because-security.com"], "ports": {"numbers": [443]}}}],
"reading": "one TLS rule naming the store host, no http rule; the model-free smoke task names no Git host and no Model host",
"also": "Egress.apply reads every policy back and raises on a mismatch, so each completed Run is itself the equality check"
},
"status_table": "ousast plane status ax-e2e-smoke printed the attribution table (facts: done, calls 0) in every run; the table is in the smoke-run-*.txt files",
"reconciler_lines": {"after_2_4": 549, "bound": 550, "note": "both paths coexist until 2.6; the receiver wiring goes then"},
"gates": {
"2.1": "pytest 2020 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.2": "pytest 2024 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.3": "pytest 2027 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.4": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
"2.5": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0 (first attempt 2026-10-02: 11 failed + 5 errors, all the FileStore 1 GiB free-space refusal at 0.97 GiB free; rerun after the disk was freed)"
}
}
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - n/a n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=2
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - 0 n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=26
@@ -0,0 +1,9 @@
run finished: done
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
facts - 0 n/a n/a n/a n/a [done]
----------------------------------------------------------------------------------------------------
total - n/a n/a n/a n/a n/a
smoke exit=0 seconds=24
@@ -0,0 +1,36 @@
{
"store": "s3://sast-memory",
"objects_under_run": {
"runs/ax-e2e-smoke/facts/done.json": 149,
"runs/ax-e2e-smoke/facts/facts.json": 451,
"runs/ax-e2e-smoke/facts/output.tar": 10240,
"runs/ax-e2e-smoke/facts/summary.json": 238,
"runs/ax-e2e-smoke/state.json": 350
},
"tar_bytes": 10240,
"tar_members": {
"facts.json": 451,
"summary.json": 238,
"units.jsonl": 291
},
"extracted_files": {
"facts.json": 451,
"units.jsonl": 291,
"summary.json": 238
},
"members_equal_extracted": true,
"member_bytes_total": 980,
"extracted_bytes_total": 980,
"done_json": {
"bytes": 10240,
"files": 3,
"finished": "2026-10-02T19:39:40+00:00",
"published": [
"facts.json",
"summary.json"
],
"status": "done",
"task": "facts"
},
"tar_put_at_last_modified": "2026-10-02T19:39:38+00:00"
}
@@ -0,0 +1,58 @@
== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (21:36:18)
cluster exists; kept
== agent substrate (ate-system) (21:36:18)
substrate ready; kept
NAME READY STATUS RESTARTS AGE
ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (13h ago) 3d1h
ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (13h ago) 3d1h
ate-controller-c95586684-g7nvn 1/1 Running 0 3d3h
atelet-4f9c293-9zpfg 1/1 Running 0 3d3h
atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d1h
atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (13h ago) 3d3h
postgres-0 2/2 Running 3 (3d3h ago) 3d3h
rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d3h ago) 3d3h
rustfs-bucket-init-nvfr6 0/1 Completed 0 3d3h
== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (21:36:18)
gateway present; kept
== ax control plane (ax-system) (21:36:19)
ax ready; kept
NAME READY STATUS RESTARTS AGE
ax-redis-86df786c87-pz5tp 1/1 Running 0 3d3h
ax-server-b479bdc76-hwkqb 1/1 Running 0 3d3h
ousast-pool-6656b55f5b-5brkn 1/1 Running 0 66m
ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d3h
== artifact receiver Service (ousast-receiver.ax-system -> this host) (21:36:19)
service/ousast-receiver unchanged
endpointslice.discovery.k8s.io/ousast-receiver-host configured
== worker pool (gVisor): pools.default of the profile (21:36:20)
pool ousast-pool exists (2/2 ready); kept
pod/ousast-pool-6656b55f5b-5brkn condition met
pod/ousast-pool-6656b55f5b-9wbrw condition met
== runner image localhost:5001/ousast-runner (21:36:21)
sha256:0501baf180b29f7aa7bbb8888bebabfc5d62429b7bdaa7f5a62b5b188f8ad5f6
localhost:5001/ousast-runner:dev
runner image localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
== ax CLI from the deployed checkout (a release CLI skews from the server) (21:36:32)
ax CLI newer than the checkout; kept
== smoke task (21:36:32)
task.ax.io/ousast-smoke created
workspace.ax.io/ousast-smoke-ws unchanged
task.ax.io/ousast-smoke resumed
NAME ATESPACE PHASE ACTOR WORKER-IP AGE
ousast-smoke default Running ousast-smoke 10.244.0.29 4s
== footprint (21:36:38)
total used free shared buff/cache available
Mem: 7877 5202 382 63 2903 2674
kind-registry 32.45MiB / 7.693GiB 0.00%
up.sh exit 0
@@ -0,0 +1,792 @@
{
"id": "2026-10-02-pre-push-operational-1",
"task": "pre-push-safety-net 17.7",
"requirements": [
"9.1",
"9.2",
"9.3",
"9.4",
"9.5",
"9.6",
"6.3"
],
"question": "Does the hook installed on any repository give honest, advisory-only results, and never block a push?",
"subjects": "The audit's three shallow clones (kennethreitz/records, expressjs/serve-static, unrolled/secure), each with the same planted commit c2 on c1, copied fresh from the audit's scratch clones.",
"instrument_proof": "Every replay and hook push records snapshots[].bytes_read and quick_tier files/bytes_read. The head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head (records 43,577; serve-static 61,525; secure 103,023), and the quick tier reads exactly the changed source files.",
"host": "Joern is not on the host, so host runs cannot run the engine; that is the case a first-time pip user hits, and the output must say so. The Docker engine run uses the 2026-09-14 openultrasast:dev image with the worktree src mounted read-only, as the audit did; the image was not rebuilt (1.1 GB disk free, below the 4 GB threshold).",
"before": {
"source": "/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready/gap-report.md (audit at main 49e11e0)",
"replay_host": {
"records": {
"exit": 0,
"wall_seconds": 1.06,
"files_bytes_read": [
21,
43577
],
"findings_shown": 0,
"terminal": "generic: Some checks could not be completed. Review the comparison details...",
"skip_reasons_named_in_terminal": 0
},
"serve-static": {
"exit": 0,
"wall_seconds": 1.1,
"files_bytes_read": [
26,
61525
],
"findings_shown": 0,
"terminal": "same generic line",
"skip_reasons_named_in_terminal": 0
},
"secure": {
"exit": 0,
"wall_seconds": 1.04,
"files_bytes_read": [
19,
103023
],
"findings_shown": 0,
"terminal": "same generic line; Go not named",
"skip_reasons_named_in_terminal": 0
}
},
"docker_engine_serve_static": {
"deadline_30": {
"exit": 0,
"wall_seconds": 31.3,
"terminal": "Analysis did not finish. Rerun with a longer analysis deadline."
},
"deadline_300": {
"exit": 0,
"wall_seconds": 261.7,
"raw_engine_findings": 2,
"admitted": 0,
"dispositions": "injection and output_encoding at debug.js:2: comparison_unknown/base_incomplete + capability_unavailable",
"terminal": "No actionable defects reported; Details: /out/... (container path)"
}
},
"quick_scan": {
"serve-static": {
"exit": 0,
"wall_seconds": 0.46,
"findings": "js-eval, js-xss-response at debug.js:2"
},
"records": {
"exit": 0,
"wall_seconds": 0.49,
"findings": "2 pre-existing (setup.py os-command); planted %-formatted SQL missed"
},
"secure": {
"exit": 0,
"wall_seconds": 0.48,
"findings": 0,
"terminal": "No quick-mode findings were emitted (silent zero for Go)"
}
},
"hook": {
"ousast_not_on_path": {
"push_exit": "rejected",
"terminal": ".git/hooks/pre-push: 13: exec: ousast: not found"
},
"first_push_new_branch": {
"push_exit": 0,
"wall_seconds": 0.58,
"terminal": "No analysis result is available. (does not say new branch)"
},
"update": {
"push_exit": 0,
"wall_seconds": 1.36,
"terminal": "No actionable defects reported; analysis is incomplete."
},
"bad_OUSAST_PUSH_DEADLINE_or_cache_inside_repo": {
"exit": 2,
"effect": "would block the push"
},
"install": "ops/install-pre-push only (checkout); not in the wheel"
}
},
"verdict": {
"never_blocks": "PASS in this sample: 10 hook pushes across the three repositories all exited 0, including `ousast` absent from PATH (rejected before) and OUSAST_PUSH_DEADLINE=abc (exit 2 before). Blocking stays an explicit opt-in (unit-tested, not exercised here).",
"never_hides": "PASS in this sample: every incomplete run printed one plain `Skipped:` line per reason (2-5 per run), with no hex path or sha256 token. Go is named as not covered (`language not covered: go (1 changed file(s), 10 in the repository)`, and `notice: language not covered: go (10 files)` in `scan --mode quick`). The first push of a new branch says there is no base and what it checked instead.",
"quick_tier_within_deadline": "On the host, every replay and hook push finished in 0.69-1.21 s against the 30 s deadline. The quick tier read exactly the changed source file (records 1 file / 17,449 B; serve-static 1 / 87 B; secure 1 / 193 B).",
"planted_js_eval": "SURFACES. Quick tier: js-eval (CWE-95) and js-xss-response (CWE-79) at debug.js:2 on the host, 0.9 s, in replay and in the hook push. Engine (Docker, deadline 300): injection and output_encoding at debug.js:2, now novelty `new` / `file_added_in_head` (before: `unknown` / `base_incomplete`). The only veto is `capability_unavailable`, so it is shown as an advisory engine finding, not an alert.",
"planted_python_sql": "STILL MISSED. No quick rule matches the %-formatted db.query string in records.py, and the host has no Joern, so nothing in the pre-push path reports it. Recorded as a coverage gap, with no rule edit.",
"planted_go_command": "NOT DETECTED, NOW NAMED: Go has no engine frontend and no quick rules.",
"engine_deadline": "Not solved. The Docker engine took 190.9 s wall time (head 107.6 s, base 80.7 s) against a 300 s deadline. The audit measured 261.7 s on a different image/src combination. A 30 s hook deadline still cannot hold it. --engine background starts it after the push, and the next push showed the earlier result. On the host that result says 'the engine did not run (Joern unavailable)', which is honest."
},
"coverage_gaps": [
"records: the planted Python SQL injection (`db.query(\"... '%s'\" % name)`) is matched by no quick rule, before or after. Recorded as a quick-rule coverage gap; no rule was edited for a planted sample (that would be hand-tuning). On the host the engine cannot run, so nothing in the pre-push path reports it.",
"secure: Go has no engine frontend and no quick rules. It is now named in the terminal (language not covered: go), not detected.",
"The engine still cannot finish inside the 30 s hook deadline (the audit measured 260 s on 26 files); --engine background moves it after the push, graph reuse remains the later speed task."
],
"after": {
"replay_host": {
"records": {
"pre-push --engine inline": {
"exit": 0,
"wall_seconds": 1.206878771,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"changed lines could not be matched one-to-one to base lines in records.py",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
],
"snapshots_files_bytes": [
[
21,
43577
],
[
21,
43482
]
],
"quick_tier_files_bytes": [
[
1,
17449
]
],
"quick_tier_seconds": [
0.041
],
"coverage_reasons_raw": [
"7265636f7264732e7079:ambiguous_line_correspondence",
"base_counterpart_unresolved",
"cpg_build_failed",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.483
},
"pre-push --engine off": {
"exit": 0,
"wall_seconds": 0.954010589,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"changed lines could not be matched one-to-one to base lines in records.py",
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
],
"snapshots_files_bytes": [
[
21,
43577
],
[
21,
43482
]
],
"quick_tier_files_bytes": [
[
1,
17449
]
],
"quick_tier_seconds": [
0.042
],
"coverage_reasons_raw": [
"7265636f7264732e7079:ambiguous_line_correspondence",
"engine_off",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.498
},
"scan --mode quick": {
"exit": 0,
"wall_seconds": 0.407089313,
"terminal": [
"file_targets=9",
"findings=2"
]
}
},
"serve-static": {
"pre-push --engine inline": {
"exit": 0,
"wall_seconds": 0.892543629,
"deadline_seconds": 30,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [],
"skip_lines": [
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
],
"snapshots_files_bytes": [
[
26,
61525
],
[
25,
61438
]
],
"quick_tier_files_bytes": [
[
1,
87
]
],
"quick_tier_seconds": [
0.017
],
"coverage_reasons_raw": [
"base_counterpart_unresolved",
"cpg_build_failed",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.496
},
"pre-push --engine off": {
"exit": 0,
"wall_seconds": 0.878791036,
"deadline_seconds": 30,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [],
"skip_lines": [
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
],
"snapshots_files_bytes": [
[
26,
61525
],
[
25,
61438
]
],
"quick_tier_files_bytes": [
[
1,
87
]
],
"quick_tier_seconds": [
0.016
],
"coverage_reasons_raw": [
"engine_off",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.471
},
"scan --mode quick": {
"exit": 0,
"wall_seconds": 0.394269229,
"terminal": [
"file_targets=4",
"findings=2"
]
}
},
"secure": {
"pre-push --engine inline": {
"exit": 0,
"wall_seconds": 0.762186729,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"engine has no frontend for at least one language in this push",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
],
"snapshots_files_bytes": [
[
19,
103023
],
[
18,
102830
]
],
"quick_tier_files_bytes": [
[
1,
193
]
],
"quick_tier_seconds": [
0.015
],
"coverage_reasons_raw": [
"language_not_covered:go:1:10",
"base_counterpart_unresolved",
"cpg_build_failed",
"frontend_unsupported",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.352
},
"pre-push --engine off": {
"exit": 0,
"wall_seconds": 0.694374205,
"deadline_seconds": 30,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
],
"snapshots_files_bytes": [
[
19,
103023
],
[
18,
102830
]
],
"quick_tier_files_bytes": [
[
1,
193
]
],
"quick_tier_seconds": [
0.016
],
"coverage_reasons_raw": [
"language_not_covered:go:1:10",
"engine_off",
"analysis_incomplete",
"base_comparison_unavailable"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 0.311
},
"scan --mode quick": {
"exit": 0,
"wall_seconds": 0.372512766,
"terminal": [
"file_targets=10",
"findings=0",
"notice: language not covered: go (10 files)"
]
}
}
},
"hook_host": {
"records": {
"new-branch": {
"push_exit": 0,
"wall_seconds": 0.873912782,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"New branch: instead, quick rules checked the files its last commit changed (compared with c925575).",
"Quick rules: no pattern match on changed code (2 file(s), 18416 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
" * [new branch] HEAD~1 -> main"
]
},
"update": {
"push_exit": 0,
"wall_seconds": 0.890046656,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"changed lines could not be matched one-to-one to base lines in records.py",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
" ea42736..ad3d7db HEAD -> main"
]
},
"install_uninstall": [
"install rc=0",
"uninstall rc=0"
]
},
"serve-static": {
"new-branch": {
"push_exit": 0,
"wall_seconds": 0.904302643,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"New branch: instead, quick rules checked the files its last commit changed (compared with 76312e1).",
"Quick rules: no pattern match on changed code (0 file(s), 0 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" * [new branch] HEAD~1 -> main"
]
},
"update": {
"push_exit": 0,
"wall_seconds": 0.92386555,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [],
"skip_lines": [
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" 74be78a..468213d HEAD -> main"
]
},
"no-ousast": {
"push_exit": 0,
"wall_seconds": 0.017937758,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [],
"other_lines": [
"ousast: pre-push check did not run: 'ousast' is not on PATH. Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" 468213d..aad3dfe HEAD -> main"
]
},
"bad-deadline": {
"push_exit": 0,
"wall_seconds": 0.300776,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [],
"other_lines": [
"usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]",
" [--comparison-base COMPARISON_BASE]",
" [--model-config MODEL_CONFIG] [--prior-hook PRIOR_HOOK]",
" --artifact ARTIFACT [--cache-dir CACHE_DIR]",
" [--deadline DEADLINE]",
" [--cancellation-allowance CANCELLATION_ALLOWANCE]",
" [--max-regions MAX_REGIONS]",
" [--mode {advisory,blocking}]",
" [--incomplete-coverage {allow,block}]",
" [--engine {inline,background,off}]",
" [--background-deadline BACKGROUND_DEADLINE]",
" [--experimental-declarations EXPERIMENTAL_DECLARATIONS]",
" [--experimental-record-vetoes]",
" path",
"ousast pre-push: error: argument --deadline: invalid float value: 'abc'",
"ousast: pre-push check did not run: invalid arguments or settings (message above). Push continues.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" aad3dfe..6fcfec7 HEAD -> main"
]
},
"background-1": {
"push_exit": 0,
"wall_seconds": 0.915950565,
"quick_findings_shown": [],
"engine_findings_shown": [
"Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result."
],
"skip_lines": [
"engine checks run in the background after this push; the next push shows their result",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 93 bytes read).",
"No analysis result is available.",
"Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" 6fcfec7..c5b6c4d HEAD -> main"
]
},
"background-2": {
"push_exit": 0,
"wall_seconds": 0.879868436,
"quick_findings_shown": [],
"engine_findings_shown": [
"Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable)."
],
"skip_lines": [
"engine checks were switched off for this run (--engine off)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable).",
"Quick rules: no pattern match on changed code (1 file(s), 99 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
" c5b6c4d..3eee5ef HEAD -> main"
]
},
"install_uninstall": [
"install rc=0",
"uninstall rc=0"
]
},
"secure": {
"new-branch": {
"push_exit": 0,
"wall_seconds": 0.96295861,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"New branch: instead, quick rules checked the files its last commit changed (compared with ba42bc3).",
"Quick rules: no pattern match on changed code (1 file(s), 41228 bytes read).",
"No analysis result is available.",
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
" * [new branch] HEAD~1 -> main"
]
},
"update": {
"push_exit": 0,
"wall_seconds": 0.747049238,
"quick_findings_shown": [],
"engine_findings_shown": [],
"skip_lines": [
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
"the changed code has no counterpart in the base revision to compare against",
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
"engine has no frontend for at least one language in this push",
"no base revision was analysed, so no engine finding can be called new or worsened"
],
"other_lines": [
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
"No actionable defects reported; analysis is incomplete.",
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
" 740abc8..64739c4 HEAD -> main"
]
},
"install_uninstall": [
"install rc=0",
"uninstall rc=0"
]
}
},
"docker_engine_serve_static": {
"log": [
"MemAvailable=3283MB disk_free=1159MiB",
"base=74be78a8ffad679edfe135c457ce141114d96fcc head=468213d8a737ad03a1f712c792c1db8476e154df",
"exit=0 elapsed=190.895648822s"
],
"exit": 0,
"wall_seconds": 190.895648822,
"deadline_seconds": 300,
"quick_findings_shown": [
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
],
"engine_findings_shown": [
"Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert."
],
"skip_lines": [
"engine findings are not qualified as alerts yet: no capability has passed independent evaluation",
"the capability registry does not match this build (eligibility provenance mismatch), so no capability can be used",
"20 engine check(s) depend on dynamic or external code the engine cannot follow"
],
"other_lines": [
"Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert.",
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
"No actionable defects reported; analysis is incomplete.",
"Notice: Some checks are not enabled for normal alerts. Review capability coverage in the details."
],
"snapshots_files_bytes": [
[
26,
61525
],
[
25,
61438
]
],
"quick_tier_files_bytes": [
[
1,
87
]
],
"quick_tier_seconds": [
0.038
],
"coverage_reasons_raw": [
"capability_unavailable",
"eligibility:eligibility_provenance_mismatch",
"dynamic_external_or_depth_context_unresolved:contributor-scan:7ba72698b15c888c0835cad443fc1f177f1d2e13ed2658b17e816cc0b81415de",
"dynamic_external_or_depth_context_unresolved:contributor-scan:392beff95d0e245cbaef2676813d7f69d35599fb32977c8f197e6df46738fc75",
"dynamic_external_or_depth_context_unresolved:contributor-scan:ec16a67e984a2f82c3ce9a04dbc44eae1e950393b1c1c4fae88485446f1e2f95",
"dynamic_external_or_depth_context_unresolved:contributor-scan:0a3bd3f145260792e9a5d4af2546020689cfd08d0b1e82264d7fe31dda7210e7",
"dynamic_external_or_depth_context_unresolved:contributor-scan:1551c000ba35ba4e5488013f51171b27ab10df8b80f29509610670192cff8a30",
"dynamic_external_or_depth_context_unresolved:contributor-scan:5b687fa7647cefd89d064b3e1b5e5a62842f8fd23104900e8af8d5b14fc800fd",
"dynamic_external_or_depth_context_unresolved:contributor-scan:f122da538137e0a3a3e598c24e6af57c5e0b90ff48dacb4fd1b9d82ac20c35ef",
"dynamic_external_or_depth_context_unresolved:contributor-scan:8ab9ddcb1f4431fa3a3c1cce4cc3d289bcb5d363d47aac1bcf35bc7db86ce866",
"dynamic_external_or_depth_context_unresolved:contributor-scan:19f1031c2c010d91fb41e23955cd5f18eaa5ede355e68fab9e18c8249c21c2ee",
"dynamic_external_or_depth_context_unresolved:contributor-scan:d31a861124274a09e3cb8f3398831ca299cc0cfad72ea385083c1fd88b7a2c45",
"dynamic_external_or_depth_context_unresolved:contributor-scan:c8a1c8042befd3ff6f296921666949850a1c54d63c77dd44296e7be96e621127",
"dynamic_external_or_depth_context_unresolved:contributor-scan:3bb50c50f77027f1e0347f490e35fdd0a75ae586fdf92775c753877833a9d099",
"dynamic_external_or_depth_context_unresolved:contributor-scan:c32cb93cad379fee7b814900e6673aa5ae470c27acd55d33f855987bccb1ab8e",
"dynamic_external_or_depth_context_unresolved:contributor-scan:300fba1acc39cb0ab32ebbd66a8674daecf496b3d8c675d27f7f71cf9f788c9e",
"dynamic_external_or_depth_context_unresolved:contributor-scan:40c99dfe6d923602dc3aa6353f3b5bc017c0d1a67595c19ddf8206ef431db23d",
"dynamic_external_or_depth_context_unresolved:contributor-scan:0e99666c9be7c4704f6c6f858844d721ed8f0b4e3978d53073bc2c1b64d3f075",
"dynamic_external_or_depth_context_unresolved:contributor-scan:da903214d018e0b1bd7a3a3b635b3e0fb2a23fd44527f41a9483c0bf3913f2df",
"dynamic_external_or_depth_context_unresolved:contributor-scan:79d89b41b86aadcc0423baa786e0518ccff2a5b7853ff05986682ecbd636af52",
"dynamic_external_or_depth_context_unresolved:contributor-scan:7d3331ca3c06d1013c758d3648bfebb1223385955e77df977266fc621f62e733",
"dynamic_external_or_depth_context_unresolved:contributor-scan:54fd93450c8db7e6daef06fe795babb26726402211ca8f6d0a8679a375786041",
"analysis_incomplete"
],
"admitted_alerts": 0,
"push_disposition": "allow",
"total_seconds_internal": 189.327,
"dispositions": [
{
"family": "injection",
"site": "debug.js:2:debug",
"novelty": "new",
"reason": "file_added_in_head",
"vetoes": [
"capability_unavailable"
]
},
{
"family": "output_encoding",
"site": "debug.js:2:debug",
"novelty": "new",
"reason": "file_added_in_head",
"vetoes": [
"capability_unavailable"
]
}
],
"timings": {
"deadline_seconds": 300.0,
"cancellation_allowance_seconds": 2.0,
"resolution_seconds": 0.0,
"provenance_seconds": 0.3,
"snapshot_seconds": 0.3,
"discovery_seconds": 0.3,
"preparation_seconds": 0.6,
"quick_seconds": 0.0,
"head_seconds": 107.6,
"base_seconds": 80.7,
"admission_seconds": 0.0,
"cleanup_seconds": 0.0,
"total_seconds": 189.3
}
}
},
"build": {
"worktree_head": "591818d + hook install-hint fix (this commit)",
"wheel_sha256": "5b725adb052239c2afaec237fecf582f69fc5ea61799dc67f0c07e09e10908c9",
"installed_with": "uv pip install <wheel> into a scratch venv"
},
"inputs": {
"records": {
"language": "python",
"tracked_files": 21,
"tracked_bytes": 43577,
"planted": "records.py: db.query(\"... '%s'\" % name)"
},
"serve-static": {
"language": "javascript",
"tracked_files": 26,
"tracked_bytes": 61525,
"planted": "new debug.js: res.end(String(eval(req.query.code)))"
},
"secure": {
"language": "go",
"tracked_files": 19,
"tracked_bytes": 103023,
"planted": "new run.go: exec.Command(\"sh\", \"-c\", r.URL.Query().Get(\"cmd\"))"
}
},
"notes": [
"The Docker engine run used the worktree src at 591818d mounted over the stale 2026-09-14 openultrasast:dev image (not rebuilt: 1.1 GB disk free, below the 4 GB threshold). `eligibility_provenance_mismatch` still appears; the audit already suspected src/image skew, and it is not investigated here.",
"The run shows host paths after `Details:` because ops/ousast-docker mounts the artifact directory at its own path. Before, it showed the container path /out/...",
"The hook's 'not on PATH' hint first said 'pip install openultrasast', which is not on PyPI. The first proof run caught it. It was corrected to the git URL, and the replay and hook proofs were rerun on the rebuilt wheel recorded under build. The Docker engine run predates the hint fix and does not involve the hook script.",
"Host runs read their input. Each head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head, and the quick tier lists the files it read."
]
}
@@ -0,0 +1,351 @@
{
"created": "2026-10-03",
"note": "COUNTS ONLY. Real (vulnerable function, fixed function) pairs where the function survives the fix, mined at zero model cost from reviewed GitHub advisories for the six compiled families, added as pointer-pair corpus benchmarks/pairs/advisory-fixes-2 (98 pairs, source `advisory-fixes-2` in learn/sources.toml), labelled, given static feature records and built into the memory store; their excerpts embedded. No chat-model call. Candidate, path, function and repository names stay in the catalog; the two pairs the reserved-population test flagged are withheld (never printed: found by bisection with the test's output discarded).",
"method": {
"enumeration": "gh api /advisories?type=reviewed&ecosystem=<eco>&cwes=<family CWEs>&published=>2025-06-01, per ecosystem (pip, npm, maven, composer, go, rubygems) x family; newest first",
"advisories_enumerated": 6594,
"advisories_published_after_2026_06_01": 2400,
"automated_screen": "single family by CWE; a linked fix commit (at most 3, one repository); repository not named anywhere under benchmarks/ (populations v1/v2, every pair corpus; excluded v3 file names never opened), no WordPress plugin, one pair per repository; SPDX licence permissive; one-parent commit with <= 4 non-test source files and <= 120 changed source lines; the function holding most changed CODE lines (comments and blanks excluded) inside its own matcher span, declared exactly once on both sides, body changed, >= 25% of the file's changed code lines (second pass; the first pass counted lines by nearest declaration above)",
"file_reads": "the two versions of each changed source file through the contents API (metadata + two raw files per file; no clone, no scanner, engine, plane task or model on any of these repositories)",
"manual_review": "every automated candidate's function diff was read (agent, no model call) and kept only when the named function itself holds the fix",
"reserved_population_guard": "tests/test_independent_population.py::test_no_reserved_repository_is_referenced_by_the_development_tree, run with its output discarded; bisection over catalog subsets, 23 runs",
"quotas": "20 per family on the second pass (14 on the first); advisories past a met quota were not examined",
"mechanism": "family default (injection/output_encoding/deserialization source_reaches_sink, path path_join_user_input, access_control missing_auth_guard, untrusted_destination validation_strength for CWE-601 else source_reaches_sink); provenance = agent"
},
"screened": {
"advisories_examined_second_pass": 2983,
"outcomes_second_pass": {
"no fix commit linked": 1247,
"repository already in benchmarks/ (populations v1/v2 or a pair corpus)": 520,
"language outside the feature vocabulary (go/ruby)": 332,
"CWEs span several families": 253,
"repository already kept (one pair per repository)": 140,
"licence not permissive or unrecorded": 135,
"large commit (refactor or release, not the fix)": 125,
"kept": 110,
"no function declared on both sides with a changed body": 51,
"fix spread over several commits or repositories": 25,
"manual review: the chosen function does not hold the fix (comment-only, misattributed, partial or family mismatch)": 13,
"no source file changed": 11,
"merge or root commit": 11,
"language outside the feature vocabulary (go/ruby/c#/...)": 9,
"WordPress plugin (population v3 safety)": 1
},
"candidates_manually_reviewed": 145,
"kept_after_review": 100,
"review_drops": {
"total": 45,
"function misattributed (matcher span overrun or an undetected enclosing declaration)": 13,
"change in the function unrelated to the advisory (refactor, rename, formatting, another advisory)": 11,
"partial: the function only calls or configures the real fix": 7,
"comment or documentation change only": 6,
"one pair per repository (a reviewed pair of the same repository kept)": 4,
"second pass chose another function of an advisory already reviewed": 2,
"family mismatch": 1,
"unsure": 1
},
"reserved_population_drops": 2,
"kept": 98
},
"kept": {
"total": 98,
"by_family": {
"access_control": 17,
"deserialization": 11,
"injection": 20,
"output_encoding": 16,
"path": 17,
"untrusted_destination": 17
},
"post_cutoff_by_family": {
"access_control": 13,
"deserialization": 2,
"injection": 20,
"output_encoding": 16,
"path": 17,
"untrusted_destination": 11
},
"post_cutoff_total": 79,
"by_language": {
"typescript": 34,
"python": 30,
"javascript": 15,
"java": 10,
"php": 9
},
"by_ecosystem": {
"npm": 48,
"pip": 32,
"maven": 9,
"composer": 9
},
"licences": {
"MIT": 67,
"Apache-2.0": 24,
"BSD-3-Clause": 5,
"ISC": 1,
"Unlicense": 1
},
"split": {
"train": 51,
"holdout": 47
},
"published_range": [
"2025-06-06",
"2026-10-02"
],
"verification": {
"verified": 98,
"excerpt_bytes_read": 526134
},
"known_limit": "excerpts inherit the label builder's matcher: some spans run past the function's end into the next declaration; the label is path::function and the changed lines were checked inside the span"
},
"labels": {
"command": "ousast learn labels --root . --out <scratch> --snapshot <scratch>",
"rows": [
6589,
6981
],
"new_pair_rows": 392,
"new_pin_rows_by_label": {
"positive": 98,
"negative": 98
},
"fixed_side_negative": "98 of 98 pairs (absent_fixed_side 0)"
},
"features": {
"command": "benchmarks/learn/build_features.py --labels <the 392 new rows> --units <units.json of `ousast plane harvest --catalog benchmarks/pairs/advisory-fixes-2/catalog.toml` (manifests written, no Run started)> --kinds pairs",
"records": 392,
"instrument_states_plane_all_families": {
"agree:missing": 74,
"agree:none": 122,
"delta:not_applicable": 196,
"engine:missing": 128,
"engine:none": 68,
"entry_points:not_applicable": 196,
"facts:not_applicable": 196,
"language:ran": 196,
"model_sinks:missing": 196,
"quick:ran": 196,
"roles:ran": 196,
"source:ran": 196,
"verify:missing": 74,
"verify:none": 122
},
"note": "quick rules and deterministic roles ran on each side; engine, verify, agree and model sinks never ran for these pairs (missing/none: null features, never zeros)"
},
"memory_build": {
"command": "ousast learn memory build --labels <labels> --units <merged units.json> --profile static --root . (exit 2: adjudications 9 of 43 anonymous functions, unchanged since 2026-10-01; rows kept)",
"examples": [
937,
1133
],
"pairs_examples": [
772,
968
],
"pairs_counts_after": {
"absent_fixed_side": 0,
"examples": 968,
"labels": 1936,
"no_excerpt": 0,
"no_record": 968,
"recorded": 968,
"unread": 0
}
},
"positives_only_groups": {
"how": "the 2026-10-02 gap record's count: static example rows of the store, per family, repository groups with a positive and no negative",
"before": 99,
"after": 99,
"before_note": "the store before this harvest held 937 examples (the 10-02 gap record counted 904 and 98; the 10-02 rebuild added one population-v2 positive without a negative)",
"why_unchanged": "the 99 are data gaps of other corpora (94 vibe-py pairs whose fixed side lost the function, 3 dev-php, 2 population-v2); a new repository group arrives with both labels, so it adds paired groups and leaves those counts as they were",
"paired_groups_total": [
151,
249
],
"positives_only_share_of_positive_groups_six_families": [
0.4034,
0.2857
],
"by_family_before_after": {
"access_control": {
"groups_pos_only": [
12,
12
],
"groups_pos": [
32,
49
],
"groups_both": [
20,
37
],
"examples_neg": [
20,
37
]
},
"config_secrets": {
"groups_pos_only": [
3,
3
],
"groups_pos": [
5,
5
],
"groups_both": [
2,
2
],
"examples_neg": [
13,
13
]
},
"deserialization": {
"groups_pos_only": [
20,
20
],
"groups_pos": [
41,
52
],
"groups_both": [
21,
32
],
"examples_neg": [
21,
32
]
},
"injection": {
"groups_pos_only": [
11,
11
],
"groups_pos": [
44,
64
],
"groups_both": [
33,
53
],
"examples_neg": [
70,
90
]
},
"memory": {
"groups_pos_only": [
0,
0
],
"groups_pos": [
5,
5
],
"groups_both": [
5,
5
],
"examples_neg": [
173,
173
]
},
"output_encoding": {
"groups_pos_only": [
14,
14
],
"groups_pos": [
36,
52
],
"groups_both": [
22,
38
],
"examples_neg": [
61,
77
]
},
"path": {
"groups_pos_only": [
12,
12
],
"groups_pos": [
33,
50
],
"groups_both": [
21,
38
],
"examples_neg": [
21,
38
]
},
"unknown": {
"groups_pos_only": [
0,
0
],
"groups_pos": [
2,
2
],
"groups_both": [
2,
2
],
"examples_neg": [
2,
2
]
},
"untrusted_destination": {
"groups_pos_only": [
27,
27
],
"groups_pos": [
52,
69
],
"groups_both": [
25,
42
],
"examples_neg": [
36,
53
]
}
}
},
"embeddings": {
"command": "ousast learn memory embed --budget-usd 0.20 (dry run first: 66,005 estimated tokens, $0.0013)",
"requested": 1004,
"cached": 781,
"embedded": 223,
"calls": 7,
"tokens": 71163,
"usd": 0.001423,
"note": "embedded = the new pairs' excerpts plus the excerpts the 10-02 rebuild left without a vector; OpenRouter, the embedding model .env names"
},
"spend": {
"chat_model_calls": 0,
"embedding_usd": 0.001423,
"github_api": "metadata and raw contents only"
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,496 @@
{
"pairs": [
{
"name": "prefect-token-validation-e216171",
"ok": true,
"bytes_read": 3697
},
{
"name": "arcadedb-getdatabasegroupsconfiguration-04110c0",
"ok": true,
"bytes_read": 3896
},
{
"name": "langflow-get-flow-by-id-or-endpoint-name-2c9f498",
"ok": true,
"bytes_read": 3948
},
{
"name": "grav-plugin-api-update-26f529c",
"ok": true,
"bytes_read": 5598
},
{
"name": "google-maps-mcp-main-00d8725",
"ok": true,
"bytes_read": 3371
},
{
"name": "camofox-mcp-createmcphttpapp-599f56e",
"ok": true,
"bytes_read": 5587
},
{
"name": "spring-graphql-isoverridefor-63a89ed",
"ok": true,
"bytes_read": 2045
},
{
"name": "jenkins-dofilltimezonenameitems-0586de4",
"ok": true,
"bytes_read": 2093
},
{
"name": "netlicensing-mcp-dispatch-fbbb1d5",
"ok": true,
"bytes_read": 2610
},
{
"name": "starlette-admin-validate-where-af05b45",
"ok": true,
"bytes_read": 2061
},
{
"name": "next-auth-initauth-d008b9b",
"ok": true,
"bytes_read": 8193
},
{
"name": "deepseek-mcp-server-starthttptransport-dab07ed",
"ok": true,
"bytes_read": 2394
},
{
"name": "network-ai-routerequest-a59c13a",
"ok": true,
"bytes_read": 5945
},
{
"name": "device-builder-parse-args-9e294f7",
"ok": true,
"bytes_read": 15243
},
{
"name": "typespec-start-30d6f65",
"ok": true,
"bytes_read": 2109
},
{
"name": "tomcat-findsecurityconstraints-8bafd79",
"ok": true,
"bytes_read": 18199
},
{
"name": "vm2-makebuiltinsfromlegacyoptions-b0f5066",
"ok": true,
"bytes_read": 3539
},
{
"name": "inlong-filtersensitive-ebcd6d3",
"ok": true,
"bytes_read": 6474
},
{
"name": "h2o-3-validatejdbcurl-0298ee3",
"ok": true,
"bytes_read": 4919
},
{
"name": "skops-load-model-29d61ea",
"ok": true,
"bytes_read": 1847
},
{
"name": "picklescan-list-globals-58983e1",
"ok": true,
"bytes_read": 6975
},
{
"name": "sglang-deserialize-49afb3d",
"ok": true,
"bytes_read": 4334
},
{
"name": "pdfminer-six-load-data-b808ee0",
"ok": true,
"bytes_read": 2399
},
{
"name": "reverb-handle-9ec26f8",
"ok": true,
"bytes_read": 3342
},
{
"name": "transformers-is-kernel-a7f8e7f",
"ok": true,
"bytes_read": 1921
},
{
"name": "opentelemetry-java-instrumentation-read-9cf4fba",
"ok": true,
"bytes_read": 2075
},
{
"name": "keras-from-config-55888d3",
"ok": true,
"bytes_read": 3154
},
{
"name": "airflow-handle-event-submit-a1ab065",
"ok": true,
"bytes_read": 4828
},
{
"name": "sequelize-partialescape-5deadd2",
"ok": true,
"bytes_read": 6826
},
{
"name": "electerm-mv-aa77881",
"ok": true,
"bytes_read": 1258
},
{
"name": "systeminformation-checklinuxdchpinterfaces-bbfddde",
"ok": true,
"bytes_read": 2208
},
{
"name": "prompty-buildagent-c27402d",
"ok": true,
"bytes_read": 3198
},
{
"name": "datamodel-code-generator-get-field-extras-17fc235",
"ok": true,
"bytes_read": 2471
},
{
"name": "spring-data-mongodb-bindablevaluefor-24d5afc",
"ok": true,
"bytes_read": 9895
},
{
"name": "spring-ai-emitlucenestring-2b20cfc",
"ok": true,
"bytes_read": 4037
},
{
"name": "aws-cdk-write-baa9e1d",
"ok": true,
"bytes_read": 1461
},
{
"name": "danger-js-localgetfileatsha-087a729",
"ok": true,
"bytes_read": 1724
},
{
"name": "argos-javascript-gitfetch-8355f3a",
"ok": true,
"bytes_read": 1319
},
{
"name": "cakephp-transformfunctionexpression-138f2f6",
"ok": true,
"bytes_read": 6754
},
{
"name": "orval-getroutepath-8ef1bfd",
"ok": true,
"bytes_read": 2607
},
{
"name": "typeorm-handler-41d1c62",
"ok": true,
"bytes_read": 11977
},
{
"name": "velocity-js-isblockedpathkey-f8e47a6",
"ok": true,
"bytes_read": 1539
},
{
"name": "jsonata-evaluategroupexpression-4b217d5",
"ok": true,
"bytes_read": 7399
},
{
"name": "nuxt-getislandcontext-5b60017",
"ok": true,
"bytes_read": 6805
},
{
"name": "gitpython-option-candidates-96a888f",
"ok": true,
"bytes_read": 3421
},
{
"name": "mermaid-addnamespace-12d472c",
"ok": true,
"bytes_read": 9722
},
{
"name": "php-codesniffer-getblamecontent-7a3a6bb",
"ok": true,
"bytes_read": 2040
},
{
"name": "xmldom-serializetostring-1cde3e3",
"ok": true,
"bytes_read": 23297
},
{
"name": "trix-fromjson-9c0a993",
"ok": true,
"bytes_read": 1295
},
{
"name": "oauthlib-prepare-token-revocation-request-c888359",
"ok": true,
"bytes_read": 4800
},
{
"name": "suneditor-submitsrc-9d43a5e",
"ok": true,
"bytes_read": 2319
},
{
"name": "mathlive-atomtomathml-5fe1c46",
"ok": true,
"bytes_read": 32529
},
{
"name": "genql-comment-2dbc80f",
"ok": true,
"bytes_read": 1633
},
{
"name": "angular-visitelementlike-417a407",
"ok": true,
"bytes_read": 8480
},
{
"name": "cyberchef-objtotable-85db3be",
"ok": true,
"bytes_read": 2556
},
{
"name": "mdc-isanchorlinkallowed-61d636c",
"ok": true,
"bytes_read": 1795
},
{
"name": "node-xhtml-purifier-node-21d461a",
"ok": true,
"bytes_read": 7486
},
{
"name": "orpc-init-daabded",
"ok": true,
"bytes_read": 6240
},
{
"name": "mkdocs-material-mountsearchsuggest-52fb6be",
"ok": true,
"bytes_read": 4493
},
{
"name": "jodit-isremovablenode-49a31f4",
"ok": true,
"bytes_read": 3921
},
{
"name": "prowler-transform-6db407e",
"ok": true,
"bytes_read": 5481
},
{
"name": "commonmark-parseattributes-dfcdf45",
"ok": true,
"bytes_read": 6437
},
{
"name": "svgo-fn-3db3ef3",
"ok": true,
"bytes_read": 6895
},
{
"name": "tiptap-mergeattributes-01d7af8",
"ok": true,
"bytes_read": 3690
},
{
"name": "nogginlessdom-matchfilesnapshot-785e6ac",
"ok": true,
"bytes_read": 3023
},
{
"name": "gradio-preprocess-97d541f",
"ok": true,
"bytes_read": 2581
},
{
"name": "gemini-bridge-resolve-path-8f3b85a",
"ok": true,
"bytes_read": 2006
},
{
"name": "libreoffice-convert-convertwithoptions-b78f17d",
"ok": true,
"bytes_read": 8912
},
{
"name": "atomic-agents-stack-do-get-ec474f4",
"ok": true,
"bytes_read": 3493
},
{
"name": "browse-mcp-downloadurl-5352a4a",
"ok": true,
"bytes_read": 5302
},
{
"name": "notebooklm-mcp-makeslug-13828d9",
"ok": true,
"bytes_read": 2103
},
{
"name": "langchain-nvidia-url-to-b64-string-b483c7c",
"ok": true,
"bytes_read": 2318
},
{
"name": "redocly-cli-iteratepathitems-5041204",
"ok": true,
"bytes_read": 4412
},
{
"name": "pip-filename-10dfb6b",
"ok": true,
"bytes_read": 1419
},
{
"name": "lib-get-db-path-d665042",
"ok": true,
"bytes_read": 3879
},
{
"name": "blender-mcp-download-polyhaven-asset-30a3308",
"ok": true,
"bytes_read": 35668
},
{
"name": "datasets-set-feature-f989ef9",
"ok": true,
"bytes_read": 4882
},
{
"name": "postcss-loadfile-7beca13",
"ok": true,
"bytes_read": 1712
},
{
"name": "vitest-configureserver-51edf2b",
"ok": true,
"bytes_read": 3287
},
{
"name": "next-js-getfilepath-968b9fc",
"ok": true,
"bytes_read": 2462
},
{
"name": "composer-install-53b8bb4",
"ok": true,
"bytes_read": 4127
},
{
"name": "vue-cli-mayproxy-72ba750",
"ok": true,
"bytes_read": 1203
},
{
"name": "h3-redirectback-459a1c6",
"ok": true,
"bytes_read": 2750
},
{
"name": "axios-setproxy-fb3befb",
"ok": true,
"bytes_read": 4403
},
{
"name": "rembg-s-command-07ad0d4",
"ok": true,
"bytes_read": 21204
},
{
"name": "http-server-getpreviousurl-2008fb6",
"ok": true,
"bytes_read": 4395
},
{
"name": "authlib-get-authorization-grant-3be0846",
"ok": true,
"bytes_read": 2631
},
{
"name": "java-client-overrideserverurl-2b9cd44",
"ok": true,
"bytes_read": 1382
},
{
"name": "n8n-mcp-validatewebhookurl-9639f75",
"ok": true,
"bytes_read": 13458
},
{
"name": "ufo-withoutbase-5cd9e67",
"ok": true,
"bytes_read": 2154
},
{
"name": "symfony-dogenerate-4b63c3a",
"ok": true,
"bytes_read": 16981
},
{
"name": "php-weasyprint-isoptionurl-9582dcf",
"ok": true,
"bytes_read": 1090
},
{
"name": "compliance-trestle-check-metadata-endpoints-d107cd1",
"ok": true,
"bytes_read": 2412
},
{
"name": "vllm-preprocess-audio-54503ec",
"ok": true,
"bytes_read": 4629
},
{
"name": "phpspreadsheet-webservice-7ef7b25",
"ok": true,
"bytes_read": 4441
},
{
"name": "frontmcp-dofetch-077201e",
"ok": true,
"bytes_read": 3737
},
{
"name": "ip-address-islinklocal-d03e960",
"ok": true,
"bytes_read": 1489
},
{
"name": "pyjwt-fetch-data-0a795b8",
"ok": true,
"bytes_read": 3385
}
],
"verified": 98,
"bytes_read": 526134
}
+114 -67
View File
@@ -1,14 +1,22 @@
# Architecture
How a scan works, how a finding earns its place, how severity and the project score are
decided, and how the ruleset improves itself. State as of 2026-10-02; module paths are under
`src/openultrasast/`.
This page covers four things:
- how a scan works;
- how a finding earns its place;
- how severity and the project score are decided;
- how the ruleset improves itself.
State as of 2026-10-02; module paths are under `src/openultrasast/`.
## The scan pipeline
The CLI is thin; `HarnessRuntime` owns the lifecycle and writes an event trace for every stage
(`trace/events.jsonl`). The stages a mode requests are `MODE_STAGES` in `stages.py`:
`quick` runs STATIC, `standard` adds MAP, `deep` adds REGRESS.
The CLI is thin. `HarnessRuntime` owns the lifecycle. It writes an event trace for every stage
(`trace/events.jsonl`). `MODE_STAGES` in `stages.py` lists the stages each mode requests:
- `quick` runs STATIC;
- `standard` adds MAP;
- `deep` adds REGRESS.
```
policy_load ─▶ ruleset_load ─▶ policy_check (fail loud on an ungoverned CWE)
@@ -28,14 +36,19 @@ policy_load ─▶ ruleset_load ─▶ policy_check (fail loud on an
```
**The model layer** (`model/`, `cpg/`) is the core of `standard`. It builds one code property
graph per scan with Joern (`joern-parse`, then batched `joern --script` queries from
`cpg/queries/*.sc`) and dispatches each question to the arbiter of its spec type: taint
(source to sink with no discharging sanitizer), dominance (does a guard govern the obligated
operation) and configuration (constant abstraction over a value). Taint facts ship for C,
Java, JavaScript, PHP and Python (`ruleset/semantic/*.toml`). Without Joern the layer records
`cpg_unavailable` and the scan is otherwise unchanged; the Docker image ships Joern and
`php-cli` (which the PHP frontend needs). The representations and techniques behind each stage
are explained in [Detection techniques](detection-techniques.md).
graph (CPG: a graph of the program's syntax, control flow and data flow) per scan with Joern.
It runs `joern-parse`, then batched `joern --script` queries from `cpg/queries/*.sc`. Each
question goes to the arbiter (a checker that decides one kind of question) for its spec type:
- taint (untrusted input reaching a dangerous call): source to sink with no discharging sanitizer;
- dominance: does a guard govern the obligated operation;
- configuration: constant abstraction over a value.
Taint facts ship for C, Java, JavaScript, PHP and Python (`ruleset/semantic/*.toml`). Without
Joern, the layer records `cpg_unavailable` and the rest of the scan is unchanged. The Docker
image ships Joern and `php-cli`, which the PHP frontend needs. [Detection
techniques](detection-techniques.md) explains the representations and techniques behind each
stage.
The model layer reports on a rung ladder:
@@ -47,23 +60,29 @@ model_corroborated the graph found the shape; a judge agreed on the residual
suspicion proposed, nothing corroborated it
```
The LLM (`model/pipeline.py`, `residual_question`) is asked only the residual question the graph
cannot settle, for a flow the graph already found; it can confirm or contradict, never create a
flow. When the graph resolved nothing, the enumerator's candidates (at most
`MAX_JUDGED_CANDIDATES`, 8) are put to it, and anything it affirms is reported at `suspicion`.
Without a provider key neither question is asked and only the graph's entailed findings are
reported.
The LLM answers only the residual question: the part the graph cannot settle
(`model/pipeline.py`, `residual_question`). It is asked only about a flow the graph already
found. It can confirm or contradict that flow, but never create one. When the graph resolved
nothing, the LLM sees the enumerator's candidates (at most `MAX_JUDGED_CANDIDATES`, 8).
Anything it affirms is reported at `suspicion`. Without a provider key, neither question is
asked. Only the graph's entailed findings are reported then.
**REGRESS** (`regress/`, `sandbox/`) runs only when `docker info` succeeds. Each promoted
candidate gets a small language recipe (Python, JavaScript, C) that loads the candidate's
source inside a container with no network, a read-only `/workspace`, a non-root user and the
`[sandbox]` limits. A structural deny-list rejects snippets mentioning sockets, curl, host
networking or writes under `/workspace` before anything runs.
candidate gets a small language recipe (Python, JavaScript, C). The recipe loads the
candidate's source inside a container with:
- no network;
- a read-only `/workspace`;
- a non-root user;
- the `[sandbox]` limits.
Before anything runs, a structural deny-list rejects snippets that mention sockets, curl, host
networking or writes under `/workspace`.
## Evidence ladder: how a false positive is eliminated
Evidence level is a state machine, not a label a model may assign. The verifier
(`verification.py`) runs on independent context and enforces:
(`verification.py`) runs on independent context and enforces it:
```
suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_explained
@@ -75,31 +94,36 @@ suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_
call-graph, route, CLI, parser or dynamic evidence;
- `static_corroboration` with function-level reachability: `ACCEPTED`.
A rejected finding becomes a **scoped false-positive learning** (`calibration.py`) with a
reason from a fixed taxonomy (`unreachable_path`, `missing_attacker_control`,
A rejected finding becomes a **scoped false-positive learning** (`calibration.py`). It has a
scope and a reason from a fixed taxonomy: `unreachable_path`, `missing_attacker_control`,
`sanitizer_disproved`, `static_rule_mismatch`, `incorrect_model_assumption`, `duplicate`,
`insufficient_impact`, `unsupported`, `contradicted`, `unverified`) and a scope. A rejection in
`auth/` demotes only `auth/...`, never the whole class
`insufficient_impact`, `unsupported`, `contradicted`, `unverified`. A rejection in `auth/`
demotes only `auth/...`, never the whole class
(`tests/test_calibration.py::test_scoped_false_positive_demotion_does_not_suppress_class_globally`).
## How a false negative is surfaced
Every expected vulnerability that no finding matched becomes a `BenchmarkCalibrationRecord` in
`calibration_records.json`, with a `next_improvement_candidate` (a static rule, SARIF source,
entry-point mapping, retrieval package, hunter prompt, dynamic reproducer or skill route).
Once a miss is closed the detection gate keeps it closed in CI.
An expected vulnerability that no finding matched becomes a `BenchmarkCalibrationRecord` in
`calibration_records.json`. Each record names a `next_improvement_candidate`: a static rule,
SARIF source, entry-point mapping, retrieval package, hunter prompt, dynamic reproducer or
skill route. Once a miss is closed, the detection gate keeps it closed in CI.
## Central CWE policy and the project score
**Policy** (`policy/verycode.py`): a vendored `CWE_Score.tsv` (verycode-policies) is the single
source of truth. Each CWE carries a flaw category, a severity (0-5) and `static`/`dynamic` scope
flags. A finding's severity comes only from policy, keyed on CWE. CWEs that are not `static`
resolve to 0 and are report-only. `assert_rules_resolve` runs as the `policy_check` stage: an
enabled rule naming an ungoverned CWE aborts the scan. Verycode has no CWE-120, so the C/C++
source of truth. Each CWE carries:
- a flaw category;
- a severity (0-5);
- `static`/`dynamic` scope flags.
A finding's severity comes only from policy, keyed on CWE. CWEs that are not `static` resolve
to 0 and are report-only. `assert_rules_resolve` runs as the `policy_check` stage. An enabled
rule that names an ungoverned CWE aborts the scan. Verycode has no CWE-120, so the C/C++
buffer rules carry CWE-121.
**Score** (`scoring/project_score.py`): each finding's penalty is
`SEV_WEIGHT[severity] x REACH_MULT[reachability]`, and the score is `100 * e^(-total/k)` with
`SEV_WEIGHT[severity] x REACH_MULT[reachability]`. The score is `100 * e^(-total/k)` with
`k = 60`.
| Severity | Weight | | Reachability | Multiplier |
@@ -109,10 +133,13 @@ buffer rules carry CWE-121.
| 3 | 10 | | `unknown` | 0.4 |
| 2 / 1 / 0 | 2 / 1 / 0 | | | |
A confirmed false positive lowers a finding's reachability multiplier instead of deleting the
rule. A severity-5 reachable finding always fails the score gate; `score < min_score`
(default 80) fails it only with `[score] blocking = true`. The `score` stage writes
`score.json` and merges it into `manifest.json`.
A confirmed false positive lowers a finding's reachability multiplier. The rule is not
deleted. The score gate works like this:
- a severity-5 reachable finding always fails it;
- `score < min_score` (default 80) fails it only with `[score] blocking = true`.
The `score` stage writes `score.json` and merges it into `manifest.json`.
## Self-improvement
@@ -124,41 +151,61 @@ scan ─▶ verify ─▶ non-accepted findings ─▶ scoped learnings
next scan ─▶ rank ─▶ calibrate (demote rejected scopes) ─▶ findings
```
`record_calibration` (after `verify`) merges this run's non-accepted outcomes into the ledger,
de-duplicated by finding ID; `calibrate` (after `rank`) demotes the priority of every scope
that produced a rejection and writes `applied_calibrations.json`. Demotion is scoped and
reversible (`tests/test_pipeline_calibration.py`).
1. `record_calibration` runs after `verify`. It merges this run's non-accepted outcomes into
the ledger, de-duplicated by finding ID.
2. `calibrate` runs after `rank`. It demotes the priority of every scope that produced a
rejection and writes `applied_calibrations.json`.
Demotion is scoped and reversible (`tests/test_pipeline_calibration.py`).
### Rule-level loop: `ousast improve`
Each round (`improve/evolve.py`): benchmark with the current ledger, per-rule signals
(`rule_signals.json`), bounded proposals, validation (`improve/validator.py`), replay smoke,
re-benchmark, and a hard gate. The levers are rule status (`enabled -> shadow` before
`disabled`) and score constants; the loop can never edit pattern text
or the authoritative CWE severity. A round is accepted only if recall >= 90%, FP < 10%, the
project score did not regress and no matched finding was lost; otherwise the ledger is not
written. With `--pair-catalog`, every round is also gated per provenance profile on the
catalog's holdout pairs (`--no-pair-gate` skips this and is not for accepted ledgers). The same
detection gate runs in CI (`python -m openultrasast.gate`).
Each round (`improve/evolve.py`) runs these steps:
1. benchmark with the current ledger;
2. per-rule signals (`rule_signals.json`);
3. bounded proposals;
4. validation (`improve/validator.py`);
5. replay smoke;
6. re-benchmark;
7. a hard gate.
The levers are rule status (`enabled -> shadow` before `disabled`) and score constants. The
loop can never edit pattern text or the authoritative CWE severity. A round is accepted only if
all of these hold:
- recall >= 90%;
- FP < 10%;
- the project score did not regress;
- no matched finding was lost.
Otherwise the ledger is not written. With `--pair-catalog`, each round is also gated per
provenance profile on the catalog's holdout pairs. `--no-pair-gate` skips this check; do not
use it for accepted ledgers. The same detection gate runs in CI (`python -m openultrasast.gate`).
### Proposals from plane memory
With `--memory [STORE]`, `ousast improve` also reads the rows plane runs stored per repository
and pin (`improve/memory.py`). Two deterministic rules turn them into rule-status proposals: a
rule whose alerts are repeatedly false across repositories is shadowed, and a shadow rule that
keeps hitting agreed, declared vulnerable sites is re-enabled. Rows from holdout pairs, from the
gated manifest's own cases and from any `--qualify-population` are dropped before any rule sees
them. Each proposal records the rows it came from (`memory_proposals.jsonl` next to the
journal) and goes through the same validator and gate.
With `--memory [STORE]`, `ousast improve` also reads the rows that plane runs stored per
repository and pin (`improve/memory.py`). Two deterministic rules turn these rows into
rule-status proposals:
- a rule whose alerts are repeatedly false across repositories is shadowed;
- a shadow rule that keeps hitting agreed, declared vulnerable sites is re-enabled.
Some rows are dropped before any rule sees them: rows from holdout pairs, from the gated
manifest's own cases and from any `--qualify-population`. Each proposal records the rows it
came from (`memory_proposals.jsonl` next to the journal). It then goes through the same
validator and gate.
### The loop as a plane Run
`ousast plane workspaces <population> --validation-set ... --loop` appends the improvement loop
to a generated Run: `alerts` per case (the quick scan on the vulnerable and fixed pins), then
the steps `snapshot` (task `memory-snapshot`), `measure`, `propose` and `improve` (`plane/tasks/loop.py`), each model-free
with a zero budget. The snapshot applies the same train-on-test guard before any row leaves the
store. A recorded live run: `benchmarks/measurements/2026-09-30-harnessx-removal-loop/`.
to a generated Run. The Run holds `alerts` per case: the quick scan on the vulnerable and fixed
pins. Then come the steps `snapshot` (task `memory-snapshot`), `measure`, `propose` and
`improve` (`plane/tasks/loop.py`). Each step is model-free with a zero budget. The snapshot
applies the same train-on-test guard before any row leaves the store. A recorded live run:
`benchmarks/measurements/2026-09-30-harnessx-removal-loop/`.
For PHP, which quick mode covered with zero rules when that run was made, `ousast plane
When that run was made, quick mode covered PHP with zero rules. So for PHP, `ousast plane
alerts-engine` produces a Run's `alerts` from the engine image on the host instead
(`benchmarks/measurements/2026-09-30-php-engine-alerts/`).
+256 -67
View File
@@ -1,51 +1,77 @@
# The learned decision engine (in development)
**Status 2026-10-02: in development, not adopted.** No scan, pre-push check or report uses it;
every user-facing path is today's deterministic one until a compiled program is adopted. The
design is the `learned-decision-engine` specification (maintainer tooling); the code is in
**Status 2026-10-02: in development, not adopted.** No scan, pre-push check or report uses it.
Every user-facing path stays on today's deterministic one until a compiled program is adopted.
The design is the `learned-decision-engine` specification (maintainer tooling). The code is in
`src/openultrasast/learn/`.
## Why
Every detection gain before the engine was a hand edit tuned on the cases that exposed it, and
engine tuning measured on population v1 did not transfer to v2. The reasoning, and the step-by-step
chain that replaced hand edits, are on [Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
Before the engine, every detection gain was a hand edit. Each edit was tuned on the cases that
exposed the miss. Engine tuning measured on population v1 did not transfer to v2. [Where we
stand](where-we-stand.md#2-our-approach-to-detection-step-by-step) gives the reasoning and the
step-by-step chain that replaced hand edits.
## Design
- **Signals, not verdicts.** Every instrument (quick rules, the Joern engine, model sink and
role classification, the plane's verify passes and agreement, repository facts such as
callers) contributes a feature record per candidate `(path, function, family)`, including an
explicit state when it did not run or failed. None decides alone.
- **Signals, not verdicts.** Every instrument contributes a feature record per candidate
`(path, function, family)`. The instruments are quick rules, the Joern engine, model sink and
role classification, the plane's verify passes and agreement, and repository facts such as
callers. The record includes an explicit state when an instrument did not run or failed. No
instrument decides alone.
- **Labels from ground truth.** `ousast learn labels` derives labels only from sources on a
fail-closed allow-list (`learn/sources.toml`): the vulnerable and fixed sides of pairs, the
declared sites and fix ranges of spent populations, benign deltas and recorded adjudications.
fail-closed allow-list (`learn/sources.toml`). Fail-closed means a source not on the list is
never used. The sources are:
- the vulnerable and fixed sides of pairs;
- the declared sites and fix ranges of spent populations;
- benign deltas;
- recorded adjudications.
"Rejected by the plane" is never a negative. Reserved populations (v3) are not on the list.
- **An AI classifier with local memory, compiled in house in the DSPy style.** The program is an
instruction, a few demonstrations and up to six retrieved labelled examples, then the
candidate's code excerpt and signals; the model (`deepseek-flash`) answers
vulnerable / not vulnerable / unsure with a rationale, sampled `k` times, giving a score `s`.
Compilation bootstraps demonstrations and searches instructions on a compile split, scored by
a pre-registered balanced Brier score (`learn/compile.toml`).
- **Retrieval** from the memory store: nearest neighbours by Gower distance on the signal
profile, re-ranked by cosine similarity of code embeddings (OpenAI `text-embedding-3-small`
through OpenRouter, cached by excerpt hash), under a label/family/repository balance rule. An
evaluation-boundary filter keeps a candidate's own repository group out of what it is shown.
- **An AI classifier with local memory, compiled in house in the DSPy style.** The program has
an instruction, a few demonstrations and up to six retrieved labelled examples. Then come the
candidate's code excerpt and signals. The model (`deepseek-flash`) answers vulnerable / not
vulnerable / unsure with a rationale. It is sampled `k` times, which gives a score `s`.
Compilation bootstraps demonstrations and searches instructions on a compile split. A
pre-registered balanced Brier score (`learn/compile.toml`) scores the candidates. The Brier score
measures how far predicted probabilities are from the true labels.
- **Retrieval** from the memory store works in three steps:
1. find nearest neighbours by Gower distance (a distance over mixed feature types) on the
signal profile;
2. re-rank them by cosine similarity of code embeddings (OpenAI `text-embedding-3-small`
through OpenRouter, cached by excerpt hash);
3. apply a label/family/repository balance rule.
An evaluation-boundary filter keeps a candidate's own repository group out of what it is
shown.
- **Repository-grouped evaluation.** The compile split (25% of repository groups) is never
evaluated. The rest is evaluated by outer grouped 5-fold cross-validation: the Platt
calibration map is cross-fitted on the other folds and the operating points are chosen on the
other folds (nested). One compile per profile, not one per fold. Intervals are 95%
repository-cluster bootstrap.
- **Two operating points.** BLOCK is offered only when calibration holds (slope CI contains 1,
intercept CI contains 0, ECE <= 0.05) and a fold reaches the required precision; otherwise the
engine is ADVISORY only. An `unsure` answer can never BLOCK.
evaluated. The rest is evaluated by outer grouped 5-fold cross-validation:
- the Platt calibration map (which turns the score into a probability) is cross-fitted on
the other folds;
- the operating points are chosen on the other folds (nested).
There is one compile per profile, not one per fold. Intervals are 95% repository-cluster
bootstrap.
- **Two operating points.** BLOCK is offered only when both of these hold:
- calibration holds: slope CI contains 1, intercept CI contains 0, ECE <= 0.05;
- a fold reaches the required precision.
Otherwise the engine is ADVISORY only. An `unsure` answer can never BLOCK. ECE (expected
calibration error) is the average gap between predicted and observed rates.
## First measured slice (2026-10-01)
Record: `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json` (counts
only). Family **injection** only; memory of 724 static examples from all families; input profile
v1 with the engine signals and function length withheld (a leak audit found them separating the
sides of a pair by construction: `2026-10-01-decision-engine-leak-audit/`); framework priors off.
only). The setup:
- family **injection** only;
- memory of 724 static examples from all families;
- input profile v1, with the engine signals and function length withheld;
- framework priors off.
The signals were withheld because a leak audit found them separating the sides of a pair by
construction (`2026-10-01-decision-engine-leak-audit/`). AUC below is the chance the engine
ranks a real bug above a fixed one.
| Measure | Value (95% repository-cluster CI) |
| --- | --- |
@@ -62,11 +88,17 @@ sides of a pair by construction: `2026-10-01-decision-engine-leak-audit/`); fram
## Second slice: six families (2026-10-02)
Record: `benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json` (counts only).
After the advisory-fix harvest the memory holds 904 static examples (724 before). One program was
compiled per evaluable family (injection recompiled, since the memory changed) and evaluated on
that family's own candidates, out of repository as above: compile split never evaluated, outer
grouped 5-fold, calibration cross-fitted, operating points nested, 95% repository-cluster
bootstrap intervals (2,000 resamples); input profile v1, priors off, k = 5.
After the advisory-fix harvest, the memory holds 904 static examples (724 before).
One program was compiled per evaluable family. Injection was recompiled, since the memory
changed. Each program was evaluated on its family's own candidates, out of repository as above:
- compile split never evaluated;
- outer grouped 5-fold;
- calibration cross-fitted;
- operating points nested;
- 95% repository-cluster bootstrap intervals (2,000 resamples);
- input profile v1, priors off, k = 5.
| Family | Candidates (groups; positive) | Pooled AUC of `s` (95% CI) | Within-pair AUC (paired groups) | ADVISORY recall / precision | Canary agreement |
| --- | --- | --- | --- | --- | --- |
@@ -77,45 +109,174 @@ bootstrap intervals (2,000 resamples); input profile v1, priors off, k = 5.
| path | 35 (21; 21) | 0.89 (0.79 - 0.97) | 0.83 (14) | 0.86 / 0.69 | 0.90 |
| untrusted destination | 62 (37; 38) | 0.85 (0.76 - 0.94) | 0.72 (18) | 0.84 / 0.73 | 0.87 |
- **BLOCK is offered for no family.** Calibration holds only for output encoding (ECE 0.045), and
there no fold reaches the required precision (`precision_unreachable`: no fold reaches a Wilson
lower bound of 0.95). For the other five families calibration does not hold (ECE 0.071 to 0.119).
- **The pooled AUC is the more favourable number.** Every family's pool includes candidates that
carry only a positive label, with no fixed-side counterpart (7 to 19 per family; there are no
single-label negatives), so the pooled AUC is inflated by positives that never had to be told
apart from their own fix. The within-pair AUC over the paired groups, 0.72 to 0.93, is the
stricter reading (`paired_groups_check` in the record).
- **BLOCK is offered for no family.** Calibration holds only for output encoding (ECE 0.045).
There, no fold reaches the required precision (`precision_unreachable`: no fold reaches a
Wilson lower bound of 0.95). For the other five families, calibration does not hold (ECE
0.071 to 0.119).
- **The pooled AUC is the more favourable number.** Every family's pool includes candidates
that carry only a positive label, with no fixed-side counterpart (7 to 19 per family; there
are no single-label negatives). These positives never had to be told apart from their own
fix, so they inflate the pooled AUC. The within-pair AUC over the paired groups, 0.72 to
0.93, is the stricter reading (`paired_groups_check` in the record).
- **Re-run agreement (canary, 30 candidates per family) is below 0.9 for three families:**
injection 0.77, access control 0.83, untrusted destination 0.87.
- **One memory-size point for injection** (paired: the same program, candidates and folds, with a
retrieval memory of 724 vs 904 examples): AUC 0.76 vs 0.77, within the intervals. No injection
example was added, so the new examples reach an injection prompt only as contrast.
- **Spend:** $2.88 metered at list price for DeepSeek under an $8 ceiling, $0.0009 for
embeddings; $0.0027 to $0.0032 per evaluated candidate. The access-control evaluation replayed
from the response cache at $0 with identical predictions.
- **An instrument fix is recorded:** compiled demonstrations ignored the near-duplicate rule
(cosine >= 0.98) until the pre-call boundary assertion stopped the untrusted-destination
evaluation; demonstrations are now swapped or dropped like retrieved examples, and the stopped
part was replayed after the fix.
- **One memory-size point for injection.** This run is paired: the same program, candidates
and folds, with a retrieval memory of 724 vs 904 examples. AUC was 0.76 vs 0.77, within the
intervals. No injection example was added. So the new examples reach an injection prompt
only as contrast.
- **Spend:** $2.88 metered at list price for DeepSeek under an $8 ceiling, and $0.0009 for
embeddings. That is $0.0027 to $0.0032 per evaluated candidate. The access-control
evaluation replayed from the response cache at $0 with identical predictions.
- **An instrument fix is recorded.** Compiled demonstrations ignored the near-duplicate rule
(cosine >= 0.98). The pre-call boundary assertion caught this and stopped the
untrusted-destination evaluation. Demonstrations are now swapped or dropped like retrieved
examples. The stopped part was replayed after the fix.
**Not adopted.** What is still ahead is listed once, on
[Where we stand](where-we-stand.md#5-open-items).
**Not adopted.** [Where we stand](where-we-stand.md#5-open-items) lists, in one place, what is
still ahead.
## Experiments
Changes to prompts, models, pass counts, features or sampling are compared by controlled
experiments from a pre-registered manifest (`plane/experiments/<id>.yaml`), so nothing is adopted
on one run's number. Two are registered. exp-001 (known callers) has not run. exp-002 (a retrieval
ensemble against today's temperature sampling) ran on 160 paired candidates in 56 repository
groups and was **rejected**: within-pair AUC 0.815 against 0.741 and $0.0021 against $0.0051 per
candidate, with re-run agreement 0.875 against 0.906 and an interval spanning zero
(`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`). The reading is on
[Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
experiments. Each experiment comes from a pre-registered manifest
(`plane/experiments/<id>.yaml`). This way nothing is adopted on one run's number.
Four experiments are registered:
- exp-001 (known callers) has not run.
- exp-002 compared a retrieval ensemble against today's temperature sampling. It ran on 160
paired candidates in 56 repository groups and was **rejected**. Within-pair AUC was 0.815
against 0.741 and the cost was $0.0021 against $0.0051 per candidate. The record shows re-run agreement
0.875 against 0.906 and an interval spanning zero
(`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`).
- exp-003 compared two BLOCK rules on the same scores. It was **inconclusive**. Neither rule flags anything.
- exp-004 tested one pooled BLOCK threshold with a per-family floor. It was **inconclusive**.
### exp-003: precision-bound blocking (2026-10-03)
The question: is the calibration gate what keeps BLOCK out of reach?
Arm A is today's rule: a calibrated probability, offered only when calibration holds.
Arm B picks a raw-score threshold per family, on training folds only.
It takes the lowest threshold whose Wilson 95% lower bound on precision reaches 0.95.
The held-out repository group is then scored with that threshold.
Arm C relaxes the bound to point precision 0.95 over at least 10 flags.
Arm C was registered as secondary and does not decide.
- **Registered first.** The manifest and units were committed and registered before the run.
- **Primary metric:** arm B's held-out Wilson lower bound on flagged precision, per family.
- **Primary population:** paired units only. Positives-only groups are excluded; pooled figures are secondary.
- **Zero cost.** Scores came only from the response cache, behind a zero-ceiling meter.
The meter recorded 0 client calls and $0.0.
- **Exclusions.** 142 of 353 units had an uncached response and were excluded, not asked.
Path lost all 39 units. Injection and access control lost none.
- **Evaluated:** 211 units in 89 repository groups, 138 of them paired.
| Family | Evaluated (excluded) | Paired positives | Flagged A / B / C | Coverage A / B / C |
| --- | --- | --- | --- | --- |
| access control | 41 (0) | 16 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
| deserialization | 14 (34) | 2 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
| injection | 119 (0) | 38 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
| output encoding | 24 (21) | 8 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
| path | 0 (39) | 0 | 0 / 0 / 0 | none |
| untrusted destination | 13 (48) | 4 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
No arm chose a threshold on any fold, so flagged precision is undefined everywhere.
Arm A failed calibration where it was checked: ECE 0.077 (access control), 0.1067 (injection).
The other families had too few units for a calibration check.
**Why B flags nothing.** A Wilson lower bound of 0.95 needs 73 flags without one error.
Injection, the largest family, holds 38 paired positives across all five folds.
The registration predicted this zero before the run.
On this data the bound, not the calibration gate, makes BLOCK unreachable.
**The zero is the rule's, not the instrument's.** The scores are real: 48 distinct values for injection.
In access control, the 10 highest-scored paired units are all positive.
In injection, a negative appears at rank 4 of the paired ranking.
Pooled rankings look cleaner because positives-only units fill the top.
Injection answered `unsure` for 42 of 119 units, and `unsure` never blocks.
Record: `benchmarks/experiments/exp-003-precision-bound-blocking/result.json`.
**Supplementary: all 353 units (2026-10-03).** This is not a new decision; exp-003's verdict stands.
The 142 excluded units had their responses filled with the same programs and request keys.
The fill cost 697 client calls and $0.331759 under a $1.00 ceiling.
The replay then decided all 353 units at zero cost: 244 paired, 146 repository groups.
| Family | Paired units (positives) | Flagged A / B / C | ECE (arm A) |
| --- | --- | --- | --- |
| access control | 32 (16) | 0 / 0 / 0 | 0.077 |
| deserialization | 34 (17) | 0 / 0 / 0 | 0.0596 |
| injection | 76 (38) | 0 / 0 / 0 | 0.1067 |
| output encoding | 32 (16) | 0 / 0 / 0 | 0.0761 |
| path | 32 (16) | 0 / 0 / 0 | 0.062 |
| untrusted destination | 38 (19) | 0 / 0 / 0 | 0.0454 |
Still no arm flags any unit, so the registered rule reads inconclusive.
Calibration holds only for untrusted destination.
There arm A still finds no BLOCK point that reaches the bound.
Record: `benchmarks/experiments/exp-003-precision-bound-blocking/result-complete-cache.json`.
### exp-004: pooled block gate (2026-10-03)
The experiment tested one pooled BLOCK threshold across families.
Each family needed at least 10 clean flags to block.
Each family also had its own quiet threshold.
The run evaluated 501 units across 220 repository groups.
Of these, 390 units were paired.
The cache fill cost $1.238223 for 2,073 calls.
The replay cost $0.0 and made 0 client calls.
The verdict was **inconclusive**.
No fold found a `t_block` threshold.
The instrument check confirmed real scores.
The pooled top 30 contained 27 positives.
The first negative appeared at rank 12.
The bound needs 73 clean flags without an error.
A within-pair win means the vulnerable side scored above its fixed counterpart.
| Family | Within-pair wins | Pairs |
| --- | --- | --- |
| access control | 22 | 28 |
| deserialization | 18 | 25 |
| injection | 30 | 53 |
| output encoding | 18 | 28 |
| path | 25 | 29 |
| untrusted destination | 18 | 32 |
Precision at the top of the ranking now limits blocking.
Data volume alone cannot resolve this.
The next levers are graph evidence in the prompt and the combiner.
Records: `benchmarks/experiments/exp-004-pooled-block-gate/record.json` and `benchmarks/experiments/exp-004-pooled-block-gate/result.json`.
### exp-004: pooled block gate with a family floor (registered, not run)
This registration describes the setup before the run reported above.
The maintainer amended the BLOCK rule on 2026-10-03 (requirements, Req 6.4).
Precision is now shown across all families together, not per family.
Each family's score is first mapped by its own Platt fit, on training folds.
One threshold, t_block, then serves every family.
It is the lowest one whose pooled Wilson lower bound reaches 0.95 on training folds.
A family blocks only with at least 10 held-out flags and no error.
Otherwise its flags stay advisory.
Below a per-family t_quiet, which keeps 90% of positives, findings are silent.
No held-out label moves a threshold.
- **Primary metric:** the pooled held-out Wilson lower bound of flagged precision, paired units, before the floor.
- **Decision:** reject below point precision 0.95; adopt if the bound reaches 0.95 and a family passes the floor.
- **Units:** frozen from the memory after the negatives harvest.
- **Cost:** the run replayed the cache after a paid cache fill.
Manifest: `plane/experiments/exp-004-pooled-block-gate.yaml`.
The reading is on [Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
## Commands
`ousast learn` is a maintainer command group over the engine's data. Paid steps take
`--budget-usd`; `--replay-only` answers every call from the response cache at no cost.
`--budget-usd`. `--replay-only` answers every call from the response cache at no cost.
| Command | Does |
| --- | --- |
@@ -127,8 +288,36 @@ candidate, with re-run agreement 0.875 against 0.906 and an interval spanning ze
| `ousast learn curve --program ID` | the metric over memory size on a fixed candidate subset |
| `ousast learn audit-leaks` | how well each signal alone separates the sides of a pair (no model) |
| `ousast learn experiment register\|units\|run\|analyse MANIFEST` | A/B experiments from a pre-registered manifest (`plane/experiments/<id>.yaml`): record its digest, freeze the units (no model), run the arms paired (arm A may replay from the cache at $0), analyse (repository bootstrap, exact McNemar, two looks, the adoption verdict) |
| `ousast learn experiment fill MANIFEST --budget-usd X` | Fill missing responses for a registered `kind: rule` experiment's frozen units before its zero-cost run; `--dry-run` inspects and estimates without calls or writes |
`experiment fill` accepts `--memory URL`, `--profile static|plane` (default `static`),
`--manifest-model PATH` (default `plane/models/deepseek-flash.yaml`) and `--out PATH`.
Freeze, commit and register the units first. Fill uses the incumbent programs and the
same retrieval, sampling and cache keys as replay. The first missing request is the
smoke call; a successful smoke response is kept. Any provider failure stops the fill.
The JSON report on stdout gives each family's units, distinct requests, initially
cached and missing responses, newly filled responses and still-missing responses,
plus total client calls and metered USD. `--out` also saves it, except during dry-run,
which writes nothing. Exit codes are 0 for completion or dry-run, 1 for a provider/cache
failure, 2 for invalid setup and 3 for budget exhaustion. Resume a partial fill with a
new invocation and budget; that budget covers only the new invocation, including smoke.
Dry-run uses each family's mean billed cached response cost when recorded usage exists.
Model prices account for cache-hit input, cache-miss input, and output tokens.
Without cached usage, it uses the program's input-token estimate and **300 output tokens**.
Each family's `estimate_method` reports `measured` or `estimated`.
It reports potential retries separately as `conditional_retries`: an unseen answer
may need one parse retry, whose necessity cannot be known without calling the model.
Known invalid cached answers include their retry requests in the inventory.
Fill reserves at least **$0.01** and the largest observed call cost before each call.
For the production client it also reserves a conservative input-byte token bound plus
the program's enforced output-token limit; hidden provider retries are disabled.
Consequently a small budget may stop before making any call, and a fill may finish
with unused budget. Calls without token usage stop as failures, never as free calls.
The memory store is the plane's (`OUSAST_MEMORY`; see [ops/ax/README.md](ops/ax/README.md)).
The harvests that produced the plane signals ran as plane Runs generated by
`ousast plane harvest` (`benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`
and, for the advisory-fix pairs, `2026-10-02-decision-engine-harvest-advisory/record.json`).
`ousast plane harvest`. Records: `benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`
and, for the advisory-fix pairs, `2026-10-02-decision-engine-harvest-advisory/record.json`.
+387 -243
View File
@@ -1,17 +1,21 @@
# Deployment
How the agentic plane is deployed today, and what moving it to a separate Kubernetes cluster
would take. State as of 2026-10-02 (v2.0.0). A scan needs none of this: `ousast scan` and
`ousast pre-push` run from the Python package (and the Docker image for Joern, see
[Engine and pre-push](ops/README.md)). The plane is the maintainers' surface for model work over
whole repositories, driven by `ousast plane`.
This page covers two things. First, how the agentic plane is deployed today. Second, what moving it to a
separate Kubernetes cluster would take. It describes the state as of 2026-10-02 (v2.0.0).
Two words are used strictly on this page. **Implemented** means the code on `main` does it and it
has run on the maintainer's host. **Planned** means it has not been done; the guidance is derived
from the code and the host's lessons, and the code changes it needs are named with their file paths.
A scan needs none of this. `ousast scan` and `ousast pre-push` run from the Python package. Joern runs
from the Docker image (see [Engine and pre-push](ops/README.md)). The plane is the maintainers' tool for
model work over whole repositories. `ousast plane` drives it.
This page uses two words strictly:
- **Implemented** means the code on `main` does it, and it has run on the maintainer's host.
- **Planned** means it has not been done. The guidance comes from the code and from what the host taught
us. Each code change it needs is named with its file path.
**No deployment of the plane to a cluster other than the maintainer's single-node kind cluster has
been made.** Section B is guidance, not a record.
been made.** Section B is guidance, not a record. (kind runs a Kubernetes cluster inside Docker
containers on one machine.)
## What runs where today (implemented)
@@ -27,11 +31,11 @@ flowchart LR
sub["Agent Substrate (ate-system): atenet-router, atenet-egress"]
act["Task actors in gVisor workers (WorkerPool ousast-pool)"]
svc["Service ousast-receiver + EndpointSlice to the host"]
reg["Registry localhost:5001: runner image by digest"]
reg["Registry named by the profile: runner image by digest"]
end
mem[("Memory store: file store on the host, or an S3 bucket (RustFS tested)")]
rec -- "ax apply, resume, get, delete" --> ax
rec -- "EgressPolicy per actor; start request via port-forward" --> sub
rec -- "EgressPolicy per actor; start request through the kube-context tunnel" --> sub
sub --> act
act -- "HTTP by Service name through the gateway" --> svc
svc --> rcv
@@ -46,19 +50,26 @@ flowchart LR
| How actors reach the receiver | the Service `ousast-receiver.ax-system` (port 80) with a selector-less EndpointSlice whose endpoint is the kind network's gateway address on the host; the runner dials the ClusterIP with the Service name as `Host`, which the egress gateway allows by hostname | `ops/ax/receiver-service.yaml.tmpl`, rendered by `up.sh` |
| ax control plane, Agent Substrate, egress gateway, router | the kind cluster (`ax-system`, `ate-system`) | installed by `ops/ax/up.sh` from the upstream checkouts under `~/.cache/ousast/ax-src/` |
| Tasks | one actor per Task in a gVisor worker; two warm workers of 1 CPU / 1.5 GiB | `ops/ax/workerpool.yaml.tmpl` |
| Runner image | the kind-local registry `localhost:5001`, pinned by digest (`~/.cache/ousast/ax-src/runner-image`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through a `kubectl port-forward` to `svc/atenet-router` | `src/openultrasast/plane/router.py` |
| Runner image | the registry the profile names (`registry` in `ops/k8s/profiles/kind.toml`: the kind-local one), pinned by digest in the profile's images file (`ops/k8s/profiles/kind-images.json`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through the tunnel `router.py` opens to `svc/atenet-router` in the profile's kube context | `src/openultrasast/plane/router.py` |
| Memory store | `OUSAST_MEMORY`: a file store under `~/ousast-results/plane/memory` by default, or `s3://<bucket>[/<prefix>]` on any reachable S3-compatible server (RustFS is the tested one) | `src/openultrasast/plane/memory.py`, [Memory](memory.md), [RustFS setup](rustfs.md) |
| Run state and artifacts | `$OUSAST_RESULTS/plane/<run>/` on the host (default `~/ousast-results/plane/`) | `reconciler.py` (`run_dir`) |
Only the reconciler talks to the memory store: a sandboxed task cannot read it, so the loop's
`memory-snapshot` is seeded by the reconciler before the Run ([The plane on ax](plane.md)).
Terms in the table:
- **ax** is Google's controller that runs agent tasks on Kubernetes.
- **Agent Substrate** is the layer below ax that owns the machines and runs sandboxes.
- **gVisor** is a sandbox runtime that isolates each actor from the node's kernel.
- An **actor** is one running Task inside a sandbox.
Only the reconciler talks to the memory store. A sandboxed task cannot read it. So the reconciler seeds
the loop's `memory-snapshot` before the Run starts ([The plane on ax](plane.md)).
## A. This host: the local setup (implemented)
What `ops/ax/up.sh` needs on the PATH: `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` (it
refuses to start when one is missing). Tools live in `~/go/bin` and `~/.local/bin`; sources and
rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
`ops/ax/up.sh` needs these tools on the PATH: `docker`, `go`, `kubectl`, `kind`, `ko` and `ax`. It
refuses to start when one is missing. Tools live in `~/go/bin` and `~/.local/bin`. Sources and rendered
files live in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
1. **Bring up the cluster and the plane.**
@@ -66,25 +77,39 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
ops/ax/up.sh
```
Idempotent: each step is skipped when already done. It creates the kind cluster
(`KIND_CLUSTER_NAME`, default `ousast`) with its registry (`KO_DOCKER_REPO`, default
`localhost:5001`), installs Agent Substrate, applies the egress gateway (the agentgateway
variant, one prebuilt image) when the install left it out, deploys the ax control plane, applies
the receiver Service with the host's kind-gateway address as its endpoint, applies the gVisor
worker pool, builds and pushes the runner image and writes its digest pin to
`~/.cache/ousast/ax-src/runner-image`, installs the `ax` CLI from the deployed checkout (a release
CLI skews from the server), and runs one smoke Task. It ends by printing the host's memory and the
kind containers' usage.
The script is idempotent: it skips each step that is already done. In order, it:
- creates the kind cluster (`KIND_CLUSTER_NAME`, default `ousast`) with its registry
(`KO_DOCKER_REPO`, default: the `registry` of `ops/k8s/profiles/kind.toml`);
- installs Agent Substrate;
- applies the egress gateway if the install left it out (the agentgateway variant, one prebuilt image);
- deploys the ax control plane;
- applies the receiver Service, with the host's kind-gateway address as its endpoint;
- applies the gVisor worker pool;
- builds and pushes the runner image, and writes its digest pin to `~/.cache/ousast/ax-src/runner-image`;
- installs the `ax` CLI from the deployed checkout (a release CLI does not match the server version);
- runs one smoke Task.
At the end it prints the host's memory and the kind containers' usage.
2. **Check it.**
```bash
uv run ousast plane doctor
uv run ousast plane doctor --profile kind
```
Four checks: the kind context is reachable, every pod in `ate-system` and in `ax-system` is ready,
and the registry at `localhost:5001` holds `ousast-runner`. Each failing line names
`ops/ax/up.sh` as the fix. `doctor` is informational: `ousast plane run` does not call it.
It checks the profile you pass (`--profile`, default `kind`; the profiles are in `ops/k8s/profiles/`):
- its configured addresses, printed on the first line;
- the kube context is reachable;
- every pod in `ate-system` is ready;
- every pod in `ax-system` is ready;
- the runner image of the profile's images file is in its registry (a local registry over its v2
API, a remote one through `docker manifest inspect`);
- the memory store opens (an S3 bucket is verified).
Each failing line names the fix (`ops/ax/up.sh` for the kind profile). `doctor` is informational:
`ousast plane run` does not call it.
3. **Run the end-to-end smoke Run.**
@@ -92,23 +117,27 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
ops/ax/smoke-run.sh
```
It renders `ops/ax/smoke-run.yaml.tmpl` with the current runner digest and submits the Run
`ax-e2e-smoke` through the reconciler: one model-free `repo-facts` Task (`budget: {usd: 0,
calls: 0}`) over a files-only Workspace, resumed, started through the router, its `facts.json` and
`summary.json` delivered to the host, then `ousast plane status ax-e2e-smoke` prints the attribution
table. The script calls the repository's `.venv/bin/ousast`, so `uv sync` first.
It renders `ops/ax/smoke-run.yaml.tmpl` with the current runner digest. It then submits the Run
`ax-e2e-smoke` through the reconciler. The Run has one model-free `repo-facts` Task (`budget: {usd: 0,
calls: 0}`) over a files-only Workspace, resumed, started through the router, its `facts.json`
and `summary.json` delivered to the host. Then `ousast plane status ax-e2e-smoke`
prints the attribution table. The script calls the repository's `.venv/bin/ousast`, so run `uv sync`
first.
4. **Keys and the store.** Put `DEEPSEEK_API_KEY` (chat) and `OPENROUTER_API_KEY` (embeddings) in
`.env` or export them; a Task's Model names the variable (`secretKey.key`), and the key travels
only in that Task's start request. Set `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT`,
`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` (optional `S3_REGION`, `S3_BUCKET`) for the S3 store;
the bucket must be set up once by an admin as [RustFS setup](rustfs.md) describes, and the store
verifies it at every start.
4. **Keys and the store.**
- Put `DEEPSEEK_API_KEY` (chat) and `OPENROUTER_API_KEY` (embeddings) in `.env`, or export them.
- A Task's Model names the variable (`secretKey.key`). The key travels only in that Task's start
request.
- For the S3 store, set `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY`. `S3_REGION` and `S3_BUCKET` are optional.
- An admin must set up the bucket once, as [RustFS setup](rustfs.md) describes. The store verifies
it at every start.
5. **Run a real Run**, as in [Examples](examples.md#7-running-the-agentic-plane): `ousast plane run
plane/runs/validation-46.yaml`, then `status` and `remember`.
6. **Tear down** with `ops/ax/down.sh`, which deletes the cluster and its registry.
6. **Tear down** with `ops/ax/down.sh`. It deletes the cluster and its registry.
**Footprint** (measured 2026-09-29, idle, two workers; [ax on this host](ops/ax/README.md#measured-footprint-2026-09-29-idle-two-workers)):
@@ -119,44 +148,70 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
| node volume (images, containerd) | 4.9 GB on disk |
| worker pool | 2 gVisor workers, 1 CPU / 1.5 GiB limit each |
Disk is the binding constraint: the Substrate install and the `ko` builds need several GB of Go
caches (`go clean -cache -modcache` afterwards), and every Task is its own actor template with a
golden actor (about 24 MB each under `/var/lib/ate/actors` on the node). The failures the host
produced, and what each one taught, are listed in
[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run).
Disk is the tightest limit, for two reasons:
- The Substrate install and the `ko` builds need several GB of Go caches. Run `go clean -cache -modcache`
afterwards.
- Every Task is its own actor template with a golden actor (a saved snapshot the actor boots from). Each
takes about 24 MB under `/var/lib/ate/actors` on the node.
[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run) lists the
failures the host produced and what each one taught.
## B. A separate Kubernetes cluster (planned)
The target: a cluster that runs Agent Substrate with its egress gateway and the ax control plane,
pulls the runner image from a registry of yours, receives task artifacts without a developer
laptop in the path, and keeps its memory in your S3 bucket. The provider key already travels only
in the start request through `atenet-router`, which works the same through a port-forward to any
cluster. Everything else below is either a step you take or a code change that is not made yet.
The target cluster would do four things:
- run Agent Substrate with its egress gateway and the ax control plane;
- pull the runner image from a registry of yours;
- receive task artifacts without a developer laptop in the path;
- keep its memory in your S3 bucket.
The provider key already travels only in the start request through `atenet-router`. That works the same
through the kube-context tunnel to any cluster. Everything else below is either a step you take or a code
change that is not made yet.
### B.1 Cluster with Agent Substrate, its egress gateway and ax
- Install Agent Substrate on the cluster following its own instructions. `ops/ax/up.sh` uses
Substrate's kind helpers (`hack/create-kind-cluster.sh`, `hack/install-ate-kind.sh`), which do
not apply to another cluster.
- Install Agent Substrate on the cluster with its own instructions. `ops/ax/up.sh` uses Substrate's kind
helpers (`hack/create-kind-cluster.sh`, `hack/install-ate-kind.sh`). These do not apply to another
cluster.
- Apply the egress gateway. On this host it is the agentgateway variant
(`manifests/ate-install/agentgateway-egress` in the Substrate checkout, applied with
`kubectl kustomize --load-restrictor=LoadRestrictionsNone`), which needs no Rust build; the
Envoy variant does. Without the gateway an actor has no network at all, and the per-task
EgressPolicies the reconciler writes (`src/openultrasast/plane/egress.py`) work unchanged
against the Substrate-installed gateway: deny by default, hostnames only, plain HTTP to the
receiver, TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts.
(`manifests/ate-install/agentgateway-egress` in the Substrate checkout). It is applied with
`kubectl kustomize --load-restrictor=LoadRestrictionsNone`. It needs no Rust build; the Envoy variant
does.
- Without the gateway an actor has no network at all.
- The reconciler writes one EgressPolicy per task (`src/openultrasast/plane/egress.py`). These work
unchanged against the gateway that Substrate installs. The rules are:
- deny by default;
- hostnames only;
- plain HTTP to the receiver;
- TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts.
- Deploy ax (`make deploy AX_IMAGE_REPO=<your registry>` in the ax checkout) **after** B.2.
- Apply a WorkerPool (B.6). Substrate installs none by itself ("no free workers").
- **gVisor on the nodes (planned, checked by hand on the day k3s exists).** Substrate does not use a Kubernetes
`RuntimeClass`: the `ateom-gvisor` worker runs `runsc` itself, from the gVisor release tarball named in
`manifests/ate-install/sandboxconfig-gvisor.yaml` of the Substrate checkout
(`gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd`, sha256-pinned per architecture), which
the atelet fetches on the worker node. The node must reach that URL, or a copied `SandboxConfig` must name a
mirrored copy with the same sha256 and the ActorTemplates must name that config. The check is the atelet's
fetch line in `kubectl -n ate-system logs` of the worker pod; it is labelled planned until that log has been
read on a k3s node (plane-on-kubernetes task 1.7, design open question 3).
### B.2 ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET`
ax's deploy manifest (`deploy/ax-server.yaml` in the ax checkout) sets `AX_SNAPSHOTS_BUCKET` to
the ax authors' own bucket (`gs://dberkov-gke-dev3/ate-env/` in the checkout used here). Point it
at **your own bucket before deploying ax**. On this host the bucket named there did not exist in
the S3 server, deleted golden actors piled up in `DELETING` and filled the disk during a 600-task
harvest (2026-10-01); creating the bucket let deletes complete, and even then a deleted actor's
directory stays on the node, so long Runs need a janitor that removes directories no live actor
owns.
Point `AX_SNAPSHOTS_BUCKET` at **your own bucket before deploying ax**. ax's deploy manifest
(`deploy/ax-server.yaml` in the ax checkout) sets it to the ax authors' own bucket. In the checkout used
here that is `gs://dberkov-gke-dev3/ate-env/`.
This host shows what goes wrong otherwise (2026-10-01):
1. The bucket named there did not exist in the S3 server.
2. Deleted golden actors piled up in `DELETING`.
3. They filled the disk during a 600-task harvest.
Creating the bucket let deletes complete. Even then, a deleted actor's directory stays on the node. So
long Runs need a janitor that removes directories no live actor owns.
### B.3 A registry the workers can pull from, the runner image pinned by digest
@@ -168,83 +223,109 @@ owns.
docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' <registry>/ousast-runner:dev
```
- Substrate rejects tags: every Task's `image` must be the `<registry>/ousast-runner@sha256:...`
reference. Write it into a file (on this host, `~/.cache/ousast/ax-src/runner-image`) and pass
`--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`, which re-pin the
Tasks they generate. The committed templates under `plane/tasks/` and the generated
`plane/tasks/validation-46.yaml` carry `localhost:5001/ousast-runner@sha256:...` and need the same
re-pin (regenerate, or replace the reference) before they run elsewhere.
- Workers must be able to pull from the registry (credentials, if any, are the cluster's concern;
ax's default runner image on `gcr.io` needs credentials and is not used here).
- **Not yet configurable:** `ousast plane doctor` checks the registry at `localhost:5001` by name
(`src/openultrasast/plane/doctor.py:45`), so its fourth check fails on another cluster even when
the image is in place. `doctor` is informational, `plane run` does not depend on it.
- Substrate rejects tags. Every Task's `image` must be the `<registry>/ousast-runner@sha256:...`
reference.
- Write that reference into a file. On this host the file is `~/.cache/ousast/ax-src/runner-image`.
- Pass `--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`. They re-pin the
Tasks they generate.
- The committed templates under `plane/tasks/` and the generated `plane/tasks/validation-46.yaml` carry
the kind profile's registry and digest. They need the same re-pin before they run elsewhere.
Regenerate them, or replace the reference.
- Workers must be able to pull from the registry. Credentials, if any, are the cluster's concern. ax's
default runner image on `gcr.io` needs credentials and is not used here.
- `ousast plane doctor --profile <name>` looks up the runner reference of the profile's images file in
the profile's `registry` (implemented 2026-10-02, plane-on-kubernetes task 1.2). `doctor` is
informational; `plane run` does not depend on it.
### B.4 One configurable kube context
### B.4 One configurable kube context (implemented 2026-10-02)
The reconciler, the egress client and the router port-forward all use the context
`kind-$KIND_CLUSTER_NAME` (`kind-ousast` by default). **Not yet configurable**: the prefix `kind-`
is hard-coded in `src/openultrasast/plane/doctor.py:39`, `src/openultrasast/plane/egress.py:115`
and `:148`, and `src/openultrasast/plane/router.py:145`; `ops/ax/up.sh:24` builds the same name.
The planned change is one variable (for example `OUSAST_KUBE_CONTEXT`) read in those three
modules. Until then the only way to drive another cluster from the unchanged code is to give its
kubeconfig context a name of that form (`kubectl config rename-context <ctx> kind-<name>` and
`KIND_CLUSTER_NAME=<name>`), which is a workaround, not a supported configuration. The egress
client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names) able to use that
context.
The reconciler, the egress client and the router tunnel take the kube context from the profile.
- The setting is `kube_context` in `ops/k8s/profiles/<name>.toml`; `OUSAST_KUBE_CONTEXT` overrides it
(plane-on-kubernetes task 1.2).
- `doctor.py`, `egress.py` and `router.py` are handed the context. They build none themselves.
- `tests/test_plane_literals.py` fails on any module, manifest or page that names this host's cluster,
registry or addresses.
The egress client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names). It must be
able to use that context.
### B.5 The receiver: in-cluster instead of on the host
Today the receiver is a thread inside the reconciler process (`egress.py`, `Receiver`), and the
reconciler learns of a task's completion by watching that object's `delivered` set in memory
(`reconciler.py`). Actors reach it through the Service `ousast-receiver.ax-system` whose
EndpointSlice points at the host. Three paths, in order of effort:
Today the receiver is a thread inside the reconciler process (`egress.py`, `Receiver`). The reconciler
learns that a task finished by watching that object's `delivered` set in memory (`reconciler.py`).
Actors reach it through the Service `ousast-receiver.ax-system`. That Service's EndpointSlice points at
the host.
1. **Interim, works with the unchanged code:** keep the host receiver and render
`ops/ax/receiver-service.yaml.tmpl` with an endpoint address the egress gateway pod can reach
(`KIND_GATEWAY` is just an IPv4 address in the template, `OUSAST_ARTIFACT_PORT` the port). The
reconciler's host must be reachable from the cluster (a VPN or a routed address), and the
gateway dials the address the actor connected to, so the Service's ClusterIP must still map to it.
2. **Planned: the receiver as an in-cluster Deployment** behind the same Service name, with the
reconciler reading deliveries and serving inputs from it instead of from its own thread. This
needs code: the receiver and its `delivered`/`inputs` state are in-process today, and the
runner's `GET /inputs/<producer>/<artifact>` is served by that same process.
3. **Follow-on: presigned-URL delivery to the S3 store.** The runner would `PUT` its output tar to a
presigned URL on the memory store's bucket and fetch inputs the same way, and the reconciler
would read from the bucket. Nothing of this exists; the EgressPolicy would then have to allow
the store's hostname for the task.
There are three paths, from least to most effort:
1. **Interim, works with the unchanged code:** keep the host receiver.
Render `ops/ax/receiver-service.yaml.tmpl` with an endpoint address the egress gateway pod can reach.
In the template, `KIND_GATEWAY` is just an IPv4 address and `OUSAST_ARTIFACT_PORT` is the port.
The cluster must be able to reach the reconciler's host, through a VPN or a routed address.
The gateway dials the address the actor connected to. So the Service's ClusterIP must still map to it.
2. **Planned: the receiver as an in-cluster Deployment** behind the same Service name.
The reconciler would read deliveries from it and serve inputs from it, instead of from its own thread.
This needs code. The receiver and its `delivered`/`inputs` state are in-process today.
The same process serves the runner's `GET /inputs/<producer>/<artifact>`.
3. **Follow-on: presigned-URL delivery to the S3 store.** (A presigned URL grants one upload or download
without credentials.)
The runner would `PUT` its output tar to a presigned URL on the memory store's bucket.
It would fetch inputs the same way, and the reconciler would read from the bucket.
None of this exists. The EgressPolicy would then have to allow the store's hostname for the task.
### B.6 Worker pool sizing
`ops/ax/workerpool.yaml.tmpl` declares two warm gVisor workers of 1 CPU / 1.5 GiB (requests 250m /
1.5 GiB) for the 7 GB host, selected onto nodes by `ate.dev/substrate-version`. An actor occupies a
whole worker, so the pool size is the number of tasks that can run at once; set `--workers` on
`ousast plane run` to match. Size the workers above the largest Task's limits (`verify`: requests
250m / 384Mi, limits 1 CPU / 1 GiB; `roles` and the others are in `plane/tasks/`), and keep the
golden-actor storage on the node in mind (B.2).
`ops/ax/workerpool.yaml.tmpl` declares two warm gVisor workers for the 7 GB host. Each has 1 CPU /
1.5 GiB (requests 250m / 1.5 GiB). They are placed on nodes by `ate.dev/substrate-version`.
An actor occupies a whole worker. So the pool size is the number of tasks that can run at once. Set
`--workers` on `ousast plane run` to match.
Size the workers above the largest Task's limits. For `verify` these are requests 250m / 384Mi and
limits 1 CPU / 1 GiB. `roles` and the others are in `plane/tasks/`. Also keep the golden-actor storage
on the node in mind (B.2).
### B.7 The S3 memory store reachable from where the reconciler runs
The `s3://` store already works against any reachable S3-compatible server with versioning,
lifecycle rules, object tags and S3 Select (RustFS is the tested one; the bucket setup and the
agent policy are on [RustFS setup](rustfs.md)). Only the reconciler reads and writes it, so today
it must be reachable from the operator's host; if the reconciler moves into the cluster, from its
pod. No Task talks to the store directly; the day one does (B.5, path 3), that Task's EgressPolicy
must allow the store's hostname (the policies allow hostnames only, never addresses), the way a
Model declares its hosts today through `openultrasast.io/egress-hosts`.
The `s3://` store already works against any reachable S3-compatible server. The server must support:
- versioning;
- lifecycle rules;
- object tags;
- S3 Select (a server-side query over JSON objects).
RustFS is the tested server. The bucket setup and the agent policy are on [RustFS setup](rustfs.md).
Only the reconciler reads and writes the store. So today the operator's host must reach it. If the
reconciler moves into the cluster, its pod must reach it.
No Task talks to the store directly. If one ever does (B.5, path 3), its EgressPolicy must allow the
store's hostname. The policies allow hostnames only, never addresses. A Model already declares its hosts
this way today, through `openultrasast.io/egress-hosts`.
### B.8 Secrets in Kubernetes Secrets, injected into the reconciler's environment
The reconciler reads the variable a Model's `secretKey.key` names (`DEEPSEEK_API_KEY`,
`OPENROUTER_API_KEY`) and the store's `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`
(optional `AWS_SESSION_TOKEN`, `S3_REGION`, `S3_BUCKET`) from its own environment or `.env`. When
the reconciler runs in the cluster, keep them in Kubernetes Secrets and inject them into **the
reconciler's** pod environment (`envFrom` with `secretRef`, or `valueFrom.secretKeyRef` per
variable). Never put a value into a manifest: the Model manifests name a variable, the rendered
Task carries no key, and the key reaches an actor only in the start request, where the runner keeps
it in memory and redacts it from echoed stderr. The `secretKey.name` values in `plane/models/`
(`deepseek`, `openrouter`) are the names such Secrets would carry. Running the reconciler
in-cluster is itself planned: today it is a host process, and its state directory
The reconciler reads these variables from its own environment or `.env`:
- the variable a Model's `secretKey.key` names (`DEEPSEEK_API_KEY`, `OPENROUTER_API_KEY`);
- the store's `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`;
- optionally `AWS_SESSION_TOKEN`, `S3_REGION` and `S3_BUCKET`.
When the reconciler runs in the cluster, keep these in Kubernetes Secrets. Inject them into **the
reconciler's** pod environment. Use `envFrom` with `secretRef`, or `valueFrom.secretKeyRef` per variable.
Never put a value into a manifest. The key stays out of every manifest:
- the Model manifests name a variable;
- the rendered Task carries no key;
- the key reaches an actor only in the start request;
- the runner keeps it in memory and redacts it from echoed stderr.
The `secretKey.name` values in `plane/models/` (`deepseek`, `openrouter`) are the names such Secrets
would carry.
Running the reconciler in-cluster is itself planned. Today it is a host process. Its state directory
(`OUSAST_RESULTS`) would need a persistent volume.
### What is implemented and what is planned
@@ -255,10 +336,10 @@ in-cluster is itself planned: today it is a host process, and its state director
| Per-task EgressPolicy, credentials only in the start request, per-task budgets and attribution | implemented |
| S3 memory store against any reachable S3-compatible server, verified at startup | implemented (RustFS tested) |
| Local bring-up, doctor, smoke Run (`ops/ax/`) | implemented |
| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates still name `localhost:5001` |
| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates name the kind profile's registry (`tests/test_plane_literals.py` checks it) |
| Deployment to a cluster other than this host's kind cluster | **not done** |
| One configurable kube context | **planned**; `kind-` hard-coded in `doctor.py`, `egress.py`, `router.py` |
| `doctor` registry check against a configurable registry | **planned**; `localhost:5001` hard-coded in `doctor.py:45` |
| One configurable kube context | implemented (2026-10-02): `kube_context` in the profile, `OUSAST_KUBE_CONTEXT` overrides |
| `doctor` registry check against a configurable registry | implemented (2026-10-02): the profile's `registry` and images file |
| Receiver as an in-cluster Deployment | **planned** (code change) |
| Presigned-URL delivery to the S3 store | **follow-on**, not started |
| Reconciler running in-cluster with Secrets injected into its environment | **planned** |
@@ -266,12 +347,20 @@ in-cluster is itself planned: today it is a host process, and its state director
## Production topology: ax on a Kubernetes cluster
The maintainer's questions, answered from the code and the upstream checkouts under
`~/.cache/ousast/ax-src/`: how is it hosted in ax, is there one container, how does ax relate to
Kubernetes, and what scaling the pre-push hook would take. The decisions of 2026-10-02 set the
target: **two profiles of one code base**, `kind` (this laptop, kept as a first-class local
execution mode, where every change is proven first) and `k3s` (production). Everything marked
**planned** is not built; the rest is what runs on this host today.
This section answers the maintainer's questions from the code and the upstream checkouts under
`~/.cache/ousast/ax-src/`:
- How is it hosted in ax?
- Is there one container?
- How does ax relate to Kubernetes?
- What would scaling the pre-push hook take?
The decisions of 2026-10-02 set the target: **two profiles of one code base**.
- `kind` is this laptop. It stays a first-class local execution mode. Every change is proven here first.
- `k3s` is production. (k3s is a small Kubernetes distribution.)
Everything marked **planned** is not built. The rest is what runs on this host today.
```mermaid
flowchart LR
@@ -287,7 +376,7 @@ flowchart LR
ctl["ate-api-server, ate-controller, atelet"]
rt["atenet-router"]
eg["atenet-egress gateway"]
wp["WorkerPool: gVisor workers, runsc RuntimeClass, HPA (planned)"]
wp["WorkerPool: gVisor workers, runsc in the worker, HPA (planned)"]
ewp["Engine WorkerPool: Joern as a task (planned)"]
end
end
@@ -308,124 +397,173 @@ flowchart LR
cli -- "polls: status, attribution" --> s3
```
No inbound path exists in this picture: the CLI reaches ax-server and the router from outside, the
tasks push their artifacts to the store through the egress gateway, and the CLI reads the store.
Today's EndpointSlice back to a laptop (`ops/ax/receiver-service.yaml.tmpl`) is the kind profile's
device and does not appear in production.
This picture has no inbound path:
- the CLI reaches ax-server and the router from outside;
- the tasks push their artifacts to the store through the egress gateway;
- the CLI reads the store.
Today's EndpointSlice back to a laptop (`ops/ax/receiver-service.yaml.tmpl`) belongs to the kind
profile only. It does not appear in production.
### ax, Kubernetes and Agent Substrate in three sentences
**ax is a Kubernetes application**: a Deployment `ax-server` with a Redis beside it in the
namespace `ax-system` (`ax/deploy/ax-server.yaml` in the ax checkout; its controller runs with
`--template=default-template --template-atespace=ax-system`), and the `ax` CLI reaches it through a
`kubectl port-forward` to `svc/ax-server` (`ax/internal/tunnel/tunnel.go`), so `ousast plane run`
needs a kube context or an ingress, not a public endpoint. **ax does not run containers itself**:
it turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`,
`CreateActorTemplate`), built from the Task's image and environment, and the image must be pinned
by digest (Substrate's `SandboxConfig` validation rejects tags: "All images must include a
digest"). **Agent Substrate is the layer that owns the machines**: in the namespace `ate-system`
it runs `ate-api-server`, `ate-controller`, the `atelet` DaemonSet, `atenet-router`, the
`atenet-egress` gateway, a Postgres and a RustFS (`substrate/manifests/ate-install/`), and three
CRDs under `ate.dev/v1alpha1`, `WorkerPool`, `SandboxConfig` and `CSIDriverConfig`; a
`WorkerPool` keeps warm worker pods (`ko://.../cmd/ateom-gvisor`), and an actor is one gVisor
sandbox inside one of those workers, booted from the golden snapshot that
[The plane on ax](plane.md) describes, suspended and resumed by the controller. Names are limited
to 63 bytes (`MaxNameLength` in `ax/pkg/apis/v1alpha1/types.go`), which is why the generated Task
names are short.
**ax is a Kubernetes application.**
**What k3s changes** (planned; nothing has been applied to a k3s cluster): k3s runs containerd
with no Docker socket, so the host-Docker path of `ousast plane alerts-engine` does not exist
there and the engine must become a task (below); Substrate's gVisor workers need `runsc` and a
`RuntimeClass` on the nodes they are scheduled to; k3s ships Traefik (the ingress the CLI can use
instead of a kubeconfig) and local-path storage. Images are built by GitHub Actions from tagged
commits, published to GitHub Container Registry and digest-pinned in the manifests; a private
package needs an `imagePullSecret` on the worker ServiceAccount. Substrate's and ax's own images
are built with `ko` from their checkouts and pushed to the same registry. The kind profile keeps
`localhost:5001` and `ops/ax/up.sh`.
- It is a Deployment `ax-server` with a Redis beside it, in the namespace `ax-system`
(`ax/deploy/ax-server.yaml` in the ax checkout).
- Its controller runs with `--template=default-template --template-atespace=ax-system`.
- The `ax` CLI reaches it through a tunnel to `svc/ax-server` over the kube context
(`ax/internal/tunnel/tunnel.go`).
- So `ousast plane run` needs a kube context or an ingress, not a public endpoint.
**ax does not run containers itself.**
- It turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`,
`CreateActorTemplate`).
- The ActorTemplate is built from the Task's image and environment.
- The image must be pinned by digest. Substrate's `SandboxConfig` validation rejects tags: "All images
must include a digest".
**Agent Substrate is the layer that owns the machines.**
- In the namespace `ate-system` it runs `ate-api-server`, `ate-controller`, the `atelet` DaemonSet,
`atenet-router`, the `atenet-egress` gateway, a Postgres and a RustFS
(`substrate/manifests/ate-install/`).
- It defines three CRDs (custom Kubernetes resource types) under `ate.dev/v1alpha1`: `WorkerPool`,
`SandboxConfig` and `CSIDriverConfig`.
- A `WorkerPool` keeps warm worker pods (`ko://.../cmd/ateom-gvisor`).
- An actor is one gVisor sandbox inside one of those workers.
- The actor boots from the golden snapshot that [The plane on ax](plane.md) describes. The controller
suspends and resumes it.
- Names are limited to 63 bytes (`MaxNameLength` in `ax/pkg/apis/v1alpha1/types.go`). That is why the
generated Task names are short.
**What k3s changes** (planned; nothing has been applied to a k3s cluster):
- k3s runs containerd with no Docker socket. So the host-Docker path of `ousast plane alerts-engine`
does not exist there. The engine must become a task (below).
- Substrate's gVisor workers need no Kubernetes `RuntimeClass`: the worker fetches a pinned gVisor
tarball and runs `runsc` inside its own pod. The node checklist is pod security, tarball
reachability and local-path storage (B.1).
- k3s ships Traefik and local-path storage. Traefik is an ingress the CLI can use instead of a kubeconfig.
- GitHub Actions builds the images from tagged commits. They are published to GitHub Container Registry
and digest-pinned in the manifests.
- A private package needs an `imagePullSecret` on the worker ServiceAccount.
- Substrate's and ax's own images are built with `ko` from their checkouts. They are pushed to the same
registry.
- The kind profile keeps its local registry (`ops/k8s/profiles/kind.toml`) and `ops/ax/up.sh`.
### One image, and what concurrency equals
There is **one container image** for every Task: `plane/Dockerfile.runner` (`python:3.12-slim`
plus this package, PID 1 `/usr/local/bin/ax-task-runner`, which is
`openultrasast.plane.runner`). Each Task manifest under `plane/tasks/` names the same image by
digest (the pin is kept in `~/.cache/ousast/ax-src/runner-image`) and selects its work with
`spec.command`: the runner starts `python -m openultrasast.plane.tasks.<command[0]>`
(`runner.py`, `task_module`). `repo-facts`, `verify`, `agree`, `features`, `remember`, `alerts`,
`loop` and `roles` are modules in that image, not images of their own; the engine would be the
second image (planned). So the production answer to "how many containers" is: one image, one
actor per Task, and **an actor occupies a whole worker** (its limits are the ActorTemplate's, the
worker holds one actor at a time). Concurrency is therefore the number of workers in the
`WorkerPool`, nothing else: `ops/ax/workerpool.yaml.tmpl` declares `replicas: 2` of 1 CPU /
1.5 GiB, so at most two Tasks run at once on this host, and `ousast plane run --workers` must not
exceed the pool. Substrate ships an autoscaled pool
(`substrate/demos/autoscaled-workerpool/README.md`: an HPA on the external metric
`ate_workerpool_workers{state=at_capacity}` through prometheus-adapter, writing
`WorkerPool.spec.replicas`; its README says the demo is kind-only today) and request parking
(`substrate/demos/parking/`: the router holds a resume until a worker frees up). Neither is
applied by `ops/ax/up.sh`; the production profile's pool manifests and autoscaling rule are
There is **one container image** for every Task: `plane/Dockerfile.runner`. It is `python:3.12-slim`
plus this package. PID 1 is `/usr/local/bin/ax-task-runner`, which is `openultrasast.plane.runner`.
Each Task manifest under `plane/tasks/` names the same image by digest. The pin is kept in
`~/.cache/ousast/ax-src/runner-image`. The manifest picks its work with `spec.command`. The runner
starts `python -m openultrasast.plane.tasks.<command[0]>` (`runner.py`, `task_module`).
`repo-facts`, `verify`, `agree`, `features`, `remember`, `alerts`, `loop` and `roles` are modules in
that image. They are not images of their own. The engine would be the second image (planned).
So the production answer to "how many containers" is:
- one image;
- one actor per Task;
- **an actor occupies a whole worker**. Its limits are the ActorTemplate's, and the worker holds one
actor at a time.
Concurrency is therefore the number of workers in the `WorkerPool`, nothing else.
`ops/ax/workerpool.yaml.tmpl` declares `replicas: 2` of 1 CPU / 1.5 GiB. So at most two Tasks run at
once on this host. `ousast plane run --workers` must not exceed the pool.
Substrate ships two scaling aids:
- **An autoscaled pool** (`substrate/demos/autoscaled-workerpool/README.md`). An HPA (Horizontal Pod
Autoscaler) reads the external metric `ate_workerpool_workers{state=at_capacity}` through
prometheus-adapter. It writes `WorkerPool.spec.replicas`. Its README says the demo is kind-only today.
- **Request parking** (`substrate/demos/parking/`). The router holds a resume until a worker frees up.
`ops/ax/up.sh` applies neither. The production profile's pool manifests and autoscaling rule are
**planned**.
### The three components that are bound to the host today
| Component | Today (the kind profile) | Production (the k3s profile, planned) |
| --- | --- | --- |
| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against `kind-ousast`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against the profile's `kube_context`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
| Artifact receiver (`plane/egress.py`, the `Receiver` thread inside the reconciler) | reached through the Service `ousast-receiver.ax-system` whose EndpointSlice points at the kind gateway address on the host (`ops/ax/receiver-service.yaml.tmpl`) | **no receiver in remote mode.** Tasks deliver their artifacts and fetch their inputs through presigned URLs on the S3 memory store (B.5, path 3; the store's hostname enters each Task's EgressPolicy, B.7), and the CLI reads completion from the store. The kind profile keeps the receiver thread; an in-cluster receiver Deployment (B.5, path 2) is not the chosen direction |
| Joern engine (`plane/engine_alerts.py`, the host's `openultrasast:dev` container, run with `--memory 3g`) | `ousast plane alerts-engine` runs it on the host, one container at a time, and marks the Run's `alerts` tasks done | **an `engine` task on a second, larger `WorkerPool`** (the engine does not fit a 1.5 GiB worker, and k3s has no Docker socket), from an engine image digest-pinned in GHCR, emitting the same `alerts.jsonl` rows and coverage; `alerts-engine` on the host stays as the development path |
The S3 memory store (RustFS tested) is already external; in remote mode it is also the artifact
path, so it must be reachable from the CLI's host and, through the egress gateway, from the
workers.
The S3 memory store (RustFS tested) is already external. In remote mode it is also the artifact path.
So the CLI's host must reach it, and the workers must reach it through the egress gateway.
### Scaling the pre-push hook in production
**Today** `ousast pre-push` is a local, deterministic check: it materialises base and head,
discovers the changed regions, runs the evidence ranker and arbiter on head and compares with base,
all within `--deadline` (30 s by default) and in `--mode advisory` unless blocking is opted into
([Pre-push: the delta check](scanning.md#pre-push-the-delta-check-experimental),
`src/openultrasast/push/`, [Engine and pre-push](ops/README.md#experimental-pre-push-integration)).
**The plane is not in that path**: no hook submits a Run, no model is called without an explicit
`--model-config`, and nothing leaves the developer's machine.
**Today** `ousast pre-push` is a local, deterministic check. It does four things:
1. materialises base and head;
2. finds the changed regions;
3. runs the evidence ranker and arbiter on head;
4. compares the result with base.
All of this runs within `--deadline` (30 s by default). It runs in `--mode advisory` unless you opt
into blocking. See [Pre-push: the delta check](scanning.md#pre-push-the-delta-check-experimental),
`src/openultrasast/push/` and [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
**The plane is not in that path.**
- No hook submits a Run.
- No model is called without an explicit `--model-config`.
- Nothing leaves the developer's machine.
**The production shape** (planned, none of it built) splits the work by latency:
1. **Blocking, inside the hook's deadline**: what runs today, the quick rules and the delta
engine on the changed regions. It stays local and deterministic, so the push is never held for
a model or a cluster.
2. **Asynchronous, on the plane**: the hook or the CI job calls the CLI in remote mode with a
*delta Run* whose candidates are the functions the push changed; the Run is the per-case chain
of [The plane on ax](plane.md#per-case) (`facts`, `verify` a and b, `agree`, `verify` c on the
disputed only, `final`). The caller either waits, with its own deadline, for the CLI to print
the status and attribution table, or returns at once and a later `ousast plane status <run>`
reports from the store; a commit status check or a review comment would be built on that. The
worker pool is sized to the push rate or autoscaled on `at_capacity` workers. An unattended
in-cluster reconciler that accepts submissions without a CLI session is the later option named
above.
1. **Blocking, inside the hook's deadline**: what runs today, the quick rules and the delta engine on
the changed regions. It stays local and deterministic. So the push never waits for a model or a
cluster.
2. **Asynchronous, on the plane**: the hook or the CI job calls the CLI in remote mode with a *delta
Run*.
- The Run's candidates are the functions the push changed.
- The Run is the per-case chain of [The plane on ax](plane.md#per-case): `facts`, `verify` a and b,
`agree`, `verify` c on the disputed only, `final`.
- The caller can wait, with its own deadline, for the CLI to print the status and attribution table.
- Or the caller returns at once, and a later `ousast plane status <run>` reports from the store.
- A commit status check or a review comment would be built on that.
- The worker pool is sized to the push rate, or autoscaled on `at_capacity` workers.
- An unattended in-cluster reconciler that accepts submissions without a CLI session is the later
option named above.
**Cost per push, estimated from measured rates.** Two measured per-candidate rates exist for one
model verdict: `$0.003215` (`spend.cost_per_candidate_evaluate_usd` in
`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`) and `$0.0021`
(exp-002 arm A, `overall.usd_per_candidate.A` in
`benchmarks/measurements/2026-10-02-exp-002/record.json`). A push that changes five functions needs
two verify passes per candidate: 5 x 2 x $0.0021 = $0.021, or 5 x 2 x $0.003215 = $0.032 at the
higher rate, plus a third pass on the disputed candidates only. That is where "about two cents per
push" comes from; it is arithmetic over the decision engine's measured rates, not a measurement of
a push Run, and `ousast plane status` would print the real figure per Task.
**Cost per push, estimated from measured rates.** Two measured per-candidate rates exist for one model
verdict:
- `$0.003215` (`spend.cost_per_candidate_evaluate_usd` in
`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`);
- `$0.0021` (exp-002 arm A, `overall.usd_per_candidate.A` in
`benchmarks/measurements/2026-10-02-exp-002/record.json`).
A push that changes five functions needs two verify passes per candidate. That gives 5 x 2 x $0.0021 =
$0.021, or 5 x 2 x $0.003215 = $0.032 at the higher rate. A third pass runs on the disputed candidates
only. That is where "about two cents per push" comes from. It is arithmetic over the decision engine's
measured rates, not a measurement of a push Run. `ousast plane status` would print the real figure per
Task.
**What is not built** (each is a named gap, not a configuration):
- The delta-Run command. `ousast plane scan <repo> --base <commit> --head <commit>` **does not
exist**; `ousast plane` has `run`, `status`, `doctor`, `remember`, `memory-normalise`,
exist**. `ousast plane` has `run`, `status`, `doctor`, `remember`, `memory-normalise`,
`workspaces`, `harvest` and `alerts-engine` (`src/openultrasast/cli.py`).
- A candidates task for an arbitrary repository. `repo-facts` takes its candidates from a
population's validation set; a step that derives them from a diff is not written.
population's validation set. No step derives them from a diff yet.
- Workspace generation from a URL and a commit outside a population. `plane/generate.py` is
population-driven (`read_cases` reads a validation set and a population TOML), so a Run for a
population-driven: `read_cases` reads a validation set and a population TOML. So a Run for a
repository that is not in a population cannot be generated today.
- The remote execution mode itself: the `local`/`remote` switch, configurable cluster context,
ax-server and router addresses (the `kind-` prefix is hard-coded, B.4), presigned delivery from
the runner to the store, and the CLI reading completion and `summary.json` from the store
instead of its receiver thread.
- The remote execution mode itself:
- the `local`/`remote` switch;
- a configurable cluster context and ax-server and router addresses (the `kind-` prefix is
hard-coded, B.4);
- presigned delivery from the runner to the store;
- the CLI reading completion and `summary.json` from the store instead of its receiver thread.
- The engine task and its WorkerPool, the GHCR build, the production pool and autoscaling
manifests, and the k3s runbook and checklist.
- The check or comment that carries the asynchronous verdict back to the push.
@@ -433,8 +571,7 @@ a push Run, and `ousast plane status` would print the real figure per Task.
### Sizing
The worker is the unit. Every Task's `limits` fit the 1 CPU / 1.5 GiB worker of
`ops/ax/workerpool.yaml.tmpl`; the requests are what the Task declares, the worker is what it
occupies.
`ops/ax/workerpool.yaml.tmpl`. The requests are what the Task declares. The worker is what it occupies.
| Task (`plane/tasks/*.yaml`) | Requests | Limits | Per push |
| --- | --- | --- | --- |
@@ -446,17 +583,18 @@ occupies.
| `roles` | 250m / 384Mi | 1 CPU / 1Gi | not in a push Run (harvest Runs only) |
| `engine` (planned) | from a measured run | above 3 GiB (the host run uses `--memory 3g`) | its own pool, not the default one |
- **Actors per push**: 4 (`facts`, `va`, `vb`, `agree`), 6 with a tie-break (`vc`, `final`);
- **Actors per push**: 4 (`facts`, `va`, `vb`, `agree`), or 6 with a tie-break (`vc`, `final`).
`features` and `remember` add two more when the verdict is to be kept.
- **Critical path**: `facts` then `va` and `vb` side by side, then `agree`, then (sometimes) `vc`
and `final`: three to five Task boots in series, each a golden-snapshot restore plus the model
calls of the verify passes.
- **One actor per worker**: `agree` asks for 256Mi and still holds a 1.5 GiB worker while it runs,
because the worker's capacity, not the Task's request, is the unit. A pool of `W` workers runs
at most `W/2` pushes through their verify phase at once; on this host (`W = 2`) that is one
push at a time, and every further push waits (the reconciler's `--workers` bound, or Substrate's
request parking if the pool is saturated). Autoscaling the pool on `at_capacity` workers is the
upstream answer and is **planned** here.
- **Critical path**: `facts`, then `va` and `vb` side by side, then `agree`, then sometimes `vc`
and `final`. That is three to five Task boots in series. Each boot is a golden-snapshot restore plus
the model calls of the verify passes.
- **One actor per worker**: `agree` asks for 256Mi but still holds a 1.5 GiB worker while it runs. The
worker's capacity, not the Task's request, is the unit.
- A pool of `W` workers runs at most `W/2` pushes through their verify phase at once.
- On this host (`W = 2`) that is one push at a time.
- Every further push waits, on the reconciler's `--workers` bound or on Substrate's request parking
if the pool is saturated.
- Autoscaling the pool on `at_capacity` workers is the upstream answer. It is **planned** here.
## C. The manifests under `plane/`
@@ -472,6 +610,12 @@ occupies.
| `ops/ax/smoke-task.yaml.tmpl`, `smoke-run.yaml.tmpl` | yes | rendered with `RUNNER_IMAGE` from your pin file |
| `ops/ax/up.sh`, `down.sh` | no | kind-specific; they do not apply to another cluster |
The Run and Task contract itself (`plane/runner.py` as PID 1 of the runner image, `AX_TASK_YAML`
and `AX_WORKSPACES_YAML`, `/healthz` and `/readyz`, the start request, the tar delivery as
completion) does not depend on where the cluster is: [The plane on ax](plane.md).
The Run and Task contract does not depend on where the cluster is. That contract covers:
- `plane/runner.py` as PID 1 of the runner image;
- `AX_TASK_YAML` and `AX_WORKSPACES_YAML`;
- `/healthz` and `/readyz`;
- the start request;
- the tar delivery as completion.
See [The plane on ax](plane.md).
File diff suppressed because it is too large Load Diff
+75 -52
View File
@@ -1,7 +1,8 @@
# Evaluation
What has been measured on code the tool was not tuned on, and how. State as of 2026-10-02.
Every number below is quoted from a committed record; the path is given with it.
This page shows what has been measured on code the tool was not tuned on, and how. State as of
2026-10-02. Every number below is quoted from a committed record, and the record's path is
given with it.
## Instruments
@@ -11,32 +12,47 @@ Every number below is quoted from a committed record; the path is given with it.
| Pair corpus | fire on the vulnerable function, stay silent on its fix | `ousast pairs`, `benchmarks/pairs/` |
| Independent populations | real repositories at vulnerable, fixed and benign pins, scored under a pre-registered protocol | `benchmarks/independent/` |
The detection gate and the local pairs are CI gates. They catch regressions; they do not
Recall means the share of known bugs found. FP means false positives.
The detection gate and the local pairs are CI gates. They catch regressions. They do not
estimate recall on real code, because the rules were written against them.
## Pair corpora
`benchmarks/pairs/catalog.toml` and the slice directories (`local`, `github`, `sast`, `vfc`,
`vibe-py`, `vfc-js`, `agent-vfc`, `owasp`) are runnable with `ousast pairs --slice <name>`; each
pair carries a provenance profile and a train/holdout split (`--profile`, `--split`). Pointer
pairs are not vendored: `--pointers` fetches them into `~/.cache/openultrasast/pairs/`.
`benchmarks/pairs/README.md` documents the schema and the review tiers.
A pair is a vulnerable function and its fixed version. You can run the catalog
`benchmarks/pairs/catalog.toml` and the slice directories with `ousast pairs --slice <name>`.
The slices are `local`, `github`, `sast`, `vfc`, `vibe-py`, `vfc-js`, `agent-vfc` and `owasp`.
- Each pair carries a provenance profile and a train/holdout split (`--profile`, `--split`).
- Pointer pairs are not vendored. `--pointers` fetches them into `~/.cache/openultrasast/pairs/`.
- `benchmarks/pairs/README.md` documents the schema and the review tiers.
**`advisory-fixes`** (`benchmarks/pairs/advisory-fixes/`, 2026-10-01) holds 97 pointer pairs,
one per repository: a public CVE/GHSA fix commit whose advisory-named function is declared on
both sides and changed by the fix. It was assembled to give the decision engine real negatives
for families the corpus had few of (access control, deserialization, output encoding, path,
untrusted destination). It is a label source of the decision engine
(`src/openultrasast/learn/sources.toml`), not an `ousast pairs` slice; the research log and
one per repository. Each is a public CVE/GHSA fix commit. The function the advisory names is
declared on both sides and changed by the fix.
It was assembled to give the decision engine real negatives (fixed, non-vulnerable code). It
targets families the corpus had few of: access control, deserialization, output encoding,
path, untrusted destination. It is a label source of the decision engine
(`src/openultrasast/learn/sources.toml`), not an `ousast pairs` slice. The research log and
every rejection are in `research-2026-10-01.md`.
## Independent populations and the M4 gates
The pre-push safety net's qualification milestone (M4) requires, together: at least 95%
actionable precision, at least 90% supported recall, at least 95% supported-check completion,
the runtime gates, and zero fixed-side or benign false alerts. A population is frozen before
any of its cases is scanned, scored once under a pre-registered protocol, and then **spent**:
it may inform changes, but it can no longer qualify anything.
M4 is the qualification milestone of the pre-push safety net. It requires all of these
together:
- at least 95% actionable precision;
- at least 90% supported recall;
- at least 95% supported-check completion;
- the runtime gates;
- zero fixed-side or benign false alerts.
A population is a set of real repositories. Its lifecycle has three steps:
1. It is frozen before any of its cases is scanned.
2. It is scored once under a pre-registered protocol.
3. It is then **spent**: it may inform changes, but it can no longer qualify anything.
| Population | Status | Result | Record |
| --- | --- | --- | --- |
@@ -45,9 +61,9 @@ it may inform changes, but it can no longer qualify anything.
| v2, model-driven pipeline | exploratory (v2 already spent) | stopped after 7 of 17 cases when the account emptied: recall 4/7; precision not adjudicated; no gate measurable | `results-v2-model-pipeline.json` |
| v3 (PHP, 15 cases; one SSRF slot left empty by the selection rule) | frozen, untouched | none: it is the one-time final check, run once under `protocol-v3.md` after a prediction is committed | `population-v3-php.toml`, `freeze-v3-php.json` |
v3 is reserved: development and analysis work never reads its cases, and
`tests/test_independent_population.py` fails when anything in the development tree (a recipe,
catalog, manifest or measurement) names one of its repositories.
v3 is reserved. Development and analysis work never reads its cases.
`tests/test_independent_population.py` fails when anything in the development tree names one
of its repositories. That covers a recipe, catalog, manifest or measurement.
```mermaid
stateDiagram-v2
@@ -66,9 +82,10 @@ stateDiagram-v2
## Folds: how learned decisions are kept out of their own training data
The decision engine is evaluated on repositories it was not shown (`src/openultrasast/learn/folds.py`).
Every split is by repository group (the label builder's normalised `owner/name`, merged across
URLs, forks and shared advisories), across all corpora, seeded and deterministic.
The decision engine is evaluated on repositories it was not shown
(`src/openultrasast/learn/folds.py`). Every split is by repository group. A group is the label
builder's normalised `owner/name`, merged across URLs, forks and shared advisories. Splits
cover all corpora and are seeded and deterministic.
```mermaid
flowchart TD
@@ -81,44 +98,50 @@ flowchart TD
rest --> lofo["Leave-one-framework-out: frameworks with at least 10 groups"]
```
`assert_disjoint` runs before a compile: the compile split and the evaluation folds share no
group. A framework with fewer than 10 groups gets `insufficient data` instead of a fold. As of
2026-10-02 the outer folds have been run for the six evaluable families; leave-one-source-out
and leave-one-framework-out have not
`assert_disjoint` runs before a compile. It checks that the compile split and the evaluation
folds share no group. A framework with fewer than 10 groups gets `insufficient data` instead of
a fold. As of 2026-10-02, the outer folds have been run for the six evaluable families.
Leave-one-source-out and leave-one-framework-out have not
([decision-engine.md](decision-engine.md#second-slice-six-families-2026-10-02)).
## The plane increment
The first increment of the agentic plane ran the model-driven pipeline's validation set (46
candidates in 15 cases of v2, 20 declared vulnerable sites) on google/ax and compared it with
the script pipeline on the same cases, candidates, pins and recorded triage. The pre-registered
gate: at least 16 of 20 declared sites agreed, at a cost per candidate under $0.022.
The first increment of the agentic plane ran on google/ax. It used the model-driven pipeline's
validation set: 46 candidates in 15 cases of v2, with 20 declared vulnerable sites. It was
compared with the script pipeline on the same cases, candidates, pins and recorded triage.
The pre-registered gate: at least 16 of 20 declared sites agreed, at a cost per candidate under
$0.022.
- **Increment 1** (2026-09-29, `benchmarks/independent/plane-increment-1.json`): 14 of 20
declared sites agreed against the reference's 16; $0.0194 per candidate with recorded triage.
**NOT MET** on recall.
*Correction (2026-09-30, in the record):* the first version read agree's
`declared_sites_matched` (coverage by candidates, 19) as "found in either pass" and concluded
detection was unchanged. Recounted from the agreed rows, the plane's passes flagged 17 of 20
declared sites in at least one pass against the reference's 19.
- **Increment 2** (2026-09-30, `benchmarks/independent/plane-increment-2.json`): after the design
revisit, a tie-break pass c on the 10 disputed candidates with 2-of-3 agreement. 16 of 20
declared sites agreed (reference 16) at $0.0205 per candidate over 46 ($0.02195 over the 43
after triage, against the reference's $0.022). **MET**, with two caveats the record carries:
the tie-break was chosen after seeing the first measurement on the same validation set, so
this compares plane with script and qualifies nothing on an independent population; and the
plane's passes still flag fewer declared sites than the reference (17 vs 19). Precision is
not part of this gate and remains the open problem.
declared sites agreed, against the reference's 16. Cost was $0.0194 per candidate with
recorded triage. **NOT MET** on recall.
*Correction (2026-09-30, in the record):* the first version misread agree's
`declared_sites_matched`. That field is coverage by candidates (19), but it was read as
"found in either pass". So the first version concluded detection was unchanged. Recounted
from the agreed rows, the plane's passes flagged 17 of 20 declared sites in at least one
pass. The reference flagged 19.
- **Increment 2** (2026-09-30, `benchmarks/independent/plane-increment-2.json`): came after the
design revisit. It added a tie-break pass c on the 10 disputed candidates, with 2-of-3
agreement. 16 of 20 declared sites agreed (reference 16). Cost was $0.0205 per candidate over
46, and $0.02195 over the 43 after triage (reference: $0.022). **MET**, with two
caveats the record carries:
1. The tie-break was chosen after seeing the first measurement on the same validation set.
So this compares plane with script and qualifies nothing on an independent population.
2. The plane's passes still flag fewer declared sites than the reference (17 vs 19).
Precision is not part of this gate and remains the open problem.
## The decision engine
Two slices have been measured out of repository (injection, 2026-10-01; six families,
2026-10-02). The engine is not adopted. The state against the gates, in one place with the plane
increment and the populations above, is on [Where we stand](where-we-stand.md#1-detection-capability-and-goals-no-go-measured);
the per-family tables are on [decision-engine.md](decision-engine.md).
Two slices have been measured out of repository: injection (2026-10-01) and six families
(2026-10-02). The engine is not adopted. [Where we
stand](where-we-stand.md#1-detection-capability-and-goals-no-go-measured) shows the state
against the gates in one place, with the plane increment and the populations above. The
per-family tables are on [decision-engine.md](decision-engine.md).
## Removal of the earlier agentic extra
The removal of the earlier agentic extra (2026-09-30) was checked for byte-identical
deterministic outputs (pair replay, quick benchmark, standard scan, `improve --dry-run`):
The earlier agentic extra was removed on 2026-09-30. The removal was checked for byte-identical
deterministic outputs: pair replay, quick benchmark, standard scan, `improve --dry-run`. Records:
`benchmarks/measurements/2026-09-30-harnessx-removal-baseline/` and `-equality/`.
+28 -17
View File
@@ -1,7 +1,7 @@
# End-to-end examples
Each example is self-contained. Artifacts land under
`<target>/.openultrasast/runs/<scan-id>/` and `.openultrasast/` is gitignored.
Each example stands on its own. Artifacts land under `<target>/.openultrasast/runs/<scan-id>/`.
`.openultrasast/` is gitignored.
## 1. Quick scan a local repo (deterministic, no keys)
@@ -21,7 +21,7 @@ ousast scan . --mode standard # Joern model layer, container network off:
ousast-with-judge scan . --mode standard # network on, so the model is asked (needs DEEPSEEK_API_KEY)
```
Without a key the graph still decides: its entailed findings are reported, and the LLM's residual
Without a key, the graph still decides. Its entailed findings are reported. The LLM's residual
and suspicion questions go unasked.
## 3. CI gate
@@ -52,10 +52,11 @@ uv run ousast pairs --split holdout --profile human # filter by declared split
uv run python -m openultrasast.pair_gate # CI: local pairs must all pass
```
Each case is two isolated trees (vuln file vs patched file, same relative path).
`pair_correct` means the harness fired on the vuln side and stayed silent on the
fix. Misses and fix-side leaks become `miss`/`fp` signals for `ousast improve`.
The catalog and public dataset pointers live in `benchmarks/pairs/`.
Each case is two isolated trees: the vuln file and the patched file, at the same relative path.
- `pair_correct` means the harness fired on the vuln side and stayed silent on the fix.
- Misses become `miss` signals and fix-side leaks become `fp` signals for `ousast improve`.
- The catalog and public dataset pointers live in `benchmarks/pairs/`.
## 6. Self-improve the ruleset from benchmark feedback
@@ -68,10 +69,11 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml
## 7. Running the agentic plane
Model work over whole repositories runs as a Run manifest on ax, one isolated actor
per task (bring-up: [ops/ax/README.md](ops/ax/README.md)). The provider key is
read from the environment (or `.env`, which never overrides an exported variable) by
`ousast` and sent only in each task's start request.
Model work over whole repositories runs as a Run manifest on ax. Each task gets one isolated
actor. Bring-up is in [ops/ax/README.md](ops/ax/README.md).
`ousast` reads the provider key from the environment, or from `.env`. `.env` never overrides an
exported variable. The key is sent only in each task's start request.
```bash
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
@@ -83,16 +85,25 @@ uv run ousast plane remember validation-46 # re-ingest the run's m
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
```
Each task in the Run carries its own Model and `budget: {usd, calls}`; a task that
reaches its ceiling stops as `unfinished` and resumes on a rerun with a larger
budget. `ousast plane status` prints the token attribution table: calls, prompt,
cache-hit and output tokens and USD per task.
Each task in the Run carries its own Model and `budget: {usd, calls}`.
- A task that reaches its ceiling stops as `unfinished`. It resumes on a rerun with a larger
budget.
- `ousast plane status` prints the token attribution table. It shows calls, prompt, cache-hit and
output tokens, and USD per task.
## 8. Drive it from an MCP client (OpenCode / IDE)
MCP (Model Context Protocol) lets an editor or agent call the tool's functions.
```jsonc
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
```
Then: `openultrasast.scan {path}` → `run_dir` → `openultrasast.findings {run_dir}` →
`openultrasast.explain {run_dir, finding_id}`. See the README section "Agent integrations".
Then call the tools in this order:
1. `openultrasast.scan {path}` returns a `run_dir`.
2. `openultrasast.findings {run_dir}` lists the findings.
3. `openultrasast.explain {run_dir, finding_id}` explains one.
See the README section "Agent integrations".
+24 -18
View File
@@ -1,13 +1,13 @@
# OpenUltraSAST
OpenUltraSAST is an experimental static security analyser meant to work as a safety net on
repositories it has never seen: it scans a checkout, or the commits a `git push` would publish,
and separates what it can prove from what it merely suspects. A scan that could not look at
something records a degradation instead of reporting a clean result.
OpenUltraSAST is an experimental static security analyser. It is meant to be a safety net on
repositories it has never seen. It scans a checkout, or the commits a `git push` would publish. It
keeps what it can prove apart from what it only suspects. When a scan could not look at
something, it records a degradation. It does not report a clean result.
**State as of 2026-10-02 (v2.0.0):** the pre-push safety net is **NO-GO** for rollout.
No hook capability is enabled, and no independent population has passed the qualification gates
(see [Evaluation](evaluation.md)).
No hook capability is enabled. No independent population (a case set frozen before any scan) has
passed the qualification gates (see [Evaluation](evaluation.md)).
## Three layers
@@ -17,6 +17,9 @@ No hook capability is enabled, and no independent population has passed the qual
| **The agentic plane on ax** | Model-driven work over whole repositories, run as a DAG of tasks on google/ax, one isolated actor per task. What the runs learn is kept in a memory store. Maintainer surface: `ousast plane`. | [The plane on ax](plane.md), [Memory](memory.md) |
| **The decision engine** | A learned classifier that weighs every instrument's signals against similar labelled cases from memory. **In development, not adopted**: no scan, pre-push check or report uses it. Maintainer surface: `ousast learn`. | [Memory and detection](memory-and-detection.md), [Decision engine](decision-engine.md) |
A code property graph is a graph of the program's syntax, control flow and data flow. Joern builds
it.
## How they connect
```mermaid
@@ -33,31 +36,34 @@ flowchart LR
engine -. "intended, not adopted: BLOCK / ADVISORY" .-> report
```
- The **scanners** produce findings and, inside plane Runs, the `alerts` that become memory rows.
- The **plane** runs model work (verify passes, roles) and model-free bookkeeping (facts, agreement,
feature records) per case; its `remember` task turns a case's artifacts into rows of the
[memory store](memory.md).
- Memory feeds two consumers. `ousast improve --memory` turns stored rows into rule-status
proposals that pass through the same validator and gate as every other edit. The decision engine
retrieves labelled examples from the same store.
- The **scanners** produce findings. Inside plane Runs, they also produce the `alerts` that become
memory rows.
- The **plane** runs model work per case: verify passes and roles. It also runs model-free
bookkeeping per case: facts, agreement and feature records. Its `remember` task turns a case's
artifacts into rows of the [memory store](memory.md).
- Memory feeds two consumers:
1. `ousast improve --memory` turns stored rows into rule-status proposals. They pass through the
same validator and gate as every other edit.
2. The decision engine retrieves labelled examples from the same store.
- The decision engine's output (dotted line) is the intended path. Nothing user-facing reads it
today.
## Where to go next
- The current state in one page, every figure with its record: capability against the M4 gates, the
step-by-step approach, ax on the server, token economics before and after, and the open items:
[Where we stand](where-we-stand.md).
- The current state in one page, every figure with its record: [Where we stand](where-we-stand.md).
It covers capability against the M4 gates, the step-by-step approach, ax on the server, token
economics before and after, and the open items.
- Run a scan: [Examples](examples.md), and the commands in [Scanning](scanning.md).
- Understand a finding's evidence and score: [Architecture](architecture.md).
- The program analysis under a scan, what each representation can see and what it misses:
[Detection techniques](detection-techniques.md).
- Operate the plane and its store: [ax on this host](ops/ax/README.md) and
[RustFS setup](rustfs.md).
- Where the plane runs today and what a separate Kubernetes cluster would take (planned, not
- Where the plane runs today, and what a separate Kubernetes cluster would take (planned, not
done): [Deployment](deployment.md).
- Where the model spend goes, per Task at runtime and in the maintainers' own coding sessions,
and what is reused instead of paid again: [Token ergonomics](token-ergonomics.md).
- Trust boundaries and hardening: [Threat model](threat-model.md).
Every number in these pages is quoted from a committed record whose path is given next to it.
Every number in these pages is quoted from a committed record. The record's path is given next to
the number.
+139 -85
View File
@@ -1,11 +1,12 @@
# Memory and detection
How what the system remembers is meant to turn into better decisions on code it has not seen.
This is the design of the learned decision engine (`src/openultrasast/learn/`), which is **in
development and not adopted**: no scan, pre-push check or report uses it today. Every user-facing
decision is still the deterministic one described in [Scanning](scanning.md) and
This page shows how the system's memory is meant to improve decisions on code it has not seen. It
describes the design of the learned decision engine (`src/openultrasast/learn/`).
The engine is **in development and not adopted**. No scan, pre-push check or report uses it today.
Every user-facing decision is still the deterministic one described in [Scanning](scanning.md) and
[Architecture](architecture.md). The engine's commands and status are on
[Decision engine](decision-engine.md); the store itself is on [Memory](memory.md).
[Decision engine](decision-engine.md). The store itself is on [Memory](memory.md).
## From instruments to a decision
@@ -25,89 +26,128 @@ flowchart TD
op -- "otherwise" --> adv["ADVISORY"]
```
The intended place for these decisions is changed code: `learn/features.py` builds feature records
for a whole-repository scan or for a base..head delta (`build_for_scan` with `Delta`), the same
delta `ousast pre-push` checks. Integration into scan and pre-push is still ahead.
These decisions are meant for changed code. `learn/features.py` builds feature records in two ways:
- for a whole-repository scan;
- for a base..head delta (`build_for_scan` with `Delta`). This is the same delta `ousast pre-push` checks.
Integration into scan and pre-push is still ahead.
## 1. Instruments produce signals, not verdicts
Each instrument contributes one block to a candidate's feature record (`learn/schema.py`,
`INSTRUMENTS`): `language`, `quick` (quick rules), `engine` (the Joern model layer), `facts` (repo
facts, such as callers and function span), `source`, `entry_points`, `roles` (model source, sink
and sanitizer roles), `delta`, `verify` and `agree` (the plane's verify passes and their
agreement), and `model_sinks` (the archived stage-1 classifier, `benchmarks/archive/model_sinks.py`, whose
logic now lives in `plane/tasks/roles.py`). A candidate is `(path, function, family)` of one repository and pin
where at least one instrument emitted a signal of that family (`learn/features.py`).
`INSTRUMENTS`). The blocks are:
An instrument that did not run, had no coverage or failed records that state (`none`, `failed`,
`not_applicable`), never a zero, so "silent" and "did not look" stay distinguishable. None of the
instruments decides alone.
- `language`;
- `quick` (quick rules);
- `engine` (the Joern model layer);
- `facts` (repo facts, such as callers and function span);
- `source`;
- `entry_points`;
- `roles` (model source, sink and sanitizer roles);
- `delta`;
- `verify` and `agree` (the plane's verify passes and their agreement);
- `model_sinks` (the archived stage-1 classifier, `benchmarks/archive/model_sinks.py`). Its logic now
lives in `plane/tasks/roles.py`.
**Why profile v1 withholds the engine and function length.** A leak audit
(`ousast learn audit-leaks`, `benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`)
found that both separate the vulnerable and fixed sides of a pair by construction: a fix removes
a sink, and a guard lengthens the function. That holds on the pair corpus, not on unseen code, so
`INPUT_PROFILES["v1"]` drops the `engine` block and `facts.function_lines` from prompts and from
the retrieval distance.
A candidate is `(path, function, family)` of one repository and pin. At least one instrument must have
emitted a signal of that family there (`learn/features.py`).
An instrument may not run, may have no coverage, or may fail. It then records that state (`none`,
`failed`, `not_applicable`), never a zero. So "silent" and "did not look" stay distinguishable. No
instrument decides alone.
**Why profile v1 withholds the engine and function length.** A leak audit found that both separate the
vulnerable and fixed sides of a pair by construction (`ousast learn audit-leaks`,
`benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`):
- a fix removes a sink;
- a guard lengthens the function.
That holds on the pair corpus, not on unseen code. So `INPUT_PROFILES["v1"]` drops the `engine` block
and `facts.function_lines` from prompts and from the retrieval distance.
## 2. Labels come from ground truth only
`ousast learn labels` (`learn/labels.py`) reads only the sources on a fail-closed allow-list
(`learn/sources.toml`): `population-v1`, `population-v2`, `dev-php` (reviewed recipe sites),
`pairs`, `advisory-fixes`, `adjudications` and `assumed_benign`. A file the list does not name is
refused before it is opened; the reserved population v3 is excluded unread.
(`learn/sources.toml`). The sources are `population-v1`, `population-v2`, `dev-php` (reviewed recipe
sites), `pairs`, `advisory-fixes`, `adjudications` and `assumed_benign`. A file the list does not name
is refused before it is opened. The reserved population v3 is excluded unread.
- **Positives** are declared sites and the vulnerable side of a fix; **negatives** are the same
function on the fixed side where the fix changed it, and recorded `False`/`FP` adjudications.
A vulnerable-pin candidate matching no site is unlabelled, never negative.
- **Assumed benign** (maintainer decision 2026-09-30): ordinary non-security commits of the training
repositories, at least 90 days from any fix, with no later security commit touching the same
files. They are their own source with weight 0.5, conditional on a signal at the tip and none at
the base, and reported separately, never merged with verified negatives.
- **Plane verdicts are features, not labels.** "Agreed", "rejected" or "disputed" is the output of
the verify instrument; it enters the record as the `verify` and `agree` blocks. Using it as a
label would teach the engine to reproduce the plane, mistakes included, so "rejected by the
plane" is never a negative (`learn/labels.py`: "A plane verdict ... is never a label").
- **Positives** are declared sites and the vulnerable side of a fix.
- **Negatives** are two kinds:
- the same function on the fixed side, where the fix changed it;
- recorded `False`/`FP` adjudications.
- A vulnerable-pin candidate matching no site is unlabelled, never negative.
- **Assumed benign** (maintainer decision 2026-09-30) covers ordinary non-security commits of the
training repositories. Each is at least 90 days from any fix. No later security commit touches the
same files.
- They are their own source with weight 0.5, conditional on a signal at the tip and none at the base.
- They are reported separately, never merged with verified negatives.
- **Plane verdicts are features, not labels.** "Agreed", "rejected" or "disputed" is the output of the
verify instrument. It enters the record as the `verify` and `agree` blocks. Using it as a label would
teach the engine to reproduce the plane, mistakes included. So "rejected by the plane" is never a
negative (`learn/labels.py`: "A plane verdict ... is never a label").
`ousast learn memory build` joins each label with the candidate's feature record and an excerpt of
its function and stores one `example` row (no rationale) plus the excerpt blob.
its function. It stores one `example` row (no rationale) plus the excerpt blob.
## 3. Retrieval of similar labelled cases
`learn/retrieve.py`, no model:
`learn/retrieve.py` uses no model. It works in three steps:
1. **Signal profile.** A Gower distance over the allow-listed features of the active profile,
per instrument block, with missing values explicit (a different state counts as distance 1).
The nearest 40 examples of the candidate's family and the nearest 40 of other families go on.
2. **Code embeddings.** A cosine re-rank of the excerpts' embeddings (OpenAI
`text-embedding-3-small` through OpenRouter, 1,536 dimensions, cached by excerpt hash):
`r = 0.5 (1 - D) + 0.5 cos`. Without the candidate's vector the retrieval is signals only.
3. **Balance.** Six examples: at most half per label, at most 2 per repository group, at most 2
contrast examples from other families.
1. **Signal profile.** It computes a Gower distance (a distance for mixed-type features) over the
allow-listed features of the active profile, per instrument block. Missing values are explicit: a
different state counts as distance 1. The nearest 40 examples of the candidate's family go on. So do
the nearest 40 of other families.
2. **Code embeddings.** It re-ranks by cosine over the excerpts' embeddings. The embeddings come from
OpenAI `text-embedding-3-small` through OpenRouter (1,536 dimensions, cached by excerpt hash). The
formula is `r = 0.5 (1 - D) + 0.5 cos`. Without the candidate's vector, the retrieval uses signals only.
3. **Balance.** It picks six examples:
- at most half per label;
- at most 2 per repository group;
- at most 2 contrast examples from other families.
**The evaluation boundary.** `eligible()` is the only way an example enters a prompt, for retrieved
examples and compiled demonstrations alike: never the candidate's own repository group, never a
group the fold evaluates, never the held-out source or framework, and (outside deployment) never
a near-duplicate of the candidate (cosine >= 0.98). The program repeats the check on the ids it
actually rendered before any call.
**The evaluation boundary.** `eligible()` is the only way an example enters a prompt. This holds for
retrieved examples and compiled demonstrations alike. An eligible example is never:
- from the candidate's own repository group;
- from a group the fold evaluates;
- from the held-out source or framework;
- a near-duplicate of the candidate (cosine >= 0.98), outside deployment.
The program repeats the check on the ids it actually rendered, before any call.
## 4. The AI classifier, compiled
`learn/program.py` and `learn/compile.py`, DSPy-style and in-house. The prompt is the instruction
plus the compiled demonstrations (a prefix identical for every candidate, so the provider's prefix
cache hits), then the retrieved examples, then the candidate's excerpt, signals, roles and family.
The model (`deepseek-flash`) answers `vulnerable`, `not_vulnerable` or `unsure` with a confidence
and a rationale that must cite a line of the code. It is sampled `k` times; the score `s` is the
mean over samples, and a majority `unsure` can never BLOCK. Compilation bootstraps demonstrations
and searches instructions on a compile split of repository groups, scored by a pre-registered
balanced Brier score (`learn/compile.toml`); the program is stored as `programs/<id>.json`.
The classifier lives in `learn/program.py` and `learn/compile.py`. It is DSPy-style (DSPy is a
framework that compiles prompts from examples) and built in-house.
The prompt has this order:
1. the instruction plus the compiled demonstrations. This prefix is identical for every candidate, so
the provider's prefix cache hits;
2. the retrieved examples;
3. the candidate's excerpt, signals, roles and family.
The model (`deepseek-flash`) answers `vulnerable`, `not_vulnerable` or `unsure`. It adds a confidence
and a rationale. The rationale must cite a line of the code. The model is sampled `k` times. The score
`s` is the mean over samples. A majority `unsure` can never BLOCK.
Compilation bootstraps demonstrations and searches instructions on a compile split of repository groups.
A pre-registered balanced Brier score scores it (`learn/compile.toml`). The program is stored as
`programs/<id>.json`.
## 5. Calibration and the two operating points
`learn/calibrate.py`: Platt scaling of `s`, cross-fitted so that no map is checked on the fold it
was fitted on. Calibration **holds** when the slope interval contains 1, the intercept interval
contains 0 and ECE <= 0.05. **BLOCK** is offered only when calibration holds and a fold reaches the
required precision; otherwise the engine offers **ADVISORY** only.
`learn/calibrate.py` applies Platt scaling (a logistic map from score to probability) to `s`. It is
cross-fitted, so no map is checked on the fold it was fitted on.
Calibration **holds** when the slope interval contains 1, the intercept interval contains 0 and ECE
(expected calibration error) <= 0.05.
**BLOCK** is offered only when calibration holds and a fold reaches the required precision. Otherwise
the engine offers **ADVISORY** only.
## The loop: memory improves decisions without retraining
@@ -125,25 +165,36 @@ flowchart LR
gate -- "accepted round" --> ledger["Rule status in the ledger"]
```
- **Decisions.** The compiled part of the program is its instruction and demonstrations; the
examples are retrieved from memory for each candidate. A new `example` row is therefore eligible
for retrieval in the next decision without recompiling the program, subject to the same
evaluation boundary. `ousast learn curve` measures the metric over memory size on a fixed
candidate subset.
- **Rules.** `ousast improve --memory` reads the run rows (alerts, verdicts, coverage) and proposes
rule-status edits only (shadow a rule that is repeatedly false across repositories; re-enable a
shadow rule that keeps hitting agreed declared sites). Rows from holdout pairs, the gated
manifest's own cases and any `--qualify-population` are dropped first, and every proposal goes
through the same validator and gate as any other edit (`improve/memory.py`).
- **Decisions.**
- The compiled part of the program is its instruction and demonstrations.
- The examples are retrieved from memory for each candidate.
- So a new `example` row is eligible for retrieval in the next decision without recompiling. The same
evaluation boundary applies.
- `ousast learn curve` measures the metric over memory size on a fixed candidate subset.
- **Rules.** `ousast improve --memory` reads the run rows (alerts, verdicts, coverage). It proposes
rule-status edits only:
- shadow a rule that is repeatedly false across repositories;
- re-enable a shadow rule that keeps hitting agreed declared sites.
Some rows are dropped first: rows from holdout pairs, the gated manifest's own cases and any
`--qualify-population`. Every proposal goes through the same validator and gate as any other edit
(`improve/memory.py`).
## What the first slice measured
Record: `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json` (counts
only). Family **injection**; memory of 724 static examples from all families; input profile v1;
framework priors off; out-of-repository evaluation (compile split never evaluated, outer grouped
5-fold, calibration cross-fitted, operating points nested); 95% repository-cluster bootstrap
intervals. The slice's spend ($0.003215 per candidate at evaluation, k = 5, and its replay at
$0) is on [Token ergonomics](token-ergonomics.md#measured-costs).
only). The setup:
- family **injection**;
- memory of 724 static examples from all families;
- input profile v1;
- framework priors off;
- out-of-repository evaluation: compile split never evaluated, outer grouped 5-fold, calibration
cross-fitted, operating points nested;
- 95% repository-cluster bootstrap intervals.
The slice's spend is on [Token ergonomics](token-ergonomics.md#measured-costs). It was $0.003215 per
candidate at evaluation, k = 5, and its replay cost $0.
| Measure | Value | What it means |
| --- | --- | --- |
@@ -152,9 +203,12 @@ $0) is on [Token ergonomics](token-ergonomics.md#measured-costs).
| ADVISORY | recall 0.90 (0.82 - 0.97), precision 0.54 (0.44 - 0.65) | precision is at the positive share (0.53), so at this threshold ADVISORY is **not discriminating**: it flags nearly as indiscriminately as flagging everything would |
| BLOCK | not offered | calibration does not hold (ECE 0.064 > 0.05), and every outer fold reported `precision_unreachable` |
**The engine is not adopted.** A second slice (2026-10-02) evaluated the six evaluable families
the same way: pooled AUC 0.77 to 0.96 per family, within-pair AUC 0.72 to 0.93, BLOCK offered for
none, re-run agreement below 0.9 for three families
([Decision engine](decision-engine.md#second-slice-six-families-2026-10-02)). Leave-one-source-out
and leave-one-framework-out, integration into scan and pre-push, and the one-time check on
population v3 are still ahead.
**The engine is not adopted.** A second slice (2026-10-02) evaluated the six evaluable families the
same way ([Decision engine](decision-engine.md#second-slice-six-families-2026-10-02)). The results: pooled AUC 0.77 to 0.96 per family, within-pair AUC 0.72 to 0.93, BLOCK offered for
none. Re-run agreement was below 0.9 for three families.
Still ahead:
- leave-one-source-out and leave-one-framework-out;
- integration into scan and pre-push;
- the one-time check on population v3.
+50 -40
View File
@@ -1,8 +1,8 @@
# Memory
One store, keyed by repository and pin, holds what plane runs and the decision engine learned.
The code is `src/openultrasast/plane/memory.py`; the same layout serves both backends.
How the stored rows feed detection is on [Memory and detection](memory-and-detection.md).
One store holds what plane runs and the decision engine learned. It is keyed by repository and pin
(a fixed commit). The code is `src/openultrasast/plane/memory.py`. The same layout serves both backends.
[Memory and detection](memory-and-detection.md) shows how the stored rows feed detection.
## What is stored
@@ -17,7 +17,7 @@ programs/<program id>.json compiled decision-engine programs
```
**Rows.** Every row carries `id`, `kind`, `repo`, `pin`, `run`, `task`, `population`, `split`
and `image` (`ROW_FIELDS`); `id` is the sha256 of (kind, run, task, subject), so a repeated row
and `image` (`ROW_FIELDS`). `id` is the sha256 of (kind, run, task, subject). So a repeated row
replaces itself. The kinds (`KINDS`) and who writes them:
| Kind | Written by | Holds |
@@ -35,12 +35,15 @@ replaces itself. The kinds (`KINDS`) and who writes them:
| `experiment_result` | `ousast learn experiment analyse --record` | the paired estimates, the looks and the adoption verdict (counts and intervals only) |
| `label`, `decision` | reserved for later decision-engine tasks | declared in `KINDS`; not written by the code on main yet |
**Blobs** (`put_blob`) are content-addressed: a 64-hex name under a fixed prefix. Excerpts are
written by `learn memory build` (`learn/examples.py`), embeddings by `learn memory embed`
(`learn/embeddings.py`, keyed by excerpt sha so an excerpt is embedded once), responses by the
program's caller (`learn/program.py`, keyed by the sha256 of model, parameters digest, messages,
temperature, sample and format, so a rerun replays at no cost), and compiled programs by
`learn compile` (`learn/compile.py`).
**Blobs** (`put_blob`) are content-addressed: each has a 64-hex name under a fixed prefix.
Content-addressed means the name is the hash of the content. Who writes each blob:
- Excerpts: `learn memory build` (`learn/examples.py`).
- Embeddings: `learn memory embed` (`learn/embeddings.py`). They are keyed by excerpt sha, so an
excerpt is embedded once.
- Responses: the program's caller (`learn/program.py`). The key is the sha256 of model, parameters
digest, messages, temperature, sample and format. So a rerun replays at no cost.
- Compiled programs: `learn compile` (`learn/compile.py`).
## Write path
@@ -60,13 +63,14 @@ flowchart LR
put --> store[("Memory store")]
```
- `ingest` validates every row first: a malformed file fails before anything is written.
- Rows are merged into their repository + pin object by `id`. Every object is written whole and the
index last, so an interrupted ingest leaves nothing a repeat would not repair.
- `ingest` validates every row first. A malformed file fails before anything is written.
- Rows are merged by `id` into their repository + pin object.
- Every object is written whole, and the index last. So a repeat repairs anything an interrupted
ingest left behind.
- Each row object carries tags (`TAG_FIELDS`: `repo`, `pin`, `kind`, `family`, `run`,
`population`, `split`) listing the distinct values of its rows (`*` when they exceed 256
characters).
- The plane's store failures never change a run's result: an ingest can be repeated by hand
`population`, `split`). The tags list the distinct values of its rows. A tag is `*` when the
values exceed 256 characters.
- The plane's store failures never change a run's result. An ingest can be repeated by hand
(`cli.py`, `_plane_memory`).
## Read path
@@ -85,14 +89,14 @@ flowchart LR
rules --> gate{"Unchanged validator and gate"}
```
- **Seed** (`memory.seed`): a Run task whose `openultrasast.io/memory-key` matches stored facts gets
them and is marked done; a changed pin, candidate set or runner image finds nothing and
recomputes. The loop's `memory-snapshot` is seeded the same way, because a sandboxed task cannot
read the store.
- **Seed** (`memory.seed`).
- A Run task gets stored facts when its `openultrasast.io/memory-key` matches them. It is then marked done.
- A changed pin, candidate set or runner image finds nothing and recomputes.
- The loop's `memory-snapshot` is seeded the same way. A sandboxed task cannot read the store.
- **Retrieval** (`learn/retrieve.py`): see
[Memory and detection](memory-and-detection.md#3-retrieval-of-similar-labelled-cases).
- **`improve --memory`** (`improve/memory.py`): rows from holdout pairs, from the gated manifest's
own cases and from any `--qualify-population` are dropped before any rule sees them. See
- **`improve --memory`** (`improve/memory.py`). Some rows are dropped before any rule sees them:
rows from holdout pairs, from the gated manifest's own cases, and from any `--qualify-population`. See
[Architecture](architecture.md#proposals-from-plane-memory).
## Backends: FileStore and the S3 store
@@ -109,26 +113,32 @@ flowchart LR
### Verify at startup, no fallback
The S3 store never configures its bucket. Each time an `S3Store` is opened for a bucket and
prefix (once per process), `verify_bucket()` checks, using reads plus one small probe object at
`<prefix>/_probe/select.jsonl`:
The S3 store never configures its bucket. It only checks it. An `S3Store` is opened for a bucket and
prefix once per process. Each time, `verify_bucket()` runs these checks. It uses reads plus one small
probe object at `<prefix>/_probe/select.jsonl`.
1. versioning is `Enabled` (provenance cites object versions);
2. an enabled lifecycle rule expires `<prefix>/runs/` after a number of days, as a plain prefix
filter, and no enabled rule expires the whole store;
3. S3 Select answers a probe query over JSON Lines;
4. the probe's object tags are readable.
1. Versioning is `Enabled`. Provenance cites object versions.
2. An enabled lifecycle rule expires `<prefix>/runs/` after a number of days. It uses a plain prefix
filter. No enabled rule expires the whole store.
3. S3 Select (a server-side query over JSON objects) answers a probe query over JSON Lines.
4. The probe's object tags are readable.
Anything missing is named, with the admin command that fixes it, in one `MemoryStoreError`, and
the store does not start. There is no local or fetch-and-filter fallback: a filtered read that
S3 Select cannot answer raises a `MemoryStoreError` instead of returning "no rows". The exact
messages are listed on [RustFS setup](rustfs.md#what-the-store-verifies-at-startup).
If anything is missing, the store does not start. One `MemoryStoreError` names each problem and the
admin command that fixes it.
There is no local or fetch-and-filter fallback. A filtered read that S3 Select cannot answer raises a
`MemoryStoreError`. It does not return "no rows". [RustFS setup](rustfs.md#what-the-store-verifies-at-startup)
lists the exact messages.
### Queryable fields per kind
RustFS's Select infers an object's JSON schema from its leading rows, so a `where` field missing
there fails with `EvaluatorBindingDoesNotExist` even when a later row has it, and a column that is
null in the first 1000 rows breaks Select on the whole object (measured 2026-10-02; `memory.py`,
`S3Store._select`). The store therefore writes every row with its kind's full set of queryable
fields (`QUERY_FIELDS`), using `""` where a field does not apply, and refuses a `where` on an
undeclared field. Existing rows are rewritten once with `ousast plane memory-normalise`.
RustFS's Select infers an object's JSON schema from its leading rows. This causes two failures
(measured 2026-10-02; `memory.py`, `S3Store._select`):
- A `where` field missing from the leading rows fails with `EvaluatorBindingDoesNotExist`. This
happens even when a later row has the field.
- A column that is null in the first 1000 rows breaks Select on the whole object.
So the store writes every row with its kind's full set of queryable fields (`QUERY_FIELDS`). It uses
`""` where a field does not apply. It refuses a `where` on an undeclared field. Existing rows are
rewritten once with `ousast plane memory-normalise`.
+83 -57
View File
@@ -1,23 +1,34 @@
# The plane on ax
Model-driven work over whole repositories runs on a separate agentic plane: a **Run**
(`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG of steps, and each step is
an ax **Task** (`plane/tasks/`) that binds its Workspaces (`plane/workspaces/`), at most one ax
**Model** (`plane/models/`: `deepseek-flash` for chat, `openrouter-embedding` for embeddings) and
its own `budget: {usd, calls}`. ax (google/ax over Agent Substrate, in a single-node kind cluster
on the maintainer's host) is the only executor; there is no local subprocess path. Bring-up and
the host's lessons are in [ax on this host](ops/ax/README.md); where each part runs today (the
reconciler and the receiver on the host, the tasks in gVisor on Agent Substrate) and what a
separate Kubernetes cluster would take (planned) are on [Deployment](deployment.md); this page
draws the flow.
Model-driven work over whole repositories runs on a separate agentic plane. The plane has these parts:
- A **Run** (`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG (a graph of steps
with no cycles).
- Each step is an ax **Task** (`plane/tasks/`).
- A Task binds its Workspaces (`plane/workspaces/`).
- A Task binds at most one ax **Model** (`plane/models/`). `deepseek-flash` serves chat.
`openrouter-embedding` serves embeddings.
- A Task carries its own `budget: {usd, calls}`.
ax is the only executor. It is google/ax, Google's controller that runs agent tasks, over Agent Substrate (its
sandboxed actor runtime). It runs in a single-node kind cluster (Kubernetes in Docker) on the maintainer's host.
There is no local subprocess path.
Where to read more:
- Bring-up and the host's lessons are in [ax on this host](ops/ax/README.md).
- [Deployment](deployment.md) shows where each part runs today. The reconciler and the receiver run on the host.
The tasks run in gVisor (a sandboxing kernel) on Agent Substrate. It also covers what a separate Kubernetes
cluster would take (planned).
- This page draws the flow.
Sources: `src/openultrasast/plane/reconciler.py`, `router.py`, `egress.py`, `runner.py`,
`generate.py`, `memory.py`, and `src/openultrasast/plane/tasks/`.
## One task, end to end
`ousast plane run RUN.yaml` (the reconciler) does this for every ready task; `--workers` bounds
the tasks in flight, and a rerun skips tasks already done.
`ousast plane run RUN.yaml` (the reconciler) does this for every ready task. `--workers` bounds the
tasks in flight. A rerun skips tasks already done.
```mermaid
sequenceDiagram
@@ -48,44 +59,55 @@ sequenceDiagram
What each step guarantees:
- **Credentials** (`router.py`, `runner.py`). The reconciler reads the variable the Model's
`secretKey.key` names from the operator's environment or `.env` (which never overrides an
exported variable). The key exists only in the start request; the runner keeps it in memory,
passes it to the command's environment, redacts it from echoed stderr and never writes it.
- **Egress** (`egress.py`). One EgressPolicy per actor, deny by default, hostnames only: plain
HTTP to the artifact receiver, TLS passthrough to the source hosts of the bound Workspaces and
to the hosts the bound Model declares (`openultrasast.io/egress-hosts`). It is written between
`ax apply` and the resume, and deleted with the actor.
- **Inputs** (`reconciler.py`). Artifacts other tasks produced are never rendered into the
manifest; the runner fetches them from the receiver after the start, and the receiver serves a
running task its declared inputs only.
- **Completion** (`runner.py`). ax has no Completed phase: the runner's delivery of the output tar
is the task's completion. A completion marker stops a resumed actor from repeating billed work;
when the marker says the delivery failed, a later boot retries the delivery alone.
- **Budgets** (`plane/budget.py`). The metered client refuses the next call once a task's `usd` or
`calls` ceiling is reached; the task ends `unfinished` and resumes on a rerun with a larger
budget. Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
- **Attribution.** `ousast plane status RUN` prints per task its status, `calls`, `prompt`,
`cache_hit` and `output` tokens and `usd` from the tasks' `summary.json`, and writes
`attribution.json` in the run directory (`$OUSAST_RESULTS/plane/RUN/`, default
`~/ousast-results/plane/`).
- **Credentials** (`router.py`, `runner.py`).
- The reconciler reads the variable that the Model's `secretKey.key` names.
- It reads it from the operator's environment or `.env`. `.env` never overrides an exported variable.
- The key exists only in the start request.
- The runner keeps it in memory and passes it to the command's environment.
- The runner redacts it from echoed stderr and never writes it.
- **Egress** (`egress.py`). Egress means outbound network traffic.
- Each actor gets one EgressPolicy. It denies by default and names hostnames only.
- It allows plain HTTP to the artifact receiver.
- It allows TLS passthrough to the source hosts of the bound Workspaces.
- It allows TLS passthrough to the hosts the bound Model declares (`openultrasast.io/egress-hosts`).
- The reconciler writes it between `ax apply` and the resume. It is deleted with the actor.
- **Inputs** (`reconciler.py`).
- Artifacts that other tasks produced are never rendered into the manifest.
- The runner fetches them from the receiver after the start.
- The receiver serves a running task its declared inputs only.
- **Completion** (`runner.py`).
- ax has no Completed phase. The runner's delivery of the output tar is the task's completion.
- A completion marker stops a resumed actor from repeating billed work.
- If the marker says the delivery failed, a later boot retries only the delivery.
- **Budgets** (`plane/budget.py`).
- The metered client refuses the next call once a task reaches its `usd` or `calls` ceiling.
- The task then ends `unfinished`. It resumes on a rerun with a larger budget.
- Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
- **Attribution.**
- `ousast plane status RUN` prints these per task: status, `calls`, `prompt`, `cache_hit` and
`output` tokens, and `usd`. It reads them from the tasks' `summary.json`.
- It writes `attribution.json` in the run directory (`$OUSAST_RESULTS/plane/RUN/`, default
`~/ousast-results/plane/`).
!!! note "The golden snapshot"
Agent Substrate boots every new actor template once as a *golden* actor to snapshot it, with
the same image, Task and environment, and inside the sandbox nothing tells that boot from the
real one. So the runner's command never starts by itself: it waits for the start request,
which the reconciler sends through `atenet-router` to the task's actor, never to the golden
one. Workspaces are prepared only after the start, so the golden boot touches no network and a
started actor already runs under its egress policy (a clone at boot failed on the live cluster
while the actor was being restored, 2026-09-29; `runner.py`, `_run_once`). The first resume of
a new image can time out while the snapshot is built, which is why the reconciler retries it
Agent Substrate boots every new actor template once as a *golden* actor and snapshots it. That boot
uses the same image, Task and environment. Inside the sandbox, nothing tells that boot from the real one.
So the runner's command never starts by itself. It waits for the start request. The reconciler sends
that request through `atenet-router` to the task's actor, never to the golden one.
Workspaces are prepared only after the start. So the golden boot touches no network. A started actor
already runs under its egress policy. A clone at boot once failed on the live cluster while the actor
was being restored (2026-09-29; `runner.py`, `_run_once`).
The first resume of a new image can time out while the snapshot is built. So the reconciler retries it
for up to `OUSAST_RESUME_TIMEOUT` seconds (900).
## The Runs: validation and loop
`ousast plane workspaces POPULATION --validation-set ...` (`generate.py`) writes one chain per
case; `--loop` adds the `alerts` step per case and the loop's singleton steps. Every step except
the verify passes (and `roles` in harvest Runs) is model-free.
`ousast plane workspaces POPULATION --validation-set ...` (`generate.py`) writes one chain per case.
`--loop` adds the `alerts` step per case and the loop's singleton steps. Every step is model-free
except the verify passes (and `roles` in harvest Runs).
### Per case
@@ -119,23 +141,27 @@ flowchart LR
improve --> outcome["gate.json, journal.json, and rule_policy.json only if accepted"]
```
- **Before the Run** `ousast plane run` seeds from the store (`memory.seed`): a task whose
`openultrasast.io/memory-key` (repository, pin, candidates digest, runner image digest) has
stored facts gets them and is marked done, and the sandboxed `memory-snapshot` task gets the
store's rows after the guard. **After the Run** it ingests every delivered `memory.jsonl`
(`ousast plane remember RUN` repeats this by hand). See [Memory](memory.md).
- The loop's rules and its guard are described in
[Architecture](architecture.md#proposals-from-plane-memory). Adopting an accepted ledger stays a
maintainer commit (`src/openultrasast/plane/tasks/loop.py`).
- **Before the Run**, `ousast plane run` seeds from the store (`memory.seed`).
- Each task has an `openultrasast.io/memory-key`: repository, pin, candidates digest and runner image digest.
- If the store has facts for that key, the task gets them and is marked done.
- The sandboxed `memory-snapshot` task gets the store's rows after the guard.
- **After the Run**, it ingests every delivered `memory.jsonl`. `ousast plane remember RUN` repeats
this by hand. See [Memory](memory.md).
- [Architecture](architecture.md#proposals-from-plane-memory) describes the loop's rules and its guard.
Adopting an accepted ledger stays a maintainer commit (`src/openultrasast/plane/tasks/loop.py`).
- `ousast plane harvest` writes the decision engine's harvest Runs (verify a/b with agree, and
model `roles`); `ousast plane alerts-engine` produces a Run's `alerts` from the Joern engine
image on the host for PHP and other languages quick mode does not cover.
model `roles`).
- `ousast plane alerts-engine` produces a Run's `alerts` from the Joern engine image on the host. It
covers PHP and other languages that quick mode does not cover.
## Where this runs in production
ax is a Kubernetes application over Agent Substrate, every Task runs the one runner image as an actor that
occupies a whole worker, and two profiles of this code base are the target: `kind` on a laptop (today) and `k3s`
in production (planned), where the same CLI submits Runs remotely and the tasks deliver to the S3 store:
ax is a Kubernetes application over Agent Substrate. Every Task runs the one runner image as an actor. Each actor
occupies a whole worker. This code base targets two profiles:
- `kind` on a laptop (today).
- `k3s` in production (planned). There the same CLI submits Runs remotely, and the tasks deliver to the S3 store.
[Production topology: ax on a Kubernetes cluster](deployment.md#production-topology-ax-on-a-kubernetes-cluster)
draws it and sizes the pre-push hook at scale.
+74 -50
View File
@@ -1,16 +1,22 @@
# RustFS setup for the memory store
The plane's memory store can live in an S3-compatible bucket: `OUSAST_MEMORY=s3://sast-memory`
selects `S3Store` (`src/openultrasast/plane/memory.py`), which talks to the server through boto3.
Any S3-compatible store with bucket versioning, lifecycle rules, object tags and S3 Select works;
RustFS is the server the contract tests run against. The maintainer's server is RustFS at
`files.because-security.com`, bucket `sast-memory`, store at the bucket root. What the store keeps
there is described on [Memory](memory.md).
The plane's memory store can live in an S3-compatible bucket. RustFS is an S3-compatible object
storage server.
**The store never configures its bucket.** An admin sets the bucket up once with admin
credentials. The agent account the store runs as can read the bucket's settings but not change
them, and reads and writes objects freely. Every time the store opens, it checks the setup and
refuses to start if anything is missing.
- `OUSAST_MEMORY=s3://sast-memory` selects `S3Store` (`src/openultrasast/plane/memory.py`).
- `S3Store` talks to the server through boto3 (the AWS SDK for Python).
- Any S3-compatible store works if it has bucket versioning, lifecycle rules, object tags and S3 Select
(a server-side query over JSON objects).
- RustFS is the server the contract tests run against.
- The maintainer's server is RustFS at `files.because-security.com`. The bucket is `sast-memory`. The
store sits at the bucket root.
[Memory](memory.md) describes what the store keeps there.
**The store never configures its bucket.** An admin sets the bucket up once with admin credentials.
The store runs as an agent account. That account can read the bucket's settings but not change them.
It reads and writes objects freely. Every time the store opens, it checks the setup. It refuses to start
if anything is missing.
```mermaid
flowchart LR
@@ -35,8 +41,8 @@ aws s3api put-bucket-versioning \
--versioning-configuration Status=Enabled
```
**2. Expire raw run outputs under `runs/` after 30 days.** Everything else (`repos/`, `facts/`,
`index.jsonl` and the blobs) is kept, so the rule must be a plain prefix filter on `runs/`.
**2. Expire raw run outputs under `runs/` after 30 days.** Everything else is kept: `repos/`, `facts/`,
`index.jsonl` and the blobs. So the rule must be a plain prefix filter on `runs/`.
```bash
aws s3api put-bucket-lifecycle-configuration \
@@ -46,9 +52,10 @@ aws s3api put-bucket-lifecycle-configuration \
```
`put-bucket-lifecycle-configuration` **replaces every rule on the bucket**. If the bucket already
has rules, merge this one into them first. The rule id `ousast-runs-expiry` and the 30 days are the
store's defaults (`RUNS_RULE_ID`, `RUNS_EXPIRE_DAYS`); the store accepts any enabled, untagged
prefix rule on `runs/` with a number of days.
has rules, merge this one into them first.
The rule id `ousast-runs-expiry` and the 30 days are the store's defaults (`RUNS_RULE_ID`,
`RUNS_EXPIRE_DAYS`). The store accepts any enabled, untagged prefix rule on `runs/` with a number of days.
To read the result back:
@@ -59,8 +66,8 @@ aws s3api get-bucket-lifecycle-configuration --endpoint-url https://files.becaus
## The agent policy
Attach this policy to the agent account (the account whose keys go into `.env`). It is the
complete policy: nothing else is needed, and it grants no `Put*` on versioning or lifecycle.
Attach this policy to the agent account. That is the account whose keys go into `.env`. It is the
complete policy, and nothing else is needed. It grants no `Put*` on versioning or lifecycle.
```json
{
@@ -102,14 +109,14 @@ complete policy: nothing else is needed, and it grants no `Put*` on versioning o
| `BucketSettingsReadOnly` | `arn:aws:s3:::sast-memory` | The agent can list the bucket and its versions and *read* its location, versioning and lifecycle settings, which is what the startup check needs; it cannot change them. |
| `ObjectsReadWrite` | `arn:aws:s3:::sast-memory/*` | The agent reads, writes and deletes objects and their versions and reads and writes object tags; this covers row objects, blobs, the index, the Select probe, and the `kind` tag the store filters by. |
So the agent writes objects freely but cannot change bucket settings: turning versioning off or
So the agent writes objects freely but cannot change bucket settings. Turning versioning off or
removing the expiry rule needs the admin.
## The `.env` settings
The store reads its endpoint and credentials from `.env` in the working directory or from the
environment (`s3_settings`), never from a manifest, and never prints them. The names are the ones
the AWS SDKs and CLI use for credentials, so one `.env` serves both the store and `aws s3api`.
environment (`s3_settings`). It never reads them from a manifest. It never prints them. The names are
the ones the AWS SDKs and CLI use for credentials. So one `.env` serves both the store and `aws s3api`.
| Variable | Value for this server | Notes |
| --- | --- | --- |
@@ -121,26 +128,33 @@ the AWS SDKs and CLI use for credentials, so one `.env` serves both the store an
| `S3_REGION` | optional (`us-east-1` when unset) | the SigV4 signing region; spares a GetBucketLocation call |
| `S3_BUCKET` | `sast-memory` | the bucket `s3://` names when the URL has none; also the real-server tests' default bucket |
The client signs with SigV4 and uses path-style addressing (`https://host/bucket/key`), which
RustFS needs. The SDK's own checksum trailers are off, because not every S3-compatible server
accepts them.
The client signs with SigV4 (AWS's request-signing scheme). It uses path-style addressing
(`https://host/bucket/key`), which RustFS needs. The SDK's own checksum trailers are off. Not every
S3-compatible server accepts them.
- **Never commit `.env`.** It is gitignored (`.gitignore`: `.env`, `.env.*`, except
`.env.example`). `.env.example` lists these names with empty values.
- **`.env` never overrides a variable already exported in your shell** (`config.load_dotenv`).
A stale `export AWS_SECRET_ACCESS_KEY=...` in your profile silently wins over the file; `unset`
A stale `export AWS_SECRET_ACCESS_KEY=...` in your profile silently wins over the file. `unset`
it when a change in `.env` seems to have no effect.
- The store needs the `s3` extra (`uv sync --extra s3`, which installs boto3); without it opening
the store fails with `OUSAST_MEMORY=s3://... needs boto3: install openultrasast[s3]`. A missing
variable fails with `OUSAST_MEMORY=s3://... needs S3_ENDPOINT, AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY in .env or the environment` (naming only the missing ones); an endpoint
without a scheme fails with `S3_ENDPOINT must be a URL with a scheme (https://host[:port])`.
- The store needs the `s3` extra (`uv sync --extra s3`, which installs boto3). These are the
startup errors:
- Without the extra, opening the store fails with
`OUSAST_MEMORY=s3://... needs boto3: install openultrasast[s3]`.
- A missing variable fails with `OUSAST_MEMORY=s3://... needs S3_ENDPOINT, AWS_ACCESS_KEY_ID,
AWS_SECRET_ACCESS_KEY in .env or the environment`. The message names only the missing ones.
- An endpoint without a scheme fails with
`S3_ENDPOINT must be a URL with a scheme (https://host[:port])`.
## What the store verifies at startup
`S3Store.verify_bucket()` runs once per process for each bucket and prefix. It reads the
versioning status and the lifecycle rules, writes a two-line probe object at
`_probe/select.jsonl` (under the prefix, if any), queries it with S3 Select and reads its tags.
`S3Store.verify_bucket()` runs once per process for each bucket and prefix. It does four things:
1. It reads the versioning status.
2. It reads the lifecycle rules.
3. It writes a two-line probe object at `_probe/select.jsonl` (under the prefix, if any).
4. It queries the probe with S3 Select and reads its tags.
Every missing piece is collected into one error:
```text
@@ -148,8 +162,8 @@ memory s3://sast-memory: the bucket is not set up for the store. The store never
- <one line per missing piece>
```
The lines, exactly as the code writes them for this bucket at the root (`<failure>` is the
server's error code and the first 200 characters of its message):
The table lists the lines exactly as the code writes them for this bucket at the root. `<failure>` is
the server's error code and the first 200 characters of its message.
| Check | Refusal line |
| --- | --- |
@@ -161,32 +175,42 @@ server's error code and the first 200 characters of its message):
| tags unreadable | `object tags cannot be read on _probe/select.jsonl (<failure>): the store filters row objects by their kind tag, so the agent needs s3:GetObjectTagging on arn:aws:s3:::sast-memory/*` |
| S3 Select missing | `S3 Select (SelectObjectContent over JSON Lines) does not answer on _probe/select.jsonl (<failure>). The store requires it and has no local fallback: use a server with S3 Select (RustFS has it; not every S3-compatible server does) and let the agent s3:PutObject and s3:GetObject` |
The admin commands inside the messages use `--endpoint-url "$S3_ENDPOINT"`, which is the full
URL from `.env`, so they run as written once `.env` is exported.
The admin commands inside the messages use `--endpoint-url "$S3_ENDPOINT"`. That is the full URL from
`.env`. So they run as written once `.env` is exported.
There is **no fallback**: the store does not start in a degraded mode, does not drop to a local
store, and does not fetch and filter objects itself.
There is **no fallback**. The store does not:
- start in a degraded mode;
- drop to a local store;
- fetch and filter objects itself.
## The RustFS Select limit and the fixed-schema rule
RustFS's S3 Select infers an object's JSON schema from its leading rows. A `where` field that is
absent from those rows is unbound, even if a later row has it: the query fails with
RustFS's S3 Select infers an object's JSON schema from its leading rows. A `where` field absent from
those rows is unbound, even if a later row has it. The query then fails with
`EvaluatorBindingDoesNotExist` instead of matching (measured 2026-10-02 with the field first
present at row 5000). The store does not read that as "no rows", since that answer could silently
drop matches; it raises:
present at row 5000).
The store does not read that failure as "no rows". That answer could silently drop matches. It raises:
```text
memory s3://sast-memory: S3 Select on <key> failed (EvaluatorBindingDoesNotExist: ...) (a field of [<fields>] is absent from the object's leading rows, which the server reads as its schema); the store has no local fallback
```
The rule that follows: a field the store filters on is present in every row of its kind from
the first row on, as `""` where it does not apply; each row kind has a fixed set of queryable
fields, enforced at write time (see [Memory](memory.md#queryable-fields-per-kind)).
The rule that follows has two parts (see [Memory](memory.md#queryable-fields-per-kind)):
- A field the store filters on is present in every row of its kind from the first row on. It is `""`
where it does not apply.
- Each row kind has a fixed set of queryable fields. The store enforces it at write time.
## Checks against the real server
The contract tests that talk to a real server run only with `OUSAST_MEMORY_TEST_S3=1` (bucket
`OUSAST_MEMORY_TEST_BUCKET`, else `S3_BUCKET`). They verify the bucket at its root, write only
under a fresh `contract-<id>/` prefix, and remove every version and delete marker under it
afterwards; they never configure the bucket. More on the host setup:
[ax on this host](ops/ax/README.md).
The contract tests that talk to a real server run only with `OUSAST_MEMORY_TEST_S3=1`. They use the
bucket `OUSAST_MEMORY_TEST_BUCKET`, else `S3_BUCKET`. The tests:
- verify the bucket at its root;
- write only under a fresh `contract-<id>/` prefix;
- remove every version and delete marker under it afterwards;
- never configure the bucket.
More on the host setup: [ax on this host](ops/ax/README.md).
+330 -39
View File
@@ -1,13 +1,21 @@
# Scanning
`ousast scan PATH --mode quick|standard|deep` scans a checkout; `ousast pre-push` checks only
what a push (or an explicit `--base`/`--head` pair) changes. The stage-by-stage pipeline,
evidence ladder and score are in [Architecture](architecture.md); this page shows the flows.
This page shows the scan flows.
- `ousast scan PATH --mode quick|standard|deep` scans a checkout.
- `ousast pre-push` checks only what a push changes, or what an explicit `--base`/`--head` pair
changes.
The stage-by-stage pipeline, the evidence ladder and the score are in
[Architecture](architecture.md).
## The three scan modes
The stages a mode requests are `MODE_STAGES` in `src/openultrasast/stages.py`: `quick` runs
STATIC, `standard` adds MAP, `deep` adds REGRESS.
The stages each mode requests are listed in `MODE_STAGES` in `src/openultrasast/stages.py`:
- `quick` runs STATIC.
- `standard` adds MAP.
- `deep` adds REGRESS.
```mermaid
flowchart TD
@@ -38,7 +46,7 @@ Sources: `README.md` (scan modes), `src/openultrasast/cli.py` (`run_stage` calls
## Quick rules per language
Rules run only against files of their language. The bundled ruleset lives in
Each rule runs only against files of its language. The bundled ruleset lives in
`src/openultrasast/ruleset/<language>/rules.toml`.
| Language | Classes (CWE) |
@@ -49,21 +57,31 @@ Rules run only against files of their language. The bundled ruleset lives in
| Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
| PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules |
The detection gate (`python -m openultrasast.gate`) enforces at least 90% recall and under 10%
false positives per language on the bundled corpora in `benchmarks/manifests/`. Those corpora are
cheat-sheet fixtures: they catch regressions, they do not estimate recall on real code. **PHP is
not in the gate**; its numbers are in-sample development numbers
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). See `README.md` for the
gate's per-language output.
The detection gate (`python -m openultrasast.gate`) enforces two limits per language on the
bundled corpora in `benchmarks/manifests/`: at least 90% recall and under 10% false positives.
- Those corpora are cheat-sheet fixtures. They catch regressions. They do not estimate recall on
real code.
- **PHP is not in the gate**. Its numbers are in-sample development numbers
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`).
See `README.md` for the gate's per-language output.
## The Joern engine
The model layer (`src/openultrasast/model/`, `src/openultrasast/cpg/`) is the core of `standard`.
It builds one code property graph per scan with Joern (`joern-parse`, then batched
`joern --script` queries from `cpg/queries/*.sc`) and sends each question to the arbiter of its
type: **taint** (source to sink with no discharging sanitizer), **dominance** (does a guard govern
the obligated operation) and **configuration** (constant abstraction over a value). Taint facts
ship for C, Java, JavaScript, PHP and Python (`src/openultrasast/ruleset/semantic/*.toml`).
It builds one code property graph per scan with Joern. A code property graph is a graph of the
program's syntax, control flow and data flow. The build runs `joern-parse`, then batched
`joern --script` queries from `cpg/queries/*.sc`.
The layer sends each question to the arbiter (the decider) of its type:
- **taint**: a flow from source to sink with no discharging sanitizer;
- **dominance**: does a guard govern the obligated operation;
- **configuration**: constant abstraction over a value.
Taint facts ship for C, Java, JavaScript, PHP and Python
(`src/openultrasast/ruleset/semantic/*.toml`).
```mermaid
flowchart LR
@@ -79,25 +97,33 @@ flowchart LR
rung -- "residual question only" --> judge["LLM judge: confirm or contradict, never create a flow"]
```
The Docker image ships Joern and `php-cli` (which the PHP frontend needs); without Joern the
layer records `cpg_unavailable` and the scan is otherwise unchanged. Engine installation and the
Joern features used are in [Engine and pre-push](ops/README.md).
How the graph is built, queried and refined, with the records behind each refinement, is in
[Detection techniques](detection-techniques.md).
The Docker image ships Joern and `php-cli`. The PHP frontend needs `php-cli`. Without Joern, the
layer records `cpg_unavailable`; the rest of the scan is unchanged.
- Engine installation and the Joern features used: [Engine and pre-push](ops/README.md).
- How the graph is built, queried and refined, with the records behind each refinement:
[Detection techniques](detection-techniques.md).
## Framework priors
Rule and taint-fact entries that know a framework or library carry a `framework` or `library` tag
naming a row of `src/openultrasast/ruleset/frameworks.toml` (for example `wordpress`, `flask`,
`django`, `express`, `spring`). The loaders take `priors=`: `"all"` (the default and today's
behaviour), `"off"` (language-level knowledge only) or a set of ids, so the contribution of
framework knowledge can be measured and switched off. The decision engine's first slice ran with
priors off ([Decision engine](decision-engine.md#first-measured-slice-2026-10-01)).
Some rule and taint-fact entries know a framework or library. They carry a `framework` or
`library` tag. The tag names a row of `src/openultrasast/ruleset/frameworks.toml` (for example
`wordpress`, `flask`, `django`, `express`, `spring`).
The loaders take `priors=` with one of three values:
- `"all"`: the default and today's behaviour;
- `"off"`: language-level knowledge only;
- a set of ids.
So the contribution of framework knowledge can be measured and switched off. The decision
engine's first slice ran with priors off
([Decision engine](decision-engine.md#first-measured-slice-2026-10-01)).
## Pre-push: the delta check (experimental)
`ousast pre-push` analyses the commits a `git push` would publish, or an explicit
`--base`/`--head` pair, and compares head with base so that only new or worsened defects are
`--base`/`--head` pair. It compares head with base. Only new or worsened defects become
candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`,
[Engine and pre-push](ops/README.md#explicit-local-replay).
@@ -105,23 +131,288 @@ candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`,
flowchart TD
input["git push refs on stdin, or --base / --head"] --> resolve["Resolve both commit identities"]
resolve --> snap["Materialise tracked bytes in private scratch storage"]
snap --> regions["Discover changed regions, up to --max-regions"]
regions --> engine["Evidence ranker and arbiter on head"]
snap --> quick["Quick rules on the changed head lines (advisory, under 1 s)"]
quick --> eng{"--engine"}
eng -- "off" --> art
eng -- "background" --> bg["Detached engine run; result shown on the next push"]
bg --> art
eng -- "inline, the default" --> engine["Engine on head: changed regions up to --max-regions, ranker, arbiter"]
engine --> cmp["Targeted base comparison: new or worsened only"]
cmp --> admit{"Admission policy: capability registry"}
admit -- "registry empty today" --> diag["Diagnostic candidates in the artifact"]
admit -- "registry empty today" --> diag["Advisory engine findings, not alerts"]
admit -- "an enabled capability" --> alert["Normal alert"]
diag --> art["JSON artifact outside the repository"]
diag --> art["Terminal lines + JSON artifact outside the repository"]
alert --> art
art --> exit{"--mode"}
exit -- "advisory, the default" --> allow["Exit 0: allow"]
exit -- "blocking, explicit opt-in" --> block["May block the push"]
```
- `--deadline` (30 s by default) bounds preparation, both revisions and reporting together;
`--incomplete-coverage block` opts into failing when coverage is incomplete.
- The default capability registry is empty pending independent qualification, so no normal alert
is emitted today; every candidate stays diagnostic.
- `--deadline` (30 s by default) bounds preparation, the quick rules, both revisions and reporting together.
- The quick rules run first on the changed lines. Their matches are shown as advisory pattern
matches, never as alerts, and never block.
- The default capability registry is empty until independent qualification passes. So no normal
alert is emitted today. An engine finding the push introduced is shown as an advisory engine
finding; every other candidate stays diagnostic in the artifact.
- A model is called only with an explicit `--model-config`.
- Hook installation (`ops/install-pre-push`) and the private cache (`--cache-dir`) are documented
in [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
- The private cache (`--cache-dir`) and the manual install recipe are documented in
[Engine and pre-push](ops/README.md#experimental-pre-push-integration).
### First run: install the hook on any repository
The hook is advisory. It never rejects a push unless you set `OUSAST_PUSH_MODE=blocking`. A check
that cannot run says so in one line and lets the push through: a missing `ousast`, a missing
Docker image, an invalid setting, a crash.
1. Install the tool. The package is not on PyPI. Install it from the repository (or from a
release wheel on the GitHub releases page):
```bash
pip install "git+https://github.com/norandom/OpenUltraSAST"
```
2. Install the hook in the repository you want checked. Git's effective hook directory is used,
including `core.hooksPath`, and that setting is never edited:
```bash
cd /path/to/your/repository
ousast pre-push install # refuses if a pre-push hook already exists
ousast pre-push install --force # moves an existing hook to pre-push.before-ousast and chains it
```
3. Push as usual. Without the Joern engine, you get the quick rules and one line saying the
engine did not run. For the engine, choose one:
- a host install with `joern-parse` on the PATH (`pyinfra @local ops/joern.py`, see
[Engine and pre-push](ops/README.md));
- Docker: `docker pull ghcr.io/norandom/openultrasast:2.0.1`, then
`ousast pre-push install --docker`. The wrapper `ousast-docker` is installed beside the
hook and runs the check in the container (repository read-only, no network,
`--memory 3g`). The container runs the code of its image's version: the quick-rule
tier, the plain skip lines and `--engine` need an image built from a version that has them
(`docker build -t openultrasast:dev .`, then `OUSAST_IMAGE=openultrasast:dev`).
4. Remove it with `ousast pre-push uninstall`. Only a hook this tool wrote is removed, and a
chained previous hook is restored.
A push on the 2026-10-02 proof repository (serve-static, a new `debug.js` that evaluates
`req.query.code`, no Joern on the host) prints:
```text
Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.
- js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review
- js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review
No actionable defects reported; analysis is incomplete.
Skipped: the changed code has no counterpart in the base revision to compare against
Skipped: engine did not run: the code graph could not be built (is Joern installed? ...)
Skipped: no base revision was analysed, so no engine finding can be called new or worsened
Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.
Details: /home/you/.cache/openultrasast/push/push-12345.json
```
The hook reads these settings from the environment of `git push`:
| Variable | Default | Effect |
| --- | --- | --- |
| `OUSAST_PUSH_DEADLINE` | `30` | seconds for the whole check |
| `OUSAST_PUSH_MODE` | `advisory` | `blocking` is the only way a result can reject a push |
| `OUSAST_INCOMPLETE_COVERAGE` | `allow` | with `blocking`: `block` also rejects incomplete coverage |
| `OUSAST_PUSH_ENGINE` | `inline` | `background`: return after the quick rules, run the engine detached; `off`: quick rules only |
| `OUSAST_COMPARISON_BASE` | unset | a local ref to compare a new branch against |
| `OUSAST_ARTIFACT_DIR` | `~/.cache/openultrasast/push` | where the JSON results go; outside the repository |
| `OUSAST_COMMAND` | `ousast` | the command the hook runs, for example the Docker wrapper |
| `OUSAST_IMAGE` | `ghcr.io/norandom/openultrasast:2.0.1` | the Docker wrapper's image |
| `OUSAST_DOCKER_MEMORY` | `3g` | the Docker wrapper's memory limit |
**A new branch.** Git gives the hook no remote base for a branch the remote does not have yet.
The check says so ("new branch: the remote has no base to compare against"), runs the quick rules
over the files the branch's last commit changed instead, and says that too. Set
`OUSAST_COMPARISON_BASE=main` to compare a new branch against your local `main`.
**The engine and the deadline.** The engine does not fit a 30 s deadline today. The 2026-10-02
audit measured 260 s on a 26-file repository; graph reuse is the planned fix. When it cannot
finish, one line says so. With `OUSAST_PUSH_ENGINE=background`, the hook returns after the quick
rules and starts one detached engine run (`--background-deadline`, 900 s by default). The next push
in the same repository prints `Engine result for <head> from the previous push: ...` once. Only
one background run happens at a time. The Docker wrapper cannot run the engine in the background,
because the container ends with the hook.
**The artifact.** Every run writes one JSON file, and the terminal prints its path after
`Details:`. It records the following:
- `snapshots[].bytes_read`: proof that the input was read;
- `quick_tier`: the quick-rule matches, with the files and bytes read;
- `admission.coverage_reasons`: the raw skip reasons;
- `admission.dispositions`: every engine candidate and its vetoes;
- `engine_background`: background runs started or shown;
- `timings`.
Files accumulate, one per push; delete old ones when you like.
### Skip reasons
The terminal prints one `Skipped:` line per distinct reason in plain words. The artifact keeps the
raw reason. Repository paths are hex-encoded there, and per-question reasons end in a hash.
| Raw reason (artifact) | Terminal line says | What to do |
| --- | --- | --- |
| `cpg_build_failed`, `cpg_unavailable` | the engine did not run: the code graph could not be built | install Joern, or use the Docker wrapper |
| `cpg_empty` | the engine read no code | check that the changed files are in a supported language |
| `frontend_unsupported` | no engine frontend for a language in this push | none: that language gets the quick rules at most |
| `language_not_covered:<lang>:<changed>:<total>` | language not covered: `<lang>` (n changed files, m in the repository) | none: neither the engine nor the quick rules cover it, so silence is not a clean result |
| `deadline_exhausted` | the engine did not finish within the deadline | raise `OUSAST_PUSH_DEADLINE`, or set `OUSAST_PUSH_ENGINE=background` |
| `change_context_deadline_exhausted` | the deadline ran out while relating the change to the code | as above |
| `report_deadline_exhausted` | the deadline ran out while saving the details | as above |
| `engine_off`, `engine_background` | the engine was switched off, or runs after the push | none |
| `resolution:missing_base` | new branch: the remote has no base to compare against | set `OUSAST_COMPARISON_BASE` |
| `resolution:unsupported_base`, `resolution:no_merge_base`, `resolution:ambiguous_base`, `resolution:missing_target`, `resolution:unsupported_target`, `resolution:resolution_error` | the pushed ref or its remote base could not be resolved locally | fetch the remote, then push again |
| `base_comparison_unavailable`, `base_unavailable` | no base revision was analysed, so nothing can be called new | see the line before it for the cause |
| `base_counterpart_unresolved` | the changed code has no counterpart in the base to compare against | none: expected for new files |
| `<path>:ambiguous_line_correspondence`, `<path>:line_correspondence_unavailable`, `<path>:ambiguous_path_rename_correspondence` | changed lines or a renamed file could not be matched to the base | none: findings there stay diagnostic |
| `unmatched_added_deleted_path_correspondence` | files were both added and deleted, so a move cannot be told from a new file | push the move separately from other edits |
| `declaration_paths_unavailable` | project declaration files could not be listed | rerun |
| `capability_unavailable`, `capability_disabled`, `capability_unevaluated`, `capability_ambiguous` | engine findings are not qualified as alerts yet | none: they show as advisory engine findings |
| `eligibility:<reason>` | the capability registry does not match this build | reinstall a consistent build |
| `comparison_unknown`, `base_incomplete` and the other novelty reasons | whether this push introduced some engine findings could not be established | none: they stay diagnostic |
| `dynamic_external_or_depth_context_unresolved:...` | n engine checks depend on dynamic or external code the engine cannot follow | none |
| `query_failed`, `query_budget_reserved`, `query_too_expensive`, `regions_truncated`, `files_unparsed` | some engine questions were not answered | raise the deadline or `--max-regions` |
| `cross_partition_semantics_unresolved` | flows between languages are not followed | none |
| `dependency_unresolved`, `change_unsupported` | engine findings depend on unresolved code or have no supported link to the change | none |
| `snapshot:<reason>` | some files could not be materialised for analysis | see the reason (symlinks, submodules, LFS) |
| `<stage>_failed:<Error>`, `push_input_failed:<Error>` | the named step failed | rerun; report it if it repeats |
A reason not in this table is still printed as its own words, followed by "see the details file".
### In CI
The same check runs on a pushed range in CI. This GitHub Actions job is advisory: it never fails
the build, and it uploads the artifact.
```yaml
name: ousast pre-push (advisory)
on: push
jobs:
pre-push:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install "git+https://github.com/norandom/OpenUltraSAST"
- name: Check what this push changed
if: github.event.before != '0000000000000000000000000000000000000000'
run: |
ousast pre-push . --base ${{ github.event.before }} --head ${{ github.sha }} \
--artifact "$RUNNER_TEMP/ousast-pre-push.json" --engine off --deadline 120 || true
- uses: actions/upload-artifact@v4
if: always()
with:
name: ousast-pre-push
path: ${{ runner.temp }}/ousast-pre-push.json
```
A branch's first push has no `github.event.before` to compare against, and the job skips it. For
the engine in CI, run the job in the container image instead of `pip install`, and drop
`--engine off`.
### `ousast pre-push --help`
```text
usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]
[--comparison-base COMPARISON_BASE] [--model-config MODEL_CONFIG]
[--prior-hook PRIOR_HOOK] --artifact ARTIFACT [--cache-dir CACHE_DIR]
[--deadline DEADLINE] [--cancellation-allowance CANCELLATION_ALLOWANCE]
[--max-regions MAX_REGIONS] [--mode {advisory,blocking}]
[--incomplete-coverage {allow,block}] [--engine {inline,background,off}]
[--background-deadline BACKGROUND_DEADLINE]
[--experimental-declarations EXPERIMENTAL_DECLARATIONS]
[--experimental-record-vetoes]
path
Check what a push changes: quick rules on the changed lines, then the Joern engine on head and
base within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with
'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall
[REPO]'.
positional arguments:
path the repository to check (the hook passes '.')
options:
-h, --help show this help message and exit
--base BASE explicit replay: the base revision (with --head; instead of --remote)
--head HEAD explicit replay: the head revision to check against --base
--remote NAME URL Git hook mode: the remote name and URL Git passes to pre-push; the push
input is read from stdin
--comparison-base COMPARISON_BASE
hook mode: a local ref to compare a NEW branch against (the remote has no
base for it); the hook reads OUSAST_COMPARISON_BASE
--model-config MODEL_CONFIG
explicit optional witness selection endpoint configuration
--prior-hook PRIOR_HOOK
explicitly chain an existing hook with the same input and arguments
--artifact ARTIFACT where to write the complete JSON result; must be outside the repository
--cache-dir CACHE_DIR
reuse compatible local artifacts in a private directory outside the
repository
--deadline DEADLINE seconds for the whole check: resolution, quick rules, engine and reporting
(default 30; hook: OUSAST_PUSH_DEADLINE)
--cancellation-allowance CANCELLATION_ALLOWANCE
seconds after the deadline to stop work and save the result (default 2)
--max-regions MAX_REGIONS
most code regions the engine examines, highest-ranked first (default 500)
--mode {advisory,blocking}
advisory (default) never blocks; blocking is the explicit opt-in that
rejects a push for an admitted alert (hook: OUSAST_PUSH_MODE)
--incomplete-coverage {allow,block}
with --mode blocking only: also block when coverage is incomplete (hook:
OUSAST_INCOMPLETE_COVERAGE)
--engine {inline,background,off}
inline (default): run the Joern engine inside the deadline; background:
return after the quick rules and run the engine detached, showing its
result on the next run; off: quick rules only
--background-deadline BACKGROUND_DEADLINE
seconds the detached engine run of --engine background may take (default
900)
--experimental-declarations EXPERIMENTAL_DECLARATIONS
explicit --base/--head replay only: unreviewed capability declarations
used to render the experimental evaluation explanation; never consulted by
admission and cannot enable any capability
--experimental-record-vetoes
explicit --base/--head replay only: record each comparison and admission
veto per raw finding instead of applying it; output is labeled
experimental, changes no result and enables no hook capability
```
```text
usage: ousast pre-push install [-h] [--force] [--docker] [repository]
Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory
(core.hooksPath is honoured, never edited).
positional arguments:
repository the repository (default: current directory)
options:
-h, --help show this help message and exit
--force install: replace our hook, or move a foreign hook to pre-push.before-ousast and
chain it; uninstall: remove a hook this tool did not write
--docker also install the ousast-docker wrapper beside the hook, so the check runs in the
container image
```
```text
usage: ousast pre-push uninstall [-h] [--force] [repository]
Remove the OpenUltraSAST pre-push hook and restore a chained previous hook.
positional arguments:
repository the repository (default: current directory)
options:
-h, --help show this help message and exit
--force install: replace our hook, or move a foreign hook to pre-push.before-ousast and
chain it; uninstall: remove a hook this tool did not write
```
+75 -52
View File
@@ -1,78 +1,101 @@
# Threat model & hardening
OpenUltraSAST analyzes **untrusted code**. This document states what it trusts, what
it does not, and the controls that keep a scan safe to run in CI and safe to share.
OpenUltraSAST analyzes **untrusted code**. This page states what it trusts and what it does not.
It also lists the controls that keep a scan safe to run in CI and safe to share.
## Trust boundaries
- **Scanned code is untrusted input.** Quick and standard modes never import, build or run
the target: quick reads source as text, and standard additionally parses it into a code
property graph with Joern (`joern-parse`, a parser, not a build) and may send code excerpts
to the configured model. Only `deep` mode executes target-derived code: its REGRESS stage
loads promoted candidates inside the Docker sandbox below, and it runs only when `docker
info` succeeds (otherwise the stage is skipped and recorded as `sandbox_unavailable`).
- **The ruleset and CWE policy are trusted, governed data.** They change only through
reviewed commits or the bounded self-improvement loop (`ousast improve`), which can
flip rule *status* and tune score constants but can never edit pattern text or the
- **Scanned code is untrusted input.** Quick and standard modes never import, build or run the
target.
- Quick mode reads source as text.
- Standard mode also parses it into a code property graph (a graph of the program's syntax,
control flow and data flow) with Joern. It uses `joern-parse`, a parser, not a build.
Standard mode may send code excerpts to the configured model.
- Only `deep` mode executes target-derived code. Its REGRESS stage loads promoted candidates
inside the Docker sandbox described below. It runs only when `docker
info` succeeds (otherwise the stage is skipped and recorded as `sandbox_unavailable`).
- **The ruleset and CWE policy are trusted, governed data.** They change in only two ways:
through reviewed commits, or through the bounded self-improvement loop (`ousast improve`). The
loop can flip rule *status* and tune score constants. It can never edit pattern text or the
authoritative 0–5 severity. See [architecture.md](architecture.md#self-improvement).
- **Provider credentials are the operator's.** The core install (PyYAML only) makes no
network calls on its own. A local scan calls a model only in standard or deep mode, and
only when a key is available: the model layer asks the residual question the graph cannot
settle, and `[models] hunter` enables the tool hunter. LLM calls go to DeepSeek
(`DEEPSEEK_API_KEY`); OpenRouter (`OPENROUTER_API_KEY`) serves embeddings. Keys come from
the environment or a gitignored `.env`, which never overrides an exported variable. The
pre-push check calls a model only with an explicit `--model-config`. Agentic work runs on the plane (`ousast plane run`): `ousast` reads
the variable a task's Model names (`secretKey.key`) from the operator's environment or
`.env` and sends the key only in that task's start request, through Agent Substrate's
router to that task's actor. It is not baked into an image or task template, and it is
never logged or written to artifacts.
- **Provider credentials are the operator's.** The core install (PyYAML only) makes no network
calls on its own.
- A local scan calls a model only in standard or deep mode, and only when a key is available.
The model layer asks the residual question the graph cannot settle. `[models] hunter`
enables the tool hunter.
- LLM calls go to DeepSeek (`DEEPSEEK_API_KEY`). OpenRouter (`OPENROUTER_API_KEY`) serves
embeddings.
- Keys come from the environment or a gitignored `.env`. `.env` never overrides an exported
variable.
- The pre-push check calls a model only with an explicit `--model-config`.
- Agentic work runs on the plane (`ousast plane run`). `ousast` reads the variable a task's
Model names (`secretKey.key`) from the operator's environment or `.env`. It sends the key
only in that task's start request. The request goes through Agent Substrate's router to that
task's actor. The key is not baked into an image or task template. It is never logged or
written to artifacts.
## Controls
### Secret redaction
Scan artifacts can quote source that contains live credentials. `redaction.py` masks
recognizable secret shapes (provider API keys, AWS/GitHub/Slack/Google tokens, bearer
tokens, URL-embedded credentials, PEM private keys, and `key = value` assignments for
sensitive names) before traces (`trace/events.jsonl`) and the markdown report are
written. On by default; disable with `[hardening] redact_secrets = false`.
recognizable secret shapes before traces (`trace/events.jsonl`) and the markdown report are
written. It masks:
- provider API keys;
- AWS/GitHub/Slack/Google tokens;
- bearer tokens;
- URL-embedded credentials;
- PEM private keys;
- `key = value` assignments for sensitive names.
Redaction is on by default. Turn it off with `[hardening] redact_secrets = false`.
### Plane egress
Each plane task runs in its own actor behind Agent Substrate's egress gateway, which
denies by default. The task's EgressPolicy (`src/openultrasast/plane/egress.py`) allows exactly three
things: plain HTTP to the artifact receiver, and TLS passthrough to the Git hosts of the
task's bound Workspaces and to the hosts its bound Model declares
(`openultrasast.io/egress-hosts`). There are no wildcards, no IP addresses and no TLS
interception. The policy is deleted with the actor.
Each plane task runs in its own actor behind Agent Substrate's egress gateway. The gateway denies
by default. The task's EgressPolicy (`src/openultrasast/plane/egress.py`) allows exactly three
things:
1. plain HTTP to the artifact receiver;
2. TLS passthrough to the Git hosts of the task's bound Workspaces;
3. TLS passthrough to the hosts its bound Model declares (`openultrasast.io/egress-hosts`).
There are no wildcards, no IP addresses and no TLS interception. The policy is deleted with the
actor.
### Cost & CI budgets
- **Agentic spend** is bounded per plane task by the Run's `budget: {usd, calls}`. The
metered client refuses the next call once either ceiling is reached (the task is
`unfinished` and resumes on a rerun with a larger budget). A usd budget on a Model
without prices is refused rather than metered as zero, and an account or
authentication error fails the task and starts nothing further. The standard-mode tool
hunter is bounded by the per-tier hunter budgets.
- **Agentic spend** is bounded per plane task by the Run's `budget: {usd, calls}`.
- The metered client refuses the next call once either ceiling is reached. The task is then
`unfinished`. It resumes on a rerun with a larger budget.
- A usd budget on a Model without prices is refused. It is not metered as zero.
- An account or authentication error fails the task and starts nothing further.
- The standard-mode tool hunter is bounded by the per-tier hunter budgets.
- **Output size** is bounded by `[hardening] max_findings` (0 = unlimited). Truncation is
severity-ordered and disclosed as a `budget` degradation in the manifest.
severity-ordered. It is disclosed as a `budget` degradation in the manifest.
### Provider reliability
LLM/embedding calls retry transient failures (HTTP 429/5xx, connection errors, timeouts)
with exponential backoff; non-transient errors (e.g. 4xx, malformed JSON) fail fast.
LLM/embedding calls retry transient failures with exponential backoff. Transient failures are
HTTP 429/5xx, connection errors and timeouts. Non-transient errors fail fast, for example 4xx or
malformed JSON.
### Visible degradation & determinism
When an optional engine is missing (Joern: `cpg_unavailable`; Docker: `sandbox_unavailable`),
the scan falls back to its deterministic equivalent and records a `degradations` entry in the
manifest — it is never silently downgraded. Without a provider key the model layer asks no
question and reports only what the graph entailed. Runs are reproducible: fixed config,
artifact manifests, prompt hashes, and model identifiers are recorded.
A scan is never silently downgraded. When an optional engine is missing, the scan falls back to
its deterministic equivalent. It records a `degradations` entry in the manifest (Joern:
`cpg_unavailable`; Docker: `sandbox_unavailable`).
Without a provider key, the model layer asks no question. It reports only what the graph
entailed.
Runs are reproducible. The fixed config, artifact manifests, prompt hashes and model identifiers
are recorded.
### MCP surface
`ousast mcp` exposes only the ten narrow project tools. No tool runs arbitrary shell,
Docker, or internal hunter tools, and no tool accepts a free-form command argument.
`ousast mcp` exposes only the ten narrow project tools. No tool runs arbitrary shell, Docker or
internal hunter tools. No tool accepts a free-form command argument.
## Sandbox (deep mode)
The REGRESS stage (`regress/`, `sandbox/runner.py`) runs each snippet with `docker run`
and never mounts the host Docker socket:
The REGRESS stage (`regress/`, `sandbox/runner.py`) runs each snippet with `docker run`. It never
mounts the host Docker socket.
| Control | Setting |
|---|---|
@@ -84,5 +107,5 @@ and never mounts the host Docker socket:
| pids | `[sandbox] pids_limit`, default 512 |
| timeout | `[sandbox] timeout_seconds`, default 300 s |
Before anything runs, a structural deny-list (`regress/safety.py`) rejects a snippet that
mentions sockets, curl, host networking or writes under `/workspace`.
A structural deny-list (`regress/safety.py`) checks each snippet before anything runs. It rejects
a snippet that mentions sockets, curl, host networking or writes under `/workspace`.
+200 -128
View File
@@ -1,18 +1,26 @@
# Token ergonomics
Model tokens are the one running cost of this project, and they are spent in two places: by the
**plane** when a Run verifies candidates or compiles the decision engine, and by the maintainers'
own **agentic coding sessions**. The maintainer's framing (2026-09-29): "the token saving strategy
is a routing concern. ax can handle that and structure our services in small components." So the
runtime half is routing, one Model per Task with its own budget, and reuse of what was already
paid for; the development half is tooling that keeps bulk file contents out of the session.
Model tokens are the one running cost of this project. They are spent in two places:
Every number on this page comes from a committed record, named next to it. Records are counts
only; none holds code, prompts or responses.
- by the **plane** (the system that runs scans as small tasks) when a Run verifies candidates
or compiles the decision engine;
- by the maintainers' own **agentic coding sessions**.
The before-and-after summary (script pipeline against plane, and the development share) is on
[Where we stand](where-we-stand.md#4-token-economics-before-and-after); this page holds the
mechanisms and the detail table.
The maintainer's framing (2026-09-29): "the token saving strategy is a routing concern. ax can
handle that and structure our services in small components." Here ax is Google's controller
that runs agent tasks on Kubernetes. So the two halves look like this:
- **Runtime** is routing: one Model per Task with its own budget, and reuse of what was
already paid for.
- **Development** is tooling that keeps bulk file contents out of the session.
Every number on this page comes from a committed record, named next to it. Records hold counts
only. None holds code, prompts or responses.
The before-and-after summary is on
[Where we stand](where-we-stand.md#4-token-economics-before-and-after). It compares the script
pipeline with the plane and gives the development share. This page holds the mechanisms and the
detail table.
## A. Runtime: the plane's token economy
@@ -37,46 +45,67 @@ flowchart LR
### One Model per Task, one budget per Task
A Run is a DAG of ax Tasks; each Task binds at most one Model and its own `budget: {usd, calls}`
([The plane on ax](plane.md)). The budget is enforced by the metered client
(`src/openultrasast/plane/budget.py`, `MeteredClient`): `check()` runs before every call and
raises `BudgetExhausted` once the spent `usd` or the call count reaches the ceiling, so the call
that crosses the ceiling completes and the next one is refused. The task then ends `unfinished`
and resumes from its `units.jsonl` on a rerun with a larger budget; an HTTP 401/402 or
"Insufficient Balance" is an `AccountError`, which fails the task and starts nothing further.
Usage is attributed per call: the wrapped client appends one usage row per call
(`prompt_tokens`, `prompt_cache_hit_tokens`, `completion_tokens`), rows are absorbed even when
the call raises, and they are priced by the Model's price list. An unpriced Model cannot run
under a `usd` budget, and `summary()` reports `usd: None` for it, never 0. Model-free tasks run
with `{usd: 0, calls: 0}`, so a stray call fails loudly.
A Run is a DAG (a graph of steps with no cycles) of ax Tasks. Each Task binds at most one Model.
Each Task has its own `budget: {usd, calls}` ([The plane on ax](plane.md)).
`ousast plane status RUN` is the attribution table: one row per task with `calls`, `prompt`,
`cache_hit`, `output` and `usd` from each task's `summary.json`, a `subtotal` row per Model and a
`total` (a missing value makes a sum `n/a`, never a smaller number); `--units` adds the per-unit
rows. The same table is written as `attribution.json` in the run directory
(`src/openultrasast/plane/reconciler.py`, `attribution()`/`status()`). Provider credentials
reach a task only in the start request: `router.py` reads the variable the Model's
`secretKey.key` names, the runner keeps the value in memory, passes it to the command's
environment and redacts it from echoed stderr; it never appears in a manifest, a rendered Task
or a file.
The metered client enforces the budget (`src/openultrasast/plane/budget.py`, `MeteredClient`):
1. `check()` runs before every call.
2. It raises `BudgetExhausted` once the spent `usd` or the call count reaches the ceiling.
3. So the call that crosses the ceiling completes, and the next one is refused.
4. The task then ends `unfinished`. A rerun with a larger budget resumes from its `units.jsonl`.
An HTTP 401/402 or "Insufficient Balance" is an `AccountError`. It fails the task and starts
nothing further.
Usage is attributed per call:
- The wrapped client appends one usage row per call (`prompt_tokens`,
`prompt_cache_hit_tokens`, `completion_tokens`).
- Rows are absorbed even when the call raises.
- Rows are priced by the Model's price list.
- An unpriced Model cannot run under a `usd` budget. `summary()` reports `usd: None` for it,
never 0.
- Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
`ousast plane status RUN` prints the attribution table. It reads each task's `summary.json`
and shows:
- one row per task with `calls`, `prompt`, `cache_hit`, `output` and `usd`;
- a `subtotal` row per Model;
- a `total`. A missing value makes a sum `n/a`, never a smaller number.
`--units` adds the per-unit rows. The same table is written as `attribution.json` in the run
directory (`src/openultrasast/plane/reconciler.py`, `attribution()`/`status()`).
Provider credentials reach a task only in the start request:
- `router.py` reads the variable that the Model's `secretKey.key` names.
- The runner keeps the value in memory and passes it to the command's environment.
- The runner redacts it from echoed stderr.
- It never appears in a manifest, a rendered Task or a file.
### Pay once, reuse
- **Repo facts are computed model-free and reused.** `repo-facts` reads source text only (no
model, no git) and its `facts.json` is byte-identical across runs over the same tree. The
memory store keeps a facts entry under the key `(repo, pin, candidates digest, runner image
digest)` (`src/openultrasast/plane/memory.py`, `memory_key()`); when a later Run's task
carries the same key, `seed()` writes the stored `facts.json`, a `done` summary with
`calls: 0` and no `usd`, and marks the task done without starting an actor. A changed pin,
candidate set or image finds nothing and recomputes.
- **Content-addressed memory** (`src/openultrasast/learn/`). Excerpts are stored as
`excerpts/<sha256 of the normalised text>.txt`; an embedding lives at
`embeddings/<model>/<excerpt sha>.json` and "an excerpt is embedded once; a rerun makes no
call" (`embeddings.py`). Model responses are cached under the sha256 of
`{model, messages, params, temperature, sample}` (`program.py`, `request_key()`), and a
compiled program is `programs/<sha256 of its artifact>.json` with the digests of the memory
snapshot and of the response cache it was built from (`compile.py`).
- **Replay is free.** With no client, the response cache replays at $0 and a miss raises
- **Repo facts are computed model-free and reused.**
- `repo-facts` reads source text only (no model, no git).
- Its `facts.json` is byte-identical across runs over the same tree.
- The memory store keeps a facts entry under the key `(repo, pin, candidates digest, runner image
digest)` (`src/openultrasast/plane/memory.py`, `memory_key()`).
- A later Run's task may carry the same key. Then `seed()` writes the stored `facts.json` and a
`done` summary with `calls: 0` and no `usd`. It marks the task done without starting an
actor.
- A changed pin, candidate set or image finds nothing and recomputes.
- **Content-addressed memory** (`src/openultrasast/learn/`). Each item is stored under a hash
of its content.
- Excerpts are stored as `excerpts/<sha256 of the normalised text>.txt`.
- An embedding lives at `embeddings/<model>/<excerpt sha>.json`. The source says: "an excerpt
is embedded once; a rerun makes no call" (`embeddings.py`).
- Model responses are cached under the sha256 of `{model, messages, params, temperature, sample}`
(`program.py`, `request_key()`).
- A compiled program is `programs/<sha256 of its artifact>.json`. It records the digests of
the memory snapshot and of the response cache it was built from (`compile.py`).
- **Replay is free.** With no client, the response cache replays at $0. A miss raises
`ReplayMiss` instead of calling. The injection-slice record states it: "evaluate re-run
replay-only on the committed code: 550 responses replayed, $0, identical overall and
calibration" (`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`,
@@ -84,26 +113,35 @@ or a file.
### Cheap before expensive
- **Triage first.** Verification runs only over the candidates the recorded triage kept; the
plane applies that recording without a model call and prices it as a separate line
(`recorded_triage_usd`), so every per-candidate cost below is reported both with and without
it (`src/openultrasast/plane/tasks/agree.py`).
- **Triage first.** Verification runs only over the candidates the recorded triage kept.
- The plane applies that recording without a model call.
- It prices the recording as a separate line (`recorded_triage_usd`).
- So every per-candidate cost below is reported both with and without it
(`src/openultrasast/plane/tasks/agree.py`).
- **Hunts are batched per file.** `verify` runs one tool hunt per file for up to `PER_HUNT = 6`
candidates, with the candidates' known callers (capped at 8) in the prompt
candidates. The prompt includes the candidates' known callers, capped at 8
(`src/openultrasast/plane/tasks/verify.py`, `units_of()`).
- **The tie-break pass runs only on disputes.** `agree` compares passes a and b; pass c is
started with `inputs.only` set to the case's `disputed.json`, an empty list ends it `done`
with no model call, and the final verdict is 2-of-3 ("pass c alone never decides",
`agree.py`).
- **Cache-friendly prompt order.** The decision engine's prompt is the compiled prefix
(instruction and fixed demonstrations, identical for every candidate so the provider's
prefix cache hits), then the retrieved examples, then the candidate (`learn/program.py`).
Its self-consistency `k` is tied to that cache: "k = 5 when samples 2..k hit the prefix cache
for >= 0.8 of their prompt tokens"; the measured repeat-hit share was 0.96 (injection-slice
record, `smoke.k_rule`, `smoke.k5_repeat_hit_share_measured`). The verify hunts share a prefix
too: in the first plane increment 2,385,152 of 3,820,395 prompt tokens were cache hits
(`benchmarks/independent/plane-increment-1.json`, `detail.cache_hit_tokens`,
`detail.prompt_tokens`; its reading: "Cost fell 17% (62% of prompt tokens were cache hits)").
- **The tie-break pass runs only on disputes.**
- `agree` compares passes a and b.
- Pass c starts with `inputs.only` set to the case's `disputed.json`.
- An empty list ends it `done` with no model call.
- The final verdict is 2-of-3 ("pass c alone never decides", `agree.py`).
- **Cache-friendly prompt order.** The decision engine's prompt has three parts, in this order
(`learn/program.py`):
1. The compiled prefix: instruction and fixed demonstrations. It is identical for every
candidate, so the provider's prefix cache hits.
2. The retrieved examples.
3. The candidate.
The self-consistency `k` (the number of samples per candidate) is tied to that cache: "k = 5
when samples 2..k hit the prefix cache for >= 0.8 of their prompt tokens". The measured
repeat-hit share was 0.96 (injection-slice record, `smoke.k_rule`,
`smoke.k5_repeat_hit_share_measured`).
The verify hunts share a prefix too. In the first plane increment, 2,385,152 of 3,820,395
prompt tokens were cache hits (`benchmarks/independent/plane-increment-1.json`,
`detail.cache_hit_tokens`, `detail.prompt_tokens`). The record's reading: "Cost fell 17% (62%
of prompt tokens were cache hits)".
### Measured costs
@@ -116,37 +154,41 @@ or a file.
| Decision engine harvest: `verify` part | $2.5373 over 1,961 calls, 474 tasks done, 10,921,167 prompt tokens of which 6,275,060 cache hits, 306,941 output tokens, task budgets summing to $8.624 under a $10 ceiling (`verify`) | `benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json` |
| Decision engine harvest: `roles` part | $1.4036 over 1,034 calls, 151 tasks done, 2,398,669 prompt tokens of which 329,600 cache hits, 370,155 output tokens, task budgets summing to $4.917 under a $10 ceiling (`roles`) | same record |
The increment-2 record's note on the two denominators: "the reference divides by the 46
The increment-2 record notes that the two denominators differ: "the reference divides by the 46
candidates before triage; on that basis the plane is 7% cheaper, on the 43 after triage it is
0.2% under".
!!! note "The meter prices at list; the account moved less"
The attribution table prices every call at the Model's list price
(`plane/models/deepseek-flash.yaml`). The harvest record: "Spend is the plane's attribution
table (`ousast plane status`, task meters priced at plane/models/deepseek-flash.yaml); the
DeepSeek account moved less (balance below)", with the balance at $19.63 before, $18.75
after `verify` and $18.23 after `roles`. The injection-slice record names the ratio: "the
provider charged ~1/3 of the list-price meter (balance moved $0.30 for $0.91 metered):
DeepSeek's off-peak discount window, presumably; the meter prices at list"
(`spend.balance_note`, balance $18.2 to $17.9). Budgets and gates are set against the
meter, never against the balance.
(`plane/models/deepseek-flash.yaml`). The harvest record says: "Spend is the plane's
attribution table (`ousast plane status`, task meters priced at
plane/models/deepseek-flash.yaml); the DeepSeek account moved less (balance below)".
The balance was $19.63 before, $18.75 after `verify` and $18.23 after `roles`.
The injection-slice record names the ratio: "the provider charged ~1/3 of the list-price
meter (balance moved $0.30 for $0.91 metered): DeepSeek's off-peak discount window,
presumably; the meter prices at list" (`spend.balance_note`, balance $18.2 to $17.9).
Budgets and gates are set against the meter, never against the balance.
### Not done yet
- **No model-side token budget in ax itself.** The plane's requirements state what ax does
not yet carry: "task dependencies, artifacts, token budgets (on ax's roadmap), a DeepSeek
provider"; the thin `Run` layer carries them and dissolves into ax as ax gains them. Until
then the ceiling lives in the metered client, not in the executor.
- **Triage is applied, not run.** The plane has no triage task; it filters by the recorded
triage and adds that recording's cost as its own line. A triage task is planned as a later
specification.
- **`k` and the contrast examples are not knobs.** `k` is a compiled setting in {1, 3, 5}
(`learn/compile.toml`, "k and lam remain A/B variants"), changed only through the decision
engine's A/B experiments, its requirement that "changes to prompts, models, pass counts, rule
sets and features [are] compared by controlled experiments, so that nothing is adopted on one
run's number" (Requirement 4 of its specification). The retrieval cap of at most two
contrast examples from other families is a module constant (`learn/retrieve.py`,
`CONTRAST = 2`), a maintainer decision, not an experiment variant yet.
- **No model-side token budget in ax itself.**
- The plane's requirements state what ax does not yet carry: "task dependencies, artifacts,
token budgets (on ax's roadmap), a DeepSeek provider".
- The thin `Run` layer carries them. It dissolves into ax as ax gains them.
- Until then the ceiling lives in the metered client, not in the executor.
- **Triage is applied, not run.**
- The plane has no triage task.
- It filters by the recorded triage and adds that recording's cost as its own line.
- A triage task is planned as a later specification.
- **`k` and the contrast examples are not knobs.**
- `k` is a compiled setting in {1, 3, 5} (`learn/compile.toml`, "k and lam remain A/B
variants").
- It changes only through the decision engine's A/B experiments. Requirement 4 of its
specification asks that "changes to prompts, models, pass counts, rule sets and features
[are] compared by controlled experiments, so that nothing is adopted on one run's number".
- Retrieval adds at most two contrast examples from other families. This cap is a module
constant (`learn/retrieve.py`, `CONTRAST = 2`).
- The cap is a maintainer decision, not an experiment variant yet.
## B. Development: the maintainers' own sessions
@@ -165,28 +207,47 @@ flowchart LR
### The measurement: tool I/O dominates
`benchmarks/dev/token_report.py` reads a Claude Code session transcript (`.jsonl`; without an
argument, the newest under `~/.claude/projects/`) and reports, per content kind, characters and
share: `tool_use_input` (the tool call's input as JSON), `tool_result`, `thinking`,
`assistant:text`, `user:text`, plus the tool-call count and the top five tools. Tokens are
approximated at four characters each. It prints to stdout and writes nothing; what gets
committed is the counts-only summary. The one in `benchmarks/independent/plane-increment-1.json`
(`dev_token_report`, for "the whole development session (c09e30b2), not the increment alone"):
7.3M unique characters (~1.81M tokens), `tool_use_input_pct` 45.4, `tool_result_pct` 40.1,
`tool_io_pct` 85.5 against `previous_tool_io_pct` 90, over 2,701 tool calls. The reasoning the
session is paid for (assistant text and thinking) is the small remainder. That is why the
remedies below target file contents, not prose.
`benchmarks/dev/token_report.py` reads a Claude Code session transcript (`.jsonl`). Without an
argument it reads the newest one under `~/.claude/projects/`.
It reports characters and share per content kind:
- `tool_use_input` (the tool call's input as JSON);
- `tool_result`;
- `thinking`;
- `assistant:text`;
- `user:text`.
It also reports the tool-call count and the top five tools. Tokens are approximated at four
characters each. It prints to stdout and writes nothing. Only the counts-only summary is
committed.
The committed summary is in `benchmarks/independent/plane-increment-1.json`
(`dev_token_report`). It covers "the whole development session (c09e30b2), not the increment
alone". It counts 7.3M unique characters (~1.81M tokens). The shares
are `tool_use_input_pct` 45.4, `tool_result_pct` 40.1, and `tool_io_pct` 85.5 against
`previous_tool_io_pct` 90, over 2,701 tool calls.
The reasoning the session is paid for (assistant text and thinking) is the small remainder.
That is why the remedies below target file contents, not prose.
### The remedies in `benchmarks/dev/`
**`read_guard.py`**, a `PreToolUse` hook for agentic coding sessions. It reads the hook event on
stdin and, for the `Read` tool only, denies a whole-file read of a file longer than 350 lines
(`LIMIT = 350`); a read with `offset` or `limit`, a missing path, an unreadable file or a file
within the limit passes. It always exits 0 ("never fails the tool"). A denial is one JSON line
whose `permissionDecisionReason` tells the agent to read a range or to ask the bulk reader. The
script's docstring records that it is wired in `.claude/settings.local.json` (a local, untracked
file) as `PreToolUse` with matcher `Read`; the setting has this shape, with the command
repo-relative because hooks run from the project directory:
**`read_guard.py`** is a `PreToolUse` hook (a script that runs before each tool call) for
agentic coding sessions.
- It reads the hook event on stdin.
- For the `Read` tool only, it denies a whole-file read of a file longer than 350 lines
(`LIMIT = 350`).
- These pass: a read with `offset` or `limit`, a missing path, an unreadable file, or a file
within the limit.
- It always exits 0 ("never fails the tool").
- A denial is one JSON line. Its `permissionDecisionReason` tells the agent to read a range or
to ask the bulk reader.
The script's docstring records how it is wired. It sits in `.claude/settings.local.json` (a
local, untracked file) as `PreToolUse` with matcher `Read`. The setting has this shape. The
command is repo-relative because hooks run from the project directory:
```json
{
@@ -203,37 +264,48 @@ repo-relative because hooks run from the project directory:
}
```
**`bulk_read.py`**, the portal pattern: a cheap model reads the files and answers one question,
citing `path:line` for every claim; it reports what the files say and decides nothing.
**`bulk_read.py`** follows the portal pattern. A cheap model reads the files and answers one
question. It cites `path:line` for every claim. It reports what the files say and decides
nothing.
```text
.venv/bin/python benchmarks/dev/bulk_read.py "<question>" FILE [FILE ...] [--max-usd 0.05]
```
Files go to the model in 400-line chunks with a `--- path (lines a-b)` header and numbered
lines; the client is the project's detector client (`deepseek-flash` through `DEEPSEEK_API_KEY`,
otherwise OpenRouter), each call has a 90 s timeout and no tools. `--max-usd` (default 0.05) is
checked before every chunk; at the ceiling the answer ends with
`[stopped at the $X ceiling before: <chunk header>]`. The answer goes to stdout, the summary
`[bulk_read: N chunk(s), $X.XXXX]` to stderr. Without a chat client it exits 2.
How it works:
- Files go to the model in 400-line chunks, with a `--- path (lines a-b)` header and numbered
lines.
- The client is the project's detector client: `deepseek-flash` through `DEEPSEEK_API_KEY`,
otherwise OpenRouter.
- Each call has a 90 s timeout and no tools.
- `--max-usd` (default 0.05) is checked before every chunk. At the ceiling the answer ends with
`[stopped at the $X ceiling before: <chunk header>]`.
- The answer goes to stdout. The summary `[bulk_read: N chunk(s), $X.XXXX]` goes to stderr.
- Without a chat client it exits 2.
**`token_report.py`**: `python benchmarks/dev/token_report.py [TRANSCRIPT.jsonl]`, described
above.
### Practices
- **Scripts into files, not heredocs.** A heredoc is `tool_use_input` and its output
`tool_result`; both are paid again at every later turn of the session. A script on disk is
read once, by the interpreter.
- **Terse outputs.** Commands print counts, exit codes and the lines that matter; a harness
fails loudly with the exit code rather than printing a table that has to be read back.
- **Subagents with worktrees** for work whose output need not enter the main context: a
subagent's reads and command output stay in its own context and only its conclusion comes
back; a git worktree keeps its edits off the main checkout.
- **Scripts into files, not heredocs.** A heredoc (a script typed inline in a shell command) is
`tool_use_input`, and its output is `tool_result`. Both are paid again at every later turn of
the session. A script on disk is read once, by the interpreter.
- **Terse outputs.** Commands print counts, exit codes and the lines that matter. A harness
fails loudly with the exit code. It does not print a table that has to be read back.
- **Subagents with worktrees** for work whose output need not enter the main context.
- A subagent's reads and command output stay in its own context.
- Only its conclusion comes back.
- A git worktree (a separate checkout of the same repository) keeps its edits off the main
checkout.
- **Counts-only measurement records.** A record under `benchmarks/measurements/` or
`benchmarks/independent/` holds numbers and their keys, never code, prompts or responses, so
it can be quoted on a page like this one and reread by a model at a few hundred tokens.
`benchmarks/independent/` holds numbers and their keys. It never holds code, prompts or
responses. So it can be quoted on a page like this one. A model can reread it at a few
hundred tokens.
Related: [The plane on ax](plane.md) for the Run and Task contract, [Memory](memory.md) for
the store the reuse keys live in, [Evaluation](evaluation.md) for the gates these costs are
measured against.
Related:
- [The plane on ax](plane.md) for the Run and Task contract;
- [Memory](memory.md) for the store the reuse keys live in;
- [Evaluation](evaluation.md) for the gates these costs are measured against.
+261 -176
View File
@@ -1,33 +1,50 @@
# Where we stand
State as of 2026-10-02 (v2.0.0). This page answers five questions in one place. How far is
detection from its gates. How does the project now try to close that gap. How does the model work
run on ax. What did the tokens cost before and after that move. What is still open. Every figure
is quoted from a committed record whose path stands next to it; detail tables stay on the pages
linked from each section.
State as of 2026-10-02 (v2.0.0). This page answers five questions in one place:
The goal has not changed: a safety net on repositories the tool has never seen, which says
BLOCK or ADVISORY about the code a push would change and stays silent on fixes and on benign
commits. Two independent populations have been spent against that goal, with engine recall 1 of
1. How far is detection from its gates?
2. How does the project now try to close that gap?
3. How does the model work run on ax?
4. What did the tokens cost before and after that move?
5. What is still open?
Every figure is quoted from a committed record. The record's path stands next to the figure.
Detail tables stay on the pages linked from each section.
The goal has not changed. The tool should be a safety net on repositories it has never seen. It
should say BLOCK or ADVISORY about the code a push would change. It should stay silent on fixes
and on benign commits.
Two independent populations have been spent against that goal. An independent population is a
set of cases frozen before any scan. Engine recall was 1 of
11 and then 0 of 17. The hand edits that closed misses on the first population did not carry to
the second. So the question is no longer which rule to add next. It is how a change to detection
can be shown to work on code it was not tuned on, at a cost per candidate that can be paid for
every push.
the second. So the question is no longer which rule to add next. The question is this: how can a
change to detection be shown to work on code it was not tuned on? And can that be done at a cost
per candidate that can be paid for every push?
The answer this page records: the pre-push safety net is NO-GO against every M4 gate, and the
project has stopped trying to reach the gates by hand. Every detection change now has to win a
controlled measurement on held-out repositories, run on an agentic plane whose cost per candidate
is known. The pieces of that method exist and have been measured. None of them has produced a
BLOCK decision yet.
The answer this page records:
- The pre-push safety net is NO-GO against every M4 gate.
- The project has stopped trying to reach the gates by hand.
- Every detection change now has to win a controlled measurement on held-out repositories. The
measurement runs on an agentic plane whose cost per candidate is known.
- The pieces of that method exist and have been measured. None of them has produced a BLOCK
decision yet.
## 1. Detection capability and goals: NO-GO, measured
No M4 gate is met on an independent population. The qualification milestone for the pre-push
safety net (M4) requires, together: at least 95% actionable precision, at least 90% supported
recall, at least 95% supported-check completion, the runtime gates, and zero fixed-side or benign
false alerts. A population is frozen before any
case is scanned, scored once under a pre-registered protocol, then spent; it can inform changes
but can qualify nothing afterwards ([Evaluation](evaluation.md#independent-populations-and-the-m4-gates)).
No M4 gate is met on an independent population. M4 is the qualification milestone for the
pre-push safety net. It requires all of these together:
- at least 95% actionable precision;
- at least 90% supported recall;
- at least 95% supported-check completion;
- the runtime gates;
- zero fixed-side or benign false alerts.
A population is frozen before any case is scanned. It is scored once under a pre-registered
protocol, then spent. After that it can inform changes but can qualify nothing
([Evaluation](evaluation.md#independent-populations-and-the-m4-gates)).
| Gate | Required | Latest independent result | Record |
| --- | --- | --- | --- |
@@ -36,156 +53,217 @@ but can qualify nothing afterwards ([Evaluation](evaluation.md#independent-popul
| Supported-check completion | >= 95% | 13,206 of 108,374 | `results-v2.json` |
| Fixed-side and benign alerts | zero | 0 fixed-side, which the record calls met trivially since nothing was detected; 3 benign alerts | `results-v2.json` |
Three later measurements narrow the picture without changing the verdict.
Three later measurements narrow the picture. They do not change the verdict.
- The plane reproduces the script pipeline's agreement at a known cost. On the model-driven
pipeline's validation set (46 candidates, 15 cases of v2, 20 declared vulnerable sites), the
plane's verify passes a and b plus a tie-break on the 10 disputed candidates agreed on 16 of 20
declared sites. The script reference agreed on 16 as well. The cost was $0.0205 per candidate
(`benchmarks/independent/plane-increment-2.json`). The record carries two caveats. The tie-break
was chosen after seeing the first measurement on the same set, so this compares plane with
script and qualifies nothing. And the passes flag fewer declared sites than the reference in
any pass, 17 against 19; that count is a correction in `plane-increment-1.json`, whose first
reading took agree's `declared_sites_matched` (19, coverage by candidates) for "found in either
pass".
- The learned decision engine ranks, but may not block. On six families, evaluated on
repositories the compiled program was not shown, the pooled AUC of its score runs from 0.77
(injection) to 0.96 (access control). The stricter reading is the within-pair AUC, which asks
whether the vulnerable side of a fix scores above its own fixed side: 0.72 to 0.93. BLOCK is
offered for no family; calibration holds only for output encoding, and there no fold reaches
the required precision (`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
The per-family table is on [Decision engine](decision-engine.md#second-slice-six-families-2026-10-02).
- PHP coverage is in-sample. The PHP quick rules were written and tightened while looking at
the pins and fixtures they are scored on, so the record says so itself (`provenance.in_sample` in
`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`): a development check, not
a recall estimate. On real PHP code the Joern engine is the instrument.
- **The plane reproduces the script pipeline's agreement at a known cost.** The test used the
model-driven pipeline's validation set: 46 candidates, 15 cases of v2, 20 declared vulnerable
sites. The plane ran verify passes a and b, plus a tie-break on the 10 disputed candidates. It
agreed on 16 of 20 declared sites. The script reference agreed on 16 as well. The cost was
$0.0205 per candidate (`benchmarks/independent/plane-increment-2.json`). The record carries two
caveats:
1. The tie-break was chosen after seeing the first measurement on the same set. So this compares
plane with script and qualifies nothing.
2. The passes flag fewer declared sites than the reference in any pass: 17 against 19 (passes against reference). That
count is a correction in `plane-increment-1.json`. Its first reading took agree's
`declared_sites_matched` (19, coverage by candidates) to mean "found in either pass".
- **The learned decision engine ranks, but may not block.** It was evaluated on six families, on
repositories the compiled program was not shown.
- The pooled AUC (area under the ROC curve) of its score runs from 0.77 (injection) to 0.96
(access control).
- The stricter reading is the within-pair AUC. It asks whether the vulnerable side of a fix
scores above its own fixed side: 0.72 to 0.93.
- BLOCK is offered for no family. Calibration holds only for output encoding. There, no fold
reaches the required precision
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
- The per-family table is on
[Decision engine](decision-engine.md#second-slice-six-families-2026-10-02).
- **PHP coverage is in-sample.** The PHP quick rules were written and tightened while looking at
the pins and fixtures they are scored on. The record says so itself (`provenance.in_sample` in
`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). It is a development
check, not a recall estimate. On real PHP code, the Joern engine is the instrument.
What has not been touched: population v3, 15 PHP cases, frozen as the one-time final check. No
development or analysis work reads its cases, a test fails when anything in the tree names one
of its repositories, and it runs once, after a prediction is committed.
What has not been touched: population v3, 15 PHP cases, frozen as the one-time final check.
- No development or analysis work reads its cases.
- A test fails when anything in the tree names one of its repositories.
- It runs once, after a prediction is committed.
## 2. Our approach to detection, step by step
The path changed because hand edits did not transfer. Every gain before 2026-09-30 was a hand
edit after a miss: a rule, a taint vocabulary entry, a guard, a prompt line, a tie-break. Each was
tuned on the cases that exposed it, and the tuning measured on population v1 did not carry to v2
(1 of 11, then 0 of 17). The conclusions drawn: hand edits do not transfer. The data that could
teach a better decision cannot be produced by hand at the scale needed, so it has to be harvested
and labelled by machinery with a boundary around it. Framework knowledge (which function is a sink in which
framework) is useful as a prior but must stay optional, or the engine learns the vocabulary
instead of the code. A reader might object that a classifier is slower to show gains than the
next rule. It is. The difference is that its gains are measured where rules were not: on
repositories it never saw.
tuned on the cases that exposed it. The tuning measured on population v1 did not carry to v2
(1 of 11, then 0 of 17).
The chain, one job per step. The design is on [Memory and detection](memory-and-detection.md);
this is the order of operations.
The conclusions drawn:
1. Instruments produce signals, never verdicts. Quick rules, the Joern engine, model sink
classification, the plane's verify passes a and b with the tie-break c, repository facts
(callers, spans) and model source/sink/sanitizer roles each write one block of a candidate's
record. None decides alone.
2. Feature records are allow-listed, and missing is not zero. A candidate is `(path, function,
family)` of one repository and pin. An instrument that did not run, had no coverage or failed
records that state; "silent" and "did not look" stay distinguishable. Input profile v1 withholds
the engine block and the function length, which a leak audit found separate the two sides of a
pair by construction (`benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`), and
since 2026-10-02 the plane's `verify.flag_b` and `ms.operation`, flagged the same way by the
audit of the plane profile (`2026-10-02-decision-engine-harvest-advisory/record.json`,
`src/openultrasast/learn/schema.py`).
3. Labels come from ground truth only. A fail-closed allow-list (`src/openultrasast/learn/sources.toml`)
names the sources: the spent populations' declared sites and fix ranges, the vulnerable and
fixed sides of pairs, including the 97 real public fixes in `benchmarks/pairs/advisory-fixes/`,
benign controls and recorded adjudications. Assumed-benign commits are their own source,
weighted 0.5 and reported separately. A plane verdict is a feature, never a label: "rejected by
the plane" is not a negative, or the engine would learn to reproduce the plane's mistakes.
4. Retrieval finds similar labelled cases. A Gower distance over the signal profile, re-ranked
by cosine similarity of code embeddings (OpenAI `text-embedding-3-small` through OpenRouter,
cached by excerpt hash). The evaluation boundary is the only door into a prompt: never the
candidate's own repository group, never a group the fold evaluates.
5. A classifier compiled in house, DSPy-style. `retrieve -> classify(k)`: an instruction and
compiled demonstrations (a fixed prefix, so the provider's cache hits), the retrieved examples,
then the candidate's excerpt and signals. The model answers vulnerable, not vulnerable or
unsure with a rationale that cites a line; `k` samples give a score. One program is compiled
per profile and family on a compile split that is never evaluated.
6. Calibration on held-out repositories. Platt scaling of the score, cross-fitted by
repository group so no map is checked on the fold that fitted it. Calibration holds when the
slope interval contains 1, the intercept interval contains 0 and ECE <= 0.05.
7. Two operating points. BLOCK only where calibration holds and a fold reaches the required
precision; otherwise ADVISORY. An unsure majority can never BLOCK.
8. A/B experiments are the only way a change is adopted. Prompts, models, pass counts,
features and sampling are compared under a pre-registered manifest with frozen units, paired
arms, a repository bootstrap and two looks. exp-002 is the first to finish: a retrieval
ensemble (five samples over disjoint example sets at temperature 0) against today's
temperature sampling. It raised re-run agreement a little (0.906 against 0.875, interval
spanning zero) and lowered the within-pair AUC from 0.815 to 0.741 at 2.4 times the cost per
candidate ($0.0051 against $0.0021). Decision: REJECTED
(`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`).
9. Extrapolation with intervals. Every number carries a 95% repository-cluster bootstrap
interval; the pooled and the within-pair readings are both reported, and the stricter one is
- Hand edits do not transfer.
- The data that could teach a better decision cannot be produced by hand at the scale needed. It
has to be harvested and labelled by machinery with a boundary around it.
- Framework knowledge (which function is a sink in which framework) is useful as a prior. But it
must stay optional. Otherwise the engine learns the vocabulary instead of the code.
A reader might object that a classifier is slower to show gains than the next rule. It is. The
difference is where its gains are measured: on repositories it never saw. Rules were not measured
there.
The chain has one job per step. The design is on [Memory and detection](memory-and-detection.md).
This list is the order of operations.
1. **Instruments produce signals, never verdicts.** Each instrument writes one block of a
candidate's record. None decides alone. The instruments are:
- quick rules;
- the Joern engine;
- model sink classification;
- the plane's verify passes a and b, with the tie-break c;
- repository facts (callers, spans);
- model source/sink/sanitizer roles.
2. **Feature records are allow-listed, and missing is not zero.** A candidate is `(path, function,
family)` of one repository and pin.
- An instrument that did not run, had no coverage or failed records that state. "Silent" and
"did not look" stay distinguishable.
- Input profile v1 withholds the engine block and the function length. A leak audit found that
they separate the two sides of a pair by construction
(`benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`).
- Since 2026-10-02 it also withholds the plane's `verify.flag_b` and `ms.operation`. The audit
of the plane profile flagged them the same way
(`2026-10-02-decision-engine-harvest-advisory/record.json`,
`src/openultrasast/learn/schema.py`).
3. **Labels come from ground truth only.** A fail-closed allow-list
(`src/openultrasast/learn/sources.toml`) names the sources:
- the spent populations' declared sites and fix ranges;
- the vulnerable and fixed sides of pairs, including the 97 real public fixes in
`benchmarks/pairs/advisory-fixes/`;
- benign controls;
- recorded adjudications.
Assumed-benign commits are their own source. They are weighted 0.5 and reported separately. A
plane verdict is a feature, never a label. "Rejected by the plane" is not a negative.
Otherwise the engine would learn to reproduce the plane's mistakes.
4. **Retrieval finds similar labelled cases.** It uses a Gower distance (a distance for mixed
numeric and categorical data) over the signal profile. Results are re-ranked by cosine
similarity of code embeddings. The embeddings are OpenAI `text-embedding-3-small` through
OpenRouter, cached by excerpt hash. The evaluation boundary is the only door into a prompt. It
never admits the candidate's own repository group, or a group the fold evaluates.
5. **A classifier compiled in house, DSPy-style.** The program is `retrieve -> classify(k)`. The
prompt holds, in order:
- an instruction and compiled demonstrations (a fixed prefix, so the provider's cache hits);
- the retrieved examples;
- the candidate's excerpt and signals.
The model answers vulnerable, not vulnerable or unsure, with a rationale that cites a line. `k`
samples give a score. One program is compiled per profile and family, on a compile split that
is never evaluated.
6. **Calibration on held-out repositories.** Platt scaling (a logistic fit) maps the score to a
probability. It is cross-fitted by repository group, so no map is checked on the fold that
fitted it. Calibration holds when three conditions are met: the slope interval contains 1, the
intercept interval contains 0 and ECE <= 0.05.
7. **Two operating points.** BLOCK applies only where calibration holds and a fold reaches the
required precision. Otherwise the point is ADVISORY. An unsure majority can never BLOCK.
8. **A/B experiments are the only way a change is adopted.** Prompts, models, pass counts,
features and sampling are compared under a pre-registered manifest. The manifest fixes frozen
units, paired arms, a repository bootstrap and two looks. exp-002 is the first to finish:
- It compared a retrieval ensemble (five samples over disjoint example sets at temperature 0)
with today's temperature sampling.
- It raised re-run agreement a little (0.906 against 0.875, interval spanning zero).
- It lowered the within-pair AUC from 0.815 to 0.741 and cost 2.4 times as much per candidate
($0.0051 against $0.0021).
- Decision: REJECTED (`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`).
9. **Extrapolation with intervals.** Every number carries a 95% repository-cluster bootstrap
interval. Both the pooled and the within-pair readings are reported. The stricter one is
named.
10. The one-time v3 check. When a program is adopted and a prediction is committed, population
v3 is scored once under `benchmarks/independent/protocol-v3.md`.
10. **The one-time v3 check.** It happens when a program is adopted and a prediction is
committed. Then population v3 is scored once under `benchmarks/independent/protocol-v3.md`.
## 3. AI integration: ax on the server
All model work runs as Tasks on google/ax over Agent Substrate, in a single-node kind cluster on
the maintainer's host, each Task in its own gVisor sandbox. There is no local subprocess path.
The reconciler (`ousast plane run`) is a host process: it applies a Run's Workspaces and Tasks,
resumes each Task, sends its start signal, collects its artifacts and records state. Flow and
contract: [The plane on ax](plane.md); bring-up and the host's lessons: [ax on this host](ops/ax/README.md).
All model work runs as Tasks on google/ax over Agent Substrate. They run in a single-node kind
cluster on the maintainer's host. Each Task runs in its own gVisor sandbox. There is no local
subprocess path.
The reconciler (`ousast plane run`) is a host process. It does five things:
1. applies a Run's Workspaces and Tasks;
2. resumes each Task;
3. sends its start signal;
4. collects its artifacts;
5. records state.
Flow and contract: [The plane on ax](plane.md). Bring-up and the host's lessons:
[ax on this host](ops/ax/README.md).
What each Run guarantees:
- One Model and one budget per Task. A Task binds at most one ax Model (`deepseek-flash` for
chat, `openrouter-embedding` for embeddings) and `budget: {usd, calls}`. The metered client
refuses the next call at the ceiling; the task ends unfinished and resumes on a rerun. Model-free
tasks run with a zero budget, so a stray call fails loudly.
- Credentials exist only in the start request. Agent Substrate boots each template once as a
golden actor, so the runner never starts by itself. It waits for `POST /ousast/v1/start`, which
the reconciler sends through `atenet-router` to that actor alone. The key is read from the
operator's environment or `.env`, kept in memory by the runner, passed to the command and
redacted from echoed stderr; no manifest, rendered Task or file carries it.
- Per-task egress. One EgressPolicy per actor through the `atenet-egress` gateway, deny by
default, hostnames only: plain HTTP to the artifact receiver, TLS passthrough to the Workspaces'
Git hosts and the Model's declared hosts. Deleted with the actor.
- Artifacts come back through the gateway. The runner posts its output directory as a tar to
the receiver, an HTTP server inside the reconciler process on the host, reached through a Service
with an EndpointSlice to the host. That delivery is the task's completion; ax has no Completed
phase.
- Memory on S3-compatible RustFS. `OUSAST_MEMORY=s3://sast-memory` on a RustFS server. The
store verifies the bucket at every start (versioning, lifecycle, tags, S3 Select) and refuses to
start when anything is missing; there is no local or fetch-and-filter fallback. Each row kind
has a fixed set of queryable fields, enforced at write time
([Memory](memory.md#verify-at-startup-no-fallback), [RustFS setup](rustfs.md)).
- **One Model and one budget per Task.** A Task binds at most one ax Model and
`budget: {usd, calls}`. The Models are `deepseek-flash` for chat and `openrouter-embedding` for
embeddings. The metered client refuses the next call at the ceiling. The task ends unfinished
and resumes on a rerun. Model-free tasks run with a zero budget, so a stray call fails loudly.
- **Credentials exist only in the start request.** Agent Substrate boots each template once as a
golden actor. So the runner never starts by itself. It waits for `POST /ousast/v1/start`. The
reconciler sends that request through `atenet-router` to that actor alone.
- The key is read from the operator's environment or `.env`.
- The runner keeps it in memory and passes it to the command.
- It is redacted from echoed stderr.
- No manifest, rendered Task or file carries it.
- **Per-task egress.** Each actor gets one EgressPolicy through the `atenet-egress` gateway. It is
deny by default and uses hostnames only. It allows plain HTTP to the artifact receiver. It allows
TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts. It is deleted with
the actor.
- **Artifacts come back through the gateway.** The runner posts its output directory as a tar to
the receiver. The receiver is an HTTP server inside the reconciler process on the host. It is
reached through a Service with an EndpointSlice to the host. That delivery is the task's
completion; ax has no Completed phase.
- **Memory on S3-compatible RustFS.** The setting is `OUSAST_MEMORY=s3://sast-memory` on a RustFS
server.
- The store verifies the bucket at every start: versioning, lifecycle, tags, S3 Select.
- It refuses to start when anything is missing. There is no local or fetch-and-filter fallback.
- Each row kind has a fixed set of queryable fields, enforced at write time
([Memory](memory.md#verify-at-startup-no-fallback), [RustFS setup](rustfs.md)).
Lessons the host taught, each from a failed live run (the full list is on
[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run)):
- Substrate installs no WorkerPool, and actor images must be pinned by digest.
- A release `ax` CLI skews from the server; build it from the deployed checkout.
- A release `ax` CLI skews from the server. Build it from the deployed checkout.
- ax has no Completed phase, so completion is our artifact delivery.
- The golden boot means the runner must wait for a start request.
- The egress gateway dials the address the actor connected to, so the receiver is a Service
- The egress gateway dials the address the actor connected to. So the receiver is a Service,
dialled by ClusterIP with its name as Host.
- ax's snapshot bucket must exist in your S3 server. When it did not, deleted golden actors piled
up in `DELETING` and filled the disk during a 600-task harvest on 2026-10-01.
up in `DELETING`. They filled the disk during a 600-task harvest on 2026-10-01.
Deployment status, in the two words [Deployment](deployment.md) uses strictly. Implemented:
ax as the only executor, per-task EgressPolicy, credentials only in the start request, per-task
budgets and attribution, the S3 store verified at startup, local bring-up, doctor and smoke Run,
re-pinning generated Tasks to another registry. Planned, not done: deployment to any cluster
other than this host's kind cluster, a configurable kube context (`kind-` is hard-coded in
`doctor.py`, `egress.py`, `router.py`), the receiver as an in-cluster Deployment, the
reconciler in-cluster with Secrets injected, presigned-URL delivery to the store.
Deployment status uses two words, as [Deployment](deployment.md) does, strictly.
Implemented:
- ax as the only executor;
- per-task EgressPolicy;
- credentials only in the start request;
- per-task budgets and attribution;
- the S3 store verified at startup;
- local bring-up, doctor and smoke Run;
- re-pinning generated Tasks to another registry.
Planned, not done:
- deployment to any cluster other than this host's kind cluster;
- a configurable kube context (`kind-` is hard-coded in `doctor.py`, `egress.py`, `router.py`);
- the receiver as an in-cluster Deployment;
- the reconciler in-cluster with Secrets injected;
- presigned-URL delivery to the store.
## 4. Token economics, before and after
Before the plane, the script pipeline cost $0.022 per candidate for two verify passes
(`benchmarks/independent/verifier-batched-check-2026-09-29.json`), and the maintainers' own
coding sessions spent about 90% of their tokens on tool input and output rather than reasoning.
After it, the per-candidate costs below are what the records show. The mechanisms (one budget
per Task, pay-once reuse, cheap-before-expensive ordering, replay at $0) are on
[Token ergonomics](token-ergonomics.md); this table is the before and after.
Before the plane, two costs stood out:
- The script pipeline cost $0.022 per candidate for two verify passes
(`benchmarks/independent/verifier-batched-check-2026-09-29.json`).
- The maintainers' own coding sessions spent about 90% of their tokens on tool input and output
rather than reasoning.
After it, the per-candidate costs below are what the records show. The mechanisms are on
[Token ergonomics](token-ergonomics.md): one budget per Task, pay-once reuse,
cheap-before-expensive ordering and replay at $0. This table is the before and after.
| Item | Before | After | Record |
| --- | --- | --- | --- |
@@ -195,63 +273,70 @@ per Task, pay-once reuse, cheap-before-expensive ordering, replay at $0) are on
| Harvest of plane signals for the engine | none existed | `verify` $2.5373 over 1,961 calls and 474 tasks; `roles` $1.4036 over 1,034 calls and 151 tasks | `2026-10-01-decision-engine-harvest/record.json` |
| Development transcript, tool I/O share | 90% | 85.5% over 2,701 tool calls (`dev_token_report`) | `plane-increment-1.json` |
Two notes from the records. The meter prices every call at the Model's list price
(`plane/models/deepseek-flash.yaml`); the DeepSeek account moved less, about one third of the
metered amount on the injection slice ($0.30 for $0.91 metered), and budgets and gates are set
against the meter, never against the balance. And the development share barely moved: 90 to 85.5
is a small gain, and the remedies (a read guard, a bulk reader, scripts in files) target file
contents because that is where the tokens go.
Two notes from the records:
1. The meter prices every call at the Model's list price (`plane/models/deepseek-flash.yaml`). The
DeepSeek account moved less: about one third of the metered amount on the injection slice
($0.30 for $0.91 metered). Budgets and gates are set against the meter, never against the
balance.
2. The development share barely moved: 90 to 85.5 is a small gain. The remedies are a read guard,
a bulk reader and scripts in files. They target file contents, because that is where the
tokens go.
## 5. Open items
Five groups, each item with its record: the engine's own gates, data, measurement debt,
infrastructure, and instrument gaps.
There are five groups. Each item has its record. The groups are the engine's own gates, data,
measurement debt, infrastructure, and instrument gaps.
The engine's gates.
- BLOCK is reachable for no family: calibration fails for five (ECE 0.071 to 0.119) and the
sixth reaches no fold with a Wilson lower bound of 0.95 on precision (slice-2 record).
- BLOCK is reachable for no family. Calibration fails for five (ECE 0.071 to 0.119). The sixth
reaches no fold with a Wilson lower bound of 0.95 on precision (slice-2 record).
- Re-run agreement is below 0.9 for three families: injection 0.77, access control 0.83,
untrusted destination 0.87. exp-002 tried the obvious variance fix and lost on the ranking.
Data.
- Positives-only groups: 98 (family, group) pairs in the 904-example memory hold positive examples
and no negative. 94 of them exist because the fixed side no longer has the function (the vibe-py
pairs). No advisory-fixes group is among them
(`benchmarks/measurements/2026-10-02-decision-engine-negatives-gap/record.json`). There are no
single-label negatives, so every family's pooled AUC is inflated by positives that never had to
be told apart from their own fix (slice-2 record, `paired_groups_check`).
- The model has seen public fixes. A verify pass named the advisory's CVE id for a public fix, an
observation the engine's design specification (maintainer tooling) records as visible
memorisation. Excerpts now redact advisory ids and security-worded comments
(`src/openultrasast/learn/excerpt.py`), but labels drawn from public advisories are partly in the
model's training data, and the within-pair numbers carry that.
- PHP is covered by the engine on real code and by in-sample quick rules; the population that
and no negative.
- 94 of them exist because the fixed side no longer has the function (the vibe-py pairs).
- No advisory-fixes group is among them
(`benchmarks/measurements/2026-10-02-decision-engine-negatives-gap/record.json`).
- There are no single-label negatives. So every family's pooled AUC is inflated by positives
that never had to be told apart from their own fix (slice-2 record, `paired_groups_check`).
- The model has seen public fixes. A verify pass named the advisory's CVE id for a public fix.
The engine's design specification (maintainer tooling) records this as visible memorisation.
Excerpts now redact advisory ids and security-worded comments
(`src/openultrasast/learn/excerpt.py`). But labels drawn from public advisories are partly in
the model's training data, and the within-pair numbers carry that.
- PHP is covered by the engine on real code and by in-sample quick rules. The population that
would measure it (v3) is reserved.
Measurement debt.
- exp-001 (known callers) is registered and has not run; the model-free candidate generation on
the 41 development repositories that its units need is not built
- exp-001 (known callers) is registered and has not run. Its units need model-free candidate
generation on the 41 development repositories. That generation is not built
(`benchmarks/experiments/exp-001-known-callers/registration.json`).
- Leave-one-source-out, leave-one-framework-out and the memory-size curve points (0, 25, 50%)
have not run; one paired point exists (724 against 904 examples, injection, AUC 0.76 against 0.77).
- The plane-profile leak audit flagged `verify.flag_b` and `ms.operation`; they are withheld from
profile v1, and whether they track the label on unseen code has to be decided before any
plane-profile program runs (`2026-10-02-decision-engine-harvest-advisory/record.json`).
have not run. One paired point exists (724 against 904 examples, injection, AUC 0.76 against 0.77).
- The plane-profile leak audit flagged `verify.flag_b` and `ms.operation`. They are withheld from
profile v1. Before any plane-profile program runs, someone must decide whether they track the
label on unseen code (`2026-10-02-decision-engine-harvest-advisory/record.json`).
- Population v3 waits for `benchmarks/independent/prediction-v3.json`, which does not exist. It is
written only after a program is adopted.
Infrastructure.
- Deployment to a separate Kubernetes cluster is planned, not done; the code changes it needs are
- Deployment to a separate Kubernetes cluster is planned, not done. The code changes it needs are
listed with file paths on [Deployment](deployment.md#what-is-implemented-and-what-is-planned).
Instrument gaps.
- A missing provider key is a quiet path in a scan. The plane refuses to start a Task whose Model
names an unset variable (`plane/router.py`, `StartError`). A scan without a key asks the model
nothing, reports only the graph's entailed findings, and records `learning_endpoint_unavailable`
in `manifest.json` (`cli.py`, after `resolve_chat_endpoint` returns nothing); `report.md` has no
line for that reason yet, so the report does not say the model layer was skipped.
- A missing provider key is a quiet path in a scan.
- The plane refuses to start a Task whose Model names an unset variable (`plane/router.py`,
`StartError`).
- A scan without a key asks the model nothing and reports only the graph's entailed findings.
It records `learning_endpoint_unavailable` in `manifest.json` (`cli.py`, after
`resolve_chat_endpoint` returns nothing).
- `report.md` has no line for that reason yet. So the report does not say the model layer was
skipped.
+1
View File
@@ -1,5 +1,6 @@
site_name: OpenUltraSAST
site_description: An experimental static security analyser and repository safety net, with an agentic plane and a learned decision engine in development.
site_url: https://norandom.github.io/OpenUltraSAST/
repo_url: https://github.com/norandom/OpenUltraSAST
edit_uri: edit/main/docs/
docs_dir: docs
+9 -2
View File
@@ -291,13 +291,20 @@ The default capability registry is empty pending independent qualification. Inst
this interface does not make the project rollout-ready. Advisory is the default;
blocking and strict incomplete-coverage handling are separate explicit choices.
Put the installed `ousast` executable on the Git process's PATH. From this tool's
checkout, explicitly install into the desired repository:
The supported install is `ousast pre-push install [REPO] [--force] [--docker]`, shipped in
the wheel; `ousast pre-push uninstall [REPO]` removes it. The first-run guide, settings, skip
reasons and a CI recipe are in [Scanning](../scanning.md#first-run-install-the-hook-on-any-repository).
If `ousast` is not on the Git process's PATH, the hook prints one line and lets the push through.
From this tool's checkout, the script installer remains available:
```sh
ops/install-pre-push /absolute/path/to/repository
```
`ops/ousast-docker` is the hook-callable Docker wrapper (`OUSAST_COMMAND=/path/to/ops/ousast-docker`,
or installed beside the hook by `--docker`). It is a script, not a shell function, so Git can run
it. It passes the push input through and mounts the artifact directory at its own path.
The installer resolves Git's effective hook directory (including `core.hooksPath`),
leaves that configuration untouched, and refuses to overwrite any existing hook.
For an existing hook, inspect it and explicitly move it aside **before** installation:
+9 -6
View File
@@ -7,12 +7,15 @@ single-node kind cluster. ax is the only executor; `ousast plane run` submits, r
ops/ax/up.sh # idempotent: kind + registry, Substrate, ax, gVisor worker pool, egress gateway,
# receiver Service, runner image (digest pinned), ax CLI, smoke Task
ousast plane doctor # kind, Substrate, ax controller, runner image
ousast plane doctor # the profile's cluster (ops/k8s/profiles/kind.toml), Substrate, ax, runner image, memory store
ops/ax/smoke-run.sh # end to end: repo-facts on ax, artifacts back on the host, attribution table
ops/ax/down.sh # deletes the cluster and its registry
Tools live in `~/go/bin` (kind, ax, ko, kubectl-ate) and `~/.local/bin` (kubectl); sources and rendered files
in `~/.cache/ousast/ax-src/` (the runner digest pin is `runner-image` there).
in `~/.cache/ousast/ax-src/`. The cluster's context, registry and pool sizes come from the kind profile
(`ops/k8s/profiles/kind.toml`); `up.sh` writes the runner's digest pin to that profile's images file
(`ops/k8s/profiles/kind-images.json`), which the templates are re-pinned from and which is committed with them.
`up.sh` and `smoke-run.sh` run the checkout's code with `OUSAST_PYTHON` (default `.venv/bin/python`).
## How a Run executes
@@ -192,16 +195,16 @@ assume this laptop and must change first (checked 2026-10-02):
| Assumption | Where | Needed in a real cluster |
|---|---|---|
| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind gateway `172.19.0.1:18090` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it |
| Registry `localhost:5001`, rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), `doctor.py:45` (hard-coded check), the runner digest pin, the `image` of every template under `plane/tasks/` | a registry the workers can pull from; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks |
| kubectl context `kind-$KIND_CLUSTER_NAME` | `doctor.py:39`, `egress.py:115` and `:148`, `router.py:145`, `up.sh:24` | one configurable context (for example `OUSAST_KUBE_CONTEXT`); not yet configurable |
| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind network's gateway address on `OUSAST_ARTIFACT_PORT` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it |
| The kind-local registry (`registry` in `ops/k8s/profiles/kind.toml`), rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), the profile's images file, the `image` of every template under `plane/tasks/` | the `k3s` profile's registry; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks |
| kubectl context | `kube_context` in the profile, read by `doctor.py`, `egress.py`, `router.py` (plane-on-kubernetes 1.2); `up.sh` reads the same profile | the `k3s` profile's context; nothing else changes |
| ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET` in ax's `deploy/ax-server.yaml` points at the ax authors' GCS bucket | ax deploy manifest | your own bucket, set before deploying ax |
| Egress gateway applied by hand (agentgateway variant, no Rust build) | this README | the Substrate-installed gateway; per-task EgressPolicies work unchanged |
| Worker pool of 2 x 1 CPU / 1.5 GiB for the 7 GB host | `workerpool.yaml.tmpl` | sized to the cluster; `--workers` to match |
| Plane memory store and results under `~/ousast-results/` on the host | `reconciler.py` (`OUSAST_RESULTS`), `memory.py` (`OUSAST_MEMORY`) | a persistent volume or bucket shared by the reconciler; the `s3://` store already works against any reachable S3-compatible server |
The provider key already travels only in the start request through `atenet-router`, which works the same through
a port-forward to any cluster.
the kube-context tunnel to any cluster.
## What a larger ax deployment needs
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# Placement probe (plane-on-kubernetes task 1.5, design open question 2): does Agent Substrate place an actor on a
# worker that fits its memory request, or on any free worker of the sandbox class?
#
# Applies a second WorkerPool `probe-pool` (1 replica, 2 CPU / 3Gi) from ops/ax/workerpool.yaml.tmpl with the worker
# image the existing pool already built (no ko build), scales the default pool to one worker so the 7 GB host holds
# both, then runs one model-free repo-facts Task asking 2560Mi (limit 3Gi) and one asking 256Mi, recording after
# each resume which worker holds the actor (`kubectl ate get workers -o json`). Everything it created is removed
# on exit and the default pool is scaled back. Usage: ops/ax/probe-placement.sh OUT_DIR (the JSON evidence lands there).
set -euo pipefail
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
OUT="${1:?usage: probe-placement.sh OUT_DIR}"
mkdir -p "$OUT"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
CTX="$(profile kube_context)"
RUNNER_IMAGE="$(profile images.runner)"
DEFAULT_REPLICAS="$(profile pools.default.replicas)"
K() { kubectl --context "$CTX" -n ax-system "$@"; }
log() { printf '%s %s\n' "$(date +%H:%M:%S)" "$*" | tee -a "$OUT/probe.log"; }
cleanup() {
rc=$?
log "cleanup (exit $rc): delete the probe tasks and probe-pool, scale ousast-pool back to $DEFAULT_REPLICAS"
for t in ousast-probe-big ousast-probe-small; do ax delete task "$t" >/dev/null 2>&1 || true; done
K delete workerpool probe-pool --ignore-not-found >/dev/null 2>&1 || true
K patch workerpool ousast-pool --type merge -p "{\"spec\":{\"replicas\":$DEFAULT_REPLICAS}}" >/dev/null 2>&1 || true
echo "probe exit $rc"
}
trap cleanup EXIT
workers() { kubectl ate --context "$CTX" get workers -o json > "$OUT/$1.json"; kubectl ate --context "$CTX" get workers | tee -a "$OUT/probe.log"; }
export POOL_NAME=probe-pool POOL_REPLICAS=1 POOL_CPU=2 POOL_MEMORY=3Gi
WORKER_IMAGE="$(K get workerpool ousast-pool -o jsonpath='{.spec.workerImage}')" # already built and pulled by the node
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
export WORKER_IMAGE SUBSTRATE_VERSION
log "default pool to 1 replica (room for a 3Gi worker on this host)"
K patch workerpool ousast-pool --type merge -p '{"spec":{"replicas":1}}'
for i in $(seq 1 60); do [ "$(K get pods -l ate.dev/worker-pool=ousast-pool --no-headers 2>/dev/null | wc -l)" = 1 ] && break; sleep 2; done
log "apply probe-pool: $POOL_REPLICAS x $POOL_CPU CPU / $POOL_MEMORY, image $WORKER_IMAGE"
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" | tee "$OUT/probe-pool.yaml" | K apply -f -
K wait --for=condition=Ready pod -l ate.dev/worker-pool=probe-pool --timeout=600s
workers workers-before
task() { # name, request memory, limit memory
cat > "$OUT/$1.yaml" <<EOF
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata: {name: $1-ws, atespace: default}
spec:
files:
- {path: "hello.py", content: "def hello():\n return greet()\n"}
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata: {name: $1, atespace: default}
spec:
image: "$RUNNER_IMAGE"
command: ["repo-facts"]
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
resources: {requests: {cpu: "250m", memory: "$2"}, limits: {cpu: "1", memory: "$3"}}
workspaces: [{name: $1-ws, path: "/workspace"}]
EOF
ax delete task "$1" >/dev/null 2>&1 || true
log "apply $1 (requests $2, limits $3)"
ax apply -f "$OUT/$1.yaml"
for i in 1 2 3 4 5 6 7 8 9; do
if ax resume task "$1" 2>&1 | tee -a "$OUT/probe.log"; then break; fi # the first resume may wait for the golden snapshot
sleep 20
done
sleep 15
ax get task "$1" | tee "$OUT/$1.status.yaml" | grep -E "phase|message" || true
workers "workers-$1"
ax delete task "$1" >/dev/null 2>&1 || true
for i in $(seq 1 30); do kubectl ate --context "$CTX" get workers | grep -q " 1/1000 " || break; sleep 2; done
}
task ousast-probe-big 2560Mi 3Gi
task ousast-probe-small 256Mi 512Mi
log "done; evidence under $OUT"
+9 -5
View File
@@ -1,14 +1,18 @@
#!/usr/bin/env bash
# Render the end-to-end smoke Run with the current runner digest and execute it through the reconciler on ax.
# Render the end-to-end smoke Run with the profile's runner digest and execute it through the reconciler on ax.
# The profile (OUSAST_PLANE_PROFILE, default kind; ops/k8s/profiles/) names the images file up.sh wrote.
set -euo pipefail
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
RUNNER_IMAGE="$(cat "$SRC/runner-image")"; export RUNNER_IMAGE
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
PY="${OUSAST_PYTHON:-$REPO/.venv/bin/python}"
RUNNER_IMAGE="$(PYTHONPATH="$REPO/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print images.runner)"
export RUNNER_IMAGE
# Actors reach the receiver by its Service name through the egress gateway (ops/ax/receiver-service.yaml.tmpl,
# applied by up.sh, endpoint this host at OUSAST_ARTIFACT_PORT): the runner dials the Service's ClusterIP on port
# 80 with that name as Host; the reconciler looks the ClusterIP up unless OUSAST_ARTIFACT_DIAL is set.
export OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}"
envsubst < "$(dirname "$0")/smoke-run.yaml.tmpl" > "$SRC/smoke-run.yaml"
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
"$REPO/.venv/bin/ousast" plane run "$SRC/smoke-run.yaml" "$@"
"$REPO/.venv/bin/ousast" plane status ax-e2e-smoke
ousast() { PYTHONPATH="$REPO/src" "$PY" -m openultrasast.cli "$@"; } # this checkout's code, whatever python runs it
ousast plane run "$SRC/smoke-run.yaml" "$@"
ousast plane status ax-e2e-smoke
+30 -12
View File
@@ -1,13 +1,21 @@
#!/usr/bin/env bash
# Bring up ax on this host: kind cluster + local registry, Agent Substrate, the ax control plane, one smoke
# Task. Idempotent enough to rerun; `down.sh` removes everything. Requires docker, go, kubectl, kind, ko, ax.
# Task. Idempotent: every step is skipped when already done; `down.sh` removes everything. Requires docker, go,
# kubectl, kind, ko, ax. The cluster's context, the registry and the pool sizes come from the kind profile
# (ops/k8s/profiles/kind.toml; OUSAST_PLANE_PROFILE selects another file with exec = "local"). This script may
# name the kind cluster because it creates it; the code reads the profile (tests/test_plane_literals.py).
set -euo pipefail
trap 'rc=$?; echo "up.sh FAILED at line $LINENO (exit $rc)"' ERR
trap 'echo "up.sh exit $?"' EXIT
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
export GOTOOLCHAIN=auto
export KIND_CLUSTER_NAME="${KIND_CLUSTER_NAME:-ousast}"
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-localhost:5001}"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
CTX="$(profile kube_context)" # kind names its context kind-<cluster>
export KIND_CLUSTER_NAME="${CTX#kind-}"
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-$(profile registry)}"
IMAGES_FILE="$(profile images)" # the digest pins the reconciler and the templates read
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
for t in docker go kubectl kind ko ax; do
@@ -19,9 +27,8 @@ clone substrate https://github.com/agent-substrate/substrate.git
clone ax https://github.com/google/ax.git
step() { printf '\n== %s (%s)\n' "$1" "$(date +%H:%M:%S)"; }
step "kind cluster '$KIND_CLUSTER_NAME' + registry $KO_DOCKER_REPO"
step "kind cluster '$KIND_CLUSTER_NAME' (context $CTX) + registry $KO_DOCKER_REPO"
# Each step is skipped when already done: the substrate helper DELETES an existing cluster before creating one.
CTX="kind-$KIND_CLUSTER_NAME"
ready() { kubectl --context "$CTX" -n "$1" get pods --no-headers 2>/dev/null | awk '$2 ~ /^[0-9]+\/[0-9]+$/ {n++; split($2,a,"/"); if (a[1]!=a[2] && $3!="Completed") bad++} END {exit !(n>0 && bad==0)}'; }
if kind get clusters 2>/dev/null | grep -qx "$KIND_CLUSTER_NAME"; then
echo "cluster exists; kept"
@@ -56,25 +63,36 @@ KIND_GATEWAY="$(docker network inspect kind --format '{{range .IPAM.Config}}{{if
export KIND_GATEWAY OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}" # the reconciler's fixed receiver port
envsubst < "$(dirname "$0")/receiver-service.yaml.tmpl" | kubectl --context "$CTX" apply -f -
step "worker pool (gVisor)"
step "worker pool (gVisor): pools.default of the profile"
# `ko apply` builds the worker image from the Substrate checkout (several GB of Go caches): only when the pool
# is absent. A size change in the profile is applied with `kubectl patch` or by deleting the pool first.
export POOL_NAME=ousast-pool POOL_REPLICAS="$(profile pools.default.replicas)" POOL_CPU="$(profile pools.default.cpu)" POOL_MEMORY="$(profile pools.default.memory)"
export WORKER_IMAGE="ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor"
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
export SUBSTRATE_VERSION
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" > "$SRC/workerpool.yaml"
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l ate.dev/worker-pool=ousast-pool --timeout=600s \
if kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" >/dev/null 2>&1; then
echo "pool $POOL_NAME exists ($(kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" -o jsonpath='{.status.readyReplicas}/{.spec.replicas}') ready); kept"
else
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
fi
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l "ate.dev/worker-pool=$POOL_NAME" --timeout=600s \
|| kubectl --context "$CTX" -n ax-system get pods
step "runner image $KO_DOCKER_REPO/ousast-runner"
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
docker build -q -f "$REPO_ROOT/plane/Dockerfile.runner" -t "$KO_DOCKER_REPO/ousast-runner:dev" "$REPO_ROOT"
docker push -q "$KO_DOCKER_REPO/ousast-runner:dev"
RUNNER_IMAGE="$(docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' "$KO_DOCKER_REPO/ousast-runner:dev" | grep "^$KO_DOCKER_REPO/ousast-runner@" | head -1)"
[ -n "$RUNNER_IMAGE" ] || { echo "no digest for the runner image" >&2; exit 2; }
export RUNNER_IMAGE; echo "$RUNNER_IMAGE" > "$SRC/runner-image" # the reconciler reads this pin
echo "runner image $RUNNER_IMAGE"
export RUNNER_IMAGE
printf '{\n "runner": "%s"\n}\n' "$RUNNER_IMAGE" > "$IMAGES_FILE" # the profile's images file: commit it with the templates
echo "runner image $RUNNER_IMAGE -> $IMAGES_FILE"
echo "re-pin the templates with: ousast plane workspaces ... --runner-image $IMAGES_FILE (or ousast plane repin, task 4.3)"
step "ax CLI from the deployed checkout (a release CLI skews from the server)"
(cd "$SRC/ax" && go install ./cmd/ax)
if [ -x "$HOME/go/bin/ax" ] && [ "$HOME/go/bin/ax" -nt "$SRC/ax/.git/HEAD" ]; then echo "ax CLI newer than the checkout; kept"; else
(cd "$SRC/ax" && go install ./cmd/ax)
fi
step "smoke task"
ax delete task ousast-smoke >/dev/null 2>&1 || true
+12 -9
View File
@@ -1,23 +1,26 @@
# Warm gVisor workers for ax tasks on this host (Agent Substrate WorkerPool). ax's per-task ActorTemplates
# set no workerSelector, so every pool of the matching sandbox class is eligible. Two workers of 1 CPU /
# 1.5 GiB fit the 7 GB host beside the control planes; an actor occupies a whole worker.
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
apiVersion: ate.dev/v1alpha1
kind: WorkerPool
metadata:
name: ousast-pool
name: ${POOL_NAME}
namespace: ax-system
labels:
workload: ousast
spec:
replicas: 2
workerImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor
replicas: ${POOL_REPLICAS}
workerImage: ${WORKER_IMAGE}
template:
nodeSelector:
ate.dev/substrate-version: "${SUBSTRATE_VERSION}"
resources:
limits:
cpu: "1"
memory: 1536Mi
cpu: "${POOL_CPU}"
memory: ${POOL_MEMORY}
requests:
cpu: 250m
memory: 1536Mi
memory: ${POOL_MEMORY}
+26
View File
@@ -0,0 +1,26 @@
# The k3s profile: the plane on the maintainer's production cluster (plane-on-kubernetes design section 1;
# requirements "Target Environment"). Documented values, not secrets: the kubeconfig for the context, the GHCR
# pull secret's content and the store's keys live outside this file. Every field is overridable by OUSAST_<FIELD>,
# a pool by OUSAST_POOL_<NAME>_<FIELD>. Nothing has been applied to a k3s cluster yet; docs/deployment.md says
# which items are planned.
exec = "remote" # the same CLI from a laptop or a CI job, through the cluster's kube context
kube_context = "ousast-k3s" # the maintainer's context name for the k3s cluster
registry = "ghcr.io/norandom" # images built by GitHub Actions from tagged commits (design section 8)
images = "images.json" # the release asset {"runner": "ghcr.io/norandom/ousast-runner@sha256:...", "engine": ...}
router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel)
ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel)
memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from the CI secrets or .env
atespace = "default"
snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch)
image_pull_secret = "ghcr-pull" # only needed while the GHCR packages are private
[pools.default] # production sizing: four verify-class workers; the HPA (design section 5) scales it
replicas = 4
cpu = "1"
memory = "1536Mi"
[pools.engine] # two engine workers sized from the measured 2.1 GiB peak under a 3 GiB limit
replicas = 2
cpu = "2"
memory = "4Gi"
+3
View File
@@ -0,0 +1,3 @@
{
"runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
}
+25
View File
@@ -0,0 +1,25 @@
# The kind profile: the plane on this laptop's single-node kind cluster, which ops/ax/up.sh creates and which is
# the development default (plane-on-kubernetes design section 1). Values only; a credential never goes here
# (keys stay in the environment or .env). Every field is overridable by OUSAST_<FIELD> in the environment, a pool
# by OUSAST_POOL_<NAME>_<FIELD>. This file may name the kind cluster; the code must not (tests/test_plane_literals.py).
exec = "local" # the reconciler runs on this host against the cluster below
kube_context = "kind-ousast" # kind's context for KIND_CLUSTER_NAME=ousast
registry = "localhost:5001" # the kind-local registry up.sh creates (Substrate rewrites it for the node)
images = "kind-images.json" # digest-pinned image references, written by up.sh after each build
router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel)
ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel)
memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from .env
atespace = "default"
snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch)
image_pull_secret = "" # the kind registry needs none
[pools.default] # ousast-pool: two gVisor workers fit the 7 GB host beside the control planes
replicas = 2
cpu = "1"
memory = "1536Mi"
[pools.engine] # engine-pool (design section 4): one worker for the Joern engine task
replicas = 1
cpu = "2"
memory = "4Gi"
+61
View File
@@ -0,0 +1,61 @@
#!/bin/sh
# OpenUltraSAST in Docker, callable from a Git hook. A script, not a shell function: Git runs
# hooks non-interactively and never sees functions from ~/.bashrc.
#
# OUSAST_COMMAND=/path/to/ousast-docker git push # or: ousast pre-push install --docker
#
# The repository is mounted READ-ONLY at its own absolute path and the artifact directory
# read-write at its own path, so every path in the arguments and in the output ("Details: ...")
# means the same thing inside and outside the container. The push input on stdin is passed
# through (-i). The container runs as your user, with no network and --memory 3g.
#
# Advisory like the hook: when Docker or the image is missing, it says the check did not run
# and exits 0. Pull the image once first; a hook never downloads ~3 GB on its own:
#
# docker pull ghcr.io/norandom/openultrasast:2.0.1
#
# --engine background does not work through this wrapper: the container ends with the hook.
image=${OUSAST_IMAGE:-ghcr.io/norandom/openultrasast:2.0.1}
if ! command -v docker >/dev/null 2>&1; then
echo "ousast-docker: check did not run: docker is not installed. Push continues."
exit 0
fi
if ! docker image inspect "$image" >/dev/null 2>&1; then
echo "ousast-docker: check did not run: image $image is not present." \
"Run 'docker pull $image' once. Push continues."
exit 0
fi
repo=$(pwd -P)
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
previous=
for argument in "$@"; do
if [ "$previous" = "--artifact" ]; then
artifact_dir=$(dirname -- "$argument")
fi
previous=$argument
done
mkdir -p -- "$artifact_dir" || exit 0
artifact_dir=$(cd -- "$artifact_dir" && pwd -P)
# A linked worktree keeps its objects in the main repository's git directory: mount that too.
common=$(git rev-parse --path-format=absolute --git-common-dir 2>/dev/null || echo "$repo/.git")
case "$common" in
"$repo"/*) extra_mount= ;;
*) extra_mount="-v $common:$common:ro" ;;
esac
# shellcheck disable=SC2086 # OUSAST_DOCKER_ARGS and extra_mount are deliberately word-split
docker run --rm -i --network none --memory "${OUSAST_DOCKER_MEMORY:-3g}" \
--user "$(id -u):$(id -g)" -e HOME=/tmp -e TMPDIR=/work --tmpfs /work:size=4g \
-v "$repo:$repo:ro" -w "$repo" -v "$artifact_dir:$artifact_dir" $extra_mount \
${OUSAST_DOCKER_ARGS:-} "$image" "$@"
status=$?
if [ "$status" -ge 125 ]; then
echo "ousast-docker: check did not run: the container could not start (docker exit $status). Push continues."
exit 0
fi
# Only an explicit blocking opt-in may turn the container's exit into a rejected push; an older
# image that rejects a newer argument must not block either.
if [ "$status" -ne 0 ] && [ "${OUSAST_PUSH_MODE:-advisory}" != blocking ]; then
echo "ousast-docker: the check exited with status $status; push continues (advisory)."
exit 0
fi
exit "$status"
+32 -11
View File
@@ -1,17 +1,38 @@
#!/bin/sh
# Explicitly copy using the non-overwriting recipe in ops/README.md.
# A prior hook is chained only after you explicitly move it to this sibling path.
# OpenUltraSAST pre-push safety net. Advisory: nothing here rejects a push unless
# OUSAST_PUSH_MODE=blocking is set explicitly. Install with `ousast pre-push install`
# (or ops/install-pre-push from a checkout); remove with `ousast pre-push uninstall`.
# A prior hook is chained only after it was moved to the sibling path "$0.before-ousast".
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
prior="$0.before-ousast"
if [ -e "$prior" ] || [ -L "$prior" ]; then
exec ousast pre-push . --artifact "$artifact_dir/push-$$.json" \
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
--mode "${OUSAST_PUSH_MODE:-advisory}" \
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
--prior-hook "$prior" --remote "$1" "$2"
# The Docker wrapper installed beside this hook (`ousast pre-push install --docker`) is used
# unless OUSAST_COMMAND names another command; otherwise `ousast` from PATH.
if [ -z "${OUSAST_COMMAND:-}" ] && [ -x "$(dirname -- "$0")/ousast-docker" ]; then
OUSAST_COMMAND="$(dirname -- "$0")/ousast-docker"
fi
exec ousast pre-push . --artifact "$artifact_dir/push-$$.json" \
command=${OUSAST_COMMAND:-ousast}
if ! command -v "$command" >/dev/null 2>&1; then
echo "ousast: pre-push check did not run: '$command' is not on PATH." \
"Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues."
if [ -e "$prior" ] || [ -L "$prior" ]; then
exec "$prior" "$@"
fi
exit 0
fi
remote_name=$1
remote_url=$2
set -- pre-push . --artifact "$artifact_dir/push-$$.json" \
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
--mode "${OUSAST_PUSH_MODE:-advisory}" \
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
--remote "$1" "$2"
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}"
# Optional settings are passed only when set, so an older ousast (or image) still runs.
if [ -n "${OUSAST_PUSH_ENGINE:-}" ]; then
set -- "$@" --engine "$OUSAST_PUSH_ENGINE"
fi
if [ -n "${OUSAST_COMPARISON_BASE:-}" ]; then
set -- "$@" --comparison-base "$OUSAST_COMPARISON_BASE"
fi
if [ -e "$prior" ] || [ -L "$prior" ]; then
set -- "$@" --prior-hook "$prior"
fi
exec "$command" "$@" --remote "$remote_name" "$remote_url"
@@ -0,0 +1,353 @@
{"family": "deserialization", "fold": "outer-1", "group": "aio-libs/aiohttp", "label": 1, "pair": "ee577ee4b6076fab", "unit": "2523f55d035d1a772a842090bc335e5d755b2fe15b376dd5b6b9e30b9dae82d1"}
{"family": "deserialization", "fold": "outer-1", "group": "aio-libs/aiohttp", "label": 0, "pair": "ee577ee4b6076fab", "unit": "ca0f6c160d7839204f6fcc1709bcbc749a02a0b3b8bb648e2a238eacbf22ed78"}
{"family": "access_control", "fold": "outer-0", "group": "alchemycms/alchemy_cms", "label": 1, "pair": "c95b6b01d0c2db2a", "unit": "15540c23be0c42472a49039658ec2ec0bbd8c6b318e35dd040a9d427370e0ef0"}
{"family": "access_control", "fold": "outer-0", "group": "alchemycms/alchemy_cms", "label": 0, "pair": "c95b6b01d0c2db2a", "unit": "542471980417e964182d7d3db59a05fe12bdd072449f3bd77f7daa5515dbc909"}
{"family": "access_control", "fold": "outer-1", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "bcc636b23dfb5f570d8864b17fe07b6b4fb52b65a40988c32011f1422f74d504"}
{"family": "access_control", "fold": "outer-1", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "dbcf1e67a1be7ccf7d51701f9b8339339164055135443044c23a446b16b3d934"}
{"family": "injection", "fold": "outer-1", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "50408f68114ecb4de3bebd836d52d9afad86b22dd1843641ac8a49b617c09e33"}
{"family": "injection", "fold": "outer-0", "group": "anxolerd/dvpwa", "label": 1, "pair": "", "unit": "cafb0f00f80dc6beeba40eb95c764bf9a3569193070ba321c28677dfa425ec88"}
{"family": "output_encoding", "fold": "outer-3", "group": "apache/logging-log4net", "label": 1, "pair": "fe0e104d111c2555", "unit": "088036fab9962be2b96f9162ea4f32190575678d3c5ba96bdfeac178bc22b2c4"}
{"family": "output_encoding", "fold": "outer-3", "group": "apache/logging-log4net", "label": 0, "pair": "fe0e104d111c2555", "unit": "7fd2b56a816e25358477eee64487fb2c77f34c652703ad0a7120c368492017a3"}
{"family": "path", "fold": "outer-0", "group": "apache/wicket", "label": 1, "pair": "b6742570f7f0e026", "unit": "7c6c1d27aee6557f1b6bf371b8209003f0d2d1122cfa35b6629eb43f38769cf4"}
{"family": "path", "fold": "outer-0", "group": "apache/wicket", "label": 0, "pair": "b6742570f7f0e026", "unit": "df8300d84dd784d0be46969ea432ef97b8ce611bae7a896ed4f349bd4d6897db"}
{"family": "access_control", "fold": "outer-0", "group": "apolloconfig/apollo", "label": 1, "pair": "0f584bb97a154488", "unit": "222a84ffc7c58d45e8f10df221762b23bf20d0287ecd20f375558134928886ca"}
{"family": "access_control", "fold": "outer-0", "group": "apolloconfig/apollo", "label": 0, "pair": "0f584bb97a154488", "unit": "25e4c9d2941f59f072385a7bde3fd0bd76f9e986ed24b06ce81962ab447f5094"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "archivy/archivy", "label": 0, "pair": "bb3d7baed62e6bb7", "unit": "e8ae88fd2b677e3f64ac94bddb012e3b903afa294ecc87774d10087c0d50489c"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "archivy/archivy", "label": 1, "pair": "bb3d7baed62e6bb7", "unit": "f6b0d241891dc1d897d74d98b6ff7ada2c1f1ce7874ca32097630c06ca53e389"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "axllent/mailpit", "label": 1, "pair": "143b900b99a46cf4", "unit": "200f96cd784ff5c39d8bba22883f22bc8a7c87c6970a7c871802f48a45548337"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "axllent/mailpit", "label": 0, "pair": "143b900b99a46cf4", "unit": "7a0d62ec424849ec4681f4333fbb077e8239054d1e964bd99b40847ad7ccdb01"}
{"family": "injection", "fold": "outer-2", "group": "browserify/static-eval", "label": 1, "pair": "c12dbc8a50f7fe9b", "unit": "04beb8a0fd054b92c2dd0ed83907330fdee6124d86114df948ac72a297a7d51f"}
{"family": "injection", "fold": "outer-2", "group": "browserify/static-eval", "label": 0, "pair": "c12dbc8a50f7fe9b", "unit": "c5029016a6fd2e3f9df59dba0ce3c173bbbb0d2b1de22d151086e01e1b272119"}
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "0e213aeabacf4205db024ea3c557e720c3aba911424555784917e967d6404867"}
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 0, "pair": "e8bc65415bd42e43", "unit": "4920230ef773595a9fd9a92a090dff42b48499a1dc5fc8831f941dae55cb2538"}
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 1, "pair": "e8bc65415bd42e43", "unit": "49209a810f207eb61382e2ceeab7076936838a5652546f21c187d066e220160b"}
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "9e2964884cdf281fa3f18ee3481aa62f9a8a0a195d605da2df5bd71876fe311a"}
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 1, "pair": "", "unit": "f9cc6f1bbcd59f5dc0de8f96248f722ea0679f8afa47c53f70832151ce7616d3"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 0, "pair": "e69463f945f9a0f4", "unit": "5feace7b28e260175f80c682e9009025aea6fae3dc75409cabbb955b560d0372"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 1, "pair": "e69463f945f9a0f4", "unit": "799a4f5a5b9324e86c90752f0239f059984f44e8ff1ec0dd518d0d9413e34c66"}
{"family": "deserialization", "fold": "outer-2", "group": "cesargb/laravel-magiclink", "label": 0, "pair": "768fc9db8a75a351", "unit": "adc6527469618f72b1dd7776814717cbd0056a0c625e05bef74199171ce73f1a"}
{"family": "deserialization", "fold": "outer-2", "group": "cesargb/laravel-magiclink", "label": 1, "pair": "768fc9db8a75a351", "unit": "da47663a57e370a75cc65b810b8a380aa26d12dcabb1d9b3c02641398ff7d348"}
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 0, "pair": "fab4ce020049724b", "unit": "b9be90242be627aa46462143ac85940a906232ebc0b47776b5729475b27d215e"}
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 1, "pair": "fab4ce020049724b", "unit": "ece12ea7d87b4a12c4da30faed9eb530b51fc879c271b68ba5304b230cda8e79"}
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 1, "pair": "9126ba83905eb8b4", "unit": "d8dc50b7bb40bbcfc3f3d4f1f8d3a4d17eb1c662d5f759efd18656a482dd2e32"}
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 0, "pair": "9126ba83905eb8b4", "unit": "f426f9307621b3f2d6a8d1669cdb4c19e962b94e34fbb394d4660cbde5921c88"}
{"family": "deserialization", "fold": "outer-3", "group": "commenthol/serialize-to-js", "label": 0, "pair": "7ac5e21d9d8f4c47", "unit": "1d1efa2fd7323c264290d4d314c957c853e1c7a9c877a54d6f03ab7d2af2ca86"}
{"family": "deserialization", "fold": "outer-3", "group": "commenthol/serialize-to-js", "label": 1, "pair": "7ac5e21d9d8f4c47", "unit": "b5dc78c7ac0b76803dc98904142c1139083fb9daafb5bff40e7b5c3f14ee03f5"}
{"family": "deserialization", "fold": "outer-0", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "0894a7212280bab755cd9d93b1f0ede2c51e4fca76b76e5ed9e24ef8f8cf0b9c"}
{"family": "injection", "fold": "outer-0", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "02694b5fb85e9e52bf5eaaab4e0b628f1f44ff3a8d5ad026cc10afdbc4222eaf"}
{"family": "injection", "fold": "outer-0", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "db9a3e888780501dd3c2c125af2f9d47f68ebf02f0e0db8370b2ea09c6f9d070"}
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9af96b099dedcc4005e5fcd19ad78f73864e1cfbafbe6953b4edb6260957943d"}
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9e7c2a786068a936a6c664cc229d6f39cea1fe57caf0806306f0cfc44515460c"}
{"family": "deserialization", "fold": "outer-4", "group": "cps-it/mailqueue", "label": 1, "pair": "07d1dc4d299a7700", "unit": "0e664e274dfb438d2f721edc81f8469ac48387dbd5c1b43973cd7e651db46db3"}
{"family": "deserialization", "fold": "outer-4", "group": "cps-it/mailqueue", "label": 0, "pair": "07d1dc4d299a7700", "unit": "da55f60378c67e91683b9720f2fa0e6659e28c7b392c050278b40713e47c84a3"}
{"family": "deserialization", "fold": "outer-0", "group": "datapizza-labs/datapizza-ai", "label": 1, "pair": "5a91049be21aa611", "unit": "1ab9936627aa59c3a60ea44074ad44b68aea4f9c04f14c1949dd9c77ca13ff0a"}
{"family": "deserialization", "fold": "outer-0", "group": "datapizza-labs/datapizza-ai", "label": 0, "pair": "5a91049be21aa611", "unit": "8df72b9eab3f2fd858c40312d0a806cb67c620bd62873826c8732ba8795d9cfa"}
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 0, "pair": "", "unit": "115d87f37bf1d4ab91fa3010303e81b127f84b817268b60b8fd4b9934efeae4f"}
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 0, "pair": "2b8101c89c839614", "unit": "7bb7d858a670fc51f81e08c141c2bca3b8b7524254ab679e33b22ce79536e895"}
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 0, "pair": "c23acf4f38945093", "unit": "82bdc0b6bb0c19e342efa63af0c9014a607461d0682cdd14bd5fea33516fbd50"}
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 1, "pair": "2b8101c89c839614", "unit": "886285661b88f14e00ff8e863b31cf7b7afb303ad4aa4aa13b1cffbf1272d5e0"}
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 1, "pair": "c23acf4f38945093", "unit": "bf74aae5a48156f183426a80cafca6da219ab13b2ae4b40fa88fca75acb72980"}
{"family": "access_control", "fold": "outer-2", "group": "delmaredigital/payload-puck", "label": 0, "pair": "d7cc37a3a54315cb", "unit": "4aecd43c6e5b81daf7aadd6a6e22f22c17c08b652a2f5121a49320ca1057b186"}
{"family": "access_control", "fold": "outer-2", "group": "delmaredigital/payload-puck", "label": 1, "pair": "d7cc37a3a54315cb", "unit": "88123ae860be2fdc28fd6a289cf26c3e0af0689f455cf8c15098e5dc76e5c8a8"}
{"family": "output_encoding", "fold": "outer-4", "group": "devbridge/jquery-autocomplete", "label": 0, "pair": "81caad96b221c2b8", "unit": "3c2dc720ed159e55629ba737665d93859def9790427289567a513c83c133e236"}
{"family": "output_encoding", "fold": "outer-4", "group": "devbridge/jquery-autocomplete", "label": 1, "pair": "81caad96b221c2b8", "unit": "e8b9595a320776c8e663fd0b168193c1e64e36a9052784d1219f2c4a91d647f2"}
{"family": "path", "fold": "outer-3", "group": "dgtlmoon/changedetection.io", "label": 0, "pair": "f9b6599018ef3b60", "unit": "7558d6ee39a311c5459e35fe6f7b12917b7f843b21ab1fbb5e485a07f53b7d78"}
{"family": "path", "fold": "outer-3", "group": "dgtlmoon/changedetection.io", "label": 1, "pair": "f9b6599018ef3b60", "unit": "b1a6cf450a074788ffca2bebd880435abac2c376e89cb328d7a22d277fdccad8"}
{"family": "output_encoding", "fold": "outer-0", "group": "dicebear/dicebear", "label": 1, "pair": "eaf2f03db2c502b5", "unit": "38dbfbf745d7cbcc52c6fff83248d84663273fe6a1f1b014eac7776c4bb8f074"}
{"family": "output_encoding", "fold": "outer-0", "group": "dicebear/dicebear", "label": 0, "pair": "eaf2f03db2c502b5", "unit": "a84d848ef56b72fc15872d88acaae1743c9cdf88095d13b1869abbec6a0d8a8f"}
{"family": "deserialization", "fold": "outer-1", "group": "docling-project/docling-core", "label": 0, "pair": "7df48bca2c9fabfc", "unit": "7b76ce8f55d61071adb367018cc3e50f04a05d08c7c4919c3806fc1cc8bf41a7"}
{"family": "deserialization", "fold": "outer-1", "group": "docling-project/docling-core", "label": 1, "pair": "7df48bca2c9fabfc", "unit": "c82ea0adc9c8366aa2b77ce73772038676fc37ff50879e873ff397958fdd7ec1"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "7f02a09b8d8f2321f2cdade74c2fa1d52442e006149a749a8be768daaa62195d"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "9b5e439943ce5c2b04d6201b7a843634aa1b13b726813c23450ff54ce03f3dd6"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "5b836889b4da4921", "unit": "a84c2ce4a4eadf265d4481ad7f6864edf1d7e3084c6ee32cbbfe454bb2946bfe"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 0, "pair": "5b836889b4da4921", "unit": "caeed7f097a3311fe2e84efd372fe4b4c140f63a80a9f07ba0b61966dbab15e0"}
{"family": "access_control", "fold": "outer-2", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "c2a2c2d85114a97f892fb1e5500fbd98622457a8c46451e1f9e958ab08a40318"}
{"family": "injection", "fold": "outer-2", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "29d8be8206c88ba7efd86b56a7dcf8b655e33b4a792c2d18b028c92c6252daab"}
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 0, "pair": "f96f0fb245948915", "unit": "0954e6931373915c204b9ec8d2b8ae6297a95667bfd41aadf9035625ad93967f"}
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 1, "pair": "f96f0fb245948915", "unit": "5e09cdae2504e834d591b6f92a808427f07550364b5419899f7e6f76084fc377"}
{"family": "output_encoding", "fold": "outer-4", "group": "ether/etherpad", "label": 0, "pair": "bf11f138fe39675e", "unit": "56f0d0b156992af56141d6368fdb0c5376fc7a44a03991d754f97202b9d77008"}
{"family": "output_encoding", "fold": "outer-4", "group": "ether/etherpad", "label": 1, "pair": "bf11f138fe39675e", "unit": "8271af5017894167556893da2070188060d06eebc87f27db45c46234c359631d"}
{"family": "path", "fold": "outer-2", "group": "fastify/fastify-static", "label": 0, "pair": "9d937c2dffb4aadc", "unit": "01ac12bb06e660d0aeef8b1e26dddc5c99d52cc333f7af6d1981eee2818e64fd"}
{"family": "path", "fold": "outer-2", "group": "fastify/fastify-static", "label": 1, "pair": "9d937c2dffb4aadc", "unit": "9612edf0f927048335e83e9925f01be9aea9b772bdbdc568ad9cc0a17055662a"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 1, "pair": "d4f648048e45f2ea", "unit": "08807c8110b8c1b8635a6883c229fb91f2b9566db8b6c252448a0db7b2ddd47e"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 0, "pair": "d4f648048e45f2ea", "unit": "10869c931053bf77ecaa98a47f56dea27d643d1abed2a4758bcf48a78547ccc9"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "662e1f36c230d7f1", "unit": "0b73332b25413902c26d5b5dee6767465eed99ff71a814582988ffe0d8ec1a30"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "8c8e46d0c7d361b784baff9d70f38744b2805ef1ee0aaacaa616bc20a2c6f32d"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "aca938682c196fd4c85b39b89eeedb97c35f6a5a017651fe9c03c5b727cab7ea"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "cec00d1509a129364da6ab8537fb3f433fd847e26d073c90311ccdaf7a1b39e7"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 1, "pair": "662e1f36c230d7f1", "unit": "e0f58a7ac5627de3e7b6cfe5add41d18b0f64b998c1cc788d42eb2cdbbc3fe41"}
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 0, "pair": "2c251a645094f389", "unit": "0d41cc634771048f72d07c4aa9feaed669d0bfe3844d61dcdc525a0f899617e2"}
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 1, "pair": "2c251a645094f389", "unit": "ca8288d0b8b94dc6b7e4c681936ea610d295fe949cba5f64f09b6de134ef9875"}
{"family": "access_control", "fold": "outer-2", "group": "forgekeep/nebula-mesh", "label": 1, "pair": "2ac5ec5011615c9b", "unit": "2f427f403af3cb58d776437736c2389805543413fdb91aad98979158c6d00de3"}
{"family": "access_control", "fold": "outer-2", "group": "forgekeep/nebula-mesh", "label": 0, "pair": "2ac5ec5011615c9b", "unit": "cddc8c5bed0841b80e7b20d0b378e858f8e4ac37fd7b890ce76611d989857f92"}
{"family": "injection", "fold": "outer-2", "group": "frangoteam/fuxa", "label": 1, "pair": "", "unit": "0c7830c2912ff285855d8a3c8e9b47db4ca4af840baa66d4b5d3f4422f46f95d"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "5383a8c2a78fabd3d17a5b3021125953b5ebbee61f8900c30d016c56b1a3bad6"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "360d8bfd2f116e98", "unit": "7c1989a4ec3622bdaebcf1161de850f22cfed041cb38830ca1ded4fa99d40466"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "ec8c706cc2b20f36", "unit": "848688dcd0fe8d46e6762cc8aee1d82e8fad8a2a044c94b547aea11049c85bfc"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "ec8c706cc2b20f36", "unit": "983cc9231f91baee4e2d626193f3b88aa5b09907f6faab870949bdc9c336b816"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "e4ad86416d5c6668", "unit": "98ff1dff2afe80709b8e61a9c95cfeea12b2110c13b7120478268b4762e2c992"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "360d8bfd2f116e98", "unit": "a1a825d23e34757c8e961e43ae791253e62682e130acdb846b58118588006a98"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "a7fa4207508b3989baabc7a1f0ff2ea5229b366c1cb106e7fc945134b8b106c8"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "e4ad86416d5c6668", "unit": "ef5c53f3114d0057e694df13dc07cbcb532c3b0d59150eda269b87b021d7a6e8"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "getgrav/grav", "label": 1, "pair": "9ae575048504c690", "unit": "176acf077fd58c3e6f940753179ffa3e8316661f355d94acb5e0345b51925ca4"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "getgrav/grav", "label": 0, "pair": "9ae575048504c690", "unit": "228b34ebc2090b5239c7821d28366ddc663e7118d015a79b45dca6618d5490fb"}
{"family": "output_encoding", "fold": "outer-0", "group": "gofiber/template", "label": 0, "pair": "a3fdf9ce3441ec86", "unit": "653236018929ab8534c27704a20ec5d358047bc66e451620d2ac32081c91d333"}
{"family": "output_encoding", "fold": "outer-0", "group": "gofiber/template", "label": 1, "pair": "a3fdf9ce3441ec86", "unit": "b70ca2c43f5929e7b885bd728349e721fd85d7605e2f722a7e8cfe0f0d477b0c"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "groundhoggwp/groundhogg", "label": 1, "pair": "", "unit": "92400e83890882a66ffdea1a405e2354a195442e176d85fb707b0442c44d45ca"}
{"family": "path", "fold": "outer-1", "group": "gtsteffaniak/filebrowser", "label": 0, "pair": "f9dcf3a0fb0f0a14", "unit": "7ca6dd4f673976b000a657f885af3463740419ce41e14f972af0c5483cb75a7d"}
{"family": "path", "fold": "outer-1", "group": "gtsteffaniak/filebrowser", "label": 1, "pair": "f9dcf3a0fb0f0a14", "unit": "86c1db3a5e9f843d6546146449171e0378ac61d6c8f44c6dd72fd379cdbb188a"}
{"family": "injection", "fold": "outer-4", "group": "handsontable/formula-parser", "label": 0, "pair": "6144baeb491f4a56", "unit": "f5c22ce104e9593e5272b6e8b3c6f3b61fa4578552ccea80e6021ee90bdb3ffc"}
{"family": "injection", "fold": "outer-4", "group": "handsontable/formula-parser", "label": 1, "pair": "6144baeb491f4a56", "unit": "f687e7e95dbe47f602a1e0002a44f94fd48e5b1c33f77ecebc6c2d814fcbf450"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "heartcombo/devise", "label": 0, "pair": "2ef4471c66e5e4ab", "unit": "ad7c8e368fe1699a044e058d5c10b41bd9fc0f91c3f3c384c5dac40af39e71f6"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "heartcombo/devise", "label": 1, "pair": "2ef4471c66e5e4ab", "unit": "d888baf7194bbe7158cb3932f58b47a9a235351d388d10df82a130ddd54c2d46"}
{"family": "output_encoding", "fold": "outer-1", "group": "honojs/hono", "label": 1, "pair": "5628c54152216646", "unit": "47d081c9f6060218967902f694e61c88f64f57c65bec6fc3073afeb0b77d117c"}
{"family": "output_encoding", "fold": "outer-1", "group": "honojs/hono", "label": 0, "pair": "5628c54152216646", "unit": "e1297bccf9c3606e0b41d762a69ff5d0a33ae4f7f1506071af6b9cb4a177ff56"}
{"family": "path", "fold": "outer-2", "group": "hs-web/hsweb-framework", "label": 1, "pair": "e52aa814c36ace1f", "unit": "2d6d74364f1982e91f27f1c995b8699df1fc8f8dce6e1e2cf274bf01b6e9494e"}
{"family": "path", "fold": "outer-2", "group": "hs-web/hsweb-framework", "label": 0, "pair": "e52aa814c36ace1f", "unit": "4727471fea2cf0a92d43b4b1503f1339a7dde87f872ba8064dacbf855c57705b"}
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 0, "pair": "6548e1527e82bd99", "unit": "5542613f6d2eb9421b8213b8bb864894f0ecf9fe64945681019230ef0d75cfaa"}
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 0, "pair": "", "unit": "6c2adee34e9e8384535ec1073eda60e2ab528ac77c36747802c303e1f5e9ffe8"}
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "7ce70e9b20107cc7ebb87d173da765fc53ffd268e521749a06a9a1f38d746845"}
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 1, "pair": "6548e1527e82bd99", "unit": "c3e54975464323e58ec96ee4d83e7bc08baf94553d553a8dc3fb28cd9c70ff21"}
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "d3173c1e43eb6402a71ea2dec1592913dc078a2194ba1c1d509ab129a3180f6f"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "internlm/lmdeploy", "label": 1, "pair": "0924befc49c7bb18", "unit": "e7e4872db40f0420547ed622fbdd7a072b64a7ba466d1ccfff68e691cee6ad99"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "internlm/lmdeploy", "label": 0, "pair": "0924befc49c7bb18", "unit": "f7d95f43fb2df20ebc4c37fceabdd00ffd8159f404ca384481f1764252382388"}
{"family": "path", "fold": "outer-1", "group": "jarofghosts/glance", "label": 1, "pair": "772e18821701d60f", "unit": "89a0496c7663bbfd67d660c9f481c12420da2dd1bdb29dc079735ee7a0a49626"}
{"family": "path", "fold": "outer-1", "group": "jarofghosts/glance", "label": 0, "pair": "772e18821701d60f", "unit": "d3241ceaa50a7d23a4650e6a2a55372b08ec96b35b0bede844bc40e16e818ebc"}
{"family": "injection", "fold": "outer-0", "group": "jas-/node-libnmap", "label": 1, "pair": "250de1302c7e541d", "unit": "5d03535f2df2acdbc319d9e6b20da65d51f3e810b80f3c8f89b5074a971a3f1c"}
{"family": "injection", "fold": "outer-0", "group": "jas-/node-libnmap", "label": 0, "pair": "250de1302c7e541d", "unit": "78f977cfc096e34b4eada7f798ad2973ef45728d0d01bfab6ac9e2a3257827ef"}
{"family": "injection", "fold": "outer-0", "group": "jashkenas/underscore", "label": 1, "pair": "c10bfb006f2da55a", "unit": "1ea4b86dbd84581b1d7b7cabbab9f28c8b213416a60778879666ff42d23fd7f1"}
{"family": "injection", "fold": "outer-0", "group": "jashkenas/underscore", "label": 0, "pair": "c10bfb006f2da55a", "unit": "7647021f742da24c660b2d407886771786e957076c4cf336def560a570dd8778"}
{"family": "access_control", "fold": "outer-0", "group": "jellyfin/jellyfin", "label": 1, "pair": "a73421efb106ab0a", "unit": "1f6f207a758582de0c0e6b331b19fe119abe18727d45ccbbe09aabb795f3a292"}
{"family": "access_control", "fold": "outer-0", "group": "jellyfin/jellyfin", "label": 0, "pair": "a73421efb106ab0a", "unit": "32fbdad39eaf5177db4ad0a11d53e785b30bc7b2376ad9d460f29a08429bd0c7"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "jenkinsci/cas-plugin", "label": 0, "pair": "0458e3fd444409f9", "unit": "b521216bb4106738b3571ad3902d08f6230aabd88cf3a8f9f26f1c8ec01304cb"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "jenkinsci/cas-plugin", "label": 1, "pair": "0458e3fd444409f9", "unit": "c87d2871907798fa2e73d31381e9d63089b0388033f096d247373fec07c2cb36"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "jenkinsci/google-login-plugin", "label": 0, "pair": "6bdccabcf5b9fdc9", "unit": "3d6ddca5eb264d88411b86755909f1d2abddd09c129cabd54c96ff26b4105316"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "jenkinsci/google-login-plugin", "label": 1, "pair": "6bdccabcf5b9fdc9", "unit": "e055b898802279bc63204cc601e29f7f1018efad7228cab0cce00ea5866ecc7b"}
{"family": "access_control", "fold": "outer-2", "group": "jenkinsci/mcp-server-plugin", "label": 1, "pair": "91bc832e70680c92", "unit": "1060c63ff0fae3af567448f3d4c60d9c159b61ccc358964243eb152a9ac38865"}
{"family": "access_control", "fold": "outer-2", "group": "jenkinsci/mcp-server-plugin", "label": 0, "pair": "91bc832e70680c92", "unit": "369dffd1e730d7997813adfbebf1481cc68987cae0d0085e47b91a26d5330cf8"}
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 1, "pair": "a4144258dffb3b8d", "unit": "7ab301e1614979f44c353e665540582493f78ef4bfc41209562c19b900710f8c"}
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 0, "pair": "a4144258dffb3b8d", "unit": "ea95e8387dff32ded9dc7b04c11a76a545b4e2ea6db0076b9b13971a59904bb3"}
{"family": "deserialization", "fold": "outer-2", "group": "jmespath/jmespath.rb", "label": 1, "pair": "858330e01d3cf9bb", "unit": "033cda2b7cffa5a5fa6abf0b7f2cd27ba78d916eb3cab8c79decccd8776f4a3f"}
{"family": "deserialization", "fold": "outer-2", "group": "jmespath/jmespath.rb", "label": 0, "pair": "858330e01d3cf9bb", "unit": "d44c4555a5ae19aa17b468ec5cff56b33a0b221df52e04d8abcb656399499dee"}
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 1, "pair": "c6a49339d4f0c798", "unit": "dc2e4cd5bd963643702d2b832ae170dd25744162e4076e36687256a09e69d089"}
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 0, "pair": "c6a49339d4f0c798", "unit": "f83cd137b85e651ca5299c5a42176b797ed4964cbef31418c96d63c52fdb65f5"}
{"family": "access_control", "fold": "outer-1", "group": "jsonbored/gittensory", "label": 1, "pair": "5ec37c0f475001ca", "unit": "5d8175fb21a4649bdc8730358e9b87adb90b2aafc4da5fee203829663e4f46a0"}
{"family": "access_control", "fold": "outer-1", "group": "jsonbored/gittensory", "label": 0, "pair": "5ec37c0f475001ca", "unit": "af845e9e332555096dd350401977b2f07015433f3db8a040891d9470f3f9e355"}
{"family": "deserialization", "fold": "outer-0", "group": "kevin1024/vcrpy", "label": 1, "pair": "c74525e844650de2", "unit": "2e444ad34ee5f47d73f0cfad0a10c70363337b4ddf9067ecd1b8072d8232652c"}
{"family": "deserialization", "fold": "outer-0", "group": "kevin1024/vcrpy", "label": 0, "pair": "c74525e844650de2", "unit": "c492d325f5ff82b5f817aa4be1440cec8595054dd0719e07a58ac5501b2ae7eb"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "koajs/koa", "label": 1, "pair": "111a5a80a3abc2c9", "unit": "c7ae403742450deb2bd051ad1231121d2117ba56fa72a038c82d58d207cb3870"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "koajs/koa", "label": 0, "pair": "111a5a80a3abc2c9", "unit": "debe34831afca6f8220b9952036a2d4570babdb5e321a2f8dbaec661a016bd50"}
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "9950b85412b021ebabb11eea4c490a660ad380d14c673a87c3fb568ba4bcfac9"}
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "656df5ff8e69dcdb1f9ba1ed52d734966eb29773bc745587483a355743b8eb5a"}
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "91652a4e776fbce70cde9e4a99c6da5e17da7f8d6d0404867c61eec04161b6cf"}
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e53bb3a217dc4737e8dd03671de4f6a895599024070a985a268c65a034295e7c"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "3d7aba22444811a337abcad0e4fc62990bb704d3a10e72d893342fa94699026b"}
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "5f18a3026a51f7140b7aaa3aa8563e8b6fdda6065d95e19e25b1dac4b9ce0e00"}
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "0fccc306ce80cb4cbfdcde0a2e56d637bcefa9dd30423d6c54a4f77860afe740"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "1e2eaa45661345f1b6759d968d7696dd0d96d6fba1350a00e4d0a2ebaaa2e391"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "8b9a8ff0e32a327df75eb8a1811464d5f34690079d5930139dd917063bdd2809"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "245d0e939b7c1ccf098c798492009e3a73ea17400fa613dd865cd369d2e7f383"}
{"family": "access_control", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "653c48df493f31c2d01c973d1e379a180fbe6a6edc83a500bc62509112175f41"}
{"family": "path", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "3719fe21d45b50b35458bcd2e263e4baee67adbd3c98adf0eaab6bc6183c5651"}
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "1209a6cc8c1548ed83d4a9a09d6a27f706a23e69cf0c261fc5e3a27536a83306"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "362c90495dc468bb732c71f62d71ec308854161de31ad18d2a870056d947ddb6"}
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "b52e53fec9707b2b442c4dc7a41d376cf76946d2c45c2eb600c9477fc6baebaa"}
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "477e3b90b92e622888dbe0e9d3a23b31271ddfa3104a5ab326eb9e9e765aa01d"}
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "660c63c4a73bfe64088ae3249e40d9c5d69a6f8a505ff671217c5708a345c5b0"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "4d60c386c801bedb61c1461fced29a3cfb50b09d6435e8617c60398343719d93"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "f1fddcc1354a29bef55610ee2b73eaa440192f2f18a0d22d08057b9d1e1b61b8"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "6d9886c38221b2f9aacf83749218ac5ebc354a6fcc4170e6c78bedb92b88296a"}
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "5550fec4a81c4c9cb22d7312d8a63e1fb2b03c4aed0c1f316d44c1eb27511bac"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8ff87dd04140f0137ea1891f220b1276156ce10a58be5a49d0c8e07be531ffb2"}
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "c6c9bde6630678377d250701073c5c223c7892e6ff8f2f8e0293dea22c5b5e7e"}
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "b9b15958973c83f8d7b58372ab5ba58eb191c400125579d5ae8c5b52d5d1368e"}
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "b95ec12e4e126ad558e944e2c260116ed6a63f5f3ab97fb5cbcdca75a931ad73"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "38a5545ac9ef67e15db720d1396a2d5915f9d06d6f4772012e5bc95f33770376"}
{"family": "access_control", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "c8bbe9be91c976be97b7ca387048a7d4537d9f6fcc6cbad0a541d1080629d5a6"}
{"family": "path", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "ffacd53f1bd75155e63b318fb9cba27ec03e754841e1eb39d1d4fa0b2c3dfca2"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8fd920175fc43b91b4397acffacb10ab8f7fd0ee38bcf2d2f3ed06a3fad0dfe9"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "86c9598ebe3dcda261a69ae545513f6ce73101f3980066e4506ece99bebea84f"}
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e5190fcc57505116f79a31c51aed69f8a1eea13a8ae3e7265f44c72418277ef9"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "2b807ff54e6c47e941124c16d0c27c691cef54f2203060a3a6ce812d478c5261"}
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "401aefb4b4791301ace2d770d321c6b26743b0aab7b96f8b410254c704be6d2e"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "22d186c93498e21418b6c1d0547a73374b13041d8afc89a80d9540fa658b6b42"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "dbfb6df14fe37f8c0efda5f16643297fb0550dfc5ad5805d4e4b844d967773b2"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "f0da8cfe4ac1e586aa7903d72e67b44500bbd5306d72b4b342d10613efc69222"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "2b8fd73943f456da266434470333a7775f55d9a41953e97a000fdaa3257b6a30"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "a612091f452e60f28f44eefec641175170dc5db04456a39a099cbb13c50b4d2c"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "4c6600facd83bbf234a32d9e28ac494fb516df6e3d9e9dc28d720ff35252d067"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "fb0ea8cab8c9430d22d1b0fdd65a69146fa95fbff94e720b4861b96538d992c6"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "e81f91a56058ac1473579767abea57064c53d7d007797a854314814e3c35ee74"}
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "5851ecd96a449f98b05f243bfda0a7cf3af8612b27f53c414ae1f4ea67242a7f"}
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "328621ef2a94632e5d37ea6267d17b766233384cde35c07e29a4eb672d40ba28"}
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "04b2eae2128608dad460ba8275142a41a44e32c4243a29e2a6fb2c9ebf3d8f1a"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "33337002ff5f5a189610b042522ceea254c9d107c1f24771b4f616a0f96f636e"}
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "02c09a783cbf058856fc2b448406f3ffa6fbfbacf9b70f4e5e22fd741a5c7699"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "956260164cb21a47c499e5598491031ef303418943f5096f225a87162b588574"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "8bd8c5aed69eae3019cc2d3654b36bdf2b31435e6a020a389cd103696a2f874c"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "1d8397ef4a9fde7b48e2e1e871bba0f960a802cd7823ecfc10e9dfaddb955ccc"}
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "c9db92476032eb4296b950a1eff8fa4aaa26f11542a01a5cf10c7b2e32dc6433"}
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "d3e0d0f9b0f4b723080989b5486395cda0a5b186a674022ea70e7fa7ab561323"}
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "44ff3b3e61c02744830c4abea872821ca7e0f94950c082740c5c48ef13604e94"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "4103e979a99233c3e71fdfe9f1cdcbab34a2c60b5351b75c5ff5dfc287b962d0"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kozea/weasyprint", "label": 1, "pair": "6768bfae38c6859d", "unit": "f376519889a00950f5af67eb15c639306ea70da7235d3be4a94dbc19cb096cf5"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kozea/weasyprint", "label": 0, "pair": "6768bfae38c6859d", "unit": "f3fea4e334f49ef6fe6b17121fef553eb8e33a9ffa13e6d6cbc4c448da996a7c"}
{"family": "access_control", "fold": "outer-1", "group": "kubev2v/migration-planner", "label": 1, "pair": "d670253fa209671c", "unit": "1aeebe2e440bdd70525c31f93180f33be93f820db5ba46a2e59c371ef83ee794"}
{"family": "access_control", "fold": "outer-1", "group": "kubev2v/migration-planner", "label": 0, "pair": "d670253fa209671c", "unit": "b11c8a8a24d3809c0507228568fe7cc05ca0a22ed28cb19019adb13cb619d63f"}
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 0, "pair": "33260f3e6c8ea6d0", "unit": "7ca409c7dbc1c38ff005624e212efc6d424cb421e27b482dc8dc0d6881f4c11a"}
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 1, "pair": "33260f3e6c8ea6d0", "unit": "fc364d7ed77c7c0508da00e5ba847ec0792f3bc373d8b4297747189ef42d1ae0"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "", "unit": "6cce78d7569ad8db7ca0a61d7078198f666502d26ea6c0afb03173a8a4bb16ef"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "82047c0513e43a83", "unit": "6d2dbed1ab1c9d8ac259dd4227be69641fa59d52e816a8dc547ade01d0b9b0c4"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "98ee23cd6bd31cd3", "unit": "ab4bb39be535d70cd95e9035bb958bfc7418576a6ff40a47551894dd140317bb"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "98ee23cd6bd31cd3", "unit": "b70068cbc646bd01d3e10986e002f648a74e9d798900dc59cf3039b749c95c18"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "ff168d95d8d9496e", "unit": "c6ec9478c125ed780d7013c2807f2cec1878f44253667dfed319c28cbfd17d4d"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "b1bfef46dcd3b9a5", "unit": "cd4d095d8380a2fb7ca02fc6abdbae5249cc06455db2929875533f4aa8b886dc"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "82047c0513e43a83", "unit": "d1bade217be8d345cf5d0f0442ee31924187acbb98779dada95274a77462f696"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "ff168d95d8d9496e", "unit": "d561db250b892b91feed7ae877900e1e25963444f73c7699576e010f06da73cf"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "b1bfef46dcd3b9a5", "unit": "ed78fd5a7ab30bcd6e8ddd27c706a19cfd89750de4fbf81c0b567946eb94db83"}
{"family": "output_encoding", "fold": "outer-2", "group": "lepture/mistune", "label": 0, "pair": "25f75d77a669b54e", "unit": "0f2825568461562f2a915a4f16ef8a4931290fd20a9801470aeb12677e66e3f1"}
{"family": "output_encoding", "fold": "outer-2", "group": "lepture/mistune", "label": 1, "pair": "25f75d77a669b54e", "unit": "bf22bb44bcff4a5baf5d550c9d3e83e5f3ca607475fb51deaf4fb6fccff3d73a"}
{"family": "output_encoding", "fold": "outer-4", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "cd80750447cf7af67ad809279718d98106c71705fef9ff7254728bfff4ece5b1"}
{"family": "path", "fold": "outer-4", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "2ce605ab5ce7ad3a1fb983cd6bddf3924e5da3010d2252a0a4640aab4e5921a0"}
{"family": "deserialization", "fold": "outer-1", "group": "litedb-org/litedb", "label": 1, "pair": "b0a899116dc36b85", "unit": "510ad48af03ba4a1a2a06dd254b135311dddb62d9fe5a2e8e8b12627586e6b13"}
{"family": "deserialization", "fold": "outer-1", "group": "litedb-org/litedb", "label": 0, "pair": "b0a899116dc36b85", "unit": "b414718a8a5f79739b88db24da6ad9da8e48bcb6b5e2dc99158848cdaa226767"}
{"family": "deserialization", "fold": "outer-4", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "8e22ee30441b20484090659b0c97423c711f5d10219041bf3a399623fc056a07"}
{"family": "injection", "fold": "outer-4", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "0f9070df9dc334f981cb01008c260b8bff78dbdeb48ce672830ed95caa8406fb"}
{"family": "injection", "fold": "outer-4", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "5e352f797b1422b0df477bce4730a9ce33cbd0837c3d63da94e4d8801a9a4b64"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 1, "pair": "dadccb128ac0e120", "unit": "505097494da5939404ad7d5b4af2b1c4a10efe94048084492d6ce60ebaa04c97"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "dadccb128ac0e120", "unit": "84ff8ca2d98a1b5d1ca4bfaabe7d19e03a396042b228c8d18d1185a178389702"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "91e261c54bd2c3c3a9ac04e48894ecabaca5b13e7db4fbd5578f9d266098117a"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "a308dfd59bc398824e2454bf6cf6cfd5b7880e5b948a82424f14753d2b6fcdbe"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "ba24eb554458bc97314ed561a7b2f4a4e643dde1205c32de4c8fcdd8c46e575b"}
{"family": "output_encoding", "fold": "outer-0", "group": "mganss/htmlsanitizer", "label": 0, "pair": "24d8f1840b67fbe5", "unit": "7ffa4a229a62266788fbf64890f81f20358b3eb8c35d3ab87d0c339d5bb4b507"}
{"family": "output_encoding", "fold": "outer-0", "group": "mganss/htmlsanitizer", "label": 1, "pair": "24d8f1840b67fbe5", "unit": "d19493ed7941e1cd25a76dc50504d97398770d07ebc302dd66d83441e946c468"}
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 0, "pair": "9f8a60eb48d74bc8", "unit": "80fce34f2377731e3de4355c7ea33bb20fc2fc00a5f388565b23018c7a3ed008"}
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 1, "pair": "9f8a60eb48d74bc8", "unit": "b68e62722a327c9d5ce6774eb96527451729410768efc0b5c7ba604ec468052d"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "miniflux/v2", "label": 0, "pair": "997f9d2efc32302d", "unit": "b587bb499945a9f5f0749eb4d57cc1027cfd21c8292bd32be4d4e1cc1b1db261"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "miniflux/v2", "label": 1, "pair": "997f9d2efc32302d", "unit": "f8d9c9a7c12797b3caf042e6bfdb89a5072c4eae1fd36be6196f9fa326cdf257"}
{"family": "output_encoding", "fold": "outer-4", "group": "mixxorz/slippers", "label": 0, "pair": "8e6b4b4e28981590", "unit": "3cce3e8f9ff3f1d261f9b96c844c1e4da07bae72d5393897b3a929be25372edc"}
{"family": "output_encoding", "fold": "outer-4", "group": "mixxorz/slippers", "label": 1, "pair": "8e6b4b4e28981590", "unit": "b105689ac24ed334294c434a77bb6d56d5d598830ffe8a5d60ab3be5290b47ad"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "0a36cd5177f3b2c20b433db6bfffe5ec5b7e74cacc34a86ac581f7a0a559432a"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "e7fc97d83013c9ff", "unit": "5ab444e64297fc19065478104c7ebdabe23cc415f22adafd0cf82c98e4c5cae3"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 0, "pair": "e7fc97d83013c9ff", "unit": "6b462ea30c1afda572600a10324249223932d36a032eacafac8c837bccbdce87"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "84d6a4202a082e0d", "unit": "8175082f40d8d61d44b54579c84a808e25b89110be5d8fd82feedf7dd5e0d5ab"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "8abefca43ee4ae2c797033da31f85c17bfa7be6b35413d70b0216415c5db9a09"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "91d213946859f33d3f243c43aa8285cc6e8ecb3db1de938f32af775a70607f8e"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "a244d9f010ddb43a09e82363dfabb0e406d411fb5de08ab7d6c9ee367f2a15c3"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "d27d174d6e8e5f1682a1c42723705e89f0d18c95ecedd0567e313daa9a0aae44"}
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 0, "pair": "84d6a4202a082e0d", "unit": "f45608ed47fa11ca25093641f6021ec06076c18676abac9b809f31b3b7713de8"}
{"family": "access_control", "fold": "outer-4", "group": "moquette-io/moquette", "label": 0, "pair": "55d1ce4f29dd1583", "unit": "a877ce3462da316bcdd82ce9db1d9887e887eaa83157922edb1663413edae1d6"}
{"family": "access_control", "fold": "outer-4", "group": "moquette-io/moquette", "label": 1, "pair": "55d1ce4f29dd1583", "unit": "aee67be0ba31f99c7a24faa1bad6178c6d37361a308cf5aa10138e6ce4bbcfeb"}
{"family": "path", "fold": "outer-0", "group": "naorsabag/openhop", "label": 1, "pair": "16e529fe8c5486b2", "unit": "1d21acd0171726775d3da4589b660645fcd43cc026042a2b27e0e0e8374eceb7"}
{"family": "path", "fold": "outer-0", "group": "naorsabag/openhop", "label": 0, "pair": "16e529fe8c5486b2", "unit": "f8e004b67ac98dc3392ea4115051e6bd10ce22eb638a762f0f16e1d4200bf129"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "nuxt-modules/og-image", "label": 1, "pair": "ef603436079b0005", "unit": "77f843b5044bb5da935df4b2620bf2abe9e7dceb3147439e883927112eaec22f"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "nuxt-modules/og-image", "label": 0, "pair": "ef603436079b0005", "unit": "f2ce95f0e44e73986784b63a702ffa4035c423056cac3667a63140a7e8d795c2"}
{"family": "deserialization", "fold": "outer-3", "group": "opensearch-project/opensearch-ruby", "label": 1, "pair": "1945f28a3fc5bd55", "unit": "64c0b7ccad09192eeaf590befe0de0087d5b48a65a4f6cef41415f3f54a8f536"}
{"family": "deserialization", "fold": "outer-3", "group": "opensearch-project/opensearch-ruby", "label": 0, "pair": "1945f28a3fc5bd55", "unit": "b55cd6759d4b7861e589052aa534bd0554586db46a329ed3beed7244d579e8c4"}
{"family": "injection", "fold": "outer-3", "group": "openssl/openssl", "label": 0, "pair": "1c9e47835232f644", "unit": "90e220d3540b88dd9efc8e66c8ab519d4cc78805f16bed585b795c78cfbb4d58"}
{"family": "injection", "fold": "outer-3", "group": "openssl/openssl", "label": 1, "pair": "1c9e47835232f644", "unit": "b5a185b05235f70f6cf27e2adba919f4d8393dbdf2d99de095548a21946aeaf3"}
{"family": "access_control", "fold": "outer-3", "group": "openwisp/openwisp-ipam", "label": 1, "pair": "9b2b34cf62a76ea3", "unit": "91a3984e47f17512db8523e04f863525c67db18631d8f11776cf61ca7221c91b"}
{"family": "access_control", "fold": "outer-3", "group": "openwisp/openwisp-ipam", "label": 0, "pair": "9b2b34cf62a76ea3", "unit": "d91f1e14282a0c3c8da57c6935a1625e881374bd205b5baa9e4a341d9109ae21"}
{"family": "access_control", "fold": "outer-3", "group": "oqtane/oqtane.framework", "label": 0, "pair": "a20cdf9dd208720d", "unit": "18a26df68e4b81d411cfbcc8155a7e6f382753d6e1ad95eece7ed24fa8c44324"}
{"family": "access_control", "fold": "outer-3", "group": "oqtane/oqtane.framework", "label": 1, "pair": "a20cdf9dd208720d", "unit": "e6fb74441fb585d7ef5855317fba795ef0f0a0e6eae9970a49556ce3070c3254"}
{"family": "path", "fold": "outer-4", "group": "owen2345/camaleon-cms", "label": 1, "pair": "d6930fe7e752b7ec", "unit": "06f0341dfbf1aa9504083737df5d9be3cedfd4e3a198799d8075a29990776800"}
{"family": "path", "fold": "outer-4", "group": "owen2345/camaleon-cms", "label": 0, "pair": "d6930fe7e752b7ec", "unit": "30e2cd8bd381d7568b78eaf982f562d278a6a8d942d5282466532c1441bdf0ad"}
{"family": "access_control", "fold": "outer-4", "group": "perses/perses", "label": 0, "pair": "2d482951d7380c4f", "unit": "5c1a721da6f85d1d5dfc8cbe5e9809b093a09c059e5abeebed2cbbee55fbd498"}
{"family": "access_control", "fold": "outer-4", "group": "perses/perses", "label": 1, "pair": "2d482951d7380c4f", "unit": "6cabd1c4b5aa94cafdd07c0c6a3f9abfdc03761ddc5b0a2bebc72a508d020b8b"}
{"family": "injection", "fold": "outer-2", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "", "unit": "2300c097374b9098671dcbcd42d02d08172e0f3064a8323223d0fe79a6106aab"}
{"family": "injection", "fold": "outer-2", "group": "pgadmin-org/pgadmin4", "label": 1, "pair": "0664148b0b192456", "unit": "d702617aaf4d19811baaa18386a3c8b0eb35cd3fd72e8d90611d34850e990858"}
{"family": "injection", "fold": "outer-2", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "0664148b0b192456", "unit": "f5ed13d811c5b16d46cbd4a3b9edb8f7d2db49acbd49d95f39f33eaab38f58f5"}
{"family": "access_control", "fold": "outer-1", "group": "publify/publify", "label": 0, "pair": "1972ea9f72b178b6", "unit": "3ddabb356d78e7c65ec8f88b6da8e4fcabc055ae73f2f6d92c50efddc5d4fcbe"}
{"family": "access_control", "fold": "outer-1", "group": "publify/publify", "label": 1, "pair": "1972ea9f72b178b6", "unit": "6875dd5e9c0ff8a1b693d29a109cb2e62e6ff4081df40868aca70e4770476757"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "04aa777130c02a06", "unit": "16aa43c87956c44a4b115245140eefed704095d37cec0cc6410377403cd2fa7a"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "23744137e0485a36f686f5f1b5666780fbed3ca522e8c6d35687a391ef504498"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "67565d99fe28c68f", "unit": "2692316792e6f02fb51578cf30d71bba6c73bf5420c36f709681c8eaa69807d4"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "2e36f73002c416a68d077bffb0335d82346c7507f2478353aeac005fce5a9cbc"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "04aa777130c02a06", "unit": "67721f67055eb6fed955ff22bfe4060344662eccb369431835a2116ad120c4f8"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "8fc1c96fa6faa4d3966ecc42876189b18ddab779d58f47cdf28a1155f48a6ac4"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "67565d99fe28c68f", "unit": "ab535098a8e709b022afee1da15dd1fc371d389056938ee4fa4971b2966c823a"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "d46f94235bc462360097e7a22e1d442d95f049306631489a4d8b25f647a95bbe"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "edd6dd4364442f66384d14e9317ef2f6234e32ab78327a45741b8a816b4332ee"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "fc7df6887ac719d4615934c0e8c3296728c2f6089f233158c59989027c802d72"}
{"family": "path", "fold": "outer-2", "group": "rack/rack", "label": 1, "pair": "c0412509f15fe3e1", "unit": "63bda2a71ad7155ad2f59aa9086efef3fd3710dc82c57f4b32ec8bf561460239"}
{"family": "path", "fold": "outer-2", "group": "rack/rack", "label": 0, "pair": "c0412509f15fe3e1", "unit": "b1c9a070c93764655786c3de5fde536c150e319efe47548614cc544b73a8ee2b"}
{"family": "deserialization", "fold": "outer-3", "group": "rails/kredis", "label": 1, "pair": "feed5f7de11289a4", "unit": "71531567b2b8609ae0d31cb177b8faea4f5fc61923e513b0154a2c941287ce2b"}
{"family": "deserialization", "fold": "outer-3", "group": "rails/kredis", "label": 0, "pair": "feed5f7de11289a4", "unit": "d6be6d397a301a07faa761163d3d9e270d71d23d3bdfd084eeafb1e62b91dd71"}
{"family": "deserialization", "fold": "outer-2", "group": "redisson/redisson", "label": 1, "pair": "ffd4b04a7bebfaf8", "unit": "54f7519d3d59b6d8d69a5f8f7bd78e360d3fabaae00b7587c37e388719169370"}
{"family": "deserialization", "fold": "outer-2", "group": "redisson/redisson", "label": 0, "pair": "ffd4b04a7bebfaf8", "unit": "c5e42c9ab2ee54e46700ab0b64e239b567fbc75ccaceaf6cbf20687813663332"}
{"family": "deserialization", "fold": "outer-0", "group": "remix-run/react-router", "label": 1, "pair": "73f2a014c6a699ed", "unit": "44ba9d9d8e75769c1444ebf37a64941a845b4bcae678ec79d64dc84e5cc067a0"}
{"family": "deserialization", "fold": "outer-0", "group": "remix-run/react-router", "label": 0, "pair": "73f2a014c6a699ed", "unit": "b430d488c486cef26b1c661f773e7e9e4a177eeea73537a033eec98d532e4eed"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "remram44/taguette", "label": 1, "pair": "cb0c638282db0d37", "unit": "a49e6ad2e7c2a5ad0d68d5d69c263306b501fc0dc9382e701fb5fe680611196b"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "remram44/taguette", "label": 0, "pair": "cb0c638282db0d37", "unit": "dee28099d3b37cbd0ee14412122ecc5f8488ed6bec01863520b87706a040fba1"}
{"family": "injection", "fold": "outer-4", "group": "roest01/node-pdf-image", "label": 0, "pair": "09bce271e8530fbc", "unit": "35eb19b4c6176b37287e2123bb98ec4045ce6bde7cc3864dde317c024b0935d8"}
{"family": "injection", "fold": "outer-4", "group": "roest01/node-pdf-image", "label": 1, "pair": "09bce271e8530fbc", "unit": "dd33291cc494bbc77d44e9b13d16f171d32c9746637506f315f375e8a9ddee95"}
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 1, "pair": "40d8d81a764c508f", "unit": "7deeed73c29c74ccbed29be049e109d363d41e4b5eea245ee57780b85f0e137a"}
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 0, "pair": "40d8d81a764c508f", "unit": "b5cdadf848e67910550666ba2a6ac67b24112e501c02dba1865766a316317a23"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "", "unit": "15dbea384b3255457a44d9382efc6944dd27f1d411d903344a5d5e5fd194b72f"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "b2ac284bd70dd684", "unit": "225890ba6694dc8f4fbab46e9c851dff8c095e906cf14db15d01ebce0891aaab"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "b6eca35de93a5be8", "unit": "42eb8427415704e060e73b5ce0943b014ddf380ca756d5ec47c558e59a79895a"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "b6eca35de93a5be8", "unit": "4c9cdcc11a18641ef41e9308712c5e8b830ec9ad5ebc3a956c8701e4f77b39c0"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "6a53939838afbdd7", "unit": "5794809e595311c86909d673a570d003aa025e4dcd6abc5641b641d935beb0b1"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "46d3ec7d49cb8ca7", "unit": "5f7a63ec650915962ee5255f5416df63ab882d6107f3b540be36d481798a8944"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "6a53939838afbdd7", "unit": "97a7018535ab511622e3ade9e53d570ffb1c80633eaa372fb76e749b3a586c8f"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "46d3ec7d49cb8ca7", "unit": "a84e9ab47f4f2b584684dbb81575758d9032fe3f2ed53bef478cb9dcfedaf677"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "b2ac284bd70dd684", "unit": "aeceb1ebc863ebbc3b69ae56c5b78d2dba005132b75cbd6af929bb0bfe25da9b"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "f42cab375ace1739", "unit": "e329d4a6ac757680bf3d292a81f1f74f8a8a9b482d1651efe9f503f6a4de8796"}
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "f42cab375ace1739", "unit": "e73c5aac41c0b93072223a3097fb175548ce893e0678ff3a22bc7571745a17e0"}
{"family": "injection", "fold": "outer-2", "group": "sgabe/dsvpwa", "label": 1, "pair": "", "unit": "52354ccdbc16f209e4692d97d5237e04ed030cf4a5e71f77b5a767962b355b4a"}
{"family": "injection", "fold": "outer-2", "group": "sgabe/dsvpwa", "label": 1, "pair": "7d23be667536afff", "unit": "b5016a5a53409491ba805ed8718599b84250b7b6914da0434b0882b0b3578dc8"}
{"family": "injection", "fold": "outer-2", "group": "sgabe/dsvpwa", "label": 0, "pair": "7d23be667536afff", "unit": "f1b01102968568d3ba1b2fdc70f4ae281f433fb20d29eff2a55fc45555ddbfcd"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 0, "pair": "1199de75f426940b", "unit": "10fdc236f43a02b7e291eb8bf6ada297a2ae98086a63b171057bc7bfc37a3377"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 0, "pair": "22112c31603bbe9f", "unit": "5eb3b7523b5a196dcaacba08d68457f9a45e31503ea02745ca3a7d0542f9881c"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 1, "pair": "22112c31603bbe9f", "unit": "6bd5bdfd81b0a3374d59cfb2c1f7fa2b27887f4672f4485e77eb5234b569983f"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 1, "pair": "1199de75f426940b", "unit": "c09dffd04708540845c8d1dd77a2bd989bd35f264abf652a2db55f8cb1faf7a9"}
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 0, "pair": "7aab025ea7b97019", "unit": "7487a34cdfdb532a5096764a47a3e565fa509605c1df0e2b105eb9c751fc5307"}
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 1, "pair": "7aab025ea7b97019", "unit": "9c0be2a58741f095fd0a9cdc60d4ca4039de1529699d5701dc4d0bcf7d5bf90c"}
{"family": "path", "fold": "outer-0", "group": "simonh1000/angular-http-server", "label": 0, "pair": "4de5441f1776f083", "unit": "a3a35d573da308cd1d0614a26616bbd37b360d9ea416894cdeffdb59abd691d4"}
{"family": "path", "fold": "outer-0", "group": "simonh1000/angular-http-server", "label": 1, "pair": "4de5441f1776f083", "unit": "dc33fc26c1bf7c67e5439ddfdad06c0faf16c796e0fe8a46b6e6a541e708676d"}
{"family": "access_control", "fold": "outer-3", "group": "smithyhq/sqladmin", "label": 1, "pair": "d76d5232a8587127", "unit": "1ec32d427822d1ddd3b0d7245f5771ddeb56c146ac8ff9959dd85760ff659a20"}
{"family": "access_control", "fold": "outer-3", "group": "smithyhq/sqladmin", "label": 0, "pair": "d76d5232a8587127", "unit": "5143b19a1a1e6b9162b80594ed7fb5c0c57ed0b8abfd793bfc71089552d57638"}
{"family": "injection", "fold": "outer-1", "group": "soyuka/pidusage", "label": 0, "pair": "5f6e814444b4acd8", "unit": "2e8aa07b1aa4bfc5349b0d4464f3f2a6500aa55ff765b0139e9901ed34151ffa"}
{"family": "injection", "fold": "outer-1", "group": "soyuka/pidusage", "label": 1, "pair": "5f6e814444b4acd8", "unit": "e26c726c08be761b22fcc6ec17fc1fa7aba4769f08957f024e5545906df62797"}
{"family": "output_encoding", "fold": "outer-2", "group": "spatie/schema-org", "label": 1, "pair": "888e955fb6cfb4fd", "unit": "6f1a431ca97afc34ceec48393eb134a8928a312e6f761eb907c6b5855cac6208"}
{"family": "output_encoding", "fold": "outer-2", "group": "spatie/schema-org", "label": 0, "pair": "888e955fb6cfb4fd", "unit": "7b62415d17569924d16238a4aa28ab1af2db6561f56be9f59d8075fd1bc4a941"}
{"family": "injection", "fold": "outer-3", "group": "strangerstudios/paid-memberships-pro", "label": 1, "pair": "", "unit": "ac905d12b92954263c046bd0138a682cea86cf8476336d6baa664ade28d34ac6"}
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 1, "pair": "218a45d869bd7e39", "unit": "309ebac468b7ab054c8396ed495770385ae3590caba43709745ba2d1db4b6d54"}
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 1, "pair": "", "unit": "3cc8613813dec13cb1b7b45b296688da883f0b36cd2c4fc1da6e81703fb59ab9"}
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 0, "pair": "218a45d869bd7e39", "unit": "9bd2331dbc260451355307e4c37bae17ad6676654c66181934e1b35fb174cc5a"}
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "b087ac4d0989e0b72698a9644ef4eac8073b4a15bb21730d63e4730b5cc58a60"}
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "c5d1e8d19560c997018fdca12eb60f1799be76c33be058b88472f5a2fc6709be"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "22befc17a75a688e2f9358d2ddf9a7b197da44cd95dbdeae5e0191b113ee54d3"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 0, "pair": "63f496d9ff53e39b", "unit": "6a98f8145776c78793b37156e9a81757841bb6de80016ab4fa575209d724db3f"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 1, "pair": "63f496d9ff53e39b", "unit": "7ac60c96c15f8517e089cc2240aedcf1fc30e54a41e35fe03c6b965bc393fbd6"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "ae39cb55ef2a87c5347e0ddc40975f142450e82de4909e93ba62a7ed0efb4134"}
{"family": "injection", "fold": "outer-4", "group": "thenables/thenify", "label": 1, "pair": "d1b19898f8deaa41", "unit": "3763c7b407ad471820b5f8fa25bde0ba6d91e15c07c56681de4b41add418b0b1"}
{"family": "injection", "fold": "outer-4", "group": "thenables/thenify", "label": 0, "pair": "d1b19898f8deaa41", "unit": "bc8b934938c30687f3d4f982392d2cfa9504a916b463ccb5c46fe8a30eede488"}
{"family": "path", "fold": "outer-1", "group": "thumbor/thumbor", "label": 1, "pair": "f5f33fa6fe7b8492", "unit": "938bbf146a9037e687190bcee82c047492dffbe90efea40a49a647f81a0f1ed0"}
{"family": "path", "fold": "outer-1", "group": "thumbor/thumbor", "label": 0, "pair": "f5f33fa6fe7b8492", "unit": "b90c9f7e0011685f1a8e853a9ed0ae37fd437ef6d6eb0ba17c240aec2c22b603"}
{"family": "deserialization", "fold": "outer-3", "group": "tomasnorre/crawler", "label": 0, "pair": "f74364dc8e35c8ba", "unit": "11e760e99c5c852676c9288e18a250febb3854e9047b51e8b9ecc9f3b423b6f5"}
{"family": "deserialization", "fold": "outer-3", "group": "tomasnorre/crawler", "label": 1, "pair": "f74364dc8e35c8ba", "unit": "5fc980194040f4c4d803075d4085618cc0260ae8f3454b26b9b704d30200ffea"}
{"family": "output_encoding", "fold": "outer-3", "group": "tornadoweb/tornado", "label": 0, "pair": "db1c51833994ec55", "unit": "c2ce8320c778dd55434d7413844797b68e6fcabc4dc509458a9d60ca70b88c60"}
{"family": "output_encoding", "fold": "outer-3", "group": "tornadoweb/tornado", "label": 1, "pair": "db1c51833994ec55", "unit": "f2d6843547af1276ae8a4dff3a7786913fa79dc69540d4fbb95c4c091e27c7f2"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "truelayer/truelayer-dotnet", "label": 1, "pair": "f4690679bf9d80aa", "unit": "1a4e24df8d08672ecde4709a77ea5c437b2198fecff8bfff373b63c9eaed6e63"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "truelayer/truelayer-dotnet", "label": 0, "pair": "f4690679bf9d80aa", "unit": "78711973ec84867671314dd7878769092b32f59860eafed50a0f6979daee76fc"}
{"family": "injection", "fold": "outer-2", "group": "ultimatemember/ultimatemember", "label": 0, "pair": "85c15e3fc2b370b2", "unit": "0990d8f41cb07c1cf217812e51a7025381de4ba68d1be3799e26241d81dcea67"}
{"family": "injection", "fold": "outer-2", "group": "ultimatemember/ultimatemember", "label": 1, "pair": "85c15e3fc2b370b2", "unit": "9da1d63f0c4da270f66eb87bf9c342ace883a0aef78c4643e254159b5da8fce3"}
{"family": "output_encoding", "fold": "outer-0", "group": "viewcomponent/view_component", "label": 0, "pair": "8f5a6abedd742bbe", "unit": "71e3ced643a244100cc3fde234f5b4b27655214c7ae0ff67ab61e3e48a8fdf78"}
{"family": "output_encoding", "fold": "outer-0", "group": "viewcomponent/view_component", "label": 1, "pair": "8f5a6abedd742bbe", "unit": "7c8cbfae87127042f9f43621345b4f7958bde27a3ff9db76254063e80735f3ce"}
{"family": "injection", "fold": "outer-1", "group": "vincentmorneau/apex-publish-static-files", "label": 1, "pair": "c9538342f2d7efca", "unit": "2b8db57d65ef1d67c457ff5a336e8a542d03387395d69d5e16815c4f00f40723"}
{"family": "injection", "fold": "outer-1", "group": "vincentmorneau/apex-publish-static-files", "label": 0, "pair": "c9538342f2d7efca", "unit": "e2ed113dc21ad5f7caa71ba4a5ece8f8f900a97fd0d00dc85f7ac79ff5bed19a"}
{"family": "injection", "fold": "outer-1", "group": "vvo/node-whereis", "label": 1, "pair": "8acd426c5771b37c", "unit": "983e146a60e061254b15dc8dd573cbba57f8dd0358f23ea1b9c313ef69069af5"}
{"family": "injection", "fold": "outer-1", "group": "vvo/node-whereis", "label": 0, "pair": "8acd426c5771b37c", "unit": "f80dfec45231e4778078cb10f00128ed445177cac0a062fc2df5ced5c0171f21"}
{"family": "deserialization", "fold": "outer-4", "group": "waterfutures/epyt-flow", "label": 0, "pair": "9b84800ce5c395cb", "unit": "9b7dfdceae03fd86ad3260a03f85bcdcff8f32f10c265d23743ff8dc98eeefb8"}
{"family": "deserialization", "fold": "outer-4", "group": "waterfutures/epyt-flow", "label": 1, "pair": "9b84800ce5c395cb", "unit": "b99dda150c0da3defa5f4c527ef6c402f5589dfb2f2f03f3e5e424910eb48c53"}
{"family": "path", "fold": "outer-4", "group": "webpack/webpack-dev-middleware", "label": 1, "pair": "ad64e8fac4ac3ee0", "unit": "36dfd01915a553d86576e8ecd3b737b8e2b297aa542f58cc83b448060e74398a"}
{"family": "path", "fold": "outer-4", "group": "webpack/webpack-dev-middleware", "label": 0, "pair": "ad64e8fac4ac3ee0", "unit": "c2b6d40f130d35bbfc16fab9411c664fa06321c59fb1d901a1987a69cb09c72e"}
{"family": "access_control", "fold": "outer-4", "group": "wintercms/winter", "label": 0, "pair": "d48c08b19f560848", "unit": "03d02c9e25eec0f380b125279fce35e8867065bf6e1ec75f9f39729ce6efdc77"}
{"family": "access_control", "fold": "outer-4", "group": "wintercms/winter", "label": 1, "pair": "d48c08b19f560848", "unit": "abbea69b7d0fa68c3ace442c25597e128cde354b417787b5913fd10d666c0d19"}
{"family": "output_encoding", "fold": "outer-3", "group": "xyproto/algernon", "label": 0, "pair": "3844ed3d703d7ef4", "unit": "445efda074e4d9f6c1634225eb08a3c3eb1257311c995ce891df2c791a6f5c8b"}
{"family": "output_encoding", "fold": "outer-3", "group": "xyproto/algernon", "label": 1, "pair": "3844ed3d703d7ef4", "unit": "8ac27b08549353ada819caaf99a5180d2f2f515b03aa8cd99a5d37b527ea3a5c"}
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 1, "pair": "a63eae7d621a985e", "unit": "04ca0b583b09c3b4f67bd2496a4607ecb134d24b7bdaf8960fa151e40ebcc447"}
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 0, "pair": "a63eae7d621a985e", "unit": "ecfa2a67726ca9d3e2621a4d8be60145095b76aa18ecf0268212341d63752871"}
{"family": "deserialization", "fold": "outer-4", "group": "zumba/json-serializer", "label": 0, "pair": "0d9f529b7f0084b6", "unit": "35aa369236f510634b3aa7b54b04e9da0f80877e515786a888e760f8fe33c524"}
{"family": "deserialization", "fold": "outer-4", "group": "zumba/json-serializer", "label": 1, "pair": "0d9f529b7f0084b6", "unit": "ef4f8613c0914eb815504e00cd6ee05df38352fd3b27826798e6623594017042"}
@@ -0,0 +1,65 @@
# Experiment exp-003 (learned-decision-engine Req 6, Req 7.3/7.4): the BLOCK rule. Registered before it runs;
# `ousast learn experiment register` records this file's sha256, and a changed file is a new experiment.
# kind: rule -- both arms decide from the SAME scores: one scoring pass of the six slice-2 incumbent programs, replayed
# from the response cache only (zero cost by construction: a zero-ceiling meter around a refusing client; a unit with any
# uncached response is excluded and counted, never asked; the meter's calls and usd must read 0 or the run is void).
#
# Threshold selection (arm B), fixed here: per family and per outer fold j, the candidate thresholds are the raw scores
# s of the training rows, i.e. the family's paired units in the OTHER four outer folds (each of those predictions is
# itself out of repository: retrieval and demonstrations never cross its own fold's boundary). The chosen threshold is
# the lowest t such that, among those training rows with s >= t whose majority verdict is not `unsure`, the Wilson 95%
# lower bound of precision is >= 0.95. No such t -> fold j gets no BLOCK. Fold j's units are then flagged when not
# `unsure` and s >= t. Fold j's labels never enter its own threshold (tests/test_learn_rules.py).
#
# Pre-registered expectation (power): a Wilson lower bound of 0.95 needs at least 73 flagged units with no false flag.
# The frozen units (353 in 146 repository groups, 244 paired) hold 38 paired positives for injection and 16-19 for each
# other family, all five folds together; the training rows of any one fold hold fewer. So arm B cannot offer a BLOCK in
# any family by construction, and the expected verdict is inconclusive with arm B flagging nothing. That is the finding
# this run would confirm: on this data the bound, not the calibration gate, is what makes BLOCK unreachable. Arm C (point precision >= 0.95 over >= 10 training flags) is a secondary,
# non-deciding arm that measures how far the held-out precision is from the bar when the bound is relaxed.
id: exp-003-precision-bound-blocking
hypothesis: >
The calibrated-probability BLOCK rule (Platt map + calibration gate, ECE <= 0.05) is the wrong instrument for M4,
which asks for precision on what is flagged. Selective blocking -- a per-family raw-score threshold chosen on training
folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95 -- reaches the M4 precision
bar on held-out repositories where the calibrated rule offers no BLOCK.
kind: rule
programs: # the slice-2 incumbents (benchmarks/measurements/2026-10-02-decision-engine-slice-2, program.id)
access_control: 745cb1d76bbb09dc1574d333d2118489be62e39886517894d3699fa565b0e8c2
deserialization: 3c0c2831be7d78198a62351661c37867363c501cb40e3ed7b0119c155af09654
injection: b207a0710e0abf053b3b9397b25f9ee1cab4679144865e3a82c1b9be2c7b4982
output_encoding: 5d768f16724562c381c09949ff3df9cbb11d58cbb7a9c0515d0dc3f858d3818e
path: aea396680a4874dcb6de70e240ca0480c83424debdd3710bc3f1ea6a612661f3
untrusted_destination: 1242b3973fc89056aa08e00843043d68235df217d8e3ef8a0ea0e0b81c142753
arms:
A:
label: today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p
rule: {name: calibrated_block}
B:
label: selective blocking -- per-family raw-score threshold from the other folds' paired units, lowest t with Wilson 95% LB of flagged precision >= 0.95
rule: {name: precision_bound_block, bound: wilson, precision: 0.95, select_on: paired}
C:
label: secondary, non-deciding -- as B with point precision >= 0.95 over at least 10 training flags
rule: {name: precision_bound_block, bound: point, precision: 0.95, min_flagged: 10, select_on: paired}
unit: candidate
pairing: paired # every unit scored once; every arm decides every unit
cluster: repository
order: {randomise: none, seed: 3}
# primary population: paired units (both sides of the candidate present). It excludes every positives-only group and any
# unpaired unit, whose positives never had to be told apart from their own fix. Pooled (every unit) is reported beside it.
populations: {primary: paired, secondary: [pooled]}
metric:
primary:
- {name: flagged_precision_lb, target: 0.95} # held-out (outer-fold) Wilson 95% LB of flagged precision, per family, arm B, paired
secondary: [flagged_precision, coverage, flagged] # coverage = flagged true positives / positives
# Decision rule (decide_rule): REJECT B if in any family B flags held-out paired units at a point precision < 0.95;
# else ADOPT B if in some family B's held-out paired Wilson LB >= 0.95; else INCONCLUSIVE (the incumbent rule stays).
# A family "reaches the M4 precision bar on held-out repositories" when its held-out paired Wilson LB >= 0.95 (the
# point precision is reported beside it).
decision: {arm: B, target_precision: 0.95}
test: {resamples: 2000} # repository-cluster bootstrap of the held-out flagged precision, beside the Wilson bound
mde: 0.0 # not a paired difference test; see the power note above
budget_usd: {total: 0} # zero cost is a hard rule
families: [access_control, deserialization, injection, output_encoding, path, untrusted_destination]
folds: {kind: outer, k: 5, seed: 0, compile_fraction: 0.25} # the incumbent evaluation's folds (learn/compile.toml seed)
units_file: plane/experiments/exp-003-precision-bound-blocking.units.jsonl # frozen by `ousast learn experiment units`
@@ -0,0 +1,501 @@
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 0, "pair": "6b0b3cab2df3397b", "unit": "0ac37b7154242dfa637140af643fb4595e85510a17ab1a12c111441b9d293238"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 1, "pair": "6b0b3cab2df3397b", "unit": "6e5e274946d0e8f13a7ef4e37f6c7ed1828d057304a9262db0311413f18e1731"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 0, "pair": "c0d0c91757f5ba95", "unit": "285891b16ee6870af50b36dec3c2ca9a90ee15d9e14851c61e66ad443e6eab59"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 1, "pair": "c0d0c91757f5ba95", "unit": "f33fb2413edf1499a524c4592b3b2f93d166be845f5b65291969a086a4bec334"}
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 1, "pair": "f56f2a107ae591be", "unit": "350149bf9cafbb547f500b714a164734d2361b22e86fc9819c7bad3507d8458a"}
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 0, "pair": "f56f2a107ae591be", "unit": "ad125319dd2d4c9d2d99a1eb34bb4cfab9f099a96e7547d0d59a8ecf1c990347"}
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 1, "pair": "ee577ee4b6076fab", "unit": "2523f55d035d1a772a842090bc335e5d755b2fe15b376dd5b6b9e30b9dae82d1"}
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 0, "pair": "ee577ee4b6076fab", "unit": "ca0f6c160d7839204f6fcc1709bcbc749a02a0b3b8bb648e2a238eacbf22ed78"}
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 1, "pair": "c95b6b01d0c2db2a", "unit": "15540c23be0c42472a49039658ec2ec0bbd8c6b318e35dd040a9d427370e0ef0"}
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 0, "pair": "c95b6b01d0c2db2a", "unit": "542471980417e964182d7d3db59a05fe12bdd072449f3bd77f7daa5515dbc909"}
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 0, "pair": "29252934ef4bcfcf", "unit": "63bae9240b31f9cf1dfe00d9b01d4d0ffda95931c9e7d040df1fd718f04d828b"}
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 1, "pair": "29252934ef4bcfcf", "unit": "cce2af1743834dcc3df58827ac263a2d63296ca1b2a2f9bdb811954fb5767c33"}
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "bcc636b23dfb5f570d8864b17fe07b6b4fb52b65a40988c32011f1422f74d504"}
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "dbcf1e67a1be7ccf7d51701f9b8339339164055135443044c23a446b16b3d934"}
{"family": "injection", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "50408f68114ecb4de3bebd836d52d9afad86b22dd1843641ac8a49b617c09e33"}
{"family": "injection", "fold": "outer-3", "group": "anxolerd/dvpwa", "label": 1, "pair": "", "unit": "cafb0f00f80dc6beeba40eb95c764bf9a3569193070ba321c28677dfa425ec88"}
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 0, "pair": "07317a2258292656", "unit": "a9f292d7cb467def582d9c0b31a397b13cdaefa77380ce971bfe9fbf535bb660"}
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 1, "pair": "07317a2258292656", "unit": "ed363d0dfa620707be477b90eb4b83a901bfd402040b918aeee6f06b2c5b795c"}
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 1, "pair": "1544b6dfa2299d60", "unit": "d45018fc4ce3dfd715c7087d4d197ad4108c0c8b6625c02c4d4da0adb9f6a140"}
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 0, "pair": "1544b6dfa2299d60", "unit": "f295c7dfb095595a4aa985c58688e4757c29be9e06b7a4ca55035ac5d06b1ef8"}
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 1, "pair": "fe0e104d111c2555", "unit": "088036fab9962be2b96f9162ea4f32190575678d3c5ba96bdfeac178bc22b2c4"}
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 0, "pair": "fe0e104d111c2555", "unit": "7fd2b56a816e25358477eee64487fb2c77f34c652703ad0a7120c368492017a3"}
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 1, "pair": "b6742570f7f0e026", "unit": "7c6c1d27aee6557f1b6bf371b8209003f0d2d1122cfa35b6629eb43f38769cf4"}
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 0, "pair": "b6742570f7f0e026", "unit": "df8300d84dd784d0be46969ea432ef97b8ce611bae7a896ed4f349bd4d6897db"}
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 1, "pair": "0f584bb97a154488", "unit": "222a84ffc7c58d45e8f10df221762b23bf20d0287ecd20f375558134928886ca"}
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 0, "pair": "0f584bb97a154488", "unit": "25e4c9d2941f59f072385a7bde3fd0bd76f9e986ed24b06ce81962ab447f5094"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 0, "pair": "b15b924b4f7b1ae1", "unit": "2fa95fce0751c97db1a12a80ddb249ff5c54ee31c7dc3cd47765099c7b673e52"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 1, "pair": "b15b924b4f7b1ae1", "unit": "52be8128053102eafbee67c2d3aac88c83ca290b943ab2f2cfdcd2630601c7d9"}
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 0, "pair": "d31ca58f2a1d27d1", "unit": "2ae40ea61d691060e7b9f6c8e910e3aa0edf16bc6f88bec81aa9a7dae5cf5603"}
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 1, "pair": "d31ca58f2a1d27d1", "unit": "4876728701698045e65eeae6e11e133c30f43b09029d7777d6d780f7d0855885"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 0, "pair": "bb3d7baed62e6bb7", "unit": "e8ae88fd2b677e3f64ac94bddb012e3b903afa294ecc87774d10087c0d50489c"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 1, "pair": "bb3d7baed62e6bb7", "unit": "f6b0d241891dc1d897d74d98b6ff7ada2c1f1ce7874ca32097630c06ca53e389"}
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 0, "pair": "d9d8cfa38981847f", "unit": "864f1800617f5218a2300ce390ba7922fe1bd0a182de115952342b61d627bb16"}
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 1, "pair": "d9d8cfa38981847f", "unit": "fdcda2307e20ef1b9247b7082a85337f457dd1f4e648bb8264077bf983c03271"}
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 0, "pair": "1f73ad2576d52a64", "unit": "0bc9634940734cf7f41628f220032fb7e2fd2ff5b584afc519875e90ea0be5a4"}
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 1, "pair": "1f73ad2576d52a64", "unit": "ddb59b2dc39715281a6d48b32e7ca4ec0354b85b953fff55139579f83905b7d6"}
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 1, "pair": "a7326e6daa507b79", "unit": "793ebf4c8e0f46d8ce3b3d090d2f6290d4167931a3b574f8269b90008e3dcde7"}
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 0, "pair": "a7326e6daa507b79", "unit": "b8d25f337fad527544c33c82f473015e08399457276fc480477f3cfcfc9c5b8a"}
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 0, "pair": "211ba072bdfa67b7", "unit": "3217f9d672a4a1d78005c8fb66e1bc60a542cb26bd8c46903402c765c123d60a"}
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 1, "pair": "211ba072bdfa67b7", "unit": "8370f9624baa0d144e56b8087eea897b98e5b2afcd6eb3a029d03e5b4b69d584"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 1, "pair": "baccf350d5cc9911", "unit": "50a3ca70700c8b41874c43ba94e6b7ffe0aa28b8930ff941a40767e1057065b5"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 0, "pair": "baccf350d5cc9911", "unit": "62927a118d3662c81450e19d49f4146b66f29a58d55ec0d1606ed521b1883f12"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 1, "pair": "697f34e19bda57f3", "unit": "775b6d900ec1ba71e4082ed49bb94439f38066628eed91461eecf0658caee78a"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 0, "pair": "697f34e19bda57f3", "unit": "9e855586c685b3c62ffe32407a82966661d5839e2cff742f2676acaba645a97a"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 1, "pair": "143b900b99a46cf4", "unit": "200f96cd784ff5c39d8bba22883f22bc8a7c87c6970a7c871802f48a45548337"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 0, "pair": "143b900b99a46cf4", "unit": "7a0d62ec424849ec4681f4333fbb077e8239054d1e964bd99b40847ad7ccdb01"}
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 1, "pair": "4b4e81797ca48cbf", "unit": "a4d17d85973d024a615b89334ad5a2f46afea660ef5d0bcf2b2196c998b1caf2"}
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 0, "pair": "4b4e81797ca48cbf", "unit": "d604e460d8f5291124ba6fab5f9c94c7dfda61ea5f6808e512162334d6d6db4f"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 1, "pair": "bcbd06e53147628b", "unit": "0b5cfe2d64771e30e50ff92cc409b8dd5e30079c252918ba03774cb5198a73b5"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 0, "pair": "bcbd06e53147628b", "unit": "b805976c35330452af6c9f814ff4617b7b9ad0e82e1fa4440a443f93f0772c4a"}
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 1, "pair": "c12dbc8a50f7fe9b", "unit": "04beb8a0fd054b92c2dd0ed83907330fdee6124d86114df948ac72a297a7d51f"}
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 0, "pair": "c12dbc8a50f7fe9b", "unit": "c5029016a6fd2e3f9df59dba0ce3c173bbbb0d2b1de22d151086e01e1b272119"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "0e213aeabacf4205db024ea3c557e720c3aba911424555784917e967d6404867"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "e8bc65415bd42e43", "unit": "4920230ef773595a9fd9a92a090dff42b48499a1dc5fc8831f941dae55cb2538"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "e8bc65415bd42e43", "unit": "49209a810f207eb61382e2ceeab7076936838a5652546f21c187d066e220160b"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "9e2964884cdf281fa3f18ee3481aa62f9a8a0a195d605da2df5bd71876fe311a"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "", "unit": "f9cc6f1bbcd59f5dc0de8f96248f722ea0679f8afa47c53f70832151ce7616d3"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 0, "pair": "e69463f945f9a0f4", "unit": "5feace7b28e260175f80c682e9009025aea6fae3dc75409cabbb955b560d0372"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 1, "pair": "e69463f945f9a0f4", "unit": "799a4f5a5b9324e86c90752f0239f059984f44e8ff1ec0dd518d0d9413e34c66"}
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 0, "pair": "71f56810b8ee6947", "unit": "c659c0e717b7dc1084e1178288f6c9b2e87fbf0ef518ba3bbaedb0fbadae15c1"}
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 1, "pair": "71f56810b8ee6947", "unit": "c8d2e035a6099fb775c8b4ecf00196c1488dbddd9eed3d2d268a4f1d2567b194"}
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 0, "pair": "768fc9db8a75a351", "unit": "adc6527469618f72b1dd7776814717cbd0056a0c625e05bef74199171ce73f1a"}
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 1, "pair": "768fc9db8a75a351", "unit": "da47663a57e370a75cc65b810b8a380aa26d12dcabb1d9b3c02641398ff7d348"}
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 0, "pair": "fab4ce020049724b", "unit": "b9be90242be627aa46462143ac85940a906232ebc0b47776b5729475b27d215e"}
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 1, "pair": "fab4ce020049724b", "unit": "ece12ea7d87b4a12c4da30faed9eb530b51fc879c271b68ba5304b230cda8e79"}
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 1, "pair": "9126ba83905eb8b4", "unit": "d8dc50b7bb40bbcfc3f3d4f1f8d3a4d17eb1c662d5f759efd18656a482dd2e32"}
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 0, "pair": "9126ba83905eb8b4", "unit": "f426f9307621b3f2d6a8d1669cdb4c19e962b94e34fbb394d4660cbde5921c88"}
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 0, "pair": "7ac5e21d9d8f4c47", "unit": "1d1efa2fd7323c264290d4d314c957c853e1c7a9c877a54d6f03ab7d2af2ca86"}
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 1, "pair": "7ac5e21d9d8f4c47", "unit": "b5dc78c7ac0b76803dc98904142c1139083fb9daafb5bff40e7b5c3f14ee03f5"}
{"family": "deserialization", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "0894a7212280bab755cd9d93b1f0ede2c51e4fca76b76e5ed9e24ef8f8cf0b9c"}
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "02694b5fb85e9e52bf5eaaab4e0b628f1f44ff3a8d5ad026cc10afdbc4222eaf"}
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "db9a3e888780501dd3c2c125af2f9d47f68ebf02f0e0db8370b2ea09c6f9d070"}
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9af96b099dedcc4005e5fcd19ad78f73864e1cfbafbe6953b4edb6260957943d"}
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9e7c2a786068a936a6c664cc229d6f39cea1fe57caf0806306f0cfc44515460c"}
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 1, "pair": "07d1dc4d299a7700", "unit": "0e664e274dfb438d2f721edc81f8469ac48387dbd5c1b43973cd7e651db46db3"}
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 0, "pair": "07d1dc4d299a7700", "unit": "da55f60378c67e91683b9720f2fa0e6659e28c7b392c050278b40713e47c84a3"}
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 1, "pair": "749eee05a507fe93", "unit": "24fbb1dffcca083382a23e48db232e0d83bbbf2ff2b325f0dc0f4c72dd1a16bc"}
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 0, "pair": "749eee05a507fe93", "unit": "640e1626f8d7d9d5c2478ede48cc7b2780370bda98ae114bae102bba892da90c"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 0, "pair": "e36bbaf8f3cfe65c", "unit": "14f7c65fb5621ac5e2068c8695ddbf482103e4320ec3b36886f6e1f08cbc312d"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 1, "pair": "e36bbaf8f3cfe65c", "unit": "c96dd379dec2ecfd99051d7a20e6ef5601821f93c46eb34ed087daf5fbed2d47"}
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 0, "pair": "b81c8c63cae892f6", "unit": "5982fe7fa1ea6bfc1c8dcb8c71cd4410de475d25cafa324147649f26a1ae1324"}
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 1, "pair": "b81c8c63cae892f6", "unit": "59916deeb8ba14922eebac61c9eac122e15d1f7590ee3cb1b97f019585fb798f"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 1, "pair": "bab3a167ab5ca505", "unit": "5ebea23aee96c80fa0d2d9239964d99614613c4ae36f709811bc19e61ebc720c"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 0, "pair": "bab3a167ab5ca505", "unit": "9bfea8d7a6ffa6d79d16f77fc355fc1bb605189513ebad561ee523d577a1498b"}
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 1, "pair": "5a91049be21aa611", "unit": "1ab9936627aa59c3a60ea44074ad44b68aea4f9c04f14c1949dd9c77ca13ff0a"}
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 0, "pair": "5a91049be21aa611", "unit": "8df72b9eab3f2fd858c40312d0a806cb67c620bd62873826c8732ba8795d9cfa"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "", "unit": "115d87f37bf1d4ab91fa3010303e81b127f84b817268b60b8fd4b9934efeae4f"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "2b8101c89c839614", "unit": "7bb7d858a670fc51f81e08c141c2bca3b8b7524254ab679e33b22ce79536e895"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "c23acf4f38945093", "unit": "82bdc0b6bb0c19e342efa63af0c9014a607461d0682cdd14bd5fea33516fbd50"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "2b8101c89c839614", "unit": "886285661b88f14e00ff8e863b31cf7b7afb303ad4aa4aa13b1cffbf1272d5e0"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "c23acf4f38945093", "unit": "bf74aae5a48156f183426a80cafca6da219ab13b2ae4b40fa88fca75acb72980"}
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 0, "pair": "d7cc37a3a54315cb", "unit": "4aecd43c6e5b81daf7aadd6a6e22f22c17c08b652a2f5121a49320ca1057b186"}
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 1, "pair": "d7cc37a3a54315cb", "unit": "88123ae860be2fdc28fd6a289cf26c3e0af0689f455cf8c15098e5dc76e5c8a8"}
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 1, "pair": "5011004f06f1e217", "unit": "6c48682d1b02ff2559d92346d192cc9dbe6a0bbe2ab2c146653110420105ff1d"}
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 0, "pair": "5011004f06f1e217", "unit": "d3592260108a838246e0cfd5ab6d48bceb0ef870e223565745dcfd68f1826840"}
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 0, "pair": "81caad96b221c2b8", "unit": "3c2dc720ed159e55629ba737665d93859def9790427289567a513c83c133e236"}
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 1, "pair": "81caad96b221c2b8", "unit": "e8b9595a320776c8e663fd0b168193c1e64e36a9052784d1219f2c4a91d647f2"}
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 0, "pair": "f9b6599018ef3b60", "unit": "7558d6ee39a311c5459e35fe6f7b12917b7f843b21ab1fbb5e485a07f53b7d78"}
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 1, "pair": "f9b6599018ef3b60", "unit": "b1a6cf450a074788ffca2bebd880435abac2c376e89cb328d7a22d277fdccad8"}
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 1, "pair": "eaf2f03db2c502b5", "unit": "38dbfbf745d7cbcc52c6fff83248d84663273fe6a1f1b014eac7776c4bb8f074"}
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 0, "pair": "eaf2f03db2c502b5", "unit": "a84d848ef56b72fc15872d88acaae1743c9cdf88095d13b1869abbec6a0d8a8f"}
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 0, "pair": "7df48bca2c9fabfc", "unit": "7b76ce8f55d61071adb367018cc3e50f04a05d08c7c4919c3806fc1cc8bf41a7"}
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 1, "pair": "7df48bca2c9fabfc", "unit": "c82ea0adc9c8366aa2b77ce73772038676fc37ff50879e873ff397958fdd7ec1"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "7f02a09b8d8f2321f2cdade74c2fa1d52442e006149a749a8be768daaa62195d"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "9b5e439943ce5c2b04d6201b7a843634aa1b13b726813c23450ff54ce03f3dd6"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "5b836889b4da4921", "unit": "a84c2ce4a4eadf265d4481ad7f6864edf1d7e3084c6ee32cbbfe454bb2946bfe"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 0, "pair": "5b836889b4da4921", "unit": "caeed7f097a3311fe2e84efd372fe4b4c140f63a80a9f07ba0b61966dbab15e0"}
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 0, "pair": "ac526f457067fad8", "unit": "50d8bafd5189fab0429bd4135d9ed400cc2fc14706226a2de98d65746ebfdcc7"}
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 1, "pair": "ac526f457067fad8", "unit": "9532b70b0df76a34170b57c74deafaba54d5d2f16a729c39b0dfd3eb2022d899"}
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 1, "pair": "c97a971a18f85244", "unit": "95c80c503ed43fe9a0370f89ea935d47badee6a9625548b9dca78e04275003dd"}
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 0, "pair": "c97a971a18f85244", "unit": "e9810872beb9c158f54048392142c2f701314f854d6dfc748553ca36ed6c08d0"}
{"family": "access_control", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "c2a2c2d85114a97f892fb1e5500fbd98622457a8c46451e1f9e958ab08a40318"}
{"family": "injection", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "29d8be8206c88ba7efd86b56a7dcf8b655e33b4a792c2d18b028c92c6252daab"}
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 0, "pair": "6a4e0bd705331747", "unit": "6db80439d89b54f60f836c415dd403fd09d21cf630da81435d4d2f9cef5d883c"}
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 1, "pair": "6a4e0bd705331747", "unit": "bf257e4f0812012be330fe119d07d29253d291161f435262787f491762c6bfb1"}
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 0, "pair": "f96f0fb245948915", "unit": "0954e6931373915c204b9ec8d2b8ae6297a95667bfd41aadf9035625ad93967f"}
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 1, "pair": "f96f0fb245948915", "unit": "5e09cdae2504e834d591b6f92a808427f07550364b5419899f7e6f76084fc377"}
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 0, "pair": "bf11f138fe39675e", "unit": "56f0d0b156992af56141d6368fdb0c5376fc7a44a03991d754f97202b9d77008"}
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 1, "pair": "bf11f138fe39675e", "unit": "8271af5017894167556893da2070188060d06eebc87f27db45c46234c359631d"}
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 0, "pair": "9d937c2dffb4aadc", "unit": "01ac12bb06e660d0aeef8b1e26dddc5c99d52cc333f7af6d1981eee2818e64fd"}
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 1, "pair": "9d937c2dffb4aadc", "unit": "9612edf0f927048335e83e9925f01be9aea9b772bdbdc568ad9cc0a17055662a"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 1, "pair": "d4f648048e45f2ea", "unit": "08807c8110b8c1b8635a6883c229fb91f2b9566db8b6c252448a0db7b2ddd47e"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 0, "pair": "d4f648048e45f2ea", "unit": "10869c931053bf77ecaa98a47f56dea27d643d1abed2a4758bcf48a78547ccc9"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "662e1f36c230d7f1", "unit": "0b73332b25413902c26d5b5dee6767465eed99ff71a814582988ffe0d8ec1a30"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "8c8e46d0c7d361b784baff9d70f38744b2805ef1ee0aaacaa616bc20a2c6f32d"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "aca938682c196fd4c85b39b89eeedb97c35f6a5a017651fe9c03c5b727cab7ea"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "cec00d1509a129364da6ab8537fb3f433fd847e26d073c90311ccdaf7a1b39e7"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 1, "pair": "662e1f36c230d7f1", "unit": "e0f58a7ac5627de3e7b6cfe5add41d18b0f64b998c1cc788d42eb2cdbbc3fe41"}
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 0, "pair": "2c251a645094f389", "unit": "0d41cc634771048f72d07c4aa9feaed669d0bfe3844d61dcdc525a0f899617e2"}
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 1, "pair": "2c251a645094f389", "unit": "ca8288d0b8b94dc6b7e4c681936ea610d295fe949cba5f64f09b6de134ef9875"}
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 1, "pair": "2ac5ec5011615c9b", "unit": "2f427f403af3cb58d776437736c2389805543413fdb91aad98979158c6d00de3"}
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 0, "pair": "2ac5ec5011615c9b", "unit": "cddc8c5bed0841b80e7b20d0b378e858f8e4ac37fd7b890ce76611d989857f92"}
{"family": "injection", "fold": "outer-3", "group": "frangoteam/fuxa", "label": 1, "pair": "", "unit": "0c7830c2912ff285855d8a3c8e9b47db4ca4af840baa66d4b5d3f4422f46f95d"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "5383a8c2a78fabd3d17a5b3021125953b5ebbee61f8900c30d016c56b1a3bad6"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "360d8bfd2f116e98", "unit": "7c1989a4ec3622bdaebcf1161de850f22cfed041cb38830ca1ded4fa99d40466"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "ec8c706cc2b20f36", "unit": "848688dcd0fe8d46e6762cc8aee1d82e8fad8a2a044c94b547aea11049c85bfc"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "ec8c706cc2b20f36", "unit": "983cc9231f91baee4e2d626193f3b88aa5b09907f6faab870949bdc9c336b816"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "e4ad86416d5c6668", "unit": "98ff1dff2afe80709b8e61a9c95cfeea12b2110c13b7120478268b4762e2c992"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "360d8bfd2f116e98", "unit": "a1a825d23e34757c8e961e43ae791253e62682e130acdb846b58118588006a98"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "a7fa4207508b3989baabc7a1f0ff2ea5229b366c1cb106e7fc945134b8b106c8"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "e4ad86416d5c6668", "unit": "ef5c53f3114d0057e694df13dc07cbcb532c3b0d59150eda269b87b021d7a6e8"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 1, "pair": "9ae575048504c690", "unit": "176acf077fd58c3e6f940753179ffa3e8316661f355d94acb5e0345b51925ca4"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 0, "pair": "9ae575048504c690", "unit": "228b34ebc2090b5239c7821d28366ddc663e7118d015a79b45dca6618d5490fb"}
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 0, "pair": "d6edb5496b3fdba2", "unit": "9acb8017ebadc480515a664181c427f37359edcc1dc6f344f368a87e0c191b3f"}
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 1, "pair": "d6edb5496b3fdba2", "unit": "c1bfe63148bc700c812fde95ad9f83d456e4fa04e10c6f5394072a08f73174c6"}
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 0, "pair": "27e1c8e8ee93a892", "unit": "47fb89b132d6de48674c37965579e727d922bef1796db70f8b9f862ac47cb9a5"}
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 1, "pair": "27e1c8e8ee93a892", "unit": "79887f0ca8b05d6599d55167d8a9faf67bdb2254d17c143141da44971077002d"}
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 0, "pair": "a3fdf9ce3441ec86", "unit": "653236018929ab8534c27704a20ec5d358047bc66e451620d2ac32081c91d333"}
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 1, "pair": "a3fdf9ce3441ec86", "unit": "b70ca2c43f5929e7b885bd728349e721fd85d7605e2f722a7e8cfe0f0d477b0c"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "groundhoggwp/groundhogg", "label": 1, "pair": "", "unit": "92400e83890882a66ffdea1a405e2354a195442e176d85fb707b0442c44d45ca"}
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 0, "pair": "f9dcf3a0fb0f0a14", "unit": "7ca6dd4f673976b000a657f885af3463740419ce41e14f972af0c5483cb75a7d"}
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 1, "pair": "f9dcf3a0fb0f0a14", "unit": "86c1db3a5e9f843d6546146449171e0378ac61d6c8f44c6dd72fd379cdbb188a"}
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 1, "pair": "3e71f79348a6b770", "unit": "200250ebceadde4cbdfec2802a4f9fbfa220dbd7fa4c2dd5ed717fb1f3ea7447"}
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 0, "pair": "3e71f79348a6b770", "unit": "b9faa5007c8cbb67ada3f1f026ca59e0ace6fbb9c3d1f1369d4521a7e323e7da"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 0, "pair": "8972a0f56ee57477", "unit": "6b987df2960828dcb1b2b3f190a903a40361d58117d63321d671aecc6c2cb570"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 1, "pair": "8972a0f56ee57477", "unit": "ca3fa87dd37435c01a7121119912c1a1e261259c11805c0b66e7d4b315640945"}
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 0, "pair": "6144baeb491f4a56", "unit": "f5c22ce104e9593e5272b6e8b3c6f3b61fa4578552ccea80e6021ee90bdb3ffc"}
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 1, "pair": "6144baeb491f4a56", "unit": "f687e7e95dbe47f602a1e0002a44f94fd48e5b1c33f77ecebc6c2d814fcbf450"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 0, "pair": "2ef4471c66e5e4ab", "unit": "ad7c8e368fe1699a044e058d5c10b41bd9fc0f91c3f3c384c5dac40af39e71f6"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 1, "pair": "2ef4471c66e5e4ab", "unit": "d888baf7194bbe7158cb3932f58b47a9a235351d388d10df82a130ddd54c2d46"}
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 1, "pair": "5628c54152216646", "unit": "47d081c9f6060218967902f694e61c88f64f57c65bec6fc3073afeb0b77d117c"}
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 0, "pair": "5628c54152216646", "unit": "e1297bccf9c3606e0b41d762a69ff5d0a33ae4f7f1506071af6b9cb4a177ff56"}
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 1, "pair": "e52aa814c36ace1f", "unit": "2d6d74364f1982e91f27f1c995b8699df1fc8f8dce6e1e2cf274bf01b6e9494e"}
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 0, "pair": "e52aa814c36ace1f", "unit": "4727471fea2cf0a92d43b4b1503f1339a7dde87f872ba8064dacbf855c57705b"}
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 1, "pair": "a07929b3fc1cfc28", "unit": "0637029194bd7378237b4dc944dc5cf9f2b63db9796f27410a74a9d567b66dc4"}
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 0, "pair": "a07929b3fc1cfc28", "unit": "ffd286c8588a77929639a89e4a742e15306b642af14aa906c721165df804ccd3"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "6548e1527e82bd99", "unit": "5542613f6d2eb9421b8213b8bb864894f0ecf9fe64945681019230ef0d75cfaa"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "", "unit": "6c2adee34e9e8384535ec1073eda60e2ab528ac77c36747802c303e1f5e9ffe8"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "7ce70e9b20107cc7ebb87d173da765fc53ffd268e521749a06a9a1f38d746845"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "6548e1527e82bd99", "unit": "c3e54975464323e58ec96ee4d83e7bc08baf94553d553a8dc3fb28cd9c70ff21"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "d3173c1e43eb6402a71ea2dec1592913dc078a2194ba1c1d509ab129a3180f6f"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 1, "pair": "0924befc49c7bb18", "unit": "e7e4872db40f0420547ed622fbdd7a072b64a7ba466d1ccfff68e691cee6ad99"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 0, "pair": "0924befc49c7bb18", "unit": "f7d95f43fb2df20ebc4c37fceabdd00ffd8159f404ca384481f1764252382388"}
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 0, "pair": "d4ee6cfeba71e398", "unit": "e79582de3031bada2617a0667ef7ff5cd7183d4aa885f55a665cf0ec88d17624"}
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 1, "pair": "d4ee6cfeba71e398", "unit": "fef35272853e7f3692efd54daa01c12862953352fa6ccbbb5e3f7fea70c63d78"}
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 1, "pair": "772e18821701d60f", "unit": "89a0496c7663bbfd67d660c9f481c12420da2dd1bdb29dc079735ee7a0a49626"}
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 0, "pair": "772e18821701d60f", "unit": "d3241ceaa50a7d23a4650e6a2a55372b08ec96b35b0bede844bc40e16e818ebc"}
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 1, "pair": "250de1302c7e541d", "unit": "5d03535f2df2acdbc319d9e6b20da65d51f3e810b80f3c8f89b5074a971a3f1c"}
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 0, "pair": "250de1302c7e541d", "unit": "78f977cfc096e34b4eada7f798ad2973ef45728d0d01bfab6ac9e2a3257827ef"}
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 1, "pair": "c10bfb006f2da55a", "unit": "1ea4b86dbd84581b1d7b7cabbab9f28c8b213416a60778879666ff42d23fd7f1"}
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 0, "pair": "c10bfb006f2da55a", "unit": "7647021f742da24c660b2d407886771786e957076c4cf336def560a570dd8778"}
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 1, "pair": "a73421efb106ab0a", "unit": "1f6f207a758582de0c0e6b331b19fe119abe18727d45ccbbe09aabb795f3a292"}
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 0, "pair": "a73421efb106ab0a", "unit": "32fbdad39eaf5177db4ad0a11d53e785b30bc7b2376ad9d460f29a08429bd0c7"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 0, "pair": "0458e3fd444409f9", "unit": "b521216bb4106738b3571ad3902d08f6230aabd88cf3a8f9f26f1c8ec01304cb"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 1, "pair": "0458e3fd444409f9", "unit": "c87d2871907798fa2e73d31381e9d63089b0388033f096d247373fec07c2cb36"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 0, "pair": "6bdccabcf5b9fdc9", "unit": "3d6ddca5eb264d88411b86755909f1d2abddd09c129cabd54c96ff26b4105316"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 1, "pair": "6bdccabcf5b9fdc9", "unit": "e055b898802279bc63204cc601e29f7f1018efad7228cab0cce00ea5866ecc7b"}
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 1, "pair": "91bc832e70680c92", "unit": "1060c63ff0fae3af567448f3d4c60d9c159b61ccc358964243eb152a9ac38865"}
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 0, "pair": "91bc832e70680c92", "unit": "369dffd1e730d7997813adfbebf1481cc68987cae0d0085e47b91a26d5330cf8"}
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 0, "pair": "137fb3496a3a75f4", "unit": "7e8b6c484999229b23d50d9f5a4c4c5b2a8d954a5d64230981b1c6705b760fbf"}
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 1, "pair": "137fb3496a3a75f4", "unit": "c461154e4dbd74e35a4ed2e85de7b076f0582beeec459be28104c38f616c7565"}
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 1, "pair": "a4144258dffb3b8d", "unit": "7ab301e1614979f44c353e665540582493f78ef4bfc41209562c19b900710f8c"}
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 0, "pair": "a4144258dffb3b8d", "unit": "ea95e8387dff32ded9dc7b04c11a76a545b4e2ea6db0076b9b13971a59904bb3"}
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 1, "pair": "858330e01d3cf9bb", "unit": "033cda2b7cffa5a5fa6abf0b7f2cd27ba78d916eb3cab8c79decccd8776f4a3f"}
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 0, "pair": "858330e01d3cf9bb", "unit": "d44c4555a5ae19aa17b468ec5cff56b33a0b221df52e04d8abcb656399499dee"}
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 1, "pair": "c6a49339d4f0c798", "unit": "dc2e4cd5bd963643702d2b832ae170dd25744162e4076e36687256a09e69d089"}
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 0, "pair": "c6a49339d4f0c798", "unit": "f83cd137b85e651ca5299c5a42176b797ed4964cbef31418c96d63c52fdb65f5"}
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 1, "pair": "4b6526d132e9ac86", "unit": "8ec3596444f3045684242840e9a1df19879383f2733aca8f8af6f6cf8a96d388"}
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 0, "pair": "4b6526d132e9ac86", "unit": "f3d39162fa14446b207aabe7bf3c913dc32a9637ac83352ea5cb2fdbb43720a8"}
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 0, "pair": "80abf1d5c4a3f6a5", "unit": "3df3c6d9941352dd2861427da56b94ee335f308d0a2a27c6f745aef60349c80f"}
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 1, "pair": "80abf1d5c4a3f6a5", "unit": "4fde61a570906e658d9b14db0e5d1f5acc0c7489951421bff2afde25033ff73c"}
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 1, "pair": "5ec37c0f475001ca", "unit": "5d8175fb21a4649bdc8730358e9b87adb90b2aafc4da5fee203829663e4f46a0"}
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 0, "pair": "5ec37c0f475001ca", "unit": "af845e9e332555096dd350401977b2f07015433f3db8a040891d9470f3f9e355"}
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 1, "pair": "68d44ef338ef3470", "unit": "3020ac6c3186d7974953b1c4da65b80141b81aca5203436ae7a2d7aca0870944"}
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 0, "pair": "68d44ef338ef3470", "unit": "8fdb85a6bfec89f0715ea81e83fd91616f92c59954c092e79272154f202b7488"}
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 1, "pair": "c74525e844650de2", "unit": "2e444ad34ee5f47d73f0cfad0a10c70363337b4ddf9067ecd1b8072d8232652c"}
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 0, "pair": "c74525e844650de2", "unit": "c492d325f5ff82b5f817aa4be1440cec8595054dd0719e07a58ac5501b2ae7eb"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 1, "pair": "111a5a80a3abc2c9", "unit": "c7ae403742450deb2bd051ad1231121d2117ba56fa72a038c82d58d207cb3870"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 0, "pair": "111a5a80a3abc2c9", "unit": "debe34831afca6f8220b9952036a2d4570babdb5e321a2f8dbaec661a016bd50"}
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "9950b85412b021ebabb11eea4c490a660ad380d14c673a87c3fb568ba4bcfac9"}
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "656df5ff8e69dcdb1f9ba1ed52d734966eb29773bc745587483a355743b8eb5a"}
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "91652a4e776fbce70cde9e4a99c6da5e17da7f8d6d0404867c61eec04161b6cf"}
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e53bb3a217dc4737e8dd03671de4f6a895599024070a985a268c65a034295e7c"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "3d7aba22444811a337abcad0e4fc62990bb704d3a10e72d893342fa94699026b"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "5f18a3026a51f7140b7aaa3aa8563e8b6fdda6065d95e19e25b1dac4b9ce0e00"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "0fccc306ce80cb4cbfdcde0a2e56d637bcefa9dd30423d6c54a4f77860afe740"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "1e2eaa45661345f1b6759d968d7696dd0d96d6fba1350a00e4d0a2ebaaa2e391"}
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "8b9a8ff0e32a327df75eb8a1811464d5f34690079d5930139dd917063bdd2809"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "245d0e939b7c1ccf098c798492009e3a73ea17400fa613dd865cd369d2e7f383"}
{"family": "access_control", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "653c48df493f31c2d01c973d1e379a180fbe6a6edc83a500bc62509112175f41"}
{"family": "path", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "3719fe21d45b50b35458bcd2e263e4baee67adbd3c98adf0eaab6bc6183c5651"}
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "1209a6cc8c1548ed83d4a9a09d6a27f706a23e69cf0c261fc5e3a27536a83306"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "362c90495dc468bb732c71f62d71ec308854161de31ad18d2a870056d947ddb6"}
{"family": "access_control", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "b52e53fec9707b2b442c4dc7a41d376cf76946d2c45c2eb600c9477fc6baebaa"}
{"family": "path", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "477e3b90b92e622888dbe0e9d3a23b31271ddfa3104a5ab326eb9e9e765aa01d"}
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "660c63c4a73bfe64088ae3249e40d9c5d69a6f8a505ff671217c5708a345c5b0"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "4d60c386c801bedb61c1461fced29a3cfb50b09d6435e8617c60398343719d93"}
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "f1fddcc1354a29bef55610ee2b73eaa440192f2f18a0d22d08057b9d1e1b61b8"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "6d9886c38221b2f9aacf83749218ac5ebc354a6fcc4170e6c78bedb92b88296a"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "5550fec4a81c4c9cb22d7312d8a63e1fb2b03c4aed0c1f316d44c1eb27511bac"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8ff87dd04140f0137ea1891f220b1276156ce10a58be5a49d0c8e07be531ffb2"}
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "f24b19cc0f67247eccb274521a62741fef8f4729f0ddc9b5c1fa561a4947182a"}
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "83523c9d83380d4cdae593a593d361493e00d18e61740f84e762aad41b991e18"}
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "c6c9bde6630678377d250701073c5c223c7892e6ff8f2f8e0293dea22c5b5e7e"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "b9b15958973c83f8d7b58372ab5ba58eb191c400125579d5ae8c5b52d5d1368e"}
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "b95ec12e4e126ad558e944e2c260116ed6a63f5f3ab97fb5cbcdca75a931ad73"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "38a5545ac9ef67e15db720d1396a2d5915f9d06d6f4772012e5bc95f33770376"}
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "c8bbe9be91c976be97b7ca387048a7d4537d9f6fcc6cbad0a541d1080629d5a6"}
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "ffacd53f1bd75155e63b318fb9cba27ec03e754841e1eb39d1d4fa0b2c3dfca2"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8fd920175fc43b91b4397acffacb10ab8f7fd0ee38bcf2d2f3ed06a3fad0dfe9"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "86c9598ebe3dcda261a69ae545513f6ce73101f3980066e4506ece99bebea84f"}
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e5190fcc57505116f79a31c51aed69f8a1eea13a8ae3e7265f44c72418277ef9"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "2b807ff54e6c47e941124c16d0c27c691cef54f2203060a3a6ce812d478c5261"}
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "401aefb4b4791301ace2d770d321c6b26743b0aab7b96f8b410254c704be6d2e"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "22d186c93498e21418b6c1d0547a73374b13041d8afc89a80d9540fa658b6b42"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "dbfb6df14fe37f8c0efda5f16643297fb0550dfc5ad5805d4e4b844d967773b2"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "f0da8cfe4ac1e586aa7903d72e67b44500bbd5306d72b4b342d10613efc69222"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "2b8fd73943f456da266434470333a7775f55d9a41953e97a000fdaa3257b6a30"}
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "a612091f452e60f28f44eefec641175170dc5db04456a39a099cbb13c50b4d2c"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "4c6600facd83bbf234a32d9e28ac494fb516df6e3d9e9dc28d720ff35252d067"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "fb0ea8cab8c9430d22d1b0fdd65a69146fa95fbff94e720b4861b96538d992c6"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "e81f91a56058ac1473579767abea57064c53d7d007797a854314814e3c35ee74"}
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "5851ecd96a449f98b05f243bfda0a7cf3af8612b27f53c414ae1f4ea67242a7f"}
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "328621ef2a94632e5d37ea6267d17b766233384cde35c07e29a4eb672d40ba28"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "04b2eae2128608dad460ba8275142a41a44e32c4243a29e2a6fb2c9ebf3d8f1a"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "33337002ff5f5a189610b042522ceea254c9d107c1f24771b4f616a0f96f636e"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "02c09a783cbf058856fc2b448406f3ffa6fbfbacf9b70f4e5e22fd741a5c7699"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "956260164cb21a47c499e5598491031ef303418943f5096f225a87162b588574"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "8bd8c5aed69eae3019cc2d3654b36bdf2b31435e6a020a389cd103696a2f874c"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "1d8397ef4a9fde7b48e2e1e871bba0f960a802cd7823ecfc10e9dfaddb955ccc"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "c9db92476032eb4296b950a1eff8fa4aaa26f11542a01a5cf10c7b2e32dc6433"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "d3e0d0f9b0f4b723080989b5486395cda0a5b186a674022ea70e7fa7ab561323"}
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "44ff3b3e61c02744830c4abea872821ca7e0f94950c082740c5c48ef13604e94"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "4103e979a99233c3e71fdfe9f1cdcbab34a2c60b5351b75c5ff5dfc287b962d0"}
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 0, "pair": "f73ecbfda2bcadef", "unit": "53e596dabf90b2992d5b3e50394791859b393beedc8f84e00484e37498627ff0"}
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 1, "pair": "f73ecbfda2bcadef", "unit": "6daaccf64706e876dd505c65a62aeacc1357f3470a5cef7b1ac874032a6fcf99"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 1, "pair": "6768bfae38c6859d", "unit": "f376519889a00950f5af67eb15c639306ea70da7235d3be4a94dbc19cb096cf5"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 0, "pair": "6768bfae38c6859d", "unit": "f3fea4e334f49ef6fe6b17121fef553eb8e33a9ffa13e6d6cbc4c448da996a7c"}
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 1, "pair": "d670253fa209671c", "unit": "1aeebe2e440bdd70525c31f93180f33be93f820db5ba46a2e59c371ef83ee794"}
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 0, "pair": "d670253fa209671c", "unit": "b11c8a8a24d3809c0507228568fe7cc05ca0a22ed28cb19019adb13cb619d63f"}
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 0, "pair": "33260f3e6c8ea6d0", "unit": "7ca409c7dbc1c38ff005624e212efc6d424cb421e27b482dc8dc0d6881f4c11a"}
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 1, "pair": "33260f3e6c8ea6d0", "unit": "fc364d7ed77c7c0508da00e5ba847ec0792f3bc373d8b4297747189ef42d1ae0"}
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 0, "pair": "32ef8148a2c08f58", "unit": "2fb2c9314a483787168a450f50430721f9527f913aa0eae07298d5334cf4f137"}
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 1, "pair": "32ef8148a2c08f58", "unit": "78dd291bb2ef83ea3a87948c0ac0027c6d667b9044093e766e2bc91bebe83936"}
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 0, "pair": "54586b5a24655adf", "unit": "2ece75a0240ed36a811c381841aa6d440af2b9774ddacb1d7a31a67ec267b0b2"}
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 1, "pair": "54586b5a24655adf", "unit": "e60f6b788a196db9bb671d621189b79a5526302a6d09c2396d67fbe0b9ab3c24"}
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 0, "pair": "474f96e2a9b3d202", "unit": "4a5e8e8ebd7ecb0e252280af0cb38a1d7794757728b912525322ac26df5ee223"}
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 1, "pair": "474f96e2a9b3d202", "unit": "6a888540443061d3b37c06f0cebc70cffdfa70e0aa940626e86cbff9829310fe"}
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 1, "pair": "a09692dc94164db8", "unit": "139cf84e79c4efd026a064b2ca0a3d7ffff87b54eae1275a07c0d69ef4981e27"}
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 0, "pair": "a09692dc94164db8", "unit": "828e88b656245a1d0865c79b2ed023a49e78607a4ac81d70ccd3690f4419d2d1"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "", "unit": "6cce78d7569ad8db7ca0a61d7078198f666502d26ea6c0afb03173a8a4bb16ef"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "82047c0513e43a83", "unit": "6d2dbed1ab1c9d8ac259dd4227be69641fa59d52e816a8dc547ade01d0b9b0c4"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "98ee23cd6bd31cd3", "unit": "ab4bb39be535d70cd95e9035bb958bfc7418576a6ff40a47551894dd140317bb"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "98ee23cd6bd31cd3", "unit": "b70068cbc646bd01d3e10986e002f648a74e9d798900dc59cf3039b749c95c18"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "ff168d95d8d9496e", "unit": "c6ec9478c125ed780d7013c2807f2cec1878f44253667dfed319c28cbfd17d4d"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "b1bfef46dcd3b9a5", "unit": "cd4d095d8380a2fb7ca02fc6abdbae5249cc06455db2929875533f4aa8b886dc"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "82047c0513e43a83", "unit": "d1bade217be8d345cf5d0f0442ee31924187acbb98779dada95274a77462f696"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "ff168d95d8d9496e", "unit": "d561db250b892b91feed7ae877900e1e25963444f73c7699576e010f06da73cf"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "b1bfef46dcd3b9a5", "unit": "ed78fd5a7ab30bcd6e8ddd27c706a19cfd89750de4fbf81c0b567946eb94db83"}
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 0, "pair": "25f75d77a669b54e", "unit": "0f2825568461562f2a915a4f16ef8a4931290fd20a9801470aeb12677e66e3f1"}
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 1, "pair": "25f75d77a669b54e", "unit": "bf22bb44bcff4a5baf5d550c9d3e83e5f3ca607475fb51deaf4fb6fccff3d73a"}
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 0, "pair": "47069df6bfb3dc00", "unit": "444c1f41fc23ad656a8bf98dae58b36d0f1daa214ab030682f26111dd389e58d"}
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 1, "pair": "47069df6bfb3dc00", "unit": "c0c8b5cca070673ea4b4bb8f08dfff86b7781780e300720c4051aded035052fe"}
{"family": "output_encoding", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "cd80750447cf7af67ad809279718d98106c71705fef9ff7254728bfff4ece5b1"}
{"family": "path", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "2ce605ab5ce7ad3a1fb983cd6bddf3924e5da3010d2252a0a4640aab4e5921a0"}
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 1, "pair": "b0a899116dc36b85", "unit": "510ad48af03ba4a1a2a06dd254b135311dddb62d9fe5a2e8e8b12627586e6b13"}
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 0, "pair": "b0a899116dc36b85", "unit": "b414718a8a5f79739b88db24da6ad9da8e48bcb6b5e2dc99158848cdaa226767"}
{"family": "deserialization", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "8e22ee30441b20484090659b0c97423c711f5d10219041bf3a399623fc056a07"}
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "0f9070df9dc334f981cb01008c260b8bff78dbdeb48ce672830ed95caa8406fb"}
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "5e352f797b1422b0df477bce4730a9ce33cbd0837c3d63da94e4d8801a9a4b64"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 1, "pair": "dadccb128ac0e120", "unit": "505097494da5939404ad7d5b4af2b1c4a10efe94048084492d6ce60ebaa04c97"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "dadccb128ac0e120", "unit": "84ff8ca2d98a1b5d1ca4bfaabe7d19e03a396042b228c8d18d1185a178389702"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "91e261c54bd2c3c3a9ac04e48894ecabaca5b13e7db4fbd5578f9d266098117a"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "a308dfd59bc398824e2454bf6cf6cfd5b7880e5b948a82424f14753d2b6fcdbe"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "ba24eb554458bc97314ed561a7b2f4a4e643dde1205c32de4c8fcdd8c46e575b"}
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 0, "pair": "24d8f1840b67fbe5", "unit": "7ffa4a229a62266788fbf64890f81f20358b3eb8c35d3ab87d0c339d5bb4b507"}
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 1, "pair": "24d8f1840b67fbe5", "unit": "d19493ed7941e1cd25a76dc50504d97398770d07ebc302dd66d83441e946c468"}
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 0, "pair": "9f8a60eb48d74bc8", "unit": "80fce34f2377731e3de4355c7ea33bb20fc2fc00a5f388565b23018c7a3ed008"}
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 1, "pair": "9f8a60eb48d74bc8", "unit": "b68e62722a327c9d5ce6774eb96527451729410768efc0b5c7ba604ec468052d"}
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 0, "pair": "2c857235d0764986", "unit": "33bdfe911cd1b512e5994ee7a3e3ab2ad1f302cab29df3b011c0d825c23536a3"}
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 1, "pair": "2c857235d0764986", "unit": "b5efdcb84a057f3cd9fd3a30a32239dc06c13536a4e84a66c7a17cb372050c97"}
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 1, "pair": "9e84429d629c0b6b", "unit": "774cfdecac67499bb35a872931d4698d1b92ebeeb42d21790469e418889954e5"}
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 0, "pair": "9e84429d629c0b6b", "unit": "a6c5c15ca1815f83dfb47428c4a8afd4e60fa1c027b9f33c668cb9a0d77c18af"}
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 0, "pair": "8e6b4b4e28981590", "unit": "3cce3e8f9ff3f1d261f9b96c844c1e4da07bae72d5393897b3a929be25372edc"}
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 1, "pair": "8e6b4b4e28981590", "unit": "b105689ac24ed334294c434a77bb6d56d5d598830ffe8a5d60ab3be5290b47ad"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "0a36cd5177f3b2c20b433db6bfffe5ec5b7e74cacc34a86ac581f7a0a559432a"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "e7fc97d83013c9ff", "unit": "5ab444e64297fc19065478104c7ebdabe23cc415f22adafd0cf82c98e4c5cae3"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "e7fc97d83013c9ff", "unit": "6b462ea30c1afda572600a10324249223932d36a032eacafac8c837bccbdce87"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "84d6a4202a082e0d", "unit": "8175082f40d8d61d44b54579c84a808e25b89110be5d8fd82feedf7dd5e0d5ab"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "8abefca43ee4ae2c797033da31f85c17bfa7be6b35413d70b0216415c5db9a09"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "91d213946859f33d3f243c43aa8285cc6e8ecb3db1de938f32af775a70607f8e"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "a244d9f010ddb43a09e82363dfabb0e406d411fb5de08ab7d6c9ee367f2a15c3"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "d27d174d6e8e5f1682a1c42723705e89f0d18c95ecedd0567e313daa9a0aae44"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "84d6a4202a082e0d", "unit": "f45608ed47fa11ca25093641f6021ec06076c18676abac9b809f31b3b7713de8"}
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 0, "pair": "55d1ce4f29dd1583", "unit": "a877ce3462da316bcdd82ce9db1d9887e887eaa83157922edb1663413edae1d6"}
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 1, "pair": "55d1ce4f29dd1583", "unit": "aee67be0ba31f99c7a24faa1bad6178c6d37361a308cf5aa10138e6ce4bbcfeb"}
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 1, "pair": "16e529fe8c5486b2", "unit": "1d21acd0171726775d3da4589b660645fcd43cc026042a2b27e0e0e8374eceb7"}
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 0, "pair": "16e529fe8c5486b2", "unit": "f8e004b67ac98dc3392ea4115051e6bd10ce22eb638a762f0f16e1d4200bf129"}
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 1, "pair": "0b34438e6e0ba63d", "unit": "bf2608b00f0f784cc42e4c607f0b1ef50903283779bfb2eec91d2ec187856d14"}
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 0, "pair": "0b34438e6e0ba63d", "unit": "c5104cbf723fa0d55675963d8aa6802780ce3d274efe0012290fc49164c3fcc4"}
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 0, "pair": "b0210cd069757ad2", "unit": "d1d40c8b48d039c1999ff957a015273c30652ca0c45c4eb088052991a2a92f28"}
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 1, "pair": "b0210cd069757ad2", "unit": "e2e804652a885f640ebf341627f3279d6c965b47ee2ce792e55045ab4d1e1dee"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 1, "pair": "ef603436079b0005", "unit": "77f843b5044bb5da935df4b2620bf2abe9e7dceb3147439e883927112eaec22f"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 0, "pair": "ef603436079b0005", "unit": "f2ce95f0e44e73986784b63a702ffa4035c423056cac3667a63140a7e8d795c2"}
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 0, "pair": "a023056d2704c727", "unit": "6028fe87bea019646fff2b31c1b2d7bd0a95eb9214fe437e2008bc5d2dfabb7d"}
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 1, "pair": "a023056d2704c727", "unit": "dd93069189c5f242db611e4bfba4341c22f283b65af666fc31343854358de87e"}
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 0, "pair": "abc80e34d5f72367", "unit": "87d4a5b81bbfdd9628109a5f820895b6fddb00eb96b0f7bbe30a8d640457bcc2"}
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 1, "pair": "abc80e34d5f72367", "unit": "b171c77b52637bc84c487d7f8bce9a54229f9dbd6605e5089e1e88943ced989c"}
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 1, "pair": "1945f28a3fc5bd55", "unit": "64c0b7ccad09192eeaf590befe0de0087d5b48a65a4f6cef41415f3f54a8f536"}
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 0, "pair": "1945f28a3fc5bd55", "unit": "b55cd6759d4b7861e589052aa534bd0554586db46a329ed3beed7244d579e8c4"}
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 0, "pair": "1c9e47835232f644", "unit": "90e220d3540b88dd9efc8e66c8ab519d4cc78805f16bed585b795c78cfbb4d58"}
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 1, "pair": "1c9e47835232f644", "unit": "b5a185b05235f70f6cf27e2adba919f4d8393dbdf2d99de095548a21946aeaf3"}
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 1, "pair": "9b2b34cf62a76ea3", "unit": "91a3984e47f17512db8523e04f863525c67db18631d8f11776cf61ca7221c91b"}
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 0, "pair": "9b2b34cf62a76ea3", "unit": "d91f1e14282a0c3c8da57c6935a1625e881374bd205b5baa9e4a341d9109ae21"}
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 0, "pair": "a20cdf9dd208720d", "unit": "18a26df68e4b81d411cfbcc8155a7e6f382753d6e1ad95eece7ed24fa8c44324"}
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 1, "pair": "a20cdf9dd208720d", "unit": "e6fb74441fb585d7ef5855317fba795ef0f0a0e6eae9970a49556ce3070c3254"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 0, "pair": "b2e833b60a0fa396", "unit": "08ccacb7434e9f8b8b05c7bb0996be0609e34860d8fc418db329aaf8d7910f2f"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 1, "pair": "b2e833b60a0fa396", "unit": "2ca7f80f0badc049ce725718e49bd60da72d316589399126a5f4c5308cf83868"}
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 1, "pair": "d6930fe7e752b7ec", "unit": "06f0341dfbf1aa9504083737df5d9be3cedfd4e3a198799d8075a29990776800"}
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 0, "pair": "d6930fe7e752b7ec", "unit": "30e2cd8bd381d7568b78eaf982f562d278a6a8d942d5282466532c1441bdf0ad"}
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 0, "pair": "e9c34a6a4bf2d333", "unit": "0ae20c7334fb27889ef8551f34c65a847dffd6b37f84f722f7e8ed64fd82e95b"}
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 1, "pair": "e9c34a6a4bf2d333", "unit": "59ab1924056704605b22b7c8360f40f745e7f488450be3647a632bf9a3b22699"}
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 0, "pair": "053a9e02feda3960", "unit": "34e18fe343b5c25ec582fc05ab091fd9ba160f658ba7678066c56db4991a951b"}
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 1, "pair": "053a9e02feda3960", "unit": "8fcb18a2a8654b6594cb2ec078e2a018d20f73d2b69a2e625338ffa80b3f72c3"}
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 0, "pair": "2d482951d7380c4f", "unit": "5c1a721da6f85d1d5dfc8cbe5e9809b093a09c059e5abeebed2cbbee55fbd498"}
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 1, "pair": "2d482951d7380c4f", "unit": "6cabd1c4b5aa94cafdd07c0c6a3f9abfdc03761ddc5b0a2bebc72a508d020b8b"}
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "", "unit": "2300c097374b9098671dcbcd42d02d08172e0f3064a8323223d0fe79a6106aab"}
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 1, "pair": "0664148b0b192456", "unit": "d702617aaf4d19811baaa18386a3c8b0eb35cd3fd72e8d90611d34850e990858"}
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "0664148b0b192456", "unit": "f5ed13d811c5b16d46cbd4a3b9edb8f7d2db49acbd49d95f39f33eaab38f58f5"}
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 0, "pair": "fd5781ba332ab81e", "unit": "59eca69b2bd99ba282eb8056d7bc789e9c691eebf62a4b42594e34f5516833ae"}
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 1, "pair": "fd5781ba332ab81e", "unit": "7a7d774a6de9e4a95fe97fff822652f1f1383922f2f0a4dcb12b8e964ca330f5"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 0, "pair": "366dc5ddcfe598dd", "unit": "0053d866cec1e99f8a4a8754cf8ece3c46df16e002c86703c68937d5f3a09b08"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 1, "pair": "366dc5ddcfe598dd", "unit": "027dbc5b9245fb804a1e2ba4445b8afa180aa2c2b3293db107ffe0ca4900ebb4"}
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 0, "pair": "d0a34bf90e99e794", "unit": "4efbb75cc4105149522d2b2220987fd4373a8ef645c2c544546da540f14109eb"}
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 1, "pair": "d0a34bf90e99e794", "unit": "fd790f533103fd1a5646a9374ae895cf6e36d578fc747db2b0d0249a54a85e9f"}
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 0, "pair": "a697aa902d141a6a", "unit": "341e8fbb46f00167a32a2d653ba09a205794084eff65afddb70f9cfff42d697e"}
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 1, "pair": "a697aa902d141a6a", "unit": "e29414494b275afc15f3a028f62402161df7f6da5123c302149e23845177f7f6"}
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 0, "pair": "8c1fc8417190a134", "unit": "2125fb101d6108627e77f61c05cadddddfebd1cddce0e66376d9b822a2eecbb4"}
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 1, "pair": "8c1fc8417190a134", "unit": "ae7bbc58026fb71f91b477946a27bbe3599be90501da9237755f3cf0fb289794"}
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 0, "pair": "1972ea9f72b178b6", "unit": "3ddabb356d78e7c65ec8f88b6da8e4fcabc055ae73f2f6d92c50efddc5d4fcbe"}
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 1, "pair": "1972ea9f72b178b6", "unit": "6875dd5e9c0ff8a1b693d29a109cb2e62e6ff4081df40868aca70e4770476757"}
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 1, "pair": "2b007a2377d633dd", "unit": "bdd71a783efb4eda9a82afbd30a16bd7608153830dbc599e65c453c18d1644b2"}
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 0, "pair": "2b007a2377d633dd", "unit": "f398ef33900624219e59de08e1cc101c43082a81c37d6d03a916cc220727ca67"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "04aa777130c02a06", "unit": "16aa43c87956c44a4b115245140eefed704095d37cec0cc6410377403cd2fa7a"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "23744137e0485a36f686f5f1b5666780fbed3ca522e8c6d35687a391ef504498"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "67565d99fe28c68f", "unit": "2692316792e6f02fb51578cf30d71bba6c73bf5420c36f709681c8eaa69807d4"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "2e36f73002c416a68d077bffb0335d82346c7507f2478353aeac005fce5a9cbc"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "04aa777130c02a06", "unit": "67721f67055eb6fed955ff22bfe4060344662eccb369431835a2116ad120c4f8"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "8fc1c96fa6faa4d3966ecc42876189b18ddab779d58f47cdf28a1155f48a6ac4"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "67565d99fe28c68f", "unit": "ab535098a8e709b022afee1da15dd1fc371d389056938ee4fa4971b2966c823a"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "d46f94235bc462360097e7a22e1d442d95f049306631489a4d8b25f647a95bbe"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "edd6dd4364442f66384d14e9317ef2f6234e32ab78327a45741b8a816b4332ee"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "fc7df6887ac719d4615934c0e8c3296728c2f6089f233158c59989027c802d72"}
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 1, "pair": "c0412509f15fe3e1", "unit": "63bda2a71ad7155ad2f59aa9086efef3fd3710dc82c57f4b32ec8bf561460239"}
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 0, "pair": "c0412509f15fe3e1", "unit": "b1c9a070c93764655786c3de5fde536c150e319efe47548614cc544b73a8ee2b"}
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 1, "pair": "feed5f7de11289a4", "unit": "71531567b2b8609ae0d31cb177b8faea4f5fc61923e513b0154a2c941287ce2b"}
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 0, "pair": "feed5f7de11289a4", "unit": "d6be6d397a301a07faa761163d3d9e270d71d23d3bdfd084eeafb1e62b91dd71"}
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 1, "pair": "31c259b4e9dccd82", "unit": "1f1ddb8a5b415461638056c70756d2d0e836de71e824c0e46474f20646eaa4d5"}
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 0, "pair": "31c259b4e9dccd82", "unit": "a6ec568d4eef212a842dded7fdef2284524d367e5d24473ddb34e6374be9fad7"}
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 1, "pair": "ffd4b04a7bebfaf8", "unit": "54f7519d3d59b6d8d69a5f8f7bd78e360d3fabaae00b7587c37e388719169370"}
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 0, "pair": "ffd4b04a7bebfaf8", "unit": "c5e42c9ab2ee54e46700ab0b64e239b567fbc75ccaceaf6cbf20687813663332"}
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 1, "pair": "2ffec9490ef1d482", "unit": "0ab883d3561162b75ad9d161f79c0d17521a23317f2f574c910465f288119483"}
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 0, "pair": "2ffec9490ef1d482", "unit": "b0af86c1642c14047248f1134181be0ec3839900917dffa360c5014119f8632c"}
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 1, "pair": "73f2a014c6a699ed", "unit": "44ba9d9d8e75769c1444ebf37a64941a845b4bcae678ec79d64dc84e5cc067a0"}
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 0, "pair": "73f2a014c6a699ed", "unit": "b430d488c486cef26b1c661f773e7e9e4a177eeea73537a033eec98d532e4eed"}
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 0, "pair": "738e4c22edb32172", "unit": "932d7312b2bb4b830179104bca076f587505c2d4ce881b2f04f992defba6dd9e"}
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 1, "pair": "738e4c22edb32172", "unit": "eee5136489c3baa3653ededbf6385d0237e2bd8b5384286780646b88ed34fb91"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 1, "pair": "cb0c638282db0d37", "unit": "a49e6ad2e7c2a5ad0d68d5d69c263306b501fc0dc9382e701fb5fe680611196b"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 0, "pair": "cb0c638282db0d37", "unit": "dee28099d3b37cbd0ee14412122ecc5f8488ed6bec01863520b87706a040fba1"}
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 0, "pair": "09bce271e8530fbc", "unit": "35eb19b4c6176b37287e2123bb98ec4045ce6bde7cc3864dde317c024b0935d8"}
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 1, "pair": "09bce271e8530fbc", "unit": "dd33291cc494bbc77d44e9b13d16f171d32c9746637506f315f375e8a9ddee95"}
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 1, "pair": "a856cd13668353a6", "unit": "6149fb0132b981620a90a23a9af8d4656d0d827403e63b48258443cacd0a9c81"}
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 0, "pair": "a856cd13668353a6", "unit": "b5aecdd6a87f8d57d28f54a013b1f96332460fe1bb44bff4ed28e166f18f9abe"}
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 1, "pair": "40d8d81a764c508f", "unit": "7deeed73c29c74ccbed29be049e109d363d41e4b5eea245ee57780b85f0e137a"}
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 0, "pair": "40d8d81a764c508f", "unit": "b5cdadf848e67910550666ba2a6ac67b24112e501c02dba1865766a316317a23"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "", "unit": "15dbea384b3255457a44d9382efc6944dd27f1d411d903344a5d5e5fd194b72f"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b2ac284bd70dd684", "unit": "225890ba6694dc8f4fbab46e9c851dff8c095e906cf14db15d01ebce0891aaab"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b6eca35de93a5be8", "unit": "42eb8427415704e060e73b5ce0943b014ddf380ca756d5ec47c558e59a79895a"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b6eca35de93a5be8", "unit": "4c9cdcc11a18641ef41e9308712c5e8b830ec9ad5ebc3a956c8701e4f77b39c0"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "6a53939838afbdd7", "unit": "5794809e595311c86909d673a570d003aa025e4dcd6abc5641b641d935beb0b1"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "46d3ec7d49cb8ca7", "unit": "5f7a63ec650915962ee5255f5416df63ab882d6107f3b540be36d481798a8944"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "6a53939838afbdd7", "unit": "97a7018535ab511622e3ade9e53d570ffb1c80633eaa372fb76e749b3a586c8f"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "46d3ec7d49cb8ca7", "unit": "a84e9ab47f4f2b584684dbb81575758d9032fe3f2ed53bef478cb9dcfedaf677"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b2ac284bd70dd684", "unit": "aeceb1ebc863ebbc3b69ae56c5b78d2dba005132b75cbd6af929bb0bfe25da9b"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "f42cab375ace1739", "unit": "e329d4a6ac757680bf3d292a81f1f74f8a8a9b482d1651efe9f503f6a4de8796"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "f42cab375ace1739", "unit": "e73c5aac41c0b93072223a3097fb175548ce893e0678ff3a22bc7571745a17e0"}
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 1, "pair": "1041a18cd1aec5f8", "unit": "c301b8c362aee3fe5ea38e18f858166bc4162ad8ace44c5fb2df192afc81da2a"}
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 0, "pair": "1041a18cd1aec5f8", "unit": "ca8e58ffcfd60bdc8626db8b9eb4f253c1b8b8affcb793c6d963b7a3ec91906b"}
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "", "unit": "52354ccdbc16f209e4692d97d5237e04ed030cf4a5e71f77b5a767962b355b4a"}
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "7d23be667536afff", "unit": "b5016a5a53409491ba805ed8718599b84250b7b6914da0434b0882b0b3578dc8"}
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 0, "pair": "7d23be667536afff", "unit": "f1b01102968568d3ba1b2fdc70f4ae281f433fb20d29eff2a55fc45555ddbfcd"}
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 0, "pair": "f21df949f8aca4fa", "unit": "15d23ab3aea4c878dd73bbae579f3ab473e8363477a97d28164ed5e1b94cb510"}
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 1, "pair": "f21df949f8aca4fa", "unit": "787bc98c651ed3db21be4c28e9c560c366ffd6ca508823d21e5d83eeaf1269db"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "1199de75f426940b", "unit": "10fdc236f43a02b7e291eb8bf6ada297a2ae98086a63b171057bc7bfc37a3377"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "22112c31603bbe9f", "unit": "5eb3b7523b5a196dcaacba08d68457f9a45e31503ea02745ca3a7d0542f9881c"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "22112c31603bbe9f", "unit": "6bd5bdfd81b0a3374d59cfb2c1f7fa2b27887f4672f4485e77eb5234b569983f"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "1199de75f426940b", "unit": "c09dffd04708540845c8d1dd77a2bd989bd35f264abf652a2db55f8cb1faf7a9"}
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 0, "pair": "7aab025ea7b97019", "unit": "7487a34cdfdb532a5096764a47a3e565fa509605c1df0e2b105eb9c751fc5307"}
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 1, "pair": "7aab025ea7b97019", "unit": "9c0be2a58741f095fd0a9cdc60d4ca4039de1529699d5701dc4d0bcf7d5bf90c"}
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 0, "pair": "4de5441f1776f083", "unit": "a3a35d573da308cd1d0614a26616bbd37b360d9ea416894cdeffdb59abd691d4"}
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 1, "pair": "4de5441f1776f083", "unit": "dc33fc26c1bf7c67e5439ddfdad06c0faf16c796e0fe8a46b6e6a541e708676d"}
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 0, "pair": "1972a9a06d7585ad", "unit": "1f472e842e894c9907536c6eb6c9c922bbb3b21c27999114588ec2a4345ab665"}
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 1, "pair": "1972a9a06d7585ad", "unit": "7e38e24f1cc4d287627b7cd6cb870888865bb87fa87bba1c4fbf84eabb213121"}
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 1, "pair": "d76d5232a8587127", "unit": "1ec32d427822d1ddd3b0d7245f5771ddeb56c146ac8ff9959dd85760ff659a20"}
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 0, "pair": "d76d5232a8587127", "unit": "5143b19a1a1e6b9162b80594ed7fb5c0c57ed0b8abfd793bfc71089552d57638"}
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 0, "pair": "5f6e814444b4acd8", "unit": "2e8aa07b1aa4bfc5349b0d4464f3f2a6500aa55ff765b0139e9901ed34151ffa"}
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 1, "pair": "5f6e814444b4acd8", "unit": "e26c726c08be761b22fcc6ec17fc1fa7aba4769f08957f024e5545906df62797"}
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 1, "pair": "888e955fb6cfb4fd", "unit": "6f1a431ca97afc34ceec48393eb134a8928a312e6f761eb907c6b5855cac6208"}
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 0, "pair": "888e955fb6cfb4fd", "unit": "7b62415d17569924d16238a4aa28ab1af2db6561f56be9f59d8075fd1bc4a941"}
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 1, "pair": "a5691395e641fd4c", "unit": "40339c123107f1231d6aa58de876cded1dfefd5b31dadc4fc86eedc6a687165f"}
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 0, "pair": "a5691395e641fd4c", "unit": "ea3828e2f59cc6f7b415a101532b566ecbd47ab123c3c7593c23c52f6773a2e3"}
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 1, "pair": "f1ebdacbd647e39d", "unit": "79628ef3d6c9e12a94922cdca548ff77d3dbdd6b595c1fec5958fcf23b609095"}
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 0, "pair": "f1ebdacbd647e39d", "unit": "f910e010c3ff2c8c17f0f80fbe89a934e4f792b601febbf318359e6dc3afc18e"}
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 0, "pair": "2dd5384f097711b5", "unit": "212fba6b5e50db045114af369a03ffbdf09ec898c10eeff3c45bfc01a5017b4c"}
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 1, "pair": "2dd5384f097711b5", "unit": "370ba3a1d0e354e882b7be92c1ebbf2e2903bdf29d442048ea3dc1f80922467f"}
{"family": "injection", "fold": "outer-4", "group": "strangerstudios/paid-memberships-pro", "label": 1, "pair": "", "unit": "ac905d12b92954263c046bd0138a682cea86cf8476336d6baa664ade28d34ac6"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "218a45d869bd7e39", "unit": "309ebac468b7ab054c8396ed495770385ae3590caba43709745ba2d1db4b6d54"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "", "unit": "3cc8613813dec13cb1b7b45b296688da883f0b36cd2c4fc1da6e81703fb59ab9"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "218a45d869bd7e39", "unit": "9bd2331dbc260451355307e4c37bae17ad6676654c66181934e1b35fb174cc5a"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "b087ac4d0989e0b72698a9644ef4eac8073b4a15bb21730d63e4730b5cc58a60"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "c5d1e8d19560c997018fdca12eb60f1799be76c33be058b88472f5a2fc6709be"}
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 0, "pair": "c863054be488a284", "unit": "425e1d9961b79bbc011c447cd050910dd796a32b248962a5b0acc3f5f19b7345"}
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 1, "pair": "c863054be488a284", "unit": "552925bba199119c7db5457b9d3c535bc9f6061cecfd2ab2f72cde690a536e1e"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "22befc17a75a688e2f9358d2ddf9a7b197da44cd95dbdeae5e0191b113ee54d3"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "63f496d9ff53e39b", "unit": "6a98f8145776c78793b37156e9a81757841bb6de80016ab4fa575209d724db3f"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 1, "pair": "63f496d9ff53e39b", "unit": "7ac60c96c15f8517e089cc2240aedcf1fc30e54a41e35fe03c6b965bc393fbd6"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "ae39cb55ef2a87c5347e0ddc40975f142450e82de4909e93ba62a7ed0efb4134"}
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 1, "pair": "d1b19898f8deaa41", "unit": "3763c7b407ad471820b5f8fa25bde0ba6d91e15c07c56681de4b41add418b0b1"}
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 0, "pair": "d1b19898f8deaa41", "unit": "bc8b934938c30687f3d4f982392d2cfa9504a916b463ccb5c46fe8a30eede488"}
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 1, "pair": "f5f33fa6fe7b8492", "unit": "938bbf146a9037e687190bcee82c047492dffbe90efea40a49a647f81a0f1ed0"}
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 0, "pair": "f5f33fa6fe7b8492", "unit": "b90c9f7e0011685f1a8e853a9ed0ae37fd437ef6d6eb0ba17c240aec2c22b603"}
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 0, "pair": "f74364dc8e35c8ba", "unit": "11e760e99c5c852676c9288e18a250febb3854e9047b51e8b9ecc9f3b423b6f5"}
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 1, "pair": "f74364dc8e35c8ba", "unit": "5fc980194040f4c4d803075d4085618cc0260ae8f3454b26b9b704d30200ffea"}
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 0, "pair": "db1c51833994ec55", "unit": "c2ce8320c778dd55434d7413844797b68e6fcabc4dc509458a9d60ca70b88c60"}
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 1, "pair": "db1c51833994ec55", "unit": "f2d6843547af1276ae8a4dff3a7786913fa79dc69540d4fbb95c4c091e27c7f2"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 1, "pair": "f4690679bf9d80aa", "unit": "1a4e24df8d08672ecde4709a77ea5c437b2198fecff8bfff373b63c9eaed6e63"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 0, "pair": "f4690679bf9d80aa", "unit": "78711973ec84867671314dd7878769092b32f59860eafed50a0f6979daee76fc"}
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 1, "pair": "d6a42c18151efb48", "unit": "01ae2279707361bfc06a005335d41b85066d0a163802ca81c031f5dbded5bb46"}
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 0, "pair": "d6a42c18151efb48", "unit": "a50dc15c313a9078badea01365987c66c5cee1ad48cea65c35c8bc2cbba25e8e"}
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 0, "pair": "85c15e3fc2b370b2", "unit": "0990d8f41cb07c1cf217812e51a7025381de4ba68d1be3799e26241d81dcea67"}
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 1, "pair": "85c15e3fc2b370b2", "unit": "9da1d63f0c4da270f66eb87bf9c342ace883a0aef78c4643e254159b5da8fce3"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 1, "pair": "c7f4f4ebebfae831", "unit": "b5cbc1490bca76679adfbe5e4d9f8067d27a226d595ba401b551b00c3ce4d45a"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 0, "pair": "c7f4f4ebebfae831", "unit": "ba2c1808eebdbe3f54c488b09bcb3d69e521916d1a3a0b5b14e02d5bc88bf016"}
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 1, "pair": "1507c8e00ed4b859", "unit": "620e9b5ed143c0bcec178db359fa601acb1d3a134b3e85c2e838b7dcf3892d85"}
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 0, "pair": "1507c8e00ed4b859", "unit": "6f5ce4008dac0471260310c60e1e2a9f76a7e4eec6ebd039835f0488cf3b19aa"}
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 0, "pair": "8f5a6abedd742bbe", "unit": "71e3ced643a244100cc3fde234f5b4b27655214c7ae0ff67ab61e3e48a8fdf78"}
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 1, "pair": "8f5a6abedd742bbe", "unit": "7c8cbfae87127042f9f43621345b4f7958bde27a3ff9db76254063e80735f3ce"}
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 1, "pair": "c9538342f2d7efca", "unit": "2b8db57d65ef1d67c457ff5a336e8a542d03387395d69d5e16815c4f00f40723"}
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 0, "pair": "c9538342f2d7efca", "unit": "e2ed113dc21ad5f7caa71ba4a5ece8f8f900a97fd0d00dc85f7ac79ff5bed19a"}
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 0, "pair": "494dd5d5e5cdf4b8", "unit": "3b8011f94ec721f68dd29afc6c8b01117c86a8b0c31f3650a6ce0a8d7c0739ff"}
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 1, "pair": "494dd5d5e5cdf4b8", "unit": "f7e2280574e9ae51daa64d8adb2955407ff6d1e156f0e2854b6c047e0a733c78"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 1, "pair": "51eae488182a123a", "unit": "20b039866ba796c34efa0e92468fb49a314ad7fe2914d9ff8b823e3e35ab26f9"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 0, "pair": "51eae488182a123a", "unit": "2ab3fef05def768ef7b4165bcedcf7d200282dba78a32e19b90a0df3aee7e3e5"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 1, "pair": "6f5a650865e08571", "unit": "e2051cc42e4f8b026c7807f966d492367310c90326e60fef1b0ee8c509c805a9"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 0, "pair": "6f5a650865e08571", "unit": "f5364d68ffe80b578e788f37f08784b0162826393b49b4a3affd70d7cc4c9e01"}
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 1, "pair": "8acd426c5771b37c", "unit": "983e146a60e061254b15dc8dd573cbba57f8dd0358f23ea1b9c313ef69069af5"}
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 0, "pair": "8acd426c5771b37c", "unit": "f80dfec45231e4778078cb10f00128ed445177cac0a062fc2df5ced5c0171f21"}
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 0, "pair": "9b84800ce5c395cb", "unit": "9b7dfdceae03fd86ad3260a03f85bcdcff8f32f10c265d23743ff8dc98eeefb8"}
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 1, "pair": "9b84800ce5c395cb", "unit": "b99dda150c0da3defa5f4c527ef6c402f5589dfb2f2f03f3e5e424910eb48c53"}
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 1, "pair": "ad64e8fac4ac3ee0", "unit": "36dfd01915a553d86576e8ecd3b737b8e2b297aa542f58cc83b448060e74398a"}
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 0, "pair": "ad64e8fac4ac3ee0", "unit": "c2b6d40f130d35bbfc16fab9411c664fa06321c59fb1d901a1987a69cb09c72e"}
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 0, "pair": "d48c08b19f560848", "unit": "03d02c9e25eec0f380b125279fce35e8867065bf6e1ec75f9f39729ce6efdc77"}
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 1, "pair": "d48c08b19f560848", "unit": "abbea69b7d0fa68c3ace442c25597e128cde354b417787b5913fd10d666c0d19"}
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 0, "pair": "bfc8188bbb673e3b", "unit": "a7cff0da55c0cc1d5c838e11fc6ce6c4b22c54b9f8e101688b218d6928eca504"}
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 1, "pair": "bfc8188bbb673e3b", "unit": "f741c56815230b9c30e2c8ef8952b895e9480e4780a89ff2cf6daf60d1fce7b6"}
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 0, "pair": "3844ed3d703d7ef4", "unit": "445efda074e4d9f6c1634225eb08a3c3eb1257311c995ce891df2c791a6f5c8b"}
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 1, "pair": "3844ed3d703d7ef4", "unit": "8ac27b08549353ada819caaf99a5180d2f2f515b03aa8cd99a5d37b527ea3a5c"}
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 1, "pair": "a63eae7d621a985e", "unit": "04ca0b583b09c3b4f67bd2496a4607ecb134d24b7bdaf8960fa151e40ebcc447"}
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 0, "pair": "a63eae7d621a985e", "unit": "ecfa2a67726ca9d3e2621a4d8be60145095b76aa18ecf0268212341d63752871"}
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 0, "pair": "0d9f529b7f0084b6", "unit": "35aa369236f510634b3aa7b54b04e9da0f80877e515786a888e760f8fe33c524"}
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 1, "pair": "0d9f529b7f0084b6", "unit": "ef4f8613c0914eb815504e00cd6ee05df38352fd3b27826798e6623594017042"}
@@ -0,0 +1,90 @@
# Experiment exp-004 (learned-decision-engine Req 6.4, amended 2026-10-03 by maintainer decision): the pooled BLOCK
# gate with a family floor. Registered before it runs; `ousast learn experiment register` records this file's sha256,
# and a changed file is a new experiment. The procedure and the decision rule are fixed here; the units are NOT: they
# are frozen at run time (`ousast learn experiment units`) from the memory after the negatives harvest (merged in
# 5b2bd9b, 1133 static examples), committed, and registered again before the run. Until then the registration row
# says `units_status: pending` and the run refuses.
#
# Scoring: kind rule, zero cost by construction -- one scoring pass of the six slice-2 incumbent programs, replayed from
# the response cache behind a zero-ceiling meter; a unit with any uncached response is excluded and counted. The
# harvest changed the memory the programs retrieve from, so most prompts are new: a paid cache fill (separately
# approved, its own ceiling, as for exp-003's supplementary analysis) must precede the run, or the exclusions will be
# most of the units.
#
# Normalisation (declared before the run): per-family Platt scaling, learn.calibrate.fit_map with its existing floors
# (own slope and intercept per family at >= 20 positive and negative repository groups; a pooled slope with a
# per-family intercept at >= 10; below that no map, so the family cannot block on that fold). Fitted on the training
# folds' paired units only. Why Platt and not rank-to-quantile: a quantile map gives every family the same flag rate
# whatever its precision, so one pooled threshold would flag the top slice of a weak family as readily as of a strong
# one. Platt puts each family on the scale of its own training precision, which is what a pooled precision bound needs.
# The map is a normaliser only: exp-003 showed calibration (ECE <= 0.05) does not hold per family, and precision is
# demonstrated here by the bound on held-out flags, not by calibration.
#
# Thresholds, per outer fold j, from the other four folds only (fold j's labels never enter a threshold or a map;
# tests/test_learn_rules.py):
# - the training units are normalised by an inner cross-fit (each training fold k by a map fitted on the training
# folds other than k), so t_block is chosen on out-of-sample scores;
# - t_block: the lowest normalised score at which the training paired units of ALL families together with p >= t
# and a verdict other than `unsure` have a Wilson 95% lower bound of precision >= 0.95; none -> no BLOCK on fold j;
# - t_quiet, per family: the highest normalised score keeping >= 90% of that family's training paired positives at
# or above it;
# - fold j's units, normalised by the map fitted on all its training units: BLOCK when not `unsure` and p >= t_block;
# ADVISORY when p >= t_quiet (or the family has no map or no t_quiet: nothing is silenced blind); silent below.
#
# Family floor F = 10: a family may block only if its own held-out paired flags (all five folds) number >= 10 with zero
# errors; otherwise its flags are reported as ADVISORY. Why 10: zero errors in 10 flags is unlikely for a family that is
# much worse than the bar (a family at 0.75 precision produces 10 clean flags with probability 0.056, at 0.80 with
# 0.107); a larger F is out of reach of the smaller families (16-19 paired positives in exp-003's units, all folds); and
# 10 is exp-003 arm C's training minimum. The floor uses held-out labels to decide which families block; it moves no
# threshold. The primary metric is computed BEFORE the floor, over every family's held-out flags, so dropping a failing
# family cannot raise it.
#
# Power note: the pooled bound needs >= 73 flagged units with no error (110 with one, 142 with two). exp-003's
# instrument check found few positives above the first negative per family on the raw score (paired: 10, 1, 3, 0, 1);
# per-family normalisation keeps each family's order, so the pooled top set can reach 73 only if the harvest's pairs
# separate better. An inconclusive verdict is a plausible outcome and will be reported as such.
id: exp-004-pooled-block-gate
hypothesis: >
One BLOCK threshold pooled across families on a per-family Platt-normalised score, chosen on training folds only as
the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95, reaches that bound on held-out
repositories' paired units, where per-family thresholds (exp-003) could not for lack of flags; a per-family floor
(>= 10 held-out flags, none wrong) keeps a family that the pooling masks from blocking.
kind: rule
programs: # the slice-2 incumbents, as exp-003
access_control: 745cb1d76bbb09dc1574d333d2118489be62e39886517894d3699fa565b0e8c2
deserialization: 3c0c2831be7d78198a62351661c37867363c501cb40e3ed7b0119c155af09654
injection: b207a0710e0abf053b3b9397b25f9ee1cab4679144865e3a82c1b9be2c7b4982
output_encoding: 5d768f16724562c381c09949ff3df9cbb11d58cbb7a9c0515d0dc3f858d3818e
path: aea396680a4874dcb6de70e240ca0480c83424debdd3710bc3f1ea6a612661f3
untrusted_destination: 1242b3973fc89056aa08e00843043d68235df217d8e3ef8a0ea0e0b81c142753
arms:
A:
label: today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p
rule: {name: calibrated_block}
B:
label: pooled gate -- one t_block over all families on a per-family Platt score (training folds, inner cross-fit), t_quiet per family at 90% recall, family floor 10 clean held-out flags
rule: {name: pooled_block_gate, precision: 0.95, recall: 0.9, floor: 10, normalise: platt, select_on: paired}
unit: candidate
pairing: paired # every unit scored once; every arm decides every unit
cluster: repository
order: {randomise: none, seed: 4}
# primary population: paired units (both sides of the candidate present); pooled (every unit) is reported beside it
populations: {primary: paired, secondary: [pooled]}
metric:
primary:
- {name: flagged_precision_lb, target: 0.95} # held-out Wilson 95% LB of flagged precision, all families pooled, arm B, paired, before the floor
secondary: [flagged_precision, coverage, flagged, quiet_recall] # quiet_recall: held-out positives at or above t_quiet / positives
# Decision rule (decide_pooled), on arm B's held-out paired flags of all families together, before the floor:
# REJECT B if their point precision < 0.95;
# else ADOPT B if their Wilson 95% LB >= 0.95 and at least one family passes the floor (>= 10 held-out paired
# flags, zero errors) -- those families block, every other family stays advisory;
# else INCONCLUSIVE (the incumbent rule stays).
# Reported, not deciding: per-family held-out precision and LB, the floor table, quiet_recall per family against 0.90,
# the after-floor totals (what users would get), the repository-cluster bootstrap interval.
decision: {arm: B, target_precision: 0.95, pooled: true}
test: {resamples: 2000} # repository-cluster bootstrap of the held-out flagged precision, beside the Wilson bound
mde: 0.0 # not a paired difference test; see the power note above
budget_usd: {total: 0} # the run itself is zero cost; the cache fill before it is a separate, approved spend
families: [access_control, deserialization, injection, output_encoding, path, untrusted_destination]
folds: {kind: outer, k: 5, seed: 0, compile_fraction: 0.25} # the incumbent evaluation's folds (learn/compile.toml seed)
units_file: plane/experiments/exp-004-pooled-block-gate.units.jsonl # frozen at run time, after the negatives harvest
@@ -0,0 +1,501 @@
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 0, "pair": "6b0b3cab2df3397b", "unit": "0ac37b7154242dfa637140af643fb4595e85510a17ab1a12c111441b9d293238"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 1, "pair": "6b0b3cab2df3397b", "unit": "6e5e274946d0e8f13a7ef4e37f6c7ed1828d057304a9262db0311413f18e1731"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 0, "pair": "c0d0c91757f5ba95", "unit": "285891b16ee6870af50b36dec3c2ca9a90ee15d9e14851c61e66ad443e6eab59"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 1, "pair": "c0d0c91757f5ba95", "unit": "f33fb2413edf1499a524c4592b3b2f93d166be845f5b65291969a086a4bec334"}
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 1, "pair": "f56f2a107ae591be", "unit": "350149bf9cafbb547f500b714a164734d2361b22e86fc9819c7bad3507d8458a"}
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 0, "pair": "f56f2a107ae591be", "unit": "ad125319dd2d4c9d2d99a1eb34bb4cfab9f099a96e7547d0d59a8ecf1c990347"}
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 1, "pair": "ee577ee4b6076fab", "unit": "2523f55d035d1a772a842090bc335e5d755b2fe15b376dd5b6b9e30b9dae82d1"}
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 0, "pair": "ee577ee4b6076fab", "unit": "ca0f6c160d7839204f6fcc1709bcbc749a02a0b3b8bb648e2a238eacbf22ed78"}
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 1, "pair": "c95b6b01d0c2db2a", "unit": "15540c23be0c42472a49039658ec2ec0bbd8c6b318e35dd040a9d427370e0ef0"}
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 0, "pair": "c95b6b01d0c2db2a", "unit": "542471980417e964182d7d3db59a05fe12bdd072449f3bd77f7daa5515dbc909"}
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 0, "pair": "29252934ef4bcfcf", "unit": "63bae9240b31f9cf1dfe00d9b01d4d0ffda95931c9e7d040df1fd718f04d828b"}
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 1, "pair": "29252934ef4bcfcf", "unit": "cce2af1743834dcc3df58827ac263a2d63296ca1b2a2f9bdb811954fb5767c33"}
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "bcc636b23dfb5f570d8864b17fe07b6b4fb52b65a40988c32011f1422f74d504"}
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "dbcf1e67a1be7ccf7d51701f9b8339339164055135443044c23a446b16b3d934"}
{"family": "injection", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "50408f68114ecb4de3bebd836d52d9afad86b22dd1843641ac8a49b617c09e33"}
{"family": "injection", "fold": "outer-3", "group": "anxolerd/dvpwa", "label": 1, "pair": "", "unit": "cafb0f00f80dc6beeba40eb95c764bf9a3569193070ba321c28677dfa425ec88"}
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 0, "pair": "07317a2258292656", "unit": "a9f292d7cb467def582d9c0b31a397b13cdaefa77380ce971bfe9fbf535bb660"}
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 1, "pair": "07317a2258292656", "unit": "ed363d0dfa620707be477b90eb4b83a901bfd402040b918aeee6f06b2c5b795c"}
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 1, "pair": "1544b6dfa2299d60", "unit": "d45018fc4ce3dfd715c7087d4d197ad4108c0c8b6625c02c4d4da0adb9f6a140"}
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 0, "pair": "1544b6dfa2299d60", "unit": "f295c7dfb095595a4aa985c58688e4757c29be9e06b7a4ca55035ac5d06b1ef8"}
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 1, "pair": "fe0e104d111c2555", "unit": "088036fab9962be2b96f9162ea4f32190575678d3c5ba96bdfeac178bc22b2c4"}
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 0, "pair": "fe0e104d111c2555", "unit": "7fd2b56a816e25358477eee64487fb2c77f34c652703ad0a7120c368492017a3"}
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 1, "pair": "b6742570f7f0e026", "unit": "7c6c1d27aee6557f1b6bf371b8209003f0d2d1122cfa35b6629eb43f38769cf4"}
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 0, "pair": "b6742570f7f0e026", "unit": "df8300d84dd784d0be46969ea432ef97b8ce611bae7a896ed4f349bd4d6897db"}
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 1, "pair": "0f584bb97a154488", "unit": "222a84ffc7c58d45e8f10df221762b23bf20d0287ecd20f375558134928886ca"}
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 0, "pair": "0f584bb97a154488", "unit": "25e4c9d2941f59f072385a7bde3fd0bd76f9e986ed24b06ce81962ab447f5094"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 0, "pair": "b15b924b4f7b1ae1", "unit": "2fa95fce0751c97db1a12a80ddb249ff5c54ee31c7dc3cd47765099c7b673e52"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 1, "pair": "b15b924b4f7b1ae1", "unit": "52be8128053102eafbee67c2d3aac88c83ca290b943ab2f2cfdcd2630601c7d9"}
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 0, "pair": "d31ca58f2a1d27d1", "unit": "2ae40ea61d691060e7b9f6c8e910e3aa0edf16bc6f88bec81aa9a7dae5cf5603"}
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 1, "pair": "d31ca58f2a1d27d1", "unit": "4876728701698045e65eeae6e11e133c30f43b09029d7777d6d780f7d0855885"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 0, "pair": "bb3d7baed62e6bb7", "unit": "e8ae88fd2b677e3f64ac94bddb012e3b903afa294ecc87774d10087c0d50489c"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 1, "pair": "bb3d7baed62e6bb7", "unit": "f6b0d241891dc1d897d74d98b6ff7ada2c1f1ce7874ca32097630c06ca53e389"}
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 0, "pair": "d9d8cfa38981847f", "unit": "864f1800617f5218a2300ce390ba7922fe1bd0a182de115952342b61d627bb16"}
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 1, "pair": "d9d8cfa38981847f", "unit": "fdcda2307e20ef1b9247b7082a85337f457dd1f4e648bb8264077bf983c03271"}
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 0, "pair": "1f73ad2576d52a64", "unit": "0bc9634940734cf7f41628f220032fb7e2fd2ff5b584afc519875e90ea0be5a4"}
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 1, "pair": "1f73ad2576d52a64", "unit": "ddb59b2dc39715281a6d48b32e7ca4ec0354b85b953fff55139579f83905b7d6"}
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 1, "pair": "a7326e6daa507b79", "unit": "793ebf4c8e0f46d8ce3b3d090d2f6290d4167931a3b574f8269b90008e3dcde7"}
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 0, "pair": "a7326e6daa507b79", "unit": "b8d25f337fad527544c33c82f473015e08399457276fc480477f3cfcfc9c5b8a"}
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 0, "pair": "211ba072bdfa67b7", "unit": "3217f9d672a4a1d78005c8fb66e1bc60a542cb26bd8c46903402c765c123d60a"}
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 1, "pair": "211ba072bdfa67b7", "unit": "8370f9624baa0d144e56b8087eea897b98e5b2afcd6eb3a029d03e5b4b69d584"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 1, "pair": "baccf350d5cc9911", "unit": "50a3ca70700c8b41874c43ba94e6b7ffe0aa28b8930ff941a40767e1057065b5"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 0, "pair": "baccf350d5cc9911", "unit": "62927a118d3662c81450e19d49f4146b66f29a58d55ec0d1606ed521b1883f12"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 1, "pair": "697f34e19bda57f3", "unit": "775b6d900ec1ba71e4082ed49bb94439f38066628eed91461eecf0658caee78a"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 0, "pair": "697f34e19bda57f3", "unit": "9e855586c685b3c62ffe32407a82966661d5839e2cff742f2676acaba645a97a"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 1, "pair": "143b900b99a46cf4", "unit": "200f96cd784ff5c39d8bba22883f22bc8a7c87c6970a7c871802f48a45548337"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 0, "pair": "143b900b99a46cf4", "unit": "7a0d62ec424849ec4681f4333fbb077e8239054d1e964bd99b40847ad7ccdb01"}
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 1, "pair": "4b4e81797ca48cbf", "unit": "a4d17d85973d024a615b89334ad5a2f46afea660ef5d0bcf2b2196c998b1caf2"}
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 0, "pair": "4b4e81797ca48cbf", "unit": "d604e460d8f5291124ba6fab5f9c94c7dfda61ea5f6808e512162334d6d6db4f"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 1, "pair": "bcbd06e53147628b", "unit": "0b5cfe2d64771e30e50ff92cc409b8dd5e30079c252918ba03774cb5198a73b5"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 0, "pair": "bcbd06e53147628b", "unit": "b805976c35330452af6c9f814ff4617b7b9ad0e82e1fa4440a443f93f0772c4a"}
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 1, "pair": "c12dbc8a50f7fe9b", "unit": "04beb8a0fd054b92c2dd0ed83907330fdee6124d86114df948ac72a297a7d51f"}
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 0, "pair": "c12dbc8a50f7fe9b", "unit": "c5029016a6fd2e3f9df59dba0ce3c173bbbb0d2b1de22d151086e01e1b272119"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "0e213aeabacf4205db024ea3c557e720c3aba911424555784917e967d6404867"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "e8bc65415bd42e43", "unit": "4920230ef773595a9fd9a92a090dff42b48499a1dc5fc8831f941dae55cb2538"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "e8bc65415bd42e43", "unit": "49209a810f207eb61382e2ceeab7076936838a5652546f21c187d066e220160b"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "9e2964884cdf281fa3f18ee3481aa62f9a8a0a195d605da2df5bd71876fe311a"}
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "", "unit": "f9cc6f1bbcd59f5dc0de8f96248f722ea0679f8afa47c53f70832151ce7616d3"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 0, "pair": "e69463f945f9a0f4", "unit": "5feace7b28e260175f80c682e9009025aea6fae3dc75409cabbb955b560d0372"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 1, "pair": "e69463f945f9a0f4", "unit": "799a4f5a5b9324e86c90752f0239f059984f44e8ff1ec0dd518d0d9413e34c66"}
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 0, "pair": "71f56810b8ee6947", "unit": "c659c0e717b7dc1084e1178288f6c9b2e87fbf0ef518ba3bbaedb0fbadae15c1"}
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 1, "pair": "71f56810b8ee6947", "unit": "c8d2e035a6099fb775c8b4ecf00196c1488dbddd9eed3d2d268a4f1d2567b194"}
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 0, "pair": "768fc9db8a75a351", "unit": "adc6527469618f72b1dd7776814717cbd0056a0c625e05bef74199171ce73f1a"}
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 1, "pair": "768fc9db8a75a351", "unit": "da47663a57e370a75cc65b810b8a380aa26d12dcabb1d9b3c02641398ff7d348"}
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 0, "pair": "fab4ce020049724b", "unit": "b9be90242be627aa46462143ac85940a906232ebc0b47776b5729475b27d215e"}
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 1, "pair": "fab4ce020049724b", "unit": "ece12ea7d87b4a12c4da30faed9eb530b51fc879c271b68ba5304b230cda8e79"}
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 1, "pair": "9126ba83905eb8b4", "unit": "d8dc50b7bb40bbcfc3f3d4f1f8d3a4d17eb1c662d5f759efd18656a482dd2e32"}
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 0, "pair": "9126ba83905eb8b4", "unit": "f426f9307621b3f2d6a8d1669cdb4c19e962b94e34fbb394d4660cbde5921c88"}
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 0, "pair": "7ac5e21d9d8f4c47", "unit": "1d1efa2fd7323c264290d4d314c957c853e1c7a9c877a54d6f03ab7d2af2ca86"}
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 1, "pair": "7ac5e21d9d8f4c47", "unit": "b5dc78c7ac0b76803dc98904142c1139083fb9daafb5bff40e7b5c3f14ee03f5"}
{"family": "deserialization", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "0894a7212280bab755cd9d93b1f0ede2c51e4fca76b76e5ed9e24ef8f8cf0b9c"}
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "02694b5fb85e9e52bf5eaaab4e0b628f1f44ff3a8d5ad026cc10afdbc4222eaf"}
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "db9a3e888780501dd3c2c125af2f9d47f68ebf02f0e0db8370b2ea09c6f9d070"}
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9af96b099dedcc4005e5fcd19ad78f73864e1cfbafbe6953b4edb6260957943d"}
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9e7c2a786068a936a6c664cc229d6f39cea1fe57caf0806306f0cfc44515460c"}
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 1, "pair": "07d1dc4d299a7700", "unit": "0e664e274dfb438d2f721edc81f8469ac48387dbd5c1b43973cd7e651db46db3"}
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 0, "pair": "07d1dc4d299a7700", "unit": "da55f60378c67e91683b9720f2fa0e6659e28c7b392c050278b40713e47c84a3"}
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 1, "pair": "749eee05a507fe93", "unit": "24fbb1dffcca083382a23e48db232e0d83bbbf2ff2b325f0dc0f4c72dd1a16bc"}
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 0, "pair": "749eee05a507fe93", "unit": "640e1626f8d7d9d5c2478ede48cc7b2780370bda98ae114bae102bba892da90c"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 0, "pair": "e36bbaf8f3cfe65c", "unit": "14f7c65fb5621ac5e2068c8695ddbf482103e4320ec3b36886f6e1f08cbc312d"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 1, "pair": "e36bbaf8f3cfe65c", "unit": "c96dd379dec2ecfd99051d7a20e6ef5601821f93c46eb34ed087daf5fbed2d47"}
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 0, "pair": "b81c8c63cae892f6", "unit": "5982fe7fa1ea6bfc1c8dcb8c71cd4410de475d25cafa324147649f26a1ae1324"}
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 1, "pair": "b81c8c63cae892f6", "unit": "59916deeb8ba14922eebac61c9eac122e15d1f7590ee3cb1b97f019585fb798f"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 1, "pair": "bab3a167ab5ca505", "unit": "5ebea23aee96c80fa0d2d9239964d99614613c4ae36f709811bc19e61ebc720c"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 0, "pair": "bab3a167ab5ca505", "unit": "9bfea8d7a6ffa6d79d16f77fc355fc1bb605189513ebad561ee523d577a1498b"}
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 1, "pair": "5a91049be21aa611", "unit": "1ab9936627aa59c3a60ea44074ad44b68aea4f9c04f14c1949dd9c77ca13ff0a"}
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 0, "pair": "5a91049be21aa611", "unit": "8df72b9eab3f2fd858c40312d0a806cb67c620bd62873826c8732ba8795d9cfa"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "", "unit": "115d87f37bf1d4ab91fa3010303e81b127f84b817268b60b8fd4b9934efeae4f"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "2b8101c89c839614", "unit": "7bb7d858a670fc51f81e08c141c2bca3b8b7524254ab679e33b22ce79536e895"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "c23acf4f38945093", "unit": "82bdc0b6bb0c19e342efa63af0c9014a607461d0682cdd14bd5fea33516fbd50"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "2b8101c89c839614", "unit": "886285661b88f14e00ff8e863b31cf7b7afb303ad4aa4aa13b1cffbf1272d5e0"}
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "c23acf4f38945093", "unit": "bf74aae5a48156f183426a80cafca6da219ab13b2ae4b40fa88fca75acb72980"}
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 0, "pair": "d7cc37a3a54315cb", "unit": "4aecd43c6e5b81daf7aadd6a6e22f22c17c08b652a2f5121a49320ca1057b186"}
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 1, "pair": "d7cc37a3a54315cb", "unit": "88123ae860be2fdc28fd6a289cf26c3e0af0689f455cf8c15098e5dc76e5c8a8"}
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 1, "pair": "5011004f06f1e217", "unit": "6c48682d1b02ff2559d92346d192cc9dbe6a0bbe2ab2c146653110420105ff1d"}
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 0, "pair": "5011004f06f1e217", "unit": "d3592260108a838246e0cfd5ab6d48bceb0ef870e223565745dcfd68f1826840"}
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 0, "pair": "81caad96b221c2b8", "unit": "3c2dc720ed159e55629ba737665d93859def9790427289567a513c83c133e236"}
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 1, "pair": "81caad96b221c2b8", "unit": "e8b9595a320776c8e663fd0b168193c1e64e36a9052784d1219f2c4a91d647f2"}
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 0, "pair": "f9b6599018ef3b60", "unit": "7558d6ee39a311c5459e35fe6f7b12917b7f843b21ab1fbb5e485a07f53b7d78"}
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 1, "pair": "f9b6599018ef3b60", "unit": "b1a6cf450a074788ffca2bebd880435abac2c376e89cb328d7a22d277fdccad8"}
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 1, "pair": "eaf2f03db2c502b5", "unit": "38dbfbf745d7cbcc52c6fff83248d84663273fe6a1f1b014eac7776c4bb8f074"}
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 0, "pair": "eaf2f03db2c502b5", "unit": "a84d848ef56b72fc15872d88acaae1743c9cdf88095d13b1869abbec6a0d8a8f"}
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 0, "pair": "7df48bca2c9fabfc", "unit": "7b76ce8f55d61071adb367018cc3e50f04a05d08c7c4919c3806fc1cc8bf41a7"}
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 1, "pair": "7df48bca2c9fabfc", "unit": "c82ea0adc9c8366aa2b77ce73772038676fc37ff50879e873ff397958fdd7ec1"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "7f02a09b8d8f2321f2cdade74c2fa1d52442e006149a749a8be768daaa62195d"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "9b5e439943ce5c2b04d6201b7a843634aa1b13b726813c23450ff54ce03f3dd6"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "5b836889b4da4921", "unit": "a84c2ce4a4eadf265d4481ad7f6864edf1d7e3084c6ee32cbbfe454bb2946bfe"}
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 0, "pair": "5b836889b4da4921", "unit": "caeed7f097a3311fe2e84efd372fe4b4c140f63a80a9f07ba0b61966dbab15e0"}
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 0, "pair": "ac526f457067fad8", "unit": "50d8bafd5189fab0429bd4135d9ed400cc2fc14706226a2de98d65746ebfdcc7"}
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 1, "pair": "ac526f457067fad8", "unit": "9532b70b0df76a34170b57c74deafaba54d5d2f16a729c39b0dfd3eb2022d899"}
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 1, "pair": "c97a971a18f85244", "unit": "95c80c503ed43fe9a0370f89ea935d47badee6a9625548b9dca78e04275003dd"}
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 0, "pair": "c97a971a18f85244", "unit": "e9810872beb9c158f54048392142c2f701314f854d6dfc748553ca36ed6c08d0"}
{"family": "access_control", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "c2a2c2d85114a97f892fb1e5500fbd98622457a8c46451e1f9e958ab08a40318"}
{"family": "injection", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "29d8be8206c88ba7efd86b56a7dcf8b655e33b4a792c2d18b028c92c6252daab"}
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 0, "pair": "6a4e0bd705331747", "unit": "6db80439d89b54f60f836c415dd403fd09d21cf630da81435d4d2f9cef5d883c"}
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 1, "pair": "6a4e0bd705331747", "unit": "bf257e4f0812012be330fe119d07d29253d291161f435262787f491762c6bfb1"}
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 0, "pair": "f96f0fb245948915", "unit": "0954e6931373915c204b9ec8d2b8ae6297a95667bfd41aadf9035625ad93967f"}
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 1, "pair": "f96f0fb245948915", "unit": "5e09cdae2504e834d591b6f92a808427f07550364b5419899f7e6f76084fc377"}
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 0, "pair": "bf11f138fe39675e", "unit": "56f0d0b156992af56141d6368fdb0c5376fc7a44a03991d754f97202b9d77008"}
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 1, "pair": "bf11f138fe39675e", "unit": "8271af5017894167556893da2070188060d06eebc87f27db45c46234c359631d"}
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 0, "pair": "9d937c2dffb4aadc", "unit": "01ac12bb06e660d0aeef8b1e26dddc5c99d52cc333f7af6d1981eee2818e64fd"}
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 1, "pair": "9d937c2dffb4aadc", "unit": "9612edf0f927048335e83e9925f01be9aea9b772bdbdc568ad9cc0a17055662a"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 1, "pair": "d4f648048e45f2ea", "unit": "08807c8110b8c1b8635a6883c229fb91f2b9566db8b6c252448a0db7b2ddd47e"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 0, "pair": "d4f648048e45f2ea", "unit": "10869c931053bf77ecaa98a47f56dea27d643d1abed2a4758bcf48a78547ccc9"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "662e1f36c230d7f1", "unit": "0b73332b25413902c26d5b5dee6767465eed99ff71a814582988ffe0d8ec1a30"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "8c8e46d0c7d361b784baff9d70f38744b2805ef1ee0aaacaa616bc20a2c6f32d"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "aca938682c196fd4c85b39b89eeedb97c35f6a5a017651fe9c03c5b727cab7ea"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "cec00d1509a129364da6ab8537fb3f433fd847e26d073c90311ccdaf7a1b39e7"}
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 1, "pair": "662e1f36c230d7f1", "unit": "e0f58a7ac5627de3e7b6cfe5add41d18b0f64b998c1cc788d42eb2cdbbc3fe41"}
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 0, "pair": "2c251a645094f389", "unit": "0d41cc634771048f72d07c4aa9feaed669d0bfe3844d61dcdc525a0f899617e2"}
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 1, "pair": "2c251a645094f389", "unit": "ca8288d0b8b94dc6b7e4c681936ea610d295fe949cba5f64f09b6de134ef9875"}
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 1, "pair": "2ac5ec5011615c9b", "unit": "2f427f403af3cb58d776437736c2389805543413fdb91aad98979158c6d00de3"}
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 0, "pair": "2ac5ec5011615c9b", "unit": "cddc8c5bed0841b80e7b20d0b378e858f8e4ac37fd7b890ce76611d989857f92"}
{"family": "injection", "fold": "outer-3", "group": "frangoteam/fuxa", "label": 1, "pair": "", "unit": "0c7830c2912ff285855d8a3c8e9b47db4ca4af840baa66d4b5d3f4422f46f95d"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "5383a8c2a78fabd3d17a5b3021125953b5ebbee61f8900c30d016c56b1a3bad6"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "360d8bfd2f116e98", "unit": "7c1989a4ec3622bdaebcf1161de850f22cfed041cb38830ca1ded4fa99d40466"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "ec8c706cc2b20f36", "unit": "848688dcd0fe8d46e6762cc8aee1d82e8fad8a2a044c94b547aea11049c85bfc"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "ec8c706cc2b20f36", "unit": "983cc9231f91baee4e2d626193f3b88aa5b09907f6faab870949bdc9c336b816"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "e4ad86416d5c6668", "unit": "98ff1dff2afe80709b8e61a9c95cfeea12b2110c13b7120478268b4762e2c992"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "360d8bfd2f116e98", "unit": "a1a825d23e34757c8e961e43ae791253e62682e130acdb846b58118588006a98"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "a7fa4207508b3989baabc7a1f0ff2ea5229b366c1cb106e7fc945134b8b106c8"}
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "e4ad86416d5c6668", "unit": "ef5c53f3114d0057e694df13dc07cbcb532c3b0d59150eda269b87b021d7a6e8"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 1, "pair": "9ae575048504c690", "unit": "176acf077fd58c3e6f940753179ffa3e8316661f355d94acb5e0345b51925ca4"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 0, "pair": "9ae575048504c690", "unit": "228b34ebc2090b5239c7821d28366ddc663e7118d015a79b45dca6618d5490fb"}
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 0, "pair": "d6edb5496b3fdba2", "unit": "9acb8017ebadc480515a664181c427f37359edcc1dc6f344f368a87e0c191b3f"}
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 1, "pair": "d6edb5496b3fdba2", "unit": "c1bfe63148bc700c812fde95ad9f83d456e4fa04e10c6f5394072a08f73174c6"}
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 0, "pair": "27e1c8e8ee93a892", "unit": "47fb89b132d6de48674c37965579e727d922bef1796db70f8b9f862ac47cb9a5"}
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 1, "pair": "27e1c8e8ee93a892", "unit": "79887f0ca8b05d6599d55167d8a9faf67bdb2254d17c143141da44971077002d"}
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 0, "pair": "a3fdf9ce3441ec86", "unit": "653236018929ab8534c27704a20ec5d358047bc66e451620d2ac32081c91d333"}
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 1, "pair": "a3fdf9ce3441ec86", "unit": "b70ca2c43f5929e7b885bd728349e721fd85d7605e2f722a7e8cfe0f0d477b0c"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "groundhoggwp/groundhogg", "label": 1, "pair": "", "unit": "92400e83890882a66ffdea1a405e2354a195442e176d85fb707b0442c44d45ca"}
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 0, "pair": "f9dcf3a0fb0f0a14", "unit": "7ca6dd4f673976b000a657f885af3463740419ce41e14f972af0c5483cb75a7d"}
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 1, "pair": "f9dcf3a0fb0f0a14", "unit": "86c1db3a5e9f843d6546146449171e0378ac61d6c8f44c6dd72fd379cdbb188a"}
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 1, "pair": "3e71f79348a6b770", "unit": "200250ebceadde4cbdfec2802a4f9fbfa220dbd7fa4c2dd5ed717fb1f3ea7447"}
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 0, "pair": "3e71f79348a6b770", "unit": "b9faa5007c8cbb67ada3f1f026ca59e0ace6fbb9c3d1f1369d4521a7e323e7da"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 0, "pair": "8972a0f56ee57477", "unit": "6b987df2960828dcb1b2b3f190a903a40361d58117d63321d671aecc6c2cb570"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 1, "pair": "8972a0f56ee57477", "unit": "ca3fa87dd37435c01a7121119912c1a1e261259c11805c0b66e7d4b315640945"}
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 0, "pair": "6144baeb491f4a56", "unit": "f5c22ce104e9593e5272b6e8b3c6f3b61fa4578552ccea80e6021ee90bdb3ffc"}
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 1, "pair": "6144baeb491f4a56", "unit": "f687e7e95dbe47f602a1e0002a44f94fd48e5b1c33f77ecebc6c2d814fcbf450"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 0, "pair": "2ef4471c66e5e4ab", "unit": "ad7c8e368fe1699a044e058d5c10b41bd9fc0f91c3f3c384c5dac40af39e71f6"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 1, "pair": "2ef4471c66e5e4ab", "unit": "d888baf7194bbe7158cb3932f58b47a9a235351d388d10df82a130ddd54c2d46"}
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 1, "pair": "5628c54152216646", "unit": "47d081c9f6060218967902f694e61c88f64f57c65bec6fc3073afeb0b77d117c"}
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 0, "pair": "5628c54152216646", "unit": "e1297bccf9c3606e0b41d762a69ff5d0a33ae4f7f1506071af6b9cb4a177ff56"}
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 1, "pair": "e52aa814c36ace1f", "unit": "2d6d74364f1982e91f27f1c995b8699df1fc8f8dce6e1e2cf274bf01b6e9494e"}
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 0, "pair": "e52aa814c36ace1f", "unit": "4727471fea2cf0a92d43b4b1503f1339a7dde87f872ba8064dacbf855c57705b"}
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 1, "pair": "a07929b3fc1cfc28", "unit": "0637029194bd7378237b4dc944dc5cf9f2b63db9796f27410a74a9d567b66dc4"}
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 0, "pair": "a07929b3fc1cfc28", "unit": "ffd286c8588a77929639a89e4a742e15306b642af14aa906c721165df804ccd3"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "6548e1527e82bd99", "unit": "5542613f6d2eb9421b8213b8bb864894f0ecf9fe64945681019230ef0d75cfaa"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "", "unit": "6c2adee34e9e8384535ec1073eda60e2ab528ac77c36747802c303e1f5e9ffe8"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "7ce70e9b20107cc7ebb87d173da765fc53ffd268e521749a06a9a1f38d746845"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "6548e1527e82bd99", "unit": "c3e54975464323e58ec96ee4d83e7bc08baf94553d553a8dc3fb28cd9c70ff21"}
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "d3173c1e43eb6402a71ea2dec1592913dc078a2194ba1c1d509ab129a3180f6f"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 1, "pair": "0924befc49c7bb18", "unit": "e7e4872db40f0420547ed622fbdd7a072b64a7ba466d1ccfff68e691cee6ad99"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 0, "pair": "0924befc49c7bb18", "unit": "f7d95f43fb2df20ebc4c37fceabdd00ffd8159f404ca384481f1764252382388"}
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 0, "pair": "d4ee6cfeba71e398", "unit": "e79582de3031bada2617a0667ef7ff5cd7183d4aa885f55a665cf0ec88d17624"}
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 1, "pair": "d4ee6cfeba71e398", "unit": "fef35272853e7f3692efd54daa01c12862953352fa6ccbbb5e3f7fea70c63d78"}
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 1, "pair": "772e18821701d60f", "unit": "89a0496c7663bbfd67d660c9f481c12420da2dd1bdb29dc079735ee7a0a49626"}
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 0, "pair": "772e18821701d60f", "unit": "d3241ceaa50a7d23a4650e6a2a55372b08ec96b35b0bede844bc40e16e818ebc"}
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 1, "pair": "250de1302c7e541d", "unit": "5d03535f2df2acdbc319d9e6b20da65d51f3e810b80f3c8f89b5074a971a3f1c"}
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 0, "pair": "250de1302c7e541d", "unit": "78f977cfc096e34b4eada7f798ad2973ef45728d0d01bfab6ac9e2a3257827ef"}
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 1, "pair": "c10bfb006f2da55a", "unit": "1ea4b86dbd84581b1d7b7cabbab9f28c8b213416a60778879666ff42d23fd7f1"}
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 0, "pair": "c10bfb006f2da55a", "unit": "7647021f742da24c660b2d407886771786e957076c4cf336def560a570dd8778"}
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 1, "pair": "a73421efb106ab0a", "unit": "1f6f207a758582de0c0e6b331b19fe119abe18727d45ccbbe09aabb795f3a292"}
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 0, "pair": "a73421efb106ab0a", "unit": "32fbdad39eaf5177db4ad0a11d53e785b30bc7b2376ad9d460f29a08429bd0c7"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 0, "pair": "0458e3fd444409f9", "unit": "b521216bb4106738b3571ad3902d08f6230aabd88cf3a8f9f26f1c8ec01304cb"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 1, "pair": "0458e3fd444409f9", "unit": "c87d2871907798fa2e73d31381e9d63089b0388033f096d247373fec07c2cb36"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 0, "pair": "6bdccabcf5b9fdc9", "unit": "3d6ddca5eb264d88411b86755909f1d2abddd09c129cabd54c96ff26b4105316"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 1, "pair": "6bdccabcf5b9fdc9", "unit": "e055b898802279bc63204cc601e29f7f1018efad7228cab0cce00ea5866ecc7b"}
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 1, "pair": "91bc832e70680c92", "unit": "1060c63ff0fae3af567448f3d4c60d9c159b61ccc358964243eb152a9ac38865"}
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 0, "pair": "91bc832e70680c92", "unit": "369dffd1e730d7997813adfbebf1481cc68987cae0d0085e47b91a26d5330cf8"}
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 0, "pair": "137fb3496a3a75f4", "unit": "7e8b6c484999229b23d50d9f5a4c4c5b2a8d954a5d64230981b1c6705b760fbf"}
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 1, "pair": "137fb3496a3a75f4", "unit": "c461154e4dbd74e35a4ed2e85de7b076f0582beeec459be28104c38f616c7565"}
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 1, "pair": "a4144258dffb3b8d", "unit": "7ab301e1614979f44c353e665540582493f78ef4bfc41209562c19b900710f8c"}
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 0, "pair": "a4144258dffb3b8d", "unit": "ea95e8387dff32ded9dc7b04c11a76a545b4e2ea6db0076b9b13971a59904bb3"}
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 1, "pair": "858330e01d3cf9bb", "unit": "033cda2b7cffa5a5fa6abf0b7f2cd27ba78d916eb3cab8c79decccd8776f4a3f"}
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 0, "pair": "858330e01d3cf9bb", "unit": "d44c4555a5ae19aa17b468ec5cff56b33a0b221df52e04d8abcb656399499dee"}
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 1, "pair": "c6a49339d4f0c798", "unit": "dc2e4cd5bd963643702d2b832ae170dd25744162e4076e36687256a09e69d089"}
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 0, "pair": "c6a49339d4f0c798", "unit": "f83cd137b85e651ca5299c5a42176b797ed4964cbef31418c96d63c52fdb65f5"}
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 1, "pair": "4b6526d132e9ac86", "unit": "8ec3596444f3045684242840e9a1df19879383f2733aca8f8af6f6cf8a96d388"}
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 0, "pair": "4b6526d132e9ac86", "unit": "f3d39162fa14446b207aabe7bf3c913dc32a9637ac83352ea5cb2fdbb43720a8"}
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 0, "pair": "80abf1d5c4a3f6a5", "unit": "3df3c6d9941352dd2861427da56b94ee335f308d0a2a27c6f745aef60349c80f"}
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 1, "pair": "80abf1d5c4a3f6a5", "unit": "4fde61a570906e658d9b14db0e5d1f5acc0c7489951421bff2afde25033ff73c"}
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 1, "pair": "5ec37c0f475001ca", "unit": "5d8175fb21a4649bdc8730358e9b87adb90b2aafc4da5fee203829663e4f46a0"}
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 0, "pair": "5ec37c0f475001ca", "unit": "af845e9e332555096dd350401977b2f07015433f3db8a040891d9470f3f9e355"}
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 1, "pair": "68d44ef338ef3470", "unit": "3020ac6c3186d7974953b1c4da65b80141b81aca5203436ae7a2d7aca0870944"}
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 0, "pair": "68d44ef338ef3470", "unit": "8fdb85a6bfec89f0715ea81e83fd91616f92c59954c092e79272154f202b7488"}
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 1, "pair": "c74525e844650de2", "unit": "2e444ad34ee5f47d73f0cfad0a10c70363337b4ddf9067ecd1b8072d8232652c"}
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 0, "pair": "c74525e844650de2", "unit": "c492d325f5ff82b5f817aa4be1440cec8595054dd0719e07a58ac5501b2ae7eb"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 1, "pair": "111a5a80a3abc2c9", "unit": "c7ae403742450deb2bd051ad1231121d2117ba56fa72a038c82d58d207cb3870"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 0, "pair": "111a5a80a3abc2c9", "unit": "debe34831afca6f8220b9952036a2d4570babdb5e321a2f8dbaec661a016bd50"}
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "9950b85412b021ebabb11eea4c490a660ad380d14c673a87c3fb568ba4bcfac9"}
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "656df5ff8e69dcdb1f9ba1ed52d734966eb29773bc745587483a355743b8eb5a"}
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "91652a4e776fbce70cde9e4a99c6da5e17da7f8d6d0404867c61eec04161b6cf"}
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e53bb3a217dc4737e8dd03671de4f6a895599024070a985a268c65a034295e7c"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "3d7aba22444811a337abcad0e4fc62990bb704d3a10e72d893342fa94699026b"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "5f18a3026a51f7140b7aaa3aa8563e8b6fdda6065d95e19e25b1dac4b9ce0e00"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "0fccc306ce80cb4cbfdcde0a2e56d637bcefa9dd30423d6c54a4f77860afe740"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "1e2eaa45661345f1b6759d968d7696dd0d96d6fba1350a00e4d0a2ebaaa2e391"}
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "8b9a8ff0e32a327df75eb8a1811464d5f34690079d5930139dd917063bdd2809"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "245d0e939b7c1ccf098c798492009e3a73ea17400fa613dd865cd369d2e7f383"}
{"family": "access_control", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "653c48df493f31c2d01c973d1e379a180fbe6a6edc83a500bc62509112175f41"}
{"family": "path", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "3719fe21d45b50b35458bcd2e263e4baee67adbd3c98adf0eaab6bc6183c5651"}
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "1209a6cc8c1548ed83d4a9a09d6a27f706a23e69cf0c261fc5e3a27536a83306"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "362c90495dc468bb732c71f62d71ec308854161de31ad18d2a870056d947ddb6"}
{"family": "access_control", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "b52e53fec9707b2b442c4dc7a41d376cf76946d2c45c2eb600c9477fc6baebaa"}
{"family": "path", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "477e3b90b92e622888dbe0e9d3a23b31271ddfa3104a5ab326eb9e9e765aa01d"}
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "660c63c4a73bfe64088ae3249e40d9c5d69a6f8a505ff671217c5708a345c5b0"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "4d60c386c801bedb61c1461fced29a3cfb50b09d6435e8617c60398343719d93"}
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "f1fddcc1354a29bef55610ee2b73eaa440192f2f18a0d22d08057b9d1e1b61b8"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "6d9886c38221b2f9aacf83749218ac5ebc354a6fcc4170e6c78bedb92b88296a"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "5550fec4a81c4c9cb22d7312d8a63e1fb2b03c4aed0c1f316d44c1eb27511bac"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8ff87dd04140f0137ea1891f220b1276156ce10a58be5a49d0c8e07be531ffb2"}
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "f24b19cc0f67247eccb274521a62741fef8f4729f0ddc9b5c1fa561a4947182a"}
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "83523c9d83380d4cdae593a593d361493e00d18e61740f84e762aad41b991e18"}
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "c6c9bde6630678377d250701073c5c223c7892e6ff8f2f8e0293dea22c5b5e7e"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "b9b15958973c83f8d7b58372ab5ba58eb191c400125579d5ae8c5b52d5d1368e"}
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "b95ec12e4e126ad558e944e2c260116ed6a63f5f3ab97fb5cbcdca75a931ad73"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "38a5545ac9ef67e15db720d1396a2d5915f9d06d6f4772012e5bc95f33770376"}
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "c8bbe9be91c976be97b7ca387048a7d4537d9f6fcc6cbad0a541d1080629d5a6"}
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "ffacd53f1bd75155e63b318fb9cba27ec03e754841e1eb39d1d4fa0b2c3dfca2"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8fd920175fc43b91b4397acffacb10ab8f7fd0ee38bcf2d2f3ed06a3fad0dfe9"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "86c9598ebe3dcda261a69ae545513f6ce73101f3980066e4506ece99bebea84f"}
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e5190fcc57505116f79a31c51aed69f8a1eea13a8ae3e7265f44c72418277ef9"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "2b807ff54e6c47e941124c16d0c27c691cef54f2203060a3a6ce812d478c5261"}
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "401aefb4b4791301ace2d770d321c6b26743b0aab7b96f8b410254c704be6d2e"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "22d186c93498e21418b6c1d0547a73374b13041d8afc89a80d9540fa658b6b42"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "dbfb6df14fe37f8c0efda5f16643297fb0550dfc5ad5805d4e4b844d967773b2"}
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "f0da8cfe4ac1e586aa7903d72e67b44500bbd5306d72b4b342d10613efc69222"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "2b8fd73943f456da266434470333a7775f55d9a41953e97a000fdaa3257b6a30"}
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "a612091f452e60f28f44eefec641175170dc5db04456a39a099cbb13c50b4d2c"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "4c6600facd83bbf234a32d9e28ac494fb516df6e3d9e9dc28d720ff35252d067"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "fb0ea8cab8c9430d22d1b0fdd65a69146fa95fbff94e720b4861b96538d992c6"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "e81f91a56058ac1473579767abea57064c53d7d007797a854314814e3c35ee74"}
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "5851ecd96a449f98b05f243bfda0a7cf3af8612b27f53c414ae1f4ea67242a7f"}
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "328621ef2a94632e5d37ea6267d17b766233384cde35c07e29a4eb672d40ba28"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "04b2eae2128608dad460ba8275142a41a44e32c4243a29e2a6fb2c9ebf3d8f1a"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "33337002ff5f5a189610b042522ceea254c9d107c1f24771b4f616a0f96f636e"}
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "02c09a783cbf058856fc2b448406f3ffa6fbfbacf9b70f4e5e22fd741a5c7699"}
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "956260164cb21a47c499e5598491031ef303418943f5096f225a87162b588574"}
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "8bd8c5aed69eae3019cc2d3654b36bdf2b31435e6a020a389cd103696a2f874c"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "1d8397ef4a9fde7b48e2e1e871bba0f960a802cd7823ecfc10e9dfaddb955ccc"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "c9db92476032eb4296b950a1eff8fa4aaa26f11542a01a5cf10c7b2e32dc6433"}
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "d3e0d0f9b0f4b723080989b5486395cda0a5b186a674022ea70e7fa7ab561323"}
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "44ff3b3e61c02744830c4abea872821ca7e0f94950c082740c5c48ef13604e94"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "4103e979a99233c3e71fdfe9f1cdcbab34a2c60b5351b75c5ff5dfc287b962d0"}
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 0, "pair": "f73ecbfda2bcadef", "unit": "53e596dabf90b2992d5b3e50394791859b393beedc8f84e00484e37498627ff0"}
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 1, "pair": "f73ecbfda2bcadef", "unit": "6daaccf64706e876dd505c65a62aeacc1357f3470a5cef7b1ac874032a6fcf99"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 1, "pair": "6768bfae38c6859d", "unit": "f376519889a00950f5af67eb15c639306ea70da7235d3be4a94dbc19cb096cf5"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 0, "pair": "6768bfae38c6859d", "unit": "f3fea4e334f49ef6fe6b17121fef553eb8e33a9ffa13e6d6cbc4c448da996a7c"}
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 1, "pair": "d670253fa209671c", "unit": "1aeebe2e440bdd70525c31f93180f33be93f820db5ba46a2e59c371ef83ee794"}
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 0, "pair": "d670253fa209671c", "unit": "b11c8a8a24d3809c0507228568fe7cc05ca0a22ed28cb19019adb13cb619d63f"}
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 0, "pair": "33260f3e6c8ea6d0", "unit": "7ca409c7dbc1c38ff005624e212efc6d424cb421e27b482dc8dc0d6881f4c11a"}
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 1, "pair": "33260f3e6c8ea6d0", "unit": "fc364d7ed77c7c0508da00e5ba847ec0792f3bc373d8b4297747189ef42d1ae0"}
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 0, "pair": "32ef8148a2c08f58", "unit": "2fb2c9314a483787168a450f50430721f9527f913aa0eae07298d5334cf4f137"}
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 1, "pair": "32ef8148a2c08f58", "unit": "78dd291bb2ef83ea3a87948c0ac0027c6d667b9044093e766e2bc91bebe83936"}
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 0, "pair": "54586b5a24655adf", "unit": "2ece75a0240ed36a811c381841aa6d440af2b9774ddacb1d7a31a67ec267b0b2"}
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 1, "pair": "54586b5a24655adf", "unit": "e60f6b788a196db9bb671d621189b79a5526302a6d09c2396d67fbe0b9ab3c24"}
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 0, "pair": "474f96e2a9b3d202", "unit": "4a5e8e8ebd7ecb0e252280af0cb38a1d7794757728b912525322ac26df5ee223"}
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 1, "pair": "474f96e2a9b3d202", "unit": "6a888540443061d3b37c06f0cebc70cffdfa70e0aa940626e86cbff9829310fe"}
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 1, "pair": "a09692dc94164db8", "unit": "139cf84e79c4efd026a064b2ca0a3d7ffff87b54eae1275a07c0d69ef4981e27"}
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 0, "pair": "a09692dc94164db8", "unit": "828e88b656245a1d0865c79b2ed023a49e78607a4ac81d70ccd3690f4419d2d1"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "", "unit": "6cce78d7569ad8db7ca0a61d7078198f666502d26ea6c0afb03173a8a4bb16ef"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "82047c0513e43a83", "unit": "6d2dbed1ab1c9d8ac259dd4227be69641fa59d52e816a8dc547ade01d0b9b0c4"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "98ee23cd6bd31cd3", "unit": "ab4bb39be535d70cd95e9035bb958bfc7418576a6ff40a47551894dd140317bb"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "98ee23cd6bd31cd3", "unit": "b70068cbc646bd01d3e10986e002f648a74e9d798900dc59cf3039b749c95c18"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "ff168d95d8d9496e", "unit": "c6ec9478c125ed780d7013c2807f2cec1878f44253667dfed319c28cbfd17d4d"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "b1bfef46dcd3b9a5", "unit": "cd4d095d8380a2fb7ca02fc6abdbae5249cc06455db2929875533f4aa8b886dc"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "82047c0513e43a83", "unit": "d1bade217be8d345cf5d0f0442ee31924187acbb98779dada95274a77462f696"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "ff168d95d8d9496e", "unit": "d561db250b892b91feed7ae877900e1e25963444f73c7699576e010f06da73cf"}
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "b1bfef46dcd3b9a5", "unit": "ed78fd5a7ab30bcd6e8ddd27c706a19cfd89750de4fbf81c0b567946eb94db83"}
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 0, "pair": "25f75d77a669b54e", "unit": "0f2825568461562f2a915a4f16ef8a4931290fd20a9801470aeb12677e66e3f1"}
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 1, "pair": "25f75d77a669b54e", "unit": "bf22bb44bcff4a5baf5d550c9d3e83e5f3ca607475fb51deaf4fb6fccff3d73a"}
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 0, "pair": "47069df6bfb3dc00", "unit": "444c1f41fc23ad656a8bf98dae58b36d0f1daa214ab030682f26111dd389e58d"}
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 1, "pair": "47069df6bfb3dc00", "unit": "c0c8b5cca070673ea4b4bb8f08dfff86b7781780e300720c4051aded035052fe"}
{"family": "output_encoding", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "cd80750447cf7af67ad809279718d98106c71705fef9ff7254728bfff4ece5b1"}
{"family": "path", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "2ce605ab5ce7ad3a1fb983cd6bddf3924e5da3010d2252a0a4640aab4e5921a0"}
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 1, "pair": "b0a899116dc36b85", "unit": "510ad48af03ba4a1a2a06dd254b135311dddb62d9fe5a2e8e8b12627586e6b13"}
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 0, "pair": "b0a899116dc36b85", "unit": "b414718a8a5f79739b88db24da6ad9da8e48bcb6b5e2dc99158848cdaa226767"}
{"family": "deserialization", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "8e22ee30441b20484090659b0c97423c711f5d10219041bf3a399623fc056a07"}
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "0f9070df9dc334f981cb01008c260b8bff78dbdeb48ce672830ed95caa8406fb"}
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "5e352f797b1422b0df477bce4730a9ce33cbd0837c3d63da94e4d8801a9a4b64"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 1, "pair": "dadccb128ac0e120", "unit": "505097494da5939404ad7d5b4af2b1c4a10efe94048084492d6ce60ebaa04c97"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "dadccb128ac0e120", "unit": "84ff8ca2d98a1b5d1ca4bfaabe7d19e03a396042b228c8d18d1185a178389702"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "91e261c54bd2c3c3a9ac04e48894ecabaca5b13e7db4fbd5578f9d266098117a"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "a308dfd59bc398824e2454bf6cf6cfd5b7880e5b948a82424f14753d2b6fcdbe"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "ba24eb554458bc97314ed561a7b2f4a4e643dde1205c32de4c8fcdd8c46e575b"}
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 0, "pair": "24d8f1840b67fbe5", "unit": "7ffa4a229a62266788fbf64890f81f20358b3eb8c35d3ab87d0c339d5bb4b507"}
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 1, "pair": "24d8f1840b67fbe5", "unit": "d19493ed7941e1cd25a76dc50504d97398770d07ebc302dd66d83441e946c468"}
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 0, "pair": "9f8a60eb48d74bc8", "unit": "80fce34f2377731e3de4355c7ea33bb20fc2fc00a5f388565b23018c7a3ed008"}
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 1, "pair": "9f8a60eb48d74bc8", "unit": "b68e62722a327c9d5ce6774eb96527451729410768efc0b5c7ba604ec468052d"}
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 0, "pair": "2c857235d0764986", "unit": "33bdfe911cd1b512e5994ee7a3e3ab2ad1f302cab29df3b011c0d825c23536a3"}
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 1, "pair": "2c857235d0764986", "unit": "b5efdcb84a057f3cd9fd3a30a32239dc06c13536a4e84a66c7a17cb372050c97"}
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 1, "pair": "9e84429d629c0b6b", "unit": "774cfdecac67499bb35a872931d4698d1b92ebeeb42d21790469e418889954e5"}
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 0, "pair": "9e84429d629c0b6b", "unit": "a6c5c15ca1815f83dfb47428c4a8afd4e60fa1c027b9f33c668cb9a0d77c18af"}
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 0, "pair": "8e6b4b4e28981590", "unit": "3cce3e8f9ff3f1d261f9b96c844c1e4da07bae72d5393897b3a929be25372edc"}
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 1, "pair": "8e6b4b4e28981590", "unit": "b105689ac24ed334294c434a77bb6d56d5d598830ffe8a5d60ab3be5290b47ad"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "0a36cd5177f3b2c20b433db6bfffe5ec5b7e74cacc34a86ac581f7a0a559432a"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "e7fc97d83013c9ff", "unit": "5ab444e64297fc19065478104c7ebdabe23cc415f22adafd0cf82c98e4c5cae3"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "e7fc97d83013c9ff", "unit": "6b462ea30c1afda572600a10324249223932d36a032eacafac8c837bccbdce87"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "84d6a4202a082e0d", "unit": "8175082f40d8d61d44b54579c84a808e25b89110be5d8fd82feedf7dd5e0d5ab"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "8abefca43ee4ae2c797033da31f85c17bfa7be6b35413d70b0216415c5db9a09"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "91d213946859f33d3f243c43aa8285cc6e8ecb3db1de938f32af775a70607f8e"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "a244d9f010ddb43a09e82363dfabb0e406d411fb5de08ab7d6c9ee367f2a15c3"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "d27d174d6e8e5f1682a1c42723705e89f0d18c95ecedd0567e313daa9a0aae44"}
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "84d6a4202a082e0d", "unit": "f45608ed47fa11ca25093641f6021ec06076c18676abac9b809f31b3b7713de8"}
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 0, "pair": "55d1ce4f29dd1583", "unit": "a877ce3462da316bcdd82ce9db1d9887e887eaa83157922edb1663413edae1d6"}
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 1, "pair": "55d1ce4f29dd1583", "unit": "aee67be0ba31f99c7a24faa1bad6178c6d37361a308cf5aa10138e6ce4bbcfeb"}
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 1, "pair": "16e529fe8c5486b2", "unit": "1d21acd0171726775d3da4589b660645fcd43cc026042a2b27e0e0e8374eceb7"}
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 0, "pair": "16e529fe8c5486b2", "unit": "f8e004b67ac98dc3392ea4115051e6bd10ce22eb638a762f0f16e1d4200bf129"}
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 1, "pair": "0b34438e6e0ba63d", "unit": "bf2608b00f0f784cc42e4c607f0b1ef50903283779bfb2eec91d2ec187856d14"}
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 0, "pair": "0b34438e6e0ba63d", "unit": "c5104cbf723fa0d55675963d8aa6802780ce3d274efe0012290fc49164c3fcc4"}
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 0, "pair": "b0210cd069757ad2", "unit": "d1d40c8b48d039c1999ff957a015273c30652ca0c45c4eb088052991a2a92f28"}
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 1, "pair": "b0210cd069757ad2", "unit": "e2e804652a885f640ebf341627f3279d6c965b47ee2ce792e55045ab4d1e1dee"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 1, "pair": "ef603436079b0005", "unit": "77f843b5044bb5da935df4b2620bf2abe9e7dceb3147439e883927112eaec22f"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 0, "pair": "ef603436079b0005", "unit": "f2ce95f0e44e73986784b63a702ffa4035c423056cac3667a63140a7e8d795c2"}
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 0, "pair": "a023056d2704c727", "unit": "6028fe87bea019646fff2b31c1b2d7bd0a95eb9214fe437e2008bc5d2dfabb7d"}
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 1, "pair": "a023056d2704c727", "unit": "dd93069189c5f242db611e4bfba4341c22f283b65af666fc31343854358de87e"}
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 0, "pair": "abc80e34d5f72367", "unit": "87d4a5b81bbfdd9628109a5f820895b6fddb00eb96b0f7bbe30a8d640457bcc2"}
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 1, "pair": "abc80e34d5f72367", "unit": "b171c77b52637bc84c487d7f8bce9a54229f9dbd6605e5089e1e88943ced989c"}
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 1, "pair": "1945f28a3fc5bd55", "unit": "64c0b7ccad09192eeaf590befe0de0087d5b48a65a4f6cef41415f3f54a8f536"}
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 0, "pair": "1945f28a3fc5bd55", "unit": "b55cd6759d4b7861e589052aa534bd0554586db46a329ed3beed7244d579e8c4"}
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 0, "pair": "1c9e47835232f644", "unit": "90e220d3540b88dd9efc8e66c8ab519d4cc78805f16bed585b795c78cfbb4d58"}
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 1, "pair": "1c9e47835232f644", "unit": "b5a185b05235f70f6cf27e2adba919f4d8393dbdf2d99de095548a21946aeaf3"}
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 1, "pair": "9b2b34cf62a76ea3", "unit": "91a3984e47f17512db8523e04f863525c67db18631d8f11776cf61ca7221c91b"}
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 0, "pair": "9b2b34cf62a76ea3", "unit": "d91f1e14282a0c3c8da57c6935a1625e881374bd205b5baa9e4a341d9109ae21"}
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 0, "pair": "a20cdf9dd208720d", "unit": "18a26df68e4b81d411cfbcc8155a7e6f382753d6e1ad95eece7ed24fa8c44324"}
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 1, "pair": "a20cdf9dd208720d", "unit": "e6fb74441fb585d7ef5855317fba795ef0f0a0e6eae9970a49556ce3070c3254"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 0, "pair": "b2e833b60a0fa396", "unit": "08ccacb7434e9f8b8b05c7bb0996be0609e34860d8fc418db329aaf8d7910f2f"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 1, "pair": "b2e833b60a0fa396", "unit": "2ca7f80f0badc049ce725718e49bd60da72d316589399126a5f4c5308cf83868"}
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 1, "pair": "d6930fe7e752b7ec", "unit": "06f0341dfbf1aa9504083737df5d9be3cedfd4e3a198799d8075a29990776800"}
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 0, "pair": "d6930fe7e752b7ec", "unit": "30e2cd8bd381d7568b78eaf982f562d278a6a8d942d5282466532c1441bdf0ad"}
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 0, "pair": "e9c34a6a4bf2d333", "unit": "0ae20c7334fb27889ef8551f34c65a847dffd6b37f84f722f7e8ed64fd82e95b"}
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 1, "pair": "e9c34a6a4bf2d333", "unit": "59ab1924056704605b22b7c8360f40f745e7f488450be3647a632bf9a3b22699"}
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 0, "pair": "053a9e02feda3960", "unit": "34e18fe343b5c25ec582fc05ab091fd9ba160f658ba7678066c56db4991a951b"}
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 1, "pair": "053a9e02feda3960", "unit": "8fcb18a2a8654b6594cb2ec078e2a018d20f73d2b69a2e625338ffa80b3f72c3"}
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 0, "pair": "2d482951d7380c4f", "unit": "5c1a721da6f85d1d5dfc8cbe5e9809b093a09c059e5abeebed2cbbee55fbd498"}
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 1, "pair": "2d482951d7380c4f", "unit": "6cabd1c4b5aa94cafdd07c0c6a3f9abfdc03761ddc5b0a2bebc72a508d020b8b"}
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "", "unit": "2300c097374b9098671dcbcd42d02d08172e0f3064a8323223d0fe79a6106aab"}
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 1, "pair": "0664148b0b192456", "unit": "d702617aaf4d19811baaa18386a3c8b0eb35cd3fd72e8d90611d34850e990858"}
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "0664148b0b192456", "unit": "f5ed13d811c5b16d46cbd4a3b9edb8f7d2db49acbd49d95f39f33eaab38f58f5"}
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 0, "pair": "fd5781ba332ab81e", "unit": "59eca69b2bd99ba282eb8056d7bc789e9c691eebf62a4b42594e34f5516833ae"}
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 1, "pair": "fd5781ba332ab81e", "unit": "7a7d774a6de9e4a95fe97fff822652f1f1383922f2f0a4dcb12b8e964ca330f5"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 0, "pair": "366dc5ddcfe598dd", "unit": "0053d866cec1e99f8a4a8754cf8ece3c46df16e002c86703c68937d5f3a09b08"}
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 1, "pair": "366dc5ddcfe598dd", "unit": "027dbc5b9245fb804a1e2ba4445b8afa180aa2c2b3293db107ffe0ca4900ebb4"}
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 0, "pair": "d0a34bf90e99e794", "unit": "4efbb75cc4105149522d2b2220987fd4373a8ef645c2c544546da540f14109eb"}
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 1, "pair": "d0a34bf90e99e794", "unit": "fd790f533103fd1a5646a9374ae895cf6e36d578fc747db2b0d0249a54a85e9f"}
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 0, "pair": "a697aa902d141a6a", "unit": "341e8fbb46f00167a32a2d653ba09a205794084eff65afddb70f9cfff42d697e"}
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 1, "pair": "a697aa902d141a6a", "unit": "e29414494b275afc15f3a028f62402161df7f6da5123c302149e23845177f7f6"}
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 0, "pair": "8c1fc8417190a134", "unit": "2125fb101d6108627e77f61c05cadddddfebd1cddce0e66376d9b822a2eecbb4"}
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 1, "pair": "8c1fc8417190a134", "unit": "ae7bbc58026fb71f91b477946a27bbe3599be90501da9237755f3cf0fb289794"}
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 0, "pair": "1972ea9f72b178b6", "unit": "3ddabb356d78e7c65ec8f88b6da8e4fcabc055ae73f2f6d92c50efddc5d4fcbe"}
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 1, "pair": "1972ea9f72b178b6", "unit": "6875dd5e9c0ff8a1b693d29a109cb2e62e6ff4081df40868aca70e4770476757"}
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 1, "pair": "2b007a2377d633dd", "unit": "bdd71a783efb4eda9a82afbd30a16bd7608153830dbc599e65c453c18d1644b2"}
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 0, "pair": "2b007a2377d633dd", "unit": "f398ef33900624219e59de08e1cc101c43082a81c37d6d03a916cc220727ca67"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "04aa777130c02a06", "unit": "16aa43c87956c44a4b115245140eefed704095d37cec0cc6410377403cd2fa7a"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "23744137e0485a36f686f5f1b5666780fbed3ca522e8c6d35687a391ef504498"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "67565d99fe28c68f", "unit": "2692316792e6f02fb51578cf30d71bba6c73bf5420c36f709681c8eaa69807d4"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "2e36f73002c416a68d077bffb0335d82346c7507f2478353aeac005fce5a9cbc"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "04aa777130c02a06", "unit": "67721f67055eb6fed955ff22bfe4060344662eccb369431835a2116ad120c4f8"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "8fc1c96fa6faa4d3966ecc42876189b18ddab779d58f47cdf28a1155f48a6ac4"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "67565d99fe28c68f", "unit": "ab535098a8e709b022afee1da15dd1fc371d389056938ee4fa4971b2966c823a"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "d46f94235bc462360097e7a22e1d442d95f049306631489a4d8b25f647a95bbe"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "edd6dd4364442f66384d14e9317ef2f6234e32ab78327a45741b8a816b4332ee"}
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "fc7df6887ac719d4615934c0e8c3296728c2f6089f233158c59989027c802d72"}
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 1, "pair": "c0412509f15fe3e1", "unit": "63bda2a71ad7155ad2f59aa9086efef3fd3710dc82c57f4b32ec8bf561460239"}
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 0, "pair": "c0412509f15fe3e1", "unit": "b1c9a070c93764655786c3de5fde536c150e319efe47548614cc544b73a8ee2b"}
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 1, "pair": "feed5f7de11289a4", "unit": "71531567b2b8609ae0d31cb177b8faea4f5fc61923e513b0154a2c941287ce2b"}
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 0, "pair": "feed5f7de11289a4", "unit": "d6be6d397a301a07faa761163d3d9e270d71d23d3bdfd084eeafb1e62b91dd71"}
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 1, "pair": "31c259b4e9dccd82", "unit": "1f1ddb8a5b415461638056c70756d2d0e836de71e824c0e46474f20646eaa4d5"}
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 0, "pair": "31c259b4e9dccd82", "unit": "a6ec568d4eef212a842dded7fdef2284524d367e5d24473ddb34e6374be9fad7"}
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 1, "pair": "ffd4b04a7bebfaf8", "unit": "54f7519d3d59b6d8d69a5f8f7bd78e360d3fabaae00b7587c37e388719169370"}
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 0, "pair": "ffd4b04a7bebfaf8", "unit": "c5e42c9ab2ee54e46700ab0b64e239b567fbc75ccaceaf6cbf20687813663332"}
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 1, "pair": "2ffec9490ef1d482", "unit": "0ab883d3561162b75ad9d161f79c0d17521a23317f2f574c910465f288119483"}
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 0, "pair": "2ffec9490ef1d482", "unit": "b0af86c1642c14047248f1134181be0ec3839900917dffa360c5014119f8632c"}
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 1, "pair": "73f2a014c6a699ed", "unit": "44ba9d9d8e75769c1444ebf37a64941a845b4bcae678ec79d64dc84e5cc067a0"}
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 0, "pair": "73f2a014c6a699ed", "unit": "b430d488c486cef26b1c661f773e7e9e4a177eeea73537a033eec98d532e4eed"}
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 0, "pair": "738e4c22edb32172", "unit": "932d7312b2bb4b830179104bca076f587505c2d4ce881b2f04f992defba6dd9e"}
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 1, "pair": "738e4c22edb32172", "unit": "eee5136489c3baa3653ededbf6385d0237e2bd8b5384286780646b88ed34fb91"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 1, "pair": "cb0c638282db0d37", "unit": "a49e6ad2e7c2a5ad0d68d5d69c263306b501fc0dc9382e701fb5fe680611196b"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 0, "pair": "cb0c638282db0d37", "unit": "dee28099d3b37cbd0ee14412122ecc5f8488ed6bec01863520b87706a040fba1"}
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 0, "pair": "09bce271e8530fbc", "unit": "35eb19b4c6176b37287e2123bb98ec4045ce6bde7cc3864dde317c024b0935d8"}
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 1, "pair": "09bce271e8530fbc", "unit": "dd33291cc494bbc77d44e9b13d16f171d32c9746637506f315f375e8a9ddee95"}
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 1, "pair": "a856cd13668353a6", "unit": "6149fb0132b981620a90a23a9af8d4656d0d827403e63b48258443cacd0a9c81"}
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 0, "pair": "a856cd13668353a6", "unit": "b5aecdd6a87f8d57d28f54a013b1f96332460fe1bb44bff4ed28e166f18f9abe"}
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 1, "pair": "40d8d81a764c508f", "unit": "7deeed73c29c74ccbed29be049e109d363d41e4b5eea245ee57780b85f0e137a"}
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 0, "pair": "40d8d81a764c508f", "unit": "b5cdadf848e67910550666ba2a6ac67b24112e501c02dba1865766a316317a23"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "", "unit": "15dbea384b3255457a44d9382efc6944dd27f1d411d903344a5d5e5fd194b72f"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b2ac284bd70dd684", "unit": "225890ba6694dc8f4fbab46e9c851dff8c095e906cf14db15d01ebce0891aaab"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b6eca35de93a5be8", "unit": "42eb8427415704e060e73b5ce0943b014ddf380ca756d5ec47c558e59a79895a"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b6eca35de93a5be8", "unit": "4c9cdcc11a18641ef41e9308712c5e8b830ec9ad5ebc3a956c8701e4f77b39c0"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "6a53939838afbdd7", "unit": "5794809e595311c86909d673a570d003aa025e4dcd6abc5641b641d935beb0b1"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "46d3ec7d49cb8ca7", "unit": "5f7a63ec650915962ee5255f5416df63ab882d6107f3b540be36d481798a8944"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "6a53939838afbdd7", "unit": "97a7018535ab511622e3ade9e53d570ffb1c80633eaa372fb76e749b3a586c8f"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "46d3ec7d49cb8ca7", "unit": "a84e9ab47f4f2b584684dbb81575758d9032fe3f2ed53bef478cb9dcfedaf677"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b2ac284bd70dd684", "unit": "aeceb1ebc863ebbc3b69ae56c5b78d2dba005132b75cbd6af929bb0bfe25da9b"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "f42cab375ace1739", "unit": "e329d4a6ac757680bf3d292a81f1f74f8a8a9b482d1651efe9f503f6a4de8796"}
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "f42cab375ace1739", "unit": "e73c5aac41c0b93072223a3097fb175548ce893e0678ff3a22bc7571745a17e0"}
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 1, "pair": "1041a18cd1aec5f8", "unit": "c301b8c362aee3fe5ea38e18f858166bc4162ad8ace44c5fb2df192afc81da2a"}
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 0, "pair": "1041a18cd1aec5f8", "unit": "ca8e58ffcfd60bdc8626db8b9eb4f253c1b8b8affcb793c6d963b7a3ec91906b"}
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "", "unit": "52354ccdbc16f209e4692d97d5237e04ed030cf4a5e71f77b5a767962b355b4a"}
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "7d23be667536afff", "unit": "b5016a5a53409491ba805ed8718599b84250b7b6914da0434b0882b0b3578dc8"}
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 0, "pair": "7d23be667536afff", "unit": "f1b01102968568d3ba1b2fdc70f4ae281f433fb20d29eff2a55fc45555ddbfcd"}
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 0, "pair": "f21df949f8aca4fa", "unit": "15d23ab3aea4c878dd73bbae579f3ab473e8363477a97d28164ed5e1b94cb510"}
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 1, "pair": "f21df949f8aca4fa", "unit": "787bc98c651ed3db21be4c28e9c560c366ffd6ca508823d21e5d83eeaf1269db"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "1199de75f426940b", "unit": "10fdc236f43a02b7e291eb8bf6ada297a2ae98086a63b171057bc7bfc37a3377"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "22112c31603bbe9f", "unit": "5eb3b7523b5a196dcaacba08d68457f9a45e31503ea02745ca3a7d0542f9881c"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "22112c31603bbe9f", "unit": "6bd5bdfd81b0a3374d59cfb2c1f7fa2b27887f4672f4485e77eb5234b569983f"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "1199de75f426940b", "unit": "c09dffd04708540845c8d1dd77a2bd989bd35f264abf652a2db55f8cb1faf7a9"}
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 0, "pair": "7aab025ea7b97019", "unit": "7487a34cdfdb532a5096764a47a3e565fa509605c1df0e2b105eb9c751fc5307"}
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 1, "pair": "7aab025ea7b97019", "unit": "9c0be2a58741f095fd0a9cdc60d4ca4039de1529699d5701dc4d0bcf7d5bf90c"}
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 0, "pair": "4de5441f1776f083", "unit": "a3a35d573da308cd1d0614a26616bbd37b360d9ea416894cdeffdb59abd691d4"}
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 1, "pair": "4de5441f1776f083", "unit": "dc33fc26c1bf7c67e5439ddfdad06c0faf16c796e0fe8a46b6e6a541e708676d"}
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 0, "pair": "1972a9a06d7585ad", "unit": "1f472e842e894c9907536c6eb6c9c922bbb3b21c27999114588ec2a4345ab665"}
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 1, "pair": "1972a9a06d7585ad", "unit": "7e38e24f1cc4d287627b7cd6cb870888865bb87fa87bba1c4fbf84eabb213121"}
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 1, "pair": "d76d5232a8587127", "unit": "1ec32d427822d1ddd3b0d7245f5771ddeb56c146ac8ff9959dd85760ff659a20"}
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 0, "pair": "d76d5232a8587127", "unit": "5143b19a1a1e6b9162b80594ed7fb5c0c57ed0b8abfd793bfc71089552d57638"}
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 0, "pair": "5f6e814444b4acd8", "unit": "2e8aa07b1aa4bfc5349b0d4464f3f2a6500aa55ff765b0139e9901ed34151ffa"}
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 1, "pair": "5f6e814444b4acd8", "unit": "e26c726c08be761b22fcc6ec17fc1fa7aba4769f08957f024e5545906df62797"}
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 1, "pair": "888e955fb6cfb4fd", "unit": "6f1a431ca97afc34ceec48393eb134a8928a312e6f761eb907c6b5855cac6208"}
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 0, "pair": "888e955fb6cfb4fd", "unit": "7b62415d17569924d16238a4aa28ab1af2db6561f56be9f59d8075fd1bc4a941"}
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 1, "pair": "a5691395e641fd4c", "unit": "40339c123107f1231d6aa58de876cded1dfefd5b31dadc4fc86eedc6a687165f"}
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 0, "pair": "a5691395e641fd4c", "unit": "ea3828e2f59cc6f7b415a101532b566ecbd47ab123c3c7593c23c52f6773a2e3"}
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 1, "pair": "f1ebdacbd647e39d", "unit": "79628ef3d6c9e12a94922cdca548ff77d3dbdd6b595c1fec5958fcf23b609095"}
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 0, "pair": "f1ebdacbd647e39d", "unit": "f910e010c3ff2c8c17f0f80fbe89a934e4f792b601febbf318359e6dc3afc18e"}
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 0, "pair": "2dd5384f097711b5", "unit": "212fba6b5e50db045114af369a03ffbdf09ec898c10eeff3c45bfc01a5017b4c"}
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 1, "pair": "2dd5384f097711b5", "unit": "370ba3a1d0e354e882b7be92c1ebbf2e2903bdf29d442048ea3dc1f80922467f"}
{"family": "injection", "fold": "outer-4", "group": "strangerstudios/paid-memberships-pro", "label": 1, "pair": "", "unit": "ac905d12b92954263c046bd0138a682cea86cf8476336d6baa664ade28d34ac6"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "218a45d869bd7e39", "unit": "309ebac468b7ab054c8396ed495770385ae3590caba43709745ba2d1db4b6d54"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "", "unit": "3cc8613813dec13cb1b7b45b296688da883f0b36cd2c4fc1da6e81703fb59ab9"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "218a45d869bd7e39", "unit": "9bd2331dbc260451355307e4c37bae17ad6676654c66181934e1b35fb174cc5a"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "b087ac4d0989e0b72698a9644ef4eac8073b4a15bb21730d63e4730b5cc58a60"}
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "c5d1e8d19560c997018fdca12eb60f1799be76c33be058b88472f5a2fc6709be"}
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 0, "pair": "c863054be488a284", "unit": "425e1d9961b79bbc011c447cd050910dd796a32b248962a5b0acc3f5f19b7345"}
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 1, "pair": "c863054be488a284", "unit": "552925bba199119c7db5457b9d3c535bc9f6061cecfd2ab2f72cde690a536e1e"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "22befc17a75a688e2f9358d2ddf9a7b197da44cd95dbdeae5e0191b113ee54d3"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "63f496d9ff53e39b", "unit": "6a98f8145776c78793b37156e9a81757841bb6de80016ab4fa575209d724db3f"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 1, "pair": "63f496d9ff53e39b", "unit": "7ac60c96c15f8517e089cc2240aedcf1fc30e54a41e35fe03c6b965bc393fbd6"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "ae39cb55ef2a87c5347e0ddc40975f142450e82de4909e93ba62a7ed0efb4134"}
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 1, "pair": "d1b19898f8deaa41", "unit": "3763c7b407ad471820b5f8fa25bde0ba6d91e15c07c56681de4b41add418b0b1"}
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 0, "pair": "d1b19898f8deaa41", "unit": "bc8b934938c30687f3d4f982392d2cfa9504a916b463ccb5c46fe8a30eede488"}
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 1, "pair": "f5f33fa6fe7b8492", "unit": "938bbf146a9037e687190bcee82c047492dffbe90efea40a49a647f81a0f1ed0"}
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 0, "pair": "f5f33fa6fe7b8492", "unit": "b90c9f7e0011685f1a8e853a9ed0ae37fd437ef6d6eb0ba17c240aec2c22b603"}
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 0, "pair": "f74364dc8e35c8ba", "unit": "11e760e99c5c852676c9288e18a250febb3854e9047b51e8b9ecc9f3b423b6f5"}
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 1, "pair": "f74364dc8e35c8ba", "unit": "5fc980194040f4c4d803075d4085618cc0260ae8f3454b26b9b704d30200ffea"}
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 0, "pair": "db1c51833994ec55", "unit": "c2ce8320c778dd55434d7413844797b68e6fcabc4dc509458a9d60ca70b88c60"}
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 1, "pair": "db1c51833994ec55", "unit": "f2d6843547af1276ae8a4dff3a7786913fa79dc69540d4fbb95c4c091e27c7f2"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 1, "pair": "f4690679bf9d80aa", "unit": "1a4e24df8d08672ecde4709a77ea5c437b2198fecff8bfff373b63c9eaed6e63"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 0, "pair": "f4690679bf9d80aa", "unit": "78711973ec84867671314dd7878769092b32f59860eafed50a0f6979daee76fc"}
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 1, "pair": "d6a42c18151efb48", "unit": "01ae2279707361bfc06a005335d41b85066d0a163802ca81c031f5dbded5bb46"}
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 0, "pair": "d6a42c18151efb48", "unit": "a50dc15c313a9078badea01365987c66c5cee1ad48cea65c35c8bc2cbba25e8e"}
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 0, "pair": "85c15e3fc2b370b2", "unit": "0990d8f41cb07c1cf217812e51a7025381de4ba68d1be3799e26241d81dcea67"}
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 1, "pair": "85c15e3fc2b370b2", "unit": "9da1d63f0c4da270f66eb87bf9c342ace883a0aef78c4643e254159b5da8fce3"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 1, "pair": "c7f4f4ebebfae831", "unit": "b5cbc1490bca76679adfbe5e4d9f8067d27a226d595ba401b551b00c3ce4d45a"}
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 0, "pair": "c7f4f4ebebfae831", "unit": "ba2c1808eebdbe3f54c488b09bcb3d69e521916d1a3a0b5b14e02d5bc88bf016"}
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 1, "pair": "1507c8e00ed4b859", "unit": "620e9b5ed143c0bcec178db359fa601acb1d3a134b3e85c2e838b7dcf3892d85"}
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 0, "pair": "1507c8e00ed4b859", "unit": "6f5ce4008dac0471260310c60e1e2a9f76a7e4eec6ebd039835f0488cf3b19aa"}
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 0, "pair": "8f5a6abedd742bbe", "unit": "71e3ced643a244100cc3fde234f5b4b27655214c7ae0ff67ab61e3e48a8fdf78"}
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 1, "pair": "8f5a6abedd742bbe", "unit": "7c8cbfae87127042f9f43621345b4f7958bde27a3ff9db76254063e80735f3ce"}
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 1, "pair": "c9538342f2d7efca", "unit": "2b8db57d65ef1d67c457ff5a336e8a542d03387395d69d5e16815c4f00f40723"}
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 0, "pair": "c9538342f2d7efca", "unit": "e2ed113dc21ad5f7caa71ba4a5ece8f8f900a97fd0d00dc85f7ac79ff5bed19a"}
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 0, "pair": "494dd5d5e5cdf4b8", "unit": "3b8011f94ec721f68dd29afc6c8b01117c86a8b0c31f3650a6ce0a8d7c0739ff"}
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 1, "pair": "494dd5d5e5cdf4b8", "unit": "f7e2280574e9ae51daa64d8adb2955407ff6d1e156f0e2854b6c047e0a733c78"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 1, "pair": "51eae488182a123a", "unit": "20b039866ba796c34efa0e92468fb49a314ad7fe2914d9ff8b823e3e35ab26f9"}
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 0, "pair": "51eae488182a123a", "unit": "2ab3fef05def768ef7b4165bcedcf7d200282dba78a32e19b90a0df3aee7e3e5"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 1, "pair": "6f5a650865e08571", "unit": "e2051cc42e4f8b026c7807f966d492367310c90326e60fef1b0ee8c509c805a9"}
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 0, "pair": "6f5a650865e08571", "unit": "f5364d68ffe80b578e788f37f08784b0162826393b49b4a3affd70d7cc4c9e01"}
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 1, "pair": "8acd426c5771b37c", "unit": "983e146a60e061254b15dc8dd573cbba57f8dd0358f23ea1b9c313ef69069af5"}
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 0, "pair": "8acd426c5771b37c", "unit": "f80dfec45231e4778078cb10f00128ed445177cac0a062fc2df5ced5c0171f21"}
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 0, "pair": "9b84800ce5c395cb", "unit": "9b7dfdceae03fd86ad3260a03f85bcdcff8f32f10c265d23743ff8dc98eeefb8"}
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 1, "pair": "9b84800ce5c395cb", "unit": "b99dda150c0da3defa5f4c527ef6c402f5589dfb2f2f03f3e5e424910eb48c53"}
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 1, "pair": "ad64e8fac4ac3ee0", "unit": "36dfd01915a553d86576e8ecd3b737b8e2b297aa542f58cc83b448060e74398a"}
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 0, "pair": "ad64e8fac4ac3ee0", "unit": "c2b6d40f130d35bbfc16fab9411c664fa06321c59fb1d901a1987a69cb09c72e"}
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 0, "pair": "d48c08b19f560848", "unit": "03d02c9e25eec0f380b125279fce35e8867065bf6e1ec75f9f39729ce6efdc77"}
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 1, "pair": "d48c08b19f560848", "unit": "abbea69b7d0fa68c3ace442c25597e128cde354b417787b5913fd10d666c0d19"}
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 0, "pair": "bfc8188bbb673e3b", "unit": "a7cff0da55c0cc1d5c838e11fc6ce6c4b22c54b9f8e101688b218d6928eca504"}
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 1, "pair": "bfc8188bbb673e3b", "unit": "f741c56815230b9c30e2c8ef8952b895e9480e4780a89ff2cf6daf60d1fce7b6"}
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 0, "pair": "3844ed3d703d7ef4", "unit": "445efda074e4d9f6c1634225eb08a3c3eb1257311c995ce891df2c791a6f5c8b"}
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 1, "pair": "3844ed3d703d7ef4", "unit": "8ac27b08549353ada819caaf99a5180d2f2f515b03aa8cd99a5d37b527ea3a5c"}
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 1, "pair": "a63eae7d621a985e", "unit": "04ca0b583b09c3b4f67bd2496a4607ecb134d24b7bdaf8960fa151e40ebcc447"}
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 0, "pair": "a63eae7d621a985e", "unit": "ecfa2a67726ca9d3e2621a4d8be60145095b76aa18ecf0268212341d63752871"}
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 0, "pair": "0d9f529b7f0084b6", "unit": "35aa369236f510634b3aa7b54b04e9da0f80877e515786a888e760f8fe33c524"}
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 1, "pair": "0d9f529b7f0084b6", "unit": "ef4f8613c0914eb815504e00cd6ee05df38352fd3b27826798e6623594017042"}
@@ -0,0 +1,61 @@
# Experiment exp-005 (learned-decision-engine design section 6): in-function graph slice.
# Register before any paid fill or scoring; register pins the manifest and frozen units by sha256.
# A changed question is a new experiment. The six slice-2 incumbents are unchanged in A;
# exp-004's cache fill covered their prompts on this exact frozen population and memory.
# B changes only ProgramSpec.slice: same inputs profile, retrieval, k, temperatures, and examples.
# Slices are computed from the same numbered excerpt for candidates and all examples/demonstrations.
#
# Leak audit: benchmarks/measurements/2026-10-03-graph-slice-leak-audit/record.json.
# Of 501 units, 220 have a slice (59 flow, 161 no-flow); 281 receive the fixed not-analysed sentence.
# The audit found no leak. This is coverage, not evidence that the prompt improves ranking.
# The units file is a byte-for-byte copy of exp-004-pooled-block-gate.units.jsonl; do not regenerate it.
#
# Primary: within-pair AUC, vulnerable vs fixed side (ties 0.5), complete pairs only;
# paired B-A percentile 95% CI, resampling repository groups with replacement, both arms together.
# Secondary: pooled top-30 precision across all families/units (raw score descending, unit ID breaks ties;
# undefined below 30 scored units), unsure rate, and recorded response USD per candidate (including A's replay).
# One final look. ADOPT B iff the primary CI is wholly positive AND mean cost/candidate <= 1.5x A;
# REJECT iff the primary CI is wholly negative; otherwise INCONCLUSIVE, including unavailable cost.
# Fill B under a $6.00 hard ceiling, then run --replay-only: scoring spends nothing further.
# An incomplete fill must not be treated as a complete experiment; any resumption uses only the remaining budget.
id: exp-005-graph-slice
hypothesis: >
Adding the in-function def-use slice to the prompt improves within-pair ranking of the six slice-2
incumbent programs on held-out repositories, at no more than 1.5 times the cost per candidate.
kind: program
programs: # the slice-2 incumbents, as exp-003
access_control: 745cb1d76bbb09dc1574d333d2118489be62e39886517894d3699fa565b0e8c2
deserialization: 3c0c2831be7d78198a62351661c37867363c501cb40e3ed7b0119c155af09654
injection: b207a0710e0abf053b3b9397b25f9ee1cab4679144865e3a82c1b9be2c7b4982
output_encoding: 5d768f16724562c381c09949ff3df9cbb11d58cbb7a9c0515d0dc3f858d3818e
path: aea396680a4874dcb6de70e240ca0480c83424debdd3710bc3f1ea6a612661f3
untrusted_destination: 1242b3973fc89056aa08e00843043d68235df217d8e3ef8a0ea0e0b81c142753
arms:
A:
label: six slice-2 incumbents unchanged, replaying exp-004's response cache
spec: {}
replay_only: true
repeats: 1
budget_usd: 0.0
B:
label: the same six programs with the in-function slice added to their prompts
spec: {slice: true}
replay_only: false
repeats: 1
budget_usd: 6.0
unit: candidate
pairing: paired
cluster: repository
order: {randomise: per_case, seed: 5}
populations: {primary: paired, secondary: [pooled]}
metric:
primary: [within_pair_auc]
secondary: [top_30_precision, unsure_rate, usd_per_candidate]
decision: {arm: B, max_cost_ratio: 1.5}
test: {primary: cluster_bootstrap_paired_diff, alpha: 0.05, sides: 2, resamples: 10000}
mde: 0.0
stopping: {looks: [1.0], boundaries: [1.959964]}
budget_usd: {total: 6.0, per_arm: {A: 0.0, B: 6.0}}
families: [access_control, deserialization, injection, output_encoding, path, untrusted_destination]
folds: {kind: outer, k: 5, seed: 0, compile_fraction: 0.25}
units_file: plane/experiments/exp-005-graph-slice.units.jsonl
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: agree
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["agree"]
resources:
requests:
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: alerts
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["alerts"]
resources:
requests:
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: features
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["features"]
resources:
requests:
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: loop
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["loop"]
resources:
requests:
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
name: remember
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["remember"]
resources:
requests:
+1 -1
View File
@@ -8,7 +8,7 @@ metadata:
name: repo-facts
atespace: default
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["repo-facts"]
resources:
requests:
+1 -1
View File
@@ -11,7 +11,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["roles"]
resources:
requests:
+120 -120
View File
@@ -5,9 +5,9 @@ metadata:
name: repo-facts-budibase-mongo-template-nosqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}'
openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -36,7 +36,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -67,7 +67,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -96,7 +96,7 @@ metadata:
name: agree-budibase-mongo-template-nosqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -123,9 +123,9 @@ metadata:
name: repo-facts-cct-studio-server-cmdi
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}'
openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -154,7 +154,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -185,7 +185,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -214,7 +214,7 @@ metadata:
name: agree-cct-studio-server-cmdi
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -241,9 +241,9 @@ metadata:
name: repo-facts-contextforge-prompt-ssti
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}'
openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -272,7 +272,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -303,7 +303,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -332,7 +332,7 @@ metadata:
name: agree-contextforge-prompt-ssti
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -359,9 +359,9 @@ metadata:
name: repo-facts-froxlor-admin-ip-second-order-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}'
openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -390,7 +390,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -421,7 +421,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -450,7 +450,7 @@ metadata:
name: agree-froxlor-admin-ip-second-order-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -477,9 +477,9 @@ metadata:
name: repo-facts-fuxa-tdengine-escape-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}'
openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -508,7 +508,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -539,7 +539,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -568,7 +568,7 @@ metadata:
name: agree-fuxa-tdengine-escape-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -595,9 +595,9 @@ metadata:
name: repo-facts-lmdeploy-media-url-ssrf
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}'
openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -626,7 +626,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -657,7 +657,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -686,7 +686,7 @@ metadata:
name: agree-lmdeploy-media-url-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -713,9 +713,9 @@ metadata:
name: repo-facts-medplum-external-auth-open-redirect
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}'
openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -744,7 +744,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -775,7 +775,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -804,7 +804,7 @@ metadata:
name: agree-medplum-external-auth-open-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -831,9 +831,9 @@ metadata:
name: repo-facts-pgadmin-maintenance-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}'
openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -862,7 +862,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -893,7 +893,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -922,7 +922,7 @@ metadata:
name: agree-pgadmin-maintenance-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -949,9 +949,9 @@ metadata:
name: repo-facts-qwed-sympy-eval
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}'
openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -980,7 +980,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1011,7 +1011,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1040,7 +1040,7 @@ metadata:
name: agree-qwed-sympy-eval
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1067,9 +1067,9 @@ metadata:
name: repo-facts-reactpress-findall-column-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}'
openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1098,7 +1098,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1129,7 +1129,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1158,7 +1158,7 @@ metadata:
name: agree-reactpress-findall-column-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1185,9 +1185,9 @@ metadata:
name: repo-facts-unleash-addon-webhook-ssrf
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}'
openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1216,7 +1216,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1247,7 +1247,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1276,7 +1276,7 @@ metadata:
name: agree-unleash-addon-webhook-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1303,9 +1303,9 @@ metadata:
name: repo-facts-winml-cli-serve-cors
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}'
openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1334,7 +1334,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1365,7 +1365,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1394,7 +1394,7 @@ metadata:
name: agree-winml-cli-serve-cors
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1421,9 +1421,9 @@ metadata:
name: repo-facts-wp-directorist-avatar-ssrf
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}'
openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1452,7 +1452,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1483,7 +1483,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1512,7 +1512,7 @@ metadata:
name: agree-wp-directorist-avatar-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1539,9 +1539,9 @@ metadata:
name: repo-facts-wp-gopay-log-filter-sqli
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}'
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1570,7 +1570,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1601,7 +1601,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1630,7 +1630,7 @@ metadata:
name: agree-wp-gopay-log-filter-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1657,9 +1657,9 @@ metadata:
name: repo-facts-wp-groundhogg-confirm-redirect
atespace: default
annotations:
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}'
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}'
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- repo-facts
env:
@@ -1688,7 +1688,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1719,7 +1719,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1748,7 +1748,7 @@ metadata:
name: agree-wp-groundhogg-confirm-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- agree
env:
@@ -1777,7 +1777,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1808,7 +1808,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1839,7 +1839,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1870,7 +1870,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1901,7 +1901,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1932,7 +1932,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1963,7 +1963,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -1994,7 +1994,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2025,7 +2025,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2056,7 +2056,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2087,7 +2087,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2118,7 +2118,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2149,7 +2149,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2180,7 +2180,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2211,7 +2211,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- verify
env:
@@ -2240,7 +2240,7 @@ metadata:
name: features-budibase-mongo-template-nosqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2263,7 +2263,7 @@ metadata:
name: remember-budibase-mongo-template-nosqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2288,7 +2288,7 @@ metadata:
name: features-cct-studio-server-cmdi
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2311,7 +2311,7 @@ metadata:
name: remember-cct-studio-server-cmdi
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2336,7 +2336,7 @@ metadata:
name: features-contextforge-prompt-ssti
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2359,7 +2359,7 @@ metadata:
name: remember-contextforge-prompt-ssti
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2384,7 +2384,7 @@ metadata:
name: features-froxlor-admin-ip-second-order-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2407,7 +2407,7 @@ metadata:
name: remember-froxlor-admin-ip-second-order-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2432,7 +2432,7 @@ metadata:
name: features-fuxa-tdengine-escape-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2455,7 +2455,7 @@ metadata:
name: remember-fuxa-tdengine-escape-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2480,7 +2480,7 @@ metadata:
name: features-lmdeploy-media-url-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2503,7 +2503,7 @@ metadata:
name: remember-lmdeploy-media-url-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2528,7 +2528,7 @@ metadata:
name: features-medplum-external-auth-open-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2551,7 +2551,7 @@ metadata:
name: remember-medplum-external-auth-open-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2576,7 +2576,7 @@ metadata:
name: features-pgadmin-maintenance-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2599,7 +2599,7 @@ metadata:
name: remember-pgadmin-maintenance-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2624,7 +2624,7 @@ metadata:
name: features-qwed-sympy-eval
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2647,7 +2647,7 @@ metadata:
name: remember-qwed-sympy-eval
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2672,7 +2672,7 @@ metadata:
name: features-reactpress-findall-column-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2695,7 +2695,7 @@ metadata:
name: remember-reactpress-findall-column-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2720,7 +2720,7 @@ metadata:
name: features-unleash-addon-webhook-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2743,7 +2743,7 @@ metadata:
name: remember-unleash-addon-webhook-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2768,7 +2768,7 @@ metadata:
name: features-winml-cli-serve-cors
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2791,7 +2791,7 @@ metadata:
name: remember-winml-cli-serve-cors
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2816,7 +2816,7 @@ metadata:
name: features-wp-directorist-avatar-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2839,7 +2839,7 @@ metadata:
name: remember-wp-directorist-avatar-ssrf
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2864,7 +2864,7 @@ metadata:
name: features-wp-gopay-log-filter-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2887,7 +2887,7 @@ metadata:
name: remember-wp-gopay-log-filter-sqli
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
@@ -2912,7 +2912,7 @@ metadata:
name: features-wp-groundhogg-confirm-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- features
env:
@@ -2935,7 +2935,7 @@ metadata:
name: remember-wp-groundhogg-confirm-redirect
atespace: default
spec:
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
command:
- remember
env:
+1 -1
View File
@@ -10,7 +10,7 @@ metadata:
annotations:
openultrasast.io/model: deepseek-flash
spec:
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
command: ["verify"]
resources:
requests:
+272 -95
View File
@@ -37,7 +37,7 @@ from .calibration import (
from .complexity import map as complexity_map
from .complexity.ledger import persist_verdicts
from .config import ModelLayerConfig, ObligationsConfig, ResolvedConfig, RetiredConfigError, load_config, load_dotenv
from .findings import StaticFinding, quick_scan_findings, write_findings
from .findings import StaticFinding, quick_scan_findings, uncovered_languages, write_findings
from .fusion import FusionDecision, fuse_findings
from .gate import FALSE_POSITIVE_CEILING, RECALL_FLOOR
from .harness import HarnessRuntime, HarnessTraceWriter, write_harness_config
@@ -103,6 +103,7 @@ class ScanOutcome:
finding_count: int
calibrations_applied: int
exit_code: int
notices: tuple[str, ...] = ()
if TYPE_CHECKING:
@@ -131,28 +132,84 @@ def main(argv: list[str] | None = None) -> int:
def _main(argv: list[str] | None) -> int:
if (argv if argv is not None else sys.argv[1:])[:1] != ["pre-push"]:
given = list(argv if argv is not None else sys.argv[1:])
if given[:1] == ["pre-push"] and given[1:2] in (["install"], ["uninstall"]):
return _hook_install(given[1], given[2:])
if given[:1] != ["pre-push"]:
load_dotenv()
parser = argparse.ArgumentParser(prog="ousast")
subparsers = parser.add_subparsers(dest="command", required=True)
replay_parser = subparsers.add_parser("pre-push", help="check Git push input or replay local revisions (experimental)")
replay_parser.add_argument("path", type=Path)
replay_parser.add_argument("--base")
replay_parser.add_argument("--head")
replay_parser.add_argument("--remote", nargs=2, metavar=("NAME", "URL"))
replay_parser.add_argument("--comparison-base")
replay_parser = subparsers.add_parser(
"pre-push",
help="check Git push input or replay local revisions (experimental, advisory)",
description="Check what a push changes: quick rules on the changed lines, then the Joern engine on head and base "
"within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with "
"'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall [REPO]'.",
)
replay_parser.add_argument("path", type=Path, help="the repository to check (the hook passes '.')")
replay_parser.add_argument("--base", help="explicit replay: the base revision (with --head; instead of --remote)")
replay_parser.add_argument("--head", help="explicit replay: the head revision to check against --base")
replay_parser.add_argument(
"--remote",
nargs=2,
metavar=("NAME", "URL"),
help="Git hook mode: the remote name and URL Git passes to pre-push; the push input is read from stdin",
)
replay_parser.add_argument(
"--comparison-base",
help="hook mode: a local ref to compare a NEW branch against (the remote has no base for it); "
"the hook reads OUSAST_COMPARISON_BASE",
)
replay_parser.add_argument("--model-config", type=Path, help="explicit optional witness selection endpoint configuration")
replay_parser.add_argument("--prior-hook", type=Path, help="explicitly chain an existing hook with the same input and arguments")
replay_parser.add_argument("--artifact", type=Path, required=True)
replay_parser.add_argument(
"--artifact", type=Path, required=True, help="where to write the complete JSON result; must be outside the repository"
)
replay_parser.add_argument(
"--cache-dir", type=Path, help="reuse compatible local artifacts in a private directory outside the repository"
)
replay_parser.add_argument("--deadline", type=float, default=30.0)
replay_parser.add_argument("--cancellation-allowance", type=float, default=2.0)
replay_parser.add_argument("--max-regions", type=int, default=500)
replay_parser.add_argument("--mode", choices=("advisory", "blocking"), default="advisory")
replay_parser.add_argument("--incomplete-coverage", choices=("allow", "block"), default="allow")
replay_parser.add_argument(
"--deadline",
type=float,
default=30.0,
help="seconds for the whole check: resolution, quick rules, engine and reporting (default 30; hook: OUSAST_PUSH_DEADLINE)",
)
replay_parser.add_argument(
"--cancellation-allowance",
type=float,
default=2.0,
help="seconds after the deadline to stop work and save the result (default 2)",
)
replay_parser.add_argument(
"--max-regions", type=int, default=500, help="most code regions the engine examines, highest-ranked first (default 500)"
)
replay_parser.add_argument(
"--mode",
choices=("advisory", "blocking"),
default="advisory",
help="advisory (default) never blocks; blocking is the explicit opt-in that rejects a push for an admitted "
"alert (hook: OUSAST_PUSH_MODE)",
)
replay_parser.add_argument(
"--incomplete-coverage",
choices=("allow", "block"),
default="allow",
help="with --mode blocking only: also block when coverage is incomplete (hook: OUSAST_INCOMPLETE_COVERAGE)",
)
replay_parser.add_argument(
"--engine",
choices=("inline", "background", "off"),
default="inline",
help="inline (default): run the Joern engine inside the deadline; background: return after the quick rules and "
"run the engine detached, showing its result on the next run; off: quick rules only",
)
replay_parser.add_argument(
"--background-deadline",
type=float,
default=900.0,
help="seconds the detached engine run of --engine background may take (default 900)",
)
replay_parser.add_argument(
"--experimental-declarations",
type=Path,
@@ -266,16 +323,21 @@ def _main(argv: list[str] | None) -> int:
learn_labels.add_argument("--include-title", action="store_true", help="the title-tier pairs as well (a sensitivity arm)")
learn_labels.add_argument("--no-assumed-benign", action="store_true", help="skip mining the assumed-benign history source")
add_learn_commands(learn_sub)
plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on this host's kind cluster")
plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on the profile's cluster (ops/k8s/profiles/)")
profile_help = "the PlaneProfile: a name under ops/k8s/profiles/ or a .toml path (default OUSAST_PLANE_PROFILE, else kind)"
plane_sub = plane.add_subparsers(dest="plane_command", required=True)
plane_run = plane_sub.add_parser("run", help="execute a Run manifest on ax; a rerun skips done tasks")
plane_run.add_argument("run_manifest", type=Path)
plane_run.add_argument("--workers", type=int, default=1, help="tasks in flight at once (serialize labels never overlap)")
plane_run.add_argument("--ax", default="ax", help="path of the ax CLI")
plane_run.add_argument("--profile", help=profile_help)
plane_status = plane_sub.add_parser("status", help="per-task status and the token attribution table of a run")
plane_status.add_argument("run")
plane_status.add_argument("--units", action="store_true", help="per-unit rows from each task's units.jsonl")
plane_sub.add_parser("doctor", help="check kind, Agent Substrate, the ax controller and the runner image")
plane_doctor = plane_sub.add_parser(
"doctor", help="check the profile's cluster, Agent Substrate, the ax controller, the runner image and the memory store"
)
plane_doctor.add_argument("--profile", help=profile_help)
plane_remember = plane_sub.add_parser("remember", help="ingest a run's memory rows into the store (OUSAST_MEMORY)")
plane_remember.add_argument("run")
plane_remember.add_argument("--plane", type=Path, default=Path("plane"), help="where runs/<run>.yaml names population and split")
@@ -313,9 +375,11 @@ def _main(argv: list[str] | None) -> int:
)
plane_harvest.add_argument("--ceiling", type=float, default=10.0, help="USD: each Run's task budgets sum to at most this")
plane_harvest.add_argument("--runner-image", type=Path, help="re-pin the task templates to the image in this file first (ops/ax/up.sh)")
plane_harvest.add_argument("--profile", help=profile_help)
plane_ws = plane_sub.add_parser("workspaces", help="one Workspace manifest per case pin of a population file")
plane_ws.add_argument("population", type=Path)
plane_ws.add_argument("--out", type=Path, default=Path("plane/workspaces"))
plane_ws.add_argument("--profile", help=profile_help)
plane_ws.add_argument("--validation-set", type=Path, help="JSON {case: [[path, function], ...]}: emit that set's increment instead")
plane_ws.add_argument("--candidates", type=Path, help="with --validation-set: per-case candidate records (independent-v2-modelsinks)")
plane_ws.add_argument("--triage", type=Path, help="with --validation-set: recorded batched-check scans (<case>--vulnerable_a.jsonl)")
@@ -339,6 +403,20 @@ def _main(argv: list[str] | None) -> int:
"--qualify-population", action="append", default=[], metavar="NAME", help="with --loop: a population whose rows never propose"
)
raw = list(argv if argv is not None else sys.argv[1:])
if _advisory_hook(raw):
# Git hook mode (stdin) without an explicit blocking opt-in: a check that cannot run must
# never reject the push. Say so in one line and allow it (Requirement 9.1).
try:
return _hook_main(parser, argv)
except SystemExit as stop:
if stop.code in (0, None):
raise
print("ousast: pre-push check did not run: invalid arguments or settings (message above). Push continues.")
return 0
except Exception as error: # noqa: BLE001 -- an advisory hook reports a crash, it never blocks on one
print(f"ousast: pre-push check did not run: unexpected {type(error).__name__}. Push continues.")
return 0
args = parser.parse_args(argv)
# Diagnosis needs the stage costs, and nothing configures logging, so the default root level of
# WARNING silently dropped every informational line. A run that reports nothing is
@@ -347,77 +425,7 @@ def _main(argv: list[str] | None) -> int:
_configure_logging()
start_profiling() # only when OUSAST_SAMPLE_PROFILE names a file
if args.command == "pre-push":
from .config import PushConfig
from .push.runner import push, replay
try:
if bool(args.base) != bool(args.head) or bool(args.base) == bool(args.remote):
parser.error("provide --base and --head together, or --remote NAME URL for Git stdin")
settings = PushConfig(
comparison_base=args.comparison_base,
deadline_seconds=args.deadline,
cancellation_allowance_seconds=args.cancellation_allowance,
mode=args.mode,
incomplete_coverage_policy=args.incomplete_coverage,
)
options: dict[str, Any] = dict(
artifact=args.artifact,
config=settings,
max_regions=args.max_regions,
cache_dir=args.cache_dir,
model_config=args.model_config,
)
if args.base and args.prior_hook:
parser.error("--prior-hook requires Git stdin mode")
if args.experimental_record_vetoes and not args.base:
parser.error("--experimental-record-vetoes requires explicit --base/--head replay; it is never a hook capability")
if args.experimental_declarations and not args.experimental_record_vetoes:
parser.error("--experimental-declarations requires --experimental-record-vetoes")
prior_data = None
if args.base:
delivery = replay(
args.path,
base=args.base,
head=args.head,
record_vetoes=args.experimental_record_vetoes,
declarations=args.experimental_declarations,
**options,
)
else:
# multiprocessing closes sys.stdin in its child; duplicate the Git pipe first.
with os.fdopen(os.dup(0), "rb") as stream:
if args.prior_hook:
from .model.contracts import ExecutionBudget
from .push.runner import _prepare
shared = ExecutionBudget(time.monotonic() + settings.deadline_seconds, settings.cancellation_allowance_seconds)
try:
prior_data = _prepare(lambda: stream.read(1024 * 1024 + 1), shared)
if len(prior_data) > 1024 * 1024:
raise ValueError("input exceeds hook integration limit")
except (ValueError, RuntimeError, TimeoutError):
print("Hook input could not be retained for all consumers. Retry with complete input and sufficient time.")
return 1
options["execution_budget"] = shared
def updates() -> str:
return (prior_data if prior_data is not None else stream.read(1024 * 1024 + 1)).decode("utf-8")
delivery = push(args.path, updates=updates, remote_name=args.remote[0], remote_url=args.remote[1], **options)
except ValueError as error:
parser.error(str(error))
print(delivery.text, end="", flush=True)
if args.prior_hook and prior_data is not None:
# This is the explicitly chained hook's own behavior, outside analysis.
# Never replace its rejection with our advisory success or impose our timeout on it.
try:
prior = subprocess.run([str(args.prior_hook.resolve()), *args.remote], input=prior_data, cwd=args.path)
except OSError:
print("Existing hook could not be started. Restore its executable path before retrying.")
return 1
if prior.returncode:
return prior.returncode if prior.returncode > 0 else 128 - prior.returncode
return delivery.exit_code
return _pre_push(args, parser)
if args.command == "scan":
return _scan(args.path, args.config, args.mode, args.fail_on)
if args.command == "benchmark":
@@ -463,6 +471,145 @@ def _main(argv: list[str] | None) -> int:
return 2
def _hook_install(action: str, argv: list[str]) -> int:
"""`ousast pre-push install|uninstall [REPO]` from the hook shipped in the package (task 17.6)."""
from .push.install import install, uninstall
parser = argparse.ArgumentParser(
prog=f"ousast pre-push {action}",
description="Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory "
"(core.hooksPath is honoured, never edited)."
if action == "install"
else "Remove the OpenUltraSAST pre-push hook and restore a chained previous hook.",
)
parser.add_argument("repository", nargs="?", type=Path, default=Path("."), help="the repository (default: current directory)")
parser.add_argument(
"--force",
action="store_true",
help="install: replace our hook, or move a foreign hook to pre-push.before-ousast and chain it; "
"uninstall: remove a hook this tool did not write",
)
if action == "install":
parser.add_argument(
"--docker",
action="store_true",
help="also install the ousast-docker wrapper beside the hook, so the check runs in the container image",
)
args = parser.parse_args(argv)
try:
code, lines = (
install(args.repository, force=args.force, docker=args.docker)
if action == "install"
else uninstall(args.repository, force=args.force)
)
except (OSError, ValueError, subprocess.SubprocessError) as error:
print(f"ousast pre-push {action}: {error}", file=sys.stderr)
return 1
for line in lines:
print(line, file=sys.stderr if code else sys.stdout)
return code
def _advisory_hook(argv: list[str]) -> bool:
"""A Git hook invocation (`--remote NAME URL`, stdin input) that did not opt into blocking."""
if argv[:1] != ["pre-push"] or "--remote" not in argv:
return False
blocking = any(a == "--mode=blocking" for a in argv) or any(
a == "--mode" and i + 1 < len(argv) and argv[i + 1] == "blocking" for i, a in enumerate(argv)
)
return not blocking
def _hook_main(parser: argparse.ArgumentParser, argv: list[str] | None) -> int:
args = parser.parse_args(argv)
_configure_logging()
return _pre_push(args, parser)
def _pre_push(args: argparse.Namespace, parser: argparse.ArgumentParser) -> int:
from .config import PushConfig
from .push.runner import push, replay
try:
if bool(args.base) != bool(args.head) or bool(args.base) == bool(args.remote):
parser.error("provide --base and --head together, or --remote NAME URL for Git stdin")
settings = PushConfig(
comparison_base=args.comparison_base,
deadline_seconds=args.deadline,
cancellation_allowance_seconds=args.cancellation_allowance,
mode=args.mode,
incomplete_coverage_policy=args.incomplete_coverage,
)
options: dict[str, Any] = dict(
artifact=args.artifact,
config=settings,
max_regions=args.max_regions,
cache_dir=args.cache_dir,
model_config=args.model_config,
)
if args.base and args.prior_hook:
parser.error("--prior-hook requires Git stdin mode")
if args.experimental_record_vetoes and not args.base:
parser.error("--experimental-record-vetoes requires explicit --base/--head replay; it is never a hook capability")
if args.experimental_declarations and not args.experimental_record_vetoes:
parser.error("--experimental-declarations requires --experimental-record-vetoes")
prior_data = None
if args.base:
delivery = replay(
args.path,
base=args.base,
head=args.head,
record_vetoes=args.experimental_record_vetoes,
declarations=args.experimental_declarations,
engine=args.engine,
background_deadline=args.background_deadline,
**options,
)
else:
# multiprocessing closes sys.stdin in its child; duplicate the Git pipe first.
with os.fdopen(os.dup(0), "rb") as stream:
if args.prior_hook:
from .model.contracts import ExecutionBudget
from .push.runner import _prepare
shared = ExecutionBudget(time.monotonic() + settings.deadline_seconds, settings.cancellation_allowance_seconds)
try:
prior_data = _prepare(lambda: stream.read(1024 * 1024 + 1), shared)
if len(prior_data) > 1024 * 1024:
raise ValueError("input exceeds hook integration limit")
except (ValueError, RuntimeError, TimeoutError):
print("Hook input could not be retained for all consumers. Retry with complete input and sufficient time.")
return 1
options["execution_budget"] = shared
def updates() -> str:
return (prior_data if prior_data is not None else stream.read(1024 * 1024 + 1)).decode("utf-8")
delivery = push(
args.path,
updates=updates,
remote_name=args.remote[0],
remote_url=args.remote[1],
engine=args.engine,
background_deadline=args.background_deadline,
**options,
)
except ValueError as error:
parser.error(str(error))
print(delivery.text, end="", flush=True)
if args.prior_hook and prior_data is not None:
# This is the explicitly chained hook's own behavior, outside analysis.
# Never replace its rejection with our advisory success or impose our timeout on it.
try:
prior = subprocess.run([str(args.prior_hook.resolve()), *args.remote], input=prior_data, cwd=args.path)
except OSError:
print("Existing hook could not be started. Restore its executable path before retrying.")
return 1
if prior.returncode:
return prior.returncode if prior.returncode > 0 else 128 - prior.returncode
return delivery.exit_code
def _scan(path: Path, config_path: Path, mode: str, fail_on: str) -> int:
outcome = _run_scan(path, config_path, mode, fail_on)
_print_scan_outcome(outcome)
@@ -495,6 +642,17 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
"preprocess",
lambda: preprocess_repository(run.target, run.root / "preprocess" / "file_targets.json", static_hints),
)
# A language nothing here covers is named, never a silent zero (pre-push 17.2, Requirement 9.2).
for language, count in uncovered_languages(targets, ruleset, engine=mode != "quick"):
runtime.state["degradations"].append(
{
"stage": "quick_findings",
"reason": "language_not_covered",
"language": language,
"count": count,
"engine": "" if mode == "quick" else " and no engine frontend",
}
)
entry_points = runtime.run_stage("entry_point_mapping", lambda: analyze_entry_points(run.target, targets))
write_entry_points(entry_points, run.root / "mapping" / "entry_points.json")
targets = attach_reachability_hints(targets, entry_points)
@@ -793,6 +951,11 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
finding_count=len(findings),
calibrations_applied=len(applied_calibrations),
exit_code=scan_exit_code(findings, verifications, fail_on, worth_fixing_verdicts=verdict_records),
notices=tuple(
f"language not covered: {item['language']} ({item['count']} files)"
for item in runtime.state["degradations"]
if item.get("reason") == "language_not_covered"
),
)
@@ -867,6 +1030,8 @@ def _print_scan_outcome(outcome: ScanOutcome) -> None:
print(f"ranked_targets={outcome.ranked_target_count}")
print(f"findings={outcome.finding_count}")
print(f"calibrations_applied={outcome.calibrations_applied}")
for notice in outcome.notices:
print(f"notice: {notice}")
def _load_static_hints(sarif_paths: tuple[str, ...]) -> list[object]:
@@ -1371,10 +1536,22 @@ def _learn_labels(args: argparse.Namespace) -> int:
def _plane(args: argparse.Namespace) -> int:
from .plane import reconciler # lazy: the plane is a maintainer surface, not the scan path
from .plane.profile import ProfileError, load_profile
try:
profile = load_profile(getattr(args, "profile", None))
except ProfileError as exc:
print(f"plane: {exc}", file=sys.stderr)
return 2
if args.plane_command == "run":
from .plane.memory import MemoryStoreError
_plane_memory("seed", args.run_manifest)
result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax)
try:
result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax, profile=profile)
except MemoryStoreError as exc: # the store is the state of record: no fallback, the Run is not submitted
print(f"memory store {profile.memory}: {exc}", file=sys.stderr)
return 1
print(f"run finished: {result}")
_plane_memory("ingest", args.run_manifest)
return 0 if result == "done" else 1
@@ -1389,14 +1566,14 @@ def _plane(args: argparse.Namespace) -> int:
if args.plane_command == "alerts-engine":
return _plane_alerts_engine(args)
if args.plane_command == "harvest":
return _plane_harvest(args)
return _plane_harvest(args, profile.atespace)
if args.plane_command == "doctor":
checks = reconciler.doctor()
checks = reconciler.doctor(profile)
for name, ok, text in checks:
print(f"[{'ok' if ok else 'FAIL'}] {name}: {text}")
return 0 if all(ok for _, ok, _ in checks) else 1
if args.validation_set is not None:
return _plane_increment(args)
return _plane_increment(args, profile.atespace)
from .plane.workspaces import workspaces
written = workspaces(args.population, args.out)
@@ -1404,7 +1581,7 @@ def _plane(args: argparse.Namespace) -> int:
return 0
def _plane_harvest(args: argparse.Namespace) -> int:
def _plane_harvest(args: argparse.Namespace, atespace: str) -> int:
from .plane.generate import repin_templates
from .plane.harvest import write_harvest
@@ -1413,7 +1590,7 @@ def _plane_harvest(args: argparse.Namespace) -> int:
command = f"ousast plane harvest --labels {args.labels.name} --name {args.name} --ceiling {args.ceiling:g}"
files = write_harvest(
args.labels, args.plane, name=args.name, command=command, catalog=args.catalog, cache=args.cache, templates=args.templates,
ceiling=args.ceiling,
ceiling=args.ceiling, atespace=atespace,
) # fmt: skip
for relative in sorted(k for k in files if k.startswith("runs/")):
print(f"{args.plane / relative}: {files[relative].splitlines()[1].lstrip('# ')}")
@@ -1494,7 +1671,7 @@ def _plane_memory(
return 0
def _plane_increment(args: argparse.Namespace) -> int:
def _plane_increment(args: argparse.Namespace, atespace: str) -> int:
from .plane.generate import LOOP_MANIFEST, Loop, increment
if args.candidates is None or args.triage is None:
@@ -1520,7 +1697,7 @@ def _plane_increment(args: argparse.Namespace) -> int:
command += "".join(f" --qualify-population {p}" for p in loop.populations)
written = increment(
args.population, args.validation_set, args.candidates, args.triage, plane=args.plane, run_name=args.run, command=command,
repos=args.repos, runner_image=args.runner_image, loop=loop,
repos=args.repos, runner_image=args.runner_image, loop=loop, atespace=atespace,
) # fmt: skip
print(f"{len(written)} manifests written under {args.plane} (Run {args.run})")
return 0
+3 -1
View File
@@ -162,7 +162,9 @@ class EngineSession:
return None
payload = json.dumps({"query": code}).encode()
call = urllib.request.Request(
f"http://127.0.0.1:{self._port}/query-sync", data=payload, headers={"Content-Type": "application/json"}
f"http://127.0.0.1:{self._port}/query-sync", # loopback: the Joern server this process started
data=payload,
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(call, timeout=timeout) as response: # noqa: S310 -- fixed loopback URL
+25
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import json
import re
from collections.abc import Sequence
from dataclasses import asdict, dataclass
from pathlib import Path
@@ -74,6 +75,30 @@ def quick_scan_findings(
return sorted(findings, key=lambda item: (_severity_sort(item.severity), -item.ranking_priority, item.path))
def uncovered_languages(
targets: Sequence[FileTarget], ruleset: Sequence[PatternRule] | None = None, *, engine: bool = False
) -> list[tuple[str, int]]:
"""Languages present in `targets` that no enabled quick rule (and, with `engine`, no engine frontend) covers.
A repository in such a language produces zero findings whatever it contains; callers name it so that
zero is never read as a clean result.
"""
from .model.partitions import FRONTENDS
rules = PATTERN_RULES if ruleset is None else ruleset
enabled = [rule for rule in rules if rule.status != "disabled"]
if any(not rule.languages for rule in enabled): # a rule without a language list applies to every file
return []
covered = {language for rule in enabled for language in rule.languages}
if engine:
covered |= set(FRONTENDS)
counts: dict[str, int] = {}
for target in targets:
if target.language not in covered and target.language != "unknown":
counts[target.language] = counts.get(target.language, 0) + 1
return sorted(counts.items())
def build_quick_hunter_prompt(target: FileTarget, source_excerpt: str) -> str:
return (
"Review this file for security-relevant issues. Return structured findings only when evidence exists.\n"
+60 -12
View File
@@ -60,18 +60,26 @@ def add_commands(learn_sub: Any) -> None:
("register", "record the manifest's and the units file's sha256 as the experiment row (both committed at HEAD)"),
("units", "freeze the units of a program experiment from the memory (no model): the families' evaluation candidates"),
("run", "run every arm on every unit, paired; arm A replays from the response cache when the manifest says so"),
("fill", "fill an experiment's response cache under a separate budget, or estimate with --dry-run"),
("analyse", "paired cluster bootstrap, exact McNemar, the two looks and the adoption verdict (counts and intervals only)"),
):
sub = experiment_sub.add_parser(name, help=text)
sub.add_argument("manifest", type=Path, help="plane/experiments/<id>.yaml")
sub.add_argument("--memory", help="the store (default OUSAST_MEMORY, then results_root()/memory)")
if name in ("units", "run"):
if name in ("units", "run", "fill"):
sub.add_argument("--profile", choices=("static", "plane"), default="static")
if name == "run":
if name in ("run", "fill"):
sub.add_argument(
"--manifest-model", type=Path, default=Path("plane/models/deepseek-flash.yaml"), help="the chat Model (prices)"
)
if name == "run":
sub.add_argument("--replay-only", action="store_true", help="replay every arm without resolving a model endpoint")
sub.add_argument("--no-early-stop", action="store_true", help="take no first look during the run")
if name == "fill":
sub.add_argument("--arm", action="append", help="arm to fill (repeatable; default: non-replay program arms)")
sub.add_argument("--budget-usd", type=float, help="required for paid fill; includes the smoke call")
sub.add_argument("--dry-run", action="store_true", help="inspect and estimate only; no calls or writes, including --out")
sub.add_argument("--out", type=Path, help="also write the fill report as JSON (ignored for --dry-run)")
if name in ("run", "analyse"):
sub.add_argument("--resamples", type=int, help="bootstrap resamples (default: the manifest's)")
sub.add_argument("--out", type=Path, help="write the counts-only report here (analyse --record: benchmarks/experiments/<id>/)")
@@ -111,7 +119,7 @@ def _experiment(args: argparse.Namespace) -> int:
from ..plane.budget import MeteredClient
from ..plane.memory import open_store
from . import experiments as ex
from .program import Caller
from .program import Caller, ReplayMiss
store = open_store(args.memory)
try:
@@ -122,11 +130,17 @@ def _experiment(args: argparse.Namespace) -> int:
print(json.dumps({k: row[k] for k in shown}, indent=2))
return 0
if args.experiment_command == "analyse":
row = ex.check_registered(store, manifest, need_units=False)
row = ex.check_registered(store, manifest, need_units=manifest.kind == "rule")
provenance = {"code_commit": ex.committed_at_head(manifest.path)}
report = ex.analyse(manifest, ex.load_outcomes(store, manifest.id), resamples=args.resamples, provenance=provenance)
outcomes = ex.load_outcomes(store, manifest.id)
if manifest.kind == "rule":
units = ex.read_units(manifest.units_file)
report = ex.analyse_rule(manifest, outcomes, units, resamples=args.resamples, provenance=provenance)
else:
report = ex.analyse(manifest, outcomes, resamples=args.resamples, provenance=provenance)
if args.record:
store.put_row(ex.result_row(manifest, str(row["pin"]), report))
result = ex.rule_result_row if manifest.kind == "rule" else ex.result_row
store.put_row(result(manifest, str(row["pin"]), report))
_write(report, args.out or (ex.RESULT_DIR / manifest.id / "result.json" if args.record else None))
return 0
from .embeddings import EmbeddingCache
@@ -143,9 +157,47 @@ def _experiment(args: argparse.Namespace) -> int:
cache = EmbeddingCache(store)
vectors = {sha: v for sha in cache.names() if (v := cache.get(sha)) is not None} or None
parameters = _model_parameters(args.manifest_model)
def excerpt_text(sha: str) -> str | None:
data = store.get_blob("excerpts", sha)
return data.decode("utf-8") if data is not None else None
if args.experiment_command == "fill":
from ..tool_hunter import ChatClient
from .fill import fill_experiment
def resolve_fill_client() -> ChatClient:
from ..config import load_config
from ..model.endpoint import resolve_chat_endpoint
resolved = resolve_chat_endpoint(load_config(None))
if resolved is None:
raise ex.ExperimentError("no chat endpoint configured; nothing was spent")
return resolved[0]
report = fill_experiment(
store,
manifest,
memory,
parameters,
excerpt_text,
vectors,
client_factory=resolve_fill_client,
budget_usd=args.budget_usd,
dry_run=args.dry_run,
arms=args.arm,
)
_write(report, None if args.dry_run else args.out)
return {"done": 0, "dry_run": 0, "unfinished": 3, "failed": 1}[report["status"]]
if manifest.kind == "rule": # zero cost: cache only, behind a zero-ceiling meter; no endpoint is resolved
caller, meter = ex.zero_cost_caller(_arm_model(manifest, manifest.arms["A"], store), parameters, store)
summary = ex.run_rule(store, manifest, memory, caller, meter, excerpt_text, vectors)
_write(summary.as_dict(), args.out)
return 0
callers: dict[str, Caller] = {}
for name, arm in manifest.arms.items():
if arm.replay_only:
if arm.replay_only or args.replay_only:
callers[name] = Caller(None, _arm_model(manifest, arm, store), parameters, store=store)
continue
from ..config import load_config
@@ -157,13 +209,9 @@ def _experiment(args: argparse.Namespace) -> int:
client = MeteredClient(resolved[0], prices=parameters, budget_usd=arm.budget_usd)
callers[name] = Caller(client, _arm_model(manifest, arm, store), parameters, store=store)
def excerpt_text(sha: str) -> str | None:
data = store.get_blob("excerpts", sha)
return data.decode("utf-8") if data is not None else None
early = not args.no_early_stop
summary = ex.run(store, manifest, memory, callers, excerpt_text, vectors, stop_early=early, resamples=args.resamples)
except ex.ExperimentError as exc:
except (ex.ExperimentError, ReplayMiss) as exc:
print(f"learn experiment {args.experiment_command}: {exc}", file=sys.stderr)
return 2
_write(summary.as_dict(), args.out)
+180 -4
View File
@@ -25,7 +25,7 @@ from collections.abc import Callable, Iterable, Mapping, Sequence
from dataclasses import dataclass, replace
from typing import Any
from .calibrate import Row, check, cross_fit, offered
from .calibrate import Row, check, cross_fit, fit_map, offered
from .examples import Example
from .folds import Fold
from .labels import PER_FAMILY_FLOOR, POOLED_FLOOR
@@ -54,6 +54,7 @@ class Prediction:
parse_failed: int = 0
p: float | None = None
point: str | None = None # BLOCK | ADVISORY | DROP, from points chosen on the other folds
pair: str = "" # the two sides of one candidate share it; "" when only one side is present
@property
def blockable(self) -> bool:
@@ -128,6 +129,179 @@ def nested_points(preds: Sequence[Prediction]) -> tuple[list[Prediction], dict[s
return out, points
BOUNDS = ("wilson", "point")
def precision_threshold(
rows: Sequence[Prediction], *, precision: float = TARGET_PRECISION, bound: str = "wilson", min_flagged: int = 1
) -> float | None:
"""The lowest raw-score threshold ``t`` at which the blockable rows with ``s >= t`` reach ``precision``: their
Wilson 95% lower bound (``bound="wilson"``) or their point precision over at least ``min_flagged`` rows
(``bound="point"``). Candidate thresholds are the rows' own scores; ``None`` when no threshold qualifies."""
if bound not in BOUNDS:
raise ValueError(f"bound {bound!r} (one of {', '.join(BOUNDS)})")
blockable = [r for r in rows if r.blockable]
for t in sorted({r.s for r in blockable}):
chosen = [r for r in blockable if r.s >= t]
hits = sum(r.label for r in chosen)
if len(chosen) < max(1, min_flagged):
continue
level = wilson_lower(hits, len(chosen)) if bound == "wilson" else hits / len(chosen)
if level >= precision:
return t
return None
def nested_precision_block(
preds: Sequence[Prediction],
*,
eligible: Callable[[Prediction], bool] | None = None,
precision: float = TARGET_PRECISION,
bound: str = "wilson",
min_flagged: int = 1,
) -> tuple[list[Prediction], dict[str, dict[str, float | None]]]:
"""Selective blocking on the raw score, per family: the threshold for outer fold j is chosen on the *other* folds'
rows of the family only (and, with ``eligible``, only those rows); the held-out fold's labels never enter it.
Every row of fold j is then decided with that threshold: ``BLOCK`` when blockable and ``s >= t``, else ``DROP``.
A family whose training folds reach no threshold offers no BLOCK on that fold. Returns the decided rows and the
thresholds by family and fold."""
thresholds: dict[str, dict[str, float | None]] = {}
for family in sorted({r.family for r in preds}):
rows = [r for r in preds if r.family == family]
thresholds[family] = {
fold: precision_threshold(
[r for r in rows if r.fold != fold and (eligible is None or eligible(r))],
precision=precision, bound=bound, min_flagged=min_flagged,
)
for fold in sorted({r.fold for r in rows})
} # fmt: skip
out = []
for r in preds:
t = thresholds[r.family][r.fold]
out.append(replace(r, point="BLOCK" if t is not None and r.blockable and r.s >= t else "DROP"))
return out, thresholds
def calibrated_block(preds: Sequence[Prediction], *, seed: int = 0, **floors: int) -> tuple[list[Prediction], dict[str, Any]]:
"""Today's BLOCK rule, per family as the slice-2 evaluation applied it: the Platt map cross-fitted on the other
folds, the calibration check on the family's held-out calibrated predictions, the BLOCK point chosen on the other
folds' calibrated predictions (nested); a row is ``BLOCK`` only when the family's calibration holds. Returns the
decided rows and, per family, the check and the calibrated thresholds by fold."""
out: list[Prediction] = []
info: dict[str, Any] = {}
for family in sorted({r.family for r in preds}):
rows = [r for r in preds if r.family == family]
ps, _ = cross_fit([Row(r.fold, r.family, r.group, r.label, r.s) for r in rows], **floors)
calibrated = [replace(r, p=p) for r, p in zip(rows, ps, strict=True)]
decided = [r for r in calibrated if r.p is not None]
checked = (
check([float(r.p) for r in decided if r.p is not None], [r.label for r in decided], [r.group for r in decided], seed=seed)
if len({r.label for r in decided}) == 2
else None
)
holds = bool(checked and checked.holds)
nested, points = nested_points(calibrated)
out += [replace(r, point="BLOCK" if holds and r.point == "BLOCK" else "DROP") for r in nested]
info[family] = {
"calibration_holds": holds, "ece": None if checked is None else round(checked.ece, 4),
"thresholds_by_fold": {f: p.block for f, p in points.items()},
"reason_by_fold": {f: ("calibration" if not holds else p.block_reason) for f, p in points.items()},
} # fmt: skip
return out, info
# --- the pooled BLOCK gate (Req 6.4, amended 2026-10-03; exp-004) ---------------------------------------------------------
FAMILY_FLOOR = 10 # a family blocks only with at least this many held-out flags and none of them wrong
def normalised(train: Sequence[Prediction], rows: Sequence[Prediction], **floors: int) -> list[float | None]:
"""``rows``' raw scores on one scale across families: the per-family Platt map of :func:`.calibrate.fit_map`
(pooled slope with a per-family intercept between the floors) fitted on ``train`` only. ``None`` for a family the
training rows cannot map (below the pooled floor): it cannot block on that fold."""
cmap = fit_map([Row(r.fold, r.family, r.group, r.label, r.s) for r in train], **floors)
return [cmap.apply(r.family, r.s) for r in rows]
def pooled_threshold(rows: Sequence[Prediction], *, precision: float = TARGET_PRECISION) -> float | None:
"""The lowest normalised score ``t`` at which the blockable rows of every family together with ``p >= t`` have a
Wilson 95% lower bound of precision >= ``precision``; candidate thresholds are the rows' own ``p``."""
blockable = [r for r in rows if r.blockable and r.p is not None]
for t in sorted({float(r.p) for r in blockable if r.p is not None}):
chosen = [r for r in blockable if (r.p or 0.0) >= t]
if wilson_lower(sum(r.label for r in chosen), len(chosen)) >= precision:
return t
return None
def quiet_threshold(rows: Sequence[Prediction], *, recall: float = TARGET_RECALL) -> float | None:
"""The highest normalised score ``t`` that keeps at least ``recall`` of the rows' positives at or above it."""
positives = sorted((float(r.p) for r in rows if r.label == 1 and r.p is not None), reverse=True)
if not positives:
return None
return positives[math.ceil(recall * len(positives) - 1e-9) - 1]
def pooled_block_gate(
preds: Sequence[Prediction],
*,
eligible: Callable[[Prediction], bool] | None = None,
precision: float = TARGET_PRECISION,
recall: float = TARGET_RECALL,
**floors: int,
) -> tuple[list[Prediction], dict[str, dict[str, Any]]]:
"""One BLOCK threshold for every family, on a normalised score, chosen per outer fold j on the other folds only.
The training rows (``eligible`` rows of the other folds) are normalised by an inner cross-fit: each training fold
k by a map fitted on the training folds other than k, so ``t_block`` is chosen on out-of-sample scores. ``t_block``
is the lowest normalised score whose pooled Wilson 95% lower bound of precision is >= ``precision``; ``t_quiet``
per family is the highest one keeping >= ``recall`` of that family's training positives. Fold j's rows are then
normalised by the map fitted on all its training rows and decided: ``BLOCK`` when blockable and ``p >= t_block``;
``ADVISORY`` when ``p >= t_quiet`` (or the family has no map, or no ``t_quiet``: nothing is silenced blind);
``DROP`` (silent) below ``t_quiet``. Fold j's labels never enter its thresholds or its map."""
chosen = [r for r in preds if eligible is None or eligible(r)]
families = sorted({r.family for r in preds})
decided: dict[int, Prediction] = {}
gates: dict[str, dict[str, Any]] = {}
for fold in sorted({r.fold for r in preds}):
train = [r for r in chosen if r.fold != fold]
inner: list[Prediction] = []
for k in sorted({r.fold for r in train}):
mine = [r for r in train if r.fold == k]
inner += [replace(r, p=p) for r, p in zip(mine, normalised([r for r in train if r.fold != k], mine, **floors), strict=True)
if p is not None] # fmt: skip
t_block = pooled_threshold(inner, precision=precision)
t_quiet = {f: quiet_threshold([r for r in inner if r.family == f], recall=recall) for f in families}
held = [(i, r) for i, r in enumerate(preds) if r.fold == fold]
for (i, r), p in zip(held, normalised(train, [r for _, r in held], **floors), strict=True):
quiet = t_quiet[r.family]
if p is not None and t_block is not None and r.blockable and p >= t_block:
point = "BLOCK"
elif p is None or quiet is None or p >= quiet:
point = "ADVISORY"
else:
point = "DROP"
decided[i] = replace(r, p=p, point=point)
gates[fold] = {"t_block": t_block, "t_quiet": t_quiet, "training_rows": len(train), "normalised_training_rows": len(inner)}
return [decided[i] for i in range(len(preds))], gates
def family_floor(rows: Sequence[Prediction], *, floor: int = FAMILY_FLOOR) -> dict[str, dict[str, Any]]:
"""Per family, its held-out ``BLOCK`` flags among ``rows`` (the primary population, every fold): it may block only
when none is wrong and there are at least ``floor``; otherwise its flags are reported as ADVISORY."""
out: dict[str, dict[str, Any]] = {}
for family in sorted({r.family for r in rows}):
flagged = [r for r in rows if r.family == family and r.point == "BLOCK"]
errors = sum(1 for r in flagged if r.label == 0)
out[family] = {"flagged": len(flagged), "errors": errors, "floor": floor, "may_block": errors == 0 and len(flagged) >= floor}
return out
def apply_floor(rows: Sequence[Prediction], floors: Mapping[str, Mapping[str, Any]]) -> list[Prediction]:
"""``BLOCK`` becomes ``ADVISORY`` in every family that does not pass :func:`family_floor`."""
return [replace(r, point="ADVISORY") if r.point == "BLOCK" and not floors.get(r.family, {}).get("may_block") else r for r in rows]
def _at(rows: Sequence[Prediction], level: str) -> dict[str, float | None]:
flagged = [r for r in rows if r.point in (("BLOCK",) if level == "block" else ("BLOCK", "ADVISORY"))]
positives = sum(r.label for r in rows)
@@ -326,8 +500,10 @@ def candidate_from_example(
__all__ = [
"PARSE_FAILURE_LIMIT", "TARGET_PRECISION", "TARGET_RECALL", "Points", "Prediction", "candidate_from_example", "choose_points",
"cluster_bootstrap", "evaluate", "explain", "floor_status", "incompatible", "learning_curve", "nested_points", "predict", "strata",
"summary", "targets", "unevaluable_families",
"BOUNDS", "FAMILY_FLOOR", "PARSE_FAILURE_LIMIT", "TARGET_PRECISION", "TARGET_RECALL", "Points", "Prediction", "apply_floor",
"calibrated_block", "family_floor", "normalised", "pooled_block_gate", "pooled_threshold", "quiet_threshold",
"candidate_from_example", "choose_points", "cluster_bootstrap", "evaluate", "explain", "floor_status", "incompatible",
"learning_curve", "nested_points", "nested_precision_block", "precision_threshold", "predict", "strata", "summary", "targets",
"unevaluable_families",
"wilson_lower",
] # fmt: skip
+435 -14
View File
@@ -35,16 +35,30 @@ import math
import random
import subprocess
from collections.abc import Callable, Iterable, Mapping, Sequence
from contextlib import suppress
from dataclasses import asdict, dataclass, field, fields, replace
from pathlib import Path
from typing import Any
import yaml
from ..plane.budget import BudgetExhausted
from ..plane.budget import BudgetExhausted, MeteredClient
from ..plane.memory import MemoryStore, row_id
from .compile import load_program, spec_of
from .evaluate import candidate_from_example
from .evaluate import (
BOUNDS,
FAMILY_FLOOR,
TARGET_PRECISION,
TARGET_RECALL,
Prediction,
apply_floor,
calibrated_block,
candidate_from_example,
family_floor,
nested_precision_block,
pooled_block_gate,
wilson_lower,
)
from .examples import EXAMPLE_KIND, Example
from .folds import Fold, compile_split, folds_digest, outer_folds
from .program import Caller, Decision, Program, ProgramSpec
@@ -57,7 +71,20 @@ RESULT_KIND = "experiment_result"
REGISTER_TASK = "register"
POPULATION = "experiments" # the envelope of every experiment row; `split` is the experiment id
RUN_TASK = "run"
ARM_KINDS = ("program", "plane")
ARM_KINDS = ("program", "plane", "rule")
# decision-rule experiments (kind: rule): one replay-only scoring pass of the incumbent programs, arms differ only in
# the BLOCK rule applied to the same scores
RULES = ("calibrated_block", "precision_bound_block", "pooled_block_gate")
NORMALISATIONS = ("platt",) # pooled_block_gate: how the scores of different families are put on one scale (Req 6.4)
RULE_PARAMETERS: Mapping[str, set[str]] = {
"precision_bound_block": {"precision", "bound", "min_flagged", "select_on"},
"pooled_block_gate": {"precision", "recall", "floor", "normalise", "select_on"},
}
POPULATIONS = ("paired", "pooled") # paired: units whose candidate has both sides present; pooled: every unit
SCORE_ARM = "score" # the arm name of a rule experiment's outcome rows (the shared scores)
MISS_VERDICT = "replay_miss" # an outcome row of a unit excluded because a response was not cached
# rule metrics, per family and population of held-out (outer-fold) units: direction only (no paired test)
RULE_METRICS: Mapping[str, int] = {"flagged_precision_lb": 1, "flagged_precision": 1, "coverage": 1, "flagged": 1, "quiet_recall": 1}
UNITS = ("candidate", "file", "repository")
PAIRINGS = ("paired", "assigned")
# O'Brien-Fleming boundaries for two equally spaced looks at alpha 0.05, two-sided (design section 6).
@@ -70,6 +97,8 @@ ADVISORY_VERDICT = "vulnerable"
METRICS: Mapping[str, tuple[int, bool]] = {
"auc": (1, False),
"within_pair_auc": (1, False),
"top_30_precision": (1, False),
"unsure_rate": (-1, True),
"advisory_recall": (1, True),
"advisory_precision": (1, False),
"canary_agreement": (1, True),
@@ -96,6 +125,7 @@ class Arm:
replay_only: bool = False
repeats: int = 1
budget_usd: float = 0.0
rule: Mapping[str, Any] = field(default_factory=dict) # rule arms: {name: calibrated_block | precision_bound_block, ...}
@dataclass(frozen=True)
@@ -119,11 +149,11 @@ class Metric:
@property
def direction(self) -> int:
return METRICS[self.name][0]
return RULE_METRICS[self.name] if self.name in RULE_METRICS else METRICS[self.name][0]
@property
def binary(self) -> bool:
return METRICS[self.name][1]
return self.name in METRICS and METRICS[self.name][1]
def as_dict(self) -> dict[str, Any]:
return {k: v for k, v in asdict(self).items() if v is not None}
@@ -198,6 +228,7 @@ def load_manifest(path: Path) -> Manifest:
bool(spec.get("replay_only", False)),
int(spec.get("repeats", 1)),
float(spec.get("budget_usd", 0.0)),
dict(spec.get("rule") or {}),
) # fmt: skip
for name, spec in dict(raw["arms"]).items()
}
@@ -210,9 +241,12 @@ def load_manifest(path: Path) -> Manifest:
primary_raw = metric["primary"]
primary = tuple(Metric.parse(m) for m in (primary_raw if isinstance(primary_raw, list) else [primary_raw]))
secondary = tuple(Metric.parse(m) for m in metric.get("secondary", []))
known = RULE_METRICS if kind == "rule" else METRICS
for m in (*primary, *secondary):
if m.name not in METRICS:
raise ExperimentError(f"{path}: metric {m.name!r} (one of {', '.join(METRICS)})")
if m.name not in known:
raise ExperimentError(f"{path}: metric {m.name!r} (one of {', '.join(known)})")
if kind == "rule":
_check_rule_manifest(path, raw, arms)
test = dict(raw.get("test") or {})
stopping = dict(raw.get("stopping") or {"looks": [1.0], "boundaries": [1.959964]})
looks = tuple(float(x) for x in stopping.get("looks", [0.5, 1.0]))
@@ -244,11 +278,46 @@ def load_manifest(path: Path) -> Manifest:
raise ExperimentError(f"{path}: missing {exc}") from exc
if manifest.id != path.stem:
raise ExperimentError(f"{path}: id {manifest.id!r} must equal the file stem")
if kind == "program" and not manifest.families:
raise ExperimentError(f"{path}: a program experiment names its families")
if kind in ("program", "rule") and not manifest.families:
raise ExperimentError(f"{path}: a {kind} experiment names its families")
if kind == "rule" and set(manifest.families) - set(manifest.programs):
raise ExperimentError(f"{path}: a rule experiment names a program for every family (programs:)")
return manifest
def _check_rule_manifest(path: Path, raw: Mapping[str, Any], arms: Mapping[str, Arm]) -> None:
"""A decision-rule experiment costs nothing by construction: no arm overrides the program, no budget, and every
arm names a known rule with known parameters; the populations are declared before the run."""
budget = raw["budget_usd"]
total = float(budget["total"] if isinstance(budget, Mapping) else budget)
if total != 0 or any(a.budget_usd for a in arms.values()):
raise ExperimentError(f"{path}: a rule experiment replays cached responses only: budget_usd must be 0")
for arm in arms.values():
if arm.spec or arm.task_env or arm.programs:
raise ExperimentError(f"{path}: arm {arm.name} of a rule experiment may change the BLOCK rule only (rule:)")
name = arm.rule.get("name")
if name not in RULES:
raise ExperimentError(f"{path}: arm {arm.name} rule.name {name!r} (one of {', '.join(RULES)})")
allowed = {"name"} | RULE_PARAMETERS.get(str(name), set())
if set(arm.rule) - allowed:
raise ExperimentError(f"{path}: arm {arm.name} rule parameters {sorted(set(arm.rule) - allowed)} are not understood")
if arm.rule.get("bound", "wilson") not in BOUNDS or arm.rule.get("select_on", "paired") not in POPULATIONS:
raise ExperimentError(f"{path}: arm {arm.name} bound in {BOUNDS} and select_on in {POPULATIONS}")
if name == "pooled_block_gate":
if arm.rule.get("normalise") not in NORMALISATIONS:
raise ExperimentError(f"{path}: arm {arm.name} normalise in {NORMALISATIONS} (declared before the run)")
if int(arm.rule.get("floor", FAMILY_FLOOR)) < 1 or not 0 < float(arm.rule.get("recall", TARGET_RECALL)) <= 1:
raise ExperimentError(f"{path}: arm {arm.name} floor >= 1 and recall in (0, 1]")
populations = dict(raw.get("populations") or {})
if populations.get("primary") not in POPULATIONS or any(p not in POPULATIONS for p in populations.get("secondary", [])):
raise ExperimentError(f"{path}: populations.primary (and secondary) in {POPULATIONS}")
decision = dict(raw.get("decision") or {})
if not 0 < float(decision.get("target_precision", 0)) <= 1 or decision.get("arm") not in arms:
raise ExperimentError(f"{path}: decision.target_precision in (0, 1] and decision.arm naming an arm")
if decision.get("pooled") and arms[str(decision["arm"])].rule.get("name") != "pooled_block_gate":
raise ExperimentError(f"{path}: decision.pooled judges a pooled_block_gate arm only")
# --- register ------------------------------------------------------------------------------------------------------------
@@ -274,8 +343,22 @@ def registration_id(experiment_id: str) -> str:
def registered(store: MemoryStore, experiment_id: str) -> dict[str, Any] | None:
"""Return the authoritative registration, preferring a set units digest over pending rows.
Within that group, use the greatest ``created`` value when all values are nonempty and mutually
comparable (strings compare lexically). Otherwise, or on a tie, use the lexically greatest ``pin``,
then the last row in store order (key/id order for MemoryStore, insertion order for an append-only store).
"""
rows = [r.row for r in store.rows(repo=experiment_repo(experiment_id), kind=EXPERIMENT_KIND) if r.row.get("task") == REGISTER_TASK]
return rows[0] if rows else None
if not rows:
return None
rows = [r for r in rows if r.get("units_digest")] or rows
rows = sorted(rows, key=lambda r: str(r.get("pin", "")))
if all(r.get("created") for r in rows):
# Incomparable timestamps retain the deterministic pin/store ordering.
with suppress(TypeError):
rows = sorted(rows, key=lambda r: r["created"])
return rows[-1]
def register(
@@ -383,7 +466,7 @@ def program_folds(memory: Sequence[Example], manifest: Manifest) -> tuple[list[F
def freeze_units(store: MemoryStore, manifest: Manifest, memory: Sequence[Example]) -> list[Unit]:
"""The evaluation candidates of the manifest's families under the incumbent's outer folds, paired by candidate
across the two pins where both sides are in memory (no model; the list is then committed and registered)."""
if manifest.kind != "program":
if manifest.kind not in ("program", "rule"):
raise ExperimentError(f"{manifest.id}: units of a plane experiment come from candidate generation, not the memory")
folds, _ = program_folds(memory, manifest)
sides: dict[str, tuple[str, str, str]] = {}
@@ -517,10 +600,10 @@ def auc(rows: Sequence[ArmUnit]) -> float | None:
def within_pair_auc(rows: Sequence[ArmUnit]) -> float | None:
"""Of the candidate pairs (vulnerable and fixed side both present), the share where the vulnerable side scores
higher (ties one half)."""
pairs: dict[str, dict[int, float]] = {}
pairs: dict[tuple[str, str, str], dict[int, float]] = {}
for r in rows:
if r.pair and r.s is not None:
pairs.setdefault(r.pair, {})[r.label] = r.s
pairs.setdefault((r.group, r.family, r.pair), {})[r.label] = r.s
done = [(p[1], p[0]) for p in pairs.values() if 0 in p and 1 in p]
if not done:
return None
@@ -535,6 +618,11 @@ def metric_value(metric: Metric, rows: Sequence[ArmUnit]) -> float | None:
return auc(rows)
if name == "within_pair_auc":
return within_pair_auc(rows)
if name == "top_30_precision":
ranked = sorted((r for r in rows if r.s is not None), key=lambda r: (-r.s if r.s is not None else 0.0, r.unit))
return _mean([float(r.label) for r in ranked[:30]]) if len(ranked) >= 30 else None
if name == "unsure_rate":
return _mean([float(r.verdict == "unsure") for r in rows])
if name == "advisory_recall":
return _mean([float(r.flag) for r in rows if r.label == 1])
if name == "advisory_precision":
@@ -558,6 +646,8 @@ def binary_value(metric: Metric, row: ArmUnit) -> bool | None:
return row.agree
if metric.name == "flag_rate":
return row.flag
if metric.name == "unsure_rate":
return row.verdict == "unsure"
return None
@@ -595,7 +685,13 @@ def paired_bootstrap(
rng = random.Random(seed)
values: list[float] = []
for _ in range(resamples if names else 0):
sample = [p for g in (rng.choice(names) for _ in names) for p in by_group[g]]
# Give each drawn cluster its own identity: repeated draws must retain
# repeated candidate pairs when within_pair_auc groups the two sides.
sample = [
(replace(a, group=f"{draw}:{g}"), replace(b, group=f"{draw}:{g}"))
for draw, g in enumerate(rng.choice(names) for _ in names)
for a, b in by_group[g]
]
a = metric_value(metric, [p[0] for p in sample])
b = metric_value(metric, [p[1] for p in sample])
if a is not None and b is not None:
@@ -704,6 +800,9 @@ def decide(manifest: Manifest, finals: Mapping[str, Estimate], cost: Estimate |
return "reject", f"the CI of {', '.join(bad)} excludes 0 against B"
first = manifest.primary[0].name
if sides[first] == "favour":
max_ratio = (manifest.raw.get("decision") or {}).get("max_cost_ratio")
if max_ratio is not None and (cost is None or cost.a is None or cost.b is None or cost.b > float(max_ratio) * cost.a):
return "inconclusive", f"B's cost per candidate is unavailable or exceeds {max_ratio}x A; the incumbent stays"
return "adopt", f"the CI of {first} excludes 0 in B's favour and no primary CI excludes 0 against B"
if manifest.cost_only and cost is not None and tost is not None and tost.ci is not None and cost.ci is not None:
within = -manifest.tost_margin <= tost.ci[0] and tost.ci[1] <= manifest.tost_margin
@@ -812,6 +911,8 @@ def run(
"""Every unit in every arm, paired, repositories in the pre-shuffled look order, a seeded coin per unit for which
arm goes first; outcomes already in the store are not decided again. After the first look's repositories the
primary is tested against its boundary and, when crossed, the run stops. A ceiling ends the run ``unfinished``."""
if manifest.kind == "rule":
raise ExperimentError(f"{manifest.id}: a rule experiment scores once at zero cost (run_rule), its arms differ only in the rule")
if manifest.kind != "program":
raise ExperimentError(f"{manifest.id}: plane arms run as plane Runs; their outcomes arrive as arm_outcome rows")
row = check_registered(store, manifest)
@@ -892,7 +993,327 @@ def run(
return summary
# --- decision-rule experiments (kind: rule) -----------------------------------------------------------------------------
class ZeroCostViolation(ExperimentError):
"""A model call reached the client of a zero-cost run, or the meter recorded spend: the run is void."""
class _RefusingChat:
"""The inner client behind a zero-cost run's meter. The meter's zero ceilings stop every call before it gets here;
reaching it means a guard was bypassed."""
usage: list[Any] = []
def complete(self, **_: Any) -> Any:
raise ZeroCostViolation("a model call reached the client of a zero-cost run")
def zero_cost_caller(model: str, prices: Mapping[str, Any], store: MemoryStore | None) -> tuple[Caller, MeteredClient]:
"""A caller that answers only from the response cache: its client is a meter with zero ceilings (USD and calls)
around a client that refuses. A response that is not cached raises ``BudgetExhausted`` before any call; the run
excludes that unit and counts it. The meter is the proof: its ``calls`` and ``usd`` must stay 0."""
meter = MeteredClient(_RefusingChat(), prices=prices, budget_usd=0.0, budget_calls=0) # type: ignore[arg-type]
return Caller(meter, model, prices, store=store), meter
def _checked_units(manifest: Manifest, memory: Sequence[Example]) -> tuple[list[Unit], dict[str, Fold], str]:
units = read_units(manifest.units_file)
folds, fdigest = program_folds(memory, manifest)
by_name = {f.name: f for f in folds}
index = {e.id for e in memory}
for unit in units:
fold = by_name.get(unit.fold)
if fold is None or unit.group not in fold.eval_groups or unit.unit not in index:
raise ExperimentError(
f"{manifest.id}: unit {unit.unit[:12]} is not under fold {unit.fold!r} of this memory; the memory changed"
)
return units, by_name, fdigest
def run_rule(
store: MemoryStore,
manifest: Manifest,
memory: Sequence[Example],
caller: Caller,
meter: MeteredClient,
excerpt_text: Callable[[str], str | None],
vectors: Mapping[str, Sequence[float]] | None = None,
*,
on_decision: Callable[[Outcome, Decision], None] | None = None,
) -> RunSummary:
"""The one scoring pass of a rule experiment: every unit decided once by its family's incumbent program, answered
only from the response cache. A unit whose responses are not all cached is excluded and counted (an outcome row
with ``verdict = replay_miss`` and no score), never asked. Raises :class:`ZeroCostViolation` when the meter shows
a call or any spend afterwards."""
if manifest.kind != "rule":
raise ExperimentError(f"{manifest.id}: run_rule runs decision-rule experiments only")
if caller.client is not meter or meter.budget_usd != 0.0 or meter.budget_calls != 0:
raise ZeroCostViolation(f"{manifest.id}: a rule experiment runs only behind a zero-ceiling meter (zero_cost_caller)")
row = check_registered(store, manifest)
units, by_name, fdigest = _checked_units(manifest, memory)
index = {e.id: e for e in memory}
programs = {f: Program(arm_spec(manifest, manifest.arms["A"], f, store), memory, excerpt_text, vectors) for f in manifest.families}
models = {spec_of(load_program(store, manifest.programs[f])).model for f in manifest.families}
if models != {caller.model}:
raise ExperimentError(f"{manifest.id}: the programs' models {sorted(models)} differ from the caller's {caller.model!r}")
candidate_of = candidate_from_example(excerpt_text, vectors)
have = {outcome_id(manifest.id, o.arm, o.unit, o.replicate) for o in load_outcomes(store, manifest.id)}
decided, skipped = {SCORE_ARM: 0}, {"already": 0, MISS_VERDICT: 0}
fold_seed = int(manifest.folds.get("seed", 0))
for unit in sorted(units, key=lambda u: (u.group, u.family, u.unit)):
if outcome_id(manifest.id, SCORE_ARM, unit.unit, 0) in have:
skipped["already"] += 1
continue
candidate = candidate_of(index[unit.unit])
if candidate is None:
raise ExperimentError(f"{manifest.id}: unit {unit.unit[:12]} has no excerpt in the store")
try:
decision = programs[unit.family].decide(candidate, by_name[unit.fold], caller, seed=fold_seed)
except BudgetExhausted: # a response is not cached: the zero ceiling stopped the call before it started
outcome = Outcome(unit.unit, unit.group, unit.family, unit.label, unit.pair, SCORE_ARM, 0, None, MISS_VERDICT, 0.0)
skipped[MISS_VERDICT] += 1
store.put_row(outcome_row(manifest.id, str(row["pin"]), outcome))
continue
outcome = Outcome(unit.unit, unit.group, unit.family, unit.label, unit.pair, SCORE_ARM, 0, decision.s, decision.verdict, 0.0,
decision.parse_failed) # fmt: skip
store.put_row(outcome_row(manifest.id, str(row["pin"]), outcome))
decided[SCORE_ARM] += 1
if on_decision is not None:
on_decision(outcome, decision)
if meter.calls or (meter.usd or 0.0) > 0 or caller.calls:
raise ZeroCostViolation(f"{manifest.id}: the meter recorded {meter.calls} calls and ${meter.usd}; the run is void")
spend = {SCORE_ARM: {"usd": meter.usd, "client_calls": meter.calls, "replayed": caller.replayed, "cache_digest": caller.cache_digest(),
"replay_priced_usd": caller.usd()}} # fmt: skip
summary = RunSummary(manifest.id, "done", len(units), decided, skipped, spend, fdigest)
store.put_row({
"id": row_id(EXPERIMENT_KIND, f"experiment-{manifest.id}", RUN_TASK, manifest.id), "kind": EXPERIMENT_KIND,
"repo": experiment_repo(manifest.id), "pin": str(row["pin"]), "run": f"experiment-{manifest.id}", "task": RUN_TASK,
"population": POPULATION, "split": manifest.id, "image": "host", "experiment": manifest.id,
"manifest_digest": manifest.digest, **summary.as_dict(),
}) # fmt: skip
return summary
def min_flagged_for_bound(target: float, z: float = 1.959964) -> int:
"""The fewest flagged units, all correct, whose Wilson lower bound reaches ``target`` (73 at 0.95)."""
n = 1
while wilson_lower(n, n, z) < target:
n += 1
return n
def _group_bootstrap_precision(rows: Sequence[Prediction], *, resamples: int, seed: int) -> list[float] | None:
by_group: dict[str, list[Prediction]] = {}
for r in rows:
by_group.setdefault(r.group, []).append(r)
names = sorted(by_group)
rng = random.Random(seed)
values = []
for _ in range(resamples if names else 0):
sample = [r for g in (rng.choice(names) for _ in names) for r in by_group[g]]
flagged = [r for r in sample if r.point == "BLOCK"]
if flagged:
values.append(sum(r.label for r in flagged) / len(flagged))
if len(values) < 2:
return None
values.sort()
return [values[int(0.025 * (len(values) - 1))], values[int(0.975 * (len(values) - 1))]]
def flag_summary(rows: Sequence[Prediction], target: float, *, resamples: int = 2000, seed: int = 0) -> dict[str, Any]:
"""Held-out BLOCK decisions of one population: flagged count, flagged precision with its Wilson 95% lower bound
and a repository-cluster bootstrap interval, coverage (the share of positives flagged), and the M4 reading."""
flagged = [r for r in rows if r.point == "BLOCK"]
hits = sum(r.label for r in flagged)
positives = sum(r.label for r in rows)
precision = hits / len(flagged) if flagged else None
lower = wilson_lower(hits, len(flagged)) if flagged else None
return {
"units": len(rows), "groups": len({r.group for r in rows}), "positives": positives, "flagged": len(flagged),
"true_flagged": hits, "flagged_precision": None if precision is None else round(precision, 4),
"flagged_precision_lb": None if lower is None else round(lower, 4),
"flagged_precision_ci": _group_bootstrap_precision(rows, resamples=resamples, seed=seed) if flagged else None,
"coverage": round(hits / positives, 4) if positives else None,
"meets_target_point": precision is not None and precision >= target,
"meets_target_lb": lower is not None and lower >= target,
} # fmt: skip
def _apply_rule(arm: Arm, preds: Sequence[Prediction], seed: int) -> tuple[list[Prediction], dict[str, Any]]:
if arm.rule["name"] == "calibrated_block":
return calibrated_block(preds, seed=seed)
select_on = str(arm.rule.get("select_on", "paired"))
if arm.rule["name"] == "pooled_block_gate": # the family floor is applied by analyse_rule, on held-out flags
rows, gates = pooled_block_gate(
preds, eligible=(lambda r: bool(r.pair)) if select_on == "paired" else None,
precision=float(arm.rule.get("precision", TARGET_PRECISION)), recall=float(arm.rule.get("recall", TARGET_RECALL)),
) # fmt: skip
info = {
f: {"t_block_by_fold": {j: g["t_block"] for j, g in gates.items()},
"t_quiet_by_fold": {j: g["t_quiet"].get(f) for j, g in gates.items()}}
for f in sorted({r.family for r in preds})
} # fmt: skip
return rows, info
decided, thresholds = nested_precision_block(
preds, eligible=(lambda r: bool(r.pair)) if select_on == "paired" else None,
precision=float(arm.rule.get("precision", TARGET_PRECISION)), bound=str(arm.rule.get("bound", "wilson")),
min_flagged=int(arm.rule.get("min_flagged", 1)),
) # fmt: skip
return decided, {f: {"thresholds_by_fold": t} for f, t in thresholds.items()}
def decide_pooled(manifest: Manifest, families: Mapping[str, Mapping[str, Any]], overall: Mapping[str, Any]) -> tuple[str, str]:
"""The pre-registered rule of a pooled gate (Req 6.4, exp-004), on the primary population's held-out flags of
every family together, before the family floor: **reject** when they flag at a point precision below the target;
**adopt** when their Wilson 95% lower bound reaches the target and at least one family passes the floor (no
held-out error, at least ``floor`` flags) -- those families block, the others stay advisory; otherwise
**inconclusive** (the incumbent rule stays)."""
decision = dict(manifest.raw["decision"])
arm, target = str(decision["arm"]), float(decision["target_precision"])
primary = str(manifest.raw["populations"]["primary"])
cell = overall[arm][primary]
if cell["flagged"] and not cell["meets_target_point"]:
return "reject", f"{arm} flags held-out {primary} units of all families together below precision {target}"
passing = sorted(f for f, v in families.items() if v[arm]["floor"]["may_block"])
if cell["meets_target_lb"] and passing:
return "adopt", f"{arm}'s pooled held-out Wilson lower bound reaches {target}; {', '.join(passing)} pass the family floor and block"
if not cell["flagged"]:
return "inconclusive", f"{arm} offers no BLOCK on held-out {primary} units; the incumbent stays"
if not cell["meets_target_lb"]:
return "inconclusive", f"{arm}'s pooled held-out Wilson lower bound is below {target}; the incumbent stays"
return "inconclusive", f"{arm} reaches {target} pooled but no family passes the floor; every family stays advisory"
def decide_rule(manifest: Manifest, families: Mapping[str, Mapping[str, Any]], overall: Mapping[str, Any] | None = None) -> tuple[str, str]:
"""The pre-registered rule: **reject** the decision arm when it flags held-out units of the primary population in
some family at a point precision below the target; **adopt** it when it does not and some family's held-out
Wilson lower bound reaches the target; otherwise **inconclusive** (the incumbent rule stays). A manifest whose
``decision.pooled`` is set is judged by :func:`decide_pooled` instead."""
decision = dict(manifest.raw["decision"])
if decision.get("pooled"):
if overall is None:
raise ExperimentError(f"{manifest.id}: a pooled decision needs the pooled-over-families totals")
return decide_pooled(manifest, families, overall)
arm, target = str(decision["arm"]), float(decision["target_precision"])
primary = str(manifest.raw["populations"]["primary"])
cells = {f: v[arm][primary] for f, v in families.items()}
low = sorted(f for f, c in cells.items() if c["flagged"] and not c["meets_target_point"])
if low:
return "reject", f"{arm} flags held-out {primary} units below precision {target} in {', '.join(low)}"
reached = sorted(f for f, c in cells.items() if c["meets_target_lb"])
if reached:
return "adopt", f"{arm}'s held-out Wilson lower bound reaches {target} in {', '.join(reached)} and nowhere flags below it"
if not any(c["flagged"] for c in cells.values()):
return "inconclusive", f"{arm} offers no BLOCK on held-out {primary} units in any family; the incumbent stays"
return "inconclusive", f"{arm} flags only at a Wilson lower bound below {target}; the incumbent stays"
def analyse_rule(
manifest: Manifest,
outcomes: Sequence[Outcome],
units: Sequence[Unit],
*,
resamples: int | None = None,
provenance: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
"""Counts and intervals only: every arm's rule applied to the same held-out scores, per family and population,
the pooled-over-families totals, the exclusions, the training capacity of each family against the Wilson bound,
and the pre-registered verdict."""
n = resamples if resamples is not None else min(manifest.resamples, 2000)
target = float(manifest.raw["decision"]["target_precision"])
populations = [str(manifest.raw["populations"]["primary"]), *[str(p) for p in manifest.raw["populations"].get("secondary", [])]]
by_unit = {u.unit: u for u in units}
scores = {o.unit: o for o in outcomes if o.arm == SCORE_ARM and o.replicate == 0 and o.unit in by_unit}
preds = [
Prediction(by_unit[u].fold, o.family, o.group, o.label, float(o.s), o.verdict, parse_failed=o.parse_failed, pair=o.pair)
for u, o in sorted(scores.items())
if o.s is not None
] # fmt: skip
excluded = {f: sum(1 for o in scores.values() if o.s is None and o.family == f) for f in manifest.families}
not_run = {f: sum(1 for u in units if u.family == f and u.unit not in scores) for f in manifest.families}
def select(rows: Sequence[Prediction], population: str) -> list[Prediction]:
return [r for r in rows if r.pair] if population == "paired" else list(rows)
decided = {name: _apply_rule(arm, preds, manifest.seed) for name, arm in manifest.arms.items()}
gated = {name for name, arm in manifest.arms.items() if arm.rule.get("name") == "pooled_block_gate"}
floors = { # held-out flags of the primary population decide which families may block; no threshold moves
name: {f: family_floor([r for r in select(decided[name][0], populations[0]) if r.family == f],
floor=int(manifest.arms[name].rule.get("floor", FAMILY_FLOOR))).get(f)
or {"flagged": 0, "errors": 0, "floor": int(manifest.arms[name].rule.get("floor", FAMILY_FLOOR)), "may_block": False}
for f in manifest.families}
for name in gated
} # fmt: skip
def quiet_recall(rows: Sequence[Prediction]) -> float | None:
positives = [r for r in rows if r.label == 1]
return round(sum(1 for r in positives if r.point in ("BLOCK", "ADVISORY")) / len(positives), 4) if positives else None
families: dict[str, dict[str, Any]] = {}
for family in manifest.families:
cell: dict[str, Any] = {}
for name, (rows, info) in decided.items():
mine = [r for r in rows if r.family == family]
cell[name] = {**info.get(family, {}), **{p: flag_summary(select(mine, p), target, resamples=n, seed=manifest.seed)
for p in populations}} # fmt: skip
if name in gated:
cell[name]["floor"] = floors[name][family]
cell[name]["quiet_recall"] = {p: quiet_recall(select(mine, p)) for p in populations}
mine = [r for r in preds if r.family == family]
folds = sorted({r.fold for r in mine})
cell["capacity"] = {
"evaluated": len(mine), "excluded_replay_miss": excluded[family], "not_run": not_run[family],
"paired_positives_by_training_folds": {f: sum(r.label for r in mine if r.pair and r.fold != f) for f in folds},
} # fmt: skip
families[family] = cell
overall = {
name: {p: flag_summary(select(rows, p), target, resamples=n, seed=manifest.seed) for p in populations}
for name, (rows, _) in decided.items()
}
after_floor = { # what users would get: BLOCK only in the families that pass the floor (secondary, not deciding)
name: {p: flag_summary(select(apply_floor(decided[name][0], floors[name]), p), target, resamples=n, seed=manifest.seed)
for p in populations}
for name in sorted(gated)
} # fmt: skip
verdict, reason = decide_rule(manifest, families, overall)
spend = {SCORE_ARM: round(sum(o.usd or 0.0 for o in outcomes if o.arm == SCORE_ARM), 6)}
return {
"experiment": manifest.id, "manifest_digest": manifest.digest, "hypothesis": manifest.hypothesis,
"arms": {a: {"label": arm.label, "rule": dict(arm.rule)} for a, arm in manifest.arms.items()},
"units": {"frozen": len(units), "evaluated": len(preds), "excluded_replay_miss": sum(excluded.values()),
"not_run": sum(not_run.values()), "paired_evaluated": sum(1 for r in preds if r.pair),
"groups": len({r.group for r in preds}), "by_label": {str(lab): sum(r.label == lab for r in preds) for lab in (0, 1)},
"unsure": sum(1 for r in preds if not r.blockable)},
"populations": {"primary": populations[0], "secondary": populations[1:]}, "target_precision": target,
"min_flagged_for_bound": min_flagged_for_bound(target), "primary": [m.as_dict() for m in manifest.primary],
"secondary": [m.as_dict() for m in manifest.secondary], "families": families, "overall": overall, "resamples": n,
**({"overall_after_floor": after_floor} if after_floor else {}), "decision": verdict, "reason": reason, "spend_usd": spend,
**dict(provenance or {}),
} # fmt: skip
def rule_result_row(manifest: Manifest, pin: str, result: Mapping[str, Any]) -> dict[str, Any]:
"""The ``experiment_result`` row of a rule experiment, with the column types of :func:`result_row`."""
primary = result["populations"]["primary"]
tests = {
f: {a: {k: cell[a][primary][k] for k in ("flagged", "flagged_precision", "flagged_precision_lb", "coverage")}
for a in manifest.arms}
for f, cell in result["families"].items()
} # fmt: skip
return {
"id": row_id(RESULT_KIND, f"experiment-{manifest.id}", "analyse", manifest.id), "kind": RESULT_KIND,
"repo": experiment_repo(manifest.id), "pin": pin, "run": f"experiment-{manifest.id}", "task": "analyse", "population": POPULATION,
"split": manifest.id, "image": "host", "experiment": manifest.id, "manifest_digest": manifest.digest,
"decision": result["decision"], "reason": result["reason"], "unit_counts": result["units"], "spend_usd": result["spend_usd"],
"tests": tests, "looks_taken": [],
} # fmt: skip
__all__ = [
"MISS_VERDICT", "POPULATIONS", "RULES", "RULE_METRICS", "SCORE_ARM", "ZeroCostViolation", "analyse_rule", "decide_rule",
"flag_summary", "min_flagged_for_bound", "rule_result_row", "run_rule", "zero_cost_caller",
"ALPHA", "ARM_KINDS", "EXPERIMENT_KIND", "MANIFEST_DIR", "METRICS", "OBF_TWO_LOOKS", "OUTCOME_KIND", "RESAMPLES", "RESULT_DIR",
"RESULT_KIND", "TOST_MARGIN", "VERDICTS", "Arm", "ArmUnit", "Estimate", "ExperimentError", "Look", "Manifest", "McNemar", "Metric",
"Outcome", "RunSummary", "Unit", "analyse", "arm_spec", "arm_units", "auc", "check_registered", "committed_at_head", "decide",
+229
View File
@@ -0,0 +1,229 @@
"""Fill experiment responses without writing scores or changing the frozen population.
Estimates use the family's mean billed cached response cost when usage is available.
Otherwise they use Prompt.estimated_tokens and 300 output tokens. Unknown answers may require one retry;
dry-run counts those separately as conditional requests, never fabricating an answer.
"""
from __future__ import annotations
import math
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass
from typing import Any
from ..model.endpoint import DeepSeekChatClient
from ..plane.budget import BudgetExhausted, MeteredClient, cost_of, prices_from
from ..plane.memory import MemoryStore
from ..tool_hunter import ChatClient, ChatResponse
from . import experiments as ex
from .examples import Example
from .program import Caller, Program, Prompt, parse_answer, request_key
DEFAULT_OUTPUT_TOKENS = 300
DEFAULT_MARGIN_USD = 0.01
@dataclass(frozen=True)
class Request:
family: str
prompt: Prompt
temperature: float
sample: str
def key(self, caller: Caller) -> str:
return request_key(caller.model, caller.params_digest, self.prompt.messages, self.temperature, self.sample, True)
def ask(self, caller: Caller) -> str:
return caller.ask(self.prompt.messages, temperature=self.temperature, sample=self.sample)
def retry(self) -> Request:
return Request(self.family, self.prompt, 0.0, f"{self.sample}:retry")
class FillMeter(MeteredClient):
"""Reserve a whole call before admission, including the first (smoke) call.
The fixed cent is a floor, never replaced by a smaller observed cost. For the
production adapter, UTF-8 bytes conservatively bound input tokens (plus 256 framing
tokens per message), and the provider enforces the output limit. Injected clients
must honour the same token/usage contract. No hidden transport retries are allowed.
"""
def __init__(self, client: ChatClient, parameters: Mapping[str, Any], budget: float, output_limit: int) -> None:
super().__init__(client, prices=parameters, budget_usd=budget)
self.limit = budget
self.output_limit = output_limit
self.largest_call_usd = 0.0
def complete(self, **kwargs: Any) -> ChatResponse:
assert self.prices is not None
messages = kwargs["messages"]
input_bound = sum(len(str(m["content"]).encode("utf-8")) + 256 for m in messages)
bound = (
input_bound * max(self.prices.input_per_m, self.prices.cache_hit_per_m) + self.output_limit * self.prices.output_per_m
) / 1_000_000
reserve = max(DEFAULT_MARGIN_USD, self.largest_call_usd, bound)
self.budget_usd = max(0.0, self.limit - reserve)
before = self.usd or 0.0
tokens_before = sum(self.usage.values())
try:
response = super().complete(**kwargs)
except BudgetExhausted:
raise
except Exception:
self.calls += 1 # attempted provider calls count even when no response was returned
raise
finally:
self.largest_call_usd = max(self.largest_call_usd, (self.usd or 0.0) - before)
if sum(self.usage.values()) <= tokens_before:
raise RuntimeError("provider returned no metered token usage")
return response
def fill_experiment(
store: MemoryStore,
manifest: ex.Manifest,
memory: Sequence[Example],
parameters: Mapping[str, Any],
excerpt_text: Callable[[str], str | None],
vectors: Mapping[str, Sequence[float]] | None = None,
*,
client: ChatClient | None = None,
client_factory: Callable[[], ChatClient] | None = None,
budget_usd: float | None = None,
dry_run: bool = False,
arms: Sequence[str] | None = None,
) -> dict[str, Any]:
"""Inspect or fill every frozen unit. The first cache miss is the smoke call.
A successful smoke is retained in the cache; any provider error stops immediately
and is reported without its message, which may contain credentials. A factory is
resolved only after validation, and only when a paid request is actually missing.
"""
from .evaluate import candidate_from_example
if manifest.kind not in ("rule", "program"):
raise ex.ExperimentError("experiment fill supports kind: rule or program only")
selected = list(dict.fromkeys(arms if arms is not None else (["A"] if manifest.kind == "rule" else manifest.paid_arms)))
if not selected or any(name not in manifest.arms for name in selected):
raise ex.ExperimentError("fill requires at least one known arm")
# Rule arms share exactly one scoring pass, regardless of the selected rule.
scoring = selected[:1] if manifest.kind == "rule" else selected
if not dry_run and (budget_usd is None or not math.isfinite(budget_usd) or budget_usd < 0):
raise ex.ExperimentError("--budget-usd must be a finite non-negative number (or use --dry-run)")
if manifest.kind == "program" and not dry_run:
ceiling = min(manifest.budget_total_usd, sum(manifest.arms[name].budget_usd for name in selected))
if budget_usd is not None and budget_usd > ceiling:
raise ex.ExperimentError(f"--budget-usd exceeds the selected program arms' manifest ceiling ({ceiling})")
prices = prices_from(parameters)
if prices is None or any(not math.isfinite(p) or p < 0 for p in (prices.input_per_m, prices.output_per_m, prices.cache_hit_per_m)):
raise ex.ExperimentError("fill requires finite, non-negative Model prices")
ex.check_registered(store, manifest)
units, folds, _ = ex._checked_units(manifest, memory)
programs = {
(name, f): Program(ex.arm_spec(manifest, manifest.arms[name], f, store), memory, excerpt_text, vectors)
for name in scoring
for f in manifest.families
}
models = {p.spec.model for p in programs.values()}
if len(models) != 1:
raise ex.ExperimentError("the selected programs must use the same model")
caller = Caller(None, next(iter(models)), parameters, store=store)
index = {e.id: e for e in memory}
candidate_of = candidate_from_example(excerpt_text, vectors)
requests: list[Request] = []
for unit in sorted(units, key=lambda u: (u.group, u.family, u.unit)):
candidate = candidate_of(index[unit.unit])
if candidate is None:
raise ex.ExperimentError(f"{manifest.id}: unit {unit.unit[:12]} has no excerpt in the store")
for name in scoring:
prepared = programs[name, unit.family].prepare_samples(candidate, folds[unit.fold], seed=int(manifest.folds.get("seed", 0)))
repeats = manifest.arms[name].repeats if manifest.kind == "program" else 1
for replicate in range(repeats):
salt = f"r{replicate}:" if replicate else ""
for prompt, _, temperature, sample in prepared:
requests.append(Request(unit.family, prompt, temperature, f"{salt}{sample}"))
# Request counts are distinct cache keys per family, even when multiple units share one.
def inventory() -> dict[str, dict[str, Request]]:
result: dict[str, dict[str, Request]] = {f: {} for f in manifest.families}
for req in requests:
result[req.family][req.key(caller)] = req
entry = caller._cached(req.key(caller))
if entry is not None and parse_answer(entry.get("content"), req.prompt.valid_lines) is None:
retry = req.retry()
result[req.family][retry.key(caller)] = retry
return result
initial = inventory()
initially_cached = {key for cells in initial.values() for key in cells if caller._cached(key) is not None}
families: dict[str, Any] = {}
for family, cells in initial.items():
usages = [
usage
for key in cells
if (entry := caller._cached(key)) is not None and isinstance(usage := entry.get("usage"), Mapping) and usage
]
outputs = [float(usage["completion_tokens"]) for usage in usages if "completion_tokens" in usage]
mean_output = sum(outputs) / len(outputs) if outputs else DEFAULT_OUTPUT_TOKENS
missing = [req for key, req in cells.items() if key not in initially_cached]
families[family] = {
"units": sum(u.family == family for u in units),
"requests": len(cells),
"cached": len(cells) - len(missing),
"missing": len(missing),
"filled": 0,
"still_missing": len(missing),
"estimate_method": "measured" if usages else "estimated",
"estimated_usd": len(missing) * sum(cost_of(usage, prices) for usage in usages) / len(usages)
if usages
else sum((r.prompt.estimated_tokens * prices.input_per_m + mean_output * prices.output_per_m) / 1_000_000 for r in missing),
"estimated_output_tokens": mean_output,
"conditional_retries": sum(not r.sample.endswith(":retry") for r in missing),
}
report: dict[str, Any] = {
"experiment": manifest.id,
"arms": selected,
"status": "dry_run" if dry_run else "done",
"families": families,
"client_calls": 0,
"usd": 0.0,
"reason": "",
}
if dry_run or not any(c["missing"] for c in families.values()):
return report
meter: FillMeter | None = None
try:
if client is None and client_factory is not None:
client = client_factory()
if client is None:
raise ex.ExperimentError("no chat endpoint configured; nothing was spent")
output_limit = max(p.spec.max_output_tokens for p in programs.values())
if isinstance(client, DeepSeekChatClient):
client = client.bounded(output_limit)
assert budget_usd is not None
meter = FillMeter(client, parameters, budget_usd, output_limit)
caller.client = meter
for req in requests:
content = req.ask(caller)
if parse_answer(content, req.prompt.valid_lines) is None:
req.retry().ask(caller)
except BudgetExhausted:
report.update(status="unfinished", reason="budget exhausted; resume with a new fill budget")
except ex.ExperimentError:
raise
except Exception:
# Never echo provider exceptions: response bodies and URLs may contain secrets.
report.update(status="failed", reason="provider or cache failure; fill stopped")
final = inventory()
for family, cells in final.items():
cached = {key for key in cells if caller._cached(key) is not None}
families[family].update(requests=len(cells), filled=len(cached - initially_cached), still_missing=len(cells) - len(cached))
if meter is not None:
report.update(client_calls=meter.calls, usd=meter.usd)
return report
# Backwards-compatible name for callers of the original rule-only API.
fill_rule = fill_experiment
+26 -13
View File
@@ -38,6 +38,7 @@ from .examples import Example
from .folds import Fold
from .retrieve import Retrieval, Target, assert_boundary, demonstrations_for, retrieve
from .schema import DEFAULT_INPUTS, features_for, instruments_for, validate_x, withheld
from .slice import function_slice, render_slice
VERDICTS = ("vulnerable", "not_vulnerable", "unsure")
UNSURE = "unsure"
@@ -150,9 +151,11 @@ def _case(
profile: str,
language: str,
inputs: str = DEFAULT_INPUTS,
slice: bool = False,
) -> str:
signals = render_signals(x, instruments, profile, inputs)
return f"Family: {family}\nLanguage: {language}\nCode:\n{code.rstrip()}\nSignals:\n{signals}\nRoles:\n{render_roles(roles)}"
flow = "\n" + render_slice(function_slice(code, language, family), language) if slice else ""
return f"Family: {family}\nLanguage: {language}\nCode:\n{code.rstrip()}{flow}\nSignals:\n{signals}\nRoles:\n{render_roles(roles)}"
def label_word(label: int) -> str:
@@ -204,6 +207,7 @@ class ProgramSpec:
priors: str = "off"
inputs: str = DEFAULT_INPUTS # the input profile: which instruments the classifier never sees (v1: the engine)
max_output_tokens: int = MAX_OUTPUT_TOKENS
slice: bool = False # opt-in in-function def-use evidence, computed from the same excerpt
sampling: str = SAMPLING[0] # temperature | retrieval_ensemble (an experiment arm's setting; never compiled)
@@ -252,7 +256,9 @@ def render_prefix(spec: ProgramSpec, demos: Sequence[Demo], examples: Mapping[st
"cited_lines": list(demo.cited_lines)} # fmt: skip
parts.append(
f"Demonstration {number}:\n"
+ _case(code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs)
+ _case(
code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs, spec.slice
)
+ f"\nAnswer: {json.dumps(answer, sort_keys=True)}"
)
return "\n\n".join(parts)
@@ -273,11 +279,14 @@ def render(
code = excerpt_text(example.excerpt_sha) or ""
blocks.append(
f"Example {number} (label: {label_word(example.label)}):\n"
+ _case(code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs)
+ _case(
code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs, spec.slice
)
)
t = candidate.target
blocks.append(
"Candidate:\n" + _case(candidate.code, t.x, t.instruments, candidate.roles, t.family, spec.profile, candidate.language, spec.inputs)
"Candidate:\n"
+ _case(candidate.code, t.x, t.instruments, candidate.roles, t.family, spec.profile, candidate.language, spec.inputs, spec.slice)
)
messages: list[dict[str, object]] = [
{"role": "system", "content": render_prefix(spec, demos, examples, excerpt_text)},
@@ -506,20 +515,24 @@ class Program:
out.append((prompt, got))
return out
def decide(self, candidate: Candidate, fold: Fold, caller: Caller, *, seed: int = 0) -> Decision:
def prepare_samples(self, candidate: Candidate, fold: Fold, *, seed: int = 0) -> list[tuple[Prompt, Retrieval, float, str]]:
"""The exact prompts, temperatures and sample labels shared by execution and cache planning."""
if self.spec.sampling == RETRIEVAL_ENSEMBLE:
prepared = self.prepare_ensemble(candidate, fold, seed=seed)
answers = [
sample_answer(prompt, caller, temperature=TEMPERATURES[0], sample=str(j), families=self.families)
for j, (prompt, _) in enumerate(prepared)
]
got = prepared[0][1]
return replace(aggregate(answers), retrieval=got.mode, neighbours=got.neighbours)
return [(prompt, got, TEMPERATURES[0], str(j)) for j, (prompt, got) in enumerate(prepared)]
if self.spec.sampling != SAMPLING[0]:
raise ValueError(f"unknown sampling {self.spec.sampling!r} (one of {', '.join(SAMPLING)})")
prompt, got = self.prepare(candidate, fold, seed=seed)
decision = classify(prompt, caller, self.spec.k, families=self.families)
return replace(decision, retrieval=got.mode, neighbours=got.neighbours)
return [(prompt, got, TEMPERATURES[0] if j == 0 else TEMPERATURES[1], str(j)) for j in range(self.spec.k)]
def decide(self, candidate: Candidate, fold: Fold, caller: Caller, *, seed: int = 0) -> Decision:
prepared = self.prepare_samples(candidate, fold, seed=seed)
answers = [
sample_answer(prompt, caller, temperature=temperature, sample=sample, families=self.families)
for prompt, _, temperature, sample in prepared
]
got = prepared[0][1]
return replace(aggregate(answers), retrieval=got.mode, neighbours=got.neighbours)
__all__ = [
+155
View File
@@ -0,0 +1,155 @@
"""Bounded, local def-use evidence, never a vulnerability verdict.
Uses the shipped FileIR and fact/taxonomy routing. Parameters are potential inputs,
as in semantic.taint; unknown call returns carry their arguments conservatively.
Sanitizers stay on paths (they do not erase connectivity). Facts currently scope
sanitizers by language, not family, so a sanitizer is not a discharge claim.
The flat IR has no branch edges or dominance: no guard is asserted, and branch
assignments are visited in textual order. No interprocedural or alias analysis.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from textwrap import dedent
from typing import Literal
from ..model.specs import taint_specs
from ..semantic.facts import load_facts
from ..semantic.ir import Bind, CallSite, FunctionIR, parse_file
from ..semantic.taint import _call_matches, _sanitizer_for, _source_in
from .excerpt import redact
Role = Literal["source", "step", "sanitizer", "guard", "sink"]
# The renderer uses two spaces; normalization strips the suffix on blank lines.
_NUMBERED = re.compile(r"^\s*(\d+)(?: (.*))?$")
@dataclass(frozen=True, order=True)
class Step:
line: int
role: Role
code: str
@dataclass(frozen=True)
class Slice:
steps: tuple[Step, ...] = ()
def function_slice(excerpt_text: str, language: str, family: str) -> Slice | None:
"""Return local paths, an empty slice, or None for an unsupported/unparseable excerpt.
Numbered excerpts retain their displayed numbers; raw code uses 1-based lines.
Diffs are never parsed. A truncated/malformed excerpt cannot establish no-flow.
Nested functions are analysed separately and never share definitions.
"""
raw = excerpt_text.split("\ndiff:\n", 1)[0].splitlines()
matches = [_NUMBERED.match(line) for line in raw]
if any(matches) and not all(matches):
return None
numbers = [int(m[1]) if m else i + 1 for i, m in enumerate(matches)]
lines = [(m[2] or "") if m else line for m, line in zip(matches, raw, strict=True)]
if len(set(numbers)) != len(numbers):
return None
code = dedent("\n".join(lines))
ir = parse_file("excerpt", code, language)
if not ir.parse_ok:
return None
spec = taint_specs(language=language).get(family)
if spec is None:
return Slice()
shown = redact(lines, language)
facts = load_facts().for_language(language)
def step(line: int, role: Role) -> Step:
return Step(numbers[line - 1], role, shown[line - 1].strip())
def paths(function: FunctionIR) -> set[Step]:
states = {name: {step(function.start_line, "source")} for name in function.params}
found: set[Step] = set()
def value(text: str, names: tuple[str, ...], constant: bool, line: int, end: int) -> set[Step]:
if constant:
return set()
result = set().union(*(states.get(name, set()) for name in names))
if _source_in(text, facts):
result.add(step(line, "source"))
if result:
# Keep only sanitizers whose call is inside this expression, not
# an unrelated call elsewhere on the same source line.
for call in function.calls:
if (
call.line >= line
and (not end or call.order <= end)
and any(
re.search(r"(?<![\w$])" + re.escape(name) + r"\s*\(", text) and _call_matches(call.name, name)
for name in spec.sanitizers
)
):
result.add(step(call.line, "sanitizer"))
return result
events: list[Bind | CallSite] = [*function.binds, *function.calls]
# Tree-sitter supplies evaluation ends, including same-line statements.
# The AST fallback supplies equivalent end-line/column positions.
events.sort(key=lambda e: (e.order if e.order else e.line, isinstance(e, Bind)))
for event in events:
if isinstance(event, Bind):
inherited = value(event.value_text, event.names, event.is_constant, event.line, event.order)
states[event.name] = inherited | {step(event.line, "step")} if inherited else set()
continue
if not any(_call_matches(event.name, name) for name in spec.sinks):
continue
reached: set[Step] = set()
for text, names, constant in zip(event.arg_texts, event.arg_names, event.arg_is_constant, strict=True):
reached.update(value(text, names, constant, event.line, event.order))
if reached:
if _sanitizer_for(event, facts):
reached.add(step(event.line, "sanitizer"))
found.update(reached | {step(event.line, "sink")})
return found
result: set[Step] = set()
for function in ir.functions:
result.update(paths(function))
order = {"source": 0, "step": 1, "sanitizer": 2, "guard": 3, "sink": 4}
return Slice(tuple(sorted(result, key=lambda s: (s.line, order[s.role]))))
def render_slice(slice_or_none: Slice | None, reason: str) -> str:
"""Fixed framing; reason is the language, never free-form provenance text."""
prefix = "Data flow inside the function:"
if slice_or_none is None:
language = (
reason
if reason
in {
"python",
"javascript",
"typescript",
"tsx",
"c",
"cpp",
"java",
"php",
"go",
"ruby",
"rust",
"csharp",
"perl",
"kotlin",
"swift",
"scala",
"lua",
"groovy",
"shell",
"other",
}
else "other"
)
return f"{prefix} not analysed ({language} is not supported)."
if not slice_or_none.steps:
return f"{prefix} no source reaches a sink of this family."
return prefix + "\n" + "\n".join(f"L{s.line} {s.role}: {s.code}" for s in slice_or_none.steps)
+12
View File
@@ -70,6 +70,18 @@ recorded = "2026-10-01"
status = "development"
exclude_tiers = ["title"]
# Batch 2: advisory fixes whose changed function survives on the fixed side, for the six compiled families (most
# published after 2026-06-01). Same slice, cache layout and checks as batch 1; screening record
# benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest/record.json.
[[source]]
id = "advisory-fixes-2"
kind = "pairs"
file = "benchmarks/pairs/advisory-fixes-2/catalog.toml"
evaluation = ["benchmarks/pairs/advisory-fixes-2/verification.json"]
recorded = "2026-10-03"
status = "development"
exclude_tiers = ["title"]
[[source]]
id = "adjudications"
kind = "adjudications"
+4
View File
@@ -111,6 +111,9 @@ class ChangeContext(Contract):
# are not encoded or decoded by this field.
path_encoding: Literal["text", "filesystem-bytes-hex"] = "text"
line_correspondences: tuple[LineCorrespondence, ...] = ()
# Paths Git reports as added (status A) in head, in this context's path encoding. A rename
# target is not added. Only a lexical fact: novelty still needs complete head evidence.
added_paths: tuple[str, ...] = ()
def decode_path(self, path: str) -> str:
"""Return the filesystem spelling for matching ordinary question paths.
@@ -130,6 +133,7 @@ class ChangeContext(Contract):
*(s.path for s in self.spans),
*(p for r in self.renames for p in (r.base_path, r.head_path)),
*(p for m in self.line_correspondences for p in (m.base_path, m.head_path)),
*self.added_paths,
]
for path in paths:
try:
+18 -1
View File
@@ -96,6 +96,21 @@ class DeepSeekChatClient:
self._disable_thinking = disable_thinking
self.endpoint = endpoint
self.usage: list[dict[str, object]] = [] # one entry per call, for the round's cost meter
self._output_limit: int | None = None
def bounded(self, max_output_tokens: int) -> DeepSeekChatClient:
"""A single-attempt, output-capped copy for callers reserving a hard per-call budget.
Empty JSON retries belong to the caller's meter, and transport failures must
stop that caller before another potentially billable attempt.
"""
from dataclasses import replace
if max_output_tokens <= 0:
raise ValueError("max_output_tokens must be positive")
client = DeepSeekChatClient(replace(self._client, max_attempts=1), disable_thinking=self._disable_thinking, endpoint=self.endpoint)
client._output_limit = max_output_tokens
return client
def complete(
self,
@@ -117,7 +132,7 @@ class DeepSeekChatClient:
logprobs=logprobs,
temperature=temperature,
)
if json_object and not (response.content or "").strip():
if self._output_limit is None and json_object and not (response.content or "").strip():
response = self._call(
model=model,
messages=messages,
@@ -147,6 +162,8 @@ class DeepSeekChatClient:
temperature: float | None = None,
) -> ChatResponse:
extra: dict[str, object] = {}
if self._output_limit is not None:
extra["max_tokens"] = self._output_limit
if self._disable_thinking:
extra["thinking"] = {"type": "disabled"} # otherwise temperature is silently ignored
if temperature is not None:
+43 -20
View File
@@ -19,7 +19,7 @@ from .layout import is_test_path, is_vendored, layout_facts
FRONTENDS = {
"php": "php",
"javascript": "javascript",
"typescript": "typescript",
"typescript": "javascript",
"python": "python",
"java": "java",
"c": "c",
@@ -109,14 +109,19 @@ class PartitionGraph:
def _dispatch(self, kind: str, requests: Mapping[str, Mapping[str, object]], method: str) -> dict[str, list[object]]:
answers: dict[str, list[object]] = {}
languages = dict.fromkeys(self.path_languages.get(str(params.get("file", "")), "") for params in requests.values())
def graph_language(path: str) -> str:
language = self.path_languages.get(path, "")
return "javascript" if language == "typescript" else language
languages = dict.fromkeys(graph_language(str(params.get("file", ""))) for params in requests.values())
for language in languages:
graph = self.graphs.get(language)
if graph is None:
continue
if self.budget is not None and time.monotonic() >= self.budget.deadline_monotonic:
break
selected = {rid: params for rid, params in requests.items() if self.path_languages.get(str(params.get("file", ""))) == language}
selected = {rid: params for rid, params in requests.items() if graph_language(str(params.get("file", ""))) == language}
if not selected:
continue
batch = getattr(graph, method, None) or getattr(graph, "run_batch", None)
@@ -133,7 +138,12 @@ class PartitionGraph:
# Carried, not normalized: it names request ids and costs, never a path.
answers.setdefault(TIMING, []).extend(rows)
elif rid == "__census__" and isinstance(rows, list):
expected = set(next(p.paths for p in self.partitions if p.language == language))
expected = {
path
for p in self.partitions
if p.language == language or (language == "javascript" and p.language == "typescript")
for path in p.paths
}
normalized = self._normalize(rows)
names = [r.get("file_names") for r in normalized if isinstance(r, Mapping)]
named = bool(names) and all(isinstance(n, list) and all(isinstance(p, str) for p in n) for n in names)
@@ -234,10 +244,17 @@ def build_partitions(
graph.path_languages[relative] = languages[0]
elif languages:
boundaries.add("ambiguous_frontend_path")
if len(paths) > 1:
# JavaScript and TypeScript share one frontend input tree and graph. Keep
# original per-file languages for scoped coverage and property limitations.
groups: dict[str, list[str]] = {}
for language in paths:
key = "javascript" if language == "typescript" else language
groups.setdefault(key, []).append(language)
if len(groups) > 1:
boundaries.add("cross_partition_semantics_unresolved")
records: list[PartitionCoverage] = []
for language, sources in sorted(paths.items()):
for language, members in sorted(groups.items()):
sources = [source for member in members for source in paths[member]]
relative_paths = tuple(sorted(p.relative_to(root).as_posix() for p in sources))
frontend = FRONTENDS.get(language)
if frontend is None:
@@ -247,7 +264,7 @@ def build_partitions(
continue
scratch = Path(tempfile.mkdtemp(prefix="ousast-input-"))
graph.roots[language] = scratch
size = 0
sizes: dict[str, int] = {}
complete = True
for source in sources:
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
@@ -268,25 +285,31 @@ def build_partitions(
complete = False
break
output.write(chunk)
size += len(chunk)
relative = source.relative_to(root).as_posix()
sizes[relative] = sizes.get(relative, 0) + len(chunk)
if not complete:
break
built = build(scratch, frontend) if complete else None
if built is not None:
graph.graphs[language] = built
graph.unparsed += tuple(graph._normalize(list(getattr(built, "unparsed", ()))))
record_limits: tuple[str, ...] = ("runtime_unspecified",) if language in {"javascript", "typescript"} else ()
if language == "typescript":
record_limits += ("typescript_property_support_unvalidated",)
boundaries.add("typescript_property_support_unvalidated")
# About TypeScript answers. YesWiki is a PHP application that ships a few `.ts` files, and this
# boundary made all 1,128 of its PHP questions unresolved.
owned_languages.setdefault("typescript_property_support_unvalidated", set()).add("typescript")
if language in {"c", "cpp"}:
record_limits += ("c_bounds_arithmetic_unmodeled",)
records.append(
PartitionCoverage(language, frontend, relative_paths, size, "built" if built is not None else "build_failed", record_limits)
)
for language in sorted(members):
relative_paths = tuple(sorted(p.relative_to(root).as_posix() for p in paths[language]))
source_size = sum(sizes.get(p, 0) for p in relative_paths)
record_limits: tuple[str, ...] = ("runtime_unspecified",) if language in {"javascript", "typescript"} else ()
if language == "typescript":
record_limits += ("typescript_property_support_unvalidated",)
boundaries.add("typescript_property_support_unvalidated")
# About TypeScript answers. YesWiki is a PHP application that ships a few `.ts` files, and this
# boundary made all 1,128 of its PHP questions unresolved.
owned_languages.setdefault("typescript_property_support_unvalidated", set()).add("typescript")
if language in {"c", "cpp"}:
record_limits += ("c_bounds_arithmetic_unmodeled",)
records.append(
PartitionCoverage(
language, frontend, relative_paths, source_size, "built" if built is not None else "build_failed", record_limits
)
)
from dataclasses import replace
graph.partitions = tuple(
+2 -1
View File
@@ -24,6 +24,7 @@ from collections.abc import Mapping, Sequence
from dataclasses import dataclass, replace
from pathlib import Path
from ..semantic.facts import TAINT_LANGUAGE_ALIAS
from .contracts import AffectedRelationship, ChangeContext, ExecutionBudget, QuestionIdentity
from .specs import config_specs, dominance_specs, taint_specs
@@ -41,7 +42,7 @@ _ACCESS_RANK: dict[str, float] = {
# Languages whose facts the model layer carries. Anything else yields no region rather than an empty scan
# that looks like a clean bill of health.
_LANGUAGES = {"python", "javascript", "typescript", "java", "c", "c_cpp", "php"}
_NORMALISE = {"typescript": "javascript", "c_cpp": "c"}
_NORMALISE = {**TAINT_LANGUAGE_ALIAS, "c_cpp": "c"}
# What a CPG calls a file's top-level code. A module body is a method like any other to the engine, and
# naming it here is what lets a region ask about the code that is not inside any function.

Some files were not shown because too many files have changed in this diff Show More