mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 04:29:27 +00:00
Compare commits
52
Commits
@@ -0,0 +1,43 @@
|
||||
name: Docs
|
||||
|
||||
# Builds the MkDocs site and publishes it to GitHub Pages on every push to main that touches it.
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths: ["docs/**", "mkdocs.yml", "README.md", ".github/workflows/docs.yml"]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: pages
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
enable-cache: true
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version-file: .python-version
|
||||
- run: uv sync --extra docs
|
||||
- run: uv run mkdocs build --strict
|
||||
- uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: site
|
||||
deploy:
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
@@ -119,6 +119,19 @@ rule hits.
|
||||
2. Without a trained model available the tools fall back to today's behaviour and say so in the report.
|
||||
3. Hand-written rules remain as signal producers; changing their enabled/shadow status no longer changes what is
|
||||
reported unless the engine's decision changes.
|
||||
4. *(Amended 2026-10-03, maintainer decision; refines the BLOCK rule of 7.3 and 7.4. Tested by exp-004.)* BLOCK
|
||||
precision is demonstrated pooled across families, not per family:
|
||||
- **Normalisation**, declared before any run: each family's raw score is put on one scale by a per-family Platt
|
||||
map (`learn.calibrate.fit_map` with its floors) fitted on training folds only. A family the training folds
|
||||
cannot map does not block on that fold.
|
||||
- **t_block**: one threshold on the normalised score for all families, chosen on training folds as the lowest
|
||||
one whose Wilson 95% lower bound on precision, over the training flags of all families together, is >= 0.95.
|
||||
It is reported by the Wilson 95% lower bound over all held-out flags of all families, before the floor.
|
||||
- **Family floor**: a family may block only if its own held-out flags have zero errors and number at least F
|
||||
(F = 10, reasoned in the exp-004 registration); otherwise that family stays ADVISORY.
|
||||
- **t_quiet**: per family, the highest threshold keeping >= 90% of the paired positives at or above it (chosen
|
||||
on training folds, its held-out recall reported); findings below it are silent.
|
||||
- No held-out label moves a threshold or a normalisation map; candidates whose verdict is `unsure` never block.
|
||||
|
||||
### Requirement 7: Built for unseen repositories -- a repo safety net
|
||||
|
||||
@@ -136,7 +149,8 @@ rule hits.
|
||||
ADVISORY, trading precision for recall; both reported with out-of-repository intervals. The decision unit is a
|
||||
candidate in changed code (`pre-push` delta), with whole-repository scans as a secondary mode.
|
||||
4. Calibration is checked on held-out repositories; where it does not hold, only ADVISORY is offered for that
|
||||
family.
|
||||
family. *(Amended 2026-10-03: once exp-004 adopts the pooled gate of 6.4, BLOCK precision is shown by its held-out
|
||||
bound and family floor, and calibration is reported, not required.)*
|
||||
5. Any adaptation to a user's own repository (facts reuse, their dismissals) is opt-in, separate from the
|
||||
generalisation model, and never contributes to its reported numbers.
|
||||
|
||||
|
||||
@@ -254,6 +254,13 @@ containerd; `Dockerfile:1-7` already argues against it).
|
||||
a 3 Gi actor lands only on `engine-pool`. If Substrate does not place by resource fit, the fallback is a
|
||||
separate atespace with its own pool if Substrate scopes pools that way; failing that, the engine stays
|
||||
`alerts-engine` on a Docker host and the k3s engine becomes a plain Kubernetes Job outside ax, recorded as such.
|
||||
**Probe outcome (task 1.5, 2026-10-02, `benchmarks/measurements/2026-10-02-k8s-01-profile/`): `placement: any`.**
|
||||
With `ousast-pool` at one 1536Mi worker and a `probe-pool` of one 3Gi worker both ACTIVE, a `repo-facts` actor
|
||||
requesting 2560Mi (limit 3Gi) was placed on the 1536Mi `ousast-pool` worker (`kubectl ate get workers`: ACTORS
|
||||
1/1000 there, 0/1000 on `probe-pool`), and so was the 256Mi actor. Substrate does not place by resource fit, so
|
||||
group 5 takes the fallback: the engine pool must be the only pool its actors can land on. Before 5.2, check whether
|
||||
Substrate scopes pools by atespace (an `engine` atespace with `engine-pool` as its only pool); if it does not, the
|
||||
k3s engine runs as a plain Kubernetes Job outside ax and `alerts-engine` stays the kind path.
|
||||
- **`alerts-engine` stays** for development on a host with Docker (kind profile), its help text and
|
||||
`docs/plane.md` row marked "development only; the plane task is `engine`" (Req 4.3). Generated Runs choose
|
||||
`engine` when the profile's `images` file has an engine image, else leave `alerts` for `alerts-engine`.
|
||||
|
||||
@@ -10,7 +10,7 @@ its kind proof and the record it writes under `benchmarks/measurements/<date>-k8
|
||||
go-ahead** with a ceiling; steps that need a hand on a cluster that does not exist yet are marked **manual**.
|
||||
|
||||
- [ ] 1. Profile and literals (design section 1; open-question checks 2, 3, 7)
|
||||
- [ ] 1.1 `PlaneProfile`
|
||||
- [x] 1.1 `PlaneProfile`
|
||||
- `src/openultrasast/plane/profile.py`: frozen dataclass with the fields of design section 1; `load_profile()`
|
||||
reads `ops/k8s/profiles/<OUSAST_PLANE_PROFILE>.toml` (a name or a path; default `kind`), then applies
|
||||
`OUSAST_<FIELD>` overrides; rejects an unknown key, a secret-looking key (`*_KEY`, `*_SECRET`), an `exec`
|
||||
@@ -19,7 +19,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
|
||||
- `tests/test_plane_profile.py`: defaults; file then env precedence; each rejection names the field; both
|
||||
committed profiles load; no field holds a value matching a credential pattern.
|
||||
- _Requirements: 1.1_
|
||||
- [ ] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
|
||||
- [x] 1.2 Literals out of `doctor.py`, `egress.py`, `router.py`, `generate.py`
|
||||
- `doctor(profile)`: context from `profile.kube_context`; the registry check resolves the `images` file's runner
|
||||
reference (`docker manifest inspect` for a remote registry, `/v2/_catalog` for a local one); `open_store`
|
||||
verifies `profile.memory`; prints every configured address (Req 1.2). `Egress(context=...)`,
|
||||
@@ -29,46 +29,46 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
|
||||
profile's context, never a built one.
|
||||
- Evidence: `grep -rn "kind-" src/openultrasast/plane` matches only comments citing the kind profile.
|
||||
- _Requirements: 1.1, 1.2_
|
||||
- [ ] 1.3 Host-literal scan test
|
||||
- [x] 1.3 Host-literal scan test
|
||||
- `tests/test_plane_literals.py`: pattern built at runtime (as `tests/test_removed_plane_references.py` does)
|
||||
over `src/`, `plane/`, `ops/` minus `ops/ax/up.sh` and `ops/k8s/profiles/`, and `docs/`, for `kind-ousast`,
|
||||
`localhost:5001`, `172.19.`, `127.0.0.1:` and `port-forward` outside `router.py` and lines tagged
|
||||
`ax-tunnel`; fails listing `path:line`.
|
||||
- Evidence: the test passes on this commit and fails when a literal is reinserted (shown once in the record).
|
||||
- _Requirements: 1.3_
|
||||
- [ ] 1.4 `up.sh` by profile
|
||||
- [x] 1.4 `up.sh` by profile
|
||||
- `ops/ax/up.sh`: `KO_DOCKER_REPO` and the context from `kind.toml` (read with `python -m
|
||||
openultrasast.plane.profile --print registry|kube_context`); writes `kind-images.json` instead of
|
||||
`runner-image`; `generate.repin_templates` accepts that file (runner key only until task 4.3).
|
||||
- Evidence: `ops/ax/up.sh` reruns idempotently on kind; `ops/ax/smoke-run.sh` passes.
|
||||
- _Requirements: 1.1, 2.4_
|
||||
- [ ] 1.5 Placement probe (open question 2, decided before the engine work)
|
||||
- [x] 1.5 Placement probe (open question 2, decided before the engine work) -- result: `placement: any`
|
||||
- `ops/ax/probe-placement.sh`: applies a second WorkerPool `probe-pool` (1 replica, 3 Gi) from the design's
|
||||
engine template, then one model-free `repo-facts` Task with `resources.requests.memory: 2560Mi`, and records
|
||||
with `kubectl ate get workers` which worker took the actor; then the same Task with 256Mi. Removed after.
|
||||
- Evidence: `record.json` says whether Substrate places by resource fit (`placement: fit|any`). `any` selects
|
||||
the design's fallback for group 5 and is written into `design.md` section 4 before group 5 starts.
|
||||
- _Requirements: 4.1_
|
||||
- [ ] 1.6 Reconciler bound (open question 7)
|
||||
- [x] 1.6 Reconciler bound (open question 7) -- 489 lines after 1.2; projected 514 during group 2: the bound moves to 550 in 2.4
|
||||
- Record `wc -l src/openultrasast/plane/reconciler.py` (477 at 17ac7ad) and the lines task 2.4 may add
|
||||
(receiver wiring removed: `-12`, `Delivery` calls: `+8` estimated). If the estimate crosses 500, the bound
|
||||
in `tests/test_plane_reconciler.py:803` moves to 550 in task 2.4 with the reason in the assertion message.
|
||||
- Evidence: the numbers in `record.json`.
|
||||
- _Requirements: 2.1_
|
||||
- [ ] 1.7 gvisor tarball reachability (open question 3) **manual**
|
||||
- [x] 1.7 gvisor tarball reachability (open question 3) **manual** -- documented as planned in `docs/deployment.md` B.1; the live check waits for k3s
|
||||
- `docs/deployment.md` (B.1, planned): the k3s node must fetch
|
||||
`gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd` (`substrate/manifests/ate-install/sandboxconfig-gvisor.yaml`)
|
||||
or a mirrored copy named in a copied SandboxConfig; the check is `kubectl -n ate-system logs` of the worker
|
||||
pod's atelet fetch line, run by hand on the day k3s exists. Labelled planned until then.
|
||||
- _Requirements: 7.2_
|
||||
- [ ] 1.8 Kind proof and record 01
|
||||
- [x] 1.8 Kind proof and record 01
|
||||
- `ousast plane doctor --profile kind` passes every line; `ousast plane doctor --profile k3s` fails only on the
|
||||
unreachable context (the expected line); `test_plane_literals` 0 offenders; the probe of 1.5; the bound of 1.6.
|
||||
- `benchmarks/measurements/<date>-k8s-01-profile/record.json`.
|
||||
- _Requirements: 1.1, 1.2, 1.3, 7.1_
|
||||
|
||||
- [ ] 2. Store delivery, receiver-less egress (design section 3)
|
||||
- [ ] 2.1 Presigned URLs and the store run directory
|
||||
- [x] 2.1 Presigned URLs and the store run directory
|
||||
- `src/openultrasast/plane/delivery.py`: `Delivery(store, run)` mints presigned PUT for
|
||||
`runs/<run>/<task>/output.tar` and GET for `runs/<run>/<producer>/<artifact>` (boto3 `generate_presigned_url`
|
||||
on `S3Client.sdk`, expiry `OUSAST_TASK_TIMEOUT + 600`); `delivered(task)` HEADs the tar; `collect(task)`
|
||||
@@ -78,7 +78,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
|
||||
- `tests/test_plane_delivery.py`: against a local S3-shaped fake (PUT/GET/HEAD by key); URLs never appear in
|
||||
logs; expiry; member rejection; declared-only re-put; state seeding.
|
||||
- _Requirements: 3.1, 2.1_
|
||||
- [ ] 2.2 Runner: `delivery` in the start body
|
||||
- [x] 2.2 Runner: `delivery` in the start body
|
||||
- `src/openultrasast/plane/runner.py`: `StartGate.offer` validates an optional `delivery {put, inputs}` (https,
|
||||
hostname, no echo); `_deliver_output` PUTs the tar to `delivery.put` (same retry and `DeliveryError`
|
||||
semantics); `fetch_inputs` GETs from `delivery.inputs[name]`; `OUSAST_ARTIFACT_URL` and `OUSAST_ARTIFACT_DIAL`
|
||||
@@ -86,14 +86,14 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
|
||||
- `tests/test_plane_runner.py`: accepted body with `delivery`; 400 on an `http` or IP URL; PUT to a local
|
||||
server; inputs fetched from presigned-shaped URLs; the marker and retry-only boot unchanged.
|
||||
- _Requirements: 3.1_
|
||||
- [ ] 2.3 Egress without the receiver
|
||||
- [x] 2.3 Egress without the receiver
|
||||
- `src/openultrasast/plane/egress.py`: `policy_for(task, workspaces, model, store_host)` emits one
|
||||
`tlsPassthrough` rule on 443 over the sorted union of store host, Git hosts, Model hosts; no `http` rule; an
|
||||
IP store host raises naming `S3_ENDPOINT`.
|
||||
- `tests/test_plane_egress.py`: for every committed template under `plane/tasks/` the rendered policy is the
|
||||
expected set; the IP refusal.
|
||||
- _Requirements: 3.2_
|
||||
- [ ] 2.4 Reconciler on `Delivery`
|
||||
- [x] 2.4 Reconciler on `Delivery`
|
||||
- `src/openultrasast/plane/reconciler.py`: `run()` opens the store first (no fallback), builds `Delivery`,
|
||||
passes `delivery` to `router.starter`/`start_task` (body extended in `router.py`), `_await` on
|
||||
`Delivery.delivered`, outcome from the collected `summary.json`; the receiver thread is kept behind
|
||||
@@ -101,7 +101,7 @@ go-ahead** with a ceiling; steps that need a hand on a cluster that does not exi
|
||||
- `tests/test_plane_reconciler.py`: the ax fake run completes through the delivery fake; a rerun from an empty
|
||||
results root resumes from the store's `state.json`.
|
||||
- _Requirements: 2.1, 3.1_
|
||||
- [ ] 2.5 Kind proof and record 02
|
||||
- [x] 2.5 Kind proof and record 02
|
||||
- `ops/ax/smoke-run.sh` with `OUSAST_DELIVERY=store`: `facts.json` and `summary.json` arrive through the store;
|
||||
the actor's policy read back has one TLS rule naming the store host; `ousast plane status ax-e2e-smoke`
|
||||
prints the table. Instrument: the tar's byte size from the store equals the extracted files' total.
|
||||
|
||||
@@ -147,6 +147,43 @@ with Express-style routes and middleware, including vulnerable, fixed and benign
|
||||
frontend/framework adaptations and new abstract capabilities, and shall report C/C++ envelope or
|
||||
unsupported-property results separately from demonstrated vulnerability detection.
|
||||
|
||||
## Requirement 9: Operate as an advisory safety net on any repository
|
||||
|
||||
Added 2026-10-02, maintainer-approved, from the operational-readiness audit. The maintainer's
|
||||
definition: a user installs the hook on ANY repository and gets honest, advisory-only results;
|
||||
nothing ever blocks a push unless blocking was explicitly chosen.
|
||||
|
||||
9.1 (added 2026-10-02, maintainer-approved) When the installed hook cannot run the check, because the
|
||||
`ousast` executable is missing or an argument or setting is invalid, the hook shall allow the push and
|
||||
print one line saying the check did not run and how to fix it; only an explicit `--mode blocking`
|
||||
opt-in may turn an analysis result into a rejected push.
|
||||
|
||||
9.2 (added 2026-10-02, maintainer-approved) When a check is skipped or incomplete, the terminal output
|
||||
shall name each distinct reason in plain words, one line per reason, with repository paths decoded and
|
||||
no opaque hash tokens; a language with no engine frontend and no quick rules shall be named with its
|
||||
file count; a first push of a new branch shall say there was no base to compare against and what was
|
||||
checked instead. The JSON artifact keeps the raw reason values.
|
||||
|
||||
9.3 (added 2026-10-02, maintainer-approved) Before the engine, within the same deadline, the Safety Net
|
||||
shall run the shipped quick pattern rules over the head side of the changed lines and show their
|
||||
findings as advisory pattern matches, labeled separately from admitted alerts. This amends 3.2 for this
|
||||
tier only: quick-rule findings are shown, never admitted, and never block.
|
||||
|
||||
9.4 (added 2026-10-02, maintainer-approved) When a finding's operation lies in a file that is new in the
|
||||
head revision, the comparison shall classify it as new rather than unknown; an engine finding whose
|
||||
novelty is established but whose capability is not yet qualified shall be shown as an advisory engine
|
||||
finding, not an admitted alert. Genuinely unknown comparisons keep their vetoes.
|
||||
|
||||
9.5 (added 2026-10-02, maintainer-approved) When the engine cannot finish within the deadline, one line
|
||||
shall say so; on explicit request (`--engine background`) the Safety Net shall start the engine
|
||||
detached after the push, record its result in the artifact directory, and show it on the next run as
|
||||
the engine result for that head from the previous push.
|
||||
|
||||
9.6 (added 2026-10-02, maintainer-approved) The hook installer and remover shall ship in the wheel
|
||||
(`ousast pre-push install|uninstall`), refuse to overwrite an existing hook unless forced, and a
|
||||
hook-callable container wrapper shall pass the push input and artifact path through; the documentation
|
||||
shall give a first-run guide, a glossary of skip reasons, a CI recipe and help text for every flag.
|
||||
|
||||
## Accepted experimental acceptance profile
|
||||
|
||||
These values are the initial implementation/evaluation profile accepted for task generation, not
|
||||
|
||||
@@ -475,7 +475,7 @@ Preserve all completed task evidence below.
|
||||
- _Depends: 13.1_
|
||||
- _Requirements: 6.2, 6.3, 8.1, 8.2_
|
||||
|
||||
- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4)
|
||||
- [x] 14. Ask every declared family whether it establishes anything (precondition for M4)
|
||||
- Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing.
|
||||
- [x] 14.1 Census the seven declared families on NodeGoat and VAmPI
|
||||
- Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only.
|
||||
@@ -575,7 +575,7 @@ Preserve all completed task evidence below.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_
|
||||
- _Requirements: 6.2, 6.3, 8.1, 8.2_
|
||||
- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds
|
||||
- [x] 13.4 Let a question inside one partition complete, whatever else the repository holds
|
||||
- **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan.
|
||||
- The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote.
|
||||
- **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it.
|
||||
@@ -587,6 +587,7 @@ Preserve all completed task evidence below.
|
||||
- _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical.
|
||||
- _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`.
|
||||
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
|
||||
- _Checked 2026-10-02 per the operational-readiness audit:_ implemented and verified on PMPro (120 of 127, above). 13.3 stays open on its own blocker; this box was left unchecked only because it waits on it.
|
||||
- _Depends: 13.3_
|
||||
- _Requirements: 2.1, 2.2, 6.3, 8.1_
|
||||
|
||||
@@ -619,6 +620,54 @@ Preserve all completed task evidence below.
|
||||
- _Requirements: 8.1, 8.2_
|
||||
|
||||
|
||||
- [x] 17. Operational readiness (2026-10-02)
|
||||
- **Named by the 2026-10-02 operational-readiness audit**, maintainer-approved as the first increment. The maintainer's definition: a user installs the hook on ANY repository and gets honest, advisory-only results; nothing ever blocks a push. The audit found the plumbing sound (snapshots, resolution, deadline accounting, advisory exit 0, the JSON artifact) and five things in the way: the hook can block a push, quick rules are not part of pre-push, the engine cannot finish a 26-file repository inside 30 s, the terminal hides every skip reason, and there is no supported install path without the source checkout. Before numbers for every item are the audit's live runs on records (Python), serve-static (JavaScript) and unrolled/secure (Go), each with one planted commit.
|
||||
- This group is plumbing and honesty. It enables no capability, edits no rule for a planted sample, and leaves graph reuse (the engine speed work decided in 11.6) as a later task.
|
||||
- _Done 2026-10-02:_ all seven subtasks done. The engine still does not fit the 30 s hook deadline (graph reuse, the 11.6 direction, stays a later task), and plane verdicts after the push were not in this increment.
|
||||
- _Requirements: 9.1–9.6_
|
||||
- [x] 17.1 Never block a push the user did not choose to block
|
||||
- The installed hook exits 0 with one line when `ousast` is missing (the check did not run, and how to install). In hook (stdin) mode, an argument or usage error (an invalid `OUSAST_PUSH_DEADLINE`, a cache directory inside the repository) is advisory: one line and exit 0. `--mode blocking` stays the explicit opt-in that can reject a push, and keeps its usage errors. The prior-hook input-retention failure keeps exit 1, because it is the chained hook's verdict that cannot be obtained, not ours.
|
||||
- Files: `ops/pre-push`, `src/openultrasast/cli.py`. Tests: `tests/test_push_hook.py` (a push with `ousast` absent from PATH succeeds and prints the line; a bad deadline in hook mode exits 0; blocking mode keeps exit 2).
|
||||
- Evidence: the audit's `hook.sh` push 1 (rejected before) and `fail.sh` (rc=2 before) rerun with exit 0.
|
||||
- _Done 2026-10-02:_ the hook prints one line and exits 0 (chaining a prior hook if present) when `ousast` (or `OUSAST_COMMAND`) is missing; in hook mode without `--mode blocking`, argparse/usage errors and unexpected exceptions print one line and exit 0. Tests: `test_missing_ousast_never_blocks_the_push`, `test_hook_mode_usage_errors_are_advisory_unless_blocking` (blocking keeps exit 2), and `test_cli_refuses_the_flag_outside_explicit_replay` updated (the experimental flag is still refused in hook mode, now advisory). Gates: 1987 passed, 42 skipped; ruff, format, mypy clean.
|
||||
- _Requirements: 9.1, 5.3, 5.4_
|
||||
- [x] 17.2 Name every skipped check in plain words
|
||||
- One terminal line per distinct coverage reason: hex-encoded paths decoded, sha256-suffixed reasons grouped with a count, JSON unchanged. "language not covered: <lang> (<n> files)" for a language with no engine frontend and no quick rules, in both `pre-push` and `scan`. A first push of a new branch says there is no base and what was checked instead. `reports.py` gains lines for the eight scan degradations `cli.py` records and the report did not render (`learning_endpoint_unavailable`, `cpg_unavailable`, `facts_unavailable`, `policy_invalid`, `model_withheld_execution_budget`, `max_findings_exceeded`, `hunter_model_unavailable`, `sandbox_unavailable`).
|
||||
- Files: `src/openultrasast/push/report.py`, `src/openultrasast/push/explain.py` (new), `src/openultrasast/reports.py`, `src/openultrasast/cli.py`. Tests: `tests/test_push_report.py`, `tests/test_reports.py`.
|
||||
- Evidence: the three repositories' terminal output names engine-missing, Go not covered and the new branch.
|
||||
- _Done 2026-10-02:_ new `push/explain.py` turns every raw coverage reason into one plain `Skipped:` line per distinct kind (hex paths decoded, sha256 tokens dropped and counted, unknown reasons still printed); the existing one-line `Notice:` keeps its next action and gains an engine-missing variant. `language_not_covered:<lang>:<changed>:<total>` is recorded for a changed language with no engine frontend and no quick rules; `ousast scan` prints `notice: language not covered: go (N files)` and its report says so. `reports.py` renders the eight previously dropped degradations and prints any reason it has no sentence for. The hook passes `OUSAST_COMPARISON_BASE`. The new-branch line names the base it lacks; what was checked instead lands with the quick tier (17.3). Tests: `tests/test_push_explain.py` (6), two in `tests/test_reports.py`; module-audit manifest regenerated for the new module. Gates: 1995 passed, 42 skipped; ruff, format, mypy clean.
|
||||
- _Requirements: 9.2, 5.2, 8.2_
|
||||
- [x] 17.3 Run the quick rules as the fast first tier
|
||||
- Inside the deadline and before the engine, the shipped quick rules run over the head side of the changed files and keep the findings on changed lines or in changed functions. They print as advisory pattern matches, labeled apart from admitted alerts, recorded in the artifact's `quick_tier`, and never block.
|
||||
- Files: `src/openultrasast/push/quick.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_quick.py` (new).
|
||||
- Evidence: serve-static's planted `eval` in the new `debug.js` is shown; whether records' planted `%`-formatted SQL is shown is recorded as is, with no rule edits for it.
|
||||
- _Done 2026-10-02:_ new `push/quick.py` runs the shipped quick rules (no shadow rules, no model) over the changed head files inside `_prepare` under the shared deadline, before the engine, keeping matches on a changed line or inside a changed function (entry-point ranges); it records `files`/`bytes_read` as proof of input. The terminal shows them under "Quick rules: N pattern match(es) on changed code ... Advisory: not verified by the engine."; the artifact keeps `quick_tier`; admission and push disposition are untouched. A new branch runs the quick tier over its last commit's changes and says so. `_analyze` gained `engine="off"`. On the audit's serve-static planted commit the `eval` in the new `debug.js` is now shown (js-eval, js-xss-response at debug.js:2, ~1.1 s on the host); the planted `%`-formatted SQL in records is NOT matched by any quick rule (coverage gap, no rule edited). Tests: `tests/test_push_quick.py` (3). Gates: 1998 passed, 42 skipped; ruff, format, mypy clean.
|
||||
- _Requirements: 9.3_
|
||||
- [x] 17.4 Classify a file new in head as new, not unknown
|
||||
- `ChangeContext` records the paths Git reports as added. An operation in an added file, when no add/delete pairing is unresolved and its head evidence is complete, is `new` with reason `file_added_in_head`, and admission accepts that reason as change evidence. An engine finding with established novelty whose only vetoes are capability vetoes renders as an advisory engine finding, not an alert. Every other unknown comparison keeps its veto.
|
||||
- Files: `src/openultrasast/model/contracts.py`, `src/openultrasast/push/snapshot.py`, `src/openultrasast/push/policy.py`, `src/openultrasast/push/report.py`. Tests: `tests/test_push_policy.py`, `tests/test_push_snapshot.py`, `tests/test_push_report.py`.
|
||||
- Evidence: unit tests on the comparison and admission; a live engine run if the host allows one (free memory >= 3 GB).
|
||||
- _Done 2026-10-02:_ `ChangeContext.added_paths` records Git status-A paths (rename targets excluded). In `_compare_evidence`, after every existing context/head/deadline check, an operation in an added file is `new`/`file_added_in_head` when the claim is entailed, undischarged and has change evidence, and no base answer holds the same operation (else `operation_moved_within_change`); an unpaired add/delete still blocks as `change_context_unresolved`. Admission accepts `(new, file_added_in_head)`; with the empty registry such a candidate's only veto is `capability_unavailable`, and the terminal now shows it under "Engine: N finding(s) this push introduced. Advisory ... not an alert." Any comparison, evidence or location veto keeps a candidate out of that line. Tests: `tests/test_push_new_file.py` (9). Live engine evidence depends on host memory (17.7). Gates: 2007 passed, 42 skipped; ruff, format, mypy clean.
|
||||
- _Requirements: 9.4, 3.4_
|
||||
- [x] 17.5 Say when the engine cannot finish, and offer it in the background
|
||||
- One line when the engine did not finish (deadline, engine not installed). `--engine background` starts the engine detached after the quick tier, writes its result under the artifact directory and, on the next run, prints "engine result for <head> from the previous push". One background engine at a time; `--engine off` skips it with a stated line. No speed work.
|
||||
- Files: `src/openultrasast/push/background.py` (new), `src/openultrasast/push/runner.py`, `src/openultrasast/cli.py`, `ops/pre-push`. Tests: `tests/test_push_background.py` (new).
|
||||
- Evidence: a hook push with `OUSAST_PUSH_ENGINE=background` followed by a second push showing the previous result.
|
||||
- _Done 2026-10-02:_ `--engine inline|background|off` (hook: `OUSAST_PUSH_ENGINE`) and `--background-deadline` (default 900 s). `off` and `background` run the quick tier and record `engine_off`/`engine_background` with a plain skip line; the deadline line names the deadline and the background option. New `push/background.py` starts one detached explicit replay (`start_new_session`, stdin closed, output to a log) per artifact directory and repository (lock file with a live-pid check: a second start reports busy), writes its artifact under `<artifact dir>/engine/<repo key>/<head>.json`, and the next run prints "Engine result for <head> from the previous push: ..." once, then marks it shown. Not usable through the Docker wrapper (the container exits with the hook); documented in 17.6. Tests: `tests/test_push_background.py` (4, one launching a real detached run). Gates: 2011 passed, 42 skipped; ruff, format, mypy clean.
|
||||
- _Requirements: 9.5, 4.1, 4.2_
|
||||
- [x] 17.6 Install, Docker and documentation for a first-time user
|
||||
- `ousast pre-push install|uninstall [--force] [--docker] [REPO]` ships the hook as package data, honours `core.hooksPath`, refuses an existing hook unless `--force` (which chains it as `pre-push.before-ousast`), and works from a pip install. `ops/ousast-docker` is a hook-callable script (not a shell function) that passes stdin and the artifact path through. The `openultrasast:dev` image is rebuilt only if `df -h /` shows at least 4 GB free; otherwise the rebuild command is documented and the record says it was not rebuilt. `docs/scanning.md` gains a first-run guide, a glossary of skip reasons, a GitHub Actions recipe and full `--help` text; README tiers 1 and 2 updated, pulling `ghcr.io/norandom/openultrasast:2.0.1` where the wrapper allows.
|
||||
- Files: `src/openultrasast/push/hook/pre-push`, `src/openultrasast/push/install.py` (new), `src/openultrasast/cli.py`, `ops/ousast-docker`, `docs/scanning.md`, `README.md`, `ops/README.md`. Tests: `tests/test_push_install.py` (new), `mkdocs build --strict`.
|
||||
- Evidence: install from the package into a scratch clone, push through it, uninstall restores the prior hook.
|
||||
- _Done 2026-10-02:_ `ousast pre-push install|uninstall [REPO] [--force] [--docker]` (`push/install.py`) installs the hook shipped as package data (`openultrasast/push/hook/pre-push`, byte-identical to `ops/pre-push`, verified in the built wheel), honours `core.hooksPath` without editing it, refuses an existing hook unless `--force` (ours is replaced; a foreign hook moves to `pre-push.before-ousast` and is chained), and uninstall removes only a hook it wrote and restores the chained one. `ops/ousast-docker` (also package data; `--docker` installs it beside the hook, which then uses it) is a script that mounts the repository read-only and the artifact directory at their own absolute paths, passes stdin with `-i`, runs as the user with no network and `--memory 3g`, defaults to `ghcr.io/norandom/openultrasast:2.0.1`, never pulls inside a hook, and exits 0 unless `OUSAST_PUSH_MODE=blocking`. The hook passes `--engine`/`--comparison-base` only when set, so an older ousast or image still runs. Every pre-push flag has help text; `docs/scanning.md` has the first-run guide, settings table, new-branch and background notes, the artifact keys, a skip-reason glossary (a test keeps it in sync with `push/explain.py`), a GitHub Actions recipe and the full `--help` text; README tiers 1 and 2 and `ops/README.md` updated (pull and tag the GHCR image instead of building). **Image not rebuilt:** `df -h /` showed 1.1 GB free (< 4 GB); rebuild with `docker build -t openultrasast:dev .`. Tests: `tests/test_push_install.py` (11) and the glossary test. Gates: 2022 passed, 42 skipped; ruff, format, mypy clean; `mkdocs build --strict` exit 0.
|
||||
- _Requirements: 9.6, 7.4_
|
||||
- [x] 17.7 Prove it on the audit's three repositories
|
||||
- Rerun records, serve-static and unrolled/secure (same planted commits) with the new build in quick mode and with the hook installed, plus one Docker engine run on serve-static if free memory >= 3 GB and free disk >= 1 GB. Prove each run read its input (files, bytes).
|
||||
- Files: `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json`. Evidence: before (the audit) and after: exit codes, wall time against the deadline, findings shown, every skip line printed. A planted sample still missed is a coverage gap, recorded, never a rule edit.
|
||||
- _Done 2026-10-02:_ `benchmarks/measurements/2026-10-02-pre-push-operational-1/record.json` (before = the audit, after = this build installed from its wheel into a scratch venv). 10 hook pushes and 6 replays, all exit 0, 0.69-1.21 s against 30 s on the host; the quick tier read exactly the changed file and every head snapshot read the full tree (records 43,577 B, serve-static 61,525 B, secure 103,023 B). The planted JS `eval` in the new `debug.js` surfaces: quick tier on the host, and as two advisory engine findings classified `new`/`file_added_in_head` in one Docker engine run (190.9 s against a 300 s deadline; MemAvailable 3.28 GB, 1.16 GB disk). **The planted Python `%`-formatted SQL is still missed**: no quick rule matches it and the host has no engine. Recorded as a coverage gap, no rule edited. Go is named as not covered. The proof caught a wrong install hint (`pip install openultrasast`, not on PyPI), corrected in the hook in this commit.
|
||||
- _Requirements: 9.1–9.6, 6.3_
|
||||
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.
|
||||
|
||||
@@ -1,38 +1,53 @@
|
||||
# OpenUltraSAST
|
||||
|
||||
OpenUltraSAST is an experimental static security analyser meant to work as a safety net on
|
||||
repositories it has never seen: it scans a checkout, or the commits a `git push` would publish,
|
||||
and separates what it can prove from what it merely suspects. A scan combines language-scoped
|
||||
pattern rules, a tree-sitter overlay, a Joern code property graph and, optionally, a model that is
|
||||
asked only the question the graph cannot settle; every finding carries the evidence that earned
|
||||
its place. A scan that could not look at something records a degradation instead of reporting a
|
||||
clean result. `ousast` is the interface; model work over whole repositories runs on a separate
|
||||
agentic plane (google/ax), and a learned decision engine is in development. Neither is needed
|
||||
for a scan.
|
||||
OpenUltraSAST is an experimental static security analyser. It is meant to be a safety net on
|
||||
repositories it has never seen. It scans a checkout, or the commits a `git push` would publish. It
|
||||
keeps what it can prove apart from what it only suspects.
|
||||
|
||||
This file is the front door; each section links to its page of the documentation site (`docs/`,
|
||||
`mkdocs.yml`; `uv sync --extra docs && uv run mkdocs serve`).
|
||||
A scan combines up to four parts:
|
||||
|
||||
- language-scoped pattern rules;
|
||||
- a tree-sitter overlay (a syntax tree of each file, built by the tree-sitter parser);
|
||||
- a Joern code property graph (a graph of the program's syntax, control flow and data flow);
|
||||
- optionally, a model. It is asked only the question the graph cannot settle.
|
||||
|
||||
Every finding carries the evidence that earned its place. When a scan could not look at
|
||||
something, it records a degradation. It does not report a clean result.
|
||||
|
||||
`ousast` is the interface. Model work over whole repositories runs on a separate agentic plane
|
||||
(google/ax). A learned decision engine is in development. A scan needs neither of them.
|
||||
|
||||
This file is the front door. Each section links to its page of the documentation site.
|
||||
The site is published at <https://norandom.github.io/OpenUltraSAST/>. Its sources are in `docs/` and
|
||||
`mkdocs.yml`; to preview it locally, run `uv sync --extra docs && uv run mkdocs serve`.
|
||||
|
||||
## Status
|
||||
|
||||
**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate:
|
||||
no hook capability is enabled, and the two independent populations spent so far gave engine recall
|
||||
1 of 11 and then 0 of 17. The learned decision engine ranks but may not block: BLOCK is offered for
|
||||
no family. The project has stopped trying to reach the gates by hand; every detection change now
|
||||
has to win a controlled measurement on held-out repositories. The one page that states all of this
|
||||
with the record behind each figure is [docs/where-we-stand.md](docs/where-we-stand.md); the
|
||||
release history is in [RELEASE_NOTES.md](RELEASE_NOTES.md).
|
||||
**v2.0.0, state as of 2026-10-02.** The pre-push safety net is **NO-GO** against every M4 gate.
|
||||
|
||||
- No hook capability is enabled.
|
||||
- Two independent populations (case sets frozen before any scan) have been spent so far. Engine
|
||||
recall was 1 of 11 and then 0 of 17.
|
||||
- The learned decision engine ranks but may not block. BLOCK is offered for no family.
|
||||
|
||||
The project has stopped trying to reach the gates by hand. Every detection change now has to win
|
||||
a controlled measurement on held-out repositories. One page states all of this, with the record
|
||||
behind each figure: [docs/where-we-stand.md](docs/where-we-stand.md). The release history is in
|
||||
[RELEASE_NOTES.md](RELEASE_NOTES.md).
|
||||
|
||||
## Install and first scan
|
||||
|
||||
Which tier you need: a deterministic scan of a checkout or a CI gate is tier 1; the Joern engine
|
||||
(`standard`, `deep`) is tier 2; running model work over whole repositories on the plane is tier 3,
|
||||
for maintainers; tier 4 does not exist yet.
|
||||
Pick the tier you need:
|
||||
|
||||
1. Tier 1: a deterministic scan of a checkout, or a CI gate.
|
||||
2. Tier 2: the Joern engine (`standard`, `deep`).
|
||||
3. Tier 3, for maintainers: model work over whole repositories on the plane.
|
||||
4. Tier 4 does not exist yet.
|
||||
|
||||
### Tier 1: quick scan, no Docker
|
||||
|
||||
The core install depends only on PyYAML. From a checkout, `uv sync` (or `pip install .`) installs
|
||||
the `ousast` entry point; `uv sync --group dev` adds the test and lint tools.
|
||||
the `ousast` entry point. `uv sync --group dev` adds the test and lint tools.
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
@@ -41,19 +56,41 @@ uv run ousast scan . --mode quick --fail-on verified # CI: exit 1 on any evide
|
||||
```
|
||||
|
||||
This runs the language-scoped pattern rules, entry-point reachability hints, ranking,
|
||||
verification and scoring: deterministic, reproducible, no network, no keys. What it cannot do:
|
||||
there is no engine, so nothing follows a value across statements or files, and the PHP quick rules
|
||||
are a low-precision, in-sample development set (per-rule precision lower bounds from 0.03 to 1.0 on
|
||||
the real pins, `benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). The optional
|
||||
`semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for the overlay.
|
||||
verification and scoring. It is deterministic and reproducible. It needs no network and no keys.
|
||||
|
||||
The same install gives you the advisory pre-push hook for any repository. It runs the quick rules
|
||||
on the lines a push changes, names every check it could not run, and never blocks a push:
|
||||
|
||||
```bash
|
||||
cd /path/to/your/repository && ousast pre-push install # remove: ousast pre-push uninstall
|
||||
```
|
||||
|
||||
What it cannot do:
|
||||
|
||||
- There is no engine. Nothing follows a value across statements or files.
|
||||
- The PHP quick rules are a low-precision, in-sample development set. Per-rule precision lower
|
||||
bounds run from 0.03 to 1.0 on the real pins
|
||||
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`).
|
||||
|
||||
The optional `semantic` extra (`uv sync --extra semantic`) installs the tree-sitter grammars for
|
||||
the overlay.
|
||||
|
||||
### Tier 2: standard and deep scans (Docker)
|
||||
|
||||
`standard` and `deep` need Joern. The Docker image `openultrasast:dev` ships the tool, Joern
|
||||
v4.0.625 and `php-cli` (needed by the PHP frontend) in one image; it is built locally from the
|
||||
`Dockerfile` (no published image yet), and it is big: a JRE plus about 2 GB of Joern. The shell
|
||||
wrapper builds it on first use and then runs `ousast` in the container with your directory mounted
|
||||
read-only and the container network off:
|
||||
`standard` and `deep` need Joern. The Docker image ships three things in one image: the tool,
|
||||
Joern v4.0.625 and `php-cli` (the PHP frontend needs it). Each release publishes it to GHCR. It is
|
||||
big: a JRE plus about 2 GB of Joern. Pull it instead of building, and tag it with the name the
|
||||
shell wrapper uses:
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/norandom/openultrasast:2.0.1
|
||||
docker tag ghcr.io/norandom/openultrasast:2.0.1 openultrasast:dev # or build: docker compose build
|
||||
```
|
||||
|
||||
The shell wrapper runs `ousast` in the container. It builds the image only if `openultrasast:dev`
|
||||
is missing. Your directory is mounted read-only and the container network is off. For the pre-push
|
||||
hook, use `ousast pre-push install --docker`: Git cannot call a shell function, so it installs the
|
||||
script `ousast-docker` beside the hook.
|
||||
|
||||
```bash
|
||||
source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1
|
||||
@@ -62,90 +99,132 @@ ousast-with-judge scan . --mode standard # network on, the model is a
|
||||
```
|
||||
|
||||
`docker compose build` and `TARGET=/path/to/repo docker compose run --rm ousast scan /target` do
|
||||
the same without the wrapper. `deep` adds REGRESS, which runs each reproduction snippet with
|
||||
`docker run` from wherever the scan runs; the engine container has no Docker socket, so run `deep`
|
||||
from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
|
||||
verified) and a working `docker`, or accept the recorded `sandbox_unavailable` degradation.
|
||||
the same without the wrapper.
|
||||
|
||||
`deep` adds REGRESS. REGRESS runs each reproduction snippet with `docker run` from wherever the
|
||||
scan runs. The engine container has no Docker socket. So run `deep` in one of two ways:
|
||||
|
||||
- from a host install with Joern on the PATH (`pyinfra @local ops/joern.py`, pinned and checksum-
|
||||
verified) and a working `docker`;
|
||||
- or accept the recorded `sandbox_unavailable` degradation.
|
||||
|
||||
Details, including the host install, are in [ops/README.md](ops/README.md).
|
||||
|
||||
Keys and store settings live in `.env` in the working directory (gitignored; `.env.example` lists
|
||||
the names). `DEEPSEEK_API_KEY` serves every LLM call; `OPENROUTER_API_KEY` is used for embeddings
|
||||
only (`ousast learn memory embed`). `.env` never overrides a variable already exported in your
|
||||
shell, so `unset` a stale export when a key change seems to have no effect. The plane's memory
|
||||
store defaults to a local file store under `~/ousast-results/plane/memory`; an S3-compatible bucket
|
||||
is optional (`uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT` and the
|
||||
AWS credentials in `.env`). Nothing in a scan needs it.
|
||||
Keys and store settings live in `.env` in the working directory. The file is gitignored;
|
||||
`.env.example` lists the names.
|
||||
|
||||
- `DEEPSEEK_API_KEY` serves every LLM call.
|
||||
- `OPENROUTER_API_KEY` is used for embeddings only (`ousast learn memory embed`).
|
||||
- `.env` never overrides a variable already exported in your shell. If a key change seems to have
|
||||
no effect, `unset` the stale export.
|
||||
- The plane's memory store defaults to a local file store under `~/ousast-results/plane/memory`.
|
||||
- An S3-compatible bucket is optional: `uv sync --extra s3`, then `OUSAST_MEMORY=s3://<bucket>`
|
||||
with `S3_ENDPOINT` and the AWS credentials in `.env`. Nothing in a scan needs it.
|
||||
|
||||
### Tier 3: the plane on ax (maintainers)
|
||||
|
||||
None of this is needed for tier 1 or 2. Today's profile is a single-node kind cluster on the
|
||||
maintainer's host. `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the
|
||||
PATH (it refuses to start when one is missing) and builds Agent Substrate and ax from their
|
||||
checkouts under `~/.cache/ousast/ax-src/`. Measured idle footprint with two workers: the kind node
|
||||
takes 1.7 GiB of memory (27% of the 7.7 GiB host) and 4.9 GB of disk for images, with several GB
|
||||
more of Go caches during the build; the worker pool is sized for that 7 GB host. The verify and
|
||||
roles tasks need `DEEPSEEK_API_KEY`, and memory lives in the local file store or the S3 store
|
||||
(RustFS is the tested server). Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
|
||||
Tier 1 and 2 need none of this. Today's profile is a single-node kind cluster on the maintainer's
|
||||
host.
|
||||
|
||||
- `ops/ax/up.sh` needs `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` on the PATH. It refuses to
|
||||
start when one is missing.
|
||||
- It builds Agent Substrate and ax from their checkouts under `~/.cache/ousast/ax-src/`.
|
||||
- Measured idle footprint with two workers: the kind node takes 1.7 GiB of memory (27% of the
|
||||
7.7 GiB host) and 4.9 GB of disk for images. The build uses several GB more for Go caches.
|
||||
- The worker pool is sized for that 7 GB host.
|
||||
- The verify and roles tasks need `DEEPSEEK_API_KEY`.
|
||||
- Memory lives in the local file store or the S3 store. RustFS is the tested server.
|
||||
|
||||
Bring-up, doctor and the smoke Run: [ops/ax/README.md](ops/ax/README.md).
|
||||
|
||||
### Tier 4: Kubernetes (k3s), planned
|
||||
|
||||
No deployment to any cluster other than that kind cluster has been made. The target is two
|
||||
profiles of one code base, `kind` locally and `k3s` in production, where the same `ousast plane run`
|
||||
submits a Run in a remote execution mode: it needs a kubeconfig context or the cluster's ingress
|
||||
(and an `imagePullSecret` on the worker ServiceAccount for a private package), runner and engine
|
||||
images published to GHCR and pinned by digest, and the S3 store as the artifact path, with no
|
||||
receiver on a laptop. The plan, and the code changes it needs with file paths, are in
|
||||
Nothing has been deployed to any cluster other than that kind cluster. The target is two profiles
|
||||
of one code base: `kind` locally and `k3s` in production. The same `ousast plane run` would
|
||||
submit a Run in a remote execution mode. It needs:
|
||||
|
||||
- a kubeconfig context or the cluster's ingress (and an `imagePullSecret` on the worker
|
||||
ServiceAccount for a private package);
|
||||
- runner and engine images published to GHCR and pinned by digest;
|
||||
- the S3 store as the artifact path, with no receiver on a laptop.
|
||||
|
||||
The plan, and the code changes it needs with file paths, are in
|
||||
[docs/deployment.md](docs/deployment.md#production-topology-ax-on-a-kubernetes-cluster).
|
||||
|
||||
## Scan modes
|
||||
|
||||
`quick` runs the pattern rules, hints, ranking, verification and score and needs nothing.
|
||||
`standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and the
|
||||
Joern model layer, which builds one code property graph per repository and decides taint,
|
||||
guard-dominance and configuration questions on it; a model is asked only the residual question
|
||||
(without a key it is skipped and only the graph's entailed findings are reported). `deep` adds
|
||||
REGRESS: promoted candidates are loaded by a small snippet in a Docker sandbox and get a
|
||||
`triggerable` or `not_triggerable` verdict. A missing engine is a recorded degradation
|
||||
(`cpg_unavailable`, `sandbox_unavailable`), never a clean result.
|
||||
There are three modes. Each adds to the one before.
|
||||
|
||||
- `quick` runs the pattern rules, hints, ranking, verification and score. It needs nothing.
|
||||
- `standard` adds the MAP stage: complexity map, semantic overlay, authorization obligations and
|
||||
the Joern model layer. The model layer builds one code property graph per repository. It decides
|
||||
taint, guard-dominance and configuration questions on that graph. A model is asked only the
|
||||
residual question. Without a key, that question is skipped and only the graph's entailed
|
||||
findings are reported.
|
||||
- `deep` adds REGRESS. Promoted candidates are loaded by a small snippet in a Docker sandbox. Each
|
||||
gets a `triggerable` or `not_triggerable` verdict.
|
||||
|
||||
A missing engine is a recorded degradation (`cpg_unavailable`, `sandbox_unavailable`), never a
|
||||
clean result.
|
||||
|
||||
`--fail-on never|findings|verified|worth-fixing` sets the exit code. `--config openultrasast.toml`
|
||||
loads settings. Every run writes these files under `<target>/.openultrasast/runs/<scan-id>/`:
|
||||
`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`, `report.sarif`
|
||||
and `trace/events.jsonl`.
|
||||
|
||||
`ousast pre-push` (experimental) analyses the commits a push would publish against their base.
|
||||
First, the quick rules run on the changed lines. Then the engine compares head with base, and only
|
||||
new or worsened defects become candidates. It is advisory by default, and it prints one line per
|
||||
check it could not run. Its capability registry is empty, so it emits no normal alert today. An
|
||||
engine finding the push introduced is shown as advisory. Install, settings, skip reasons and a CI
|
||||
recipe: [docs/scanning.md](docs/scanning.md#first-run-install-the-hook-on-any-repository).
|
||||
|
||||
`--fail-on never|findings|verified|worth-fixing` sets the exit code and `--config openultrasast.toml`
|
||||
loads settings. Every run writes `manifest.json`, `findings.json`, `verification.json`,
|
||||
`score.json`, `report.md`, `report.sarif` and `trace/events.jsonl` under
|
||||
`<target>/.openultrasast/runs/<scan-id>/`. `ousast pre-push` (experimental) analyses the commits a
|
||||
push would publish against their base so that only new or worsened defects are candidates; it is
|
||||
advisory by default and its capability registry is empty, so no normal alert is emitted today.
|
||||
Flows and the per-language rule table: [docs/scanning.md](docs/scanning.md).
|
||||
|
||||
## How detection works
|
||||
|
||||
Three representations, from cheap to expensive: text (regex quick rules, one statement at a time),
|
||||
tree (the tree-sitter CST and an IR per function, for the overlay and the obligations) and graph
|
||||
(the Joern CPG, the only one that follows a value across statements, calls and files). Taint runs
|
||||
as fact tables, queries and verdicts on the graph; absence bugs (an operation that should have
|
||||
been guarded) are found as obligations, dominance and configuration census; a model answers only
|
||||
the residual question. [docs/detection-techniques.md](docs/detection-techniques.md) explains each
|
||||
technique with the code it comes from and what it misses;
|
||||
[docs/architecture.md](docs/architecture.md) gives the stage order, the evidence ladder (a state
|
||||
machine, not a label a model may assign), the central CWE policy and the project score, and the
|
||||
self-improving loops (`ousast improve`, with `--memory` for proposals from plane rows).
|
||||
The tool uses three representations of code, from cheap to expensive:
|
||||
|
||||
Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP, and
|
||||
run only against files of their language. The detection gate (`uv run python -m openultrasast.gate`)
|
||||
enforces at least 90% recall and under 10% false positives per language on the bundled cheat-sheet
|
||||
corpora; on this tree it reports 93.62% recall (44/47) and 0.0% false positives. Those corpora show
|
||||
that a rule change did not break a known detection; they are not a recall estimate on real code,
|
||||
and PHP is not in the gate. Framework knowledge (WordPress, Flask, Django, Express, Spring) is
|
||||
tagged as an optional prior in `src/openultrasast/ruleset/frameworks.toml`, so its contribution can
|
||||
be measured and switched off.
|
||||
1. Text: regex quick rules, one statement at a time.
|
||||
2. Tree: the tree-sitter CST (concrete syntax tree) and an IR (intermediate representation) per
|
||||
function. The overlay and the obligations use it.
|
||||
3. Graph: the Joern CPG. It is the only one that follows a value across statements, calls and
|
||||
files.
|
||||
|
||||
Taint runs as fact tables, queries and verdicts on the graph. Absence bugs are operations that
|
||||
should have been guarded. They are found as obligations, dominance and configuration census. A
|
||||
model answers only the residual question.
|
||||
|
||||
- [docs/detection-techniques.md](docs/detection-techniques.md) explains each technique, with the
|
||||
code it comes from and what it misses.
|
||||
- [docs/architecture.md](docs/architecture.md) gives the stage order and the evidence ladder (a
|
||||
state machine, not a label a model may assign). It also covers the central CWE policy, the
|
||||
project score and the self-improving loops (`ousast improve`, with `--memory` for proposals from
|
||||
plane rows).
|
||||
|
||||
Quick rules cover C/C++, Python, JavaScript/TypeScript, Java with Groovy templates, and PHP. Each
|
||||
rule runs only against files of its language.
|
||||
|
||||
The detection gate (`uv run python -m openultrasast.gate`) enforces at least 90% recall and under
|
||||
10% false positives per language on the bundled cheat-sheet corpora. On this tree it reports
|
||||
93.62% recall (44/47) and 0.0% false positives. Those corpora show that a rule change did not
|
||||
break a known detection. They are not a recall estimate on real code, and PHP is not in the gate.
|
||||
|
||||
Framework knowledge (WordPress, Flask, Django, Express, Spring) is tagged as an optional prior in
|
||||
`src/openultrasast/ruleset/frameworks.toml`. So its contribution can be measured and switched off.
|
||||
|
||||
## The plane on ax
|
||||
|
||||
Model-driven work over whole repositories runs as a Run manifest, a DAG of ax Tasks, each in its
|
||||
own gVisor actor on Agent Substrate. ax is the only agentic executor; there is no local
|
||||
subprocess path. Each Task binds at most one Model and its own `usd`/`calls` budget, egress is
|
||||
deny-by-default per Task (hostnames only), and the provider key travels only in the Task's start
|
||||
request, never in a manifest or an image. Measured on the first increment: the plane agreed on 16
|
||||
of 20 declared sites at $0.0205 per candidate.
|
||||
Model-driven work over whole repositories runs as a Run manifest. A Run is a DAG of ax Tasks. Each
|
||||
Task runs in its own gVisor actor (an isolated sandbox) on Agent Substrate. ax is the only agentic
|
||||
executor; there is no local subprocess path.
|
||||
|
||||
- Each Task binds at most one Model and its own `usd`/`calls` budget.
|
||||
- Egress is deny-by-default per Task, by hostname only.
|
||||
- The provider key travels only in the Task's start request, never in a manifest or an image.
|
||||
|
||||
Measured on the first increment: the plane agreed on 16 of 20 declared sites at $0.0205 per
|
||||
candidate.
|
||||
|
||||
```bash
|
||||
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
|
||||
@@ -155,42 +234,61 @@ uv run ousast plane remember validation-46 # ingest the run's rows
|
||||
```
|
||||
|
||||
`workspaces`, `harvest`, `alerts-engine` and `memory-normalise` generate Runs, their inputs and
|
||||
the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md); what runs
|
||||
where today and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
|
||||
the store's rewrites. The flow and the task catalogue: [docs/plane.md](docs/plane.md). What runs
|
||||
where today, and what a separate cluster would take: [docs/deployment.md](docs/deployment.md).
|
||||
|
||||
HarnessX, the earlier agentic extra, was retired 2026-09-30: a `[harnessx]` section is ignored
|
||||
with one warning, and `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail
|
||||
naming the plane replacement, as do config keys nothing ever read. Fusion is deterministic.
|
||||
HarnessX, the earlier agentic extra, was retired 2026-09-30. A `[harnessx]` section is ignored
|
||||
with one warning. `[models] verifier`, `[fusion] panel_model` and `[fusion] decider_model` fail and
|
||||
name the plane replacement. Config keys that nothing ever read fail the same way. Fusion is
|
||||
deterministic.
|
||||
|
||||
## Memory
|
||||
|
||||
One store, keyed by repository and pin, holds what plane runs and the decision engine learned:
|
||||
rows (facts, verdicts, unit costs, alerts, coverage, feature records) and content-addressed blobs
|
||||
(excerpts, embeddings, cached responses, compiled programs). `OUSAST_MEMORY` selects the backend:
|
||||
a local file store, or `s3://<bucket>` on any S3-compatible server with versioning, lifecycle
|
||||
rules, object tags and S3 Select; RustFS is the tested server. An admin sets the bucket up once;
|
||||
the store verifies it at every start and refuses to run, naming each missing piece, rather than
|
||||
fall back. Layout and the read and write paths: [docs/memory.md](docs/memory.md); the bucket
|
||||
policy and admin steps: [docs/rustfs.md](docs/rustfs.md); how stored rows are meant to turn into
|
||||
better decisions: [docs/memory-and-detection.md](docs/memory-and-detection.md).
|
||||
One store holds what plane runs and the decision engine learned. It is keyed by repository and pin.
|
||||
It holds two kinds of data:
|
||||
|
||||
- rows: facts, verdicts, unit costs, alerts, coverage and feature records;
|
||||
- content-addressed blobs: excerpts, embeddings, cached responses and compiled programs.
|
||||
|
||||
`OUSAST_MEMORY` selects the backend. It is either a local file store or `s3://<bucket>` on any
|
||||
S3-compatible server. The S3 server must support versioning, lifecycle rules, object tags and S3
|
||||
Select. RustFS is the tested server.
|
||||
|
||||
An admin sets the bucket up once. The store verifies it at every start. If anything is missing, it
|
||||
refuses to run and names each missing piece. It does not fall back.
|
||||
|
||||
- Layout and the read and write paths: [docs/memory.md](docs/memory.md).
|
||||
- The bucket policy and admin steps: [docs/rustfs.md](docs/rustfs.md).
|
||||
- How stored rows are meant to turn into better decisions:
|
||||
[docs/memory-and-detection.md](docs/memory-and-detection.md).
|
||||
|
||||
## Decision engine and evaluation
|
||||
|
||||
A learned decision engine is being built to replace hand-tuned detection edits: every instrument
|
||||
produces signals, never verdicts; labels come from ground truth only; a compiled AI classifier with
|
||||
local memory turns retrieved similar cases and the candidate's signals into a calibrated
|
||||
probability with BLOCK and ADVISORY operating points. It is **in development, not adopted**: no
|
||||
scan, pre-push check or report uses it. Measured on six families, on repositories the compiled
|
||||
program was not shown, the pooled AUC of its score runs from 0.77 (injection) to 0.96 (access
|
||||
control); the stricter within-pair AUC is 0.72 to 0.93; BLOCK is offered for no family
|
||||
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`). Maintainer surface:
|
||||
`ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`. Design, slices and
|
||||
the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
|
||||
A learned decision engine is being built. It is meant to replace hand-tuned detection edits. It
|
||||
follows three rules:
|
||||
|
||||
How anything is measured on code the tool was not tuned on, the pair corpora, the independent
|
||||
populations and the M4 gates, and the fold discipline that keeps learned decisions out of their
|
||||
own training data: [docs/evaluation.md](docs/evaluation.md). Population v3 (PHP, 15 cases) is
|
||||
frozen as the one-time final check and nothing in the tree may read it.
|
||||
- Every instrument produces signals, never verdicts.
|
||||
- Labels come from ground truth only.
|
||||
- A compiled AI classifier with local memory turns retrieved similar cases and the candidate's
|
||||
signals into a calibrated probability. It has BLOCK and ADVISORY operating points.
|
||||
|
||||
It is **in development, not adopted**. No scan, pre-push check or report uses it.
|
||||
|
||||
It was measured on six families, on repositories the compiled program was not shown:
|
||||
|
||||
- The pooled AUC (area under the ROC curve) of its score runs from 0.77 (injection) to 0.96
|
||||
(access control).
|
||||
- The stricter within-pair AUC is 0.72 to 0.93 across the same families.
|
||||
- BLOCK is offered for no family
|
||||
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
|
||||
|
||||
Maintainer surface: `ousast learn labels|memory|compile|evaluate|curve|experiment|audit-leaks`.
|
||||
Design, slices and the A/B protocol: [docs/decision-engine.md](docs/decision-engine.md).
|
||||
|
||||
[docs/evaluation.md](docs/evaluation.md) explains how anything is measured on code the tool was not
|
||||
tuned on. It covers the pair corpora, the independent populations and the M4 gates. It also covers
|
||||
the fold discipline that keeps learned decisions out of their own training data. Population v3
|
||||
(PHP, 15 cases) is frozen as the one-time final check. Nothing in the tree may read it.
|
||||
|
||||
```bash
|
||||
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
|
||||
@@ -201,19 +299,26 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dr
|
||||
|
||||
## Token economics
|
||||
|
||||
Model tokens are the one running cost. At runtime the plane routes them: one Model and one budget
|
||||
per Task, a metered client that refuses the next call at the ceiling, repository facts computed
|
||||
model-free and reused, blobs replayed at $0, triage before the hunt and a tie-break only on
|
||||
disputed candidates. In development, 85.5% of the maintainers' coding-session tokens were tool
|
||||
input and output rather than reasoning (down from 90%), and the remedies in `benchmarks/dev/`
|
||||
target that. Mechanisms,
|
||||
the measured costs and the before-and-after table: [docs/token-ergonomics.md](docs/token-ergonomics.md).
|
||||
Model tokens are the one running cost. At runtime the plane routes them:
|
||||
|
||||
- one Model and one budget per Task;
|
||||
- a metered client that refuses the next call at the ceiling;
|
||||
- repository facts computed without a model and reused;
|
||||
- blobs replayed at $0, with no new model call;
|
||||
- triage before the hunt;
|
||||
- a tie-break only on disputed candidates.
|
||||
|
||||
In development, 85.5% of the maintainers' coding-session tokens were tool input and output rather
|
||||
than reasoning. That is down from 90%. The remedies in `benchmarks/dev/` target that share.
|
||||
Mechanisms, the measured costs and the before-and-after table:
|
||||
[docs/token-ergonomics.md](docs/token-ergonomics.md).
|
||||
|
||||
## Examples, threat model
|
||||
|
||||
End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
|
||||
What the tool trusts and what it does not (scanned code is untrusted input, secret redaction,
|
||||
per-task egress, budgets, the `deep` sandbox): [docs/threat-model.md](docs/threat-model.md).
|
||||
- End-to-end walkthroughs, from a quick scan to an MCP client: [docs/examples.md](docs/examples.md).
|
||||
- What the tool trusts and what it does not: [docs/threat-model.md](docs/threat-model.md). This
|
||||
covers scanned code as untrusted input, secret redaction, per-task egress, budgets and the
|
||||
`deep` sandbox.
|
||||
|
||||
## Contributing and maintainer tooling
|
||||
|
||||
@@ -228,25 +333,33 @@ uv run python dagger/ci.py # containerized CI pipeline
|
||||
```
|
||||
|
||||
An agent that can run shell commands needs nothing but the CLI. For [OpenCode](https://opencode.ai)
|
||||
there are two project skills in `.agents/skills/`: `openultrasast-scan` (modes and their
|
||||
prerequisites, `--fail-on`, the evidence ladder, recorded degradations, run artifacts) and
|
||||
`openultrasast-triage` (false-positive reasons, what an adjudication may feed, fixing and
|
||||
re-scanning). `ousast mcp` runs a narrow MCP server over stdio with ten tools (`openultrasast.scan`,
|
||||
`status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`, `explain`,
|
||||
`propose_patch`, `export_report`); no tool runs a shell, Docker or a free-form command:
|
||||
there are two project skills in `.agents/skills/`:
|
||||
|
||||
- `openultrasast-scan`: modes and their prerequisites, `--fail-on`, the evidence ladder, recorded
|
||||
degradations and run artifacts.
|
||||
- `openultrasast-triage`: false-positive reasons, what an adjudication may feed, and fixing and
|
||||
re-scanning.
|
||||
|
||||
`ousast mcp` runs a narrow MCP (Model Context Protocol) server over stdio. It has ten tools:
|
||||
`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`, `benchmark`,
|
||||
`explain`, `propose_patch` and `export_report`. No tool runs a shell, Docker or a free-form
|
||||
command.
|
||||
|
||||
```jsonc
|
||||
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
|
||||
```
|
||||
|
||||
The `kiro-*` skills, `AGENTS.md` and the Kiro-style specifications under `.kiro/` are the
|
||||
maintainer's development tooling and can be ignored by users. Other maintainer commands:
|
||||
`ousast repos` (pinned known-vulnerable checkouts) and `ousast model candidates` (what the
|
||||
candidate enumerator can reach).
|
||||
maintainer's development tooling. Users can ignore them. Other maintainer commands:
|
||||
|
||||
- `ousast repos`: pinned known-vulnerable checkouts.
|
||||
- `ousast model candidates`: what the candidate enumerator can reach.
|
||||
|
||||
## Licence
|
||||
|
||||
OpenUltraSAST is licensed under the [Apache License 2.0](LICENSE). Third-party material keeps its
|
||||
own terms: Joern (Apache-2.0) is downloaded at image build time, the tree-sitter wheels of the
|
||||
`semantic` extra are MIT, and the published benchmark cases were licence-checked one by one (v1's
|
||||
OpenCVE case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md)).
|
||||
own terms:
|
||||
|
||||
- Joern (Apache-2.0) is downloaded at image build time.
|
||||
- The tree-sitter wheels of the `semantic` extra are MIT.
|
||||
- The published benchmark cases were licence-checked one by one. v1's OpenCVE case is BUSL-1.1, see [RELEASE_NOTES.md](RELEASE_NOTES.md).
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"experiment": "exp-003-precision-bound-blocking",
|
||||
"registered": "2026-10-03",
|
||||
"store": "s3://sast-memory",
|
||||
"manifest": "plane/experiments/exp-003-precision-bound-blocking.yaml",
|
||||
"manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
|
||||
"pin": "dd7ebc119b5ad8e77c05d418eed9cbd46e193587",
|
||||
"units_status": "frozen",
|
||||
"units_file": "plane/experiments/exp-003-precision-bound-blocking.units.jsonl",
|
||||
"units_digest": "338de998b1cfa36857cd793dbc4cf7fd0a13ab517a91f6790335881e941b3006",
|
||||
"units": {
|
||||
"total": 353,
|
||||
"paired": 244,
|
||||
"groups": 146,
|
||||
"by_family": {
|
||||
"access_control": {"units": 41, "positives": 25, "paired": 32, "paired_positives": 16, "groups": 24},
|
||||
"deserialization": {"units": 48, "positives": 31, "paired": 34, "paired_positives": 17, "groups": 31},
|
||||
"injection": {"units": 119, "positives": 63, "paired": 76, "paired_positives": 38, "groups": 34},
|
||||
"output_encoding": {"units": 45, "positives": 29, "paired": 32, "paired_positives": 16, "groups": 27},
|
||||
"path": {"units": 39, "positives": 23, "paired": 32, "paired_positives": 16, "groups": 23},
|
||||
"untrusted_destination": {"units": 61, "positives": 37, "paired": 38, "paired_positives": 19, "groups": 36}
|
||||
},
|
||||
"note": "frozen model-free on the 937-example static memory; the outer folds (k 5, seed 0, compile fraction 0.25) are the incumbent evaluation's; injection and access_control equal exp-002's 160 units"
|
||||
},
|
||||
"kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
|
||||
"zero_cost": "responses replayed from the response cache only, behind a meter with zero USD and call ceilings around a refusing client; a unit with any uncached response is excluded and counted; the run is void unless the meter reads 0 calls and $0",
|
||||
"arms": {
|
||||
"A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds (slope CI contains 1, intercept CI contains 0, ECE <= 0.05); BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
|
||||
"B": "selective blocking: per family and outer fold, threshold chosen on the family's paired units in the other four folds as the lowest raw score t with Wilson 95% LB of flagged precision >= 0.95 (unsure verdicts never flag); none -> no BLOCK on that fold",
|
||||
"C": "secondary, non-deciding: as B with point precision >= 0.95 over at least 10 training flags"
|
||||
},
|
||||
"primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision, per family, arm B, paired population",
|
||||
"populations": {"primary": "paired (both sides of the candidate present; excludes every positives-only group and any unpaired unit)", "secondary": ["pooled (every unit)"]},
|
||||
"secondary_metrics": ["flagged_precision", "coverage (flagged true positives / positives)", "flagged", "repository-cluster bootstrap 95% CI of flagged precision"],
|
||||
"decision_rule": "REJECT B if in any family B flags held-out paired units at a point precision < 0.95; else ADOPT B if in some family B's held-out paired Wilson LB >= 0.95; else INCONCLUSIVE (the incumbent rule stays)",
|
||||
"m4_reading": "a family reaches the M4 precision bar on held-out repositories when its held-out paired Wilson LB >= 0.95; point precision reported beside it",
|
||||
"power": "a Wilson LB of 0.95 needs >= 73 flagged units with no false flag; the largest family holds 38 paired positives over all five folds, so arm B cannot offer a BLOCK in any family by construction; expected verdict inconclusive",
|
||||
"status": "run and analysed 2026-10-03: inconclusive -- no arm flags any held-out unit (result.json); meter 0 client calls, $0.0; 142 of 353 units excluded for an uncached response"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,981 @@
|
||||
{
|
||||
"arms": {
|
||||
"A": {
|
||||
"label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
|
||||
"rule": {
|
||||
"name": "calibrated_block"
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"label": "selective blocking -- per-family raw-score threshold from the other folds' paired units, lowest t with Wilson 95% LB of flagged precision >= 0.95",
|
||||
"rule": {
|
||||
"bound": "wilson",
|
||||
"name": "precision_bound_block",
|
||||
"precision": 0.95,
|
||||
"select_on": "paired"
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"label": "secondary, non-deciding -- as B with point precision >= 0.95 over at least 10 training flags",
|
||||
"rule": {
|
||||
"bound": "point",
|
||||
"min_flagged": 10,
|
||||
"name": "precision_bound_block",
|
||||
"precision": 0.95,
|
||||
"select_on": "paired"
|
||||
}
|
||||
}
|
||||
},
|
||||
"code_commit": "7bb97ad0bca62b5f986c5768297d8b8fca50fae4",
|
||||
"decision": "inconclusive",
|
||||
"experiment": "exp-003-precision-bound-blocking",
|
||||
"families": {
|
||||
"access_control": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.077,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 16,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 16,
|
||||
"true_flagged": 0,
|
||||
"units": 32
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 24,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 25,
|
||||
"true_flagged": 0,
|
||||
"units": 41
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 16,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 16,
|
||||
"true_flagged": 0,
|
||||
"units": 32
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 24,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 25,
|
||||
"true_flagged": 0,
|
||||
"units": 41
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 16,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 16,
|
||||
"true_flagged": 0,
|
||||
"units": 32
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 24,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 25,
|
||||
"true_flagged": 0,
|
||||
"units": 41
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 41,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 12,
|
||||
"outer-1": 13,
|
||||
"outer-2": 13,
|
||||
"outer-3": 13,
|
||||
"outer-4": 13
|
||||
}
|
||||
}
|
||||
},
|
||||
"deserialization": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": null,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 3,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 2,
|
||||
"true_flagged": 0,
|
||||
"units": 5
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 12,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 11,
|
||||
"true_flagged": 0,
|
||||
"units": 14
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 3,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 2,
|
||||
"true_flagged": 0,
|
||||
"units": 5
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 12,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 11,
|
||||
"true_flagged": 0,
|
||||
"units": 14
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 3,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 2,
|
||||
"true_flagged": 0,
|
||||
"units": 5
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 12,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 11,
|
||||
"true_flagged": 0,
|
||||
"units": 14
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 14,
|
||||
"excluded_replay_miss": 34,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 0,
|
||||
"outer-2": 2,
|
||||
"outer-3": 2,
|
||||
"outer-4": 2
|
||||
}
|
||||
}
|
||||
},
|
||||
"injection": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.1067,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 26,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 38,
|
||||
"true_flagged": 0,
|
||||
"units": 76
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 34,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 63,
|
||||
"true_flagged": 0,
|
||||
"units": 119
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 26,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 38,
|
||||
"true_flagged": 0,
|
||||
"units": 76
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 34,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 63,
|
||||
"true_flagged": 0,
|
||||
"units": 119
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 26,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 38,
|
||||
"true_flagged": 0,
|
||||
"units": 76
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 34,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 63,
|
||||
"true_flagged": 0,
|
||||
"units": 119
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 119,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 29,
|
||||
"outer-1": 33,
|
||||
"outer-2": 33,
|
||||
"outer-3": 26,
|
||||
"outer-4": 31
|
||||
}
|
||||
}
|
||||
},
|
||||
"output_encoding": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": null,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 8,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 8,
|
||||
"true_flagged": 0,
|
||||
"units": 16
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 15,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 16,
|
||||
"true_flagged": 0,
|
||||
"units": 24
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 8,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 8,
|
||||
"true_flagged": 0,
|
||||
"units": 16
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 15,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 16,
|
||||
"true_flagged": 0,
|
||||
"units": 24
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 8,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 8,
|
||||
"true_flagged": 0,
|
||||
"units": 16
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 15,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 16,
|
||||
"true_flagged": 0,
|
||||
"units": 24
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 24,
|
||||
"excluded_replay_miss": 21,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 7,
|
||||
"outer-1": 6,
|
||||
"outer-2": 6,
|
||||
"outer-3": 7,
|
||||
"outer-4": 6
|
||||
}
|
||||
}
|
||||
},
|
||||
"path": {
|
||||
"A": {
|
||||
"paired": {
|
||||
"coverage": null,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 0,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 0,
|
||||
"true_flagged": 0,
|
||||
"units": 0
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": null,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 0,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 0,
|
||||
"true_flagged": 0,
|
||||
"units": 0
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": null,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 0,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 0,
|
||||
"true_flagged": 0,
|
||||
"units": 0
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": null,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 0,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 0,
|
||||
"true_flagged": 0,
|
||||
"units": 0
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"paired": {
|
||||
"coverage": null,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 0,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 0,
|
||||
"true_flagged": 0,
|
||||
"units": 0
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": null,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 0,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 0,
|
||||
"true_flagged": 0,
|
||||
"units": 0
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 0,
|
||||
"excluded_replay_miss": 39,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {}
|
||||
}
|
||||
},
|
||||
"untrusted_destination": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": null,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 6,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 4,
|
||||
"true_flagged": 0,
|
||||
"units": 9
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 9,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 7,
|
||||
"true_flagged": 0,
|
||||
"units": 13
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 6,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 4,
|
||||
"true_flagged": 0,
|
||||
"units": 9
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 9,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 7,
|
||||
"true_flagged": 0,
|
||||
"units": 13
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 6,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 4,
|
||||
"true_flagged": 0,
|
||||
"units": 9
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 9,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 7,
|
||||
"true_flagged": 0,
|
||||
"units": 13
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 13,
|
||||
"excluded_replay_miss": 48,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 4,
|
||||
"outer-1": 3,
|
||||
"outer-2": 3,
|
||||
"outer-3": 2
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"hypothesis": "The calibrated-probability BLOCK rule (Platt map + calibration gate, ECE <= 0.05) is the wrong instrument for M4, which asks for precision on what is flagged. Selective blocking -- a per-family raw-score threshold chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95 -- reaches the M4 precision bar on held-out repositories where the calibrated rule offers no BLOCK.\n",
|
||||
"instrument_check": {
|
||||
"families": {
|
||||
"access_control": {
|
||||
"distinct_s": 30,
|
||||
"paired": {
|
||||
"blockable": 28,
|
||||
"positives_before_first_negative": 10
|
||||
},
|
||||
"pooled": {
|
||||
"blockable": 37,
|
||||
"positives_before_first_negative": 19
|
||||
},
|
||||
"verdicts": {
|
||||
"not_vulnerable": 19,
|
||||
"unsure": 4,
|
||||
"vulnerable": 18
|
||||
}
|
||||
},
|
||||
"deserialization": {
|
||||
"distinct_s": 5,
|
||||
"paired": {
|
||||
"blockable": 4,
|
||||
"positives_before_first_negative": 1
|
||||
},
|
||||
"pooled": {
|
||||
"blockable": 13,
|
||||
"positives_before_first_negative": 10
|
||||
},
|
||||
"verdicts": {
|
||||
"not_vulnerable": 2,
|
||||
"unsure": 1,
|
||||
"vulnerable": 11
|
||||
}
|
||||
},
|
||||
"injection": {
|
||||
"distinct_s": 48,
|
||||
"paired": {
|
||||
"blockable": 50,
|
||||
"positives_before_first_negative": 3
|
||||
},
|
||||
"pooled": {
|
||||
"blockable": 77,
|
||||
"positives_before_first_negative": 11
|
||||
},
|
||||
"verdicts": {
|
||||
"not_vulnerable": 46,
|
||||
"unsure": 42,
|
||||
"vulnerable": 31
|
||||
}
|
||||
},
|
||||
"output_encoding": {
|
||||
"distinct_s": 14,
|
||||
"paired": {
|
||||
"blockable": 11,
|
||||
"positives_before_first_negative": 0
|
||||
},
|
||||
"pooled": {
|
||||
"blockable": 19,
|
||||
"positives_before_first_negative": 8
|
||||
},
|
||||
"verdicts": {
|
||||
"not_vulnerable": 9,
|
||||
"unsure": 5,
|
||||
"vulnerable": 10
|
||||
}
|
||||
},
|
||||
"untrusted_destination": {
|
||||
"distinct_s": 9,
|
||||
"paired": {
|
||||
"blockable": 4,
|
||||
"positives_before_first_negative": 1
|
||||
},
|
||||
"pooled": {
|
||||
"blockable": 8,
|
||||
"positives_before_first_negative": 4
|
||||
},
|
||||
"verdicts": {
|
||||
"not_vulnerable": 3,
|
||||
"unsure": 5,
|
||||
"vulnerable": 5
|
||||
}
|
||||
}
|
||||
},
|
||||
"note": "descriptive, all folds together, ranking by raw score s among non-unsure verdicts; not nested, chooses nothing. It shows the scores are real and the zero is the rule's, not an unread input."
|
||||
},
|
||||
"manifest_digest": "5917f9a12035f09eae76978f3b5827b9fe59c8a9a5275f9ab54c3c1b55e8db3b",
|
||||
"meter": {
|
||||
"client_calls": 0,
|
||||
"decided": 211,
|
||||
"excluded_replay_miss": 142,
|
||||
"folds_digest": "a1f78e141a572a4adcc9cd57eb8a784c440c177f0b60d5ef5bed9e94c8753c5f",
|
||||
"metered_usd": 0.0,
|
||||
"replay_priced_usd_not_spent": 0.582876,
|
||||
"replayed_responses": 1056
|
||||
},
|
||||
"min_flagged_for_bound": 73,
|
||||
"overall": {
|
||||
"A": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 59,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 68,
|
||||
"true_flagged": 0,
|
||||
"units": 138
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 89,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 122,
|
||||
"true_flagged": 0,
|
||||
"units": 211
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 59,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 68,
|
||||
"true_flagged": 0,
|
||||
"units": 138
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 89,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 122,
|
||||
"true_flagged": 0,
|
||||
"units": 211
|
||||
}
|
||||
},
|
||||
"C": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 59,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 68,
|
||||
"true_flagged": 0,
|
||||
"units": 138
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 89,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 122,
|
||||
"true_flagged": 0,
|
||||
"units": 211
|
||||
}
|
||||
}
|
||||
},
|
||||
"populations": {
|
||||
"primary": "paired",
|
||||
"secondary": [
|
||||
"pooled"
|
||||
]
|
||||
},
|
||||
"primary": [
|
||||
{
|
||||
"name": "flagged_precision_lb",
|
||||
"target": 0.95
|
||||
}
|
||||
],
|
||||
"reason": "B offers no BLOCK on held-out paired units in any family; the incumbent stays",
|
||||
"resamples": 2000,
|
||||
"secondary": [
|
||||
{
|
||||
"name": "flagged_precision"
|
||||
},
|
||||
{
|
||||
"name": "coverage"
|
||||
},
|
||||
{
|
||||
"name": "flagged"
|
||||
}
|
||||
],
|
||||
"spend_usd": {
|
||||
"score": 0.0
|
||||
},
|
||||
"target_precision": 0.95,
|
||||
"units": {
|
||||
"by_label": {
|
||||
"0": 89,
|
||||
"1": 122
|
||||
},
|
||||
"evaluated": 211,
|
||||
"excluded_replay_miss": 142,
|
||||
"frozen": 353,
|
||||
"groups": 89,
|
||||
"not_run": 0,
|
||||
"paired_evaluated": 138,
|
||||
"unsure": 57
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Instrument check for exp-004: are the scores real, and how clean is the pooled top of the ranking?
|
||||
|
||||
Reads the run's arm_outcome rows from the memory store (OUSAST_MEMORY) and the frozen units file; makes no model call.
|
||||
usage: python benchmarks/experiments/exp-004-pooled-block-gate/instrument_check.py (from the repository root)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import collections
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from openultrasast.config import load_dotenv
|
||||
from openultrasast.learn import experiments as ex
|
||||
from openultrasast.plane.memory import open_store
|
||||
|
||||
EXPERIMENT = "exp-004-pooled-block-gate"
|
||||
UNITS = Path(f"plane/experiments/{EXPERIMENT}.units.jsonl")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
load_dotenv(Path(".env"))
|
||||
rows = [r.row for r in open_store(None).rows(repo=ex.experiment_repo(EXPERIMENT), kind="arm_outcome")]
|
||||
with UNITS.open() as handle:
|
||||
units = {u["unit"]: u for u in (json.loads(line) for line in handle)}
|
||||
print("arm_outcome rows", len(rows), "units file", len(units))
|
||||
by_family: dict[str, list[tuple[float | None, int, str | None, str]]] = collections.defaultdict(list)
|
||||
for row in rows:
|
||||
unit = units.get(row.get("unit"))
|
||||
if unit:
|
||||
by_family[unit["family"]].append((row.get("s"), unit["label"], unit.get("pair"), str(row.get("verdict") or "")))
|
||||
for family, xs in sorted(by_family.items()):
|
||||
scored = [x for x in xs if x[0] is not None]
|
||||
pairs: dict[str, dict[int, float]] = collections.defaultdict(dict)
|
||||
for score, label, pair, _ in scored:
|
||||
if pair:
|
||||
pairs[pair][label] = score
|
||||
full = [p for p in pairs.values() if 0 in p and 1 in p]
|
||||
wins = sum(1 for p in full if p[1] > p[0])
|
||||
ties = sum(1 for p in full if p[1] == p[0])
|
||||
top = sorted((x for x in scored if x[2]), key=lambda x: -(x[0] or 0.0))
|
||||
first_negative = next((i for i, x in enumerate(top) if x[1] == 0), len(top))
|
||||
print(
|
||||
f"{family}: n={len(xs)} distinct_scores={len({round(x[0] or 0.0, 4) for x in scored})} "
|
||||
f"unsure={sum(1 for x in xs if x[3] == 'unsure')} pairs={len(full)} within_pair_win={wins} tie={ties} "
|
||||
f"pos_above_first_neg={first_negative}"
|
||||
)
|
||||
pooled = sorted((x for xs in by_family.values() for x in xs if x[0] is not None and x[2]), key=lambda x: -(x[0] or 0.0))
|
||||
print("pooled raw top-30 labels:", "".join(str(x[1]) for x in pooled[:30]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"experiment": "exp-004-pooled-block-gate",
|
||||
"date": "2026-10-03",
|
||||
"fill": {
|
||||
"client_calls": 2073,
|
||||
"usd": 1.238223,
|
||||
"ceiling_usd": 2.0,
|
||||
"still_missing": 0,
|
||||
"dry_run_estimate_usd": 5.582,
|
||||
"note": "the dry-run estimator over-priced 4.5x (no prompt-cache pricing); measured spend is the meter's"
|
||||
},
|
||||
"run": {
|
||||
"units": 501,
|
||||
"client_calls": 0,
|
||||
"usd": 0.0,
|
||||
"replayed": 2509,
|
||||
"replay_miss": 0
|
||||
},
|
||||
"verdict": "inconclusive",
|
||||
"reason": "no t_block on any outer fold, pooled or per family; B offers no BLOCK",
|
||||
"instrument_check": {
|
||||
"per_family": {
|
||||
"access_control": {
|
||||
"n": 66,
|
||||
"distinct_scores": 41,
|
||||
"unsure": 5,
|
||||
"pairs": 28,
|
||||
"within_pair_wins": 22,
|
||||
"ties": 1,
|
||||
"positives_above_first_negative": 12
|
||||
},
|
||||
"deserialization": {
|
||||
"n": 64,
|
||||
"distinct_scores": 38,
|
||||
"unsure": 7,
|
||||
"pairs": 25,
|
||||
"within_pair_wins": 18,
|
||||
"ties": 4,
|
||||
"positives_above_first_negative": 5
|
||||
},
|
||||
"injection": {
|
||||
"n": 149,
|
||||
"distinct_scores": 52,
|
||||
"unsure": 54,
|
||||
"pairs": 53,
|
||||
"within_pair_wins": 30,
|
||||
"ties": 17,
|
||||
"positives_above_first_negative": 6
|
||||
},
|
||||
"output_encoding": {
|
||||
"n": 69,
|
||||
"distinct_scores": 31,
|
||||
"unsure": 32,
|
||||
"pairs": 28,
|
||||
"within_pair_wins": 18,
|
||||
"ties": 6,
|
||||
"positives_above_first_negative": 2
|
||||
},
|
||||
"path": {
|
||||
"n": 66,
|
||||
"distinct_scores": 43,
|
||||
"unsure": 15,
|
||||
"pairs": 29,
|
||||
"within_pair_wins": 25,
|
||||
"ties": 3,
|
||||
"positives_above_first_negative": 13
|
||||
},
|
||||
"untrusted_destination": {
|
||||
"n": 87,
|
||||
"distinct_scores": 36,
|
||||
"unsure": 33,
|
||||
"pairs": 32,
|
||||
"within_pair_wins": 18,
|
||||
"ties": 4,
|
||||
"positives_above_first_negative": 3
|
||||
}
|
||||
},
|
||||
"pooled_raw_top30_labels": "111111111110111110111111110111",
|
||||
"reading": "scores are real (31-52 distinct values per family); pooled raw top 30 holds 27 positives (0.90), first negative at rank 12; the Wilson bound needs >= 73 flags with no error, so precision at the top, not data volume alone, now binds"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"experiment": "exp-004-pooled-block-gate",
|
||||
"registered": "2026-10-03",
|
||||
"store": "s3://sast-memory",
|
||||
"manifest": "plane/experiments/exp-004-pooled-block-gate.yaml",
|
||||
"manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
|
||||
"pin": "2ad9c1db7ae673c2a2e48a0afadcaa795dfb1ced",
|
||||
"units_status": "pending",
|
||||
"units_file": "plane/experiments/exp-004-pooled-block-gate.units.jsonl",
|
||||
"units_digest": "",
|
||||
"units": "frozen at run time by `ousast learn experiment units` from the memory after the negatives harvest (merged in 5b2bd9b, 1133 static examples), committed, and registered again before the run; until then the run refuses (check_registered: the units file does not exist)",
|
||||
"requirement": "learned-decision-engine Req 6.4, amended 2026-10-03 by maintainer decision (refines the BLOCK rule of Req 7.3 and 7.4)",
|
||||
"kind": "rule: one zero-cost scoring pass of the six slice-2 incumbent programs; arms differ only in the BLOCK rule",
|
||||
"zero_cost": "the run replays the response cache behind a zero-ceiling meter; a unit with any uncached response is excluded and counted. The harvest changed the memory the programs retrieve from, so a paid cache fill, approved separately with its own ceiling, must precede the run",
|
||||
"normalisation": {
|
||||
"choice": "per-family Platt scaling: learn.calibrate.fit_map with its existing floors (own slope and intercept per family at >= 20 positive and negative repository groups, a pooled slope with a per-family intercept at >= 10, no map below that and so no BLOCK for that family on that fold), fitted on the training folds' paired units only",
|
||||
"why": "a rank-to-quantile map gives every family the same flag rate whatever its precision, so one pooled threshold would flag the top slice of a weak family as readily as of a strong one; Platt puts each family on the scale of its own training precision, which a pooled precision bound needs. The map is a normaliser only: precision is shown by the held-out bound, not by calibration (exp-003: ECE <= 0.05 does not hold per family)"
|
||||
},
|
||||
"arms": {
|
||||
"A": "today's rule: Platt map cross-fitted per family; BLOCK only when the calibration check holds; BLOCK point nested on the other folds' calibrated p (Wilson LB >= 0.95)",
|
||||
"B": "pooled gate: per outer fold j, training units normalised by an inner cross-fit; t_block = the lowest normalised score whose Wilson 95% LB of precision over the training paired flags of all families together is >= 0.95 (unsure never flags); t_quiet per family = the highest normalised score keeping >= 90% of that family's training paired positives; fold j's units BLOCK at p >= t_block, ADVISORY at p >= t_quiet (or without a map or t_quiet), silent below"
|
||||
},
|
||||
"family_floor": {
|
||||
"F": 10,
|
||||
"rule": "a family may block only if its own held-out paired flags (all five folds) number >= 10 with zero errors; otherwise its flags are reported as ADVISORY. The floor uses held-out labels to decide which families block and moves no threshold",
|
||||
"why": "zero errors in 10 flags is unlikely for a family much worse than the bar (probability 0.056 at precision 0.75, 0.107 at 0.80); a larger F is out of reach of the smaller families (16-19 paired positives in exp-003's units over all folds); 10 is exp-003 arm C's training minimum"
|
||||
},
|
||||
"primary_metric": "held-out (outer-fold) Wilson 95% lower bound of flagged precision over arm B's paired flags of all families together, before the family floor (so dropping a failing family cannot raise it)",
|
||||
"populations": {"primary": "paired (both sides of the candidate present)", "secondary": ["pooled (every unit)"]},
|
||||
"secondary_metrics": ["per-family flagged precision and Wilson LB", "the family-floor table", "quiet_recall per family (held-out positives at or above t_quiet / positives) against 0.90", "after-floor totals (what users would get)", "repository-cluster bootstrap 95% CI of flagged precision", "coverage", "flagged"],
|
||||
"decision_rule": "on arm B's held-out paired flags of all families together, before the floor: REJECT B if their point precision < 0.95; else ADOPT B if their Wilson 95% LB >= 0.95 and at least one family passes the floor (>= 10 held-out paired flags, zero errors) -- those families block, every other family stays advisory; else INCONCLUSIVE (the incumbent rule stays)",
|
||||
"power": "the pooled bound needs >= 73 flags with no error (110 with one, 142 with two). On exp-003's units (all 353 scored, result-complete-cache.json) few paired positives rank above the first negative per family on the raw score (access control 10, deserialization 7, injection 2, output encoding 0, path 8, untrusted destination 5); normalisation keeps each family's order, so the pooled top set reaches 73 only if the harvest's pairs separate better. An inconclusive verdict is a plausible outcome",
|
||||
"tests": "tests/test_learn_rules.py: test_one_pooled_threshold_blocks_where_no_family_alone_reaches_the_bound, test_no_held_out_label_moves_a_threshold_or_the_map, test_a_family_with_one_held_out_error_cannot_block, test_quiet_threshold_keeps_ninety_percent_of_positives_and_nothing_is_silenced_blind, test_the_pooled_verdict_follows_the_registered_rule, test_the_exp004_manifest_registers_the_procedure_before_its_units",
|
||||
"status": "registered 2026-10-03, units pending, not run"
|
||||
}
|
||||
@@ -0,0 +1,822 @@
|
||||
{
|
||||
"arms": {
|
||||
"A": {
|
||||
"label": "today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p",
|
||||
"rule": {
|
||||
"name": "calibrated_block"
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"label": "pooled gate -- one t_block over all families on a per-family Platt score (training folds, inner cross-fit), t_quiet per family at 90% recall, family floor 10 clean held-out flags",
|
||||
"rule": {
|
||||
"floor": 10,
|
||||
"name": "pooled_block_gate",
|
||||
"normalise": "platt",
|
||||
"precision": 0.95,
|
||||
"recall": 0.9,
|
||||
"select_on": "paired"
|
||||
}
|
||||
}
|
||||
},
|
||||
"code_commit": "90a2642019df414cb67b5dcea6e4ec235f35650a",
|
||||
"decision": "inconclusive",
|
||||
"experiment": "exp-004-pooled-block-gate",
|
||||
"families": {
|
||||
"access_control": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.0715,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 28,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 28,
|
||||
"true_flagged": 0,
|
||||
"units": 56
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 37,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 38,
|
||||
"true_flagged": 0,
|
||||
"units": 66
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"floor": {
|
||||
"errors": 0,
|
||||
"flagged": 0,
|
||||
"floor": 10,
|
||||
"may_block": false
|
||||
},
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 28,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 28,
|
||||
"true_flagged": 0,
|
||||
"units": 56
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 37,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 38,
|
||||
"true_flagged": 0,
|
||||
"units": 66
|
||||
},
|
||||
"quiet_recall": {
|
||||
"paired": 0.8929,
|
||||
"pooled": 0.9211
|
||||
},
|
||||
"t_block_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
},
|
||||
"t_quiet_by_fold": {
|
||||
"outer-0": 0.44900641988143913,
|
||||
"outer-1": 0.3723384028044276,
|
||||
"outer-2": 0.4202772526614184,
|
||||
"outer-3": 0.4481849709797749,
|
||||
"outer-4": 0.36346174463707154
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 66,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 22,
|
||||
"outer-1": 22,
|
||||
"outer-2": 22,
|
||||
"outer-3": 23,
|
||||
"outer-4": 23
|
||||
}
|
||||
}
|
||||
},
|
||||
"deserialization": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.0725,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 25,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 25,
|
||||
"true_flagged": 0,
|
||||
"units": 50
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 39,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 39,
|
||||
"true_flagged": 0,
|
||||
"units": 64
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"floor": {
|
||||
"errors": 0,
|
||||
"flagged": 0,
|
||||
"floor": 10,
|
||||
"may_block": false
|
||||
},
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 25,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 25,
|
||||
"true_flagged": 0,
|
||||
"units": 50
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 39,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 39,
|
||||
"true_flagged": 0,
|
||||
"units": 64
|
||||
},
|
||||
"quiet_recall": {
|
||||
"paired": 0.88,
|
||||
"pooled": 0.9231
|
||||
},
|
||||
"t_block_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
},
|
||||
"t_quiet_by_fold": {
|
||||
"outer-0": 0.17507194490500488,
|
||||
"outer-1": 0.22766488322270723,
|
||||
"outer-2": 0.2163745870743959,
|
||||
"outer-3": 0.3112201878893598,
|
||||
"outer-4": 0.363243722721776
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 64,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 20,
|
||||
"outer-1": 20,
|
||||
"outer-2": 20,
|
||||
"outer-3": 20,
|
||||
"outer-4": 20
|
||||
}
|
||||
}
|
||||
},
|
||||
"injection": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.1042,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 41,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 53,
|
||||
"true_flagged": 0,
|
||||
"units": 106
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 49,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 78,
|
||||
"true_flagged": 0,
|
||||
"units": 149
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"floor": {
|
||||
"errors": 0,
|
||||
"flagged": 0,
|
||||
"floor": 10,
|
||||
"may_block": false
|
||||
},
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 41,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 53,
|
||||
"true_flagged": 0,
|
||||
"units": 106
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 49,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 78,
|
||||
"true_flagged": 0,
|
||||
"units": 149
|
||||
},
|
||||
"quiet_recall": {
|
||||
"paired": 0.8679,
|
||||
"pooled": 0.8974
|
||||
},
|
||||
"t_block_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
},
|
||||
"t_quiet_by_fold": {
|
||||
"outer-0": 0.287983504097999,
|
||||
"outer-1": 0.17524866355153343,
|
||||
"outer-2": 0.12918182665453967,
|
||||
"outer-3": 0.14521040470628938,
|
||||
"outer-4": 0.23005714687697673
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 149,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 38,
|
||||
"outer-1": 44,
|
||||
"outer-2": 45,
|
||||
"outer-3": 43,
|
||||
"outer-4": 42
|
||||
}
|
||||
}
|
||||
},
|
||||
"output_encoding": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.0736,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 28,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 28,
|
||||
"true_flagged": 0,
|
||||
"units": 56
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 39,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 41,
|
||||
"true_flagged": 0,
|
||||
"units": 69
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"floor": {
|
||||
"errors": 0,
|
||||
"flagged": 0,
|
||||
"floor": 10,
|
||||
"may_block": false
|
||||
},
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 28,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 28,
|
||||
"true_flagged": 0,
|
||||
"units": 56
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 39,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 41,
|
||||
"true_flagged": 0,
|
||||
"units": 69
|
||||
},
|
||||
"quiet_recall": {
|
||||
"paired": 0.8571,
|
||||
"pooled": 0.9024
|
||||
},
|
||||
"t_block_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
},
|
||||
"t_quiet_by_fold": {
|
||||
"outer-0": 0.3723166587726901,
|
||||
"outer-1": 0.5329072757249385,
|
||||
"outer-2": 0.3785066578507705,
|
||||
"outer-3": 0.4389011844848602,
|
||||
"outer-4": 0.43855966643440264
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 69,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 23,
|
||||
"outer-1": 22,
|
||||
"outer-2": 22,
|
||||
"outer-3": 22,
|
||||
"outer-4": 23
|
||||
}
|
||||
}
|
||||
},
|
||||
"path": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.0621,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 29,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 29,
|
||||
"true_flagged": 0,
|
||||
"units": 58
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 37,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 37,
|
||||
"true_flagged": 0,
|
||||
"units": 66
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"floor": {
|
||||
"errors": 0,
|
||||
"flagged": 0,
|
||||
"floor": 10,
|
||||
"may_block": false
|
||||
},
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 29,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 29,
|
||||
"true_flagged": 0,
|
||||
"units": 58
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 37,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 37,
|
||||
"true_flagged": 0,
|
||||
"units": 66
|
||||
},
|
||||
"quiet_recall": {
|
||||
"paired": 0.8621,
|
||||
"pooled": 0.8919
|
||||
},
|
||||
"t_block_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
},
|
||||
"t_quiet_by_fold": {
|
||||
"outer-0": 0.4680507207548809,
|
||||
"outer-1": 0.3955717127313657,
|
||||
"outer-2": 0.43694703358760895,
|
||||
"outer-3": 0.4450583743716199,
|
||||
"outer-4": 0.46396657833309796
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 66,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 23,
|
||||
"outer-1": 23,
|
||||
"outer-2": 24,
|
||||
"outer-3": 23,
|
||||
"outer-4": 23
|
||||
}
|
||||
}
|
||||
},
|
||||
"untrusted_destination": {
|
||||
"A": {
|
||||
"calibration_holds": false,
|
||||
"ece": 0.1143,
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 31,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 32,
|
||||
"true_flagged": 0,
|
||||
"units": 64
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 49,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 50,
|
||||
"true_flagged": 0,
|
||||
"units": 87
|
||||
},
|
||||
"reason_by_fold": {
|
||||
"outer-0": "calibration",
|
||||
"outer-1": "calibration",
|
||||
"outer-2": "calibration",
|
||||
"outer-3": "calibration",
|
||||
"outer-4": "calibration"
|
||||
},
|
||||
"thresholds_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"floor": {
|
||||
"errors": 0,
|
||||
"flagged": 0,
|
||||
"floor": 10,
|
||||
"may_block": false
|
||||
},
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 31,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 32,
|
||||
"true_flagged": 0,
|
||||
"units": 64
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 49,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 50,
|
||||
"true_flagged": 0,
|
||||
"units": 87
|
||||
},
|
||||
"quiet_recall": {
|
||||
"paired": 1.0,
|
||||
"pooled": 1.0
|
||||
},
|
||||
"t_block_by_fold": {
|
||||
"outer-0": null,
|
||||
"outer-1": null,
|
||||
"outer-2": null,
|
||||
"outer-3": null,
|
||||
"outer-4": null
|
||||
},
|
||||
"t_quiet_by_fold": {
|
||||
"outer-0": 0.13676746287025585,
|
||||
"outer-1": 0.11255775703980551,
|
||||
"outer-2": 0.13763698150768486,
|
||||
"outer-3": 0.09787938647374639,
|
||||
"outer-4": 0.13763698150768486
|
||||
}
|
||||
},
|
||||
"capacity": {
|
||||
"evaluated": 87,
|
||||
"excluded_replay_miss": 0,
|
||||
"not_run": 0,
|
||||
"paired_positives_by_training_folds": {
|
||||
"outer-0": 26,
|
||||
"outer-1": 26,
|
||||
"outer-2": 24,
|
||||
"outer-3": 26,
|
||||
"outer-4": 26
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"hypothesis": "One BLOCK threshold pooled across families on a per-family Platt-normalised score, chosen on training folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95, reaches that bound on held-out repositories' paired units, where per-family thresholds (exp-003) could not for lack of flags; a per-family floor (>= 10 held-out flags, none wrong) keeps a family that the pooling masks from blocking.\n",
|
||||
"manifest_digest": "87fceb6580c6de629d6a6089eb5888cdfc4df0f791576184f987c14c192d85f5",
|
||||
"min_flagged_for_bound": 73,
|
||||
"overall": {
|
||||
"A": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 182,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 195,
|
||||
"true_flagged": 0,
|
||||
"units": 390
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 220,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 283,
|
||||
"true_flagged": 0,
|
||||
"units": 501
|
||||
}
|
||||
},
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 182,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 195,
|
||||
"true_flagged": 0,
|
||||
"units": 390
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 220,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 283,
|
||||
"true_flagged": 0,
|
||||
"units": 501
|
||||
}
|
||||
}
|
||||
},
|
||||
"overall_after_floor": {
|
||||
"B": {
|
||||
"paired": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 182,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 195,
|
||||
"true_flagged": 0,
|
||||
"units": 390
|
||||
},
|
||||
"pooled": {
|
||||
"coverage": 0.0,
|
||||
"flagged": 0,
|
||||
"flagged_precision": null,
|
||||
"flagged_precision_ci": null,
|
||||
"flagged_precision_lb": null,
|
||||
"groups": 220,
|
||||
"meets_target_lb": false,
|
||||
"meets_target_point": false,
|
||||
"positives": 283,
|
||||
"true_flagged": 0,
|
||||
"units": 501
|
||||
}
|
||||
}
|
||||
},
|
||||
"populations": {
|
||||
"primary": "paired",
|
||||
"secondary": [
|
||||
"pooled"
|
||||
]
|
||||
},
|
||||
"primary": [
|
||||
{
|
||||
"name": "flagged_precision_lb",
|
||||
"target": 0.95
|
||||
}
|
||||
],
|
||||
"reason": "B offers no BLOCK on held-out paired units; the incumbent stays",
|
||||
"resamples": 2000,
|
||||
"secondary": [
|
||||
{
|
||||
"name": "flagged_precision"
|
||||
},
|
||||
{
|
||||
"name": "coverage"
|
||||
},
|
||||
{
|
||||
"name": "flagged"
|
||||
},
|
||||
{
|
||||
"name": "quiet_recall"
|
||||
}
|
||||
],
|
||||
"spend_usd": {
|
||||
"score": 0.0
|
||||
},
|
||||
"target_precision": 0.95,
|
||||
"units": {
|
||||
"by_label": {
|
||||
"0": 218,
|
||||
"1": 283
|
||||
},
|
||||
"evaluated": 501,
|
||||
"excluded_replay_miss": 0,
|
||||
"frozen": 501,
|
||||
"groups": 220,
|
||||
"not_run": 0,
|
||||
"paired_evaluated": 390,
|
||||
"unsure": 146
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
{
|
||||
"arms": [
|
||||
"B"
|
||||
],
|
||||
"client_calls": 2394,
|
||||
"experiment": "exp-005-graph-slice",
|
||||
"families": {
|
||||
"access_control": {
|
||||
"cached": 0,
|
||||
"conditional_retries": 330,
|
||||
"estimate_method": "estimated",
|
||||
"estimated_output_tokens": 300,
|
||||
"estimated_usd": 0.9956892000000007,
|
||||
"filled": 330,
|
||||
"missing": 330,
|
||||
"requests": 330,
|
||||
"still_missing": 0,
|
||||
"units": 66
|
||||
},
|
||||
"deserialization": {
|
||||
"cached": 0,
|
||||
"conditional_retries": 310,
|
||||
"estimate_method": "estimated",
|
||||
"estimated_output_tokens": 300,
|
||||
"estimated_usd": 0.8843295999999988,
|
||||
"filled": 310,
|
||||
"missing": 310,
|
||||
"requests": 310,
|
||||
"still_missing": 0,
|
||||
"units": 64
|
||||
},
|
||||
"injection": {
|
||||
"cached": 0,
|
||||
"conditional_retries": 675,
|
||||
"estimate_method": "estimated",
|
||||
"estimated_output_tokens": 300,
|
||||
"estimated_usd": 2.270278999999999,
|
||||
"filled": 675,
|
||||
"missing": 675,
|
||||
"requests": 675,
|
||||
"still_missing": 0,
|
||||
"units": 149
|
||||
},
|
||||
"output_encoding": {
|
||||
"cached": 0,
|
||||
"conditional_retries": 325,
|
||||
"estimate_method": "estimated",
|
||||
"estimated_output_tokens": 300,
|
||||
"estimated_usd": 1.0014180000000004,
|
||||
"filled": 328,
|
||||
"missing": 325,
|
||||
"requests": 328,
|
||||
"still_missing": 0,
|
||||
"units": 69
|
||||
},
|
||||
"path": {
|
||||
"cached": 0,
|
||||
"conditional_retries": 320,
|
||||
"estimate_method": "estimated",
|
||||
"estimated_output_tokens": 300,
|
||||
"estimated_usd": 0.9399192000000005,
|
||||
"filled": 325,
|
||||
"missing": 320,
|
||||
"requests": 325,
|
||||
"still_missing": 0,
|
||||
"units": 66
|
||||
},
|
||||
"untrusted_destination": {
|
||||
"cached": 0,
|
||||
"conditional_retries": 425,
|
||||
"estimate_method": "estimated",
|
||||
"estimated_output_tokens": 300,
|
||||
"estimated_usd": 1.174316,
|
||||
"filled": 426,
|
||||
"missing": 425,
|
||||
"requests": 426,
|
||||
"still_missing": 0,
|
||||
"units": 87
|
||||
}
|
||||
},
|
||||
"reason": "",
|
||||
"status": "done",
|
||||
"usd": 1.5890382240000016
|
||||
}
|
||||
@@ -0,0 +1,165 @@
|
||||
{
|
||||
"alpha": 0.05,
|
||||
"arms": {
|
||||
"A": {
|
||||
"budget_usd": 0.0,
|
||||
"label": "six slice-2 incumbents unchanged, replaying exp-004's response cache",
|
||||
"repeats": 1,
|
||||
"replay_only": true
|
||||
},
|
||||
"B": {
|
||||
"budget_usd": 6.0,
|
||||
"label": "the same six programs with the in-function slice added to their prompts",
|
||||
"repeats": 1,
|
||||
"replay_only": false
|
||||
}
|
||||
},
|
||||
"code_commit": "0acbc7ec41dcaeca805c4720bc3f6d02cb940ee7",
|
||||
"decision": "inconclusive",
|
||||
"experiment": "exp-005-graph-slice",
|
||||
"hypothesis": "Adding the in-function def-use slice to the prompt improves within-pair ranking of the six slice-2 incumbent programs on held-out repositories, at no more than 1.5 times the cost per candidate.\n",
|
||||
"looks": [
|
||||
{
|
||||
"boundary": 1.959964,
|
||||
"crossed": false,
|
||||
"estimate": {
|
||||
"a": 0.7615384615384615,
|
||||
"b": 0.7461538461538462,
|
||||
"ci": [
|
||||
-0.0628019323671497,
|
||||
0.03155339805825241
|
||||
],
|
||||
"diff": -0.01538461538461533,
|
||||
"groups": 220,
|
||||
"metric": "within_pair_auc",
|
||||
"resamples": 10000,
|
||||
"se": 0.024030780235707972,
|
||||
"units": 501
|
||||
},
|
||||
"fraction": 1.0,
|
||||
"groups": 220,
|
||||
"z": -0.6402045723740141
|
||||
}
|
||||
],
|
||||
"manifest_digest": "1b1ec814cde12801491376d2086c8c2d6db70f4a86fc308c0adc5a530398e554",
|
||||
"mde": 0.0,
|
||||
"primary": [
|
||||
{
|
||||
"name": "within_pair_auc"
|
||||
}
|
||||
],
|
||||
"reason": "the CI of within_pair_auc spans 0; the incumbent stays",
|
||||
"resamples": 10000,
|
||||
"secondary": [
|
||||
{
|
||||
"name": "top_30_precision"
|
||||
},
|
||||
{
|
||||
"name": "unsure_rate"
|
||||
},
|
||||
{
|
||||
"name": "usd_per_candidate"
|
||||
}
|
||||
],
|
||||
"spend_usd": {
|
||||
"A": 1.444503,
|
||||
"B": 1.589038
|
||||
},
|
||||
"tests": {
|
||||
"top_30_precision": {
|
||||
"ci_vs_zero": "spans",
|
||||
"direction": 1,
|
||||
"estimate": {
|
||||
"a": 1.0,
|
||||
"b": 1.0,
|
||||
"ci": [
|
||||
0.0,
|
||||
0.0
|
||||
],
|
||||
"diff": 0.0,
|
||||
"groups": 220,
|
||||
"metric": "top_30_precision",
|
||||
"resamples": 10000,
|
||||
"se": 0.0006666666666666664,
|
||||
"units": 501
|
||||
},
|
||||
"mcnemar": null,
|
||||
"target": null
|
||||
},
|
||||
"unsure_rate": {
|
||||
"ci_vs_zero": "favour",
|
||||
"direction": -1,
|
||||
"estimate": {
|
||||
"a": 0.29141716566866266,
|
||||
"b": 0.23952095808383234,
|
||||
"ci": [
|
||||
-0.08583690987124465,
|
||||
-0.01996007984031939
|
||||
],
|
||||
"diff": -0.05189620758483032,
|
||||
"groups": 220,
|
||||
"metric": "unsure_rate",
|
||||
"resamples": 10000,
|
||||
"se": 0.01685991406098136,
|
||||
"units": 501
|
||||
},
|
||||
"mcnemar": {
|
||||
"b": 47,
|
||||
"c": 21,
|
||||
"concordant": 433,
|
||||
"p_value": 0.0021862431776861705
|
||||
},
|
||||
"target": null
|
||||
},
|
||||
"usd_per_candidate": {
|
||||
"ci_vs_zero": "against",
|
||||
"direction": -1,
|
||||
"estimate": {
|
||||
"a": 0.002883239520958083,
|
||||
"b": 0.0031717325349301397,
|
||||
"ci": [
|
||||
0.0002458205128205199,
|
||||
0.0003319800399201574
|
||||
],
|
||||
"diff": 0.0002884930139720565,
|
||||
"groups": 220,
|
||||
"metric": "usd_per_candidate",
|
||||
"resamples": 10000,
|
||||
"se": 2.1746161270931252e-05,
|
||||
"units": 501
|
||||
},
|
||||
"mcnemar": null,
|
||||
"target": null
|
||||
},
|
||||
"within_pair_auc": {
|
||||
"ci_vs_zero": "spans",
|
||||
"direction": 1,
|
||||
"estimate": {
|
||||
"a": 0.7615384615384615,
|
||||
"b": 0.7461538461538462,
|
||||
"ci": [
|
||||
-0.0628019323671497,
|
||||
0.03155339805825241
|
||||
],
|
||||
"diff": -0.01538461538461533,
|
||||
"groups": 220,
|
||||
"metric": "within_pair_auc",
|
||||
"resamples": 10000,
|
||||
"se": 0.024030780235707972,
|
||||
"units": 501
|
||||
},
|
||||
"mcnemar": null,
|
||||
"target": null
|
||||
}
|
||||
},
|
||||
"tost": null,
|
||||
"units": {
|
||||
"by_label": {
|
||||
"0": 218,
|
||||
"1": 283
|
||||
},
|
||||
"groups": 220,
|
||||
"outcomes": 1002,
|
||||
"paired": 501
|
||||
}
|
||||
}
|
||||
@@ -7,7 +7,7 @@ from:
|
||||
- quick rules and deterministic roles, run here on the unit's tree: a pair side materialised from its excerpt, a
|
||||
population or recipe pin exported from its clone or checkout (one at a time, deleted after, disk-guarded);
|
||||
- the engine: ``benchmarks/learn/engine_pairs.py`` records for pairs, the recorded protocol scans
|
||||
(``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>.json``) for populations;
|
||||
(``~/ousast-results/independent-v{1,2}/scans/<case>--<pin role>[_a|_b].json``) for populations;
|
||||
- verify passes a/b and ``agree`` from the harvest Run (``harvest-verify``) and, for population v2, the recorded
|
||||
``validation-46`` Run; model roles from ``harvest-roles``;
|
||||
- callers and entry-point distance on a repository (``not_applicable`` on a pair excerpt: no repository exists).
|
||||
@@ -21,6 +21,11 @@ An instrument that has not run for a unit yet (the engine job still running, a R
|
||||
memory store (``OUSAST_MEMORY``, default ``~/ousast-results/plane/memory``) as kind ``features``; a rerun replaces
|
||||
them by id. ``--counts`` writes the counts-only coverage record (no candidate, path or repository name).
|
||||
|
||||
``--dry-run`` instead compares existing local store examples with the corrected engine features, printing
|
||||
per-source transitions from not-run/none to ran (and recorded failures separately). It reads only the spent
|
||||
v1/v2 manifests, saved scans and local Git objects; it writes neither the store nor ``--counts`` and does not
|
||||
rebuild other instruments. No label snapshot, harvest units or scratch directory is needed in this mode.
|
||||
|
||||
Usage::
|
||||
|
||||
python benchmarks/learn/build_features.py --labels labels-<sha>.jsonl --units <harvest plane>/units.json --counts counts.json
|
||||
@@ -30,9 +35,11 @@ from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import tomllib
|
||||
from collections import Counter, defaultdict
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
@@ -61,6 +68,7 @@ from openultrasast.learn.features import (
|
||||
source_part,
|
||||
verify_part,
|
||||
)
|
||||
from openultrasast.learn.labels import Clone
|
||||
from openultrasast.learn.roles import RoleSet, from_model_roles, infer_for_checkout
|
||||
from openultrasast.pairs import DEFAULT_CATALOG, _materialize_side, _targets, load_pair_catalog
|
||||
from openultrasast.plane import memory
|
||||
@@ -89,6 +97,109 @@ def read_json(path: Path) -> Any:
|
||||
return json.loads(path.read_text(encoding="utf-8")) if path.is_file() else None
|
||||
|
||||
|
||||
def resolve_scan(scans: Path, case: str, side: str) -> dict[str, Any] | None:
|
||||
"""Read the v1 single scan, otherwise combine the v2 repeats for this side."""
|
||||
single = read_json(scans / f"{case}--{side}.json")
|
||||
if single is not None:
|
||||
return dict(single)
|
||||
repeats = [read_json(scans / f"{case}--{side}_{repeat}.json") for repeat in ("a", "b")]
|
||||
available = [r for r in repeats if r is not None]
|
||||
if not available:
|
||||
return None
|
||||
# Identical rule for vulnerable and fixed: a finding in either repeat counts;
|
||||
# exact duplicates count once. engine_part takes the highest rung and minimum
|
||||
# witness steps across the union. Completion is sum(completed)/sum(questions).
|
||||
# A successful repeat survives a failed/missing peer, which marks degradation.
|
||||
ran = [r for r in available if engine_for(r) is None]
|
||||
findings = {json.dumps(f, sort_keys=True): f for r in ran for f in r.get("findings", [])}
|
||||
degradations = sorted({d for r in available for d in r.get("degradations", [])})
|
||||
if len(ran) != len(repeats):
|
||||
degradations.append("repeat_missing_or_failed")
|
||||
return {
|
||||
"state": "ran" if ran else "none" if all(engine_for(r) == NONE for r in available) else "failed",
|
||||
"questions": sum(int(r.get("questions") or 0) for r in available),
|
||||
"completed": sum(int(r.get("completed") or 0) for r in available),
|
||||
"findings": list(findings.values()),
|
||||
"degradations": degradations,
|
||||
"image": "+".join(sorted({str(r.get("image") or "recorded") for r in available})),
|
||||
}
|
||||
|
||||
|
||||
def recorded_engine(language: str, findings: list, result: Mapping | None, repository: bool) -> Part:
|
||||
# Recorded findings take precedence over the shared engine language coverage.
|
||||
if language not in ENGINE_LANGUAGES and not findings:
|
||||
return NONE
|
||||
state = engine_for(result)
|
||||
if state is not None:
|
||||
return state
|
||||
result = result or {}
|
||||
questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
|
||||
return engine_part(
|
||||
findings, completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
|
||||
vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
|
||||
) # fmt: skip
|
||||
|
||||
|
||||
class PinSourceIndex(SourceIndex):
|
||||
"""Resolve findings with the same matcher as a checkout, without exporting it."""
|
||||
|
||||
def __init__(self, clone: Path, pin: str) -> None:
|
||||
super().__init__(clone)
|
||||
self.clone, self.pin = Clone(clone), pin
|
||||
|
||||
def lines(self, path: str) -> list[str]:
|
||||
if path not in self._lines:
|
||||
self._lines[path] = self.clone.git("show", f"{self.pin}:{path}").splitlines()
|
||||
return self._lines[path] or []
|
||||
|
||||
|
||||
def dry_run(store: memory.FileStore, results: Path, cache: Path) -> dict[str, Any]:
|
||||
"""Count existing examples whose engine becomes available; never write rows or blobs.
|
||||
|
||||
Only spent v1/v2 populations are read. Counts are examples, not the two
|
||||
static/plane feature records. Other sources are reported with zero changes.
|
||||
"""
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
cases = {}
|
||||
for population in ("population-v1", "population-v2"):
|
||||
manifest = root / "benchmarks" / "independent" / f"{population}.toml"
|
||||
for case in tomllib.loads(manifest.read_text(encoding="utf-8"))["case"]:
|
||||
for side in ("vulnerable", "fixed"):
|
||||
cases[(population, memory.repo_key(case["repo"]), case[side], side)] = case["id"]
|
||||
rows = [r.row for r in store.rows(kind="example")]
|
||||
if not rows:
|
||||
raise ValueError("dry run read no examples from the store")
|
||||
counts: dict[str, Counter[str]] = defaultdict(Counter)
|
||||
for row in rows:
|
||||
count = counts[row["source"]]
|
||||
count["examples"] += 1
|
||||
old = row["instruments"]["engine"]
|
||||
reason = "not_run" if old.get("version") == "missing:not-run" else "none" if old["state"] == "none" else None
|
||||
if reason is None:
|
||||
continue
|
||||
key = (row["split"], row["repo"], row["pin"], row["pin_role"])
|
||||
case_id = cases.get(key)
|
||||
if case_id is None:
|
||||
continue
|
||||
scans = results / row["split"].replace("population", "independent") / "scans"
|
||||
result = resolve_scan(scans, case_id, row["pin_role"])
|
||||
path, _, function = row["candidate"].partition("::")
|
||||
engine = {}
|
||||
if engine_for(result) is None:
|
||||
index = PinSourceIndex(cache / "independent" / case_id, row["pin"])
|
||||
lines = index.lines(path) # fail loudly if the candidate's input cannot be read
|
||||
count["source_bytes_read"] += len("\n".join(lines).encode("utf-8"))
|
||||
selected = dict(result or {})
|
||||
selected["findings"] = [f for f in selected.get("findings", []) if f["site"].partition(":")[0] == path]
|
||||
_, engine = _signals(index, [], selected, {})
|
||||
part = recorded_engine(row["language"], engine.get((path, function, row["family"]), []), result, True)
|
||||
if part.state == "ran":
|
||||
count[f"{reason}_to_ran"] += 1
|
||||
elif result is not None and part.state == "failed":
|
||||
count[f"{reason}_to_recorded_failed"] += 1
|
||||
return {source: {**{"not_run_to_ran": 0, "none_to_ran": 0}, **dict(count)} for source, count in sorted(counts.items())}
|
||||
|
||||
|
||||
def done(run: Path, entry: str) -> bool:
|
||||
state = (read_json(run / "state.json") or {}).get("tasks", {})
|
||||
return state.get(entry, {}).get("status") == "done"
|
||||
@@ -188,17 +299,7 @@ def static_parts(root: Path, index: SourceIndex, quick: Mapping, engine: Mapping
|
||||
key = (path, function, family)
|
||||
parts: dict[str, Part] = {}
|
||||
parts["quick"] = quick_part(quick.get(key, []), rules, quick_version) if covers(QUICK_LANGUAGES, language) else NONE
|
||||
if language not in ENGINE_LANGUAGES:
|
||||
parts["engine"] = NONE
|
||||
elif engine_state is not None:
|
||||
parts["engine"] = engine_state
|
||||
else:
|
||||
result = engine_result or {}
|
||||
questions, completed = int(result.get("questions") or 0), int(result.get("completed") or 0)
|
||||
parts["engine"] = engine_part(
|
||||
engine.get(key, []), completion=completed / questions if questions else None, degraded=bool(result.get("degradations")),
|
||||
vocabulary=Vocabulary.load(language), version=str(result.get("image") or "recorded"),
|
||||
) # fmt: skip
|
||||
parts["engine"] = recorded_engine(language, engine.get(key, []), engine_result, repository)
|
||||
lines = index.lines(path)
|
||||
parts["source"] = source_part(lines, language, function)
|
||||
parts["roles"] = roles_part(function_of(lines, path, language, function), inferred, language)
|
||||
@@ -237,21 +338,34 @@ def export(clone: Path, pin: str, target: Path) -> bool:
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--labels", type=Path, required=True)
|
||||
parser.add_argument("--units", type=Path, required=True, help="units.json of the harvest plane")
|
||||
parser.add_argument("--labels", type=Path)
|
||||
parser.add_argument("--units", type=Path, help="units.json of the harvest plane")
|
||||
parser.add_argument("--catalog", type=Path, default=DEFAULT_CATALOG)
|
||||
parser.add_argument("--cache", type=Path, default=Path.home() / ".cache" / "openultrasast")
|
||||
parser.add_argument("--engine", type=Path, default=RESULTS / "plane" / "harvest-engine")
|
||||
parser.add_argument("--verify-run", type=Path, default=RESULTS / "plane" / "harvest-verify")
|
||||
parser.add_argument("--roles-run", type=Path, default=RESULTS / "plane" / "harvest-roles")
|
||||
parser.add_argument("--v46", type=Path, default=RESULTS / "plane" / "validation-46")
|
||||
parser.add_argument("--scratch", type=Path, required=True)
|
||||
parser.add_argument("--scratch", type=Path)
|
||||
parser.add_argument("--min-free-gb", type=float, default=1.2)
|
||||
parser.add_argument("--counts", type=Path, required=True)
|
||||
parser.add_argument("--counts", type=Path)
|
||||
parser.add_argument("--store", help="OUSAST_MEMORY spec (default: the environment's)")
|
||||
parser.add_argument("--limit", type=int, default=0, help="stop after this many targets (a test)")
|
||||
parser.add_argument("--kinds", default="pairs,pin", help="which targets: pairs, pin (population and recipe pins)")
|
||||
parser.add_argument("--results-root", type=Path, default=RESULTS)
|
||||
parser.add_argument("--dry-run", action="store_true", help="read local stored examples and print engine changes by source; no writes")
|
||||
args = parser.parse_args()
|
||||
if args.dry_run:
|
||||
spec = args.store or os.environ.get("OUSAST_MEMORY") or ""
|
||||
if spec and not spec.startswith("file://"):
|
||||
parser.error("--dry-run requires a local file:// store (no network)")
|
||||
store = memory.open_store(spec)
|
||||
if not isinstance(store, memory.FileStore):
|
||||
parser.error("--dry-run requires a local file:// store (no network)")
|
||||
print(json.dumps({"dry_run": True, "by_source": dry_run(store, args.results_root, args.cache)}, sort_keys=True))
|
||||
return 0
|
||||
if any(getattr(args, name) is None for name in ("labels", "units", "scratch", "counts")):
|
||||
parser.error("--labels, --units, --scratch and --counts are required unless --dry-run is used")
|
||||
|
||||
labels = [r for r in jsonl(args.labels) if r["unit"] == "pin" and r["source"] not in CONDITIONAL_SOURCES and not r.get("conditional")]
|
||||
units = json.loads(args.units.read_text(encoding="utf-8"))
|
||||
@@ -314,7 +428,7 @@ def main() -> int:
|
||||
continue
|
||||
split = {r["split"] for r in rows} | {unit["source"]}
|
||||
population = "independent-v1" if "population-v1" in split else "independent-v2"
|
||||
engine_result = read_json(RESULTS / population / "scans" / f"{case_id}--{rows[0]['pin_role']}.json")
|
||||
engine_result = resolve_scan(args.results_root / population / "scans", case_id, rows[0]["pin_role"])
|
||||
stems = [(args.verify_run, n) for n in names] + [(args.v46, unit["ref"])]
|
||||
owner = next(
|
||||
(n for n in names if f"{n}-roles" in {e for e in (read_json(args.roles_run / "state.json") or {}).get("tasks", {})}),
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
"""Read-only, zero-model slice audit. Run: env -u OPENROUTER_API_KEY python benchmarks/learn/slice_leak_audit.py.
|
||||
|
||||
Only sources.toml itself is read, never its referenced corpora. Stored source IDs
|
||||
are the reporting strata (the store merges advisory pair catalogs into `pairs`).
|
||||
Ties get half credit. Both signed and best-direction scores are recorded; like
|
||||
learn.leaks, a reversed label shortcut is still a leak. Missing classes/pairs
|
||||
produce null metrics, not plausible zeros. Unsupported slices score zero for both
|
||||
indicators and remain separately counted in coverage.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from collections import Counter, defaultdict
|
||||
from collections.abc import Mapping, Sequence
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from openultrasast.config import load_dotenv
|
||||
from openultrasast.learn.examples import load_examples
|
||||
from openultrasast.learn.labels import load_sources
|
||||
from openultrasast.learn.slice import function_slice
|
||||
from openultrasast.plane.memory import MemoryStore, open_store
|
||||
|
||||
UNITS = Path("plane/experiments/exp-004-pooled-block-gate.units.jsonl")
|
||||
OUTPUT = Path("benchmarks/measurements/2026-10-03-graph-slice-leak-audit/record.json")
|
||||
THRESHOLD = 0.70
|
||||
|
||||
|
||||
def complete_pairs(rows: Sequence[Mapping[str, Any]]) -> list[tuple[Mapping[str, Any], Mapping[str, Any]]]:
|
||||
groups: dict[tuple[str, str], list[Mapping[str, Any]]] = defaultdict(list)
|
||||
for row in rows:
|
||||
if row.get("pair"):
|
||||
groups[(row["family"], row["pair"])].append(row)
|
||||
pairs = []
|
||||
for members in groups.values():
|
||||
if len(members) == 2 and {m["label"] for m in members} == {0, 1}:
|
||||
v, f = sorted(members, key=lambda m: -m["label"])
|
||||
if v["group"] != f["group"] or v["fold"] != f["fold"]:
|
||||
raise ValueError("Pair crosses group or fold")
|
||||
pairs.append((v, f))
|
||||
elif len(members) != 1:
|
||||
raise ValueError("Ambiguous pair")
|
||||
return pairs
|
||||
|
||||
|
||||
def metrics(rows: Sequence[Mapping[str, Any]], signal: str) -> dict[str, Any]:
|
||||
pairs = complete_pairs(rows)
|
||||
higher = sum(v[signal] > f[signal] for v, f in pairs)
|
||||
lower = sum(v[signal] < f[signal] for v, f in pairs)
|
||||
ties = len(pairs) - higher - lower
|
||||
pair_score = (higher + 0.5 * ties) / len(pairs) if pairs else None
|
||||
positive = [r[signal] for r in rows if r["label"] == 1]
|
||||
negative = [r[signal] for r in rows if r["label"] == 0]
|
||||
auc = (
|
||||
sum((v > f) + 0.5 * (v == f) for v in positive for f in negative) / (len(positive) * len(negative))
|
||||
if positive and negative
|
||||
else None
|
||||
)
|
||||
best_pair = max(pair_score, 1 - pair_score) if pair_score is not None else None
|
||||
best_auc = max(auc, 1 - auc) if auc is not None else None
|
||||
return {
|
||||
"rows": len(rows),
|
||||
"pairs": len(pairs),
|
||||
"positive": len(positive),
|
||||
"negative": len(negative),
|
||||
"higher_on_vulnerable": higher,
|
||||
"higher_on_fixed": lower,
|
||||
"ties": ties,
|
||||
"within_pair_accuracy": pair_score,
|
||||
"pooled_auc": auc,
|
||||
"best_direction_within_pair_accuracy": best_pair,
|
||||
"best_direction_pooled_auc": best_auc,
|
||||
"flagged": any(v is not None and v >= THRESHOLD for v in (best_pair, best_auc)),
|
||||
}
|
||||
|
||||
|
||||
def audit(store: MemoryStore, units: Sequence[Mapping[str, Any]], *, expected_rows: int = 501, expected_pairs: int = 195) -> dict[str, Any]:
|
||||
if len(units) != expected_rows or len({u["unit"] for u in units}) != expected_rows:
|
||||
raise ValueError(f"Expected {expected_rows} unique units, read {len(units)}")
|
||||
if len(complete_pairs(units)) != expected_pairs:
|
||||
raise ValueError(f"Expected {expected_pairs} complete pairs")
|
||||
examples = load_examples(store)
|
||||
index = {e.id: e for e in examples}
|
||||
if len(index) != len(examples):
|
||||
raise ValueError("Duplicate example IDs in memory")
|
||||
sources = {s.id for s in load_sources().sources}
|
||||
coverage: dict[str, Counter[str]] = defaultdict(Counter)
|
||||
rows = []
|
||||
excerpt_bytes = 0
|
||||
blobs: dict[str, str] = {}
|
||||
for unit in units:
|
||||
e = index[unit["unit"]] # missing input is fatal, never counted as no-flow
|
||||
if (e.family, e.label, e.group) != (unit["family"], unit["label"], unit["group"]):
|
||||
raise ValueError(f"Unit metadata differs from store: {e.id}")
|
||||
if e.source not in sources:
|
||||
raise ValueError(f"Source absent from sources.toml: {e.source}")
|
||||
if e.excerpt_sha not in blobs:
|
||||
# MemoryStore.get_blob(verify=True) DELETES corrupt blobs: do not use it.
|
||||
data = store.get_blob("excerpts", e.excerpt_sha, verify=False)
|
||||
if not data or hashlib.sha256(data).hexdigest() != e.excerpt_sha:
|
||||
raise ValueError(f"Missing, empty or corrupt excerpt: {e.id}")
|
||||
excerpt_bytes += len(data)
|
||||
blobs[e.excerpt_sha] = data.decode("utf-8")
|
||||
result = function_slice(blobs[e.excerpt_sha], e.language, e.family)
|
||||
status = "not-analysable" if result is None else "flow" if result.steps else "no-flow"
|
||||
coverage[e.family][status] += 1
|
||||
rows.append(
|
||||
{
|
||||
**unit,
|
||||
"source": e.source,
|
||||
"status": status,
|
||||
"has_flow": int(status == "flow"),
|
||||
"slice_length": len(result.steps) if result else 0,
|
||||
}
|
||||
)
|
||||
pairs = complete_pairs(rows)
|
||||
asymmetry = Counter(
|
||||
"both" if v["has_flow"] and f["has_flow"] else "vulnerable_only" if v["has_flow"] else "fixed_only" if f["has_flow"] else "neither"
|
||||
for v, f in pairs
|
||||
)
|
||||
scopes = {"all": rows, **{f"source:{s}": [r for r in rows if r["source"] == s] for s in sorted({r["source"] for r in rows})}}
|
||||
return {
|
||||
"cost_usd": 0,
|
||||
"threshold": THRESHOLD,
|
||||
"unit_rows_read": len(units),
|
||||
"example_rows_read": len(examples),
|
||||
"matched_rows": len(rows),
|
||||
"excerpts_read": len(blobs),
|
||||
"excerpt_units": len(rows),
|
||||
"excerpt_bytes": excerpt_bytes,
|
||||
"complete_pairs": len(pairs),
|
||||
"coverage": {f: {s: c[s] for s in ("flow", "no-flow", "not-analysable")} for f, c in sorted(coverage.items())},
|
||||
"pair_asymmetry": {k: asymmetry[k] for k in ("vulnerable_only", "fixed_only", "both", "neither")},
|
||||
"signals": {
|
||||
signal: {scope: metrics(members, signal) for scope, members in scopes.items()} for signal in ("has_flow", "slice_length")
|
||||
},
|
||||
"source_strata": "Stored source IDs from sources.toml; advisory pair catalogs are merged into pairs by the example builder.",
|
||||
"limitations": (
|
||||
"Flat IR: no dominance/guards, textual branch order, potential parameter inputs, conservative call returns; no alias analysis."
|
||||
),
|
||||
"units": rows,
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
load_dotenv(Path(".env"))
|
||||
os.environ.pop("OPENROUTER_API_KEY", None) # dotenv must not restore the removed key
|
||||
raw = UNITS.read_bytes()
|
||||
units = [json.loads(line) for line in raw.decode().splitlines() if line.strip()]
|
||||
report = audit(open_store(), units)
|
||||
report["units_bytes"] = len(raw)
|
||||
report["units_sha256"] = hashlib.sha256(raw).hexdigest()
|
||||
OUTPUT.parent.mkdir(parents=True, exist_ok=True)
|
||||
OUTPUT.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n")
|
||||
totals = Counter({s: sum(c[s] for c in report["coverage"].values()) for s in ("flow", "no-flow", "not-analysable")})
|
||||
lines = [
|
||||
f"Slice leak audit: $0; units bytes={len(raw)}",
|
||||
f"Rows read: units={len(units)}, examples={report['example_rows_read']}, matched={report['matched_rows']}",
|
||||
f"Excerpts read: {report['excerpts_read']}; units={report['excerpt_units']}; bytes={report['excerpt_bytes']}",
|
||||
f"Coverage: {dict(totals)}",
|
||||
f"Complete pairs: {report['complete_pairs']}",
|
||||
f"Pair asymmetry: {report['pair_asymmetry']}",
|
||||
f"Has flow: {report['signals']['has_flow']['all']}",
|
||||
f"Slice length: {report['signals']['slice_length']['all']}",
|
||||
f"Source strata: {', '.join(report['signals']['has_flow'])}; threshold={THRESHOLD}",
|
||||
f"Report: {OUTPUT}",
|
||||
]
|
||||
print("\n".join(lines))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"modules": 151,
|
||||
"modules": 159,
|
||||
"counts": {
|
||||
"load_bearing": 146,
|
||||
"load_bearing": 154,
|
||||
"standalone_capability": 5
|
||||
},
|
||||
"orphaned": [],
|
||||
@@ -229,6 +229,8 @@
|
||||
"model.report",
|
||||
"pairs",
|
||||
"plane.tasks.alerts",
|
||||
"push.quick",
|
||||
"push.runner",
|
||||
"rank",
|
||||
"regress.candidate",
|
||||
"reports",
|
||||
@@ -356,12 +358,14 @@
|
||||
"learn.excerpt",
|
||||
"learn.experiments",
|
||||
"learn.features",
|
||||
"learn.fill",
|
||||
"learn.folds",
|
||||
"learn.labels",
|
||||
"learn.leaks",
|
||||
"learn.program",
|
||||
"learn.retrieve",
|
||||
"learn.roles",
|
||||
"learn.slice",
|
||||
"plane.harvest",
|
||||
"plane.tasks.features",
|
||||
"plane.tasks.remember",
|
||||
@@ -410,7 +414,8 @@
|
||||
"importers": [
|
||||
"learn.cli",
|
||||
"learn.compile",
|
||||
"learn.experiments"
|
||||
"learn.experiments",
|
||||
"learn.fill"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -422,6 +427,7 @@
|
||||
"learn.compile",
|
||||
"learn.evaluate",
|
||||
"learn.experiments",
|
||||
"learn.fill",
|
||||
"learn.folds",
|
||||
"learn.program",
|
||||
"learn.retrieve"
|
||||
@@ -435,6 +441,7 @@
|
||||
"learn.embeddings",
|
||||
"learn.examples",
|
||||
"learn.labels",
|
||||
"learn.slice",
|
||||
"plane.harvest"
|
||||
]
|
||||
},
|
||||
@@ -443,7 +450,8 @@
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"learn.cli"
|
||||
"learn.cli",
|
||||
"learn.fill"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -455,6 +463,14 @@
|
||||
"plane.tasks.features"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "learn.fill",
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"learn.cli"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "learn.folds",
|
||||
"classification": "load_bearing",
|
||||
@@ -500,7 +516,8 @@
|
||||
"learn.cli",
|
||||
"learn.compile",
|
||||
"learn.evaluate",
|
||||
"learn.experiments"
|
||||
"learn.experiments",
|
||||
"learn.fill"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -539,6 +556,14 @@
|
||||
"plane.tasks.roles"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "learn.slice",
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"learn.program"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "map_gate",
|
||||
"classification": "load_bearing",
|
||||
@@ -560,6 +585,7 @@
|
||||
"pairs",
|
||||
"plane.tasks.alerts",
|
||||
"preprocess",
|
||||
"push.quick",
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
@@ -579,9 +605,12 @@
|
||||
"cli",
|
||||
"cpg.backend",
|
||||
"cpg.session",
|
||||
"findings",
|
||||
"learn.cli",
|
||||
"learn.features",
|
||||
"learn.fill",
|
||||
"learn.labels",
|
||||
"learn.slice",
|
||||
"model.candidates",
|
||||
"model.config_value",
|
||||
"model.dominance",
|
||||
@@ -602,6 +631,7 @@
|
||||
"push.cache",
|
||||
"push.contracts",
|
||||
"push.policy",
|
||||
"push.quick",
|
||||
"push.report",
|
||||
"push.reuse",
|
||||
"push.runner",
|
||||
@@ -649,6 +679,7 @@
|
||||
"push.cache",
|
||||
"push.contracts",
|
||||
"push.policy",
|
||||
"push.quick",
|
||||
"push.report",
|
||||
"push.reuse",
|
||||
"push.runner",
|
||||
@@ -672,6 +703,7 @@
|
||||
"importers": [
|
||||
"cli",
|
||||
"learn.cli",
|
||||
"learn.fill",
|
||||
"plane.budget",
|
||||
"plane.generate",
|
||||
"push.assistance",
|
||||
@@ -716,7 +748,9 @@
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"model.scan"
|
||||
"findings",
|
||||
"model.scan",
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -775,6 +809,7 @@
|
||||
"classification": "load_bearing",
|
||||
"reason": "model-grounded-detection Req 7.2-7.4: the security vocabularies the CPG queries are parameterised by",
|
||||
"importers": [
|
||||
"learn.slice",
|
||||
"model.config_value",
|
||||
"model.dominance",
|
||||
"model.pipeline",
|
||||
@@ -846,7 +881,10 @@
|
||||
"learn.embeddings",
|
||||
"learn.examples",
|
||||
"learn.experiments",
|
||||
"learn.fill",
|
||||
"learn.program",
|
||||
"plane.delivery",
|
||||
"plane.doctor",
|
||||
"plane.egress",
|
||||
"plane.engine_alerts",
|
||||
"plane.generate",
|
||||
@@ -871,12 +909,21 @@
|
||||
"learn.compile",
|
||||
"learn.embeddings",
|
||||
"learn.experiments",
|
||||
"learn.fill",
|
||||
"learn.program",
|
||||
"plane.generate",
|
||||
"plane.tasks.roles",
|
||||
"plane.tasks.verify"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "plane.delivery",
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"plane.reconciler"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "plane.doctor",
|
||||
"classification": "load_bearing",
|
||||
@@ -924,6 +971,7 @@
|
||||
"reason": "ai-service-plane Req 1.1-1.4: ax-shaped Task/Workspace/Model and the Run kind, validated before anything runs",
|
||||
"importers": [
|
||||
"learn.cli",
|
||||
"plane.delivery",
|
||||
"plane.egress",
|
||||
"plane.generate",
|
||||
"plane.harvest",
|
||||
@@ -945,12 +993,27 @@
|
||||
"learn.embeddings",
|
||||
"learn.examples",
|
||||
"learn.experiments",
|
||||
"learn.fill",
|
||||
"learn.program",
|
||||
"plane.delivery",
|
||||
"plane.doctor",
|
||||
"plane.generate",
|
||||
"plane.reconciler",
|
||||
"plane.tasks.loop",
|
||||
"plane.tasks.remember"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "plane.profile",
|
||||
"classification": "load_bearing",
|
||||
"reason": "python -m openultrasast.plane.profile",
|
||||
"importers": [
|
||||
"cli",
|
||||
"plane.doctor",
|
||||
"plane.generate",
|
||||
"plane.reconciler"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "plane.reconciler",
|
||||
"classification": "load_bearing",
|
||||
@@ -1093,6 +1156,7 @@
|
||||
"improve.evolve",
|
||||
"improve.validator",
|
||||
"plane.tasks.loop",
|
||||
"push.quick",
|
||||
"scoring.project_score",
|
||||
"semantic.overlay"
|
||||
]
|
||||
@@ -1132,6 +1196,7 @@
|
||||
"plane.tasks.alerts",
|
||||
"plane.tasks.features",
|
||||
"plane.tasks.repo_facts",
|
||||
"push.quick",
|
||||
"push.runner",
|
||||
"rank",
|
||||
"regress.candidate",
|
||||
@@ -1186,6 +1251,7 @@
|
||||
"importers": [
|
||||
"cli",
|
||||
"push.assistance",
|
||||
"push.background",
|
||||
"push.eligibility",
|
||||
"push.policy",
|
||||
"push.report",
|
||||
@@ -1203,6 +1269,14 @@
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "push.background",
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "push.cache",
|
||||
"classification": "load_bearing",
|
||||
@@ -1234,6 +1308,23 @@
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "push.explain",
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"push.report",
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "push.install",
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"cli"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "push.policy",
|
||||
"classification": "load_bearing",
|
||||
@@ -1245,12 +1336,21 @@
|
||||
"push.vetoes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "push.quick",
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
{
|
||||
"module": "push.report",
|
||||
"classification": "load_bearing",
|
||||
"reason": "pre-push-safety-net Req 5.1/5.2/5.5/7.3: compact output and complete artifacts; runner integration pending",
|
||||
"importers": [
|
||||
"push.assistance",
|
||||
"push.background",
|
||||
"push.runner"
|
||||
]
|
||||
},
|
||||
@@ -1315,7 +1415,8 @@
|
||||
"improve.evolve",
|
||||
"map_gate",
|
||||
"pairs",
|
||||
"plane.tasks.alerts"
|
||||
"plane.tasks.alerts",
|
||||
"push.quick"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -1501,6 +1602,7 @@
|
||||
"hunter_tools",
|
||||
"learn.features",
|
||||
"learn.roles",
|
||||
"learn.slice",
|
||||
"mapping",
|
||||
"model.candidates",
|
||||
"model.layout",
|
||||
@@ -1557,6 +1659,7 @@
|
||||
"hunter_tools",
|
||||
"learn.features",
|
||||
"learn.roles",
|
||||
"learn.slice",
|
||||
"mapping",
|
||||
"model.candidates",
|
||||
"model.layout",
|
||||
@@ -1590,6 +1693,7 @@
|
||||
"importers": [
|
||||
"cli",
|
||||
"hunter_tools",
|
||||
"learn.slice",
|
||||
"model.candidates",
|
||||
"pairs",
|
||||
"semantic.cst",
|
||||
@@ -1723,6 +1827,7 @@
|
||||
"classification": "load_bearing",
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"learn.slice",
|
||||
"semantic.overlay"
|
||||
]
|
||||
},
|
||||
@@ -1748,6 +1853,8 @@
|
||||
"reason": "reachable from gate, map_gate, pair_gate, cli",
|
||||
"importers": [
|
||||
"cli",
|
||||
"learn.cli",
|
||||
"learn.fill",
|
||||
"model.endpoint",
|
||||
"plane.budget",
|
||||
"plane.tasks.roles",
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
[ok] profile k3s: exec=remote kube_context=ousast-k3s registry=ghcr.io/norandom images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=ghcr-pull
|
||||
[FAIL] cluster ousast-k3s: context ousast-k3s unreachable (error: context "ousast-k3s" does not exist); docs/deployment.md lists the prerequisites
|
||||
[FAIL] runner image in ghcr.io/norandom: images file /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/images.json: No such file or directory; docs/deployment.md lists the prerequisites
|
||||
[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
|
||||
exit=1 seconds=1.1
|
||||
@@ -0,0 +1,7 @@
|
||||
[ok] profile kind: exec=local kube_context=kind-ousast registry=localhost:5001 images=/home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json router_url=(tunnel through the kube context) ax_server=(tunnel through the kube context) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)
|
||||
[ok] cluster kind-ousast: reachable
|
||||
[ok] agent substrate (ate-system): 9 pods ready
|
||||
[ok] ax controller (ax-system): 4 pods ready
|
||||
[ok] runner image in localhost:5001: ousast-runner@sha256:95edc7ed5621... present in localhost:5001
|
||||
[ok] memory store s3://sast-memory: s3://sast-memory opens (an S3 bucket is verified)
|
||||
exit=0 seconds=1.4
|
||||
@@ -0,0 +1,19 @@
|
||||
F.. [100%]
|
||||
=================================== FAILURES ===================================
|
||||
________ test_no_host_literal_outside_the_profile_and_the_kind_bring_up ________
|
||||
|
||||
def test_no_host_literal_outside_the_profile_and_the_kind_bring_up() -> None:
|
||||
kind = load_profile("kind", environ={})
|
||||
found = offenders(_files(), kind.registry)
|
||||
> assert found == [], "host-only literals (Req 1.3):\n" + "\n".join(found)
|
||||
E AssertionError: host-only literals (Req 1.3):
|
||||
E src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration
|
||||
E assert ['src/openult...emonstration'] == []
|
||||
E
|
||||
E Left contains one more item: 'src/openultrasast/plane/workspaces.py:37: [kind context] CONTEXT = "kind-ousast" # reinserted for the record 01 demonstration'
|
||||
E Use -v to get more diff
|
||||
|
||||
tests/test_plane_literals.py:81: AssertionError
|
||||
=========================== short test summary info ============================
|
||||
FAILED tests/test_plane_literals.py::test_no_host_literal_outside_the_profile_and_the_kind_bring_up
|
||||
1 failed, 2 passed in 0.26s
|
||||
@@ -0,0 +1,2 @@
|
||||
... [100%]
|
||||
3 passed in 0.23s
|
||||
@@ -0,0 +1,16 @@
|
||||
apiVersion: ax.io/v1alpha1
|
||||
kind: Workspace
|
||||
metadata: {name: ousast-probe-big-ws, atespace: default}
|
||||
spec:
|
||||
files:
|
||||
- {path: "hello.py", content: "def hello():\n return greet()\n"}
|
||||
---
|
||||
apiVersion: ax.io/v1alpha1
|
||||
kind: Task
|
||||
metadata: {name: ousast-probe-big, atespace: default}
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
command: ["repo-facts"]
|
||||
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
|
||||
resources: {requests: {cpu: "250m", memory: "2560Mi"}, limits: {cpu: "1", memory: "3Gi"}}
|
||||
workspaces: [{name: ousast-probe-big-ws, path: "/workspace"}]
|
||||
@@ -0,0 +1,16 @@
|
||||
apiVersion: ax.io/v1alpha1
|
||||
kind: Workspace
|
||||
metadata: {name: ousast-probe-small-ws, atespace: default}
|
||||
spec:
|
||||
files:
|
||||
- {path: "hello.py", content: "def hello():\n return greet()\n"}
|
||||
---
|
||||
apiVersion: ax.io/v1alpha1
|
||||
kind: Task
|
||||
metadata: {name: ousast-probe-small, atespace: default}
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
command: ["repo-facts"]
|
||||
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
|
||||
resources: {requests: {cpu: "250m", memory: "256Mi"}, limits: {cpu: "1", memory: "512Mi"}}
|
||||
workspaces: [{name: ousast-probe-small-ws, path: "/workspace"}]
|
||||
@@ -0,0 +1,26 @@
|
||||
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
|
||||
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
|
||||
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
|
||||
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
|
||||
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
|
||||
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
|
||||
apiVersion: ate.dev/v1alpha1
|
||||
kind: WorkerPool
|
||||
metadata:
|
||||
name: probe-pool
|
||||
namespace: ax-system
|
||||
labels:
|
||||
workload: ousast
|
||||
spec:
|
||||
replicas: 1
|
||||
workerImage: localhost:5001/ateom-gvisor-715889664656de67e44382a8d6ab981d@sha256:220f1cf13ea485b73a2e4270fa544901a18ae826d8f22bd54d42761944a41c5c
|
||||
template:
|
||||
nodeSelector:
|
||||
ate.dev/substrate-version: "4f9c293"
|
||||
resources:
|
||||
limits:
|
||||
cpu: "2"
|
||||
memory: 3Gi
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 3Gi
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"workers": [
|
||||
{
|
||||
"ip": "10.244.0.22",
|
||||
"labels": {
|
||||
"workload": "ousast"
|
||||
},
|
||||
"metadata": {
|
||||
"createTime": "2026-09-29T16:32:08.594663703Z",
|
||||
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
|
||||
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
|
||||
"updateTime": "2026-10-02T05:24:59.069304031Z",
|
||||
"version": "1758"
|
||||
},
|
||||
"nodeName": "ousast-control-plane",
|
||||
"sandboxClass": "gvisor",
|
||||
"status": {
|
||||
"capacity": {
|
||||
"actors": 1000,
|
||||
"resources": {
|
||||
"limits": [
|
||||
{
|
||||
"name": "cpu",
|
||||
"quantity": "1"
|
||||
},
|
||||
{
|
||||
"name": "memory",
|
||||
"quantity": "1536Mi"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"state": "WORKER_STATE_ACTIVE"
|
||||
},
|
||||
"workerNamespace": "ax-system",
|
||||
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
|
||||
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
|
||||
"workerPool": "ousast-pool"
|
||||
},
|
||||
{
|
||||
"ip": "10.244.0.28",
|
||||
"labels": {
|
||||
"workload": "ousast"
|
||||
},
|
||||
"metadata": {
|
||||
"createTime": "2026-10-02T18:29:30.486026311Z",
|
||||
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
|
||||
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
|
||||
"updateTime": "2026-10-02T18:29:30.551668391Z",
|
||||
"version": "2"
|
||||
},
|
||||
"nodeName": "ousast-control-plane",
|
||||
"sandboxClass": "gvisor",
|
||||
"status": {
|
||||
"capacity": {
|
||||
"actors": 1000,
|
||||
"resources": {
|
||||
"limits": [
|
||||
{
|
||||
"name": "cpu",
|
||||
"quantity": "2"
|
||||
},
|
||||
{
|
||||
"name": "memory",
|
||||
"quantity": "3Gi"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"state": "WORKER_STATE_ACTIVE"
|
||||
},
|
||||
"workerNamespace": "ax-system",
|
||||
"workerPod": "probe-pool-5dffd74bdc-dqlww",
|
||||
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
|
||||
"workerPool": "probe-pool"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"workers": [
|
||||
{
|
||||
"ip": "10.244.0.22",
|
||||
"labels": {
|
||||
"workload": "ousast"
|
||||
},
|
||||
"metadata": {
|
||||
"createTime": "2026-09-29T16:32:08.594663703Z",
|
||||
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
|
||||
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
|
||||
"updateTime": "2026-10-02T18:29:45.281958347Z",
|
||||
"version": "1761"
|
||||
},
|
||||
"nodeName": "ousast-control-plane",
|
||||
"sandboxClass": "gvisor",
|
||||
"status": {
|
||||
"allocated": {
|
||||
"actors": 1
|
||||
},
|
||||
"capacity": {
|
||||
"actors": 1000,
|
||||
"resources": {
|
||||
"limits": [
|
||||
{
|
||||
"name": "cpu",
|
||||
"quantity": "1"
|
||||
},
|
||||
{
|
||||
"name": "memory",
|
||||
"quantity": "1536Mi"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"state": "WORKER_STATE_ACTIVE"
|
||||
},
|
||||
"workerNamespace": "ax-system",
|
||||
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
|
||||
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
|
||||
"workerPool": "ousast-pool"
|
||||
},
|
||||
{
|
||||
"ip": "10.244.0.28",
|
||||
"labels": {
|
||||
"workload": "ousast"
|
||||
},
|
||||
"metadata": {
|
||||
"createTime": "2026-10-02T18:29:30.486026311Z",
|
||||
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
|
||||
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
|
||||
"updateTime": "2026-10-02T18:29:30.551668391Z",
|
||||
"version": "2"
|
||||
},
|
||||
"nodeName": "ousast-control-plane",
|
||||
"sandboxClass": "gvisor",
|
||||
"status": {
|
||||
"capacity": {
|
||||
"actors": 1000,
|
||||
"resources": {
|
||||
"limits": [
|
||||
{
|
||||
"name": "cpu",
|
||||
"quantity": "2"
|
||||
},
|
||||
{
|
||||
"name": "memory",
|
||||
"quantity": "3Gi"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"state": "WORKER_STATE_ACTIVE"
|
||||
},
|
||||
"workerNamespace": "ax-system",
|
||||
"workerPod": "probe-pool-5dffd74bdc-dqlww",
|
||||
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
|
||||
"workerPool": "probe-pool"
|
||||
}
|
||||
]
|
||||
}
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"workers": [
|
||||
{
|
||||
"ip": "10.244.0.22",
|
||||
"labels": {
|
||||
"workload": "ousast"
|
||||
},
|
||||
"metadata": {
|
||||
"createTime": "2026-09-29T16:32:08.594663703Z",
|
||||
"name": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
|
||||
"uid": "152cf7c0-3f4d-4634-88c5-9fe501bb403e",
|
||||
"updateTime": "2026-10-02T18:29:53.001457323Z",
|
||||
"version": "1763"
|
||||
},
|
||||
"nodeName": "ousast-control-plane",
|
||||
"sandboxClass": "gvisor",
|
||||
"status": {
|
||||
"allocated": {
|
||||
"actors": 1
|
||||
},
|
||||
"capacity": {
|
||||
"actors": 1000,
|
||||
"resources": {
|
||||
"limits": [
|
||||
{
|
||||
"name": "cpu",
|
||||
"quantity": "1"
|
||||
},
|
||||
{
|
||||
"name": "memory",
|
||||
"quantity": "1536Mi"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"state": "WORKER_STATE_ACTIVE"
|
||||
},
|
||||
"workerNamespace": "ax-system",
|
||||
"workerPod": "ousast-pool-6656b55f5b-9wbrw",
|
||||
"workerPodUid": "769f7d8b-142a-4ed5-b896-08d6a04bedb6",
|
||||
"workerPool": "ousast-pool"
|
||||
},
|
||||
{
|
||||
"ip": "10.244.0.28",
|
||||
"labels": {
|
||||
"workload": "ousast"
|
||||
},
|
||||
"metadata": {
|
||||
"createTime": "2026-10-02T18:29:30.486026311Z",
|
||||
"name": "1407a421-7dcf-4ad7-b999-70a4624bf890",
|
||||
"uid": "2d496d7a-4f32-459d-9b92-256a2c5b4800",
|
||||
"updateTime": "2026-10-02T18:30:04.825058843Z",
|
||||
"version": "4"
|
||||
},
|
||||
"nodeName": "ousast-control-plane",
|
||||
"sandboxClass": "gvisor",
|
||||
"status": {
|
||||
"capacity": {
|
||||
"actors": 1000,
|
||||
"resources": {
|
||||
"limits": [
|
||||
{
|
||||
"name": "cpu",
|
||||
"quantity": "2"
|
||||
},
|
||||
{
|
||||
"name": "memory",
|
||||
"quantity": "3Gi"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"state": "WORKER_STATE_ACTIVE"
|
||||
},
|
||||
"workerNamespace": "ax-system",
|
||||
"workerPod": "probe-pool-5dffd74bdc-dqlww",
|
||||
"workerPodUid": "1407a421-7dcf-4ad7-b999-70a4624bf890",
|
||||
"workerPool": "probe-pool"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
{
|
||||
"record": "k8s-01-profile",
|
||||
"spec": "plane-on-kubernetes",
|
||||
"tasks": ["1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7", "1.8"],
|
||||
"date": "2026-10-02",
|
||||
"host": "the maintainer's laptop: 7.9 GiB, single-node kind cluster, RustFS store reachable over https",
|
||||
"commits": {
|
||||
"1.1": "8eac1b2",
|
||||
"1.2": "77a03ba",
|
||||
"1.3": "36f5927",
|
||||
"1.4": "8099b29",
|
||||
"1.5": "86ec237",
|
||||
"1.6": "2afa20e",
|
||||
"1.7": "2cca03c",
|
||||
"1.8": "the commit that adds this line (git log: plane(1.8))"
|
||||
},
|
||||
"images": {
|
||||
"runner_before": "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884",
|
||||
"runner_after_up_sh": "localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb",
|
||||
"pin_file": "ops/k8s/profiles/kind-images.json (written by ops/ax/up.sh; the templates re-pinned to it, 128 image lines)"
|
||||
},
|
||||
"instrument_checks": {
|
||||
"code_under_test": "PYTHONPATH=<worktree>/src first; `python -c 'import openultrasast; print(openultrasast.__file__)'` printed the worktree path before every gate",
|
||||
"store_reached": {
|
||||
"how": "scratch store_probe.py: open_store('s3://sast-memory') through the worktree code (verify_bucket runs), then read the probe object",
|
||||
"open_and_verify_seconds": 0.62,
|
||||
"probe_key": "_probe/select.jsonl",
|
||||
"probe_bytes": 24,
|
||||
"probe_version_id_present": true
|
||||
},
|
||||
"smoke_delivery_sizes_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291, "task.yaml": 1148},
|
||||
"registry_resolution": "doctor HEADs /v2/ousast-runner/manifests/<digest> on the kind registry: HTTP 2xx for the new digest"
|
||||
},
|
||||
"doctor": {
|
||||
"kind": {
|
||||
"command": "ousast plane doctor --profile kind",
|
||||
"exit": 0,
|
||||
"seconds": 1.4,
|
||||
"lines": [
|
||||
"[ok] profile kind: exec=local kube_context=... registry=... images=... router_url=(tunnel) ax_server=(tunnel) memory=s3://sast-memory atespace=default snapshots_bucket=s3://ax-snapshots image_pull_secret=(none)",
|
||||
"[ok] cluster <kind context>: reachable",
|
||||
"[ok] agent substrate (ate-system): 9 pods ready",
|
||||
"[ok] ax controller (ax-system): 4 pods ready",
|
||||
"[ok] runner image in <kind registry>: ousast-runner@sha256:95edc7ed5621... present",
|
||||
"[ok] memory store s3://sast-memory: opens (an S3 bucket is verified)"
|
||||
],
|
||||
"file": "doctor-kind.txt"
|
||||
},
|
||||
"k3s": {
|
||||
"command": "ousast plane doctor --profile k3s",
|
||||
"exit": 1,
|
||||
"seconds": 1.1,
|
||||
"failing_lines": [
|
||||
"cluster ousast-k3s: context does not exist (expected: the cluster does not exist yet)",
|
||||
"runner image in ghcr.io/norandom: images file ops/k8s/profiles/images.json missing (expected: the release asset is published by task 4.1)"
|
||||
],
|
||||
"passing_lines": ["profile k3s: every address reported", "memory store s3://sast-memory: opens"],
|
||||
"note": "two expected failures, not one: the images file is the release asset and does not exist before group 4; the namespaces are not checked behind an unreachable context, so no third line fails for the same cause",
|
||||
"file": "doctor-k3s.txt"
|
||||
}
|
||||
},
|
||||
"literal_scan": {
|
||||
"test": "tests/test_plane_literals.py",
|
||||
"scope": "src/, plane/, ops/ (minus ops/ax/up.sh, ops/k8s/profiles/), docs/",
|
||||
"patterns": ["kind context", "kind registry", "kind gateway 172.19.", "loopback with port", "port-forward"],
|
||||
"exemptions": [
|
||||
"router.py for loopback-with-port and port-forward",
|
||||
"lines tagged ax-tunnel or loopback (two: the runner's own health URL, the Joern session's query URL)",
|
||||
"digest-pinned image: lines under plane/tasks/ whose registry equals the kind profile's (a third test asserts every one does)"
|
||||
],
|
||||
"offenders_at_1_3": 0,
|
||||
"offenders_before_1_3": 23,
|
||||
"reinsertion_demo": {
|
||||
"inserted": "a kind-context literal appended to src/openultrasast/plane/workspaces.py",
|
||||
"pytest_exit_with_literal": 1,
|
||||
"named": "src/openultrasast/plane/workspaces.py:37: [kind context]",
|
||||
"restored": "byte for byte (cmp)",
|
||||
"pytest_exit_restored": 0,
|
||||
"files": ["literal-reinserted-fail.txt", "literal-restored-pass.txt"]
|
||||
}
|
||||
},
|
||||
"up_sh_rerun": {
|
||||
"command": "OUSAST_PYTHON=<main venv python> ops/ax/up.sh",
|
||||
"exit": 0,
|
||||
"seconds": 38,
|
||||
"kept": ["cluster", "substrate", "egress gateway", "ax control plane", "worker pool ousast-pool (2/2 ready)", "ax CLI (newer than the checkout)"],
|
||||
"done": ["receiver Service re-applied", "runner image rebuilt and pushed (new digest above)", "kind-images.json written", "smoke Task applied and resumed"],
|
||||
"first_attempt_failure": "`profile --print images` answered `no image named ''`; fixed in profile._lookup (an images path with no sub-key is the file path) and covered by test_plane_profile",
|
||||
"disk_gb_free_before_after": [1.7, 1.5],
|
||||
"file": "up-sh-rerun.txt"
|
||||
},
|
||||
"smoke_run_1_4": {
|
||||
"command": "ops/ax/smoke-run.sh (receiver delivery: the path group 2 replaces)",
|
||||
"exit": 0,
|
||||
"seconds": 24,
|
||||
"status": "done",
|
||||
"delivered": "facts.json 451 B, summary.json 238 B, units.jsonl 291 B; summary.image = the new digest, units 2/2",
|
||||
"file": "smoke-run-1.4.txt"
|
||||
},
|
||||
"placement_probe_1_5": {
|
||||
"script": "ops/ax/probe-placement.sh",
|
||||
"pools": {"ousast-pool": "scaled to 1 worker, 1 CPU / 1536Mi", "probe-pool": "1 worker, 2 CPU / 3Gi, the already-built ateom-gvisor image (no ko build)"},
|
||||
"big_actor": {"requests_memory": "2560Mi", "limits_memory": "3Gi", "landed_on": "ousast-pool (1536Mi worker): status.allocated.actors=1; probe-pool 0"},
|
||||
"small_actor": {"requests_memory": "256Mi", "limits_memory": "512Mi", "landed_on": "ousast-pool: allocated.actors=1; probe-pool 0"},
|
||||
"placement": "any",
|
||||
"consequence": "design.md section 4: group 5 takes the fallback (an engine atespace with engine-pool as its only pool if Substrate scopes pools by atespace; else a plain Kubernetes Job on k3s and alerts-engine on kind)",
|
||||
"resume_latency_seconds": "first resume of each new template succeeded at once (golden snapshot built within the resume deadline); whole probe 46 s",
|
||||
"cleanup": "probe tasks deleted, probe-pool deleted, ousast-pool back to 2/2 (verified by kubectl get workerpool)",
|
||||
"files": ["probe/probe.log", "probe/workers-before.json", "probe/workers-ousast-probe-big.json", "probe/workers-ousast-probe-small.json", "probe/ousast-probe-big.yaml", "probe/ousast-probe-small.yaml", "probe/probe-pool.yaml"]
|
||||
},
|
||||
"reconciler_bound_1_6": {
|
||||
"lines_at_spec_commit_6bd44ca": 477,
|
||||
"lines_after_1_2": 489,
|
||||
"added_by_1_2": "the profile parameter of run() and its docstring (+12)",
|
||||
"estimate_2_4": "Delivery wiring: open the store, build Delivery, pass the delivery body to the starter, delivered() and collect() in the await/execute path, state mirroring, the OUSAST_DELIVERY=receiver switch: about +25 while the receiver is kept (group 2 keeps both paths), about -20 when 2.6 removes the receiver wiring",
|
||||
"projected_during_group_2": 514,
|
||||
"projected_after_2_6": 494,
|
||||
"decision": "the estimate crosses 500 while both paths coexist: task 2.4 moves the bound in tests/test_plane_reconciler.py to 550 with the reason in the assertion message, and 2.6 records the final count"
|
||||
},
|
||||
"gvisor_tarball_1_7": {
|
||||
"status": "planned, manual",
|
||||
"where": "docs/deployment.md B.1",
|
||||
"url": "gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd (sha256-pinned in substrate/manifests/ate-install/sandboxconfig-gvisor.yaml)",
|
||||
"check": "the atelet's fetch line in `kubectl -n ate-system logs` of a worker pod on the k3s node; labelled planned until read there"
|
||||
},
|
||||
"gates": {
|
||||
"1.1": "pytest 2002 passed + module-audit manifest regenerated (test_module_audit 6 passed), ruff 0, format 0, mypy 0",
|
||||
"1.2": "pytest 2006 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
|
||||
"1.3": "pytest 2009 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0; mkdocs --strict 0",
|
||||
"1.4": "see the commit message"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
run finished: done
|
||||
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
|
||||
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
|
||||
|
||||
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
|
||||
facts - 0 n/a n/a n/a n/a [done]
|
||||
----------------------------------------------------------------------------------------------------
|
||||
total - n/a n/a n/a n/a n/a
|
||||
smoke exit=0 seconds=24
|
||||
@@ -0,0 +1,58 @@
|
||||
|
||||
== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (20:35:35)
|
||||
cluster exists; kept
|
||||
|
||||
== agent substrate (ate-system) (20:35:35)
|
||||
substrate ready; kept
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (12h ago) 3d
|
||||
ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (12h ago) 3d
|
||||
ate-controller-c95586684-g7nvn 1/1 Running 0 3d2h
|
||||
atelet-4f9c293-9zpfg 1/1 Running 0 3d2h
|
||||
atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d
|
||||
atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (12h ago) 3d2h
|
||||
postgres-0 2/2 Running 3 (3d2h ago) 3d2h
|
||||
rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d2h ago) 3d2h
|
||||
rustfs-bucket-init-nvfr6 0/1 Completed 0 3d2h
|
||||
|
||||
== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (20:35:35)
|
||||
gateway present; kept
|
||||
|
||||
== ax control plane (ax-system) (20:35:35)
|
||||
ax ready; kept
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
ax-redis-86df786c87-pz5tp 1/1 Running 0 3d2h
|
||||
ax-server-b479bdc76-hwkqb 1/1 Running 0 3d2h
|
||||
ousast-pool-6656b55f5b-5brkn 1/1 Running 0 5m23s
|
||||
ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d2h
|
||||
|
||||
== artifact receiver Service (ousast-receiver.ax-system -> this host) (20:35:35)
|
||||
service/ousast-receiver unchanged
|
||||
endpointslice.discovery.k8s.io/ousast-receiver-host configured
|
||||
|
||||
== worker pool (gVisor): pools.default of the profile (20:35:36)
|
||||
pool ousast-pool exists (2/2 ready); kept
|
||||
pod/ousast-pool-6656b55f5b-5brkn condition met
|
||||
pod/ousast-pool-6656b55f5b-9wbrw condition met
|
||||
|
||||
== runner image localhost:5001/ousast-runner (20:35:37)
|
||||
sha256:f004a5ef8f71b1cb8a5880c509389c0569181af7b19170b574958cc3ef75cb1c
|
||||
localhost:5001/ousast-runner:dev
|
||||
runner image localhost:5001/ousast-runner@sha256:95edc7ed562189180a472df8115e6a90c8b4fcea47c06593395634bd56041bcb -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
|
||||
re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
|
||||
|
||||
== ax CLI from the deployed checkout (a release CLI skews from the server) (20:36:05)
|
||||
ax CLI newer than the checkout; kept
|
||||
|
||||
== smoke task (20:36:05)
|
||||
task.ax.io/ousast-smoke created
|
||||
workspace.ax.io/ousast-smoke-ws unchanged
|
||||
task.ax.io/ousast-smoke resumed
|
||||
NAME ATESPACE PHASE ACTOR WORKER-IP AGE
|
||||
ousast-smoke default Running ousast-smoke 10.244.0.29 5s
|
||||
|
||||
== footprint (20:36:11)
|
||||
total used free shared buff/cache available
|
||||
Mem: 7877 4797 225 63 3464 3079
|
||||
kind-registry 53.77MiB / 7.693GiB 0.00%
|
||||
up.sh exit 0
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"metadata": {
|
||||
"atespace": "default",
|
||||
"createTime": "2026-10-02T19:39:19.428045847Z",
|
||||
"name": "default",
|
||||
"uid": "741e52b7-aaf1-4a91-b4bf-fc3b20ba4a36",
|
||||
"updateTime": "2026-10-02T19:39:19.428045847Z",
|
||||
"version": "1"
|
||||
},
|
||||
"rules": [
|
||||
{
|
||||
"tlsPassthrough": {
|
||||
"hostnames": [
|
||||
"files.because-security.com"
|
||||
],
|
||||
"ports": {
|
||||
"numbers": [
|
||||
443
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
{
|
||||
"record": "k8s-02-store-delivery",
|
||||
"spec": "plane-on-kubernetes",
|
||||
"tasks": ["2.1", "2.2", "2.3", "2.4", "2.5", "2.6"],
|
||||
"date": "2026-10-02",
|
||||
"commits": {
|
||||
"2.1": "35d8032",
|
||||
"2.2": "9506a18",
|
||||
"2.3": "c49eae4",
|
||||
"2.4": "605b7af",
|
||||
"2.5": "the commit that adds this file (git log: plane(2.5))",
|
||||
"receiver_removed": "pending: task 2.6 (paused 2026-10-03 by the maintainer's reprioritisation; the receiver path still exists behind OUSAST_DELIVERY=receiver)"
|
||||
},
|
||||
"images": {
|
||||
"runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019",
|
||||
"built_by": "ops/ax/up.sh rerun (file up-sh-rebuild.txt): every step kept but the runner build; the image carries the 2.2 runner",
|
||||
"disk_gb_free_after_build": 1.3
|
||||
},
|
||||
"instrument_checks": {
|
||||
"code_under_test": "the worktree's src first on PYTHONPATH; `ousast` run as `python -m openultrasast.cli` from the worktree (ops/ax/smoke-run.sh with OUSAST_PYTHON)",
|
||||
"store_read_back": {
|
||||
"how": "scratch store_tar_check.py opens the store through the worktree code (verify_bucket runs), lists runs/ax-e2e-smoke/, downloads output.tar, compares its members with the files the reconciler extracted (file store-check.json)",
|
||||
"tar_bytes": 10240,
|
||||
"tar_members_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
|
||||
"extracted_files_bytes": {"facts.json": 451, "summary.json": 238, "units.jsonl": 291},
|
||||
"members_equal_extracted": true,
|
||||
"member_bytes_total": 980,
|
||||
"note": "a tar is 512-byte blocks: 10240 B of tar carry 980 B of files; done.json records the tar's byte size"
|
||||
},
|
||||
"no_receiver_in_the_task": "the rendered task.yaml is 996 B against 1148 B on the receiver path: no OUSAST_ARTIFACT_URL/DIAL in the Task's env; the start body carries the URLs"
|
||||
},
|
||||
"smoke_runs": {
|
||||
"store_1": {
|
||||
"command": "ops/ax/smoke-run.sh (store delivery, the default after 2.4; OUSAST_DELIVERY unset)",
|
||||
"exit": 0,
|
||||
"seconds_submit_to_done": 26,
|
||||
"tar_put_at": "2026-10-02T19:37:24+00:00 (the object's LastModified)",
|
||||
"collected_at": "2026-10-02T19:37:26+00:00 (done.json finished)",
|
||||
"put_to_collect_seconds": 2,
|
||||
"poll_interval_seconds": 3,
|
||||
"file": "smoke-run-store-1.txt"
|
||||
},
|
||||
"seeded_rerun": {
|
||||
"command": "the same, with the local results root deleted and the store's runs/ax-e2e-smoke/ intact",
|
||||
"exit": 0,
|
||||
"seconds": 2,
|
||||
"what_happened": "state.json seeded from the store: every task done, nothing submitted to ax (plane-on-kubernetes 2.1, design section 2: a rerun from another machine resumes from the store)",
|
||||
"file": "smoke-run-seeded-rerun.txt"
|
||||
},
|
||||
"store_2": {
|
||||
"command": "the same, after deleting runs/ax-e2e-smoke/ from the store (5 objects) so the task executes again, with the egress policy polled",
|
||||
"exit": 0,
|
||||
"seconds_submit_to_done": 24,
|
||||
"tar_put_at": "2026-10-02T19:39:38+00:00",
|
||||
"collected_at": "2026-10-02T19:39:40+00:00",
|
||||
"put_to_collect_seconds": 2,
|
||||
"file": "smoke-run-store-2.txt"
|
||||
}
|
||||
},
|
||||
"objects_delivered_to_the_store": {
|
||||
"runs/ax-e2e-smoke/facts/output.tar": 10240,
|
||||
"runs/ax-e2e-smoke/facts/facts.json": 451,
|
||||
"runs/ax-e2e-smoke/facts/summary.json": 238,
|
||||
"runs/ax-e2e-smoke/facts/done.json": 149,
|
||||
"runs/ax-e2e-smoke/state.json": 350,
|
||||
"note": "units.jsonl is extracted locally but not a declared output of the smoke Run, so it is not re-put"
|
||||
},
|
||||
"egress_policy_read_back": {
|
||||
"how": "kubectl ate get egress-policy ax-e2e-smoke-facts -a default -o json, polled while the actor lived (file egress-policy-read-back.json)",
|
||||
"rules": [{"tlsPassthrough": {"hostnames": ["files.because-security.com"], "ports": {"numbers": [443]}}}],
|
||||
"reading": "one TLS rule naming the store host, no http rule; the model-free smoke task names no Git host and no Model host",
|
||||
"also": "Egress.apply reads every policy back and raises on a mismatch, so each completed Run is itself the equality check"
|
||||
},
|
||||
"status_table": "ousast plane status ax-e2e-smoke printed the attribution table (facts: done, calls 0) in every run; the table is in the smoke-run-*.txt files",
|
||||
"reconciler_lines": {"after_2_4": 549, "bound": 550, "note": "both paths coexist until 2.6; the receiver wiring goes then"},
|
||||
"gates": {
|
||||
"2.1": "pytest 2020 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
|
||||
"2.2": "pytest 2024 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
|
||||
"2.3": "pytest 2027 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
|
||||
"2.4": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0",
|
||||
"2.5": "pytest 2031 passed, 42 skipped, exit 0; ruff 0; format 0; mypy 0 (first attempt 2026-10-02: 11 failed + 5 errors, all the FileStore 1 GiB free-space refusal at 0.97 GiB free; rerun after the disk was freed)"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
run finished: done
|
||||
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
|
||||
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
|
||||
|
||||
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
|
||||
facts - n/a n/a n/a n/a n/a [done]
|
||||
----------------------------------------------------------------------------------------------------
|
||||
total - n/a n/a n/a n/a n/a
|
||||
smoke exit=0 seconds=2
|
||||
@@ -0,0 +1,9 @@
|
||||
run finished: done
|
||||
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
|
||||
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
|
||||
|
||||
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
|
||||
facts - 0 n/a n/a n/a n/a [done]
|
||||
----------------------------------------------------------------------------------------------------
|
||||
total - n/a n/a n/a n/a n/a
|
||||
smoke exit=0 seconds=26
|
||||
@@ -0,0 +1,9 @@
|
||||
run finished: done
|
||||
memory s3://sast-memory: ax-e2e-smoke: 0 tasks ingested, 0 already in the index; rows none
|
||||
run ax-e2e-smoke: done (/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/k8s-impl/results/plane/ax-e2e-smoke)
|
||||
|
||||
TASK MODEL CALLS PROMPT CACHE-HIT OUTPUT USD
|
||||
facts - 0 n/a n/a n/a n/a [done]
|
||||
----------------------------------------------------------------------------------------------------
|
||||
total - n/a n/a n/a n/a n/a
|
||||
smoke exit=0 seconds=24
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"store": "s3://sast-memory",
|
||||
"objects_under_run": {
|
||||
"runs/ax-e2e-smoke/facts/done.json": 149,
|
||||
"runs/ax-e2e-smoke/facts/facts.json": 451,
|
||||
"runs/ax-e2e-smoke/facts/output.tar": 10240,
|
||||
"runs/ax-e2e-smoke/facts/summary.json": 238,
|
||||
"runs/ax-e2e-smoke/state.json": 350
|
||||
},
|
||||
"tar_bytes": 10240,
|
||||
"tar_members": {
|
||||
"facts.json": 451,
|
||||
"summary.json": 238,
|
||||
"units.jsonl": 291
|
||||
},
|
||||
"extracted_files": {
|
||||
"facts.json": 451,
|
||||
"units.jsonl": 291,
|
||||
"summary.json": 238
|
||||
},
|
||||
"members_equal_extracted": true,
|
||||
"member_bytes_total": 980,
|
||||
"extracted_bytes_total": 980,
|
||||
"done_json": {
|
||||
"bytes": 10240,
|
||||
"files": 3,
|
||||
"finished": "2026-10-02T19:39:40+00:00",
|
||||
"published": [
|
||||
"facts.json",
|
||||
"summary.json"
|
||||
],
|
||||
"status": "done",
|
||||
"task": "facts"
|
||||
},
|
||||
"tar_put_at_last_modified": "2026-10-02T19:39:38+00:00"
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
|
||||
== kind cluster 'ousast' (context kind-ousast) + registry localhost:5001 (21:36:18)
|
||||
cluster exists; kept
|
||||
|
||||
== agent substrate (ate-system) (21:36:18)
|
||||
substrate ready; kept
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
ate-api-server-647bb6f8b6-c9k5k 1/1 Running 2 (13h ago) 3d1h
|
||||
ate-api-server-647bb6f8b6-dznjn 1/1 Running 1 (13h ago) 3d1h
|
||||
ate-controller-c95586684-g7nvn 1/1 Running 0 3d3h
|
||||
atelet-4f9c293-9zpfg 1/1 Running 0 3d3h
|
||||
atenet-egress-d74b8f6bf-8zmxj 1/1 Running 0 3d1h
|
||||
atenet-router-54887bc8f9-rqs6g 2/2 Running 2 (13h ago) 3d3h
|
||||
postgres-0 2/2 Running 3 (3d3h ago) 3d3h
|
||||
rustfs-6c4c677cb6-cqvhc 1/1 Running 3 (3d3h ago) 3d3h
|
||||
rustfs-bucket-init-nvfr6 0/1 Completed 0 3d3h
|
||||
|
||||
== egress gateway (atenet-egress, agentgateway variant: one prebuilt image, nothing built) (21:36:18)
|
||||
gateway present; kept
|
||||
|
||||
== ax control plane (ax-system) (21:36:19)
|
||||
ax ready; kept
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
ax-redis-86df786c87-pz5tp 1/1 Running 0 3d3h
|
||||
ax-server-b479bdc76-hwkqb 1/1 Running 0 3d3h
|
||||
ousast-pool-6656b55f5b-5brkn 1/1 Running 0 66m
|
||||
ousast-pool-6656b55f5b-9wbrw 1/1 Running 0 3d3h
|
||||
|
||||
== artifact receiver Service (ousast-receiver.ax-system -> this host) (21:36:19)
|
||||
service/ousast-receiver unchanged
|
||||
endpointslice.discovery.k8s.io/ousast-receiver-host configured
|
||||
|
||||
== worker pool (gVisor): pools.default of the profile (21:36:20)
|
||||
pool ousast-pool exists (2/2 ready); kept
|
||||
pod/ousast-pool-6656b55f5b-5brkn condition met
|
||||
pod/ousast-pool-6656b55f5b-9wbrw condition met
|
||||
|
||||
== runner image localhost:5001/ousast-runner (21:36:21)
|
||||
sha256:0501baf180b29f7aa7bbb8888bebabfc5d62429b7bdaa7f5a62b5b188f8ad5f6
|
||||
localhost:5001/ousast-runner:dev
|
||||
runner image localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019 -> /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json
|
||||
re-pin the templates with: ousast plane workspaces ... --runner-image /home/mc/Source/OpenUltraSAST/.claude/worktrees/agent-a7d4724e0c0c407a8/ops/k8s/profiles/kind-images.json (or ousast plane repin, task 4.3)
|
||||
|
||||
== ax CLI from the deployed checkout (a release CLI skews from the server) (21:36:32)
|
||||
ax CLI newer than the checkout; kept
|
||||
|
||||
== smoke task (21:36:32)
|
||||
task.ax.io/ousast-smoke created
|
||||
workspace.ax.io/ousast-smoke-ws unchanged
|
||||
task.ax.io/ousast-smoke resumed
|
||||
NAME ATESPACE PHASE ACTOR WORKER-IP AGE
|
||||
ousast-smoke default Running ousast-smoke 10.244.0.29 4s
|
||||
|
||||
== footprint (21:36:38)
|
||||
total used free shared buff/cache available
|
||||
Mem: 7877 5202 382 63 2903 2674
|
||||
kind-registry 32.45MiB / 7.693GiB 0.00%
|
||||
up.sh exit 0
|
||||
@@ -0,0 +1,792 @@
|
||||
{
|
||||
"id": "2026-10-02-pre-push-operational-1",
|
||||
"task": "pre-push-safety-net 17.7",
|
||||
"requirements": [
|
||||
"9.1",
|
||||
"9.2",
|
||||
"9.3",
|
||||
"9.4",
|
||||
"9.5",
|
||||
"9.6",
|
||||
"6.3"
|
||||
],
|
||||
"question": "Does the hook installed on any repository give honest, advisory-only results, and never block a push?",
|
||||
"subjects": "The audit's three shallow clones (kennethreitz/records, expressjs/serve-static, unrolled/secure), each with the same planted commit c2 on c1, copied fresh from the audit's scratch clones.",
|
||||
"instrument_proof": "Every replay and hook push records snapshots[].bytes_read and quick_tier files/bytes_read. The head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head (records 43,577; serve-static 61,525; secure 103,023), and the quick tier reads exactly the changed source files.",
|
||||
"host": "Joern is not on the host, so host runs cannot run the engine; that is the case a first-time pip user hits, and the output must say so. The Docker engine run uses the 2026-09-14 openultrasast:dev image with the worktree src mounted read-only, as the audit did; the image was not rebuilt (1.1 GB disk free, below the 4 GB threshold).",
|
||||
"before": {
|
||||
"source": "/tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready/gap-report.md (audit at main 49e11e0)",
|
||||
"replay_host": {
|
||||
"records": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 1.06,
|
||||
"files_bytes_read": [
|
||||
21,
|
||||
43577
|
||||
],
|
||||
"findings_shown": 0,
|
||||
"terminal": "generic: Some checks could not be completed. Review the comparison details...",
|
||||
"skip_reasons_named_in_terminal": 0
|
||||
},
|
||||
"serve-static": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 1.1,
|
||||
"files_bytes_read": [
|
||||
26,
|
||||
61525
|
||||
],
|
||||
"findings_shown": 0,
|
||||
"terminal": "same generic line",
|
||||
"skip_reasons_named_in_terminal": 0
|
||||
},
|
||||
"secure": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 1.04,
|
||||
"files_bytes_read": [
|
||||
19,
|
||||
103023
|
||||
],
|
||||
"findings_shown": 0,
|
||||
"terminal": "same generic line; Go not named",
|
||||
"skip_reasons_named_in_terminal": 0
|
||||
}
|
||||
},
|
||||
"docker_engine_serve_static": {
|
||||
"deadline_30": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 31.3,
|
||||
"terminal": "Analysis did not finish. Rerun with a longer analysis deadline."
|
||||
},
|
||||
"deadline_300": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 261.7,
|
||||
"raw_engine_findings": 2,
|
||||
"admitted": 0,
|
||||
"dispositions": "injection and output_encoding at debug.js:2: comparison_unknown/base_incomplete + capability_unavailable",
|
||||
"terminal": "No actionable defects reported; Details: /out/... (container path)"
|
||||
}
|
||||
},
|
||||
"quick_scan": {
|
||||
"serve-static": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.46,
|
||||
"findings": "js-eval, js-xss-response at debug.js:2"
|
||||
},
|
||||
"records": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.49,
|
||||
"findings": "2 pre-existing (setup.py os-command); planted %-formatted SQL missed"
|
||||
},
|
||||
"secure": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.48,
|
||||
"findings": 0,
|
||||
"terminal": "No quick-mode findings were emitted (silent zero for Go)"
|
||||
}
|
||||
},
|
||||
"hook": {
|
||||
"ousast_not_on_path": {
|
||||
"push_exit": "rejected",
|
||||
"terminal": ".git/hooks/pre-push: 13: exec: ousast: not found"
|
||||
},
|
||||
"first_push_new_branch": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.58,
|
||||
"terminal": "No analysis result is available. (does not say new branch)"
|
||||
},
|
||||
"update": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 1.36,
|
||||
"terminal": "No actionable defects reported; analysis is incomplete."
|
||||
},
|
||||
"bad_OUSAST_PUSH_DEADLINE_or_cache_inside_repo": {
|
||||
"exit": 2,
|
||||
"effect": "would block the push"
|
||||
},
|
||||
"install": "ops/install-pre-push only (checkout); not in the wheel"
|
||||
}
|
||||
},
|
||||
"verdict": {
|
||||
"never_blocks": "PASS in this sample: 10 hook pushes across the three repositories all exited 0, including `ousast` absent from PATH (rejected before) and OUSAST_PUSH_DEADLINE=abc (exit 2 before). Blocking stays an explicit opt-in (unit-tested, not exercised here).",
|
||||
"never_hides": "PASS in this sample: every incomplete run printed one plain `Skipped:` line per reason (2-5 per run), with no hex path or sha256 token. Go is named as not covered (`language not covered: go (1 changed file(s), 10 in the repository)`, and `notice: language not covered: go (10 files)` in `scan --mode quick`). The first push of a new branch says there is no base and what it checked instead.",
|
||||
"quick_tier_within_deadline": "On the host, every replay and hook push finished in 0.69-1.21 s against the 30 s deadline. The quick tier read exactly the changed source file (records 1 file / 17,449 B; serve-static 1 / 87 B; secure 1 / 193 B).",
|
||||
"planted_js_eval": "SURFACES. Quick tier: js-eval (CWE-95) and js-xss-response (CWE-79) at debug.js:2 on the host, 0.9 s, in replay and in the hook push. Engine (Docker, deadline 300): injection and output_encoding at debug.js:2, now novelty `new` / `file_added_in_head` (before: `unknown` / `base_incomplete`). The only veto is `capability_unavailable`, so it is shown as an advisory engine finding, not an alert.",
|
||||
"planted_python_sql": "STILL MISSED. No quick rule matches the %-formatted db.query string in records.py, and the host has no Joern, so nothing in the pre-push path reports it. Recorded as a coverage gap, with no rule edit.",
|
||||
"planted_go_command": "NOT DETECTED, NOW NAMED: Go has no engine frontend and no quick rules.",
|
||||
"engine_deadline": "Not solved. The Docker engine took 190.9 s wall time (head 107.6 s, base 80.7 s) against a 300 s deadline. The audit measured 261.7 s on a different image/src combination. A 30 s hook deadline still cannot hold it. --engine background starts it after the push, and the next push showed the earlier result. On the host that result says 'the engine did not run (Joern unavailable)', which is honest."
|
||||
},
|
||||
"coverage_gaps": [
|
||||
"records: the planted Python SQL injection (`db.query(\"... '%s'\" % name)`) is matched by no quick rule, before or after. Recorded as a quick-rule coverage gap; no rule was edited for a planted sample (that would be hand-tuning). On the host the engine cannot run, so nothing in the pre-push path reports it.",
|
||||
"secure: Go has no engine frontend and no quick rules. It is now named in the terminal (language not covered: go), not detected.",
|
||||
"The engine still cannot finish inside the 30 s hook deadline (the audit measured 260 s on 26 files); --engine background moves it after the push, graph reuse remains the later speed task."
|
||||
],
|
||||
"after": {
|
||||
"replay_host": {
|
||||
"records": {
|
||||
"pre-push --engine inline": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 1.206878771,
|
||||
"deadline_seconds": 30,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"changed lines could not be matched one-to-one to base lines in records.py",
|
||||
"the changed code has no counterpart in the base revision to compare against",
|
||||
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
|
||||
"No actionable defects reported; analysis is incomplete.",
|
||||
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
|
||||
],
|
||||
"snapshots_files_bytes": [
|
||||
[
|
||||
21,
|
||||
43577
|
||||
],
|
||||
[
|
||||
21,
|
||||
43482
|
||||
]
|
||||
],
|
||||
"quick_tier_files_bytes": [
|
||||
[
|
||||
1,
|
||||
17449
|
||||
]
|
||||
],
|
||||
"quick_tier_seconds": [
|
||||
0.041
|
||||
],
|
||||
"coverage_reasons_raw": [
|
||||
"7265636f7264732e7079:ambiguous_line_correspondence",
|
||||
"base_counterpart_unresolved",
|
||||
"cpg_build_failed",
|
||||
"analysis_incomplete",
|
||||
"base_comparison_unavailable"
|
||||
],
|
||||
"admitted_alerts": 0,
|
||||
"push_disposition": "allow",
|
||||
"total_seconds_internal": 0.483
|
||||
},
|
||||
"pre-push --engine off": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.954010589,
|
||||
"deadline_seconds": 30,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"changed lines could not be matched one-to-one to base lines in records.py",
|
||||
"engine checks were switched off for this run (--engine off)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
|
||||
"No analysis result is available.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
|
||||
],
|
||||
"snapshots_files_bytes": [
|
||||
[
|
||||
21,
|
||||
43577
|
||||
],
|
||||
[
|
||||
21,
|
||||
43482
|
||||
]
|
||||
],
|
||||
"quick_tier_files_bytes": [
|
||||
[
|
||||
1,
|
||||
17449
|
||||
]
|
||||
],
|
||||
"quick_tier_seconds": [
|
||||
0.042
|
||||
],
|
||||
"coverage_reasons_raw": [
|
||||
"7265636f7264732e7079:ambiguous_line_correspondence",
|
||||
"engine_off",
|
||||
"analysis_incomplete",
|
||||
"base_comparison_unavailable"
|
||||
],
|
||||
"admitted_alerts": 0,
|
||||
"push_disposition": "allow",
|
||||
"total_seconds_internal": 0.498
|
||||
},
|
||||
"scan --mode quick": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.407089313,
|
||||
"terminal": [
|
||||
"file_targets=9",
|
||||
"findings=2"
|
||||
]
|
||||
}
|
||||
},
|
||||
"serve-static": {
|
||||
"pre-push --engine inline": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.892543629,
|
||||
"deadline_seconds": 30,
|
||||
"quick_findings_shown": [
|
||||
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
|
||||
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
|
||||
],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"the changed code has no counterpart in the base revision to compare against",
|
||||
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
|
||||
"No actionable defects reported; analysis is incomplete.",
|
||||
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
|
||||
],
|
||||
"snapshots_files_bytes": [
|
||||
[
|
||||
26,
|
||||
61525
|
||||
],
|
||||
[
|
||||
25,
|
||||
61438
|
||||
]
|
||||
],
|
||||
"quick_tier_files_bytes": [
|
||||
[
|
||||
1,
|
||||
87
|
||||
]
|
||||
],
|
||||
"quick_tier_seconds": [
|
||||
0.017
|
||||
],
|
||||
"coverage_reasons_raw": [
|
||||
"base_counterpart_unresolved",
|
||||
"cpg_build_failed",
|
||||
"analysis_incomplete",
|
||||
"base_comparison_unavailable"
|
||||
],
|
||||
"admitted_alerts": 0,
|
||||
"push_disposition": "allow",
|
||||
"total_seconds_internal": 0.496
|
||||
},
|
||||
"pre-push --engine off": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.878791036,
|
||||
"deadline_seconds": 30,
|
||||
"quick_findings_shown": [
|
||||
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
|
||||
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
|
||||
],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"engine checks were switched off for this run (--engine off)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
|
||||
"No analysis result is available.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
|
||||
],
|
||||
"snapshots_files_bytes": [
|
||||
[
|
||||
26,
|
||||
61525
|
||||
],
|
||||
[
|
||||
25,
|
||||
61438
|
||||
]
|
||||
],
|
||||
"quick_tier_files_bytes": [
|
||||
[
|
||||
1,
|
||||
87
|
||||
]
|
||||
],
|
||||
"quick_tier_seconds": [
|
||||
0.016
|
||||
],
|
||||
"coverage_reasons_raw": [
|
||||
"engine_off",
|
||||
"analysis_incomplete",
|
||||
"base_comparison_unavailable"
|
||||
],
|
||||
"admitted_alerts": 0,
|
||||
"push_disposition": "allow",
|
||||
"total_seconds_internal": 0.471
|
||||
},
|
||||
"scan --mode quick": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.394269229,
|
||||
"terminal": [
|
||||
"file_targets=4",
|
||||
"findings=2"
|
||||
]
|
||||
}
|
||||
},
|
||||
"secure": {
|
||||
"pre-push --engine inline": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.762186729,
|
||||
"deadline_seconds": 30,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
|
||||
"the changed code has no counterpart in the base revision to compare against",
|
||||
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
|
||||
"engine has no frontend for at least one language in this push",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
|
||||
"No actionable defects reported; analysis is incomplete.",
|
||||
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details."
|
||||
],
|
||||
"snapshots_files_bytes": [
|
||||
[
|
||||
19,
|
||||
103023
|
||||
],
|
||||
[
|
||||
18,
|
||||
102830
|
||||
]
|
||||
],
|
||||
"quick_tier_files_bytes": [
|
||||
[
|
||||
1,
|
||||
193
|
||||
]
|
||||
],
|
||||
"quick_tier_seconds": [
|
||||
0.015
|
||||
],
|
||||
"coverage_reasons_raw": [
|
||||
"language_not_covered:go:1:10",
|
||||
"base_counterpart_unresolved",
|
||||
"cpg_build_failed",
|
||||
"frontend_unsupported",
|
||||
"analysis_incomplete",
|
||||
"base_comparison_unavailable"
|
||||
],
|
||||
"admitted_alerts": 0,
|
||||
"push_disposition": "allow",
|
||||
"total_seconds_internal": 0.352
|
||||
},
|
||||
"pre-push --engine off": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.694374205,
|
||||
"deadline_seconds": 30,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
|
||||
"engine checks were switched off for this run (--engine off)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
|
||||
"No analysis result is available.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying."
|
||||
],
|
||||
"snapshots_files_bytes": [
|
||||
[
|
||||
19,
|
||||
103023
|
||||
],
|
||||
[
|
||||
18,
|
||||
102830
|
||||
]
|
||||
],
|
||||
"quick_tier_files_bytes": [
|
||||
[
|
||||
1,
|
||||
193
|
||||
]
|
||||
],
|
||||
"quick_tier_seconds": [
|
||||
0.016
|
||||
],
|
||||
"coverage_reasons_raw": [
|
||||
"language_not_covered:go:1:10",
|
||||
"engine_off",
|
||||
"analysis_incomplete",
|
||||
"base_comparison_unavailable"
|
||||
],
|
||||
"admitted_alerts": 0,
|
||||
"push_disposition": "allow",
|
||||
"total_seconds_internal": 0.311
|
||||
},
|
||||
"scan --mode quick": {
|
||||
"exit": 0,
|
||||
"wall_seconds": 0.372512766,
|
||||
"terminal": [
|
||||
"file_targets=10",
|
||||
"findings=0",
|
||||
"notice: language not covered: go (10 files)"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"hook_host": {
|
||||
"records": {
|
||||
"new-branch": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.873912782,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"New branch: instead, quick rules checked the files its last commit changed (compared with c925575).",
|
||||
"Quick rules: no pattern match on changed code (2 file(s), 18416 bytes read).",
|
||||
"No analysis result is available.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
|
||||
" * [new branch] HEAD~1 -> main"
|
||||
]
|
||||
},
|
||||
"update": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.890046656,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"changed lines could not be matched one-to-one to base lines in records.py",
|
||||
"the changed code has no counterpart in the base revision to compare against",
|
||||
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 17449 bytes read).",
|
||||
"No actionable defects reported; analysis is incomplete.",
|
||||
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-records.git",
|
||||
" ea42736..ad3d7db HEAD -> main"
|
||||
]
|
||||
},
|
||||
"install_uninstall": [
|
||||
"install rc=0",
|
||||
"uninstall rc=0"
|
||||
]
|
||||
},
|
||||
"serve-static": {
|
||||
"new-branch": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.904302643,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"New branch: instead, quick rules checked the files its last commit changed (compared with 76312e1).",
|
||||
"Quick rules: no pattern match on changed code (0 file(s), 0 bytes read).",
|
||||
"No analysis result is available.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
|
||||
" * [new branch] HEAD~1 -> main"
|
||||
]
|
||||
},
|
||||
"update": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.92386555,
|
||||
"quick_findings_shown": [
|
||||
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
|
||||
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
|
||||
],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"the changed code has no counterpart in the base revision to compare against",
|
||||
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
|
||||
"No actionable defects reported; analysis is incomplete.",
|
||||
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
|
||||
" 74be78a..468213d HEAD -> main"
|
||||
]
|
||||
},
|
||||
"no-ousast": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.017937758,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [],
|
||||
"other_lines": [
|
||||
"ousast: pre-push check did not run: 'ousast' is not on PATH. Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
|
||||
" 468213d..aad3dfe HEAD -> main"
|
||||
]
|
||||
},
|
||||
"bad-deadline": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.300776,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [],
|
||||
"other_lines": [
|
||||
"usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]",
|
||||
" [--comparison-base COMPARISON_BASE]",
|
||||
" [--model-config MODEL_CONFIG] [--prior-hook PRIOR_HOOK]",
|
||||
" --artifact ARTIFACT [--cache-dir CACHE_DIR]",
|
||||
" [--deadline DEADLINE]",
|
||||
" [--cancellation-allowance CANCELLATION_ALLOWANCE]",
|
||||
" [--max-regions MAX_REGIONS]",
|
||||
" [--mode {advisory,blocking}]",
|
||||
" [--incomplete-coverage {allow,block}]",
|
||||
" [--engine {inline,background,off}]",
|
||||
" [--background-deadline BACKGROUND_DEADLINE]",
|
||||
" [--experimental-declarations EXPERIMENTAL_DECLARATIONS]",
|
||||
" [--experimental-record-vetoes]",
|
||||
" path",
|
||||
"ousast pre-push: error: argument --deadline: invalid float value: 'abc'",
|
||||
"ousast: pre-push check did not run: invalid arguments or settings (message above). Push continues.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
|
||||
" aad3dfe..6fcfec7 HEAD -> main"
|
||||
]
|
||||
},
|
||||
"background-1": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.915950565,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [
|
||||
"Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result."
|
||||
],
|
||||
"skip_lines": [
|
||||
"engine checks run in the background after this push; the next push shows their result",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 93 bytes read).",
|
||||
"No analysis result is available.",
|
||||
"Engine: running in the background for c5b6c4d7060f (deadline 900.0 s); the next push shows its result.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
|
||||
" 6fcfec7..c5b6c4d HEAD -> main"
|
||||
]
|
||||
},
|
||||
"background-2": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.879868436,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [
|
||||
"Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable)."
|
||||
],
|
||||
"skip_lines": [
|
||||
"engine checks were switched off for this run (--engine off)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Engine result for c5b6c4d7060f from the previous push: the engine did not run (Joern unavailable).",
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 99 bytes read).",
|
||||
"No analysis result is available.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-serve-static.git",
|
||||
" c5b6c4d..3eee5ef HEAD -> main"
|
||||
]
|
||||
},
|
||||
"install_uninstall": [
|
||||
"install rc=0",
|
||||
"uninstall rc=0"
|
||||
]
|
||||
},
|
||||
"secure": {
|
||||
"new-branch": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.96295861,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"new branch: the remote has no base to compare against, so the engine comparison did not run (OUSAST_COMPARISON_BASE=<local branch> compares against a local branch instead)",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"New branch: instead, quick rules checked the files its last commit changed (compared with ba42bc3).",
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 41228 bytes read).",
|
||||
"No analysis result is available.",
|
||||
"Notice: Some checks could not be completed. Review the comparison details and resolve the recorded gaps before retrying.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
|
||||
" * [new branch] HEAD~1 -> main"
|
||||
]
|
||||
},
|
||||
"update": {
|
||||
"push_exit": 0,
|
||||
"wall_seconds": 0.747049238,
|
||||
"quick_findings_shown": [],
|
||||
"engine_findings_shown": [],
|
||||
"skip_lines": [
|
||||
"language not covered: go (1 changed file(s), 10 in the repository): no engine frontend and no quick rules, so silence here is not a clean result",
|
||||
"the changed code has no counterpart in the base revision to compare against",
|
||||
"engine did not run: the code graph could not be built (is Joern installed? 'joern-parse' must be on PATH, or run the hook through ops/ousast-docker)",
|
||||
"engine has no frontend for at least one language in this push",
|
||||
"no base revision was analysed, so no engine finding can be called new or worsened"
|
||||
],
|
||||
"other_lines": [
|
||||
"Quick rules: no pattern match on changed code (1 file(s), 193 bytes read).",
|
||||
"No actionable defects reported; analysis is incomplete.",
|
||||
"Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.",
|
||||
"To /tmp/claude-1000/-home-mc-Source-OpenUltraSAST/c09e30b2-39a4-4564-bc31-6ea71ffaa198/scratchpad/ready-impl/bare-secure.git",
|
||||
" 740abc8..64739c4 HEAD -> main"
|
||||
]
|
||||
},
|
||||
"install_uninstall": [
|
||||
"install rc=0",
|
||||
"uninstall rc=0"
|
||||
]
|
||||
}
|
||||
},
|
||||
"docker_engine_serve_static": {
|
||||
"log": [
|
||||
"MemAvailable=3283MB disk_free=1159MiB",
|
||||
"base=74be78a8ffad679edfe135c457ce141114d96fcc head=468213d8a737ad03a1f712c792c1db8476e154df",
|
||||
"exit=0 elapsed=190.895648822s"
|
||||
],
|
||||
"exit": 0,
|
||||
"wall_seconds": 190.895648822,
|
||||
"deadline_seconds": 300,
|
||||
"quick_findings_shown": [
|
||||
"js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review",
|
||||
"js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review"
|
||||
],
|
||||
"engine_findings_shown": [
|
||||
"Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert."
|
||||
],
|
||||
"skip_lines": [
|
||||
"engine findings are not qualified as alerts yet: no capability has passed independent evaluation",
|
||||
"the capability registry does not match this build (eligibility provenance mismatch), so no capability can be used",
|
||||
"20 engine check(s) depend on dynamic or external code the engine cannot follow"
|
||||
],
|
||||
"other_lines": [
|
||||
"Engine: 2 finding(s) this push introduced. Advisory: the capability is not yet qualified, so not an alert.",
|
||||
"Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.",
|
||||
"No actionable defects reported; analysis is incomplete.",
|
||||
"Notice: Some checks are not enabled for normal alerts. Review capability coverage in the details."
|
||||
],
|
||||
"snapshots_files_bytes": [
|
||||
[
|
||||
26,
|
||||
61525
|
||||
],
|
||||
[
|
||||
25,
|
||||
61438
|
||||
]
|
||||
],
|
||||
"quick_tier_files_bytes": [
|
||||
[
|
||||
1,
|
||||
87
|
||||
]
|
||||
],
|
||||
"quick_tier_seconds": [
|
||||
0.038
|
||||
],
|
||||
"coverage_reasons_raw": [
|
||||
"capability_unavailable",
|
||||
"eligibility:eligibility_provenance_mismatch",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:7ba72698b15c888c0835cad443fc1f177f1d2e13ed2658b17e816cc0b81415de",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:392beff95d0e245cbaef2676813d7f69d35599fb32977c8f197e6df46738fc75",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:ec16a67e984a2f82c3ce9a04dbc44eae1e950393b1c1c4fae88485446f1e2f95",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:0a3bd3f145260792e9a5d4af2546020689cfd08d0b1e82264d7fe31dda7210e7",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:1551c000ba35ba4e5488013f51171b27ab10df8b80f29509610670192cff8a30",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:5b687fa7647cefd89d064b3e1b5e5a62842f8fd23104900e8af8d5b14fc800fd",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:f122da538137e0a3a3e598c24e6af57c5e0b90ff48dacb4fd1b9d82ac20c35ef",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:8ab9ddcb1f4431fa3a3c1cce4cc3d289bcb5d363d47aac1bcf35bc7db86ce866",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:19f1031c2c010d91fb41e23955cd5f18eaa5ede355e68fab9e18c8249c21c2ee",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:d31a861124274a09e3cb8f3398831ca299cc0cfad72ea385083c1fd88b7a2c45",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:c8a1c8042befd3ff6f296921666949850a1c54d63c77dd44296e7be96e621127",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:3bb50c50f77027f1e0347f490e35fdd0a75ae586fdf92775c753877833a9d099",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:c32cb93cad379fee7b814900e6673aa5ae470c27acd55d33f855987bccb1ab8e",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:300fba1acc39cb0ab32ebbd66a8674daecf496b3d8c675d27f7f71cf9f788c9e",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:40c99dfe6d923602dc3aa6353f3b5bc017c0d1a67595c19ddf8206ef431db23d",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:0e99666c9be7c4704f6c6f858844d721ed8f0b4e3978d53073bc2c1b64d3f075",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:da903214d018e0b1bd7a3a3b635b3e0fb2a23fd44527f41a9483c0bf3913f2df",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:79d89b41b86aadcc0423baa786e0518ccff2a5b7853ff05986682ecbd636af52",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:7d3331ca3c06d1013c758d3648bfebb1223385955e77df977266fc621f62e733",
|
||||
"dynamic_external_or_depth_context_unresolved:contributor-scan:54fd93450c8db7e6daef06fe795babb26726402211ca8f6d0a8679a375786041",
|
||||
"analysis_incomplete"
|
||||
],
|
||||
"admitted_alerts": 0,
|
||||
"push_disposition": "allow",
|
||||
"total_seconds_internal": 189.327,
|
||||
"dispositions": [
|
||||
{
|
||||
"family": "injection",
|
||||
"site": "debug.js:2:debug",
|
||||
"novelty": "new",
|
||||
"reason": "file_added_in_head",
|
||||
"vetoes": [
|
||||
"capability_unavailable"
|
||||
]
|
||||
},
|
||||
{
|
||||
"family": "output_encoding",
|
||||
"site": "debug.js:2:debug",
|
||||
"novelty": "new",
|
||||
"reason": "file_added_in_head",
|
||||
"vetoes": [
|
||||
"capability_unavailable"
|
||||
]
|
||||
}
|
||||
],
|
||||
"timings": {
|
||||
"deadline_seconds": 300.0,
|
||||
"cancellation_allowance_seconds": 2.0,
|
||||
"resolution_seconds": 0.0,
|
||||
"provenance_seconds": 0.3,
|
||||
"snapshot_seconds": 0.3,
|
||||
"discovery_seconds": 0.3,
|
||||
"preparation_seconds": 0.6,
|
||||
"quick_seconds": 0.0,
|
||||
"head_seconds": 107.6,
|
||||
"base_seconds": 80.7,
|
||||
"admission_seconds": 0.0,
|
||||
"cleanup_seconds": 0.0,
|
||||
"total_seconds": 189.3
|
||||
}
|
||||
}
|
||||
},
|
||||
"build": {
|
||||
"worktree_head": "591818d + hook install-hint fix (this commit)",
|
||||
"wheel_sha256": "5b725adb052239c2afaec237fecf582f69fc5ea61799dc67f0c07e09e10908c9",
|
||||
"installed_with": "uv pip install <wheel> into a scratch venv"
|
||||
},
|
||||
"inputs": {
|
||||
"records": {
|
||||
"language": "python",
|
||||
"tracked_files": 21,
|
||||
"tracked_bytes": 43577,
|
||||
"planted": "records.py: db.query(\"... '%s'\" % name)"
|
||||
},
|
||||
"serve-static": {
|
||||
"language": "javascript",
|
||||
"tracked_files": 26,
|
||||
"tracked_bytes": 61525,
|
||||
"planted": "new debug.js: res.end(String(eval(req.query.code)))"
|
||||
},
|
||||
"secure": {
|
||||
"language": "go",
|
||||
"tracked_files": 19,
|
||||
"tracked_bytes": 103023,
|
||||
"planted": "new run.go: exec.Command(\"sh\", \"-c\", r.URL.Query().Get(\"cmd\"))"
|
||||
}
|
||||
},
|
||||
"notes": [
|
||||
"The Docker engine run used the worktree src at 591818d mounted over the stale 2026-09-14 openultrasast:dev image (not rebuilt: 1.1 GB disk free, below the 4 GB threshold). `eligibility_provenance_mismatch` still appears; the audit already suspected src/image skew, and it is not investigated here.",
|
||||
"The run shows host paths after `Details:` because ops/ousast-docker mounts the artifact directory at its own path. Before, it showed the container path /out/...",
|
||||
"The hook's 'not on PATH' hint first said 'pip install openultrasast', which is not on PyPI. The first proof run caught it. It was corrected to the git URL, and the replay and hook proofs were rerun on the rebuilt wheel recorded under build. The Docker engine run predates the hint fix and does not involve the hook script.",
|
||||
"Host runs read their input. Each head snapshot's bytes equal `git ls-files | xargs cat | wc -c` at head, and the quick tier lists the files it read."
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,351 @@
|
||||
{
|
||||
"created": "2026-10-03",
|
||||
"note": "COUNTS ONLY. Real (vulnerable function, fixed function) pairs where the function survives the fix, mined at zero model cost from reviewed GitHub advisories for the six compiled families, added as pointer-pair corpus benchmarks/pairs/advisory-fixes-2 (98 pairs, source `advisory-fixes-2` in learn/sources.toml), labelled, given static feature records and built into the memory store; their excerpts embedded. No chat-model call. Candidate, path, function and repository names stay in the catalog; the two pairs the reserved-population test flagged are withheld (never printed: found by bisection with the test's output discarded).",
|
||||
"method": {
|
||||
"enumeration": "gh api /advisories?type=reviewed&ecosystem=<eco>&cwes=<family CWEs>&published=>2025-06-01, per ecosystem (pip, npm, maven, composer, go, rubygems) x family; newest first",
|
||||
"advisories_enumerated": 6594,
|
||||
"advisories_published_after_2026_06_01": 2400,
|
||||
"automated_screen": "single family by CWE; a linked fix commit (at most 3, one repository); repository not named anywhere under benchmarks/ (populations v1/v2, every pair corpus; excluded v3 file names never opened), no WordPress plugin, one pair per repository; SPDX licence permissive; one-parent commit with <= 4 non-test source files and <= 120 changed source lines; the function holding most changed CODE lines (comments and blanks excluded) inside its own matcher span, declared exactly once on both sides, body changed, >= 25% of the file's changed code lines (second pass; the first pass counted lines by nearest declaration above)",
|
||||
"file_reads": "the two versions of each changed source file through the contents API (metadata + two raw files per file; no clone, no scanner, engine, plane task or model on any of these repositories)",
|
||||
"manual_review": "every automated candidate's function diff was read (agent, no model call) and kept only when the named function itself holds the fix",
|
||||
"reserved_population_guard": "tests/test_independent_population.py::test_no_reserved_repository_is_referenced_by_the_development_tree, run with its output discarded; bisection over catalog subsets, 23 runs",
|
||||
"quotas": "20 per family on the second pass (14 on the first); advisories past a met quota were not examined",
|
||||
"mechanism": "family default (injection/output_encoding/deserialization source_reaches_sink, path path_join_user_input, access_control missing_auth_guard, untrusted_destination validation_strength for CWE-601 else source_reaches_sink); provenance = agent"
|
||||
},
|
||||
"screened": {
|
||||
"advisories_examined_second_pass": 2983,
|
||||
"outcomes_second_pass": {
|
||||
"no fix commit linked": 1247,
|
||||
"repository already in benchmarks/ (populations v1/v2 or a pair corpus)": 520,
|
||||
"language outside the feature vocabulary (go/ruby)": 332,
|
||||
"CWEs span several families": 253,
|
||||
"repository already kept (one pair per repository)": 140,
|
||||
"licence not permissive or unrecorded": 135,
|
||||
"large commit (refactor or release, not the fix)": 125,
|
||||
"kept": 110,
|
||||
"no function declared on both sides with a changed body": 51,
|
||||
"fix spread over several commits or repositories": 25,
|
||||
"manual review: the chosen function does not hold the fix (comment-only, misattributed, partial or family mismatch)": 13,
|
||||
"no source file changed": 11,
|
||||
"merge or root commit": 11,
|
||||
"language outside the feature vocabulary (go/ruby/c#/...)": 9,
|
||||
"WordPress plugin (population v3 safety)": 1
|
||||
},
|
||||
"candidates_manually_reviewed": 145,
|
||||
"kept_after_review": 100,
|
||||
"review_drops": {
|
||||
"total": 45,
|
||||
"function misattributed (matcher span overrun or an undetected enclosing declaration)": 13,
|
||||
"change in the function unrelated to the advisory (refactor, rename, formatting, another advisory)": 11,
|
||||
"partial: the function only calls or configures the real fix": 7,
|
||||
"comment or documentation change only": 6,
|
||||
"one pair per repository (a reviewed pair of the same repository kept)": 4,
|
||||
"second pass chose another function of an advisory already reviewed": 2,
|
||||
"family mismatch": 1,
|
||||
"unsure": 1
|
||||
},
|
||||
"reserved_population_drops": 2,
|
||||
"kept": 98
|
||||
},
|
||||
"kept": {
|
||||
"total": 98,
|
||||
"by_family": {
|
||||
"access_control": 17,
|
||||
"deserialization": 11,
|
||||
"injection": 20,
|
||||
"output_encoding": 16,
|
||||
"path": 17,
|
||||
"untrusted_destination": 17
|
||||
},
|
||||
"post_cutoff_by_family": {
|
||||
"access_control": 13,
|
||||
"deserialization": 2,
|
||||
"injection": 20,
|
||||
"output_encoding": 16,
|
||||
"path": 17,
|
||||
"untrusted_destination": 11
|
||||
},
|
||||
"post_cutoff_total": 79,
|
||||
"by_language": {
|
||||
"typescript": 34,
|
||||
"python": 30,
|
||||
"javascript": 15,
|
||||
"java": 10,
|
||||
"php": 9
|
||||
},
|
||||
"by_ecosystem": {
|
||||
"npm": 48,
|
||||
"pip": 32,
|
||||
"maven": 9,
|
||||
"composer": 9
|
||||
},
|
||||
"licences": {
|
||||
"MIT": 67,
|
||||
"Apache-2.0": 24,
|
||||
"BSD-3-Clause": 5,
|
||||
"ISC": 1,
|
||||
"Unlicense": 1
|
||||
},
|
||||
"split": {
|
||||
"train": 51,
|
||||
"holdout": 47
|
||||
},
|
||||
"published_range": [
|
||||
"2025-06-06",
|
||||
"2026-10-02"
|
||||
],
|
||||
"verification": {
|
||||
"verified": 98,
|
||||
"excerpt_bytes_read": 526134
|
||||
},
|
||||
"known_limit": "excerpts inherit the label builder's matcher: some spans run past the function's end into the next declaration; the label is path::function and the changed lines were checked inside the span"
|
||||
},
|
||||
"labels": {
|
||||
"command": "ousast learn labels --root . --out <scratch> --snapshot <scratch>",
|
||||
"rows": [
|
||||
6589,
|
||||
6981
|
||||
],
|
||||
"new_pair_rows": 392,
|
||||
"new_pin_rows_by_label": {
|
||||
"positive": 98,
|
||||
"negative": 98
|
||||
},
|
||||
"fixed_side_negative": "98 of 98 pairs (absent_fixed_side 0)"
|
||||
},
|
||||
"features": {
|
||||
"command": "benchmarks/learn/build_features.py --labels <the 392 new rows> --units <units.json of `ousast plane harvest --catalog benchmarks/pairs/advisory-fixes-2/catalog.toml` (manifests written, no Run started)> --kinds pairs",
|
||||
"records": 392,
|
||||
"instrument_states_plane_all_families": {
|
||||
"agree:missing": 74,
|
||||
"agree:none": 122,
|
||||
"delta:not_applicable": 196,
|
||||
"engine:missing": 128,
|
||||
"engine:none": 68,
|
||||
"entry_points:not_applicable": 196,
|
||||
"facts:not_applicable": 196,
|
||||
"language:ran": 196,
|
||||
"model_sinks:missing": 196,
|
||||
"quick:ran": 196,
|
||||
"roles:ran": 196,
|
||||
"source:ran": 196,
|
||||
"verify:missing": 74,
|
||||
"verify:none": 122
|
||||
},
|
||||
"note": "quick rules and deterministic roles ran on each side; engine, verify, agree and model sinks never ran for these pairs (missing/none: null features, never zeros)"
|
||||
},
|
||||
"memory_build": {
|
||||
"command": "ousast learn memory build --labels <labels> --units <merged units.json> --profile static --root . (exit 2: adjudications 9 of 43 anonymous functions, unchanged since 2026-10-01; rows kept)",
|
||||
"examples": [
|
||||
937,
|
||||
1133
|
||||
],
|
||||
"pairs_examples": [
|
||||
772,
|
||||
968
|
||||
],
|
||||
"pairs_counts_after": {
|
||||
"absent_fixed_side": 0,
|
||||
"examples": 968,
|
||||
"labels": 1936,
|
||||
"no_excerpt": 0,
|
||||
"no_record": 968,
|
||||
"recorded": 968,
|
||||
"unread": 0
|
||||
}
|
||||
},
|
||||
"positives_only_groups": {
|
||||
"how": "the 2026-10-02 gap record's count: static example rows of the store, per family, repository groups with a positive and no negative",
|
||||
"before": 99,
|
||||
"after": 99,
|
||||
"before_note": "the store before this harvest held 937 examples (the 10-02 gap record counted 904 and 98; the 10-02 rebuild added one population-v2 positive without a negative)",
|
||||
"why_unchanged": "the 99 are data gaps of other corpora (94 vibe-py pairs whose fixed side lost the function, 3 dev-php, 2 population-v2); a new repository group arrives with both labels, so it adds paired groups and leaves those counts as they were",
|
||||
"paired_groups_total": [
|
||||
151,
|
||||
249
|
||||
],
|
||||
"positives_only_share_of_positive_groups_six_families": [
|
||||
0.4034,
|
||||
0.2857
|
||||
],
|
||||
"by_family_before_after": {
|
||||
"access_control": {
|
||||
"groups_pos_only": [
|
||||
12,
|
||||
12
|
||||
],
|
||||
"groups_pos": [
|
||||
32,
|
||||
49
|
||||
],
|
||||
"groups_both": [
|
||||
20,
|
||||
37
|
||||
],
|
||||
"examples_neg": [
|
||||
20,
|
||||
37
|
||||
]
|
||||
},
|
||||
"config_secrets": {
|
||||
"groups_pos_only": [
|
||||
3,
|
||||
3
|
||||
],
|
||||
"groups_pos": [
|
||||
5,
|
||||
5
|
||||
],
|
||||
"groups_both": [
|
||||
2,
|
||||
2
|
||||
],
|
||||
"examples_neg": [
|
||||
13,
|
||||
13
|
||||
]
|
||||
},
|
||||
"deserialization": {
|
||||
"groups_pos_only": [
|
||||
20,
|
||||
20
|
||||
],
|
||||
"groups_pos": [
|
||||
41,
|
||||
52
|
||||
],
|
||||
"groups_both": [
|
||||
21,
|
||||
32
|
||||
],
|
||||
"examples_neg": [
|
||||
21,
|
||||
32
|
||||
]
|
||||
},
|
||||
"injection": {
|
||||
"groups_pos_only": [
|
||||
11,
|
||||
11
|
||||
],
|
||||
"groups_pos": [
|
||||
44,
|
||||
64
|
||||
],
|
||||
"groups_both": [
|
||||
33,
|
||||
53
|
||||
],
|
||||
"examples_neg": [
|
||||
70,
|
||||
90
|
||||
]
|
||||
},
|
||||
"memory": {
|
||||
"groups_pos_only": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"groups_pos": [
|
||||
5,
|
||||
5
|
||||
],
|
||||
"groups_both": [
|
||||
5,
|
||||
5
|
||||
],
|
||||
"examples_neg": [
|
||||
173,
|
||||
173
|
||||
]
|
||||
},
|
||||
"output_encoding": {
|
||||
"groups_pos_only": [
|
||||
14,
|
||||
14
|
||||
],
|
||||
"groups_pos": [
|
||||
36,
|
||||
52
|
||||
],
|
||||
"groups_both": [
|
||||
22,
|
||||
38
|
||||
],
|
||||
"examples_neg": [
|
||||
61,
|
||||
77
|
||||
]
|
||||
},
|
||||
"path": {
|
||||
"groups_pos_only": [
|
||||
12,
|
||||
12
|
||||
],
|
||||
"groups_pos": [
|
||||
33,
|
||||
50
|
||||
],
|
||||
"groups_both": [
|
||||
21,
|
||||
38
|
||||
],
|
||||
"examples_neg": [
|
||||
21,
|
||||
38
|
||||
]
|
||||
},
|
||||
"unknown": {
|
||||
"groups_pos_only": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"groups_pos": [
|
||||
2,
|
||||
2
|
||||
],
|
||||
"groups_both": [
|
||||
2,
|
||||
2
|
||||
],
|
||||
"examples_neg": [
|
||||
2,
|
||||
2
|
||||
]
|
||||
},
|
||||
"untrusted_destination": {
|
||||
"groups_pos_only": [
|
||||
27,
|
||||
27
|
||||
],
|
||||
"groups_pos": [
|
||||
52,
|
||||
69
|
||||
],
|
||||
"groups_both": [
|
||||
25,
|
||||
42
|
||||
],
|
||||
"examples_neg": [
|
||||
36,
|
||||
53
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"embeddings": {
|
||||
"command": "ousast learn memory embed --budget-usd 0.20 (dry run first: 66,005 estimated tokens, $0.0013)",
|
||||
"requested": 1004,
|
||||
"cached": 781,
|
||||
"embedded": 223,
|
||||
"calls": 7,
|
||||
"tokens": 71163,
|
||||
"usd": 0.001423,
|
||||
"note": "embedded = the new pairs' excerpts plus the excerpts the 10-02 rebuild left without a vector; OpenRouter, the embedding model .env names"
|
||||
},
|
||||
"spend": {
|
||||
"chat_model_calls": 0,
|
||||
"embedding_usd": 0.001423,
|
||||
"github_api": "metadata and raw contents only"
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,496 @@
|
||||
{
|
||||
"pairs": [
|
||||
{
|
||||
"name": "prefect-token-validation-e216171",
|
||||
"ok": true,
|
||||
"bytes_read": 3697
|
||||
},
|
||||
{
|
||||
"name": "arcadedb-getdatabasegroupsconfiguration-04110c0",
|
||||
"ok": true,
|
||||
"bytes_read": 3896
|
||||
},
|
||||
{
|
||||
"name": "langflow-get-flow-by-id-or-endpoint-name-2c9f498",
|
||||
"ok": true,
|
||||
"bytes_read": 3948
|
||||
},
|
||||
{
|
||||
"name": "grav-plugin-api-update-26f529c",
|
||||
"ok": true,
|
||||
"bytes_read": 5598
|
||||
},
|
||||
{
|
||||
"name": "google-maps-mcp-main-00d8725",
|
||||
"ok": true,
|
||||
"bytes_read": 3371
|
||||
},
|
||||
{
|
||||
"name": "camofox-mcp-createmcphttpapp-599f56e",
|
||||
"ok": true,
|
||||
"bytes_read": 5587
|
||||
},
|
||||
{
|
||||
"name": "spring-graphql-isoverridefor-63a89ed",
|
||||
"ok": true,
|
||||
"bytes_read": 2045
|
||||
},
|
||||
{
|
||||
"name": "jenkins-dofilltimezonenameitems-0586de4",
|
||||
"ok": true,
|
||||
"bytes_read": 2093
|
||||
},
|
||||
{
|
||||
"name": "netlicensing-mcp-dispatch-fbbb1d5",
|
||||
"ok": true,
|
||||
"bytes_read": 2610
|
||||
},
|
||||
{
|
||||
"name": "starlette-admin-validate-where-af05b45",
|
||||
"ok": true,
|
||||
"bytes_read": 2061
|
||||
},
|
||||
{
|
||||
"name": "next-auth-initauth-d008b9b",
|
||||
"ok": true,
|
||||
"bytes_read": 8193
|
||||
},
|
||||
{
|
||||
"name": "deepseek-mcp-server-starthttptransport-dab07ed",
|
||||
"ok": true,
|
||||
"bytes_read": 2394
|
||||
},
|
||||
{
|
||||
"name": "network-ai-routerequest-a59c13a",
|
||||
"ok": true,
|
||||
"bytes_read": 5945
|
||||
},
|
||||
{
|
||||
"name": "device-builder-parse-args-9e294f7",
|
||||
"ok": true,
|
||||
"bytes_read": 15243
|
||||
},
|
||||
{
|
||||
"name": "typespec-start-30d6f65",
|
||||
"ok": true,
|
||||
"bytes_read": 2109
|
||||
},
|
||||
{
|
||||
"name": "tomcat-findsecurityconstraints-8bafd79",
|
||||
"ok": true,
|
||||
"bytes_read": 18199
|
||||
},
|
||||
{
|
||||
"name": "vm2-makebuiltinsfromlegacyoptions-b0f5066",
|
||||
"ok": true,
|
||||
"bytes_read": 3539
|
||||
},
|
||||
{
|
||||
"name": "inlong-filtersensitive-ebcd6d3",
|
||||
"ok": true,
|
||||
"bytes_read": 6474
|
||||
},
|
||||
{
|
||||
"name": "h2o-3-validatejdbcurl-0298ee3",
|
||||
"ok": true,
|
||||
"bytes_read": 4919
|
||||
},
|
||||
{
|
||||
"name": "skops-load-model-29d61ea",
|
||||
"ok": true,
|
||||
"bytes_read": 1847
|
||||
},
|
||||
{
|
||||
"name": "picklescan-list-globals-58983e1",
|
||||
"ok": true,
|
||||
"bytes_read": 6975
|
||||
},
|
||||
{
|
||||
"name": "sglang-deserialize-49afb3d",
|
||||
"ok": true,
|
||||
"bytes_read": 4334
|
||||
},
|
||||
{
|
||||
"name": "pdfminer-six-load-data-b808ee0",
|
||||
"ok": true,
|
||||
"bytes_read": 2399
|
||||
},
|
||||
{
|
||||
"name": "reverb-handle-9ec26f8",
|
||||
"ok": true,
|
||||
"bytes_read": 3342
|
||||
},
|
||||
{
|
||||
"name": "transformers-is-kernel-a7f8e7f",
|
||||
"ok": true,
|
||||
"bytes_read": 1921
|
||||
},
|
||||
{
|
||||
"name": "opentelemetry-java-instrumentation-read-9cf4fba",
|
||||
"ok": true,
|
||||
"bytes_read": 2075
|
||||
},
|
||||
{
|
||||
"name": "keras-from-config-55888d3",
|
||||
"ok": true,
|
||||
"bytes_read": 3154
|
||||
},
|
||||
{
|
||||
"name": "airflow-handle-event-submit-a1ab065",
|
||||
"ok": true,
|
||||
"bytes_read": 4828
|
||||
},
|
||||
{
|
||||
"name": "sequelize-partialescape-5deadd2",
|
||||
"ok": true,
|
||||
"bytes_read": 6826
|
||||
},
|
||||
{
|
||||
"name": "electerm-mv-aa77881",
|
||||
"ok": true,
|
||||
"bytes_read": 1258
|
||||
},
|
||||
{
|
||||
"name": "systeminformation-checklinuxdchpinterfaces-bbfddde",
|
||||
"ok": true,
|
||||
"bytes_read": 2208
|
||||
},
|
||||
{
|
||||
"name": "prompty-buildagent-c27402d",
|
||||
"ok": true,
|
||||
"bytes_read": 3198
|
||||
},
|
||||
{
|
||||
"name": "datamodel-code-generator-get-field-extras-17fc235",
|
||||
"ok": true,
|
||||
"bytes_read": 2471
|
||||
},
|
||||
{
|
||||
"name": "spring-data-mongodb-bindablevaluefor-24d5afc",
|
||||
"ok": true,
|
||||
"bytes_read": 9895
|
||||
},
|
||||
{
|
||||
"name": "spring-ai-emitlucenestring-2b20cfc",
|
||||
"ok": true,
|
||||
"bytes_read": 4037
|
||||
},
|
||||
{
|
||||
"name": "aws-cdk-write-baa9e1d",
|
||||
"ok": true,
|
||||
"bytes_read": 1461
|
||||
},
|
||||
{
|
||||
"name": "danger-js-localgetfileatsha-087a729",
|
||||
"ok": true,
|
||||
"bytes_read": 1724
|
||||
},
|
||||
{
|
||||
"name": "argos-javascript-gitfetch-8355f3a",
|
||||
"ok": true,
|
||||
"bytes_read": 1319
|
||||
},
|
||||
{
|
||||
"name": "cakephp-transformfunctionexpression-138f2f6",
|
||||
"ok": true,
|
||||
"bytes_read": 6754
|
||||
},
|
||||
{
|
||||
"name": "orval-getroutepath-8ef1bfd",
|
||||
"ok": true,
|
||||
"bytes_read": 2607
|
||||
},
|
||||
{
|
||||
"name": "typeorm-handler-41d1c62",
|
||||
"ok": true,
|
||||
"bytes_read": 11977
|
||||
},
|
||||
{
|
||||
"name": "velocity-js-isblockedpathkey-f8e47a6",
|
||||
"ok": true,
|
||||
"bytes_read": 1539
|
||||
},
|
||||
{
|
||||
"name": "jsonata-evaluategroupexpression-4b217d5",
|
||||
"ok": true,
|
||||
"bytes_read": 7399
|
||||
},
|
||||
{
|
||||
"name": "nuxt-getislandcontext-5b60017",
|
||||
"ok": true,
|
||||
"bytes_read": 6805
|
||||
},
|
||||
{
|
||||
"name": "gitpython-option-candidates-96a888f",
|
||||
"ok": true,
|
||||
"bytes_read": 3421
|
||||
},
|
||||
{
|
||||
"name": "mermaid-addnamespace-12d472c",
|
||||
"ok": true,
|
||||
"bytes_read": 9722
|
||||
},
|
||||
{
|
||||
"name": "php-codesniffer-getblamecontent-7a3a6bb",
|
||||
"ok": true,
|
||||
"bytes_read": 2040
|
||||
},
|
||||
{
|
||||
"name": "xmldom-serializetostring-1cde3e3",
|
||||
"ok": true,
|
||||
"bytes_read": 23297
|
||||
},
|
||||
{
|
||||
"name": "trix-fromjson-9c0a993",
|
||||
"ok": true,
|
||||
"bytes_read": 1295
|
||||
},
|
||||
{
|
||||
"name": "oauthlib-prepare-token-revocation-request-c888359",
|
||||
"ok": true,
|
||||
"bytes_read": 4800
|
||||
},
|
||||
{
|
||||
"name": "suneditor-submitsrc-9d43a5e",
|
||||
"ok": true,
|
||||
"bytes_read": 2319
|
||||
},
|
||||
{
|
||||
"name": "mathlive-atomtomathml-5fe1c46",
|
||||
"ok": true,
|
||||
"bytes_read": 32529
|
||||
},
|
||||
{
|
||||
"name": "genql-comment-2dbc80f",
|
||||
"ok": true,
|
||||
"bytes_read": 1633
|
||||
},
|
||||
{
|
||||
"name": "angular-visitelementlike-417a407",
|
||||
"ok": true,
|
||||
"bytes_read": 8480
|
||||
},
|
||||
{
|
||||
"name": "cyberchef-objtotable-85db3be",
|
||||
"ok": true,
|
||||
"bytes_read": 2556
|
||||
},
|
||||
{
|
||||
"name": "mdc-isanchorlinkallowed-61d636c",
|
||||
"ok": true,
|
||||
"bytes_read": 1795
|
||||
},
|
||||
{
|
||||
"name": "node-xhtml-purifier-node-21d461a",
|
||||
"ok": true,
|
||||
"bytes_read": 7486
|
||||
},
|
||||
{
|
||||
"name": "orpc-init-daabded",
|
||||
"ok": true,
|
||||
"bytes_read": 6240
|
||||
},
|
||||
{
|
||||
"name": "mkdocs-material-mountsearchsuggest-52fb6be",
|
||||
"ok": true,
|
||||
"bytes_read": 4493
|
||||
},
|
||||
{
|
||||
"name": "jodit-isremovablenode-49a31f4",
|
||||
"ok": true,
|
||||
"bytes_read": 3921
|
||||
},
|
||||
{
|
||||
"name": "prowler-transform-6db407e",
|
||||
"ok": true,
|
||||
"bytes_read": 5481
|
||||
},
|
||||
{
|
||||
"name": "commonmark-parseattributes-dfcdf45",
|
||||
"ok": true,
|
||||
"bytes_read": 6437
|
||||
},
|
||||
{
|
||||
"name": "svgo-fn-3db3ef3",
|
||||
"ok": true,
|
||||
"bytes_read": 6895
|
||||
},
|
||||
{
|
||||
"name": "tiptap-mergeattributes-01d7af8",
|
||||
"ok": true,
|
||||
"bytes_read": 3690
|
||||
},
|
||||
{
|
||||
"name": "nogginlessdom-matchfilesnapshot-785e6ac",
|
||||
"ok": true,
|
||||
"bytes_read": 3023
|
||||
},
|
||||
{
|
||||
"name": "gradio-preprocess-97d541f",
|
||||
"ok": true,
|
||||
"bytes_read": 2581
|
||||
},
|
||||
{
|
||||
"name": "gemini-bridge-resolve-path-8f3b85a",
|
||||
"ok": true,
|
||||
"bytes_read": 2006
|
||||
},
|
||||
{
|
||||
"name": "libreoffice-convert-convertwithoptions-b78f17d",
|
||||
"ok": true,
|
||||
"bytes_read": 8912
|
||||
},
|
||||
{
|
||||
"name": "atomic-agents-stack-do-get-ec474f4",
|
||||
"ok": true,
|
||||
"bytes_read": 3493
|
||||
},
|
||||
{
|
||||
"name": "browse-mcp-downloadurl-5352a4a",
|
||||
"ok": true,
|
||||
"bytes_read": 5302
|
||||
},
|
||||
{
|
||||
"name": "notebooklm-mcp-makeslug-13828d9",
|
||||
"ok": true,
|
||||
"bytes_read": 2103
|
||||
},
|
||||
{
|
||||
"name": "langchain-nvidia-url-to-b64-string-b483c7c",
|
||||
"ok": true,
|
||||
"bytes_read": 2318
|
||||
},
|
||||
{
|
||||
"name": "redocly-cli-iteratepathitems-5041204",
|
||||
"ok": true,
|
||||
"bytes_read": 4412
|
||||
},
|
||||
{
|
||||
"name": "pip-filename-10dfb6b",
|
||||
"ok": true,
|
||||
"bytes_read": 1419
|
||||
},
|
||||
{
|
||||
"name": "lib-get-db-path-d665042",
|
||||
"ok": true,
|
||||
"bytes_read": 3879
|
||||
},
|
||||
{
|
||||
"name": "blender-mcp-download-polyhaven-asset-30a3308",
|
||||
"ok": true,
|
||||
"bytes_read": 35668
|
||||
},
|
||||
{
|
||||
"name": "datasets-set-feature-f989ef9",
|
||||
"ok": true,
|
||||
"bytes_read": 4882
|
||||
},
|
||||
{
|
||||
"name": "postcss-loadfile-7beca13",
|
||||
"ok": true,
|
||||
"bytes_read": 1712
|
||||
},
|
||||
{
|
||||
"name": "vitest-configureserver-51edf2b",
|
||||
"ok": true,
|
||||
"bytes_read": 3287
|
||||
},
|
||||
{
|
||||
"name": "next-js-getfilepath-968b9fc",
|
||||
"ok": true,
|
||||
"bytes_read": 2462
|
||||
},
|
||||
{
|
||||
"name": "composer-install-53b8bb4",
|
||||
"ok": true,
|
||||
"bytes_read": 4127
|
||||
},
|
||||
{
|
||||
"name": "vue-cli-mayproxy-72ba750",
|
||||
"ok": true,
|
||||
"bytes_read": 1203
|
||||
},
|
||||
{
|
||||
"name": "h3-redirectback-459a1c6",
|
||||
"ok": true,
|
||||
"bytes_read": 2750
|
||||
},
|
||||
{
|
||||
"name": "axios-setproxy-fb3befb",
|
||||
"ok": true,
|
||||
"bytes_read": 4403
|
||||
},
|
||||
{
|
||||
"name": "rembg-s-command-07ad0d4",
|
||||
"ok": true,
|
||||
"bytes_read": 21204
|
||||
},
|
||||
{
|
||||
"name": "http-server-getpreviousurl-2008fb6",
|
||||
"ok": true,
|
||||
"bytes_read": 4395
|
||||
},
|
||||
{
|
||||
"name": "authlib-get-authorization-grant-3be0846",
|
||||
"ok": true,
|
||||
"bytes_read": 2631
|
||||
},
|
||||
{
|
||||
"name": "java-client-overrideserverurl-2b9cd44",
|
||||
"ok": true,
|
||||
"bytes_read": 1382
|
||||
},
|
||||
{
|
||||
"name": "n8n-mcp-validatewebhookurl-9639f75",
|
||||
"ok": true,
|
||||
"bytes_read": 13458
|
||||
},
|
||||
{
|
||||
"name": "ufo-withoutbase-5cd9e67",
|
||||
"ok": true,
|
||||
"bytes_read": 2154
|
||||
},
|
||||
{
|
||||
"name": "symfony-dogenerate-4b63c3a",
|
||||
"ok": true,
|
||||
"bytes_read": 16981
|
||||
},
|
||||
{
|
||||
"name": "php-weasyprint-isoptionurl-9582dcf",
|
||||
"ok": true,
|
||||
"bytes_read": 1090
|
||||
},
|
||||
{
|
||||
"name": "compliance-trestle-check-metadata-endpoints-d107cd1",
|
||||
"ok": true,
|
||||
"bytes_read": 2412
|
||||
},
|
||||
{
|
||||
"name": "vllm-preprocess-audio-54503ec",
|
||||
"ok": true,
|
||||
"bytes_read": 4629
|
||||
},
|
||||
{
|
||||
"name": "phpspreadsheet-webservice-7ef7b25",
|
||||
"ok": true,
|
||||
"bytes_read": 4441
|
||||
},
|
||||
{
|
||||
"name": "frontmcp-dofetch-077201e",
|
||||
"ok": true,
|
||||
"bytes_read": 3737
|
||||
},
|
||||
{
|
||||
"name": "ip-address-islinklocal-d03e960",
|
||||
"ok": true,
|
||||
"bytes_read": 1489
|
||||
},
|
||||
{
|
||||
"name": "pyjwt-fetch-data-0a795b8",
|
||||
"ok": true,
|
||||
"bytes_read": 3385
|
||||
}
|
||||
],
|
||||
"verified": 98,
|
||||
"bytes_read": 526134
|
||||
}
|
||||
+114
-67
@@ -1,14 +1,22 @@
|
||||
# Architecture
|
||||
|
||||
How a scan works, how a finding earns its place, how severity and the project score are
|
||||
decided, and how the ruleset improves itself. State as of 2026-10-02; module paths are under
|
||||
`src/openultrasast/`.
|
||||
This page covers four things:
|
||||
|
||||
- how a scan works;
|
||||
- how a finding earns its place;
|
||||
- how severity and the project score are decided;
|
||||
- how the ruleset improves itself.
|
||||
|
||||
State as of 2026-10-02; module paths are under `src/openultrasast/`.
|
||||
|
||||
## The scan pipeline
|
||||
|
||||
The CLI is thin; `HarnessRuntime` owns the lifecycle and writes an event trace for every stage
|
||||
(`trace/events.jsonl`). The stages a mode requests are `MODE_STAGES` in `stages.py`:
|
||||
`quick` runs STATIC, `standard` adds MAP, `deep` adds REGRESS.
|
||||
The CLI is thin. `HarnessRuntime` owns the lifecycle. It writes an event trace for every stage
|
||||
(`trace/events.jsonl`). `MODE_STAGES` in `stages.py` lists the stages each mode requests:
|
||||
|
||||
- `quick` runs STATIC;
|
||||
- `standard` adds MAP;
|
||||
- `deep` adds REGRESS.
|
||||
|
||||
```
|
||||
policy_load ─▶ ruleset_load ─▶ policy_check (fail loud on an ungoverned CWE)
|
||||
@@ -28,14 +36,19 @@ policy_load ─▶ ruleset_load ─▶ policy_check (fail loud on an
|
||||
```
|
||||
|
||||
**The model layer** (`model/`, `cpg/`) is the core of `standard`. It builds one code property
|
||||
graph per scan with Joern (`joern-parse`, then batched `joern --script` queries from
|
||||
`cpg/queries/*.sc`) and dispatches each question to the arbiter of its spec type: taint
|
||||
(source to sink with no discharging sanitizer), dominance (does a guard govern the obligated
|
||||
operation) and configuration (constant abstraction over a value). Taint facts ship for C,
|
||||
Java, JavaScript, PHP and Python (`ruleset/semantic/*.toml`). Without Joern the layer records
|
||||
`cpg_unavailable` and the scan is otherwise unchanged; the Docker image ships Joern and
|
||||
`php-cli` (which the PHP frontend needs). The representations and techniques behind each stage
|
||||
are explained in [Detection techniques](detection-techniques.md).
|
||||
graph (CPG: a graph of the program's syntax, control flow and data flow) per scan with Joern.
|
||||
It runs `joern-parse`, then batched `joern --script` queries from `cpg/queries/*.sc`. Each
|
||||
question goes to the arbiter (a checker that decides one kind of question) for its spec type:
|
||||
|
||||
- taint (untrusted input reaching a dangerous call): source to sink with no discharging sanitizer;
|
||||
- dominance: does a guard govern the obligated operation;
|
||||
- configuration: constant abstraction over a value.
|
||||
|
||||
Taint facts ship for C, Java, JavaScript, PHP and Python (`ruleset/semantic/*.toml`). Without
|
||||
Joern, the layer records `cpg_unavailable` and the rest of the scan is unchanged. The Docker
|
||||
image ships Joern and `php-cli`, which the PHP frontend needs. [Detection
|
||||
techniques](detection-techniques.md) explains the representations and techniques behind each
|
||||
stage.
|
||||
|
||||
The model layer reports on a rung ladder:
|
||||
|
||||
@@ -47,23 +60,29 @@ model_corroborated the graph found the shape; a judge agreed on the residual
|
||||
suspicion proposed, nothing corroborated it
|
||||
```
|
||||
|
||||
The LLM (`model/pipeline.py`, `residual_question`) is asked only the residual question the graph
|
||||
cannot settle, for a flow the graph already found; it can confirm or contradict, never create a
|
||||
flow. When the graph resolved nothing, the enumerator's candidates (at most
|
||||
`MAX_JUDGED_CANDIDATES`, 8) are put to it, and anything it affirms is reported at `suspicion`.
|
||||
Without a provider key neither question is asked and only the graph's entailed findings are
|
||||
reported.
|
||||
The LLM answers only the residual question: the part the graph cannot settle
|
||||
(`model/pipeline.py`, `residual_question`). It is asked only about a flow the graph already
|
||||
found. It can confirm or contradict that flow, but never create one. When the graph resolved
|
||||
nothing, the LLM sees the enumerator's candidates (at most `MAX_JUDGED_CANDIDATES`, 8).
|
||||
Anything it affirms is reported at `suspicion`. Without a provider key, neither question is
|
||||
asked. Only the graph's entailed findings are reported then.
|
||||
|
||||
**REGRESS** (`regress/`, `sandbox/`) runs only when `docker info` succeeds. Each promoted
|
||||
candidate gets a small language recipe (Python, JavaScript, C) that loads the candidate's
|
||||
source inside a container with no network, a read-only `/workspace`, a non-root user and the
|
||||
`[sandbox]` limits. A structural deny-list rejects snippets mentioning sockets, curl, host
|
||||
networking or writes under `/workspace` before anything runs.
|
||||
candidate gets a small language recipe (Python, JavaScript, C). The recipe loads the
|
||||
candidate's source inside a container with:
|
||||
|
||||
- no network;
|
||||
- a read-only `/workspace`;
|
||||
- a non-root user;
|
||||
- the `[sandbox]` limits.
|
||||
|
||||
Before anything runs, a structural deny-list rejects snippets that mention sockets, curl, host
|
||||
networking or writes under `/workspace`.
|
||||
|
||||
## Evidence ladder: how a false positive is eliminated
|
||||
|
||||
Evidence level is a state machine, not a label a model may assign. The verifier
|
||||
(`verification.py`) runs on independent context and enforces:
|
||||
(`verification.py`) runs on independent context and enforces it:
|
||||
|
||||
```
|
||||
suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_explained
|
||||
@@ -75,31 +94,36 @@ suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_
|
||||
call-graph, route, CLI, parser or dynamic evidence;
|
||||
- `static_corroboration` with function-level reachability: `ACCEPTED`.
|
||||
|
||||
A rejected finding becomes a **scoped false-positive learning** (`calibration.py`) with a
|
||||
reason from a fixed taxonomy (`unreachable_path`, `missing_attacker_control`,
|
||||
A rejected finding becomes a **scoped false-positive learning** (`calibration.py`). It has a
|
||||
scope and a reason from a fixed taxonomy: `unreachable_path`, `missing_attacker_control`,
|
||||
`sanitizer_disproved`, `static_rule_mismatch`, `incorrect_model_assumption`, `duplicate`,
|
||||
`insufficient_impact`, `unsupported`, `contradicted`, `unverified`) and a scope. A rejection in
|
||||
`auth/` demotes only `auth/...`, never the whole class
|
||||
`insufficient_impact`, `unsupported`, `contradicted`, `unverified`. A rejection in `auth/`
|
||||
demotes only `auth/...`, never the whole class
|
||||
(`tests/test_calibration.py::test_scoped_false_positive_demotion_does_not_suppress_class_globally`).
|
||||
|
||||
## How a false negative is surfaced
|
||||
|
||||
Every expected vulnerability that no finding matched becomes a `BenchmarkCalibrationRecord` in
|
||||
`calibration_records.json`, with a `next_improvement_candidate` (a static rule, SARIF source,
|
||||
entry-point mapping, retrieval package, hunter prompt, dynamic reproducer or skill route).
|
||||
Once a miss is closed the detection gate keeps it closed in CI.
|
||||
An expected vulnerability that no finding matched becomes a `BenchmarkCalibrationRecord` in
|
||||
`calibration_records.json`. Each record names a `next_improvement_candidate`: a static rule,
|
||||
SARIF source, entry-point mapping, retrieval package, hunter prompt, dynamic reproducer or
|
||||
skill route. Once a miss is closed, the detection gate keeps it closed in CI.
|
||||
|
||||
## Central CWE policy and the project score
|
||||
|
||||
**Policy** (`policy/verycode.py`): a vendored `CWE_Score.tsv` (verycode-policies) is the single
|
||||
source of truth. Each CWE carries a flaw category, a severity (0-5) and `static`/`dynamic` scope
|
||||
flags. A finding's severity comes only from policy, keyed on CWE. CWEs that are not `static`
|
||||
resolve to 0 and are report-only. `assert_rules_resolve` runs as the `policy_check` stage: an
|
||||
enabled rule naming an ungoverned CWE aborts the scan. Verycode has no CWE-120, so the C/C++
|
||||
source of truth. Each CWE carries:
|
||||
|
||||
- a flaw category;
|
||||
- a severity (0-5);
|
||||
- `static`/`dynamic` scope flags.
|
||||
|
||||
A finding's severity comes only from policy, keyed on CWE. CWEs that are not `static` resolve
|
||||
to 0 and are report-only. `assert_rules_resolve` runs as the `policy_check` stage. An enabled
|
||||
rule that names an ungoverned CWE aborts the scan. Verycode has no CWE-120, so the C/C++
|
||||
buffer rules carry CWE-121.
|
||||
|
||||
**Score** (`scoring/project_score.py`): each finding's penalty is
|
||||
`SEV_WEIGHT[severity] x REACH_MULT[reachability]`, and the score is `100 * e^(-total/k)` with
|
||||
`SEV_WEIGHT[severity] x REACH_MULT[reachability]`. The score is `100 * e^(-total/k)` with
|
||||
`k = 60`.
|
||||
|
||||
| Severity | Weight | | Reachability | Multiplier |
|
||||
@@ -109,10 +133,13 @@ buffer rules carry CWE-121.
|
||||
| 3 | 10 | | `unknown` | 0.4 |
|
||||
| 2 / 1 / 0 | 2 / 1 / 0 | | | |
|
||||
|
||||
A confirmed false positive lowers a finding's reachability multiplier instead of deleting the
|
||||
rule. A severity-5 reachable finding always fails the score gate; `score < min_score`
|
||||
(default 80) fails it only with `[score] blocking = true`. The `score` stage writes
|
||||
`score.json` and merges it into `manifest.json`.
|
||||
A confirmed false positive lowers a finding's reachability multiplier. The rule is not
|
||||
deleted. The score gate works like this:
|
||||
|
||||
- a severity-5 reachable finding always fails it;
|
||||
- `score < min_score` (default 80) fails it only with `[score] blocking = true`.
|
||||
|
||||
The `score` stage writes `score.json` and merges it into `manifest.json`.
|
||||
|
||||
## Self-improvement
|
||||
|
||||
@@ -124,41 +151,61 @@ scan ─▶ verify ─▶ non-accepted findings ─▶ scoped learnings
|
||||
next scan ─▶ rank ─▶ calibrate (demote rejected scopes) ─▶ findings
|
||||
```
|
||||
|
||||
`record_calibration` (after `verify`) merges this run's non-accepted outcomes into the ledger,
|
||||
de-duplicated by finding ID; `calibrate` (after `rank`) demotes the priority of every scope
|
||||
that produced a rejection and writes `applied_calibrations.json`. Demotion is scoped and
|
||||
reversible (`tests/test_pipeline_calibration.py`).
|
||||
1. `record_calibration` runs after `verify`. It merges this run's non-accepted outcomes into
|
||||
the ledger, de-duplicated by finding ID.
|
||||
2. `calibrate` runs after `rank`. It demotes the priority of every scope that produced a
|
||||
rejection and writes `applied_calibrations.json`.
|
||||
|
||||
Demotion is scoped and reversible (`tests/test_pipeline_calibration.py`).
|
||||
|
||||
### Rule-level loop: `ousast improve`
|
||||
|
||||
Each round (`improve/evolve.py`): benchmark with the current ledger, per-rule signals
|
||||
(`rule_signals.json`), bounded proposals, validation (`improve/validator.py`), replay smoke,
|
||||
re-benchmark, and a hard gate. The levers are rule status (`enabled -> shadow` before
|
||||
`disabled`) and score constants; the loop can never edit pattern text
|
||||
or the authoritative CWE severity. A round is accepted only if recall >= 90%, FP < 10%, the
|
||||
project score did not regress and no matched finding was lost; otherwise the ledger is not
|
||||
written. With `--pair-catalog`, every round is also gated per provenance profile on the
|
||||
catalog's holdout pairs (`--no-pair-gate` skips this and is not for accepted ledgers). The same
|
||||
detection gate runs in CI (`python -m openultrasast.gate`).
|
||||
Each round (`improve/evolve.py`) runs these steps:
|
||||
|
||||
1. benchmark with the current ledger;
|
||||
2. per-rule signals (`rule_signals.json`);
|
||||
3. bounded proposals;
|
||||
4. validation (`improve/validator.py`);
|
||||
5. replay smoke;
|
||||
6. re-benchmark;
|
||||
7. a hard gate.
|
||||
|
||||
The levers are rule status (`enabled -> shadow` before `disabled`) and score constants. The
|
||||
loop can never edit pattern text or the authoritative CWE severity. A round is accepted only if
|
||||
all of these hold:
|
||||
|
||||
- recall >= 90%;
|
||||
- FP < 10%;
|
||||
- the project score did not regress;
|
||||
- no matched finding was lost.
|
||||
|
||||
Otherwise the ledger is not written. With `--pair-catalog`, each round is also gated per
|
||||
provenance profile on the catalog's holdout pairs. `--no-pair-gate` skips this check; do not
|
||||
use it for accepted ledgers. The same detection gate runs in CI (`python -m openultrasast.gate`).
|
||||
|
||||
### Proposals from plane memory
|
||||
|
||||
With `--memory [STORE]`, `ousast improve` also reads the rows plane runs stored per repository
|
||||
and pin (`improve/memory.py`). Two deterministic rules turn them into rule-status proposals: a
|
||||
rule whose alerts are repeatedly false across repositories is shadowed, and a shadow rule that
|
||||
keeps hitting agreed, declared vulnerable sites is re-enabled. Rows from holdout pairs, from the
|
||||
gated manifest's own cases and from any `--qualify-population` are dropped before any rule sees
|
||||
them. Each proposal records the rows it came from (`memory_proposals.jsonl` next to the
|
||||
journal) and goes through the same validator and gate.
|
||||
With `--memory [STORE]`, `ousast improve` also reads the rows that plane runs stored per
|
||||
repository and pin (`improve/memory.py`). Two deterministic rules turn these rows into
|
||||
rule-status proposals:
|
||||
|
||||
- a rule whose alerts are repeatedly false across repositories is shadowed;
|
||||
- a shadow rule that keeps hitting agreed, declared vulnerable sites is re-enabled.
|
||||
|
||||
Some rows are dropped before any rule sees them: rows from holdout pairs, from the gated
|
||||
manifest's own cases and from any `--qualify-population`. Each proposal records the rows it
|
||||
came from (`memory_proposals.jsonl` next to the journal). It then goes through the same
|
||||
validator and gate.
|
||||
|
||||
### The loop as a plane Run
|
||||
|
||||
`ousast plane workspaces <population> --validation-set ... --loop` appends the improvement loop
|
||||
to a generated Run: `alerts` per case (the quick scan on the vulnerable and fixed pins), then
|
||||
the steps `snapshot` (task `memory-snapshot`), `measure`, `propose` and `improve` (`plane/tasks/loop.py`), each model-free
|
||||
with a zero budget. The snapshot applies the same train-on-test guard before any row leaves the
|
||||
store. A recorded live run: `benchmarks/measurements/2026-09-30-harnessx-removal-loop/`.
|
||||
to a generated Run. The Run holds `alerts` per case: the quick scan on the vulnerable and fixed
|
||||
pins. Then come the steps `snapshot` (task `memory-snapshot`), `measure`, `propose` and
|
||||
`improve` (`plane/tasks/loop.py`). Each step is model-free with a zero budget. The snapshot
|
||||
applies the same train-on-test guard before any row leaves the store. A recorded live run:
|
||||
`benchmarks/measurements/2026-09-30-harnessx-removal-loop/`.
|
||||
|
||||
For PHP, which quick mode covered with zero rules when that run was made, `ousast plane
|
||||
When that run was made, quick mode covered PHP with zero rules. So for PHP, `ousast plane
|
||||
alerts-engine` produces a Run's `alerts` from the engine image on the host instead
|
||||
(`benchmarks/measurements/2026-09-30-php-engine-alerts/`).
|
||||
|
||||
+256
-67
@@ -1,51 +1,77 @@
|
||||
# The learned decision engine (in development)
|
||||
|
||||
**Status 2026-10-02: in development, not adopted.** No scan, pre-push check or report uses it;
|
||||
every user-facing path is today's deterministic one until a compiled program is adopted. The
|
||||
design is the `learned-decision-engine` specification (maintainer tooling); the code is in
|
||||
**Status 2026-10-02: in development, not adopted.** No scan, pre-push check or report uses it.
|
||||
Every user-facing path stays on today's deterministic one until a compiled program is adopted.
|
||||
The design is the `learned-decision-engine` specification (maintainer tooling). The code is in
|
||||
`src/openultrasast/learn/`.
|
||||
|
||||
## Why
|
||||
|
||||
Every detection gain before the engine was a hand edit tuned on the cases that exposed it, and
|
||||
engine tuning measured on population v1 did not transfer to v2. The reasoning, and the step-by-step
|
||||
chain that replaced hand edits, are on [Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
|
||||
Before the engine, every detection gain was a hand edit. Each edit was tuned on the cases that
|
||||
exposed the miss. Engine tuning measured on population v1 did not transfer to v2. [Where we
|
||||
stand](where-we-stand.md#2-our-approach-to-detection-step-by-step) gives the reasoning and the
|
||||
step-by-step chain that replaced hand edits.
|
||||
|
||||
## Design
|
||||
|
||||
- **Signals, not verdicts.** Every instrument (quick rules, the Joern engine, model sink and
|
||||
role classification, the plane's verify passes and agreement, repository facts such as
|
||||
callers) contributes a feature record per candidate `(path, function, family)`, including an
|
||||
explicit state when it did not run or failed. None decides alone.
|
||||
- **Signals, not verdicts.** Every instrument contributes a feature record per candidate
|
||||
`(path, function, family)`. The instruments are quick rules, the Joern engine, model sink and
|
||||
role classification, the plane's verify passes and agreement, and repository facts such as
|
||||
callers. The record includes an explicit state when an instrument did not run or failed. No
|
||||
instrument decides alone.
|
||||
- **Labels from ground truth.** `ousast learn labels` derives labels only from sources on a
|
||||
fail-closed allow-list (`learn/sources.toml`): the vulnerable and fixed sides of pairs, the
|
||||
declared sites and fix ranges of spent populations, benign deltas and recorded adjudications.
|
||||
fail-closed allow-list (`learn/sources.toml`). Fail-closed means a source not on the list is
|
||||
never used. The sources are:
|
||||
- the vulnerable and fixed sides of pairs;
|
||||
- the declared sites and fix ranges of spent populations;
|
||||
- benign deltas;
|
||||
- recorded adjudications.
|
||||
|
||||
"Rejected by the plane" is never a negative. Reserved populations (v3) are not on the list.
|
||||
- **An AI classifier with local memory, compiled in house in the DSPy style.** The program is an
|
||||
instruction, a few demonstrations and up to six retrieved labelled examples, then the
|
||||
candidate's code excerpt and signals; the model (`deepseek-flash`) answers
|
||||
vulnerable / not vulnerable / unsure with a rationale, sampled `k` times, giving a score `s`.
|
||||
Compilation bootstraps demonstrations and searches instructions on a compile split, scored by
|
||||
a pre-registered balanced Brier score (`learn/compile.toml`).
|
||||
- **Retrieval** from the memory store: nearest neighbours by Gower distance on the signal
|
||||
profile, re-ranked by cosine similarity of code embeddings (OpenAI `text-embedding-3-small`
|
||||
through OpenRouter, cached by excerpt hash), under a label/family/repository balance rule. An
|
||||
evaluation-boundary filter keeps a candidate's own repository group out of what it is shown.
|
||||
- **An AI classifier with local memory, compiled in house in the DSPy style.** The program has
|
||||
an instruction, a few demonstrations and up to six retrieved labelled examples. Then come the
|
||||
candidate's code excerpt and signals. The model (`deepseek-flash`) answers vulnerable / not
|
||||
vulnerable / unsure with a rationale. It is sampled `k` times, which gives a score `s`.
|
||||
Compilation bootstraps demonstrations and searches instructions on a compile split. A
|
||||
pre-registered balanced Brier score (`learn/compile.toml`) scores the candidates. The Brier score
|
||||
measures how far predicted probabilities are from the true labels.
|
||||
- **Retrieval** from the memory store works in three steps:
|
||||
1. find nearest neighbours by Gower distance (a distance over mixed feature types) on the
|
||||
signal profile;
|
||||
2. re-rank them by cosine similarity of code embeddings (OpenAI `text-embedding-3-small`
|
||||
through OpenRouter, cached by excerpt hash);
|
||||
3. apply a label/family/repository balance rule.
|
||||
|
||||
An evaluation-boundary filter keeps a candidate's own repository group out of what it is
|
||||
shown.
|
||||
- **Repository-grouped evaluation.** The compile split (25% of repository groups) is never
|
||||
evaluated. The rest is evaluated by outer grouped 5-fold cross-validation: the Platt
|
||||
calibration map is cross-fitted on the other folds and the operating points are chosen on the
|
||||
other folds (nested). One compile per profile, not one per fold. Intervals are 95%
|
||||
repository-cluster bootstrap.
|
||||
- **Two operating points.** BLOCK is offered only when calibration holds (slope CI contains 1,
|
||||
intercept CI contains 0, ECE <= 0.05) and a fold reaches the required precision; otherwise the
|
||||
engine is ADVISORY only. An `unsure` answer can never BLOCK.
|
||||
evaluated. The rest is evaluated by outer grouped 5-fold cross-validation:
|
||||
- the Platt calibration map (which turns the score into a probability) is cross-fitted on
|
||||
the other folds;
|
||||
- the operating points are chosen on the other folds (nested).
|
||||
|
||||
There is one compile per profile, not one per fold. Intervals are 95% repository-cluster
|
||||
bootstrap.
|
||||
- **Two operating points.** BLOCK is offered only when both of these hold:
|
||||
- calibration holds: slope CI contains 1, intercept CI contains 0, ECE <= 0.05;
|
||||
- a fold reaches the required precision.
|
||||
|
||||
Otherwise the engine is ADVISORY only. An `unsure` answer can never BLOCK. ECE (expected
|
||||
calibration error) is the average gap between predicted and observed rates.
|
||||
|
||||
## First measured slice (2026-10-01)
|
||||
|
||||
Record: `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json` (counts
|
||||
only). Family **injection** only; memory of 724 static examples from all families; input profile
|
||||
v1 with the engine signals and function length withheld (a leak audit found them separating the
|
||||
sides of a pair by construction: `2026-10-01-decision-engine-leak-audit/`); framework priors off.
|
||||
only). The setup:
|
||||
|
||||
- family **injection** only;
|
||||
- memory of 724 static examples from all families;
|
||||
- input profile v1, with the engine signals and function length withheld;
|
||||
- framework priors off.
|
||||
|
||||
The signals were withheld because a leak audit found them separating the sides of a pair by
|
||||
construction (`2026-10-01-decision-engine-leak-audit/`). AUC below is the chance the engine
|
||||
ranks a real bug above a fixed one.
|
||||
|
||||
| Measure | Value (95% repository-cluster CI) |
|
||||
| --- | --- |
|
||||
@@ -62,11 +88,17 @@ sides of a pair by construction: `2026-10-01-decision-engine-leak-audit/`); fram
|
||||
## Second slice: six families (2026-10-02)
|
||||
|
||||
Record: `benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json` (counts only).
|
||||
After the advisory-fix harvest the memory holds 904 static examples (724 before). One program was
|
||||
compiled per evaluable family (injection recompiled, since the memory changed) and evaluated on
|
||||
that family's own candidates, out of repository as above: compile split never evaluated, outer
|
||||
grouped 5-fold, calibration cross-fitted, operating points nested, 95% repository-cluster
|
||||
bootstrap intervals (2,000 resamples); input profile v1, priors off, k = 5.
|
||||
After the advisory-fix harvest, the memory holds 904 static examples (724 before).
|
||||
|
||||
One program was compiled per evaluable family. Injection was recompiled, since the memory
|
||||
changed. Each program was evaluated on its family's own candidates, out of repository as above:
|
||||
|
||||
- compile split never evaluated;
|
||||
- outer grouped 5-fold;
|
||||
- calibration cross-fitted;
|
||||
- operating points nested;
|
||||
- 95% repository-cluster bootstrap intervals (2,000 resamples);
|
||||
- input profile v1, priors off, k = 5.
|
||||
|
||||
| Family | Candidates (groups; positive) | Pooled AUC of `s` (95% CI) | Within-pair AUC (paired groups) | ADVISORY recall / precision | Canary agreement |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
@@ -77,45 +109,174 @@ bootstrap intervals (2,000 resamples); input profile v1, priors off, k = 5.
|
||||
| path | 35 (21; 21) | 0.89 (0.79 - 0.97) | 0.83 (14) | 0.86 / 0.69 | 0.90 |
|
||||
| untrusted destination | 62 (37; 38) | 0.85 (0.76 - 0.94) | 0.72 (18) | 0.84 / 0.73 | 0.87 |
|
||||
|
||||
- **BLOCK is offered for no family.** Calibration holds only for output encoding (ECE 0.045), and
|
||||
there no fold reaches the required precision (`precision_unreachable`: no fold reaches a Wilson
|
||||
lower bound of 0.95). For the other five families calibration does not hold (ECE 0.071 to 0.119).
|
||||
- **The pooled AUC is the more favourable number.** Every family's pool includes candidates that
|
||||
carry only a positive label, with no fixed-side counterpart (7 to 19 per family; there are no
|
||||
single-label negatives), so the pooled AUC is inflated by positives that never had to be told
|
||||
apart from their own fix. The within-pair AUC over the paired groups, 0.72 to 0.93, is the
|
||||
stricter reading (`paired_groups_check` in the record).
|
||||
- **BLOCK is offered for no family.** Calibration holds only for output encoding (ECE 0.045).
|
||||
There, no fold reaches the required precision (`precision_unreachable`: no fold reaches a
|
||||
Wilson lower bound of 0.95). For the other five families, calibration does not hold (ECE
|
||||
0.071 to 0.119).
|
||||
- **The pooled AUC is the more favourable number.** Every family's pool includes candidates
|
||||
that carry only a positive label, with no fixed-side counterpart (7 to 19 per family; there
|
||||
are no single-label negatives). These positives never had to be told apart from their own
|
||||
fix, so they inflate the pooled AUC. The within-pair AUC over the paired groups, 0.72 to
|
||||
0.93, is the stricter reading (`paired_groups_check` in the record).
|
||||
- **Re-run agreement (canary, 30 candidates per family) is below 0.9 for three families:**
|
||||
injection 0.77, access control 0.83, untrusted destination 0.87.
|
||||
- **One memory-size point for injection** (paired: the same program, candidates and folds, with a
|
||||
retrieval memory of 724 vs 904 examples): AUC 0.76 vs 0.77, within the intervals. No injection
|
||||
example was added, so the new examples reach an injection prompt only as contrast.
|
||||
- **Spend:** $2.88 metered at list price for DeepSeek under an $8 ceiling, $0.0009 for
|
||||
embeddings; $0.0027 to $0.0032 per evaluated candidate. The access-control evaluation replayed
|
||||
from the response cache at $0 with identical predictions.
|
||||
- **An instrument fix is recorded:** compiled demonstrations ignored the near-duplicate rule
|
||||
(cosine >= 0.98) until the pre-call boundary assertion stopped the untrusted-destination
|
||||
evaluation; demonstrations are now swapped or dropped like retrieved examples, and the stopped
|
||||
part was replayed after the fix.
|
||||
- **One memory-size point for injection.** This run is paired: the same program, candidates
|
||||
and folds, with a retrieval memory of 724 vs 904 examples. AUC was 0.76 vs 0.77, within the
|
||||
intervals. No injection example was added. So the new examples reach an injection prompt
|
||||
only as contrast.
|
||||
- **Spend:** $2.88 metered at list price for DeepSeek under an $8 ceiling, and $0.0009 for
|
||||
embeddings. That is $0.0027 to $0.0032 per evaluated candidate. The access-control
|
||||
evaluation replayed from the response cache at $0 with identical predictions.
|
||||
- **An instrument fix is recorded.** Compiled demonstrations ignored the near-duplicate rule
|
||||
(cosine >= 0.98). The pre-call boundary assertion caught this and stopped the
|
||||
untrusted-destination evaluation. Demonstrations are now swapped or dropped like retrieved
|
||||
examples. The stopped part was replayed after the fix.
|
||||
|
||||
**Not adopted.** What is still ahead is listed once, on
|
||||
[Where we stand](where-we-stand.md#5-open-items).
|
||||
**Not adopted.** [Where we stand](where-we-stand.md#5-open-items) lists, in one place, what is
|
||||
still ahead.
|
||||
|
||||
## Experiments
|
||||
|
||||
Changes to prompts, models, pass counts, features or sampling are compared by controlled
|
||||
experiments from a pre-registered manifest (`plane/experiments/<id>.yaml`), so nothing is adopted
|
||||
on one run's number. Two are registered. exp-001 (known callers) has not run. exp-002 (a retrieval
|
||||
ensemble against today's temperature sampling) ran on 160 paired candidates in 56 repository
|
||||
groups and was **rejected**: within-pair AUC 0.815 against 0.741 and $0.0021 against $0.0051 per
|
||||
candidate, with re-run agreement 0.875 against 0.906 and an interval spanning zero
|
||||
(`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`). The reading is on
|
||||
[Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
|
||||
experiments. Each experiment comes from a pre-registered manifest
|
||||
(`plane/experiments/<id>.yaml`). This way nothing is adopted on one run's number.
|
||||
|
||||
Four experiments are registered:
|
||||
|
||||
- exp-001 (known callers) has not run.
|
||||
- exp-002 compared a retrieval ensemble against today's temperature sampling. It ran on 160
|
||||
paired candidates in 56 repository groups and was **rejected**. Within-pair AUC was 0.815
|
||||
against 0.741 and the cost was $0.0021 against $0.0051 per candidate. The record shows re-run agreement
|
||||
0.875 against 0.906 and an interval spanning zero
|
||||
(`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`).
|
||||
- exp-003 compared two BLOCK rules on the same scores. It was **inconclusive**. Neither rule flags anything.
|
||||
- exp-004 tested one pooled BLOCK threshold with a per-family floor. It was **inconclusive**.
|
||||
|
||||
### exp-003: precision-bound blocking (2026-10-03)
|
||||
|
||||
The question: is the calibration gate what keeps BLOCK out of reach?
|
||||
Arm A is today's rule: a calibrated probability, offered only when calibration holds.
|
||||
Arm B picks a raw-score threshold per family, on training folds only.
|
||||
It takes the lowest threshold whose Wilson 95% lower bound on precision reaches 0.95.
|
||||
The held-out repository group is then scored with that threshold.
|
||||
Arm C relaxes the bound to point precision 0.95 over at least 10 flags.
|
||||
Arm C was registered as secondary and does not decide.
|
||||
|
||||
- **Registered first.** The manifest and units were committed and registered before the run.
|
||||
- **Primary metric:** arm B's held-out Wilson lower bound on flagged precision, per family.
|
||||
- **Primary population:** paired units only. Positives-only groups are excluded; pooled figures are secondary.
|
||||
- **Zero cost.** Scores came only from the response cache, behind a zero-ceiling meter.
|
||||
The meter recorded 0 client calls and $0.0.
|
||||
- **Exclusions.** 142 of 353 units had an uncached response and were excluded, not asked.
|
||||
Path lost all 39 units. Injection and access control lost none.
|
||||
- **Evaluated:** 211 units in 89 repository groups, 138 of them paired.
|
||||
|
||||
| Family | Evaluated (excluded) | Paired positives | Flagged A / B / C | Coverage A / B / C |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| access control | 41 (0) | 16 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
|
||||
| deserialization | 14 (34) | 2 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
|
||||
| injection | 119 (0) | 38 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
|
||||
| output encoding | 24 (21) | 8 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
|
||||
| path | 0 (39) | 0 | 0 / 0 / 0 | none |
|
||||
| untrusted destination | 13 (48) | 4 | 0 / 0 / 0 | 0.0 / 0.0 / 0.0 |
|
||||
|
||||
No arm chose a threshold on any fold, so flagged precision is undefined everywhere.
|
||||
Arm A failed calibration where it was checked: ECE 0.077 (access control), 0.1067 (injection).
|
||||
The other families had too few units for a calibration check.
|
||||
|
||||
**Why B flags nothing.** A Wilson lower bound of 0.95 needs 73 flags without one error.
|
||||
Injection, the largest family, holds 38 paired positives across all five folds.
|
||||
The registration predicted this zero before the run.
|
||||
On this data the bound, not the calibration gate, makes BLOCK unreachable.
|
||||
|
||||
**The zero is the rule's, not the instrument's.** The scores are real: 48 distinct values for injection.
|
||||
In access control, the 10 highest-scored paired units are all positive.
|
||||
In injection, a negative appears at rank 4 of the paired ranking.
|
||||
Pooled rankings look cleaner because positives-only units fill the top.
|
||||
Injection answered `unsure` for 42 of 119 units, and `unsure` never blocks.
|
||||
|
||||
Record: `benchmarks/experiments/exp-003-precision-bound-blocking/result.json`.
|
||||
|
||||
**Supplementary: all 353 units (2026-10-03).** This is not a new decision; exp-003's verdict stands.
|
||||
The 142 excluded units had their responses filled with the same programs and request keys.
|
||||
The fill cost 697 client calls and $0.331759 under a $1.00 ceiling.
|
||||
The replay then decided all 353 units at zero cost: 244 paired, 146 repository groups.
|
||||
|
||||
| Family | Paired units (positives) | Flagged A / B / C | ECE (arm A) |
|
||||
| --- | --- | --- | --- |
|
||||
| access control | 32 (16) | 0 / 0 / 0 | 0.077 |
|
||||
| deserialization | 34 (17) | 0 / 0 / 0 | 0.0596 |
|
||||
| injection | 76 (38) | 0 / 0 / 0 | 0.1067 |
|
||||
| output encoding | 32 (16) | 0 / 0 / 0 | 0.0761 |
|
||||
| path | 32 (16) | 0 / 0 / 0 | 0.062 |
|
||||
| untrusted destination | 38 (19) | 0 / 0 / 0 | 0.0454 |
|
||||
|
||||
Still no arm flags any unit, so the registered rule reads inconclusive.
|
||||
Calibration holds only for untrusted destination.
|
||||
There arm A still finds no BLOCK point that reaches the bound.
|
||||
Record: `benchmarks/experiments/exp-003-precision-bound-blocking/result-complete-cache.json`.
|
||||
|
||||
### exp-004: pooled block gate (2026-10-03)
|
||||
|
||||
The experiment tested one pooled BLOCK threshold across families.
|
||||
Each family needed at least 10 clean flags to block.
|
||||
Each family also had its own quiet threshold.
|
||||
The run evaluated 501 units across 220 repository groups.
|
||||
Of these, 390 units were paired.
|
||||
The cache fill cost $1.238223 for 2,073 calls.
|
||||
The replay cost $0.0 and made 0 client calls.
|
||||
|
||||
The verdict was **inconclusive**.
|
||||
No fold found a `t_block` threshold.
|
||||
The instrument check confirmed real scores.
|
||||
The pooled top 30 contained 27 positives.
|
||||
The first negative appeared at rank 12.
|
||||
The bound needs 73 clean flags without an error.
|
||||
|
||||
A within-pair win means the vulnerable side scored above its fixed counterpart.
|
||||
|
||||
| Family | Within-pair wins | Pairs |
|
||||
| --- | --- | --- |
|
||||
| access control | 22 | 28 |
|
||||
| deserialization | 18 | 25 |
|
||||
| injection | 30 | 53 |
|
||||
| output encoding | 18 | 28 |
|
||||
| path | 25 | 29 |
|
||||
| untrusted destination | 18 | 32 |
|
||||
|
||||
Precision at the top of the ranking now limits blocking.
|
||||
Data volume alone cannot resolve this.
|
||||
The next levers are graph evidence in the prompt and the combiner.
|
||||
|
||||
Records: `benchmarks/experiments/exp-004-pooled-block-gate/record.json` and `benchmarks/experiments/exp-004-pooled-block-gate/result.json`.
|
||||
|
||||
### exp-004: pooled block gate with a family floor (registered, not run)
|
||||
|
||||
This registration describes the setup before the run reported above.
|
||||
|
||||
The maintainer amended the BLOCK rule on 2026-10-03 (requirements, Req 6.4).
|
||||
Precision is now shown across all families together, not per family.
|
||||
Each family's score is first mapped by its own Platt fit, on training folds.
|
||||
One threshold, t_block, then serves every family.
|
||||
It is the lowest one whose pooled Wilson lower bound reaches 0.95 on training folds.
|
||||
A family blocks only with at least 10 held-out flags and no error.
|
||||
Otherwise its flags stay advisory.
|
||||
Below a per-family t_quiet, which keeps 90% of positives, findings are silent.
|
||||
No held-out label moves a threshold.
|
||||
|
||||
- **Primary metric:** the pooled held-out Wilson lower bound of flagged precision, paired units, before the floor.
|
||||
- **Decision:** reject below point precision 0.95; adopt if the bound reaches 0.95 and a family passes the floor.
|
||||
- **Units:** frozen from the memory after the negatives harvest.
|
||||
- **Cost:** the run replayed the cache after a paid cache fill.
|
||||
|
||||
Manifest: `plane/experiments/exp-004-pooled-block-gate.yaml`.
|
||||
|
||||
The reading is on [Where we stand](where-we-stand.md#2-our-approach-to-detection-step-by-step).
|
||||
|
||||
## Commands
|
||||
|
||||
`ousast learn` is a maintainer command group over the engine's data. Paid steps take
|
||||
`--budget-usd`; `--replay-only` answers every call from the response cache at no cost.
|
||||
`--budget-usd`. `--replay-only` answers every call from the response cache at no cost.
|
||||
|
||||
| Command | Does |
|
||||
| --- | --- |
|
||||
@@ -127,8 +288,36 @@ candidate, with re-run agreement 0.875 against 0.906 and an interval spanning ze
|
||||
| `ousast learn curve --program ID` | the metric over memory size on a fixed candidate subset |
|
||||
| `ousast learn audit-leaks` | how well each signal alone separates the sides of a pair (no model) |
|
||||
| `ousast learn experiment register\|units\|run\|analyse MANIFEST` | A/B experiments from a pre-registered manifest (`plane/experiments/<id>.yaml`): record its digest, freeze the units (no model), run the arms paired (arm A may replay from the cache at $0), analyse (repository bootstrap, exact McNemar, two looks, the adoption verdict) |
|
||||
| `ousast learn experiment fill MANIFEST --budget-usd X` | Fill missing responses for a registered `kind: rule` experiment's frozen units before its zero-cost run; `--dry-run` inspects and estimates without calls or writes |
|
||||
|
||||
`experiment fill` accepts `--memory URL`, `--profile static|plane` (default `static`),
|
||||
`--manifest-model PATH` (default `plane/models/deepseek-flash.yaml`) and `--out PATH`.
|
||||
Freeze, commit and register the units first. Fill uses the incumbent programs and the
|
||||
same retrieval, sampling and cache keys as replay. The first missing request is the
|
||||
smoke call; a successful smoke response is kept. Any provider failure stops the fill.
|
||||
|
||||
The JSON report on stdout gives each family's units, distinct requests, initially
|
||||
cached and missing responses, newly filled responses and still-missing responses,
|
||||
plus total client calls and metered USD. `--out` also saves it, except during dry-run,
|
||||
which writes nothing. Exit codes are 0 for completion or dry-run, 1 for a provider/cache
|
||||
failure, 2 for invalid setup and 3 for budget exhaustion. Resume a partial fill with a
|
||||
new invocation and budget; that budget covers only the new invocation, including smoke.
|
||||
|
||||
Dry-run uses each family's mean billed cached response cost when recorded usage exists.
|
||||
Model prices account for cache-hit input, cache-miss input, and output tokens.
|
||||
Without cached usage, it uses the program's input-token estimate and **300 output tokens**.
|
||||
Each family's `estimate_method` reports `measured` or `estimated`.
|
||||
It reports potential retries separately as `conditional_retries`: an unseen answer
|
||||
may need one parse retry, whose necessity cannot be known without calling the model.
|
||||
Known invalid cached answers include their retry requests in the inventory.
|
||||
|
||||
Fill reserves at least **$0.01** and the largest observed call cost before each call.
|
||||
For the production client it also reserves a conservative input-byte token bound plus
|
||||
the program's enforced output-token limit; hidden provider retries are disabled.
|
||||
Consequently a small budget may stop before making any call, and a fill may finish
|
||||
with unused budget. Calls without token usage stop as failures, never as free calls.
|
||||
|
||||
The memory store is the plane's (`OUSAST_MEMORY`; see [ops/ax/README.md](ops/ax/README.md)).
|
||||
The harvests that produced the plane signals ran as plane Runs generated by
|
||||
`ousast plane harvest` (`benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`
|
||||
and, for the advisory-fix pairs, `2026-10-02-decision-engine-harvest-advisory/record.json`).
|
||||
`ousast plane harvest`. Records: `benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`
|
||||
and, for the advisory-fix pairs, `2026-10-02-decision-engine-harvest-advisory/record.json`.
|
||||
|
||||
+387
-243
@@ -1,17 +1,21 @@
|
||||
# Deployment
|
||||
|
||||
How the agentic plane is deployed today, and what moving it to a separate Kubernetes cluster
|
||||
would take. State as of 2026-10-02 (v2.0.0). A scan needs none of this: `ousast scan` and
|
||||
`ousast pre-push` run from the Python package (and the Docker image for Joern, see
|
||||
[Engine and pre-push](ops/README.md)). The plane is the maintainers' surface for model work over
|
||||
whole repositories, driven by `ousast plane`.
|
||||
This page covers two things. First, how the agentic plane is deployed today. Second, what moving it to a
|
||||
separate Kubernetes cluster would take. It describes the state as of 2026-10-02 (v2.0.0).
|
||||
|
||||
Two words are used strictly on this page. **Implemented** means the code on `main` does it and it
|
||||
has run on the maintainer's host. **Planned** means it has not been done; the guidance is derived
|
||||
from the code and the host's lessons, and the code changes it needs are named with their file paths.
|
||||
A scan needs none of this. `ousast scan` and `ousast pre-push` run from the Python package. Joern runs
|
||||
from the Docker image (see [Engine and pre-push](ops/README.md)). The plane is the maintainers' tool for
|
||||
model work over whole repositories. `ousast plane` drives it.
|
||||
|
||||
This page uses two words strictly:
|
||||
|
||||
- **Implemented** means the code on `main` does it, and it has run on the maintainer's host.
|
||||
- **Planned** means it has not been done. The guidance comes from the code and from what the host taught
|
||||
us. Each code change it needs is named with its file path.
|
||||
|
||||
**No deployment of the plane to a cluster other than the maintainer's single-node kind cluster has
|
||||
been made.** Section B is guidance, not a record.
|
||||
been made.** Section B is guidance, not a record. (kind runs a Kubernetes cluster inside Docker
|
||||
containers on one machine.)
|
||||
|
||||
## What runs where today (implemented)
|
||||
|
||||
@@ -27,11 +31,11 @@ flowchart LR
|
||||
sub["Agent Substrate (ate-system): atenet-router, atenet-egress"]
|
||||
act["Task actors in gVisor workers (WorkerPool ousast-pool)"]
|
||||
svc["Service ousast-receiver + EndpointSlice to the host"]
|
||||
reg["Registry localhost:5001: runner image by digest"]
|
||||
reg["Registry named by the profile: runner image by digest"]
|
||||
end
|
||||
mem[("Memory store: file store on the host, or an S3 bucket (RustFS tested)")]
|
||||
rec -- "ax apply, resume, get, delete" --> ax
|
||||
rec -- "EgressPolicy per actor; start request via port-forward" --> sub
|
||||
rec -- "EgressPolicy per actor; start request through the kube-context tunnel" --> sub
|
||||
sub --> act
|
||||
act -- "HTTP by Service name through the gateway" --> svc
|
||||
svc --> rcv
|
||||
@@ -46,19 +50,26 @@ flowchart LR
|
||||
| How actors reach the receiver | the Service `ousast-receiver.ax-system` (port 80) with a selector-less EndpointSlice whose endpoint is the kind network's gateway address on the host; the runner dials the ClusterIP with the Service name as `Host`, which the egress gateway allows by hostname | `ops/ax/receiver-service.yaml.tmpl`, rendered by `up.sh` |
|
||||
| ax control plane, Agent Substrate, egress gateway, router | the kind cluster (`ax-system`, `ate-system`) | installed by `ops/ax/up.sh` from the upstream checkouts under `~/.cache/ousast/ax-src/` |
|
||||
| Tasks | one actor per Task in a gVisor worker; two warm workers of 1 CPU / 1.5 GiB | `ops/ax/workerpool.yaml.tmpl` |
|
||||
| Runner image | the kind-local registry `localhost:5001`, pinned by digest (`~/.cache/ousast/ax-src/runner-image`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
|
||||
| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through a `kubectl port-forward` to `svc/atenet-router` | `src/openultrasast/plane/router.py` |
|
||||
| Runner image | the registry the profile names (`registry` in `ops/k8s/profiles/kind.toml`: the kind-local one), pinned by digest in the profile's images file (`ops/k8s/profiles/kind-images.json`) | `plane/Dockerfile.runner`; the `image` of every template under `plane/tasks/` |
|
||||
| Provider keys | read from the operator's environment or `.env` (which never overrides an exported variable), sent only in the start request through the tunnel `router.py` opens to `svc/atenet-router` in the profile's kube context | `src/openultrasast/plane/router.py` |
|
||||
| Memory store | `OUSAST_MEMORY`: a file store under `~/ousast-results/plane/memory` by default, or `s3://<bucket>[/<prefix>]` on any reachable S3-compatible server (RustFS is the tested one) | `src/openultrasast/plane/memory.py`, [Memory](memory.md), [RustFS setup](rustfs.md) |
|
||||
| Run state and artifacts | `$OUSAST_RESULTS/plane/<run>/` on the host (default `~/ousast-results/plane/`) | `reconciler.py` (`run_dir`) |
|
||||
|
||||
Only the reconciler talks to the memory store: a sandboxed task cannot read it, so the loop's
|
||||
`memory-snapshot` is seeded by the reconciler before the Run ([The plane on ax](plane.md)).
|
||||
Terms in the table:
|
||||
|
||||
- **ax** is Google's controller that runs agent tasks on Kubernetes.
|
||||
- **Agent Substrate** is the layer below ax that owns the machines and runs sandboxes.
|
||||
- **gVisor** is a sandbox runtime that isolates each actor from the node's kernel.
|
||||
- An **actor** is one running Task inside a sandbox.
|
||||
|
||||
Only the reconciler talks to the memory store. A sandboxed task cannot read it. So the reconciler seeds
|
||||
the loop's `memory-snapshot` before the Run starts ([The plane on ax](plane.md)).
|
||||
|
||||
## A. This host: the local setup (implemented)
|
||||
|
||||
What `ops/ax/up.sh` needs on the PATH: `docker`, `go`, `kubectl`, `kind`, `ko` and `ax` (it
|
||||
refuses to start when one is missing). Tools live in `~/go/bin` and `~/.local/bin`; sources and
|
||||
rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
|
||||
`ops/ax/up.sh` needs these tools on the PATH: `docker`, `go`, `kubectl`, `kind`, `ko` and `ax`. It
|
||||
refuses to start when one is missing. Tools live in `~/go/bin` and `~/.local/bin`. Sources and rendered
|
||||
files live in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
|
||||
|
||||
1. **Bring up the cluster and the plane.**
|
||||
|
||||
@@ -66,25 +77,39 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
|
||||
ops/ax/up.sh
|
||||
```
|
||||
|
||||
Idempotent: each step is skipped when already done. It creates the kind cluster
|
||||
(`KIND_CLUSTER_NAME`, default `ousast`) with its registry (`KO_DOCKER_REPO`, default
|
||||
`localhost:5001`), installs Agent Substrate, applies the egress gateway (the agentgateway
|
||||
variant, one prebuilt image) when the install left it out, deploys the ax control plane, applies
|
||||
the receiver Service with the host's kind-gateway address as its endpoint, applies the gVisor
|
||||
worker pool, builds and pushes the runner image and writes its digest pin to
|
||||
`~/.cache/ousast/ax-src/runner-image`, installs the `ax` CLI from the deployed checkout (a release
|
||||
CLI skews from the server), and runs one smoke Task. It ends by printing the host's memory and the
|
||||
kind containers' usage.
|
||||
The script is idempotent: it skips each step that is already done. In order, it:
|
||||
|
||||
- creates the kind cluster (`KIND_CLUSTER_NAME`, default `ousast`) with its registry
|
||||
(`KO_DOCKER_REPO`, default: the `registry` of `ops/k8s/profiles/kind.toml`);
|
||||
- installs Agent Substrate;
|
||||
- applies the egress gateway if the install left it out (the agentgateway variant, one prebuilt image);
|
||||
- deploys the ax control plane;
|
||||
- applies the receiver Service, with the host's kind-gateway address as its endpoint;
|
||||
- applies the gVisor worker pool;
|
||||
- builds and pushes the runner image, and writes its digest pin to `~/.cache/ousast/ax-src/runner-image`;
|
||||
- installs the `ax` CLI from the deployed checkout (a release CLI does not match the server version);
|
||||
- runs one smoke Task.
|
||||
|
||||
At the end it prints the host's memory and the kind containers' usage.
|
||||
|
||||
2. **Check it.**
|
||||
|
||||
```bash
|
||||
uv run ousast plane doctor
|
||||
uv run ousast plane doctor --profile kind
|
||||
```
|
||||
|
||||
Four checks: the kind context is reachable, every pod in `ate-system` and in `ax-system` is ready,
|
||||
and the registry at `localhost:5001` holds `ousast-runner`. Each failing line names
|
||||
`ops/ax/up.sh` as the fix. `doctor` is informational: `ousast plane run` does not call it.
|
||||
It checks the profile you pass (`--profile`, default `kind`; the profiles are in `ops/k8s/profiles/`):
|
||||
|
||||
- its configured addresses, printed on the first line;
|
||||
- the kube context is reachable;
|
||||
- every pod in `ate-system` is ready;
|
||||
- every pod in `ax-system` is ready;
|
||||
- the runner image of the profile's images file is in its registry (a local registry over its v2
|
||||
API, a remote one through `docker manifest inspect`);
|
||||
- the memory store opens (an S3 bucket is verified).
|
||||
|
||||
Each failing line names the fix (`ops/ax/up.sh` for the kind profile). `doctor` is informational:
|
||||
`ousast plane run` does not call it.
|
||||
|
||||
3. **Run the end-to-end smoke Run.**
|
||||
|
||||
@@ -92,23 +117,27 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
|
||||
ops/ax/smoke-run.sh
|
||||
```
|
||||
|
||||
It renders `ops/ax/smoke-run.yaml.tmpl` with the current runner digest and submits the Run
|
||||
`ax-e2e-smoke` through the reconciler: one model-free `repo-facts` Task (`budget: {usd: 0,
|
||||
calls: 0}`) over a files-only Workspace, resumed, started through the router, its `facts.json` and
|
||||
`summary.json` delivered to the host, then `ousast plane status ax-e2e-smoke` prints the attribution
|
||||
table. The script calls the repository's `.venv/bin/ousast`, so `uv sync` first.
|
||||
It renders `ops/ax/smoke-run.yaml.tmpl` with the current runner digest. It then submits the Run
|
||||
`ax-e2e-smoke` through the reconciler. The Run has one model-free `repo-facts` Task (`budget: {usd: 0,
|
||||
calls: 0}`) over a files-only Workspace, resumed, started through the router, its `facts.json`
|
||||
and `summary.json` delivered to the host. Then `ousast plane status ax-e2e-smoke`
|
||||
prints the attribution table. The script calls the repository's `.venv/bin/ousast`, so run `uv sync`
|
||||
first.
|
||||
|
||||
4. **Keys and the store.** Put `DEEPSEEK_API_KEY` (chat) and `OPENROUTER_API_KEY` (embeddings) in
|
||||
`.env` or export them; a Task's Model names the variable (`secretKey.key`), and the key travels
|
||||
only in that Task's start request. Set `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT`,
|
||||
`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` (optional `S3_REGION`, `S3_BUCKET`) for the S3 store;
|
||||
the bucket must be set up once by an admin as [RustFS setup](rustfs.md) describes, and the store
|
||||
verifies it at every start.
|
||||
4. **Keys and the store.**
|
||||
|
||||
- Put `DEEPSEEK_API_KEY` (chat) and `OPENROUTER_API_KEY` (embeddings) in `.env`, or export them.
|
||||
- A Task's Model names the variable (`secretKey.key`). The key travels only in that Task's start
|
||||
request.
|
||||
- For the S3 store, set `OUSAST_MEMORY=s3://<bucket>` with `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID` and
|
||||
`AWS_SECRET_ACCESS_KEY`. `S3_REGION` and `S3_BUCKET` are optional.
|
||||
- An admin must set up the bucket once, as [RustFS setup](rustfs.md) describes. The store verifies
|
||||
it at every start.
|
||||
|
||||
5. **Run a real Run**, as in [Examples](examples.md#7-running-the-agentic-plane): `ousast plane run
|
||||
plane/runs/validation-46.yaml`, then `status` and `remember`.
|
||||
|
||||
6. **Tear down** with `ops/ax/down.sh`, which deletes the cluster and its registry.
|
||||
6. **Tear down** with `ops/ax/down.sh`. It deletes the cluster and its registry.
|
||||
|
||||
**Footprint** (measured 2026-09-29, idle, two workers; [ax on this host](ops/ax/README.md#measured-footprint-2026-09-29-idle-two-workers)):
|
||||
|
||||
@@ -119,44 +148,70 @@ rendered files in `~/.cache/ousast/ax-src/` (`OUSAST_AX_SRC`).
|
||||
| node volume (images, containerd) | 4.9 GB on disk |
|
||||
| worker pool | 2 gVisor workers, 1 CPU / 1.5 GiB limit each |
|
||||
|
||||
Disk is the binding constraint: the Substrate install and the `ko` builds need several GB of Go
|
||||
caches (`go clean -cache -modcache` afterwards), and every Task is its own actor template with a
|
||||
golden actor (about 24 MB each under `/var/lib/ate/actors` on the node). The failures the host
|
||||
produced, and what each one taught, are listed in
|
||||
[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run).
|
||||
Disk is the tightest limit, for two reasons:
|
||||
|
||||
- The Substrate install and the `ko` builds need several GB of Go caches. Run `go clean -cache -modcache`
|
||||
afterwards.
|
||||
- Every Task is its own actor template with a golden actor (a saved snapshot the actor boots from). Each
|
||||
takes about 24 MB under `/var/lib/ate/actors` on the node.
|
||||
|
||||
[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run) lists the
|
||||
failures the host produced and what each one taught.
|
||||
|
||||
## B. A separate Kubernetes cluster (planned)
|
||||
|
||||
The target: a cluster that runs Agent Substrate with its egress gateway and the ax control plane,
|
||||
pulls the runner image from a registry of yours, receives task artifacts without a developer
|
||||
laptop in the path, and keeps its memory in your S3 bucket. The provider key already travels only
|
||||
in the start request through `atenet-router`, which works the same through a port-forward to any
|
||||
cluster. Everything else below is either a step you take or a code change that is not made yet.
|
||||
The target cluster would do four things:
|
||||
|
||||
- run Agent Substrate with its egress gateway and the ax control plane;
|
||||
- pull the runner image from a registry of yours;
|
||||
- receive task artifacts without a developer laptop in the path;
|
||||
- keep its memory in your S3 bucket.
|
||||
|
||||
The provider key already travels only in the start request through `atenet-router`. That works the same
|
||||
through the kube-context tunnel to any cluster. Everything else below is either a step you take or a code
|
||||
change that is not made yet.
|
||||
|
||||
### B.1 Cluster with Agent Substrate, its egress gateway and ax
|
||||
|
||||
- Install Agent Substrate on the cluster following its own instructions. `ops/ax/up.sh` uses
|
||||
Substrate's kind helpers (`hack/create-kind-cluster.sh`, `hack/install-ate-kind.sh`), which do
|
||||
not apply to another cluster.
|
||||
- Install Agent Substrate on the cluster with its own instructions. `ops/ax/up.sh` uses Substrate's kind
|
||||
helpers (`hack/create-kind-cluster.sh`, `hack/install-ate-kind.sh`). These do not apply to another
|
||||
cluster.
|
||||
- Apply the egress gateway. On this host it is the agentgateway variant
|
||||
(`manifests/ate-install/agentgateway-egress` in the Substrate checkout, applied with
|
||||
`kubectl kustomize --load-restrictor=LoadRestrictionsNone`), which needs no Rust build; the
|
||||
Envoy variant does. Without the gateway an actor has no network at all, and the per-task
|
||||
EgressPolicies the reconciler writes (`src/openultrasast/plane/egress.py`) work unchanged
|
||||
against the Substrate-installed gateway: deny by default, hostnames only, plain HTTP to the
|
||||
receiver, TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts.
|
||||
(`manifests/ate-install/agentgateway-egress` in the Substrate checkout). It is applied with
|
||||
`kubectl kustomize --load-restrictor=LoadRestrictionsNone`. It needs no Rust build; the Envoy variant
|
||||
does.
|
||||
- Without the gateway an actor has no network at all.
|
||||
- The reconciler writes one EgressPolicy per task (`src/openultrasast/plane/egress.py`). These work
|
||||
unchanged against the gateway that Substrate installs. The rules are:
|
||||
- deny by default;
|
||||
- hostnames only;
|
||||
- plain HTTP to the receiver;
|
||||
- TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts.
|
||||
- Deploy ax (`make deploy AX_IMAGE_REPO=<your registry>` in the ax checkout) **after** B.2.
|
||||
- Apply a WorkerPool (B.6). Substrate installs none by itself ("no free workers").
|
||||
- **gVisor on the nodes (planned, checked by hand on the day k3s exists).** Substrate does not use a Kubernetes
|
||||
`RuntimeClass`: the `ateom-gvisor` worker runs `runsc` itself, from the gVisor release tarball named in
|
||||
`manifests/ate-install/sandboxconfig-gvisor.yaml` of the Substrate checkout
|
||||
(`gs://gvisor/releases/nightly/2026-09-02/<arch>/gvisor.tar.zstd`, sha256-pinned per architecture), which
|
||||
the atelet fetches on the worker node. The node must reach that URL, or a copied `SandboxConfig` must name a
|
||||
mirrored copy with the same sha256 and the ActorTemplates must name that config. The check is the atelet's
|
||||
fetch line in `kubectl -n ate-system logs` of the worker pod; it is labelled planned until that log has been
|
||||
read on a k3s node (plane-on-kubernetes task 1.7, design open question 3).
|
||||
|
||||
### B.2 ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET`
|
||||
|
||||
ax's deploy manifest (`deploy/ax-server.yaml` in the ax checkout) sets `AX_SNAPSHOTS_BUCKET` to
|
||||
the ax authors' own bucket (`gs://dberkov-gke-dev3/ate-env/` in the checkout used here). Point it
|
||||
at **your own bucket before deploying ax**. On this host the bucket named there did not exist in
|
||||
the S3 server, deleted golden actors piled up in `DELETING` and filled the disk during a 600-task
|
||||
harvest (2026-10-01); creating the bucket let deletes complete, and even then a deleted actor's
|
||||
directory stays on the node, so long Runs need a janitor that removes directories no live actor
|
||||
owns.
|
||||
Point `AX_SNAPSHOTS_BUCKET` at **your own bucket before deploying ax**. ax's deploy manifest
|
||||
(`deploy/ax-server.yaml` in the ax checkout) sets it to the ax authors' own bucket. In the checkout used
|
||||
here that is `gs://dberkov-gke-dev3/ate-env/`.
|
||||
|
||||
This host shows what goes wrong otherwise (2026-10-01):
|
||||
|
||||
1. The bucket named there did not exist in the S3 server.
|
||||
2. Deleted golden actors piled up in `DELETING`.
|
||||
3. They filled the disk during a 600-task harvest.
|
||||
|
||||
Creating the bucket let deletes complete. Even then, a deleted actor's directory stays on the node. So
|
||||
long Runs need a janitor that removes directories no live actor owns.
|
||||
|
||||
### B.3 A registry the workers can pull from, the runner image pinned by digest
|
||||
|
||||
@@ -168,83 +223,109 @@ owns.
|
||||
docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' <registry>/ousast-runner:dev
|
||||
```
|
||||
|
||||
- Substrate rejects tags: every Task's `image` must be the `<registry>/ousast-runner@sha256:...`
|
||||
reference. Write it into a file (on this host, `~/.cache/ousast/ax-src/runner-image`) and pass
|
||||
`--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`, which re-pin the
|
||||
Tasks they generate. The committed templates under `plane/tasks/` and the generated
|
||||
`plane/tasks/validation-46.yaml` carry `localhost:5001/ousast-runner@sha256:...` and need the same
|
||||
re-pin (regenerate, or replace the reference) before they run elsewhere.
|
||||
- Workers must be able to pull from the registry (credentials, if any, are the cluster's concern;
|
||||
ax's default runner image on `gcr.io` needs credentials and is not used here).
|
||||
- **Not yet configurable:** `ousast plane doctor` checks the registry at `localhost:5001` by name
|
||||
(`src/openultrasast/plane/doctor.py:45`), so its fourth check fails on another cluster even when
|
||||
the image is in place. `doctor` is informational, `plane run` does not depend on it.
|
||||
- Substrate rejects tags. Every Task's `image` must be the `<registry>/ousast-runner@sha256:...`
|
||||
reference.
|
||||
- Write that reference into a file. On this host the file is `~/.cache/ousast/ax-src/runner-image`.
|
||||
- Pass `--runner-image FILE` to `ousast plane workspaces` and `ousast plane harvest`. They re-pin the
|
||||
Tasks they generate.
|
||||
- The committed templates under `plane/tasks/` and the generated `plane/tasks/validation-46.yaml` carry
|
||||
the kind profile's registry and digest. They need the same re-pin before they run elsewhere.
|
||||
Regenerate them, or replace the reference.
|
||||
- Workers must be able to pull from the registry. Credentials, if any, are the cluster's concern. ax's
|
||||
default runner image on `gcr.io` needs credentials and is not used here.
|
||||
- `ousast plane doctor --profile <name>` looks up the runner reference of the profile's images file in
|
||||
the profile's `registry` (implemented 2026-10-02, plane-on-kubernetes task 1.2). `doctor` is
|
||||
informational; `plane run` does not depend on it.
|
||||
|
||||
### B.4 One configurable kube context
|
||||
### B.4 One configurable kube context (implemented 2026-10-02)
|
||||
|
||||
The reconciler, the egress client and the router port-forward all use the context
|
||||
`kind-$KIND_CLUSTER_NAME` (`kind-ousast` by default). **Not yet configurable**: the prefix `kind-`
|
||||
is hard-coded in `src/openultrasast/plane/doctor.py:39`, `src/openultrasast/plane/egress.py:115`
|
||||
and `:148`, and `src/openultrasast/plane/router.py:145`; `ops/ax/up.sh:24` builds the same name.
|
||||
The planned change is one variable (for example `OUSAST_KUBE_CONTEXT`) read in those three
|
||||
modules. Until then the only way to drive another cluster from the unchanged code is to give its
|
||||
kubeconfig context a name of that form (`kubectl config rename-context <ctx> kind-<name>` and
|
||||
`KIND_CLUSTER_NAME=<name>`), which is a workaround, not a supported configuration. The egress
|
||||
client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names) able to use that
|
||||
context.
|
||||
The reconciler, the egress client and the router tunnel take the kube context from the profile.
|
||||
|
||||
- The setting is `kube_context` in `ops/k8s/profiles/<name>.toml`; `OUSAST_KUBE_CONTEXT` overrides it
|
||||
(plane-on-kubernetes task 1.2).
|
||||
- `doctor.py`, `egress.py` and `router.py` are handed the context. They build none themselves.
|
||||
- `tests/test_plane_literals.py` fails on any module, manifest or page that names this host's cluster,
|
||||
registry or addresses.
|
||||
|
||||
The egress client also needs `kubectl-ate` (or the executable `OUSAST_KUBECTL_ATE` names). It must be
|
||||
able to use that context.
|
||||
|
||||
### B.5 The receiver: in-cluster instead of on the host
|
||||
|
||||
Today the receiver is a thread inside the reconciler process (`egress.py`, `Receiver`), and the
|
||||
reconciler learns of a task's completion by watching that object's `delivered` set in memory
|
||||
(`reconciler.py`). Actors reach it through the Service `ousast-receiver.ax-system` whose
|
||||
EndpointSlice points at the host. Three paths, in order of effort:
|
||||
Today the receiver is a thread inside the reconciler process (`egress.py`, `Receiver`). The reconciler
|
||||
learns that a task finished by watching that object's `delivered` set in memory (`reconciler.py`).
|
||||
Actors reach it through the Service `ousast-receiver.ax-system`. That Service's EndpointSlice points at
|
||||
the host.
|
||||
|
||||
1. **Interim, works with the unchanged code:** keep the host receiver and render
|
||||
`ops/ax/receiver-service.yaml.tmpl` with an endpoint address the egress gateway pod can reach
|
||||
(`KIND_GATEWAY` is just an IPv4 address in the template, `OUSAST_ARTIFACT_PORT` the port). The
|
||||
reconciler's host must be reachable from the cluster (a VPN or a routed address), and the
|
||||
gateway dials the address the actor connected to, so the Service's ClusterIP must still map to it.
|
||||
2. **Planned: the receiver as an in-cluster Deployment** behind the same Service name, with the
|
||||
reconciler reading deliveries and serving inputs from it instead of from its own thread. This
|
||||
needs code: the receiver and its `delivered`/`inputs` state are in-process today, and the
|
||||
runner's `GET /inputs/<producer>/<artifact>` is served by that same process.
|
||||
3. **Follow-on: presigned-URL delivery to the S3 store.** The runner would `PUT` its output tar to a
|
||||
presigned URL on the memory store's bucket and fetch inputs the same way, and the reconciler
|
||||
would read from the bucket. Nothing of this exists; the EgressPolicy would then have to allow
|
||||
the store's hostname for the task.
|
||||
There are three paths, from least to most effort:
|
||||
|
||||
1. **Interim, works with the unchanged code:** keep the host receiver.
|
||||
Render `ops/ax/receiver-service.yaml.tmpl` with an endpoint address the egress gateway pod can reach.
|
||||
In the template, `KIND_GATEWAY` is just an IPv4 address and `OUSAST_ARTIFACT_PORT` is the port.
|
||||
The cluster must be able to reach the reconciler's host, through a VPN or a routed address.
|
||||
The gateway dials the address the actor connected to. So the Service's ClusterIP must still map to it.
|
||||
2. **Planned: the receiver as an in-cluster Deployment** behind the same Service name.
|
||||
The reconciler would read deliveries from it and serve inputs from it, instead of from its own thread.
|
||||
This needs code. The receiver and its `delivered`/`inputs` state are in-process today.
|
||||
The same process serves the runner's `GET /inputs/<producer>/<artifact>`.
|
||||
3. **Follow-on: presigned-URL delivery to the S3 store.** (A presigned URL grants one upload or download
|
||||
without credentials.)
|
||||
The runner would `PUT` its output tar to a presigned URL on the memory store's bucket.
|
||||
It would fetch inputs the same way, and the reconciler would read from the bucket.
|
||||
None of this exists. The EgressPolicy would then have to allow the store's hostname for the task.
|
||||
|
||||
### B.6 Worker pool sizing
|
||||
|
||||
`ops/ax/workerpool.yaml.tmpl` declares two warm gVisor workers of 1 CPU / 1.5 GiB (requests 250m /
|
||||
1.5 GiB) for the 7 GB host, selected onto nodes by `ate.dev/substrate-version`. An actor occupies a
|
||||
whole worker, so the pool size is the number of tasks that can run at once; set `--workers` on
|
||||
`ousast plane run` to match. Size the workers above the largest Task's limits (`verify`: requests
|
||||
250m / 384Mi, limits 1 CPU / 1 GiB; `roles` and the others are in `plane/tasks/`), and keep the
|
||||
golden-actor storage on the node in mind (B.2).
|
||||
`ops/ax/workerpool.yaml.tmpl` declares two warm gVisor workers for the 7 GB host. Each has 1 CPU /
|
||||
1.5 GiB (requests 250m / 1.5 GiB). They are placed on nodes by `ate.dev/substrate-version`.
|
||||
|
||||
An actor occupies a whole worker. So the pool size is the number of tasks that can run at once. Set
|
||||
`--workers` on `ousast plane run` to match.
|
||||
|
||||
Size the workers above the largest Task's limits. For `verify` these are requests 250m / 384Mi and
|
||||
limits 1 CPU / 1 GiB. `roles` and the others are in `plane/tasks/`. Also keep the golden-actor storage
|
||||
on the node in mind (B.2).
|
||||
|
||||
### B.7 The S3 memory store reachable from where the reconciler runs
|
||||
|
||||
The `s3://` store already works against any reachable S3-compatible server with versioning,
|
||||
lifecycle rules, object tags and S3 Select (RustFS is the tested one; the bucket setup and the
|
||||
agent policy are on [RustFS setup](rustfs.md)). Only the reconciler reads and writes it, so today
|
||||
it must be reachable from the operator's host; if the reconciler moves into the cluster, from its
|
||||
pod. No Task talks to the store directly; the day one does (B.5, path 3), that Task's EgressPolicy
|
||||
must allow the store's hostname (the policies allow hostnames only, never addresses), the way a
|
||||
Model declares its hosts today through `openultrasast.io/egress-hosts`.
|
||||
The `s3://` store already works against any reachable S3-compatible server. The server must support:
|
||||
|
||||
- versioning;
|
||||
- lifecycle rules;
|
||||
- object tags;
|
||||
- S3 Select (a server-side query over JSON objects).
|
||||
|
||||
RustFS is the tested server. The bucket setup and the agent policy are on [RustFS setup](rustfs.md).
|
||||
|
||||
Only the reconciler reads and writes the store. So today the operator's host must reach it. If the
|
||||
reconciler moves into the cluster, its pod must reach it.
|
||||
|
||||
No Task talks to the store directly. If one ever does (B.5, path 3), its EgressPolicy must allow the
|
||||
store's hostname. The policies allow hostnames only, never addresses. A Model already declares its hosts
|
||||
this way today, through `openultrasast.io/egress-hosts`.
|
||||
|
||||
### B.8 Secrets in Kubernetes Secrets, injected into the reconciler's environment
|
||||
|
||||
The reconciler reads the variable a Model's `secretKey.key` names (`DEEPSEEK_API_KEY`,
|
||||
`OPENROUTER_API_KEY`) and the store's `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`
|
||||
(optional `AWS_SESSION_TOKEN`, `S3_REGION`, `S3_BUCKET`) from its own environment or `.env`. When
|
||||
the reconciler runs in the cluster, keep them in Kubernetes Secrets and inject them into **the
|
||||
reconciler's** pod environment (`envFrom` with `secretRef`, or `valueFrom.secretKeyRef` per
|
||||
variable). Never put a value into a manifest: the Model manifests name a variable, the rendered
|
||||
Task carries no key, and the key reaches an actor only in the start request, where the runner keeps
|
||||
it in memory and redacts it from echoed stderr. The `secretKey.name` values in `plane/models/`
|
||||
(`deepseek`, `openrouter`) are the names such Secrets would carry. Running the reconciler
|
||||
in-cluster is itself planned: today it is a host process, and its state directory
|
||||
The reconciler reads these variables from its own environment or `.env`:
|
||||
|
||||
- the variable a Model's `secretKey.key` names (`DEEPSEEK_API_KEY`, `OPENROUTER_API_KEY`);
|
||||
- the store's `S3_ENDPOINT`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`;
|
||||
- optionally `AWS_SESSION_TOKEN`, `S3_REGION` and `S3_BUCKET`.
|
||||
|
||||
When the reconciler runs in the cluster, keep these in Kubernetes Secrets. Inject them into **the
|
||||
reconciler's** pod environment. Use `envFrom` with `secretRef`, or `valueFrom.secretKeyRef` per variable.
|
||||
|
||||
Never put a value into a manifest. The key stays out of every manifest:
|
||||
|
||||
- the Model manifests name a variable;
|
||||
- the rendered Task carries no key;
|
||||
- the key reaches an actor only in the start request;
|
||||
- the runner keeps it in memory and redacts it from echoed stderr.
|
||||
|
||||
The `secretKey.name` values in `plane/models/` (`deepseek`, `openrouter`) are the names such Secrets
|
||||
would carry.
|
||||
|
||||
Running the reconciler in-cluster is itself planned. Today it is a host process. Its state directory
|
||||
(`OUSAST_RESULTS`) would need a persistent volume.
|
||||
|
||||
### What is implemented and what is planned
|
||||
@@ -255,10 +336,10 @@ in-cluster is itself planned: today it is a host process, and its state director
|
||||
| Per-task EgressPolicy, credentials only in the start request, per-task budgets and attribution | implemented |
|
||||
| S3 memory store against any reachable S3-compatible server, verified at startup | implemented (RustFS tested) |
|
||||
| Local bring-up, doctor, smoke Run (`ops/ax/`) | implemented |
|
||||
| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates still name `localhost:5001` |
|
||||
| Re-pinning generated Tasks to another registry (`--runner-image`) | implemented; the committed templates name the kind profile's registry (`tests/test_plane_literals.py` checks it) |
|
||||
| Deployment to a cluster other than this host's kind cluster | **not done** |
|
||||
| One configurable kube context | **planned**; `kind-` hard-coded in `doctor.py`, `egress.py`, `router.py` |
|
||||
| `doctor` registry check against a configurable registry | **planned**; `localhost:5001` hard-coded in `doctor.py:45` |
|
||||
| One configurable kube context | implemented (2026-10-02): `kube_context` in the profile, `OUSAST_KUBE_CONTEXT` overrides |
|
||||
| `doctor` registry check against a configurable registry | implemented (2026-10-02): the profile's `registry` and images file |
|
||||
| Receiver as an in-cluster Deployment | **planned** (code change) |
|
||||
| Presigned-URL delivery to the S3 store | **follow-on**, not started |
|
||||
| Reconciler running in-cluster with Secrets injected into its environment | **planned** |
|
||||
@@ -266,12 +347,20 @@ in-cluster is itself planned: today it is a host process, and its state director
|
||||
|
||||
## Production topology: ax on a Kubernetes cluster
|
||||
|
||||
The maintainer's questions, answered from the code and the upstream checkouts under
|
||||
`~/.cache/ousast/ax-src/`: how is it hosted in ax, is there one container, how does ax relate to
|
||||
Kubernetes, and what scaling the pre-push hook would take. The decisions of 2026-10-02 set the
|
||||
target: **two profiles of one code base**, `kind` (this laptop, kept as a first-class local
|
||||
execution mode, where every change is proven first) and `k3s` (production). Everything marked
|
||||
**planned** is not built; the rest is what runs on this host today.
|
||||
This section answers the maintainer's questions from the code and the upstream checkouts under
|
||||
`~/.cache/ousast/ax-src/`:
|
||||
|
||||
- How is it hosted in ax?
|
||||
- Is there one container?
|
||||
- How does ax relate to Kubernetes?
|
||||
- What would scaling the pre-push hook take?
|
||||
|
||||
The decisions of 2026-10-02 set the target: **two profiles of one code base**.
|
||||
|
||||
- `kind` is this laptop. It stays a first-class local execution mode. Every change is proven here first.
|
||||
- `k3s` is production. (k3s is a small Kubernetes distribution.)
|
||||
|
||||
Everything marked **planned** is not built. The rest is what runs on this host today.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -287,7 +376,7 @@ flowchart LR
|
||||
ctl["ate-api-server, ate-controller, atelet"]
|
||||
rt["atenet-router"]
|
||||
eg["atenet-egress gateway"]
|
||||
wp["WorkerPool: gVisor workers, runsc RuntimeClass, HPA (planned)"]
|
||||
wp["WorkerPool: gVisor workers, runsc in the worker, HPA (planned)"]
|
||||
ewp["Engine WorkerPool: Joern as a task (planned)"]
|
||||
end
|
||||
end
|
||||
@@ -308,124 +397,173 @@ flowchart LR
|
||||
cli -- "polls: status, attribution" --> s3
|
||||
```
|
||||
|
||||
No inbound path exists in this picture: the CLI reaches ax-server and the router from outside, the
|
||||
tasks push their artifacts to the store through the egress gateway, and the CLI reads the store.
|
||||
Today's EndpointSlice back to a laptop (`ops/ax/receiver-service.yaml.tmpl`) is the kind profile's
|
||||
device and does not appear in production.
|
||||
This picture has no inbound path:
|
||||
|
||||
- the CLI reaches ax-server and the router from outside;
|
||||
- the tasks push their artifacts to the store through the egress gateway;
|
||||
- the CLI reads the store.
|
||||
|
||||
Today's EndpointSlice back to a laptop (`ops/ax/receiver-service.yaml.tmpl`) belongs to the kind
|
||||
profile only. It does not appear in production.
|
||||
|
||||
### ax, Kubernetes and Agent Substrate in three sentences
|
||||
|
||||
**ax is a Kubernetes application**: a Deployment `ax-server` with a Redis beside it in the
|
||||
namespace `ax-system` (`ax/deploy/ax-server.yaml` in the ax checkout; its controller runs with
|
||||
`--template=default-template --template-atespace=ax-system`), and the `ax` CLI reaches it through a
|
||||
`kubectl port-forward` to `svc/ax-server` (`ax/internal/tunnel/tunnel.go`), so `ousast plane run`
|
||||
needs a kube context or an ingress, not a public endpoint. **ax does not run containers itself**:
|
||||
it turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`,
|
||||
`CreateActorTemplate`), built from the Task's image and environment, and the image must be pinned
|
||||
by digest (Substrate's `SandboxConfig` validation rejects tags: "All images must include a
|
||||
digest"). **Agent Substrate is the layer that owns the machines**: in the namespace `ate-system`
|
||||
it runs `ate-api-server`, `ate-controller`, the `atelet` DaemonSet, `atenet-router`, the
|
||||
`atenet-egress` gateway, a Postgres and a RustFS (`substrate/manifests/ate-install/`), and three
|
||||
CRDs under `ate.dev/v1alpha1`, `WorkerPool`, `SandboxConfig` and `CSIDriverConfig`; a
|
||||
`WorkerPool` keeps warm worker pods (`ko://.../cmd/ateom-gvisor`), and an actor is one gVisor
|
||||
sandbox inside one of those workers, booted from the golden snapshot that
|
||||
[The plane on ax](plane.md) describes, suspended and resumed by the controller. Names are limited
|
||||
to 63 bytes (`MaxNameLength` in `ax/pkg/apis/v1alpha1/types.go`), which is why the generated Task
|
||||
names are short.
|
||||
**ax is a Kubernetes application.**
|
||||
|
||||
**What k3s changes** (planned; nothing has been applied to a k3s cluster): k3s runs containerd
|
||||
with no Docker socket, so the host-Docker path of `ousast plane alerts-engine` does not exist
|
||||
there and the engine must become a task (below); Substrate's gVisor workers need `runsc` and a
|
||||
`RuntimeClass` on the nodes they are scheduled to; k3s ships Traefik (the ingress the CLI can use
|
||||
instead of a kubeconfig) and local-path storage. Images are built by GitHub Actions from tagged
|
||||
commits, published to GitHub Container Registry and digest-pinned in the manifests; a private
|
||||
package needs an `imagePullSecret` on the worker ServiceAccount. Substrate's and ax's own images
|
||||
are built with `ko` from their checkouts and pushed to the same registry. The kind profile keeps
|
||||
`localhost:5001` and `ops/ax/up.sh`.
|
||||
- It is a Deployment `ax-server` with a Redis beside it, in the namespace `ax-system`
|
||||
(`ax/deploy/ax-server.yaml` in the ax checkout).
|
||||
- Its controller runs with `--template=default-template --template-atespace=ax-system`.
|
||||
- The `ax` CLI reaches it through a tunnel to `svc/ax-server` over the kube context
|
||||
(`ax/internal/tunnel/tunnel.go`).
|
||||
- So `ousast plane run` needs a kube context or an ingress, not a public endpoint.
|
||||
|
||||
**ax does not run containers itself.**
|
||||
|
||||
- It turns every Task into an ActorTemplate on Agent Substrate (`ax/internal/substrate/client.go`,
|
||||
`CreateActorTemplate`).
|
||||
- The ActorTemplate is built from the Task's image and environment.
|
||||
- The image must be pinned by digest. Substrate's `SandboxConfig` validation rejects tags: "All images
|
||||
must include a digest".
|
||||
|
||||
**Agent Substrate is the layer that owns the machines.**
|
||||
|
||||
- In the namespace `ate-system` it runs `ate-api-server`, `ate-controller`, the `atelet` DaemonSet,
|
||||
`atenet-router`, the `atenet-egress` gateway, a Postgres and a RustFS
|
||||
(`substrate/manifests/ate-install/`).
|
||||
- It defines three CRDs (custom Kubernetes resource types) under `ate.dev/v1alpha1`: `WorkerPool`,
|
||||
`SandboxConfig` and `CSIDriverConfig`.
|
||||
- A `WorkerPool` keeps warm worker pods (`ko://.../cmd/ateom-gvisor`).
|
||||
- An actor is one gVisor sandbox inside one of those workers.
|
||||
- The actor boots from the golden snapshot that [The plane on ax](plane.md) describes. The controller
|
||||
suspends and resumes it.
|
||||
- Names are limited to 63 bytes (`MaxNameLength` in `ax/pkg/apis/v1alpha1/types.go`). That is why the
|
||||
generated Task names are short.
|
||||
|
||||
**What k3s changes** (planned; nothing has been applied to a k3s cluster):
|
||||
|
||||
- k3s runs containerd with no Docker socket. So the host-Docker path of `ousast plane alerts-engine`
|
||||
does not exist there. The engine must become a task (below).
|
||||
- Substrate's gVisor workers need no Kubernetes `RuntimeClass`: the worker fetches a pinned gVisor
|
||||
tarball and runs `runsc` inside its own pod. The node checklist is pod security, tarball
|
||||
reachability and local-path storage (B.1).
|
||||
- k3s ships Traefik and local-path storage. Traefik is an ingress the CLI can use instead of a kubeconfig.
|
||||
- GitHub Actions builds the images from tagged commits. They are published to GitHub Container Registry
|
||||
and digest-pinned in the manifests.
|
||||
- A private package needs an `imagePullSecret` on the worker ServiceAccount.
|
||||
- Substrate's and ax's own images are built with `ko` from their checkouts. They are pushed to the same
|
||||
registry.
|
||||
- The kind profile keeps its local registry (`ops/k8s/profiles/kind.toml`) and `ops/ax/up.sh`.
|
||||
|
||||
### One image, and what concurrency equals
|
||||
|
||||
There is **one container image** for every Task: `plane/Dockerfile.runner` (`python:3.12-slim`
|
||||
plus this package, PID 1 `/usr/local/bin/ax-task-runner`, which is
|
||||
`openultrasast.plane.runner`). Each Task manifest under `plane/tasks/` names the same image by
|
||||
digest (the pin is kept in `~/.cache/ousast/ax-src/runner-image`) and selects its work with
|
||||
`spec.command`: the runner starts `python -m openultrasast.plane.tasks.<command[0]>`
|
||||
(`runner.py`, `task_module`). `repo-facts`, `verify`, `agree`, `features`, `remember`, `alerts`,
|
||||
`loop` and `roles` are modules in that image, not images of their own; the engine would be the
|
||||
second image (planned). So the production answer to "how many containers" is: one image, one
|
||||
actor per Task, and **an actor occupies a whole worker** (its limits are the ActorTemplate's, the
|
||||
worker holds one actor at a time). Concurrency is therefore the number of workers in the
|
||||
`WorkerPool`, nothing else: `ops/ax/workerpool.yaml.tmpl` declares `replicas: 2` of 1 CPU /
|
||||
1.5 GiB, so at most two Tasks run at once on this host, and `ousast plane run --workers` must not
|
||||
exceed the pool. Substrate ships an autoscaled pool
|
||||
(`substrate/demos/autoscaled-workerpool/README.md`: an HPA on the external metric
|
||||
`ate_workerpool_workers{state=at_capacity}` through prometheus-adapter, writing
|
||||
`WorkerPool.spec.replicas`; its README says the demo is kind-only today) and request parking
|
||||
(`substrate/demos/parking/`: the router holds a resume until a worker frees up). Neither is
|
||||
applied by `ops/ax/up.sh`; the production profile's pool manifests and autoscaling rule are
|
||||
There is **one container image** for every Task: `plane/Dockerfile.runner`. It is `python:3.12-slim`
|
||||
plus this package. PID 1 is `/usr/local/bin/ax-task-runner`, which is `openultrasast.plane.runner`.
|
||||
|
||||
Each Task manifest under `plane/tasks/` names the same image by digest. The pin is kept in
|
||||
`~/.cache/ousast/ax-src/runner-image`. The manifest picks its work with `spec.command`. The runner
|
||||
starts `python -m openultrasast.plane.tasks.<command[0]>` (`runner.py`, `task_module`).
|
||||
|
||||
`repo-facts`, `verify`, `agree`, `features`, `remember`, `alerts`, `loop` and `roles` are modules in
|
||||
that image. They are not images of their own. The engine would be the second image (planned).
|
||||
|
||||
So the production answer to "how many containers" is:
|
||||
|
||||
- one image;
|
||||
- one actor per Task;
|
||||
- **an actor occupies a whole worker**. Its limits are the ActorTemplate's, and the worker holds one
|
||||
actor at a time.
|
||||
|
||||
Concurrency is therefore the number of workers in the `WorkerPool`, nothing else.
|
||||
`ops/ax/workerpool.yaml.tmpl` declares `replicas: 2` of 1 CPU / 1.5 GiB. So at most two Tasks run at
|
||||
once on this host. `ousast plane run --workers` must not exceed the pool.
|
||||
|
||||
Substrate ships two scaling aids:
|
||||
|
||||
- **An autoscaled pool** (`substrate/demos/autoscaled-workerpool/README.md`). An HPA (Horizontal Pod
|
||||
Autoscaler) reads the external metric `ate_workerpool_workers{state=at_capacity}` through
|
||||
prometheus-adapter. It writes `WorkerPool.spec.replicas`. Its README says the demo is kind-only today.
|
||||
- **Request parking** (`substrate/demos/parking/`). The router holds a resume until a worker frees up.
|
||||
|
||||
`ops/ax/up.sh` applies neither. The production profile's pool manifests and autoscaling rule are
|
||||
**planned**.
|
||||
|
||||
### The three components that are bound to the host today
|
||||
|
||||
| Component | Today (the kind profile) | Production (the k3s profile, planned) |
|
||||
| --- | --- | --- |
|
||||
| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against `kind-ousast`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
|
||||
| Reconciler (`ousast plane run`, `src/openultrasast/plane/reconciler.py`) | a process on the operator's host; calls `ax` and `kubectl` against the profile's `kube_context`, holds the Run's state under `OUSAST_RESULTS` | **the reconciler stays the CLI.** An execution mode `local` or `remote`: in `remote`, the same `ousast plane run` from a laptop or a CI job submits the Run to ax-server in k3s (over the cluster's ingress or a kubeconfig), sends the start requests, then polls the store and prints the same `status` and attribution table. Credentials come from the caller's environment as today (CI secrets in a job; B.8's rule, no value in a manifest). An unattended in-cluster reconciler (a Deployment with a submit API, or a Job per Run) is a **later option**, not the first step |
|
||||
| Artifact receiver (`plane/egress.py`, the `Receiver` thread inside the reconciler) | reached through the Service `ousast-receiver.ax-system` whose EndpointSlice points at the kind gateway address on the host (`ops/ax/receiver-service.yaml.tmpl`) | **no receiver in remote mode.** Tasks deliver their artifacts and fetch their inputs through presigned URLs on the S3 memory store (B.5, path 3; the store's hostname enters each Task's EgressPolicy, B.7), and the CLI reads completion from the store. The kind profile keeps the receiver thread; an in-cluster receiver Deployment (B.5, path 2) is not the chosen direction |
|
||||
| Joern engine (`plane/engine_alerts.py`, the host's `openultrasast:dev` container, run with `--memory 3g`) | `ousast plane alerts-engine` runs it on the host, one container at a time, and marks the Run's `alerts` tasks done | **an `engine` task on a second, larger `WorkerPool`** (the engine does not fit a 1.5 GiB worker, and k3s has no Docker socket), from an engine image digest-pinned in GHCR, emitting the same `alerts.jsonl` rows and coverage; `alerts-engine` on the host stays as the development path |
|
||||
|
||||
The S3 memory store (RustFS tested) is already external; in remote mode it is also the artifact
|
||||
path, so it must be reachable from the CLI's host and, through the egress gateway, from the
|
||||
workers.
|
||||
The S3 memory store (RustFS tested) is already external. In remote mode it is also the artifact path.
|
||||
So the CLI's host must reach it, and the workers must reach it through the egress gateway.
|
||||
|
||||
### Scaling the pre-push hook in production
|
||||
|
||||
**Today** `ousast pre-push` is a local, deterministic check: it materialises base and head,
|
||||
discovers the changed regions, runs the evidence ranker and arbiter on head and compares with base,
|
||||
all within `--deadline` (30 s by default) and in `--mode advisory` unless blocking is opted into
|
||||
([Pre-push: the delta check](scanning.md#pre-push-the-delta-check-experimental),
|
||||
`src/openultrasast/push/`, [Engine and pre-push](ops/README.md#experimental-pre-push-integration)).
|
||||
**The plane is not in that path**: no hook submits a Run, no model is called without an explicit
|
||||
`--model-config`, and nothing leaves the developer's machine.
|
||||
**Today** `ousast pre-push` is a local, deterministic check. It does four things:
|
||||
|
||||
1. materialises base and head;
|
||||
2. finds the changed regions;
|
||||
3. runs the evidence ranker and arbiter on head;
|
||||
4. compares the result with base.
|
||||
|
||||
All of this runs within `--deadline` (30 s by default). It runs in `--mode advisory` unless you opt
|
||||
into blocking. See [Pre-push: the delta check](scanning.md#pre-push-the-delta-check-experimental),
|
||||
`src/openultrasast/push/` and [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
|
||||
|
||||
**The plane is not in that path.**
|
||||
|
||||
- No hook submits a Run.
|
||||
- No model is called without an explicit `--model-config`.
|
||||
- Nothing leaves the developer's machine.
|
||||
|
||||
**The production shape** (planned, none of it built) splits the work by latency:
|
||||
|
||||
1. **Blocking, inside the hook's deadline**: what runs today, the quick rules and the delta
|
||||
engine on the changed regions. It stays local and deterministic, so the push is never held for
|
||||
a model or a cluster.
|
||||
2. **Asynchronous, on the plane**: the hook or the CI job calls the CLI in remote mode with a
|
||||
*delta Run* whose candidates are the functions the push changed; the Run is the per-case chain
|
||||
of [The plane on ax](plane.md#per-case) (`facts`, `verify` a and b, `agree`, `verify` c on the
|
||||
disputed only, `final`). The caller either waits, with its own deadline, for the CLI to print
|
||||
the status and attribution table, or returns at once and a later `ousast plane status <run>`
|
||||
reports from the store; a commit status check or a review comment would be built on that. The
|
||||
worker pool is sized to the push rate or autoscaled on `at_capacity` workers. An unattended
|
||||
in-cluster reconciler that accepts submissions without a CLI session is the later option named
|
||||
above.
|
||||
1. **Blocking, inside the hook's deadline**: what runs today, the quick rules and the delta engine on
|
||||
the changed regions. It stays local and deterministic. So the push never waits for a model or a
|
||||
cluster.
|
||||
2. **Asynchronous, on the plane**: the hook or the CI job calls the CLI in remote mode with a *delta
|
||||
Run*.
|
||||
- The Run's candidates are the functions the push changed.
|
||||
- The Run is the per-case chain of [The plane on ax](plane.md#per-case): `facts`, `verify` a and b,
|
||||
`agree`, `verify` c on the disputed only, `final`.
|
||||
- The caller can wait, with its own deadline, for the CLI to print the status and attribution table.
|
||||
- Or the caller returns at once, and a later `ousast plane status <run>` reports from the store.
|
||||
- A commit status check or a review comment would be built on that.
|
||||
- The worker pool is sized to the push rate, or autoscaled on `at_capacity` workers.
|
||||
- An unattended in-cluster reconciler that accepts submissions without a CLI session is the later
|
||||
option named above.
|
||||
|
||||
**Cost per push, estimated from measured rates.** Two measured per-candidate rates exist for one
|
||||
model verdict: `$0.003215` (`spend.cost_per_candidate_evaluate_usd` in
|
||||
`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`) and `$0.0021`
|
||||
(exp-002 arm A, `overall.usd_per_candidate.A` in
|
||||
`benchmarks/measurements/2026-10-02-exp-002/record.json`). A push that changes five functions needs
|
||||
two verify passes per candidate: 5 x 2 x $0.0021 = $0.021, or 5 x 2 x $0.003215 = $0.032 at the
|
||||
higher rate, plus a third pass on the disputed candidates only. That is where "about two cents per
|
||||
push" comes from; it is arithmetic over the decision engine's measured rates, not a measurement of
|
||||
a push Run, and `ousast plane status` would print the real figure per Task.
|
||||
**Cost per push, estimated from measured rates.** Two measured per-candidate rates exist for one model
|
||||
verdict:
|
||||
|
||||
- `$0.003215` (`spend.cost_per_candidate_evaluate_usd` in
|
||||
`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`);
|
||||
- `$0.0021` (exp-002 arm A, `overall.usd_per_candidate.A` in
|
||||
`benchmarks/measurements/2026-10-02-exp-002/record.json`).
|
||||
|
||||
A push that changes five functions needs two verify passes per candidate. That gives 5 x 2 x $0.0021 =
|
||||
$0.021, or 5 x 2 x $0.003215 = $0.032 at the higher rate. A third pass runs on the disputed candidates
|
||||
only. That is where "about two cents per push" comes from. It is arithmetic over the decision engine's
|
||||
measured rates, not a measurement of a push Run. `ousast plane status` would print the real figure per
|
||||
Task.
|
||||
|
||||
**What is not built** (each is a named gap, not a configuration):
|
||||
|
||||
- The delta-Run command. `ousast plane scan <repo> --base <commit> --head <commit>` **does not
|
||||
exist**; `ousast plane` has `run`, `status`, `doctor`, `remember`, `memory-normalise`,
|
||||
exist**. `ousast plane` has `run`, `status`, `doctor`, `remember`, `memory-normalise`,
|
||||
`workspaces`, `harvest` and `alerts-engine` (`src/openultrasast/cli.py`).
|
||||
- A candidates task for an arbitrary repository. `repo-facts` takes its candidates from a
|
||||
population's validation set; a step that derives them from a diff is not written.
|
||||
population's validation set. No step derives them from a diff yet.
|
||||
- Workspace generation from a URL and a commit outside a population. `plane/generate.py` is
|
||||
population-driven (`read_cases` reads a validation set and a population TOML), so a Run for a
|
||||
population-driven: `read_cases` reads a validation set and a population TOML. So a Run for a
|
||||
repository that is not in a population cannot be generated today.
|
||||
- The remote execution mode itself: the `local`/`remote` switch, configurable cluster context,
|
||||
ax-server and router addresses (the `kind-` prefix is hard-coded, B.4), presigned delivery from
|
||||
the runner to the store, and the CLI reading completion and `summary.json` from the store
|
||||
instead of its receiver thread.
|
||||
- The remote execution mode itself:
|
||||
- the `local`/`remote` switch;
|
||||
- a configurable cluster context and ax-server and router addresses (the `kind-` prefix is
|
||||
hard-coded, B.4);
|
||||
- presigned delivery from the runner to the store;
|
||||
- the CLI reading completion and `summary.json` from the store instead of its receiver thread.
|
||||
- The engine task and its WorkerPool, the GHCR build, the production pool and autoscaling
|
||||
manifests, and the k3s runbook and checklist.
|
||||
- The check or comment that carries the asynchronous verdict back to the push.
|
||||
@@ -433,8 +571,7 @@ a push Run, and `ousast plane status` would print the real figure per Task.
|
||||
### Sizing
|
||||
|
||||
The worker is the unit. Every Task's `limits` fit the 1 CPU / 1.5 GiB worker of
|
||||
`ops/ax/workerpool.yaml.tmpl`; the requests are what the Task declares, the worker is what it
|
||||
occupies.
|
||||
`ops/ax/workerpool.yaml.tmpl`. The requests are what the Task declares. The worker is what it occupies.
|
||||
|
||||
| Task (`plane/tasks/*.yaml`) | Requests | Limits | Per push |
|
||||
| --- | --- | --- | --- |
|
||||
@@ -446,17 +583,18 @@ occupies.
|
||||
| `roles` | 250m / 384Mi | 1 CPU / 1Gi | not in a push Run (harvest Runs only) |
|
||||
| `engine` (planned) | from a measured run | above 3 GiB (the host run uses `--memory 3g`) | its own pool, not the default one |
|
||||
|
||||
- **Actors per push**: 4 (`facts`, `va`, `vb`, `agree`), 6 with a tie-break (`vc`, `final`);
|
||||
- **Actors per push**: 4 (`facts`, `va`, `vb`, `agree`), or 6 with a tie-break (`vc`, `final`).
|
||||
`features` and `remember` add two more when the verdict is to be kept.
|
||||
- **Critical path**: `facts` then `va` and `vb` side by side, then `agree`, then (sometimes) `vc`
|
||||
and `final`: three to five Task boots in series, each a golden-snapshot restore plus the model
|
||||
calls of the verify passes.
|
||||
- **One actor per worker**: `agree` asks for 256Mi and still holds a 1.5 GiB worker while it runs,
|
||||
because the worker's capacity, not the Task's request, is the unit. A pool of `W` workers runs
|
||||
at most `W/2` pushes through their verify phase at once; on this host (`W = 2`) that is one
|
||||
push at a time, and every further push waits (the reconciler's `--workers` bound, or Substrate's
|
||||
request parking if the pool is saturated). Autoscaling the pool on `at_capacity` workers is the
|
||||
upstream answer and is **planned** here.
|
||||
- **Critical path**: `facts`, then `va` and `vb` side by side, then `agree`, then sometimes `vc`
|
||||
and `final`. That is three to five Task boots in series. Each boot is a golden-snapshot restore plus
|
||||
the model calls of the verify passes.
|
||||
- **One actor per worker**: `agree` asks for 256Mi but still holds a 1.5 GiB worker while it runs. The
|
||||
worker's capacity, not the Task's request, is the unit.
|
||||
- A pool of `W` workers runs at most `W/2` pushes through their verify phase at once.
|
||||
- On this host (`W = 2`) that is one push at a time.
|
||||
- Every further push waits, on the reconciler's `--workers` bound or on Substrate's request parking
|
||||
if the pool is saturated.
|
||||
- Autoscaling the pool on `at_capacity` workers is the upstream answer. It is **planned** here.
|
||||
|
||||
## C. The manifests under `plane/`
|
||||
|
||||
@@ -472,6 +610,12 @@ occupies.
|
||||
| `ops/ax/smoke-task.yaml.tmpl`, `smoke-run.yaml.tmpl` | yes | rendered with `RUNNER_IMAGE` from your pin file |
|
||||
| `ops/ax/up.sh`, `down.sh` | no | kind-specific; they do not apply to another cluster |
|
||||
|
||||
The Run and Task contract itself (`plane/runner.py` as PID 1 of the runner image, `AX_TASK_YAML`
|
||||
and `AX_WORKSPACES_YAML`, `/healthz` and `/readyz`, the start request, the tar delivery as
|
||||
completion) does not depend on where the cluster is: [The plane on ax](plane.md).
|
||||
The Run and Task contract does not depend on where the cluster is. That contract covers:
|
||||
|
||||
- `plane/runner.py` as PID 1 of the runner image;
|
||||
- `AX_TASK_YAML` and `AX_WORKSPACES_YAML`;
|
||||
- `/healthz` and `/readyz`;
|
||||
- the start request;
|
||||
- the tar delivery as completion.
|
||||
|
||||
See [The plane on ax](plane.md).
|
||||
|
||||
+549
-348
File diff suppressed because it is too large
Load Diff
+75
-52
@@ -1,7 +1,8 @@
|
||||
# Evaluation
|
||||
|
||||
What has been measured on code the tool was not tuned on, and how. State as of 2026-10-02.
|
||||
Every number below is quoted from a committed record; the path is given with it.
|
||||
This page shows what has been measured on code the tool was not tuned on, and how. State as of
|
||||
2026-10-02. Every number below is quoted from a committed record, and the record's path is
|
||||
given with it.
|
||||
|
||||
## Instruments
|
||||
|
||||
@@ -11,32 +12,47 @@ Every number below is quoted from a committed record; the path is given with it.
|
||||
| Pair corpus | fire on the vulnerable function, stay silent on its fix | `ousast pairs`, `benchmarks/pairs/` |
|
||||
| Independent populations | real repositories at vulnerable, fixed and benign pins, scored under a pre-registered protocol | `benchmarks/independent/` |
|
||||
|
||||
The detection gate and the local pairs are CI gates. They catch regressions; they do not
|
||||
Recall means the share of known bugs found. FP means false positives.
|
||||
|
||||
The detection gate and the local pairs are CI gates. They catch regressions. They do not
|
||||
estimate recall on real code, because the rules were written against them.
|
||||
|
||||
## Pair corpora
|
||||
|
||||
`benchmarks/pairs/catalog.toml` and the slice directories (`local`, `github`, `sast`, `vfc`,
|
||||
`vibe-py`, `vfc-js`, `agent-vfc`, `owasp`) are runnable with `ousast pairs --slice <name>`; each
|
||||
pair carries a provenance profile and a train/holdout split (`--profile`, `--split`). Pointer
|
||||
pairs are not vendored: `--pointers` fetches them into `~/.cache/openultrasast/pairs/`.
|
||||
`benchmarks/pairs/README.md` documents the schema and the review tiers.
|
||||
A pair is a vulnerable function and its fixed version. You can run the catalog
|
||||
`benchmarks/pairs/catalog.toml` and the slice directories with `ousast pairs --slice <name>`.
|
||||
The slices are `local`, `github`, `sast`, `vfc`, `vibe-py`, `vfc-js`, `agent-vfc` and `owasp`.
|
||||
|
||||
- Each pair carries a provenance profile and a train/holdout split (`--profile`, `--split`).
|
||||
- Pointer pairs are not vendored. `--pointers` fetches them into `~/.cache/openultrasast/pairs/`.
|
||||
- `benchmarks/pairs/README.md` documents the schema and the review tiers.
|
||||
|
||||
**`advisory-fixes`** (`benchmarks/pairs/advisory-fixes/`, 2026-10-01) holds 97 pointer pairs,
|
||||
one per repository: a public CVE/GHSA fix commit whose advisory-named function is declared on
|
||||
both sides and changed by the fix. It was assembled to give the decision engine real negatives
|
||||
for families the corpus had few of (access control, deserialization, output encoding, path,
|
||||
untrusted destination). It is a label source of the decision engine
|
||||
(`src/openultrasast/learn/sources.toml`), not an `ousast pairs` slice; the research log and
|
||||
one per repository. Each is a public CVE/GHSA fix commit. The function the advisory names is
|
||||
declared on both sides and changed by the fix.
|
||||
|
||||
It was assembled to give the decision engine real negatives (fixed, non-vulnerable code). It
|
||||
targets families the corpus had few of: access control, deserialization, output encoding,
|
||||
path, untrusted destination. It is a label source of the decision engine
|
||||
(`src/openultrasast/learn/sources.toml`), not an `ousast pairs` slice. The research log and
|
||||
every rejection are in `research-2026-10-01.md`.
|
||||
|
||||
## Independent populations and the M4 gates
|
||||
|
||||
The pre-push safety net's qualification milestone (M4) requires, together: at least 95%
|
||||
actionable precision, at least 90% supported recall, at least 95% supported-check completion,
|
||||
the runtime gates, and zero fixed-side or benign false alerts. A population is frozen before
|
||||
any of its cases is scanned, scored once under a pre-registered protocol, and then **spent**:
|
||||
it may inform changes, but it can no longer qualify anything.
|
||||
M4 is the qualification milestone of the pre-push safety net. It requires all of these
|
||||
together:
|
||||
|
||||
- at least 95% actionable precision;
|
||||
- at least 90% supported recall;
|
||||
- at least 95% supported-check completion;
|
||||
- the runtime gates;
|
||||
- zero fixed-side or benign false alerts.
|
||||
|
||||
A population is a set of real repositories. Its lifecycle has three steps:
|
||||
|
||||
1. It is frozen before any of its cases is scanned.
|
||||
2. It is scored once under a pre-registered protocol.
|
||||
3. It is then **spent**: it may inform changes, but it can no longer qualify anything.
|
||||
|
||||
| Population | Status | Result | Record |
|
||||
| --- | --- | --- | --- |
|
||||
@@ -45,9 +61,9 @@ it may inform changes, but it can no longer qualify anything.
|
||||
| v2, model-driven pipeline | exploratory (v2 already spent) | stopped after 7 of 17 cases when the account emptied: recall 4/7; precision not adjudicated; no gate measurable | `results-v2-model-pipeline.json` |
|
||||
| v3 (PHP, 15 cases; one SSRF slot left empty by the selection rule) | frozen, untouched | none: it is the one-time final check, run once under `protocol-v3.md` after a prediction is committed | `population-v3-php.toml`, `freeze-v3-php.json` |
|
||||
|
||||
v3 is reserved: development and analysis work never reads its cases, and
|
||||
`tests/test_independent_population.py` fails when anything in the development tree (a recipe,
|
||||
catalog, manifest or measurement) names one of its repositories.
|
||||
v3 is reserved. Development and analysis work never reads its cases.
|
||||
`tests/test_independent_population.py` fails when anything in the development tree names one
|
||||
of its repositories. That covers a recipe, catalog, manifest or measurement.
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
@@ -66,9 +82,10 @@ stateDiagram-v2
|
||||
|
||||
## Folds: how learned decisions are kept out of their own training data
|
||||
|
||||
The decision engine is evaluated on repositories it was not shown (`src/openultrasast/learn/folds.py`).
|
||||
Every split is by repository group (the label builder's normalised `owner/name`, merged across
|
||||
URLs, forks and shared advisories), across all corpora, seeded and deterministic.
|
||||
The decision engine is evaluated on repositories it was not shown
|
||||
(`src/openultrasast/learn/folds.py`). Every split is by repository group. A group is the label
|
||||
builder's normalised `owner/name`, merged across URLs, forks and shared advisories. Splits
|
||||
cover all corpora and are seeded and deterministic.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
@@ -81,44 +98,50 @@ flowchart TD
|
||||
rest --> lofo["Leave-one-framework-out: frameworks with at least 10 groups"]
|
||||
```
|
||||
|
||||
`assert_disjoint` runs before a compile: the compile split and the evaluation folds share no
|
||||
group. A framework with fewer than 10 groups gets `insufficient data` instead of a fold. As of
|
||||
2026-10-02 the outer folds have been run for the six evaluable families; leave-one-source-out
|
||||
and leave-one-framework-out have not
|
||||
`assert_disjoint` runs before a compile. It checks that the compile split and the evaluation
|
||||
folds share no group. A framework with fewer than 10 groups gets `insufficient data` instead of
|
||||
a fold. As of 2026-10-02, the outer folds have been run for the six evaluable families.
|
||||
Leave-one-source-out and leave-one-framework-out have not
|
||||
([decision-engine.md](decision-engine.md#second-slice-six-families-2026-10-02)).
|
||||
|
||||
## The plane increment
|
||||
|
||||
The first increment of the agentic plane ran the model-driven pipeline's validation set (46
|
||||
candidates in 15 cases of v2, 20 declared vulnerable sites) on google/ax and compared it with
|
||||
the script pipeline on the same cases, candidates, pins and recorded triage. The pre-registered
|
||||
gate: at least 16 of 20 declared sites agreed, at a cost per candidate under $0.022.
|
||||
The first increment of the agentic plane ran on google/ax. It used the model-driven pipeline's
|
||||
validation set: 46 candidates in 15 cases of v2, with 20 declared vulnerable sites. It was
|
||||
compared with the script pipeline on the same cases, candidates, pins and recorded triage.
|
||||
|
||||
The pre-registered gate: at least 16 of 20 declared sites agreed, at a cost per candidate under
|
||||
$0.022.
|
||||
|
||||
- **Increment 1** (2026-09-29, `benchmarks/independent/plane-increment-1.json`): 14 of 20
|
||||
declared sites agreed against the reference's 16; $0.0194 per candidate with recorded triage.
|
||||
**NOT MET** on recall.
|
||||
*Correction (2026-09-30, in the record):* the first version read agree's
|
||||
`declared_sites_matched` (coverage by candidates, 19) as "found in either pass" and concluded
|
||||
detection was unchanged. Recounted from the agreed rows, the plane's passes flagged 17 of 20
|
||||
declared sites in at least one pass against the reference's 19.
|
||||
- **Increment 2** (2026-09-30, `benchmarks/independent/plane-increment-2.json`): after the design
|
||||
revisit, a tie-break pass c on the 10 disputed candidates with 2-of-3 agreement. 16 of 20
|
||||
declared sites agreed (reference 16) at $0.0205 per candidate over 46 ($0.02195 over the 43
|
||||
after triage, against the reference's $0.022). **MET**, with two caveats the record carries:
|
||||
the tie-break was chosen after seeing the first measurement on the same validation set, so
|
||||
this compares plane with script and qualifies nothing on an independent population; and the
|
||||
plane's passes still flag fewer declared sites than the reference (17 vs 19). Precision is
|
||||
not part of this gate and remains the open problem.
|
||||
declared sites agreed, against the reference's 16. Cost was $0.0194 per candidate with
|
||||
recorded triage. **NOT MET** on recall.
|
||||
*Correction (2026-09-30, in the record):* the first version misread agree's
|
||||
`declared_sites_matched`. That field is coverage by candidates (19), but it was read as
|
||||
"found in either pass". So the first version concluded detection was unchanged. Recounted
|
||||
from the agreed rows, the plane's passes flagged 17 of 20 declared sites in at least one
|
||||
pass. The reference flagged 19.
|
||||
- **Increment 2** (2026-09-30, `benchmarks/independent/plane-increment-2.json`): came after the
|
||||
design revisit. It added a tie-break pass c on the 10 disputed candidates, with 2-of-3
|
||||
agreement. 16 of 20 declared sites agreed (reference 16). Cost was $0.0205 per candidate over
|
||||
46, and $0.02195 over the 43 after triage (reference: $0.022). **MET**, with two
|
||||
caveats the record carries:
|
||||
1. The tie-break was chosen after seeing the first measurement on the same validation set.
|
||||
So this compares plane with script and qualifies nothing on an independent population.
|
||||
2. The plane's passes still flag fewer declared sites than the reference (17 vs 19).
|
||||
|
||||
Precision is not part of this gate and remains the open problem.
|
||||
|
||||
## The decision engine
|
||||
|
||||
Two slices have been measured out of repository (injection, 2026-10-01; six families,
|
||||
2026-10-02). The engine is not adopted. The state against the gates, in one place with the plane
|
||||
increment and the populations above, is on [Where we stand](where-we-stand.md#1-detection-capability-and-goals-no-go-measured);
|
||||
the per-family tables are on [decision-engine.md](decision-engine.md).
|
||||
Two slices have been measured out of repository: injection (2026-10-01) and six families
|
||||
(2026-10-02). The engine is not adopted. [Where we
|
||||
stand](where-we-stand.md#1-detection-capability-and-goals-no-go-measured) shows the state
|
||||
against the gates in one place, with the plane increment and the populations above. The
|
||||
per-family tables are on [decision-engine.md](decision-engine.md).
|
||||
|
||||
## Removal of the earlier agentic extra
|
||||
|
||||
The removal of the earlier agentic extra (2026-09-30) was checked for byte-identical
|
||||
deterministic outputs (pair replay, quick benchmark, standard scan, `improve --dry-run`):
|
||||
The earlier agentic extra was removed on 2026-09-30. The removal was checked for byte-identical
|
||||
deterministic outputs: pair replay, quick benchmark, standard scan, `improve --dry-run`. Records:
|
||||
`benchmarks/measurements/2026-09-30-harnessx-removal-baseline/` and `-equality/`.
|
||||
|
||||
+28
-17
@@ -1,7 +1,7 @@
|
||||
# End-to-end examples
|
||||
|
||||
Each example is self-contained. Artifacts land under
|
||||
`<target>/.openultrasast/runs/<scan-id>/` and `.openultrasast/` is gitignored.
|
||||
Each example stands on its own. Artifacts land under `<target>/.openultrasast/runs/<scan-id>/`.
|
||||
`.openultrasast/` is gitignored.
|
||||
|
||||
## 1. Quick scan a local repo (deterministic, no keys)
|
||||
|
||||
@@ -21,7 +21,7 @@ ousast scan . --mode standard # Joern model layer, container network off:
|
||||
ousast-with-judge scan . --mode standard # network on, so the model is asked (needs DEEPSEEK_API_KEY)
|
||||
```
|
||||
|
||||
Without a key the graph still decides: its entailed findings are reported, and the LLM's residual
|
||||
Without a key, the graph still decides. Its entailed findings are reported. The LLM's residual
|
||||
and suspicion questions go unasked.
|
||||
|
||||
## 3. CI gate
|
||||
@@ -52,10 +52,11 @@ uv run ousast pairs --split holdout --profile human # filter by declared split
|
||||
uv run python -m openultrasast.pair_gate # CI: local pairs must all pass
|
||||
```
|
||||
|
||||
Each case is two isolated trees (vuln file vs patched file, same relative path).
|
||||
`pair_correct` means the harness fired on the vuln side and stayed silent on the
|
||||
fix. Misses and fix-side leaks become `miss`/`fp` signals for `ousast improve`.
|
||||
The catalog and public dataset pointers live in `benchmarks/pairs/`.
|
||||
Each case is two isolated trees: the vuln file and the patched file, at the same relative path.
|
||||
|
||||
- `pair_correct` means the harness fired on the vuln side and stayed silent on the fix.
|
||||
- Misses become `miss` signals and fix-side leaks become `fp` signals for `ousast improve`.
|
||||
- The catalog and public dataset pointers live in `benchmarks/pairs/`.
|
||||
|
||||
## 6. Self-improve the ruleset from benchmark feedback
|
||||
|
||||
@@ -68,10 +69,11 @@ uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml
|
||||
|
||||
## 7. Running the agentic plane
|
||||
|
||||
Model work over whole repositories runs as a Run manifest on ax, one isolated actor
|
||||
per task (bring-up: [ops/ax/README.md](ops/ax/README.md)). The provider key is
|
||||
read from the environment (or `.env`, which never overrides an exported variable) by
|
||||
`ousast` and sent only in each task's start request.
|
||||
Model work over whole repositories runs as a Run manifest on ax. Each task gets one isolated
|
||||
actor. Bring-up is in [ops/ax/README.md](ops/ax/README.md).
|
||||
|
||||
`ousast` reads the provider key from the environment, or from `.env`. `.env` never overrides an
|
||||
exported variable. The key is sent only in each task's start request.
|
||||
|
||||
```bash
|
||||
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
|
||||
@@ -83,16 +85,25 @@ uv run ousast plane remember validation-46 # re-ingest the run's m
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
|
||||
```
|
||||
|
||||
Each task in the Run carries its own Model and `budget: {usd, calls}`; a task that
|
||||
reaches its ceiling stops as `unfinished` and resumes on a rerun with a larger
|
||||
budget. `ousast plane status` prints the token attribution table: calls, prompt,
|
||||
cache-hit and output tokens and USD per task.
|
||||
Each task in the Run carries its own Model and `budget: {usd, calls}`.
|
||||
|
||||
- A task that reaches its ceiling stops as `unfinished`. It resumes on a rerun with a larger
|
||||
budget.
|
||||
- `ousast plane status` prints the token attribution table. It shows calls, prompt, cache-hit and
|
||||
output tokens, and USD per task.
|
||||
|
||||
## 8. Drive it from an MCP client (OpenCode / IDE)
|
||||
|
||||
MCP (Model Context Protocol) lets an editor or agent call the tool's functions.
|
||||
|
||||
```jsonc
|
||||
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
|
||||
```
|
||||
|
||||
Then: `openultrasast.scan {path}` → `run_dir` → `openultrasast.findings {run_dir}` →
|
||||
`openultrasast.explain {run_dir, finding_id}`. See the README section "Agent integrations".
|
||||
Then call the tools in this order:
|
||||
|
||||
1. `openultrasast.scan {path}` returns a `run_dir`.
|
||||
2. `openultrasast.findings {run_dir}` lists the findings.
|
||||
3. `openultrasast.explain {run_dir, finding_id}` explains one.
|
||||
|
||||
See the README section "Agent integrations".
|
||||
|
||||
+24
-18
@@ -1,13 +1,13 @@
|
||||
# OpenUltraSAST
|
||||
|
||||
OpenUltraSAST is an experimental static security analyser meant to work as a safety net on
|
||||
repositories it has never seen: it scans a checkout, or the commits a `git push` would publish,
|
||||
and separates what it can prove from what it merely suspects. A scan that could not look at
|
||||
something records a degradation instead of reporting a clean result.
|
||||
OpenUltraSAST is an experimental static security analyser. It is meant to be a safety net on
|
||||
repositories it has never seen. It scans a checkout, or the commits a `git push` would publish. It
|
||||
keeps what it can prove apart from what it only suspects. When a scan could not look at
|
||||
something, it records a degradation. It does not report a clean result.
|
||||
|
||||
**State as of 2026-10-02 (v2.0.0):** the pre-push safety net is **NO-GO** for rollout.
|
||||
No hook capability is enabled, and no independent population has passed the qualification gates
|
||||
(see [Evaluation](evaluation.md)).
|
||||
No hook capability is enabled. No independent population (a case set frozen before any scan) has
|
||||
passed the qualification gates (see [Evaluation](evaluation.md)).
|
||||
|
||||
## Three layers
|
||||
|
||||
@@ -17,6 +17,9 @@ No hook capability is enabled, and no independent population has passed the qual
|
||||
| **The agentic plane on ax** | Model-driven work over whole repositories, run as a DAG of tasks on google/ax, one isolated actor per task. What the runs learn is kept in a memory store. Maintainer surface: `ousast plane`. | [The plane on ax](plane.md), [Memory](memory.md) |
|
||||
| **The decision engine** | A learned classifier that weighs every instrument's signals against similar labelled cases from memory. **In development, not adopted**: no scan, pre-push check or report uses it. Maintainer surface: `ousast learn`. | [Memory and detection](memory-and-detection.md), [Decision engine](decision-engine.md) |
|
||||
|
||||
A code property graph is a graph of the program's syntax, control flow and data flow. Joern builds
|
||||
it.
|
||||
|
||||
## How they connect
|
||||
|
||||
```mermaid
|
||||
@@ -33,31 +36,34 @@ flowchart LR
|
||||
engine -. "intended, not adopted: BLOCK / ADVISORY" .-> report
|
||||
```
|
||||
|
||||
- The **scanners** produce findings and, inside plane Runs, the `alerts` that become memory rows.
|
||||
- The **plane** runs model work (verify passes, roles) and model-free bookkeeping (facts, agreement,
|
||||
feature records) per case; its `remember` task turns a case's artifacts into rows of the
|
||||
[memory store](memory.md).
|
||||
- Memory feeds two consumers. `ousast improve --memory` turns stored rows into rule-status
|
||||
proposals that pass through the same validator and gate as every other edit. The decision engine
|
||||
retrieves labelled examples from the same store.
|
||||
- The **scanners** produce findings. Inside plane Runs, they also produce the `alerts` that become
|
||||
memory rows.
|
||||
- The **plane** runs model work per case: verify passes and roles. It also runs model-free
|
||||
bookkeeping per case: facts, agreement and feature records. Its `remember` task turns a case's
|
||||
artifacts into rows of the [memory store](memory.md).
|
||||
- Memory feeds two consumers:
|
||||
1. `ousast improve --memory` turns stored rows into rule-status proposals. They pass through the
|
||||
same validator and gate as every other edit.
|
||||
2. The decision engine retrieves labelled examples from the same store.
|
||||
- The decision engine's output (dotted line) is the intended path. Nothing user-facing reads it
|
||||
today.
|
||||
|
||||
## Where to go next
|
||||
|
||||
- The current state in one page, every figure with its record: capability against the M4 gates, the
|
||||
step-by-step approach, ax on the server, token economics before and after, and the open items:
|
||||
[Where we stand](where-we-stand.md).
|
||||
- The current state in one page, every figure with its record: [Where we stand](where-we-stand.md).
|
||||
It covers capability against the M4 gates, the step-by-step approach, ax on the server, token
|
||||
economics before and after, and the open items.
|
||||
- Run a scan: [Examples](examples.md), and the commands in [Scanning](scanning.md).
|
||||
- Understand a finding's evidence and score: [Architecture](architecture.md).
|
||||
- The program analysis under a scan, what each representation can see and what it misses:
|
||||
[Detection techniques](detection-techniques.md).
|
||||
- Operate the plane and its store: [ax on this host](ops/ax/README.md) and
|
||||
[RustFS setup](rustfs.md).
|
||||
- Where the plane runs today and what a separate Kubernetes cluster would take (planned, not
|
||||
- Where the plane runs today, and what a separate Kubernetes cluster would take (planned, not
|
||||
done): [Deployment](deployment.md).
|
||||
- Where the model spend goes, per Task at runtime and in the maintainers' own coding sessions,
|
||||
and what is reused instead of paid again: [Token ergonomics](token-ergonomics.md).
|
||||
- Trust boundaries and hardening: [Threat model](threat-model.md).
|
||||
|
||||
Every number in these pages is quoted from a committed record whose path is given next to it.
|
||||
Every number in these pages is quoted from a committed record. The record's path is given next to
|
||||
the number.
|
||||
|
||||
+139
-85
@@ -1,11 +1,12 @@
|
||||
# Memory and detection
|
||||
|
||||
How what the system remembers is meant to turn into better decisions on code it has not seen.
|
||||
This is the design of the learned decision engine (`src/openultrasast/learn/`), which is **in
|
||||
development and not adopted**: no scan, pre-push check or report uses it today. Every user-facing
|
||||
decision is still the deterministic one described in [Scanning](scanning.md) and
|
||||
This page shows how the system's memory is meant to improve decisions on code it has not seen. It
|
||||
describes the design of the learned decision engine (`src/openultrasast/learn/`).
|
||||
|
||||
The engine is **in development and not adopted**. No scan, pre-push check or report uses it today.
|
||||
Every user-facing decision is still the deterministic one described in [Scanning](scanning.md) and
|
||||
[Architecture](architecture.md). The engine's commands and status are on
|
||||
[Decision engine](decision-engine.md); the store itself is on [Memory](memory.md).
|
||||
[Decision engine](decision-engine.md). The store itself is on [Memory](memory.md).
|
||||
|
||||
## From instruments to a decision
|
||||
|
||||
@@ -25,89 +26,128 @@ flowchart TD
|
||||
op -- "otherwise" --> adv["ADVISORY"]
|
||||
```
|
||||
|
||||
The intended place for these decisions is changed code: `learn/features.py` builds feature records
|
||||
for a whole-repository scan or for a base..head delta (`build_for_scan` with `Delta`), the same
|
||||
delta `ousast pre-push` checks. Integration into scan and pre-push is still ahead.
|
||||
These decisions are meant for changed code. `learn/features.py` builds feature records in two ways:
|
||||
|
||||
- for a whole-repository scan;
|
||||
- for a base..head delta (`build_for_scan` with `Delta`). This is the same delta `ousast pre-push` checks.
|
||||
|
||||
Integration into scan and pre-push is still ahead.
|
||||
|
||||
## 1. Instruments produce signals, not verdicts
|
||||
|
||||
Each instrument contributes one block to a candidate's feature record (`learn/schema.py`,
|
||||
`INSTRUMENTS`): `language`, `quick` (quick rules), `engine` (the Joern model layer), `facts` (repo
|
||||
facts, such as callers and function span), `source`, `entry_points`, `roles` (model source, sink
|
||||
and sanitizer roles), `delta`, `verify` and `agree` (the plane's verify passes and their
|
||||
agreement), and `model_sinks` (the archived stage-1 classifier, `benchmarks/archive/model_sinks.py`, whose
|
||||
logic now lives in `plane/tasks/roles.py`). A candidate is `(path, function, family)` of one repository and pin
|
||||
where at least one instrument emitted a signal of that family (`learn/features.py`).
|
||||
`INSTRUMENTS`). The blocks are:
|
||||
|
||||
An instrument that did not run, had no coverage or failed records that state (`none`, `failed`,
|
||||
`not_applicable`), never a zero, so "silent" and "did not look" stay distinguishable. None of the
|
||||
instruments decides alone.
|
||||
- `language`;
|
||||
- `quick` (quick rules);
|
||||
- `engine` (the Joern model layer);
|
||||
- `facts` (repo facts, such as callers and function span);
|
||||
- `source`;
|
||||
- `entry_points`;
|
||||
- `roles` (model source, sink and sanitizer roles);
|
||||
- `delta`;
|
||||
- `verify` and `agree` (the plane's verify passes and their agreement);
|
||||
- `model_sinks` (the archived stage-1 classifier, `benchmarks/archive/model_sinks.py`). Its logic now
|
||||
lives in `plane/tasks/roles.py`.
|
||||
|
||||
**Why profile v1 withholds the engine and function length.** A leak audit
|
||||
(`ousast learn audit-leaks`, `benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`)
|
||||
found that both separate the vulnerable and fixed sides of a pair by construction: a fix removes
|
||||
a sink, and a guard lengthens the function. That holds on the pair corpus, not on unseen code, so
|
||||
`INPUT_PROFILES["v1"]` drops the `engine` block and `facts.function_lines` from prompts and from
|
||||
the retrieval distance.
|
||||
A candidate is `(path, function, family)` of one repository and pin. At least one instrument must have
|
||||
emitted a signal of that family there (`learn/features.py`).
|
||||
|
||||
An instrument may not run, may have no coverage, or may fail. It then records that state (`none`,
|
||||
`failed`, `not_applicable`), never a zero. So "silent" and "did not look" stay distinguishable. No
|
||||
instrument decides alone.
|
||||
|
||||
**Why profile v1 withholds the engine and function length.** A leak audit found that both separate the
|
||||
vulnerable and fixed sides of a pair by construction (`ousast learn audit-leaks`,
|
||||
`benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`):
|
||||
|
||||
- a fix removes a sink;
|
||||
- a guard lengthens the function.
|
||||
|
||||
That holds on the pair corpus, not on unseen code. So `INPUT_PROFILES["v1"]` drops the `engine` block
|
||||
and `facts.function_lines` from prompts and from the retrieval distance.
|
||||
|
||||
## 2. Labels come from ground truth only
|
||||
|
||||
`ousast learn labels` (`learn/labels.py`) reads only the sources on a fail-closed allow-list
|
||||
(`learn/sources.toml`): `population-v1`, `population-v2`, `dev-php` (reviewed recipe sites),
|
||||
`pairs`, `advisory-fixes`, `adjudications` and `assumed_benign`. A file the list does not name is
|
||||
refused before it is opened; the reserved population v3 is excluded unread.
|
||||
(`learn/sources.toml`). The sources are `population-v1`, `population-v2`, `dev-php` (reviewed recipe
|
||||
sites), `pairs`, `advisory-fixes`, `adjudications` and `assumed_benign`. A file the list does not name
|
||||
is refused before it is opened. The reserved population v3 is excluded unread.
|
||||
|
||||
- **Positives** are declared sites and the vulnerable side of a fix; **negatives** are the same
|
||||
function on the fixed side where the fix changed it, and recorded `False`/`FP` adjudications.
|
||||
A vulnerable-pin candidate matching no site is unlabelled, never negative.
|
||||
- **Assumed benign** (maintainer decision 2026-09-30): ordinary non-security commits of the training
|
||||
repositories, at least 90 days from any fix, with no later security commit touching the same
|
||||
files. They are their own source with weight 0.5, conditional on a signal at the tip and none at
|
||||
the base, and reported separately, never merged with verified negatives.
|
||||
- **Plane verdicts are features, not labels.** "Agreed", "rejected" or "disputed" is the output of
|
||||
the verify instrument; it enters the record as the `verify` and `agree` blocks. Using it as a
|
||||
label would teach the engine to reproduce the plane, mistakes included, so "rejected by the
|
||||
plane" is never a negative (`learn/labels.py`: "A plane verdict ... is never a label").
|
||||
- **Positives** are declared sites and the vulnerable side of a fix.
|
||||
- **Negatives** are two kinds:
|
||||
- the same function on the fixed side, where the fix changed it;
|
||||
- recorded `False`/`FP` adjudications.
|
||||
- A vulnerable-pin candidate matching no site is unlabelled, never negative.
|
||||
- **Assumed benign** (maintainer decision 2026-09-30) covers ordinary non-security commits of the
|
||||
training repositories. Each is at least 90 days from any fix. No later security commit touches the
|
||||
same files.
|
||||
- They are their own source with weight 0.5, conditional on a signal at the tip and none at the base.
|
||||
- They are reported separately, never merged with verified negatives.
|
||||
- **Plane verdicts are features, not labels.** "Agreed", "rejected" or "disputed" is the output of the
|
||||
verify instrument. It enters the record as the `verify` and `agree` blocks. Using it as a label would
|
||||
teach the engine to reproduce the plane, mistakes included. So "rejected by the plane" is never a
|
||||
negative (`learn/labels.py`: "A plane verdict ... is never a label").
|
||||
|
||||
`ousast learn memory build` joins each label with the candidate's feature record and an excerpt of
|
||||
its function and stores one `example` row (no rationale) plus the excerpt blob.
|
||||
its function. It stores one `example` row (no rationale) plus the excerpt blob.
|
||||
|
||||
## 3. Retrieval of similar labelled cases
|
||||
|
||||
`learn/retrieve.py`, no model:
|
||||
`learn/retrieve.py` uses no model. It works in three steps:
|
||||
|
||||
1. **Signal profile.** A Gower distance over the allow-listed features of the active profile,
|
||||
per instrument block, with missing values explicit (a different state counts as distance 1).
|
||||
The nearest 40 examples of the candidate's family and the nearest 40 of other families go on.
|
||||
2. **Code embeddings.** A cosine re-rank of the excerpts' embeddings (OpenAI
|
||||
`text-embedding-3-small` through OpenRouter, 1,536 dimensions, cached by excerpt hash):
|
||||
`r = 0.5 (1 - D) + 0.5 cos`. Without the candidate's vector the retrieval is signals only.
|
||||
3. **Balance.** Six examples: at most half per label, at most 2 per repository group, at most 2
|
||||
contrast examples from other families.
|
||||
1. **Signal profile.** It computes a Gower distance (a distance for mixed-type features) over the
|
||||
allow-listed features of the active profile, per instrument block. Missing values are explicit: a
|
||||
different state counts as distance 1. The nearest 40 examples of the candidate's family go on. So do
|
||||
the nearest 40 of other families.
|
||||
2. **Code embeddings.** It re-ranks by cosine over the excerpts' embeddings. The embeddings come from
|
||||
OpenAI `text-embedding-3-small` through OpenRouter (1,536 dimensions, cached by excerpt hash). The
|
||||
formula is `r = 0.5 (1 - D) + 0.5 cos`. Without the candidate's vector, the retrieval uses signals only.
|
||||
3. **Balance.** It picks six examples:
|
||||
- at most half per label;
|
||||
- at most 2 per repository group;
|
||||
- at most 2 contrast examples from other families.
|
||||
|
||||
**The evaluation boundary.** `eligible()` is the only way an example enters a prompt, for retrieved
|
||||
examples and compiled demonstrations alike: never the candidate's own repository group, never a
|
||||
group the fold evaluates, never the held-out source or framework, and (outside deployment) never
|
||||
a near-duplicate of the candidate (cosine >= 0.98). The program repeats the check on the ids it
|
||||
actually rendered before any call.
|
||||
**The evaluation boundary.** `eligible()` is the only way an example enters a prompt. This holds for
|
||||
retrieved examples and compiled demonstrations alike. An eligible example is never:
|
||||
|
||||
- from the candidate's own repository group;
|
||||
- from a group the fold evaluates;
|
||||
- from the held-out source or framework;
|
||||
- a near-duplicate of the candidate (cosine >= 0.98), outside deployment.
|
||||
|
||||
The program repeats the check on the ids it actually rendered, before any call.
|
||||
|
||||
## 4. The AI classifier, compiled
|
||||
|
||||
`learn/program.py` and `learn/compile.py`, DSPy-style and in-house. The prompt is the instruction
|
||||
plus the compiled demonstrations (a prefix identical for every candidate, so the provider's prefix
|
||||
cache hits), then the retrieved examples, then the candidate's excerpt, signals, roles and family.
|
||||
The model (`deepseek-flash`) answers `vulnerable`, `not_vulnerable` or `unsure` with a confidence
|
||||
and a rationale that must cite a line of the code. It is sampled `k` times; the score `s` is the
|
||||
mean over samples, and a majority `unsure` can never BLOCK. Compilation bootstraps demonstrations
|
||||
and searches instructions on a compile split of repository groups, scored by a pre-registered
|
||||
balanced Brier score (`learn/compile.toml`); the program is stored as `programs/<id>.json`.
|
||||
The classifier lives in `learn/program.py` and `learn/compile.py`. It is DSPy-style (DSPy is a
|
||||
framework that compiles prompts from examples) and built in-house.
|
||||
|
||||
The prompt has this order:
|
||||
|
||||
1. the instruction plus the compiled demonstrations. This prefix is identical for every candidate, so
|
||||
the provider's prefix cache hits;
|
||||
2. the retrieved examples;
|
||||
3. the candidate's excerpt, signals, roles and family.
|
||||
|
||||
The model (`deepseek-flash`) answers `vulnerable`, `not_vulnerable` or `unsure`. It adds a confidence
|
||||
and a rationale. The rationale must cite a line of the code. The model is sampled `k` times. The score
|
||||
`s` is the mean over samples. A majority `unsure` can never BLOCK.
|
||||
|
||||
Compilation bootstraps demonstrations and searches instructions on a compile split of repository groups.
|
||||
A pre-registered balanced Brier score scores it (`learn/compile.toml`). The program is stored as
|
||||
`programs/<id>.json`.
|
||||
|
||||
## 5. Calibration and the two operating points
|
||||
|
||||
`learn/calibrate.py`: Platt scaling of `s`, cross-fitted so that no map is checked on the fold it
|
||||
was fitted on. Calibration **holds** when the slope interval contains 1, the intercept interval
|
||||
contains 0 and ECE <= 0.05. **BLOCK** is offered only when calibration holds and a fold reaches the
|
||||
required precision; otherwise the engine offers **ADVISORY** only.
|
||||
`learn/calibrate.py` applies Platt scaling (a logistic map from score to probability) to `s`. It is
|
||||
cross-fitted, so no map is checked on the fold it was fitted on.
|
||||
|
||||
Calibration **holds** when the slope interval contains 1, the intercept interval contains 0 and ECE
|
||||
(expected calibration error) <= 0.05.
|
||||
|
||||
**BLOCK** is offered only when calibration holds and a fold reaches the required precision. Otherwise
|
||||
the engine offers **ADVISORY** only.
|
||||
|
||||
## The loop: memory improves decisions without retraining
|
||||
|
||||
@@ -125,25 +165,36 @@ flowchart LR
|
||||
gate -- "accepted round" --> ledger["Rule status in the ledger"]
|
||||
```
|
||||
|
||||
- **Decisions.** The compiled part of the program is its instruction and demonstrations; the
|
||||
examples are retrieved from memory for each candidate. A new `example` row is therefore eligible
|
||||
for retrieval in the next decision without recompiling the program, subject to the same
|
||||
evaluation boundary. `ousast learn curve` measures the metric over memory size on a fixed
|
||||
candidate subset.
|
||||
- **Rules.** `ousast improve --memory` reads the run rows (alerts, verdicts, coverage) and proposes
|
||||
rule-status edits only (shadow a rule that is repeatedly false across repositories; re-enable a
|
||||
shadow rule that keeps hitting agreed declared sites). Rows from holdout pairs, the gated
|
||||
manifest's own cases and any `--qualify-population` are dropped first, and every proposal goes
|
||||
through the same validator and gate as any other edit (`improve/memory.py`).
|
||||
- **Decisions.**
|
||||
- The compiled part of the program is its instruction and demonstrations.
|
||||
- The examples are retrieved from memory for each candidate.
|
||||
- So a new `example` row is eligible for retrieval in the next decision without recompiling. The same
|
||||
evaluation boundary applies.
|
||||
- `ousast learn curve` measures the metric over memory size on a fixed candidate subset.
|
||||
- **Rules.** `ousast improve --memory` reads the run rows (alerts, verdicts, coverage). It proposes
|
||||
rule-status edits only:
|
||||
- shadow a rule that is repeatedly false across repositories;
|
||||
- re-enable a shadow rule that keeps hitting agreed declared sites.
|
||||
|
||||
Some rows are dropped first: rows from holdout pairs, the gated manifest's own cases and any
|
||||
`--qualify-population`. Every proposal goes through the same validator and gate as any other edit
|
||||
(`improve/memory.py`).
|
||||
|
||||
## What the first slice measured
|
||||
|
||||
Record: `benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json` (counts
|
||||
only). Family **injection**; memory of 724 static examples from all families; input profile v1;
|
||||
framework priors off; out-of-repository evaluation (compile split never evaluated, outer grouped
|
||||
5-fold, calibration cross-fitted, operating points nested); 95% repository-cluster bootstrap
|
||||
intervals. The slice's spend ($0.003215 per candidate at evaluation, k = 5, and its replay at
|
||||
$0) is on [Token ergonomics](token-ergonomics.md#measured-costs).
|
||||
only). The setup:
|
||||
|
||||
- family **injection**;
|
||||
- memory of 724 static examples from all families;
|
||||
- input profile v1;
|
||||
- framework priors off;
|
||||
- out-of-repository evaluation: compile split never evaluated, outer grouped 5-fold, calibration
|
||||
cross-fitted, operating points nested;
|
||||
- 95% repository-cluster bootstrap intervals.
|
||||
|
||||
The slice's spend is on [Token ergonomics](token-ergonomics.md#measured-costs). It was $0.003215 per
|
||||
candidate at evaluation, k = 5, and its replay cost $0.
|
||||
|
||||
| Measure | Value | What it means |
|
||||
| --- | --- | --- |
|
||||
@@ -152,9 +203,12 @@ $0) is on [Token ergonomics](token-ergonomics.md#measured-costs).
|
||||
| ADVISORY | recall 0.90 (0.82 - 0.97), precision 0.54 (0.44 - 0.65) | precision is at the positive share (0.53), so at this threshold ADVISORY is **not discriminating**: it flags nearly as indiscriminately as flagging everything would |
|
||||
| BLOCK | not offered | calibration does not hold (ECE 0.064 > 0.05), and every outer fold reported `precision_unreachable` |
|
||||
|
||||
**The engine is not adopted.** A second slice (2026-10-02) evaluated the six evaluable families
|
||||
the same way: pooled AUC 0.77 to 0.96 per family, within-pair AUC 0.72 to 0.93, BLOCK offered for
|
||||
none, re-run agreement below 0.9 for three families
|
||||
([Decision engine](decision-engine.md#second-slice-six-families-2026-10-02)). Leave-one-source-out
|
||||
and leave-one-framework-out, integration into scan and pre-push, and the one-time check on
|
||||
population v3 are still ahead.
|
||||
**The engine is not adopted.** A second slice (2026-10-02) evaluated the six evaluable families the
|
||||
same way ([Decision engine](decision-engine.md#second-slice-six-families-2026-10-02)). The results: pooled AUC 0.77 to 0.96 per family, within-pair AUC 0.72 to 0.93, BLOCK offered for
|
||||
none. Re-run agreement was below 0.9 for three families.
|
||||
|
||||
Still ahead:
|
||||
|
||||
- leave-one-source-out and leave-one-framework-out;
|
||||
- integration into scan and pre-push;
|
||||
- the one-time check on population v3.
|
||||
|
||||
+50
-40
@@ -1,8 +1,8 @@
|
||||
# Memory
|
||||
|
||||
One store, keyed by repository and pin, holds what plane runs and the decision engine learned.
|
||||
The code is `src/openultrasast/plane/memory.py`; the same layout serves both backends.
|
||||
How the stored rows feed detection is on [Memory and detection](memory-and-detection.md).
|
||||
One store holds what plane runs and the decision engine learned. It is keyed by repository and pin
|
||||
(a fixed commit). The code is `src/openultrasast/plane/memory.py`. The same layout serves both backends.
|
||||
[Memory and detection](memory-and-detection.md) shows how the stored rows feed detection.
|
||||
|
||||
## What is stored
|
||||
|
||||
@@ -17,7 +17,7 @@ programs/<program id>.json compiled decision-engine programs
|
||||
```
|
||||
|
||||
**Rows.** Every row carries `id`, `kind`, `repo`, `pin`, `run`, `task`, `population`, `split`
|
||||
and `image` (`ROW_FIELDS`); `id` is the sha256 of (kind, run, task, subject), so a repeated row
|
||||
and `image` (`ROW_FIELDS`). `id` is the sha256 of (kind, run, task, subject). So a repeated row
|
||||
replaces itself. The kinds (`KINDS`) and who writes them:
|
||||
|
||||
| Kind | Written by | Holds |
|
||||
@@ -35,12 +35,15 @@ replaces itself. The kinds (`KINDS`) and who writes them:
|
||||
| `experiment_result` | `ousast learn experiment analyse --record` | the paired estimates, the looks and the adoption verdict (counts and intervals only) |
|
||||
| `label`, `decision` | reserved for later decision-engine tasks | declared in `KINDS`; not written by the code on main yet |
|
||||
|
||||
**Blobs** (`put_blob`) are content-addressed: a 64-hex name under a fixed prefix. Excerpts are
|
||||
written by `learn memory build` (`learn/examples.py`), embeddings by `learn memory embed`
|
||||
(`learn/embeddings.py`, keyed by excerpt sha so an excerpt is embedded once), responses by the
|
||||
program's caller (`learn/program.py`, keyed by the sha256 of model, parameters digest, messages,
|
||||
temperature, sample and format, so a rerun replays at no cost), and compiled programs by
|
||||
`learn compile` (`learn/compile.py`).
|
||||
**Blobs** (`put_blob`) are content-addressed: each has a 64-hex name under a fixed prefix.
|
||||
Content-addressed means the name is the hash of the content. Who writes each blob:
|
||||
|
||||
- Excerpts: `learn memory build` (`learn/examples.py`).
|
||||
- Embeddings: `learn memory embed` (`learn/embeddings.py`). They are keyed by excerpt sha, so an
|
||||
excerpt is embedded once.
|
||||
- Responses: the program's caller (`learn/program.py`). The key is the sha256 of model, parameters
|
||||
digest, messages, temperature, sample and format. So a rerun replays at no cost.
|
||||
- Compiled programs: `learn compile` (`learn/compile.py`).
|
||||
|
||||
## Write path
|
||||
|
||||
@@ -60,13 +63,14 @@ flowchart LR
|
||||
put --> store[("Memory store")]
|
||||
```
|
||||
|
||||
- `ingest` validates every row first: a malformed file fails before anything is written.
|
||||
- Rows are merged into their repository + pin object by `id`. Every object is written whole and the
|
||||
index last, so an interrupted ingest leaves nothing a repeat would not repair.
|
||||
- `ingest` validates every row first. A malformed file fails before anything is written.
|
||||
- Rows are merged by `id` into their repository + pin object.
|
||||
- Every object is written whole, and the index last. So a repeat repairs anything an interrupted
|
||||
ingest left behind.
|
||||
- Each row object carries tags (`TAG_FIELDS`: `repo`, `pin`, `kind`, `family`, `run`,
|
||||
`population`, `split`) listing the distinct values of its rows (`*` when they exceed 256
|
||||
characters).
|
||||
- The plane's store failures never change a run's result: an ingest can be repeated by hand
|
||||
`population`, `split`). The tags list the distinct values of its rows. A tag is `*` when the
|
||||
values exceed 256 characters.
|
||||
- The plane's store failures never change a run's result. An ingest can be repeated by hand
|
||||
(`cli.py`, `_plane_memory`).
|
||||
|
||||
## Read path
|
||||
@@ -85,14 +89,14 @@ flowchart LR
|
||||
rules --> gate{"Unchanged validator and gate"}
|
||||
```
|
||||
|
||||
- **Seed** (`memory.seed`): a Run task whose `openultrasast.io/memory-key` matches stored facts gets
|
||||
them and is marked done; a changed pin, candidate set or runner image finds nothing and
|
||||
recomputes. The loop's `memory-snapshot` is seeded the same way, because a sandboxed task cannot
|
||||
read the store.
|
||||
- **Seed** (`memory.seed`).
|
||||
- A Run task gets stored facts when its `openultrasast.io/memory-key` matches them. It is then marked done.
|
||||
- A changed pin, candidate set or runner image finds nothing and recomputes.
|
||||
- The loop's `memory-snapshot` is seeded the same way. A sandboxed task cannot read the store.
|
||||
- **Retrieval** (`learn/retrieve.py`): see
|
||||
[Memory and detection](memory-and-detection.md#3-retrieval-of-similar-labelled-cases).
|
||||
- **`improve --memory`** (`improve/memory.py`): rows from holdout pairs, from the gated manifest's
|
||||
own cases and from any `--qualify-population` are dropped before any rule sees them. See
|
||||
- **`improve --memory`** (`improve/memory.py`). Some rows are dropped before any rule sees them:
|
||||
rows from holdout pairs, from the gated manifest's own cases, and from any `--qualify-population`. See
|
||||
[Architecture](architecture.md#proposals-from-plane-memory).
|
||||
|
||||
## Backends: FileStore and the S3 store
|
||||
@@ -109,26 +113,32 @@ flowchart LR
|
||||
|
||||
### Verify at startup, no fallback
|
||||
|
||||
The S3 store never configures its bucket. Each time an `S3Store` is opened for a bucket and
|
||||
prefix (once per process), `verify_bucket()` checks, using reads plus one small probe object at
|
||||
`<prefix>/_probe/select.jsonl`:
|
||||
The S3 store never configures its bucket. It only checks it. An `S3Store` is opened for a bucket and
|
||||
prefix once per process. Each time, `verify_bucket()` runs these checks. It uses reads plus one small
|
||||
probe object at `<prefix>/_probe/select.jsonl`.
|
||||
|
||||
1. versioning is `Enabled` (provenance cites object versions);
|
||||
2. an enabled lifecycle rule expires `<prefix>/runs/` after a number of days, as a plain prefix
|
||||
filter, and no enabled rule expires the whole store;
|
||||
3. S3 Select answers a probe query over JSON Lines;
|
||||
4. the probe's object tags are readable.
|
||||
1. Versioning is `Enabled`. Provenance cites object versions.
|
||||
2. An enabled lifecycle rule expires `<prefix>/runs/` after a number of days. It uses a plain prefix
|
||||
filter. No enabled rule expires the whole store.
|
||||
3. S3 Select (a server-side query over JSON objects) answers a probe query over JSON Lines.
|
||||
4. The probe's object tags are readable.
|
||||
|
||||
Anything missing is named, with the admin command that fixes it, in one `MemoryStoreError`, and
|
||||
the store does not start. There is no local or fetch-and-filter fallback: a filtered read that
|
||||
S3 Select cannot answer raises a `MemoryStoreError` instead of returning "no rows". The exact
|
||||
messages are listed on [RustFS setup](rustfs.md#what-the-store-verifies-at-startup).
|
||||
If anything is missing, the store does not start. One `MemoryStoreError` names each problem and the
|
||||
admin command that fixes it.
|
||||
|
||||
There is no local or fetch-and-filter fallback. A filtered read that S3 Select cannot answer raises a
|
||||
`MemoryStoreError`. It does not return "no rows". [RustFS setup](rustfs.md#what-the-store-verifies-at-startup)
|
||||
lists the exact messages.
|
||||
|
||||
### Queryable fields per kind
|
||||
|
||||
RustFS's Select infers an object's JSON schema from its leading rows, so a `where` field missing
|
||||
there fails with `EvaluatorBindingDoesNotExist` even when a later row has it, and a column that is
|
||||
null in the first 1000 rows breaks Select on the whole object (measured 2026-10-02; `memory.py`,
|
||||
`S3Store._select`). The store therefore writes every row with its kind's full set of queryable
|
||||
fields (`QUERY_FIELDS`), using `""` where a field does not apply, and refuses a `where` on an
|
||||
undeclared field. Existing rows are rewritten once with `ousast plane memory-normalise`.
|
||||
RustFS's Select infers an object's JSON schema from its leading rows. This causes two failures
|
||||
(measured 2026-10-02; `memory.py`, `S3Store._select`):
|
||||
|
||||
- A `where` field missing from the leading rows fails with `EvaluatorBindingDoesNotExist`. This
|
||||
happens even when a later row has the field.
|
||||
- A column that is null in the first 1000 rows breaks Select on the whole object.
|
||||
|
||||
So the store writes every row with its kind's full set of queryable fields (`QUERY_FIELDS`). It uses
|
||||
`""` where a field does not apply. It refuses a `where` on an undeclared field. Existing rows are
|
||||
rewritten once with `ousast plane memory-normalise`.
|
||||
|
||||
+83
-57
@@ -1,23 +1,34 @@
|
||||
# The plane on ax
|
||||
|
||||
Model-driven work over whole repositories runs on a separate agentic plane: a **Run**
|
||||
(`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG of steps, and each step is
|
||||
an ax **Task** (`plane/tasks/`) that binds its Workspaces (`plane/workspaces/`), at most one ax
|
||||
**Model** (`plane/models/`: `deepseek-flash` for chat, `openrouter-embedding` for embeddings) and
|
||||
its own `budget: {usd, calls}`. ax (google/ax over Agent Substrate, in a single-node kind cluster
|
||||
on the maintainer's host) is the only executor; there is no local subprocess path. Bring-up and
|
||||
the host's lessons are in [ax on this host](ops/ax/README.md); where each part runs today (the
|
||||
reconciler and the receiver on the host, the tasks in gVisor on Agent Substrate) and what a
|
||||
separate Kubernetes cluster would take (planned) are on [Deployment](deployment.md); this page
|
||||
draws the flow.
|
||||
Model-driven work over whole repositories runs on a separate agentic plane. The plane has these parts:
|
||||
|
||||
- A **Run** (`plane/runs/*.yaml`, kind `openultrasast.io/v1alpha1 Run`) is a DAG (a graph of steps
|
||||
with no cycles).
|
||||
- Each step is an ax **Task** (`plane/tasks/`).
|
||||
- A Task binds its Workspaces (`plane/workspaces/`).
|
||||
- A Task binds at most one ax **Model** (`plane/models/`). `deepseek-flash` serves chat.
|
||||
`openrouter-embedding` serves embeddings.
|
||||
- A Task carries its own `budget: {usd, calls}`.
|
||||
|
||||
ax is the only executor. It is google/ax, Google's controller that runs agent tasks, over Agent Substrate (its
|
||||
sandboxed actor runtime). It runs in a single-node kind cluster (Kubernetes in Docker) on the maintainer's host.
|
||||
There is no local subprocess path.
|
||||
|
||||
Where to read more:
|
||||
|
||||
- Bring-up and the host's lessons are in [ax on this host](ops/ax/README.md).
|
||||
- [Deployment](deployment.md) shows where each part runs today. The reconciler and the receiver run on the host.
|
||||
The tasks run in gVisor (a sandboxing kernel) on Agent Substrate. It also covers what a separate Kubernetes
|
||||
cluster would take (planned).
|
||||
- This page draws the flow.
|
||||
|
||||
Sources: `src/openultrasast/plane/reconciler.py`, `router.py`, `egress.py`, `runner.py`,
|
||||
`generate.py`, `memory.py`, and `src/openultrasast/plane/tasks/`.
|
||||
|
||||
## One task, end to end
|
||||
|
||||
`ousast plane run RUN.yaml` (the reconciler) does this for every ready task; `--workers` bounds
|
||||
the tasks in flight, and a rerun skips tasks already done.
|
||||
`ousast plane run RUN.yaml` (the reconciler) does this for every ready task. `--workers` bounds the
|
||||
tasks in flight. A rerun skips tasks already done.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
@@ -48,44 +59,55 @@ sequenceDiagram
|
||||
|
||||
What each step guarantees:
|
||||
|
||||
- **Credentials** (`router.py`, `runner.py`). The reconciler reads the variable the Model's
|
||||
`secretKey.key` names from the operator's environment or `.env` (which never overrides an
|
||||
exported variable). The key exists only in the start request; the runner keeps it in memory,
|
||||
passes it to the command's environment, redacts it from echoed stderr and never writes it.
|
||||
- **Egress** (`egress.py`). One EgressPolicy per actor, deny by default, hostnames only: plain
|
||||
HTTP to the artifact receiver, TLS passthrough to the source hosts of the bound Workspaces and
|
||||
to the hosts the bound Model declares (`openultrasast.io/egress-hosts`). It is written between
|
||||
`ax apply` and the resume, and deleted with the actor.
|
||||
- **Inputs** (`reconciler.py`). Artifacts other tasks produced are never rendered into the
|
||||
manifest; the runner fetches them from the receiver after the start, and the receiver serves a
|
||||
running task its declared inputs only.
|
||||
- **Completion** (`runner.py`). ax has no Completed phase: the runner's delivery of the output tar
|
||||
is the task's completion. A completion marker stops a resumed actor from repeating billed work;
|
||||
when the marker says the delivery failed, a later boot retries the delivery alone.
|
||||
- **Budgets** (`plane/budget.py`). The metered client refuses the next call once a task's `usd` or
|
||||
`calls` ceiling is reached; the task ends `unfinished` and resumes on a rerun with a larger
|
||||
budget. Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
|
||||
- **Attribution.** `ousast plane status RUN` prints per task its status, `calls`, `prompt`,
|
||||
`cache_hit` and `output` tokens and `usd` from the tasks' `summary.json`, and writes
|
||||
`attribution.json` in the run directory (`$OUSAST_RESULTS/plane/RUN/`, default
|
||||
`~/ousast-results/plane/`).
|
||||
- **Credentials** (`router.py`, `runner.py`).
|
||||
- The reconciler reads the variable that the Model's `secretKey.key` names.
|
||||
- It reads it from the operator's environment or `.env`. `.env` never overrides an exported variable.
|
||||
- The key exists only in the start request.
|
||||
- The runner keeps it in memory and passes it to the command's environment.
|
||||
- The runner redacts it from echoed stderr and never writes it.
|
||||
- **Egress** (`egress.py`). Egress means outbound network traffic.
|
||||
- Each actor gets one EgressPolicy. It denies by default and names hostnames only.
|
||||
- It allows plain HTTP to the artifact receiver.
|
||||
- It allows TLS passthrough to the source hosts of the bound Workspaces.
|
||||
- It allows TLS passthrough to the hosts the bound Model declares (`openultrasast.io/egress-hosts`).
|
||||
- The reconciler writes it between `ax apply` and the resume. It is deleted with the actor.
|
||||
- **Inputs** (`reconciler.py`).
|
||||
- Artifacts that other tasks produced are never rendered into the manifest.
|
||||
- The runner fetches them from the receiver after the start.
|
||||
- The receiver serves a running task its declared inputs only.
|
||||
- **Completion** (`runner.py`).
|
||||
- ax has no Completed phase. The runner's delivery of the output tar is the task's completion.
|
||||
- A completion marker stops a resumed actor from repeating billed work.
|
||||
- If the marker says the delivery failed, a later boot retries only the delivery.
|
||||
- **Budgets** (`plane/budget.py`).
|
||||
- The metered client refuses the next call once a task reaches its `usd` or `calls` ceiling.
|
||||
- The task then ends `unfinished`. It resumes on a rerun with a larger budget.
|
||||
- Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
|
||||
- **Attribution.**
|
||||
- `ousast plane status RUN` prints these per task: status, `calls`, `prompt`, `cache_hit` and
|
||||
`output` tokens, and `usd`. It reads them from the tasks' `summary.json`.
|
||||
- It writes `attribution.json` in the run directory (`$OUSAST_RESULTS/plane/RUN/`, default
|
||||
`~/ousast-results/plane/`).
|
||||
|
||||
!!! note "The golden snapshot"
|
||||
Agent Substrate boots every new actor template once as a *golden* actor to snapshot it, with
|
||||
the same image, Task and environment, and inside the sandbox nothing tells that boot from the
|
||||
real one. So the runner's command never starts by itself: it waits for the start request,
|
||||
which the reconciler sends through `atenet-router` to the task's actor, never to the golden
|
||||
one. Workspaces are prepared only after the start, so the golden boot touches no network and a
|
||||
started actor already runs under its egress policy (a clone at boot failed on the live cluster
|
||||
while the actor was being restored, 2026-09-29; `runner.py`, `_run_once`). The first resume of
|
||||
a new image can time out while the snapshot is built, which is why the reconciler retries it
|
||||
Agent Substrate boots every new actor template once as a *golden* actor and snapshots it. That boot
|
||||
uses the same image, Task and environment. Inside the sandbox, nothing tells that boot from the real one.
|
||||
|
||||
So the runner's command never starts by itself. It waits for the start request. The reconciler sends
|
||||
that request through `atenet-router` to the task's actor, never to the golden one.
|
||||
|
||||
Workspaces are prepared only after the start. So the golden boot touches no network. A started actor
|
||||
already runs under its egress policy. A clone at boot once failed on the live cluster while the actor
|
||||
was being restored (2026-09-29; `runner.py`, `_run_once`).
|
||||
|
||||
The first resume of a new image can time out while the snapshot is built. So the reconciler retries it
|
||||
for up to `OUSAST_RESUME_TIMEOUT` seconds (900).
|
||||
|
||||
## The Runs: validation and loop
|
||||
|
||||
`ousast plane workspaces POPULATION --validation-set ...` (`generate.py`) writes one chain per
|
||||
case; `--loop` adds the `alerts` step per case and the loop's singleton steps. Every step except
|
||||
the verify passes (and `roles` in harvest Runs) is model-free.
|
||||
`ousast plane workspaces POPULATION --validation-set ...` (`generate.py`) writes one chain per case.
|
||||
`--loop` adds the `alerts` step per case and the loop's singleton steps. Every step is model-free
|
||||
except the verify passes (and `roles` in harvest Runs).
|
||||
|
||||
### Per case
|
||||
|
||||
@@ -119,23 +141,27 @@ flowchart LR
|
||||
improve --> outcome["gate.json, journal.json, and rule_policy.json only if accepted"]
|
||||
```
|
||||
|
||||
- **Before the Run** `ousast plane run` seeds from the store (`memory.seed`): a task whose
|
||||
`openultrasast.io/memory-key` (repository, pin, candidates digest, runner image digest) has
|
||||
stored facts gets them and is marked done, and the sandboxed `memory-snapshot` task gets the
|
||||
store's rows after the guard. **After the Run** it ingests every delivered `memory.jsonl`
|
||||
(`ousast plane remember RUN` repeats this by hand). See [Memory](memory.md).
|
||||
- The loop's rules and its guard are described in
|
||||
[Architecture](architecture.md#proposals-from-plane-memory). Adopting an accepted ledger stays a
|
||||
maintainer commit (`src/openultrasast/plane/tasks/loop.py`).
|
||||
- **Before the Run**, `ousast plane run` seeds from the store (`memory.seed`).
|
||||
- Each task has an `openultrasast.io/memory-key`: repository, pin, candidates digest and runner image digest.
|
||||
- If the store has facts for that key, the task gets them and is marked done.
|
||||
- The sandboxed `memory-snapshot` task gets the store's rows after the guard.
|
||||
- **After the Run**, it ingests every delivered `memory.jsonl`. `ousast plane remember RUN` repeats
|
||||
this by hand. See [Memory](memory.md).
|
||||
- [Architecture](architecture.md#proposals-from-plane-memory) describes the loop's rules and its guard.
|
||||
Adopting an accepted ledger stays a maintainer commit (`src/openultrasast/plane/tasks/loop.py`).
|
||||
- `ousast plane harvest` writes the decision engine's harvest Runs (verify a/b with agree, and
|
||||
model `roles`); `ousast plane alerts-engine` produces a Run's `alerts` from the Joern engine
|
||||
image on the host for PHP and other languages quick mode does not cover.
|
||||
model `roles`).
|
||||
- `ousast plane alerts-engine` produces a Run's `alerts` from the Joern engine image on the host. It
|
||||
covers PHP and other languages that quick mode does not cover.
|
||||
|
||||
## Where this runs in production
|
||||
|
||||
ax is a Kubernetes application over Agent Substrate, every Task runs the one runner image as an actor that
|
||||
occupies a whole worker, and two profiles of this code base are the target: `kind` on a laptop (today) and `k3s`
|
||||
in production (planned), where the same CLI submits Runs remotely and the tasks deliver to the S3 store:
|
||||
ax is a Kubernetes application over Agent Substrate. Every Task runs the one runner image as an actor. Each actor
|
||||
occupies a whole worker. This code base targets two profiles:
|
||||
|
||||
- `kind` on a laptop (today).
|
||||
- `k3s` in production (planned). There the same CLI submits Runs remotely, and the tasks deliver to the S3 store.
|
||||
|
||||
[Production topology: ax on a Kubernetes cluster](deployment.md#production-topology-ax-on-a-kubernetes-cluster)
|
||||
draws it and sizes the pre-push hook at scale.
|
||||
|
||||
|
||||
+74
-50
@@ -1,16 +1,22 @@
|
||||
# RustFS setup for the memory store
|
||||
|
||||
The plane's memory store can live in an S3-compatible bucket: `OUSAST_MEMORY=s3://sast-memory`
|
||||
selects `S3Store` (`src/openultrasast/plane/memory.py`), which talks to the server through boto3.
|
||||
Any S3-compatible store with bucket versioning, lifecycle rules, object tags and S3 Select works;
|
||||
RustFS is the server the contract tests run against. The maintainer's server is RustFS at
|
||||
`files.because-security.com`, bucket `sast-memory`, store at the bucket root. What the store keeps
|
||||
there is described on [Memory](memory.md).
|
||||
The plane's memory store can live in an S3-compatible bucket. RustFS is an S3-compatible object
|
||||
storage server.
|
||||
|
||||
**The store never configures its bucket.** An admin sets the bucket up once with admin
|
||||
credentials. The agent account the store runs as can read the bucket's settings but not change
|
||||
them, and reads and writes objects freely. Every time the store opens, it checks the setup and
|
||||
refuses to start if anything is missing.
|
||||
- `OUSAST_MEMORY=s3://sast-memory` selects `S3Store` (`src/openultrasast/plane/memory.py`).
|
||||
- `S3Store` talks to the server through boto3 (the AWS SDK for Python).
|
||||
- Any S3-compatible store works if it has bucket versioning, lifecycle rules, object tags and S3 Select
|
||||
(a server-side query over JSON objects).
|
||||
- RustFS is the server the contract tests run against.
|
||||
- The maintainer's server is RustFS at `files.because-security.com`. The bucket is `sast-memory`. The
|
||||
store sits at the bucket root.
|
||||
|
||||
[Memory](memory.md) describes what the store keeps there.
|
||||
|
||||
**The store never configures its bucket.** An admin sets the bucket up once with admin credentials.
|
||||
The store runs as an agent account. That account can read the bucket's settings but not change them.
|
||||
It reads and writes objects freely. Every time the store opens, it checks the setup. It refuses to start
|
||||
if anything is missing.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -35,8 +41,8 @@ aws s3api put-bucket-versioning \
|
||||
--versioning-configuration Status=Enabled
|
||||
```
|
||||
|
||||
**2. Expire raw run outputs under `runs/` after 30 days.** Everything else (`repos/`, `facts/`,
|
||||
`index.jsonl` and the blobs) is kept, so the rule must be a plain prefix filter on `runs/`.
|
||||
**2. Expire raw run outputs under `runs/` after 30 days.** Everything else is kept: `repos/`, `facts/`,
|
||||
`index.jsonl` and the blobs. So the rule must be a plain prefix filter on `runs/`.
|
||||
|
||||
```bash
|
||||
aws s3api put-bucket-lifecycle-configuration \
|
||||
@@ -46,9 +52,10 @@ aws s3api put-bucket-lifecycle-configuration \
|
||||
```
|
||||
|
||||
`put-bucket-lifecycle-configuration` **replaces every rule on the bucket**. If the bucket already
|
||||
has rules, merge this one into them first. The rule id `ousast-runs-expiry` and the 30 days are the
|
||||
store's defaults (`RUNS_RULE_ID`, `RUNS_EXPIRE_DAYS`); the store accepts any enabled, untagged
|
||||
prefix rule on `runs/` with a number of days.
|
||||
has rules, merge this one into them first.
|
||||
|
||||
The rule id `ousast-runs-expiry` and the 30 days are the store's defaults (`RUNS_RULE_ID`,
|
||||
`RUNS_EXPIRE_DAYS`). The store accepts any enabled, untagged prefix rule on `runs/` with a number of days.
|
||||
|
||||
To read the result back:
|
||||
|
||||
@@ -59,8 +66,8 @@ aws s3api get-bucket-lifecycle-configuration --endpoint-url https://files.becaus
|
||||
|
||||
## The agent policy
|
||||
|
||||
Attach this policy to the agent account (the account whose keys go into `.env`). It is the
|
||||
complete policy: nothing else is needed, and it grants no `Put*` on versioning or lifecycle.
|
||||
Attach this policy to the agent account. That is the account whose keys go into `.env`. It is the
|
||||
complete policy, and nothing else is needed. It grants no `Put*` on versioning or lifecycle.
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -102,14 +109,14 @@ complete policy: nothing else is needed, and it grants no `Put*` on versioning o
|
||||
| `BucketSettingsReadOnly` | `arn:aws:s3:::sast-memory` | The agent can list the bucket and its versions and *read* its location, versioning and lifecycle settings, which is what the startup check needs; it cannot change them. |
|
||||
| `ObjectsReadWrite` | `arn:aws:s3:::sast-memory/*` | The agent reads, writes and deletes objects and their versions and reads and writes object tags; this covers row objects, blobs, the index, the Select probe, and the `kind` tag the store filters by. |
|
||||
|
||||
So the agent writes objects freely but cannot change bucket settings: turning versioning off or
|
||||
So the agent writes objects freely but cannot change bucket settings. Turning versioning off or
|
||||
removing the expiry rule needs the admin.
|
||||
|
||||
## The `.env` settings
|
||||
|
||||
The store reads its endpoint and credentials from `.env` in the working directory or from the
|
||||
environment (`s3_settings`), never from a manifest, and never prints them. The names are the ones
|
||||
the AWS SDKs and CLI use for credentials, so one `.env` serves both the store and `aws s3api`.
|
||||
environment (`s3_settings`). It never reads them from a manifest. It never prints them. The names are
|
||||
the ones the AWS SDKs and CLI use for credentials. So one `.env` serves both the store and `aws s3api`.
|
||||
|
||||
| Variable | Value for this server | Notes |
|
||||
| --- | --- | --- |
|
||||
@@ -121,26 +128,33 @@ the AWS SDKs and CLI use for credentials, so one `.env` serves both the store an
|
||||
| `S3_REGION` | optional (`us-east-1` when unset) | the SigV4 signing region; spares a GetBucketLocation call |
|
||||
| `S3_BUCKET` | `sast-memory` | the bucket `s3://` names when the URL has none; also the real-server tests' default bucket |
|
||||
|
||||
The client signs with SigV4 and uses path-style addressing (`https://host/bucket/key`), which
|
||||
RustFS needs. The SDK's own checksum trailers are off, because not every S3-compatible server
|
||||
accepts them.
|
||||
The client signs with SigV4 (AWS's request-signing scheme). It uses path-style addressing
|
||||
(`https://host/bucket/key`), which RustFS needs. The SDK's own checksum trailers are off. Not every
|
||||
S3-compatible server accepts them.
|
||||
|
||||
- **Never commit `.env`.** It is gitignored (`.gitignore`: `.env`, `.env.*`, except
|
||||
`.env.example`). `.env.example` lists these names with empty values.
|
||||
- **`.env` never overrides a variable already exported in your shell** (`config.load_dotenv`).
|
||||
A stale `export AWS_SECRET_ACCESS_KEY=...` in your profile silently wins over the file; `unset`
|
||||
A stale `export AWS_SECRET_ACCESS_KEY=...` in your profile silently wins over the file. `unset`
|
||||
it when a change in `.env` seems to have no effect.
|
||||
- The store needs the `s3` extra (`uv sync --extra s3`, which installs boto3); without it opening
|
||||
the store fails with `OUSAST_MEMORY=s3://... needs boto3: install openultrasast[s3]`. A missing
|
||||
variable fails with `OUSAST_MEMORY=s3://... needs S3_ENDPOINT, AWS_ACCESS_KEY_ID,
|
||||
AWS_SECRET_ACCESS_KEY in .env or the environment` (naming only the missing ones); an endpoint
|
||||
without a scheme fails with `S3_ENDPOINT must be a URL with a scheme (https://host[:port])`.
|
||||
- The store needs the `s3` extra (`uv sync --extra s3`, which installs boto3). These are the
|
||||
startup errors:
|
||||
- Without the extra, opening the store fails with
|
||||
`OUSAST_MEMORY=s3://... needs boto3: install openultrasast[s3]`.
|
||||
- A missing variable fails with `OUSAST_MEMORY=s3://... needs S3_ENDPOINT, AWS_ACCESS_KEY_ID,
|
||||
AWS_SECRET_ACCESS_KEY in .env or the environment`. The message names only the missing ones.
|
||||
- An endpoint without a scheme fails with
|
||||
`S3_ENDPOINT must be a URL with a scheme (https://host[:port])`.
|
||||
|
||||
## What the store verifies at startup
|
||||
|
||||
`S3Store.verify_bucket()` runs once per process for each bucket and prefix. It reads the
|
||||
versioning status and the lifecycle rules, writes a two-line probe object at
|
||||
`_probe/select.jsonl` (under the prefix, if any), queries it with S3 Select and reads its tags.
|
||||
`S3Store.verify_bucket()` runs once per process for each bucket and prefix. It does four things:
|
||||
|
||||
1. It reads the versioning status.
|
||||
2. It reads the lifecycle rules.
|
||||
3. It writes a two-line probe object at `_probe/select.jsonl` (under the prefix, if any).
|
||||
4. It queries the probe with S3 Select and reads its tags.
|
||||
|
||||
Every missing piece is collected into one error:
|
||||
|
||||
```text
|
||||
@@ -148,8 +162,8 @@ memory s3://sast-memory: the bucket is not set up for the store. The store never
|
||||
- <one line per missing piece>
|
||||
```
|
||||
|
||||
The lines, exactly as the code writes them for this bucket at the root (`<failure>` is the
|
||||
server's error code and the first 200 characters of its message):
|
||||
The table lists the lines exactly as the code writes them for this bucket at the root. `<failure>` is
|
||||
the server's error code and the first 200 characters of its message.
|
||||
|
||||
| Check | Refusal line |
|
||||
| --- | --- |
|
||||
@@ -161,32 +175,42 @@ server's error code and the first 200 characters of its message):
|
||||
| tags unreadable | `object tags cannot be read on _probe/select.jsonl (<failure>): the store filters row objects by their kind tag, so the agent needs s3:GetObjectTagging on arn:aws:s3:::sast-memory/*` |
|
||||
| S3 Select missing | `S3 Select (SelectObjectContent over JSON Lines) does not answer on _probe/select.jsonl (<failure>). The store requires it and has no local fallback: use a server with S3 Select (RustFS has it; not every S3-compatible server does) and let the agent s3:PutObject and s3:GetObject` |
|
||||
|
||||
The admin commands inside the messages use `--endpoint-url "$S3_ENDPOINT"`, which is the full
|
||||
URL from `.env`, so they run as written once `.env` is exported.
|
||||
The admin commands inside the messages use `--endpoint-url "$S3_ENDPOINT"`. That is the full URL from
|
||||
`.env`. So they run as written once `.env` is exported.
|
||||
|
||||
There is **no fallback**: the store does not start in a degraded mode, does not drop to a local
|
||||
store, and does not fetch and filter objects itself.
|
||||
There is **no fallback**. The store does not:
|
||||
|
||||
- start in a degraded mode;
|
||||
- drop to a local store;
|
||||
- fetch and filter objects itself.
|
||||
|
||||
## The RustFS Select limit and the fixed-schema rule
|
||||
|
||||
RustFS's S3 Select infers an object's JSON schema from its leading rows. A `where` field that is
|
||||
absent from those rows is unbound, even if a later row has it: the query fails with
|
||||
RustFS's S3 Select infers an object's JSON schema from its leading rows. A `where` field absent from
|
||||
those rows is unbound, even if a later row has it. The query then fails with
|
||||
`EvaluatorBindingDoesNotExist` instead of matching (measured 2026-10-02 with the field first
|
||||
present at row 5000). The store does not read that as "no rows", since that answer could silently
|
||||
drop matches; it raises:
|
||||
present at row 5000).
|
||||
|
||||
The store does not read that failure as "no rows". That answer could silently drop matches. It raises:
|
||||
|
||||
```text
|
||||
memory s3://sast-memory: S3 Select on <key> failed (EvaluatorBindingDoesNotExist: ...) (a field of [<fields>] is absent from the object's leading rows, which the server reads as its schema); the store has no local fallback
|
||||
```
|
||||
|
||||
The rule that follows: a field the store filters on is present in every row of its kind from
|
||||
the first row on, as `""` where it does not apply; each row kind has a fixed set of queryable
|
||||
fields, enforced at write time (see [Memory](memory.md#queryable-fields-per-kind)).
|
||||
The rule that follows has two parts (see [Memory](memory.md#queryable-fields-per-kind)):
|
||||
|
||||
- A field the store filters on is present in every row of its kind from the first row on. It is `""`
|
||||
where it does not apply.
|
||||
- Each row kind has a fixed set of queryable fields. The store enforces it at write time.
|
||||
|
||||
## Checks against the real server
|
||||
|
||||
The contract tests that talk to a real server run only with `OUSAST_MEMORY_TEST_S3=1` (bucket
|
||||
`OUSAST_MEMORY_TEST_BUCKET`, else `S3_BUCKET`). They verify the bucket at its root, write only
|
||||
under a fresh `contract-<id>/` prefix, and remove every version and delete marker under it
|
||||
afterwards; they never configure the bucket. More on the host setup:
|
||||
[ax on this host](ops/ax/README.md).
|
||||
The contract tests that talk to a real server run only with `OUSAST_MEMORY_TEST_S3=1`. They use the
|
||||
bucket `OUSAST_MEMORY_TEST_BUCKET`, else `S3_BUCKET`. The tests:
|
||||
|
||||
- verify the bucket at its root;
|
||||
- write only under a fresh `contract-<id>/` prefix;
|
||||
- remove every version and delete marker under it afterwards;
|
||||
- never configure the bucket.
|
||||
|
||||
More on the host setup: [ax on this host](ops/ax/README.md).
|
||||
|
||||
+330
-39
@@ -1,13 +1,21 @@
|
||||
# Scanning
|
||||
|
||||
`ousast scan PATH --mode quick|standard|deep` scans a checkout; `ousast pre-push` checks only
|
||||
what a push (or an explicit `--base`/`--head` pair) changes. The stage-by-stage pipeline,
|
||||
evidence ladder and score are in [Architecture](architecture.md); this page shows the flows.
|
||||
This page shows the scan flows.
|
||||
|
||||
- `ousast scan PATH --mode quick|standard|deep` scans a checkout.
|
||||
- `ousast pre-push` checks only what a push changes, or what an explicit `--base`/`--head` pair
|
||||
changes.
|
||||
|
||||
The stage-by-stage pipeline, the evidence ladder and the score are in
|
||||
[Architecture](architecture.md).
|
||||
|
||||
## The three scan modes
|
||||
|
||||
The stages a mode requests are `MODE_STAGES` in `src/openultrasast/stages.py`: `quick` runs
|
||||
STATIC, `standard` adds MAP, `deep` adds REGRESS.
|
||||
The stages each mode requests are listed in `MODE_STAGES` in `src/openultrasast/stages.py`:
|
||||
|
||||
- `quick` runs STATIC.
|
||||
- `standard` adds MAP.
|
||||
- `deep` adds REGRESS.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
@@ -38,7 +46,7 @@ Sources: `README.md` (scan modes), `src/openultrasast/cli.py` (`run_stage` calls
|
||||
|
||||
## Quick rules per language
|
||||
|
||||
Rules run only against files of their language. The bundled ruleset lives in
|
||||
Each rule runs only against files of its language. The bundled ruleset lives in
|
||||
`src/openultrasast/ruleset/<language>/rules.toml`.
|
||||
|
||||
| Language | Classes (CWE) |
|
||||
@@ -49,21 +57,31 @@ Rules run only against files of their language. The bundled ruleset lives in
|
||||
| Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
|
||||
| PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules |
|
||||
|
||||
The detection gate (`python -m openultrasast.gate`) enforces at least 90% recall and under 10%
|
||||
false positives per language on the bundled corpora in `benchmarks/manifests/`. Those corpora are
|
||||
cheat-sheet fixtures: they catch regressions, they do not estimate recall on real code. **PHP is
|
||||
not in the gate**; its numbers are in-sample development numbers
|
||||
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). See `README.md` for the
|
||||
gate's per-language output.
|
||||
The detection gate (`python -m openultrasast.gate`) enforces two limits per language on the
|
||||
bundled corpora in `benchmarks/manifests/`: at least 90% recall and under 10% false positives.
|
||||
|
||||
- Those corpora are cheat-sheet fixtures. They catch regressions. They do not estimate recall on
|
||||
real code.
|
||||
- **PHP is not in the gate**. Its numbers are in-sample development numbers
|
||||
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`).
|
||||
|
||||
See `README.md` for the gate's per-language output.
|
||||
|
||||
## The Joern engine
|
||||
|
||||
The model layer (`src/openultrasast/model/`, `src/openultrasast/cpg/`) is the core of `standard`.
|
||||
It builds one code property graph per scan with Joern (`joern-parse`, then batched
|
||||
`joern --script` queries from `cpg/queries/*.sc`) and sends each question to the arbiter of its
|
||||
type: **taint** (source to sink with no discharging sanitizer), **dominance** (does a guard govern
|
||||
the obligated operation) and **configuration** (constant abstraction over a value). Taint facts
|
||||
ship for C, Java, JavaScript, PHP and Python (`src/openultrasast/ruleset/semantic/*.toml`).
|
||||
It builds one code property graph per scan with Joern. A code property graph is a graph of the
|
||||
program's syntax, control flow and data flow. The build runs `joern-parse`, then batched
|
||||
`joern --script` queries from `cpg/queries/*.sc`.
|
||||
|
||||
The layer sends each question to the arbiter (the decider) of its type:
|
||||
|
||||
- **taint**: a flow from source to sink with no discharging sanitizer;
|
||||
- **dominance**: does a guard govern the obligated operation;
|
||||
- **configuration**: constant abstraction over a value.
|
||||
|
||||
Taint facts ship for C, Java, JavaScript, PHP and Python
|
||||
(`src/openultrasast/ruleset/semantic/*.toml`).
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
@@ -79,25 +97,33 @@ flowchart LR
|
||||
rung -- "residual question only" --> judge["LLM judge: confirm or contradict, never create a flow"]
|
||||
```
|
||||
|
||||
The Docker image ships Joern and `php-cli` (which the PHP frontend needs); without Joern the
|
||||
layer records `cpg_unavailable` and the scan is otherwise unchanged. Engine installation and the
|
||||
Joern features used are in [Engine and pre-push](ops/README.md).
|
||||
How the graph is built, queried and refined, with the records behind each refinement, is in
|
||||
[Detection techniques](detection-techniques.md).
|
||||
The Docker image ships Joern and `php-cli`. The PHP frontend needs `php-cli`. Without Joern, the
|
||||
layer records `cpg_unavailable`; the rest of the scan is unchanged.
|
||||
|
||||
- Engine installation and the Joern features used: [Engine and pre-push](ops/README.md).
|
||||
- How the graph is built, queried and refined, with the records behind each refinement:
|
||||
[Detection techniques](detection-techniques.md).
|
||||
|
||||
## Framework priors
|
||||
|
||||
Rule and taint-fact entries that know a framework or library carry a `framework` or `library` tag
|
||||
naming a row of `src/openultrasast/ruleset/frameworks.toml` (for example `wordpress`, `flask`,
|
||||
`django`, `express`, `spring`). The loaders take `priors=`: `"all"` (the default and today's
|
||||
behaviour), `"off"` (language-level knowledge only) or a set of ids, so the contribution of
|
||||
framework knowledge can be measured and switched off. The decision engine's first slice ran with
|
||||
priors off ([Decision engine](decision-engine.md#first-measured-slice-2026-10-01)).
|
||||
Some rule and taint-fact entries know a framework or library. They carry a `framework` or
|
||||
`library` tag. The tag names a row of `src/openultrasast/ruleset/frameworks.toml` (for example
|
||||
`wordpress`, `flask`, `django`, `express`, `spring`).
|
||||
|
||||
The loaders take `priors=` with one of three values:
|
||||
|
||||
- `"all"`: the default and today's behaviour;
|
||||
- `"off"`: language-level knowledge only;
|
||||
- a set of ids.
|
||||
|
||||
So the contribution of framework knowledge can be measured and switched off. The decision
|
||||
engine's first slice ran with priors off
|
||||
([Decision engine](decision-engine.md#first-measured-slice-2026-10-01)).
|
||||
|
||||
## Pre-push: the delta check (experimental)
|
||||
|
||||
`ousast pre-push` analyses the commits a `git push` would publish, or an explicit
|
||||
`--base`/`--head` pair, and compares head with base so that only new or worsened defects are
|
||||
`--base`/`--head` pair. It compares head with base. Only new or worsened defects become
|
||||
candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`,
|
||||
[Engine and pre-push](ops/README.md#explicit-local-replay).
|
||||
|
||||
@@ -105,23 +131,288 @@ candidates. Sources: `ousast pre-push --help`, `src/openultrasast/push/`,
|
||||
flowchart TD
|
||||
input["git push refs on stdin, or --base / --head"] --> resolve["Resolve both commit identities"]
|
||||
resolve --> snap["Materialise tracked bytes in private scratch storage"]
|
||||
snap --> regions["Discover changed regions, up to --max-regions"]
|
||||
regions --> engine["Evidence ranker and arbiter on head"]
|
||||
snap --> quick["Quick rules on the changed head lines (advisory, under 1 s)"]
|
||||
quick --> eng{"--engine"}
|
||||
eng -- "off" --> art
|
||||
eng -- "background" --> bg["Detached engine run; result shown on the next push"]
|
||||
bg --> art
|
||||
eng -- "inline, the default" --> engine["Engine on head: changed regions up to --max-regions, ranker, arbiter"]
|
||||
engine --> cmp["Targeted base comparison: new or worsened only"]
|
||||
cmp --> admit{"Admission policy: capability registry"}
|
||||
admit -- "registry empty today" --> diag["Diagnostic candidates in the artifact"]
|
||||
admit -- "registry empty today" --> diag["Advisory engine findings, not alerts"]
|
||||
admit -- "an enabled capability" --> alert["Normal alert"]
|
||||
diag --> art["JSON artifact outside the repository"]
|
||||
diag --> art["Terminal lines + JSON artifact outside the repository"]
|
||||
alert --> art
|
||||
art --> exit{"--mode"}
|
||||
exit -- "advisory, the default" --> allow["Exit 0: allow"]
|
||||
exit -- "blocking, explicit opt-in" --> block["May block the push"]
|
||||
```
|
||||
|
||||
- `--deadline` (30 s by default) bounds preparation, both revisions and reporting together;
|
||||
`--incomplete-coverage block` opts into failing when coverage is incomplete.
|
||||
- The default capability registry is empty pending independent qualification, so no normal alert
|
||||
is emitted today; every candidate stays diagnostic.
|
||||
- `--deadline` (30 s by default) bounds preparation, the quick rules, both revisions and reporting together.
|
||||
- The quick rules run first on the changed lines. Their matches are shown as advisory pattern
|
||||
matches, never as alerts, and never block.
|
||||
- The default capability registry is empty until independent qualification passes. So no normal
|
||||
alert is emitted today. An engine finding the push introduced is shown as an advisory engine
|
||||
finding; every other candidate stays diagnostic in the artifact.
|
||||
- A model is called only with an explicit `--model-config`.
|
||||
- Hook installation (`ops/install-pre-push`) and the private cache (`--cache-dir`) are documented
|
||||
in [Engine and pre-push](ops/README.md#experimental-pre-push-integration).
|
||||
- The private cache (`--cache-dir`) and the manual install recipe are documented in
|
||||
[Engine and pre-push](ops/README.md#experimental-pre-push-integration).
|
||||
|
||||
### First run: install the hook on any repository
|
||||
|
||||
The hook is advisory. It never rejects a push unless you set `OUSAST_PUSH_MODE=blocking`. A check
|
||||
that cannot run says so in one line and lets the push through: a missing `ousast`, a missing
|
||||
Docker image, an invalid setting, a crash.
|
||||
|
||||
1. Install the tool. The package is not on PyPI. Install it from the repository (or from a
|
||||
release wheel on the GitHub releases page):
|
||||
|
||||
```bash
|
||||
pip install "git+https://github.com/norandom/OpenUltraSAST"
|
||||
```
|
||||
|
||||
2. Install the hook in the repository you want checked. Git's effective hook directory is used,
|
||||
including `core.hooksPath`, and that setting is never edited:
|
||||
|
||||
```bash
|
||||
cd /path/to/your/repository
|
||||
ousast pre-push install # refuses if a pre-push hook already exists
|
||||
ousast pre-push install --force # moves an existing hook to pre-push.before-ousast and chains it
|
||||
```
|
||||
|
||||
3. Push as usual. Without the Joern engine, you get the quick rules and one line saying the
|
||||
engine did not run. For the engine, choose one:
|
||||
|
||||
- a host install with `joern-parse` on the PATH (`pyinfra @local ops/joern.py`, see
|
||||
[Engine and pre-push](ops/README.md));
|
||||
- Docker: `docker pull ghcr.io/norandom/openultrasast:2.0.1`, then
|
||||
`ousast pre-push install --docker`. The wrapper `ousast-docker` is installed beside the
|
||||
hook and runs the check in the container (repository read-only, no network,
|
||||
`--memory 3g`). The container runs the code of its image's version: the quick-rule
|
||||
tier, the plain skip lines and `--engine` need an image built from a version that has them
|
||||
(`docker build -t openultrasast:dev .`, then `OUSAST_IMAGE=openultrasast:dev`).
|
||||
|
||||
4. Remove it with `ousast pre-push uninstall`. Only a hook this tool wrote is removed, and a
|
||||
chained previous hook is restored.
|
||||
|
||||
A push on the 2026-10-02 proof repository (serve-static, a new `debug.js` that evaluates
|
||||
`req.query.code`, no Joern on the host) prints:
|
||||
|
||||
```text
|
||||
Quick rules: 2 pattern match(es) on changed code (1 file(s), 87 bytes read). Advisory: not verified by the engine.
|
||||
- js-eval (CWE-95, critical) at debug.js:2: Dynamic JavaScript evaluation needs review
|
||||
- js-xss-response (CWE-79, medium) at debug.js:2: Unsanitized HTTP response reflection needs review
|
||||
No actionable defects reported; analysis is incomplete.
|
||||
Skipped: the changed code has no counterpart in the base revision to compare against
|
||||
Skipped: engine did not run: the code graph could not be built (is Joern installed? ...)
|
||||
Skipped: no base revision was analysed, so no engine finding can be called new or worsened
|
||||
Notice: Engine checks did not run. Install Joern, or run the hook through ops/ousast-docker. Review the details.
|
||||
Details: /home/you/.cache/openultrasast/push/push-12345.json
|
||||
```
|
||||
|
||||
The hook reads these settings from the environment of `git push`:
|
||||
|
||||
| Variable | Default | Effect |
|
||||
| --- | --- | --- |
|
||||
| `OUSAST_PUSH_DEADLINE` | `30` | seconds for the whole check |
|
||||
| `OUSAST_PUSH_MODE` | `advisory` | `blocking` is the only way a result can reject a push |
|
||||
| `OUSAST_INCOMPLETE_COVERAGE` | `allow` | with `blocking`: `block` also rejects incomplete coverage |
|
||||
| `OUSAST_PUSH_ENGINE` | `inline` | `background`: return after the quick rules, run the engine detached; `off`: quick rules only |
|
||||
| `OUSAST_COMPARISON_BASE` | unset | a local ref to compare a new branch against |
|
||||
| `OUSAST_ARTIFACT_DIR` | `~/.cache/openultrasast/push` | where the JSON results go; outside the repository |
|
||||
| `OUSAST_COMMAND` | `ousast` | the command the hook runs, for example the Docker wrapper |
|
||||
| `OUSAST_IMAGE` | `ghcr.io/norandom/openultrasast:2.0.1` | the Docker wrapper's image |
|
||||
| `OUSAST_DOCKER_MEMORY` | `3g` | the Docker wrapper's memory limit |
|
||||
|
||||
**A new branch.** Git gives the hook no remote base for a branch the remote does not have yet.
|
||||
The check says so ("new branch: the remote has no base to compare against"), runs the quick rules
|
||||
over the files the branch's last commit changed instead, and says that too. Set
|
||||
`OUSAST_COMPARISON_BASE=main` to compare a new branch against your local `main`.
|
||||
|
||||
**The engine and the deadline.** The engine does not fit a 30 s deadline today. The 2026-10-02
|
||||
audit measured 260 s on a 26-file repository; graph reuse is the planned fix. When it cannot
|
||||
finish, one line says so. With `OUSAST_PUSH_ENGINE=background`, the hook returns after the quick
|
||||
rules and starts one detached engine run (`--background-deadline`, 900 s by default). The next push
|
||||
in the same repository prints `Engine result for <head> from the previous push: ...` once. Only
|
||||
one background run happens at a time. The Docker wrapper cannot run the engine in the background,
|
||||
because the container ends with the hook.
|
||||
|
||||
**The artifact.** Every run writes one JSON file, and the terminal prints its path after
|
||||
`Details:`. It records the following:
|
||||
|
||||
- `snapshots[].bytes_read`: proof that the input was read;
|
||||
- `quick_tier`: the quick-rule matches, with the files and bytes read;
|
||||
- `admission.coverage_reasons`: the raw skip reasons;
|
||||
- `admission.dispositions`: every engine candidate and its vetoes;
|
||||
- `engine_background`: background runs started or shown;
|
||||
- `timings`.
|
||||
|
||||
Files accumulate, one per push; delete old ones when you like.
|
||||
|
||||
### Skip reasons
|
||||
|
||||
The terminal prints one `Skipped:` line per distinct reason in plain words. The artifact keeps the
|
||||
raw reason. Repository paths are hex-encoded there, and per-question reasons end in a hash.
|
||||
|
||||
| Raw reason (artifact) | Terminal line says | What to do |
|
||||
| --- | --- | --- |
|
||||
| `cpg_build_failed`, `cpg_unavailable` | the engine did not run: the code graph could not be built | install Joern, or use the Docker wrapper |
|
||||
| `cpg_empty` | the engine read no code | check that the changed files are in a supported language |
|
||||
| `frontend_unsupported` | no engine frontend for a language in this push | none: that language gets the quick rules at most |
|
||||
| `language_not_covered:<lang>:<changed>:<total>` | language not covered: `<lang>` (n changed files, m in the repository) | none: neither the engine nor the quick rules cover it, so silence is not a clean result |
|
||||
| `deadline_exhausted` | the engine did not finish within the deadline | raise `OUSAST_PUSH_DEADLINE`, or set `OUSAST_PUSH_ENGINE=background` |
|
||||
| `change_context_deadline_exhausted` | the deadline ran out while relating the change to the code | as above |
|
||||
| `report_deadline_exhausted` | the deadline ran out while saving the details | as above |
|
||||
| `engine_off`, `engine_background` | the engine was switched off, or runs after the push | none |
|
||||
| `resolution:missing_base` | new branch: the remote has no base to compare against | set `OUSAST_COMPARISON_BASE` |
|
||||
| `resolution:unsupported_base`, `resolution:no_merge_base`, `resolution:ambiguous_base`, `resolution:missing_target`, `resolution:unsupported_target`, `resolution:resolution_error` | the pushed ref or its remote base could not be resolved locally | fetch the remote, then push again |
|
||||
| `base_comparison_unavailable`, `base_unavailable` | no base revision was analysed, so nothing can be called new | see the line before it for the cause |
|
||||
| `base_counterpart_unresolved` | the changed code has no counterpart in the base to compare against | none: expected for new files |
|
||||
| `<path>:ambiguous_line_correspondence`, `<path>:line_correspondence_unavailable`, `<path>:ambiguous_path_rename_correspondence` | changed lines or a renamed file could not be matched to the base | none: findings there stay diagnostic |
|
||||
| `unmatched_added_deleted_path_correspondence` | files were both added and deleted, so a move cannot be told from a new file | push the move separately from other edits |
|
||||
| `declaration_paths_unavailable` | project declaration files could not be listed | rerun |
|
||||
| `capability_unavailable`, `capability_disabled`, `capability_unevaluated`, `capability_ambiguous` | engine findings are not qualified as alerts yet | none: they show as advisory engine findings |
|
||||
| `eligibility:<reason>` | the capability registry does not match this build | reinstall a consistent build |
|
||||
| `comparison_unknown`, `base_incomplete` and the other novelty reasons | whether this push introduced some engine findings could not be established | none: they stay diagnostic |
|
||||
| `dynamic_external_or_depth_context_unresolved:...` | n engine checks depend on dynamic or external code the engine cannot follow | none |
|
||||
| `query_failed`, `query_budget_reserved`, `query_too_expensive`, `regions_truncated`, `files_unparsed` | some engine questions were not answered | raise the deadline or `--max-regions` |
|
||||
| `cross_partition_semantics_unresolved` | flows between languages are not followed | none |
|
||||
| `dependency_unresolved`, `change_unsupported` | engine findings depend on unresolved code or have no supported link to the change | none |
|
||||
| `snapshot:<reason>` | some files could not be materialised for analysis | see the reason (symlinks, submodules, LFS) |
|
||||
| `<stage>_failed:<Error>`, `push_input_failed:<Error>` | the named step failed | rerun; report it if it repeats |
|
||||
|
||||
A reason not in this table is still printed as its own words, followed by "see the details file".
|
||||
|
||||
### In CI
|
||||
|
||||
The same check runs on a pushed range in CI. This GitHub Actions job is advisory: it never fails
|
||||
the build, and it uploads the artifact.
|
||||
|
||||
```yaml
|
||||
name: ousast pre-push (advisory)
|
||||
on: push
|
||||
jobs:
|
||||
pre-push:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- run: pip install "git+https://github.com/norandom/OpenUltraSAST"
|
||||
- name: Check what this push changed
|
||||
if: github.event.before != '0000000000000000000000000000000000000000'
|
||||
run: |
|
||||
ousast pre-push . --base ${{ github.event.before }} --head ${{ github.sha }} \
|
||||
--artifact "$RUNNER_TEMP/ousast-pre-push.json" --engine off --deadline 120 || true
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: ousast-pre-push
|
||||
path: ${{ runner.temp }}/ousast-pre-push.json
|
||||
```
|
||||
|
||||
A branch's first push has no `github.event.before` to compare against, and the job skips it. For
|
||||
the engine in CI, run the job in the container image instead of `pip install`, and drop
|
||||
`--engine off`.
|
||||
|
||||
### `ousast pre-push --help`
|
||||
|
||||
```text
|
||||
usage: ousast pre-push [-h] [--base BASE] [--head HEAD] [--remote NAME URL]
|
||||
[--comparison-base COMPARISON_BASE] [--model-config MODEL_CONFIG]
|
||||
[--prior-hook PRIOR_HOOK] --artifact ARTIFACT [--cache-dir CACHE_DIR]
|
||||
[--deadline DEADLINE] [--cancellation-allowance CANCELLATION_ALLOWANCE]
|
||||
[--max-regions MAX_REGIONS] [--mode {advisory,blocking}]
|
||||
[--incomplete-coverage {allow,block}] [--engine {inline,background,off}]
|
||||
[--background-deadline BACKGROUND_DEADLINE]
|
||||
[--experimental-declarations EXPERIMENTAL_DECLARATIONS]
|
||||
[--experimental-record-vetoes]
|
||||
path
|
||||
|
||||
Check what a push changes: quick rules on the changed lines, then the Joern engine on head and
|
||||
base within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with
|
||||
'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall
|
||||
[REPO]'.
|
||||
|
||||
positional arguments:
|
||||
path the repository to check (the hook passes '.')
|
||||
|
||||
options:
|
||||
-h, --help show this help message and exit
|
||||
--base BASE explicit replay: the base revision (with --head; instead of --remote)
|
||||
--head HEAD explicit replay: the head revision to check against --base
|
||||
--remote NAME URL Git hook mode: the remote name and URL Git passes to pre-push; the push
|
||||
input is read from stdin
|
||||
--comparison-base COMPARISON_BASE
|
||||
hook mode: a local ref to compare a NEW branch against (the remote has no
|
||||
base for it); the hook reads OUSAST_COMPARISON_BASE
|
||||
--model-config MODEL_CONFIG
|
||||
explicit optional witness selection endpoint configuration
|
||||
--prior-hook PRIOR_HOOK
|
||||
explicitly chain an existing hook with the same input and arguments
|
||||
--artifact ARTIFACT where to write the complete JSON result; must be outside the repository
|
||||
--cache-dir CACHE_DIR
|
||||
reuse compatible local artifacts in a private directory outside the
|
||||
repository
|
||||
--deadline DEADLINE seconds for the whole check: resolution, quick rules, engine and reporting
|
||||
(default 30; hook: OUSAST_PUSH_DEADLINE)
|
||||
--cancellation-allowance CANCELLATION_ALLOWANCE
|
||||
seconds after the deadline to stop work and save the result (default 2)
|
||||
--max-regions MAX_REGIONS
|
||||
most code regions the engine examines, highest-ranked first (default 500)
|
||||
--mode {advisory,blocking}
|
||||
advisory (default) never blocks; blocking is the explicit opt-in that
|
||||
rejects a push for an admitted alert (hook: OUSAST_PUSH_MODE)
|
||||
--incomplete-coverage {allow,block}
|
||||
with --mode blocking only: also block when coverage is incomplete (hook:
|
||||
OUSAST_INCOMPLETE_COVERAGE)
|
||||
--engine {inline,background,off}
|
||||
inline (default): run the Joern engine inside the deadline; background:
|
||||
return after the quick rules and run the engine detached, showing its
|
||||
result on the next run; off: quick rules only
|
||||
--background-deadline BACKGROUND_DEADLINE
|
||||
seconds the detached engine run of --engine background may take (default
|
||||
900)
|
||||
--experimental-declarations EXPERIMENTAL_DECLARATIONS
|
||||
explicit --base/--head replay only: unreviewed capability declarations
|
||||
used to render the experimental evaluation explanation; never consulted by
|
||||
admission and cannot enable any capability
|
||||
--experimental-record-vetoes
|
||||
explicit --base/--head replay only: record each comparison and admission
|
||||
veto per raw finding instead of applying it; output is labeled
|
||||
experimental, changes no result and enables no hook capability
|
||||
```
|
||||
|
||||
```text
|
||||
usage: ousast pre-push install [-h] [--force] [--docker] [repository]
|
||||
|
||||
Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory
|
||||
(core.hooksPath is honoured, never edited).
|
||||
|
||||
positional arguments:
|
||||
repository the repository (default: current directory)
|
||||
|
||||
options:
|
||||
-h, --help show this help message and exit
|
||||
--force install: replace our hook, or move a foreign hook to pre-push.before-ousast and
|
||||
chain it; uninstall: remove a hook this tool did not write
|
||||
--docker also install the ousast-docker wrapper beside the hook, so the check runs in the
|
||||
container image
|
||||
```
|
||||
|
||||
```text
|
||||
usage: ousast pre-push uninstall [-h] [--force] [repository]
|
||||
|
||||
Remove the OpenUltraSAST pre-push hook and restore a chained previous hook.
|
||||
|
||||
positional arguments:
|
||||
repository the repository (default: current directory)
|
||||
|
||||
options:
|
||||
-h, --help show this help message and exit
|
||||
--force install: replace our hook, or move a foreign hook to pre-push.before-ousast and
|
||||
chain it; uninstall: remove a hook this tool did not write
|
||||
```
|
||||
|
||||
+75
-52
@@ -1,78 +1,101 @@
|
||||
# Threat model & hardening
|
||||
|
||||
OpenUltraSAST analyzes **untrusted code**. This document states what it trusts, what
|
||||
it does not, and the controls that keep a scan safe to run in CI and safe to share.
|
||||
OpenUltraSAST analyzes **untrusted code**. This page states what it trusts and what it does not.
|
||||
It also lists the controls that keep a scan safe to run in CI and safe to share.
|
||||
|
||||
## Trust boundaries
|
||||
|
||||
- **Scanned code is untrusted input.** Quick and standard modes never import, build or run
|
||||
the target: quick reads source as text, and standard additionally parses it into a code
|
||||
property graph with Joern (`joern-parse`, a parser, not a build) and may send code excerpts
|
||||
to the configured model. Only `deep` mode executes target-derived code: its REGRESS stage
|
||||
loads promoted candidates inside the Docker sandbox below, and it runs only when `docker
|
||||
info` succeeds (otherwise the stage is skipped and recorded as `sandbox_unavailable`).
|
||||
- **The ruleset and CWE policy are trusted, governed data.** They change only through
|
||||
reviewed commits or the bounded self-improvement loop (`ousast improve`), which can
|
||||
flip rule *status* and tune score constants but can never edit pattern text or the
|
||||
- **Scanned code is untrusted input.** Quick and standard modes never import, build or run the
|
||||
target.
|
||||
- Quick mode reads source as text.
|
||||
- Standard mode also parses it into a code property graph (a graph of the program's syntax,
|
||||
control flow and data flow) with Joern. It uses `joern-parse`, a parser, not a build.
|
||||
Standard mode may send code excerpts to the configured model.
|
||||
- Only `deep` mode executes target-derived code. Its REGRESS stage loads promoted candidates
|
||||
inside the Docker sandbox described below. It runs only when `docker
|
||||
info` succeeds (otherwise the stage is skipped and recorded as `sandbox_unavailable`).
|
||||
- **The ruleset and CWE policy are trusted, governed data.** They change in only two ways:
|
||||
through reviewed commits, or through the bounded self-improvement loop (`ousast improve`). The
|
||||
loop can flip rule *status* and tune score constants. It can never edit pattern text or the
|
||||
authoritative 0–5 severity. See [architecture.md](architecture.md#self-improvement).
|
||||
- **Provider credentials are the operator's.** The core install (PyYAML only) makes no
|
||||
network calls on its own. A local scan calls a model only in standard or deep mode, and
|
||||
only when a key is available: the model layer asks the residual question the graph cannot
|
||||
settle, and `[models] hunter` enables the tool hunter. LLM calls go to DeepSeek
|
||||
(`DEEPSEEK_API_KEY`); OpenRouter (`OPENROUTER_API_KEY`) serves embeddings. Keys come from
|
||||
the environment or a gitignored `.env`, which never overrides an exported variable. The
|
||||
pre-push check calls a model only with an explicit `--model-config`. Agentic work runs on the plane (`ousast plane run`): `ousast` reads
|
||||
the variable a task's Model names (`secretKey.key`) from the operator's environment or
|
||||
`.env` and sends the key only in that task's start request, through Agent Substrate's
|
||||
router to that task's actor. It is not baked into an image or task template, and it is
|
||||
never logged or written to artifacts.
|
||||
- **Provider credentials are the operator's.** The core install (PyYAML only) makes no network
|
||||
calls on its own.
|
||||
- A local scan calls a model only in standard or deep mode, and only when a key is available.
|
||||
The model layer asks the residual question the graph cannot settle. `[models] hunter`
|
||||
enables the tool hunter.
|
||||
- LLM calls go to DeepSeek (`DEEPSEEK_API_KEY`). OpenRouter (`OPENROUTER_API_KEY`) serves
|
||||
embeddings.
|
||||
- Keys come from the environment or a gitignored `.env`. `.env` never overrides an exported
|
||||
variable.
|
||||
- The pre-push check calls a model only with an explicit `--model-config`.
|
||||
- Agentic work runs on the plane (`ousast plane run`). `ousast` reads the variable a task's
|
||||
Model names (`secretKey.key`) from the operator's environment or `.env`. It sends the key
|
||||
only in that task's start request. The request goes through Agent Substrate's router to that
|
||||
task's actor. The key is not baked into an image or task template. It is never logged or
|
||||
written to artifacts.
|
||||
|
||||
## Controls
|
||||
|
||||
### Secret redaction
|
||||
Scan artifacts can quote source that contains live credentials. `redaction.py` masks
|
||||
recognizable secret shapes (provider API keys, AWS/GitHub/Slack/Google tokens, bearer
|
||||
tokens, URL-embedded credentials, PEM private keys, and `key = value` assignments for
|
||||
sensitive names) before traces (`trace/events.jsonl`) and the markdown report are
|
||||
written. On by default; disable with `[hardening] redact_secrets = false`.
|
||||
recognizable secret shapes before traces (`trace/events.jsonl`) and the markdown report are
|
||||
written. It masks:
|
||||
|
||||
- provider API keys;
|
||||
- AWS/GitHub/Slack/Google tokens;
|
||||
- bearer tokens;
|
||||
- URL-embedded credentials;
|
||||
- PEM private keys;
|
||||
- `key = value` assignments for sensitive names.
|
||||
|
||||
Redaction is on by default. Turn it off with `[hardening] redact_secrets = false`.
|
||||
|
||||
### Plane egress
|
||||
Each plane task runs in its own actor behind Agent Substrate's egress gateway, which
|
||||
denies by default. The task's EgressPolicy (`src/openultrasast/plane/egress.py`) allows exactly three
|
||||
things: plain HTTP to the artifact receiver, and TLS passthrough to the Git hosts of the
|
||||
task's bound Workspaces and to the hosts its bound Model declares
|
||||
(`openultrasast.io/egress-hosts`). There are no wildcards, no IP addresses and no TLS
|
||||
interception. The policy is deleted with the actor.
|
||||
Each plane task runs in its own actor behind Agent Substrate's egress gateway. The gateway denies
|
||||
by default. The task's EgressPolicy (`src/openultrasast/plane/egress.py`) allows exactly three
|
||||
things:
|
||||
|
||||
1. plain HTTP to the artifact receiver;
|
||||
2. TLS passthrough to the Git hosts of the task's bound Workspaces;
|
||||
3. TLS passthrough to the hosts its bound Model declares (`openultrasast.io/egress-hosts`).
|
||||
|
||||
There are no wildcards, no IP addresses and no TLS interception. The policy is deleted with the
|
||||
actor.
|
||||
|
||||
### Cost & CI budgets
|
||||
- **Agentic spend** is bounded per plane task by the Run's `budget: {usd, calls}`. The
|
||||
metered client refuses the next call once either ceiling is reached (the task is
|
||||
`unfinished` and resumes on a rerun with a larger budget). A usd budget on a Model
|
||||
without prices is refused rather than metered as zero, and an account or
|
||||
authentication error fails the task and starts nothing further. The standard-mode tool
|
||||
hunter is bounded by the per-tier hunter budgets.
|
||||
- **Agentic spend** is bounded per plane task by the Run's `budget: {usd, calls}`.
|
||||
- The metered client refuses the next call once either ceiling is reached. The task is then
|
||||
`unfinished`. It resumes on a rerun with a larger budget.
|
||||
- A usd budget on a Model without prices is refused. It is not metered as zero.
|
||||
- An account or authentication error fails the task and starts nothing further.
|
||||
- The standard-mode tool hunter is bounded by the per-tier hunter budgets.
|
||||
- **Output size** is bounded by `[hardening] max_findings` (0 = unlimited). Truncation is
|
||||
severity-ordered and disclosed as a `budget` degradation in the manifest.
|
||||
severity-ordered. It is disclosed as a `budget` degradation in the manifest.
|
||||
|
||||
### Provider reliability
|
||||
LLM/embedding calls retry transient failures (HTTP 429/5xx, connection errors, timeouts)
|
||||
with exponential backoff; non-transient errors (e.g. 4xx, malformed JSON) fail fast.
|
||||
LLM/embedding calls retry transient failures with exponential backoff. Transient failures are
|
||||
HTTP 429/5xx, connection errors and timeouts. Non-transient errors fail fast, for example 4xx or
|
||||
malformed JSON.
|
||||
|
||||
### Visible degradation & determinism
|
||||
When an optional engine is missing (Joern: `cpg_unavailable`; Docker: `sandbox_unavailable`),
|
||||
the scan falls back to its deterministic equivalent and records a `degradations` entry in the
|
||||
manifest — it is never silently downgraded. Without a provider key the model layer asks no
|
||||
question and reports only what the graph entailed. Runs are reproducible: fixed config,
|
||||
artifact manifests, prompt hashes, and model identifiers are recorded.
|
||||
A scan is never silently downgraded. When an optional engine is missing, the scan falls back to
|
||||
its deterministic equivalent. It records a `degradations` entry in the manifest (Joern:
|
||||
`cpg_unavailable`; Docker: `sandbox_unavailable`).
|
||||
|
||||
Without a provider key, the model layer asks no question. It reports only what the graph
|
||||
entailed.
|
||||
|
||||
Runs are reproducible. The fixed config, artifact manifests, prompt hashes and model identifiers
|
||||
are recorded.
|
||||
|
||||
### MCP surface
|
||||
`ousast mcp` exposes only the ten narrow project tools. No tool runs arbitrary shell,
|
||||
Docker, or internal hunter tools, and no tool accepts a free-form command argument.
|
||||
`ousast mcp` exposes only the ten narrow project tools. No tool runs arbitrary shell, Docker or
|
||||
internal hunter tools. No tool accepts a free-form command argument.
|
||||
|
||||
## Sandbox (deep mode)
|
||||
|
||||
The REGRESS stage (`regress/`, `sandbox/runner.py`) runs each snippet with `docker run`
|
||||
and never mounts the host Docker socket:
|
||||
The REGRESS stage (`regress/`, `sandbox/runner.py`) runs each snippet with `docker run`. It never
|
||||
mounts the host Docker socket.
|
||||
|
||||
| Control | Setting |
|
||||
|---|---|
|
||||
@@ -84,5 +107,5 @@ and never mounts the host Docker socket:
|
||||
| pids | `[sandbox] pids_limit`, default 512 |
|
||||
| timeout | `[sandbox] timeout_seconds`, default 300 s |
|
||||
|
||||
Before anything runs, a structural deny-list (`regress/safety.py`) rejects a snippet that
|
||||
mentions sockets, curl, host networking or writes under `/workspace`.
|
||||
A structural deny-list (`regress/safety.py`) checks each snippet before anything runs. It rejects
|
||||
a snippet that mentions sockets, curl, host networking or writes under `/workspace`.
|
||||
|
||||
+200
-128
@@ -1,18 +1,26 @@
|
||||
# Token ergonomics
|
||||
|
||||
Model tokens are the one running cost of this project, and they are spent in two places: by the
|
||||
**plane** when a Run verifies candidates or compiles the decision engine, and by the maintainers'
|
||||
own **agentic coding sessions**. The maintainer's framing (2026-09-29): "the token saving strategy
|
||||
is a routing concern. ax can handle that and structure our services in small components." So the
|
||||
runtime half is routing, one Model per Task with its own budget, and reuse of what was already
|
||||
paid for; the development half is tooling that keeps bulk file contents out of the session.
|
||||
Model tokens are the one running cost of this project. They are spent in two places:
|
||||
|
||||
Every number on this page comes from a committed record, named next to it. Records are counts
|
||||
only; none holds code, prompts or responses.
|
||||
- by the **plane** (the system that runs scans as small tasks) when a Run verifies candidates
|
||||
or compiles the decision engine;
|
||||
- by the maintainers' own **agentic coding sessions**.
|
||||
|
||||
The before-and-after summary (script pipeline against plane, and the development share) is on
|
||||
[Where we stand](where-we-stand.md#4-token-economics-before-and-after); this page holds the
|
||||
mechanisms and the detail table.
|
||||
The maintainer's framing (2026-09-29): "the token saving strategy is a routing concern. ax can
|
||||
handle that and structure our services in small components." Here ax is Google's controller
|
||||
that runs agent tasks on Kubernetes. So the two halves look like this:
|
||||
|
||||
- **Runtime** is routing: one Model per Task with its own budget, and reuse of what was
|
||||
already paid for.
|
||||
- **Development** is tooling that keeps bulk file contents out of the session.
|
||||
|
||||
Every number on this page comes from a committed record, named next to it. Records hold counts
|
||||
only. None holds code, prompts or responses.
|
||||
|
||||
The before-and-after summary is on
|
||||
[Where we stand](where-we-stand.md#4-token-economics-before-and-after). It compares the script
|
||||
pipeline with the plane and gives the development share. This page holds the mechanisms and the
|
||||
detail table.
|
||||
|
||||
## A. Runtime: the plane's token economy
|
||||
|
||||
@@ -37,46 +45,67 @@ flowchart LR
|
||||
|
||||
### One Model per Task, one budget per Task
|
||||
|
||||
A Run is a DAG of ax Tasks; each Task binds at most one Model and its own `budget: {usd, calls}`
|
||||
([The plane on ax](plane.md)). The budget is enforced by the metered client
|
||||
(`src/openultrasast/plane/budget.py`, `MeteredClient`): `check()` runs before every call and
|
||||
raises `BudgetExhausted` once the spent `usd` or the call count reaches the ceiling, so the call
|
||||
that crosses the ceiling completes and the next one is refused. The task then ends `unfinished`
|
||||
and resumes from its `units.jsonl` on a rerun with a larger budget; an HTTP 401/402 or
|
||||
"Insufficient Balance" is an `AccountError`, which fails the task and starts nothing further.
|
||||
Usage is attributed per call: the wrapped client appends one usage row per call
|
||||
(`prompt_tokens`, `prompt_cache_hit_tokens`, `completion_tokens`), rows are absorbed even when
|
||||
the call raises, and they are priced by the Model's price list. An unpriced Model cannot run
|
||||
under a `usd` budget, and `summary()` reports `usd: None` for it, never 0. Model-free tasks run
|
||||
with `{usd: 0, calls: 0}`, so a stray call fails loudly.
|
||||
A Run is a DAG (a graph of steps with no cycles) of ax Tasks. Each Task binds at most one Model.
|
||||
Each Task has its own `budget: {usd, calls}` ([The plane on ax](plane.md)).
|
||||
|
||||
`ousast plane status RUN` is the attribution table: one row per task with `calls`, `prompt`,
|
||||
`cache_hit`, `output` and `usd` from each task's `summary.json`, a `subtotal` row per Model and a
|
||||
`total` (a missing value makes a sum `n/a`, never a smaller number); `--units` adds the per-unit
|
||||
rows. The same table is written as `attribution.json` in the run directory
|
||||
(`src/openultrasast/plane/reconciler.py`, `attribution()`/`status()`). Provider credentials
|
||||
reach a task only in the start request: `router.py` reads the variable the Model's
|
||||
`secretKey.key` names, the runner keeps the value in memory, passes it to the command's
|
||||
environment and redacts it from echoed stderr; it never appears in a manifest, a rendered Task
|
||||
or a file.
|
||||
The metered client enforces the budget (`src/openultrasast/plane/budget.py`, `MeteredClient`):
|
||||
|
||||
1. `check()` runs before every call.
|
||||
2. It raises `BudgetExhausted` once the spent `usd` or the call count reaches the ceiling.
|
||||
3. So the call that crosses the ceiling completes, and the next one is refused.
|
||||
4. The task then ends `unfinished`. A rerun with a larger budget resumes from its `units.jsonl`.
|
||||
|
||||
An HTTP 401/402 or "Insufficient Balance" is an `AccountError`. It fails the task and starts
|
||||
nothing further.
|
||||
|
||||
Usage is attributed per call:
|
||||
|
||||
- The wrapped client appends one usage row per call (`prompt_tokens`,
|
||||
`prompt_cache_hit_tokens`, `completion_tokens`).
|
||||
- Rows are absorbed even when the call raises.
|
||||
- Rows are priced by the Model's price list.
|
||||
- An unpriced Model cannot run under a `usd` budget. `summary()` reports `usd: None` for it,
|
||||
never 0.
|
||||
- Model-free tasks run with `{usd: 0, calls: 0}`, so a stray call fails loudly.
|
||||
|
||||
`ousast plane status RUN` prints the attribution table. It reads each task's `summary.json`
|
||||
and shows:
|
||||
|
||||
- one row per task with `calls`, `prompt`, `cache_hit`, `output` and `usd`;
|
||||
- a `subtotal` row per Model;
|
||||
- a `total`. A missing value makes a sum `n/a`, never a smaller number.
|
||||
|
||||
`--units` adds the per-unit rows. The same table is written as `attribution.json` in the run
|
||||
directory (`src/openultrasast/plane/reconciler.py`, `attribution()`/`status()`).
|
||||
|
||||
Provider credentials reach a task only in the start request:
|
||||
|
||||
- `router.py` reads the variable that the Model's `secretKey.key` names.
|
||||
- The runner keeps the value in memory and passes it to the command's environment.
|
||||
- The runner redacts it from echoed stderr.
|
||||
- It never appears in a manifest, a rendered Task or a file.
|
||||
|
||||
### Pay once, reuse
|
||||
|
||||
- **Repo facts are computed model-free and reused.** `repo-facts` reads source text only (no
|
||||
model, no git) and its `facts.json` is byte-identical across runs over the same tree. The
|
||||
memory store keeps a facts entry under the key `(repo, pin, candidates digest, runner image
|
||||
digest)` (`src/openultrasast/plane/memory.py`, `memory_key()`); when a later Run's task
|
||||
carries the same key, `seed()` writes the stored `facts.json`, a `done` summary with
|
||||
`calls: 0` and no `usd`, and marks the task done without starting an actor. A changed pin,
|
||||
candidate set or image finds nothing and recomputes.
|
||||
- **Content-addressed memory** (`src/openultrasast/learn/`). Excerpts are stored as
|
||||
`excerpts/<sha256 of the normalised text>.txt`; an embedding lives at
|
||||
`embeddings/<model>/<excerpt sha>.json` and "an excerpt is embedded once; a rerun makes no
|
||||
call" (`embeddings.py`). Model responses are cached under the sha256 of
|
||||
`{model, messages, params, temperature, sample}` (`program.py`, `request_key()`), and a
|
||||
compiled program is `programs/<sha256 of its artifact>.json` with the digests of the memory
|
||||
snapshot and of the response cache it was built from (`compile.py`).
|
||||
- **Replay is free.** With no client, the response cache replays at $0 and a miss raises
|
||||
- **Repo facts are computed model-free and reused.**
|
||||
- `repo-facts` reads source text only (no model, no git).
|
||||
- Its `facts.json` is byte-identical across runs over the same tree.
|
||||
- The memory store keeps a facts entry under the key `(repo, pin, candidates digest, runner image
|
||||
digest)` (`src/openultrasast/plane/memory.py`, `memory_key()`).
|
||||
- A later Run's task may carry the same key. Then `seed()` writes the stored `facts.json` and a
|
||||
`done` summary with `calls: 0` and no `usd`. It marks the task done without starting an
|
||||
actor.
|
||||
- A changed pin, candidate set or image finds nothing and recomputes.
|
||||
- **Content-addressed memory** (`src/openultrasast/learn/`). Each item is stored under a hash
|
||||
of its content.
|
||||
- Excerpts are stored as `excerpts/<sha256 of the normalised text>.txt`.
|
||||
- An embedding lives at `embeddings/<model>/<excerpt sha>.json`. The source says: "an excerpt
|
||||
is embedded once; a rerun makes no call" (`embeddings.py`).
|
||||
- Model responses are cached under the sha256 of `{model, messages, params, temperature, sample}`
|
||||
(`program.py`, `request_key()`).
|
||||
- A compiled program is `programs/<sha256 of its artifact>.json`. It records the digests of
|
||||
the memory snapshot and of the response cache it was built from (`compile.py`).
|
||||
- **Replay is free.** With no client, the response cache replays at $0. A miss raises
|
||||
`ReplayMiss` instead of calling. The injection-slice record states it: "evaluate re-run
|
||||
replay-only on the committed code: 550 responses replayed, $0, identical overall and
|
||||
calibration" (`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`,
|
||||
@@ -84,26 +113,35 @@ or a file.
|
||||
|
||||
### Cheap before expensive
|
||||
|
||||
- **Triage first.** Verification runs only over the candidates the recorded triage kept; the
|
||||
plane applies that recording without a model call and prices it as a separate line
|
||||
(`recorded_triage_usd`), so every per-candidate cost below is reported both with and without
|
||||
it (`src/openultrasast/plane/tasks/agree.py`).
|
||||
- **Triage first.** Verification runs only over the candidates the recorded triage kept.
|
||||
- The plane applies that recording without a model call.
|
||||
- It prices the recording as a separate line (`recorded_triage_usd`).
|
||||
- So every per-candidate cost below is reported both with and without it
|
||||
(`src/openultrasast/plane/tasks/agree.py`).
|
||||
- **Hunts are batched per file.** `verify` runs one tool hunt per file for up to `PER_HUNT = 6`
|
||||
candidates, with the candidates' known callers (capped at 8) in the prompt
|
||||
candidates. The prompt includes the candidates' known callers, capped at 8
|
||||
(`src/openultrasast/plane/tasks/verify.py`, `units_of()`).
|
||||
- **The tie-break pass runs only on disputes.** `agree` compares passes a and b; pass c is
|
||||
started with `inputs.only` set to the case's `disputed.json`, an empty list ends it `done`
|
||||
with no model call, and the final verdict is 2-of-3 ("pass c alone never decides",
|
||||
`agree.py`).
|
||||
- **Cache-friendly prompt order.** The decision engine's prompt is the compiled prefix
|
||||
(instruction and fixed demonstrations, identical for every candidate so the provider's
|
||||
prefix cache hits), then the retrieved examples, then the candidate (`learn/program.py`).
|
||||
Its self-consistency `k` is tied to that cache: "k = 5 when samples 2..k hit the prefix cache
|
||||
for >= 0.8 of their prompt tokens"; the measured repeat-hit share was 0.96 (injection-slice
|
||||
record, `smoke.k_rule`, `smoke.k5_repeat_hit_share_measured`). The verify hunts share a prefix
|
||||
too: in the first plane increment 2,385,152 of 3,820,395 prompt tokens were cache hits
|
||||
(`benchmarks/independent/plane-increment-1.json`, `detail.cache_hit_tokens`,
|
||||
`detail.prompt_tokens`; its reading: "Cost fell 17% (62% of prompt tokens were cache hits)").
|
||||
- **The tie-break pass runs only on disputes.**
|
||||
- `agree` compares passes a and b.
|
||||
- Pass c starts with `inputs.only` set to the case's `disputed.json`.
|
||||
- An empty list ends it `done` with no model call.
|
||||
- The final verdict is 2-of-3 ("pass c alone never decides", `agree.py`).
|
||||
- **Cache-friendly prompt order.** The decision engine's prompt has three parts, in this order
|
||||
(`learn/program.py`):
|
||||
1. The compiled prefix: instruction and fixed demonstrations. It is identical for every
|
||||
candidate, so the provider's prefix cache hits.
|
||||
2. The retrieved examples.
|
||||
3. The candidate.
|
||||
|
||||
The self-consistency `k` (the number of samples per candidate) is tied to that cache: "k = 5
|
||||
when samples 2..k hit the prefix cache for >= 0.8 of their prompt tokens". The measured
|
||||
repeat-hit share was 0.96 (injection-slice record, `smoke.k_rule`,
|
||||
`smoke.k5_repeat_hit_share_measured`).
|
||||
|
||||
The verify hunts share a prefix too. In the first plane increment, 2,385,152 of 3,820,395
|
||||
prompt tokens were cache hits (`benchmarks/independent/plane-increment-1.json`,
|
||||
`detail.cache_hit_tokens`, `detail.prompt_tokens`). The record's reading: "Cost fell 17% (62%
|
||||
of prompt tokens were cache hits)".
|
||||
|
||||
### Measured costs
|
||||
|
||||
@@ -116,37 +154,41 @@ or a file.
|
||||
| Decision engine harvest: `verify` part | $2.5373 over 1,961 calls, 474 tasks done, 10,921,167 prompt tokens of which 6,275,060 cache hits, 306,941 output tokens, task budgets summing to $8.624 under a $10 ceiling (`verify`) | `benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json` |
|
||||
| Decision engine harvest: `roles` part | $1.4036 over 1,034 calls, 151 tasks done, 2,398,669 prompt tokens of which 329,600 cache hits, 370,155 output tokens, task budgets summing to $4.917 under a $10 ceiling (`roles`) | same record |
|
||||
|
||||
The increment-2 record's note on the two denominators: "the reference divides by the 46
|
||||
The increment-2 record notes that the two denominators differ: "the reference divides by the 46
|
||||
candidates before triage; on that basis the plane is 7% cheaper, on the 43 after triage it is
|
||||
0.2% under".
|
||||
|
||||
!!! note "The meter prices at list; the account moved less"
|
||||
The attribution table prices every call at the Model's list price
|
||||
(`plane/models/deepseek-flash.yaml`). The harvest record: "Spend is the plane's attribution
|
||||
table (`ousast plane status`, task meters priced at plane/models/deepseek-flash.yaml); the
|
||||
DeepSeek account moved less (balance below)", with the balance at $19.63 before, $18.75
|
||||
after `verify` and $18.23 after `roles`. The injection-slice record names the ratio: "the
|
||||
provider charged ~1/3 of the list-price meter (balance moved $0.30 for $0.91 metered):
|
||||
DeepSeek's off-peak discount window, presumably; the meter prices at list"
|
||||
(`spend.balance_note`, balance $18.2 to $17.9). Budgets and gates are set against the
|
||||
meter, never against the balance.
|
||||
(`plane/models/deepseek-flash.yaml`). The harvest record says: "Spend is the plane's
|
||||
attribution table (`ousast plane status`, task meters priced at
|
||||
plane/models/deepseek-flash.yaml); the DeepSeek account moved less (balance below)".
|
||||
The balance was $19.63 before, $18.75 after `verify` and $18.23 after `roles`.
|
||||
The injection-slice record names the ratio: "the provider charged ~1/3 of the list-price
|
||||
meter (balance moved $0.30 for $0.91 metered): DeepSeek's off-peak discount window,
|
||||
presumably; the meter prices at list" (`spend.balance_note`, balance $18.2 to $17.9).
|
||||
Budgets and gates are set against the meter, never against the balance.
|
||||
|
||||
### Not done yet
|
||||
|
||||
- **No model-side token budget in ax itself.** The plane's requirements state what ax does
|
||||
not yet carry: "task dependencies, artifacts, token budgets (on ax's roadmap), a DeepSeek
|
||||
provider"; the thin `Run` layer carries them and dissolves into ax as ax gains them. Until
|
||||
then the ceiling lives in the metered client, not in the executor.
|
||||
- **Triage is applied, not run.** The plane has no triage task; it filters by the recorded
|
||||
triage and adds that recording's cost as its own line. A triage task is planned as a later
|
||||
specification.
|
||||
- **`k` and the contrast examples are not knobs.** `k` is a compiled setting in {1, 3, 5}
|
||||
(`learn/compile.toml`, "k and lam remain A/B variants"), changed only through the decision
|
||||
engine's A/B experiments, its requirement that "changes to prompts, models, pass counts, rule
|
||||
sets and features [are] compared by controlled experiments, so that nothing is adopted on one
|
||||
run's number" (Requirement 4 of its specification). The retrieval cap of at most two
|
||||
contrast examples from other families is a module constant (`learn/retrieve.py`,
|
||||
`CONTRAST = 2`), a maintainer decision, not an experiment variant yet.
|
||||
- **No model-side token budget in ax itself.**
|
||||
- The plane's requirements state what ax does not yet carry: "task dependencies, artifacts,
|
||||
token budgets (on ax's roadmap), a DeepSeek provider".
|
||||
- The thin `Run` layer carries them. It dissolves into ax as ax gains them.
|
||||
- Until then the ceiling lives in the metered client, not in the executor.
|
||||
- **Triage is applied, not run.**
|
||||
- The plane has no triage task.
|
||||
- It filters by the recorded triage and adds that recording's cost as its own line.
|
||||
- A triage task is planned as a later specification.
|
||||
- **`k` and the contrast examples are not knobs.**
|
||||
- `k` is a compiled setting in {1, 3, 5} (`learn/compile.toml`, "k and lam remain A/B
|
||||
variants").
|
||||
- It changes only through the decision engine's A/B experiments. Requirement 4 of its
|
||||
specification asks that "changes to prompts, models, pass counts, rule sets and features
|
||||
[are] compared by controlled experiments, so that nothing is adopted on one run's number".
|
||||
- Retrieval adds at most two contrast examples from other families. This cap is a module
|
||||
constant (`learn/retrieve.py`, `CONTRAST = 2`).
|
||||
- The cap is a maintainer decision, not an experiment variant yet.
|
||||
|
||||
## B. Development: the maintainers' own sessions
|
||||
|
||||
@@ -165,28 +207,47 @@ flowchart LR
|
||||
|
||||
### The measurement: tool I/O dominates
|
||||
|
||||
`benchmarks/dev/token_report.py` reads a Claude Code session transcript (`.jsonl`; without an
|
||||
argument, the newest under `~/.claude/projects/`) and reports, per content kind, characters and
|
||||
share: `tool_use_input` (the tool call's input as JSON), `tool_result`, `thinking`,
|
||||
`assistant:text`, `user:text`, plus the tool-call count and the top five tools. Tokens are
|
||||
approximated at four characters each. It prints to stdout and writes nothing; what gets
|
||||
committed is the counts-only summary. The one in `benchmarks/independent/plane-increment-1.json`
|
||||
(`dev_token_report`, for "the whole development session (c09e30b2), not the increment alone"):
|
||||
7.3M unique characters (~1.81M tokens), `tool_use_input_pct` 45.4, `tool_result_pct` 40.1,
|
||||
`tool_io_pct` 85.5 against `previous_tool_io_pct` 90, over 2,701 tool calls. The reasoning the
|
||||
session is paid for (assistant text and thinking) is the small remainder. That is why the
|
||||
remedies below target file contents, not prose.
|
||||
`benchmarks/dev/token_report.py` reads a Claude Code session transcript (`.jsonl`). Without an
|
||||
argument it reads the newest one under `~/.claude/projects/`.
|
||||
|
||||
It reports characters and share per content kind:
|
||||
|
||||
- `tool_use_input` (the tool call's input as JSON);
|
||||
- `tool_result`;
|
||||
- `thinking`;
|
||||
- `assistant:text`;
|
||||
- `user:text`.
|
||||
|
||||
It also reports the tool-call count and the top five tools. Tokens are approximated at four
|
||||
characters each. It prints to stdout and writes nothing. Only the counts-only summary is
|
||||
committed.
|
||||
|
||||
The committed summary is in `benchmarks/independent/plane-increment-1.json`
|
||||
(`dev_token_report`). It covers "the whole development session (c09e30b2), not the increment
|
||||
alone". It counts 7.3M unique characters (~1.81M tokens). The shares
|
||||
are `tool_use_input_pct` 45.4, `tool_result_pct` 40.1, and `tool_io_pct` 85.5 against
|
||||
`previous_tool_io_pct` 90, over 2,701 tool calls.
|
||||
|
||||
The reasoning the session is paid for (assistant text and thinking) is the small remainder.
|
||||
That is why the remedies below target file contents, not prose.
|
||||
|
||||
### The remedies in `benchmarks/dev/`
|
||||
|
||||
**`read_guard.py`**, a `PreToolUse` hook for agentic coding sessions. It reads the hook event on
|
||||
stdin and, for the `Read` tool only, denies a whole-file read of a file longer than 350 lines
|
||||
(`LIMIT = 350`); a read with `offset` or `limit`, a missing path, an unreadable file or a file
|
||||
within the limit passes. It always exits 0 ("never fails the tool"). A denial is one JSON line
|
||||
whose `permissionDecisionReason` tells the agent to read a range or to ask the bulk reader. The
|
||||
script's docstring records that it is wired in `.claude/settings.local.json` (a local, untracked
|
||||
file) as `PreToolUse` with matcher `Read`; the setting has this shape, with the command
|
||||
repo-relative because hooks run from the project directory:
|
||||
**`read_guard.py`** is a `PreToolUse` hook (a script that runs before each tool call) for
|
||||
agentic coding sessions.
|
||||
|
||||
- It reads the hook event on stdin.
|
||||
- For the `Read` tool only, it denies a whole-file read of a file longer than 350 lines
|
||||
(`LIMIT = 350`).
|
||||
- These pass: a read with `offset` or `limit`, a missing path, an unreadable file, or a file
|
||||
within the limit.
|
||||
- It always exits 0 ("never fails the tool").
|
||||
- A denial is one JSON line. Its `permissionDecisionReason` tells the agent to read a range or
|
||||
to ask the bulk reader.
|
||||
|
||||
The script's docstring records how it is wired. It sits in `.claude/settings.local.json` (a
|
||||
local, untracked file) as `PreToolUse` with matcher `Read`. The setting has this shape. The
|
||||
command is repo-relative because hooks run from the project directory:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -203,37 +264,48 @@ repo-relative because hooks run from the project directory:
|
||||
}
|
||||
```
|
||||
|
||||
**`bulk_read.py`**, the portal pattern: a cheap model reads the files and answers one question,
|
||||
citing `path:line` for every claim; it reports what the files say and decides nothing.
|
||||
**`bulk_read.py`** follows the portal pattern. A cheap model reads the files and answers one
|
||||
question. It cites `path:line` for every claim. It reports what the files say and decides
|
||||
nothing.
|
||||
|
||||
```text
|
||||
.venv/bin/python benchmarks/dev/bulk_read.py "<question>" FILE [FILE ...] [--max-usd 0.05]
|
||||
```
|
||||
|
||||
Files go to the model in 400-line chunks with a `--- path (lines a-b)` header and numbered
|
||||
lines; the client is the project's detector client (`deepseek-flash` through `DEEPSEEK_API_KEY`,
|
||||
otherwise OpenRouter), each call has a 90 s timeout and no tools. `--max-usd` (default 0.05) is
|
||||
checked before every chunk; at the ceiling the answer ends with
|
||||
`[stopped at the $X ceiling before: <chunk header>]`. The answer goes to stdout, the summary
|
||||
`[bulk_read: N chunk(s), $X.XXXX]` to stderr. Without a chat client it exits 2.
|
||||
How it works:
|
||||
|
||||
- Files go to the model in 400-line chunks, with a `--- path (lines a-b)` header and numbered
|
||||
lines.
|
||||
- The client is the project's detector client: `deepseek-flash` through `DEEPSEEK_API_KEY`,
|
||||
otherwise OpenRouter.
|
||||
- Each call has a 90 s timeout and no tools.
|
||||
- `--max-usd` (default 0.05) is checked before every chunk. At the ceiling the answer ends with
|
||||
`[stopped at the $X ceiling before: <chunk header>]`.
|
||||
- The answer goes to stdout. The summary `[bulk_read: N chunk(s), $X.XXXX]` goes to stderr.
|
||||
- Without a chat client it exits 2.
|
||||
|
||||
**`token_report.py`**: `python benchmarks/dev/token_report.py [TRANSCRIPT.jsonl]`, described
|
||||
above.
|
||||
|
||||
### Practices
|
||||
|
||||
- **Scripts into files, not heredocs.** A heredoc is `tool_use_input` and its output
|
||||
`tool_result`; both are paid again at every later turn of the session. A script on disk is
|
||||
read once, by the interpreter.
|
||||
- **Terse outputs.** Commands print counts, exit codes and the lines that matter; a harness
|
||||
fails loudly with the exit code rather than printing a table that has to be read back.
|
||||
- **Subagents with worktrees** for work whose output need not enter the main context: a
|
||||
subagent's reads and command output stay in its own context and only its conclusion comes
|
||||
back; a git worktree keeps its edits off the main checkout.
|
||||
- **Scripts into files, not heredocs.** A heredoc (a script typed inline in a shell command) is
|
||||
`tool_use_input`, and its output is `tool_result`. Both are paid again at every later turn of
|
||||
the session. A script on disk is read once, by the interpreter.
|
||||
- **Terse outputs.** Commands print counts, exit codes and the lines that matter. A harness
|
||||
fails loudly with the exit code. It does not print a table that has to be read back.
|
||||
- **Subagents with worktrees** for work whose output need not enter the main context.
|
||||
- A subagent's reads and command output stay in its own context.
|
||||
- Only its conclusion comes back.
|
||||
- A git worktree (a separate checkout of the same repository) keeps its edits off the main
|
||||
checkout.
|
||||
- **Counts-only measurement records.** A record under `benchmarks/measurements/` or
|
||||
`benchmarks/independent/` holds numbers and their keys, never code, prompts or responses, so
|
||||
it can be quoted on a page like this one and reread by a model at a few hundred tokens.
|
||||
`benchmarks/independent/` holds numbers and their keys. It never holds code, prompts or
|
||||
responses. So it can be quoted on a page like this one. A model can reread it at a few
|
||||
hundred tokens.
|
||||
|
||||
Related: [The plane on ax](plane.md) for the Run and Task contract, [Memory](memory.md) for
|
||||
the store the reuse keys live in, [Evaluation](evaluation.md) for the gates these costs are
|
||||
measured against.
|
||||
Related:
|
||||
|
||||
- [The plane on ax](plane.md) for the Run and Task contract;
|
||||
- [Memory](memory.md) for the store the reuse keys live in;
|
||||
- [Evaluation](evaluation.md) for the gates these costs are measured against.
|
||||
|
||||
+261
-176
@@ -1,33 +1,50 @@
|
||||
# Where we stand
|
||||
|
||||
State as of 2026-10-02 (v2.0.0). This page answers five questions in one place. How far is
|
||||
detection from its gates. How does the project now try to close that gap. How does the model work
|
||||
run on ax. What did the tokens cost before and after that move. What is still open. Every figure
|
||||
is quoted from a committed record whose path stands next to it; detail tables stay on the pages
|
||||
linked from each section.
|
||||
State as of 2026-10-02 (v2.0.0). This page answers five questions in one place:
|
||||
|
||||
The goal has not changed: a safety net on repositories the tool has never seen, which says
|
||||
BLOCK or ADVISORY about the code a push would change and stays silent on fixes and on benign
|
||||
commits. Two independent populations have been spent against that goal, with engine recall 1 of
|
||||
1. How far is detection from its gates?
|
||||
2. How does the project now try to close that gap?
|
||||
3. How does the model work run on ax?
|
||||
4. What did the tokens cost before and after that move?
|
||||
5. What is still open?
|
||||
|
||||
Every figure is quoted from a committed record. The record's path stands next to the figure.
|
||||
Detail tables stay on the pages linked from each section.
|
||||
|
||||
The goal has not changed. The tool should be a safety net on repositories it has never seen. It
|
||||
should say BLOCK or ADVISORY about the code a push would change. It should stay silent on fixes
|
||||
and on benign commits.
|
||||
|
||||
Two independent populations have been spent against that goal. An independent population is a
|
||||
set of cases frozen before any scan. Engine recall was 1 of
|
||||
11 and then 0 of 17. The hand edits that closed misses on the first population did not carry to
|
||||
the second. So the question is no longer which rule to add next. It is how a change to detection
|
||||
can be shown to work on code it was not tuned on, at a cost per candidate that can be paid for
|
||||
every push.
|
||||
the second. So the question is no longer which rule to add next. The question is this: how can a
|
||||
change to detection be shown to work on code it was not tuned on? And can that be done at a cost
|
||||
per candidate that can be paid for every push?
|
||||
|
||||
The answer this page records: the pre-push safety net is NO-GO against every M4 gate, and the
|
||||
project has stopped trying to reach the gates by hand. Every detection change now has to win a
|
||||
controlled measurement on held-out repositories, run on an agentic plane whose cost per candidate
|
||||
is known. The pieces of that method exist and have been measured. None of them has produced a
|
||||
BLOCK decision yet.
|
||||
The answer this page records:
|
||||
|
||||
- The pre-push safety net is NO-GO against every M4 gate.
|
||||
- The project has stopped trying to reach the gates by hand.
|
||||
- Every detection change now has to win a controlled measurement on held-out repositories. The
|
||||
measurement runs on an agentic plane whose cost per candidate is known.
|
||||
- The pieces of that method exist and have been measured. None of them has produced a BLOCK
|
||||
decision yet.
|
||||
|
||||
## 1. Detection capability and goals: NO-GO, measured
|
||||
|
||||
No M4 gate is met on an independent population. The qualification milestone for the pre-push
|
||||
safety net (M4) requires, together: at least 95% actionable precision, at least 90% supported
|
||||
recall, at least 95% supported-check completion, the runtime gates, and zero fixed-side or benign
|
||||
false alerts. A population is frozen before any
|
||||
case is scanned, scored once under a pre-registered protocol, then spent; it can inform changes
|
||||
but can qualify nothing afterwards ([Evaluation](evaluation.md#independent-populations-and-the-m4-gates)).
|
||||
No M4 gate is met on an independent population. M4 is the qualification milestone for the
|
||||
pre-push safety net. It requires all of these together:
|
||||
|
||||
- at least 95% actionable precision;
|
||||
- at least 90% supported recall;
|
||||
- at least 95% supported-check completion;
|
||||
- the runtime gates;
|
||||
- zero fixed-side or benign false alerts.
|
||||
|
||||
A population is frozen before any case is scanned. It is scored once under a pre-registered
|
||||
protocol, then spent. After that it can inform changes but can qualify nothing
|
||||
([Evaluation](evaluation.md#independent-populations-and-the-m4-gates)).
|
||||
|
||||
| Gate | Required | Latest independent result | Record |
|
||||
| --- | --- | --- | --- |
|
||||
@@ -36,156 +53,217 @@ but can qualify nothing afterwards ([Evaluation](evaluation.md#independent-popul
|
||||
| Supported-check completion | >= 95% | 13,206 of 108,374 | `results-v2.json` |
|
||||
| Fixed-side and benign alerts | zero | 0 fixed-side, which the record calls met trivially since nothing was detected; 3 benign alerts | `results-v2.json` |
|
||||
|
||||
Three later measurements narrow the picture without changing the verdict.
|
||||
Three later measurements narrow the picture. They do not change the verdict.
|
||||
|
||||
- The plane reproduces the script pipeline's agreement at a known cost. On the model-driven
|
||||
pipeline's validation set (46 candidates, 15 cases of v2, 20 declared vulnerable sites), the
|
||||
plane's verify passes a and b plus a tie-break on the 10 disputed candidates agreed on 16 of 20
|
||||
declared sites. The script reference agreed on 16 as well. The cost was $0.0205 per candidate
|
||||
(`benchmarks/independent/plane-increment-2.json`). The record carries two caveats. The tie-break
|
||||
was chosen after seeing the first measurement on the same set, so this compares plane with
|
||||
script and qualifies nothing. And the passes flag fewer declared sites than the reference in
|
||||
any pass, 17 against 19; that count is a correction in `plane-increment-1.json`, whose first
|
||||
reading took agree's `declared_sites_matched` (19, coverage by candidates) for "found in either
|
||||
pass".
|
||||
- The learned decision engine ranks, but may not block. On six families, evaluated on
|
||||
repositories the compiled program was not shown, the pooled AUC of its score runs from 0.77
|
||||
(injection) to 0.96 (access control). The stricter reading is the within-pair AUC, which asks
|
||||
whether the vulnerable side of a fix scores above its own fixed side: 0.72 to 0.93. BLOCK is
|
||||
offered for no family; calibration holds only for output encoding, and there no fold reaches
|
||||
the required precision (`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
|
||||
The per-family table is on [Decision engine](decision-engine.md#second-slice-six-families-2026-10-02).
|
||||
- PHP coverage is in-sample. The PHP quick rules were written and tightened while looking at
|
||||
the pins and fixtures they are scored on, so the record says so itself (`provenance.in_sample` in
|
||||
`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`): a development check, not
|
||||
a recall estimate. On real PHP code the Joern engine is the instrument.
|
||||
- **The plane reproduces the script pipeline's agreement at a known cost.** The test used the
|
||||
model-driven pipeline's validation set: 46 candidates, 15 cases of v2, 20 declared vulnerable
|
||||
sites. The plane ran verify passes a and b, plus a tie-break on the 10 disputed candidates. It
|
||||
agreed on 16 of 20 declared sites. The script reference agreed on 16 as well. The cost was
|
||||
$0.0205 per candidate (`benchmarks/independent/plane-increment-2.json`). The record carries two
|
||||
caveats:
|
||||
1. The tie-break was chosen after seeing the first measurement on the same set. So this compares
|
||||
plane with script and qualifies nothing.
|
||||
2. The passes flag fewer declared sites than the reference in any pass: 17 against 19 (passes against reference). That
|
||||
count is a correction in `plane-increment-1.json`. Its first reading took agree's
|
||||
`declared_sites_matched` (19, coverage by candidates) to mean "found in either pass".
|
||||
- **The learned decision engine ranks, but may not block.** It was evaluated on six families, on
|
||||
repositories the compiled program was not shown.
|
||||
- The pooled AUC (area under the ROC curve) of its score runs from 0.77 (injection) to 0.96
|
||||
(access control).
|
||||
- The stricter reading is the within-pair AUC. It asks whether the vulnerable side of a fix
|
||||
scores above its own fixed side: 0.72 to 0.93.
|
||||
- BLOCK is offered for no family. Calibration holds only for output encoding. There, no fold
|
||||
reaches the required precision
|
||||
(`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`).
|
||||
- The per-family table is on
|
||||
[Decision engine](decision-engine.md#second-slice-six-families-2026-10-02).
|
||||
- **PHP coverage is in-sample.** The PHP quick rules were written and tightened while looking at
|
||||
the pins and fixtures they are scored on. The record says so itself (`provenance.in_sample` in
|
||||
`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`). It is a development
|
||||
check, not a recall estimate. On real PHP code, the Joern engine is the instrument.
|
||||
|
||||
What has not been touched: population v3, 15 PHP cases, frozen as the one-time final check. No
|
||||
development or analysis work reads its cases, a test fails when anything in the tree names one
|
||||
of its repositories, and it runs once, after a prediction is committed.
|
||||
What has not been touched: population v3, 15 PHP cases, frozen as the one-time final check.
|
||||
|
||||
- No development or analysis work reads its cases.
|
||||
- A test fails when anything in the tree names one of its repositories.
|
||||
- It runs once, after a prediction is committed.
|
||||
|
||||
## 2. Our approach to detection, step by step
|
||||
|
||||
The path changed because hand edits did not transfer. Every gain before 2026-09-30 was a hand
|
||||
edit after a miss: a rule, a taint vocabulary entry, a guard, a prompt line, a tie-break. Each was
|
||||
tuned on the cases that exposed it, and the tuning measured on population v1 did not carry to v2
|
||||
(1 of 11, then 0 of 17). The conclusions drawn: hand edits do not transfer. The data that could
|
||||
teach a better decision cannot be produced by hand at the scale needed, so it has to be harvested
|
||||
and labelled by machinery with a boundary around it. Framework knowledge (which function is a sink in which
|
||||
framework) is useful as a prior but must stay optional, or the engine learns the vocabulary
|
||||
instead of the code. A reader might object that a classifier is slower to show gains than the
|
||||
next rule. It is. The difference is that its gains are measured where rules were not: on
|
||||
repositories it never saw.
|
||||
tuned on the cases that exposed it. The tuning measured on population v1 did not carry to v2
|
||||
(1 of 11, then 0 of 17).
|
||||
|
||||
The chain, one job per step. The design is on [Memory and detection](memory-and-detection.md);
|
||||
this is the order of operations.
|
||||
The conclusions drawn:
|
||||
|
||||
1. Instruments produce signals, never verdicts. Quick rules, the Joern engine, model sink
|
||||
classification, the plane's verify passes a and b with the tie-break c, repository facts
|
||||
(callers, spans) and model source/sink/sanitizer roles each write one block of a candidate's
|
||||
record. None decides alone.
|
||||
2. Feature records are allow-listed, and missing is not zero. A candidate is `(path, function,
|
||||
family)` of one repository and pin. An instrument that did not run, had no coverage or failed
|
||||
records that state; "silent" and "did not look" stay distinguishable. Input profile v1 withholds
|
||||
the engine block and the function length, which a leak audit found separate the two sides of a
|
||||
pair by construction (`benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`), and
|
||||
since 2026-10-02 the plane's `verify.flag_b` and `ms.operation`, flagged the same way by the
|
||||
audit of the plane profile (`2026-10-02-decision-engine-harvest-advisory/record.json`,
|
||||
`src/openultrasast/learn/schema.py`).
|
||||
3. Labels come from ground truth only. A fail-closed allow-list (`src/openultrasast/learn/sources.toml`)
|
||||
names the sources: the spent populations' declared sites and fix ranges, the vulnerable and
|
||||
fixed sides of pairs, including the 97 real public fixes in `benchmarks/pairs/advisory-fixes/`,
|
||||
benign controls and recorded adjudications. Assumed-benign commits are their own source,
|
||||
weighted 0.5 and reported separately. A plane verdict is a feature, never a label: "rejected by
|
||||
the plane" is not a negative, or the engine would learn to reproduce the plane's mistakes.
|
||||
4. Retrieval finds similar labelled cases. A Gower distance over the signal profile, re-ranked
|
||||
by cosine similarity of code embeddings (OpenAI `text-embedding-3-small` through OpenRouter,
|
||||
cached by excerpt hash). The evaluation boundary is the only door into a prompt: never the
|
||||
candidate's own repository group, never a group the fold evaluates.
|
||||
5. A classifier compiled in house, DSPy-style. `retrieve -> classify(k)`: an instruction and
|
||||
compiled demonstrations (a fixed prefix, so the provider's cache hits), the retrieved examples,
|
||||
then the candidate's excerpt and signals. The model answers vulnerable, not vulnerable or
|
||||
unsure with a rationale that cites a line; `k` samples give a score. One program is compiled
|
||||
per profile and family on a compile split that is never evaluated.
|
||||
6. Calibration on held-out repositories. Platt scaling of the score, cross-fitted by
|
||||
repository group so no map is checked on the fold that fitted it. Calibration holds when the
|
||||
slope interval contains 1, the intercept interval contains 0 and ECE <= 0.05.
|
||||
7. Two operating points. BLOCK only where calibration holds and a fold reaches the required
|
||||
precision; otherwise ADVISORY. An unsure majority can never BLOCK.
|
||||
8. A/B experiments are the only way a change is adopted. Prompts, models, pass counts,
|
||||
features and sampling are compared under a pre-registered manifest with frozen units, paired
|
||||
arms, a repository bootstrap and two looks. exp-002 is the first to finish: a retrieval
|
||||
ensemble (five samples over disjoint example sets at temperature 0) against today's
|
||||
temperature sampling. It raised re-run agreement a little (0.906 against 0.875, interval
|
||||
spanning zero) and lowered the within-pair AUC from 0.815 to 0.741 at 2.4 times the cost per
|
||||
candidate ($0.0051 against $0.0021). Decision: REJECTED
|
||||
(`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`).
|
||||
9. Extrapolation with intervals. Every number carries a 95% repository-cluster bootstrap
|
||||
interval; the pooled and the within-pair readings are both reported, and the stricter one is
|
||||
- Hand edits do not transfer.
|
||||
- The data that could teach a better decision cannot be produced by hand at the scale needed. It
|
||||
has to be harvested and labelled by machinery with a boundary around it.
|
||||
- Framework knowledge (which function is a sink in which framework) is useful as a prior. But it
|
||||
must stay optional. Otherwise the engine learns the vocabulary instead of the code.
|
||||
|
||||
A reader might object that a classifier is slower to show gains than the next rule. It is. The
|
||||
difference is where its gains are measured: on repositories it never saw. Rules were not measured
|
||||
there.
|
||||
|
||||
The chain has one job per step. The design is on [Memory and detection](memory-and-detection.md).
|
||||
This list is the order of operations.
|
||||
|
||||
1. **Instruments produce signals, never verdicts.** Each instrument writes one block of a
|
||||
candidate's record. None decides alone. The instruments are:
|
||||
- quick rules;
|
||||
- the Joern engine;
|
||||
- model sink classification;
|
||||
- the plane's verify passes a and b, with the tie-break c;
|
||||
- repository facts (callers, spans);
|
||||
- model source/sink/sanitizer roles.
|
||||
2. **Feature records are allow-listed, and missing is not zero.** A candidate is `(path, function,
|
||||
family)` of one repository and pin.
|
||||
- An instrument that did not run, had no coverage or failed records that state. "Silent" and
|
||||
"did not look" stay distinguishable.
|
||||
- Input profile v1 withholds the engine block and the function length. A leak audit found that
|
||||
they separate the two sides of a pair by construction
|
||||
(`benchmarks/measurements/2026-10-01-decision-engine-leak-audit/`).
|
||||
- Since 2026-10-02 it also withholds the plane's `verify.flag_b` and `ms.operation`. The audit
|
||||
of the plane profile flagged them the same way
|
||||
(`2026-10-02-decision-engine-harvest-advisory/record.json`,
|
||||
`src/openultrasast/learn/schema.py`).
|
||||
3. **Labels come from ground truth only.** A fail-closed allow-list
|
||||
(`src/openultrasast/learn/sources.toml`) names the sources:
|
||||
- the spent populations' declared sites and fix ranges;
|
||||
- the vulnerable and fixed sides of pairs, including the 97 real public fixes in
|
||||
`benchmarks/pairs/advisory-fixes/`;
|
||||
- benign controls;
|
||||
- recorded adjudications.
|
||||
|
||||
Assumed-benign commits are their own source. They are weighted 0.5 and reported separately. A
|
||||
plane verdict is a feature, never a label. "Rejected by the plane" is not a negative.
|
||||
Otherwise the engine would learn to reproduce the plane's mistakes.
|
||||
4. **Retrieval finds similar labelled cases.** It uses a Gower distance (a distance for mixed
|
||||
numeric and categorical data) over the signal profile. Results are re-ranked by cosine
|
||||
similarity of code embeddings. The embeddings are OpenAI `text-embedding-3-small` through
|
||||
OpenRouter, cached by excerpt hash. The evaluation boundary is the only door into a prompt. It
|
||||
never admits the candidate's own repository group, or a group the fold evaluates.
|
||||
5. **A classifier compiled in house, DSPy-style.** The program is `retrieve -> classify(k)`. The
|
||||
prompt holds, in order:
|
||||
- an instruction and compiled demonstrations (a fixed prefix, so the provider's cache hits);
|
||||
- the retrieved examples;
|
||||
- the candidate's excerpt and signals.
|
||||
|
||||
The model answers vulnerable, not vulnerable or unsure, with a rationale that cites a line. `k`
|
||||
samples give a score. One program is compiled per profile and family, on a compile split that
|
||||
is never evaluated.
|
||||
6. **Calibration on held-out repositories.** Platt scaling (a logistic fit) maps the score to a
|
||||
probability. It is cross-fitted by repository group, so no map is checked on the fold that
|
||||
fitted it. Calibration holds when three conditions are met: the slope interval contains 1, the
|
||||
intercept interval contains 0 and ECE <= 0.05.
|
||||
7. **Two operating points.** BLOCK applies only where calibration holds and a fold reaches the
|
||||
required precision. Otherwise the point is ADVISORY. An unsure majority can never BLOCK.
|
||||
8. **A/B experiments are the only way a change is adopted.** Prompts, models, pass counts,
|
||||
features and sampling are compared under a pre-registered manifest. The manifest fixes frozen
|
||||
units, paired arms, a repository bootstrap and two looks. exp-002 is the first to finish:
|
||||
- It compared a retrieval ensemble (five samples over disjoint example sets at temperature 0)
|
||||
with today's temperature sampling.
|
||||
- It raised re-run agreement a little (0.906 against 0.875, interval spanning zero).
|
||||
- It lowered the within-pair AUC from 0.815 to 0.741 and cost 2.4 times as much per candidate
|
||||
($0.0051 against $0.0021).
|
||||
- Decision: REJECTED (`benchmarks/experiments/exp-002-retrieval-ensemble/result.json`).
|
||||
9. **Extrapolation with intervals.** Every number carries a 95% repository-cluster bootstrap
|
||||
interval. Both the pooled and the within-pair readings are reported. The stricter one is
|
||||
named.
|
||||
10. The one-time v3 check. When a program is adopted and a prediction is committed, population
|
||||
v3 is scored once under `benchmarks/independent/protocol-v3.md`.
|
||||
10. **The one-time v3 check.** It happens when a program is adopted and a prediction is
|
||||
committed. Then population v3 is scored once under `benchmarks/independent/protocol-v3.md`.
|
||||
|
||||
## 3. AI integration: ax on the server
|
||||
|
||||
All model work runs as Tasks on google/ax over Agent Substrate, in a single-node kind cluster on
|
||||
the maintainer's host, each Task in its own gVisor sandbox. There is no local subprocess path.
|
||||
The reconciler (`ousast plane run`) is a host process: it applies a Run's Workspaces and Tasks,
|
||||
resumes each Task, sends its start signal, collects its artifacts and records state. Flow and
|
||||
contract: [The plane on ax](plane.md); bring-up and the host's lessons: [ax on this host](ops/ax/README.md).
|
||||
All model work runs as Tasks on google/ax over Agent Substrate. They run in a single-node kind
|
||||
cluster on the maintainer's host. Each Task runs in its own gVisor sandbox. There is no local
|
||||
subprocess path.
|
||||
|
||||
The reconciler (`ousast plane run`) is a host process. It does five things:
|
||||
|
||||
1. applies a Run's Workspaces and Tasks;
|
||||
2. resumes each Task;
|
||||
3. sends its start signal;
|
||||
4. collects its artifacts;
|
||||
5. records state.
|
||||
|
||||
Flow and contract: [The plane on ax](plane.md). Bring-up and the host's lessons:
|
||||
[ax on this host](ops/ax/README.md).
|
||||
|
||||
What each Run guarantees:
|
||||
|
||||
- One Model and one budget per Task. A Task binds at most one ax Model (`deepseek-flash` for
|
||||
chat, `openrouter-embedding` for embeddings) and `budget: {usd, calls}`. The metered client
|
||||
refuses the next call at the ceiling; the task ends unfinished and resumes on a rerun. Model-free
|
||||
tasks run with a zero budget, so a stray call fails loudly.
|
||||
- Credentials exist only in the start request. Agent Substrate boots each template once as a
|
||||
golden actor, so the runner never starts by itself. It waits for `POST /ousast/v1/start`, which
|
||||
the reconciler sends through `atenet-router` to that actor alone. The key is read from the
|
||||
operator's environment or `.env`, kept in memory by the runner, passed to the command and
|
||||
redacted from echoed stderr; no manifest, rendered Task or file carries it.
|
||||
- Per-task egress. One EgressPolicy per actor through the `atenet-egress` gateway, deny by
|
||||
default, hostnames only: plain HTTP to the artifact receiver, TLS passthrough to the Workspaces'
|
||||
Git hosts and the Model's declared hosts. Deleted with the actor.
|
||||
- Artifacts come back through the gateway. The runner posts its output directory as a tar to
|
||||
the receiver, an HTTP server inside the reconciler process on the host, reached through a Service
|
||||
with an EndpointSlice to the host. That delivery is the task's completion; ax has no Completed
|
||||
phase.
|
||||
- Memory on S3-compatible RustFS. `OUSAST_MEMORY=s3://sast-memory` on a RustFS server. The
|
||||
store verifies the bucket at every start (versioning, lifecycle, tags, S3 Select) and refuses to
|
||||
start when anything is missing; there is no local or fetch-and-filter fallback. Each row kind
|
||||
has a fixed set of queryable fields, enforced at write time
|
||||
([Memory](memory.md#verify-at-startup-no-fallback), [RustFS setup](rustfs.md)).
|
||||
- **One Model and one budget per Task.** A Task binds at most one ax Model and
|
||||
`budget: {usd, calls}`. The Models are `deepseek-flash` for chat and `openrouter-embedding` for
|
||||
embeddings. The metered client refuses the next call at the ceiling. The task ends unfinished
|
||||
and resumes on a rerun. Model-free tasks run with a zero budget, so a stray call fails loudly.
|
||||
- **Credentials exist only in the start request.** Agent Substrate boots each template once as a
|
||||
golden actor. So the runner never starts by itself. It waits for `POST /ousast/v1/start`. The
|
||||
reconciler sends that request through `atenet-router` to that actor alone.
|
||||
- The key is read from the operator's environment or `.env`.
|
||||
- The runner keeps it in memory and passes it to the command.
|
||||
- It is redacted from echoed stderr.
|
||||
- No manifest, rendered Task or file carries it.
|
||||
- **Per-task egress.** Each actor gets one EgressPolicy through the `atenet-egress` gateway. It is
|
||||
deny by default and uses hostnames only. It allows plain HTTP to the artifact receiver. It allows
|
||||
TLS passthrough to the Workspaces' Git hosts and the Model's declared hosts. It is deleted with
|
||||
the actor.
|
||||
- **Artifacts come back through the gateway.** The runner posts its output directory as a tar to
|
||||
the receiver. The receiver is an HTTP server inside the reconciler process on the host. It is
|
||||
reached through a Service with an EndpointSlice to the host. That delivery is the task's
|
||||
completion; ax has no Completed phase.
|
||||
- **Memory on S3-compatible RustFS.** The setting is `OUSAST_MEMORY=s3://sast-memory` on a RustFS
|
||||
server.
|
||||
- The store verifies the bucket at every start: versioning, lifecycle, tags, S3 Select.
|
||||
- It refuses to start when anything is missing. There is no local or fetch-and-filter fallback.
|
||||
- Each row kind has a fixed set of queryable fields, enforced at write time
|
||||
([Memory](memory.md#verify-at-startup-no-fallback), [RustFS setup](rustfs.md)).
|
||||
|
||||
Lessons the host taught, each from a failed live run (the full list is on
|
||||
[ax on this host](ops/ax/README.md#what-this-host-taught-each-one-cost-a-failed-live-run)):
|
||||
|
||||
- Substrate installs no WorkerPool, and actor images must be pinned by digest.
|
||||
- A release `ax` CLI skews from the server; build it from the deployed checkout.
|
||||
- A release `ax` CLI skews from the server. Build it from the deployed checkout.
|
||||
- ax has no Completed phase, so completion is our artifact delivery.
|
||||
- The golden boot means the runner must wait for a start request.
|
||||
- The egress gateway dials the address the actor connected to, so the receiver is a Service
|
||||
- The egress gateway dials the address the actor connected to. So the receiver is a Service,
|
||||
dialled by ClusterIP with its name as Host.
|
||||
- ax's snapshot bucket must exist in your S3 server. When it did not, deleted golden actors piled
|
||||
up in `DELETING` and filled the disk during a 600-task harvest on 2026-10-01.
|
||||
up in `DELETING`. They filled the disk during a 600-task harvest on 2026-10-01.
|
||||
|
||||
Deployment status, in the two words [Deployment](deployment.md) uses strictly. Implemented:
|
||||
ax as the only executor, per-task EgressPolicy, credentials only in the start request, per-task
|
||||
budgets and attribution, the S3 store verified at startup, local bring-up, doctor and smoke Run,
|
||||
re-pinning generated Tasks to another registry. Planned, not done: deployment to any cluster
|
||||
other than this host's kind cluster, a configurable kube context (`kind-` is hard-coded in
|
||||
`doctor.py`, `egress.py`, `router.py`), the receiver as an in-cluster Deployment, the
|
||||
reconciler in-cluster with Secrets injected, presigned-URL delivery to the store.
|
||||
Deployment status uses two words, as [Deployment](deployment.md) does, strictly.
|
||||
|
||||
Implemented:
|
||||
|
||||
- ax as the only executor;
|
||||
- per-task EgressPolicy;
|
||||
- credentials only in the start request;
|
||||
- per-task budgets and attribution;
|
||||
- the S3 store verified at startup;
|
||||
- local bring-up, doctor and smoke Run;
|
||||
- re-pinning generated Tasks to another registry.
|
||||
|
||||
Planned, not done:
|
||||
|
||||
- deployment to any cluster other than this host's kind cluster;
|
||||
- a configurable kube context (`kind-` is hard-coded in `doctor.py`, `egress.py`, `router.py`);
|
||||
- the receiver as an in-cluster Deployment;
|
||||
- the reconciler in-cluster with Secrets injected;
|
||||
- presigned-URL delivery to the store.
|
||||
|
||||
## 4. Token economics, before and after
|
||||
|
||||
Before the plane, the script pipeline cost $0.022 per candidate for two verify passes
|
||||
(`benchmarks/independent/verifier-batched-check-2026-09-29.json`), and the maintainers' own
|
||||
coding sessions spent about 90% of their tokens on tool input and output rather than reasoning.
|
||||
After it, the per-candidate costs below are what the records show. The mechanisms (one budget
|
||||
per Task, pay-once reuse, cheap-before-expensive ordering, replay at $0) are on
|
||||
[Token ergonomics](token-ergonomics.md); this table is the before and after.
|
||||
Before the plane, two costs stood out:
|
||||
|
||||
- The script pipeline cost $0.022 per candidate for two verify passes
|
||||
(`benchmarks/independent/verifier-batched-check-2026-09-29.json`).
|
||||
- The maintainers' own coding sessions spent about 90% of their tokens on tool input and output
|
||||
rather than reasoning.
|
||||
|
||||
After it, the per-candidate costs below are what the records show. The mechanisms are on
|
||||
[Token ergonomics](token-ergonomics.md): one budget per Task, pay-once reuse,
|
||||
cheap-before-expensive ordering and replay at $0. This table is the before and after.
|
||||
|
||||
| Item | Before | After | Record |
|
||||
| --- | --- | --- | --- |
|
||||
@@ -195,63 +273,70 @@ per Task, pay-once reuse, cheap-before-expensive ordering, replay at $0) are on
|
||||
| Harvest of plane signals for the engine | none existed | `verify` $2.5373 over 1,961 calls and 474 tasks; `roles` $1.4036 over 1,034 calls and 151 tasks | `2026-10-01-decision-engine-harvest/record.json` |
|
||||
| Development transcript, tool I/O share | 90% | 85.5% over 2,701 tool calls (`dev_token_report`) | `plane-increment-1.json` |
|
||||
|
||||
Two notes from the records. The meter prices every call at the Model's list price
|
||||
(`plane/models/deepseek-flash.yaml`); the DeepSeek account moved less, about one third of the
|
||||
metered amount on the injection slice ($0.30 for $0.91 metered), and budgets and gates are set
|
||||
against the meter, never against the balance. And the development share barely moved: 90 to 85.5
|
||||
is a small gain, and the remedies (a read guard, a bulk reader, scripts in files) target file
|
||||
contents because that is where the tokens go.
|
||||
Two notes from the records:
|
||||
|
||||
1. The meter prices every call at the Model's list price (`plane/models/deepseek-flash.yaml`). The
|
||||
DeepSeek account moved less: about one third of the metered amount on the injection slice
|
||||
($0.30 for $0.91 metered). Budgets and gates are set against the meter, never against the
|
||||
balance.
|
||||
2. The development share barely moved: 90 to 85.5 is a small gain. The remedies are a read guard,
|
||||
a bulk reader and scripts in files. They target file contents, because that is where the
|
||||
tokens go.
|
||||
|
||||
## 5. Open items
|
||||
|
||||
Five groups, each item with its record: the engine's own gates, data, measurement debt,
|
||||
infrastructure, and instrument gaps.
|
||||
There are five groups. Each item has its record. The groups are the engine's own gates, data,
|
||||
measurement debt, infrastructure, and instrument gaps.
|
||||
|
||||
The engine's gates.
|
||||
|
||||
- BLOCK is reachable for no family: calibration fails for five (ECE 0.071 to 0.119) and the
|
||||
sixth reaches no fold with a Wilson lower bound of 0.95 on precision (slice-2 record).
|
||||
- BLOCK is reachable for no family. Calibration fails for five (ECE 0.071 to 0.119). The sixth
|
||||
reaches no fold with a Wilson lower bound of 0.95 on precision (slice-2 record).
|
||||
- Re-run agreement is below 0.9 for three families: injection 0.77, access control 0.83,
|
||||
untrusted destination 0.87. exp-002 tried the obvious variance fix and lost on the ranking.
|
||||
|
||||
Data.
|
||||
|
||||
- Positives-only groups: 98 (family, group) pairs in the 904-example memory hold positive examples
|
||||
and no negative. 94 of them exist because the fixed side no longer has the function (the vibe-py
|
||||
pairs). No advisory-fixes group is among them
|
||||
(`benchmarks/measurements/2026-10-02-decision-engine-negatives-gap/record.json`). There are no
|
||||
single-label negatives, so every family's pooled AUC is inflated by positives that never had to
|
||||
be told apart from their own fix (slice-2 record, `paired_groups_check`).
|
||||
- The model has seen public fixes. A verify pass named the advisory's CVE id for a public fix, an
|
||||
observation the engine's design specification (maintainer tooling) records as visible
|
||||
memorisation. Excerpts now redact advisory ids and security-worded comments
|
||||
(`src/openultrasast/learn/excerpt.py`), but labels drawn from public advisories are partly in the
|
||||
model's training data, and the within-pair numbers carry that.
|
||||
- PHP is covered by the engine on real code and by in-sample quick rules; the population that
|
||||
and no negative.
|
||||
- 94 of them exist because the fixed side no longer has the function (the vibe-py pairs).
|
||||
- No advisory-fixes group is among them
|
||||
(`benchmarks/measurements/2026-10-02-decision-engine-negatives-gap/record.json`).
|
||||
- There are no single-label negatives. So every family's pooled AUC is inflated by positives
|
||||
that never had to be told apart from their own fix (slice-2 record, `paired_groups_check`).
|
||||
- The model has seen public fixes. A verify pass named the advisory's CVE id for a public fix.
|
||||
The engine's design specification (maintainer tooling) records this as visible memorisation.
|
||||
Excerpts now redact advisory ids and security-worded comments
|
||||
(`src/openultrasast/learn/excerpt.py`). But labels drawn from public advisories are partly in
|
||||
the model's training data, and the within-pair numbers carry that.
|
||||
- PHP is covered by the engine on real code and by in-sample quick rules. The population that
|
||||
would measure it (v3) is reserved.
|
||||
|
||||
Measurement debt.
|
||||
|
||||
- exp-001 (known callers) is registered and has not run; the model-free candidate generation on
|
||||
the 41 development repositories that its units need is not built
|
||||
- exp-001 (known callers) is registered and has not run. Its units need model-free candidate
|
||||
generation on the 41 development repositories. That generation is not built
|
||||
(`benchmarks/experiments/exp-001-known-callers/registration.json`).
|
||||
- Leave-one-source-out, leave-one-framework-out and the memory-size curve points (0, 25, 50%)
|
||||
have not run; one paired point exists (724 against 904 examples, injection, AUC 0.76 against 0.77).
|
||||
- The plane-profile leak audit flagged `verify.flag_b` and `ms.operation`; they are withheld from
|
||||
profile v1, and whether they track the label on unseen code has to be decided before any
|
||||
plane-profile program runs (`2026-10-02-decision-engine-harvest-advisory/record.json`).
|
||||
have not run. One paired point exists (724 against 904 examples, injection, AUC 0.76 against 0.77).
|
||||
- The plane-profile leak audit flagged `verify.flag_b` and `ms.operation`. They are withheld from
|
||||
profile v1. Before any plane-profile program runs, someone must decide whether they track the
|
||||
label on unseen code (`2026-10-02-decision-engine-harvest-advisory/record.json`).
|
||||
- Population v3 waits for `benchmarks/independent/prediction-v3.json`, which does not exist. It is
|
||||
written only after a program is adopted.
|
||||
|
||||
Infrastructure.
|
||||
|
||||
- Deployment to a separate Kubernetes cluster is planned, not done; the code changes it needs are
|
||||
- Deployment to a separate Kubernetes cluster is planned, not done. The code changes it needs are
|
||||
listed with file paths on [Deployment](deployment.md#what-is-implemented-and-what-is-planned).
|
||||
|
||||
Instrument gaps.
|
||||
|
||||
- A missing provider key is a quiet path in a scan. The plane refuses to start a Task whose Model
|
||||
names an unset variable (`plane/router.py`, `StartError`). A scan without a key asks the model
|
||||
nothing, reports only the graph's entailed findings, and records `learning_endpoint_unavailable`
|
||||
in `manifest.json` (`cli.py`, after `resolve_chat_endpoint` returns nothing); `report.md` has no
|
||||
line for that reason yet, so the report does not say the model layer was skipped.
|
||||
- A missing provider key is a quiet path in a scan.
|
||||
- The plane refuses to start a Task whose Model names an unset variable (`plane/router.py`,
|
||||
`StartError`).
|
||||
- A scan without a key asks the model nothing and reports only the graph's entailed findings.
|
||||
It records `learning_endpoint_unavailable` in `manifest.json` (`cli.py`, after
|
||||
`resolve_chat_endpoint` returns nothing).
|
||||
- `report.md` has no line for that reason yet. So the report does not say the model layer was
|
||||
skipped.
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
site_name: OpenUltraSAST
|
||||
site_description: An experimental static security analyser and repository safety net, with an agentic plane and a learned decision engine in development.
|
||||
site_url: https://norandom.github.io/OpenUltraSAST/
|
||||
repo_url: https://github.com/norandom/OpenUltraSAST
|
||||
edit_uri: edit/main/docs/
|
||||
docs_dir: docs
|
||||
|
||||
+9
-2
@@ -291,13 +291,20 @@ The default capability registry is empty pending independent qualification. Inst
|
||||
this interface does not make the project rollout-ready. Advisory is the default;
|
||||
blocking and strict incomplete-coverage handling are separate explicit choices.
|
||||
|
||||
Put the installed `ousast` executable on the Git process's PATH. From this tool's
|
||||
checkout, explicitly install into the desired repository:
|
||||
The supported install is `ousast pre-push install [REPO] [--force] [--docker]`, shipped in
|
||||
the wheel; `ousast pre-push uninstall [REPO]` removes it. The first-run guide, settings, skip
|
||||
reasons and a CI recipe are in [Scanning](../scanning.md#first-run-install-the-hook-on-any-repository).
|
||||
If `ousast` is not on the Git process's PATH, the hook prints one line and lets the push through.
|
||||
From this tool's checkout, the script installer remains available:
|
||||
|
||||
```sh
|
||||
ops/install-pre-push /absolute/path/to/repository
|
||||
```
|
||||
|
||||
`ops/ousast-docker` is the hook-callable Docker wrapper (`OUSAST_COMMAND=/path/to/ops/ousast-docker`,
|
||||
or installed beside the hook by `--docker`). It is a script, not a shell function, so Git can run
|
||||
it. It passes the push input through and mounts the artifact directory at its own path.
|
||||
|
||||
The installer resolves Git's effective hook directory (including `core.hooksPath`),
|
||||
leaves that configuration untouched, and refuses to overwrite any existing hook.
|
||||
For an existing hook, inspect it and explicitly move it aside **before** installation:
|
||||
|
||||
+9
-6
@@ -7,12 +7,15 @@ single-node kind cluster. ax is the only executor; `ousast plane run` submits, r
|
||||
|
||||
ops/ax/up.sh # idempotent: kind + registry, Substrate, ax, gVisor worker pool, egress gateway,
|
||||
# receiver Service, runner image (digest pinned), ax CLI, smoke Task
|
||||
ousast plane doctor # kind, Substrate, ax controller, runner image
|
||||
ousast plane doctor # the profile's cluster (ops/k8s/profiles/kind.toml), Substrate, ax, runner image, memory store
|
||||
ops/ax/smoke-run.sh # end to end: repo-facts on ax, artifacts back on the host, attribution table
|
||||
ops/ax/down.sh # deletes the cluster and its registry
|
||||
|
||||
Tools live in `~/go/bin` (kind, ax, ko, kubectl-ate) and `~/.local/bin` (kubectl); sources and rendered files
|
||||
in `~/.cache/ousast/ax-src/` (the runner digest pin is `runner-image` there).
|
||||
in `~/.cache/ousast/ax-src/`. The cluster's context, registry and pool sizes come from the kind profile
|
||||
(`ops/k8s/profiles/kind.toml`); `up.sh` writes the runner's digest pin to that profile's images file
|
||||
(`ops/k8s/profiles/kind-images.json`), which the templates are re-pinned from and which is committed with them.
|
||||
`up.sh` and `smoke-run.sh` run the checkout's code with `OUSAST_PYTHON` (default `.venv/bin/python`).
|
||||
|
||||
## How a Run executes
|
||||
|
||||
@@ -192,16 +195,16 @@ assume this laptop and must change first (checked 2026-10-02):
|
||||
|
||||
| Assumption | Where | Needed in a real cluster |
|
||||
|---|---|---|
|
||||
| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind gateway `172.19.0.1:18090` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it |
|
||||
| Registry `localhost:5001`, rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), `doctor.py:45` (hard-coded check), the runner digest pin, the `image` of every template under `plane/tasks/` | a registry the workers can pull from; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks |
|
||||
| kubectl context `kind-$KIND_CLUSTER_NAME` | `doctor.py:39`, `egress.py:115` and `:148`, `router.py:145`, `up.sh:24` | one configurable context (for example `OUSAST_KUBE_CONTEXT`); not yet configurable |
|
||||
| Artifact receiver runs on the developer host, reached through the `ousast-receiver` Service with an EndpointSlice to the kind network's gateway address on `OUSAST_ARTIFACT_PORT` | `egress.py`, `receiver-service.yaml.tmpl` | the receiver as an in-cluster Deployment (or object storage, e.g. Substrate's S3-compatible store), with the reconciler reading from it |
|
||||
| The kind-local registry (`registry` in `ops/k8s/profiles/kind.toml`), rewritten by Substrate for kind | `up.sh` (`KO_DOCKER_REPO`), the profile's images file, the `image` of every template under `plane/tasks/` | the `k3s` profile's registry; keep digest pins (Substrate rejects tags); `--runner-image FILE` re-pins generated Tasks |
|
||||
| kubectl context | `kube_context` in the profile, read by `doctor.py`, `egress.py`, `router.py` (plane-on-kubernetes 1.2); `up.sh` reads the same profile | the `k3s` profile's context; nothing else changes |
|
||||
| ax's snapshot bucket: `AX_SNAPSHOTS_BUCKET` in ax's `deploy/ax-server.yaml` points at the ax authors' GCS bucket | ax deploy manifest | your own bucket, set before deploying ax |
|
||||
| Egress gateway applied by hand (agentgateway variant, no Rust build) | this README | the Substrate-installed gateway; per-task EgressPolicies work unchanged |
|
||||
| Worker pool of 2 x 1 CPU / 1.5 GiB for the 7 GB host | `workerpool.yaml.tmpl` | sized to the cluster; `--workers` to match |
|
||||
| Plane memory store and results under `~/ousast-results/` on the host | `reconciler.py` (`OUSAST_RESULTS`), `memory.py` (`OUSAST_MEMORY`) | a persistent volume or bucket shared by the reconciler; the `s3://` store already works against any reachable S3-compatible server |
|
||||
|
||||
The provider key already travels only in the start request through `atenet-router`, which works the same through
|
||||
a port-forward to any cluster.
|
||||
the kube-context tunnel to any cluster.
|
||||
|
||||
## What a larger ax deployment needs
|
||||
|
||||
|
||||
Executable
+81
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env bash
|
||||
# Placement probe (plane-on-kubernetes task 1.5, design open question 2): does Agent Substrate place an actor on a
|
||||
# worker that fits its memory request, or on any free worker of the sandbox class?
|
||||
#
|
||||
# Applies a second WorkerPool `probe-pool` (1 replica, 2 CPU / 3Gi) from ops/ax/workerpool.yaml.tmpl with the worker
|
||||
# image the existing pool already built (no ko build), scales the default pool to one worker so the 7 GB host holds
|
||||
# both, then runs one model-free repo-facts Task asking 2560Mi (limit 3Gi) and one asking 256Mi, recording after
|
||||
# each resume which worker holds the actor (`kubectl ate get workers -o json`). Everything it created is removed
|
||||
# on exit and the default pool is scaled back. Usage: ops/ax/probe-placement.sh OUT_DIR (the JSON evidence lands there).
|
||||
set -euo pipefail
|
||||
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
|
||||
OUT="${1:?usage: probe-placement.sh OUT_DIR}"
|
||||
mkdir -p "$OUT"
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
|
||||
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
|
||||
CTX="$(profile kube_context)"
|
||||
RUNNER_IMAGE="$(profile images.runner)"
|
||||
DEFAULT_REPLICAS="$(profile pools.default.replicas)"
|
||||
K() { kubectl --context "$CTX" -n ax-system "$@"; }
|
||||
log() { printf '%s %s\n' "$(date +%H:%M:%S)" "$*" | tee -a "$OUT/probe.log"; }
|
||||
|
||||
cleanup() {
|
||||
rc=$?
|
||||
log "cleanup (exit $rc): delete the probe tasks and probe-pool, scale ousast-pool back to $DEFAULT_REPLICAS"
|
||||
for t in ousast-probe-big ousast-probe-small; do ax delete task "$t" >/dev/null 2>&1 || true; done
|
||||
K delete workerpool probe-pool --ignore-not-found >/dev/null 2>&1 || true
|
||||
K patch workerpool ousast-pool --type merge -p "{\"spec\":{\"replicas\":$DEFAULT_REPLICAS}}" >/dev/null 2>&1 || true
|
||||
echo "probe exit $rc"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
workers() { kubectl ate --context "$CTX" get workers -o json > "$OUT/$1.json"; kubectl ate --context "$CTX" get workers | tee -a "$OUT/probe.log"; }
|
||||
|
||||
export POOL_NAME=probe-pool POOL_REPLICAS=1 POOL_CPU=2 POOL_MEMORY=3Gi
|
||||
WORKER_IMAGE="$(K get workerpool ousast-pool -o jsonpath='{.spec.workerImage}')" # already built and pulled by the node
|
||||
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
|
||||
export WORKER_IMAGE SUBSTRATE_VERSION
|
||||
log "default pool to 1 replica (room for a 3Gi worker on this host)"
|
||||
K patch workerpool ousast-pool --type merge -p '{"spec":{"replicas":1}}'
|
||||
for i in $(seq 1 60); do [ "$(K get pods -l ate.dev/worker-pool=ousast-pool --no-headers 2>/dev/null | wc -l)" = 1 ] && break; sleep 2; done
|
||||
log "apply probe-pool: $POOL_REPLICAS x $POOL_CPU CPU / $POOL_MEMORY, image $WORKER_IMAGE"
|
||||
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" | tee "$OUT/probe-pool.yaml" | K apply -f -
|
||||
K wait --for=condition=Ready pod -l ate.dev/worker-pool=probe-pool --timeout=600s
|
||||
workers workers-before
|
||||
|
||||
task() { # name, request memory, limit memory
|
||||
cat > "$OUT/$1.yaml" <<EOF
|
||||
apiVersion: ax.io/v1alpha1
|
||||
kind: Workspace
|
||||
metadata: {name: $1-ws, atespace: default}
|
||||
spec:
|
||||
files:
|
||||
- {path: "hello.py", content: "def hello():\n return greet()\n"}
|
||||
---
|
||||
apiVersion: ax.io/v1alpha1
|
||||
kind: Task
|
||||
metadata: {name: $1, atespace: default}
|
||||
spec:
|
||||
image: "$RUNNER_IMAGE"
|
||||
command: ["repo-facts"]
|
||||
env: [{name: OUSAST_OUTPUT_DIR, value: "/workspace/.ousast-out/probe"}]
|
||||
resources: {requests: {cpu: "250m", memory: "$2"}, limits: {cpu: "1", memory: "$3"}}
|
||||
workspaces: [{name: $1-ws, path: "/workspace"}]
|
||||
EOF
|
||||
ax delete task "$1" >/dev/null 2>&1 || true
|
||||
log "apply $1 (requests $2, limits $3)"
|
||||
ax apply -f "$OUT/$1.yaml"
|
||||
for i in 1 2 3 4 5 6 7 8 9; do
|
||||
if ax resume task "$1" 2>&1 | tee -a "$OUT/probe.log"; then break; fi # the first resume may wait for the golden snapshot
|
||||
sleep 20
|
||||
done
|
||||
sleep 15
|
||||
ax get task "$1" | tee "$OUT/$1.status.yaml" | grep -E "phase|message" || true
|
||||
workers "workers-$1"
|
||||
ax delete task "$1" >/dev/null 2>&1 || true
|
||||
for i in $(seq 1 30); do kubectl ate --context "$CTX" get workers | grep -q " 1/1000 " || break; sleep 2; done
|
||||
}
|
||||
task ousast-probe-big 2560Mi 3Gi
|
||||
task ousast-probe-small 256Mi 512Mi
|
||||
log "done; evidence under $OUT"
|
||||
+9
-5
@@ -1,14 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
# Render the end-to-end smoke Run with the current runner digest and execute it through the reconciler on ax.
|
||||
# Render the end-to-end smoke Run with the profile's runner digest and execute it through the reconciler on ax.
|
||||
# The profile (OUSAST_PLANE_PROFILE, default kind; ops/k8s/profiles/) names the images file up.sh wrote.
|
||||
set -euo pipefail
|
||||
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
|
||||
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
|
||||
RUNNER_IMAGE="$(cat "$SRC/runner-image")"; export RUNNER_IMAGE
|
||||
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
PY="${OUSAST_PYTHON:-$REPO/.venv/bin/python}"
|
||||
RUNNER_IMAGE="$(PYTHONPATH="$REPO/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print images.runner)"
|
||||
export RUNNER_IMAGE
|
||||
# Actors reach the receiver by its Service name through the egress gateway (ops/ax/receiver-service.yaml.tmpl,
|
||||
# applied by up.sh, endpoint this host at OUSAST_ARTIFACT_PORT): the runner dials the Service's ClusterIP on port
|
||||
# 80 with that name as Host; the reconciler looks the ClusterIP up unless OUSAST_ARTIFACT_DIAL is set.
|
||||
export OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}"
|
||||
envsubst < "$(dirname "$0")/smoke-run.yaml.tmpl" > "$SRC/smoke-run.yaml"
|
||||
REPO="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
"$REPO/.venv/bin/ousast" plane run "$SRC/smoke-run.yaml" "$@"
|
||||
"$REPO/.venv/bin/ousast" plane status ax-e2e-smoke
|
||||
ousast() { PYTHONPATH="$REPO/src" "$PY" -m openultrasast.cli "$@"; } # this checkout's code, whatever python runs it
|
||||
ousast plane run "$SRC/smoke-run.yaml" "$@"
|
||||
ousast plane status ax-e2e-smoke
|
||||
|
||||
+30
-12
@@ -1,13 +1,21 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bring up ax on this host: kind cluster + local registry, Agent Substrate, the ax control plane, one smoke
|
||||
# Task. Idempotent enough to rerun; `down.sh` removes everything. Requires docker, go, kubectl, kind, ko, ax.
|
||||
# Task. Idempotent: every step is skipped when already done; `down.sh` removes everything. Requires docker, go,
|
||||
# kubectl, kind, ko, ax. The cluster's context, the registry and the pool sizes come from the kind profile
|
||||
# (ops/k8s/profiles/kind.toml; OUSAST_PLANE_PROFILE selects another file with exec = "local"). This script may
|
||||
# name the kind cluster because it creates it; the code reads the profile (tests/test_plane_literals.py).
|
||||
set -euo pipefail
|
||||
trap 'rc=$?; echo "up.sh FAILED at line $LINENO (exit $rc)"' ERR
|
||||
trap 'echo "up.sh exit $?"' EXIT
|
||||
export PATH="$HOME/go/bin:$HOME/.local/bin:$PATH"
|
||||
export GOTOOLCHAIN=auto
|
||||
export KIND_CLUSTER_NAME="${KIND_CLUSTER_NAME:-ousast}"
|
||||
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-localhost:5001}"
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
PY="${OUSAST_PYTHON:-$REPO_ROOT/.venv/bin/python}"
|
||||
profile() { PYTHONPATH="$REPO_ROOT/src" "$PY" -m openultrasast.plane.profile --profile "${OUSAST_PLANE_PROFILE:-kind}" --print "$@"; }
|
||||
CTX="$(profile kube_context)" # kind names its context kind-<cluster>
|
||||
export KIND_CLUSTER_NAME="${CTX#kind-}"
|
||||
export KO_DOCKER_REPO="${KO_DOCKER_REPO:-$(profile registry)}"
|
||||
IMAGES_FILE="$(profile images)" # the digest pins the reconciler and the templates read
|
||||
SRC="${OUSAST_AX_SRC:-$HOME/.cache/ousast/ax-src}"
|
||||
|
||||
for t in docker go kubectl kind ko ax; do
|
||||
@@ -19,9 +27,8 @@ clone substrate https://github.com/agent-substrate/substrate.git
|
||||
clone ax https://github.com/google/ax.git
|
||||
step() { printf '\n== %s (%s)\n' "$1" "$(date +%H:%M:%S)"; }
|
||||
|
||||
step "kind cluster '$KIND_CLUSTER_NAME' + registry $KO_DOCKER_REPO"
|
||||
step "kind cluster '$KIND_CLUSTER_NAME' (context $CTX) + registry $KO_DOCKER_REPO"
|
||||
# Each step is skipped when already done: the substrate helper DELETES an existing cluster before creating one.
|
||||
CTX="kind-$KIND_CLUSTER_NAME"
|
||||
ready() { kubectl --context "$CTX" -n "$1" get pods --no-headers 2>/dev/null | awk '$2 ~ /^[0-9]+\/[0-9]+$/ {n++; split($2,a,"/"); if (a[1]!=a[2] && $3!="Completed") bad++} END {exit !(n>0 && bad==0)}'; }
|
||||
if kind get clusters 2>/dev/null | grep -qx "$KIND_CLUSTER_NAME"; then
|
||||
echo "cluster exists; kept"
|
||||
@@ -56,25 +63,36 @@ KIND_GATEWAY="$(docker network inspect kind --format '{{range .IPAM.Config}}{{if
|
||||
export KIND_GATEWAY OUSAST_ARTIFACT_PORT="${OUSAST_ARTIFACT_PORT:-18090}" # the reconciler's fixed receiver port
|
||||
envsubst < "$(dirname "$0")/receiver-service.yaml.tmpl" | kubectl --context "$CTX" apply -f -
|
||||
|
||||
step "worker pool (gVisor)"
|
||||
step "worker pool (gVisor): pools.default of the profile"
|
||||
# `ko apply` builds the worker image from the Substrate checkout (several GB of Go caches): only when the pool
|
||||
# is absent. A size change in the profile is applied with `kubectl patch` or by deleting the pool first.
|
||||
export POOL_NAME=ousast-pool POOL_REPLICAS="$(profile pools.default.replicas)" POOL_CPU="$(profile pools.default.cpu)" POOL_MEMORY="$(profile pools.default.memory)"
|
||||
export WORKER_IMAGE="ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor"
|
||||
SUBSTRATE_VERSION="$(kubectl --context "$CTX" get nodes -o jsonpath='{.items[0].metadata.labels.ate\.dev/substrate-version}')"
|
||||
export SUBSTRATE_VERSION
|
||||
envsubst < "$(dirname "$0")/workerpool.yaml.tmpl" > "$SRC/workerpool.yaml"
|
||||
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
|
||||
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l ate.dev/worker-pool=ousast-pool --timeout=600s \
|
||||
if kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" >/dev/null 2>&1; then
|
||||
echo "pool $POOL_NAME exists ($(kubectl --context "$CTX" -n ax-system get workerpool "$POOL_NAME" -o jsonpath='{.status.readyReplicas}/{.spec.replicas}') ready); kept"
|
||||
else
|
||||
(cd "$SRC/substrate" && KO_DOCKER_REPO="$KO_DOCKER_REPO" ko apply -f "$SRC/workerpool.yaml")
|
||||
fi
|
||||
kubectl --context "$CTX" -n ax-system wait --for=condition=Ready pod -l "ate.dev/worker-pool=$POOL_NAME" --timeout=600s \
|
||||
|| kubectl --context "$CTX" -n ax-system get pods
|
||||
|
||||
step "runner image $KO_DOCKER_REPO/ousast-runner"
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||
docker build -q -f "$REPO_ROOT/plane/Dockerfile.runner" -t "$KO_DOCKER_REPO/ousast-runner:dev" "$REPO_ROOT"
|
||||
docker push -q "$KO_DOCKER_REPO/ousast-runner:dev"
|
||||
RUNNER_IMAGE="$(docker inspect --format '{{range .RepoDigests}}{{println .}}{{end}}' "$KO_DOCKER_REPO/ousast-runner:dev" | grep "^$KO_DOCKER_REPO/ousast-runner@" | head -1)"
|
||||
[ -n "$RUNNER_IMAGE" ] || { echo "no digest for the runner image" >&2; exit 2; }
|
||||
export RUNNER_IMAGE; echo "$RUNNER_IMAGE" > "$SRC/runner-image" # the reconciler reads this pin
|
||||
echo "runner image $RUNNER_IMAGE"
|
||||
export RUNNER_IMAGE
|
||||
printf '{\n "runner": "%s"\n}\n' "$RUNNER_IMAGE" > "$IMAGES_FILE" # the profile's images file: commit it with the templates
|
||||
echo "runner image $RUNNER_IMAGE -> $IMAGES_FILE"
|
||||
echo "re-pin the templates with: ousast plane workspaces ... --runner-image $IMAGES_FILE (or ousast plane repin, task 4.3)"
|
||||
|
||||
step "ax CLI from the deployed checkout (a release CLI skews from the server)"
|
||||
(cd "$SRC/ax" && go install ./cmd/ax)
|
||||
if [ -x "$HOME/go/bin/ax" ] && [ "$HOME/go/bin/ax" -nt "$SRC/ax/.git/HEAD" ]; then echo "ax CLI newer than the checkout; kept"; else
|
||||
(cd "$SRC/ax" && go install ./cmd/ax)
|
||||
fi
|
||||
|
||||
step "smoke task"
|
||||
ax delete task ousast-smoke >/dev/null 2>&1 || true
|
||||
|
||||
@@ -1,23 +1,26 @@
|
||||
# Warm gVisor workers for ax tasks on this host (Agent Substrate WorkerPool). ax's per-task ActorTemplates
|
||||
# set no workerSelector, so every pool of the matching sandbox class is eligible. Two workers of 1 CPU /
|
||||
# 1.5 GiB fit the 7 GB host beside the control planes; an actor occupies a whole worker.
|
||||
# Warm gVisor workers for ax tasks (Agent Substrate WorkerPool), rendered by ops/ax/up.sh (and by
|
||||
# ops/ax/probe-placement.sh for the placement probe) from the profile's pool: POOL_NAME, POOL_REPLICAS, POOL_CPU,
|
||||
# POOL_MEMORY (ops/k8s/profiles/<profile>.toml, [pools.default]) and WORKER_IMAGE (ko:// for a build, or a
|
||||
# digest-pinned reference of a built worker image). ax's per-task ActorTemplates set no workerSelector, so every
|
||||
# pool of the matching sandbox class is eligible. An actor occupies a whole worker. Replaced by the renderer of
|
||||
# plane-on-kubernetes task 6.1 (ops/k8s/base/workerpool-*.yaml.tmpl).
|
||||
apiVersion: ate.dev/v1alpha1
|
||||
kind: WorkerPool
|
||||
metadata:
|
||||
name: ousast-pool
|
||||
name: ${POOL_NAME}
|
||||
namespace: ax-system
|
||||
labels:
|
||||
workload: ousast
|
||||
spec:
|
||||
replicas: 2
|
||||
workerImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor
|
||||
replicas: ${POOL_REPLICAS}
|
||||
workerImage: ${WORKER_IMAGE}
|
||||
template:
|
||||
nodeSelector:
|
||||
ate.dev/substrate-version: "${SUBSTRATE_VERSION}"
|
||||
resources:
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 1536Mi
|
||||
cpu: "${POOL_CPU}"
|
||||
memory: ${POOL_MEMORY}
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 1536Mi
|
||||
memory: ${POOL_MEMORY}
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# The k3s profile: the plane on the maintainer's production cluster (plane-on-kubernetes design section 1;
|
||||
# requirements "Target Environment"). Documented values, not secrets: the kubeconfig for the context, the GHCR
|
||||
# pull secret's content and the store's keys live outside this file. Every field is overridable by OUSAST_<FIELD>,
|
||||
# a pool by OUSAST_POOL_<NAME>_<FIELD>. Nothing has been applied to a k3s cluster yet; docs/deployment.md says
|
||||
# which items are planned.
|
||||
|
||||
exec = "remote" # the same CLI from a laptop or a CI job, through the cluster's kube context
|
||||
kube_context = "ousast-k3s" # the maintainer's context name for the k3s cluster
|
||||
registry = "ghcr.io/norandom" # images built by GitHub Actions from tagged commits (design section 8)
|
||||
images = "images.json" # the release asset {"runner": "ghcr.io/norandom/ousast-runner@sha256:...", "engine": ...}
|
||||
router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel)
|
||||
ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel)
|
||||
memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from the CI secrets or .env
|
||||
atespace = "default"
|
||||
snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch)
|
||||
image_pull_secret = "ghcr-pull" # only needed while the GHCR packages are private
|
||||
|
||||
[pools.default] # production sizing: four verify-class workers; the HPA (design section 5) scales it
|
||||
replicas = 4
|
||||
cpu = "1"
|
||||
memory = "1536Mi"
|
||||
|
||||
[pools.engine] # two engine workers sized from the measured 2.1 GiB peak under a 3 GiB limit
|
||||
replicas = 2
|
||||
cpu = "2"
|
||||
memory = "4Gi"
|
||||
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"runner": "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
# The kind profile: the plane on this laptop's single-node kind cluster, which ops/ax/up.sh creates and which is
|
||||
# the development default (plane-on-kubernetes design section 1). Values only; a credential never goes here
|
||||
# (keys stay in the environment or .env). Every field is overridable by OUSAST_<FIELD> in the environment, a pool
|
||||
# by OUSAST_POOL_<NAME>_<FIELD>. This file may name the kind cluster; the code must not (tests/test_plane_literals.py).
|
||||
|
||||
exec = "local" # the reconciler runs on this host against the cluster below
|
||||
kube_context = "kind-ousast" # kind's context for KIND_CLUSTER_NAME=ousast
|
||||
registry = "localhost:5001" # the kind-local registry up.sh creates (Substrate rewrites it for the node)
|
||||
images = "kind-images.json" # digest-pinned image references, written by up.sh after each build
|
||||
router_url = "" # empty: port-forward to atenet-router through the kube context (ax-tunnel)
|
||||
ax_server = "" # empty: the ax CLI's own tunnel through the kube context (ax-tunnel)
|
||||
memory = "s3://sast-memory" # the memory store; S3_ENDPOINT and the keys come from .env
|
||||
atespace = "default"
|
||||
snapshots_bucket = "s3://ax-snapshots" # ax's AX_SNAPSHOTS_BUCKET, ours (rendered into the ax-server patch)
|
||||
image_pull_secret = "" # the kind registry needs none
|
||||
|
||||
[pools.default] # ousast-pool: two gVisor workers fit the 7 GB host beside the control planes
|
||||
replicas = 2
|
||||
cpu = "1"
|
||||
memory = "1536Mi"
|
||||
|
||||
[pools.engine] # engine-pool (design section 4): one worker for the Joern engine task
|
||||
replicas = 1
|
||||
cpu = "2"
|
||||
memory = "4Gi"
|
||||
Executable
+61
@@ -0,0 +1,61 @@
|
||||
#!/bin/sh
|
||||
# OpenUltraSAST in Docker, callable from a Git hook. A script, not a shell function: Git runs
|
||||
# hooks non-interactively and never sees functions from ~/.bashrc.
|
||||
#
|
||||
# OUSAST_COMMAND=/path/to/ousast-docker git push # or: ousast pre-push install --docker
|
||||
#
|
||||
# The repository is mounted READ-ONLY at its own absolute path and the artifact directory
|
||||
# read-write at its own path, so every path in the arguments and in the output ("Details: ...")
|
||||
# means the same thing inside and outside the container. The push input on stdin is passed
|
||||
# through (-i). The container runs as your user, with no network and --memory 3g.
|
||||
#
|
||||
# Advisory like the hook: when Docker or the image is missing, it says the check did not run
|
||||
# and exits 0. Pull the image once first; a hook never downloads ~3 GB on its own:
|
||||
#
|
||||
# docker pull ghcr.io/norandom/openultrasast:2.0.1
|
||||
#
|
||||
# --engine background does not work through this wrapper: the container ends with the hook.
|
||||
image=${OUSAST_IMAGE:-ghcr.io/norandom/openultrasast:2.0.1}
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
echo "ousast-docker: check did not run: docker is not installed. Push continues."
|
||||
exit 0
|
||||
fi
|
||||
if ! docker image inspect "$image" >/dev/null 2>&1; then
|
||||
echo "ousast-docker: check did not run: image $image is not present." \
|
||||
"Run 'docker pull $image' once. Push continues."
|
||||
exit 0
|
||||
fi
|
||||
repo=$(pwd -P)
|
||||
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
|
||||
previous=
|
||||
for argument in "$@"; do
|
||||
if [ "$previous" = "--artifact" ]; then
|
||||
artifact_dir=$(dirname -- "$argument")
|
||||
fi
|
||||
previous=$argument
|
||||
done
|
||||
mkdir -p -- "$artifact_dir" || exit 0
|
||||
artifact_dir=$(cd -- "$artifact_dir" && pwd -P)
|
||||
# A linked worktree keeps its objects in the main repository's git directory: mount that too.
|
||||
common=$(git rev-parse --path-format=absolute --git-common-dir 2>/dev/null || echo "$repo/.git")
|
||||
case "$common" in
|
||||
"$repo"/*) extra_mount= ;;
|
||||
*) extra_mount="-v $common:$common:ro" ;;
|
||||
esac
|
||||
# shellcheck disable=SC2086 # OUSAST_DOCKER_ARGS and extra_mount are deliberately word-split
|
||||
docker run --rm -i --network none --memory "${OUSAST_DOCKER_MEMORY:-3g}" \
|
||||
--user "$(id -u):$(id -g)" -e HOME=/tmp -e TMPDIR=/work --tmpfs /work:size=4g \
|
||||
-v "$repo:$repo:ro" -w "$repo" -v "$artifact_dir:$artifact_dir" $extra_mount \
|
||||
${OUSAST_DOCKER_ARGS:-} "$image" "$@"
|
||||
status=$?
|
||||
if [ "$status" -ge 125 ]; then
|
||||
echo "ousast-docker: check did not run: the container could not start (docker exit $status). Push continues."
|
||||
exit 0
|
||||
fi
|
||||
# Only an explicit blocking opt-in may turn the container's exit into a rejected push; an older
|
||||
# image that rejects a newer argument must not block either.
|
||||
if [ "$status" -ne 0 ] && [ "${OUSAST_PUSH_MODE:-advisory}" != blocking ]; then
|
||||
echo "ousast-docker: the check exited with status $status; push continues (advisory)."
|
||||
exit 0
|
||||
fi
|
||||
exit "$status"
|
||||
+32
-11
@@ -1,17 +1,38 @@
|
||||
#!/bin/sh
|
||||
# Explicitly copy using the non-overwriting recipe in ops/README.md.
|
||||
# A prior hook is chained only after you explicitly move it to this sibling path.
|
||||
# OpenUltraSAST pre-push safety net. Advisory: nothing here rejects a push unless
|
||||
# OUSAST_PUSH_MODE=blocking is set explicitly. Install with `ousast pre-push install`
|
||||
# (or ops/install-pre-push from a checkout); remove with `ousast pre-push uninstall`.
|
||||
# A prior hook is chained only after it was moved to the sibling path "$0.before-ousast".
|
||||
artifact_dir=${OUSAST_ARTIFACT_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/openultrasast/push}
|
||||
prior="$0.before-ousast"
|
||||
if [ -e "$prior" ] || [ -L "$prior" ]; then
|
||||
exec ousast pre-push . --artifact "$artifact_dir/push-$$.json" \
|
||||
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
|
||||
--mode "${OUSAST_PUSH_MODE:-advisory}" \
|
||||
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
|
||||
--prior-hook "$prior" --remote "$1" "$2"
|
||||
# The Docker wrapper installed beside this hook (`ousast pre-push install --docker`) is used
|
||||
# unless OUSAST_COMMAND names another command; otherwise `ousast` from PATH.
|
||||
if [ -z "${OUSAST_COMMAND:-}" ] && [ -x "$(dirname -- "$0")/ousast-docker" ]; then
|
||||
OUSAST_COMMAND="$(dirname -- "$0")/ousast-docker"
|
||||
fi
|
||||
exec ousast pre-push . --artifact "$artifact_dir/push-$$.json" \
|
||||
command=${OUSAST_COMMAND:-ousast}
|
||||
if ! command -v "$command" >/dev/null 2>&1; then
|
||||
echo "ousast: pre-push check did not run: '$command' is not on PATH." \
|
||||
"Install it (pip install git+https://github.com/norandom/OpenUltraSAST) or set OUSAST_COMMAND. Push continues."
|
||||
if [ -e "$prior" ] || [ -L "$prior" ]; then
|
||||
exec "$prior" "$@"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
remote_name=$1
|
||||
remote_url=$2
|
||||
set -- pre-push . --artifact "$artifact_dir/push-$$.json" \
|
||||
--deadline "${OUSAST_PUSH_DEADLINE:-30}" \
|
||||
--mode "${OUSAST_PUSH_MODE:-advisory}" \
|
||||
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}" \
|
||||
--remote "$1" "$2"
|
||||
--incomplete-coverage "${OUSAST_INCOMPLETE_COVERAGE:-allow}"
|
||||
# Optional settings are passed only when set, so an older ousast (or image) still runs.
|
||||
if [ -n "${OUSAST_PUSH_ENGINE:-}" ]; then
|
||||
set -- "$@" --engine "$OUSAST_PUSH_ENGINE"
|
||||
fi
|
||||
if [ -n "${OUSAST_COMPARISON_BASE:-}" ]; then
|
||||
set -- "$@" --comparison-base "$OUSAST_COMPARISON_BASE"
|
||||
fi
|
||||
if [ -e "$prior" ] || [ -L "$prior" ]; then
|
||||
set -- "$@" --prior-hook "$prior"
|
||||
fi
|
||||
exec "$command" "$@" --remote "$remote_name" "$remote_url"
|
||||
|
||||
@@ -0,0 +1,353 @@
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "aio-libs/aiohttp", "label": 1, "pair": "ee577ee4b6076fab", "unit": "2523f55d035d1a772a842090bc335e5d755b2fe15b376dd5b6b9e30b9dae82d1"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "aio-libs/aiohttp", "label": 0, "pair": "ee577ee4b6076fab", "unit": "ca0f6c160d7839204f6fcc1709bcbc749a02a0b3b8bb648e2a238eacbf22ed78"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "alchemycms/alchemy_cms", "label": 1, "pair": "c95b6b01d0c2db2a", "unit": "15540c23be0c42472a49039658ec2ec0bbd8c6b318e35dd040a9d427370e0ef0"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "alchemycms/alchemy_cms", "label": 0, "pair": "c95b6b01d0c2db2a", "unit": "542471980417e964182d7d3db59a05fe12bdd072449f3bd77f7daa5515dbc909"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "bcc636b23dfb5f570d8864b17fe07b6b4fb52b65a40988c32011f1422f74d504"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "dbcf1e67a1be7ccf7d51701f9b8339339164055135443044c23a446b16b3d934"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "50408f68114ecb4de3bebd836d52d9afad86b22dd1843641ac8a49b617c09e33"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "anxolerd/dvpwa", "label": 1, "pair": "", "unit": "cafb0f00f80dc6beeba40eb95c764bf9a3569193070ba321c28677dfa425ec88"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "apache/logging-log4net", "label": 1, "pair": "fe0e104d111c2555", "unit": "088036fab9962be2b96f9162ea4f32190575678d3c5ba96bdfeac178bc22b2c4"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "apache/logging-log4net", "label": 0, "pair": "fe0e104d111c2555", "unit": "7fd2b56a816e25358477eee64487fb2c77f34c652703ad0a7120c368492017a3"}
|
||||
{"family": "path", "fold": "outer-0", "group": "apache/wicket", "label": 1, "pair": "b6742570f7f0e026", "unit": "7c6c1d27aee6557f1b6bf371b8209003f0d2d1122cfa35b6629eb43f38769cf4"}
|
||||
{"family": "path", "fold": "outer-0", "group": "apache/wicket", "label": 0, "pair": "b6742570f7f0e026", "unit": "df8300d84dd784d0be46969ea432ef97b8ce611bae7a896ed4f349bd4d6897db"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "apolloconfig/apollo", "label": 1, "pair": "0f584bb97a154488", "unit": "222a84ffc7c58d45e8f10df221762b23bf20d0287ecd20f375558134928886ca"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "apolloconfig/apollo", "label": 0, "pair": "0f584bb97a154488", "unit": "25e4c9d2941f59f072385a7bde3fd0bd76f9e986ed24b06ce81962ab447f5094"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "archivy/archivy", "label": 0, "pair": "bb3d7baed62e6bb7", "unit": "e8ae88fd2b677e3f64ac94bddb012e3b903afa294ecc87774d10087c0d50489c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "archivy/archivy", "label": 1, "pair": "bb3d7baed62e6bb7", "unit": "f6b0d241891dc1d897d74d98b6ff7ada2c1f1ce7874ca32097630c06ca53e389"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "axllent/mailpit", "label": 1, "pair": "143b900b99a46cf4", "unit": "200f96cd784ff5c39d8bba22883f22bc8a7c87c6970a7c871802f48a45548337"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "axllent/mailpit", "label": 0, "pair": "143b900b99a46cf4", "unit": "7a0d62ec424849ec4681f4333fbb077e8239054d1e964bd99b40847ad7ccdb01"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "browserify/static-eval", "label": 1, "pair": "c12dbc8a50f7fe9b", "unit": "04beb8a0fd054b92c2dd0ed83907330fdee6124d86114df948ac72a297a7d51f"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "browserify/static-eval", "label": 0, "pair": "c12dbc8a50f7fe9b", "unit": "c5029016a6fd2e3f9df59dba0ce3c173bbbb0d2b1de22d151086e01e1b272119"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "0e213aeabacf4205db024ea3c557e720c3aba911424555784917e967d6404867"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 0, "pair": "e8bc65415bd42e43", "unit": "4920230ef773595a9fd9a92a090dff42b48499a1dc5fc8831f941dae55cb2538"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 1, "pair": "e8bc65415bd42e43", "unit": "49209a810f207eb61382e2ceeab7076936838a5652546f21c187d066e220160b"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "9e2964884cdf281fa3f18ee3481aa62f9a8a0a195d605da2df5bd71876fe311a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "budibase/budibase", "label": 1, "pair": "", "unit": "f9cc6f1bbcd59f5dc0de8f96248f722ea0679f8afa47c53f70832151ce7616d3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 0, "pair": "e69463f945f9a0f4", "unit": "5feace7b28e260175f80c682e9009025aea6fae3dc75409cabbb955b560d0372"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 1, "pair": "e69463f945f9a0f4", "unit": "799a4f5a5b9324e86c90752f0239f059984f44e8ff1ec0dd518d0d9413e34c66"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "cesargb/laravel-magiclink", "label": 0, "pair": "768fc9db8a75a351", "unit": "adc6527469618f72b1dd7776814717cbd0056a0c625e05bef74199171ce73f1a"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "cesargb/laravel-magiclink", "label": 1, "pair": "768fc9db8a75a351", "unit": "da47663a57e370a75cc65b810b8a380aa26d12dcabb1d9b3c02641398ff7d348"}
|
||||
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 0, "pair": "fab4ce020049724b", "unit": "b9be90242be627aa46462143ac85940a906232ebc0b47776b5729475b27d215e"}
|
||||
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 1, "pair": "fab4ce020049724b", "unit": "ece12ea7d87b4a12c4da30faed9eb530b51fc879c271b68ba5304b230cda8e79"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 1, "pair": "9126ba83905eb8b4", "unit": "d8dc50b7bb40bbcfc3f3d4f1f8d3a4d17eb1c662d5f759efd18656a482dd2e32"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 0, "pair": "9126ba83905eb8b4", "unit": "f426f9307621b3f2d6a8d1669cdb4c19e962b94e34fbb394d4660cbde5921c88"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "commenthol/serialize-to-js", "label": 0, "pair": "7ac5e21d9d8f4c47", "unit": "1d1efa2fd7323c264290d4d314c957c853e1c7a9c877a54d6f03ab7d2af2ca86"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "commenthol/serialize-to-js", "label": 1, "pair": "7ac5e21d9d8f4c47", "unit": "b5dc78c7ac0b76803dc98904142c1139083fb9daafb5bff40e7b5c3f14ee03f5"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "0894a7212280bab755cd9d93b1f0ede2c51e4fca76b76e5ed9e24ef8f8cf0b9c"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "02694b5fb85e9e52bf5eaaab4e0b628f1f44ff3a8d5ad026cc10afdbc4222eaf"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "db9a3e888780501dd3c2c125af2f9d47f68ebf02f0e0db8370b2ea09c6f9d070"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9af96b099dedcc4005e5fcd19ad78f73864e1cfbafbe6953b4edb6260957943d"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9e7c2a786068a936a6c664cc229d6f39cea1fe57caf0806306f0cfc44515460c"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "cps-it/mailqueue", "label": 1, "pair": "07d1dc4d299a7700", "unit": "0e664e274dfb438d2f721edc81f8469ac48387dbd5c1b43973cd7e651db46db3"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "cps-it/mailqueue", "label": 0, "pair": "07d1dc4d299a7700", "unit": "da55f60378c67e91683b9720f2fa0e6659e28c7b392c050278b40713e47c84a3"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "datapizza-labs/datapizza-ai", "label": 1, "pair": "5a91049be21aa611", "unit": "1ab9936627aa59c3a60ea44074ad44b68aea4f9c04f14c1949dd9c77ca13ff0a"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "datapizza-labs/datapizza-ai", "label": 0, "pair": "5a91049be21aa611", "unit": "8df72b9eab3f2fd858c40312d0a806cb67c620bd62873826c8732ba8795d9cfa"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 0, "pair": "", "unit": "115d87f37bf1d4ab91fa3010303e81b127f84b817268b60b8fd4b9934efeae4f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 0, "pair": "2b8101c89c839614", "unit": "7bb7d858a670fc51f81e08c141c2bca3b8b7524254ab679e33b22ce79536e895"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 0, "pair": "c23acf4f38945093", "unit": "82bdc0b6bb0c19e342efa63af0c9014a607461d0682cdd14bd5fea33516fbd50"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 1, "pair": "2b8101c89c839614", "unit": "886285661b88f14e00ff8e863b31cf7b7afb303ad4aa4aa13b1cffbf1272d5e0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "davila7/claude-code-templates", "label": 1, "pair": "c23acf4f38945093", "unit": "bf74aae5a48156f183426a80cafca6da219ab13b2ae4b40fa88fca75acb72980"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "delmaredigital/payload-puck", "label": 0, "pair": "d7cc37a3a54315cb", "unit": "4aecd43c6e5b81daf7aadd6a6e22f22c17c08b652a2f5121a49320ca1057b186"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "delmaredigital/payload-puck", "label": 1, "pair": "d7cc37a3a54315cb", "unit": "88123ae860be2fdc28fd6a289cf26c3e0af0689f455cf8c15098e5dc76e5c8a8"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "devbridge/jquery-autocomplete", "label": 0, "pair": "81caad96b221c2b8", "unit": "3c2dc720ed159e55629ba737665d93859def9790427289567a513c83c133e236"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "devbridge/jquery-autocomplete", "label": 1, "pair": "81caad96b221c2b8", "unit": "e8b9595a320776c8e663fd0b168193c1e64e36a9052784d1219f2c4a91d647f2"}
|
||||
{"family": "path", "fold": "outer-3", "group": "dgtlmoon/changedetection.io", "label": 0, "pair": "f9b6599018ef3b60", "unit": "7558d6ee39a311c5459e35fe6f7b12917b7f843b21ab1fbb5e485a07f53b7d78"}
|
||||
{"family": "path", "fold": "outer-3", "group": "dgtlmoon/changedetection.io", "label": 1, "pair": "f9b6599018ef3b60", "unit": "b1a6cf450a074788ffca2bebd880435abac2c376e89cb328d7a22d277fdccad8"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "dicebear/dicebear", "label": 1, "pair": "eaf2f03db2c502b5", "unit": "38dbfbf745d7cbcc52c6fff83248d84663273fe6a1f1b014eac7776c4bb8f074"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "dicebear/dicebear", "label": 0, "pair": "eaf2f03db2c502b5", "unit": "a84d848ef56b72fc15872d88acaae1743c9cdf88095d13b1869abbec6a0d8a8f"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "docling-project/docling-core", "label": 0, "pair": "7df48bca2c9fabfc", "unit": "7b76ce8f55d61071adb367018cc3e50f04a05d08c7c4919c3806fc1cc8bf41a7"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "docling-project/docling-core", "label": 1, "pair": "7df48bca2c9fabfc", "unit": "c82ea0adc9c8366aa2b77ce73772038676fc37ff50879e873ff397958fdd7ec1"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "7f02a09b8d8f2321f2cdade74c2fa1d52442e006149a749a8be768daaa62195d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "9b5e439943ce5c2b04d6201b7a843634aa1b13b726813c23450ff54ce03f3dd6"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "5b836889b4da4921", "unit": "a84c2ce4a4eadf265d4481ad7f6864edf1d7e3084c6ee32cbbfe454bb2946bfe"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 0, "pair": "5b836889b4da4921", "unit": "caeed7f097a3311fe2e84efd372fe4b4c140f63a80a9f07ba0b61966dbab15e0"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "c2a2c2d85114a97f892fb1e5500fbd98622457a8c46451e1f9e958ab08a40318"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "29d8be8206c88ba7efd86b56a7dcf8b655e33b4a792c2d18b028c92c6252daab"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 0, "pair": "f96f0fb245948915", "unit": "0954e6931373915c204b9ec8d2b8ae6297a95667bfd41aadf9035625ad93967f"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 1, "pair": "f96f0fb245948915", "unit": "5e09cdae2504e834d591b6f92a808427f07550364b5419899f7e6f76084fc377"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "ether/etherpad", "label": 0, "pair": "bf11f138fe39675e", "unit": "56f0d0b156992af56141d6368fdb0c5376fc7a44a03991d754f97202b9d77008"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "ether/etherpad", "label": 1, "pair": "bf11f138fe39675e", "unit": "8271af5017894167556893da2070188060d06eebc87f27db45c46234c359631d"}
|
||||
{"family": "path", "fold": "outer-2", "group": "fastify/fastify-static", "label": 0, "pair": "9d937c2dffb4aadc", "unit": "01ac12bb06e660d0aeef8b1e26dddc5c99d52cc333f7af6d1981eee2818e64fd"}
|
||||
{"family": "path", "fold": "outer-2", "group": "fastify/fastify-static", "label": 1, "pair": "9d937c2dffb4aadc", "unit": "9612edf0f927048335e83e9925f01be9aea9b772bdbdc568ad9cc0a17055662a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 1, "pair": "d4f648048e45f2ea", "unit": "08807c8110b8c1b8635a6883c229fb91f2b9566db8b6c252448a0db7b2ddd47e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 0, "pair": "d4f648048e45f2ea", "unit": "10869c931053bf77ecaa98a47f56dea27d643d1abed2a4758bcf48a78547ccc9"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "662e1f36c230d7f1", "unit": "0b73332b25413902c26d5b5dee6767465eed99ff71a814582988ffe0d8ec1a30"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "8c8e46d0c7d361b784baff9d70f38744b2805ef1ee0aaacaa616bc20a2c6f32d"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "aca938682c196fd4c85b39b89eeedb97c35f6a5a017651fe9c03c5b727cab7ea"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "cec00d1509a129364da6ab8537fb3f433fd847e26d073c90311ccdaf7a1b39e7"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 1, "pair": "662e1f36c230d7f1", "unit": "e0f58a7ac5627de3e7b6cfe5add41d18b0f64b998c1cc788d42eb2cdbbc3fe41"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 0, "pair": "2c251a645094f389", "unit": "0d41cc634771048f72d07c4aa9feaed669d0bfe3844d61dcdc525a0f899617e2"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 1, "pair": "2c251a645094f389", "unit": "ca8288d0b8b94dc6b7e4c681936ea610d295fe949cba5f64f09b6de134ef9875"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "forgekeep/nebula-mesh", "label": 1, "pair": "2ac5ec5011615c9b", "unit": "2f427f403af3cb58d776437736c2389805543413fdb91aad98979158c6d00de3"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "forgekeep/nebula-mesh", "label": 0, "pair": "2ac5ec5011615c9b", "unit": "cddc8c5bed0841b80e7b20d0b378e858f8e4ac37fd7b890ce76611d989857f92"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "frangoteam/fuxa", "label": 1, "pair": "", "unit": "0c7830c2912ff285855d8a3c8e9b47db4ca4af840baa66d4b5d3f4422f46f95d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "5383a8c2a78fabd3d17a5b3021125953b5ebbee61f8900c30d016c56b1a3bad6"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "360d8bfd2f116e98", "unit": "7c1989a4ec3622bdaebcf1161de850f22cfed041cb38830ca1ded4fa99d40466"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "ec8c706cc2b20f36", "unit": "848688dcd0fe8d46e6762cc8aee1d82e8fad8a2a044c94b547aea11049c85bfc"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "ec8c706cc2b20f36", "unit": "983cc9231f91baee4e2d626193f3b88aa5b09907f6faab870949bdc9c336b816"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "e4ad86416d5c6668", "unit": "98ff1dff2afe80709b8e61a9c95cfeea12b2110c13b7120478268b4762e2c992"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "360d8bfd2f116e98", "unit": "a1a825d23e34757c8e961e43ae791253e62682e130acdb846b58118588006a98"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "a7fa4207508b3989baabc7a1f0ff2ea5229b366c1cb106e7fc945134b8b106c8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "e4ad86416d5c6668", "unit": "ef5c53f3114d0057e694df13dc07cbcb532c3b0d59150eda269b87b021d7a6e8"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "getgrav/grav", "label": 1, "pair": "9ae575048504c690", "unit": "176acf077fd58c3e6f940753179ffa3e8316661f355d94acb5e0345b51925ca4"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "getgrav/grav", "label": 0, "pair": "9ae575048504c690", "unit": "228b34ebc2090b5239c7821d28366ddc663e7118d015a79b45dca6618d5490fb"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "gofiber/template", "label": 0, "pair": "a3fdf9ce3441ec86", "unit": "653236018929ab8534c27704a20ec5d358047bc66e451620d2ac32081c91d333"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "gofiber/template", "label": 1, "pair": "a3fdf9ce3441ec86", "unit": "b70ca2c43f5929e7b885bd728349e721fd85d7605e2f722a7e8cfe0f0d477b0c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "groundhoggwp/groundhogg", "label": 1, "pair": "", "unit": "92400e83890882a66ffdea1a405e2354a195442e176d85fb707b0442c44d45ca"}
|
||||
{"family": "path", "fold": "outer-1", "group": "gtsteffaniak/filebrowser", "label": 0, "pair": "f9dcf3a0fb0f0a14", "unit": "7ca6dd4f673976b000a657f885af3463740419ce41e14f972af0c5483cb75a7d"}
|
||||
{"family": "path", "fold": "outer-1", "group": "gtsteffaniak/filebrowser", "label": 1, "pair": "f9dcf3a0fb0f0a14", "unit": "86c1db3a5e9f843d6546146449171e0378ac61d6c8f44c6dd72fd379cdbb188a"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "handsontable/formula-parser", "label": 0, "pair": "6144baeb491f4a56", "unit": "f5c22ce104e9593e5272b6e8b3c6f3b61fa4578552ccea80e6021ee90bdb3ffc"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "handsontable/formula-parser", "label": 1, "pair": "6144baeb491f4a56", "unit": "f687e7e95dbe47f602a1e0002a44f94fd48e5b1c33f77ecebc6c2d814fcbf450"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "heartcombo/devise", "label": 0, "pair": "2ef4471c66e5e4ab", "unit": "ad7c8e368fe1699a044e058d5c10b41bd9fc0f91c3f3c384c5dac40af39e71f6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "heartcombo/devise", "label": 1, "pair": "2ef4471c66e5e4ab", "unit": "d888baf7194bbe7158cb3932f58b47a9a235351d388d10df82a130ddd54c2d46"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "honojs/hono", "label": 1, "pair": "5628c54152216646", "unit": "47d081c9f6060218967902f694e61c88f64f57c65bec6fc3073afeb0b77d117c"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "honojs/hono", "label": 0, "pair": "5628c54152216646", "unit": "e1297bccf9c3606e0b41d762a69ff5d0a33ae4f7f1506071af6b9cb4a177ff56"}
|
||||
{"family": "path", "fold": "outer-2", "group": "hs-web/hsweb-framework", "label": 1, "pair": "e52aa814c36ace1f", "unit": "2d6d74364f1982e91f27f1c995b8699df1fc8f8dce6e1e2cf274bf01b6e9494e"}
|
||||
{"family": "path", "fold": "outer-2", "group": "hs-web/hsweb-framework", "label": 0, "pair": "e52aa814c36ace1f", "unit": "4727471fea2cf0a92d43b4b1503f1339a7dde87f872ba8064dacbf855c57705b"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 0, "pair": "6548e1527e82bd99", "unit": "5542613f6d2eb9421b8213b8bb864894f0ecf9fe64945681019230ef0d75cfaa"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 0, "pair": "", "unit": "6c2adee34e9e8384535ec1073eda60e2ab528ac77c36747802c303e1f5e9ffe8"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "7ce70e9b20107cc7ebb87d173da765fc53ffd268e521749a06a9a1f38d746845"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 1, "pair": "6548e1527e82bd99", "unit": "c3e54975464323e58ec96ee4d83e7bc08baf94553d553a8dc3fb28cd9c70ff21"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "d3173c1e43eb6402a71ea2dec1592913dc078a2194ba1c1d509ab129a3180f6f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "internlm/lmdeploy", "label": 1, "pair": "0924befc49c7bb18", "unit": "e7e4872db40f0420547ed622fbdd7a072b64a7ba466d1ccfff68e691cee6ad99"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "internlm/lmdeploy", "label": 0, "pair": "0924befc49c7bb18", "unit": "f7d95f43fb2df20ebc4c37fceabdd00ffd8159f404ca384481f1764252382388"}
|
||||
{"family": "path", "fold": "outer-1", "group": "jarofghosts/glance", "label": 1, "pair": "772e18821701d60f", "unit": "89a0496c7663bbfd67d660c9f481c12420da2dd1bdb29dc079735ee7a0a49626"}
|
||||
{"family": "path", "fold": "outer-1", "group": "jarofghosts/glance", "label": 0, "pair": "772e18821701d60f", "unit": "d3241ceaa50a7d23a4650e6a2a55372b08ec96b35b0bede844bc40e16e818ebc"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "jas-/node-libnmap", "label": 1, "pair": "250de1302c7e541d", "unit": "5d03535f2df2acdbc319d9e6b20da65d51f3e810b80f3c8f89b5074a971a3f1c"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "jas-/node-libnmap", "label": 0, "pair": "250de1302c7e541d", "unit": "78f977cfc096e34b4eada7f798ad2973ef45728d0d01bfab6ac9e2a3257827ef"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "jashkenas/underscore", "label": 1, "pair": "c10bfb006f2da55a", "unit": "1ea4b86dbd84581b1d7b7cabbab9f28c8b213416a60778879666ff42d23fd7f1"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "jashkenas/underscore", "label": 0, "pair": "c10bfb006f2da55a", "unit": "7647021f742da24c660b2d407886771786e957076c4cf336def560a570dd8778"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "jellyfin/jellyfin", "label": 1, "pair": "a73421efb106ab0a", "unit": "1f6f207a758582de0c0e6b331b19fe119abe18727d45ccbbe09aabb795f3a292"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "jellyfin/jellyfin", "label": 0, "pair": "a73421efb106ab0a", "unit": "32fbdad39eaf5177db4ad0a11d53e785b30bc7b2376ad9d460f29a08429bd0c7"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "jenkinsci/cas-plugin", "label": 0, "pair": "0458e3fd444409f9", "unit": "b521216bb4106738b3571ad3902d08f6230aabd88cf3a8f9f26f1c8ec01304cb"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "jenkinsci/cas-plugin", "label": 1, "pair": "0458e3fd444409f9", "unit": "c87d2871907798fa2e73d31381e9d63089b0388033f096d247373fec07c2cb36"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "jenkinsci/google-login-plugin", "label": 0, "pair": "6bdccabcf5b9fdc9", "unit": "3d6ddca5eb264d88411b86755909f1d2abddd09c129cabd54c96ff26b4105316"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "jenkinsci/google-login-plugin", "label": 1, "pair": "6bdccabcf5b9fdc9", "unit": "e055b898802279bc63204cc601e29f7f1018efad7228cab0cce00ea5866ecc7b"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jenkinsci/mcp-server-plugin", "label": 1, "pair": "91bc832e70680c92", "unit": "1060c63ff0fae3af567448f3d4c60d9c159b61ccc358964243eb152a9ac38865"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jenkinsci/mcp-server-plugin", "label": 0, "pair": "91bc832e70680c92", "unit": "369dffd1e730d7997813adfbebf1481cc68987cae0d0085e47b91a26d5330cf8"}
|
||||
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 1, "pair": "a4144258dffb3b8d", "unit": "7ab301e1614979f44c353e665540582493f78ef4bfc41209562c19b900710f8c"}
|
||||
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 0, "pair": "a4144258dffb3b8d", "unit": "ea95e8387dff32ded9dc7b04c11a76a545b4e2ea6db0076b9b13971a59904bb3"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jmespath/jmespath.rb", "label": 1, "pair": "858330e01d3cf9bb", "unit": "033cda2b7cffa5a5fa6abf0b7f2cd27ba78d916eb3cab8c79decccd8776f4a3f"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jmespath/jmespath.rb", "label": 0, "pair": "858330e01d3cf9bb", "unit": "d44c4555a5ae19aa17b468ec5cff56b33a0b221df52e04d8abcb656399499dee"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 1, "pair": "c6a49339d4f0c798", "unit": "dc2e4cd5bd963643702d2b832ae170dd25744162e4076e36687256a09e69d089"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 0, "pair": "c6a49339d4f0c798", "unit": "f83cd137b85e651ca5299c5a42176b797ed4964cbef31418c96d63c52fdb65f5"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "jsonbored/gittensory", "label": 1, "pair": "5ec37c0f475001ca", "unit": "5d8175fb21a4649bdc8730358e9b87adb90b2aafc4da5fee203829663e4f46a0"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "jsonbored/gittensory", "label": 0, "pair": "5ec37c0f475001ca", "unit": "af845e9e332555096dd350401977b2f07015433f3db8a040891d9470f3f9e355"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kevin1024/vcrpy", "label": 1, "pair": "c74525e844650de2", "unit": "2e444ad34ee5f47d73f0cfad0a10c70363337b4ddf9067ecd1b8072d8232652c"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kevin1024/vcrpy", "label": 0, "pair": "c74525e844650de2", "unit": "c492d325f5ff82b5f817aa4be1440cec8595054dd0719e07a58ac5501b2ae7eb"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "koajs/koa", "label": 1, "pair": "111a5a80a3abc2c9", "unit": "c7ae403742450deb2bd051ad1231121d2117ba56fa72a038c82d58d207cb3870"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "koajs/koa", "label": 0, "pair": "111a5a80a3abc2c9", "unit": "debe34831afca6f8220b9952036a2d4570babdb5e321a2f8dbaec661a016bd50"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "9950b85412b021ebabb11eea4c490a660ad380d14c673a87c3fb568ba4bcfac9"}
|
||||
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "656df5ff8e69dcdb1f9ba1ed52d734966eb29773bc745587483a355743b8eb5a"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "91652a4e776fbce70cde9e4a99c6da5e17da7f8d6d0404867c61eec04161b6cf"}
|
||||
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e53bb3a217dc4737e8dd03671de4f6a895599024070a985a268c65a034295e7c"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "3d7aba22444811a337abcad0e4fc62990bb704d3a10e72d893342fa94699026b"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "5f18a3026a51f7140b7aaa3aa8563e8b6fdda6065d95e19e25b1dac4b9ce0e00"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "0fccc306ce80cb4cbfdcde0a2e56d637bcefa9dd30423d6c54a4f77860afe740"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "1e2eaa45661345f1b6759d968d7696dd0d96d6fba1350a00e4d0a2ebaaa2e391"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "8b9a8ff0e32a327df75eb8a1811464d5f34690079d5930139dd917063bdd2809"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "245d0e939b7c1ccf098c798492009e3a73ea17400fa613dd865cd369d2e7f383"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "653c48df493f31c2d01c973d1e379a180fbe6a6edc83a500bc62509112175f41"}
|
||||
{"family": "path", "fold": "outer-4", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "3719fe21d45b50b35458bcd2e263e4baee67adbd3c98adf0eaab6bc6183c5651"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "1209a6cc8c1548ed83d4a9a09d6a27f706a23e69cf0c261fc5e3a27536a83306"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "362c90495dc468bb732c71f62d71ec308854161de31ad18d2a870056d947ddb6"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "b52e53fec9707b2b442c4dc7a41d376cf76946d2c45c2eb600c9477fc6baebaa"}
|
||||
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "477e3b90b92e622888dbe0e9d3a23b31271ddfa3104a5ab326eb9e9e765aa01d"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "660c63c4a73bfe64088ae3249e40d9c5d69a6f8a505ff671217c5708a345c5b0"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "4d60c386c801bedb61c1461fced29a3cfb50b09d6435e8617c60398343719d93"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "f1fddcc1354a29bef55610ee2b73eaa440192f2f18a0d22d08057b9d1e1b61b8"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "6d9886c38221b2f9aacf83749218ac5ebc354a6fcc4170e6c78bedb92b88296a"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "5550fec4a81c4c9cb22d7312d8a63e1fb2b03c4aed0c1f316d44c1eb27511bac"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8ff87dd04140f0137ea1891f220b1276156ce10a58be5a49d0c8e07be531ffb2"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "c6c9bde6630678377d250701073c5c223c7892e6ff8f2f8e0293dea22c5b5e7e"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "b9b15958973c83f8d7b58372ab5ba58eb191c400125579d5ae8c5b52d5d1368e"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "b95ec12e4e126ad558e944e2c260116ed6a63f5f3ab97fb5cbcdca75a931ad73"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "38a5545ac9ef67e15db720d1396a2d5915f9d06d6f4772012e5bc95f33770376"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "c8bbe9be91c976be97b7ca387048a7d4537d9f6fcc6cbad0a541d1080629d5a6"}
|
||||
{"family": "path", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "ffacd53f1bd75155e63b318fb9cba27ec03e754841e1eb39d1d4fa0b2c3dfca2"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8fd920175fc43b91b4397acffacb10ab8f7fd0ee38bcf2d2f3ed06a3fad0dfe9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "86c9598ebe3dcda261a69ae545513f6ce73101f3980066e4506ece99bebea84f"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e5190fcc57505116f79a31c51aed69f8a1eea13a8ae3e7265f44c72418277ef9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "2b807ff54e6c47e941124c16d0c27c691cef54f2203060a3a6ce812d478c5261"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "401aefb4b4791301ace2d770d321c6b26743b0aab7b96f8b410254c704be6d2e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "22d186c93498e21418b6c1d0547a73374b13041d8afc89a80d9540fa658b6b42"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "dbfb6df14fe37f8c0efda5f16643297fb0550dfc5ad5805d4e4b844d967773b2"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "f0da8cfe4ac1e586aa7903d72e67b44500bbd5306d72b4b342d10613efc69222"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "2b8fd73943f456da266434470333a7775f55d9a41953e97a000fdaa3257b6a30"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "a612091f452e60f28f44eefec641175170dc5db04456a39a099cbb13c50b4d2c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "4c6600facd83bbf234a32d9e28ac494fb516df6e3d9e9dc28d720ff35252d067"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "fb0ea8cab8c9430d22d1b0fdd65a69146fa95fbff94e720b4861b96538d992c6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "e81f91a56058ac1473579767abea57064c53d7d007797a854314814e3c35ee74"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "5851ecd96a449f98b05f243bfda0a7cf3af8612b27f53c414ae1f4ea67242a7f"}
|
||||
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "328621ef2a94632e5d37ea6267d17b766233384cde35c07e29a4eb672d40ba28"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "04b2eae2128608dad460ba8275142a41a44e32c4243a29e2a6fb2c9ebf3d8f1a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "33337002ff5f5a189610b042522ceea254c9d107c1f24771b4f616a0f96f636e"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "02c09a783cbf058856fc2b448406f3ffa6fbfbacf9b70f4e5e22fd741a5c7699"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "956260164cb21a47c499e5598491031ef303418943f5096f225a87162b588574"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "8bd8c5aed69eae3019cc2d3654b36bdf2b31435e6a020a389cd103696a2f874c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "1d8397ef4a9fde7b48e2e1e871bba0f960a802cd7823ecfc10e9dfaddb955ccc"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "c9db92476032eb4296b950a1eff8fa4aaa26f11542a01a5cf10c7b2e32dc6433"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "d3e0d0f9b0f4b723080989b5486395cda0a5b186a674022ea70e7fa7ab561323"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "44ff3b3e61c02744830c4abea872821ca7e0f94950c082740c5c48ef13604e94"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "4103e979a99233c3e71fdfe9f1cdcbab34a2c60b5351b75c5ff5dfc287b962d0"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kozea/weasyprint", "label": 1, "pair": "6768bfae38c6859d", "unit": "f376519889a00950f5af67eb15c639306ea70da7235d3be4a94dbc19cb096cf5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kozea/weasyprint", "label": 0, "pair": "6768bfae38c6859d", "unit": "f3fea4e334f49ef6fe6b17121fef553eb8e33a9ffa13e6d6cbc4c448da996a7c"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "kubev2v/migration-planner", "label": 1, "pair": "d670253fa209671c", "unit": "1aeebe2e440bdd70525c31f93180f33be93f820db5ba46a2e59c371ef83ee794"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "kubev2v/migration-planner", "label": 0, "pair": "d670253fa209671c", "unit": "b11c8a8a24d3809c0507228568fe7cc05ca0a22ed28cb19019adb13cb619d63f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 0, "pair": "33260f3e6c8ea6d0", "unit": "7ca409c7dbc1c38ff005624e212efc6d424cb421e27b482dc8dc0d6881f4c11a"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 1, "pair": "33260f3e6c8ea6d0", "unit": "fc364d7ed77c7c0508da00e5ba847ec0792f3bc373d8b4297747189ef42d1ae0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "", "unit": "6cce78d7569ad8db7ca0a61d7078198f666502d26ea6c0afb03173a8a4bb16ef"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "82047c0513e43a83", "unit": "6d2dbed1ab1c9d8ac259dd4227be69641fa59d52e816a8dc547ade01d0b9b0c4"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "98ee23cd6bd31cd3", "unit": "ab4bb39be535d70cd95e9035bb958bfc7418576a6ff40a47551894dd140317bb"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "98ee23cd6bd31cd3", "unit": "b70068cbc646bd01d3e10986e002f648a74e9d798900dc59cf3039b749c95c18"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "ff168d95d8d9496e", "unit": "c6ec9478c125ed780d7013c2807f2cec1878f44253667dfed319c28cbfd17d4d"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "b1bfef46dcd3b9a5", "unit": "cd4d095d8380a2fb7ca02fc6abdbae5249cc06455db2929875533f4aa8b886dc"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "82047c0513e43a83", "unit": "d1bade217be8d345cf5d0f0442ee31924187acbb98779dada95274a77462f696"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "ff168d95d8d9496e", "unit": "d561db250b892b91feed7ae877900e1e25963444f73c7699576e010f06da73cf"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "b1bfef46dcd3b9a5", "unit": "ed78fd5a7ab30bcd6e8ddd27c706a19cfd89750de4fbf81c0b567946eb94db83"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "lepture/mistune", "label": 0, "pair": "25f75d77a669b54e", "unit": "0f2825568461562f2a915a4f16ef8a4931290fd20a9801470aeb12677e66e3f1"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "lepture/mistune", "label": 1, "pair": "25f75d77a669b54e", "unit": "bf22bb44bcff4a5baf5d550c9d3e83e5f3ca607475fb51deaf4fb6fccff3d73a"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "cd80750447cf7af67ad809279718d98106c71705fef9ff7254728bfff4ece5b1"}
|
||||
{"family": "path", "fold": "outer-4", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "2ce605ab5ce7ad3a1fb983cd6bddf3924e5da3010d2252a0a4640aab4e5921a0"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "litedb-org/litedb", "label": 1, "pair": "b0a899116dc36b85", "unit": "510ad48af03ba4a1a2a06dd254b135311dddb62d9fe5a2e8e8b12627586e6b13"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "litedb-org/litedb", "label": 0, "pair": "b0a899116dc36b85", "unit": "b414718a8a5f79739b88db24da6ad9da8e48bcb6b5e2dc99158848cdaa226767"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "8e22ee30441b20484090659b0c97423c711f5d10219041bf3a399623fc056a07"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "0f9070df9dc334f981cb01008c260b8bff78dbdeb48ce672830ed95caa8406fb"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "5e352f797b1422b0df477bce4730a9ce33cbd0837c3d63da94e4d8801a9a4b64"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 1, "pair": "dadccb128ac0e120", "unit": "505097494da5939404ad7d5b4af2b1c4a10efe94048084492d6ce60ebaa04c97"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "dadccb128ac0e120", "unit": "84ff8ca2d98a1b5d1ca4bfaabe7d19e03a396042b228c8d18d1185a178389702"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "91e261c54bd2c3c3a9ac04e48894ecabaca5b13e7db4fbd5578f9d266098117a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "a308dfd59bc398824e2454bf6cf6cfd5b7880e5b948a82424f14753d2b6fcdbe"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "ba24eb554458bc97314ed561a7b2f4a4e643dde1205c32de4c8fcdd8c46e575b"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "mganss/htmlsanitizer", "label": 0, "pair": "24d8f1840b67fbe5", "unit": "7ffa4a229a62266788fbf64890f81f20358b3eb8c35d3ab87d0c339d5bb4b507"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "mganss/htmlsanitizer", "label": 1, "pair": "24d8f1840b67fbe5", "unit": "d19493ed7941e1cd25a76dc50504d97398770d07ebc302dd66d83441e946c468"}
|
||||
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 0, "pair": "9f8a60eb48d74bc8", "unit": "80fce34f2377731e3de4355c7ea33bb20fc2fc00a5f388565b23018c7a3ed008"}
|
||||
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 1, "pair": "9f8a60eb48d74bc8", "unit": "b68e62722a327c9d5ce6774eb96527451729410768efc0b5c7ba604ec468052d"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "miniflux/v2", "label": 0, "pair": "997f9d2efc32302d", "unit": "b587bb499945a9f5f0749eb4d57cc1027cfd21c8292bd32be4d4e1cc1b1db261"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "miniflux/v2", "label": 1, "pair": "997f9d2efc32302d", "unit": "f8d9c9a7c12797b3caf042e6bfdb89a5072c4eae1fd36be6196f9fa326cdf257"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "mixxorz/slippers", "label": 0, "pair": "8e6b4b4e28981590", "unit": "3cce3e8f9ff3f1d261f9b96c844c1e4da07bae72d5393897b3a929be25372edc"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "mixxorz/slippers", "label": 1, "pair": "8e6b4b4e28981590", "unit": "b105689ac24ed334294c434a77bb6d56d5d598830ffe8a5d60ab3be5290b47ad"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "0a36cd5177f3b2c20b433db6bfffe5ec5b7e74cacc34a86ac581f7a0a559432a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "e7fc97d83013c9ff", "unit": "5ab444e64297fc19065478104c7ebdabe23cc415f22adafd0cf82c98e4c5cae3"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 0, "pair": "e7fc97d83013c9ff", "unit": "6b462ea30c1afda572600a10324249223932d36a032eacafac8c837bccbdce87"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "84d6a4202a082e0d", "unit": "8175082f40d8d61d44b54579c84a808e25b89110be5d8fd82feedf7dd5e0d5ab"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "8abefca43ee4ae2c797033da31f85c17bfa7be6b35413d70b0216415c5db9a09"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "91d213946859f33d3f243c43aa8285cc6e8ecb3db1de938f32af775a70607f8e"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "a244d9f010ddb43a09e82363dfabb0e406d411fb5de08ab7d6c9ee367f2a15c3"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "d27d174d6e8e5f1682a1c42723705e89f0d18c95ecedd0567e313daa9a0aae44"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "mongo-express/mongo-express", "label": 0, "pair": "84d6a4202a082e0d", "unit": "f45608ed47fa11ca25093641f6021ec06076c18676abac9b809f31b3b7713de8"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "moquette-io/moquette", "label": 0, "pair": "55d1ce4f29dd1583", "unit": "a877ce3462da316bcdd82ce9db1d9887e887eaa83157922edb1663413edae1d6"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "moquette-io/moquette", "label": 1, "pair": "55d1ce4f29dd1583", "unit": "aee67be0ba31f99c7a24faa1bad6178c6d37361a308cf5aa10138e6ce4bbcfeb"}
|
||||
{"family": "path", "fold": "outer-0", "group": "naorsabag/openhop", "label": 1, "pair": "16e529fe8c5486b2", "unit": "1d21acd0171726775d3da4589b660645fcd43cc026042a2b27e0e0e8374eceb7"}
|
||||
{"family": "path", "fold": "outer-0", "group": "naorsabag/openhop", "label": 0, "pair": "16e529fe8c5486b2", "unit": "f8e004b67ac98dc3392ea4115051e6bd10ce22eb638a762f0f16e1d4200bf129"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "nuxt-modules/og-image", "label": 1, "pair": "ef603436079b0005", "unit": "77f843b5044bb5da935df4b2620bf2abe9e7dceb3147439e883927112eaec22f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "nuxt-modules/og-image", "label": 0, "pair": "ef603436079b0005", "unit": "f2ce95f0e44e73986784b63a702ffa4035c423056cac3667a63140a7e8d795c2"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "opensearch-project/opensearch-ruby", "label": 1, "pair": "1945f28a3fc5bd55", "unit": "64c0b7ccad09192eeaf590befe0de0087d5b48a65a4f6cef41415f3f54a8f536"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "opensearch-project/opensearch-ruby", "label": 0, "pair": "1945f28a3fc5bd55", "unit": "b55cd6759d4b7861e589052aa534bd0554586db46a329ed3beed7244d579e8c4"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "openssl/openssl", "label": 0, "pair": "1c9e47835232f644", "unit": "90e220d3540b88dd9efc8e66c8ab519d4cc78805f16bed585b795c78cfbb4d58"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "openssl/openssl", "label": 1, "pair": "1c9e47835232f644", "unit": "b5a185b05235f70f6cf27e2adba919f4d8393dbdf2d99de095548a21946aeaf3"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "openwisp/openwisp-ipam", "label": 1, "pair": "9b2b34cf62a76ea3", "unit": "91a3984e47f17512db8523e04f863525c67db18631d8f11776cf61ca7221c91b"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "openwisp/openwisp-ipam", "label": 0, "pair": "9b2b34cf62a76ea3", "unit": "d91f1e14282a0c3c8da57c6935a1625e881374bd205b5baa9e4a341d9109ae21"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "oqtane/oqtane.framework", "label": 0, "pair": "a20cdf9dd208720d", "unit": "18a26df68e4b81d411cfbcc8155a7e6f382753d6e1ad95eece7ed24fa8c44324"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "oqtane/oqtane.framework", "label": 1, "pair": "a20cdf9dd208720d", "unit": "e6fb74441fb585d7ef5855317fba795ef0f0a0e6eae9970a49556ce3070c3254"}
|
||||
{"family": "path", "fold": "outer-4", "group": "owen2345/camaleon-cms", "label": 1, "pair": "d6930fe7e752b7ec", "unit": "06f0341dfbf1aa9504083737df5d9be3cedfd4e3a198799d8075a29990776800"}
|
||||
{"family": "path", "fold": "outer-4", "group": "owen2345/camaleon-cms", "label": 0, "pair": "d6930fe7e752b7ec", "unit": "30e2cd8bd381d7568b78eaf982f562d278a6a8d942d5282466532c1441bdf0ad"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "perses/perses", "label": 0, "pair": "2d482951d7380c4f", "unit": "5c1a721da6f85d1d5dfc8cbe5e9809b093a09c059e5abeebed2cbbee55fbd498"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "perses/perses", "label": 1, "pair": "2d482951d7380c4f", "unit": "6cabd1c4b5aa94cafdd07c0c6a3f9abfdc03761ddc5b0a2bebc72a508d020b8b"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "", "unit": "2300c097374b9098671dcbcd42d02d08172e0f3064a8323223d0fe79a6106aab"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "pgadmin-org/pgadmin4", "label": 1, "pair": "0664148b0b192456", "unit": "d702617aaf4d19811baaa18386a3c8b0eb35cd3fd72e8d90611d34850e990858"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "0664148b0b192456", "unit": "f5ed13d811c5b16d46cbd4a3b9edb8f7d2db49acbd49d95f39f33eaab38f58f5"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "publify/publify", "label": 0, "pair": "1972ea9f72b178b6", "unit": "3ddabb356d78e7c65ec8f88b6da8e4fcabc055ae73f2f6d92c50efddc5d4fcbe"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "publify/publify", "label": 1, "pair": "1972ea9f72b178b6", "unit": "6875dd5e9c0ff8a1b693d29a109cb2e62e6ff4081df40868aca70e4770476757"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "04aa777130c02a06", "unit": "16aa43c87956c44a4b115245140eefed704095d37cec0cc6410377403cd2fa7a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "23744137e0485a36f686f5f1b5666780fbed3ca522e8c6d35687a391ef504498"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "67565d99fe28c68f", "unit": "2692316792e6f02fb51578cf30d71bba6c73bf5420c36f709681c8eaa69807d4"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "2e36f73002c416a68d077bffb0335d82346c7507f2478353aeac005fce5a9cbc"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "04aa777130c02a06", "unit": "67721f67055eb6fed955ff22bfe4060344662eccb369431835a2116ad120c4f8"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "8fc1c96fa6faa4d3966ecc42876189b18ddab779d58f47cdf28a1155f48a6ac4"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "67565d99fe28c68f", "unit": "ab535098a8e709b022afee1da15dd1fc371d389056938ee4fa4971b2966c823a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "d46f94235bc462360097e7a22e1d442d95f049306631489a4d8b25f647a95bbe"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "edd6dd4364442f66384d14e9317ef2f6234e32ab78327a45741b8a816b4332ee"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "fc7df6887ac719d4615934c0e8c3296728c2f6089f233158c59989027c802d72"}
|
||||
{"family": "path", "fold": "outer-2", "group": "rack/rack", "label": 1, "pair": "c0412509f15fe3e1", "unit": "63bda2a71ad7155ad2f59aa9086efef3fd3710dc82c57f4b32ec8bf561460239"}
|
||||
{"family": "path", "fold": "outer-2", "group": "rack/rack", "label": 0, "pair": "c0412509f15fe3e1", "unit": "b1c9a070c93764655786c3de5fde536c150e319efe47548614cc544b73a8ee2b"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "rails/kredis", "label": 1, "pair": "feed5f7de11289a4", "unit": "71531567b2b8609ae0d31cb177b8faea4f5fc61923e513b0154a2c941287ce2b"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "rails/kredis", "label": 0, "pair": "feed5f7de11289a4", "unit": "d6be6d397a301a07faa761163d3d9e270d71d23d3bdfd084eeafb1e62b91dd71"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "redisson/redisson", "label": 1, "pair": "ffd4b04a7bebfaf8", "unit": "54f7519d3d59b6d8d69a5f8f7bd78e360d3fabaae00b7587c37e388719169370"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "redisson/redisson", "label": 0, "pair": "ffd4b04a7bebfaf8", "unit": "c5e42c9ab2ee54e46700ab0b64e239b567fbc75ccaceaf6cbf20687813663332"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "remix-run/react-router", "label": 1, "pair": "73f2a014c6a699ed", "unit": "44ba9d9d8e75769c1444ebf37a64941a845b4bcae678ec79d64dc84e5cc067a0"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "remix-run/react-router", "label": 0, "pair": "73f2a014c6a699ed", "unit": "b430d488c486cef26b1c661f773e7e9e4a177eeea73537a033eec98d532e4eed"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "remram44/taguette", "label": 1, "pair": "cb0c638282db0d37", "unit": "a49e6ad2e7c2a5ad0d68d5d69c263306b501fc0dc9382e701fb5fe680611196b"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "remram44/taguette", "label": 0, "pair": "cb0c638282db0d37", "unit": "dee28099d3b37cbd0ee14412122ecc5f8488ed6bec01863520b87706a040fba1"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "roest01/node-pdf-image", "label": 0, "pair": "09bce271e8530fbc", "unit": "35eb19b4c6176b37287e2123bb98ec4045ce6bde7cc3864dde317c024b0935d8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "roest01/node-pdf-image", "label": 1, "pair": "09bce271e8530fbc", "unit": "dd33291cc494bbc77d44e9b13d16f171d32c9746637506f315f375e8a9ddee95"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 1, "pair": "40d8d81a764c508f", "unit": "7deeed73c29c74ccbed29be049e109d363d41e4b5eea245ee57780b85f0e137a"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 0, "pair": "40d8d81a764c508f", "unit": "b5cdadf848e67910550666ba2a6ac67b24112e501c02dba1865766a316317a23"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "", "unit": "15dbea384b3255457a44d9382efc6944dd27f1d411d903344a5d5e5fd194b72f"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "b2ac284bd70dd684", "unit": "225890ba6694dc8f4fbab46e9c851dff8c095e906cf14db15d01ebce0891aaab"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "b6eca35de93a5be8", "unit": "42eb8427415704e060e73b5ce0943b014ddf380ca756d5ec47c558e59a79895a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "b6eca35de93a5be8", "unit": "4c9cdcc11a18641ef41e9308712c5e8b830ec9ad5ebc3a956c8701e4f77b39c0"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "6a53939838afbdd7", "unit": "5794809e595311c86909d673a570d003aa025e4dcd6abc5641b641d935beb0b1"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "46d3ec7d49cb8ca7", "unit": "5f7a63ec650915962ee5255f5416df63ab882d6107f3b540be36d481798a8944"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "6a53939838afbdd7", "unit": "97a7018535ab511622e3ade9e53d570ffb1c80633eaa372fb76e749b3a586c8f"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "46d3ec7d49cb8ca7", "unit": "a84e9ab47f4f2b584684dbb81575758d9032fe3f2ed53bef478cb9dcfedaf677"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "b2ac284bd70dd684", "unit": "aeceb1ebc863ebbc3b69ae56c5b78d2dba005132b75cbd6af929bb0bfe25da9b"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 1, "pair": "f42cab375ace1739", "unit": "e329d4a6ac757680bf3d292a81f1f74f8a8a9b482d1651efe9f503f6a4de8796"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "saltcorn/saltcorn", "label": 0, "pair": "f42cab375ace1739", "unit": "e73c5aac41c0b93072223a3097fb175548ce893e0678ff3a22bc7571745a17e0"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "sgabe/dsvpwa", "label": 1, "pair": "", "unit": "52354ccdbc16f209e4692d97d5237e04ed030cf4a5e71f77b5a767962b355b4a"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "sgabe/dsvpwa", "label": 1, "pair": "7d23be667536afff", "unit": "b5016a5a53409491ba805ed8718599b84250b7b6914da0434b0882b0b3578dc8"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "sgabe/dsvpwa", "label": 0, "pair": "7d23be667536afff", "unit": "f1b01102968568d3ba1b2fdc70f4ae281f433fb20d29eff2a55fc45555ddbfcd"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 0, "pair": "1199de75f426940b", "unit": "10fdc236f43a02b7e291eb8bf6ada297a2ae98086a63b171057bc7bfc37a3377"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 0, "pair": "22112c31603bbe9f", "unit": "5eb3b7523b5a196dcaacba08d68457f9a45e31503ea02745ca3a7d0542f9881c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 1, "pair": "22112c31603bbe9f", "unit": "6bd5bdfd81b0a3374d59cfb2c1f7fa2b27887f4672f4485e77eb5234b569983f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "signalk/signalk-server", "label": 1, "pair": "1199de75f426940b", "unit": "c09dffd04708540845c8d1dd77a2bd989bd35f264abf652a2db55f8cb1faf7a9"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 0, "pair": "7aab025ea7b97019", "unit": "7487a34cdfdb532a5096764a47a3e565fa509605c1df0e2b105eb9c751fc5307"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 1, "pair": "7aab025ea7b97019", "unit": "9c0be2a58741f095fd0a9cdc60d4ca4039de1529699d5701dc4d0bcf7d5bf90c"}
|
||||
{"family": "path", "fold": "outer-0", "group": "simonh1000/angular-http-server", "label": 0, "pair": "4de5441f1776f083", "unit": "a3a35d573da308cd1d0614a26616bbd37b360d9ea416894cdeffdb59abd691d4"}
|
||||
{"family": "path", "fold": "outer-0", "group": "simonh1000/angular-http-server", "label": 1, "pair": "4de5441f1776f083", "unit": "dc33fc26c1bf7c67e5439ddfdad06c0faf16c796e0fe8a46b6e6a541e708676d"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "smithyhq/sqladmin", "label": 1, "pair": "d76d5232a8587127", "unit": "1ec32d427822d1ddd3b0d7245f5771ddeb56c146ac8ff9959dd85760ff659a20"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "smithyhq/sqladmin", "label": 0, "pair": "d76d5232a8587127", "unit": "5143b19a1a1e6b9162b80594ed7fb5c0c57ed0b8abfd793bfc71089552d57638"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "soyuka/pidusage", "label": 0, "pair": "5f6e814444b4acd8", "unit": "2e8aa07b1aa4bfc5349b0d4464f3f2a6500aa55ff765b0139e9901ed34151ffa"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "soyuka/pidusage", "label": 1, "pair": "5f6e814444b4acd8", "unit": "e26c726c08be761b22fcc6ec17fc1fa7aba4769f08957f024e5545906df62797"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "spatie/schema-org", "label": 1, "pair": "888e955fb6cfb4fd", "unit": "6f1a431ca97afc34ceec48393eb134a8928a312e6f761eb907c6b5855cac6208"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "spatie/schema-org", "label": 0, "pair": "888e955fb6cfb4fd", "unit": "7b62415d17569924d16238a4aa28ab1af2db6561f56be9f59d8075fd1bc4a941"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "strangerstudios/paid-memberships-pro", "label": 1, "pair": "", "unit": "ac905d12b92954263c046bd0138a682cea86cf8476336d6baa664ade28d34ac6"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 1, "pair": "218a45d869bd7e39", "unit": "309ebac468b7ab054c8396ed495770385ae3590caba43709745ba2d1db4b6d54"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 1, "pair": "", "unit": "3cc8613813dec13cb1b7b45b296688da883f0b36cd2c4fc1da6e81703fb59ab9"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 0, "pair": "218a45d869bd7e39", "unit": "9bd2331dbc260451355307e4c37bae17ad6676654c66181934e1b35fb174cc5a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "b087ac4d0989e0b72698a9644ef4eac8073b4a15bb21730d63e4730b5cc58a60"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "c5d1e8d19560c997018fdca12eb60f1799be76c33be058b88472f5a2fc6709be"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "22befc17a75a688e2f9358d2ddf9a7b197da44cd95dbdeae5e0191b113ee54d3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 0, "pair": "63f496d9ff53e39b", "unit": "6a98f8145776c78793b37156e9a81757841bb6de80016ab4fa575209d724db3f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 1, "pair": "63f496d9ff53e39b", "unit": "7ac60c96c15f8517e089cc2240aedcf1fc30e54a41e35fe03c6b965bc393fbd6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "ae39cb55ef2a87c5347e0ddc40975f142450e82de4909e93ba62a7ed0efb4134"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "thenables/thenify", "label": 1, "pair": "d1b19898f8deaa41", "unit": "3763c7b407ad471820b5f8fa25bde0ba6d91e15c07c56681de4b41add418b0b1"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "thenables/thenify", "label": 0, "pair": "d1b19898f8deaa41", "unit": "bc8b934938c30687f3d4f982392d2cfa9504a916b463ccb5c46fe8a30eede488"}
|
||||
{"family": "path", "fold": "outer-1", "group": "thumbor/thumbor", "label": 1, "pair": "f5f33fa6fe7b8492", "unit": "938bbf146a9037e687190bcee82c047492dffbe90efea40a49a647f81a0f1ed0"}
|
||||
{"family": "path", "fold": "outer-1", "group": "thumbor/thumbor", "label": 0, "pair": "f5f33fa6fe7b8492", "unit": "b90c9f7e0011685f1a8e853a9ed0ae37fd437ef6d6eb0ba17c240aec2c22b603"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "tomasnorre/crawler", "label": 0, "pair": "f74364dc8e35c8ba", "unit": "11e760e99c5c852676c9288e18a250febb3854e9047b51e8b9ecc9f3b423b6f5"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "tomasnorre/crawler", "label": 1, "pair": "f74364dc8e35c8ba", "unit": "5fc980194040f4c4d803075d4085618cc0260ae8f3454b26b9b704d30200ffea"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "tornadoweb/tornado", "label": 0, "pair": "db1c51833994ec55", "unit": "c2ce8320c778dd55434d7413844797b68e6fcabc4dc509458a9d60ca70b88c60"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "tornadoweb/tornado", "label": 1, "pair": "db1c51833994ec55", "unit": "f2d6843547af1276ae8a4dff3a7786913fa79dc69540d4fbb95c4c091e27c7f2"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "truelayer/truelayer-dotnet", "label": 1, "pair": "f4690679bf9d80aa", "unit": "1a4e24df8d08672ecde4709a77ea5c437b2198fecff8bfff373b63c9eaed6e63"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "truelayer/truelayer-dotnet", "label": 0, "pair": "f4690679bf9d80aa", "unit": "78711973ec84867671314dd7878769092b32f59860eafed50a0f6979daee76fc"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ultimatemember/ultimatemember", "label": 0, "pair": "85c15e3fc2b370b2", "unit": "0990d8f41cb07c1cf217812e51a7025381de4ba68d1be3799e26241d81dcea67"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ultimatemember/ultimatemember", "label": 1, "pair": "85c15e3fc2b370b2", "unit": "9da1d63f0c4da270f66eb87bf9c342ace883a0aef78c4643e254159b5da8fce3"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "viewcomponent/view_component", "label": 0, "pair": "8f5a6abedd742bbe", "unit": "71e3ced643a244100cc3fde234f5b4b27655214c7ae0ff67ab61e3e48a8fdf78"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "viewcomponent/view_component", "label": 1, "pair": "8f5a6abedd742bbe", "unit": "7c8cbfae87127042f9f43621345b4f7958bde27a3ff9db76254063e80735f3ce"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "vincentmorneau/apex-publish-static-files", "label": 1, "pair": "c9538342f2d7efca", "unit": "2b8db57d65ef1d67c457ff5a336e8a542d03387395d69d5e16815c4f00f40723"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "vincentmorneau/apex-publish-static-files", "label": 0, "pair": "c9538342f2d7efca", "unit": "e2ed113dc21ad5f7caa71ba4a5ece8f8f900a97fd0d00dc85f7ac79ff5bed19a"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "vvo/node-whereis", "label": 1, "pair": "8acd426c5771b37c", "unit": "983e146a60e061254b15dc8dd573cbba57f8dd0358f23ea1b9c313ef69069af5"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "vvo/node-whereis", "label": 0, "pair": "8acd426c5771b37c", "unit": "f80dfec45231e4778078cb10f00128ed445177cac0a062fc2df5ced5c0171f21"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "waterfutures/epyt-flow", "label": 0, "pair": "9b84800ce5c395cb", "unit": "9b7dfdceae03fd86ad3260a03f85bcdcff8f32f10c265d23743ff8dc98eeefb8"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "waterfutures/epyt-flow", "label": 1, "pair": "9b84800ce5c395cb", "unit": "b99dda150c0da3defa5f4c527ef6c402f5589dfb2f2f03f3e5e424910eb48c53"}
|
||||
{"family": "path", "fold": "outer-4", "group": "webpack/webpack-dev-middleware", "label": 1, "pair": "ad64e8fac4ac3ee0", "unit": "36dfd01915a553d86576e8ecd3b737b8e2b297aa542f58cc83b448060e74398a"}
|
||||
{"family": "path", "fold": "outer-4", "group": "webpack/webpack-dev-middleware", "label": 0, "pair": "ad64e8fac4ac3ee0", "unit": "c2b6d40f130d35bbfc16fab9411c664fa06321c59fb1d901a1987a69cb09c72e"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "wintercms/winter", "label": 0, "pair": "d48c08b19f560848", "unit": "03d02c9e25eec0f380b125279fce35e8867065bf6e1ec75f9f39729ce6efdc77"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "wintercms/winter", "label": 1, "pair": "d48c08b19f560848", "unit": "abbea69b7d0fa68c3ace442c25597e128cde354b417787b5913fd10d666c0d19"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "xyproto/algernon", "label": 0, "pair": "3844ed3d703d7ef4", "unit": "445efda074e4d9f6c1634225eb08a3c3eb1257311c995ce891df2c791a6f5c8b"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "xyproto/algernon", "label": 1, "pair": "3844ed3d703d7ef4", "unit": "8ac27b08549353ada819caaf99a5180d2f2f515b03aa8cd99a5d37b527ea3a5c"}
|
||||
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 1, "pair": "a63eae7d621a985e", "unit": "04ca0b583b09c3b4f67bd2496a4607ecb134d24b7bdaf8960fa151e40ebcc447"}
|
||||
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 0, "pair": "a63eae7d621a985e", "unit": "ecfa2a67726ca9d3e2621a4d8be60145095b76aa18ecf0268212341d63752871"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "zumba/json-serializer", "label": 0, "pair": "0d9f529b7f0084b6", "unit": "35aa369236f510634b3aa7b54b04e9da0f80877e515786a888e760f8fe33c524"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "zumba/json-serializer", "label": 1, "pair": "0d9f529b7f0084b6", "unit": "ef4f8613c0914eb815504e00cd6ee05df38352fd3b27826798e6623594017042"}
|
||||
@@ -0,0 +1,65 @@
|
||||
# Experiment exp-003 (learned-decision-engine Req 6, Req 7.3/7.4): the BLOCK rule. Registered before it runs;
|
||||
# `ousast learn experiment register` records this file's sha256, and a changed file is a new experiment.
|
||||
# kind: rule -- both arms decide from the SAME scores: one scoring pass of the six slice-2 incumbent programs, replayed
|
||||
# from the response cache only (zero cost by construction: a zero-ceiling meter around a refusing client; a unit with any
|
||||
# uncached response is excluded and counted, never asked; the meter's calls and usd must read 0 or the run is void).
|
||||
#
|
||||
# Threshold selection (arm B), fixed here: per family and per outer fold j, the candidate thresholds are the raw scores
|
||||
# s of the training rows, i.e. the family's paired units in the OTHER four outer folds (each of those predictions is
|
||||
# itself out of repository: retrieval and demonstrations never cross its own fold's boundary). The chosen threshold is
|
||||
# the lowest t such that, among those training rows with s >= t whose majority verdict is not `unsure`, the Wilson 95%
|
||||
# lower bound of precision is >= 0.95. No such t -> fold j gets no BLOCK. Fold j's units are then flagged when not
|
||||
# `unsure` and s >= t. Fold j's labels never enter its own threshold (tests/test_learn_rules.py).
|
||||
#
|
||||
# Pre-registered expectation (power): a Wilson lower bound of 0.95 needs at least 73 flagged units with no false flag.
|
||||
# The frozen units (353 in 146 repository groups, 244 paired) hold 38 paired positives for injection and 16-19 for each
|
||||
# other family, all five folds together; the training rows of any one fold hold fewer. So arm B cannot offer a BLOCK in
|
||||
# any family by construction, and the expected verdict is inconclusive with arm B flagging nothing. That is the finding
|
||||
# this run would confirm: on this data the bound, not the calibration gate, is what makes BLOCK unreachable. Arm C (point precision >= 0.95 over >= 10 training flags) is a secondary,
|
||||
# non-deciding arm that measures how far the held-out precision is from the bar when the bound is relaxed.
|
||||
id: exp-003-precision-bound-blocking
|
||||
hypothesis: >
|
||||
The calibrated-probability BLOCK rule (Platt map + calibration gate, ECE <= 0.05) is the wrong instrument for M4,
|
||||
which asks for precision on what is flagged. Selective blocking -- a per-family raw-score threshold chosen on training
|
||||
folds only as the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95 -- reaches the M4 precision
|
||||
bar on held-out repositories where the calibrated rule offers no BLOCK.
|
||||
kind: rule
|
||||
programs: # the slice-2 incumbents (benchmarks/measurements/2026-10-02-decision-engine-slice-2, program.id)
|
||||
access_control: 745cb1d76bbb09dc1574d333d2118489be62e39886517894d3699fa565b0e8c2
|
||||
deserialization: 3c0c2831be7d78198a62351661c37867363c501cb40e3ed7b0119c155af09654
|
||||
injection: b207a0710e0abf053b3b9397b25f9ee1cab4679144865e3a82c1b9be2c7b4982
|
||||
output_encoding: 5d768f16724562c381c09949ff3df9cbb11d58cbb7a9c0515d0dc3f858d3818e
|
||||
path: aea396680a4874dcb6de70e240ca0480c83424debdd3710bc3f1ea6a612661f3
|
||||
untrusted_destination: 1242b3973fc89056aa08e00843043d68235df217d8e3ef8a0ea0e0b81c142753
|
||||
arms:
|
||||
A:
|
||||
label: today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p
|
||||
rule: {name: calibrated_block}
|
||||
B:
|
||||
label: selective blocking -- per-family raw-score threshold from the other folds' paired units, lowest t with Wilson 95% LB of flagged precision >= 0.95
|
||||
rule: {name: precision_bound_block, bound: wilson, precision: 0.95, select_on: paired}
|
||||
C:
|
||||
label: secondary, non-deciding -- as B with point precision >= 0.95 over at least 10 training flags
|
||||
rule: {name: precision_bound_block, bound: point, precision: 0.95, min_flagged: 10, select_on: paired}
|
||||
unit: candidate
|
||||
pairing: paired # every unit scored once; every arm decides every unit
|
||||
cluster: repository
|
||||
order: {randomise: none, seed: 3}
|
||||
# primary population: paired units (both sides of the candidate present). It excludes every positives-only group and any
|
||||
# unpaired unit, whose positives never had to be told apart from their own fix. Pooled (every unit) is reported beside it.
|
||||
populations: {primary: paired, secondary: [pooled]}
|
||||
metric:
|
||||
primary:
|
||||
- {name: flagged_precision_lb, target: 0.95} # held-out (outer-fold) Wilson 95% LB of flagged precision, per family, arm B, paired
|
||||
secondary: [flagged_precision, coverage, flagged] # coverage = flagged true positives / positives
|
||||
# Decision rule (decide_rule): REJECT B if in any family B flags held-out paired units at a point precision < 0.95;
|
||||
# else ADOPT B if in some family B's held-out paired Wilson LB >= 0.95; else INCONCLUSIVE (the incumbent rule stays).
|
||||
# A family "reaches the M4 precision bar on held-out repositories" when its held-out paired Wilson LB >= 0.95 (the
|
||||
# point precision is reported beside it).
|
||||
decision: {arm: B, target_precision: 0.95}
|
||||
test: {resamples: 2000} # repository-cluster bootstrap of the held-out flagged precision, beside the Wilson bound
|
||||
mde: 0.0 # not a paired difference test; see the power note above
|
||||
budget_usd: {total: 0} # zero cost is a hard rule
|
||||
families: [access_control, deserialization, injection, output_encoding, path, untrusted_destination]
|
||||
folds: {kind: outer, k: 5, seed: 0, compile_fraction: 0.25} # the incumbent evaluation's folds (learn/compile.toml seed)
|
||||
units_file: plane/experiments/exp-003-precision-bound-blocking.units.jsonl # frozen by `ousast learn experiment units`
|
||||
@@ -0,0 +1,501 @@
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 0, "pair": "6b0b3cab2df3397b", "unit": "0ac37b7154242dfa637140af643fb4595e85510a17ab1a12c111441b9d293238"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 1, "pair": "6b0b3cab2df3397b", "unit": "6e5e274946d0e8f13a7ef4e37f6c7ed1828d057304a9262db0311413f18e1731"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 0, "pair": "c0d0c91757f5ba95", "unit": "285891b16ee6870af50b36dec3c2ca9a90ee15d9e14851c61e66ad443e6eab59"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 1, "pair": "c0d0c91757f5ba95", "unit": "f33fb2413edf1499a524c4592b3b2f93d166be845f5b65291969a086a4bec334"}
|
||||
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 1, "pair": "f56f2a107ae591be", "unit": "350149bf9cafbb547f500b714a164734d2361b22e86fc9819c7bad3507d8458a"}
|
||||
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 0, "pair": "f56f2a107ae591be", "unit": "ad125319dd2d4c9d2d99a1eb34bb4cfab9f099a96e7547d0d59a8ecf1c990347"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 1, "pair": "ee577ee4b6076fab", "unit": "2523f55d035d1a772a842090bc335e5d755b2fe15b376dd5b6b9e30b9dae82d1"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 0, "pair": "ee577ee4b6076fab", "unit": "ca0f6c160d7839204f6fcc1709bcbc749a02a0b3b8bb648e2a238eacbf22ed78"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 1, "pair": "c95b6b01d0c2db2a", "unit": "15540c23be0c42472a49039658ec2ec0bbd8c6b318e35dd040a9d427370e0ef0"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 0, "pair": "c95b6b01d0c2db2a", "unit": "542471980417e964182d7d3db59a05fe12bdd072449f3bd77f7daa5515dbc909"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 0, "pair": "29252934ef4bcfcf", "unit": "63bae9240b31f9cf1dfe00d9b01d4d0ffda95931c9e7d040df1fd718f04d828b"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 1, "pair": "29252934ef4bcfcf", "unit": "cce2af1743834dcc3df58827ac263a2d63296ca1b2a2f9bdb811954fb5767c33"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "bcc636b23dfb5f570d8864b17fe07b6b4fb52b65a40988c32011f1422f74d504"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "dbcf1e67a1be7ccf7d51701f9b8339339164055135443044c23a446b16b3d934"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "50408f68114ecb4de3bebd836d52d9afad86b22dd1843641ac8a49b617c09e33"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "anxolerd/dvpwa", "label": 1, "pair": "", "unit": "cafb0f00f80dc6beeba40eb95c764bf9a3569193070ba321c28677dfa425ec88"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 0, "pair": "07317a2258292656", "unit": "a9f292d7cb467def582d9c0b31a397b13cdaefa77380ce971bfe9fbf535bb660"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 1, "pair": "07317a2258292656", "unit": "ed363d0dfa620707be477b90eb4b83a901bfd402040b918aeee6f06b2c5b795c"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 1, "pair": "1544b6dfa2299d60", "unit": "d45018fc4ce3dfd715c7087d4d197ad4108c0c8b6625c02c4d4da0adb9f6a140"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 0, "pair": "1544b6dfa2299d60", "unit": "f295c7dfb095595a4aa985c58688e4757c29be9e06b7a4ca55035ac5d06b1ef8"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 1, "pair": "fe0e104d111c2555", "unit": "088036fab9962be2b96f9162ea4f32190575678d3c5ba96bdfeac178bc22b2c4"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 0, "pair": "fe0e104d111c2555", "unit": "7fd2b56a816e25358477eee64487fb2c77f34c652703ad0a7120c368492017a3"}
|
||||
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 1, "pair": "b6742570f7f0e026", "unit": "7c6c1d27aee6557f1b6bf371b8209003f0d2d1122cfa35b6629eb43f38769cf4"}
|
||||
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 0, "pair": "b6742570f7f0e026", "unit": "df8300d84dd784d0be46969ea432ef97b8ce611bae7a896ed4f349bd4d6897db"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 1, "pair": "0f584bb97a154488", "unit": "222a84ffc7c58d45e8f10df221762b23bf20d0287ecd20f375558134928886ca"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 0, "pair": "0f584bb97a154488", "unit": "25e4c9d2941f59f072385a7bde3fd0bd76f9e986ed24b06ce81962ab447f5094"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 0, "pair": "b15b924b4f7b1ae1", "unit": "2fa95fce0751c97db1a12a80ddb249ff5c54ee31c7dc3cd47765099c7b673e52"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 1, "pair": "b15b924b4f7b1ae1", "unit": "52be8128053102eafbee67c2d3aac88c83ca290b943ab2f2cfdcd2630601c7d9"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 0, "pair": "d31ca58f2a1d27d1", "unit": "2ae40ea61d691060e7b9f6c8e910e3aa0edf16bc6f88bec81aa9a7dae5cf5603"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 1, "pair": "d31ca58f2a1d27d1", "unit": "4876728701698045e65eeae6e11e133c30f43b09029d7777d6d780f7d0855885"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 0, "pair": "bb3d7baed62e6bb7", "unit": "e8ae88fd2b677e3f64ac94bddb012e3b903afa294ecc87774d10087c0d50489c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 1, "pair": "bb3d7baed62e6bb7", "unit": "f6b0d241891dc1d897d74d98b6ff7ada2c1f1ce7874ca32097630c06ca53e389"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 0, "pair": "d9d8cfa38981847f", "unit": "864f1800617f5218a2300ce390ba7922fe1bd0a182de115952342b61d627bb16"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 1, "pair": "d9d8cfa38981847f", "unit": "fdcda2307e20ef1b9247b7082a85337f457dd1f4e648bb8264077bf983c03271"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 0, "pair": "1f73ad2576d52a64", "unit": "0bc9634940734cf7f41628f220032fb7e2fd2ff5b584afc519875e90ea0be5a4"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 1, "pair": "1f73ad2576d52a64", "unit": "ddb59b2dc39715281a6d48b32e7ca4ec0354b85b953fff55139579f83905b7d6"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 1, "pair": "a7326e6daa507b79", "unit": "793ebf4c8e0f46d8ce3b3d090d2f6290d4167931a3b574f8269b90008e3dcde7"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 0, "pair": "a7326e6daa507b79", "unit": "b8d25f337fad527544c33c82f473015e08399457276fc480477f3cfcfc9c5b8a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 0, "pair": "211ba072bdfa67b7", "unit": "3217f9d672a4a1d78005c8fb66e1bc60a542cb26bd8c46903402c765c123d60a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 1, "pair": "211ba072bdfa67b7", "unit": "8370f9624baa0d144e56b8087eea897b98e5b2afcd6eb3a029d03e5b4b69d584"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 1, "pair": "baccf350d5cc9911", "unit": "50a3ca70700c8b41874c43ba94e6b7ffe0aa28b8930ff941a40767e1057065b5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 0, "pair": "baccf350d5cc9911", "unit": "62927a118d3662c81450e19d49f4146b66f29a58d55ec0d1606ed521b1883f12"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 1, "pair": "697f34e19bda57f3", "unit": "775b6d900ec1ba71e4082ed49bb94439f38066628eed91461eecf0658caee78a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 0, "pair": "697f34e19bda57f3", "unit": "9e855586c685b3c62ffe32407a82966661d5839e2cff742f2676acaba645a97a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 1, "pair": "143b900b99a46cf4", "unit": "200f96cd784ff5c39d8bba22883f22bc8a7c87c6970a7c871802f48a45548337"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 0, "pair": "143b900b99a46cf4", "unit": "7a0d62ec424849ec4681f4333fbb077e8239054d1e964bd99b40847ad7ccdb01"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 1, "pair": "4b4e81797ca48cbf", "unit": "a4d17d85973d024a615b89334ad5a2f46afea660ef5d0bcf2b2196c998b1caf2"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 0, "pair": "4b4e81797ca48cbf", "unit": "d604e460d8f5291124ba6fab5f9c94c7dfda61ea5f6808e512162334d6d6db4f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 1, "pair": "bcbd06e53147628b", "unit": "0b5cfe2d64771e30e50ff92cc409b8dd5e30079c252918ba03774cb5198a73b5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 0, "pair": "bcbd06e53147628b", "unit": "b805976c35330452af6c9f814ff4617b7b9ad0e82e1fa4440a443f93f0772c4a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 1, "pair": "c12dbc8a50f7fe9b", "unit": "04beb8a0fd054b92c2dd0ed83907330fdee6124d86114df948ac72a297a7d51f"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 0, "pair": "c12dbc8a50f7fe9b", "unit": "c5029016a6fd2e3f9df59dba0ce3c173bbbb0d2b1de22d151086e01e1b272119"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "0e213aeabacf4205db024ea3c557e720c3aba911424555784917e967d6404867"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "e8bc65415bd42e43", "unit": "4920230ef773595a9fd9a92a090dff42b48499a1dc5fc8831f941dae55cb2538"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "e8bc65415bd42e43", "unit": "49209a810f207eb61382e2ceeab7076936838a5652546f21c187d066e220160b"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "9e2964884cdf281fa3f18ee3481aa62f9a8a0a195d605da2df5bd71876fe311a"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "", "unit": "f9cc6f1bbcd59f5dc0de8f96248f722ea0679f8afa47c53f70832151ce7616d3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 0, "pair": "e69463f945f9a0f4", "unit": "5feace7b28e260175f80c682e9009025aea6fae3dc75409cabbb955b560d0372"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 1, "pair": "e69463f945f9a0f4", "unit": "799a4f5a5b9324e86c90752f0239f059984f44e8ff1ec0dd518d0d9413e34c66"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 0, "pair": "71f56810b8ee6947", "unit": "c659c0e717b7dc1084e1178288f6c9b2e87fbf0ef518ba3bbaedb0fbadae15c1"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 1, "pair": "71f56810b8ee6947", "unit": "c8d2e035a6099fb775c8b4ecf00196c1488dbddd9eed3d2d268a4f1d2567b194"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 0, "pair": "768fc9db8a75a351", "unit": "adc6527469618f72b1dd7776814717cbd0056a0c625e05bef74199171ce73f1a"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 1, "pair": "768fc9db8a75a351", "unit": "da47663a57e370a75cc65b810b8a380aa26d12dcabb1d9b3c02641398ff7d348"}
|
||||
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 0, "pair": "fab4ce020049724b", "unit": "b9be90242be627aa46462143ac85940a906232ebc0b47776b5729475b27d215e"}
|
||||
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 1, "pair": "fab4ce020049724b", "unit": "ece12ea7d87b4a12c4da30faed9eb530b51fc879c271b68ba5304b230cda8e79"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 1, "pair": "9126ba83905eb8b4", "unit": "d8dc50b7bb40bbcfc3f3d4f1f8d3a4d17eb1c662d5f759efd18656a482dd2e32"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 0, "pair": "9126ba83905eb8b4", "unit": "f426f9307621b3f2d6a8d1669cdb4c19e962b94e34fbb394d4660cbde5921c88"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 0, "pair": "7ac5e21d9d8f4c47", "unit": "1d1efa2fd7323c264290d4d314c957c853e1c7a9c877a54d6f03ab7d2af2ca86"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 1, "pair": "7ac5e21d9d8f4c47", "unit": "b5dc78c7ac0b76803dc98904142c1139083fb9daafb5bff40e7b5c3f14ee03f5"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "0894a7212280bab755cd9d93b1f0ede2c51e4fca76b76e5ed9e24ef8f8cf0b9c"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "02694b5fb85e9e52bf5eaaab4e0b628f1f44ff3a8d5ad026cc10afdbc4222eaf"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "db9a3e888780501dd3c2c125af2f9d47f68ebf02f0e0db8370b2ea09c6f9d070"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9af96b099dedcc4005e5fcd19ad78f73864e1cfbafbe6953b4edb6260957943d"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9e7c2a786068a936a6c664cc229d6f39cea1fe57caf0806306f0cfc44515460c"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 1, "pair": "07d1dc4d299a7700", "unit": "0e664e274dfb438d2f721edc81f8469ac48387dbd5c1b43973cd7e651db46db3"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 0, "pair": "07d1dc4d299a7700", "unit": "da55f60378c67e91683b9720f2fa0e6659e28c7b392c050278b40713e47c84a3"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 1, "pair": "749eee05a507fe93", "unit": "24fbb1dffcca083382a23e48db232e0d83bbbf2ff2b325f0dc0f4c72dd1a16bc"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 0, "pair": "749eee05a507fe93", "unit": "640e1626f8d7d9d5c2478ede48cc7b2780370bda98ae114bae102bba892da90c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 0, "pair": "e36bbaf8f3cfe65c", "unit": "14f7c65fb5621ac5e2068c8695ddbf482103e4320ec3b36886f6e1f08cbc312d"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 1, "pair": "e36bbaf8f3cfe65c", "unit": "c96dd379dec2ecfd99051d7a20e6ef5601821f93c46eb34ed087daf5fbed2d47"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 0, "pair": "b81c8c63cae892f6", "unit": "5982fe7fa1ea6bfc1c8dcb8c71cd4410de475d25cafa324147649f26a1ae1324"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 1, "pair": "b81c8c63cae892f6", "unit": "59916deeb8ba14922eebac61c9eac122e15d1f7590ee3cb1b97f019585fb798f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 1, "pair": "bab3a167ab5ca505", "unit": "5ebea23aee96c80fa0d2d9239964d99614613c4ae36f709811bc19e61ebc720c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 0, "pair": "bab3a167ab5ca505", "unit": "9bfea8d7a6ffa6d79d16f77fc355fc1bb605189513ebad561ee523d577a1498b"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 1, "pair": "5a91049be21aa611", "unit": "1ab9936627aa59c3a60ea44074ad44b68aea4f9c04f14c1949dd9c77ca13ff0a"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 0, "pair": "5a91049be21aa611", "unit": "8df72b9eab3f2fd858c40312d0a806cb67c620bd62873826c8732ba8795d9cfa"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "", "unit": "115d87f37bf1d4ab91fa3010303e81b127f84b817268b60b8fd4b9934efeae4f"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "2b8101c89c839614", "unit": "7bb7d858a670fc51f81e08c141c2bca3b8b7524254ab679e33b22ce79536e895"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "c23acf4f38945093", "unit": "82bdc0b6bb0c19e342efa63af0c9014a607461d0682cdd14bd5fea33516fbd50"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "2b8101c89c839614", "unit": "886285661b88f14e00ff8e863b31cf7b7afb303ad4aa4aa13b1cffbf1272d5e0"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "c23acf4f38945093", "unit": "bf74aae5a48156f183426a80cafca6da219ab13b2ae4b40fa88fca75acb72980"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 0, "pair": "d7cc37a3a54315cb", "unit": "4aecd43c6e5b81daf7aadd6a6e22f22c17c08b652a2f5121a49320ca1057b186"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 1, "pair": "d7cc37a3a54315cb", "unit": "88123ae860be2fdc28fd6a289cf26c3e0af0689f455cf8c15098e5dc76e5c8a8"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 1, "pair": "5011004f06f1e217", "unit": "6c48682d1b02ff2559d92346d192cc9dbe6a0bbe2ab2c146653110420105ff1d"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 0, "pair": "5011004f06f1e217", "unit": "d3592260108a838246e0cfd5ab6d48bceb0ef870e223565745dcfd68f1826840"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 0, "pair": "81caad96b221c2b8", "unit": "3c2dc720ed159e55629ba737665d93859def9790427289567a513c83c133e236"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 1, "pair": "81caad96b221c2b8", "unit": "e8b9595a320776c8e663fd0b168193c1e64e36a9052784d1219f2c4a91d647f2"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 0, "pair": "f9b6599018ef3b60", "unit": "7558d6ee39a311c5459e35fe6f7b12917b7f843b21ab1fbb5e485a07f53b7d78"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 1, "pair": "f9b6599018ef3b60", "unit": "b1a6cf450a074788ffca2bebd880435abac2c376e89cb328d7a22d277fdccad8"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 1, "pair": "eaf2f03db2c502b5", "unit": "38dbfbf745d7cbcc52c6fff83248d84663273fe6a1f1b014eac7776c4bb8f074"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 0, "pair": "eaf2f03db2c502b5", "unit": "a84d848ef56b72fc15872d88acaae1743c9cdf88095d13b1869abbec6a0d8a8f"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 0, "pair": "7df48bca2c9fabfc", "unit": "7b76ce8f55d61071adb367018cc3e50f04a05d08c7c4919c3806fc1cc8bf41a7"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 1, "pair": "7df48bca2c9fabfc", "unit": "c82ea0adc9c8366aa2b77ce73772038676fc37ff50879e873ff397958fdd7ec1"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "7f02a09b8d8f2321f2cdade74c2fa1d52442e006149a749a8be768daaa62195d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "9b5e439943ce5c2b04d6201b7a843634aa1b13b726813c23450ff54ce03f3dd6"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "5b836889b4da4921", "unit": "a84c2ce4a4eadf265d4481ad7f6864edf1d7e3084c6ee32cbbfe454bb2946bfe"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 0, "pair": "5b836889b4da4921", "unit": "caeed7f097a3311fe2e84efd372fe4b4c140f63a80a9f07ba0b61966dbab15e0"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 0, "pair": "ac526f457067fad8", "unit": "50d8bafd5189fab0429bd4135d9ed400cc2fc14706226a2de98d65746ebfdcc7"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 1, "pair": "ac526f457067fad8", "unit": "9532b70b0df76a34170b57c74deafaba54d5d2f16a729c39b0dfd3eb2022d899"}
|
||||
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 1, "pair": "c97a971a18f85244", "unit": "95c80c503ed43fe9a0370f89ea935d47badee6a9625548b9dca78e04275003dd"}
|
||||
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 0, "pair": "c97a971a18f85244", "unit": "e9810872beb9c158f54048392142c2f701314f854d6dfc748553ca36ed6c08d0"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "c2a2c2d85114a97f892fb1e5500fbd98622457a8c46451e1f9e958ab08a40318"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "29d8be8206c88ba7efd86b56a7dcf8b655e33b4a792c2d18b028c92c6252daab"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 0, "pair": "6a4e0bd705331747", "unit": "6db80439d89b54f60f836c415dd403fd09d21cf630da81435d4d2f9cef5d883c"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 1, "pair": "6a4e0bd705331747", "unit": "bf257e4f0812012be330fe119d07d29253d291161f435262787f491762c6bfb1"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 0, "pair": "f96f0fb245948915", "unit": "0954e6931373915c204b9ec8d2b8ae6297a95667bfd41aadf9035625ad93967f"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 1, "pair": "f96f0fb245948915", "unit": "5e09cdae2504e834d591b6f92a808427f07550364b5419899f7e6f76084fc377"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 0, "pair": "bf11f138fe39675e", "unit": "56f0d0b156992af56141d6368fdb0c5376fc7a44a03991d754f97202b9d77008"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 1, "pair": "bf11f138fe39675e", "unit": "8271af5017894167556893da2070188060d06eebc87f27db45c46234c359631d"}
|
||||
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 0, "pair": "9d937c2dffb4aadc", "unit": "01ac12bb06e660d0aeef8b1e26dddc5c99d52cc333f7af6d1981eee2818e64fd"}
|
||||
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 1, "pair": "9d937c2dffb4aadc", "unit": "9612edf0f927048335e83e9925f01be9aea9b772bdbdc568ad9cc0a17055662a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 1, "pair": "d4f648048e45f2ea", "unit": "08807c8110b8c1b8635a6883c229fb91f2b9566db8b6c252448a0db7b2ddd47e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 0, "pair": "d4f648048e45f2ea", "unit": "10869c931053bf77ecaa98a47f56dea27d643d1abed2a4758bcf48a78547ccc9"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "662e1f36c230d7f1", "unit": "0b73332b25413902c26d5b5dee6767465eed99ff71a814582988ffe0d8ec1a30"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "8c8e46d0c7d361b784baff9d70f38744b2805ef1ee0aaacaa616bc20a2c6f32d"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "aca938682c196fd4c85b39b89eeedb97c35f6a5a017651fe9c03c5b727cab7ea"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "cec00d1509a129364da6ab8537fb3f433fd847e26d073c90311ccdaf7a1b39e7"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 1, "pair": "662e1f36c230d7f1", "unit": "e0f58a7ac5627de3e7b6cfe5add41d18b0f64b998c1cc788d42eb2cdbbc3fe41"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 0, "pair": "2c251a645094f389", "unit": "0d41cc634771048f72d07c4aa9feaed669d0bfe3844d61dcdc525a0f899617e2"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 1, "pair": "2c251a645094f389", "unit": "ca8288d0b8b94dc6b7e4c681936ea610d295fe949cba5f64f09b6de134ef9875"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 1, "pair": "2ac5ec5011615c9b", "unit": "2f427f403af3cb58d776437736c2389805543413fdb91aad98979158c6d00de3"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 0, "pair": "2ac5ec5011615c9b", "unit": "cddc8c5bed0841b80e7b20d0b378e858f8e4ac37fd7b890ce76611d989857f92"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "frangoteam/fuxa", "label": 1, "pair": "", "unit": "0c7830c2912ff285855d8a3c8e9b47db4ca4af840baa66d4b5d3f4422f46f95d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "5383a8c2a78fabd3d17a5b3021125953b5ebbee61f8900c30d016c56b1a3bad6"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "360d8bfd2f116e98", "unit": "7c1989a4ec3622bdaebcf1161de850f22cfed041cb38830ca1ded4fa99d40466"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "ec8c706cc2b20f36", "unit": "848688dcd0fe8d46e6762cc8aee1d82e8fad8a2a044c94b547aea11049c85bfc"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "ec8c706cc2b20f36", "unit": "983cc9231f91baee4e2d626193f3b88aa5b09907f6faab870949bdc9c336b816"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "e4ad86416d5c6668", "unit": "98ff1dff2afe80709b8e61a9c95cfeea12b2110c13b7120478268b4762e2c992"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "360d8bfd2f116e98", "unit": "a1a825d23e34757c8e961e43ae791253e62682e130acdb846b58118588006a98"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "a7fa4207508b3989baabc7a1f0ff2ea5229b366c1cb106e7fc945134b8b106c8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "e4ad86416d5c6668", "unit": "ef5c53f3114d0057e694df13dc07cbcb532c3b0d59150eda269b87b021d7a6e8"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 1, "pair": "9ae575048504c690", "unit": "176acf077fd58c3e6f940753179ffa3e8316661f355d94acb5e0345b51925ca4"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 0, "pair": "9ae575048504c690", "unit": "228b34ebc2090b5239c7821d28366ddc663e7118d015a79b45dca6618d5490fb"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 0, "pair": "d6edb5496b3fdba2", "unit": "9acb8017ebadc480515a664181c427f37359edcc1dc6f344f368a87e0c191b3f"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 1, "pair": "d6edb5496b3fdba2", "unit": "c1bfe63148bc700c812fde95ad9f83d456e4fa04e10c6f5394072a08f73174c6"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 0, "pair": "27e1c8e8ee93a892", "unit": "47fb89b132d6de48674c37965579e727d922bef1796db70f8b9f862ac47cb9a5"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 1, "pair": "27e1c8e8ee93a892", "unit": "79887f0ca8b05d6599d55167d8a9faf67bdb2254d17c143141da44971077002d"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 0, "pair": "a3fdf9ce3441ec86", "unit": "653236018929ab8534c27704a20ec5d358047bc66e451620d2ac32081c91d333"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 1, "pair": "a3fdf9ce3441ec86", "unit": "b70ca2c43f5929e7b885bd728349e721fd85d7605e2f722a7e8cfe0f0d477b0c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "groundhoggwp/groundhogg", "label": 1, "pair": "", "unit": "92400e83890882a66ffdea1a405e2354a195442e176d85fb707b0442c44d45ca"}
|
||||
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 0, "pair": "f9dcf3a0fb0f0a14", "unit": "7ca6dd4f673976b000a657f885af3463740419ce41e14f972af0c5483cb75a7d"}
|
||||
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 1, "pair": "f9dcf3a0fb0f0a14", "unit": "86c1db3a5e9f843d6546146449171e0378ac61d6c8f44c6dd72fd379cdbb188a"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 1, "pair": "3e71f79348a6b770", "unit": "200250ebceadde4cbdfec2802a4f9fbfa220dbd7fa4c2dd5ed717fb1f3ea7447"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 0, "pair": "3e71f79348a6b770", "unit": "b9faa5007c8cbb67ada3f1f026ca59e0ace6fbb9c3d1f1369d4521a7e323e7da"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 0, "pair": "8972a0f56ee57477", "unit": "6b987df2960828dcb1b2b3f190a903a40361d58117d63321d671aecc6c2cb570"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 1, "pair": "8972a0f56ee57477", "unit": "ca3fa87dd37435c01a7121119912c1a1e261259c11805c0b66e7d4b315640945"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 0, "pair": "6144baeb491f4a56", "unit": "f5c22ce104e9593e5272b6e8b3c6f3b61fa4578552ccea80e6021ee90bdb3ffc"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 1, "pair": "6144baeb491f4a56", "unit": "f687e7e95dbe47f602a1e0002a44f94fd48e5b1c33f77ecebc6c2d814fcbf450"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 0, "pair": "2ef4471c66e5e4ab", "unit": "ad7c8e368fe1699a044e058d5c10b41bd9fc0f91c3f3c384c5dac40af39e71f6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 1, "pair": "2ef4471c66e5e4ab", "unit": "d888baf7194bbe7158cb3932f58b47a9a235351d388d10df82a130ddd54c2d46"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 1, "pair": "5628c54152216646", "unit": "47d081c9f6060218967902f694e61c88f64f57c65bec6fc3073afeb0b77d117c"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 0, "pair": "5628c54152216646", "unit": "e1297bccf9c3606e0b41d762a69ff5d0a33ae4f7f1506071af6b9cb4a177ff56"}
|
||||
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 1, "pair": "e52aa814c36ace1f", "unit": "2d6d74364f1982e91f27f1c995b8699df1fc8f8dce6e1e2cf274bf01b6e9494e"}
|
||||
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 0, "pair": "e52aa814c36ace1f", "unit": "4727471fea2cf0a92d43b4b1503f1339a7dde87f872ba8064dacbf855c57705b"}
|
||||
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 1, "pair": "a07929b3fc1cfc28", "unit": "0637029194bd7378237b4dc944dc5cf9f2b63db9796f27410a74a9d567b66dc4"}
|
||||
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 0, "pair": "a07929b3fc1cfc28", "unit": "ffd286c8588a77929639a89e4a742e15306b642af14aa906c721165df804ccd3"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "6548e1527e82bd99", "unit": "5542613f6d2eb9421b8213b8bb864894f0ecf9fe64945681019230ef0d75cfaa"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "", "unit": "6c2adee34e9e8384535ec1073eda60e2ab528ac77c36747802c303e1f5e9ffe8"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "7ce70e9b20107cc7ebb87d173da765fc53ffd268e521749a06a9a1f38d746845"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "6548e1527e82bd99", "unit": "c3e54975464323e58ec96ee4d83e7bc08baf94553d553a8dc3fb28cd9c70ff21"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "d3173c1e43eb6402a71ea2dec1592913dc078a2194ba1c1d509ab129a3180f6f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 1, "pair": "0924befc49c7bb18", "unit": "e7e4872db40f0420547ed622fbdd7a072b64a7ba466d1ccfff68e691cee6ad99"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 0, "pair": "0924befc49c7bb18", "unit": "f7d95f43fb2df20ebc4c37fceabdd00ffd8159f404ca384481f1764252382388"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 0, "pair": "d4ee6cfeba71e398", "unit": "e79582de3031bada2617a0667ef7ff5cd7183d4aa885f55a665cf0ec88d17624"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 1, "pair": "d4ee6cfeba71e398", "unit": "fef35272853e7f3692efd54daa01c12862953352fa6ccbbb5e3f7fea70c63d78"}
|
||||
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 1, "pair": "772e18821701d60f", "unit": "89a0496c7663bbfd67d660c9f481c12420da2dd1bdb29dc079735ee7a0a49626"}
|
||||
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 0, "pair": "772e18821701d60f", "unit": "d3241ceaa50a7d23a4650e6a2a55372b08ec96b35b0bede844bc40e16e818ebc"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 1, "pair": "250de1302c7e541d", "unit": "5d03535f2df2acdbc319d9e6b20da65d51f3e810b80f3c8f89b5074a971a3f1c"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 0, "pair": "250de1302c7e541d", "unit": "78f977cfc096e34b4eada7f798ad2973ef45728d0d01bfab6ac9e2a3257827ef"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 1, "pair": "c10bfb006f2da55a", "unit": "1ea4b86dbd84581b1d7b7cabbab9f28c8b213416a60778879666ff42d23fd7f1"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 0, "pair": "c10bfb006f2da55a", "unit": "7647021f742da24c660b2d407886771786e957076c4cf336def560a570dd8778"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 1, "pair": "a73421efb106ab0a", "unit": "1f6f207a758582de0c0e6b331b19fe119abe18727d45ccbbe09aabb795f3a292"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 0, "pair": "a73421efb106ab0a", "unit": "32fbdad39eaf5177db4ad0a11d53e785b30bc7b2376ad9d460f29a08429bd0c7"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 0, "pair": "0458e3fd444409f9", "unit": "b521216bb4106738b3571ad3902d08f6230aabd88cf3a8f9f26f1c8ec01304cb"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 1, "pair": "0458e3fd444409f9", "unit": "c87d2871907798fa2e73d31381e9d63089b0388033f096d247373fec07c2cb36"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 0, "pair": "6bdccabcf5b9fdc9", "unit": "3d6ddca5eb264d88411b86755909f1d2abddd09c129cabd54c96ff26b4105316"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 1, "pair": "6bdccabcf5b9fdc9", "unit": "e055b898802279bc63204cc601e29f7f1018efad7228cab0cce00ea5866ecc7b"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 1, "pair": "91bc832e70680c92", "unit": "1060c63ff0fae3af567448f3d4c60d9c159b61ccc358964243eb152a9ac38865"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 0, "pair": "91bc832e70680c92", "unit": "369dffd1e730d7997813adfbebf1481cc68987cae0d0085e47b91a26d5330cf8"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 0, "pair": "137fb3496a3a75f4", "unit": "7e8b6c484999229b23d50d9f5a4c4c5b2a8d954a5d64230981b1c6705b760fbf"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 1, "pair": "137fb3496a3a75f4", "unit": "c461154e4dbd74e35a4ed2e85de7b076f0582beeec459be28104c38f616c7565"}
|
||||
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 1, "pair": "a4144258dffb3b8d", "unit": "7ab301e1614979f44c353e665540582493f78ef4bfc41209562c19b900710f8c"}
|
||||
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 0, "pair": "a4144258dffb3b8d", "unit": "ea95e8387dff32ded9dc7b04c11a76a545b4e2ea6db0076b9b13971a59904bb3"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 1, "pair": "858330e01d3cf9bb", "unit": "033cda2b7cffa5a5fa6abf0b7f2cd27ba78d916eb3cab8c79decccd8776f4a3f"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 0, "pair": "858330e01d3cf9bb", "unit": "d44c4555a5ae19aa17b468ec5cff56b33a0b221df52e04d8abcb656399499dee"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 1, "pair": "c6a49339d4f0c798", "unit": "dc2e4cd5bd963643702d2b832ae170dd25744162e4076e36687256a09e69d089"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 0, "pair": "c6a49339d4f0c798", "unit": "f83cd137b85e651ca5299c5a42176b797ed4964cbef31418c96d63c52fdb65f5"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 1, "pair": "4b6526d132e9ac86", "unit": "8ec3596444f3045684242840e9a1df19879383f2733aca8f8af6f6cf8a96d388"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 0, "pair": "4b6526d132e9ac86", "unit": "f3d39162fa14446b207aabe7bf3c913dc32a9637ac83352ea5cb2fdbb43720a8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 0, "pair": "80abf1d5c4a3f6a5", "unit": "3df3c6d9941352dd2861427da56b94ee335f308d0a2a27c6f745aef60349c80f"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 1, "pair": "80abf1d5c4a3f6a5", "unit": "4fde61a570906e658d9b14db0e5d1f5acc0c7489951421bff2afde25033ff73c"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 1, "pair": "5ec37c0f475001ca", "unit": "5d8175fb21a4649bdc8730358e9b87adb90b2aafc4da5fee203829663e4f46a0"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 0, "pair": "5ec37c0f475001ca", "unit": "af845e9e332555096dd350401977b2f07015433f3db8a040891d9470f3f9e355"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 1, "pair": "68d44ef338ef3470", "unit": "3020ac6c3186d7974953b1c4da65b80141b81aca5203436ae7a2d7aca0870944"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 0, "pair": "68d44ef338ef3470", "unit": "8fdb85a6bfec89f0715ea81e83fd91616f92c59954c092e79272154f202b7488"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 1, "pair": "c74525e844650de2", "unit": "2e444ad34ee5f47d73f0cfad0a10c70363337b4ddf9067ecd1b8072d8232652c"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 0, "pair": "c74525e844650de2", "unit": "c492d325f5ff82b5f817aa4be1440cec8595054dd0719e07a58ac5501b2ae7eb"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 1, "pair": "111a5a80a3abc2c9", "unit": "c7ae403742450deb2bd051ad1231121d2117ba56fa72a038c82d58d207cb3870"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 0, "pair": "111a5a80a3abc2c9", "unit": "debe34831afca6f8220b9952036a2d4570babdb5e321a2f8dbaec661a016bd50"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "9950b85412b021ebabb11eea4c490a660ad380d14c673a87c3fb568ba4bcfac9"}
|
||||
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "656df5ff8e69dcdb1f9ba1ed52d734966eb29773bc745587483a355743b8eb5a"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "91652a4e776fbce70cde9e4a99c6da5e17da7f8d6d0404867c61eec04161b6cf"}
|
||||
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e53bb3a217dc4737e8dd03671de4f6a895599024070a985a268c65a034295e7c"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "3d7aba22444811a337abcad0e4fc62990bb704d3a10e72d893342fa94699026b"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "5f18a3026a51f7140b7aaa3aa8563e8b6fdda6065d95e19e25b1dac4b9ce0e00"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "0fccc306ce80cb4cbfdcde0a2e56d637bcefa9dd30423d6c54a4f77860afe740"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "1e2eaa45661345f1b6759d968d7696dd0d96d6fba1350a00e4d0a2ebaaa2e391"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "8b9a8ff0e32a327df75eb8a1811464d5f34690079d5930139dd917063bdd2809"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "245d0e939b7c1ccf098c798492009e3a73ea17400fa613dd865cd369d2e7f383"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "653c48df493f31c2d01c973d1e379a180fbe6a6edc83a500bc62509112175f41"}
|
||||
{"family": "path", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "3719fe21d45b50b35458bcd2e263e4baee67adbd3c98adf0eaab6bc6183c5651"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "1209a6cc8c1548ed83d4a9a09d6a27f706a23e69cf0c261fc5e3a27536a83306"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "362c90495dc468bb732c71f62d71ec308854161de31ad18d2a870056d947ddb6"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "b52e53fec9707b2b442c4dc7a41d376cf76946d2c45c2eb600c9477fc6baebaa"}
|
||||
{"family": "path", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "477e3b90b92e622888dbe0e9d3a23b31271ddfa3104a5ab326eb9e9e765aa01d"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "660c63c4a73bfe64088ae3249e40d9c5d69a6f8a505ff671217c5708a345c5b0"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "4d60c386c801bedb61c1461fced29a3cfb50b09d6435e8617c60398343719d93"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "f1fddcc1354a29bef55610ee2b73eaa440192f2f18a0d22d08057b9d1e1b61b8"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "6d9886c38221b2f9aacf83749218ac5ebc354a6fcc4170e6c78bedb92b88296a"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "5550fec4a81c4c9cb22d7312d8a63e1fb2b03c4aed0c1f316d44c1eb27511bac"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8ff87dd04140f0137ea1891f220b1276156ce10a58be5a49d0c8e07be531ffb2"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "f24b19cc0f67247eccb274521a62741fef8f4729f0ddc9b5c1fa561a4947182a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "83523c9d83380d4cdae593a593d361493e00d18e61740f84e762aad41b991e18"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "c6c9bde6630678377d250701073c5c223c7892e6ff8f2f8e0293dea22c5b5e7e"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "b9b15958973c83f8d7b58372ab5ba58eb191c400125579d5ae8c5b52d5d1368e"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "b95ec12e4e126ad558e944e2c260116ed6a63f5f3ab97fb5cbcdca75a931ad73"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "38a5545ac9ef67e15db720d1396a2d5915f9d06d6f4772012e5bc95f33770376"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "c8bbe9be91c976be97b7ca387048a7d4537d9f6fcc6cbad0a541d1080629d5a6"}
|
||||
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "ffacd53f1bd75155e63b318fb9cba27ec03e754841e1eb39d1d4fa0b2c3dfca2"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8fd920175fc43b91b4397acffacb10ab8f7fd0ee38bcf2d2f3ed06a3fad0dfe9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "86c9598ebe3dcda261a69ae545513f6ce73101f3980066e4506ece99bebea84f"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e5190fcc57505116f79a31c51aed69f8a1eea13a8ae3e7265f44c72418277ef9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "2b807ff54e6c47e941124c16d0c27c691cef54f2203060a3a6ce812d478c5261"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "401aefb4b4791301ace2d770d321c6b26743b0aab7b96f8b410254c704be6d2e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "22d186c93498e21418b6c1d0547a73374b13041d8afc89a80d9540fa658b6b42"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "dbfb6df14fe37f8c0efda5f16643297fb0550dfc5ad5805d4e4b844d967773b2"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "f0da8cfe4ac1e586aa7903d72e67b44500bbd5306d72b4b342d10613efc69222"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "2b8fd73943f456da266434470333a7775f55d9a41953e97a000fdaa3257b6a30"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "a612091f452e60f28f44eefec641175170dc5db04456a39a099cbb13c50b4d2c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "4c6600facd83bbf234a32d9e28ac494fb516df6e3d9e9dc28d720ff35252d067"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "fb0ea8cab8c9430d22d1b0fdd65a69146fa95fbff94e720b4861b96538d992c6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "e81f91a56058ac1473579767abea57064c53d7d007797a854314814e3c35ee74"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "5851ecd96a449f98b05f243bfda0a7cf3af8612b27f53c414ae1f4ea67242a7f"}
|
||||
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "328621ef2a94632e5d37ea6267d17b766233384cde35c07e29a4eb672d40ba28"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "04b2eae2128608dad460ba8275142a41a44e32c4243a29e2a6fb2c9ebf3d8f1a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "33337002ff5f5a189610b042522ceea254c9d107c1f24771b4f616a0f96f636e"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "02c09a783cbf058856fc2b448406f3ffa6fbfbacf9b70f4e5e22fd741a5c7699"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "956260164cb21a47c499e5598491031ef303418943f5096f225a87162b588574"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "8bd8c5aed69eae3019cc2d3654b36bdf2b31435e6a020a389cd103696a2f874c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "1d8397ef4a9fde7b48e2e1e871bba0f960a802cd7823ecfc10e9dfaddb955ccc"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "c9db92476032eb4296b950a1eff8fa4aaa26f11542a01a5cf10c7b2e32dc6433"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "d3e0d0f9b0f4b723080989b5486395cda0a5b186a674022ea70e7fa7ab561323"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "44ff3b3e61c02744830c4abea872821ca7e0f94950c082740c5c48ef13604e94"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "4103e979a99233c3e71fdfe9f1cdcbab34a2c60b5351b75c5ff5dfc287b962d0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 0, "pair": "f73ecbfda2bcadef", "unit": "53e596dabf90b2992d5b3e50394791859b393beedc8f84e00484e37498627ff0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 1, "pair": "f73ecbfda2bcadef", "unit": "6daaccf64706e876dd505c65a62aeacc1357f3470a5cef7b1ac874032a6fcf99"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 1, "pair": "6768bfae38c6859d", "unit": "f376519889a00950f5af67eb15c639306ea70da7235d3be4a94dbc19cb096cf5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 0, "pair": "6768bfae38c6859d", "unit": "f3fea4e334f49ef6fe6b17121fef553eb8e33a9ffa13e6d6cbc4c448da996a7c"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 1, "pair": "d670253fa209671c", "unit": "1aeebe2e440bdd70525c31f93180f33be93f820db5ba46a2e59c371ef83ee794"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 0, "pair": "d670253fa209671c", "unit": "b11c8a8a24d3809c0507228568fe7cc05ca0a22ed28cb19019adb13cb619d63f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 0, "pair": "33260f3e6c8ea6d0", "unit": "7ca409c7dbc1c38ff005624e212efc6d424cb421e27b482dc8dc0d6881f4c11a"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 1, "pair": "33260f3e6c8ea6d0", "unit": "fc364d7ed77c7c0508da00e5ba847ec0792f3bc373d8b4297747189ef42d1ae0"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 0, "pair": "32ef8148a2c08f58", "unit": "2fb2c9314a483787168a450f50430721f9527f913aa0eae07298d5334cf4f137"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 1, "pair": "32ef8148a2c08f58", "unit": "78dd291bb2ef83ea3a87948c0ac0027c6d667b9044093e766e2bc91bebe83936"}
|
||||
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 0, "pair": "54586b5a24655adf", "unit": "2ece75a0240ed36a811c381841aa6d440af2b9774ddacb1d7a31a67ec267b0b2"}
|
||||
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 1, "pair": "54586b5a24655adf", "unit": "e60f6b788a196db9bb671d621189b79a5526302a6d09c2396d67fbe0b9ab3c24"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 0, "pair": "474f96e2a9b3d202", "unit": "4a5e8e8ebd7ecb0e252280af0cb38a1d7794757728b912525322ac26df5ee223"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 1, "pair": "474f96e2a9b3d202", "unit": "6a888540443061d3b37c06f0cebc70cffdfa70e0aa940626e86cbff9829310fe"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 1, "pair": "a09692dc94164db8", "unit": "139cf84e79c4efd026a064b2ca0a3d7ffff87b54eae1275a07c0d69ef4981e27"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 0, "pair": "a09692dc94164db8", "unit": "828e88b656245a1d0865c79b2ed023a49e78607a4ac81d70ccd3690f4419d2d1"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "", "unit": "6cce78d7569ad8db7ca0a61d7078198f666502d26ea6c0afb03173a8a4bb16ef"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "82047c0513e43a83", "unit": "6d2dbed1ab1c9d8ac259dd4227be69641fa59d52e816a8dc547ade01d0b9b0c4"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "98ee23cd6bd31cd3", "unit": "ab4bb39be535d70cd95e9035bb958bfc7418576a6ff40a47551894dd140317bb"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "98ee23cd6bd31cd3", "unit": "b70068cbc646bd01d3e10986e002f648a74e9d798900dc59cf3039b749c95c18"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "ff168d95d8d9496e", "unit": "c6ec9478c125ed780d7013c2807f2cec1878f44253667dfed319c28cbfd17d4d"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "b1bfef46dcd3b9a5", "unit": "cd4d095d8380a2fb7ca02fc6abdbae5249cc06455db2929875533f4aa8b886dc"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "82047c0513e43a83", "unit": "d1bade217be8d345cf5d0f0442ee31924187acbb98779dada95274a77462f696"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "ff168d95d8d9496e", "unit": "d561db250b892b91feed7ae877900e1e25963444f73c7699576e010f06da73cf"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "b1bfef46dcd3b9a5", "unit": "ed78fd5a7ab30bcd6e8ddd27c706a19cfd89750de4fbf81c0b567946eb94db83"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 0, "pair": "25f75d77a669b54e", "unit": "0f2825568461562f2a915a4f16ef8a4931290fd20a9801470aeb12677e66e3f1"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 1, "pair": "25f75d77a669b54e", "unit": "bf22bb44bcff4a5baf5d550c9d3e83e5f3ca607475fb51deaf4fb6fccff3d73a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 0, "pair": "47069df6bfb3dc00", "unit": "444c1f41fc23ad656a8bf98dae58b36d0f1daa214ab030682f26111dd389e58d"}
|
||||
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 1, "pair": "47069df6bfb3dc00", "unit": "c0c8b5cca070673ea4b4bb8f08dfff86b7781780e300720c4051aded035052fe"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "cd80750447cf7af67ad809279718d98106c71705fef9ff7254728bfff4ece5b1"}
|
||||
{"family": "path", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "2ce605ab5ce7ad3a1fb983cd6bddf3924e5da3010d2252a0a4640aab4e5921a0"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 1, "pair": "b0a899116dc36b85", "unit": "510ad48af03ba4a1a2a06dd254b135311dddb62d9fe5a2e8e8b12627586e6b13"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 0, "pair": "b0a899116dc36b85", "unit": "b414718a8a5f79739b88db24da6ad9da8e48bcb6b5e2dc99158848cdaa226767"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "8e22ee30441b20484090659b0c97423c711f5d10219041bf3a399623fc056a07"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "0f9070df9dc334f981cb01008c260b8bff78dbdeb48ce672830ed95caa8406fb"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "5e352f797b1422b0df477bce4730a9ce33cbd0837c3d63da94e4d8801a9a4b64"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 1, "pair": "dadccb128ac0e120", "unit": "505097494da5939404ad7d5b4af2b1c4a10efe94048084492d6ce60ebaa04c97"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "dadccb128ac0e120", "unit": "84ff8ca2d98a1b5d1ca4bfaabe7d19e03a396042b228c8d18d1185a178389702"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "91e261c54bd2c3c3a9ac04e48894ecabaca5b13e7db4fbd5578f9d266098117a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "a308dfd59bc398824e2454bf6cf6cfd5b7880e5b948a82424f14753d2b6fcdbe"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "ba24eb554458bc97314ed561a7b2f4a4e643dde1205c32de4c8fcdd8c46e575b"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 0, "pair": "24d8f1840b67fbe5", "unit": "7ffa4a229a62266788fbf64890f81f20358b3eb8c35d3ab87d0c339d5bb4b507"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 1, "pair": "24d8f1840b67fbe5", "unit": "d19493ed7941e1cd25a76dc50504d97398770d07ebc302dd66d83441e946c468"}
|
||||
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 0, "pair": "9f8a60eb48d74bc8", "unit": "80fce34f2377731e3de4355c7ea33bb20fc2fc00a5f388565b23018c7a3ed008"}
|
||||
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 1, "pair": "9f8a60eb48d74bc8", "unit": "b68e62722a327c9d5ce6774eb96527451729410768efc0b5c7ba604ec468052d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 0, "pair": "2c857235d0764986", "unit": "33bdfe911cd1b512e5994ee7a3e3ab2ad1f302cab29df3b011c0d825c23536a3"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 1, "pair": "2c857235d0764986", "unit": "b5efdcb84a057f3cd9fd3a30a32239dc06c13536a4e84a66c7a17cb372050c97"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 1, "pair": "9e84429d629c0b6b", "unit": "774cfdecac67499bb35a872931d4698d1b92ebeeb42d21790469e418889954e5"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 0, "pair": "9e84429d629c0b6b", "unit": "a6c5c15ca1815f83dfb47428c4a8afd4e60fa1c027b9f33c668cb9a0d77c18af"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 0, "pair": "8e6b4b4e28981590", "unit": "3cce3e8f9ff3f1d261f9b96c844c1e4da07bae72d5393897b3a929be25372edc"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 1, "pair": "8e6b4b4e28981590", "unit": "b105689ac24ed334294c434a77bb6d56d5d598830ffe8a5d60ab3be5290b47ad"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "0a36cd5177f3b2c20b433db6bfffe5ec5b7e74cacc34a86ac581f7a0a559432a"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "e7fc97d83013c9ff", "unit": "5ab444e64297fc19065478104c7ebdabe23cc415f22adafd0cf82c98e4c5cae3"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "e7fc97d83013c9ff", "unit": "6b462ea30c1afda572600a10324249223932d36a032eacafac8c837bccbdce87"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "84d6a4202a082e0d", "unit": "8175082f40d8d61d44b54579c84a808e25b89110be5d8fd82feedf7dd5e0d5ab"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "8abefca43ee4ae2c797033da31f85c17bfa7be6b35413d70b0216415c5db9a09"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "91d213946859f33d3f243c43aa8285cc6e8ecb3db1de938f32af775a70607f8e"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "a244d9f010ddb43a09e82363dfabb0e406d411fb5de08ab7d6c9ee367f2a15c3"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "d27d174d6e8e5f1682a1c42723705e89f0d18c95ecedd0567e313daa9a0aae44"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "84d6a4202a082e0d", "unit": "f45608ed47fa11ca25093641f6021ec06076c18676abac9b809f31b3b7713de8"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 0, "pair": "55d1ce4f29dd1583", "unit": "a877ce3462da316bcdd82ce9db1d9887e887eaa83157922edb1663413edae1d6"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 1, "pair": "55d1ce4f29dd1583", "unit": "aee67be0ba31f99c7a24faa1bad6178c6d37361a308cf5aa10138e6ce4bbcfeb"}
|
||||
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 1, "pair": "16e529fe8c5486b2", "unit": "1d21acd0171726775d3da4589b660645fcd43cc026042a2b27e0e0e8374eceb7"}
|
||||
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 0, "pair": "16e529fe8c5486b2", "unit": "f8e004b67ac98dc3392ea4115051e6bd10ce22eb638a762f0f16e1d4200bf129"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 1, "pair": "0b34438e6e0ba63d", "unit": "bf2608b00f0f784cc42e4c607f0b1ef50903283779bfb2eec91d2ec187856d14"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 0, "pair": "0b34438e6e0ba63d", "unit": "c5104cbf723fa0d55675963d8aa6802780ce3d274efe0012290fc49164c3fcc4"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 0, "pair": "b0210cd069757ad2", "unit": "d1d40c8b48d039c1999ff957a015273c30652ca0c45c4eb088052991a2a92f28"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 1, "pair": "b0210cd069757ad2", "unit": "e2e804652a885f640ebf341627f3279d6c965b47ee2ce792e55045ab4d1e1dee"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 1, "pair": "ef603436079b0005", "unit": "77f843b5044bb5da935df4b2620bf2abe9e7dceb3147439e883927112eaec22f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 0, "pair": "ef603436079b0005", "unit": "f2ce95f0e44e73986784b63a702ffa4035c423056cac3667a63140a7e8d795c2"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 0, "pair": "a023056d2704c727", "unit": "6028fe87bea019646fff2b31c1b2d7bd0a95eb9214fe437e2008bc5d2dfabb7d"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 1, "pair": "a023056d2704c727", "unit": "dd93069189c5f242db611e4bfba4341c22f283b65af666fc31343854358de87e"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 0, "pair": "abc80e34d5f72367", "unit": "87d4a5b81bbfdd9628109a5f820895b6fddb00eb96b0f7bbe30a8d640457bcc2"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 1, "pair": "abc80e34d5f72367", "unit": "b171c77b52637bc84c487d7f8bce9a54229f9dbd6605e5089e1e88943ced989c"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 1, "pair": "1945f28a3fc5bd55", "unit": "64c0b7ccad09192eeaf590befe0de0087d5b48a65a4f6cef41415f3f54a8f536"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 0, "pair": "1945f28a3fc5bd55", "unit": "b55cd6759d4b7861e589052aa534bd0554586db46a329ed3beed7244d579e8c4"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 0, "pair": "1c9e47835232f644", "unit": "90e220d3540b88dd9efc8e66c8ab519d4cc78805f16bed585b795c78cfbb4d58"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 1, "pair": "1c9e47835232f644", "unit": "b5a185b05235f70f6cf27e2adba919f4d8393dbdf2d99de095548a21946aeaf3"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 1, "pair": "9b2b34cf62a76ea3", "unit": "91a3984e47f17512db8523e04f863525c67db18631d8f11776cf61ca7221c91b"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 0, "pair": "9b2b34cf62a76ea3", "unit": "d91f1e14282a0c3c8da57c6935a1625e881374bd205b5baa9e4a341d9109ae21"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 0, "pair": "a20cdf9dd208720d", "unit": "18a26df68e4b81d411cfbcc8155a7e6f382753d6e1ad95eece7ed24fa8c44324"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 1, "pair": "a20cdf9dd208720d", "unit": "e6fb74441fb585d7ef5855317fba795ef0f0a0e6eae9970a49556ce3070c3254"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 0, "pair": "b2e833b60a0fa396", "unit": "08ccacb7434e9f8b8b05c7bb0996be0609e34860d8fc418db329aaf8d7910f2f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 1, "pair": "b2e833b60a0fa396", "unit": "2ca7f80f0badc049ce725718e49bd60da72d316589399126a5f4c5308cf83868"}
|
||||
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 1, "pair": "d6930fe7e752b7ec", "unit": "06f0341dfbf1aa9504083737df5d9be3cedfd4e3a198799d8075a29990776800"}
|
||||
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 0, "pair": "d6930fe7e752b7ec", "unit": "30e2cd8bd381d7568b78eaf982f562d278a6a8d942d5282466532c1441bdf0ad"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 0, "pair": "e9c34a6a4bf2d333", "unit": "0ae20c7334fb27889ef8551f34c65a847dffd6b37f84f722f7e8ed64fd82e95b"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 1, "pair": "e9c34a6a4bf2d333", "unit": "59ab1924056704605b22b7c8360f40f745e7f488450be3647a632bf9a3b22699"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 0, "pair": "053a9e02feda3960", "unit": "34e18fe343b5c25ec582fc05ab091fd9ba160f658ba7678066c56db4991a951b"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 1, "pair": "053a9e02feda3960", "unit": "8fcb18a2a8654b6594cb2ec078e2a018d20f73d2b69a2e625338ffa80b3f72c3"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 0, "pair": "2d482951d7380c4f", "unit": "5c1a721da6f85d1d5dfc8cbe5e9809b093a09c059e5abeebed2cbbee55fbd498"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 1, "pair": "2d482951d7380c4f", "unit": "6cabd1c4b5aa94cafdd07c0c6a3f9abfdc03761ddc5b0a2bebc72a508d020b8b"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "", "unit": "2300c097374b9098671dcbcd42d02d08172e0f3064a8323223d0fe79a6106aab"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 1, "pair": "0664148b0b192456", "unit": "d702617aaf4d19811baaa18386a3c8b0eb35cd3fd72e8d90611d34850e990858"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "0664148b0b192456", "unit": "f5ed13d811c5b16d46cbd4a3b9edb8f7d2db49acbd49d95f39f33eaab38f58f5"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 0, "pair": "fd5781ba332ab81e", "unit": "59eca69b2bd99ba282eb8056d7bc789e9c691eebf62a4b42594e34f5516833ae"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 1, "pair": "fd5781ba332ab81e", "unit": "7a7d774a6de9e4a95fe97fff822652f1f1383922f2f0a4dcb12b8e964ca330f5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 0, "pair": "366dc5ddcfe598dd", "unit": "0053d866cec1e99f8a4a8754cf8ece3c46df16e002c86703c68937d5f3a09b08"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 1, "pair": "366dc5ddcfe598dd", "unit": "027dbc5b9245fb804a1e2ba4445b8afa180aa2c2b3293db107ffe0ca4900ebb4"}
|
||||
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 0, "pair": "d0a34bf90e99e794", "unit": "4efbb75cc4105149522d2b2220987fd4373a8ef645c2c544546da540f14109eb"}
|
||||
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 1, "pair": "d0a34bf90e99e794", "unit": "fd790f533103fd1a5646a9374ae895cf6e36d578fc747db2b0d0249a54a85e9f"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 0, "pair": "a697aa902d141a6a", "unit": "341e8fbb46f00167a32a2d653ba09a205794084eff65afddb70f9cfff42d697e"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 1, "pair": "a697aa902d141a6a", "unit": "e29414494b275afc15f3a028f62402161df7f6da5123c302149e23845177f7f6"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 0, "pair": "8c1fc8417190a134", "unit": "2125fb101d6108627e77f61c05cadddddfebd1cddce0e66376d9b822a2eecbb4"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 1, "pair": "8c1fc8417190a134", "unit": "ae7bbc58026fb71f91b477946a27bbe3599be90501da9237755f3cf0fb289794"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 0, "pair": "1972ea9f72b178b6", "unit": "3ddabb356d78e7c65ec8f88b6da8e4fcabc055ae73f2f6d92c50efddc5d4fcbe"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 1, "pair": "1972ea9f72b178b6", "unit": "6875dd5e9c0ff8a1b693d29a109cb2e62e6ff4081df40868aca70e4770476757"}
|
||||
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 1, "pair": "2b007a2377d633dd", "unit": "bdd71a783efb4eda9a82afbd30a16bd7608153830dbc599e65c453c18d1644b2"}
|
||||
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 0, "pair": "2b007a2377d633dd", "unit": "f398ef33900624219e59de08e1cc101c43082a81c37d6d03a916cc220727ca67"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "04aa777130c02a06", "unit": "16aa43c87956c44a4b115245140eefed704095d37cec0cc6410377403cd2fa7a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "23744137e0485a36f686f5f1b5666780fbed3ca522e8c6d35687a391ef504498"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "67565d99fe28c68f", "unit": "2692316792e6f02fb51578cf30d71bba6c73bf5420c36f709681c8eaa69807d4"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "2e36f73002c416a68d077bffb0335d82346c7507f2478353aeac005fce5a9cbc"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "04aa777130c02a06", "unit": "67721f67055eb6fed955ff22bfe4060344662eccb369431835a2116ad120c4f8"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "8fc1c96fa6faa4d3966ecc42876189b18ddab779d58f47cdf28a1155f48a6ac4"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "67565d99fe28c68f", "unit": "ab535098a8e709b022afee1da15dd1fc371d389056938ee4fa4971b2966c823a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "d46f94235bc462360097e7a22e1d442d95f049306631489a4d8b25f647a95bbe"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "edd6dd4364442f66384d14e9317ef2f6234e32ab78327a45741b8a816b4332ee"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "fc7df6887ac719d4615934c0e8c3296728c2f6089f233158c59989027c802d72"}
|
||||
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 1, "pair": "c0412509f15fe3e1", "unit": "63bda2a71ad7155ad2f59aa9086efef3fd3710dc82c57f4b32ec8bf561460239"}
|
||||
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 0, "pair": "c0412509f15fe3e1", "unit": "b1c9a070c93764655786c3de5fde536c150e319efe47548614cc544b73a8ee2b"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 1, "pair": "feed5f7de11289a4", "unit": "71531567b2b8609ae0d31cb177b8faea4f5fc61923e513b0154a2c941287ce2b"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 0, "pair": "feed5f7de11289a4", "unit": "d6be6d397a301a07faa761163d3d9e270d71d23d3bdfd084eeafb1e62b91dd71"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 1, "pair": "31c259b4e9dccd82", "unit": "1f1ddb8a5b415461638056c70756d2d0e836de71e824c0e46474f20646eaa4d5"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 0, "pair": "31c259b4e9dccd82", "unit": "a6ec568d4eef212a842dded7fdef2284524d367e5d24473ddb34e6374be9fad7"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 1, "pair": "ffd4b04a7bebfaf8", "unit": "54f7519d3d59b6d8d69a5f8f7bd78e360d3fabaae00b7587c37e388719169370"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 0, "pair": "ffd4b04a7bebfaf8", "unit": "c5e42c9ab2ee54e46700ab0b64e239b567fbc75ccaceaf6cbf20687813663332"}
|
||||
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 1, "pair": "2ffec9490ef1d482", "unit": "0ab883d3561162b75ad9d161f79c0d17521a23317f2f574c910465f288119483"}
|
||||
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 0, "pair": "2ffec9490ef1d482", "unit": "b0af86c1642c14047248f1134181be0ec3839900917dffa360c5014119f8632c"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 1, "pair": "73f2a014c6a699ed", "unit": "44ba9d9d8e75769c1444ebf37a64941a845b4bcae678ec79d64dc84e5cc067a0"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 0, "pair": "73f2a014c6a699ed", "unit": "b430d488c486cef26b1c661f773e7e9e4a177eeea73537a033eec98d532e4eed"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 0, "pair": "738e4c22edb32172", "unit": "932d7312b2bb4b830179104bca076f587505c2d4ce881b2f04f992defba6dd9e"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 1, "pair": "738e4c22edb32172", "unit": "eee5136489c3baa3653ededbf6385d0237e2bd8b5384286780646b88ed34fb91"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 1, "pair": "cb0c638282db0d37", "unit": "a49e6ad2e7c2a5ad0d68d5d69c263306b501fc0dc9382e701fb5fe680611196b"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 0, "pair": "cb0c638282db0d37", "unit": "dee28099d3b37cbd0ee14412122ecc5f8488ed6bec01863520b87706a040fba1"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 0, "pair": "09bce271e8530fbc", "unit": "35eb19b4c6176b37287e2123bb98ec4045ce6bde7cc3864dde317c024b0935d8"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 1, "pair": "09bce271e8530fbc", "unit": "dd33291cc494bbc77d44e9b13d16f171d32c9746637506f315f375e8a9ddee95"}
|
||||
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 1, "pair": "a856cd13668353a6", "unit": "6149fb0132b981620a90a23a9af8d4656d0d827403e63b48258443cacd0a9c81"}
|
||||
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 0, "pair": "a856cd13668353a6", "unit": "b5aecdd6a87f8d57d28f54a013b1f96332460fe1bb44bff4ed28e166f18f9abe"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 1, "pair": "40d8d81a764c508f", "unit": "7deeed73c29c74ccbed29be049e109d363d41e4b5eea245ee57780b85f0e137a"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 0, "pair": "40d8d81a764c508f", "unit": "b5cdadf848e67910550666ba2a6ac67b24112e501c02dba1865766a316317a23"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "", "unit": "15dbea384b3255457a44d9382efc6944dd27f1d411d903344a5d5e5fd194b72f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b2ac284bd70dd684", "unit": "225890ba6694dc8f4fbab46e9c851dff8c095e906cf14db15d01ebce0891aaab"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b6eca35de93a5be8", "unit": "42eb8427415704e060e73b5ce0943b014ddf380ca756d5ec47c558e59a79895a"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b6eca35de93a5be8", "unit": "4c9cdcc11a18641ef41e9308712c5e8b830ec9ad5ebc3a956c8701e4f77b39c0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "6a53939838afbdd7", "unit": "5794809e595311c86909d673a570d003aa025e4dcd6abc5641b641d935beb0b1"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "46d3ec7d49cb8ca7", "unit": "5f7a63ec650915962ee5255f5416df63ab882d6107f3b540be36d481798a8944"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "6a53939838afbdd7", "unit": "97a7018535ab511622e3ade9e53d570ffb1c80633eaa372fb76e749b3a586c8f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "46d3ec7d49cb8ca7", "unit": "a84e9ab47f4f2b584684dbb81575758d9032fe3f2ed53bef478cb9dcfedaf677"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b2ac284bd70dd684", "unit": "aeceb1ebc863ebbc3b69ae56c5b78d2dba005132b75cbd6af929bb0bfe25da9b"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "f42cab375ace1739", "unit": "e329d4a6ac757680bf3d292a81f1f74f8a8a9b482d1651efe9f503f6a4de8796"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "f42cab375ace1739", "unit": "e73c5aac41c0b93072223a3097fb175548ce893e0678ff3a22bc7571745a17e0"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 1, "pair": "1041a18cd1aec5f8", "unit": "c301b8c362aee3fe5ea38e18f858166bc4162ad8ace44c5fb2df192afc81da2a"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 0, "pair": "1041a18cd1aec5f8", "unit": "ca8e58ffcfd60bdc8626db8b9eb4f253c1b8b8affcb793c6d963b7a3ec91906b"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "", "unit": "52354ccdbc16f209e4692d97d5237e04ed030cf4a5e71f77b5a767962b355b4a"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "7d23be667536afff", "unit": "b5016a5a53409491ba805ed8718599b84250b7b6914da0434b0882b0b3578dc8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 0, "pair": "7d23be667536afff", "unit": "f1b01102968568d3ba1b2fdc70f4ae281f433fb20d29eff2a55fc45555ddbfcd"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 0, "pair": "f21df949f8aca4fa", "unit": "15d23ab3aea4c878dd73bbae579f3ab473e8363477a97d28164ed5e1b94cb510"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 1, "pair": "f21df949f8aca4fa", "unit": "787bc98c651ed3db21be4c28e9c560c366ffd6ca508823d21e5d83eeaf1269db"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "1199de75f426940b", "unit": "10fdc236f43a02b7e291eb8bf6ada297a2ae98086a63b171057bc7bfc37a3377"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "22112c31603bbe9f", "unit": "5eb3b7523b5a196dcaacba08d68457f9a45e31503ea02745ca3a7d0542f9881c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "22112c31603bbe9f", "unit": "6bd5bdfd81b0a3374d59cfb2c1f7fa2b27887f4672f4485e77eb5234b569983f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "1199de75f426940b", "unit": "c09dffd04708540845c8d1dd77a2bd989bd35f264abf652a2db55f8cb1faf7a9"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 0, "pair": "7aab025ea7b97019", "unit": "7487a34cdfdb532a5096764a47a3e565fa509605c1df0e2b105eb9c751fc5307"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 1, "pair": "7aab025ea7b97019", "unit": "9c0be2a58741f095fd0a9cdc60d4ca4039de1529699d5701dc4d0bcf7d5bf90c"}
|
||||
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 0, "pair": "4de5441f1776f083", "unit": "a3a35d573da308cd1d0614a26616bbd37b360d9ea416894cdeffdb59abd691d4"}
|
||||
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 1, "pair": "4de5441f1776f083", "unit": "dc33fc26c1bf7c67e5439ddfdad06c0faf16c796e0fe8a46b6e6a541e708676d"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 0, "pair": "1972a9a06d7585ad", "unit": "1f472e842e894c9907536c6eb6c9c922bbb3b21c27999114588ec2a4345ab665"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 1, "pair": "1972a9a06d7585ad", "unit": "7e38e24f1cc4d287627b7cd6cb870888865bb87fa87bba1c4fbf84eabb213121"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 1, "pair": "d76d5232a8587127", "unit": "1ec32d427822d1ddd3b0d7245f5771ddeb56c146ac8ff9959dd85760ff659a20"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 0, "pair": "d76d5232a8587127", "unit": "5143b19a1a1e6b9162b80594ed7fb5c0c57ed0b8abfd793bfc71089552d57638"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 0, "pair": "5f6e814444b4acd8", "unit": "2e8aa07b1aa4bfc5349b0d4464f3f2a6500aa55ff765b0139e9901ed34151ffa"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 1, "pair": "5f6e814444b4acd8", "unit": "e26c726c08be761b22fcc6ec17fc1fa7aba4769f08957f024e5545906df62797"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 1, "pair": "888e955fb6cfb4fd", "unit": "6f1a431ca97afc34ceec48393eb134a8928a312e6f761eb907c6b5855cac6208"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 0, "pair": "888e955fb6cfb4fd", "unit": "7b62415d17569924d16238a4aa28ab1af2db6561f56be9f59d8075fd1bc4a941"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 1, "pair": "a5691395e641fd4c", "unit": "40339c123107f1231d6aa58de876cded1dfefd5b31dadc4fc86eedc6a687165f"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 0, "pair": "a5691395e641fd4c", "unit": "ea3828e2f59cc6f7b415a101532b566ecbd47ab123c3c7593c23c52f6773a2e3"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 1, "pair": "f1ebdacbd647e39d", "unit": "79628ef3d6c9e12a94922cdca548ff77d3dbdd6b595c1fec5958fcf23b609095"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 0, "pair": "f1ebdacbd647e39d", "unit": "f910e010c3ff2c8c17f0f80fbe89a934e4f792b601febbf318359e6dc3afc18e"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 0, "pair": "2dd5384f097711b5", "unit": "212fba6b5e50db045114af369a03ffbdf09ec898c10eeff3c45bfc01a5017b4c"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 1, "pair": "2dd5384f097711b5", "unit": "370ba3a1d0e354e882b7be92c1ebbf2e2903bdf29d442048ea3dc1f80922467f"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "strangerstudios/paid-memberships-pro", "label": 1, "pair": "", "unit": "ac905d12b92954263c046bd0138a682cea86cf8476336d6baa664ade28d34ac6"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "218a45d869bd7e39", "unit": "309ebac468b7ab054c8396ed495770385ae3590caba43709745ba2d1db4b6d54"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "", "unit": "3cc8613813dec13cb1b7b45b296688da883f0b36cd2c4fc1da6e81703fb59ab9"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "218a45d869bd7e39", "unit": "9bd2331dbc260451355307e4c37bae17ad6676654c66181934e1b35fb174cc5a"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "b087ac4d0989e0b72698a9644ef4eac8073b4a15bb21730d63e4730b5cc58a60"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "c5d1e8d19560c997018fdca12eb60f1799be76c33be058b88472f5a2fc6709be"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 0, "pair": "c863054be488a284", "unit": "425e1d9961b79bbc011c447cd050910dd796a32b248962a5b0acc3f5f19b7345"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 1, "pair": "c863054be488a284", "unit": "552925bba199119c7db5457b9d3c535bc9f6061cecfd2ab2f72cde690a536e1e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "22befc17a75a688e2f9358d2ddf9a7b197da44cd95dbdeae5e0191b113ee54d3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "63f496d9ff53e39b", "unit": "6a98f8145776c78793b37156e9a81757841bb6de80016ab4fa575209d724db3f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 1, "pair": "63f496d9ff53e39b", "unit": "7ac60c96c15f8517e089cc2240aedcf1fc30e54a41e35fe03c6b965bc393fbd6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "ae39cb55ef2a87c5347e0ddc40975f142450e82de4909e93ba62a7ed0efb4134"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 1, "pair": "d1b19898f8deaa41", "unit": "3763c7b407ad471820b5f8fa25bde0ba6d91e15c07c56681de4b41add418b0b1"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 0, "pair": "d1b19898f8deaa41", "unit": "bc8b934938c30687f3d4f982392d2cfa9504a916b463ccb5c46fe8a30eede488"}
|
||||
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 1, "pair": "f5f33fa6fe7b8492", "unit": "938bbf146a9037e687190bcee82c047492dffbe90efea40a49a647f81a0f1ed0"}
|
||||
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 0, "pair": "f5f33fa6fe7b8492", "unit": "b90c9f7e0011685f1a8e853a9ed0ae37fd437ef6d6eb0ba17c240aec2c22b603"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 0, "pair": "f74364dc8e35c8ba", "unit": "11e760e99c5c852676c9288e18a250febb3854e9047b51e8b9ecc9f3b423b6f5"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 1, "pair": "f74364dc8e35c8ba", "unit": "5fc980194040f4c4d803075d4085618cc0260ae8f3454b26b9b704d30200ffea"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 0, "pair": "db1c51833994ec55", "unit": "c2ce8320c778dd55434d7413844797b68e6fcabc4dc509458a9d60ca70b88c60"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 1, "pair": "db1c51833994ec55", "unit": "f2d6843547af1276ae8a4dff3a7786913fa79dc69540d4fbb95c4c091e27c7f2"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 1, "pair": "f4690679bf9d80aa", "unit": "1a4e24df8d08672ecde4709a77ea5c437b2198fecff8bfff373b63c9eaed6e63"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 0, "pair": "f4690679bf9d80aa", "unit": "78711973ec84867671314dd7878769092b32f59860eafed50a0f6979daee76fc"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 1, "pair": "d6a42c18151efb48", "unit": "01ae2279707361bfc06a005335d41b85066d0a163802ca81c031f5dbded5bb46"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 0, "pair": "d6a42c18151efb48", "unit": "a50dc15c313a9078badea01365987c66c5cee1ad48cea65c35c8bc2cbba25e8e"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 0, "pair": "85c15e3fc2b370b2", "unit": "0990d8f41cb07c1cf217812e51a7025381de4ba68d1be3799e26241d81dcea67"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 1, "pair": "85c15e3fc2b370b2", "unit": "9da1d63f0c4da270f66eb87bf9c342ace883a0aef78c4643e254159b5da8fce3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 1, "pair": "c7f4f4ebebfae831", "unit": "b5cbc1490bca76679adfbe5e4d9f8067d27a226d595ba401b551b00c3ce4d45a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 0, "pair": "c7f4f4ebebfae831", "unit": "ba2c1808eebdbe3f54c488b09bcb3d69e521916d1a3a0b5b14e02d5bc88bf016"}
|
||||
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 1, "pair": "1507c8e00ed4b859", "unit": "620e9b5ed143c0bcec178db359fa601acb1d3a134b3e85c2e838b7dcf3892d85"}
|
||||
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 0, "pair": "1507c8e00ed4b859", "unit": "6f5ce4008dac0471260310c60e1e2a9f76a7e4eec6ebd039835f0488cf3b19aa"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 0, "pair": "8f5a6abedd742bbe", "unit": "71e3ced643a244100cc3fde234f5b4b27655214c7ae0ff67ab61e3e48a8fdf78"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 1, "pair": "8f5a6abedd742bbe", "unit": "7c8cbfae87127042f9f43621345b4f7958bde27a3ff9db76254063e80735f3ce"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 1, "pair": "c9538342f2d7efca", "unit": "2b8db57d65ef1d67c457ff5a336e8a542d03387395d69d5e16815c4f00f40723"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 0, "pair": "c9538342f2d7efca", "unit": "e2ed113dc21ad5f7caa71ba4a5ece8f8f900a97fd0d00dc85f7ac79ff5bed19a"}
|
||||
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 0, "pair": "494dd5d5e5cdf4b8", "unit": "3b8011f94ec721f68dd29afc6c8b01117c86a8b0c31f3650a6ce0a8d7c0739ff"}
|
||||
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 1, "pair": "494dd5d5e5cdf4b8", "unit": "f7e2280574e9ae51daa64d8adb2955407ff6d1e156f0e2854b6c047e0a733c78"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 1, "pair": "51eae488182a123a", "unit": "20b039866ba796c34efa0e92468fb49a314ad7fe2914d9ff8b823e3e35ab26f9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 0, "pair": "51eae488182a123a", "unit": "2ab3fef05def768ef7b4165bcedcf7d200282dba78a32e19b90a0df3aee7e3e5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 1, "pair": "6f5a650865e08571", "unit": "e2051cc42e4f8b026c7807f966d492367310c90326e60fef1b0ee8c509c805a9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 0, "pair": "6f5a650865e08571", "unit": "f5364d68ffe80b578e788f37f08784b0162826393b49b4a3affd70d7cc4c9e01"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 1, "pair": "8acd426c5771b37c", "unit": "983e146a60e061254b15dc8dd573cbba57f8dd0358f23ea1b9c313ef69069af5"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 0, "pair": "8acd426c5771b37c", "unit": "f80dfec45231e4778078cb10f00128ed445177cac0a062fc2df5ced5c0171f21"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 0, "pair": "9b84800ce5c395cb", "unit": "9b7dfdceae03fd86ad3260a03f85bcdcff8f32f10c265d23743ff8dc98eeefb8"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 1, "pair": "9b84800ce5c395cb", "unit": "b99dda150c0da3defa5f4c527ef6c402f5589dfb2f2f03f3e5e424910eb48c53"}
|
||||
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 1, "pair": "ad64e8fac4ac3ee0", "unit": "36dfd01915a553d86576e8ecd3b737b8e2b297aa542f58cc83b448060e74398a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 0, "pair": "ad64e8fac4ac3ee0", "unit": "c2b6d40f130d35bbfc16fab9411c664fa06321c59fb1d901a1987a69cb09c72e"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 0, "pair": "d48c08b19f560848", "unit": "03d02c9e25eec0f380b125279fce35e8867065bf6e1ec75f9f39729ce6efdc77"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 1, "pair": "d48c08b19f560848", "unit": "abbea69b7d0fa68c3ace442c25597e128cde354b417787b5913fd10d666c0d19"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 0, "pair": "bfc8188bbb673e3b", "unit": "a7cff0da55c0cc1d5c838e11fc6ce6c4b22c54b9f8e101688b218d6928eca504"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 1, "pair": "bfc8188bbb673e3b", "unit": "f741c56815230b9c30e2c8ef8952b895e9480e4780a89ff2cf6daf60d1fce7b6"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 0, "pair": "3844ed3d703d7ef4", "unit": "445efda074e4d9f6c1634225eb08a3c3eb1257311c995ce891df2c791a6f5c8b"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 1, "pair": "3844ed3d703d7ef4", "unit": "8ac27b08549353ada819caaf99a5180d2f2f515b03aa8cd99a5d37b527ea3a5c"}
|
||||
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 1, "pair": "a63eae7d621a985e", "unit": "04ca0b583b09c3b4f67bd2496a4607ecb134d24b7bdaf8960fa151e40ebcc447"}
|
||||
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 0, "pair": "a63eae7d621a985e", "unit": "ecfa2a67726ca9d3e2621a4d8be60145095b76aa18ecf0268212341d63752871"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 0, "pair": "0d9f529b7f0084b6", "unit": "35aa369236f510634b3aa7b54b04e9da0f80877e515786a888e760f8fe33c524"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 1, "pair": "0d9f529b7f0084b6", "unit": "ef4f8613c0914eb815504e00cd6ee05df38352fd3b27826798e6623594017042"}
|
||||
@@ -0,0 +1,90 @@
|
||||
# Experiment exp-004 (learned-decision-engine Req 6.4, amended 2026-10-03 by maintainer decision): the pooled BLOCK
|
||||
# gate with a family floor. Registered before it runs; `ousast learn experiment register` records this file's sha256,
|
||||
# and a changed file is a new experiment. The procedure and the decision rule are fixed here; the units are NOT: they
|
||||
# are frozen at run time (`ousast learn experiment units`) from the memory after the negatives harvest (merged in
|
||||
# 5b2bd9b, 1133 static examples), committed, and registered again before the run. Until then the registration row
|
||||
# says `units_status: pending` and the run refuses.
|
||||
#
|
||||
# Scoring: kind rule, zero cost by construction -- one scoring pass of the six slice-2 incumbent programs, replayed from
|
||||
# the response cache behind a zero-ceiling meter; a unit with any uncached response is excluded and counted. The
|
||||
# harvest changed the memory the programs retrieve from, so most prompts are new: a paid cache fill (separately
|
||||
# approved, its own ceiling, as for exp-003's supplementary analysis) must precede the run, or the exclusions will be
|
||||
# most of the units.
|
||||
#
|
||||
# Normalisation (declared before the run): per-family Platt scaling, learn.calibrate.fit_map with its existing floors
|
||||
# (own slope and intercept per family at >= 20 positive and negative repository groups; a pooled slope with a
|
||||
# per-family intercept at >= 10; below that no map, so the family cannot block on that fold). Fitted on the training
|
||||
# folds' paired units only. Why Platt and not rank-to-quantile: a quantile map gives every family the same flag rate
|
||||
# whatever its precision, so one pooled threshold would flag the top slice of a weak family as readily as of a strong
|
||||
# one. Platt puts each family on the scale of its own training precision, which is what a pooled precision bound needs.
|
||||
# The map is a normaliser only: exp-003 showed calibration (ECE <= 0.05) does not hold per family, and precision is
|
||||
# demonstrated here by the bound on held-out flags, not by calibration.
|
||||
#
|
||||
# Thresholds, per outer fold j, from the other four folds only (fold j's labels never enter a threshold or a map;
|
||||
# tests/test_learn_rules.py):
|
||||
# - the training units are normalised by an inner cross-fit (each training fold k by a map fitted on the training
|
||||
# folds other than k), so t_block is chosen on out-of-sample scores;
|
||||
# - t_block: the lowest normalised score at which the training paired units of ALL families together with p >= t
|
||||
# and a verdict other than `unsure` have a Wilson 95% lower bound of precision >= 0.95; none -> no BLOCK on fold j;
|
||||
# - t_quiet, per family: the highest normalised score keeping >= 90% of that family's training paired positives at
|
||||
# or above it;
|
||||
# - fold j's units, normalised by the map fitted on all its training units: BLOCK when not `unsure` and p >= t_block;
|
||||
# ADVISORY when p >= t_quiet (or the family has no map or no t_quiet: nothing is silenced blind); silent below.
|
||||
#
|
||||
# Family floor F = 10: a family may block only if its own held-out paired flags (all five folds) number >= 10 with zero
|
||||
# errors; otherwise its flags are reported as ADVISORY. Why 10: zero errors in 10 flags is unlikely for a family that is
|
||||
# much worse than the bar (a family at 0.75 precision produces 10 clean flags with probability 0.056, at 0.80 with
|
||||
# 0.107); a larger F is out of reach of the smaller families (16-19 paired positives in exp-003's units, all folds); and
|
||||
# 10 is exp-003 arm C's training minimum. The floor uses held-out labels to decide which families block; it moves no
|
||||
# threshold. The primary metric is computed BEFORE the floor, over every family's held-out flags, so dropping a failing
|
||||
# family cannot raise it.
|
||||
#
|
||||
# Power note: the pooled bound needs >= 73 flagged units with no error (110 with one, 142 with two). exp-003's
|
||||
# instrument check found few positives above the first negative per family on the raw score (paired: 10, 1, 3, 0, 1);
|
||||
# per-family normalisation keeps each family's order, so the pooled top set can reach 73 only if the harvest's pairs
|
||||
# separate better. An inconclusive verdict is a plausible outcome and will be reported as such.
|
||||
id: exp-004-pooled-block-gate
|
||||
hypothesis: >
|
||||
One BLOCK threshold pooled across families on a per-family Platt-normalised score, chosen on training folds only as
|
||||
the lowest one whose Wilson 95% lower bound on flagged precision is >= 0.95, reaches that bound on held-out
|
||||
repositories' paired units, where per-family thresholds (exp-003) could not for lack of flags; a per-family floor
|
||||
(>= 10 held-out flags, none wrong) keeps a family that the pooling masks from blocking.
|
||||
kind: rule
|
||||
programs: # the slice-2 incumbents, as exp-003
|
||||
access_control: 745cb1d76bbb09dc1574d333d2118489be62e39886517894d3699fa565b0e8c2
|
||||
deserialization: 3c0c2831be7d78198a62351661c37867363c501cb40e3ed7b0119c155af09654
|
||||
injection: b207a0710e0abf053b3b9397b25f9ee1cab4679144865e3a82c1b9be2c7b4982
|
||||
output_encoding: 5d768f16724562c381c09949ff3df9cbb11d58cbb7a9c0515d0dc3f858d3818e
|
||||
path: aea396680a4874dcb6de70e240ca0480c83424debdd3710bc3f1ea6a612661f3
|
||||
untrusted_destination: 1242b3973fc89056aa08e00843043d68235df217d8e3ef8a0ea0e0b81c142753
|
||||
arms:
|
||||
A:
|
||||
label: today's rule -- Platt map cross-fitted per family, BLOCK only when the calibration check holds, BLOCK point nested on the other folds' calibrated p
|
||||
rule: {name: calibrated_block}
|
||||
B:
|
||||
label: pooled gate -- one t_block over all families on a per-family Platt score (training folds, inner cross-fit), t_quiet per family at 90% recall, family floor 10 clean held-out flags
|
||||
rule: {name: pooled_block_gate, precision: 0.95, recall: 0.9, floor: 10, normalise: platt, select_on: paired}
|
||||
unit: candidate
|
||||
pairing: paired # every unit scored once; every arm decides every unit
|
||||
cluster: repository
|
||||
order: {randomise: none, seed: 4}
|
||||
# primary population: paired units (both sides of the candidate present); pooled (every unit) is reported beside it
|
||||
populations: {primary: paired, secondary: [pooled]}
|
||||
metric:
|
||||
primary:
|
||||
- {name: flagged_precision_lb, target: 0.95} # held-out Wilson 95% LB of flagged precision, all families pooled, arm B, paired, before the floor
|
||||
secondary: [flagged_precision, coverage, flagged, quiet_recall] # quiet_recall: held-out positives at or above t_quiet / positives
|
||||
# Decision rule (decide_pooled), on arm B's held-out paired flags of all families together, before the floor:
|
||||
# REJECT B if their point precision < 0.95;
|
||||
# else ADOPT B if their Wilson 95% LB >= 0.95 and at least one family passes the floor (>= 10 held-out paired
|
||||
# flags, zero errors) -- those families block, every other family stays advisory;
|
||||
# else INCONCLUSIVE (the incumbent rule stays).
|
||||
# Reported, not deciding: per-family held-out precision and LB, the floor table, quiet_recall per family against 0.90,
|
||||
# the after-floor totals (what users would get), the repository-cluster bootstrap interval.
|
||||
decision: {arm: B, target_precision: 0.95, pooled: true}
|
||||
test: {resamples: 2000} # repository-cluster bootstrap of the held-out flagged precision, beside the Wilson bound
|
||||
mde: 0.0 # not a paired difference test; see the power note above
|
||||
budget_usd: {total: 0} # the run itself is zero cost; the cache fill before it is a separate, approved spend
|
||||
families: [access_control, deserialization, injection, output_encoding, path, untrusted_destination]
|
||||
folds: {kind: outer, k: 5, seed: 0, compile_fraction: 0.25} # the incumbent evaluation's folds (learn/compile.toml seed)
|
||||
units_file: plane/experiments/exp-004-pooled-block-gate.units.jsonl # frozen at run time, after the negatives harvest
|
||||
@@ -0,0 +1,501 @@
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 0, "pair": "6b0b3cab2df3397b", "unit": "0ac37b7154242dfa637140af643fb4595e85510a17ab1a12c111441b9d293238"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "adonisjs/http-server", "label": 1, "pair": "6b0b3cab2df3397b", "unit": "6e5e274946d0e8f13a7ef4e37f6c7ed1828d057304a9262db0311413f18e1731"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 0, "pair": "c0d0c91757f5ba95", "unit": "285891b16ee6870af50b36dec3c2ca9a90ee15d9e14851c61e66ad443e6eab59"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "agentfront/frontmcp", "label": 1, "pair": "c0d0c91757f5ba95", "unit": "f33fb2413edf1499a524c4592b3b2f93d166be845f5b65291969a086a4bec334"}
|
||||
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 1, "pair": "f56f2a107ae591be", "unit": "350149bf9cafbb547f500b714a164734d2361b22e86fc9819c7bad3507d8458a"}
|
||||
{"family": "path", "fold": "outer-2", "group": "ahujasid/blender-mcp", "label": 0, "pair": "f56f2a107ae591be", "unit": "ad125319dd2d4c9d2d99a1eb34bb4cfab9f099a96e7547d0d59a8ecf1c990347"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 1, "pair": "ee577ee4b6076fab", "unit": "2523f55d035d1a772a842090bc335e5d755b2fe15b376dd5b6b9e30b9dae82d1"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "aio-libs/aiohttp", "label": 0, "pair": "ee577ee4b6076fab", "unit": "ca0f6c160d7839204f6fcc1709bcbc749a02a0b3b8bb648e2a238eacbf22ed78"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 1, "pair": "c95b6b01d0c2db2a", "unit": "15540c23be0c42472a49039658ec2ec0bbd8c6b318e35dd040a9d427370e0ef0"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "alchemycms/alchemy_cms", "label": 0, "pair": "c95b6b01d0c2db2a", "unit": "542471980417e964182d7d3db59a05fe12bdd072449f3bd77f7daa5515dbc909"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 0, "pair": "29252934ef4bcfcf", "unit": "63bae9240b31f9cf1dfe00d9b01d4d0ffda95931c9e7d040df1fd718f04d828b"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "angular/angular", "label": 1, "pair": "29252934ef4bcfcf", "unit": "cce2af1743834dcc3df58827ac263a2d63296ca1b2a2f9bdb811954fb5767c33"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "bcc636b23dfb5f570d8864b17fe07b6b4fb52b65a40988c32011f1422f74d504"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "dbcf1e67a1be7ccf7d51701f9b8339339164055135443044c23a446b16b3d934"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "anotherik/threatbyte", "label": 1, "pair": "", "unit": "50408f68114ecb4de3bebd836d52d9afad86b22dd1843641ac8a49b617c09e33"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "anxolerd/dvpwa", "label": 1, "pair": "", "unit": "cafb0f00f80dc6beeba40eb95c764bf9a3569193070ba321c28677dfa425ec88"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 0, "pair": "07317a2258292656", "unit": "a9f292d7cb467def582d9c0b31a397b13cdaefa77380ce971bfe9fbf535bb660"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "apache/airflow", "label": 1, "pair": "07317a2258292656", "unit": "ed363d0dfa620707be477b90eb4b83a901bfd402040b918aeee6f06b2c5b795c"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 1, "pair": "1544b6dfa2299d60", "unit": "d45018fc4ce3dfd715c7087d4d197ad4108c0c8b6625c02c4d4da0adb9f6a140"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "apache/inlong", "label": 0, "pair": "1544b6dfa2299d60", "unit": "f295c7dfb095595a4aa985c58688e4757c29be9e06b7a4ca55035ac5d06b1ef8"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 1, "pair": "fe0e104d111c2555", "unit": "088036fab9962be2b96f9162ea4f32190575678d3c5ba96bdfeac178bc22b2c4"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "apache/logging-log4net", "label": 0, "pair": "fe0e104d111c2555", "unit": "7fd2b56a816e25358477eee64487fb2c77f34c652703ad0a7120c368492017a3"}
|
||||
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 1, "pair": "b6742570f7f0e026", "unit": "7c6c1d27aee6557f1b6bf371b8209003f0d2d1122cfa35b6629eb43f38769cf4"}
|
||||
{"family": "path", "fold": "outer-2", "group": "apache/wicket", "label": 0, "pair": "b6742570f7f0e026", "unit": "df8300d84dd784d0be46969ea432ef97b8ce611bae7a896ed4f349bd4d6897db"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 1, "pair": "0f584bb97a154488", "unit": "222a84ffc7c58d45e8f10df221762b23bf20d0287ecd20f375558134928886ca"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "apolloconfig/apollo", "label": 0, "pair": "0f584bb97a154488", "unit": "25e4c9d2941f59f072385a7bde3fd0bd76f9e986ed24b06ce81962ab447f5094"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 0, "pair": "b15b924b4f7b1ae1", "unit": "2fa95fce0751c97db1a12a80ddb249ff5c54ee31c7dc3cd47765099c7b673e52"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "appium/java-client", "label": 1, "pair": "b15b924b4f7b1ae1", "unit": "52be8128053102eafbee67c2d3aac88c83ca290b943ab2f2cfdcd2630601c7d9"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 0, "pair": "d31ca58f2a1d27d1", "unit": "2ae40ea61d691060e7b9f6c8e910e3aa0edf16bc6f88bec81aa9a7dae5cf5603"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "arcadedata/arcadedb", "label": 1, "pair": "d31ca58f2a1d27d1", "unit": "4876728701698045e65eeae6e11e133c30f43b09029d7777d6d780f7d0855885"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 0, "pair": "bb3d7baed62e6bb7", "unit": "e8ae88fd2b677e3f64ac94bddb012e3b903afa294ecc87774d10087c0d50489c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "archivy/archivy", "label": 1, "pair": "bb3d7baed62e6bb7", "unit": "f6b0d241891dc1d897d74d98b6ff7ada2c1f1ce7874ca32097630c06ca53e389"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 0, "pair": "d9d8cfa38981847f", "unit": "864f1800617f5218a2300ce390ba7922fe1bd0a182de115952342b61d627bb16"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "argos-ci/argos-javascript", "label": 1, "pair": "d9d8cfa38981847f", "unit": "fdcda2307e20ef1b9247b7082a85337f457dd1f4e648bb8264077bf983c03271"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 0, "pair": "1f73ad2576d52a64", "unit": "0bc9634940734cf7f41628f220032fb7e2fd2ff5b584afc519875e90ea0be5a4"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "arikusi/deepseek-mcp-server", "label": 1, "pair": "1f73ad2576d52a64", "unit": "ddb59b2dc39715281a6d48b32e7ca4ec0354b85b953fff55139579f83905b7d6"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 1, "pair": "a7326e6daa507b79", "unit": "793ebf4c8e0f46d8ce3b3d090d2f6290d4167931a3b574f8269b90008e3dcde7"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "arthurkatcher/google-maps-mcp", "label": 0, "pair": "a7326e6daa507b79", "unit": "b8d25f337fad527544c33c82f473015e08399457276fc480477f3cfcfc9c5b8a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 0, "pair": "211ba072bdfa67b7", "unit": "3217f9d672a4a1d78005c8fb66e1bc60a542cb26bd8c46903402c765c123d60a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "asymmetric-effort/nogginlessdom", "label": 1, "pair": "211ba072bdfa67b7", "unit": "8370f9624baa0d144e56b8087eea897b98e5b2afcd6eb3a029d03e5b4b69d584"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 1, "pair": "baccf350d5cc9911", "unit": "50a3ca70700c8b41874c43ba94e6b7ffe0aa28b8930ff941a40767e1057065b5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "authlib/authlib", "label": 0, "pair": "baccf350d5cc9911", "unit": "62927a118d3662c81450e19d49f4146b66f29a58d55ec0d1606ed521b1883f12"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 1, "pair": "697f34e19bda57f3", "unit": "775b6d900ec1ba71e4082ed49bb94439f38066628eed91461eecf0658caee78a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "axios/axios", "label": 0, "pair": "697f34e19bda57f3", "unit": "9e855586c685b3c62ffe32407a82966661d5839e2cff742f2676acaba645a97a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 1, "pair": "143b900b99a46cf4", "unit": "200f96cd784ff5c39d8bba22883f22bc8a7c87c6970a7c871802f48a45548337"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "axllent/mailpit", "label": 0, "pair": "143b900b99a46cf4", "unit": "7a0d62ec424849ec4681f4333fbb077e8239054d1e964bd99b40847ad7ccdb01"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 1, "pair": "4b4e81797ca48cbf", "unit": "a4d17d85973d024a615b89334ad5a2f46afea660ef5d0bcf2b2196c998b1caf2"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "basecamp/trix", "label": 0, "pair": "4b4e81797ca48cbf", "unit": "d604e460d8f5291124ba6fab5f9c94c7dfda61ea5f6808e512162334d6d6db4f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 1, "pair": "bcbd06e53147628b", "unit": "0b5cfe2d64771e30e50ff92cc409b8dd5e30079c252918ba03774cb5198a73b5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "beaugunderson/ip-address", "label": 0, "pair": "bcbd06e53147628b", "unit": "b805976c35330452af6c9f814ff4617b7b9ad0e82e1fa4440a443f93f0772c4a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 1, "pair": "c12dbc8a50f7fe9b", "unit": "04beb8a0fd054b92c2dd0ed83907330fdee6124d86114df948ac72a297a7d51f"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "browserify/static-eval", "label": 0, "pair": "c12dbc8a50f7fe9b", "unit": "c5029016a6fd2e3f9df59dba0ce3c173bbbb0d2b1de22d151086e01e1b272119"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "0e213aeabacf4205db024ea3c557e720c3aba911424555784917e967d6404867"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "e8bc65415bd42e43", "unit": "4920230ef773595a9fd9a92a090dff42b48499a1dc5fc8831f941dae55cb2538"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "e8bc65415bd42e43", "unit": "49209a810f207eb61382e2ceeab7076936838a5652546f21c187d066e220160b"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 0, "pair": "", "unit": "9e2964884cdf281fa3f18ee3481aa62f9a8a0a195d605da2df5bd71876fe311a"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "budibase/budibase", "label": 1, "pair": "", "unit": "f9cc6f1bbcd59f5dc0de8f96248f722ea0679f8afa47c53f70832151ce7616d3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 0, "pair": "e69463f945f9a0f4", "unit": "5feace7b28e260175f80c682e9009025aea6fae3dc75409cabbb955b560d0372"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "cakephp/authentication", "label": 1, "pair": "e69463f945f9a0f4", "unit": "799a4f5a5b9324e86c90752f0239f059984f44e8ff1ec0dd518d0d9413e34c66"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 0, "pair": "71f56810b8ee6947", "unit": "c659c0e717b7dc1084e1178288f6c9b2e87fbf0ef518ba3bbaedb0fbadae15c1"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "cakephp/cakephp", "label": 1, "pair": "71f56810b8ee6947", "unit": "c8d2e035a6099fb775c8b4ecf00196c1488dbddd9eed3d2d268a4f1d2567b194"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 0, "pair": "768fc9db8a75a351", "unit": "adc6527469618f72b1dd7776814717cbd0056a0c625e05bef74199171ce73f1a"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "cesargb/laravel-magiclink", "label": 1, "pair": "768fc9db8a75a351", "unit": "da47663a57e370a75cc65b810b8a380aa26d12dcabb1d9b3c02641398ff7d348"}
|
||||
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 0, "pair": "fab4ce020049724b", "unit": "b9be90242be627aa46462143ac85940a906232ebc0b47776b5729475b27d215e"}
|
||||
{"family": "path", "fold": "outer-0", "group": "christopy/serve-here.js", "label": 1, "pair": "fab4ce020049724b", "unit": "ece12ea7d87b4a12c4da30faed9eb530b51fc879c271b68ba5304b230cda8e79"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 1, "pair": "9126ba83905eb8b4", "unit": "d8dc50b7bb40bbcfc3f3d4f1f8d3a4d17eb1c662d5f759efd18656a482dd2e32"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "chrome-php/chrome", "label": 0, "pair": "9126ba83905eb8b4", "unit": "f426f9307621b3f2d6a8d1669cdb4c19e962b94e34fbb394d4660cbde5921c88"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 0, "pair": "7ac5e21d9d8f4c47", "unit": "1d1efa2fd7323c264290d4d314c957c853e1c7a9c877a54d6f03ab7d2af2ca86"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "commenthol/serialize-to-js", "label": 1, "pair": "7ac5e21d9d8f4c47", "unit": "b5dc78c7ac0b76803dc98904142c1139083fb9daafb5bff40e7b5c3f14ee03f5"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "0894a7212280bab755cd9d93b1f0ede2c51e4fca76b76e5ed9e24ef8f8cf0b9c"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "02694b5fb85e9e52bf5eaaab4e0b628f1f44ff3a8d5ad026cc10afdbc4222eaf"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "contrast-security-oss/djangoat", "label": 1, "pair": "", "unit": "db9a3e888780501dd3c2c125af2f9d47f68ebf02f0e0db8370b2ea09c6f9d070"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9af96b099dedcc4005e5fcd19ad78f73864e1cfbafbe6953b4edb6260957943d"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "contrast-security-oss/vulnpy", "label": 1, "pair": "", "unit": "9e7c2a786068a936a6c664cc229d6f39cea1fe57caf0806306f0cfc44515460c"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 1, "pair": "07d1dc4d299a7700", "unit": "0e664e274dfb438d2f721edc81f8469ac48387dbd5c1b43973cd7e651db46db3"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "cps-it/mailqueue", "label": 0, "pair": "07d1dc4d299a7700", "unit": "da55f60378c67e91683b9720f2fa0e6659e28c7b392c050278b40713e47c84a3"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 1, "pair": "749eee05a507fe93", "unit": "24fbb1dffcca083382a23e48db232e0d83bbbf2ff2b325f0dc0f4c72dd1a16bc"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "cstigler/node-xhtml-purifier", "label": 0, "pair": "749eee05a507fe93", "unit": "640e1626f8d7d9d5c2478ede48cc7b2780370bda98ae114bae102bba892da90c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 0, "pair": "e36bbaf8f3cfe65c", "unit": "14f7c65fb5621ac5e2068c8695ddbf482103e4320ec3b36886f6e1f08cbc312d"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "czlonkowski/n8n-mcp", "label": 1, "pair": "e36bbaf8f3cfe65c", "unit": "c96dd379dec2ecfd99051d7a20e6ef5601821f93c46eb34ed087daf5fbed2d47"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 0, "pair": "b81c8c63cae892f6", "unit": "5982fe7fa1ea6bfc1c8dcb8c71cd4410de475d25cafa324147649f26a1ae1324"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "danger/danger-js", "label": 1, "pair": "b81c8c63cae892f6", "unit": "59916deeb8ba14922eebac61c9eac122e15d1f7590ee3cb1b97f019585fb798f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 1, "pair": "bab3a167ab5ca505", "unit": "5ebea23aee96c80fa0d2d9239964d99614613c4ae36f709811bc19e61ebc720c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "danielgatis/rembg", "label": 0, "pair": "bab3a167ab5ca505", "unit": "9bfea8d7a6ffa6d79d16f77fc355fc1bb605189513ebad561ee523d577a1498b"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 1, "pair": "5a91049be21aa611", "unit": "1ab9936627aa59c3a60ea44074ad44b68aea4f9c04f14c1949dd9c77ca13ff0a"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "datapizza-labs/datapizza-ai", "label": 0, "pair": "5a91049be21aa611", "unit": "8df72b9eab3f2fd858c40312d0a806cb67c620bd62873826c8732ba8795d9cfa"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "", "unit": "115d87f37bf1d4ab91fa3010303e81b127f84b817268b60b8fd4b9934efeae4f"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "2b8101c89c839614", "unit": "7bb7d858a670fc51f81e08c141c2bca3b8b7524254ab679e33b22ce79536e895"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 0, "pair": "c23acf4f38945093", "unit": "82bdc0b6bb0c19e342efa63af0c9014a607461d0682cdd14bd5fea33516fbd50"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "2b8101c89c839614", "unit": "886285661b88f14e00ff8e863b31cf7b7afb303ad4aa4aa13b1cffbf1272d5e0"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "davila7/claude-code-templates", "label": 1, "pair": "c23acf4f38945093", "unit": "bf74aae5a48156f183426a80cafca6da219ab13b2ae4b40fa88fca75acb72980"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 0, "pair": "d7cc37a3a54315cb", "unit": "4aecd43c6e5b81daf7aadd6a6e22f22c17c08b652a2f5121a49320ca1057b186"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "delmaredigital/payload-puck", "label": 1, "pair": "d7cc37a3a54315cb", "unit": "88123ae860be2fdc28fd6a289cf26c3e0af0689f455cf8c15098e5dc76e5c8a8"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 1, "pair": "5011004f06f1e217", "unit": "6c48682d1b02ff2559d92346d192cc9dbe6a0bbe2ab2c146653110420105ff1d"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dep0we/atomic-agents-stack", "label": 0, "pair": "5011004f06f1e217", "unit": "d3592260108a838246e0cfd5ab6d48bceb0ef870e223565745dcfd68f1826840"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 0, "pair": "81caad96b221c2b8", "unit": "3c2dc720ed159e55629ba737665d93859def9790427289567a513c83c133e236"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "devbridge/jquery-autocomplete", "label": 1, "pair": "81caad96b221c2b8", "unit": "e8b9595a320776c8e663fd0b168193c1e64e36a9052784d1219f2c4a91d647f2"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 0, "pair": "f9b6599018ef3b60", "unit": "7558d6ee39a311c5459e35fe6f7b12917b7f843b21ab1fbb5e485a07f53b7d78"}
|
||||
{"family": "path", "fold": "outer-4", "group": "dgtlmoon/changedetection.io", "label": 1, "pair": "f9b6599018ef3b60", "unit": "b1a6cf450a074788ffca2bebd880435abac2c376e89cb328d7a22d277fdccad8"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 1, "pair": "eaf2f03db2c502b5", "unit": "38dbfbf745d7cbcc52c6fff83248d84663273fe6a1f1b014eac7776c4bb8f074"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "dicebear/dicebear", "label": 0, "pair": "eaf2f03db2c502b5", "unit": "a84d848ef56b72fc15872d88acaae1743c9cdf88095d13b1869abbec6a0d8a8f"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 0, "pair": "7df48bca2c9fabfc", "unit": "7b76ce8f55d61071adb367018cc3e50f04a05d08c7c4919c3806fc1cc8bf41a7"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "docling-project/docling-core", "label": 1, "pair": "7df48bca2c9fabfc", "unit": "c82ea0adc9c8366aa2b77ce73772038676fc37ff50879e873ff397958fdd7ec1"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "7f02a09b8d8f2321f2cdade74c2fa1d52442e006149a749a8be768daaa62195d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "", "unit": "9b5e439943ce5c2b04d6201b7a843634aa1b13b726813c23450ff54ce03f3dd6"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 1, "pair": "5b836889b4da4921", "unit": "a84c2ce4a4eadf265d4481ad7f6864edf1d7e3084c6ee32cbbfe454bb2946bfe"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "dolevf/damn-vulnerable-graphql-application", "label": 0, "pair": "5b836889b4da4921", "unit": "caeed7f097a3311fe2e84efd372fe4b4c140f63a80a9f07ba0b61966dbab15e0"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 0, "pair": "ac526f457067fad8", "unit": "50d8bafd5189fab0429bd4135d9ed400cc2fc14706226a2de98d65746ebfdcc7"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "electerm/electerm", "label": 1, "pair": "ac526f457067fad8", "unit": "9532b70b0df76a34170b57c74deafaba54d5d2f16a729c39b0dfd3eb2022d899"}
|
||||
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 1, "pair": "c97a971a18f85244", "unit": "95c80c503ed43fe9a0370f89ea935d47badee6a9625548b9dca78e04275003dd"}
|
||||
{"family": "path", "fold": "outer-1", "group": "elyin/gemini-bridge", "label": 0, "pair": "c97a971a18f85244", "unit": "e9810872beb9c158f54048392142c2f701314f854d6dfc748553ca36ed6c08d0"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "c2a2c2d85114a97f892fb1e5500fbd98622457a8c46451e1f9e958ab08a40318"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "erev0s/vampi", "label": 1, "pair": "", "unit": "29d8be8206c88ba7efd86b56a7dcf8b655e33b4a792c2d18b028c92c6252daab"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 0, "pair": "6a4e0bd705331747", "unit": "6db80439d89b54f60f836c415dd403fd09d21cf630da81435d4d2f9cef5d883c"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "esphome/device-builder", "label": 1, "pair": "6a4e0bd705331747", "unit": "bf257e4f0812012be330fe119d07d29253d291161f435262787f491762c6bfb1"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 0, "pair": "f96f0fb245948915", "unit": "0954e6931373915c204b9ec8d2b8ae6297a95667bfd41aadf9035625ad93967f"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "etaminstudio/prosemirror_to_html", "label": 1, "pair": "f96f0fb245948915", "unit": "5e09cdae2504e834d591b6f92a808427f07550364b5419899f7e6f76084fc377"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 0, "pair": "bf11f138fe39675e", "unit": "56f0d0b156992af56141d6368fdb0c5376fc7a44a03991d754f97202b9d77008"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "ether/etherpad", "label": 1, "pair": "bf11f138fe39675e", "unit": "8271af5017894167556893da2070188060d06eebc87f27db45c46234c359631d"}
|
||||
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 0, "pair": "9d937c2dffb4aadc", "unit": "01ac12bb06e660d0aeef8b1e26dddc5c99d52cc333f7af6d1981eee2818e64fd"}
|
||||
{"family": "path", "fold": "outer-0", "group": "fastify/fastify-static", "label": 1, "pair": "9d937c2dffb4aadc", "unit": "9612edf0f927048335e83e9925f01be9aea9b772bdbdc568ad9cc0a17055662a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 1, "pair": "d4f648048e45f2ea", "unit": "08807c8110b8c1b8635a6883c229fb91f2b9566db8b6c252448a0db7b2ddd47e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "feathersjs/feathers", "label": 0, "pair": "d4f648048e45f2ea", "unit": "10869c931053bf77ecaa98a47f56dea27d643d1abed2a4758bcf48a78547ccc9"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "662e1f36c230d7f1", "unit": "0b73332b25413902c26d5b5dee6767465eed99ff71a814582988ffe0d8ec1a30"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "8c8e46d0c7d361b784baff9d70f38744b2805ef1ee0aaacaa616bc20a2c6f32d"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "aca938682c196fd4c85b39b89eeedb97c35f6a5a017651fe9c03c5b727cab7ea"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 0, "pair": "", "unit": "cec00d1509a129364da6ab8537fb3f433fd847e26d073c90311ccdaf7a1b39e7"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "fecommunity/reactpress", "label": 1, "pair": "662e1f36c230d7f1", "unit": "e0f58a7ac5627de3e7b6cfe5add41d18b0f64b998c1cc788d42eb2cdbbc3fe41"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 0, "pair": "2c251a645094f389", "unit": "0d41cc634771048f72d07c4aa9feaed669d0bfe3844d61dcdc525a0f899617e2"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "filebrowser/filebrowser", "label": 1, "pair": "2c251a645094f389", "unit": "ca8288d0b8b94dc6b7e4c681936ea610d295fe949cba5f64f09b6de134ef9875"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 1, "pair": "2ac5ec5011615c9b", "unit": "2f427f403af3cb58d776437736c2389805543413fdb91aad98979158c6d00de3"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "forgekeep/nebula-mesh", "label": 0, "pair": "2ac5ec5011615c9b", "unit": "cddc8c5bed0841b80e7b20d0b378e858f8e4ac37fd7b890ce76611d989857f92"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "frangoteam/fuxa", "label": 1, "pair": "", "unit": "0c7830c2912ff285855d8a3c8e9b47db4ca4af840baa66d4b5d3f4422f46f95d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "5383a8c2a78fabd3d17a5b3021125953b5ebbee61f8900c30d016c56b1a3bad6"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "360d8bfd2f116e98", "unit": "7c1989a4ec3622bdaebcf1161de850f22cfed041cb38830ca1ded4fa99d40466"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "ec8c706cc2b20f36", "unit": "848688dcd0fe8d46e6762cc8aee1d82e8fad8a2a044c94b547aea11049c85bfc"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "ec8c706cc2b20f36", "unit": "983cc9231f91baee4e2d626193f3b88aa5b09907f6faab870949bdc9c336b816"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "e4ad86416d5c6668", "unit": "98ff1dff2afe80709b8e61a9c95cfeea12b2110c13b7120478268b4762e2c992"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "360d8bfd2f116e98", "unit": "a1a825d23e34757c8e961e43ae791253e62682e130acdb846b58118588006a98"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 0, "pair": "", "unit": "a7fa4207508b3989baabc7a1f0ff2ea5229b366c1cb106e7fc945134b8b106c8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "froxlor/froxlor", "label": 1, "pair": "e4ad86416d5c6668", "unit": "ef5c53f3114d0057e694df13dc07cbcb532c3b0d59150eda269b87b021d7a6e8"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 1, "pair": "9ae575048504c690", "unit": "176acf077fd58c3e6f940753179ffa3e8316661f355d94acb5e0345b51925ca4"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "getgrav/grav", "label": 0, "pair": "9ae575048504c690", "unit": "228b34ebc2090b5239c7821d28366ddc663e7118d015a79b45dca6618d5490fb"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 0, "pair": "d6edb5496b3fdba2", "unit": "9acb8017ebadc480515a664181c427f37359edcc1dc6f344f368a87e0c191b3f"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "getgrav/grav-plugin-api", "label": 1, "pair": "d6edb5496b3fdba2", "unit": "c1bfe63148bc700c812fde95ad9f83d456e4fa04e10c6f5394072a08f73174c6"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 0, "pair": "27e1c8e8ee93a892", "unit": "47fb89b132d6de48674c37965579e727d922bef1796db70f8b9f862ac47cb9a5"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "gitpython-developers/gitpython", "label": 1, "pair": "27e1c8e8ee93a892", "unit": "79887f0ca8b05d6599d55167d8a9faf67bdb2254d17c143141da44971077002d"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 0, "pair": "a3fdf9ce3441ec86", "unit": "653236018929ab8534c27704a20ec5d358047bc66e451620d2ac32081c91d333"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "gofiber/template", "label": 1, "pair": "a3fdf9ce3441ec86", "unit": "b70ca2c43f5929e7b885bd728349e721fd85d7605e2f722a7e8cfe0f0d477b0c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "groundhoggwp/groundhogg", "label": 1, "pair": "", "unit": "92400e83890882a66ffdea1a405e2354a195442e176d85fb707b0442c44d45ca"}
|
||||
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 0, "pair": "f9dcf3a0fb0f0a14", "unit": "7ca6dd4f673976b000a657f885af3463740419ce41e14f972af0c5483cb75a7d"}
|
||||
{"family": "path", "fold": "outer-4", "group": "gtsteffaniak/filebrowser", "label": 1, "pair": "f9dcf3a0fb0f0a14", "unit": "86c1db3a5e9f843d6546146449171e0378ac61d6c8f44c6dd72fd379cdbb188a"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 1, "pair": "3e71f79348a6b770", "unit": "200250ebceadde4cbdfec2802a4f9fbfa220dbd7fa4c2dd5ed717fb1f3ea7447"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "h2oai/h2o-3", "label": 0, "pair": "3e71f79348a6b770", "unit": "b9faa5007c8cbb67ada3f1f026ca59e0ace6fbb9c3d1f1369d4521a7e323e7da"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 0, "pair": "8972a0f56ee57477", "unit": "6b987df2960828dcb1b2b3f190a903a40361d58117d63321d671aecc6c2cb570"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "h3js/h3", "label": 1, "pair": "8972a0f56ee57477", "unit": "ca3fa87dd37435c01a7121119912c1a1e261259c11805c0b66e7d4b315640945"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 0, "pair": "6144baeb491f4a56", "unit": "f5c22ce104e9593e5272b6e8b3c6f3b61fa4578552ccea80e6021ee90bdb3ffc"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "handsontable/formula-parser", "label": 1, "pair": "6144baeb491f4a56", "unit": "f687e7e95dbe47f602a1e0002a44f94fd48e5b1c33f77ecebc6c2d814fcbf450"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 0, "pair": "2ef4471c66e5e4ab", "unit": "ad7c8e368fe1699a044e058d5c10b41bd9fc0f91c3f3c384c5dac40af39e71f6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "heartcombo/devise", "label": 1, "pair": "2ef4471c66e5e4ab", "unit": "d888baf7194bbe7158cb3932f58b47a9a235351d388d10df82a130ddd54c2d46"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 1, "pair": "5628c54152216646", "unit": "47d081c9f6060218967902f694e61c88f64f57c65bec6fc3073afeb0b77d117c"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "honojs/hono", "label": 0, "pair": "5628c54152216646", "unit": "e1297bccf9c3606e0b41d762a69ff5d0a33ae4f7f1506071af6b9cb4a177ff56"}
|
||||
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 1, "pair": "e52aa814c36ace1f", "unit": "2d6d74364f1982e91f27f1c995b8699df1fc8f8dce6e1e2cf274bf01b6e9494e"}
|
||||
{"family": "path", "fold": "outer-4", "group": "hs-web/hsweb-framework", "label": 0, "pair": "e52aa814c36ace1f", "unit": "4727471fea2cf0a92d43b4b1503f1339a7dde87f872ba8064dacbf855c57705b"}
|
||||
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 1, "pair": "a07929b3fc1cfc28", "unit": "0637029194bd7378237b4dc944dc5cf9f2b63db9796f27410a74a9d567b66dc4"}
|
||||
{"family": "path", "fold": "outer-2", "group": "huggingface/datasets", "label": 0, "pair": "a07929b3fc1cfc28", "unit": "ffd286c8588a77929639a89e4a742e15306b642af14aa906c721165df804ccd3"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "6548e1527e82bd99", "unit": "5542613f6d2eb9421b8213b8bb864894f0ecf9fe64945681019230ef0d75cfaa"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 0, "pair": "", "unit": "6c2adee34e9e8384535ec1073eda60e2ab528ac77c36747802c303e1f5e9ffe8"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "7ce70e9b20107cc7ebb87d173da765fc53ffd268e521749a06a9a1f38d746845"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "6548e1527e82bd99", "unit": "c3e54975464323e58ec96ee4d83e7bc08baf94553d553a8dc3fb28cd9c70ff21"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "ibm/mcp-context-forge", "label": 1, "pair": "", "unit": "d3173c1e43eb6402a71ea2dec1592913dc078a2194ba1c1d509ab129a3180f6f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 1, "pair": "0924befc49c7bb18", "unit": "e7e4872db40f0420547ed622fbdd7a072b64a7ba466d1ccfff68e691cee6ad99"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "internlm/lmdeploy", "label": 0, "pair": "0924befc49c7bb18", "unit": "f7d95f43fb2df20ebc4c37fceabdd00ffd8159f404ca384481f1764252382388"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 0, "pair": "d4ee6cfeba71e398", "unit": "e79582de3031bada2617a0667ef7ff5cd7183d4aa885f55a665cf0ec88d17624"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "iximiuz/node-diskusage-ng", "label": 1, "pair": "d4ee6cfeba71e398", "unit": "fef35272853e7f3692efd54daa01c12862953352fa6ccbbb5e3f7fea70c63d78"}
|
||||
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 1, "pair": "772e18821701d60f", "unit": "89a0496c7663bbfd67d660c9f481c12420da2dd1bdb29dc079735ee7a0a49626"}
|
||||
{"family": "path", "fold": "outer-0", "group": "jarofghosts/glance", "label": 0, "pair": "772e18821701d60f", "unit": "d3241ceaa50a7d23a4650e6a2a55372b08ec96b35b0bede844bc40e16e818ebc"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 1, "pair": "250de1302c7e541d", "unit": "5d03535f2df2acdbc319d9e6b20da65d51f3e810b80f3c8f89b5074a971a3f1c"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "jas-/node-libnmap", "label": 0, "pair": "250de1302c7e541d", "unit": "78f977cfc096e34b4eada7f798ad2973ef45728d0d01bfab6ac9e2a3257827ef"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 1, "pair": "c10bfb006f2da55a", "unit": "1ea4b86dbd84581b1d7b7cabbab9f28c8b213416a60778879666ff42d23fd7f1"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "jashkenas/underscore", "label": 0, "pair": "c10bfb006f2da55a", "unit": "7647021f742da24c660b2d407886771786e957076c4cf336def560a570dd8778"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 1, "pair": "a73421efb106ab0a", "unit": "1f6f207a758582de0c0e6b331b19fe119abe18727d45ccbbe09aabb795f3a292"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jellyfin/jellyfin", "label": 0, "pair": "a73421efb106ab0a", "unit": "32fbdad39eaf5177db4ad0a11d53e785b30bc7b2376ad9d460f29a08429bd0c7"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 0, "pair": "0458e3fd444409f9", "unit": "b521216bb4106738b3571ad3902d08f6230aabd88cf3a8f9f26f1c8ec01304cb"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/cas-plugin", "label": 1, "pair": "0458e3fd444409f9", "unit": "c87d2871907798fa2e73d31381e9d63089b0388033f096d247373fec07c2cb36"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 0, "pair": "6bdccabcf5b9fdc9", "unit": "3d6ddca5eb264d88411b86755909f1d2abddd09c129cabd54c96ff26b4105316"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "jenkinsci/google-login-plugin", "label": 1, "pair": "6bdccabcf5b9fdc9", "unit": "e055b898802279bc63204cc601e29f7f1018efad7228cab0cce00ea5866ecc7b"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 1, "pair": "91bc832e70680c92", "unit": "1060c63ff0fae3af567448f3d4c60d9c159b61ccc358964243eb152a9ac38865"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "jenkinsci/mcp-server-plugin", "label": 0, "pair": "91bc832e70680c92", "unit": "369dffd1e730d7997813adfbebf1481cc68987cae0d0085e47b91a26d5330cf8"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 0, "pair": "137fb3496a3a75f4", "unit": "7e8b6c484999229b23d50d9f5a4c4c5b2a8d954a5d64230981b1c6705b760fbf"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "jihong88/suneditor", "label": 1, "pair": "137fb3496a3a75f4", "unit": "c461154e4dbd74e35a4ed2e85de7b076f0582beeec459be28104c38f616c7565"}
|
||||
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 1, "pair": "a4144258dffb3b8d", "unit": "7ab301e1614979f44c353e665540582493f78ef4bfc41209562c19b900710f8c"}
|
||||
{"family": "path", "fold": "outer-4", "group": "jknack/handlebars.java", "label": 0, "pair": "a4144258dffb3b8d", "unit": "ea95e8387dff32ded9dc7b04c11a76a545b4e2ea6db0076b9b13971a59904bb3"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 1, "pair": "858330e01d3cf9bb", "unit": "033cda2b7cffa5a5fa6abf0b7f2cd27ba78d916eb3cab8c79decccd8776f4a3f"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "jmespath/jmespath.rb", "label": 0, "pair": "858330e01d3cf9bb", "unit": "d44c4555a5ae19aa17b468ec5cff56b33a0b221df52e04d8abcb656399499dee"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 1, "pair": "c6a49339d4f0c798", "unit": "dc2e4cd5bd963643702d2b832ae170dd25744162e4076e36687256a09e69d089"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "jooby-project/jooby", "label": 0, "pair": "c6a49339d4f0c798", "unit": "f83cd137b85e651ca5299c5a42176b797ed4964cbef31418c96d63c52fdb65f5"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 1, "pair": "4b6526d132e9ac86", "unit": "8ec3596444f3045684242840e9a1df19879383f2733aca8f8af6f6cf8a96d388"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "jowilf/starlette-admin", "label": 0, "pair": "4b6526d132e9ac86", "unit": "f3d39162fa14446b207aabe7bf3c913dc32a9637ac83352ea5cb2fdbb43720a8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 0, "pair": "80abf1d5c4a3f6a5", "unit": "3df3c6d9941352dd2861427da56b94ee335f308d0a2a27c6f745aef60349c80f"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "jsonata-js/jsonata", "label": 1, "pair": "80abf1d5c4a3f6a5", "unit": "4fde61a570906e658d9b14db0e5d1f5acc0c7489951421bff2afde25033ff73c"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 1, "pair": "5ec37c0f475001ca", "unit": "5d8175fb21a4649bdc8730358e9b87adb90b2aafc4da5fee203829663e4f46a0"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "jsonbored/gittensory", "label": 0, "pair": "5ec37c0f475001ca", "unit": "af845e9e332555096dd350401977b2f07015433f3db8a040891d9470f3f9e355"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 1, "pair": "68d44ef338ef3470", "unit": "3020ac6c3186d7974953b1c4da65b80141b81aca5203436ae7a2d7aca0870944"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "keras-team/keras", "label": 0, "pair": "68d44ef338ef3470", "unit": "8fdb85a6bfec89f0715ea81e83fd91616f92c59954c092e79272154f202b7488"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 1, "pair": "c74525e844650de2", "unit": "2e444ad34ee5f47d73f0cfad0a10c70363337b4ddf9067ecd1b8072d8232652c"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kevin1024/vcrpy", "label": 0, "pair": "c74525e844650de2", "unit": "c492d325f5ff82b5f817aa4be1440cec8595054dd0719e07a58ac5501b2ae7eb"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 1, "pair": "111a5a80a3abc2c9", "unit": "c7ae403742450deb2bd051ad1231121d2117ba56fa72a038c82d58d207cb3870"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "koajs/koa", "label": 0, "pair": "111a5a80a3abc2c9", "unit": "debe34831afca6f8220b9952036a2d4570babdb5e321a2f8dbaec661a016bd50"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "9950b85412b021ebabb11eea4c490a660ad380d14c673a87c3fb568ba4bcfac9"}
|
||||
{"family": "path", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "656df5ff8e69dcdb1f9ba1ed52d734966eb29773bc745587483a355743b8eb5a"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "91652a4e776fbce70cde9e4a99c6da5e17da7f8d6d0404867c61eec04161b6cf"}
|
||||
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e53bb3a217dc4737e8dd03671de4f6a895599024070a985a268c65a034295e7c"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "3d7aba22444811a337abcad0e4fc62990bb704d3a10e72d893342fa94699026b"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "5f18a3026a51f7140b7aaa3aa8563e8b6fdda6065d95e19e25b1dac4b9ce0e00"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "0fccc306ce80cb4cbfdcde0a2e56d637bcefa9dd30423d6c54a4f77860afe740"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-logistics-dispatch-fastapi", "label": 1, "pair": "", "unit": "1e2eaa45661345f1b6759d968d7696dd0d96d6fba1350a00e4d0a2ebaaa2e391"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "8b9a8ff0e32a327df75eb8a1811464d5f34690079d5930139dd917063bdd2809"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "245d0e939b7c1ccf098c798492009e3a73ea17400fa613dd865cd369d2e7f383"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "653c48df493f31c2d01c973d1e379a180fbe6a6edc83a500bc62509112175f41"}
|
||||
{"family": "path", "fold": "outer-2", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "3719fe21d45b50b35458bcd2e263e4baee67adbd3c98adf0eaab6bc6183c5651"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "1209a6cc8c1548ed83d4a9a09d6a27f706a23e69cf0c261fc5e3a27536a83306"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-claude-code-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "362c90495dc468bb732c71f62d71ec308854161de31ad18d2a870056d947ddb6"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "b52e53fec9707b2b442c4dc7a41d376cf76946d2c45c2eb600c9477fc6baebaa"}
|
||||
{"family": "path", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "477e3b90b92e622888dbe0e9d3a23b31271ddfa3104a5ab326eb9e9e765aa01d"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "660c63c4a73bfe64088ae3249e40d9c5d69a6f8a505ff671217c5708a345c5b0"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "4d60c386c801bedb61c1461fced29a3cfb50b09d6435e8617c60398343719d93"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "f1fddcc1354a29bef55610ee2b73eaa440192f2f18a0d22d08057b9d1e1b61b8"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "6d9886c38221b2f9aacf83749218ac5ebc354a6fcc4170e6c78bedb92b88296a"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "5550fec4a81c4c9cb22d7312d8a63e1fb2b03c4aed0c1f316d44c1eb27511bac"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8ff87dd04140f0137ea1891f220b1276156ce10a58be5a49d0c8e07be531ffb2"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "f24b19cc0f67247eccb274521a62741fef8f4729f0ddc9b5c1fa561a4947182a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "83523c9d83380d4cdae593a593d361493e00d18e61740f84e762aad41b991e18"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "c6c9bde6630678377d250701073c5c223c7892e6ff8f2f8e0293dea22c5b5e7e"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "b9b15958973c83f8d7b58372ab5ba58eb191c400125579d5ae8c5b52d5d1368e"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "b95ec12e4e126ad558e944e2c260116ed6a63f5f3ab97fb5cbcdca75a931ad73"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "38a5545ac9ef67e15db720d1396a2d5915f9d06d6f4772012e5bc95f33770376"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "c8bbe9be91c976be97b7ca387048a7d4537d9f6fcc6cbad0a541d1080629d5a6"}
|
||||
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-codex-high-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "ffacd53f1bd75155e63b318fb9cba27ec03e754841e1eb39d1d4fa0b2c3dfca2"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "8fd920175fc43b91b4397acffacb10ab8f7fd0ee38bcf2d2f3ed06a3fad0dfe9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "86c9598ebe3dcda261a69ae545513f6ce73101f3980066e4506ece99bebea84f"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "e5190fcc57505116f79a31c51aed69f8a1eea13a8ae3e7265f44c72418277ef9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-codex-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "2b807ff54e6c47e941124c16d0c27c691cef54f2203060a3a6ce812d478c5261"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "401aefb4b4791301ace2d770d321c6b26743b0aab7b96f8b410254c704be6d2e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "22d186c93498e21418b6c1d0547a73374b13041d8afc89a80d9540fa658b6b42"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "dbfb6df14fe37f8c0efda5f16643297fb0550dfc5ad5805d4e4b844d967773b2"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "f0da8cfe4ac1e586aa7903d72e67b44500bbd5306d72b4b342d10613efc69222"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "kolega-ai-dev/vc-codex-seeded-v2-support-desk-fastapi", "label": 1, "pair": "", "unit": "2b8fd73943f456da266434470333a7775f55d9a41953e97a000fdaa3257b6a30"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "a612091f452e60f28f44eefec641175170dc5db04456a39a099cbb13c50b4d2c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-crm-saas-django", "label": 1, "pair": "", "unit": "4c6600facd83bbf234a32d9e28ac494fb516df6e3d9e9dc28d720ff35252d067"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "fb0ea8cab8c9430d22d1b0fdd65a69146fa95fbff94e720b4861b96538d992c6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-education-lms-django", "label": 1, "pair": "", "unit": "e81f91a56058ac1473579767abea57064c53d7d007797a854314814e3c35ee74"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "5851ecd96a449f98b05f243bfda0a7cf3af8612b27f53c414ae1f4ea67242a7f"}
|
||||
{"family": "path", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-fintech-lending-fastapi", "label": 1, "pair": "", "unit": "328621ef2a94632e5d37ea6267d17b766233384cde35c07e29a4eb672d40ba28"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "04b2eae2128608dad460ba8275142a41a44e32c4243a29e2a6fb2c9ebf3d8f1a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-healthcare-clinic-django", "label": 1, "pair": "", "unit": "33337002ff5f5a189610b042522ceea254c9d107c1f24771b4f616a0f96f636e"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "02c09a783cbf058856fc2b448406f3ffa6fbfbacf9b70f4e5e22fd741a5c7699"}
|
||||
{"family": "untrusted_destination", "fold": "outer-0", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-hr-payroll-django", "label": 1, "pair": "", "unit": "956260164cb21a47c499e5598491031ef303418943f5096f225a87162b588574"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "8bd8c5aed69eae3019cc2d3654b36bdf2b31435e6a020a389cd103696a2f874c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-legal-case-django", "label": 1, "pair": "", "unit": "1d8397ef4a9fde7b48e2e1e871bba0f960a802cd7823ecfc10e9dfaddb955ccc"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "c9db92476032eb4296b950a1eff8fa4aaa26f11542a01a5cf10c7b2e32dc6433"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-marketplace-commerce-fastapi", "label": 1, "pair": "", "unit": "d3e0d0f9b0f4b723080989b5486395cda0a5b186a674022ea70e7fa7ab561323"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "44ff3b3e61c02744830c4abea872821ca7e0f94950c082740c5c48ef13604e94"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kolega-ai-dev/vc-kimi-code-seeded-v2-property-management-fastapi", "label": 1, "pair": "", "unit": "4103e979a99233c3e71fdfe9f1cdcbab34a2c60b5351b75c5ff5dfc287b962d0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 0, "pair": "f73ecbfda2bcadef", "unit": "53e596dabf90b2992d5b3e50394791859b393beedc8f84e00484e37498627ff0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "koxudaxi/datamodel-code-generator", "label": 1, "pair": "f73ecbfda2bcadef", "unit": "6daaccf64706e876dd505c65a62aeacc1357f3470a5cef7b1ac874032a6fcf99"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 1, "pair": "6768bfae38c6859d", "unit": "f376519889a00950f5af67eb15c639306ea70da7235d3be4a94dbc19cb096cf5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "kozea/weasyprint", "label": 0, "pair": "6768bfae38c6859d", "unit": "f3fea4e334f49ef6fe6b17121fef553eb8e33a9ffa13e6d6cbc4c448da996a7c"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 1, "pair": "d670253fa209671c", "unit": "1aeebe2e440bdd70525c31f93180f33be93f820db5ba46a2e59c371ef83ee794"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "kubev2v/migration-planner", "label": 0, "pair": "d670253fa209671c", "unit": "b11c8a8a24d3809c0507228568fe7cc05ca0a22ed28cb19019adb13cb619d63f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 0, "pair": "33260f3e6c8ea6d0", "unit": "7ca409c7dbc1c38ff005624e212efc6d424cb421e27b482dc8dc0d6881f4c11a"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "kylefarris/clamscan", "label": 1, "pair": "33260f3e6c8ea6d0", "unit": "fc364d7ed77c7c0508da00e5ba847ec0792f3bc373d8b4297747189ef42d1ae0"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 0, "pair": "32ef8148a2c08f58", "unit": "2fb2c9314a483787168a450f50430721f9527f913aa0eae07298d5334cf4f137"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "labs64/netlicensing-mcp", "label": 1, "pair": "32ef8148a2c08f58", "unit": "78dd291bb2ef83ea3a87948c0ac0027c6d667b9044093e766e2bc91bebe83936"}
|
||||
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 0, "pair": "54586b5a24655adf", "unit": "2ece75a0240ed36a811c381841aa6d440af2b9774ddacb1d7a31a67ec267b0b2"}
|
||||
{"family": "path", "fold": "outer-3", "group": "langchain-ai/langchain-nvidia", "label": 1, "pair": "54586b5a24655adf", "unit": "e60f6b788a196db9bb671d621189b79a5526302a6d09c2396d67fbe0b9ab3c24"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 0, "pair": "474f96e2a9b3d202", "unit": "4a5e8e8ebd7ecb0e252280af0cb38a1d7794757728b912525322ac26df5ee223"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "langflow-ai/langflow", "label": 1, "pair": "474f96e2a9b3d202", "unit": "6a888540443061d3b37c06f0cebc70cffdfa70e0aa940626e86cbff9829310fe"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 1, "pair": "a09692dc94164db8", "unit": "139cf84e79c4efd026a064b2ca0a3d7ffff87b54eae1275a07c0d69ef4981e27"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "laravel/reverb", "label": 0, "pair": "a09692dc94164db8", "unit": "828e88b656245a1d0865c79b2ed023a49e78607a4ac81d70ccd3690f4419d2d1"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "", "unit": "6cce78d7569ad8db7ca0a61d7078198f666502d26ea6c0afb03173a8a4bb16ef"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "82047c0513e43a83", "unit": "6d2dbed1ab1c9d8ac259dd4227be69641fa59d52e816a8dc547ade01d0b9b0c4"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "98ee23cd6bd31cd3", "unit": "ab4bb39be535d70cd95e9035bb958bfc7418576a6ff40a47551894dd140317bb"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "98ee23cd6bd31cd3", "unit": "b70068cbc646bd01d3e10986e002f648a74e9d798900dc59cf3039b749c95c18"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "ff168d95d8d9496e", "unit": "c6ec9478c125ed780d7013c2807f2cec1878f44253667dfed319c28cbfd17d4d"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "b1bfef46dcd3b9a5", "unit": "cd4d095d8380a2fb7ca02fc6abdbae5249cc06455db2929875533f4aa8b886dc"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "82047c0513e43a83", "unit": "d1bade217be8d345cf5d0f0442ee31924187acbb98779dada95274a77462f696"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 0, "pair": "ff168d95d8d9496e", "unit": "d561db250b892b91feed7ae877900e1e25963444f73c7699576e010f06da73cf"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "learnpress/learnpress", "label": 1, "pair": "b1bfef46dcd3b9a5", "unit": "ed78fd5a7ab30bcd6e8ddd27c706a19cfd89750de4fbf81c0b567946eb94db83"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 0, "pair": "25f75d77a669b54e", "unit": "0f2825568461562f2a915a4f16ef8a4931290fd20a9801470aeb12677e66e3f1"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "lepture/mistune", "label": 1, "pair": "25f75d77a669b54e", "unit": "bf22bb44bcff4a5baf5d550c9d3e83e5f3ca607475fb51deaf4fb6fccff3d73a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 0, "pair": "47069df6bfb3dc00", "unit": "444c1f41fc23ad656a8bf98dae58b36d0f1daa214ab030682f26111dd389e58d"}
|
||||
{"family": "path", "fold": "outer-1", "group": "linuxfabrik/lib", "label": 1, "pair": "47069df6bfb3dc00", "unit": "c0c8b5cca070673ea4b4bb8f08dfff86b7781780e300720c4051aded035052fe"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "cd80750447cf7af67ad809279718d98106c71705fef9ff7254728bfff4ece5b1"}
|
||||
{"family": "path", "fold": "outer-2", "group": "linuxuser255/vulnerable_python_apps", "label": 1, "pair": "", "unit": "2ce605ab5ce7ad3a1fb983cd6bddf3924e5da3010d2252a0a4640aab4e5921a0"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 1, "pair": "b0a899116dc36b85", "unit": "510ad48af03ba4a1a2a06dd254b135311dddb62d9fe5a2e8e8b12627586e6b13"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "litedb-org/litedb", "label": 0, "pair": "b0a899116dc36b85", "unit": "b414718a8a5f79739b88db24da6ad9da8e48bcb6b5e2dc99158848cdaa226767"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "8e22ee30441b20484090659b0c97423c711f5d10219041bf3a399623fc056a07"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "0f9070df9dc334f981cb01008c260b8bff78dbdeb48ce672830ed95caa8406fb"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "manuelz120/extremely-vulnerable-flask-app", "label": 1, "pair": "", "unit": "5e352f797b1422b0df477bce4730a9ce33cbd0837c3d63da94e4d8801a9a4b64"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 1, "pair": "dadccb128ac0e120", "unit": "505097494da5939404ad7d5b4af2b1c4a10efe94048084492d6ce60ebaa04c97"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "dadccb128ac0e120", "unit": "84ff8ca2d98a1b5d1ca4bfaabe7d19e03a396042b228c8d18d1185a178389702"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "91e261c54bd2c3c3a9ac04e48894ecabaca5b13e7db4fbd5578f9d266098117a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "a308dfd59bc398824e2454bf6cf6cfd5b7880e5b948a82424f14753d2b6fcdbe"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "medplum/medplum", "label": 0, "pair": "", "unit": "ba24eb554458bc97314ed561a7b2f4a4e643dde1205c32de4c8fcdd8c46e575b"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 0, "pair": "24d8f1840b67fbe5", "unit": "7ffa4a229a62266788fbf64890f81f20358b3eb8c35d3ab87d0c339d5bb4b507"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "mganss/htmlsanitizer", "label": 1, "pair": "24d8f1840b67fbe5", "unit": "d19493ed7941e1cd25a76dc50504d97398770d07ebc302dd66d83441e946c468"}
|
||||
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 0, "pair": "9f8a60eb48d74bc8", "unit": "80fce34f2377731e3de4355c7ea33bb20fc2fc00a5f388565b23018c7a3ed008"}
|
||||
{"family": "path", "fold": "outer-3", "group": "mguinness/elfinder.aspnet", "label": 1, "pair": "9f8a60eb48d74bc8", "unit": "b68e62722a327c9d5ce6774eb96527451729410768efc0b5c7ba604ec468052d"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 0, "pair": "2c857235d0764986", "unit": "33bdfe911cd1b512e5994ee7a3e3ab2ad1f302cab29df3b011c0d825c23536a3"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "microsoft/prompty", "label": 1, "pair": "2c857235d0764986", "unit": "b5efdcb84a057f3cd9fd3a30a32239dc06c13536a4e84a66c7a17cb372050c97"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 1, "pair": "9e84429d629c0b6b", "unit": "774cfdecac67499bb35a872931d4698d1b92ebeeb42d21790469e418889954e5"}
|
||||
{"family": "output_encoding", "fold": "outer-0", "group": "middleapi/orpc", "label": 0, "pair": "9e84429d629c0b6b", "unit": "a6c5c15ca1815f83dfb47428c4a8afd4e60fa1c027b9f33c668cb9a0d77c18af"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 0, "pair": "8e6b4b4e28981590", "unit": "3cce3e8f9ff3f1d261f9b96c844c1e4da07bae72d5393897b3a929be25372edc"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "mixxorz/slippers", "label": 1, "pair": "8e6b4b4e28981590", "unit": "b105689ac24ed334294c434a77bb6d56d5d598830ffe8a5d60ab3be5290b47ad"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "0a36cd5177f3b2c20b433db6bfffe5ec5b7e74cacc34a86ac581f7a0a559432a"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "e7fc97d83013c9ff", "unit": "5ab444e64297fc19065478104c7ebdabe23cc415f22adafd0cf82c98e4c5cae3"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "e7fc97d83013c9ff", "unit": "6b462ea30c1afda572600a10324249223932d36a032eacafac8c837bccbdce87"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "84d6a4202a082e0d", "unit": "8175082f40d8d61d44b54579c84a808e25b89110be5d8fd82feedf7dd5e0d5ab"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "8abefca43ee4ae2c797033da31f85c17bfa7be6b35413d70b0216415c5db9a09"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "91d213946859f33d3f243c43aa8285cc6e8ecb3db1de938f32af775a70607f8e"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "a244d9f010ddb43a09e82363dfabb0e406d411fb5de08ab7d6c9ee367f2a15c3"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 1, "pair": "", "unit": "d27d174d6e8e5f1682a1c42723705e89f0d18c95ecedd0567e313daa9a0aae44"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "mongo-express/mongo-express", "label": 0, "pair": "84d6a4202a082e0d", "unit": "f45608ed47fa11ca25093641f6021ec06076c18676abac9b809f31b3b7713de8"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 0, "pair": "55d1ce4f29dd1583", "unit": "a877ce3462da316bcdd82ce9db1d9887e887eaa83157922edb1663413edae1d6"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "moquette-io/moquette", "label": 1, "pair": "55d1ce4f29dd1583", "unit": "aee67be0ba31f99c7a24faa1bad6178c6d37361a308cf5aa10138e6ce4bbcfeb"}
|
||||
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 1, "pair": "16e529fe8c5486b2", "unit": "1d21acd0171726775d3da4589b660645fcd43cc026042a2b27e0e0e8374eceb7"}
|
||||
{"family": "path", "fold": "outer-4", "group": "naorsabag/openhop", "label": 0, "pair": "16e529fe8c5486b2", "unit": "f8e004b67ac98dc3392ea4115051e6bd10ce22eb638a762f0f16e1d4200bf129"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 1, "pair": "0b34438e6e0ba63d", "unit": "bf2608b00f0f784cc42e4c607f0b1ef50903283779bfb2eec91d2ec187856d14"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "nextauthjs/next-auth", "label": 0, "pair": "0b34438e6e0ba63d", "unit": "c5104cbf723fa0d55675963d8aa6802780ce3d274efe0012290fc49164c3fcc4"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 0, "pair": "b0210cd069757ad2", "unit": "d1d40c8b48d039c1999ff957a015273c30652ca0c45c4eb088052991a2a92f28"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "nuxt-content/mdc", "label": 1, "pair": "b0210cd069757ad2", "unit": "e2e804652a885f640ebf341627f3279d6c965b47ee2ce792e55045ab4d1e1dee"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 1, "pair": "ef603436079b0005", "unit": "77f843b5044bb5da935df4b2620bf2abe9e7dceb3147439e883927112eaec22f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "nuxt-modules/og-image", "label": 0, "pair": "ef603436079b0005", "unit": "f2ce95f0e44e73986784b63a702ffa4035c423056cac3667a63140a7e8d795c2"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 0, "pair": "a023056d2704c727", "unit": "6028fe87bea019646fff2b31c1b2d7bd0a95eb9214fe437e2008bc5d2dfabb7d"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "nuxt/nuxt", "label": 1, "pair": "a023056d2704c727", "unit": "dd93069189c5f242db611e4bfba4341c22f283b65af666fc31343854358de87e"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 0, "pair": "abc80e34d5f72367", "unit": "87d4a5b81bbfdd9628109a5f820895b6fddb00eb96b0f7bbe30a8d640457bcc2"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "oauthlib/oauthlib", "label": 1, "pair": "abc80e34d5f72367", "unit": "b171c77b52637bc84c487d7f8bce9a54229f9dbd6605e5089e1e88943ced989c"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 1, "pair": "1945f28a3fc5bd55", "unit": "64c0b7ccad09192eeaf590befe0de0087d5b48a65a4f6cef41415f3f54a8f536"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "opensearch-project/opensearch-ruby", "label": 0, "pair": "1945f28a3fc5bd55", "unit": "b55cd6759d4b7861e589052aa534bd0554586db46a329ed3beed7244d579e8c4"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 0, "pair": "1c9e47835232f644", "unit": "90e220d3540b88dd9efc8e66c8ab519d4cc78805f16bed585b795c78cfbb4d58"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "openssl/openssl", "label": 1, "pair": "1c9e47835232f644", "unit": "b5a185b05235f70f6cf27e2adba919f4d8393dbdf2d99de095548a21946aeaf3"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 1, "pair": "9b2b34cf62a76ea3", "unit": "91a3984e47f17512db8523e04f863525c67db18631d8f11776cf61ca7221c91b"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "openwisp/openwisp-ipam", "label": 0, "pair": "9b2b34cf62a76ea3", "unit": "d91f1e14282a0c3c8da57c6935a1625e881374bd205b5baa9e4a341d9109ae21"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 0, "pair": "a20cdf9dd208720d", "unit": "18a26df68e4b81d411cfbcc8155a7e6f382753d6e1ad95eece7ed24fa8c44324"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "oqtane/oqtane.framework", "label": 1, "pair": "a20cdf9dd208720d", "unit": "e6fb74441fb585d7ef5855317fba795ef0f0a0e6eae9970a49556ce3070c3254"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 0, "pair": "b2e833b60a0fa396", "unit": "08ccacb7434e9f8b8b05c7bb0996be0609e34860d8fc418db329aaf8d7910f2f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "oscal-compass/compliance-trestle", "label": 1, "pair": "b2e833b60a0fa396", "unit": "2ca7f80f0badc049ce725718e49bd60da72d316589399126a5f4c5308cf83868"}
|
||||
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 1, "pair": "d6930fe7e752b7ec", "unit": "06f0341dfbf1aa9504083737df5d9be3cedfd4e3a198799d8075a29990776800"}
|
||||
{"family": "path", "fold": "outer-0", "group": "owen2345/camaleon-cms", "label": 0, "pair": "d6930fe7e752b7ec", "unit": "30e2cd8bd381d7568b78eaf982f562d278a6a8d942d5282466532c1441bdf0ad"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 0, "pair": "e9c34a6a4bf2d333", "unit": "0ae20c7334fb27889ef8551f34c65a847dffd6b37f84f722f7e8ed64fd82e95b"}
|
||||
{"family": "access_control", "fold": "outer-4", "group": "patriksimek/vm2", "label": 1, "pair": "e9c34a6a4bf2d333", "unit": "59ab1924056704605b22b7c8360f40f745e7f488450be3647a632bf9a3b22699"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 0, "pair": "053a9e02feda3960", "unit": "34e18fe343b5c25ec582fc05ab091fd9ba160f658ba7678066c56db4991a951b"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "pdfminer/pdfminer.six", "label": 1, "pair": "053a9e02feda3960", "unit": "8fcb18a2a8654b6594cb2ec078e2a018d20f73d2b69a2e625338ffa80b3f72c3"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 0, "pair": "2d482951d7380c4f", "unit": "5c1a721da6f85d1d5dfc8cbe5e9809b093a09c059e5abeebed2cbbee55fbd498"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "perses/perses", "label": 1, "pair": "2d482951d7380c4f", "unit": "6cabd1c4b5aa94cafdd07c0c6a3f9abfdc03761ddc5b0a2bebc72a508d020b8b"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "", "unit": "2300c097374b9098671dcbcd42d02d08172e0f3064a8323223d0fe79a6106aab"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 1, "pair": "0664148b0b192456", "unit": "d702617aaf4d19811baaa18386a3c8b0eb35cd3fd72e8d90611d34850e990858"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "pgadmin-org/pgadmin4", "label": 0, "pair": "0664148b0b192456", "unit": "f5ed13d811c5b16d46cbd4a3b9edb8f7d2db49acbd49d95f39f33eaab38f58f5"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 0, "pair": "fd5781ba332ab81e", "unit": "59eca69b2bd99ba282eb8056d7bc789e9c691eebf62a4b42594e34f5516833ae"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "phpcsstandards/php_codesniffer", "label": 1, "pair": "fd5781ba332ab81e", "unit": "7a7d774a6de9e4a95fe97fff822652f1f1383922f2f0a4dcb12b8e964ca330f5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 0, "pair": "366dc5ddcfe598dd", "unit": "0053d866cec1e99f8a4a8754cf8ece3c46df16e002c86703c68937d5f3a09b08"}
|
||||
{"family": "untrusted_destination", "fold": "outer-3", "group": "phpoffice/phpspreadsheet", "label": 1, "pair": "366dc5ddcfe598dd", "unit": "027dbc5b9245fb804a1e2ba4445b8afa180aa2c2b3293db107ffe0ca4900ebb4"}
|
||||
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 0, "pair": "d0a34bf90e99e794", "unit": "4efbb75cc4105149522d2b2220987fd4373a8ef645c2c544546da540f14109eb"}
|
||||
{"family": "path", "fold": "outer-1", "group": "postcss/postcss", "label": 1, "pair": "d0a34bf90e99e794", "unit": "fd790f533103fd1a5646a9374ae895cf6e36d578fc747db2b0d0249a54a85e9f"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 0, "pair": "a697aa902d141a6a", "unit": "341e8fbb46f00167a32a2d653ba09a205794084eff65afddb70f9cfff42d697e"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "powsybl/powsybl-core", "label": 1, "pair": "a697aa902d141a6a", "unit": "e29414494b275afc15f3a028f62402161df7f6da5123c302149e23845177f7f6"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 0, "pair": "8c1fc8417190a134", "unit": "2125fb101d6108627e77f61c05cadddddfebd1cddce0e66376d9b822a2eecbb4"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "prowler-cloud/prowler", "label": 1, "pair": "8c1fc8417190a134", "unit": "ae7bbc58026fb71f91b477946a27bbe3599be90501da9237755f3cf0fb289794"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 0, "pair": "1972ea9f72b178b6", "unit": "3ddabb356d78e7c65ec8f88b6da8e4fcabc055ae73f2f6d92c50efddc5d4fcbe"}
|
||||
{"family": "access_control", "fold": "outer-3", "group": "publify/publify", "label": 1, "pair": "1972ea9f72b178b6", "unit": "6875dd5e9c0ff8a1b693d29a109cb2e62e6ff4081df40868aca70e4770476757"}
|
||||
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 1, "pair": "2b007a2377d633dd", "unit": "bdd71a783efb4eda9a82afbd30a16bd7608153830dbc599e65c453c18d1644b2"}
|
||||
{"family": "path", "fold": "outer-0", "group": "pypa/pip", "label": 0, "pair": "2b007a2377d633dd", "unit": "f398ef33900624219e59de08e1cc101c43082a81c37d6d03a916cc220727ca67"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "04aa777130c02a06", "unit": "16aa43c87956c44a4b115245140eefed704095d37cec0cc6410377403cd2fa7a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "23744137e0485a36f686f5f1b5666780fbed3ca522e8c6d35687a391ef504498"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "67565d99fe28c68f", "unit": "2692316792e6f02fb51578cf30d71bba6c73bf5420c36f709681c8eaa69807d4"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "2e36f73002c416a68d077bffb0335d82346c7507f2478353aeac005fce5a9cbc"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "04aa777130c02a06", "unit": "67721f67055eb6fed955ff22bfe4060344662eccb369431835a2116ad120c4f8"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "8fc1c96fa6faa4d3966ecc42876189b18ddab779d58f47cdf28a1155f48a6ac4"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 1, "pair": "67565d99fe28c68f", "unit": "ab535098a8e709b022afee1da15dd1fc371d389056938ee4fa4971b2966c823a"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "d46f94235bc462360097e7a22e1d442d95f049306631489a4d8b25f647a95bbe"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "edd6dd4364442f66384d14e9317ef2f6234e32ab78327a45741b8a816b4332ee"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "qwed-ai/qwed-verification", "label": 0, "pair": "", "unit": "fc7df6887ac719d4615934c0e8c3296728c2f6089f233158c59989027c802d72"}
|
||||
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 1, "pair": "c0412509f15fe3e1", "unit": "63bda2a71ad7155ad2f59aa9086efef3fd3710dc82c57f4b32ec8bf561460239"}
|
||||
{"family": "path", "fold": "outer-3", "group": "rack/rack", "label": 0, "pair": "c0412509f15fe3e1", "unit": "b1c9a070c93764655786c3de5fde536c150e319efe47548614cc544b73a8ee2b"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 1, "pair": "feed5f7de11289a4", "unit": "71531567b2b8609ae0d31cb177b8faea4f5fc61923e513b0154a2c941287ce2b"}
|
||||
{"family": "deserialization", "fold": "outer-4", "group": "rails/kredis", "label": 0, "pair": "feed5f7de11289a4", "unit": "d6be6d397a301a07faa761163d3d9e270d71d23d3bdfd084eeafb1e62b91dd71"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 1, "pair": "31c259b4e9dccd82", "unit": "1f1ddb8a5b415461638056c70756d2d0e836de71e824c0e46474f20646eaa4d5"}
|
||||
{"family": "access_control", "fold": "outer-1", "group": "redf0x1/camofox-mcp", "label": 0, "pair": "31c259b4e9dccd82", "unit": "a6ec568d4eef212a842dded7fdef2284524d367e5d24473ddb34e6374be9fad7"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 1, "pair": "ffd4b04a7bebfaf8", "unit": "54f7519d3d59b6d8d69a5f8f7bd78e360d3fabaae00b7587c37e388719169370"}
|
||||
{"family": "deserialization", "fold": "outer-0", "group": "redisson/redisson", "label": 0, "pair": "ffd4b04a7bebfaf8", "unit": "c5e42c9ab2ee54e46700ab0b64e239b567fbc75ccaceaf6cbf20687813663332"}
|
||||
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 1, "pair": "2ffec9490ef1d482", "unit": "0ab883d3561162b75ad9d161f79c0d17521a23317f2f574c910465f288119483"}
|
||||
{"family": "path", "fold": "outer-3", "group": "redocly/redocly-cli", "label": 0, "pair": "2ffec9490ef1d482", "unit": "b0af86c1642c14047248f1134181be0ec3839900917dffa360c5014119f8632c"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 1, "pair": "73f2a014c6a699ed", "unit": "44ba9d9d8e75769c1444ebf37a64941a845b4bcae678ec79d64dc84e5cc067a0"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "remix-run/react-router", "label": 0, "pair": "73f2a014c6a699ed", "unit": "b430d488c486cef26b1c661f773e7e9e4a177eeea73537a033eec98d532e4eed"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 0, "pair": "738e4c22edb32172", "unit": "932d7312b2bb4b830179104bca076f587505c2d4ce881b2f04f992defba6dd9e"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "remorses/genql", "label": 1, "pair": "738e4c22edb32172", "unit": "eee5136489c3baa3653ededbf6385d0237e2bd8b5384286780646b88ed34fb91"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 1, "pair": "cb0c638282db0d37", "unit": "a49e6ad2e7c2a5ad0d68d5d69c263306b501fc0dc9382e701fb5fe680611196b"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "remram44/taguette", "label": 0, "pair": "cb0c638282db0d37", "unit": "dee28099d3b37cbd0ee14412122ecc5f8488ed6bec01863520b87706a040fba1"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 0, "pair": "09bce271e8530fbc", "unit": "35eb19b4c6176b37287e2123bb98ec4045ce6bde7cc3864dde317c024b0935d8"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "roest01/node-pdf-image", "label": 1, "pair": "09bce271e8530fbc", "unit": "dd33291cc494bbc77d44e9b13d16f171d32c9746637506f315f375e8a9ddee95"}
|
||||
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 1, "pair": "a856cd13668353a6", "unit": "6149fb0132b981620a90a23a9af8d4656d0d827403e63b48258443cacd0a9c81"}
|
||||
{"family": "path", "fold": "outer-0", "group": "roomi-fields/notebooklm-mcp", "label": 0, "pair": "a856cd13668353a6", "unit": "b5aecdd6a87f8d57d28f54a013b1f96332460fe1bb44bff4ed28e166f18f9abe"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 1, "pair": "40d8d81a764c508f", "unit": "7deeed73c29c74ccbed29be049e109d363d41e4b5eea245ee57780b85f0e137a"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "rrainn/portprocesses", "label": 0, "pair": "40d8d81a764c508f", "unit": "b5cdadf848e67910550666ba2a6ac67b24112e501c02dba1865766a316317a23"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "", "unit": "15dbea384b3255457a44d9382efc6944dd27f1d411d903344a5d5e5fd194b72f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b2ac284bd70dd684", "unit": "225890ba6694dc8f4fbab46e9c851dff8c095e906cf14db15d01ebce0891aaab"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "b6eca35de93a5be8", "unit": "42eb8427415704e060e73b5ce0943b014ddf380ca756d5ec47c558e59a79895a"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b6eca35de93a5be8", "unit": "4c9cdcc11a18641ef41e9308712c5e8b830ec9ad5ebc3a956c8701e4f77b39c0"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "6a53939838afbdd7", "unit": "5794809e595311c86909d673a570d003aa025e4dcd6abc5641b641d935beb0b1"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "46d3ec7d49cb8ca7", "unit": "5f7a63ec650915962ee5255f5416df63ab882d6107f3b540be36d481798a8944"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "6a53939838afbdd7", "unit": "97a7018535ab511622e3ade9e53d570ffb1c80633eaa372fb76e749b3a586c8f"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "46d3ec7d49cb8ca7", "unit": "a84e9ab47f4f2b584684dbb81575758d9032fe3f2ed53bef478cb9dcfedaf677"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "b2ac284bd70dd684", "unit": "aeceb1ebc863ebbc3b69ae56c5b78d2dba005132b75cbd6af929bb0bfe25da9b"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 1, "pair": "f42cab375ace1739", "unit": "e329d4a6ac757680bf3d292a81f1f74f8a8a9b482d1651efe9f503f6a4de8796"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "saltcorn/saltcorn", "label": 0, "pair": "f42cab375ace1739", "unit": "e73c5aac41c0b93072223a3097fb175548ce893e0678ff3a22bc7571745a17e0"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 1, "pair": "1041a18cd1aec5f8", "unit": "c301b8c362aee3fe5ea38e18f858166bc4162ad8ace44c5fb2df192afc81da2a"}
|
||||
{"family": "injection", "fold": "outer-1", "group": "sebhildebrandt/systeminformation", "label": 0, "pair": "1041a18cd1aec5f8", "unit": "ca8e58ffcfd60bdc8626db8b9eb4f253c1b8b8affcb793c6d963b7a3ec91906b"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "", "unit": "52354ccdbc16f209e4692d97d5237e04ed030cf4a5e71f77b5a767962b355b4a"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 1, "pair": "7d23be667536afff", "unit": "b5016a5a53409491ba805ed8718599b84250b7b6914da0434b0882b0b3578dc8"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "sgabe/dsvpwa", "label": 0, "pair": "7d23be667536afff", "unit": "f1b01102968568d3ba1b2fdc70f4ae281f433fb20d29eff2a55fc45555ddbfcd"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 0, "pair": "f21df949f8aca4fa", "unit": "15d23ab3aea4c878dd73bbae579f3ab473e8363477a97d28164ed5e1b94cb510"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "shepherdwind/velocity.js", "label": 1, "pair": "f21df949f8aca4fa", "unit": "787bc98c651ed3db21be4c28e9c560c366ffd6ca508823d21e5d83eeaf1269db"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "1199de75f426940b", "unit": "10fdc236f43a02b7e291eb8bf6ada297a2ae98086a63b171057bc7bfc37a3377"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 0, "pair": "22112c31603bbe9f", "unit": "5eb3b7523b5a196dcaacba08d68457f9a45e31503ea02745ca3a7d0542f9881c"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "22112c31603bbe9f", "unit": "6bd5bdfd81b0a3374d59cfb2c1f7fa2b27887f4672f4485e77eb5234b569983f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "signalk/signalk-server", "label": 1, "pair": "1199de75f426940b", "unit": "c09dffd04708540845c8d1dd77a2bd989bd35f264abf652a2db55f8cb1faf7a9"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 0, "pair": "7aab025ea7b97019", "unit": "7487a34cdfdb532a5096764a47a3e565fa509605c1df0e2b105eb9c751fc5307"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "silverstripe/silverstripe-versioned", "label": 1, "pair": "7aab025ea7b97019", "unit": "9c0be2a58741f095fd0a9cdc60d4ca4039de1529699d5701dc4d0bcf7d5bf90c"}
|
||||
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 0, "pair": "4de5441f1776f083", "unit": "a3a35d573da308cd1d0614a26616bbd37b360d9ea416894cdeffdb59abd691d4"}
|
||||
{"family": "path", "fold": "outer-2", "group": "simonh1000/angular-http-server", "label": 1, "pair": "4de5441f1776f083", "unit": "dc33fc26c1bf7c67e5439ddfdad06c0faf16c796e0fe8a46b6e6a541e708676d"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 0, "pair": "1972a9a06d7585ad", "unit": "1f472e842e894c9907536c6eb6c9c922bbb3b21c27999114588ec2a4345ab665"}
|
||||
{"family": "deserialization", "fold": "outer-3", "group": "skops-dev/skops", "label": 1, "pair": "1972a9a06d7585ad", "unit": "7e38e24f1cc4d287627b7cd6cb870888865bb87fa87bba1c4fbf84eabb213121"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 1, "pair": "d76d5232a8587127", "unit": "1ec32d427822d1ddd3b0d7245f5771ddeb56c146ac8ff9959dd85760ff659a20"}
|
||||
{"family": "access_control", "fold": "outer-2", "group": "smithyhq/sqladmin", "label": 0, "pair": "d76d5232a8587127", "unit": "5143b19a1a1e6b9162b80594ed7fb5c0c57ed0b8abfd793bfc71089552d57638"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 0, "pair": "5f6e814444b4acd8", "unit": "2e8aa07b1aa4bfc5349b0d4464f3f2a6500aa55ff765b0139e9901ed34151ffa"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "soyuka/pidusage", "label": 1, "pair": "5f6e814444b4acd8", "unit": "e26c726c08be761b22fcc6ec17fc1fa7aba4769f08957f024e5545906df62797"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 1, "pair": "888e955fb6cfb4fd", "unit": "6f1a431ca97afc34ceec48393eb134a8928a312e6f761eb907c6b5855cac6208"}
|
||||
{"family": "output_encoding", "fold": "outer-3", "group": "spatie/schema-org", "label": 0, "pair": "888e955fb6cfb4fd", "unit": "7b62415d17569924d16238a4aa28ab1af2db6561f56be9f59d8075fd1bc4a941"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 1, "pair": "a5691395e641fd4c", "unit": "40339c123107f1231d6aa58de876cded1dfefd5b31dadc4fc86eedc6a687165f"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "spring-projects/spring-data-mongodb", "label": 0, "pair": "a5691395e641fd4c", "unit": "ea3828e2f59cc6f7b415a101532b566ecbd47ab123c3c7593c23c52f6773a2e3"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 1, "pair": "f1ebdacbd647e39d", "unit": "79628ef3d6c9e12a94922cdca548ff77d3dbdd6b595c1fec5958fcf23b609095"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "spring-projects/spring-graphql", "label": 0, "pair": "f1ebdacbd647e39d", "unit": "f910e010c3ff2c8c17f0f80fbe89a934e4f792b601febbf318359e6dc3afc18e"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 0, "pair": "2dd5384f097711b5", "unit": "212fba6b5e50db045114af369a03ffbdf09ec898c10eeff3c45bfc01a5017b4c"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "squidfunk/mkdocs-material", "label": 1, "pair": "2dd5384f097711b5", "unit": "370ba3a1d0e354e882b7be92c1ebbf2e2903bdf29d442048ea3dc1f80922467f"}
|
||||
{"family": "injection", "fold": "outer-4", "group": "strangerstudios/paid-memberships-pro", "label": 1, "pair": "", "unit": "ac905d12b92954263c046bd0138a682cea86cf8476336d6baa664ade28d34ac6"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "218a45d869bd7e39", "unit": "309ebac468b7ab054c8396ed495770385ae3590caba43709745ba2d1db4b6d54"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 1, "pair": "", "unit": "3cc8613813dec13cb1b7b45b296688da883f0b36cd2c4fc1da6e81703fb59ab9"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "218a45d869bd7e39", "unit": "9bd2331dbc260451355307e4c37bae17ad6676654c66181934e1b35fb174cc5a"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "b087ac4d0989e0b72698a9644ef4eac8073b4a15bb21730d63e4730b5cc58a60"}
|
||||
{"family": "injection", "fold": "outer-0", "group": "sulu/sulu", "label": 0, "pair": "", "unit": "c5d1e8d19560c997018fdca12eb60f1799be76c33be058b88472f5a2fc6709be"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 0, "pair": "c863054be488a284", "unit": "425e1d9961b79bbc011c447cd050910dd796a32b248962a5b0acc3f5f19b7345"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "svg/svgo", "label": 1, "pair": "c863054be488a284", "unit": "552925bba199119c7db5457b9d3c535bc9f6061cecfd2ab2f72cde690a536e1e"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "22befc17a75a688e2f9358d2ddf9a7b197da44cd95dbdeae5e0191b113ee54d3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "63f496d9ff53e39b", "unit": "6a98f8145776c78793b37156e9a81757841bb6de80016ab4fa575209d724db3f"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 1, "pair": "63f496d9ff53e39b", "unit": "7ac60c96c15f8517e089cc2240aedcf1fc30e54a41e35fe03c6b965bc393fbd6"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "themegrill/user-registration", "label": 0, "pair": "", "unit": "ae39cb55ef2a87c5347e0ddc40975f142450e82de4909e93ba62a7ed0efb4134"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 1, "pair": "d1b19898f8deaa41", "unit": "3763c7b407ad471820b5f8fa25bde0ba6d91e15c07c56681de4b41add418b0b1"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "thenables/thenify", "label": 0, "pair": "d1b19898f8deaa41", "unit": "bc8b934938c30687f3d4f982392d2cfa9504a916b463ccb5c46fe8a30eede488"}
|
||||
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 1, "pair": "f5f33fa6fe7b8492", "unit": "938bbf146a9037e687190bcee82c047492dffbe90efea40a49a647f81a0f1ed0"}
|
||||
{"family": "path", "fold": "outer-3", "group": "thumbor/thumbor", "label": 0, "pair": "f5f33fa6fe7b8492", "unit": "b90c9f7e0011685f1a8e853a9ed0ae37fd437ef6d6eb0ba17c240aec2c22b603"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 0, "pair": "f74364dc8e35c8ba", "unit": "11e760e99c5c852676c9288e18a250febb3854e9047b51e8b9ecc9f3b423b6f5"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "tomasnorre/crawler", "label": 1, "pair": "f74364dc8e35c8ba", "unit": "5fc980194040f4c4d803075d4085618cc0260ae8f3454b26b9b704d30200ffea"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 0, "pair": "db1c51833994ec55", "unit": "c2ce8320c778dd55434d7413844797b68e6fcabc4dc509458a9d60ca70b88c60"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "tornadoweb/tornado", "label": 1, "pair": "db1c51833994ec55", "unit": "f2d6843547af1276ae8a4dff3a7786913fa79dc69540d4fbb95c4c091e27c7f2"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 1, "pair": "f4690679bf9d80aa", "unit": "1a4e24df8d08672ecde4709a77ea5c437b2198fecff8bfff373b63c9eaed6e63"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "truelayer/truelayer-dotnet", "label": 0, "pair": "f4690679bf9d80aa", "unit": "78711973ec84867671314dd7878769092b32f59860eafed50a0f6979daee76fc"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 1, "pair": "d6a42c18151efb48", "unit": "01ae2279707361bfc06a005335d41b85066d0a163802ca81c031f5dbded5bb46"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "typeorm/typeorm", "label": 0, "pair": "d6a42c18151efb48", "unit": "a50dc15c313a9078badea01365987c66c5cee1ad48cea65c35c8bc2cbba25e8e"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 0, "pair": "85c15e3fc2b370b2", "unit": "0990d8f41cb07c1cf217812e51a7025381de4ba68d1be3799e26241d81dcea67"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "ultimatemember/ultimatemember", "label": 1, "pair": "85c15e3fc2b370b2", "unit": "9da1d63f0c4da270f66eb87bf9c342ace883a0aef78c4643e254159b5da8fce3"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 1, "pair": "c7f4f4ebebfae831", "unit": "b5cbc1490bca76679adfbe5e4d9f8067d27a226d595ba401b551b00c3ce4d45a"}
|
||||
{"family": "untrusted_destination", "fold": "outer-1", "group": "unjs/ufo", "label": 0, "pair": "c7f4f4ebebfae831", "unit": "ba2c1808eebdbe3f54c488b09bcb3d69e521916d1a3a0b5b14e02d5bc88bf016"}
|
||||
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 1, "pair": "1507c8e00ed4b859", "unit": "620e9b5ed143c0bcec178db359fa601acb1d3a134b3e85c2e838b7dcf3892d85"}
|
||||
{"family": "path", "fold": "outer-1", "group": "vercel/next.js", "label": 0, "pair": "1507c8e00ed4b859", "unit": "6f5ce4008dac0471260310c60e1e2a9f76a7e4eec6ebd039835f0488cf3b19aa"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 0, "pair": "8f5a6abedd742bbe", "unit": "71e3ced643a244100cc3fde234f5b4b27655214c7ae0ff67ab61e3e48a8fdf78"}
|
||||
{"family": "output_encoding", "fold": "outer-1", "group": "viewcomponent/view_component", "label": 1, "pair": "8f5a6abedd742bbe", "unit": "7c8cbfae87127042f9f43621345b4f7958bde27a3ff9db76254063e80735f3ce"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 1, "pair": "c9538342f2d7efca", "unit": "2b8db57d65ef1d67c457ff5a336e8a542d03387395d69d5e16815c4f00f40723"}
|
||||
{"family": "injection", "fold": "outer-3", "group": "vincentmorneau/apex-publish-static-files", "label": 0, "pair": "c9538342f2d7efca", "unit": "e2ed113dc21ad5f7caa71ba4a5ece8f8f900a97fd0d00dc85f7ac79ff5bed19a"}
|
||||
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 0, "pair": "494dd5d5e5cdf4b8", "unit": "3b8011f94ec721f68dd29afc6c8b01117c86a8b0c31f3650a6ce0a8d7c0739ff"}
|
||||
{"family": "path", "fold": "outer-2", "group": "vitest-dev/vitest", "label": 1, "pair": "494dd5d5e5cdf4b8", "unit": "f7e2280574e9ae51daa64d8adb2955407ff6d1e156f0e2854b6c047e0a733c78"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 1, "pair": "51eae488182a123a", "unit": "20b039866ba796c34efa0e92468fb49a314ad7fe2914d9ff8b823e3e35ab26f9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-4", "group": "vllm-project/vllm", "label": 0, "pair": "51eae488182a123a", "unit": "2ab3fef05def768ef7b4165bcedcf7d200282dba78a32e19b90a0df3aee7e3e5"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 1, "pair": "6f5a650865e08571", "unit": "e2051cc42e4f8b026c7807f966d492367310c90326e60fef1b0ee8c509c805a9"}
|
||||
{"family": "untrusted_destination", "fold": "outer-2", "group": "vuejs/vue-cli", "label": 0, "pair": "6f5a650865e08571", "unit": "f5364d68ffe80b578e788f37f08784b0162826393b49b4a3affd70d7cc4c9e01"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 1, "pair": "8acd426c5771b37c", "unit": "983e146a60e061254b15dc8dd573cbba57f8dd0358f23ea1b9c313ef69069af5"}
|
||||
{"family": "injection", "fold": "outer-2", "group": "vvo/node-whereis", "label": 0, "pair": "8acd426c5771b37c", "unit": "f80dfec45231e4778078cb10f00128ed445177cac0a062fc2df5ced5c0171f21"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 0, "pair": "9b84800ce5c395cb", "unit": "9b7dfdceae03fd86ad3260a03f85bcdcff8f32f10c265d23743ff8dc98eeefb8"}
|
||||
{"family": "deserialization", "fold": "outer-2", "group": "waterfutures/epyt-flow", "label": 1, "pair": "9b84800ce5c395cb", "unit": "b99dda150c0da3defa5f4c527ef6c402f5589dfb2f2f03f3e5e424910eb48c53"}
|
||||
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 1, "pair": "ad64e8fac4ac3ee0", "unit": "36dfd01915a553d86576e8ecd3b737b8e2b297aa542f58cc83b448060e74398a"}
|
||||
{"family": "path", "fold": "outer-1", "group": "webpack/webpack-dev-middleware", "label": 0, "pair": "ad64e8fac4ac3ee0", "unit": "c2b6d40f130d35bbfc16fab9411c664fa06321c59fb1d901a1987a69cb09c72e"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 0, "pair": "d48c08b19f560848", "unit": "03d02c9e25eec0f380b125279fce35e8867065bf6e1ec75f9f39729ce6efdc77"}
|
||||
{"family": "access_control", "fold": "outer-0", "group": "wintercms/winter", "label": 1, "pair": "d48c08b19f560848", "unit": "abbea69b7d0fa68c3ace442c25597e128cde354b417787b5913fd10d666c0d19"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 0, "pair": "bfc8188bbb673e3b", "unit": "a7cff0da55c0cc1d5c838e11fc6ce6c4b22c54b9f8e101688b218d6928eca504"}
|
||||
{"family": "output_encoding", "fold": "outer-2", "group": "xdan/jodit", "label": 1, "pair": "bfc8188bbb673e3b", "unit": "f741c56815230b9c30e2c8ef8952b895e9480e4780a89ff2cf6daf60d1fce7b6"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 0, "pair": "3844ed3d703d7ef4", "unit": "445efda074e4d9f6c1634225eb08a3c3eb1257311c995ce891df2c791a6f5c8b"}
|
||||
{"family": "output_encoding", "fold": "outer-4", "group": "xyproto/algernon", "label": 1, "pair": "3844ed3d703d7ef4", "unit": "8ac27b08549353ada819caaf99a5180d2f2f515b03aa8cd99a5d37b527ea3a5c"}
|
||||
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 1, "pair": "a63eae7d621a985e", "unit": "04ca0b583b09c3b4f67bd2496a4607ecb134d24b7bdaf8960fa151e40ebcc447"}
|
||||
{"family": "path", "fold": "outer-3", "group": "yiisoft/yii2", "label": 0, "pair": "a63eae7d621a985e", "unit": "ecfa2a67726ca9d3e2621a4d8be60145095b76aa18ecf0268212341d63752871"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 0, "pair": "0d9f529b7f0084b6", "unit": "35aa369236f510634b3aa7b54b04e9da0f80877e515786a888e760f8fe33c524"}
|
||||
{"family": "deserialization", "fold": "outer-1", "group": "zumba/json-serializer", "label": 1, "pair": "0d9f529b7f0084b6", "unit": "ef4f8613c0914eb815504e00cd6ee05df38352fd3b27826798e6623594017042"}
|
||||
@@ -0,0 +1,61 @@
|
||||
# Experiment exp-005 (learned-decision-engine design section 6): in-function graph slice.
|
||||
# Register before any paid fill or scoring; register pins the manifest and frozen units by sha256.
|
||||
# A changed question is a new experiment. The six slice-2 incumbents are unchanged in A;
|
||||
# exp-004's cache fill covered their prompts on this exact frozen population and memory.
|
||||
# B changes only ProgramSpec.slice: same inputs profile, retrieval, k, temperatures, and examples.
|
||||
# Slices are computed from the same numbered excerpt for candidates and all examples/demonstrations.
|
||||
#
|
||||
# Leak audit: benchmarks/measurements/2026-10-03-graph-slice-leak-audit/record.json.
|
||||
# Of 501 units, 220 have a slice (59 flow, 161 no-flow); 281 receive the fixed not-analysed sentence.
|
||||
# The audit found no leak. This is coverage, not evidence that the prompt improves ranking.
|
||||
# The units file is a byte-for-byte copy of exp-004-pooled-block-gate.units.jsonl; do not regenerate it.
|
||||
#
|
||||
# Primary: within-pair AUC, vulnerable vs fixed side (ties 0.5), complete pairs only;
|
||||
# paired B-A percentile 95% CI, resampling repository groups with replacement, both arms together.
|
||||
# Secondary: pooled top-30 precision across all families/units (raw score descending, unit ID breaks ties;
|
||||
# undefined below 30 scored units), unsure rate, and recorded response USD per candidate (including A's replay).
|
||||
# One final look. ADOPT B iff the primary CI is wholly positive AND mean cost/candidate <= 1.5x A;
|
||||
# REJECT iff the primary CI is wholly negative; otherwise INCONCLUSIVE, including unavailable cost.
|
||||
# Fill B under a $6.00 hard ceiling, then run --replay-only: scoring spends nothing further.
|
||||
# An incomplete fill must not be treated as a complete experiment; any resumption uses only the remaining budget.
|
||||
id: exp-005-graph-slice
|
||||
hypothesis: >
|
||||
Adding the in-function def-use slice to the prompt improves within-pair ranking of the six slice-2
|
||||
incumbent programs on held-out repositories, at no more than 1.5 times the cost per candidate.
|
||||
kind: program
|
||||
programs: # the slice-2 incumbents, as exp-003
|
||||
access_control: 745cb1d76bbb09dc1574d333d2118489be62e39886517894d3699fa565b0e8c2
|
||||
deserialization: 3c0c2831be7d78198a62351661c37867363c501cb40e3ed7b0119c155af09654
|
||||
injection: b207a0710e0abf053b3b9397b25f9ee1cab4679144865e3a82c1b9be2c7b4982
|
||||
output_encoding: 5d768f16724562c381c09949ff3df9cbb11d58cbb7a9c0515d0dc3f858d3818e
|
||||
path: aea396680a4874dcb6de70e240ca0480c83424debdd3710bc3f1ea6a612661f3
|
||||
untrusted_destination: 1242b3973fc89056aa08e00843043d68235df217d8e3ef8a0ea0e0b81c142753
|
||||
arms:
|
||||
A:
|
||||
label: six slice-2 incumbents unchanged, replaying exp-004's response cache
|
||||
spec: {}
|
||||
replay_only: true
|
||||
repeats: 1
|
||||
budget_usd: 0.0
|
||||
B:
|
||||
label: the same six programs with the in-function slice added to their prompts
|
||||
spec: {slice: true}
|
||||
replay_only: false
|
||||
repeats: 1
|
||||
budget_usd: 6.0
|
||||
unit: candidate
|
||||
pairing: paired
|
||||
cluster: repository
|
||||
order: {randomise: per_case, seed: 5}
|
||||
populations: {primary: paired, secondary: [pooled]}
|
||||
metric:
|
||||
primary: [within_pair_auc]
|
||||
secondary: [top_30_precision, unsure_rate, usd_per_candidate]
|
||||
decision: {arm: B, max_cost_ratio: 1.5}
|
||||
test: {primary: cluster_bootstrap_paired_diff, alpha: 0.05, sides: 2, resamples: 10000}
|
||||
mde: 0.0
|
||||
stopping: {looks: [1.0], boundaries: [1.959964]}
|
||||
budget_usd: {total: 6.0, per_arm: {A: 0.0, B: 6.0}}
|
||||
families: [access_control, deserialization, injection, output_encoding, path, untrusted_destination]
|
||||
folds: {kind: outer, k: 5, seed: 0, compile_fraction: 0.25}
|
||||
units_file: plane/experiments/exp-005-graph-slice.units.jsonl
|
||||
@@ -7,7 +7,7 @@ metadata:
|
||||
name: agree
|
||||
atespace: default
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["agree"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
@@ -7,7 +7,7 @@ metadata:
|
||||
name: alerts
|
||||
atespace: default
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["alerts"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
@@ -7,7 +7,7 @@ metadata:
|
||||
name: features
|
||||
atespace: default
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["features"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
@@ -7,7 +7,7 @@ metadata:
|
||||
name: loop
|
||||
atespace: default
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["loop"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
@@ -7,7 +7,7 @@ metadata:
|
||||
name: remember
|
||||
atespace: default
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["remember"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
@@ -8,7 +8,7 @@ metadata:
|
||||
name: repo-facts
|
||||
atespace: default
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["repo-facts"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
@@ -11,7 +11,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["roles"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
+120
-120
@@ -5,9 +5,9 @@ metadata:
|
||||
name: repo-facts-budibase-mongo-template-nosqli
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"b4ef709df34ab08c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b7e63df737eede85d951aa9665b0c0778ece7ccb","repo":"github.com/Budibase/budibase"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -36,7 +36,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -67,7 +67,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -96,7 +96,7 @@ metadata:
|
||||
name: agree-budibase-mongo-template-nosqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -123,9 +123,9 @@ metadata:
|
||||
name: repo-facts-cct-studio-server-cmdi
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"6c9fa4a6e47cda37","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"acdb5893412a81eaec9862ee24a4613bf296ab45","repo":"github.com/davila7/claude-code-templates"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -154,7 +154,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -185,7 +185,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -214,7 +214,7 @@ metadata:
|
||||
name: agree-cct-studio-server-cmdi
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -241,9 +241,9 @@ metadata:
|
||||
name: repo-facts-contextforge-prompt-ssti
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"b272af6456664ebc","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"0f8e0ad679f1fff685f486e17f642ac863c8ca9e","repo":"github.com/IBM/mcp-context-forge"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -272,7 +272,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -303,7 +303,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -332,7 +332,7 @@ metadata:
|
||||
name: agree-contextforge-prompt-ssti
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -359,9 +359,9 @@ metadata:
|
||||
name: repo-facts-froxlor-admin-ip-second-order-sqli
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"c52240d980ef0105","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9bf03de643a2edee4642bfc6c94db1f05253da02","repo":"github.com/froxlor/froxlor"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -390,7 +390,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -421,7 +421,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -450,7 +450,7 @@ metadata:
|
||||
name: agree-froxlor-admin-ip-second-order-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -477,9 +477,9 @@ metadata:
|
||||
name: repo-facts-fuxa-tdengine-escape-sqli
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"ec699ee54d38a8ee","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"193a888f8e3f805325072829b1644a2c768c948b","repo":"github.com/frangoteam/FUXA"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -508,7 +508,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -539,7 +539,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -568,7 +568,7 @@ metadata:
|
||||
name: agree-fuxa-tdengine-escape-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -595,9 +595,9 @@ metadata:
|
||||
name: repo-facts-lmdeploy-media-url-ssrf
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"1337c9c717f5ac87","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"71648d239c978452720a6d6692fe914efc8a7b94","repo":"github.com/InternLM/lmdeploy"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -626,7 +626,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -657,7 +657,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -686,7 +686,7 @@ metadata:
|
||||
name: agree-lmdeploy-media-url-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -713,9 +713,9 @@ metadata:
|
||||
name: repo-facts-medplum-external-auth-open-redirect
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"17ba609d5b1a1a00","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"5d4a8654b02aa67c444f96ae1eb5663c09854006","repo":"github.com/medplum/medplum"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -744,7 +744,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -775,7 +775,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -804,7 +804,7 @@ metadata:
|
||||
name: agree-medplum-external-auth-open-redirect
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -831,9 +831,9 @@ metadata:
|
||||
name: repo-facts-pgadmin-maintenance-sqli
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"7f1b83a88024c2d4","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"d112dc3b962218efc3613fa727d54f0fc5b1d527","repo":"github.com/pgadmin-org/pgadmin4"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -862,7 +862,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -893,7 +893,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -922,7 +922,7 @@ metadata:
|
||||
name: agree-pgadmin-maintenance-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -949,9 +949,9 @@ metadata:
|
||||
name: repo-facts-qwed-sympy-eval
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"f071e8db5f5517ab","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"06801f68cbf9c9b5e54cc17f2a61cb2792da7a26","repo":"github.com/QWED-AI/qwed-verification"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -980,7 +980,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1011,7 +1011,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1040,7 +1040,7 @@ metadata:
|
||||
name: agree-qwed-sympy-eval
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -1067,9 +1067,9 @@ metadata:
|
||||
name: repo-facts-reactpress-findall-column-sqli
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"ecb82a10249c6fed","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"c3dd10681b3d2b5e224f7ac647b3769557603975","repo":"github.com/fecommunity/reactpress"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -1098,7 +1098,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1129,7 +1129,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1158,7 +1158,7 @@ metadata:
|
||||
name: agree-reactpress-findall-column-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -1185,9 +1185,9 @@ metadata:
|
||||
name: repo-facts-unleash-addon-webhook-ssrf
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"4d3debd51fd86f9c","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"b9f3217d327f24094b4352130baa30387f85cffe","repo":"github.com/Unleash/unleash"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -1216,7 +1216,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1247,7 +1247,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1276,7 +1276,7 @@ metadata:
|
||||
name: agree-unleash-addon-webhook-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -1303,9 +1303,9 @@ metadata:
|
||||
name: repo-facts-winml-cli-serve-cors
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"e3b0c44298fc1c14","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"02179d185f0e3ac23518236b5ca1f61fd37ef680","repo":"github.com/microsoft/winml-cli"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -1334,7 +1334,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1365,7 +1365,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1394,7 +1394,7 @@ metadata:
|
||||
name: agree-winml-cli-serve-cors
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -1421,9 +1421,9 @@ metadata:
|
||||
name: repo-facts-wp-directorist-avatar-ssrf
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"d6963021f588f39e","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"77cfc9bc8b453f97ab7e720330706fa4ef056867","repo":"github.com/sovware/directorist"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -1452,7 +1452,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1483,7 +1483,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1512,7 +1512,7 @@ metadata:
|
||||
name: agree-wp-directorist-avatar-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -1539,9 +1539,9 @@ metadata:
|
||||
name: repo-facts-wp-gopay-log-filter-sqli
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"626bc8b5f8d33de6eefd921e3a05ed52ac320bcd","repo":"github.com/gopaycommunity/gopay-woocommerce-integration"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -1570,7 +1570,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1601,7 +1601,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1630,7 +1630,7 @@ metadata:
|
||||
name: agree-wp-gopay-log-filter-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -1657,9 +1657,9 @@ metadata:
|
||||
name: repo-facts-wp-groundhogg-confirm-redirect
|
||||
atespace: default
|
||||
annotations:
|
||||
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}'
|
||||
openultrasast.io/memory-key: '{"candidates":"fde4d17144a13d73","image":"sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019","pin":"9472a1dd79d4556c40eece501a61930793f170d3","repo":"github.com/groundhoggwp/groundhogg"}'
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- repo-facts
|
||||
env:
|
||||
@@ -1688,7 +1688,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1719,7 +1719,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1748,7 +1748,7 @@ metadata:
|
||||
name: agree-wp-groundhogg-confirm-redirect
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- agree
|
||||
env:
|
||||
@@ -1777,7 +1777,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1808,7 +1808,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1839,7 +1839,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1870,7 +1870,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1901,7 +1901,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1932,7 +1932,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1963,7 +1963,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -1994,7 +1994,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2025,7 +2025,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2056,7 +2056,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2087,7 +2087,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2118,7 +2118,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2149,7 +2149,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2180,7 +2180,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2211,7 +2211,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- verify
|
||||
env:
|
||||
@@ -2240,7 +2240,7 @@ metadata:
|
||||
name: features-budibase-mongo-template-nosqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2263,7 +2263,7 @@ metadata:
|
||||
name: remember-budibase-mongo-template-nosqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2288,7 +2288,7 @@ metadata:
|
||||
name: features-cct-studio-server-cmdi
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2311,7 +2311,7 @@ metadata:
|
||||
name: remember-cct-studio-server-cmdi
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2336,7 +2336,7 @@ metadata:
|
||||
name: features-contextforge-prompt-ssti
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2359,7 +2359,7 @@ metadata:
|
||||
name: remember-contextforge-prompt-ssti
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2384,7 +2384,7 @@ metadata:
|
||||
name: features-froxlor-admin-ip-second-order-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2407,7 +2407,7 @@ metadata:
|
||||
name: remember-froxlor-admin-ip-second-order-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2432,7 +2432,7 @@ metadata:
|
||||
name: features-fuxa-tdengine-escape-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2455,7 +2455,7 @@ metadata:
|
||||
name: remember-fuxa-tdengine-escape-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2480,7 +2480,7 @@ metadata:
|
||||
name: features-lmdeploy-media-url-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2503,7 +2503,7 @@ metadata:
|
||||
name: remember-lmdeploy-media-url-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2528,7 +2528,7 @@ metadata:
|
||||
name: features-medplum-external-auth-open-redirect
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2551,7 +2551,7 @@ metadata:
|
||||
name: remember-medplum-external-auth-open-redirect
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2576,7 +2576,7 @@ metadata:
|
||||
name: features-pgadmin-maintenance-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2599,7 +2599,7 @@ metadata:
|
||||
name: remember-pgadmin-maintenance-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2624,7 +2624,7 @@ metadata:
|
||||
name: features-qwed-sympy-eval
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2647,7 +2647,7 @@ metadata:
|
||||
name: remember-qwed-sympy-eval
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2672,7 +2672,7 @@ metadata:
|
||||
name: features-reactpress-findall-column-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2695,7 +2695,7 @@ metadata:
|
||||
name: remember-reactpress-findall-column-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2720,7 +2720,7 @@ metadata:
|
||||
name: features-unleash-addon-webhook-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2743,7 +2743,7 @@ metadata:
|
||||
name: remember-unleash-addon-webhook-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2768,7 +2768,7 @@ metadata:
|
||||
name: features-winml-cli-serve-cors
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2791,7 +2791,7 @@ metadata:
|
||||
name: remember-winml-cli-serve-cors
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2816,7 +2816,7 @@ metadata:
|
||||
name: features-wp-directorist-avatar-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2839,7 +2839,7 @@ metadata:
|
||||
name: remember-wp-directorist-avatar-ssrf
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2864,7 +2864,7 @@ metadata:
|
||||
name: features-wp-gopay-log-filter-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2887,7 +2887,7 @@ metadata:
|
||||
name: remember-wp-gopay-log-filter-sqli
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
@@ -2912,7 +2912,7 @@ metadata:
|
||||
name: features-wp-groundhogg-confirm-redirect
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- features
|
||||
env:
|
||||
@@ -2935,7 +2935,7 @@ metadata:
|
||||
name: remember-wp-groundhogg-confirm-redirect
|
||||
atespace: default
|
||||
spec:
|
||||
image: localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884
|
||||
image: localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019
|
||||
command:
|
||||
- remember
|
||||
env:
|
||||
|
||||
@@ -10,7 +10,7 @@ metadata:
|
||||
annotations:
|
||||
openultrasast.io/model: deepseek-flash
|
||||
spec:
|
||||
image: "localhost:5001/ousast-runner@sha256:412db0a1504af8f1b23a395af8d1dcbd152c1bb6d16594f2b733067163a31884"
|
||||
image: "localhost:5001/ousast-runner@sha256:d2a55293999cc34581ef1f02a1ffb0c2dd8fc0cbe001ac952df81600e868f019"
|
||||
command: ["verify"]
|
||||
resources:
|
||||
requests:
|
||||
|
||||
+272
-95
@@ -37,7 +37,7 @@ from .calibration import (
|
||||
from .complexity import map as complexity_map
|
||||
from .complexity.ledger import persist_verdicts
|
||||
from .config import ModelLayerConfig, ObligationsConfig, ResolvedConfig, RetiredConfigError, load_config, load_dotenv
|
||||
from .findings import StaticFinding, quick_scan_findings, write_findings
|
||||
from .findings import StaticFinding, quick_scan_findings, uncovered_languages, write_findings
|
||||
from .fusion import FusionDecision, fuse_findings
|
||||
from .gate import FALSE_POSITIVE_CEILING, RECALL_FLOOR
|
||||
from .harness import HarnessRuntime, HarnessTraceWriter, write_harness_config
|
||||
@@ -103,6 +103,7 @@ class ScanOutcome:
|
||||
finding_count: int
|
||||
calibrations_applied: int
|
||||
exit_code: int
|
||||
notices: tuple[str, ...] = ()
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -131,28 +132,84 @@ def main(argv: list[str] | None = None) -> int:
|
||||
|
||||
|
||||
def _main(argv: list[str] | None) -> int:
|
||||
if (argv if argv is not None else sys.argv[1:])[:1] != ["pre-push"]:
|
||||
given = list(argv if argv is not None else sys.argv[1:])
|
||||
if given[:1] == ["pre-push"] and given[1:2] in (["install"], ["uninstall"]):
|
||||
return _hook_install(given[1], given[2:])
|
||||
if given[:1] != ["pre-push"]:
|
||||
load_dotenv()
|
||||
parser = argparse.ArgumentParser(prog="ousast")
|
||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
replay_parser = subparsers.add_parser("pre-push", help="check Git push input or replay local revisions (experimental)")
|
||||
replay_parser.add_argument("path", type=Path)
|
||||
replay_parser.add_argument("--base")
|
||||
replay_parser.add_argument("--head")
|
||||
replay_parser.add_argument("--remote", nargs=2, metavar=("NAME", "URL"))
|
||||
replay_parser.add_argument("--comparison-base")
|
||||
replay_parser = subparsers.add_parser(
|
||||
"pre-push",
|
||||
help="check Git push input or replay local revisions (experimental, advisory)",
|
||||
description="Check what a push changes: quick rules on the changed lines, then the Joern engine on head and base "
|
||||
"within one deadline. Advisory: exit 0 unless --mode blocking. Install the Git hook with "
|
||||
"'ousast pre-push install [REPO] [--force] [--docker]'; remove it with 'ousast pre-push uninstall [REPO]'.",
|
||||
)
|
||||
replay_parser.add_argument("path", type=Path, help="the repository to check (the hook passes '.')")
|
||||
replay_parser.add_argument("--base", help="explicit replay: the base revision (with --head; instead of --remote)")
|
||||
replay_parser.add_argument("--head", help="explicit replay: the head revision to check against --base")
|
||||
replay_parser.add_argument(
|
||||
"--remote",
|
||||
nargs=2,
|
||||
metavar=("NAME", "URL"),
|
||||
help="Git hook mode: the remote name and URL Git passes to pre-push; the push input is read from stdin",
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--comparison-base",
|
||||
help="hook mode: a local ref to compare a NEW branch against (the remote has no base for it); "
|
||||
"the hook reads OUSAST_COMPARISON_BASE",
|
||||
)
|
||||
replay_parser.add_argument("--model-config", type=Path, help="explicit optional witness selection endpoint configuration")
|
||||
replay_parser.add_argument("--prior-hook", type=Path, help="explicitly chain an existing hook with the same input and arguments")
|
||||
replay_parser.add_argument("--artifact", type=Path, required=True)
|
||||
replay_parser.add_argument(
|
||||
"--artifact", type=Path, required=True, help="where to write the complete JSON result; must be outside the repository"
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--cache-dir", type=Path, help="reuse compatible local artifacts in a private directory outside the repository"
|
||||
)
|
||||
replay_parser.add_argument("--deadline", type=float, default=30.0)
|
||||
replay_parser.add_argument("--cancellation-allowance", type=float, default=2.0)
|
||||
replay_parser.add_argument("--max-regions", type=int, default=500)
|
||||
replay_parser.add_argument("--mode", choices=("advisory", "blocking"), default="advisory")
|
||||
replay_parser.add_argument("--incomplete-coverage", choices=("allow", "block"), default="allow")
|
||||
replay_parser.add_argument(
|
||||
"--deadline",
|
||||
type=float,
|
||||
default=30.0,
|
||||
help="seconds for the whole check: resolution, quick rules, engine and reporting (default 30; hook: OUSAST_PUSH_DEADLINE)",
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--cancellation-allowance",
|
||||
type=float,
|
||||
default=2.0,
|
||||
help="seconds after the deadline to stop work and save the result (default 2)",
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--max-regions", type=int, default=500, help="most code regions the engine examines, highest-ranked first (default 500)"
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--mode",
|
||||
choices=("advisory", "blocking"),
|
||||
default="advisory",
|
||||
help="advisory (default) never blocks; blocking is the explicit opt-in that rejects a push for an admitted "
|
||||
"alert (hook: OUSAST_PUSH_MODE)",
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--incomplete-coverage",
|
||||
choices=("allow", "block"),
|
||||
default="allow",
|
||||
help="with --mode blocking only: also block when coverage is incomplete (hook: OUSAST_INCOMPLETE_COVERAGE)",
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--engine",
|
||||
choices=("inline", "background", "off"),
|
||||
default="inline",
|
||||
help="inline (default): run the Joern engine inside the deadline; background: return after the quick rules and "
|
||||
"run the engine detached, showing its result on the next run; off: quick rules only",
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--background-deadline",
|
||||
type=float,
|
||||
default=900.0,
|
||||
help="seconds the detached engine run of --engine background may take (default 900)",
|
||||
)
|
||||
replay_parser.add_argument(
|
||||
"--experimental-declarations",
|
||||
type=Path,
|
||||
@@ -266,16 +323,21 @@ def _main(argv: list[str] | None) -> int:
|
||||
learn_labels.add_argument("--include-title", action="store_true", help="the title-tier pairs as well (a sensitivity arm)")
|
||||
learn_labels.add_argument("--no-assumed-benign", action="store_true", help="skip mining the assumed-benign history source")
|
||||
add_learn_commands(learn_sub)
|
||||
plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on this host's kind cluster")
|
||||
plane = subparsers.add_parser("plane", help="maintainer: the ax-backed service plane on the profile's cluster (ops/k8s/profiles/)")
|
||||
profile_help = "the PlaneProfile: a name under ops/k8s/profiles/ or a .toml path (default OUSAST_PLANE_PROFILE, else kind)"
|
||||
plane_sub = plane.add_subparsers(dest="plane_command", required=True)
|
||||
plane_run = plane_sub.add_parser("run", help="execute a Run manifest on ax; a rerun skips done tasks")
|
||||
plane_run.add_argument("run_manifest", type=Path)
|
||||
plane_run.add_argument("--workers", type=int, default=1, help="tasks in flight at once (serialize labels never overlap)")
|
||||
plane_run.add_argument("--ax", default="ax", help="path of the ax CLI")
|
||||
plane_run.add_argument("--profile", help=profile_help)
|
||||
plane_status = plane_sub.add_parser("status", help="per-task status and the token attribution table of a run")
|
||||
plane_status.add_argument("run")
|
||||
plane_status.add_argument("--units", action="store_true", help="per-unit rows from each task's units.jsonl")
|
||||
plane_sub.add_parser("doctor", help="check kind, Agent Substrate, the ax controller and the runner image")
|
||||
plane_doctor = plane_sub.add_parser(
|
||||
"doctor", help="check the profile's cluster, Agent Substrate, the ax controller, the runner image and the memory store"
|
||||
)
|
||||
plane_doctor.add_argument("--profile", help=profile_help)
|
||||
plane_remember = plane_sub.add_parser("remember", help="ingest a run's memory rows into the store (OUSAST_MEMORY)")
|
||||
plane_remember.add_argument("run")
|
||||
plane_remember.add_argument("--plane", type=Path, default=Path("plane"), help="where runs/<run>.yaml names population and split")
|
||||
@@ -313,9 +375,11 @@ def _main(argv: list[str] | None) -> int:
|
||||
)
|
||||
plane_harvest.add_argument("--ceiling", type=float, default=10.0, help="USD: each Run's task budgets sum to at most this")
|
||||
plane_harvest.add_argument("--runner-image", type=Path, help="re-pin the task templates to the image in this file first (ops/ax/up.sh)")
|
||||
plane_harvest.add_argument("--profile", help=profile_help)
|
||||
plane_ws = plane_sub.add_parser("workspaces", help="one Workspace manifest per case pin of a population file")
|
||||
plane_ws.add_argument("population", type=Path)
|
||||
plane_ws.add_argument("--out", type=Path, default=Path("plane/workspaces"))
|
||||
plane_ws.add_argument("--profile", help=profile_help)
|
||||
plane_ws.add_argument("--validation-set", type=Path, help="JSON {case: [[path, function], ...]}: emit that set's increment instead")
|
||||
plane_ws.add_argument("--candidates", type=Path, help="with --validation-set: per-case candidate records (independent-v2-modelsinks)")
|
||||
plane_ws.add_argument("--triage", type=Path, help="with --validation-set: recorded batched-check scans (<case>--vulnerable_a.jsonl)")
|
||||
@@ -339,6 +403,20 @@ def _main(argv: list[str] | None) -> int:
|
||||
"--qualify-population", action="append", default=[], metavar="NAME", help="with --loop: a population whose rows never propose"
|
||||
)
|
||||
|
||||
raw = list(argv if argv is not None else sys.argv[1:])
|
||||
if _advisory_hook(raw):
|
||||
# Git hook mode (stdin) without an explicit blocking opt-in: a check that cannot run must
|
||||
# never reject the push. Say so in one line and allow it (Requirement 9.1).
|
||||
try:
|
||||
return _hook_main(parser, argv)
|
||||
except SystemExit as stop:
|
||||
if stop.code in (0, None):
|
||||
raise
|
||||
print("ousast: pre-push check did not run: invalid arguments or settings (message above). Push continues.")
|
||||
return 0
|
||||
except Exception as error: # noqa: BLE001 -- an advisory hook reports a crash, it never blocks on one
|
||||
print(f"ousast: pre-push check did not run: unexpected {type(error).__name__}. Push continues.")
|
||||
return 0
|
||||
args = parser.parse_args(argv)
|
||||
# Diagnosis needs the stage costs, and nothing configures logging, so the default root level of
|
||||
# WARNING silently dropped every informational line. A run that reports nothing is
|
||||
@@ -347,77 +425,7 @@ def _main(argv: list[str] | None) -> int:
|
||||
_configure_logging()
|
||||
start_profiling() # only when OUSAST_SAMPLE_PROFILE names a file
|
||||
if args.command == "pre-push":
|
||||
from .config import PushConfig
|
||||
from .push.runner import push, replay
|
||||
|
||||
try:
|
||||
if bool(args.base) != bool(args.head) or bool(args.base) == bool(args.remote):
|
||||
parser.error("provide --base and --head together, or --remote NAME URL for Git stdin")
|
||||
settings = PushConfig(
|
||||
comparison_base=args.comparison_base,
|
||||
deadline_seconds=args.deadline,
|
||||
cancellation_allowance_seconds=args.cancellation_allowance,
|
||||
mode=args.mode,
|
||||
incomplete_coverage_policy=args.incomplete_coverage,
|
||||
)
|
||||
options: dict[str, Any] = dict(
|
||||
artifact=args.artifact,
|
||||
config=settings,
|
||||
max_regions=args.max_regions,
|
||||
cache_dir=args.cache_dir,
|
||||
model_config=args.model_config,
|
||||
)
|
||||
if args.base and args.prior_hook:
|
||||
parser.error("--prior-hook requires Git stdin mode")
|
||||
if args.experimental_record_vetoes and not args.base:
|
||||
parser.error("--experimental-record-vetoes requires explicit --base/--head replay; it is never a hook capability")
|
||||
if args.experimental_declarations and not args.experimental_record_vetoes:
|
||||
parser.error("--experimental-declarations requires --experimental-record-vetoes")
|
||||
prior_data = None
|
||||
if args.base:
|
||||
delivery = replay(
|
||||
args.path,
|
||||
base=args.base,
|
||||
head=args.head,
|
||||
record_vetoes=args.experimental_record_vetoes,
|
||||
declarations=args.experimental_declarations,
|
||||
**options,
|
||||
)
|
||||
else:
|
||||
# multiprocessing closes sys.stdin in its child; duplicate the Git pipe first.
|
||||
with os.fdopen(os.dup(0), "rb") as stream:
|
||||
if args.prior_hook:
|
||||
from .model.contracts import ExecutionBudget
|
||||
from .push.runner import _prepare
|
||||
|
||||
shared = ExecutionBudget(time.monotonic() + settings.deadline_seconds, settings.cancellation_allowance_seconds)
|
||||
try:
|
||||
prior_data = _prepare(lambda: stream.read(1024 * 1024 + 1), shared)
|
||||
if len(prior_data) > 1024 * 1024:
|
||||
raise ValueError("input exceeds hook integration limit")
|
||||
except (ValueError, RuntimeError, TimeoutError):
|
||||
print("Hook input could not be retained for all consumers. Retry with complete input and sufficient time.")
|
||||
return 1
|
||||
options["execution_budget"] = shared
|
||||
|
||||
def updates() -> str:
|
||||
return (prior_data if prior_data is not None else stream.read(1024 * 1024 + 1)).decode("utf-8")
|
||||
|
||||
delivery = push(args.path, updates=updates, remote_name=args.remote[0], remote_url=args.remote[1], **options)
|
||||
except ValueError as error:
|
||||
parser.error(str(error))
|
||||
print(delivery.text, end="", flush=True)
|
||||
if args.prior_hook and prior_data is not None:
|
||||
# This is the explicitly chained hook's own behavior, outside analysis.
|
||||
# Never replace its rejection with our advisory success or impose our timeout on it.
|
||||
try:
|
||||
prior = subprocess.run([str(args.prior_hook.resolve()), *args.remote], input=prior_data, cwd=args.path)
|
||||
except OSError:
|
||||
print("Existing hook could not be started. Restore its executable path before retrying.")
|
||||
return 1
|
||||
if prior.returncode:
|
||||
return prior.returncode if prior.returncode > 0 else 128 - prior.returncode
|
||||
return delivery.exit_code
|
||||
return _pre_push(args, parser)
|
||||
if args.command == "scan":
|
||||
return _scan(args.path, args.config, args.mode, args.fail_on)
|
||||
if args.command == "benchmark":
|
||||
@@ -463,6 +471,145 @@ def _main(argv: list[str] | None) -> int:
|
||||
return 2
|
||||
|
||||
|
||||
def _hook_install(action: str, argv: list[str]) -> int:
|
||||
"""`ousast pre-push install|uninstall [REPO]` from the hook shipped in the package (task 17.6)."""
|
||||
from .push.install import install, uninstall
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
prog=f"ousast pre-push {action}",
|
||||
description="Install the advisory OpenUltraSAST pre-push hook into a repository's effective hook directory "
|
||||
"(core.hooksPath is honoured, never edited)."
|
||||
if action == "install"
|
||||
else "Remove the OpenUltraSAST pre-push hook and restore a chained previous hook.",
|
||||
)
|
||||
parser.add_argument("repository", nargs="?", type=Path, default=Path("."), help="the repository (default: current directory)")
|
||||
parser.add_argument(
|
||||
"--force",
|
||||
action="store_true",
|
||||
help="install: replace our hook, or move a foreign hook to pre-push.before-ousast and chain it; "
|
||||
"uninstall: remove a hook this tool did not write",
|
||||
)
|
||||
if action == "install":
|
||||
parser.add_argument(
|
||||
"--docker",
|
||||
action="store_true",
|
||||
help="also install the ousast-docker wrapper beside the hook, so the check runs in the container image",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
try:
|
||||
code, lines = (
|
||||
install(args.repository, force=args.force, docker=args.docker)
|
||||
if action == "install"
|
||||
else uninstall(args.repository, force=args.force)
|
||||
)
|
||||
except (OSError, ValueError, subprocess.SubprocessError) as error:
|
||||
print(f"ousast pre-push {action}: {error}", file=sys.stderr)
|
||||
return 1
|
||||
for line in lines:
|
||||
print(line, file=sys.stderr if code else sys.stdout)
|
||||
return code
|
||||
|
||||
|
||||
def _advisory_hook(argv: list[str]) -> bool:
|
||||
"""A Git hook invocation (`--remote NAME URL`, stdin input) that did not opt into blocking."""
|
||||
if argv[:1] != ["pre-push"] or "--remote" not in argv:
|
||||
return False
|
||||
blocking = any(a == "--mode=blocking" for a in argv) or any(
|
||||
a == "--mode" and i + 1 < len(argv) and argv[i + 1] == "blocking" for i, a in enumerate(argv)
|
||||
)
|
||||
return not blocking
|
||||
|
||||
|
||||
def _hook_main(parser: argparse.ArgumentParser, argv: list[str] | None) -> int:
|
||||
args = parser.parse_args(argv)
|
||||
_configure_logging()
|
||||
return _pre_push(args, parser)
|
||||
|
||||
|
||||
def _pre_push(args: argparse.Namespace, parser: argparse.ArgumentParser) -> int:
|
||||
from .config import PushConfig
|
||||
from .push.runner import push, replay
|
||||
|
||||
try:
|
||||
if bool(args.base) != bool(args.head) or bool(args.base) == bool(args.remote):
|
||||
parser.error("provide --base and --head together, or --remote NAME URL for Git stdin")
|
||||
settings = PushConfig(
|
||||
comparison_base=args.comparison_base,
|
||||
deadline_seconds=args.deadline,
|
||||
cancellation_allowance_seconds=args.cancellation_allowance,
|
||||
mode=args.mode,
|
||||
incomplete_coverage_policy=args.incomplete_coverage,
|
||||
)
|
||||
options: dict[str, Any] = dict(
|
||||
artifact=args.artifact,
|
||||
config=settings,
|
||||
max_regions=args.max_regions,
|
||||
cache_dir=args.cache_dir,
|
||||
model_config=args.model_config,
|
||||
)
|
||||
if args.base and args.prior_hook:
|
||||
parser.error("--prior-hook requires Git stdin mode")
|
||||
if args.experimental_record_vetoes and not args.base:
|
||||
parser.error("--experimental-record-vetoes requires explicit --base/--head replay; it is never a hook capability")
|
||||
if args.experimental_declarations and not args.experimental_record_vetoes:
|
||||
parser.error("--experimental-declarations requires --experimental-record-vetoes")
|
||||
prior_data = None
|
||||
if args.base:
|
||||
delivery = replay(
|
||||
args.path,
|
||||
base=args.base,
|
||||
head=args.head,
|
||||
record_vetoes=args.experimental_record_vetoes,
|
||||
declarations=args.experimental_declarations,
|
||||
engine=args.engine,
|
||||
background_deadline=args.background_deadline,
|
||||
**options,
|
||||
)
|
||||
else:
|
||||
# multiprocessing closes sys.stdin in its child; duplicate the Git pipe first.
|
||||
with os.fdopen(os.dup(0), "rb") as stream:
|
||||
if args.prior_hook:
|
||||
from .model.contracts import ExecutionBudget
|
||||
from .push.runner import _prepare
|
||||
|
||||
shared = ExecutionBudget(time.monotonic() + settings.deadline_seconds, settings.cancellation_allowance_seconds)
|
||||
try:
|
||||
prior_data = _prepare(lambda: stream.read(1024 * 1024 + 1), shared)
|
||||
if len(prior_data) > 1024 * 1024:
|
||||
raise ValueError("input exceeds hook integration limit")
|
||||
except (ValueError, RuntimeError, TimeoutError):
|
||||
print("Hook input could not be retained for all consumers. Retry with complete input and sufficient time.")
|
||||
return 1
|
||||
options["execution_budget"] = shared
|
||||
|
||||
def updates() -> str:
|
||||
return (prior_data if prior_data is not None else stream.read(1024 * 1024 + 1)).decode("utf-8")
|
||||
|
||||
delivery = push(
|
||||
args.path,
|
||||
updates=updates,
|
||||
remote_name=args.remote[0],
|
||||
remote_url=args.remote[1],
|
||||
engine=args.engine,
|
||||
background_deadline=args.background_deadline,
|
||||
**options,
|
||||
)
|
||||
except ValueError as error:
|
||||
parser.error(str(error))
|
||||
print(delivery.text, end="", flush=True)
|
||||
if args.prior_hook and prior_data is not None:
|
||||
# This is the explicitly chained hook's own behavior, outside analysis.
|
||||
# Never replace its rejection with our advisory success or impose our timeout on it.
|
||||
try:
|
||||
prior = subprocess.run([str(args.prior_hook.resolve()), *args.remote], input=prior_data, cwd=args.path)
|
||||
except OSError:
|
||||
print("Existing hook could not be started. Restore its executable path before retrying.")
|
||||
return 1
|
||||
if prior.returncode:
|
||||
return prior.returncode if prior.returncode > 0 else 128 - prior.returncode
|
||||
return delivery.exit_code
|
||||
|
||||
|
||||
def _scan(path: Path, config_path: Path, mode: str, fail_on: str) -> int:
|
||||
outcome = _run_scan(path, config_path, mode, fail_on)
|
||||
_print_scan_outcome(outcome)
|
||||
@@ -495,6 +642,17 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
|
||||
"preprocess",
|
||||
lambda: preprocess_repository(run.target, run.root / "preprocess" / "file_targets.json", static_hints),
|
||||
)
|
||||
# A language nothing here covers is named, never a silent zero (pre-push 17.2, Requirement 9.2).
|
||||
for language, count in uncovered_languages(targets, ruleset, engine=mode != "quick"):
|
||||
runtime.state["degradations"].append(
|
||||
{
|
||||
"stage": "quick_findings",
|
||||
"reason": "language_not_covered",
|
||||
"language": language,
|
||||
"count": count,
|
||||
"engine": "" if mode == "quick" else " and no engine frontend",
|
||||
}
|
||||
)
|
||||
entry_points = runtime.run_stage("entry_point_mapping", lambda: analyze_entry_points(run.target, targets))
|
||||
write_entry_points(entry_points, run.root / "mapping" / "entry_points.json")
|
||||
targets = attach_reachability_hints(targets, entry_points)
|
||||
@@ -793,6 +951,11 @@ def _run_scan(path: Path, config_path: Path, mode: str, fail_on: str) -> ScanOut
|
||||
finding_count=len(findings),
|
||||
calibrations_applied=len(applied_calibrations),
|
||||
exit_code=scan_exit_code(findings, verifications, fail_on, worth_fixing_verdicts=verdict_records),
|
||||
notices=tuple(
|
||||
f"language not covered: {item['language']} ({item['count']} files)"
|
||||
for item in runtime.state["degradations"]
|
||||
if item.get("reason") == "language_not_covered"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -867,6 +1030,8 @@ def _print_scan_outcome(outcome: ScanOutcome) -> None:
|
||||
print(f"ranked_targets={outcome.ranked_target_count}")
|
||||
print(f"findings={outcome.finding_count}")
|
||||
print(f"calibrations_applied={outcome.calibrations_applied}")
|
||||
for notice in outcome.notices:
|
||||
print(f"notice: {notice}")
|
||||
|
||||
|
||||
def _load_static_hints(sarif_paths: tuple[str, ...]) -> list[object]:
|
||||
@@ -1371,10 +1536,22 @@ def _learn_labels(args: argparse.Namespace) -> int:
|
||||
|
||||
def _plane(args: argparse.Namespace) -> int:
|
||||
from .plane import reconciler # lazy: the plane is a maintainer surface, not the scan path
|
||||
from .plane.profile import ProfileError, load_profile
|
||||
|
||||
try:
|
||||
profile = load_profile(getattr(args, "profile", None))
|
||||
except ProfileError as exc:
|
||||
print(f"plane: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
if args.plane_command == "run":
|
||||
from .plane.memory import MemoryStoreError
|
||||
|
||||
_plane_memory("seed", args.run_manifest)
|
||||
result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax)
|
||||
try:
|
||||
result = reconciler.run(args.run_manifest, workers=args.workers, ax=args.ax, profile=profile)
|
||||
except MemoryStoreError as exc: # the store is the state of record: no fallback, the Run is not submitted
|
||||
print(f"memory store {profile.memory}: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
print(f"run finished: {result}")
|
||||
_plane_memory("ingest", args.run_manifest)
|
||||
return 0 if result == "done" else 1
|
||||
@@ -1389,14 +1566,14 @@ def _plane(args: argparse.Namespace) -> int:
|
||||
if args.plane_command == "alerts-engine":
|
||||
return _plane_alerts_engine(args)
|
||||
if args.plane_command == "harvest":
|
||||
return _plane_harvest(args)
|
||||
return _plane_harvest(args, profile.atespace)
|
||||
if args.plane_command == "doctor":
|
||||
checks = reconciler.doctor()
|
||||
checks = reconciler.doctor(profile)
|
||||
for name, ok, text in checks:
|
||||
print(f"[{'ok' if ok else 'FAIL'}] {name}: {text}")
|
||||
return 0 if all(ok for _, ok, _ in checks) else 1
|
||||
if args.validation_set is not None:
|
||||
return _plane_increment(args)
|
||||
return _plane_increment(args, profile.atespace)
|
||||
from .plane.workspaces import workspaces
|
||||
|
||||
written = workspaces(args.population, args.out)
|
||||
@@ -1404,7 +1581,7 @@ def _plane(args: argparse.Namespace) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _plane_harvest(args: argparse.Namespace) -> int:
|
||||
def _plane_harvest(args: argparse.Namespace, atespace: str) -> int:
|
||||
from .plane.generate import repin_templates
|
||||
from .plane.harvest import write_harvest
|
||||
|
||||
@@ -1413,7 +1590,7 @@ def _plane_harvest(args: argparse.Namespace) -> int:
|
||||
command = f"ousast plane harvest --labels {args.labels.name} --name {args.name} --ceiling {args.ceiling:g}"
|
||||
files = write_harvest(
|
||||
args.labels, args.plane, name=args.name, command=command, catalog=args.catalog, cache=args.cache, templates=args.templates,
|
||||
ceiling=args.ceiling,
|
||||
ceiling=args.ceiling, atespace=atespace,
|
||||
) # fmt: skip
|
||||
for relative in sorted(k for k in files if k.startswith("runs/")):
|
||||
print(f"{args.plane / relative}: {files[relative].splitlines()[1].lstrip('# ')}")
|
||||
@@ -1494,7 +1671,7 @@ def _plane_memory(
|
||||
return 0
|
||||
|
||||
|
||||
def _plane_increment(args: argparse.Namespace) -> int:
|
||||
def _plane_increment(args: argparse.Namespace, atespace: str) -> int:
|
||||
from .plane.generate import LOOP_MANIFEST, Loop, increment
|
||||
|
||||
if args.candidates is None or args.triage is None:
|
||||
@@ -1520,7 +1697,7 @@ def _plane_increment(args: argparse.Namespace) -> int:
|
||||
command += "".join(f" --qualify-population {p}" for p in loop.populations)
|
||||
written = increment(
|
||||
args.population, args.validation_set, args.candidates, args.triage, plane=args.plane, run_name=args.run, command=command,
|
||||
repos=args.repos, runner_image=args.runner_image, loop=loop,
|
||||
repos=args.repos, runner_image=args.runner_image, loop=loop, atespace=atespace,
|
||||
) # fmt: skip
|
||||
print(f"{len(written)} manifests written under {args.plane} (Run {args.run})")
|
||||
return 0
|
||||
|
||||
@@ -162,7 +162,9 @@ class EngineSession:
|
||||
return None
|
||||
payload = json.dumps({"query": code}).encode()
|
||||
call = urllib.request.Request(
|
||||
f"http://127.0.0.1:{self._port}/query-sync", data=payload, headers={"Content-Type": "application/json"}
|
||||
f"http://127.0.0.1:{self._port}/query-sync", # loopback: the Joern server this process started
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(call, timeout=timeout) as response: # noqa: S310 -- fixed loopback URL
|
||||
|
||||
@@ -2,6 +2,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import asdict, dataclass
|
||||
from pathlib import Path
|
||||
|
||||
@@ -74,6 +75,30 @@ def quick_scan_findings(
|
||||
return sorted(findings, key=lambda item: (_severity_sort(item.severity), -item.ranking_priority, item.path))
|
||||
|
||||
|
||||
def uncovered_languages(
|
||||
targets: Sequence[FileTarget], ruleset: Sequence[PatternRule] | None = None, *, engine: bool = False
|
||||
) -> list[tuple[str, int]]:
|
||||
"""Languages present in `targets` that no enabled quick rule (and, with `engine`, no engine frontend) covers.
|
||||
|
||||
A repository in such a language produces zero findings whatever it contains; callers name it so that
|
||||
zero is never read as a clean result.
|
||||
"""
|
||||
from .model.partitions import FRONTENDS
|
||||
|
||||
rules = PATTERN_RULES if ruleset is None else ruleset
|
||||
enabled = [rule for rule in rules if rule.status != "disabled"]
|
||||
if any(not rule.languages for rule in enabled): # a rule without a language list applies to every file
|
||||
return []
|
||||
covered = {language for rule in enabled for language in rule.languages}
|
||||
if engine:
|
||||
covered |= set(FRONTENDS)
|
||||
counts: dict[str, int] = {}
|
||||
for target in targets:
|
||||
if target.language not in covered and target.language != "unknown":
|
||||
counts[target.language] = counts.get(target.language, 0) + 1
|
||||
return sorted(counts.items())
|
||||
|
||||
|
||||
def build_quick_hunter_prompt(target: FileTarget, source_excerpt: str) -> str:
|
||||
return (
|
||||
"Review this file for security-relevant issues. Return structured findings only when evidence exists.\n"
|
||||
|
||||
@@ -60,18 +60,26 @@ def add_commands(learn_sub: Any) -> None:
|
||||
("register", "record the manifest's and the units file's sha256 as the experiment row (both committed at HEAD)"),
|
||||
("units", "freeze the units of a program experiment from the memory (no model): the families' evaluation candidates"),
|
||||
("run", "run every arm on every unit, paired; arm A replays from the response cache when the manifest says so"),
|
||||
("fill", "fill an experiment's response cache under a separate budget, or estimate with --dry-run"),
|
||||
("analyse", "paired cluster bootstrap, exact McNemar, the two looks and the adoption verdict (counts and intervals only)"),
|
||||
):
|
||||
sub = experiment_sub.add_parser(name, help=text)
|
||||
sub.add_argument("manifest", type=Path, help="plane/experiments/<id>.yaml")
|
||||
sub.add_argument("--memory", help="the store (default OUSAST_MEMORY, then results_root()/memory)")
|
||||
if name in ("units", "run"):
|
||||
if name in ("units", "run", "fill"):
|
||||
sub.add_argument("--profile", choices=("static", "plane"), default="static")
|
||||
if name == "run":
|
||||
if name in ("run", "fill"):
|
||||
sub.add_argument(
|
||||
"--manifest-model", type=Path, default=Path("plane/models/deepseek-flash.yaml"), help="the chat Model (prices)"
|
||||
)
|
||||
if name == "run":
|
||||
sub.add_argument("--replay-only", action="store_true", help="replay every arm without resolving a model endpoint")
|
||||
sub.add_argument("--no-early-stop", action="store_true", help="take no first look during the run")
|
||||
if name == "fill":
|
||||
sub.add_argument("--arm", action="append", help="arm to fill (repeatable; default: non-replay program arms)")
|
||||
sub.add_argument("--budget-usd", type=float, help="required for paid fill; includes the smoke call")
|
||||
sub.add_argument("--dry-run", action="store_true", help="inspect and estimate only; no calls or writes, including --out")
|
||||
sub.add_argument("--out", type=Path, help="also write the fill report as JSON (ignored for --dry-run)")
|
||||
if name in ("run", "analyse"):
|
||||
sub.add_argument("--resamples", type=int, help="bootstrap resamples (default: the manifest's)")
|
||||
sub.add_argument("--out", type=Path, help="write the counts-only report here (analyse --record: benchmarks/experiments/<id>/)")
|
||||
@@ -111,7 +119,7 @@ def _experiment(args: argparse.Namespace) -> int:
|
||||
from ..plane.budget import MeteredClient
|
||||
from ..plane.memory import open_store
|
||||
from . import experiments as ex
|
||||
from .program import Caller
|
||||
from .program import Caller, ReplayMiss
|
||||
|
||||
store = open_store(args.memory)
|
||||
try:
|
||||
@@ -122,11 +130,17 @@ def _experiment(args: argparse.Namespace) -> int:
|
||||
print(json.dumps({k: row[k] for k in shown}, indent=2))
|
||||
return 0
|
||||
if args.experiment_command == "analyse":
|
||||
row = ex.check_registered(store, manifest, need_units=False)
|
||||
row = ex.check_registered(store, manifest, need_units=manifest.kind == "rule")
|
||||
provenance = {"code_commit": ex.committed_at_head(manifest.path)}
|
||||
report = ex.analyse(manifest, ex.load_outcomes(store, manifest.id), resamples=args.resamples, provenance=provenance)
|
||||
outcomes = ex.load_outcomes(store, manifest.id)
|
||||
if manifest.kind == "rule":
|
||||
units = ex.read_units(manifest.units_file)
|
||||
report = ex.analyse_rule(manifest, outcomes, units, resamples=args.resamples, provenance=provenance)
|
||||
else:
|
||||
report = ex.analyse(manifest, outcomes, resamples=args.resamples, provenance=provenance)
|
||||
if args.record:
|
||||
store.put_row(ex.result_row(manifest, str(row["pin"]), report))
|
||||
result = ex.rule_result_row if manifest.kind == "rule" else ex.result_row
|
||||
store.put_row(result(manifest, str(row["pin"]), report))
|
||||
_write(report, args.out or (ex.RESULT_DIR / manifest.id / "result.json" if args.record else None))
|
||||
return 0
|
||||
from .embeddings import EmbeddingCache
|
||||
@@ -143,9 +157,47 @@ def _experiment(args: argparse.Namespace) -> int:
|
||||
cache = EmbeddingCache(store)
|
||||
vectors = {sha: v for sha in cache.names() if (v := cache.get(sha)) is not None} or None
|
||||
parameters = _model_parameters(args.manifest_model)
|
||||
|
||||
def excerpt_text(sha: str) -> str | None:
|
||||
data = store.get_blob("excerpts", sha)
|
||||
return data.decode("utf-8") if data is not None else None
|
||||
|
||||
if args.experiment_command == "fill":
|
||||
from ..tool_hunter import ChatClient
|
||||
from .fill import fill_experiment
|
||||
|
||||
def resolve_fill_client() -> ChatClient:
|
||||
from ..config import load_config
|
||||
from ..model.endpoint import resolve_chat_endpoint
|
||||
|
||||
resolved = resolve_chat_endpoint(load_config(None))
|
||||
if resolved is None:
|
||||
raise ex.ExperimentError("no chat endpoint configured; nothing was spent")
|
||||
return resolved[0]
|
||||
|
||||
report = fill_experiment(
|
||||
store,
|
||||
manifest,
|
||||
memory,
|
||||
parameters,
|
||||
excerpt_text,
|
||||
vectors,
|
||||
client_factory=resolve_fill_client,
|
||||
budget_usd=args.budget_usd,
|
||||
dry_run=args.dry_run,
|
||||
arms=args.arm,
|
||||
)
|
||||
_write(report, None if args.dry_run else args.out)
|
||||
return {"done": 0, "dry_run": 0, "unfinished": 3, "failed": 1}[report["status"]]
|
||||
|
||||
if manifest.kind == "rule": # zero cost: cache only, behind a zero-ceiling meter; no endpoint is resolved
|
||||
caller, meter = ex.zero_cost_caller(_arm_model(manifest, manifest.arms["A"], store), parameters, store)
|
||||
summary = ex.run_rule(store, manifest, memory, caller, meter, excerpt_text, vectors)
|
||||
_write(summary.as_dict(), args.out)
|
||||
return 0
|
||||
callers: dict[str, Caller] = {}
|
||||
for name, arm in manifest.arms.items():
|
||||
if arm.replay_only:
|
||||
if arm.replay_only or args.replay_only:
|
||||
callers[name] = Caller(None, _arm_model(manifest, arm, store), parameters, store=store)
|
||||
continue
|
||||
from ..config import load_config
|
||||
@@ -157,13 +209,9 @@ def _experiment(args: argparse.Namespace) -> int:
|
||||
client = MeteredClient(resolved[0], prices=parameters, budget_usd=arm.budget_usd)
|
||||
callers[name] = Caller(client, _arm_model(manifest, arm, store), parameters, store=store)
|
||||
|
||||
def excerpt_text(sha: str) -> str | None:
|
||||
data = store.get_blob("excerpts", sha)
|
||||
return data.decode("utf-8") if data is not None else None
|
||||
|
||||
early = not args.no_early_stop
|
||||
summary = ex.run(store, manifest, memory, callers, excerpt_text, vectors, stop_early=early, resamples=args.resamples)
|
||||
except ex.ExperimentError as exc:
|
||||
except (ex.ExperimentError, ReplayMiss) as exc:
|
||||
print(f"learn experiment {args.experiment_command}: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
_write(summary.as_dict(), args.out)
|
||||
|
||||
@@ -25,7 +25,7 @@ from collections.abc import Callable, Iterable, Mapping, Sequence
|
||||
from dataclasses import dataclass, replace
|
||||
from typing import Any
|
||||
|
||||
from .calibrate import Row, check, cross_fit, offered
|
||||
from .calibrate import Row, check, cross_fit, fit_map, offered
|
||||
from .examples import Example
|
||||
from .folds import Fold
|
||||
from .labels import PER_FAMILY_FLOOR, POOLED_FLOOR
|
||||
@@ -54,6 +54,7 @@ class Prediction:
|
||||
parse_failed: int = 0
|
||||
p: float | None = None
|
||||
point: str | None = None # BLOCK | ADVISORY | DROP, from points chosen on the other folds
|
||||
pair: str = "" # the two sides of one candidate share it; "" when only one side is present
|
||||
|
||||
@property
|
||||
def blockable(self) -> bool:
|
||||
@@ -128,6 +129,179 @@ def nested_points(preds: Sequence[Prediction]) -> tuple[list[Prediction], dict[s
|
||||
return out, points
|
||||
|
||||
|
||||
BOUNDS = ("wilson", "point")
|
||||
|
||||
|
||||
def precision_threshold(
|
||||
rows: Sequence[Prediction], *, precision: float = TARGET_PRECISION, bound: str = "wilson", min_flagged: int = 1
|
||||
) -> float | None:
|
||||
"""The lowest raw-score threshold ``t`` at which the blockable rows with ``s >= t`` reach ``precision``: their
|
||||
Wilson 95% lower bound (``bound="wilson"``) or their point precision over at least ``min_flagged`` rows
|
||||
(``bound="point"``). Candidate thresholds are the rows' own scores; ``None`` when no threshold qualifies."""
|
||||
if bound not in BOUNDS:
|
||||
raise ValueError(f"bound {bound!r} (one of {', '.join(BOUNDS)})")
|
||||
blockable = [r for r in rows if r.blockable]
|
||||
for t in sorted({r.s for r in blockable}):
|
||||
chosen = [r for r in blockable if r.s >= t]
|
||||
hits = sum(r.label for r in chosen)
|
||||
if len(chosen) < max(1, min_flagged):
|
||||
continue
|
||||
level = wilson_lower(hits, len(chosen)) if bound == "wilson" else hits / len(chosen)
|
||||
if level >= precision:
|
||||
return t
|
||||
return None
|
||||
|
||||
|
||||
def nested_precision_block(
|
||||
preds: Sequence[Prediction],
|
||||
*,
|
||||
eligible: Callable[[Prediction], bool] | None = None,
|
||||
precision: float = TARGET_PRECISION,
|
||||
bound: str = "wilson",
|
||||
min_flagged: int = 1,
|
||||
) -> tuple[list[Prediction], dict[str, dict[str, float | None]]]:
|
||||
"""Selective blocking on the raw score, per family: the threshold for outer fold j is chosen on the *other* folds'
|
||||
rows of the family only (and, with ``eligible``, only those rows); the held-out fold's labels never enter it.
|
||||
Every row of fold j is then decided with that threshold: ``BLOCK`` when blockable and ``s >= t``, else ``DROP``.
|
||||
A family whose training folds reach no threshold offers no BLOCK on that fold. Returns the decided rows and the
|
||||
thresholds by family and fold."""
|
||||
thresholds: dict[str, dict[str, float | None]] = {}
|
||||
for family in sorted({r.family for r in preds}):
|
||||
rows = [r for r in preds if r.family == family]
|
||||
thresholds[family] = {
|
||||
fold: precision_threshold(
|
||||
[r for r in rows if r.fold != fold and (eligible is None or eligible(r))],
|
||||
precision=precision, bound=bound, min_flagged=min_flagged,
|
||||
)
|
||||
for fold in sorted({r.fold for r in rows})
|
||||
} # fmt: skip
|
||||
out = []
|
||||
for r in preds:
|
||||
t = thresholds[r.family][r.fold]
|
||||
out.append(replace(r, point="BLOCK" if t is not None and r.blockable and r.s >= t else "DROP"))
|
||||
return out, thresholds
|
||||
|
||||
|
||||
def calibrated_block(preds: Sequence[Prediction], *, seed: int = 0, **floors: int) -> tuple[list[Prediction], dict[str, Any]]:
|
||||
"""Today's BLOCK rule, per family as the slice-2 evaluation applied it: the Platt map cross-fitted on the other
|
||||
folds, the calibration check on the family's held-out calibrated predictions, the BLOCK point chosen on the other
|
||||
folds' calibrated predictions (nested); a row is ``BLOCK`` only when the family's calibration holds. Returns the
|
||||
decided rows and, per family, the check and the calibrated thresholds by fold."""
|
||||
out: list[Prediction] = []
|
||||
info: dict[str, Any] = {}
|
||||
for family in sorted({r.family for r in preds}):
|
||||
rows = [r for r in preds if r.family == family]
|
||||
ps, _ = cross_fit([Row(r.fold, r.family, r.group, r.label, r.s) for r in rows], **floors)
|
||||
calibrated = [replace(r, p=p) for r, p in zip(rows, ps, strict=True)]
|
||||
decided = [r for r in calibrated if r.p is not None]
|
||||
checked = (
|
||||
check([float(r.p) for r in decided if r.p is not None], [r.label for r in decided], [r.group for r in decided], seed=seed)
|
||||
if len({r.label for r in decided}) == 2
|
||||
else None
|
||||
)
|
||||
holds = bool(checked and checked.holds)
|
||||
nested, points = nested_points(calibrated)
|
||||
out += [replace(r, point="BLOCK" if holds and r.point == "BLOCK" else "DROP") for r in nested]
|
||||
info[family] = {
|
||||
"calibration_holds": holds, "ece": None if checked is None else round(checked.ece, 4),
|
||||
"thresholds_by_fold": {f: p.block for f, p in points.items()},
|
||||
"reason_by_fold": {f: ("calibration" if not holds else p.block_reason) for f, p in points.items()},
|
||||
} # fmt: skip
|
||||
return out, info
|
||||
|
||||
|
||||
# --- the pooled BLOCK gate (Req 6.4, amended 2026-10-03; exp-004) ---------------------------------------------------------
|
||||
|
||||
FAMILY_FLOOR = 10 # a family blocks only with at least this many held-out flags and none of them wrong
|
||||
|
||||
|
||||
def normalised(train: Sequence[Prediction], rows: Sequence[Prediction], **floors: int) -> list[float | None]:
|
||||
"""``rows``' raw scores on one scale across families: the per-family Platt map of :func:`.calibrate.fit_map`
|
||||
(pooled slope with a per-family intercept between the floors) fitted on ``train`` only. ``None`` for a family the
|
||||
training rows cannot map (below the pooled floor): it cannot block on that fold."""
|
||||
cmap = fit_map([Row(r.fold, r.family, r.group, r.label, r.s) for r in train], **floors)
|
||||
return [cmap.apply(r.family, r.s) for r in rows]
|
||||
|
||||
|
||||
def pooled_threshold(rows: Sequence[Prediction], *, precision: float = TARGET_PRECISION) -> float | None:
|
||||
"""The lowest normalised score ``t`` at which the blockable rows of every family together with ``p >= t`` have a
|
||||
Wilson 95% lower bound of precision >= ``precision``; candidate thresholds are the rows' own ``p``."""
|
||||
blockable = [r for r in rows if r.blockable and r.p is not None]
|
||||
for t in sorted({float(r.p) for r in blockable if r.p is not None}):
|
||||
chosen = [r for r in blockable if (r.p or 0.0) >= t]
|
||||
if wilson_lower(sum(r.label for r in chosen), len(chosen)) >= precision:
|
||||
return t
|
||||
return None
|
||||
|
||||
|
||||
def quiet_threshold(rows: Sequence[Prediction], *, recall: float = TARGET_RECALL) -> float | None:
|
||||
"""The highest normalised score ``t`` that keeps at least ``recall`` of the rows' positives at or above it."""
|
||||
positives = sorted((float(r.p) for r in rows if r.label == 1 and r.p is not None), reverse=True)
|
||||
if not positives:
|
||||
return None
|
||||
return positives[math.ceil(recall * len(positives) - 1e-9) - 1]
|
||||
|
||||
|
||||
def pooled_block_gate(
|
||||
preds: Sequence[Prediction],
|
||||
*,
|
||||
eligible: Callable[[Prediction], bool] | None = None,
|
||||
precision: float = TARGET_PRECISION,
|
||||
recall: float = TARGET_RECALL,
|
||||
**floors: int,
|
||||
) -> tuple[list[Prediction], dict[str, dict[str, Any]]]:
|
||||
"""One BLOCK threshold for every family, on a normalised score, chosen per outer fold j on the other folds only.
|
||||
|
||||
The training rows (``eligible`` rows of the other folds) are normalised by an inner cross-fit: each training fold
|
||||
k by a map fitted on the training folds other than k, so ``t_block`` is chosen on out-of-sample scores. ``t_block``
|
||||
is the lowest normalised score whose pooled Wilson 95% lower bound of precision is >= ``precision``; ``t_quiet``
|
||||
per family is the highest one keeping >= ``recall`` of that family's training positives. Fold j's rows are then
|
||||
normalised by the map fitted on all its training rows and decided: ``BLOCK`` when blockable and ``p >= t_block``;
|
||||
``ADVISORY`` when ``p >= t_quiet`` (or the family has no map, or no ``t_quiet``: nothing is silenced blind);
|
||||
``DROP`` (silent) below ``t_quiet``. Fold j's labels never enter its thresholds or its map."""
|
||||
chosen = [r for r in preds if eligible is None or eligible(r)]
|
||||
families = sorted({r.family for r in preds})
|
||||
decided: dict[int, Prediction] = {}
|
||||
gates: dict[str, dict[str, Any]] = {}
|
||||
for fold in sorted({r.fold for r in preds}):
|
||||
train = [r for r in chosen if r.fold != fold]
|
||||
inner: list[Prediction] = []
|
||||
for k in sorted({r.fold for r in train}):
|
||||
mine = [r for r in train if r.fold == k]
|
||||
inner += [replace(r, p=p) for r, p in zip(mine, normalised([r for r in train if r.fold != k], mine, **floors), strict=True)
|
||||
if p is not None] # fmt: skip
|
||||
t_block = pooled_threshold(inner, precision=precision)
|
||||
t_quiet = {f: quiet_threshold([r for r in inner if r.family == f], recall=recall) for f in families}
|
||||
held = [(i, r) for i, r in enumerate(preds) if r.fold == fold]
|
||||
for (i, r), p in zip(held, normalised(train, [r for _, r in held], **floors), strict=True):
|
||||
quiet = t_quiet[r.family]
|
||||
if p is not None and t_block is not None and r.blockable and p >= t_block:
|
||||
point = "BLOCK"
|
||||
elif p is None or quiet is None or p >= quiet:
|
||||
point = "ADVISORY"
|
||||
else:
|
||||
point = "DROP"
|
||||
decided[i] = replace(r, p=p, point=point)
|
||||
gates[fold] = {"t_block": t_block, "t_quiet": t_quiet, "training_rows": len(train), "normalised_training_rows": len(inner)}
|
||||
return [decided[i] for i in range(len(preds))], gates
|
||||
|
||||
|
||||
def family_floor(rows: Sequence[Prediction], *, floor: int = FAMILY_FLOOR) -> dict[str, dict[str, Any]]:
|
||||
"""Per family, its held-out ``BLOCK`` flags among ``rows`` (the primary population, every fold): it may block only
|
||||
when none is wrong and there are at least ``floor``; otherwise its flags are reported as ADVISORY."""
|
||||
out: dict[str, dict[str, Any]] = {}
|
||||
for family in sorted({r.family for r in rows}):
|
||||
flagged = [r for r in rows if r.family == family and r.point == "BLOCK"]
|
||||
errors = sum(1 for r in flagged if r.label == 0)
|
||||
out[family] = {"flagged": len(flagged), "errors": errors, "floor": floor, "may_block": errors == 0 and len(flagged) >= floor}
|
||||
return out
|
||||
|
||||
|
||||
def apply_floor(rows: Sequence[Prediction], floors: Mapping[str, Mapping[str, Any]]) -> list[Prediction]:
|
||||
"""``BLOCK`` becomes ``ADVISORY`` in every family that does not pass :func:`family_floor`."""
|
||||
return [replace(r, point="ADVISORY") if r.point == "BLOCK" and not floors.get(r.family, {}).get("may_block") else r for r in rows]
|
||||
|
||||
|
||||
def _at(rows: Sequence[Prediction], level: str) -> dict[str, float | None]:
|
||||
flagged = [r for r in rows if r.point in (("BLOCK",) if level == "block" else ("BLOCK", "ADVISORY"))]
|
||||
positives = sum(r.label for r in rows)
|
||||
@@ -326,8 +500,10 @@ def candidate_from_example(
|
||||
|
||||
|
||||
__all__ = [
|
||||
"PARSE_FAILURE_LIMIT", "TARGET_PRECISION", "TARGET_RECALL", "Points", "Prediction", "candidate_from_example", "choose_points",
|
||||
"cluster_bootstrap", "evaluate", "explain", "floor_status", "incompatible", "learning_curve", "nested_points", "predict", "strata",
|
||||
"summary", "targets", "unevaluable_families",
|
||||
"BOUNDS", "FAMILY_FLOOR", "PARSE_FAILURE_LIMIT", "TARGET_PRECISION", "TARGET_RECALL", "Points", "Prediction", "apply_floor",
|
||||
"calibrated_block", "family_floor", "normalised", "pooled_block_gate", "pooled_threshold", "quiet_threshold",
|
||||
"candidate_from_example", "choose_points", "cluster_bootstrap", "evaluate", "explain", "floor_status", "incompatible",
|
||||
"learning_curve", "nested_points", "nested_precision_block", "precision_threshold", "predict", "strata", "summary", "targets",
|
||||
"unevaluable_families",
|
||||
"wilson_lower",
|
||||
] # fmt: skip
|
||||
|
||||
@@ -35,16 +35,30 @@ import math
|
||||
import random
|
||||
import subprocess
|
||||
from collections.abc import Callable, Iterable, Mapping, Sequence
|
||||
from contextlib import suppress
|
||||
from dataclasses import asdict, dataclass, field, fields, replace
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from ..plane.budget import BudgetExhausted
|
||||
from ..plane.budget import BudgetExhausted, MeteredClient
|
||||
from ..plane.memory import MemoryStore, row_id
|
||||
from .compile import load_program, spec_of
|
||||
from .evaluate import candidate_from_example
|
||||
from .evaluate import (
|
||||
BOUNDS,
|
||||
FAMILY_FLOOR,
|
||||
TARGET_PRECISION,
|
||||
TARGET_RECALL,
|
||||
Prediction,
|
||||
apply_floor,
|
||||
calibrated_block,
|
||||
candidate_from_example,
|
||||
family_floor,
|
||||
nested_precision_block,
|
||||
pooled_block_gate,
|
||||
wilson_lower,
|
||||
)
|
||||
from .examples import EXAMPLE_KIND, Example
|
||||
from .folds import Fold, compile_split, folds_digest, outer_folds
|
||||
from .program import Caller, Decision, Program, ProgramSpec
|
||||
@@ -57,7 +71,20 @@ RESULT_KIND = "experiment_result"
|
||||
REGISTER_TASK = "register"
|
||||
POPULATION = "experiments" # the envelope of every experiment row; `split` is the experiment id
|
||||
RUN_TASK = "run"
|
||||
ARM_KINDS = ("program", "plane")
|
||||
ARM_KINDS = ("program", "plane", "rule")
|
||||
# decision-rule experiments (kind: rule): one replay-only scoring pass of the incumbent programs, arms differ only in
|
||||
# the BLOCK rule applied to the same scores
|
||||
RULES = ("calibrated_block", "precision_bound_block", "pooled_block_gate")
|
||||
NORMALISATIONS = ("platt",) # pooled_block_gate: how the scores of different families are put on one scale (Req 6.4)
|
||||
RULE_PARAMETERS: Mapping[str, set[str]] = {
|
||||
"precision_bound_block": {"precision", "bound", "min_flagged", "select_on"},
|
||||
"pooled_block_gate": {"precision", "recall", "floor", "normalise", "select_on"},
|
||||
}
|
||||
POPULATIONS = ("paired", "pooled") # paired: units whose candidate has both sides present; pooled: every unit
|
||||
SCORE_ARM = "score" # the arm name of a rule experiment's outcome rows (the shared scores)
|
||||
MISS_VERDICT = "replay_miss" # an outcome row of a unit excluded because a response was not cached
|
||||
# rule metrics, per family and population of held-out (outer-fold) units: direction only (no paired test)
|
||||
RULE_METRICS: Mapping[str, int] = {"flagged_precision_lb": 1, "flagged_precision": 1, "coverage": 1, "flagged": 1, "quiet_recall": 1}
|
||||
UNITS = ("candidate", "file", "repository")
|
||||
PAIRINGS = ("paired", "assigned")
|
||||
# O'Brien-Fleming boundaries for two equally spaced looks at alpha 0.05, two-sided (design section 6).
|
||||
@@ -70,6 +97,8 @@ ADVISORY_VERDICT = "vulnerable"
|
||||
METRICS: Mapping[str, tuple[int, bool]] = {
|
||||
"auc": (1, False),
|
||||
"within_pair_auc": (1, False),
|
||||
"top_30_precision": (1, False),
|
||||
"unsure_rate": (-1, True),
|
||||
"advisory_recall": (1, True),
|
||||
"advisory_precision": (1, False),
|
||||
"canary_agreement": (1, True),
|
||||
@@ -96,6 +125,7 @@ class Arm:
|
||||
replay_only: bool = False
|
||||
repeats: int = 1
|
||||
budget_usd: float = 0.0
|
||||
rule: Mapping[str, Any] = field(default_factory=dict) # rule arms: {name: calibrated_block | precision_bound_block, ...}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -119,11 +149,11 @@ class Metric:
|
||||
|
||||
@property
|
||||
def direction(self) -> int:
|
||||
return METRICS[self.name][0]
|
||||
return RULE_METRICS[self.name] if self.name in RULE_METRICS else METRICS[self.name][0]
|
||||
|
||||
@property
|
||||
def binary(self) -> bool:
|
||||
return METRICS[self.name][1]
|
||||
return self.name in METRICS and METRICS[self.name][1]
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {k: v for k, v in asdict(self).items() if v is not None}
|
||||
@@ -198,6 +228,7 @@ def load_manifest(path: Path) -> Manifest:
|
||||
bool(spec.get("replay_only", False)),
|
||||
int(spec.get("repeats", 1)),
|
||||
float(spec.get("budget_usd", 0.0)),
|
||||
dict(spec.get("rule") or {}),
|
||||
) # fmt: skip
|
||||
for name, spec in dict(raw["arms"]).items()
|
||||
}
|
||||
@@ -210,9 +241,12 @@ def load_manifest(path: Path) -> Manifest:
|
||||
primary_raw = metric["primary"]
|
||||
primary = tuple(Metric.parse(m) for m in (primary_raw if isinstance(primary_raw, list) else [primary_raw]))
|
||||
secondary = tuple(Metric.parse(m) for m in metric.get("secondary", []))
|
||||
known = RULE_METRICS if kind == "rule" else METRICS
|
||||
for m in (*primary, *secondary):
|
||||
if m.name not in METRICS:
|
||||
raise ExperimentError(f"{path}: metric {m.name!r} (one of {', '.join(METRICS)})")
|
||||
if m.name not in known:
|
||||
raise ExperimentError(f"{path}: metric {m.name!r} (one of {', '.join(known)})")
|
||||
if kind == "rule":
|
||||
_check_rule_manifest(path, raw, arms)
|
||||
test = dict(raw.get("test") or {})
|
||||
stopping = dict(raw.get("stopping") or {"looks": [1.0], "boundaries": [1.959964]})
|
||||
looks = tuple(float(x) for x in stopping.get("looks", [0.5, 1.0]))
|
||||
@@ -244,11 +278,46 @@ def load_manifest(path: Path) -> Manifest:
|
||||
raise ExperimentError(f"{path}: missing {exc}") from exc
|
||||
if manifest.id != path.stem:
|
||||
raise ExperimentError(f"{path}: id {manifest.id!r} must equal the file stem")
|
||||
if kind == "program" and not manifest.families:
|
||||
raise ExperimentError(f"{path}: a program experiment names its families")
|
||||
if kind in ("program", "rule") and not manifest.families:
|
||||
raise ExperimentError(f"{path}: a {kind} experiment names its families")
|
||||
if kind == "rule" and set(manifest.families) - set(manifest.programs):
|
||||
raise ExperimentError(f"{path}: a rule experiment names a program for every family (programs:)")
|
||||
return manifest
|
||||
|
||||
|
||||
def _check_rule_manifest(path: Path, raw: Mapping[str, Any], arms: Mapping[str, Arm]) -> None:
|
||||
"""A decision-rule experiment costs nothing by construction: no arm overrides the program, no budget, and every
|
||||
arm names a known rule with known parameters; the populations are declared before the run."""
|
||||
budget = raw["budget_usd"]
|
||||
total = float(budget["total"] if isinstance(budget, Mapping) else budget)
|
||||
if total != 0 or any(a.budget_usd for a in arms.values()):
|
||||
raise ExperimentError(f"{path}: a rule experiment replays cached responses only: budget_usd must be 0")
|
||||
for arm in arms.values():
|
||||
if arm.spec or arm.task_env or arm.programs:
|
||||
raise ExperimentError(f"{path}: arm {arm.name} of a rule experiment may change the BLOCK rule only (rule:)")
|
||||
name = arm.rule.get("name")
|
||||
if name not in RULES:
|
||||
raise ExperimentError(f"{path}: arm {arm.name} rule.name {name!r} (one of {', '.join(RULES)})")
|
||||
allowed = {"name"} | RULE_PARAMETERS.get(str(name), set())
|
||||
if set(arm.rule) - allowed:
|
||||
raise ExperimentError(f"{path}: arm {arm.name} rule parameters {sorted(set(arm.rule) - allowed)} are not understood")
|
||||
if arm.rule.get("bound", "wilson") not in BOUNDS or arm.rule.get("select_on", "paired") not in POPULATIONS:
|
||||
raise ExperimentError(f"{path}: arm {arm.name} bound in {BOUNDS} and select_on in {POPULATIONS}")
|
||||
if name == "pooled_block_gate":
|
||||
if arm.rule.get("normalise") not in NORMALISATIONS:
|
||||
raise ExperimentError(f"{path}: arm {arm.name} normalise in {NORMALISATIONS} (declared before the run)")
|
||||
if int(arm.rule.get("floor", FAMILY_FLOOR)) < 1 or not 0 < float(arm.rule.get("recall", TARGET_RECALL)) <= 1:
|
||||
raise ExperimentError(f"{path}: arm {arm.name} floor >= 1 and recall in (0, 1]")
|
||||
populations = dict(raw.get("populations") or {})
|
||||
if populations.get("primary") not in POPULATIONS or any(p not in POPULATIONS for p in populations.get("secondary", [])):
|
||||
raise ExperimentError(f"{path}: populations.primary (and secondary) in {POPULATIONS}")
|
||||
decision = dict(raw.get("decision") or {})
|
||||
if not 0 < float(decision.get("target_precision", 0)) <= 1 or decision.get("arm") not in arms:
|
||||
raise ExperimentError(f"{path}: decision.target_precision in (0, 1] and decision.arm naming an arm")
|
||||
if decision.get("pooled") and arms[str(decision["arm"])].rule.get("name") != "pooled_block_gate":
|
||||
raise ExperimentError(f"{path}: decision.pooled judges a pooled_block_gate arm only")
|
||||
|
||||
|
||||
# --- register ------------------------------------------------------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -274,8 +343,22 @@ def registration_id(experiment_id: str) -> str:
|
||||
|
||||
|
||||
def registered(store: MemoryStore, experiment_id: str) -> dict[str, Any] | None:
|
||||
"""Return the authoritative registration, preferring a set units digest over pending rows.
|
||||
|
||||
Within that group, use the greatest ``created`` value when all values are nonempty and mutually
|
||||
comparable (strings compare lexically). Otherwise, or on a tie, use the lexically greatest ``pin``,
|
||||
then the last row in store order (key/id order for MemoryStore, insertion order for an append-only store).
|
||||
"""
|
||||
rows = [r.row for r in store.rows(repo=experiment_repo(experiment_id), kind=EXPERIMENT_KIND) if r.row.get("task") == REGISTER_TASK]
|
||||
return rows[0] if rows else None
|
||||
if not rows:
|
||||
return None
|
||||
rows = [r for r in rows if r.get("units_digest")] or rows
|
||||
rows = sorted(rows, key=lambda r: str(r.get("pin", "")))
|
||||
if all(r.get("created") for r in rows):
|
||||
# Incomparable timestamps retain the deterministic pin/store ordering.
|
||||
with suppress(TypeError):
|
||||
rows = sorted(rows, key=lambda r: r["created"])
|
||||
return rows[-1]
|
||||
|
||||
|
||||
def register(
|
||||
@@ -383,7 +466,7 @@ def program_folds(memory: Sequence[Example], manifest: Manifest) -> tuple[list[F
|
||||
def freeze_units(store: MemoryStore, manifest: Manifest, memory: Sequence[Example]) -> list[Unit]:
|
||||
"""The evaluation candidates of the manifest's families under the incumbent's outer folds, paired by candidate
|
||||
across the two pins where both sides are in memory (no model; the list is then committed and registered)."""
|
||||
if manifest.kind != "program":
|
||||
if manifest.kind not in ("program", "rule"):
|
||||
raise ExperimentError(f"{manifest.id}: units of a plane experiment come from candidate generation, not the memory")
|
||||
folds, _ = program_folds(memory, manifest)
|
||||
sides: dict[str, tuple[str, str, str]] = {}
|
||||
@@ -517,10 +600,10 @@ def auc(rows: Sequence[ArmUnit]) -> float | None:
|
||||
def within_pair_auc(rows: Sequence[ArmUnit]) -> float | None:
|
||||
"""Of the candidate pairs (vulnerable and fixed side both present), the share where the vulnerable side scores
|
||||
higher (ties one half)."""
|
||||
pairs: dict[str, dict[int, float]] = {}
|
||||
pairs: dict[tuple[str, str, str], dict[int, float]] = {}
|
||||
for r in rows:
|
||||
if r.pair and r.s is not None:
|
||||
pairs.setdefault(r.pair, {})[r.label] = r.s
|
||||
pairs.setdefault((r.group, r.family, r.pair), {})[r.label] = r.s
|
||||
done = [(p[1], p[0]) for p in pairs.values() if 0 in p and 1 in p]
|
||||
if not done:
|
||||
return None
|
||||
@@ -535,6 +618,11 @@ def metric_value(metric: Metric, rows: Sequence[ArmUnit]) -> float | None:
|
||||
return auc(rows)
|
||||
if name == "within_pair_auc":
|
||||
return within_pair_auc(rows)
|
||||
if name == "top_30_precision":
|
||||
ranked = sorted((r for r in rows if r.s is not None), key=lambda r: (-r.s if r.s is not None else 0.0, r.unit))
|
||||
return _mean([float(r.label) for r in ranked[:30]]) if len(ranked) >= 30 else None
|
||||
if name == "unsure_rate":
|
||||
return _mean([float(r.verdict == "unsure") for r in rows])
|
||||
if name == "advisory_recall":
|
||||
return _mean([float(r.flag) for r in rows if r.label == 1])
|
||||
if name == "advisory_precision":
|
||||
@@ -558,6 +646,8 @@ def binary_value(metric: Metric, row: ArmUnit) -> bool | None:
|
||||
return row.agree
|
||||
if metric.name == "flag_rate":
|
||||
return row.flag
|
||||
if metric.name == "unsure_rate":
|
||||
return row.verdict == "unsure"
|
||||
return None
|
||||
|
||||
|
||||
@@ -595,7 +685,13 @@ def paired_bootstrap(
|
||||
rng = random.Random(seed)
|
||||
values: list[float] = []
|
||||
for _ in range(resamples if names else 0):
|
||||
sample = [p for g in (rng.choice(names) for _ in names) for p in by_group[g]]
|
||||
# Give each drawn cluster its own identity: repeated draws must retain
|
||||
# repeated candidate pairs when within_pair_auc groups the two sides.
|
||||
sample = [
|
||||
(replace(a, group=f"{draw}:{g}"), replace(b, group=f"{draw}:{g}"))
|
||||
for draw, g in enumerate(rng.choice(names) for _ in names)
|
||||
for a, b in by_group[g]
|
||||
]
|
||||
a = metric_value(metric, [p[0] for p in sample])
|
||||
b = metric_value(metric, [p[1] for p in sample])
|
||||
if a is not None and b is not None:
|
||||
@@ -704,6 +800,9 @@ def decide(manifest: Manifest, finals: Mapping[str, Estimate], cost: Estimate |
|
||||
return "reject", f"the CI of {', '.join(bad)} excludes 0 against B"
|
||||
first = manifest.primary[0].name
|
||||
if sides[first] == "favour":
|
||||
max_ratio = (manifest.raw.get("decision") or {}).get("max_cost_ratio")
|
||||
if max_ratio is not None and (cost is None or cost.a is None or cost.b is None or cost.b > float(max_ratio) * cost.a):
|
||||
return "inconclusive", f"B's cost per candidate is unavailable or exceeds {max_ratio}x A; the incumbent stays"
|
||||
return "adopt", f"the CI of {first} excludes 0 in B's favour and no primary CI excludes 0 against B"
|
||||
if manifest.cost_only and cost is not None and tost is not None and tost.ci is not None and cost.ci is not None:
|
||||
within = -manifest.tost_margin <= tost.ci[0] and tost.ci[1] <= manifest.tost_margin
|
||||
@@ -812,6 +911,8 @@ def run(
|
||||
"""Every unit in every arm, paired, repositories in the pre-shuffled look order, a seeded coin per unit for which
|
||||
arm goes first; outcomes already in the store are not decided again. After the first look's repositories the
|
||||
primary is tested against its boundary and, when crossed, the run stops. A ceiling ends the run ``unfinished``."""
|
||||
if manifest.kind == "rule":
|
||||
raise ExperimentError(f"{manifest.id}: a rule experiment scores once at zero cost (run_rule), its arms differ only in the rule")
|
||||
if manifest.kind != "program":
|
||||
raise ExperimentError(f"{manifest.id}: plane arms run as plane Runs; their outcomes arrive as arm_outcome rows")
|
||||
row = check_registered(store, manifest)
|
||||
@@ -892,7 +993,327 @@ def run(
|
||||
return summary
|
||||
|
||||
|
||||
# --- decision-rule experiments (kind: rule) -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ZeroCostViolation(ExperimentError):
|
||||
"""A model call reached the client of a zero-cost run, or the meter recorded spend: the run is void."""
|
||||
|
||||
|
||||
class _RefusingChat:
|
||||
"""The inner client behind a zero-cost run's meter. The meter's zero ceilings stop every call before it gets here;
|
||||
reaching it means a guard was bypassed."""
|
||||
|
||||
usage: list[Any] = []
|
||||
|
||||
def complete(self, **_: Any) -> Any:
|
||||
raise ZeroCostViolation("a model call reached the client of a zero-cost run")
|
||||
|
||||
|
||||
def zero_cost_caller(model: str, prices: Mapping[str, Any], store: MemoryStore | None) -> tuple[Caller, MeteredClient]:
|
||||
"""A caller that answers only from the response cache: its client is a meter with zero ceilings (USD and calls)
|
||||
around a client that refuses. A response that is not cached raises ``BudgetExhausted`` before any call; the run
|
||||
excludes that unit and counts it. The meter is the proof: its ``calls`` and ``usd`` must stay 0."""
|
||||
meter = MeteredClient(_RefusingChat(), prices=prices, budget_usd=0.0, budget_calls=0) # type: ignore[arg-type]
|
||||
return Caller(meter, model, prices, store=store), meter
|
||||
|
||||
|
||||
def _checked_units(manifest: Manifest, memory: Sequence[Example]) -> tuple[list[Unit], dict[str, Fold], str]:
|
||||
units = read_units(manifest.units_file)
|
||||
folds, fdigest = program_folds(memory, manifest)
|
||||
by_name = {f.name: f for f in folds}
|
||||
index = {e.id for e in memory}
|
||||
for unit in units:
|
||||
fold = by_name.get(unit.fold)
|
||||
if fold is None or unit.group not in fold.eval_groups or unit.unit not in index:
|
||||
raise ExperimentError(
|
||||
f"{manifest.id}: unit {unit.unit[:12]} is not under fold {unit.fold!r} of this memory; the memory changed"
|
||||
)
|
||||
return units, by_name, fdigest
|
||||
|
||||
|
||||
def run_rule(
|
||||
store: MemoryStore,
|
||||
manifest: Manifest,
|
||||
memory: Sequence[Example],
|
||||
caller: Caller,
|
||||
meter: MeteredClient,
|
||||
excerpt_text: Callable[[str], str | None],
|
||||
vectors: Mapping[str, Sequence[float]] | None = None,
|
||||
*,
|
||||
on_decision: Callable[[Outcome, Decision], None] | None = None,
|
||||
) -> RunSummary:
|
||||
"""The one scoring pass of a rule experiment: every unit decided once by its family's incumbent program, answered
|
||||
only from the response cache. A unit whose responses are not all cached is excluded and counted (an outcome row
|
||||
with ``verdict = replay_miss`` and no score), never asked. Raises :class:`ZeroCostViolation` when the meter shows
|
||||
a call or any spend afterwards."""
|
||||
if manifest.kind != "rule":
|
||||
raise ExperimentError(f"{manifest.id}: run_rule runs decision-rule experiments only")
|
||||
if caller.client is not meter or meter.budget_usd != 0.0 or meter.budget_calls != 0:
|
||||
raise ZeroCostViolation(f"{manifest.id}: a rule experiment runs only behind a zero-ceiling meter (zero_cost_caller)")
|
||||
row = check_registered(store, manifest)
|
||||
units, by_name, fdigest = _checked_units(manifest, memory)
|
||||
index = {e.id: e for e in memory}
|
||||
programs = {f: Program(arm_spec(manifest, manifest.arms["A"], f, store), memory, excerpt_text, vectors) for f in manifest.families}
|
||||
models = {spec_of(load_program(store, manifest.programs[f])).model for f in manifest.families}
|
||||
if models != {caller.model}:
|
||||
raise ExperimentError(f"{manifest.id}: the programs' models {sorted(models)} differ from the caller's {caller.model!r}")
|
||||
candidate_of = candidate_from_example(excerpt_text, vectors)
|
||||
have = {outcome_id(manifest.id, o.arm, o.unit, o.replicate) for o in load_outcomes(store, manifest.id)}
|
||||
decided, skipped = {SCORE_ARM: 0}, {"already": 0, MISS_VERDICT: 0}
|
||||
fold_seed = int(manifest.folds.get("seed", 0))
|
||||
for unit in sorted(units, key=lambda u: (u.group, u.family, u.unit)):
|
||||
if outcome_id(manifest.id, SCORE_ARM, unit.unit, 0) in have:
|
||||
skipped["already"] += 1
|
||||
continue
|
||||
candidate = candidate_of(index[unit.unit])
|
||||
if candidate is None:
|
||||
raise ExperimentError(f"{manifest.id}: unit {unit.unit[:12]} has no excerpt in the store")
|
||||
try:
|
||||
decision = programs[unit.family].decide(candidate, by_name[unit.fold], caller, seed=fold_seed)
|
||||
except BudgetExhausted: # a response is not cached: the zero ceiling stopped the call before it started
|
||||
outcome = Outcome(unit.unit, unit.group, unit.family, unit.label, unit.pair, SCORE_ARM, 0, None, MISS_VERDICT, 0.0)
|
||||
skipped[MISS_VERDICT] += 1
|
||||
store.put_row(outcome_row(manifest.id, str(row["pin"]), outcome))
|
||||
continue
|
||||
outcome = Outcome(unit.unit, unit.group, unit.family, unit.label, unit.pair, SCORE_ARM, 0, decision.s, decision.verdict, 0.0,
|
||||
decision.parse_failed) # fmt: skip
|
||||
store.put_row(outcome_row(manifest.id, str(row["pin"]), outcome))
|
||||
decided[SCORE_ARM] += 1
|
||||
if on_decision is not None:
|
||||
on_decision(outcome, decision)
|
||||
if meter.calls or (meter.usd or 0.0) > 0 or caller.calls:
|
||||
raise ZeroCostViolation(f"{manifest.id}: the meter recorded {meter.calls} calls and ${meter.usd}; the run is void")
|
||||
spend = {SCORE_ARM: {"usd": meter.usd, "client_calls": meter.calls, "replayed": caller.replayed, "cache_digest": caller.cache_digest(),
|
||||
"replay_priced_usd": caller.usd()}} # fmt: skip
|
||||
summary = RunSummary(manifest.id, "done", len(units), decided, skipped, spend, fdigest)
|
||||
store.put_row({
|
||||
"id": row_id(EXPERIMENT_KIND, f"experiment-{manifest.id}", RUN_TASK, manifest.id), "kind": EXPERIMENT_KIND,
|
||||
"repo": experiment_repo(manifest.id), "pin": str(row["pin"]), "run": f"experiment-{manifest.id}", "task": RUN_TASK,
|
||||
"population": POPULATION, "split": manifest.id, "image": "host", "experiment": manifest.id,
|
||||
"manifest_digest": manifest.digest, **summary.as_dict(),
|
||||
}) # fmt: skip
|
||||
return summary
|
||||
|
||||
|
||||
def min_flagged_for_bound(target: float, z: float = 1.959964) -> int:
|
||||
"""The fewest flagged units, all correct, whose Wilson lower bound reaches ``target`` (73 at 0.95)."""
|
||||
n = 1
|
||||
while wilson_lower(n, n, z) < target:
|
||||
n += 1
|
||||
return n
|
||||
|
||||
|
||||
def _group_bootstrap_precision(rows: Sequence[Prediction], *, resamples: int, seed: int) -> list[float] | None:
|
||||
by_group: dict[str, list[Prediction]] = {}
|
||||
for r in rows:
|
||||
by_group.setdefault(r.group, []).append(r)
|
||||
names = sorted(by_group)
|
||||
rng = random.Random(seed)
|
||||
values = []
|
||||
for _ in range(resamples if names else 0):
|
||||
sample = [r for g in (rng.choice(names) for _ in names) for r in by_group[g]]
|
||||
flagged = [r for r in sample if r.point == "BLOCK"]
|
||||
if flagged:
|
||||
values.append(sum(r.label for r in flagged) / len(flagged))
|
||||
if len(values) < 2:
|
||||
return None
|
||||
values.sort()
|
||||
return [values[int(0.025 * (len(values) - 1))], values[int(0.975 * (len(values) - 1))]]
|
||||
|
||||
|
||||
def flag_summary(rows: Sequence[Prediction], target: float, *, resamples: int = 2000, seed: int = 0) -> dict[str, Any]:
|
||||
"""Held-out BLOCK decisions of one population: flagged count, flagged precision with its Wilson 95% lower bound
|
||||
and a repository-cluster bootstrap interval, coverage (the share of positives flagged), and the M4 reading."""
|
||||
flagged = [r for r in rows if r.point == "BLOCK"]
|
||||
hits = sum(r.label for r in flagged)
|
||||
positives = sum(r.label for r in rows)
|
||||
precision = hits / len(flagged) if flagged else None
|
||||
lower = wilson_lower(hits, len(flagged)) if flagged else None
|
||||
return {
|
||||
"units": len(rows), "groups": len({r.group for r in rows}), "positives": positives, "flagged": len(flagged),
|
||||
"true_flagged": hits, "flagged_precision": None if precision is None else round(precision, 4),
|
||||
"flagged_precision_lb": None if lower is None else round(lower, 4),
|
||||
"flagged_precision_ci": _group_bootstrap_precision(rows, resamples=resamples, seed=seed) if flagged else None,
|
||||
"coverage": round(hits / positives, 4) if positives else None,
|
||||
"meets_target_point": precision is not None and precision >= target,
|
||||
"meets_target_lb": lower is not None and lower >= target,
|
||||
} # fmt: skip
|
||||
|
||||
|
||||
def _apply_rule(arm: Arm, preds: Sequence[Prediction], seed: int) -> tuple[list[Prediction], dict[str, Any]]:
|
||||
if arm.rule["name"] == "calibrated_block":
|
||||
return calibrated_block(preds, seed=seed)
|
||||
select_on = str(arm.rule.get("select_on", "paired"))
|
||||
if arm.rule["name"] == "pooled_block_gate": # the family floor is applied by analyse_rule, on held-out flags
|
||||
rows, gates = pooled_block_gate(
|
||||
preds, eligible=(lambda r: bool(r.pair)) if select_on == "paired" else None,
|
||||
precision=float(arm.rule.get("precision", TARGET_PRECISION)), recall=float(arm.rule.get("recall", TARGET_RECALL)),
|
||||
) # fmt: skip
|
||||
info = {
|
||||
f: {"t_block_by_fold": {j: g["t_block"] for j, g in gates.items()},
|
||||
"t_quiet_by_fold": {j: g["t_quiet"].get(f) for j, g in gates.items()}}
|
||||
for f in sorted({r.family for r in preds})
|
||||
} # fmt: skip
|
||||
return rows, info
|
||||
decided, thresholds = nested_precision_block(
|
||||
preds, eligible=(lambda r: bool(r.pair)) if select_on == "paired" else None,
|
||||
precision=float(arm.rule.get("precision", TARGET_PRECISION)), bound=str(arm.rule.get("bound", "wilson")),
|
||||
min_flagged=int(arm.rule.get("min_flagged", 1)),
|
||||
) # fmt: skip
|
||||
return decided, {f: {"thresholds_by_fold": t} for f, t in thresholds.items()}
|
||||
|
||||
|
||||
def decide_pooled(manifest: Manifest, families: Mapping[str, Mapping[str, Any]], overall: Mapping[str, Any]) -> tuple[str, str]:
|
||||
"""The pre-registered rule of a pooled gate (Req 6.4, exp-004), on the primary population's held-out flags of
|
||||
every family together, before the family floor: **reject** when they flag at a point precision below the target;
|
||||
**adopt** when their Wilson 95% lower bound reaches the target and at least one family passes the floor (no
|
||||
held-out error, at least ``floor`` flags) -- those families block, the others stay advisory; otherwise
|
||||
**inconclusive** (the incumbent rule stays)."""
|
||||
decision = dict(manifest.raw["decision"])
|
||||
arm, target = str(decision["arm"]), float(decision["target_precision"])
|
||||
primary = str(manifest.raw["populations"]["primary"])
|
||||
cell = overall[arm][primary]
|
||||
if cell["flagged"] and not cell["meets_target_point"]:
|
||||
return "reject", f"{arm} flags held-out {primary} units of all families together below precision {target}"
|
||||
passing = sorted(f for f, v in families.items() if v[arm]["floor"]["may_block"])
|
||||
if cell["meets_target_lb"] and passing:
|
||||
return "adopt", f"{arm}'s pooled held-out Wilson lower bound reaches {target}; {', '.join(passing)} pass the family floor and block"
|
||||
if not cell["flagged"]:
|
||||
return "inconclusive", f"{arm} offers no BLOCK on held-out {primary} units; the incumbent stays"
|
||||
if not cell["meets_target_lb"]:
|
||||
return "inconclusive", f"{arm}'s pooled held-out Wilson lower bound is below {target}; the incumbent stays"
|
||||
return "inconclusive", f"{arm} reaches {target} pooled but no family passes the floor; every family stays advisory"
|
||||
|
||||
|
||||
def decide_rule(manifest: Manifest, families: Mapping[str, Mapping[str, Any]], overall: Mapping[str, Any] | None = None) -> tuple[str, str]:
|
||||
"""The pre-registered rule: **reject** the decision arm when it flags held-out units of the primary population in
|
||||
some family at a point precision below the target; **adopt** it when it does not and some family's held-out
|
||||
Wilson lower bound reaches the target; otherwise **inconclusive** (the incumbent rule stays). A manifest whose
|
||||
``decision.pooled`` is set is judged by :func:`decide_pooled` instead."""
|
||||
decision = dict(manifest.raw["decision"])
|
||||
if decision.get("pooled"):
|
||||
if overall is None:
|
||||
raise ExperimentError(f"{manifest.id}: a pooled decision needs the pooled-over-families totals")
|
||||
return decide_pooled(manifest, families, overall)
|
||||
arm, target = str(decision["arm"]), float(decision["target_precision"])
|
||||
primary = str(manifest.raw["populations"]["primary"])
|
||||
cells = {f: v[arm][primary] for f, v in families.items()}
|
||||
low = sorted(f for f, c in cells.items() if c["flagged"] and not c["meets_target_point"])
|
||||
if low:
|
||||
return "reject", f"{arm} flags held-out {primary} units below precision {target} in {', '.join(low)}"
|
||||
reached = sorted(f for f, c in cells.items() if c["meets_target_lb"])
|
||||
if reached:
|
||||
return "adopt", f"{arm}'s held-out Wilson lower bound reaches {target} in {', '.join(reached)} and nowhere flags below it"
|
||||
if not any(c["flagged"] for c in cells.values()):
|
||||
return "inconclusive", f"{arm} offers no BLOCK on held-out {primary} units in any family; the incumbent stays"
|
||||
return "inconclusive", f"{arm} flags only at a Wilson lower bound below {target}; the incumbent stays"
|
||||
|
||||
|
||||
def analyse_rule(
|
||||
manifest: Manifest,
|
||||
outcomes: Sequence[Outcome],
|
||||
units: Sequence[Unit],
|
||||
*,
|
||||
resamples: int | None = None,
|
||||
provenance: Mapping[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Counts and intervals only: every arm's rule applied to the same held-out scores, per family and population,
|
||||
the pooled-over-families totals, the exclusions, the training capacity of each family against the Wilson bound,
|
||||
and the pre-registered verdict."""
|
||||
n = resamples if resamples is not None else min(manifest.resamples, 2000)
|
||||
target = float(manifest.raw["decision"]["target_precision"])
|
||||
populations = [str(manifest.raw["populations"]["primary"]), *[str(p) for p in manifest.raw["populations"].get("secondary", [])]]
|
||||
by_unit = {u.unit: u for u in units}
|
||||
scores = {o.unit: o for o in outcomes if o.arm == SCORE_ARM and o.replicate == 0 and o.unit in by_unit}
|
||||
preds = [
|
||||
Prediction(by_unit[u].fold, o.family, o.group, o.label, float(o.s), o.verdict, parse_failed=o.parse_failed, pair=o.pair)
|
||||
for u, o in sorted(scores.items())
|
||||
if o.s is not None
|
||||
] # fmt: skip
|
||||
excluded = {f: sum(1 for o in scores.values() if o.s is None and o.family == f) for f in manifest.families}
|
||||
not_run = {f: sum(1 for u in units if u.family == f and u.unit not in scores) for f in manifest.families}
|
||||
|
||||
def select(rows: Sequence[Prediction], population: str) -> list[Prediction]:
|
||||
return [r for r in rows if r.pair] if population == "paired" else list(rows)
|
||||
|
||||
decided = {name: _apply_rule(arm, preds, manifest.seed) for name, arm in manifest.arms.items()}
|
||||
gated = {name for name, arm in manifest.arms.items() if arm.rule.get("name") == "pooled_block_gate"}
|
||||
floors = { # held-out flags of the primary population decide which families may block; no threshold moves
|
||||
name: {f: family_floor([r for r in select(decided[name][0], populations[0]) if r.family == f],
|
||||
floor=int(manifest.arms[name].rule.get("floor", FAMILY_FLOOR))).get(f)
|
||||
or {"flagged": 0, "errors": 0, "floor": int(manifest.arms[name].rule.get("floor", FAMILY_FLOOR)), "may_block": False}
|
||||
for f in manifest.families}
|
||||
for name in gated
|
||||
} # fmt: skip
|
||||
|
||||
def quiet_recall(rows: Sequence[Prediction]) -> float | None:
|
||||
positives = [r for r in rows if r.label == 1]
|
||||
return round(sum(1 for r in positives if r.point in ("BLOCK", "ADVISORY")) / len(positives), 4) if positives else None
|
||||
|
||||
families: dict[str, dict[str, Any]] = {}
|
||||
for family in manifest.families:
|
||||
cell: dict[str, Any] = {}
|
||||
for name, (rows, info) in decided.items():
|
||||
mine = [r for r in rows if r.family == family]
|
||||
cell[name] = {**info.get(family, {}), **{p: flag_summary(select(mine, p), target, resamples=n, seed=manifest.seed)
|
||||
for p in populations}} # fmt: skip
|
||||
if name in gated:
|
||||
cell[name]["floor"] = floors[name][family]
|
||||
cell[name]["quiet_recall"] = {p: quiet_recall(select(mine, p)) for p in populations}
|
||||
mine = [r for r in preds if r.family == family]
|
||||
folds = sorted({r.fold for r in mine})
|
||||
cell["capacity"] = {
|
||||
"evaluated": len(mine), "excluded_replay_miss": excluded[family], "not_run": not_run[family],
|
||||
"paired_positives_by_training_folds": {f: sum(r.label for r in mine if r.pair and r.fold != f) for f in folds},
|
||||
} # fmt: skip
|
||||
families[family] = cell
|
||||
overall = {
|
||||
name: {p: flag_summary(select(rows, p), target, resamples=n, seed=manifest.seed) for p in populations}
|
||||
for name, (rows, _) in decided.items()
|
||||
}
|
||||
after_floor = { # what users would get: BLOCK only in the families that pass the floor (secondary, not deciding)
|
||||
name: {p: flag_summary(select(apply_floor(decided[name][0], floors[name]), p), target, resamples=n, seed=manifest.seed)
|
||||
for p in populations}
|
||||
for name in sorted(gated)
|
||||
} # fmt: skip
|
||||
verdict, reason = decide_rule(manifest, families, overall)
|
||||
spend = {SCORE_ARM: round(sum(o.usd or 0.0 for o in outcomes if o.arm == SCORE_ARM), 6)}
|
||||
return {
|
||||
"experiment": manifest.id, "manifest_digest": manifest.digest, "hypothesis": manifest.hypothesis,
|
||||
"arms": {a: {"label": arm.label, "rule": dict(arm.rule)} for a, arm in manifest.arms.items()},
|
||||
"units": {"frozen": len(units), "evaluated": len(preds), "excluded_replay_miss": sum(excluded.values()),
|
||||
"not_run": sum(not_run.values()), "paired_evaluated": sum(1 for r in preds if r.pair),
|
||||
"groups": len({r.group for r in preds}), "by_label": {str(lab): sum(r.label == lab for r in preds) for lab in (0, 1)},
|
||||
"unsure": sum(1 for r in preds if not r.blockable)},
|
||||
"populations": {"primary": populations[0], "secondary": populations[1:]}, "target_precision": target,
|
||||
"min_flagged_for_bound": min_flagged_for_bound(target), "primary": [m.as_dict() for m in manifest.primary],
|
||||
"secondary": [m.as_dict() for m in manifest.secondary], "families": families, "overall": overall, "resamples": n,
|
||||
**({"overall_after_floor": after_floor} if after_floor else {}), "decision": verdict, "reason": reason, "spend_usd": spend,
|
||||
**dict(provenance or {}),
|
||||
} # fmt: skip
|
||||
|
||||
|
||||
def rule_result_row(manifest: Manifest, pin: str, result: Mapping[str, Any]) -> dict[str, Any]:
|
||||
"""The ``experiment_result`` row of a rule experiment, with the column types of :func:`result_row`."""
|
||||
primary = result["populations"]["primary"]
|
||||
tests = {
|
||||
f: {a: {k: cell[a][primary][k] for k in ("flagged", "flagged_precision", "flagged_precision_lb", "coverage")}
|
||||
for a in manifest.arms}
|
||||
for f, cell in result["families"].items()
|
||||
} # fmt: skip
|
||||
return {
|
||||
"id": row_id(RESULT_KIND, f"experiment-{manifest.id}", "analyse", manifest.id), "kind": RESULT_KIND,
|
||||
"repo": experiment_repo(manifest.id), "pin": pin, "run": f"experiment-{manifest.id}", "task": "analyse", "population": POPULATION,
|
||||
"split": manifest.id, "image": "host", "experiment": manifest.id, "manifest_digest": manifest.digest,
|
||||
"decision": result["decision"], "reason": result["reason"], "unit_counts": result["units"], "spend_usd": result["spend_usd"],
|
||||
"tests": tests, "looks_taken": [],
|
||||
} # fmt: skip
|
||||
|
||||
|
||||
__all__ = [
|
||||
"MISS_VERDICT", "POPULATIONS", "RULES", "RULE_METRICS", "SCORE_ARM", "ZeroCostViolation", "analyse_rule", "decide_rule",
|
||||
"flag_summary", "min_flagged_for_bound", "rule_result_row", "run_rule", "zero_cost_caller",
|
||||
"ALPHA", "ARM_KINDS", "EXPERIMENT_KIND", "MANIFEST_DIR", "METRICS", "OBF_TWO_LOOKS", "OUTCOME_KIND", "RESAMPLES", "RESULT_DIR",
|
||||
"RESULT_KIND", "TOST_MARGIN", "VERDICTS", "Arm", "ArmUnit", "Estimate", "ExperimentError", "Look", "Manifest", "McNemar", "Metric",
|
||||
"Outcome", "RunSummary", "Unit", "analyse", "arm_spec", "arm_units", "auc", "check_registered", "committed_at_head", "decide",
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
"""Fill experiment responses without writing scores or changing the frozen population.
|
||||
|
||||
Estimates use the family's mean billed cached response cost when usage is available.
|
||||
Otherwise they use Prompt.estimated_tokens and 300 output tokens. Unknown answers may require one retry;
|
||||
dry-run counts those separately as conditional requests, never fabricating an answer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from collections.abc import Callable, Mapping, Sequence
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from ..model.endpoint import DeepSeekChatClient
|
||||
from ..plane.budget import BudgetExhausted, MeteredClient, cost_of, prices_from
|
||||
from ..plane.memory import MemoryStore
|
||||
from ..tool_hunter import ChatClient, ChatResponse
|
||||
from . import experiments as ex
|
||||
from .examples import Example
|
||||
from .program import Caller, Program, Prompt, parse_answer, request_key
|
||||
|
||||
DEFAULT_OUTPUT_TOKENS = 300
|
||||
DEFAULT_MARGIN_USD = 0.01
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Request:
|
||||
family: str
|
||||
prompt: Prompt
|
||||
temperature: float
|
||||
sample: str
|
||||
|
||||
def key(self, caller: Caller) -> str:
|
||||
return request_key(caller.model, caller.params_digest, self.prompt.messages, self.temperature, self.sample, True)
|
||||
|
||||
def ask(self, caller: Caller) -> str:
|
||||
return caller.ask(self.prompt.messages, temperature=self.temperature, sample=self.sample)
|
||||
|
||||
def retry(self) -> Request:
|
||||
return Request(self.family, self.prompt, 0.0, f"{self.sample}:retry")
|
||||
|
||||
|
||||
class FillMeter(MeteredClient):
|
||||
"""Reserve a whole call before admission, including the first (smoke) call.
|
||||
|
||||
The fixed cent is a floor, never replaced by a smaller observed cost. For the
|
||||
production adapter, UTF-8 bytes conservatively bound input tokens (plus 256 framing
|
||||
tokens per message), and the provider enforces the output limit. Injected clients
|
||||
must honour the same token/usage contract. No hidden transport retries are allowed.
|
||||
"""
|
||||
|
||||
def __init__(self, client: ChatClient, parameters: Mapping[str, Any], budget: float, output_limit: int) -> None:
|
||||
super().__init__(client, prices=parameters, budget_usd=budget)
|
||||
self.limit = budget
|
||||
self.output_limit = output_limit
|
||||
self.largest_call_usd = 0.0
|
||||
|
||||
def complete(self, **kwargs: Any) -> ChatResponse:
|
||||
assert self.prices is not None
|
||||
messages = kwargs["messages"]
|
||||
input_bound = sum(len(str(m["content"]).encode("utf-8")) + 256 for m in messages)
|
||||
bound = (
|
||||
input_bound * max(self.prices.input_per_m, self.prices.cache_hit_per_m) + self.output_limit * self.prices.output_per_m
|
||||
) / 1_000_000
|
||||
reserve = max(DEFAULT_MARGIN_USD, self.largest_call_usd, bound)
|
||||
self.budget_usd = max(0.0, self.limit - reserve)
|
||||
before = self.usd or 0.0
|
||||
tokens_before = sum(self.usage.values())
|
||||
try:
|
||||
response = super().complete(**kwargs)
|
||||
except BudgetExhausted:
|
||||
raise
|
||||
except Exception:
|
||||
self.calls += 1 # attempted provider calls count even when no response was returned
|
||||
raise
|
||||
finally:
|
||||
self.largest_call_usd = max(self.largest_call_usd, (self.usd or 0.0) - before)
|
||||
if sum(self.usage.values()) <= tokens_before:
|
||||
raise RuntimeError("provider returned no metered token usage")
|
||||
return response
|
||||
|
||||
|
||||
def fill_experiment(
|
||||
store: MemoryStore,
|
||||
manifest: ex.Manifest,
|
||||
memory: Sequence[Example],
|
||||
parameters: Mapping[str, Any],
|
||||
excerpt_text: Callable[[str], str | None],
|
||||
vectors: Mapping[str, Sequence[float]] | None = None,
|
||||
*,
|
||||
client: ChatClient | None = None,
|
||||
client_factory: Callable[[], ChatClient] | None = None,
|
||||
budget_usd: float | None = None,
|
||||
dry_run: bool = False,
|
||||
arms: Sequence[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Inspect or fill every frozen unit. The first cache miss is the smoke call.
|
||||
|
||||
A successful smoke is retained in the cache; any provider error stops immediately
|
||||
and is reported without its message, which may contain credentials. A factory is
|
||||
resolved only after validation, and only when a paid request is actually missing.
|
||||
"""
|
||||
from .evaluate import candidate_from_example
|
||||
|
||||
if manifest.kind not in ("rule", "program"):
|
||||
raise ex.ExperimentError("experiment fill supports kind: rule or program only")
|
||||
selected = list(dict.fromkeys(arms if arms is not None else (["A"] if manifest.kind == "rule" else manifest.paid_arms)))
|
||||
if not selected or any(name not in manifest.arms for name in selected):
|
||||
raise ex.ExperimentError("fill requires at least one known arm")
|
||||
# Rule arms share exactly one scoring pass, regardless of the selected rule.
|
||||
scoring = selected[:1] if manifest.kind == "rule" else selected
|
||||
if not dry_run and (budget_usd is None or not math.isfinite(budget_usd) or budget_usd < 0):
|
||||
raise ex.ExperimentError("--budget-usd must be a finite non-negative number (or use --dry-run)")
|
||||
if manifest.kind == "program" and not dry_run:
|
||||
ceiling = min(manifest.budget_total_usd, sum(manifest.arms[name].budget_usd for name in selected))
|
||||
if budget_usd is not None and budget_usd > ceiling:
|
||||
raise ex.ExperimentError(f"--budget-usd exceeds the selected program arms' manifest ceiling ({ceiling})")
|
||||
prices = prices_from(parameters)
|
||||
if prices is None or any(not math.isfinite(p) or p < 0 for p in (prices.input_per_m, prices.output_per_m, prices.cache_hit_per_m)):
|
||||
raise ex.ExperimentError("fill requires finite, non-negative Model prices")
|
||||
ex.check_registered(store, manifest)
|
||||
units, folds, _ = ex._checked_units(manifest, memory)
|
||||
programs = {
|
||||
(name, f): Program(ex.arm_spec(manifest, manifest.arms[name], f, store), memory, excerpt_text, vectors)
|
||||
for name in scoring
|
||||
for f in manifest.families
|
||||
}
|
||||
models = {p.spec.model for p in programs.values()}
|
||||
if len(models) != 1:
|
||||
raise ex.ExperimentError("the selected programs must use the same model")
|
||||
caller = Caller(None, next(iter(models)), parameters, store=store)
|
||||
index = {e.id: e for e in memory}
|
||||
candidate_of = candidate_from_example(excerpt_text, vectors)
|
||||
requests: list[Request] = []
|
||||
for unit in sorted(units, key=lambda u: (u.group, u.family, u.unit)):
|
||||
candidate = candidate_of(index[unit.unit])
|
||||
if candidate is None:
|
||||
raise ex.ExperimentError(f"{manifest.id}: unit {unit.unit[:12]} has no excerpt in the store")
|
||||
for name in scoring:
|
||||
prepared = programs[name, unit.family].prepare_samples(candidate, folds[unit.fold], seed=int(manifest.folds.get("seed", 0)))
|
||||
repeats = manifest.arms[name].repeats if manifest.kind == "program" else 1
|
||||
for replicate in range(repeats):
|
||||
salt = f"r{replicate}:" if replicate else ""
|
||||
for prompt, _, temperature, sample in prepared:
|
||||
requests.append(Request(unit.family, prompt, temperature, f"{salt}{sample}"))
|
||||
|
||||
# Request counts are distinct cache keys per family, even when multiple units share one.
|
||||
def inventory() -> dict[str, dict[str, Request]]:
|
||||
result: dict[str, dict[str, Request]] = {f: {} for f in manifest.families}
|
||||
for req in requests:
|
||||
result[req.family][req.key(caller)] = req
|
||||
entry = caller._cached(req.key(caller))
|
||||
if entry is not None and parse_answer(entry.get("content"), req.prompt.valid_lines) is None:
|
||||
retry = req.retry()
|
||||
result[req.family][retry.key(caller)] = retry
|
||||
return result
|
||||
|
||||
initial = inventory()
|
||||
initially_cached = {key for cells in initial.values() for key in cells if caller._cached(key) is not None}
|
||||
families: dict[str, Any] = {}
|
||||
for family, cells in initial.items():
|
||||
usages = [
|
||||
usage
|
||||
for key in cells
|
||||
if (entry := caller._cached(key)) is not None and isinstance(usage := entry.get("usage"), Mapping) and usage
|
||||
]
|
||||
outputs = [float(usage["completion_tokens"]) for usage in usages if "completion_tokens" in usage]
|
||||
mean_output = sum(outputs) / len(outputs) if outputs else DEFAULT_OUTPUT_TOKENS
|
||||
missing = [req for key, req in cells.items() if key not in initially_cached]
|
||||
families[family] = {
|
||||
"units": sum(u.family == family for u in units),
|
||||
"requests": len(cells),
|
||||
"cached": len(cells) - len(missing),
|
||||
"missing": len(missing),
|
||||
"filled": 0,
|
||||
"still_missing": len(missing),
|
||||
"estimate_method": "measured" if usages else "estimated",
|
||||
"estimated_usd": len(missing) * sum(cost_of(usage, prices) for usage in usages) / len(usages)
|
||||
if usages
|
||||
else sum((r.prompt.estimated_tokens * prices.input_per_m + mean_output * prices.output_per_m) / 1_000_000 for r in missing),
|
||||
"estimated_output_tokens": mean_output,
|
||||
"conditional_retries": sum(not r.sample.endswith(":retry") for r in missing),
|
||||
}
|
||||
report: dict[str, Any] = {
|
||||
"experiment": manifest.id,
|
||||
"arms": selected,
|
||||
"status": "dry_run" if dry_run else "done",
|
||||
"families": families,
|
||||
"client_calls": 0,
|
||||
"usd": 0.0,
|
||||
"reason": "",
|
||||
}
|
||||
if dry_run or not any(c["missing"] for c in families.values()):
|
||||
return report
|
||||
meter: FillMeter | None = None
|
||||
try:
|
||||
if client is None and client_factory is not None:
|
||||
client = client_factory()
|
||||
if client is None:
|
||||
raise ex.ExperimentError("no chat endpoint configured; nothing was spent")
|
||||
output_limit = max(p.spec.max_output_tokens for p in programs.values())
|
||||
if isinstance(client, DeepSeekChatClient):
|
||||
client = client.bounded(output_limit)
|
||||
assert budget_usd is not None
|
||||
meter = FillMeter(client, parameters, budget_usd, output_limit)
|
||||
caller.client = meter
|
||||
for req in requests:
|
||||
content = req.ask(caller)
|
||||
if parse_answer(content, req.prompt.valid_lines) is None:
|
||||
req.retry().ask(caller)
|
||||
except BudgetExhausted:
|
||||
report.update(status="unfinished", reason="budget exhausted; resume with a new fill budget")
|
||||
except ex.ExperimentError:
|
||||
raise
|
||||
except Exception:
|
||||
# Never echo provider exceptions: response bodies and URLs may contain secrets.
|
||||
report.update(status="failed", reason="provider or cache failure; fill stopped")
|
||||
final = inventory()
|
||||
for family, cells in final.items():
|
||||
cached = {key for key in cells if caller._cached(key) is not None}
|
||||
families[family].update(requests=len(cells), filled=len(cached - initially_cached), still_missing=len(cells) - len(cached))
|
||||
if meter is not None:
|
||||
report.update(client_calls=meter.calls, usd=meter.usd)
|
||||
return report
|
||||
|
||||
|
||||
# Backwards-compatible name for callers of the original rule-only API.
|
||||
fill_rule = fill_experiment
|
||||
@@ -38,6 +38,7 @@ from .examples import Example
|
||||
from .folds import Fold
|
||||
from .retrieve import Retrieval, Target, assert_boundary, demonstrations_for, retrieve
|
||||
from .schema import DEFAULT_INPUTS, features_for, instruments_for, validate_x, withheld
|
||||
from .slice import function_slice, render_slice
|
||||
|
||||
VERDICTS = ("vulnerable", "not_vulnerable", "unsure")
|
||||
UNSURE = "unsure"
|
||||
@@ -150,9 +151,11 @@ def _case(
|
||||
profile: str,
|
||||
language: str,
|
||||
inputs: str = DEFAULT_INPUTS,
|
||||
slice: bool = False,
|
||||
) -> str:
|
||||
signals = render_signals(x, instruments, profile, inputs)
|
||||
return f"Family: {family}\nLanguage: {language}\nCode:\n{code.rstrip()}\nSignals:\n{signals}\nRoles:\n{render_roles(roles)}"
|
||||
flow = "\n" + render_slice(function_slice(code, language, family), language) if slice else ""
|
||||
return f"Family: {family}\nLanguage: {language}\nCode:\n{code.rstrip()}{flow}\nSignals:\n{signals}\nRoles:\n{render_roles(roles)}"
|
||||
|
||||
|
||||
def label_word(label: int) -> str:
|
||||
@@ -204,6 +207,7 @@ class ProgramSpec:
|
||||
priors: str = "off"
|
||||
inputs: str = DEFAULT_INPUTS # the input profile: which instruments the classifier never sees (v1: the engine)
|
||||
max_output_tokens: int = MAX_OUTPUT_TOKENS
|
||||
slice: bool = False # opt-in in-function def-use evidence, computed from the same excerpt
|
||||
sampling: str = SAMPLING[0] # temperature | retrieval_ensemble (an experiment arm's setting; never compiled)
|
||||
|
||||
|
||||
@@ -252,7 +256,9 @@ def render_prefix(spec: ProgramSpec, demos: Sequence[Demo], examples: Mapping[st
|
||||
"cited_lines": list(demo.cited_lines)} # fmt: skip
|
||||
parts.append(
|
||||
f"Demonstration {number}:\n"
|
||||
+ _case(code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs)
|
||||
+ _case(
|
||||
code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs, spec.slice
|
||||
)
|
||||
+ f"\nAnswer: {json.dumps(answer, sort_keys=True)}"
|
||||
)
|
||||
return "\n\n".join(parts)
|
||||
@@ -273,11 +279,14 @@ def render(
|
||||
code = excerpt_text(example.excerpt_sha) or ""
|
||||
blocks.append(
|
||||
f"Example {number} (label: {label_word(example.label)}):\n"
|
||||
+ _case(code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs)
|
||||
+ _case(
|
||||
code, example.x, example.instruments, example.roles, example.family, spec.profile, example.language, spec.inputs, spec.slice
|
||||
)
|
||||
)
|
||||
t = candidate.target
|
||||
blocks.append(
|
||||
"Candidate:\n" + _case(candidate.code, t.x, t.instruments, candidate.roles, t.family, spec.profile, candidate.language, spec.inputs)
|
||||
"Candidate:\n"
|
||||
+ _case(candidate.code, t.x, t.instruments, candidate.roles, t.family, spec.profile, candidate.language, spec.inputs, spec.slice)
|
||||
)
|
||||
messages: list[dict[str, object]] = [
|
||||
{"role": "system", "content": render_prefix(spec, demos, examples, excerpt_text)},
|
||||
@@ -506,20 +515,24 @@ class Program:
|
||||
out.append((prompt, got))
|
||||
return out
|
||||
|
||||
def decide(self, candidate: Candidate, fold: Fold, caller: Caller, *, seed: int = 0) -> Decision:
|
||||
def prepare_samples(self, candidate: Candidate, fold: Fold, *, seed: int = 0) -> list[tuple[Prompt, Retrieval, float, str]]:
|
||||
"""The exact prompts, temperatures and sample labels shared by execution and cache planning."""
|
||||
if self.spec.sampling == RETRIEVAL_ENSEMBLE:
|
||||
prepared = self.prepare_ensemble(candidate, fold, seed=seed)
|
||||
answers = [
|
||||
sample_answer(prompt, caller, temperature=TEMPERATURES[0], sample=str(j), families=self.families)
|
||||
for j, (prompt, _) in enumerate(prepared)
|
||||
]
|
||||
got = prepared[0][1]
|
||||
return replace(aggregate(answers), retrieval=got.mode, neighbours=got.neighbours)
|
||||
return [(prompt, got, TEMPERATURES[0], str(j)) for j, (prompt, got) in enumerate(prepared)]
|
||||
if self.spec.sampling != SAMPLING[0]:
|
||||
raise ValueError(f"unknown sampling {self.spec.sampling!r} (one of {', '.join(SAMPLING)})")
|
||||
prompt, got = self.prepare(candidate, fold, seed=seed)
|
||||
decision = classify(prompt, caller, self.spec.k, families=self.families)
|
||||
return replace(decision, retrieval=got.mode, neighbours=got.neighbours)
|
||||
return [(prompt, got, TEMPERATURES[0] if j == 0 else TEMPERATURES[1], str(j)) for j in range(self.spec.k)]
|
||||
|
||||
def decide(self, candidate: Candidate, fold: Fold, caller: Caller, *, seed: int = 0) -> Decision:
|
||||
prepared = self.prepare_samples(candidate, fold, seed=seed)
|
||||
answers = [
|
||||
sample_answer(prompt, caller, temperature=temperature, sample=sample, families=self.families)
|
||||
for prompt, _, temperature, sample in prepared
|
||||
]
|
||||
got = prepared[0][1]
|
||||
return replace(aggregate(answers), retrieval=got.mode, neighbours=got.neighbours)
|
||||
|
||||
|
||||
__all__ = [
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
"""Bounded, local def-use evidence, never a vulnerability verdict.
|
||||
|
||||
Uses the shipped FileIR and fact/taxonomy routing. Parameters are potential inputs,
|
||||
as in semantic.taint; unknown call returns carry their arguments conservatively.
|
||||
Sanitizers stay on paths (they do not erase connectivity). Facts currently scope
|
||||
sanitizers by language, not family, so a sanitizer is not a discharge claim.
|
||||
The flat IR has no branch edges or dominance: no guard is asserted, and branch
|
||||
assignments are visited in textual order. No interprocedural or alias analysis.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from textwrap import dedent
|
||||
from typing import Literal
|
||||
|
||||
from ..model.specs import taint_specs
|
||||
from ..semantic.facts import load_facts
|
||||
from ..semantic.ir import Bind, CallSite, FunctionIR, parse_file
|
||||
from ..semantic.taint import _call_matches, _sanitizer_for, _source_in
|
||||
from .excerpt import redact
|
||||
|
||||
Role = Literal["source", "step", "sanitizer", "guard", "sink"]
|
||||
# The renderer uses two spaces; normalization strips the suffix on blank lines.
|
||||
_NUMBERED = re.compile(r"^\s*(\d+)(?: (.*))?$")
|
||||
|
||||
|
||||
@dataclass(frozen=True, order=True)
|
||||
class Step:
|
||||
line: int
|
||||
role: Role
|
||||
code: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Slice:
|
||||
steps: tuple[Step, ...] = ()
|
||||
|
||||
|
||||
def function_slice(excerpt_text: str, language: str, family: str) -> Slice | None:
|
||||
"""Return local paths, an empty slice, or None for an unsupported/unparseable excerpt.
|
||||
|
||||
Numbered excerpts retain their displayed numbers; raw code uses 1-based lines.
|
||||
Diffs are never parsed. A truncated/malformed excerpt cannot establish no-flow.
|
||||
Nested functions are analysed separately and never share definitions.
|
||||
"""
|
||||
raw = excerpt_text.split("\ndiff:\n", 1)[0].splitlines()
|
||||
matches = [_NUMBERED.match(line) for line in raw]
|
||||
if any(matches) and not all(matches):
|
||||
return None
|
||||
numbers = [int(m[1]) if m else i + 1 for i, m in enumerate(matches)]
|
||||
lines = [(m[2] or "") if m else line for m, line in zip(matches, raw, strict=True)]
|
||||
if len(set(numbers)) != len(numbers):
|
||||
return None
|
||||
code = dedent("\n".join(lines))
|
||||
ir = parse_file("excerpt", code, language)
|
||||
if not ir.parse_ok:
|
||||
return None
|
||||
spec = taint_specs(language=language).get(family)
|
||||
if spec is None:
|
||||
return Slice()
|
||||
shown = redact(lines, language)
|
||||
facts = load_facts().for_language(language)
|
||||
|
||||
def step(line: int, role: Role) -> Step:
|
||||
return Step(numbers[line - 1], role, shown[line - 1].strip())
|
||||
|
||||
def paths(function: FunctionIR) -> set[Step]:
|
||||
states = {name: {step(function.start_line, "source")} for name in function.params}
|
||||
found: set[Step] = set()
|
||||
|
||||
def value(text: str, names: tuple[str, ...], constant: bool, line: int, end: int) -> set[Step]:
|
||||
if constant:
|
||||
return set()
|
||||
result = set().union(*(states.get(name, set()) for name in names))
|
||||
if _source_in(text, facts):
|
||||
result.add(step(line, "source"))
|
||||
if result:
|
||||
# Keep only sanitizers whose call is inside this expression, not
|
||||
# an unrelated call elsewhere on the same source line.
|
||||
for call in function.calls:
|
||||
if (
|
||||
call.line >= line
|
||||
and (not end or call.order <= end)
|
||||
and any(
|
||||
re.search(r"(?<![\w$])" + re.escape(name) + r"\s*\(", text) and _call_matches(call.name, name)
|
||||
for name in spec.sanitizers
|
||||
)
|
||||
):
|
||||
result.add(step(call.line, "sanitizer"))
|
||||
return result
|
||||
|
||||
events: list[Bind | CallSite] = [*function.binds, *function.calls]
|
||||
# Tree-sitter supplies evaluation ends, including same-line statements.
|
||||
# The AST fallback supplies equivalent end-line/column positions.
|
||||
events.sort(key=lambda e: (e.order if e.order else e.line, isinstance(e, Bind)))
|
||||
for event in events:
|
||||
if isinstance(event, Bind):
|
||||
inherited = value(event.value_text, event.names, event.is_constant, event.line, event.order)
|
||||
states[event.name] = inherited | {step(event.line, "step")} if inherited else set()
|
||||
continue
|
||||
if not any(_call_matches(event.name, name) for name in spec.sinks):
|
||||
continue
|
||||
reached: set[Step] = set()
|
||||
for text, names, constant in zip(event.arg_texts, event.arg_names, event.arg_is_constant, strict=True):
|
||||
reached.update(value(text, names, constant, event.line, event.order))
|
||||
if reached:
|
||||
if _sanitizer_for(event, facts):
|
||||
reached.add(step(event.line, "sanitizer"))
|
||||
found.update(reached | {step(event.line, "sink")})
|
||||
return found
|
||||
|
||||
result: set[Step] = set()
|
||||
for function in ir.functions:
|
||||
result.update(paths(function))
|
||||
order = {"source": 0, "step": 1, "sanitizer": 2, "guard": 3, "sink": 4}
|
||||
return Slice(tuple(sorted(result, key=lambda s: (s.line, order[s.role]))))
|
||||
|
||||
|
||||
def render_slice(slice_or_none: Slice | None, reason: str) -> str:
|
||||
"""Fixed framing; reason is the language, never free-form provenance text."""
|
||||
prefix = "Data flow inside the function:"
|
||||
if slice_or_none is None:
|
||||
language = (
|
||||
reason
|
||||
if reason
|
||||
in {
|
||||
"python",
|
||||
"javascript",
|
||||
"typescript",
|
||||
"tsx",
|
||||
"c",
|
||||
"cpp",
|
||||
"java",
|
||||
"php",
|
||||
"go",
|
||||
"ruby",
|
||||
"rust",
|
||||
"csharp",
|
||||
"perl",
|
||||
"kotlin",
|
||||
"swift",
|
||||
"scala",
|
||||
"lua",
|
||||
"groovy",
|
||||
"shell",
|
||||
"other",
|
||||
}
|
||||
else "other"
|
||||
)
|
||||
return f"{prefix} not analysed ({language} is not supported)."
|
||||
if not slice_or_none.steps:
|
||||
return f"{prefix} no source reaches a sink of this family."
|
||||
return prefix + "\n" + "\n".join(f"L{s.line} {s.role}: {s.code}" for s in slice_or_none.steps)
|
||||
@@ -70,6 +70,18 @@ recorded = "2026-10-01"
|
||||
status = "development"
|
||||
exclude_tiers = ["title"]
|
||||
|
||||
# Batch 2: advisory fixes whose changed function survives on the fixed side, for the six compiled families (most
|
||||
# published after 2026-06-01). Same slice, cache layout and checks as batch 1; screening record
|
||||
# benchmarks/measurements/2026-10-03-decision-engine-negatives-harvest/record.json.
|
||||
[[source]]
|
||||
id = "advisory-fixes-2"
|
||||
kind = "pairs"
|
||||
file = "benchmarks/pairs/advisory-fixes-2/catalog.toml"
|
||||
evaluation = ["benchmarks/pairs/advisory-fixes-2/verification.json"]
|
||||
recorded = "2026-10-03"
|
||||
status = "development"
|
||||
exclude_tiers = ["title"]
|
||||
|
||||
[[source]]
|
||||
id = "adjudications"
|
||||
kind = "adjudications"
|
||||
|
||||
@@ -111,6 +111,9 @@ class ChangeContext(Contract):
|
||||
# are not encoded or decoded by this field.
|
||||
path_encoding: Literal["text", "filesystem-bytes-hex"] = "text"
|
||||
line_correspondences: tuple[LineCorrespondence, ...] = ()
|
||||
# Paths Git reports as added (status A) in head, in this context's path encoding. A rename
|
||||
# target is not added. Only a lexical fact: novelty still needs complete head evidence.
|
||||
added_paths: tuple[str, ...] = ()
|
||||
|
||||
def decode_path(self, path: str) -> str:
|
||||
"""Return the filesystem spelling for matching ordinary question paths.
|
||||
@@ -130,6 +133,7 @@ class ChangeContext(Contract):
|
||||
*(s.path for s in self.spans),
|
||||
*(p for r in self.renames for p in (r.base_path, r.head_path)),
|
||||
*(p for m in self.line_correspondences for p in (m.base_path, m.head_path)),
|
||||
*self.added_paths,
|
||||
]
|
||||
for path in paths:
|
||||
try:
|
||||
|
||||
@@ -96,6 +96,21 @@ class DeepSeekChatClient:
|
||||
self._disable_thinking = disable_thinking
|
||||
self.endpoint = endpoint
|
||||
self.usage: list[dict[str, object]] = [] # one entry per call, for the round's cost meter
|
||||
self._output_limit: int | None = None
|
||||
|
||||
def bounded(self, max_output_tokens: int) -> DeepSeekChatClient:
|
||||
"""A single-attempt, output-capped copy for callers reserving a hard per-call budget.
|
||||
|
||||
Empty JSON retries belong to the caller's meter, and transport failures must
|
||||
stop that caller before another potentially billable attempt.
|
||||
"""
|
||||
from dataclasses import replace
|
||||
|
||||
if max_output_tokens <= 0:
|
||||
raise ValueError("max_output_tokens must be positive")
|
||||
client = DeepSeekChatClient(replace(self._client, max_attempts=1), disable_thinking=self._disable_thinking, endpoint=self.endpoint)
|
||||
client._output_limit = max_output_tokens
|
||||
return client
|
||||
|
||||
def complete(
|
||||
self,
|
||||
@@ -117,7 +132,7 @@ class DeepSeekChatClient:
|
||||
logprobs=logprobs,
|
||||
temperature=temperature,
|
||||
)
|
||||
if json_object and not (response.content or "").strip():
|
||||
if self._output_limit is None and json_object and not (response.content or "").strip():
|
||||
response = self._call(
|
||||
model=model,
|
||||
messages=messages,
|
||||
@@ -147,6 +162,8 @@ class DeepSeekChatClient:
|
||||
temperature: float | None = None,
|
||||
) -> ChatResponse:
|
||||
extra: dict[str, object] = {}
|
||||
if self._output_limit is not None:
|
||||
extra["max_tokens"] = self._output_limit
|
||||
if self._disable_thinking:
|
||||
extra["thinking"] = {"type": "disabled"} # otherwise temperature is silently ignored
|
||||
if temperature is not None:
|
||||
|
||||
@@ -19,7 +19,7 @@ from .layout import is_test_path, is_vendored, layout_facts
|
||||
FRONTENDS = {
|
||||
"php": "php",
|
||||
"javascript": "javascript",
|
||||
"typescript": "typescript",
|
||||
"typescript": "javascript",
|
||||
"python": "python",
|
||||
"java": "java",
|
||||
"c": "c",
|
||||
@@ -109,14 +109,19 @@ class PartitionGraph:
|
||||
|
||||
def _dispatch(self, kind: str, requests: Mapping[str, Mapping[str, object]], method: str) -> dict[str, list[object]]:
|
||||
answers: dict[str, list[object]] = {}
|
||||
languages = dict.fromkeys(self.path_languages.get(str(params.get("file", "")), "") for params in requests.values())
|
||||
|
||||
def graph_language(path: str) -> str:
|
||||
language = self.path_languages.get(path, "")
|
||||
return "javascript" if language == "typescript" else language
|
||||
|
||||
languages = dict.fromkeys(graph_language(str(params.get("file", ""))) for params in requests.values())
|
||||
for language in languages:
|
||||
graph = self.graphs.get(language)
|
||||
if graph is None:
|
||||
continue
|
||||
if self.budget is not None and time.monotonic() >= self.budget.deadline_monotonic:
|
||||
break
|
||||
selected = {rid: params for rid, params in requests.items() if self.path_languages.get(str(params.get("file", ""))) == language}
|
||||
selected = {rid: params for rid, params in requests.items() if graph_language(str(params.get("file", ""))) == language}
|
||||
if not selected:
|
||||
continue
|
||||
batch = getattr(graph, method, None) or getattr(graph, "run_batch", None)
|
||||
@@ -133,7 +138,12 @@ class PartitionGraph:
|
||||
# Carried, not normalized: it names request ids and costs, never a path.
|
||||
answers.setdefault(TIMING, []).extend(rows)
|
||||
elif rid == "__census__" and isinstance(rows, list):
|
||||
expected = set(next(p.paths for p in self.partitions if p.language == language))
|
||||
expected = {
|
||||
path
|
||||
for p in self.partitions
|
||||
if p.language == language or (language == "javascript" and p.language == "typescript")
|
||||
for path in p.paths
|
||||
}
|
||||
normalized = self._normalize(rows)
|
||||
names = [r.get("file_names") for r in normalized if isinstance(r, Mapping)]
|
||||
named = bool(names) and all(isinstance(n, list) and all(isinstance(p, str) for p in n) for n in names)
|
||||
@@ -234,10 +244,17 @@ def build_partitions(
|
||||
graph.path_languages[relative] = languages[0]
|
||||
elif languages:
|
||||
boundaries.add("ambiguous_frontend_path")
|
||||
if len(paths) > 1:
|
||||
# JavaScript and TypeScript share one frontend input tree and graph. Keep
|
||||
# original per-file languages for scoped coverage and property limitations.
|
||||
groups: dict[str, list[str]] = {}
|
||||
for language in paths:
|
||||
key = "javascript" if language == "typescript" else language
|
||||
groups.setdefault(key, []).append(language)
|
||||
if len(groups) > 1:
|
||||
boundaries.add("cross_partition_semantics_unresolved")
|
||||
records: list[PartitionCoverage] = []
|
||||
for language, sources in sorted(paths.items()):
|
||||
for language, members in sorted(groups.items()):
|
||||
sources = [source for member in members for source in paths[member]]
|
||||
relative_paths = tuple(sorted(p.relative_to(root).as_posix() for p in sources))
|
||||
frontend = FRONTENDS.get(language)
|
||||
if frontend is None:
|
||||
@@ -247,7 +264,7 @@ def build_partitions(
|
||||
continue
|
||||
scratch = Path(tempfile.mkdtemp(prefix="ousast-input-"))
|
||||
graph.roots[language] = scratch
|
||||
size = 0
|
||||
sizes: dict[str, int] = {}
|
||||
complete = True
|
||||
for source in sources:
|
||||
if budget is not None and time.monotonic() >= budget.deadline_monotonic:
|
||||
@@ -268,25 +285,31 @@ def build_partitions(
|
||||
complete = False
|
||||
break
|
||||
output.write(chunk)
|
||||
size += len(chunk)
|
||||
relative = source.relative_to(root).as_posix()
|
||||
sizes[relative] = sizes.get(relative, 0) + len(chunk)
|
||||
if not complete:
|
||||
break
|
||||
built = build(scratch, frontend) if complete else None
|
||||
if built is not None:
|
||||
graph.graphs[language] = built
|
||||
graph.unparsed += tuple(graph._normalize(list(getattr(built, "unparsed", ()))))
|
||||
record_limits: tuple[str, ...] = ("runtime_unspecified",) if language in {"javascript", "typescript"} else ()
|
||||
if language == "typescript":
|
||||
record_limits += ("typescript_property_support_unvalidated",)
|
||||
boundaries.add("typescript_property_support_unvalidated")
|
||||
# About TypeScript answers. YesWiki is a PHP application that ships a few `.ts` files, and this
|
||||
# boundary made all 1,128 of its PHP questions unresolved.
|
||||
owned_languages.setdefault("typescript_property_support_unvalidated", set()).add("typescript")
|
||||
if language in {"c", "cpp"}:
|
||||
record_limits += ("c_bounds_arithmetic_unmodeled",)
|
||||
records.append(
|
||||
PartitionCoverage(language, frontend, relative_paths, size, "built" if built is not None else "build_failed", record_limits)
|
||||
)
|
||||
for language in sorted(members):
|
||||
relative_paths = tuple(sorted(p.relative_to(root).as_posix() for p in paths[language]))
|
||||
source_size = sum(sizes.get(p, 0) for p in relative_paths)
|
||||
record_limits: tuple[str, ...] = ("runtime_unspecified",) if language in {"javascript", "typescript"} else ()
|
||||
if language == "typescript":
|
||||
record_limits += ("typescript_property_support_unvalidated",)
|
||||
boundaries.add("typescript_property_support_unvalidated")
|
||||
# About TypeScript answers. YesWiki is a PHP application that ships a few `.ts` files, and this
|
||||
# boundary made all 1,128 of its PHP questions unresolved.
|
||||
owned_languages.setdefault("typescript_property_support_unvalidated", set()).add("typescript")
|
||||
if language in {"c", "cpp"}:
|
||||
record_limits += ("c_bounds_arithmetic_unmodeled",)
|
||||
records.append(
|
||||
PartitionCoverage(
|
||||
language, frontend, relative_paths, source_size, "built" if built is not None else "build_failed", record_limits
|
||||
)
|
||||
)
|
||||
from dataclasses import replace
|
||||
|
||||
graph.partitions = tuple(
|
||||
|
||||
@@ -24,6 +24,7 @@ from collections.abc import Mapping, Sequence
|
||||
from dataclasses import dataclass, replace
|
||||
from pathlib import Path
|
||||
|
||||
from ..semantic.facts import TAINT_LANGUAGE_ALIAS
|
||||
from .contracts import AffectedRelationship, ChangeContext, ExecutionBudget, QuestionIdentity
|
||||
from .specs import config_specs, dominance_specs, taint_specs
|
||||
|
||||
@@ -41,7 +42,7 @@ _ACCESS_RANK: dict[str, float] = {
|
||||
# Languages whose facts the model layer carries. Anything else yields no region rather than an empty scan
|
||||
# that looks like a clean bill of health.
|
||||
_LANGUAGES = {"python", "javascript", "typescript", "java", "c", "c_cpp", "php"}
|
||||
_NORMALISE = {"typescript": "javascript", "c_cpp": "c"}
|
||||
_NORMALISE = {**TAINT_LANGUAGE_ALIAS, "c_cpp": "c"}
|
||||
|
||||
# What a CPG calls a file's top-level code. A module body is a method like any other to the engine, and
|
||||
# naming it here is what lets a region ask about the code that is not inside any function.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user